In this episode, host João Tomé is joined by Will Allen, Cloudflare’s VP of Product Management, to discuss Pay-Per-Crawl and our new permission-based model for AI bots. These updates, launched on July 1, 2025 — what we call Content Independence Day — aim to reshape how AI models access and reward content, shifting from an opt-out to an opt-in approach. Cloudflare’s CEO, Matthew Prince, also appears in a video at the start, reflecting on Content Independence Day.
We explore how AI Overviews are changing the old “traffic for content” model and how Cloudflare is helping creators take control through tools like AI Audit. Plus: the future of trustworthy content, bot authentication, and the rise of a fairer content economy.
Send us your questions for a future episode on these topics at ThisWeekinNET@cloudflare.com.
Check the mentioned blog posts here: https://blog.cloudflare.com/tag/pay-per-crawl/
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Cardiac Cowboys
The heart was always off-limits to surgeons. Cutting into it spelled instant death for the patient. That is, until a ragtag group of doctors scattered across the Midwest and Texas decided to throw out the rule book. Working in makeshift laboratories and home garages, using medical devices made from scavenged machine parts and beer tubes, these men and women invented the field of open heart surgery. Odds are, someone you know is alive because of them. So why has history left them behind? Presented by Chris Pine, CARDIAC COWBOYS tells the gripping true story behind the birth of heart surgery, and the young, Greatest Generation doctors who made it happen. For years, they competed and feuded, racing to be the first, the best, and the most prolific. Some appeared on the cover of Time Magazine, operated on kings and advised presidents. Others ended up disgraced, penniless, and convicted of felonies. Together, they ignited a revolution in medicine, and changed the world.
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.