Erin welcomes Carolyn Shelby, the Principal SEO at Yoast and a renowned authority in technical and enterprise SEO. Carolyn brings decades of hands-on experience from her pioneering days in digital marketing, working with brands like Disney’s ESPN, Tribune Publishing, and major nonprofits. The conversation kicks off with a surprising twist—Carolyn’s unique title as Queen of the micronation Ladonia—before diving into her role at Yoast and their latest innovation: the LLMs.txt file generator.
Carolyn explains how this new file helps websites communicate their most valuable content directly to large language models like ChatGPT and Google’s AI, streamlining the way future search agents discover and answer questions with information from your site. We explore what inspired Yoast’s push to roll out LLMs.txt to over 13 million sites, what website owners should include in their files, potential industry pushback, the adoption challenge with search giants, and how this moment could change the way websites optimize for AI-driven search results.
Key Segments:
- [00:01:46] Introducing Carolyn Shelby, Senior SEO at Yoast
- [00:03:09] Queen of the Micronation Ladonia?
- [00:07:56] What is the LLMS.txt file?
- [00:08:59] LLMS.txt is a Treasure Map
- [00:14:38] A New File, along with Robots.txt and Sitemap.xml
- [00:15:41] What inspired Yoast to create this LLM text file?
- [00:17:12] EDGE of the Web Sponsor: PreWriter.AI
- [00:18:22] LLMS.txt proposed by Jeremy Howard (Sept, 2024)
- [00:22:37] Standard Uniformity and Acceptance?
- [00:24:43] Housekeeping
- [00:29:37] LLM Markdown Effort Questioned: Exploitation?
- [00:31:41] LLMs Lack Memory at Inference
- [00:34:07] EDGE of The Web Sponsor: Inlinks (WAIKAY)
- [00:36:09] Pushback on the LLMS.txt file
Thanks to Our Sponsors!
PreWriter.AI: https://edgeofthewebradio.com/prewriter
Inlinks WAIKAY https://edgeofthewebradio.com/waikay
Follow Our Guest
Twitter: @cshel
LinkedIn: https://www.linkedin.com/in/cshel/
Resources
Learn about Ladonia (DONATE!): https://www.ladonia.org/about/
Carolyn’s Posts on LLMS.txt:
https://www.cshel.com/ai-seo/how-llms-interpret-content-structuring-for-ai-search-unedited-version/
https://searchengineland.com/llms-txt-isnt-robots-txt-its-a-treasure-map-for-ai-456586
Popular Podcasts
CrimeLess: Hillbilly Heist
It’s 1996 in rural North Carolina, and an oddball crew makes history when they pull off America’s third largest cash heist. But it’s all downhill from there. Join host Johnny Knoxville as he unspools a wild and woolly tale about a group of regular ‘ol folks who risked it all for a chance at a better life. CrimeLess: Hillbilly Heist answers the question: what would you do with 17.3 million dollars? The answer includes diamond rings, mansions, velvet Elvis paintings, plus a run for the border, murder-for-hire-plots, and FBI busts.
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.