Erin welcomes Carolyn Shelby, the Principal SEO at Yoast and a renowned authority in technical and enterprise SEO. Carolyn brings decades of hands-on experience from her pioneering days in digital marketing, working with brands like Disney's ESPN, Tribune Publishing, and major nonprofits. The conversation kicks off with a surprising twist—Carolyn's unique title as Queen of the micronation Ladonia—before diving into her role at Yoast and their latest innovation: the LLMs.txt file generator.
Carolyn explains how this new file helps websites communicate their most valuable content directly to large language models like ChatGPT and Google's AI, streamlining the way future search agents discover and answer questions with information from your site. We explore what inspired Yoast's push to roll out LLMs.txt to over 13 million sites, what website owners should include in their files, potential industry pushback, the adoption challenge with search giants, and how this moment could change the way websites optimize for AI-driven search results.
Key Segments:
- [00:01:46] Introducing Carolyn Shelby, Senior SEO at Yoast
- [00:03:09] Queen of the Micronation Ladonia?
- [00:07:56] What is the LLMS.txt file?
- [00:08:59] LLMS.txt is a Treasure Map
- [00:14:38] A New File, along with Robots.txt and Sitemap.xml
- [00:15:41] What inspired Yoast to create this LLM text file?
- [00:17:12] EDGE of the Web Sponsor: PreWriter.AI
- [00:18:22] LLMS.txt proposed by Jeremy Howard (Sept, 2024)
- [00:22:37] Standard Uniformity and Acceptance?
- [00:24:43] Housekeeping
- [00:29:37] LLM Markdown Effort Questioned: Exploitation?
- [00:31:41] LLMs Lack Memory at Inference
- [00:34:07] EDGE of The Web Sponsor: Inlinks (WAIKAY)
- [00:36:09] Pushback on the LLMS.txt file
Thanks to Our Sponsors!
PreWriter.AI: https://edgeofthewebradio.com/prewriter
Inlinks WAIKAY https://edgeofthewebradio.com/waikay
Follow Our Guest
Twitter: @cshel
LinkedIn: https://www.linkedin.com/in/cshel/
Resources
Learn about Ladonia (DONATE!): https://www.ladonia.org/about/
Carolyn's Posts on LLMS.txt:
https://www.cshel.com/ai-seo/how-llms-interpret-content-structuring-for-ai-search-unedited-version/
https://searchengineland.com/llms-txt-isnt-robots-txt-its-a-treasure-map-for-ai-456586
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Are You A Charlotte?
In 1997, actress Kristin Davis’ life was forever changed when she took on the role of Charlotte York in Sex and the City. As we watched Carrie, Samantha, Miranda and Charlotte navigate relationships in NYC, the show helped push once unacceptable conversation topics out of the shadows and altered the narrative around women and sex. We all saw ourselves in them as they searched for fulfillment in life, sex and friendships. Now, Kristin Davis wants to connect with you, the fans, and share untold stories and all the behind the scenes. Together, with Kristin and special guests, what will begin with Sex and the City will evolve into talks about themes that are still so relevant today. "Are you a Charlotte?" is much more than just rewatching this beloved show, it brings the past and the present together as we talk with heart, humor and of course some optimism.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.