Search Off the Record

Should you convert your website into Markdown to help Large Language Models (LLMs) understand your content better? Is "llms.txt" worth the effort for SEO?
In this episode of Search Off the Record, Martin Splitt and John Mueller from the Google Search Relations team dive deep into the history of Markdown, its rise in the AI era, and whether it holds any real weight for search engine discovery.
In this episode, you'll learn:<...

In this episode of Search Off the Record, Martin Splitt and John Mueller from Google's Search Relations team dive deep into the world of AI-assisted development. They explore the reality of "Vibe Coding", the process of building apps and websites using natural language instead of manual syntax. Whether you're a developer looking to offload tedious setup tasks or an SEO expert trying to understand how AI-generated sites impact ...

In this episode of Search Off the Record, Martin speaks with Nikola Todorovic (director of Software Engineering at Google Search) about how AI is changing Google Search. They discuss the evolution from traditional search to AI Overviews and AI Mode, how Google tests and launches search changes, and why query behaviour is becoming more conversational and complex.

Nikola also explains the role of machine learning in Search, how featu...

In this episode of Search Off the Record, Martin and Gary turn a simple robots.txt question into a data‑driven deep dive using HTTP Archive, WebPageTest, custom JavaScript metrics, and BigQuery. They explore how millions of real robots.txt files are actually written in 2025–2026, which directives and user‑agents are most common, and what that means for modern crawling and AI bots.

Perfect for beginner to mid‑level developers ...

In this episode of Search Off the Record, Gary and Martin dig into what "page size" and "page weight" actually mean for developers, users, and search engines.

They discuss exploding web page sizes: median mobile homepages hit 2.3 MB in 2025 Web Almanac (up 3x from 2015), key insights for developers on page weight definitions, Googlebot's crawl limits, HTML bloat from structured data/images, and why size still hurts UX on slow conne...

Developers often talk about Googlebot as if it were a single program you could just run as "googlebot.exe", but that is not how Google's crawling actually works. In this episode of Search Off the Record, Martin and Gary from the Search Relations team unpack how Google's crawling infrastructure is really built and operated.​
They cover why "Googlebot" is a misnomer and how it relates to a central crawling software-as-a-service u...

Martin and Gary unpack how HTML parsing really works, why the HTML standard is so lenient, and how messy markup can silently break key SEO signals like hreflang and rel=canonical. They revisit validators and cross‑browser hacks from the Netscape/IE days, and discuss whether semantic HTML and strict validity truly matter for search. You'll also hear when link hints like preload, prefetch, and DNS prefetch help performance (and indir...

In this episode of Search Off the Record, Martin and Gary from the Google Search Relations team tackle a deceptively simple question: do you still need a website in 2026? Starting from the recurring industry claim that "the web is dead," they explore how the web has evolved through the rise of apps, AI chatbots, and social platforms, and why the answer almost always ends up being "it depends." Tune in for an engaging d...

​Join Martin and Gary as they dive into Search Off the Record's Episode 103, unpacking the 2025 Year-End report on crawling issues. Discover fascinating insights on faceted navigation, action parameters, irrelevant parameters and more, highlighting the biggest challenges faced by web crawlers last year. With humor and expert analysis, this episode reveals critical takeaways for webmasters and SEO professionals. Don't m...

In this episode, John Mueller and Danny Sullivan continue their conversation about the changing landscape of Search. They discuss the practicalities of hiring an SEO professional in the age of AI: Do you really need one? How do you evaluate them? Danny also shares his thoughts on third-party SEO tools, "domain scores," and why creators shouldn't rush to break their content into "bite-sized chunks" just to please Large Language Mode...