{"id":11148,"date":"2026-07-07T17:03:38","date_gmt":"2026-07-07T09:03:38","guid":{"rendered":"http:\/\/longzhuplatform.com\/?p=11148"},"modified":"2026-07-07T17:03:38","modified_gmt":"2026-07-07T09:03:38","slug":"the-web-is-growing-a-second-layer-almost-a-third-head-via-sejournal-demirie","status":"publish","type":"post","link":"http:\/\/longzhuplatform.com\/?p=11148","title":{"rendered":"The Web Is Growing A Second Layer \u2013 Almost A Third Head via @sejournal, @demirie"},"content":{"rendered":"<p><\/p> <div id=\"narrow-cont\"> <p>The last few weeks have been noisy. Google shipped something called the Open Knowledge Format. Then Google Developers announced the Agentic Resource Discovery (ARD) specification.<\/p> <p>Meanwhile, every SEO LinkedIn feed is lit up with someone either declaring markdown the future of the web or explaining why you should ignore all of it.<\/p> <p>The truth, as per usual, sits somewhere more interesting than either camp.<\/p> <p>The web is developing a parallel machine-readable infrastructure (MCP\/WebMCP, OKF, ARD, LLMs.txt\u2026) and SEOs who understand what each layer actually does, rather than treating it all as \u201cAI SEO\u201d or a silver bullet, will make better decisions about where to spend their time.<\/p> <h2>First: The Layer Cake<\/h2> <p>There are at least six distinct things being discussed under the umbrella of \u201cmaking your site AI-ready.\u201d They sit at different layers and serve different purposes:<\/p> <p><iframe class=\"sej-iframe-auto-height\" id=\"in-content-iframe\" scrolling=\"no\" src=\"https:\/\/www.searchenginejournal.com\/wp-json\/sscats\/v2\/tk\/Middle_Post_Text\"><\/iframe><\/p> <ul> <li><strong>Crawlable HTML Pages:<\/strong> Still the foundation. Nothing has changed here. Everything else sits on top.<\/li> <li><strong>Schema.org\/Structured Data: <\/strong>Semantic hints baked into HTML that tell machines explicitly what a page is about. It is, in essence, a vocabulary.<\/li> <li><strong>LLMs.txt:<\/strong> Essentially a navigation file. Its purpose is to essentially tell an AI agent that\u2019s already on your site which pages matter. But as John Mueller puts it on the Search Off the Record podcast:<\/li> <\/ul> <blockquote> <p><em>\u201cIf someone is already on your website, maybe some kind of automated system is helpful. Where if it goes, I want to go to Martin\u2019s Splitt and buy a photograph, then the LLM system can go to your website and can look around, like, how do you buy a photograph? Maybe he has some guidelines for me as an agent for buying photographs. That kind of makes sense.\u201d<\/em><\/p> <\/blockquote> <ul> <li><strong>MCP\/WebMCP:<\/strong> Before ARD came into play, we were presented with another solution for the challenge of interoperability. An MCP, in its simplest explanation, is a standardized way for an AI to connect to your services to extract knowledge or take action. WebMCP, as the name itself suggests, gives websites a way to engage with agents directly. WebMCP is for live browser interactions on a webpage; MCP is for tools and services beyond the page.<\/li> <li><strong>Open Knowledge Format (OKF):<\/strong> A bundle of markdown files with YAML frontmatter.<\/li> <li><strong>Agentic Resource Discovery (ARD): <\/strong>A new open spec for how agents find and verify tools, skills, and other agents across the web. Here, the focus is not your content; it\u2019s your capabilities.<\/li> <\/ul> <p>For ecommerce, there\u2019s another layer worth naming separately \u2013 the product feed \u2013 quite possibly the future of retail discovery.<\/p> <p>Each layer does something different.<\/p> <p>I could keep adding to this list; there\u2019s a new layer popping up every five minutes. I\u2019m stopping here. It\u2019s ballooning.<\/p> <h3>What OKF Actually Is (And Isn\u2019t)<\/h3> <p>Google published the OKF spec quietly, bolted to a rebrand of Dataplex into Knowledge Catalog.<\/p> <p>The format itself is almost disarmingly simple: a directory of markdown files, each with a small YAML header declaring a type, title, description, resource, and some tags. The files link to each other like any markdown document would. That\u2019s it.<\/p> <p>As Google\u2019s own blog puts it, OKF is \u201cjust markdown, just files, just YAML frontmatter.\u201d<\/p> <p>SEO Suganthan Mohanadasan has a clear breakdown of this. He describes OKF as one floor in a stack that now includes sitemap.xml (which URLs exist), LLMs.txt (which pages you most want read), and OKF (the library itself). They stack rather than compete.<\/p> <p>The confusion sets in not when you look at what OKF is, but what it does and in which layer of the agentic and search mayhem it sits.<\/p> <p>In my mind, OKF is not a retrieval system. It doesn\u2019t replace crawling. And, personally, I do not see a future where AI systems no longer ingest massive amounts of HTML or where search and RAG are not a multistep complex pipeline that consists of self-reported and \u201cunbased\u201d signals.<\/p> <p>Any self-reported system can and will be gamed. So thinking you can just slam a bunch of markdown files on your site and be THE preferred choice in retrieval and discovery is far-fetched.<\/p> <p>OKF is a higher-signal source among many. It may reduce parsing cost and improve signal quality, but it doesn\u2019t replace existing pipelines.<\/p> <p>It\u2019s also worth being honest here: OKF was built for data teams, not marketing sites.<\/p> <p>It arrived as a way to share internal knowledge, i.e., table schemas, runbooks, metric definitions, between AI agents inside organizations. Pointing it at a public website to me seems a bit like we are yet again repurposing.<\/p> <p>Francois Vanderseypen makes the most precise point about what OKF actually is and isn\u2019t: a directed graph of markdown files is a web of documents, not a knowledge graph (at least not in its purest sense). A real KG has explicit, queryable, typed relations.\u00a0 OKF leaves what a link implies entirely up to the producer, and an LLM still has to infer the semantics every single time it reads it.<\/p> <figure id=\"attachment_581263\" class=\"wp-caption aligncenter\" style=\"width: 1086px\"><img decoding=\"async\" src=\"https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815.png\"  width=\"1086\" height=\"1136\" class=\"size-full wp-image-581263\" srcset=\"https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815-384x402.png 384w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815-425x445.png 425w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815-480x502.png 480w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815-680x711.png 680w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815-768x803.png 768w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815-850x889.png 850w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815-1024x1071.png 1024w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.29.38-pm-815.png 1086w\" sizes=\"auto, (max-width: 1086px) 100vw, 1086px\" loading=\"lazy\" title=\"The Web Is Growing A Second Layer \u2013 Almost A Third Head via @sejournal, @demirie\u63d2\u56fe\" alt=\"The Web Is Growing A Second Layer \u2013 Almost A Third Head via @sejournal, @demirie\u63d2\u56fe\" \/><figcaption class=\"wp-caption-text\">Screenshot from LinkedIn, July 2026<\/figcaption><\/figure> <p>For me, this points to the crux of how I understand the web and what we do as SEOs. OKF doesn\u2019t change the stack. It adds one more input into it.<\/p> <p>It\u2019s not a shortcut. There are no shortcuts.<\/p> <h2>The Schema.org Parallel, And Why It Matters<\/h2> <p>One of the patterns to understand here is the one Schema.org already went through.<\/p> <p>Structured data followed a predictable arc:<\/p> <p>Adoption \u2013 ranking boost \u2013 widespread use (and gaming) \u2013 platform learning \u2013 reduced dependency as a ranking signal.<\/p> <p>FAQ schema had a moment in SERPs, then Google discontinued the FAQ rich result. The platforms learn from the signals, fold the lessons into the algorithm, and the explicit markup becomes less necessary.<\/p> <p>OKF and LLMs.txt may follow the same path. They\u2019re most valuable early, as clear signals in a world where AI systems are still learning to parse the web.<\/p> <p>Over time, if the formats work, the systems learn. Explicit markup becomes redundant or remains a verification layer. For example, in ecommerce, in particular, schema and feed alignment has become more and more important. Another notch in the call for co-ownership of the product feed between SEO and paid teams!<\/p> <p>There\u2019s also a subtler point worth making here about the relationship between schema.org and discovery. Jarno van Driel\u2019s deep dive on product variants in Search Engine Journal illustrates this well: For years, Google Search and Google Merchant Center had conflicting structured data requirements, forcing publishers to duplicate markup. Schema.org evolves to close gaps, but it\u2019s slow, it\u2019s complex, and implementation is still often a mess.<\/p> <p>Structured data has never been a plug-and-play ranking lever. OKF won\u2019t be either.<\/p> <h2>Should You Convert Your Site To Markdown?<\/h2> <p>It\u2019s a big fat no from me. That doesn\u2019t mean I won\u2019t test it and apply carefully!<\/p> <p>And John Mueller said it on the Search Off the Record podcast:<\/p> <blockquote> <p><em>\u201cWhen it comes to things like a search engine or probably also in generic LLM system, having a website that uses normal HTML for the pages is critical. Because a search engine or crawler can just go to that page. It can recognise all of the other links that are within the website.\u201d<\/em><\/p> <\/blockquote> <p>The structural information in HTML \u2013 nav links, footers, header hierarchies, internal links \u2013 is how crawlers understand your site\u2019s shape. Markdown files strip all of that out. You\u2019d be breaking discovery in order to marginally improve machine readability of individual pages.<\/p> <p>Recently, on LinkedIn, I even saw a piece of research showing how \u201cYour navigation might be eating your LLM (it\u2019s ChatGPT Deep Research in fact) reading budget.\u201d Interesting findings, but please don\u2019t remove your navigation to \u201csave some tokens\u201d!<\/p> <figure id=\"attachment_581264\" class=\"wp-caption aligncenter\" style=\"width: 1088px\"><img decoding=\"async\" src=\"https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838.png\"  width=\"1088\" height=\"1042\" class=\"size-full wp-image-581264\" srcset=\"https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838-384x368.png 384w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838-425x407.png 425w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838-480x460.png 480w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838-680x651.png 680w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838-768x736.png 768w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838-850x814.png 850w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838-1024x981.png 1024w, https:\/\/cdn.searchenginejournal.com\/wp-content\/uploads\/2026\/06\/screenshot-2026-07-01-at-11.38.24-pm-838.png 1088w\" sizes=\"auto, (max-width: 1088px) 100vw, 1088px\" loading=\"lazy\" title=\"The Web Is Growing A Second Layer \u2013 Almost A Third Head via @sejournal, @demirie\u63d2\u56fe1\" alt=\"The Web Is Growing A Second Layer \u2013 Almost A Third Head via @sejournal, @demirie\u63d2\u56fe1\" \/><figcaption class=\"wp-caption-text\">Screenshot from LinkedIn, July 2026<\/figcaption><\/figure> <p>Jono Alderson makes this point brilliantly: \u201cA page is not just a container for words. It\u2019s an editorial artifact.\u201d Hierarchy, emphasis, placement, what comes first, what\u2019s prominent, what\u2019s tucked in a footnote \u2026 these aren\u2019t pretty decorations for humans. \u201cThey are signals about meaning.\u201d<\/p> <p>\u201cWhen you flatten a page into markdown, you don\u2019t just remove clutter. You remove judgment, and you remove context.\u201d And the moment you publish a machine-only representation, you\u2019ve created a second candidate version of reality.<\/p> <p>The boring fix still works: Semantic HTML, clear structure, sensible hierarchy, content that exists when the page loads.<\/p> <p>John Mueller covers the markdown debate extensively in the podcast: The parallel versions problem, the dynamic rendering lessons we already learned the hard way, and why maintaining a shadow version of your site for AI doubles your maintenance burden and creates a debugging nightmare nobody will tell you about.<\/p> <p>The one exception Mueller carves out is developer documentation:<\/p> <blockquote> <p><em>\u201cIf you have something like developer documentation, where, again, if the agent or the LLM system already knows about your website and the user says, how do I usethis API? Then if you give the LLM system a Markdown file, it\u2019s a lot easier for it to understand.\u201d<\/em><\/p> <\/blockquote> <p>Now, I can definitely see a straightforward use case there.<\/p> <h2>What ARD Is Actually Doing<\/h2> <p>The Agentic Resource Discovery specification, announced by Google on June 17, 2026, is a different beast entirely. It arrived only a couple of days behind OKF, not a coincidence, and is already making huge waves.<\/p> <p>The problem ARD solves is a coordination one. Right now, an agent has to be wired to each tool, MCP server, or API it uses before it can do anything with it.<\/p> <p>That works when you\u2019re connecting a handful of known services. It stops scaling the moment the number of available capabilities grows beyond what any team can pre-configure by hand.<\/p> <p>ARD moves that discovery out of setup and into runtime. The agent finds what it needs when it needs it, rather than only knowing what it was told about in advance.<\/p> <p>It\u2019s built on two primitives:<\/p> <ul> <li><strong>Catalogs: <\/strong>An ai-catalog.json file hosted on your domain, describing your available capabilities (MCP servers, A2A agents, OpenAPI tools). Ownership of the domain acts as the cryptographic foundation for identity and trust.<\/li> <li><strong>Registries: <\/strong>Search engines for the agentic web. They crawl catalogs, index them, and return matching capabilities with the metadata needed to verify the publisher before connecting.<\/li> <\/ul> <p>If OKF is about packaging knowledge for consumption, ARD is about advertising capabilities for connection.<\/p> <p>These are parallel efforts at different layers of the emerging agentic stack. Both shipped within inches of each other and now adopted with the speed of light by some very big players in the game, i.e., Hugging Face and their Discover Tool.<\/p> <p>It\u2019s possibly a more pragmatic bet than the formal logic layer that came before it and never reached web scale. Time will tell.<\/p> <h2>A Gap Worth Watching<\/h2> <p>Within days of both specs shipping, a contributor opened companion issues on the ARD and OKF repos pointing out something basic was missing: There\u2019s no agreed media type for an OKF bundle, so a catalog can list one but can\u2019t actually recognize it as OKF without sniffing the contents.<\/p> <p>In the meantime, publishers are already advertising bundles in production using their own interim types, which, as the issue itself notes, won\u2019t agree with each other.<\/p> <p>On the face of it, this looks like a small ask, just a request for a shared label.<\/p> <p>After a bit of a dive into this particular rabbit hole, it turns out that\u2019s quite normal practice. Waiting for full agreement before anyone ships anything is exactly how a spec dies in committee, and shipping fast and patching as real adoption surfaces is an age-old strategy.<\/p> <p>Application\/json itself wasn\u2019t formally registered until 2006, roughly five years after JSON was already in wide, informal use. Nobody worried about that, because the cost of the label being unsettled was low: A parser might reject something or fall back ungracefully.<\/p> <p>But OKF is different, because what happens after the fetch is different. The artifact behind the label is a bundle an autonomous agent is meant to ingest, verify, and potentially act on, inside a discovery system built specifically for agent-to-agent and agent-to-tool connection. Get the type wrong here, or leave an agent to infer it, and the risk isn\u2019t a parse error; it\u2019s a system acting on something it shouldn\u2019t have trusted, with no one checking the result first.<\/p> <p>I wonder about the risk involved in settling this later rather than sooner in this case. I guess it depends on how fast it gets resolved relative to how fast adoption runs ahead of it.<\/p> <h2>What This Means If You\u2019re An SEO<\/h2> <p>A few honest conclusions and my current thinking:<\/p> <p><strong>For most marketing and content sites, not much has changed. <\/strong>HTML, well-structured for humans, is still the right foundation. A contact-us form and a clean site architecture will serve you better than any OKF bundle ever will. Discovery still depends on links, authority, user signals \u2026 and indexing.<\/p> <p><strong>LLMs.txt is a signpost, not an SEO tool. <\/strong>It\u2019s useful for helping an agent navigate within your site once it\u2019s already there. It very likely doesn\u2019t make a big difference in how agents find you in the first place. And, probably never will.<\/p> <p><strong>MCP\/WebMCP.\u00a0<\/strong>Neither is urgent for most marketing sites today, but if you\u2019re building anything with programmatic interfaces or ecommerce flows you want agents to navigate, this is the direction the infrastructure is heading.<\/p> <p><strong>OKF makes a lot of sense if you\u2019re sitting on structured internal knowledge<\/strong>, i.e., documentation, API references, product specs \u2026 and you want to make it easier for agents to consume. The free OKF generator Suganthan built will produce a bundle and give you a graph view of your internal link structure as a side benefit. The structural audit alone seems worth it. But I will be running it on my website first, not on my client\u2019s website.<\/p> <p><strong>ARD is worth watching if you\u2019re building services with programmatic interfaces.<\/strong> If you have tools, agents, or APIs you want discoverable by other agents, ARD is the emerging standard for how that gets done. Just know the identity layer underneath it, what an agent is actually looking at when it finds your catalog entry, is still being settled in real time, so I\u2019d treat this as infrastructure to watch closely rather than build critical paths on just yet.<\/p> <p><strong>The schema adoption cycle might repeat.<\/strong> These formats are most valuable now, as early signals. Implement them if you can do it cheaply. Don\u2019t build your strategy around them holding value forever and don\u2019t bank on them as a silver bullet.<\/p> <p><strong>Ultimately, be aware of the shiny things<\/strong> \u2013 if your company has bigger fish to fry, i.e., a terrible website, a brand no one knows or cares for, an audience you don\u2019t understand \u2026 then deal with this first before you get caught up in any of these new shiny things.<\/p> <h2>The Underlying Shift<\/h2> <p>What all of this points to is a web that\u2019s genuinely growing a second layer or a third head, one written for machines alongside the one written for browsers and humans.<\/p> <p>Sitemap.xml told crawlers which URLs existed. Robots.txt told them where not to go. LLMs.txt, OKF, and ARD are similar infrastructure for agentic systems: navigation hints, content packaging, and capability discovery.<\/p> <p>None of it is mandatory today. None of it replaces solid HTML, authoritative content, sensible structure, or the thing that actually sits underneath all of it: a brand worth finding.<\/p> <p>But the SEOs who understand what each layer actually does, rather than treating it as a single undifferentiated \u201cAI SEO\u201d category, will make better bets on where to spend their time.<\/p> <p>My money is on the second layer, a parallel infrastructure written for machines, not a replacement for what already exists.<\/p> <p>The third head scenario, where agentic systems fully diverge from the human web, would require a different set of bets than any of us are currently making.<\/p> <p><em>Big thanks to Jarno van Driel, Jono Alderson, Chris Green, Suganthan Mohanadasan, Kristine Schachinger, Gianluca Fiorelli, Victor Pan, Renee Bigelow (and anyone else I\u2019ve missed) for some brilliant discussions on this topic over the last few weeks.<\/em><\/p> <p><strong>More Resources:<\/strong><\/p> <hr\/> <p><em>Featured Image: Collagery\/Shutterstock<\/em><\/p> <\/div> <p>Generative AI,SEO#Web #Growing #Layer #sejournal #demirie1783415018<\/p> ","protected":false},"excerpt":{"rendered":"<p>The last few weeks have been noisy. Google shipped something called the Open Knowledge Format. Then Google Developers announced the Agentic Resource Discovery (ARD) specification. Meanwhile, every SEO LinkedIn feed is lit up with someone either declaring markdown the future of the web or explaining why you should ignore all of it. The truth, as [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":11149,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16],"tags":[38377,24980,11169,80,345],"class_list":["post-11148","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-accessibility","tag-demirie","tag-growing","tag-layer","tag-sejournal","tag-web"],"acf":[],"_links":{"self":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts\/11148","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=11148"}],"version-history":[{"count":0,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts\/11148\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/media\/11149"}],"wp:attachment":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=11148"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=11148"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=11148"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}