{"id":8314,"date":"2026-05-17T20:14:26","date_gmt":"2026-05-17T12:14:26","guid":{"rendered":"http:\/\/longzhuplatform.com\/?p=8314"},"modified":"2026-05-17T20:14:26","modified_gmt":"2026-05-17T12:14:26","slug":"google-agent-the-webs-new-visitor-just-got-an-identity-via-sejournal-slobodanmanic","status":"publish","type":"post","link":"http:\/\/longzhuplatform.com\/?p=8314","title":{"rendered":"Google-Agent: The Web\u2019s New Visitor Just Got An Identity via @sejournal, @slobodanmanic"},"content":{"rendered":"<p><\/p> <div id=\"narrow-cont\"> <p>On March 20, 2026, Google quietly\u00a0added a new entry\u00a0to its official list of web fetchers. Not a crawler. Not a training bot. An agent.<\/p> <p>Google-Agent is the user agent string for AI systems running on Google infrastructure that browse websites on behalf of users. When someone asks an AI assistant to research a product, fill out a form, or compare options across websites, Google-Agent is the thing that actually visits the page.\u00a0Project Mariner, Google\u2019s experimental AI browsing tool, is the first product using it.<\/p> <p>This is not Googlebot. Googlebot crawls the web continuously, indexing pages for search. Google-Agent only shows up when a human asks it to. That distinction changes everything about how it operates.<\/p> <h2 id=\"robotstxt-does-not-apply\">Robots.txt Does Not Apply<\/h2> <p>Google classifies Google-Agent as a\u00a0user-triggered fetcher. The category includes tools like Google Read Aloud (text-to-speech), NotebookLM (document analysis), and Feedfetcher (RSS). All of them share one property: a human initiated the request. Google\u2019s position is that user-triggered fetchers \u201cgenerally ignore robots.txt rules\u201d because the fetch was requested by a person.<\/p> <p><iframe class=\"sej-iframe-auto-height\" id=\"in-content-iframe\" scrolling=\"no\" src=\"https:\/\/www.searchenginejournal.com\/wp-json\/sscats\/v2\/tk\/Middle_Post_Text\"><\/iframe><\/p> <p>The logic: If you type a URL into Chrome, the browser fetches the page regardless of what robots.txt says. Google-Agent operates on the same principle. The agent is the user\u2019s proxy, not an autonomous crawler.<\/p> <p>This is a meaningful departure from how OpenAI and Anthropic handle similar traffic.\u00a0ChatGPT-User\u00a0and\u00a0Claude-User\u00a0both function as user-triggered fetchers, but they respect robots.txt directives. If you block ChatGPT-User in robots.txt, ChatGPT won\u2019t fetch your page when a user asks it to browse. Google made a different call.<\/p> <p>Website owners who relied on robots.txt as a universal access control mechanism now have a gap. If you need to restrict access from Google-Agent, you\u2019ll need server-side authentication or access controls. The same tools you\u2019d use to block a human visitor.<\/p> <h2 id=\"cryptographic-identity-web-bot-auth\">Cryptographic Identity: Web Bot Auth<\/h2> <p>The more significant development is buried in a single line of Google\u2019s documentation: Google-Agent is experimenting with the <code>web-bot-auth<\/code> protocol using the identity <code>https:\/\/agent.bot.goog<\/code>.<\/p> <p>Web Bot Auth is an IETF draft standard that works like a digital passport for bots. Each agent holds a private key, publishes its public key in a directory, and cryptographically signs every HTTP request. The website verifies the signature and knows, with cryptographic certainty, that the visitor is who it claims to be.<\/p> <p>User agent strings can be spoofed by anyone. Web Bot Auth cannot. Google adopting this protocol, even experimentally, signals where agent identity is heading. Akamai, Cloudflare, and Amazon (AgentCore Browser) already support it. Google brings the critical mass.<\/p> <p>This matters because the web is about to have an identity problem. As agent traffic increases, websites need to distinguish between legitimate AI agents acting on behalf of real users and scrapers pretending to be agents. IP verification helps, but cryptographic signatures scale better and are harder to fake.<\/p> <h2 id=\"what-this-means-for-your-website\">What This Means For Your Website<\/h2> <p>Google-Agent creates a three-tier visitor model for the web:<\/p> <ol> <li><strong>Human visitors<\/strong> browsing directly.<\/li> <li><strong>Crawlers<\/strong> indexing content for search and training (Googlebot, GPTBot, Google-Extended).<\/li> <li><strong>Agents<\/strong> acting on behalf of specific humans in real time (Google-Agent, ChatGPT-User, Claude-User).<\/li> <\/ol> <p>Each tier has different access rules, different intentions, and different expectations. A crawler wants to index your content. An agent wants to complete a task. It might be reading a product page, comparing prices, filling out a contact form, or booking an appointment.<\/p> <p>Here\u2019s what to do now:<\/p> <p><strong>Monitor your logs.<\/strong> Google-Agent identifies itself with a user agent string containing <code>compatible; Google-Agent<\/code>. Google publishes IP ranges for verification. Start tracking how often agents visit, which pages they hit, and what they attempt to do.<\/p> <p><strong>Check your CDN and firewall rules.<\/strong> If your security tools aggressively block non-browser traffic, Google-Agent may be getting rejected before it reaches your server. Verify that Google\u2019s published IP ranges are permitted.<\/p> <p><strong>Test your forms and flows.<\/strong> Google-Agent can submit forms and navigate multi-step processes. If your checkout, booking, or contact forms rely on JavaScript patterns that confuse automated systems, agent visitors will fail silently. Semantic HTML and clear labels remain the foundation.<\/p> <p><strong>Accept that robots.txt is no longer a complete access control tool.<\/strong> For content you genuinely need to restrict, use authentication. robots.txt was designed for crawlers. The agent era needs different boundaries.<\/p> <h2 id=\"the-hybrid-web-isnt-coming-its-logged\">The Hybrid Web Isn\u2019t Coming. It\u2019s Logged<\/h2> <p>A year ago, the idea that AI agents would browse websites alongside humans was a conference talk prediction. Today, it has a user agent string, published IP ranges, a cryptographic identity protocol, and an entry in Google\u2019s official documentation.<\/p> <p>The web didn\u2019t split into human and machine. It merged. Every page you publish now serves both audiences simultaneously, and Google just made it possible to see exactly when the non-human audience shows up.<\/p> <p><strong>More Resources:<\/strong><\/p> <hr\/> <p><em>This post was originally published on No Hacks.<\/em><\/p> <hr\/> <p><em>Featured Image: Summit Art Creations\/Shutterstock<\/em><\/p> <\/div> <p>Generative AI,SEO#GoogleAgent #Webs #Visitor #Identity #sejournal #slobodanmanic1779020066<\/p> ","protected":false},"excerpt":{"rendered":"<p>On March 20, 2026, Google quietly\u00a0added a new entry\u00a0to its official list of web fetchers. Not a crawler. Not a training bot. An agent. Google-Agent is the user agent string for AI systems running on Google infrastructure that browse websites on behalf of users. When someone asks an AI assistant to research a product, fill [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":8315,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16],"tags":[19847,24992,80,18507,31450,31449],"class_list":["post-8314","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-accessibility","tag-googleagent","tag-identity","tag-sejournal","tag-slobodanmanic","tag-visitor","tag-webs"],"acf":[],"_links":{"self":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts\/8314","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=8314"}],"version-history":[{"count":0,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts\/8314\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/media\/8315"}],"wp:attachment":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=8314"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=8314"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=8314"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}