Google's Top Crawling Challenges In 2025

Gary Illyes, along with Martin Splitt, of Google posted a podcast explaining the top crawling challenges Google noticed amongst its 2025 year of crawling. The top challenges Google had with crawling included faceted navigation, action parameters, irrelevant parameters, calendar parameters and other “weird” parameters.

Here is the podcast embed:

These issues with crawling can impact a site’s performance because bots might go in a loop of the site and cause server issues because of the load the bot is putting on the server resources. And as Gary said, “once it discovers a set of URLs, it cannot make a decision about whether that URL space is good or not unless it crawled a large chunk of that URL space.”

Here is how Gary Illyes put the challenges by percentage:

• Faceted Navigation was 50%: This occurs on websites (often e-commerce) that allow users to filter and sort items by various dimensions like price, category, or manufacturer. These combinations create a massive number of unique URL patterns. Googlebot may try to crawl all of them to determine their value, potentially crashing the server or rendering the site useless for users due to heavy load.
• Action Paramters was 25%: These are URL parameters that trigger a specific action rather than changing the page content significantly. Common examples include parameters like ?add_to_cart=true or ?add_to_wishlist=true. Adding these parameters doubles or triples the URL space (e.g., a product page URL vs. the same URL with an “add to cart” parameter), causing the crawler to waste resources on identical content. These are often injected by CMS plugins, such as those for WordPress.
• Irrelevant Parameters was 10%: Like UTM tracking parameters or parameters that Googlebot generally ignores or finds irrelevant to the content’s state, such as Session IDs and UTM parameters. Googlebot struggles to determine if these random strings change the page content. It may crawl aggressively to test whether the parameters are meaningful, especially if standard naming conventions.
• WordPress Plugins or Widgets was 5%: Where maybe these widgets add sort of event tracking or other things. This was a big challenge for Google because of the open source nature of it.
• Other “Weird Stuff” was 2%: This catch-all category includes rare technical errors, such as accidentally double-encoding URLs (e.g., percent-encoding a URL that was already encoded). The crawler decodes the URL once but is left with a still-encoded string, often leading to errors or broken pages that the crawler attempts to process anyway.

This was an interesting podcast – here is the transcript if you want it.

Forum discussion at X.

Image credit Lizzi Sassman