Google Request Indexing Issue With Robots.txt Files?

Google may have a bug or issue with using the request indexing or recrawling your robots.txt files within Google Search Console. Supposedly, the feature is not working properly for all sites, but it is working for some sites as well.

This was spotted by Jay Cooper who posted about it on Bluesky and wrote, “Is there a known issue with the “request a recrawl” function in the robots.txt report at the moment? It throws an error every time for me, across several properties. Wondering whether it’s on my end or a temporary glitch.”

Here is the screenshot:

You can see, this is resulting in an error that says Re-crawl request failed and then says “Unknown error. Please wait a moment and try again.”

John Mueller from Google replied first saying it worked for his site but then later posted on Bluesky, “.. ah, I see the error on some of my other sites though. I’ll poke around.”

So if you are trying to use this tool on your robots.txt and are having issues, you may not be alone.

Here are those posts on Bluesky:

Forum discussion at Bluesky.