{"id":8202,"date":"2026-05-16T00:33:44","date_gmt":"2026-05-15T16:33:44","guid":{"rendered":"http:\/\/longzhuplatform.com\/?p=8202"},"modified":"2026-05-16T00:33:44","modified_gmt":"2026-05-15T16:33:44","slug":"how-soft-404s-and-indexing-issues-caused-a-90-traffic-collapse","status":"publish","type":"post","link":"http:\/\/longzhuplatform.com\/?p=8202","title":{"rendered":"How soft 404s and indexing issues caused a 90% traffic collapse"},"content":{"rendered":"

<\/p>

When a website migration goes wrong, the consequences can be a devastating loss of organic traffic and revenue. But what happens when the damage isn\u2019t immediately visible? What if Google is silently deprioritizing your content, page by page, until your traffic has evaporated?<\/p>

This is the case study of how a multinational media organization lost 90% of its traffic following a domain migration, and how addressing a seemingly harmless technical issue \u2014 soft 404 errors \u2014 helped unlock suppressed traffic potential across 13 country-specific domains.<\/p>

While this case study examines events from 2021\u20132023, the lessons learned remain timeless and directly applicable to any site facing indexing challenges today.<\/p>

The catastrophic drop<\/h2>

In January, 2022, the Brazilian localization of a cryptocurrency news website completed a domain migration. After the transition, traffic didn\u2019t just drop \u2014 it plummeted. Comparing December 2021 to December 2022, both sessions and pageviews had fallen approximately 90% year-over-year<\/strong>.<\/p>

\"Image\"Image<\/figure> <\/div>

According to Google Search Console data, the old domain (xx.com.br) was receiving between 15,000 to 25,000 clicks per day before migration. After migrating to the new subdomain structure (br.xx.com) in January, traffic collapsed and never recovered. It stabilized at around 2,000 to 4,000 clicks per day \u2014 a sustained loss that persisted for over a year.<\/p>

\"Image\"Image<\/figure> <\/div>

The migration coincided with three major Google algorithm updates in June 2021: the core update, spam update, and page experience update. While these updates caused the expected temporary volatility, the Brazilian site showed no signs of recovery.<\/p>

Your customers search everywhere. Make sure your brand shows up<\/span>. <\/p>

The SEO toolkit you know, plus the AI visibility data you need. <\/p> <\/p><\/div>

Start Free Trial<\/span> <\/p>

Get started with<\/p>

\"Semrush\"Semrush <\/div> <\/p><\/div> <\/p>

The migration problem: More than just redirects<\/h2>

Domain migrations typically show an initial traffic drop as Google recrawls and reassesses the site. That\u2019s expected. <\/p>

Normally, this traffic recovers within weeks or months. In this case, there were no signs of recovery.<\/p>

The root cause? The old domain continued to be crawled by Google long after the migration. <\/p>

According to the team\u2019s analysis, proper redirect implementation and technical migration protocols weren\u2019t fully implemented, causing Google to split its crawl budget between two domains rather than consolidating authority on the new one.<\/p>

In mid-August 2022, after addressing the migration issues with the SEO and IT teams, there was a subtle uptick \u2014 a peak of 12 clicks and 37 impressions on Aug. 29, 2022. While modest, this represented the first signs of recovery and indicated that Google was beginning to properly recognize the new domain.<\/p>

\"Image\"Image<\/figure>

Using Facebook Prophet forecasting on pre-migration data, the team estimated that without the migration issues, the Brazilian site would have exceeded 2 million monthly clicks by early 2022. Instead, it was generating a fraction of that traffic.<\/p>

\"Image\"Image<\/figure> <\/div>

Understanding the indexing bottleneck<\/h2>

While fixing the migration was critical, it revealed a deeper problem affecting not just Brazil, but all 13 of the site\u2019s country domains: a massive indexing backlog.<\/p>

Google\u2019s page processing follows four stages:<\/p>

  • Crawl<\/strong>: Google discovers and reads pages.<\/li>
  • Render<\/strong>: The page code is rendered.<\/li>
  • Index<\/strong>: Pages wait in a queue to be stored in Google\u2019s index.<\/li>
  • Rank<\/strong>: Pages appear in search results with rankings.<\/li> <\/ul>

    The Brazilian site was taking an average of 2 minutes for Google to crawl new articles (an acceptable amount of time for a news site). However, indexing these articles was taking 24 hours. For time-sensitive cryptocurrency news, this delay was catastrophic. By the time the site\u2019s articles were indexed, the news cycle had already moved on.<\/p>

    The scale of the site migration problem: 513,000 crawled, but not indexed, pages<\/strong><\/h2>

    In January 2023, Google Search Console revealed alarming indexing issues across all domains:<\/p>

    • Crawled \u2013 currently not indexed:<\/strong> 513,369 pages (Brazil alone)<\/li>
    • Soft 404:<\/strong> 1,193 pages and growing rapidly<\/li>
    • Alternate page with proper canonical tag:<\/strong> 2,532 pages<\/li>
    • Discovered \u2013 currently not indexed: <\/strong>524 pages<\/li> <\/ul>
      \"Image\"Image<\/figure> <\/div>

      The \u201cCrawled \u2013 currently not indexed\u201d issue was particularly concerning. These were pages that Google had successfully crawled but chose not to index. This typically happens when Google considers a page low-quality, duplicate, or not worth the crawl budget.<\/p>

      \"Image\"Image<\/figure> <\/div>

      Upon investigation, the team discovered that converter pages (e.g., \u201c\/usd-to-thor?amount=250\u201d or \u201c\/eur-to-signaturechain?amount=1000\u201d) were being automatically generated at scale. These thin content pages were consuming Google\u2019s crawl budget, causing it to deprioritize the entire domain.<\/p>

      The soft 404 time bomb<\/h2>

      While fixing the migration and removing low-quality pages was important, the most insidious issue was the proliferation of soft 404 errors.<\/p>

      A soft 404 occurs when a page returns a 200 (success) status code but actually contains no meaningful content \u2014 essentially a \u201cpage not found\u201d that doesn\u2019t properly signal its emptiness to search engines. Unlike hard 404s, which clearly communicate that the page doesn\u2019t exist, soft 404s confuse search engines and waste crawl budgets.<\/p>

      \"Image\"Image<\/figure> <\/div>

      The data revealed this wasn\u2019t isolated to Brazil. Soft 404 errors were growing exponentially across multiple domains:<\/p>

      • xx.com (main site): 90,400 affected pages<\/li>
      • es.xx.com (Spain): 17,700 pages<\/li>
      • kr.xx.com (Korea): 15,400 pages<\/li>
      • fr.xx.com (France): 15,100 pages<\/li>
      • de.xx.com (Germany): 8,010 pages<\/li> <\/ul>
        \"Image\"Image<\/figure>

        Specifically for France, Google Search Console data showed a direct correlation: As soft 404 errors began accumulating in October 2022, total crawl requests dropped from 60,000\u201370,000 per day to just 20,000\u201330,000 per day. Google was literally giving up on crawling the site efficiently.<\/p>

        The crawl budget crisis<\/strong><\/h2>

        The concept of crawl budget is critical to understanding why soft 404s matter so much.<\/p>

        Search engines allocate a finite amount of resources to crawl each website. If Google wastes time crawling broken, empty, or duplicate pages, it has less capacity to discover and index your valuable content. <\/p>

        For news sites publishing dozens of articles daily, this creates a vicious cycle: New content doesn\u2019t get indexed quickly, engagement drops, Google further reduces crawl budget, and the problem compounds.<\/p>

        In January 2023, Google was wasting significant resources crawling pages that provided no value. This meant:<\/p>

        • Slower indexing of new, timely content.<\/li>
        • Reduced visibility in search results.<\/li>
        • Lost traffic opportunities.<\/li>
        • Degraded domain authority in Google\u2019s eyes.<\/li> <\/ul>

          The systematic fix: Addressing root causes of site migration problems<\/h2>

          Starting Jan. 31, 2023, the team implemented a comprehensive technical SEO remediation plan focused on three priorities:<\/p>

          Urgent: Soft 404 resolution<\/h3>

          The team identified the source of soft 404 errors and implemented proper HTTP status codes. <\/strong>Pages that truly didn\u2019t exist began returning proper 404 or 410 status codes. <\/strong>Pages with content were fixed to render properly.<\/p>

          High priority: Crawl budget optimization<\/h3>
          • Removed or noindexed automatically generated currency converter pages.<\/li>
          • Implemented stricter URL parameter handling.<\/li>
          • Used robots.txt to block low-value URL patterns.<\/li>
          • Set up proper canonicalization for variant pages.<\/li> <\/ul>

            Medium priority: Core Web Vitals<\/h3>

            While user experience metrics were important, the team recognized that fixing indexing issues would have a more immediate impact than optimizing page speed. Core Web Vitals improvements were addressed, but not at the expense of resolving indexing bottlenecks.<\/p>

            <\/p>

            Get the newsletter search marketers rely on.<\/p> <\/p><\/div> <\/p><\/div> <\/div>

            <\/p>

            The results: Dramatic recovery across all domains<\/h2>

            Weeks after implementing the fixes, the impact was measurable:<\/p>

            \"Image\"Image<\/figure> <\/div>

            Brazil (br.xx.com)<\/h3>
            • Crawled \u2013 currently not indexed:<\/strong> Dropped from 513,000 to 220,000 pages (57% reduction).<\/li>
            • Soft 404 errors:<\/strong> Reduced from 1,193 to 370 pages (69% reduction).<\/li>
            • Traffic recovery:<\/strong> Visible upward trajectory starting early 2023.<\/li> <\/ul>
              \"Image\"Image<\/figure>

              Germany (de.xx.com)<\/h3>
              • Indexed pages:<\/strong> Increased from ~150,000 to 370,748.<\/li>
              • Total clicks:<\/strong> Rose from ~8,000\/day average to sustained 12,000-15,000\/day.<\/li>
              • Google Discover traffic share:<\/strong> Jumped from 42% to 58%.<\/li> <\/ul>

                Poland (pl.xx.com)<\/h3>
                • Indexed pages: <\/strong>Grew from ~100,000 to 135,556.<\/li>
                • Total clicks:<\/strong> Increased significantly with multiple traffic spikes above 30,000\/day.<\/li>
                • Google Discover traffic share:<\/strong> Rose from 15% to 86%.<\/li> <\/ul>
                  \"Image\"Image<\/figure> <\/div>

                  Spain (es.xx.com)<\/h3>
                  • Google Discover clicks:<\/strong> Increased from ~450,000 to 912,721 total.<\/li>
                  • Traffic distribution:<\/strong> Discover now represents 65% of total traffic.<\/li> <\/ul>

                    All domains combined<\/h3>
                    \"Image\"Image<\/figure>

                    By late April 2023, soft 404 errors across all domains had dropped from a peak of approximately 120,000 pages to under 20,000 \u2014 an 83% reduction.<\/p>

                    Most remarkably, the biggest traffic gains came from Google Discover \u2014 Google\u2019s personalized content recommendation feed. As indexing health improved, Google began trusting the domains enough to recommend their content more aggressively to users.<\/p>

                    The Core Web Vitals paradox<\/h2>

                    Interestingly, improvements to Core Web Vitals (page speed, interactivity, and visual stability) showed mixed results<\/strong>:<\/p>

                    Desktop improvements:<\/strong><\/p>

                    • Germany: 25.1% \u2192 97.1% good URLs<\/li>
                    • Poland: 20.5% \u2192 68.9% good URLs<\/li>
                    • Korea: 15% \u2192 84.6% good URLs<\/li> <\/ul>

                      Mobile challenges:<\/strong><\/p>

                      • Brazil: 0% \u2192 0% (no improvement)<\/li>
                      • Argentina: 0% \u2192 0%<\/li>
                      • Thailand: 0% \u2192 0%<\/li>
                      • Korea: 93.4% \u2192 0.5% (severe regression)<\/li>
                      • Turkey: 94% \u2192 0% (severe regression)<\/li> <\/ul>

                        The team\u2019s hypothesis: Core Web Vitals performance is heavily influenced by regional factors like CDN proximity, server location, network quality, and device capabilities. Countries with poor mobile infrastructure or greater server distance showed minimal improvement despite technical optimizations.<\/p>

                        This reinforced an important lesson: Not all technical SEO issues affect all markets equally. A one-size-fits-all approach would have wasted resources by optimizing for metrics that couldn\u2019t improve without infrastructure investment, while the real wins came from addressing indexing fundamentals.<\/p>

                        Key technical SEO lessons<\/h2>

                        1. Indexing issues trump almost everything else<\/h3>

                        No amount of content quality, backlinks, or page speed optimization matters if Google isn\u2019t indexing your pages. Before optimizing what\u2019s visible, ensure your content is actually being indexed.<\/p>

                        2. Soft 404s are silent killers<\/h3>

                        Unlike hard 404s that immediately alert you to problems, soft 404s quietly accumulate, degrading your crawl budget until you notice traffic declining. Regular monitoring of Google Search Console\u2018s \u201cPages\u201d report is essential.<\/p>

                        3. Domain migrations require exhaustive validation<\/h3>

                        The Brazilian site\u2019s migration issues persisted for over a year. A proper migration protocol should include:<\/p>

                        • Complete redirect mapping verification.<\/li>
                        • Confirmation of old domain deindexing.<\/li>
                        • Search Console property setup and validation.<\/li>
                        • Multi-week monitoring of both old and new domains.<\/li>
                        • Crawl rate and indexing speed tracking.<\/li> <\/ul>

                          4. Crawl budget is real for high-volume sites<\/h3>

                          For sites publishing 10+ articles daily across multiple domains, crawl budget optimization is not optional. Automatically generated pages, URL parameters, and infinite scroll implementations can quickly consume available crawl resources.<\/p>

                          5. Regional differences demand regional solutions<\/h3>

                          Core Web Vitals data showed that Brazil, Argentina, and Thailand couldn\u2019t achieve the same performance as European markets. Instead of forcing uniform standards, prioritize fixes tailored to each market that can actually succeed.<\/p>

                          6. Google Discover is increasingly critical<\/h3>

                          For news and timely content publishers, Google Discover accounts for a substantial share of traffic in some markets. But Discover only promotes content from sites Google trusts \u2014 and technical issues like soft 404s directly erode that trust.<\/p>

                          Practical site migration implementation guide<\/h2>

                          For teams facing similar challenges, here\u2019s a systematic approach:<\/p>

                          Weeks 1-2: Audit and prioritize<\/h3>
                          • Access Google Search Console for all properties.<\/li>
                          • Export \u201cPage indexing\u201d reports for all domains.<\/li>
                          • Identify the scale of each issue category.<\/li>
                          • Calculate the trend (growing, stable, or declining).<\/li>
                          • Prioritize based on issue volume and growth rate.<\/li> <\/ul>

                            Weeks 3-4: Fix soft 404s<\/h3>
                            • Sample 20\u201330 URLs from the soft 404 report.<\/li>
                            • Identify common patterns (empty pages, broken functionality, etc.).<\/li>
                            • Implement proper HTTP status codes (404, 410, or fix the content).<\/li>
                            • Validate fixes in Google Search Console.<\/li>
                            • Monitor for reduction in affected pages.<\/li> <\/ul>

                              Weeks 5-8: Address crawled but not indexed<\/h3>
                              • Analyze URLs to identify auto-generated content.<\/li>
                              • Implement robots.txt rules or noindex tags for low-value pages.<\/li>
                              • Review and strengthen internal linking to important pages.<\/li>
                              • Ensure proper canonicalization across variants.<\/li>
                              • Request reindexing via Search Console for key pages.<\/li> <\/ul>

                                Weeks 9-12: Monitor and optimize<\/h3>
                                • Track indexing coverage weekly.<\/li>
                                • Monitor crawl rate changes in Search Console.<\/li>
                                • Measure organic traffic recovery.<\/li>
                                • Identify remaining outlier issues.<\/li>
                                • Document learnings for future migrations.<\/li> <\/ul>

                                  Calculating the traffic loss from migration issues<\/h2>

                                  How significant was this suppressed traffic opportunity?<\/p>

                                  According to Facebook Prophet forecasting based on pre-migration data, the Brazilian site was trending toward 20,000+ daily clicks. At the time of fix implementation in early 2023, it was receiving approximately 5,000\u20137,000 daily clicks. This represented roughly 6575% of potential traffic being suppressed \u2014 or conversely, the site was only achieving 25\u201335% of its forecasted potential.<\/p>

                                  More broadly, across all 13 domains, the soft 404 and indexing issues prevented approximately 500,000 pages from being indexed. Given average click-through rates for indexed pages, this represented millions of potential monthly impressions and hundreds of thousands of potential clicks being left on the table.<\/p>

                                  See the complete picture<\/span> of your search visibility. <\/p>

                                  Track, optimize, and win in Google and AI search from one platform. <\/p> <\/p><\/div>

                                  Start Free Trial<\/span> <\/p>

                                  Get started with<\/p>

                                  \"Semrush\"Semrush <\/div> <\/p><\/div> <\/p>

                                  Technical debt compounds<\/h2>

                                  The most important lesson from this case study is that technical SEO issues don\u2019t stay static \u2014 they compound. What starts as a few hundred soft 404s becomes thousands, then tens of thousands. <\/p>

                                  Google\u2019s response isn\u2019t immediate punishment, but gradual deprioritization. Traffic doesn\u2019t crash overnight; it bleeds slowly.<\/p>

                                  For the Brazilian site, it took over a year to recognize the full scope of the problem. During that year, competitors filled the gap, topical authority eroded, and recovery became exponentially harder.<\/p>

                                  The good news? Once identified and systematically addressed, these issues are fixable. Within 12 weeks of implementing the remediation plan, every domain showed measurable improvement. Some saw traffic double or triple.<\/p>

                                  Technical SEO is often seen as unglamorous maintenance work. But as this case demonstrates, it\u2019s the foundation upon which all other optimization rests. Before worrying about AI-generated content, E-E-A-T signals, or the latest algorithm update, ensure Google can actually find, crawl, and index your content.<\/p>

                                  Because the best content in the world is worthless if it\u2019s trapped outside search engine indexes.<\/p> <\/div>

                                  Contributing authors are invited to create content for Search Engine Land and are chosen for their expertise and contribution to the search community. Our contributors work under the oversight of the editorial staff and contributions are checked for quality and relevance to our readers. Search Engine Land is owned by Semrush. Contributor was not asked to make any direct or indirect mentions of Semrush. The opinions they express are their own.<\/em> <\/p>

                                  Opinion#soft #404s #indexing #issues #caused #traffic #collapse1778862824<\/p> ","protected":false},"excerpt":{"rendered":"

                                  When a website migration goes wrong, the consequences can be a devastating loss of organic traffic and revenue. But what happens when the damage isn\u2019t immediately visible? What if Google is silently deprioritizing your content, page by page, until your traffic has evaporated? This is the case study of how a multinational media organization lost […]<\/p>\n","protected":false},"author":1,"featured_media":8203,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[18],"tags":[30813,2929,9977,9249,1747,155,30812,441],"class_list":["post-8202","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-careers","tag-404s","tag-caused","tag-collapse","tag-indexing","tag-issues","tag-opinion","tag-soft","tag-traffic"],"acf":[],"_links":{"self":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts\/8202","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=8202"}],"version-history":[{"count":0,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/posts\/8202\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=\/wp\/v2\/media\/8203"}],"wp:attachment":[{"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=8202"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=8202"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/longzhuplatform.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=8202"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}