Search Console is reporting soft 404s on pages I never created
A catch-all that answers unknown paths with your homepage and a 200 turns every typo and dead link into an indexable duplicate of your homepage. Test it with one curl against a made-up path. The fix is a real 404 page, a robots.txt and a sitemap.xml.
A coverage report full of URLs we never made
Search Console filled with soft 404s and duplicate pages that nobody had created. The URLs in the report were typos, dead links and stale paths, and every one of them was a real indexable page as far as Google was concerned, because every one of them returned a copy of the homepage.
On one site in the network, checked on 2026-09-19, every unknown path returned HTTP 200 with the homepage. There was no robots.txt, no sitemap.xml and no 404 page. That combination is not a handful of bad URLs. It is a soft 404 across the entire URL space, because the site had no way of saying that any path was wrong.
The reports that follow look strange until the cause is clear. Duplicate pages appear for URLs that were never linked from the site. Pages you did not write show up as candidates for indexing. The count in the report grows without anyone publishing anything, since it tracks the URLs Google has tried rather than the pages that exist. is the figure worth recording before you change anything, so you have a baseline to compare against afterwards.
A catch-all route answers every address with 200
A catch-all route answers unknown paths with the homepage and a 200. That is the intended behaviour for client side routing, where an unmatched path has to load the application shell before the app can read the URL and decide what to render.
A crawler cannot see that intent. It sees a request, a success status and a page of content. So every typo in an inbound link, every dead URL left over from an old structure, and every mangled path some scraper invented becomes a page that exists, returns successfully, and duplicates the homepage word for word. The site is effectively generating an unbounded number of copies of one page, and it is doing it on demand.
Google has a name for this. It reports these URLs as "Soft 404" in the Pages report under Indexing. The label describes a page that should not exist and is returning a success status instead of an error.
Two things get confused with it, and the distinctions matter because the fixes are different.
A thin page is not a soft 404. A thin page is a real page with too little on it, and the remedy is content or consolidation. A soft 404 is a page that should not exist at all, and the remedy is a correct status code.
A deliberate noindex is not a soft 404 either. A noindexed page returns 200 and asks not to be indexed. A soft 404 returns 200 and asks to be indexed, while containing something that is not a page. The difference is the request the page is making of the crawler, not the status it returns.
Request one nonsense URL
One command tells you whether any site has this, and it is worth running against every site you operate rather than only the one in the report:
curl -o /dev/null -w "%{http_code}" https://<domain>/404testA 404 is the correct answer. A 200 means every wrong URL on that domain is a live page, and the Search Console report is a lagging indicator of a problem that is already fully present.
Confirm what is being served by looking at the body rather than the status, since a 200 on its own does not say which page answered:
curl -s https://<domain>/404test | head -c 80Then check whether the other two signals exist at all, because their absence is what turns a routing quirk into a site-wide indexing problem:
curl -sI https://<domain>/robots.txt | head -1
curl -sI https://<domain>/sitemap.xml | head -1The three changes we made
These are the three changes that were made on the affected site, in the order they matter.
- Add a real
404.htmlso unknown paths return a genuine 404 instead of the homepage. The status code is the primary signal and it is the one a crawler acts on. - Mark that page
noindex, follow. The status code does the work, and the meta directive covers the case where the page is reached and read anyway, whilefollowkeeps any links on it usable. - Add a
robots.txtthat allows everything and points at the sitemap. An absentrobots.txtis not a permissive one, it is a missing file, and it leaves nowhere to declare the sitemap. - Add a
sitemap.xmllisting the real URLs. This is the positive half of the fix. The 404 page says which paths are not pages, and the sitemap says which ones are. - Re-run the single-command test and confirm it now returns 404, then confirm the body of an unknown path is no longer the homepage.
- Go back to the Pages report under Indexing and ask Google to recheck the URLs listed as Soft 404. The report will lag the fix, so expect the count to fall over
rather than immediately.
Test a missing URL before launch
Run the 404 test on every site at launch, before anything is submitted for indexing. It costs one command and it is the difference between a clean index and a report full of pages you never wrote.
Treat a 404 page, a robots.txt and a sitemap.xml as launch requirements rather than as polish. The site described here was live and returning 200 for everything, with none of the three present, which is a configuration that behaves perfectly for visitors and badly for crawlers.
If you need client side routing, scope the catch-all to the app's own routes and let everything else fall through to the 404. An unscoped catch-all is the mechanism behind this failure, and it is also the mechanism behind a deleted endpoint appearing to still exist.
Check the test again after any change to routing or middleware. A catch-all can be reintroduced by a change that has nothing to do with indexing, and nothing will complain, because from a visitor's point of view the site still works.
The distinction to hold onto. A thin page is a real page with too little on it. A soft 404 is not a page at all and is returning a success status. A noindexed page returns 200 and asks not to be indexed, which is the opposite request from the one a soft 404 makes.
How long Google takes to forget the phantoms
Returning a correct 404 does not remove anything already indexed. Those URLs were crawled and accepted while the site was answering 200, and they leave the index on Google's schedule rather than yours.
A sitemap does not cause indexing. It declares which URLs are real, and it makes the absence of everything else meaningful. Pages still have to earn their place.
None of this addresses thin pages. If a real page has too little on it, a 404 page and a sitemap change nothing about how it is judged, and the report will keep saying so under a different label.
The fix also does not help any other site you run. This was found on one site in the network by running a single command, which means the command is the transferable part. Run it on each domain separately, because each one has its own routing configuration and its own catch-all.
Finally, adding a 404 page does not tell you which unmatched paths your application still needs served as the shell. Tighten the catch-all deliberately, and confirm the app's own routes still load before treating the change as finished.
Questions about soft 404 reports
What is a soft 404?
A page that should not exist and is returning a success status instead of an error. Google reports these as "Soft 404" in the Pages report under Indexing.
Why is Search Console showing duplicate pages I never created?
A catch-all that answers unknown paths with your homepage and a 200 turns every typo, dead link and stale URL into a real indexable page that duplicates your homepage. The crawler sees a success status and a page of content, so it treats the URL as real.
How do I test for this in one command?
Run curl -o /dev/null -w "%{http_code}" https://
Is a soft 404 the same as a thin page?
No. A thin page is a real page with too little on it, and the remedy is content. A soft 404 is a page that should not exist at all, and the remedy is a correct status code.
How is this different from a page I deliberately set to noindex?
A noindexed page returns 200 and asks not to be indexed. A soft 404 returns 200 and asks to be indexed, while containing something that is not a page.
What exactly was changed to fix it?
A real 404.html marked noindex, follow so unknown paths return a genuine 404, a robots.txt that allows everything and points at the sitemap, and a sitemap.xml listing the real URLs.