Fix Crawl Anomaly Errors in Google Search Console:
Crawl Anomaly what does this mean in Google search console :
In Google search console crawl anomaly error is an crawl error which is an unspecified anomaly error occurred while Google Bot fetching your website URL throwing and error level of 400 or 500 levels are served to Google bot.
Failed Crawl Anomaly Error Means :
Fetch failed crawl anomaly error means the fetched webpage or url faced an error level of 4xx error or 5xx level error response code and Google bot left the page leaving behind not crawling the url and flagged as crawl anomaly error in Google search console and the page will not be a part of Google indexed until crawl anomaly error is resolved as Google bot facing problem while fetching and downloading the document.
What to do to Solve Crawl Anomaly Error and Fix:
IF you see your website facing crawl issue refers to a fetch failed crawl anomaly error then you need to check with you server logs first and see why crawl anomaly error occurred and figure out which type of error level response code i.e. 4xx level or 5xx level error. To do this the best way is debugging crawl anomaly error by url inspection tool and url inspection tool gives you all details about your webpage and tells you how google bot crawled the webpage including http response code,page is mobile frinedly, chrome browser issues, snapshot of how google bot sees your webpage and how google bot sees your webpage, canonical url, google declared canonical, AMP issues and everything.
If it is with the 404 then you need to check any redirection happening with the URL which is taking too long to return 200 ok response codes, which sometimes have multiple redirection loops.
If it’s a 500 server error then you need to make sure your website fast by reducing number of requests to the server which is making or slowing down Google Bot to take time and return a 500 error which is at server level and let Google bot crawl the webpage fast.
The main reason why fetch will be failed and raise crawl anomaly error in Google search console is Google doesn’t want to keep pages which take long time to load or time spent downloading the page is too high and Google is making web faster with mobile first indexing which means the mobile version of the webpage should load very fast so that user get what he is looking for in Google search index.
Many of the website owners are seen facing this Google crawl anomaly error when their website is moved and enable to mobile first indexing. Google says mobile first indexing is only the part of responsive and mobile friendliness at the same time Google also expects to return fast web pages from Google mobile version of index.
Google Crawl anomaly Error:
The URL Error Anomaly Alerts Can Include:
1. Server error
2. Soft 404
3. Access denied
4. Not found
5. Not followed
6. Page is 404
7. Not Mobile friendly
8. Which makes your webpage or url loads slow
Understand & Know What is Crawl Anomaly Error:
It’s easy and simple, if URL throws 404 page or 500 internal server error that will be a crawl anomaly error. To check whether it’s a 4xx level or 500 level error open the listed URLs in crawled anomaly in search console and open in browser and check with the status:
1. If its 404 then redirect the URL to related URL with 301 redirect
2. If its 500 internal server error makes sure you return a 300 ok response code.
Related Crawl Errors Coverage:
Errors Levels which fall in Crawl Anomaly:
There are only two types of error levels which make your website URL gets de-indexed from Google search by flagging your website URLs in this category of Crawl Anomaly.
1. 4xx level Errors:
Mostly it is with the 404 or access denied which makes Google unable to crawl the webpage (URL)
2. 5xx level Errors:
Which makes Google not to crawl by crashing the server by overloading requests.
After Crawl Anomaly Error URls De-indexing (De-indexed):
After crawl anomaly error URLs will not be a part of Google index they will be dropped from Google search indexed as crawl anomaly error means Google is unable to fetch the URL and will be flagged as fetch failed crawl anomaly error.
Crawl Anomaly Error Increases (Daily Graph):
If its 4xx level error or 5xx level error: Once Google search console faces trouble crawling your webpage the graph gets increasing, as once fetch fails and crawl anomaly, Google Bot falls back crawling your website and understands the server is getting overloaded and doesn’t want to crash server and flags other URLs in this category of error.
How To Fix Crawl Anomaly Error:
Ways to fix crawl Anomaly error make sure you make your webpage crawl by Google by submitting to Fetch as Google and see whether the URL is returning 200 ok response or not.
Mostly by submitting or fetch as Google the URL will be crawled successfully and gets indexed. By doing a fetch as Google check with the headers of the webpage it is returning what Google is expecting to be returned.
If there is any other unexpected headers returning then have a look at it and make Google bot crawl your webpage and get rid of crawl anomaly error.