How To Fix The Dreaded Crawled and Not Indexed Issue
So how many of you have pages stuck in Crawled but not indexed when you look in your Google search console? I had this with several pages, it was really random and only started happening all off a sudden, and I searched everywhere for an answer.
Most websites said it was either:
- Thin content
- Duplicate content
- Google doesn't index every page.
- The site is new
Other answers I found quite insulting as well and not helpful in the slightest. 'Google has chosen not to index this content right now'. Well yeah, no s*it sherlock.
But anyway, I ruled out the first 2 common reasons straight away, and in my opinion while number 3 is true, it is also a bit of a nonsense as well. And while the site is new, I was not having an issue before, so why now? Something was going on and I needed to find the reason why this had started happening, as there is always a reason.
I tried doing some internal linking to the pages to see if that helps, which it did not. I even got a few backlinks pointing at it and that did not help either. So what did help I hear you ask? Well, this is what I did.
How I Fixed my Crawled and Not Indexed Issues:
What I decided to do was run all the pages that had this issue through Google Page Insights, and guess what I found? They were all in the red for the performance metric for Mobile viewing. Desktop was 90+, but mobile was in the red. I had recently installed a script which slowed down the pages dramatically in Google's eyes, even though I did not really notice it.
So all I did was remove the script and the performance metric went from 32 to 75+ on each of the pages, and I resubmitted them to be indexed in Search console and a week later every single one was indexed. Google was not indexing them because they were slow as a hedgehog in Google's 3G simulated phones eyes. To be honest, it made sense, the site was slow loading, so why would they index it? I knew there had to be a reason, Google are not doing it on purpose, they just do not tell you the reason as usual as they are about as helpful as ET on crack sometimes.
So run your pages through Google Page Insights and see if you have that issue. I am not saying this will be the reason for your pages not being indexed, as it may be another reason, but for me this was 100% the reason and it fixed the issue.
As long as your pages are not thin on content and are not duplicated, there will be a reason why this is happening, and page insights is a good place to start to try and work out the issue.
If that is not the issue with your webpages, you will need to unleash your inner Sherlock Holmes, as I guarantee Google are not choosing to not index it for no reason.