Indexing your URLs in Google is a vital step in the process of getting free organic traffic to your website.
If your URLs aren’t in Google in the first place, they won’t be found, so making sure all the content you want to be visible is visible, should be one of the first things to consider if you want more organic traffic from. your website.
NB You probably won’t want ALL of your web pages to be indexed. The majority of sites will have some level of irrelevant, duplicate, or sensitive content that shouldn’t be indexed – so an unindexed URL isn’t necessarily a bad thing.
Here are three different ways to check which of your URLs are indexed.
Site: Search Google
There are many different search commands that can be used in Google to check information about yours and other people’s websites, but one of the most useful and commonly used commands is the search command. « site: ».
This will tell you how many pages of a particular domain or subdomain are indexed in Google.
In this case, we can see that our website has 2,850 pages indexed according to Google.
- Google Search Console Coverage Report
Of course, just checking the live Google index doesn’t tell you too much.
You will want to know if this is the number of pages you could and should have indexed.
Google Search Console will tell you the number of valid URLs (i.e. valid for indexing) on your domain
Ideally, this should roughly match the number of URLs returned by a site: search command.
In this case, we can see that there is 3 160 valid pages.
This is MORE than what we can see in the live index; and about a ten% difference.
There will probably always be a difference between these two numbers and we cannot trust either one to be 100% accurate, but when we see such a differential there may be an opportunity to investigate. more.
In this case, we might want to compare the number of pages indexed in the live Google index to what Google is reporting.
Google Search Console URL Inspection
Sometimes we may want to check if an individual URL is indexed, rather than the index status of every unique URL on the domain.
In these situations, we can use the URL inspection tool on Search Console
More importantly, it will tell us if the URL is indexed, and if not, why.
It will also give us other potentially useful information such as;
- If the URL was found in the XML sitemap
- Date of last URL crawl
- The user declared and selected the canonicals by Google
- Site exploration
Another way to assess the indexability of your domains is to perform a scan.
Exploring your website with a tool like Screaming Frog can give you several different data points relating to your site; One of the most useful is whether a page is indexable or not.
NB Indexable is not the same as indexed – but all indexable pages must be indexed (unless they are unintentionally indexed).
Pages may not be indexable due to rules in the robots.txt file, status codes (for example 404 or 301), lack of index tags or canonical tags. Crawl data can be used to diagnose why a page may not be indexable.
Tips for indexing your pages
Once you’ve fixed the glaring issues with your domain’s indexability, you’ll want to make sure that in the future your pages will be indexed as quickly as possible and will stay indexed for as long as they’re active.
To do this, you will;
- Make sure all indexable URLs are submitted in XML sitemap
When / if all else fails – the XML sitemap is one place Google can access a list of all of your indexable URLs, so make sure this is maintained properly and new URLs are added.
Dynamic sitemaps can help, but you’ll want to make sure the settings exclude any canonicalized, unindexed, blocked, or non-200 URLs.
- Submit new URLs through GSC
It might not speed up indexing, but for something that takes a few seconds, it’s worth a try.
When you inspect a URL through Search Console, you can request indexing, this will add the URL to Google’s crawl queue, faster than it could have been otherwise.
- Add internal links
If you add a new page to your business that you want to explore, be sure to implement internal links pointing to that page. These internal links should be both based on templates (main navigation, footer, breadcrumbs) and in content (e.g. crosslinks from related pages or blog posts, and well sure, from the home page).
Making sure your URLs are indexable is the minimum basic requirement to make sure they get traffic, but it’s something that is overlooked by many sites.