Google has introduced a new feature where you will get all the data regarding the Indexed and Unindexed URLs of your site in detail. As per google blog   It shows how many pages are currently indexed. The legend shows the latest count and the graph shows up to one year of data.

Advanced Tab

The follwoing options :

  • Ever crawled: The cumulative total of URLs on your site that Google has ever crawled. Not all crawled URLs get indexed, and Google may discover some URLs by other means such as inbound links from other sites. This number should increase over time as new pages are added to your site.
  • Total indexed: The total number of URLs currently in Google’s index. These URLs are available to appear in search results, along with other URLs Google may discover by other means. This number will change over time, as new pages are added and indexed, and old pages are removed. The number of indexed URLs is almost always significantly smaller than the number of crawled URLs, because it does not include URLs that have been identified as duplicates or non-canonical, or less useful, or that contain a meta noindex tag.

To see basic data plus additional useful information, click Advanced. Click the checkbox next to the data you want, and then click Update. The following additional data is available:

  • Blocked by robots: The total number of URLs that Google cannot crawl because they are disallowed in your robots.txt file.
  • Not selected: Pages that are not indexed because they are substantially similar to other pages, or that have been redirected to another URL. More information.

How to use this data

  • A steady increase in the number of crawled and indexed pages indicates that Google can regularly access your content, and that your site is being indexed.
  • If you see a sudden drop in the number of indexed pages, it may mean that your server is down or overloaded, or that Google is having trouble accessing your content.
  • A high number of duplicate pages could mean that your site has problems with canonicalization,duplicate content, or automatically generated pages, or that it has been hacked. In many cases, Google will send you a message when we detect problems with your site, so make sure to set yournotification preferences.
  • Try comparing data:
    • To see the percentage of your site that’s accessible to Google, compare the number of crawled pages with the number of pages blocked by robots.txt.
    • To see how much of your site consists of pages perceived to contain unique content, compare the number of indexed pages with the number of pages marked Not Selected.
    • Sudden spikes or dips that appear in several charts can indicate problems with site configuration, redirects, or security.

Post By Gishore J Kallarackal (2,121 Posts)

Gishore J Kallarackal is the founder of techgurulive. The purpose of this site is to share information about free resources that techies can use for reference. You can follow me on the social web, subscribe to the RSS Feed or sign up for the email newsletter for your daily dose of tech tips & tutorials. You can content me via @twitter or e-mail.

Website: → Techgurulive