Usually, when someone requests a page that doesn’t exist, a server will return a 404 (not found) error. This HTTP response code clearly tells both browsers and search engines that the page doesn’t exist. As a result, the content of the page (if any) won’t be crawled or indexed by search engines.

In addition to returning a 404 code in response to a request for a page that doesn’t exist, the server will also display a 404 page. This may be a standard “File Not Found” message, or it could be a custom page designed to provide the user with additional information. The content of the page is entirely unrelated to the HTTP response returned by the server. Just because a page displays a 404 File Not Found message doesn’t mean that it’s a 404 page. It’s like a giraffe wearing a name tag that says “dog.” Just because it says it’s a dog, doesn’t mean it’s actually a dog. Similarly, just because a page says 404, doesn’t mean it’s returning a 404. You can use Fetch as Googlebot (or other tools available on the web) to verify whether the URL is actually returning the correct code.

Returning a code other than 404 or 410 for a non-existent page (or redirecting users to another page, such as the homepage, instead of returning a 404) can be problematic. Firstly, it tells search engines that there’s a real page at that URL. As a result, that URL may be crawled and its content indexed. Because of the time Googlebot spends on non-existent pages, your unique URLs may not be discovered as quickly or visited as frequently and your site’s crawl coverage may be impacted (also, you probably don’t want your site to rank well for the search query [File not found]).

We recommend that you always return a 404 (Not found) or a 410 (Gone) response code in response to a request for a non-existing page. You can improve the user experience by configuring your site to display a custom 404 page when returning a 404 response code. For example, you could create a page containing a list of your most popular pages, or a link to your home page, or a feedback link. You can also use the Webmaster Tools Custom 404 widgetto add a search box and more site search options to your site. But it’s important to remember that it’s not enough to just create a page that displays a 404 message. You also need to return the correct 404 or 410 HTTP response code.

Post By Gishore J Kallarackal (2,121 Posts)

Gishore J Kallarackal is the founder of techgurulive. The purpose of this site is to share information about free resources that techies can use for reference. You can follow me on the social web, subscribe to the RSS Feed or sign up for the email newsletter for your daily dose of tech tips & tutorials. You can content me via @twitter or e-mail.

Website: → Techgurulive

Connect