Serving 404 Errors Correctly in ASP.Net

You've developed a brand new site. It's great; there is lots of good content; it's standards compliant; And as it's in .Net you have used the customErrorPages features of the framework. Brilliant. So why does your error page get indexed when you delete files from the site?


The .Net framework detects that your page is no longer there and so sends an HTTP Status Code 302 to the browser along with a redirect to the error page. The error page is then successfully served within the framework and as such sends an HTTP Status Code 200 to the browser. A user may well be able to read the text on the screen that says their file was not found but the bot crawling the site will think the page is still available as it received an HTTP Status Code 200.


To resolve this you need to do two things:

1. Check the page exists - if it doesn't server the error page.
2. Send the correct error code to the browser.

Check the page exists

In your global.asax you need to check the request is a valid one so within the application_BeginRequest code block add the following:

   1:          protected void Application_BeginRequest(object sender, EventArgs e)

   2:          {

   3:              //get the url

   4:              string url = Request.RawUrl;

   5:              //check the document exists on the file system

   6:              if ((!System.IO.File.Exists(Server.MapPath(url))))

   7:              {

   8:                  //if the document does not exist serve the error page

   9:                  Server.Transfer("/Error/FileNotFound.aspx");

  10:              }

  11:          }

Send the correct error status

Now that you can tell whether your page exists you can send the correct status, in your error page page_load event simply add:

   1:          protected void Page_Load(object sender, EventArgs e)

   2:          {

   3:              Response.StatusCode = 404;

   4:              Response.Status = "404 Not Found";

   5:   

   6:          }