HTTP Status Codes: Page Not Found

HTTP Status Codes-Page not found

It’s annoying landing on a page to be hit with a “page not found” message. There are various reasons why you’re not always directed to the right page, but the page the user lands on must serve a purpose. HTTP status codes (or server codes) are a reputable way to inform users and search engines that the page has moved or no longer exists. There are different server codes, and it’s essential to use the right one to avoid damaging your website’s performance.

From performing technical SEO audits, it’s not uncommon to see incorrect or missing HTTP status codes. In this blog, I will explain what status codes are so you know which ones to use and when.

What are status codes?

HTTP status codes inform search engines that the page has been updated, so they know how to respond when crawling the page.

Each server code is represented by numbers – 100s to 500s and describes a different request. It’s essential for organic search that the correct code has been set, and it’s also important to update other marketing teams if a specific code has been set to ensure they’re promoting a valid URL or pausing campaigns.

Before I explain the different status codes, I want to discuss how they impact SEO.

As mentioned, server codes send a message to search engines about the page and the current status so they can respond accordingly. These codes tell search engines whether to continue to crawl the page if the page has moved to a different URL and still index it or if the website is experiencing server issues. Status codes also send the message that the page is fine and can be crawled. For your website to perform well, you must set status codes.

Having a website with multiple 404 pages or redirecting a page to an irrelevant section of the site isn’t the best practice. It will affect organic performance while providing a negative user experience. Users landing on these pages are likely to leave the website immediately, causing high bounce rates and forcing them to shop or browse elsewhere. Search engines will negatively view this.

Imagine visiting a clothing shop to find out it’s been changed to a hardware shop or closed down, but there’s been no update on it? You’d be pretty annoyed. So, updating the status of your web pages is essential.

This is not an exhaustive list but should still be comprehensive enough to help you get to grips with the many available options.

http status codes
Image Source: https://devqa.io/

1xx status codes

The 1xx status codes signify that the page so far looks promising, and search engines will continue to review it and carry on with the request. Not many SEOs experience a 1xx status code, but they can still be helpful to learn.

Some common examples are:

100 Continue

This interim response suggests that the client should continue the request or ignore the response if the request is already finished.

103 Early Hints

A 103 status code is primarily intended for the Link header, allowing the user agent to start preloading resources while the server prepares a response.

2xx status codes

The 2xx status codes are the ones you want to see.

200 OK

It notifies SEO professionals that the page request is received, it’s been reviewed, and everything is good. While a 200 server code is ideal, you can’t assume that the page is 100% perfect. The page could still have errors, including:

  • Duplicate content
  • Missing page titles, H1 headers and meta descriptions
  • Page titles and meta descriptions are too long
  • Duplicate page titles, meta descriptions and H1s
  • Thin or missing content on the page
  • The canonical tags haven’t been set up correctly

 

It’s always worth checking these pages and not assuming that a 200 code is the answer to your SEO problems. The pages under the 200 status code umbrella must be indexable; otherwise, you’re sending a message to Google that the page can be ignored and isn’t important when in fact, it holds significant value to your website.

3xx status codes

The 3xx codes change the URL path and website structure, meaning pages have moved and need redirecting. There are various 300 codes, so make sure you’re using the correct one – remember, you’re telling Google the current status of a page and how you want it handled.

Think about your crawl budget and current SEO efforts to improve website performance. The most popular are 301s and 302s.

301 Moved Permanently

The page has permanently moved and is being redirected to a different page. A 301 sends a message that the page has been moved; therefore, it won’t be crawled and will continue following the redirect instructions. The original page won’t be indexable.

When applying a redirect, please ensure that you send users to another relevant page and don’t automatically reassign the page to the homepage. Think about what you’re targeting, what users are searching for, and the relevancy of that page’s internal and external links.

You don’t want to lose the page strength, rankings and current position by redirecting it to the homepage or an irrelevant page on the site. Also, it’s not guaranteed that the page will maintain its strength and ranking once it’s been redirected.

Review the page’s backlinks to ensure they’re relevant for the destination page. Contact the site owner or editor to update their content with the new URL if they’re not. Also, review the internal links coming from the original page, and all pages are using the correct URL.

My advice is to redirect the page to the main category or sub-category, depending on how your website is structured. If there aren’t any relevant pages, then opt for the homepage – or consider leaving it as a 404 error.

302 Found

A 302 tells search engines that the page has been temporarily moved and it’s currently unavailable. However, they can still attempt to crawl the page. A 302 status code, indicates it’s for a temporary redirect and not a permanent one.

A common example is for a non-logged in user trying to access a client portal – in this case, they would be temporarily redirected to a login or sign up page. This is expected behaviour and a good reason to use a 302.

If the page redirect becomes permanent, update the server code to a 301.

While the page is a 302, it will maintain its strength and popularity and won’t be transferred to the destination page. The page will still be indexable. Likewise, with a 301 page, review the current links to ensure they’re relevant and updated.

303 See Other

A 303 status code is a way to redirect web applications to a new URL with a GET request, particularly after an HTTP POST has been completed.

304 Not Modified

A 304 tells search engines that the page has not been modified since the last visit. You are essentially saying that the page in question does not need to be crawled again by the Google bot and that they can use the cache version.

The HTTP 304 status code is lesser-known compared to other 3XX status codes, but it offers the possibility to save on your crawl budget.

307 Temporary Redirect

A 307 is an HTTP response status code that indicates that the URL the user is requesting has been transferred to a provisional location and will be back soon.

While a 302 is slightly ambiguous, a 307 states specifically that the requested URL has been moved to an interim location and will be back after some time.

308 Permanent Redirect

This status code means that the resource is now permanently found at another URL typically specified by the Location: HTTP Response header. 

It has the same meaning as the 301 Moved Permanently HTTP response code, except that the user agent must not change the HTTP method used: if a POST was used in the first request, a POST must be used in the second request.

301 status code
Image Source: https://trafficradius.com.au/

4xx status codes

You’ve probably come across 4xx status codes. These are ones to avoid, as they signal a bad request to search engines, and it won’t be processed until rectified. This can happen when a user types in the wrong URL or the page no longer exists, and no redirect has been set.

The most common 4xx status code is a 404 – page not found.

400 Bad Request

In this case, the server cannot or will not process the request due to something that is perceived to be a client error (this could be a deformed request syntax, invalid request message framing, or a deceptive request routing).

403 Forbidden

You may encounter a 403 forbidden error if the client does not have access rights to the content; therefore, it is unauthorised, so the server declines to give the requested resource.

404 Not Found

We’ve all experienced it and are likely to have been frustrated that we’ve been sent to a page that doesn’t exist (out of no fault of our own). This is a client error and must be corrected.

A 404 can have damaging effects on your website. It’s being flagged as a bad request, and it can even force users to leave the website and visit a competitor – something you don’t want. If this is in place for an extended period, the page will eventually lose its popularity, impacting overall website performance.

A 404 is a non-existent page; Google is aware that the page should be non-indexable and doesn’t exist. Eventually, the page won’t be indexed, but all performance achievements will be lost. To mitigate this loss, you can use either a 301 or 302 redirect as needed. 

A 404 can also be beneficial, as you’re signalling to Google that the page currently doesn’t exist and that it isn’t problematic.

When hosting a 404 page, you can customise it to avoid a negative experience. The page can include internal links to other relevant pages, a personable message about the page, and the design can mirror the main website. Feel free to have a bit of fun with the page too, as long as the message is clear and it returns a 404 status code, there’s no reason why you can’t add a bit of humour. 

It can be considered best practice to update all internal links pointing to a 404 page to avoid sending users to a non-existent page.

If you’re unable to redirect a 404 page, for example, there are no relevant or similar pages on the website, it will never return, and there are no crucial links, then you may be ok to leave it as a 404. Though, depending on the circumstances, I may still recommend redirecting the URL after some time.

Another thing to note about 404s, which is vital, is avoiding a “soft 404”. This refers to a 404 page that doesn’t exist but responds to a 200 status code. You’re signalling that the page is good and exists (even though it doesn’t) and allowing Google to crawl it; therefore, wasting your crawl budget. The page can still be visible in the search results.

410 Gone

I want to mention the 410 status code option. A 410 provides more information to Google that the page has gone and won’t be returning. Google will respond instantly, and the page will be de-indexed, whereas, with a 404, there can be delays.

Search engines will review the page to ensure that it no longer exists.

Again, ensure no links are pointing to a 410 page and update any that are.

429 Too Many Requests

The user has sent too many requests in a given time, and the server cannot cope and therefore send this response code.

If you are crawling a site when this happens, adjust your crawl speed accordingly and try again.

marvel 404 status code
Image Source: https://www.techadvisor.com/

5xx status codes

5xx status codes are caused by server errors and cannot respond to an HTTP request. This doesn’t necessarily mean your server causes the issue.

5xx server code can be damaging to your overall website performance, in particular, SEO. It will cause a negative user experience, and your organic rankings will be affected. While these codes prevent Google from crawling the site, the issues are challenging to resolve and often require development support.

Popular 5xx status codes include:

502: Bad Gateway – The server received an invalid response from another server. Generally, this can be a website fault.

503: Service Unavailable – The request can’t be processed due to maintenance on the server. However, Google recognises this issue and won’t de-index or penalise the page.

Semrush has great information on how to fix 503 status code errors.

504: Gateway Timeout – the request is timed-out after a certain period due to delays with the server acting as a gateway.

510: Not Extended – the search engine cannot understand the request due to not having the correct extension.

To Conclude

To identify HTTP status codes on your website, perform a technical SEO site audit to detect and resolve the pages accordingly.

When a status code has been assigned, ensure that you’re updating the marketing and other team members who will be promoting web pages. For example, paid media campaigns, email marketing, PR campaigns etc. You don’t want to advertise a page with the wrong URL or a page that doesn’t exist.

There are other HTTP status codes, but the ones mentioned in this article are the most common. Think about the value of that page, the impact of removing a page, the user experience and how you want search engines like Google to review the page. If possible, avoid deleting a page completely, as you’ll lose all equity and efforts to promote it.

Facebook
Twitter
LinkedIn