Why We Should All Be Mining the SERPs

Why We Should All Be Mining the SERPs

SERPs, or search engine results pages, are a great resource to learn what Google thinks our customers want.

Google Think It’s Smarter Than Us

Did you know that Google algorithmically rewrites our meta descriptions for SERPs? This means that even though we enter the meta descriptions we’d like to use, Google is still rewriting them.

In fact, Google rewrites our meta description around 84% of the time.

So, what can we do?

We can analyse and deconstruct the language that is being used in SERPs to improve our own descriptions.

We know that we’re in the age of semantic search, and this means we need to be paying attention to user intent and query intent so help improve visibility and CTR (click through rate).

Crawl the SERPs

To help us mine the SERP, there is one tool in particular that is often mentioned. That is Jupyter Notebook.

Jupyter does take some getting used to, but it is worth it.

Once you’re ready, this is what you need to do:

  • Get all of your SERP content into a spreadsheet,
  • Make everything lowercase,
  • Try to avoid duplication,
  • Remove any stop words,
  • Count how may times each word appears.

Doing this will show you which topics your competitors are covering on their landing pages.

You can then so the same test on your own website and use your learnings to identify and then fill in any content gaps.

What’s The Point?

84% google rewrites

Google rewrites our meta description around 84% of the time

By doing this, we can get clues as to how Google is interpreting the content and context of a search.

This leads us to create landing pages that are aligned with the intent of a searcher.

We can also use Google’s algorithm to discover topical areas of interest that can then become content pillars in our content strategies.

What Else can be Mined?

SERPs aren’t the only thing that can be scraped and used to our advantage. We can also scrape the following:

  • Product reviews,
  • Social media pages,
  • Google My Business reviews,
  • Reddit,
  • YouTube comments,
  • Competitor and top ranking pages.

These will then give us more and more insights into how algorithms understand search intent.