Most SEOs will be comfortable/familiar with using Google Search Console as part of their SEO toolkit. But there are some great ways to better utilize this data without introducing additional cost or complexity into your reporting.
In this blog, we’ll cover tips and tricks to getting the most out of Google Search Console data when conducting time-based SEO tests.
Why Use Google Search Console?
Google Search Console is a treasure-trove of essential SEO data, and what’s more - it’s free! The Performance Report and the ability to filter pages and URLs using regex create a compelling way to compare/segment data quickly.
One caveat we need to be aware of here is that if you are using the data directly from the User Interface, you are limited to 1,000 rows of data.
This means that the accuracy and depth of data may suffer. If you are running relatively small tests, this would be a huge issue. However, you can side-step these limitations by using Google Data Studio and connecting it to Google Search Console.
Your Testing Hypothesis
As we are discussing testing, we are looking to see the impact of a particular change made at a specific time on specific pages. If you haven’t already, it’s time to create a hypothesis. The more detailed, the better, but it needs to cover (at least) the two questions:
- Which pages would you expect to see the impact on?
- Is this measurable within the Google Search console?
If you know the URLs and are confident that you’ll see an impact in Clicks, Impressions, or Click Through Rate (CTR), then keep this information in your head, and let’s get into the testing!
Time-Based Testing Vs Split-Testing
There are different methods to test changes within SEO. Split-testing removes variables by testing x2 different buckets of pages and measures them over the same time, while a time-based test compares a period before a change (the control period) with the period after (the test period).
Split-testing is desirable in many circumstances but has more exacting requirements on the number of pages, traffic to those pages, and often additional software to facilitate the changes themself. Even if you are split-testing, there are times when you need to run a time-based test:
- When you do not have a large enough selection of pages to measure the impact of a split-test
- Where you test pages have fewer than 1,000 daily visits*
- You cannot wait for the test pages to be indexed and the test to run
Time-based testing is not perfect; seasonality and external factors (algorithm updates, above-the-line campaigns) can significantly impact results. But for a quick way to test the impact of deployments, you can’t often find better.
*You can work around this by pulling the data in Google Data Studio - more information here.
Using Google Search Console to Run a Time-Based Test
Select the testing window
When was the test deployed & how long did it take for google to index a significant portion of the change/pages.
For example, 2-6 weeks post-deployment should be what you are aiming for here. You would then compare this with the equal period before the deployment.
Avoid a testing window that is too long; the longer you’re monitoring, the more exposed you are to other factors that could influence the test.
Next, look at the performance data within this window, did anything else take place around the time of the test which could have impacted the test/reporting?
Were there any other large deployments that took place during this time, or were there any known algorithm updates? The Semrush Sensor can help identify anything significant in the last 30 days.
It is almost impossible to control these elements, but being aware enables you to place the appropriate caveats on the results and set expectations. You can also use page group (and search query) segmentation to help remove portions of traffic that you know will upset your data. More on that coming up!
Create Regex to Segment Page Groups & Queries
When monitoring the impact of a change to a limited set of pages, Google Search Console doesn’t always make things easy. But since the introduction of Regex filters, you can easily filter, so you are just reporting on those URLs you are interested in.
Regex on Pages
The key is to know which pages you are testing and create a regular expression that matches just those pages. This will depend wildly on your website’s URL structure and the scope of the test, but here are some examples you can use:
- .*keyword.* - match anything containing a keyword
- .*/directory-name/.* - match any URL which contains a specific directory
- .*[0-9]{8}.* - match any URL with an 8-digit string
If you have a list of URLs, you can use the following sheet (please create a copy) to generate the Regex you’ll need:
Just note that you’ll be limited on the number of characters in the filter - so if the URLs are overly long or you have too many, this method may not work for you.
Either way, Regex is pretty powerful, and I haven’t encountered a scenario where I haven’t been able to create a filter that would help me report on just what I needed.
Regex on Queries
Filtering out keywords to remove brand noise - or keywords you know aren’t part of the test pages - is as simple as using regex.
Ruling out brand impact & above-the-line activity can sometimes be as simple as viewing the brand trend against the non-brand trend. Assuming there is a noticeable difference in branded traffic (a sharp uplift), that will enable you to identify the probable impact of other activities.
In the below example, we’re comparing the brand regex with the non-brand regex. This can be achieved with as per the following:
This will enable you to compare brand/non-brand performance, and you’ll see the impact of each on the overall performance.
You can then apply page filters, query filters, and date range simultaneously, which should help you start to understand how a particular change is performing.
It is not as sophisticated as some of the analyses you can run on A/B testing, but generally, it can identify changes that were not likely due to the test itself.
Comparing the Performance
If despite all the filtering & comparisons, you’re finding that seasonality is too much of a factor - or the data is too noisy, there is one other step. The year-over-year performance will also help if you want to test your change against last year’s expectations.
GSC has pre-built year-over-year comparisons, or you can create your own.
You cannot use different types of comparison (queries or pages) when using a date comparison, but you can download the data to CSV/Google Sheets and then filter it as a spreadsheet. This is generally more powerful, but less convenient!
Causal Impact (when time-based isn’t working)
If you struggle to see the impact in Google Search Console data, this method of analysis isn’t working for you. There is always a Causal Impact.
Google developed causal Impact, and without getting too technical, it helps us to understand if a change could be positive/negative based on the expected performance. You can read more here if you’re interested, but really what we want is this.
Apply the page filters you need in Google Search Console, download the data, uploaded it to the Causal Impact tool & set the date ranges.
The outputs will look something like this - it predicts what the performance should be after the change and then compares it with what actually happened.
Again, without getting too into the detail, what we really need (at least) here is to review the final graph. If the shared blue area is all the above (or below) the dotted “0” line, then the test is statistically significant. Above the line means that the change caused a positive result, whereas below the line is likely a negative one.
If the shaded line isn’t all the way above or below, then we’re looking at a test that looks like it could be positive or negative, but it is not statistically significant.
Wrapping up
So we’ve walked through the different ways to add greater power to how you use Google Search Console to report on changes/tests.
There are more advanced methods of doing this and some powerful tools to help you. Tools like SplitSignal can take a lot of the legwork/analysis here. Investing in the right tooling can level up your sophistication and accelerate your returns if you want to focus more on creating strong hypotheses and making the changes happen (rather than analysis itself).
But if you had only been using basic search reports (without filters) before, then hopefully, you have several new tools at your disposal. So start using the already open data and start interrogating the data behind your SEO changes today.