We should be working smarter, not harder. If you want to get into technical SEO and data to gain actionable business insights, the gateway drug is the Google Search Console API. The great thing about the Search Console API is that it gives you a lot more granular information than you’d get through the interface. For example, if you use the Search Console interface, you can only get 1000 rows worth of data, whereas the API will give you 5000 rows.
The API lets you combine different metrics and segments; this helps to answer more complicated questions. So for example, you can find out page-level keyword rankings, along with clicks/impressions per keyword, per page. That is useful, and you can’t do it through the interface. To experiment with the Search Console API, but not necessarily be a coder, the place to start is the API Explorer that Google provides. It’s available at developers.google.com, and it lets you access your search console data through an API key, but you don’t have to write code to do it.
You can construct a query, send it to the API, get the information back and do it all in a browser. Now, once you do that, what will happen is you’ll get a response which is a JSON object. JSON, when you first look at it is kind of scary, but it isn’t really. It is just a concise way for computers to communicate with each other efficiently. Many APIs respond with JSON, and it is relatively ubiquitous. You can make a JSON object more human-readable with a variety of tools. The tool I recommend is called konklone.io, and you can paste your JSON data into it, and it will convert it to a CSV, which can be ready by Excel. You can then parse the data however you need to, to gain insight.
If you don’t want to experiment with the API and you want a solution that works, an excellent plug-in for Google Sheets is the Search Analytics plug-in. This plug-in will just let you put your Search Console log-in details in and let you construct a query and set it all up as a timed task. The Google Sheet is then populated periodically depending upon the query you’ve set up. You can use it to do an analysis or back up your Search Console data.
All of this is easy to do, and you can get some interesting insights from the data. An example of something you can do if you categorise your content is to highlight which parts of your site get the highest click-through-rate. Something else you can do is to work out the average click-through-rate by position for your site content and build a custom click-through-rate curve for your site. That would be useful for forecasting and making predictions about future performance.
Combine Google Search Console data with another source of data like whether the search engine positions are visible above the fold or the type of results shown (e.g. maps, rich snippet result, local pack etc.) you can get useful insight. All of that information (and a lot more besides) is available from the Authoritas API.
Also, if you want to experiment with this type of thing without having to code a solution, you could use Screaming Frog to access other APIs. Screaming Frog is useful because it offers access to several different APIs, including:
Google Analytics,
Google Search Console,
Google PageSpeed Insights
MOZ
Ahrefs
Majestic SEO
Some of these APIs are not free, requiring at least an API to access them. The good thing about Screaming Frog is that once you’ve got a list of URLs that you’re interested in, you can put them into Screaming Frog via list mode and then pull all the data on a page-by-page level. That type of data is beneficial for answering interesting questions. You can even use the data you get to build a custom data set to answer questions.
For example, at Purplex Marketing, we are always trying to answer questions in a data-led way. Using this approach, we’ve built our a custom dataset using six months worth of data to try and answer questions like:
Does word count have an impact on rankings?
Has Core Web Vitals started to affect rankings?
What page speed factors are the most beneficial for ranking well?
Is <title> length a factor in ranking?
Trying to work out if there is a correlation between these things and position is interesting, but complicated mathematically. So this is something we are building up over time, and we are hoping to build a mathematical model to try and answer all of these questions. That will allow us to get into the nuts-and-bolts of this and is super interesting. Mathematical models are an advanced topic, but it is the type of places you can end up on the slippery slopes of accessing APIs. It all starts with a straightforward web request, and if you get hooked, that is it. Good luck!
Thank you Sandy Lee for all of your great tips on how to best use APIs! You can watch the full Tea Time SEO talk here:
If you’re crawling a particularly large website, consider enabling a crawl limit. You can use this to check for duplicate content issues which may not be picked up otherwise. Then, when you’re making recommendations and implementing them, you are making full use of your developer or resource, as you’ve got a comprehensive view on all the issues. Once this is complete, it should be easier to re-crawl the website and repeat the process to see if there are any more duplicate issues remaining.
Secondly, you need to make sure you’ve got the entire logs. With CDNs, load balancers and other complex hosting setups, it’s very easy to forget about a server hidden somewhere.
