Ultimate SEO Guides - Keyword Research
Starting Keyword Research can feel like a mammoth task, where do you start? What should you prioritise? What data do you need to get? Charlie Williams, Barb Davids and our own CEO, Laurence O’Toole joined us for Tea Time SEO in November 2020 and shared their methods to keyword research in SEO to help us get started.
A big thank-you to our co-authors for giving up their time and sharing their expertise with us. If you would like to become involved with our Ultimate SEO Guide series then please get in touch.
Chapter 1. Where do you start?
Keyword research is one of my favourite parts of SEO. You don’t want to be always doing it, it can take a long time, it’s hard work, but the strategy you can develop – is fantastic. It’s one of the most powerful market research tools we have available to us in any discipline.
The most important point when doing keyword research is to categorise your keywords. Run your keyword research process through a number of steps that help make sure the keyword ideas are something that you can deliver on. Where you can fulfil what the search is trying to achieve. When you can marry that up, that’s when good things happen with your content.
The first step is knowing what you’re going to be doing this research for because your aim has a bearing on what you’re going to do. The tips to follow are on how to expand your horizons to find more ideas (this is where you need to find ideas on what the audience wants to know), grouping keywords into topics, finding common themes, and mapping intent, so you know what kind of content is going to rank, and what to build.
Build on What is Already Working
First, build on what’s working already by using Search Console data. There’s a huge amount of information there.
If your site has any kind of history, any old articles or lots of commercial pages – whatever it is – there’s going to be loads of queries in Search Console which you can use and build upon to develop new ideas or expand your current content. There is a treasure trove of keywords in GSC that people sometimes forget about, because they’re rushing to find new things in traditional keyword research tools.
Go to the Performance report, start looking for keywords where you’re seen relevant queries but aren’t addressing directly, and then start working out if you should expand the page to address these queries or create something new. As always with good research, find the common themes that you can build good stuff around.
You can use the Search Console API as well. APIs have also been discussed on Tea Time SEO, but the Search Console API can help you find more ideas. You can use an add-on for Google Sheets – Search Analytics for Sheets – or build a table in Data Studio, so you can start querying your pages. See all the keywords they appeared for, and find where you can put new content out there.
Answer Your Public
Build content around questions, not topics. There’s a small demand for many questions, but often lots of questions with the same intent, which added together equals a lot of demand.
The best part of using questions is it makes you build a page or section around them. It forces you to answer the question, rather than just writing about the keyword, which if you’re not Wikipedia or a dictionary website, isn’t the aim of your content. You don’t want to write about this keyword, you want to write what the user wants to get out of it when they type that keyword in, and questions are a great way of doing that. Of course, you can also use questions as subheadings throughout your article or posts, or even e-commerce and product pages. You have loads of great angles with questions.
To get ideas, you can use many of the most famous keyword tools. (have you tried the Authoritas FAQ Explorer yet?) You can scrape the People Also Ask questions directly from the SERPs to get lots of ideas. There is also a tool for this called AlsoAsked. It’s free, go in and put your seed term in. It pulls out a visual which can help get buy-in from people higher up the chain. It’s a nice way to build more keyword ideas.
There are also loads of free keyword tools that use this kind of stuff. For example keywordtool.io. Don’t just use questions like who, what, when, why, how. Expand it out to include prepositions, comparison terms and other modifiers you see in your niche. Things like “best” and so on. That’s another great way of finding question-style topics to build your content around.
Use those who know your audience
Finally on expansion, most times as SEOs, we’re not the experts on the business itself, so make sure you speak to those who are the experts.
A great way to start a new project is to have a set of questions which you can ask the experts so you can learn from them. Common customer queries, pain points they face, barriers to purchase and so on. Your secret weapon for this is to actually sit down with them, buy them a drink and talk to them.
It’s a great way to get real answers and you will get better information than by just sending an email. Speak to the experts, whether you’re in-house or agency side.
Find the Topics and Keyword Exemplars
My fourth tip is about grouping keywords. You’ve found topics to help build better content experiences. Find all the angles of a topic. You’re looking to find and group keywords with the same intents, where you just build one page and target all those keywords that have the same user end-goal. Find where there are multiple pages we can build and a whole range of potential articles around a topic.
This used to be called the hub and spoke model in SEO. Thanks to HubSpot their article, you can see it’s often referred to as topic clusters or content hubs. The idea is we build hubs of information. There was a great article about this by Samuel Schmitt, it’s an example of how he used topic clusters to build huge amounts of traffic by turning a large article into a pillar page and a series of subpages.
By doing the research and studying the different subtopics in one area you can create a hub so Google thinks you’re not just an expert on one keyword, but on a whole series. The idea of a cluster is about building authority in that topic through a series of articles that interlink with each other.
SEO tools can help you out with finding these topics. Ahrefs use the parent topic mechanism they have, where they put a bunch of keywords under a parent topic, because if you rank for that keyword, you tend to rank for the other ones as well. It’s an idea of building a topic out, so do that when you do your keyword research. Another tool that does this is Moz, where you can group keywords by lexical similarity to show topics.
Map Out Your Keywords By Intent
Last but not least, mapping keywords out by intent. Intent has become a really big thing in SEO in the last couple of years but I still don’t see people using it enough. We want to know what the intent of a search is, or what Google thinks the intent of that search is. We want to know what kind of content we have to build to satisfy that search.
Back in 2007, Rand Fishkin, named four kinds of search intent: navigational, informational, commercial investigation, and transactional. Lots of work since then has expanded this. There’s an article from Content Harmony, where Kane Jamison expanded on this idea to about nine different intents. The idea is to build out the potential intents and the cool thing is there are tools that will give you this intent. SISTRIX, use the intents that are listed in the Google Search Quality Evaluator Guidelines as a way of breaking intent down.
This interpretation is based on SERP features. If you have a map pack, there’s probably some ‘visit-in-person’ intent. If they are shopping ads or product ads, then there’s likely to be some commercial intent, and so on. You can learn more about User Intent in our guide here.
I don’t mind how you work it out, what tool you use, but know the intent of your keyword before you start producing content or even think about targeting it. Because if you don’t, how do you know whether you’re actually in a position to address that intent or not, and actually start ranking?
I’ve got an example below of the SERPs. If you look at the SERPs for ‘keyword research tools’, there’s actually mixed intent here. There are some articles and tools directly. You should ask yourself if you can produce the right kind of page that’s going to rank for this keyword. If nothing else, know if your keyword is informational or commercial in intent. If you do that, you can break things down.
By grouping keywords by topic and by intent, you can end up with massive keyword spreadsheets. I like to group all my keywords by topic, see how much search volume there is by topic, and whether it’ll be worth investing in or not. We also can tell how many of those searches are commercial and how many are informational. You can then weigh up the content type and whether it’s worth investing in producing that content or not.
Chapter 2. Keyword URL Mapping
There are many tools, and methodologies when it comes to Keyword URL Mapping, there’s no right or wrong way. The point is trying to reach your goal in what you’re trying to accomplish with your website, and not every page has to have a keyword. For example, you’re probably not going to assign a keyword to the contact page, because you don’t necessarily want that page to come up in search results. So say a user is looking for baby diapers and comes across the contact page, that’s really not enough information, so you don’t want them to land on a page as it doesn’t give them what they’re looking for. So there’s some pages you’ll want to ‘de-index’ or ‘no index’.
One way to find content gaps is your competitors. You can look to see how they’re getting traffic and want to find how you’re missing out on the traffic. A lot of tools have some form of keyword gap or content gap finder. What you do is put in your competitor URLs and your URL, and it shows you keywords where traffic is going to your competitor sites and not yours. Often these tools can tell you the top keywords, where all your competitors have traffic and you don’t, or just one of them has traffic and you don’t. This is a good way to find new keywords.
Another example is from Steve Toth of seonotebook.com, a really good resource for SEO. If you go into Google and type site colon, a competitor name or even maybe somebody in another industry and how often they’re doing their content. You can see in the example below.
You can see a list of everything that Google indexed, and if it has a date, typically blog posts, you get a sense of everything that they published during that time or how often and you can see the sameness of it, or the commonalities. If you use the SEO Minion Chrome extension, you can extract that into an Excel sheet so that you can do more research on and use filters. Keep an eye out for commonalities.
Here’s an example of a keyword URL map. There are, of course, tonnes of them out there, in terms of how granular they get. The base one that I use a lot of the time you’ll see, has a list of all of your URL pages, page type, and a keyword you want to rank for.
What it does then is pull from the above screenshot to the below screen. It’s another sheet in the same workbook. It has a list of all the keywords, keyword difficulties and the search volume. So, when I plug in a keyword into the worksheet on the top, it auto-populates from the other sheet, so I don’t have to keep going back and forth, and see what numbers I have for each keyword.
The other sheet is from all the research and all the keywords that you found using Charlie’s tips you plug that all in. You usually get this from a keyword tool.
Keyword difficulty does differ from tool to tool, based on their methodology, but essentially, it tells you how hard it may be to rank for that keyword. And it gives you the search volume, usually globally. Now, when I work in the top sheet, the one on the bottom turns green because usually you end up with thousands of keywords. So you want to see which one you’ve used or are looking to use. Finally, what I do is add in a call to action, so you could have a call to action for every page. I’ll put in another column for what the call to action will be. Then you just start plugging away one by one, matching them up. If there is something that you’re already doing, like Charlie mentioned, with Google Search Console, you can pull that here and see what the keywords are first. See if it matches, if you don’t like it you can go find a new keyword. This is the part that’s time-consuming because you’re looking and thinking about search intent and which keywords you want and can rank for.
Chapter 3. Keyword Research is the easiest part of the process
Keyword research is something I’ve been doing for 20 years and it’s the easiest part of the SEO process, but it’s also the easiest part to get wrong. You make a lot of decisions based on the research you do, so you get it wrong at your peril. It’s like building a house. Keyword Research is the foundations, your SEO strategy and getting the metrics, organising, drawing the insights and setting your strategy is putting your walls and your roof on your house. With wobbly foundations, everything is going to fall apart around your ears, so for me, there’s four areas I look at as I start the process.
Pick a Keyword Research Tool
First, you need to pick a keyword research tool. There’s three types, manual, what I call ‘pretend automated tools’, and fully automated solutions. You then need to harvest the keyword data, so keyword metrics, user intent, consumer questions, the SERP features, everything Charlie and Barb have discussed. I’ll concentrate more on how you then orchestrate that data. How do you take all that data and use it in a way that gives you insights at scale, set an SEO strategy and implement quickly across your site?
Manual Keyword Research Tools
Manual research tools, we all know you can manually research tools. There’s a gazillion browser plugins and lots of keyword research tools. And what will it do for you? It’s going to produce lists for you. You can put in your domain, page, competitors’ pages, and you can do it all manually.
“Automated” Keyword Research Tools
If you like spreadsheets of millions of keywords and want something automated to draw some insights out for you, then you want to use something that’s more sophisticated than the plugins that give you lists. So if you look at what I call semi automated tools, a lot of SEO platforms like Authoritas will have tools that do content gap analysis for you. You’ll put in your domain or find your competitors and it will find content gaps or keywords they’re ranking for, and you’re not ranking for and distil that down for you. However, you’ve still got a list, but perhaps, a better refined list with some keyword opportunities. At the end of the day, you still have to do a lot of work.
Fully Automated Keyword Research Tools
We have a tool, which is designed for big sites and it does content gap analysis at scale. Our tool does the harvesting of keyword data from your top hundred competitors (it’s like doing 10,000 Venn diagrams at once).
It compares you to everybody and then everybody else to everybody else, then clusters all those keywords by topic. It then maps keywords to your pages, so you can understand straight away which topics and themes have the greatest potential. It can recommend the best pages for you to optimise and effectively suggest new pages to create, and it will do that at scale. The idea here is that it gives you a prioritisation matrix of topics that you can go after, based on what the best potential traffic is for you and how strong you are against the competition. So that’s what I call a fully automated solution.
Harvest Keyword Data
The next step is to implement it. So wherever you get data from you’ll end up with lists or spreadsheets full of data with keywords, search volumes, SERP features, intent and competitor data. You can go out and get questions that relate to those keywords, which you can do with our Frequently Asked Question Explorer. What’s interesting is the user intent. Some keywords have obviously one intent, but some do not. It’s not always black and white, it’s not binary. Sometimes keywords have a mix of research and transactional with a hint of navigational, but it’s good if you’re mapping those different types of keywords, clustering them together and matching them to your site.
Organise and draw insights
Once you’ve got all this data, what do you do with it to try and draw insights? One key thing you can do is find questions that are essential to your theme by mapping keywords and search volumes to the questions that come up. So, for every keyword if you focus on the questions that come up on related searches, you can map your terms and questions together. For example a keyword like, ‘keyword ranking API’, there will be plenty of related questions that come up in the SERPs for keywords related to Google SERPs API, Google keyword ranking API, keyword API, etc. You can capture those questions and link the two together, linking the head terms with the people also ask questions and put them into a graph.
Using a tool called Graph Commons, you can create two Google Sheets and you can set your nodes in your graph and edges that link the nodes. The nodes are keywords and questions, and all you’re doing is saying, “These questions come up for these keywords, these questions come up for these keywords.” It will then cluster them. What it allows you to do is relate your head terms to the questions. So you can see the questions that are most central to your theme. There’s an alternative which is called Kumu.io, they are both free and work by just importing data from Google Sheets, there’s really easy templates to follow.
Set SEO Strategy and Implement
Of course, it all comes down to set an SEO strategy. For me, what that looks like is to do things at scale, analyse your whole industry, which keywords your page was ranking and keywords you’re competing with, and which keywords they’re ranking for. But rather than run it manually, do it at scale, automate it across all your pages on your website, and look at the ones which have the highest potential for growth and where you’re relatively strong against your competitors.
If you can look at the matrix, what you’ve got is a matrix of high potential and high relative strength. So top right is where you’re relatively strong and have great potential, those are the topics and the pages where you should focus first – they are your ‘Quick Wins’. The ones to the top left, have great potential, but you’re a bit weaker, so you need to ‘Build Authority’. The ones to the bottom right are high relative strength, but low potential, that means they are worth ‘Maintaining’. If it’s bottom, left don’t bother, you’ll never get to this – they are ‘Low/No ROI’.
So, that’s one way you can automate your content strategy and you can produce that matrix manually by using the tools that we’ve already outlined. Another way of looking at the tools is actually to look at that research, can you produce an editorial guide to help customers, editors and your SEO teams understand what kind of content they should be producing. For example you look at the modifier types for the keywords: Colour, superlatives, comparatives, singulars, descriptors, so cheap, safe, size, etcetera. You can look at the keyword modifiers, and then look at the smell of the SERP. What does Google want? What does the SERP smell like for that set of phrases?
You can then see the successful content types that dominate the SERPs. So for example, for superlatives; ‘top’, ‘best’, ‘cool’, ‘good’; it will come as no surprise that buyers’ guides and articles are ranking. So, editorial reviews with lots of content snippets and images rank and perform really well. But if you’re looking at ‘second-hand cars’, ‘used pre-owned cars’, then e-commerce category pages, and classified sites dominate.
This just helps you produce a template, if you like, for thinking about how you roll out your content strategy. The last two things I want to mention is you can also map this to your buyer journey and think about different stages of your journey. If you’re buying a car; orientation, selection, financing, trade and ownership, are the different stages of your buyer journey. At different stages, you’ve got different head terms and consumers’ questions that are essential to those themes. The SERP landscape, the pages that are dominating the SERPs are different. There are editorial articles flowing through to listing sites and you have different competitors. That’s another way of segmenting and thinking about your strategy.
When it comes to implementing your SEO strategy, you’ve mapped everything to the page. A classic example is an e-commerce category page, you’ve got to implement the content and add schema markup. But, people will stuff content into ecommerce category pages, but you’ve got to be careful that you don’t alter the intent or the signals of that page because if you’ve got too much content, it looks like an article. Then you may not rank where you want for the phrases that have the intent you like. The other thing you could do in this area, is add consumers questions that answer specific product-related or brand-related questions, which will be more useful.
The last thing, is a case study with Price Minister, where if you do everything at scale, you can analyse millions of keywords, find the best opportunities, find them in new content clusters where you’re not ranking, match those back to your product catalogue, and create SEO-friendly landing pages at scale. This is a great way for big sites to automate elements of SEO strategy.
About the Co-Authors
A search veteran who’s worked just about every kind of role over more than a dozen years in the industry, Charlie runs Chopped Digital, an SEO & content strategy consultancy dedicated to helping create better sites and smarter SEOs. He also thinks keyword research is *amazing* for finding the content your audience wants from you.
Barb Davids is a digital marketer, SEO strategist and owner of Compass Digital Strategies. Driven by data and analytics, she works hard to get business-changing results for her clients, such as 256% more website traffic and 22% more leads. Her own business as a result has pivoted and begun to offer online courses so that business owners can work to gain more site traffic and leads at a more budget friendly cost.
An entrepreneur with 20+ years’ experience in consumer and B2B internet product development, sales and marketing. His current venture Authoritas is a big data science-driven SaaS platform for Content Marketing and SEO Professionals worldwide.