The Complete Guide to Keyword & AI Prompt Research (2026)

Written By: Ben Poulton
Last Updated: 7 September 2026

Key Takeaways

Summarise With AI

ChatGPT Claude Perplexity Google AI

Opens the selected AI with this article's URL prefilled.

Written By

Ben Poulton

Ben is the founder of Intellar, an SEO consultancy working with service and ecommerce brands across Australia. He writes about technical SEO, AI search and the workflows behind both. More about Ben

Research & Data Pulls

The first step of any keyword research task is to compile data dumps of relevant topics matching your business niche.

This is typically a very easy thing to do, however, this process can go quite granular, and filtering keyword data is where things become tedious.

A full, unfiltered export is likely to not give you much insight at all.

I’ve seen many keyword research documents throughout my years in SEO and sadly most seem to be just that, a giant data dump with no real insight.

That said, research and pulling data is the starting point, so here is how I do it.

Start with a few seed keywords. These are the broad products, services or topics you want to research, such as “toys” or “football boots”. Keep your country and audience in mind as you build the list. Australian customers may use different terms, or be looking for different products, to customers overseas.

Paid SEO Tools

To kick this off, let’s talk about the paid SEO tools for keyword research.

While much can be done with free versions or manual research, the tools available on the market are incredibly powerful.

Having a subscription to market leading SEO tools really helps you drill into data sets and pull out quality results quickly.

Ahrefs

I use Ahrefs, so for the purpose of this guide, I’ll reference that platform.

Plugging a website into Ahrefs’ Site Explorer is a great place to start as Ahrefs shows you keywords that a website ranks for in its database. It also includes historical data and comparative keyword metrics against competitors.

This is my go to tool as it takes much of the guess work out from the get go.

You can even drill down by folder path, enabling you to see keywords (along with SERP features) that your URL or a competitor’s URL ranks for in the selected database.

This is particularly helpful when you want concise keyword sets of subtopics from domains that may cover a whole range of niches.

Department stores for example, we can include all URLs within the “Toys” category only. In Site Explorer, enter www.bigw.com.au/toys/, choose Path, open Organic keywords and select Australia:

Ahrefs Site Explorer showing BIG W’s toys folder in Path mode, with Australian organic keywords

From here you can export keywords with search volume, SERP features, Keyword Difficulty (KD), CPC and even the associated ranking URL.

Use the country selector to switch markets, and the filters to do things like:

remove branded queries
add/remove specific URLs
add/remove specific keywords
include keywords that drive a certain amount of estimated traffic.

All this will help you go granular into sub-topic keyword sets and uncover some gold.

There is a heap of information here that is useful for much more than just SEO too.

This process will give you a top-level understanding of where the volume and ranking opportunity is beyond the retailer’s own brand.

This type of data can inform the products you stock, push harder on ads or get competitive with price on.

As you can see below, with a few basic filters I can already see at a glance what the top traffic driving keywords are for this website in the toys category, excluding the retailer’s name.

Use Add filter → Keyword → Doesn’t contain to exclude “big w” and “bigw” as two rules under All rules, then apply the filter and click Show results. Sort by Traffic from highest to lowest. Product brands such as LEGO remain in this example:

Ahrefs Australian toys keywords sorted by estimated traffic after excluding big w and bigw

This kind of information is invaluable.

Outside of direct research on a domain, Ahrefs’ Keywords Explorer can be used to discover content ideas, additional keywords and questions. Enter a seed keyword and country, then use Matching terms for variations containing your seed words, Questions for question-based searches, and Related terms for ideas from top-ranking pages.

Run your keywords and topics through here to uncover even more data.

You can also do a keyword gap analysis in Ahrefs’ Content gap report by comparing your site with relevant competitors. Look for useful topics they rank for that your site doesn’t cover. Check the ranking pages before adding them to your list, as a competitor may sell products or serve an audience that you don’t.

When comparing the numbers, search volume is an estimate of searches, and keyword difficulty is a tool’s estimate of ranking competition. They help you shortlist keywords, but you still need to look at the search results and decide whether you can create a useful page for that audience.

Answer the Public

Answer the Public combines Google’s Autosuggest with a question scraper.

This makes it the ideal tool for learning what questions your target audience is asking about a topic. It is the best tool out there (in my opinion) for generating quick lists of informational queries.

Set your country, type in a top-level keyword and then you’ll be given a hefty list of:

Questions
Prepositions
Comparisons
Alphabetical matches

However, in this list we are mainly concerned with the list of questions and prepositions, as those keywords can trigger SERP features and are often (surprisingly so) ignored by competing brands.

Some of these informational queries are low in volume and low in competition. Check the results before assuming that applies to every question.

And as a topical set, you can create a sizeable keyword list with a decent total search volume.

Once the data for your keyword has loaded, you will see a wheel of questions, and also a “data” tab that splits out different words, like below:

Ok now we have our list, the next steps are to:

Export the CSV file and filter all “questions” and “prepositions” keywords so those are the only types that remain
Delete any keywords that are not relevant or malformed
Run these keywords through Ahrefs’ Keywords Explorer
Export all terms and the associated data points

Now you’ll have a solid list of highly relevant and very targeted keywords.

Don’t shy away from zero volume keywords either, these can be a low competition gold mine!

Alsoasked

AlsoAsked works in a very similar way to Answer the Public.

The main difference here however, is that AlsoAsked only pulls Google’s “People Also Ask” results.

This tool goes one step further though.

When you input your keyword, AlsoAsked returns multiple levels of results, giving you a quick insight into many sub topic variations! Here’s an example:

AlsoAsked kids toys keyword research example

The great thing about how AlsoAsked segments keyword data is that it’s basically formatted in a done for you topical hierarchy.

“What are the top 5 selling toys of all time” is a great place to start with SEO led content.

The other 7 terms here in this cluster could all make for great content too.

Check whether those questions belong on the same page before creating 8 separate pieces. If the search results and intent overlap, they may work better as sections in one guide. Covering the related questions can strengthen that guide, but it doesn’t guarantee a ranking for “what is the most popular children’s toy?”.

Google Tools

Google Keyword Planner

The Google Keyword Planner is one of the most useful sources of keyword data online, as it comes directly from Google.

There are two key aspects when it comes to using the Google Keyword Planner for keyword research.

This tool makes it very easy to export a bulk list of keywords that you input, complete with historical metrics, or discover new keywords with their keywords ideas tool.

I almost always use the Keyword Planner to pull the historical metric data for a list of keywords I’ve uploaded, such as my Answer the Public list.

So, first things first, to get the search volume for your Answer the Public list, do the following:

Click on “Get search volume and forecasts”
Paste in your list of informational keywords and click “GET STARTED”
Review the historical search-volume metrics for your selected location and date range, then download the data in a CSV file

Secondly, if you want an additional start point for some extra research, then click on the “Discover new keywords” option and paste in your core keyword(s).

You will get a list of related terms as well as historical volume estimates, which is very handy! That is how I use Google Keyword Planner for keyword research.

Keep in mind that Keyword Planner is built for Google Ads. Its competition metric describes advertisers, and its forecasts relate to paid campaigns. For organic research, use the historical search estimates and check the actual search results to assess SEO competition.

Google Search Console

One of the often underutilised tools for keyword research is Google Search Console (GSC). This is because it shows direct click and impression data from searches in Google.

A great feature is its ability to show what queries are leading to your website being displayed in the search results. To access this data, navigate to the ‘Performance’ tab, and you’ll see a list of queries.

A practical approach to uncover potential keyword opportunities is by filtering these queries based on high impressions and low clicks.

High impressions indicate that your site frequently appears for a particular query, but low clicks can mean that you don’t rank high enough, or users aren’t finding your listing attractive or relevant enough to click through on. The layout of the results and answers shown directly on Google can also affect clicks.

By identifying these high impression and low click queries, you’ve essentially found keywords that your website is already being associated with, and may be able to improve its visibility for.

For instance, if a query with high impressions and low clicks is related to a service you offer, it may be worth revisiting the corresponding page on your site.

Look at ways to better optimise the page for this query, such as:

Improving the metadata to better match the user’s intent
Including the keyword more prominently within the content
Creating new pages with supplementary content where the terms have a distinct intent that your existing pages don’t cover

Furthermore, you could also compare your site’s listing with those of competitors appearing for the same query. Analyse what they are doing differently and how you can tweak your listing to improve its click-through rate. GSC only shows your own site’s clicks, so it won’t tell you how many clicks those competitors receive.

Google Search Console, when used in this manner, can be a powerful tool to improve your SEO strategy. It helps identify the gaps in your current keyword optimisation, providing a clear direction on where to focus your optimisation efforts.

For an existing page, filter to that exact URL and compare the latest complete 28 days with the previous 28 days. Look for relevant queries gaining impressions, then check whether the page already answers them. A missing exact phrase can be useful to add, but only where it improves the copy. If the page has no data, check indexing and allow enough time for it to appear in search before drawing conclusions.

GSC also has Search Generative AI performance reports. These help you review visibility within Google’s AI features. The activity is already included in the overall performance reports, so don’t add the totals together or treat the reports as a log of users’ full prompts.

Bing Webmaster Tools & Microsoft Clarity

Bing Webmaster Tools

Bing has its own keyword data, and it’s worth adding this to your research alongside Google.

Open Keyword Research in Bing Webmaster Tools, enter your core keyword, and set your country, language, device and date range. From here, look through the related keywords, question keywords and search-volume trends. You can also review the top-ranking URLs and topics to find more ideas.

Keep the source labelled as Bing when you add these terms to your spreadsheet. Bing search volumes describe a different search audience, so use them alongside your Google research without combining the counts.

There’s also an AI Performance report showing citation activity across supported Microsoft Copilot, Bing and partner experiences. Review the cited pages and grounding queries to see which topics are already connecting AI answers with your content.

Grounding queries are phrases an AI system uses to retrieve information. The report shows a sample, so these aren’t a complete list of customer prompts. Add relevant phrases to your research, record the cited URL and check whether that page answers the topic well. A citation also doesn’t tell you that somebody visited your site or that the answer recommended your brand.

Microsoft Clarity

In Clarity, go to Dashboards > AI Visibility > Citations. Set up your project and complete domain verification if requested. You can then review grounding queries, cited pages and AI referral traffic.

Start with a relevant cited page and look at the queries associated with it. Are there useful questions or product features that the page only partly covers? Add those to your research list, along with the source, date range and proposed page update.

Keep citations and visits separate. One answer can contain several citations, while AI referral traffic measures visits to your site. Clarity’s “share of authority” compares citations within the query set where your domain appeared; it isn’t your share of the whole AI search market.

Query Topics groups related grounding queries, making larger exports easier to review. Where available, Topic Insights lets you define topics, representative prompts and competitors to test. Those tests can help you find content gaps, but they don’t measure how many customers asked each prompt. Topic Insights is in beta and availability depends on the project.

Use the findings to improve relevant pages and select prompts to track. The prompt research process below explains how to build that list.

Manual Research & Brainstorming

Google and YouTube Auto Suggest

Aside from the dedicated keyword research tools, sometimes the old fashioned method of manual searching can be a great way to discover topics, keywords and questions that your target audience has already shown interest in.

The auto suggest feature of both Google and YouTube is a great place to start.

This works the same way on both platforms, where Google and YouTube are showing you related keywords that users have searched for.

This can be a quick and easy way to find relevant terms to a core topic that might otherwise be missed when using tools like Ahrefs, Answer the Public and the Google Keyword Planner.

Finding keywords on websites like Reddit, Quora, or any other popular online forum is another great way to find topic ideas. Using Reddit as an example:

Enter any broad topic that your audience may be interested in and then choose a popular subreddit on that topic

For Reddit, keyword research is a bit of a rabbithole like approach. At least in the way I do it.

Basically you want to browse through all the current popular threads and just keep an eye out for any posts that may prompt additional questions and keyword ideas. You can also search within the subreddit itself.

This thread below might make you think of topics and questions such as:

What are laceless football boots?
Football boot reviews
Elite football shoes

The comment sections in these alone are also a goldmine of additional keywords

Using ChatGPT, you can copy/paste the entire comment section and get it to return a whole bunch of keywords.

Some examples I got from the laceless football boots thread:

Laceless boots review
Breaking in football boots
Football boot sizing
Tight-fitting football boots
How to prevent blisters with football boots
Football boots fit like a sock
Aggressive studs football boots
Tips for breaking in boots

Amazon and eBay

Amazon and eBay are useful places to research the words people use when shopping for products.

Start with a core product keyword and look at the search suggestions, category names and filters. Then read relevant listings and customer reviews for recurring questions about things like size, fit, materials, compatibility and value.

For football boots, that might lead you to terms around wide feet, playing surfaces or whether a laceless boot stays secure. Save the original wording and the page where you found it. These are research ideas to validate, rather than search-volume figures.

Prompt Research for AI Search

The questions you find during keyword research are also a great starting point for prompt research. Here, you’re looking at what a customer might ask an AI assistant when researching a product, comparing options or trying to solve a problem.

Take the football boots example above. “Laceless boots review” gives you a topic. A prompt adds the situation and the decision somebody needs help with.

Finding Prompts Worth Tracking

Start with the keywords, customer questions and forum discussions you’ve already collected. Add questions from your sales or support team where you have them. Keep the original wording and source beside each idea so you can tell where it came from.

Then check your own product, service and support pages. What features can you explain properly? What questions can you answer with evidence? This helps you choose prompts that matter to your business and that your content can realistically address.

Using the football boots topics, you might develop these example prompts:

Which football boots should I consider for wide feet on firm ground?
Are laceless football boots a good choice if my heel slips during a game?
What should I check before buying football boots for artificial grass?

These are illustrative prompts developed from the topic, not measured customer searches. Keep that distinction in your research sheet. A related keyword’s search volume doesn’t tell you how often someone asks the full prompt in ChatGPT or another AI assistant.

Choosing Your Final Prompt List

Group the candidates by the decision they help somebody make. You might have groups for choosing a product, comparing alternatives, understanding a feature, price and value, or solving a problem after purchase.

Remove near-duplicates that ask the same thing with slightly different wording. Keep a variation when it changes something meaningful, such as the customer’s budget, location, product requirements or reason for buying. Include branded and non-branded questions where each serves a useful purpose.

For each shortlisted prompt, record why it matters and which page should answer it. Mark whether the page directly answers the question, only partly covers it, or whether there’s a content gap. This gives you a practical content update list alongside your tracking list.

Tracking the Responses

Once you’ve chosen the prompts, run a consistent set over time using the platforms that matter to your audience. Record the exact wording, platform, date and relevant location settings, and keep the response and cited URLs.

Track brand mentions, citations to your website and explicit recommendations separately. Your brand could appear without your website being cited, or a page could be cited without your product being recommended.

Repeat the checks so you can look for patterns across responses. Keep the core prompt set stable and record any additions or changes. Otherwise, a change in your tracking list can look like a change in visibility. Our guide to AI SEO tools and visibility tracking covers tools you can use for this.

AI Keyword Tools

What AI Adds

AI does a GREAT job at collating data and organising data dumps from other keyword research tools. Here’s how you can use it:

Group related keywords into draft topic clusters.
Summarise recurring questions from the research you’ve collected.
Suggest additional keyword and prompt ideas for you to validate.
Help clean up exports and prepare a research spreadsheet.

Check the groups yourself before using them to plan pages. Similar wording doesn’t always mean the same search intent, and AI-generated ideas still need supporting research.

AI Keyword Tools to Try

ChatGPT – Super fast keyword ideation, clustering and sheet exports. Great for on the fly data analysis too.

Perplexity – conversational research with links to sources you can check.

Keyword Insights – keyword clustering based on SERP similarity and content brief creation.

Surfer – content research and optimisation using related terms and search-result analysis.

SparkToro – audience research to help you find the places and topics worth investigating. It complements your keyword data.

How to Slot AI Into Your Keyword Research Process

AI is best at ideating. Run a broad prompt in ChatGPT > Prompt for clustering > Trim and validate > Paste the keyword list into Ahrefs’ Keywords Explorer.

For further validation, check the original sources, compare search estimates in your keyword tools and cross-check with GSC data. Google Trends can help you check seasonality, but its 0–100 interest scale isn’t a count of searches.

Collating & Formatting Your Keyword Research

Now you’ve got the data, bring it together in one spreadsheet and remove duplicates. Keep a column for the source, market and date so you know what each number represents.

Choosing Keywords & Mapping Them to Pages

Before sorting everything by search volume, check search intent. Is somebody looking for information, comparing products, ready to buy, or trying to find a particular website? Look at the current results to see whether Google is showing guides, category pages, product pages or something else.

Group keywords that can be answered well by the same page. These keyword clusters can include a main keyword and more specific long-tail keywords. Use the search results and the actual question to decide whether they belong together, rather than giving every keyword its own URL.

Then assign a target page and an action: create, update or keep. Check whether an existing page already covers the topic before proposing another one. That helps avoid creating overlapping pages that compete for the same intent.

Prioritise the groups by relevance to your business, the opportunity in the search results and the work needed to produce something useful. Search volume and keyword difficulty can help with that decision, but the biggest number won’t always be the best opportunity.

Making the Report Useful

Keep the main working sheet simple: keyword, topic, intent, source, market, search volume, difficulty, target URL, action and priority. Use a separate tab for prompts and their tracking results so keyword estimates don’t get confused with AI response data.

All of the above can also be done in Google Sheets. If you are just wanting to showcase some numbers and graphs, it can be a great option as it’s much easier to share around.

Google has a quick guide on how to slice data in Sheets. Slicers can help you filter a report by topic or another useful category.

I hope this guide has helped you learn how to do keyword research and compile some killer keyword lists for SEO and/or even paid search!