Your cart is currently empty!
We just launched our courses for pre-order -> Buy access here ✨
Keyword suggestions are one of the most powerful ways to very quickly, with very minimal effort incorporate query semantics, and specifically to understand some of Google-suggested query paths and query refinements and augmentations for the terms you’re analysing. This can help you uncover not only what people are searching for as top searches for your seed terms, but also what Google is suggesting as next steps in their search journey. In the process, you’ll also stumble onto some key ngrams (keyword patterns), entities, and other characteristics that may provide you an alternative and enhanced view of your research topics.
All of this, can of course, help you create a better keyword universe, and in turn – enable you to more comprehensively understanding these queries, and craft content that better resonates with your audience.
In this guide, I’ll walk you through three different methods that I use to gather keyword suggestions:
Each method has its own strengths and best use case scenarios, and together, they can become a great source of new terms to consider, when conducting keyword research.
Rest assured – this tutorial is completely beginner-friendly, and all the code mentioned will be provided in a handy Google Colab Template for you to get started with it straight away.
✨ If you’re interested about taking your keyword research to the next level, consider my course on Semantic ML-enabled Keyword Research, out now on the MLforSEO Academy.
UNDERSTAND QUERY SEMANTICS
Dive into query semantics and machine learning to improve your keyword research methods. This course will teach you how to find untapped keywords, understand user intent at scale, and boost your content’s visibility using cutting-edge ML and automation techniques. Includes a ton of information on patents and practical exercises!
Google offers several query autocomplete features, designed to predict and display search queries as you type into the search bar. The user-facing product (autosuggest) is a quick and intuitive way for users to find the information they are looking for faster, but for us, organic marketers, it can be a goldmine of insights into real user search behavior and query semantics.
Autocomplete is a feature for predicting the rest of a query a user is entering, which has several benefits:
Autocomplete and predictive text technologies predict and complete words as users type, improving speed and efficiency. It originated in the 1950s to improve Chinese typewriters’ efficiency, and its development eventually influenced modern word processors and machine learning applications like Google Search. Initially created to help people with disabilities, these models are now commonly used to assist all types of users, especially in fields with specialized vocabulary like medicine.
The basic concept involves language modeling, where the system predicts the next word based on the letters typed. For example, if a user types “med”, the model might predict “medical” or “medicine”. These predictions are driven by statistical probabilities, considering which words or phrases are most likely to follow based on previous inputs.
In addition, a “frecency” model is often employed, which prioritizes words the user has typed frequently or recently. This is especially useful for organic marketers, as it can tailor suggestions based on past query formulation and implicit user feedback Google collects for users, which can be a way for organic search marketers to incorporate user search behaviour in semantic keyword research. Over time, via reinforcement learning, these models learn from the user’s input, adapting to their unique language patterns.
Here are the different APIs and API endpoints that can be used to collect query autocompletions:
In the following sections, I’ll show you several use cases for implementation of text prediction technology (autocomplete) via the different endpoints and APIs that Google offers, with the aim of enhancing your keyword collection and keyword research efforts in digital marketing.
Make a copy of the Google’s Autocomplete APIs and Endpoints – Keyword Research for Marketers Use Cases – Google Colab Template by Lazarina Stoy for MLforSEO , as that’s what we’ll be working with for the rest of this guide.
There really isn’t much you need to get started working with these APIs. But the basics apply:
Inspired by this approach, demoed by Michael Van Den Reym, I created a function that enables you to input a list of keywords in a csv file, and get an export of your seed keywords, mapped to the Google Autocomplete keyword suggestions, and a shared keyword characteristic from all of the suggested keywords for the term in a third column.
The goal is to generate a comprehensive list of keyword suggestions based on user-provided seed keywords and organize them into clusters. Each suggestion is linked back to its seed keyword for easy analysis.
By clustering these suggestions (clusters are built in the function, based on common ngrams from the suggestions Google gives), you can identify common characteristics for terms that may not be directly related to your seed keyword but are frequently associated with it. This is particularly useful for uncovering missed opportunities. For instance, when entering “business loan,” Google might suggest related terms like “calculators” or “credit scores.”
This script is great for:
To use the script:
Below is the Python code for this section, if you’d like to edit anything in it. You can modify the script to adjust how keywords are clustered, increase the batch size, or refine the stop word list. By doing this, you can scale up your research, making it easier to explore potential keyword opportunities, including semantic clusters, keyword engrams, or entities.
## Step 1: Upload Keyword File
import pandas as pd
import requests
import json
import time
import string
from nltk.tokenize import word_tokenize
from collections import Counter
from google.colab import files
import nltk
# Download necessary NLTK data
nltk.download('punkt')
# Prompt user to upload a CSV file with keywords
uploaded = files.upload()
df = pd.read_csv(list(uploaded.keys())[0]) # Read the uploaded file
keywords = df['Keywords'].tolist() # Assume the column is named 'Keywords'
## Step 2: Define Helper Functions
# Basic stopwords list for English
basic_stopwords = {
'i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you',
'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself',
'she', 'her', 'hers', 'herself', 'it', 'its', 'itself', 'they', 'them',
'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom', 'this',
'that', 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been',
'being', 'have', 'has', 'had', 'having', 'do', 'does', 'did', 'doing',
'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until',
'while', 'of', 'at', 'by', 'for', 'with', 'about', 'against', 'between',
'into', 'through', 'during', 'before', 'after', 'above', 'below', 'to',
'from', 'up', 'down', 'in', 'out', 'on', 'off', 'over', 'under', 'again',
'further', 'then', 'once', 'here', 'there', 'when', 'where', 'why',
'how', 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other',
'some', 'such', 'no', 'nor', 'not', 'only', 'own', 'same', 'so', 'than',
'too', 'very', 's', 't', 'can', 'will', 'just', 'don', 'should', 'now'
}
def get_google_suggestions(keyword, lang_code, letterlist):
"""Fetch suggestions for a given keyword and language from Google Suggest."""
suggestions = []
headers = {'User-agent': 'Mozilla/5.0'}
for letter in letterlist:
URL = f"http://suggestqueries.google.com/complete/search?client=firefox&hl={lang_code}&q={keyword} {letter}"
response = requests.get(URL, headers=headers)
result = json.loads(response.content.decode('utf-8'))
if result:
suggestions.extend(result[1])
time.sleep(0.5) # Reduced sleep for faster processing
return suggestions
def clean_and_cluster_suggestions(all_suggestions, stop_words, seed_words):
"""Clean suggestions by removing stopwords and tokenize them for clustering."""
wordlist = []
for suggestion in all_suggestions:
words = word_tokenize(str(suggestion).lower())
for word in words:
if word not in stop_words and word not in seed_words and len(word) > 1:
wordlist.append(word)
return [word for word, count in Counter(wordlist).most_common(200)]
## Step 3: Process Keywords in Batches
lang_code = "en" # Language code
batch_size = 5
letterlist = [""] + list(string.ascii_lowercase) # Include empty and alphabetical combinations
all_clusters = []
# Process keywords in batches
for i in range(0, len(keywords), batch_size):
batch_keywords = keywords[i:i + batch_size]
# Filter out empty keywords and tokenize seed words
batch_keywords = list(filter(None, batch_keywords))
seed_words = [word_tokenize(keyword.lower()) for keyword in batch_keywords]
seed_words = [item for sublist in seed_words for item in sublist] # Flatten the list
# Get suggestions for each keyword in the batch
for keyword in batch_keywords:
suggestions = get_google_suggestions(keyword, lang_code, letterlist)
most_common_words = clean_and_cluster_suggestions(suggestions, basic_stopwords, seed_words)
# Assign suggestions and common words to their seed keyword
for common_word in most_common_words:
for suggestion in suggestions:
if common_word in suggestion:
all_clusters.append([suggestion, common_word, keyword]) # Include the seed keyword here
## Step 4: Save and Download the Result
cluster_df = pd.DataFrame(all_clusters, columns=['Keyword', 'Cluster', 'Seed Keyword'])
cluster_df.to_csv("keywords_clustered.csv", index=False)
files.download("keywords_clustered.csv")
cluster_df
Extending the functionality into YouTube Search as well, I created a function that enables you to input a list of seed keywords in a CSV file and automatically fetch YouTube’s autocomplete suggestions for each term. The result is an export of your original keywords, along with a set of related suggestions retrieved from YouTube’s search engine, all neatly organized into a downloadable file.
For each seed keyword you provide, the script pulls YouTube’s autocomplete suggestions, links them back to the original keyword, and formats them into an easy-to-analyze CSV file. By organizing the suggestions this way, you can also spot common patterns or emerging search trends that might not be obvious at first glance.
For example, if you input a seed keyword like “funny videos,” the script will return related search queries like “funny videos 2024,” “funny videos for kids,” and “funny videos try not to laugh.” This allows you to quickly spot key video content ideas, helping you create videos or blog posts around these popular search terms.
This script is great for:
How to Use the Script:
Below is the Python code for this section, if you’d like to edit anything in it. Here are some quickfire ideas for editing the script:
import requests
import pandas as pd
import json
import time
from google.colab import files
def get_youtube_suggestions(keyword):
"""
Fetch YouTube autocomplete suggestions for a given keyword.
Parameters:
keyword (str): The search query string.
Returns:
list of tuples: Each tuple contains the seed keyword and its suggestion.
"""
suggestions = []
try:
url = "https://suggestqueries.google.com/complete/search"
params = {
'client': 'youtube',
'ds': 'yt',
'q': keyword,
'hl': 'en'
}
response = requests.get(url, params=params)
response.raise_for_status()
raw_data = response.text
# Extract JSON-like content from the JavaScript response
start = raw_data.find('[')
end = raw_data.rfind(']') + 1
json_data = json.loads(raw_data[start:end])
# Process suggestions
seed_keyword = json_data[0]
for item in json_data[1]:
suggestions.append((seed_keyword, item[0]))
except Exception as e:
print(f"Error fetching suggestions for '{keyword}': {e}")
return suggestions
def process_keywords(file_path):
"""
Process a list of keywords from an uploaded file and fetch YouTube suggestions.
Parameters:
file_path (str): Path to the uploaded CSV file containing a 'Keywords' column.
Returns:
DataFrame: A DataFrame containing the seed keywords and their suggestions.
"""
df = pd.read_csv(file_path)
print("Uploaded file columns:", df.columns) # Debug: Print column names
if 'Keywords' not in df.columns:
raise ValueError("The uploaded file must contain a 'Keywords' column.")
all_suggestions = []
for keyword in df['Keywords'].dropna():
suggestions = get_youtube_suggestions(keyword)
all_suggestions.extend(suggestions)
time.sleep(0.5) # To prevent hitting rate limits
result_df = pd.DataFrame(all_suggestions, columns=['Seed Keyword', 'Suggestion'])
return result_df
# Step 1: Upload the keywords file
uploaded = files.upload()
file_path = next(iter(uploaded.keys()))
# Step 2: Process the keywords and fetch suggestions
try:
suggestions_df = process_keywords(file_path)
# Step 3: Save and download the results
output_file = "youtube_autosuggestions.csv"
suggestions_df.to_csv(output_file, index=False)
files.download(output_file)
# Display first few rows of the DataFrame
suggestions_df.head()
except Exception as e:
print(f"Error: {e}")
Google’s Query Autocomplete service helps users find location-based suggestions as they type. By using the Places API, you can implement autocomplete suggestions for geographical searches, such as searching for “pizza near New York” and getting relevant suggestions like “pizza near Paris” or “pizza near Disneyland.”
Key Features of this API of why it’s so useful for keyword research:
Here’s a summary of how it works.
A Query Autocomplete request is made via a URL, where users provide an input
(search term) and, optionally, other parameters like language
, location
, and radius
. The API returns location-based suggestions based on user input.
Example request format:
https://maps.googleapis.com/maps/api/place/queryautocomplete/json?input=your_search_term&key=YOUR_API_KEY
You can specify additional parameters, such as:
There are different functions in the sheet for the Query Module and the Place Suggestion module (see below).
Note – there is a prerequisite for using this API and function. In the associated video lesson with this tutorial, I’ve demonstrated the process of enabling the API in a Google Cloud Project (one, in which there’s already billing enabled), and getting your API key. Also, prepare your replies for language code, location, search radius.
import requests
import pandas as pd
import json
import time
from google.colab import files
def get_place_autocomplete_suggestions(input_keyword, api_key, language='en', location=None, radius=50000):
"""
Fetch Google Places Query Autocomplete suggestions for a given input keyword.
Parameters:
input_keyword (str): The input text string for autocomplete.
api_key (str): The Google API key.
language (str): The language for the query results (default is 'en').
location (tuple): Latitude and longitude to bias the search (optional).
radius (int): Search radius in meters (default is 50,000 meters).
Returns:
list of tuples: Each tuple contains the input keyword and a predicted place description.
"""
suggestions = []
try:
url = "https://maps.googleapis.com/maps/api/place/queryautocomplete/json"
params = {
'input': input_keyword,
'key': api_key,
'language': language,
'radius': radius
}
# Add location bias if provided
if location:
params['location'] = f"{location[0]},{location[1]}"
response = requests.get(url, params=params)
response.raise_for_status()
data = response.json()
# Process the suggestions from the response
if data.get('status') == 'OK':
for prediction in data.get('predictions', []):
suggestions.append((input_keyword, prediction['description']))
else:
print(f"Error fetching suggestions for '{input_keyword}': {data.get('error_message', 'No suggestions found')}")
except Exception as e:
print(f"Error: {e}")
return suggestions
def process_keywords(file_path, api_key, language='en', location=None, radius=50000):
"""
Process a list of keywords from an uploaded file and fetch Google Places query suggestions.
Parameters:
file_path (str): Path to the uploaded CSV file containing a 'Keywords' column.
api_key (str): The Google API key.
language (str): The language for the query results (default is 'en').
location (tuple): Latitude and longitude to bias the search (optional).
radius (int): Search radius in meters (default is 50,000 meters).
Returns:
DataFrame: A DataFrame containing the seed keywords and their suggestions.
"""
df = pd.read_csv(file_path)
print("Uploaded file columns:", df.columns) # Debug: Print column names
if 'Keywords' not in df.columns:
raise ValueError("The uploaded file must contain a 'Keywords' column.")
all_suggestions = []
# Fetch suggestions for each keyword in the 'Keywords' column
for keyword in df['Keywords'].dropna():
suggestions = get_place_autocomplete_suggestions(keyword, api_key, language, location, radius)
all_suggestions.extend(suggestions)
time.sleep(0.5) # To prevent hitting rate limits
result_df = pd.DataFrame(all_suggestions, columns=['Seed Keyword', 'Suggestion'])
return result_df
# Step 1: Request user input for API key and parameters
api_key = input("Please enter your Google API key: ")
language = input("Enter language code (default is 'en'): ") or 'en'
location_input = input("Enter location (latitude,longitude) or press Enter to skip: ")
location = tuple(map(float, location_input.split(','))) if location_input else None
radius = int(input("Enter search radius in meters (default is 50000): ") or 50000)
# Step 2: Upload the keywords file
uploaded = files.upload()
file_path = next(iter(uploaded.keys()))
# Step 3: Process the keywords and fetch suggestions
try:
suggestions_df = process_keywords(file_path, api_key, language, location, radius)
# Step 4: Save and download the results
output_file = "place_autosuggestions.csv"
suggestions_df.to_csv(output_file, index=False)
files.download(output_file)
# Display first few rows of the DataFrame
suggestions_df.head()
except Exception as e:
print(f"Error: {e}")
Finally, I also wanted to highlight some additional no-code alternatives to Google Autocomplete:
As well as also to highlight the Vertex AI autocomplete feature under Retail, which is not no-code but might be useful to explore for those working in e-commerce and web retail.
Google’s Autocomplete APIs—across Search, YouTube, Google Maps, and Google Merchant—offer easy access to real-time keyword suggestions based on user queries. These APIs are simple to use and provide valuable insights for keyword research, content creation, and digital marketing strategies.
Google’s Autocomplete APIs pull suggestions from a wide range of platforms, including:
These APIs allow you to quickly integrate search insights into your strategy without heavy technical work.
The power of these APIs comes from Google’s machine learning, which:
Using these APIs, you can:
In short, Google Autocomplete APIs give you predictive, machine learning-powered insights to optimize your keyword research, content strategy, and marketing efforts.
Lazarina Stoy.
Beginner BERTopic FuzzyWuzzy Google Autocomplete API Google Cloud Natural Language API Google Colab (Python) Google Sheets (Apps Script) Intermediate KeyBERT kMeans OpenAI API sBERT Whisper API
Share this post on social media:
Leave a Reply