Comparing NLP APIs for Entity and Sentiment Analysis (vs Generative AI)

“Just use ChatGPT for it.” I hear this constantly about entity analysis, and I understand why, it is accessible and easy. But entity analysis sits at the centre of so many SEO projects, understanding your content at scale, the queries you rank for, the reviews customers leave, that getting it right genuinely matters. And as I am going to show you with a hands-on experiment, generative AI is quietly the wrong tool for the job. By the end of this post you will know which API to reach for, why, and how to defend that choice to a client or a sceptical colleague. This is the comparison lesson from the entity module of the Introduction to Machine Learning for SEO course, and it is one of the ones I most enjoy running, because the results are so clear-cut.

The Contenders: Purpose-Built NLP APIs vs Generative AI

On one side, pre-trained, task-specific NLP APIs from the tech giants: Google Cloud Natural Language API, Amazon Comprehend and IBM Watson NLU (you could add Microsoft Azure to this group too). On the other, the generative AI camp: ChatGPT (OpenAI’s GPT models), Google Gemini, DeepSeek R1 and Alibaba Qwen, a deliberate mix of the established players and the newer arrivals, so the test is fair.

The first thing to understand is that these NLP APIs are not single-trick tools. Each bundles a suite of modules for different natural-language tasks, and they overlap in useful and slightly different ways.

Google Cloud Natural Language API gives you entity identification (it inspects text for known entities and returns rich information on each), entity sentiment (the sentiment attached to a specific entity, so a review that mentions “price” negatively gets a negative score on that entity), document-level sentiment, and, uniquely among the three, content moderation, plus text classification and syntax analysis. Amazon Comprehend covers entities, entity sentiment and document sentiment, and adds key-phrase (keyword) extraction, language detection, PII detection and syntax analysis. IBM Watson NLU is the most feature-rich for our purposes: on top of entities, entity sentiment and document sentiment, it does entity-relationship mapping (its “relations” module, genuinely useful for understanding how the concepts in a text connect), emotion detection (joy, anger, sadness and more), key phrases, concepts, classification, metadata and semantic roles.

How you actually access them

Getting started with each is more approachable than it sounds. Google Cloud’s API is the one we lean on most, and there are free no-code MLforSEO templates for every module: you paste your URLs and scraped content into a Google Sheet, run the script from a menu, and get entity data back in seconds, complete with the entity, its type, its importance in the document, a sentiment score, the number of mentions and the different phrasings used, and metadata such as a Wikipedia URL or a Knowledge Graph identifier. Amazon Comprehend sits inside AWS: create an account, use the free tier (around 100 to 200 dollars of credits, enough to test without risk), and run one-off analyses straight from the console demo, downloading results as a CSV, or wire it into code for scale. IBM Watson NLU needs an IBM Cloud account, the API enabled for your project, and an API key plus a service URL to authenticate, whether you work in Python or Apps Script, and it rewards the setup with the richest feature set of the three. The point is not that you must master all three tomorrow, but that each is a few clicks from a first result, and together they cover almost every text-analysis job you will meet.

Whichever you pick, a purpose-built API hands you far more than a bare list of entities: the type, the salience, sentiment and magnitude, mention counts, the varied phrasings, and metadata linking the entity to a knowledge base. That structured richness is exactly what makes real downstream analysis possible, and it is exactly what a chatbot cannot give you.

The experiment: same text, same prompt

Talk is cheap, so I ran the same piece of text through every option, and the same prompt through every chatbot, to keep it fair. The results are stark.

Google Cloud’s NLP API identified 71 entities, 59 of them unique, across several types, the clear winner. Amazon Comprehend returned 22 (16 unique) and IBM Watson 17 (16 unique). Those look lower, but there is an important nuance worth understanding, because it will stop you drawing the wrong conclusion. Amazon and IBM classify some things Google labels as “other” entities, concepts like “big data”, “machine learning” or “marketing”, as key phrases instead. Run Amazon’s key-phrases model on the same text and you add roughly 50 more phrases, which completely changes the picture. So part of the gap is a labelling difference in how each provider’s database is organised, not a raw capability gap, which is precisely why you should test the same text across APIs before choosing, and why knowing each API’s modules matters.

On the generative side, DeepSeek R1 did surprisingly well with 38 unique entities across nine types, Qwen returned 25, ChatGPT 14 to 15 (mostly person, organisation and location), and Gemini, remarkably, returned zero across every model I tried, Pro, deep research and the experimental 2.0. It simply did not attempt the task. So if you are absolutely set on using generative AI, DeepSeek is the pick. But I still would not recommend it, and here is why.

Why generative AI falls short

Scalability. An NLP API processes your whole site, keyword universe or internal-link audit with a single line of code. A chatbot does one-off analysis, one text at a time, which does not survive contact with a real project. Richer data. Only a purpose-built model returns entity type, salience, mention counts, metadata and relationships; a chatbot gives you a bare list you then have to enrich yourself. Repeatability. An NLP API returns the same result every run, so anyone can reproduce your analysis; generative output varies each time, which quietly erodes trust the moment a client re-runs it and gets different numbers, and can leave you defending an analysis nobody else can replicate.

On top of that, generative models will hallucinate entities that are not in the text (especially through the API rather than the chat interface), return unstructured output that resists further data analysis, mis-attribute categories, and cost more while running slower at scale. The root cause is simple and worth stating plainly: there is no database of entities inside a GPT that it matches your text against. There is one inside Google Cloud, Amazon and IBM, built and maintained over years. Generative models are trained to generate plausible text, not to reliably extract and classify structured facts, and no amount of prompting fully fixes a mismatch that fundamental. Generative AI is superb at content transformation, summarising, rewriting, question-answering, which we cover later in the course; it is just the wrong instrument for structured text analysis.

What about sentiment analysis?

A quick but instructive detour, because sentiment is the other half of semantic analysis. In an experiment comparing Google, Amazon and IBM against a manually labelled dataset, Amazon and Google did well on positive and negative samples with good precision and recall, while IBM was more precise on identifying neutrality. Amazon and Google also agreed with each other most often (they matched most pairwise), while IBM diverged more. But here is the humbling headline: all three matched the human label only about 43% of the time.

The lesson is not that these APIs are bad, they are excellent at scale, processing hundreds of thousands of entries in minutes. It is that you still need a human review layer before you present or act on the results, and you should test a few APIs on your own data before committing, because performance genuinely varies by dataset. Generative AI for sentiment, meanwhile, is impractical and chaotic out of the box; you can nudge it with zero-shot, one-shot or few-shot prompting and a low temperature setting for more deterministic output, but the results stay questionable, and for an introductory workflow it simply is not worth fighting a model to do a job a purpose-built API does better out of the box. (Google’s Vertex AI is one route if you are set on the generative path, but it is beyond the scope of a first project.)

Free vs Paid: Cloud APIs vs Open-Source Libraries

You do not have to use a paid cloud API. Open-source options like spaCy, NLTK, Stanford NLP and Hugging Face transformers (including BERT and DistilBERT) support entity recognition too, though spaCy and NLTK are more often used for the text-processing steps around it, like tokenisation and vectorisation.

Cloud solutions win on instant plug-and-play usage, a broad suite of advanced features in one API, automatic scaling, and easy integration, especially if your data already lives in BigQuery, where Google’s Natural Language API drops in neatly. They are great for beginners and non-technical teammates, and slot into more complex architectures when you need them. Open-source and local solutions shine when you need full data control for regulated projects (healthcare, finance, or anywhere you cannot send data to a third party), when cost at very high volume is a concern, or when you want to deeply customise and fine-tune for a specialised domain. One tip I always give: run a cost test on a cloud API before ruling it out, because the price for large volumes is often surprisingly small relative to the time you save. Ultimately the choice comes down to budget, scale, data-privacy needs, the features you want, how quickly you can reach them, and how easily your team can adopt the workflow.

Why Reliable Entity Data Matters for AI Search

There is a bigger reason to care about doing entity extraction properly than tidy data. AI search systems build answers out of facts, and facts are entities and their relationships, drawn from the same kind of knowledge graphs the big providers have spent years constructing. When you analyse your content and your SERPs with the same class of purpose-built model these systems rely on, you are auditing your brand the way the machines actually read it: which entities you are associated with, how confidently, and with what sentiment. A generative chatbot cannot give you that picture reliably, because it has no entity database to match against, so it guesses. If your goal is to understand and improve how AI search perceives your brand, you want the tool that shares its underlying model of the world, not the one that happens to be easiest to open in a browser tab.

Where to Take Entity and Sentiment Analysis Next

This post makes the case for purpose-built NLP APIs. To put one to work, start with the free Entity Extraction with Google’s Natural Language API guide, and read what entity extraction is and where it fits in SEO for the concepts. Then see it all applied in the semantic analysis of customer reviews walkthrough, which uses Google Cloud and IBM Watson together. The full Entity Extraction and Analysis module inside Introduction to Machine Learning for SEO takes each API hands-on, module by module.

Related glossary terms

• Entity extraction (NER)
• Entity sentiment analysis
• Entity salience score
• Sentiment analysis
• BERT