A measure of certainty provided by a generative AI model regarding its classification.
A Confidence Score is a metric that reflects how confident the ML model is that the text aligns with the category indicated in a classification task. It is generally provided for classification results.
When comparing custom generative AI applications for text classification (e.g., using the OpenAI API), it is noted that the model’s output sometimes states that confidence scores are “N/A” and are not provided in the API’s response. In contrast, purpose-built APIs like Google Cloud NLP explicitly return a confidence score for each classification.
Sources & References
Explore other Evaluation & Metrics terms
A
Accuracy Score
A metric used in systems that predict query responses; the decision to display a short…
B
Bounce Rate
A GA4 user engagement metric used to monitor patterns in user interaction.
C
Coherence Score
Used for evaluating the quality of topics produced by algorithms like LDA; high coherence suggests…
C
CPC (Cost Per Click)
A metric used in keyword analysis and visualizations.
C
CTR (Click-Through Rate)
A metric related to user interaction with search results.
E
Emotion Scores
Specific numerical scores returned by emotion analysis (e.g., sadness score, joy score, fear score, disgust…
E
Entity Salience Score
A score assigned by the Google Natural Language API to each extracted entity, indicating its…
K
Keyword Difficulty (KD)
A metric associated with keywords, used in prioritization and visualization.
Q
Query Distance
A measure of how similar queries are to one another, often calculated using fuzzy matching.
S
Salience (Importance)
A metric indicating the importance or prominence of an entity in the context of the…
S
Search Volume (Volume)
A traditional keyword metric, referring to the monthly search volume, used in analysis and prioritization.
S
Sentiment Magnitude
A metric returned by the Google Natural Language API alongside Sentiment Score, used in entity…