A Confidence Score is a metric that reflects how confident the ML model is that the text aligns with the category indicated in a classification task. It is generally provided for classification results.
When comparing custom generative AI applications for text classification (e.g., using the OpenAI API), it is noted that the model’s output sometimes states that confidence scores are “N/A” and are not provided in the API’s response. In contrast, purpose-built APIs like Google Cloud NLP explicitly return a confidence score for each classification.