Information Gain (IG)

A measure used to evaluate how much new, meaningful information a feature, document, or phrase provides beyond what is already known; it quantifies the reduction in uncertainty/entropy when additional data is introduced.

Information Gain is a measure used by search systems to evaluate the value of content by quantifying how much new, meaningful information a feature, document, or phrase provides beyond what is already known. It quantifies the reduction in uncertainty or entropy when additional data is introduced. The IG score is often calculated as a ratio of the actual co-occurrence rate of two phrases to their expected co-occurrence rate. This score determines the strength of the relationship between phrases; if the score exceeds a threshold, the phrases are considered significantly related.