What's Kappa Score?🙄

Data Science, AI and ML
#1

Kappa Score is an interesting, but an underused metric. Also known as Cohen’s Kappa Statistic on the name of Jacob Cohen, Kappa Score is very useful when evaluating classification models. How? Read on.👁️‍🗨️

The fundamental concept behind the Kappa Score is it measures the amount of “agreement” between two values. In classification, one of this is the predicted value and other is the ground truth.:bulb:

Kappa score takes into consideration not 1, but 2 different accuracy measures. One is the usual Predictions accuracy, and the other one is the Expected accuracy.:thinking:

The expected accuracy is the accuracy which can be attained by any random predictions.

Kappa Score is calculated as:
K = (Predicted accuracy - Expected accuracy)/(1 - Expected accuracy)

So, if K = 0.4, and expected accuracy is 50%, you can say that your classifier is performing 40% better than the random predictions, meaning a prediction accuracy of 70%.:bulb:

However, if your expected accuracy itself was 70%, and the model also gave 70% accuracy, K will be 0.:small_red_triangle_down:

A low value of K means, a low level of “agreement” between the classifier and the ground truth.:bulb:

Kappa Score can also be used to compare the performance of 2 models in the same fashion.:bulb:

#datascience #machinelearning