Advantages of using ROC Curve

Data Science, AI and ML
,
#1

Why should you use ROC and not metrics like lift curve?

Lift is dependent on total response rate of the population. Hence, if the response rate of the population changes, the same model will give a different lift chart. A solution to this concern can be true lift chart (finding the ratio of lift and perfect model lift at each decile). But such ratio rarely makes sense for the business.

ROC curve on the other hand is almost independent of the response rate. This is because it has the two axis coming out from columnar calculations of confusion matrix. The numerator and denominator of both x and y axis will change on similar scale in case of response rate shift.

#2

A classifier process ideally produces only true positives and true negatives – says “yes” when it should and “no” when it shouldn’t and is never wrong. In practice these processes have some threshold between “yes” and “no” that can be tuned. You can make it say “yes” when it should 100% of the time trivially by making the threshold low – but then it will say “yes” to everything. Or if it says “no” to everything, it will always be right on the “no” cases, but never otherwise.

The ROC curve shows the tradeoff between these two across different settings of the classifier process, and that is useful in understanding something about the performance of the classifier.