The ROC curve

Intro

ROC stands for receiver operating characteristic and is a curve first designed and used at the times of World War I for radar signals purposes. It is a curve used to validate the performance of a binary classifier which depends on a threshold parameter when it varies.
The ROC plots the true positive rate against the false positive rate at each varying value of the parameter, also called threshold. See the note on the performance metrics for classification for a description of those. In this plane, the point (0, 1) represent the perfect classification; the diagonal line shows what a random guesser (a coin flip) would give, so that points above it are good results, points below are poorly classified results.

Details: definition of the ROC curve and the AUC

Discussion borrowed from Wikipedia. Let's say that we got a continuous random variable
XX
and a binary classifier which depends on a threshold
TT
, so that
x>Tx > T
yields a "positive" (1) classification and
xTx \leq T
yields a "negative" (0) classification. The pdf of
XX
is
f1(X)f_1(X)
if the point is actually positive and
f0(X)f_0(X)
if the point is actually negative, so we can write
TPR(T)=Tf1(x) dxTPR(T) = \int_T^\infty f_1(x) \ d x
and
FPR(T)=Tf0(x) dxFPR(T) = \int_T^\infty f_0(x) \ d x
(the last one because above
TT
the point is classed as positive but it is negative, so follows
f0f_0
). The ROC curve, plots TPR(T) versus FPR(T) as a parametric function of
TT
. At the same time, the other two metrics which quantify the performance of the classification can be expressed as
TNR=Tf0(x) dxTNR = \int_{-\infty}^T f_0(x) \ d x
and
TFR=Tf1(x) dxTFR = \int_{-\infty}^T f_1(x) \ dx
(the last one because below T the point is classed as negative but it is actually positive so it follows
f1f_1
).
The area under the curve (AUC) tells us how fast the curve grows (how convex it is), quantifying how good its performance is. In a typical use case, different classifiers will be evaluated by comparing their AUCs.

References

  1. 1.
    Wikipedia on the ROC
Last modified 7mo ago