ROC AUC

Compute the area under the ROC curve from raw scores.

Rather than sweeping thresholds and integrating, use the equivalent rank based definition. AUC is the probability that a randomly chosen positive is scored above a randomly chosen negative:

AUC = (sum of ranks of the positives - n_pos * (n_pos + 1) / 2) / (n_pos * n_neg)

where ranks are 1-based over the scores sorted ascending. This is the Mann-Whitney U statistic, and it gives the exact same number as the area under the curve with far less code.

Ties matter. Samples with equal scores must all receive the same rank, the average of the positions they occupy. Without that correction a model that outputs one constant score would appear to be a perfect classifier rather than a coin flip.

If either class is missing entirely, AUC is undefined, so return 0.5.

y_true holds labels in {0, 1} and scores holds any real numbers, where higher means more likely positive.

Function signature

def roc_auc(y_true, scores):

Example

Input

y_true = np.array([0, 0, 1, 1])
scores = np.array([0.1, 0.4, 0.35, 0.8])
roc_auc(y_true, scores)

Output

0.75

Reasoning

There are 4 positive and negative pairs. 0.8 beats both negatives, and 0.35 beats 0.1 but not 0.4. That is 3 of 4 pairs, so AUC is 0.75.

Constraints and notes

• Tied scores get the average of their ranks.
• Return 0.5 when one class is missing.