Average Precision

Compute the average precision (AP) of a set of scores for a binary task.

ROC AUC can look good on a very imbalanced problem even when the top of the ranking is full of negatives. Average precision focuses on the positives. It is the area under the precision-recall curve, computed as the mean of the precision at the rank of each positive:

1. Sort the examples by score from high to low. Break ties by the lower index.
2. Walk down the list. Each time you reach a positive, record the precision of the list so far: positives seen divided by examples seen.
3. AP is the mean of those precisions, over all positives.

This is also the per-class metric that object detection reports as mAP.

Function signature

def average_precision(y_true, scores):

Example

Input

y_true = np.array([1, 0, 1, 1, 0])
scores = np.array([0.9, 0.8, 0.7, 0.4, 0.3])
average_precision(y_true, scores)

Output

0.80555556

Reasoning

The order is already by score. The positives sit at ranks 1, 3 and 4, where the precision is 1/1, 2/3 and 3/4. Their mean is (1 + 0.667 + 0.75) / 3 = 0.806. The negative at rank 2 costs the most, because it pushes down every positive after it.

Constraints and notes

• y_true holds 0 and 1, with at least one positive.
• A perfect ranking, with every positive before every negative, gives 1.0.