Before you start
Remove duplicate detections of the same object.
A detector fires many times around each object. NMS keeps the most confident box and discards anything that overlaps it too much:
iou_threshold.Return the kept indices as a list, in the order they were selected, which is descending by score.
Boxes are (n, 4) as [x1, y1, x2, y2] and scores are (n,).
Note the strict comparison: a box is dropped when IoU is strictly greater than the threshold, so at exactly the threshold it survives. That is the convention torchvision uses, and it is the kind of off-by-one that silently changes a detector's precision.
Suppression is not transitive. A box only ever competes with a box that was actually kept, never with one that was already suppressed. Comparing against suppressed boxes would remove far too much.
def non_max_suppression(boxes, scores, iou_threshold):
Input
boxes = np.array([[0, 0, 10, 10], [1, 1, 11, 11], [20, 20, 30, 30]])
scores = np.array([0.9, 0.8, 0.7])
non_max_suppression(boxes, scores, 0.5)
Output
[0, 2]
Reasoning
Box 0 has the top score and is kept. Box 1 overlaps it with IoU 81 / 119 = 0.68, which is above 0.5, so it is dropped. Box 2 does not overlap box 0, so it is kept.
Check runs the visible cases against a fresh kernel. Submit runs the hidden ones too.
⇧↵run cell