Cross Entropy Loss

Implement the mean categorical cross entropy loss.

You are given probs, a (batch, classes) array of predicted probabilities where each row already sums to 1, and labels, a (batch,) array of integer class indices. The loss is the mean negative log probability assigned to the correct class:

loss = -(1 / batch) * sum_i log(probs[i, labels[i]])

A confident wrong prediction can put a probability of exactly 0 on the true class, and log(0) is -inf. Clip the probabilities into [1e-12, 1.0] before taking the log so the loss stays finite.

Function signature

def cross_entropy(probs, labels):

Example

Input

probs  = np.array([[0.7, 0.2, 0.1], [0.1, 0.8, 0.1]])
labels = np.array([0, 1])
cross_entropy(probs, labels)

Output

0.2899092476264711

Reasoning

The true classes get probabilities 0.7 and 0.8. The loss is -(log 0.7 + log 0.8) / 2 = (0.3567 + 0.2231) / 2, about 0.2899.

Constraints and notes

• Clip probabilities into [1e-12, 1.0] before the log.
• labels holds integer class indices.