One Round of AdaBoost

Run the weight update of one AdaBoost round.

AdaBoost trains weak learners one after another. After each learner, it gives the learner a vote weight alpha based on how accurate it was, then raises the weights of the samples that the learner got wrong. The next learner must then pay more attention to those hard samples.

Given the true labels y, the learner's predictions pred (both in {-1, +1}) and the current sample weights w:

err   = sum(w where pred != y) / sum(w)
alpha = 0.5 * ln((1 - err) / err)
w_new = w * exp(-alpha * y * pred)
w_new = w_new / sum(w_new)

Clip err into [1e-10, 1 - 1e-10] before you compute alpha, so that a perfect learner does not divide by zero. Return (alpha, w_new).

Function signature

def adaboost_round(y, pred, w):

Example

Input

y    = np.array([1, 1, -1, -1])
pred = np.array([1, -1, -1, -1])
w    = np.full(4, 0.25)
adaboost_round(y, pred, w)

Output

(0.5493061443, array([0.16666667, 0.5, 0.16666667, 0.16666667]))

Reasoning

One sample of four is wrong, so err = 0.25 and alpha = 0.5 * ln(3) = 0.5493. Correct samples are multiplied by exp(-alpha) = 1 / sqrt(3) and the wrong one by sqrt(3). After normalisation, the wrong sample holds half of the total weight. This is always true in AdaBoost: after the update, the last learner has an error of exactly 0.5.

Constraints and notes

• y * pred is +1 for a correct prediction and -1 for a wrong one.
• Do not modify w in place.