Gaussian Naive Bayes

Fit and predict with a Gaussian naive Bayes classifier.

For each class c estimate a prior and, for each feature independently, a mean and variance:

prior_c   = count(c) / n
mean_cj   = mean of feature j over the rows of class c
var_cj    = variance of feature j over the rows of class c   (divide by the count)

The "naive" part is assuming the features are independent given the class, which turns a joint density into a product. Take logs and the product becomes a sum:

log posterior(c) = log prior_c + sum_j log N(x_j ; mean_cj, var_cj)

log N(x ; mu, var) = -0.5 * log(2 * pi * var) - (x - mu)^2 / (2 * var)

Predict the class with the highest log posterior, breaking ties towards the smaller class label.

Two details the tests check:

• Work in log space. Multiplying many small densities underflows to zero once you have more than a few dozen features, and then every class ties at zero.
• Floor the variance at 1e-9. A feature that is constant within a class has zero variance, and dividing by it gives inf.

Classes are integers in [0, num_classes). Return predictions of shape (m,).

Function signature

def fit_predict(X_train, y_train, X_test, num_classes):

Example

Input

X_train = np.array([[1.0], [2.0], [8.0], [9.0]])
y_train = np.array([0, 0, 1, 1])
X_test  = np.array([[1.5], [8.5], [5.0]])
fit_predict(X_train, y_train, X_test, num_classes=2)

Output

array([0, 1, 0])

Reasoning

Class 0 has mean 1.5 and class 1 has mean 8.5, both with variance 0.25 and prior 0.5. 1.5 and 8.5 go to their own classes. 5.0 is exactly halfway, so the log posteriors tie and the smaller label, 0, wins.

Constraints and notes

• Work in log space.
• Floor each variance at 1e-9.
• Break ties towards the smaller class label.