Loading
The classical models that the rest of machine learning builds on. The course covers how to split and prepare data, linear regression and gradient descent, regularisation, maximum likelihood, linear classifiers, and models based on trees and neighbours.
01
A model is only useful on data it has not seen. Hold out a test set, and use cross-validation to compare models without touching it.
02
Models that use distances or gradients need features on similar scales. Categories must become numbers, and one-hot encoding does this without inventing an order.
03
Linear regression fits a weighted sum of the features by least squares. The normal equation gives the answer in closed form.
04
Gradient descent takes repeated small steps against the gradient of the loss. The learning rate sets the step size, and feature scaling decides how well it converges.
05
A penalty on the size of the weights trades a little bias for less variance. Ridge shrinks every weight, and lasso sets some weights to exactly zero.
06
Maximum likelihood picks the parameters that make the observed data most probable. Most losses are a negative log-likelihood, and naive Bayes and Poisson regression come from the same idea.
07
A linear classifier scores each class with a weighted sum. Logistic and softmax regression turn the scores into probabilities, and an SVM looks for the widest margin between the classes.
08
A decision tree splits the data on the feature and threshold that make the groups most pure. A nearest neighbour model predicts from the closest training examples.