Loading
The probability and statistics that models are built on. The course covers events and Bayes' theorem, discrete and continuous distributions, expectation and covariance, hypothesis tests, entropy and divergence, maximum likelihood, and Monte Carlo estimates of expectations and their gradients.
01
Conditional probability restricts attention to the cases where something is known to be true. Bayes' theorem turns a probability of evidence given a cause into a probability of the cause given the evidence.
Practise it
02
The binomial counts successes in a fixed number of trials, and the Poisson counts events in a fixed interval. Both come from simple assumptions about independent events.
03
The normal distribution is set by a mean and a standard deviation. Sums of many small independent effects tend towards it, which is why it appears everywhere.
04
The expectation is the long run average of a random variable, and the variance is its spread. Covariance measures how two variables move together, and the covariance matrix holds it for every pair.
05
A test asks how surprising the data would be if there were no real effect. The t-test compares two means, and Welch's version does not assume equal variances.
Practise it
06
Entropy measures how uncertain a distribution is. The KL divergence measures how much extra it costs to use one distribution in place of another, and it is never negative.
07
An expectation can be estimated by averaging over samples, with an error that shrinks like one over the square root of the sample count. Gradients of expectations need special estimators, such as the reparameterisation trick.
Practise it
08
The likelihood scores parameters by how probable they make the observed data. A model that predicts a whole distribution can be trained on it and can say how unsure it is.
Practise it