Loading
How to write deep learning code in PyTorch. The course covers tensors, autograd, nn.Module, the training loop, data loading with Dataset and DataLoader, and how gradients accumulate in .grad.
01
A tensor is an n-dimensional array with a dtype and a device. Most numpy operations have a direct tensor version.
02
PyTorch records the operations on tensors that require gradients, and backward() walks that record in reverse. A custom autograd Function lets you write the backward rule yourself.
Practise it
03
An nn.Module holds parameters, buffers and submodules, and defines forward. train() and eval() switch the behaviour of layers such as dropout and batch norm.
Practise it
04
Each step clears the gradients, computes the loss, calls backward and steps the optimiser. Evaluation runs under torch.no_grad with the model in eval mode.
Practise it
05
A Dataset returns one sample by index. A DataLoader shuffles the indices and stacks the samples into mini-batches.
Practise it
06
backward() adds to .grad instead of replacing it. You can use this to sum gradients over several micro-batches and take one step on a large effective batch.
Practise it