The guide
Training is a loop that asks one question over and over: if I nudge this number, does the loss go up or down, and by how much? A derivative is that answer, and a gradient is the answer for every parameter at once.
The chain rule is what makes it tractable. A network is functions composed, so the derivative of the whole is the product of the derivatives of the parts, multiplied backwards from the loss. That is all backpropagation is. There is no separate algorithm to learn.
The numerical gradient you write first is slow and inexact, and you will never use it in training. You will use it constantly to check that an analytic gradient is right, which is the single most useful debugging habit in this field.