The guide
In reinforcement learning there are no labels. An agent acts, the world responds with a reward, and the agent must find out which actions lead to high rewards, often many steps later.
The central object is the value function, the expected discounted return from a state. The Bellman equations say that a state's value is its reward plus the discounted value of where it leads. Policy evaluation and value iteration solve these equations when the world is known. Q-learning learns the same values from experience alone.
Policy gradient methods learn the policy directly. Their difficulty is variance, and advantages, here estimated with GAE, are the standard way to reduce it. Bandits are the simplest case of all: one state, and the whole problem is the trade between exploring and exploiting.