Reinforcement learning (RL) trains an agent to make decisions by rewarding good outcomes.
Key Concepts
- Agent: the learner and decision-maker.
- Environment: what the agent interacts with.
- State: the current situation.
- Action: what the agent can do.
- Reward: feedback signalling how good an outcome was.
- Policy: the agent's strategy for choosing actions.
The Goal
Maximise cumulative reward over time, which may require sacrificing short-term reward for long-term gain.
Exploration Versus Exploitation
Agents must balance trying new actions to learn with using known good actions.
Main Approaches
- Value-based: learn how good states or actions are (Q-learning).
- Policy-based: learn the policy directly (policy gradients).
- Actor-critic: combine both.
Successes
Game-playing systems, robotics control, recommendation, resource management and training language models from human feedback and verifiable rewards.
Challenges
- Needs many interactions; simulation helps.
- Reward design is hard — agents exploit poorly specified rewards.
- Safety during learning in real environments.