dqn

DQN (Deep Q-Network) is the foundational deep reinforcement learning algorithm that combines Q-learning with deep neural networks — using a CNN to estimate the action-value function Q(s,a) from raw pixel inputs, stabilized by experience replay and a target network.

DQN Innovations

• Experience Replay: Store transitions (s, a, r, s') in a replay buffer — sample random mini-batches for training.
• Target Network: A slowly-updated copy of the Q-network provides stable targets: y = r + \gamma max_{a'} Q_{target}(s', a').
• epsilon-Greedy: Explore with probability epsilon, exploit with probability 1-epsilon.
• Loss: L = (y - Q_\theta(s, a))^2 — minimize the temporal difference error.

Why It Matters

• Breakthrough: DQN (Mnih et al., 2015) was the first deep RL to achieve human-level performance on Atari games.
• End-to-End: Learns directly from raw pixels to actions — no hand-crafted features.
• Foundation: DQN spawned an entire family of improvements (Double DQN, Dueling DQN, Rainbow).

DQN is deep learning meets Q-learning — the algorithm that launched the deep reinforcement learning revolution.