soft actor-critic

SAC (Soft Actor-Critic) is a state-of-the-art off-policy reinforcement learning algorithm for continuous action spaces — based on maximum entropy RL, SAC simultaneously maximizes expected reward and policy entropy, achieving sample-efficient, stable learning with automatic temperature tuning.

SAC Components

• Actor: Policy network pi_theta(a|s) outputs a Gaussian distribution over continuous actions.
• Twin Critics: Two Q-networks Q_phi_1, Q_phi_2 — use the minimum to reduce overestimation bias.
• Entropy Term: Loss includes -alpha H(pi) — temperature alpha is automatically tuned.
• Off-Policy: Stores transitions in a replay buffer — high sample efficiency.

Why It Matters

• Sample Efficient: Off-policy + replay buffer makes SAC one of the most sample-efficient model-free RL algorithms.
• Stable: Entropy regularization + twin critics prevent training instability common in actor-critic methods.
• Continuous Control: State-of-art for robotics, process control, and continuous optimization tasks.

SAC is the stable explorer — combining maximum entropy RL with twin critics for robust, sample-efficient continuous control.