reinforcement learning for nas

Reinforcement Learning for NAS is the original NAS paradigm where an RL agent (controller) learns to generate neural network architectures — treating architecture specification as a sequence of decisions, with the validation accuracy of the child network as the reward signal.

How Does RL-NAS Work?

• Controller: An RNN that outputs architecture specifications token by token (layer type, kernel size, connections).
• Child Network: The architecture generated by the controller is trained from scratch.
• Reward: Validation accuracy of the trained child network.
• Policy Gradient: REINFORCE algorithm updates the controller to produce higher-reward architectures.
• Paper: Zoph & Le, "Neural Architecture Search with Reinforcement Learning" (2017).

Why It Matters

• Pioneering: The paper that launched the modern NAS field.
• Cost: Original implementation: 800 GPUs for 28 days (massive compute).
• NASNet: Cell-based search (NASNet, 2018) reduced cost by searching for repeatable cells instead of full architectures.

RL for NAS is the genesis of automated architecture design — the breakthrough that proved machines could design neural networks better than humans.