bootstrap your own latent

BYOL (Bootstrap Your Own Latent) is a self-supervised learning method that achieves state-of-the-art representation learning without negative samples — using a teacher-student architecture where the student (online network) learns to predict the teacher's (target network) representations, with the teacher updated via exponential moving average.

How Does BYOL Work?

• Two Networks: Online (student) and Target (teacher, EMA of online).
• Process: Two augmented views of the same image. Online network predicts the target network's representation for the other view.
• No Negatives: Unlike SimCLR/MoCo, BYOL doesn't need negative pairs.
• Collapse Prevention: The EMA update of the target network prevents representational collapse.

Why It Matters

• No Negatives Needed: Eliminates the dependency on large batch sizes or memory banks.
• Performance: Matches or exceeds SimCLR on ImageNet with simpler training.
• Influence: Demonstrated that contrastive negatives are not strictly necessary for good representations.

BYOL is self-supervised learning without the contrast — proving that you can learn excellent representations by simply predicting your own augmented views.