visual commonsense reasoning (vcr)

VCR (Visual Commonsense Reasoning) is a benchmark that tests "Theory of Mind" for AI — requiring models not just to answer questions about an image, but to provide the *rationale* for why that answer is correct, often involving social cues and unstated physical rules.

What Is VCR?

- Definition: A Q&A > R (Question -> Answer -> Rationale) task.
- Structure:
1. Question: "Why is person [1] pointing at person [2]?"
2. Answer: "He is accusing him of stealing."
3. Rationale: "Because person [2] is holding the object behind his back."
- Focus: Social situations, causality, temporal prediction.

Why VCR Matters

• Beyond Recognition: Understanding a scene requires knowing *intent*, not just pixel labels.
• Safety: Essential for human-robot interaction (understanding if a human is angry, hurried, or joking).
• Cognition: Bridges the gap between Computer Vision and Cognitive Science.

VCR is the empathy test for machines — pushing AI to understand the invisible "why" behind the visible "what".