synthetic data

Synthetic data is generated rather than collected, and the entire value proposition rests on one unproven claim that has to be checked every time: that training on it produces a model which performs well on real data it has never seen. Synthetic does not automatically mean private, unbiased, or realistic — a generator can memorize the real records it was trained on, amplify whatever bias was already in its source data, or fabricate combinations that could never happen in reality, and none of those failure modes shows up by just looking at the synthetic data itself. The only real test is downstream: does a model trained on it actually work in the real world it's meant to serve.

Domain randomization solved the central problem of simulation-based training by giving up on making the simulation realistic. Tobin and colleagues at OpenAI (2017) found that instead of trying to close the "reality gap" between a simulated scene and the real world by making rendering as photorealistic as possible, randomizing textures, lighting, camera position, and object colors across an enormous range during simulated training forced a robotics model to learn features that were robust to all of that variation — and because real-world conditions fell somewhere inside the wide range already seen in training, the model transferred to real cameras and real lighting without ever being trained on a single real image. The insight generalized well beyond robotics: making a generator vary wildly along irrelevant dimensions can matter more for real-world transfer than making it precisely accurate.

CTGAN showed that adapting a generative architecture to a data type's specific quirks matters more than the architecture family. Xu and colleagues (2019) found that plain GANs, built for continuous image pixels, performed poorly on tabular data because table columns mix continuous and discrete types and often follow multimodal distributions a standard GAN's assumptions don't fit — a column like "transaction amount" might have several distinct clusters rather than one smooth distribution. CTGAN addressed this with mode-specific normalization for continuous columns and conditional generation that explicitly balances rare discrete categories during training, which is why it became a standard baseline for synthetic tabular data specifically, rather than general-purpose GANs simply being pointed at a spreadsheet.

Training a frontier model substantially on synthetic text turned out to work, not just as a cost-saving trick but as a quality lever. Microsoft's Phi-1 (Gunasekar and colleagues, 2023) was trained on a comparatively small amount of carefully filtered real code plus a much larger volume of synthetic "textbook-quality" examples generated by a larger model, and it outperformed much larger models trained on orders of magnitude more unfiltered web data on coding benchmarks — evidence that curated synthetic data, generated specifically to be dense with the signal a smaller model needs, can substitute for raw data volume. Self-instruct (Wang and colleagues, 2022) applied the same bootstrap idea to instruction-following: starting from a small seed set of human-written instructions, prompting a language model to generate many more instructions and example responses in the same style, then filtering and using that synthetic set to fine-tune — a pattern now standard across instruction-tuning pipelines precisely because hand-writing enough diverse instructions at scale is expensive and generating them is comparatively cheap.

The failure mode that doesn't show up in any synthetic-data-only evaluation is memorization — a generator reproducing specific real training records rather than learning their general pattern. A generative model can score well on every standard fidelity metric while still occasionally emitting something close to verbatim a real record it was trained on, which defeats the entire privacy rationale for using synthetic data in the first place; Carlini and colleagues' work on training-data extraction from generative models established that this risk is measurable rather than theoretical, detectable through membership-inference tests that check whether a given real record is unusually easy for the model to reproduce. Differential privacy techniques applied to the generator's training process (DP-SGD, Abadi and colleagues, 2016) bound how much any single training record can influence the model's output, trading some generation fidelity for a provable limit on how much any one real record can leak back out.

A second, quieter failure mode is recursive degradation: training a new generation of models on data generated by the previous generation. As synthetic text and images from earlier models increasingly populate the web and get scraped into later training sets, each generation trained predominantly on the previous generation's output tends to lose coverage of the true distribution's tails, amplifying whatever the earlier model was already biased toward and smoothing away rare but real variation. The practical implication is that synthetic data complements real data collection rather than permanently replacing it — a pipeline that stops collecting genuine real-world examples because synthetic generation is cheaper eventually drifts away from the reality it was meant to approximate.

Read synthetic data as a hypothesis about a target distribution, not a free substitute for real collection: every generation method — physics simulation, GANs, diffusion, LLM self-instruction — is a different way of encoding an assumption about what the real data looks like, and the only way to know whether that assumption was close enough is to test the resulting model against real data it was never shown.