Does synthetic data improve AI performance when added to real data?
Yes, and the improvement is measurable. In embryo cell stage prediction, adding synthetic images to a real dataset raised classification accuracy from 94.5% to 97% — a 2.5 percentage point gain [1]. This means the model made fewer mistakes on critical tasks like identifying viable embryos for IVF. The same trend held when tested on data from a different clinic, showing the benefit is not just a fluke [1].
Even more striking: a model trained exclusively on synthetic embryo images and tested on real images achieved 92% accuracy [1]. That is only 2.5 percentage points below the model trained on real data alone (94.5%), suggesting synthetic data can sometimes stand in for real data when real data is scarce. However, the best performance always came from mixing both.
What is the 'reality gap' and why does it matter?
The 'reality gap' is the mismatch between synthetic and real data that can cause AI models to fail when deployed. In a study on forming technology (metal stamping), simple simulation models (L1) worked well for classifying failures within the synthetic domain but failed to transfer to real-world process data [4]. More complex simulations (L2 and L3) narrowed this gap by better capturing real-world physics, but they still could not fully close it — saliency maps showed models trained on synthetic data focused on different signal regions than those trained on real data [4].
This gap is not unique to manufacturing. In underwater sonar mapping, researchers explicitly state that 'synthetic data cannot replace the value of real-world data' but can be a 'valuable supplement' [5]. The key takeaway: synthetic data is excellent for augmenting real data, but relying on it alone risks poor real-world performance.
Does the quality of synthetic data affect results?
Absolutely. In the embryo study, synthetic images from a diffusion model fooled embryologists 66.6% of the time (they thought they were real), while images from a generative adversarial network (GAN) fooled them only 25.3% of the time [1]. The diffusion model also achieved a lower Fréchet inception distance (FID) score, a standard measure of image quality. When both types of synthetic data were combined, classification accuracy was higher than using either alone [1].
In blind super-resolution (enhancing low-resolution images), the Real-ESRGAN model was trained on pure synthetic data but used a sophisticated 'high-order degradation modeling' process to simulate real-world blur and noise [3]. This approach produced visually superior results on real images compared to prior methods, proving that careful design of synthetic data can narrow the reality gap. However, the authors still note that synthetic data cannot fully replace real-world examples for all scenarios.
Sources used in this answer
Merging synthetic and real embryo data for advanced AI predictions.
Adding synthetic embryo images to real data improved classification accuracy from 94.5% to 97%; a model trained only on synthetic data achieved 92% accuracy on real data.
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
LLMs can generate synthetic text and code to augment real datasets, but challenges include factual inaccuracies, distributional realism gaps, and risk of bias amplification.
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data
Real-ESRGAN, trained on pure synthetic data with high-order degradation modeling, achieved superior visual performance on real-world images compared to prior methods.
Requirements for numeric models as sources of synthetic data for predicting real-world data sets
Simple simulation models (L1) failed to transfer to real data; more complex models (L2/L3) improved domain alignment but still showed differences in saliency maps vs. real-data-trained models.
Synthetic Sonar Generation: Leveraging Algorithmic and AI-Based Approaches for Enhanced Underwater Mapping and Exploration
Synthetic sonar data cannot replace real data but serves as a valuable supplement for training AI models and enhancing analysis of real data.