Synthetic data is data generated by software instead of recorded from real events. Teams use it for four things: to test a system on situations that have not happened yet, to get training examples that arrive already labeled, to teach a model rare cases such as fraud or equipment failure, and to work with data that is too sensitive to hand over.
So when should you use it? When the real data you need is too rare, too sensitive, or does not exist yet, and you can describe what is missing well enough for someone to generate it. If more real data can be collected with reasonable effort, or a simpler change to the model would fix the problem, those routes are usually cheaper and more reliable.
The key question is whether you can describe the missing data well enough to generate it. Synthetic data can teach a model expert knowledge that is absent from the existing records, but generating examples does not verify assumptions about an unfamiliar process. Before starting a project, identify what your real data is missing. How much to generate comes later.
This article goes through each use with the condition that has to hold and the cheaper alternative when it does not. It then covers the situations where synthetic data is the wrong tool, and how to check whether it helped.
Synthetic data depends on what you know about the real process
Every synthetic dataset comes from one of three sources. A statistical model learns the patterns in your existing records and samples new records with the same patterns. A simulation follows the physics or rules of a process, such as a machine wearing down or traffic moving through an intersection, and records what happens. A generative model, such as a language model or an image model, produces new text or images from a description of what is wanted.
In all three cases, the output can only contain what went in. A statistical model can reproduce the relationships in your records, but it cannot add a pattern your records never showed. A simulation is as accurate as the rules it was given. A generative model produces the requested example based on what it learned during training. None of them observes the world.
This limit is also where its value comes from, because people often know things their data does not show. An engineer knows what a failing bearing sounds like, but the plant has recorded it only twice. A compliance team knows how a new fraud scheme works, but it has not yet hit your transactions. Synthetic data is how that knowledge becomes training and test material.
The same limit explains every condition in the rest of this article. Each use of synthetic data requires a different level of understanding of the real process, and each fails in the same way when that knowledge is missing: the model learns the generator’s assumptions and treats them as facts.
What is synthetic data used for?
The four common uses differ in how closely the generated data has to match reality. Testing needs the least. Working with sensitive data needs the most, because it has two demands that pull against each other. The sections below go in that order.
Testing situations that have not happened yet
The safest use of synthetic data is testing. The generated cases are used to find where the model breaks, before a real case finds it in production, and nothing is added to the training set.
Autonomous driving is the best-known example. Waymo describes using its simulator to prepare its driving system for rare events, sampling many variations of a situation, such as a car tailgating through an intersection, to see how the system responds across the possible outcomes. Waymo, Simulation City The same idea applies far from the road. A grid operator can test a forecasting model against a simulated heat wave larger than any in its history. A bank can run its transaction monitoring against a fraud pattern its compliance team has read about but not yet seen. A document processing system can be tested on a form layout that a supplier has announced but not yet sent.
A test scenario only has to be plausible: something a person who operates the system would agree could happen. That is a much lower bar than training data has to clear, which is why testing carries the least risk. If a generated scenario is a little unrealistic, the worst outcome is a test that finds nothing.
Someone needs to judge whether the scenario is plausible. You also need to be able to run the system on it and record the output. The failures a scenario uncovers are worth keeping as regression tests, so that later versions of the model are checked against them too.
Getting training examples that arrive already labeled
Much of the cost of training a model is labeling: someone has to mark where the defect is in each photo, which field on a form holds the invoice total, or which intent each customer message expresses. Generated data often skips that step, because the generator knows the answer when it creates the example.
A rendered image of a part with a scratch comes with the scratch’s exact outline. An invoice generated from a template comes with every field’s position and value, which is useful for training document extraction models. A simulated sensor trace comes with the exact moment the injected fault begins. A customer message written to express a specific complaint comes with its intent. Across images, documents, sensor data, and text, the label is a by-product of generation rather than a separate job.
For this to work, the generated examples need to cover the variety found in real inputs. This does not always mean photorealism. In one widely cited robotics study, a model trained only on simulated images with deliberately unrealistic, randomly varied textures located objects in real camera images to within 1.5 cm. The variation was wide enough that the real world looked like one more variation. Tobin et al., Domain Randomization What fails is narrow generation: invoices from three templates when suppliers use forty, or customer messages that are all grammatical when real ones are full of abbreviations and typos. A model trained on that data does well on the generated test set and poorly on real inputs.
The usual fix is to fine-tune the model on a small set of real, labeled examples after training on generated data. Use a separate set of real examples to test it. When you already have plenty of real, unlabeled inputs and labeling one takes seconds or minutes, two alternatives are often cheaper than building a generator. Pre-labeling uses a model to propose labels that a person only corrects. Active learning asks people to label only the examples the model is least sure about, which is often a small fraction of the total.
Teaching a model rare cases
The cases that cost the most are often the ones with the fewest examples. A plant may have years of sensor data and only a handful of recorded failures. A payment system may process millions of legitimate transactions and only a few hundred confirmed frauds. A model trained on that data learns that the rare case almost never happens. That is accurate, and it does nothing to help anyone catch the next one.
Synthetic examples of the rare case can change that, provided someone can describe what the rare case looks like in the data. A maintenance engineer who knows that a bearing failure is preceded by a rise in a specific vibration frequency over several days can turn that knowledge into realistic traces. A fraud analyst who knows the sequence of small test payments that comes before a large fraudulent one can describe it precisely enough to generate. Here the synthetic data is a way of getting an expert’s knowledge into the training set.
When nobody can describe the rare case, generating examples of it means multiplying a guess. The model will learn to recognize the guess, and the real case, when it arrives, may look different. In that situation a different approach usually works better: anomaly detection learns what normal behavior looks like and flags what departs from it, without needing examples of every way things go wrong.
Before generating examples, check whether more real ones are available. If more real examples of the rare case can be collected in a few weeks, for example by searching older records, asking other sites, or labeling cases that were handled but never recorded as such, real examples are worth more than generated ones.
Working with data you cannot hand over
A hospital wants a software team to build a triage tool, but patient records cannot leave the hospital. A bank wants a vendor to test a new scoring system, but customer data cannot be shared. A public institution wants to work with a university on a research question, but the records involve citizens. In each case, a synthetic copy of the data, with the same columns, distributions, and relationships but no real people, would let the work go ahead.
This is the most demanding use of synthetic data, because it has two requirements that pull in opposite directions. The data has to keep the relationships that matter for the work, such as how age, diagnosis, and length of stay relate to each other. It also has to avoid revealing anything about the real people it was learned from. The closer a generator follows the real data, the more useful the output and the higher the risk.
Synthetic data is not anonymous by default. A generator can memorize individual records and reproduce them almost unchanged, especially records that are unusual. Someone with access to the synthetic data can sometimes tell whether a specific person was in the original data, which is called a membership inference attack. A study of several generative models presented at USENIX Security found that synthetic data either failed to prevent such inference attacks or lost much of its usefulness in the process, and that its privacy protection was harder to predict than that of traditional anonymization. Stadler, Oprisanu and Troncoso, USENIX Security 2022
Under Turkey’s personal data protection law (KVKK), data counts as anonymized only if it cannot be linked to an identified or identifiable person, even when it is matched with other data. KVKK regulation on deletion, destruction, and anonymization European and UK regulators take a similar view: whether a synthetic dataset is anonymous depends on an assessment of how identifiable people are, not on the method used to produce it. Ganev, When Synthetic Data Met Regulation
In practice, that means testing the output before it is released. Look for records that are exact or near copies of real ones, look for rare combinations of values that point to one person, and, where the risk justifies it, run membership inference tests. Generators trained with differential privacy offer a formal guarantee, at a cost in how closely the output follows the real data.
Which records would you release?
A hospital wants to share generated patient records with a software team. Mark the generated rows you would hold back.
| Age | District | Diagnosis | Days | Decision |
|---|---|---|---|---|
| Real data | ||||
| 41 | Çankaya | Appendicitis | 2 | |
| 94 | Yusufeli | Wilson's disease | 11 | |
| Only one real patient is over 90 and lives in Yusufeli. | ||||
| Generated data | ||||
| 63 | Üsküdar | Type 2 diabetes | 5 | |
| 41 | Çankaya | Appendicitis | 3 | |
| 93 | Yusufeli | Heart failure | 9 | |
| 52 | Karşıyaka | Pneumonia | 7 | |
Holding back 0 of 4 rows.
When those checks are not practical, two alternatives are often simpler. For software development and testing, masking or pseudonymization (replacing names and identifiers with codes) can be enough, though under KVKK pseudonymized data is still personal data and has to be handled as such. For training models, the work can often go to the data rather than the data to the work: the model is developed and trained inside the institution’s own environment, on the real records, and only the trained model or its results leave.
When synthetic data is the wrong tool
The uses above share a pattern: something specific is missing, and someone knows enough to generate it. Three common situations do not fit that pattern, and in each one a different route is cheaper or safer.
Imbalanced data usually needs a different fix
When one class is much rarer than the other, such as fraudulent against legitimate transactions, the first instinct is often to generate more examples of the rare class. Techniques like SMOTE, which creates new minority examples by interpolating between existing ones, are the standard version of that instinct.
The research is less favorable than the method’s popularity suggests. A study across 73 datasets found that balancing helped weak classifiers, but that the best results came from strong gradient-boosting models trained without any balancing at all. Elor and Averbuch-Elor, To SMOTE, or not to SMOTE? In medical risk prediction, a study of logistic regression models found that oversampling, undersampling, and SMOTE all made the predicted probabilities badly overstate risk, without improving the model’s ability to tell cases apart. van den Goorbergh et al., JAMIA 2022
Three cheaper changes usually come first. Judge the model by a metric that reflects the rare class, such as precision and recall for that class, because a fraud model that never flags anything can be 99.9 percent accurate. Give the rare class more weight during training, which most libraries support with one setting. And choose the decision threshold based on what a missed case and a false alarm each cost, rather than leaving it at the default. If the model still misses the rare case after that, it is a rare-case problem, and the conditions in the previous section apply.
More real data is within reach
Generating synthetic data looks cheap, because the generator can produce thousands of examples in minutes. The cost is elsewhere. Someone with domain knowledge has to review samples. Statistical checks have to compare the generated data with the real data. If the data involves people, privacy tests have to run. And the model has to be trained and evaluated twice, with and without the generated data, to show the data helped.
Collecting real data is often more practical than it first appears. A few more weeks of logging, an older archive that was never processed, a second site with the same equipment, or a few days of labeling by people who already do the work can produce a few thousand real examples. Those examples need no realism check, carry no generator assumptions, and can also be used for testing. A useful question before starting a synthetic data project is how long it would take to get 500 more real examples of the case you care about. If the answer is weeks rather than years, start there.
Nobody understands the process well enough to generate examples
Synthetic data reflects what its builder knows. When the cause of a failure is unknown, when customer behavior in a new market has not been observed, or when a process has never been measured, there is nothing reliable to turn into examples. A generator built anyway encodes assumptions. The model learns them as facts, and if the model is also tested on generated data, the test confirms the assumptions instead of checking them.
Synthetic data does the most damage here, because the finished dataset looks complete and nobody can see the gap in knowledge behind it. The better first step is to learn: instrument the process, observe a few real cases, and talk to the people who handle them. Once the gap can be described, generating data for it becomes an option again.
Synthetic or real?
Pick what you want synthetic data for, then answer two or three questions about your situation.
Start
What do you want synthetic data for?
How do you know synthetic data helped?
The check is the same for every use except testing: train the model twice, once with the generated data and once without, and compare the two on real examples only. The real test examples must be ones the generator never saw, not even as inspiration. This is often called train-synthetic, test-real evaluation, and it is the most direct evidence that generated data transfers to the real task. Clearbox, utility evaluation
Look at the cases the synthetic data was meant to fix, not only at the overall score. Data generated for a rare failure should improve how often the model catches that failure. If the overall score rises slightly but the rare case is still missed, the data did not do its job. If neither improves, drop the data, even if generating it took effort.
Above all, keep generated data out of the test set. A model tested on data from the same generator it was trained on is being asked to recognize the generator’s habits, and it usually can. The scores look good and say nothing about real performance. This is how a synthetic data project can make a model look better in testing while making it worse in use.
Write down what is missing before you generate anything
Before building a generator, try writing the gap in one sentence. “We need more data” does not count. “We have twelve recorded seal failures on line three, and the model has to recognize the vibration pattern that comes before them, which our maintenance lead can describe” does.
If you can write that sentence, you also know what to generate, who should review it, and which real cases to test it on. If you cannot, more data will not help, because what is missing is an understanding of the case. Getting that usually starts with the real cases you already have.
