Synthetic Data: When, Why, and How

Healthcare is where synthetic data has its clearest value proposition: real patient data is the most information-rich, privacy-sensitive, and access-restricted data in any industry. At HCLTech, we've used synthetic data to train models when we couldn't get HIPAA-compliant access to real patient data, to augment datasets for rare disease scenarios, and to test pipelines before real data was available. Here's what I've learned about when it works and when it doesn't.

Why Synthetic Data Exists

Synthetic data addresses four distinct problems:

  1. Privacy and compliance. You can't share real patient records, financial transactions, or PII-containing customer data with your AI vendor, training pipeline, or annotation team. This is the primary driver in healthcare, finance, and insurance.
  2. Data scarcity. Rare diseases, edge cases, novel failure modes - you simply don't have enough examples. Synthetic augmentation can expand a dataset of 200 rare disease cases to 2,000.
  3. Class imbalance. Your fraud detection model has 1,000 fraud cases and 1,000,000 legitimate transactions. Synthetic minority class generation (SMOTE and its variants) can rebalance the training set.
  4. Cost and speed. Synthetic data with known labels is far cheaper than expert annotation (radiology reads, legal document tagging).

Types of Synthetic Data

Tabular synthetic data: CTGAN, TVAE, and Gretel.ai generate synthetic rows that preserve statistical distributions and correlations without exposing real individuals.

from ctgan import CTGAN
import pandas as pd

real_df = pd.read_csv("patient_records.csv")
categorical_cols = ["diagnosis_code", "gender", "insurance_type"]

model = CTGAN(epochs=300, verbose=True)
model.fit(real_df, discrete_columns=categorical_cols)

synthetic_df = model.sample(10000)  # Generate 10K synthetic patients
print("Real mean age:", real_df["age"].mean())
print("Synthetic mean age:", synthetic_df["age"].mean())

Text synthetic data: LLM-generated text for instruction tuning datasets, Q&A pairs, classification training data, or expanding small labeled datasets. GPT-4o generated training data is often good enough for downstream task training.

from openai import OpenAI
import json

client = OpenAI()

def generate_synthetic_examples(
    task_description: str,
    real_examples: list,
    n_to_generate: int = 50
) -> list:
    examples_str = json.dumps(real_examples[:5], indent=2)
    prompt = (
        f'Generate {n_to_generate} diverse training examples for: {task_description}\n\n'
        f'Here are real examples for reference:\n{examples_str}\n\n'
        'Generate varied examples covering different phrasings and edge cases.\n'
        'Return as a JSON array with the same structure as the examples above.'
    )
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content)["examples"]

Image synthetic data: Stable Diffusion fine-tuning for product photos or document scans. Specialized medical imaging GANs (ProstateGAN, CheXpert-GAN) for clinical images.

Time series synthetic data: TimeGAN and its successors generate synthetic sequences that preserve temporal dynamics. Useful for synthetic sensor data, financial time series, clinical vitals.

Healthcare Use Cases

EHR augmentation for rare conditions. A hospital might have 50 confirmed cases of a rare autoimmune condition. Synthetic augmentation using CTGAN or a conditional VAE can generate plausible patient records with similar clinical profiles. At HCLTech, this approach improved rare-disease classifier performance by augmenting training sets 10-20x.

De-identification for research datasets. Synthetic data generated to match the statistical properties of de-identified data - without direct link to real individuals - enables more useful research sharing. The HL7 FHIR Synthea project generates realistic but entirely synthetic patient records for exactly this purpose.

import subprocess

# Synthea generates realistic synthetic FHIR patient data
subprocess.run([
    "java", "-jar", "synthea-with-dependencies.jar",
    "-p", "1000",  # Generate 1000 patients
    "--exporter.fhir.export", "true",
    "--generate.diabetes.prevalence", "0.15"
])

Adversarial testing. Generate synthetic examples that probe model failure modes: rare demographic combinations, unusual lab value patterns, contradictory clinical notes. You're using synthetic data not to train, but to test.

Privacy Evaluation: The Metrics That Matter

"Synthetic data is private" is a dangerous oversimplification. Key privacy metrics:

  • Membership inference attack (MIA): Given a synthetic record, can an attacker determine whether a specific real individual was in the training set? High MIA success rate = privacy leakage.
  • Nearest neighbor distance (NND): How close are synthetic records to real records? Low NND = synthetic records are near-copies of real ones = privacy risk.
  • Attribute inference: Given partial attributes, can an attacker infer the remaining attributes of a real individual?

Gretel.ai and SDV (Synthetic Data Vault) both include privacy evaluation modules. Don't skip this - a synthetic dataset that fails MIA attacks is a HIPAA violation waiting to happen.

When Synthetic Data Fails

Distribution shift between synthetic and real test data. A model trained on synthetic data may perform well on synthetic test data but poorly on real data. Always test on real held-out data, never just on synthetic test data.

Mode collapse in rare cases. GANs are notorious for mode collapse - the generator learns to produce a limited variety of outputs. Rare events in training data may be systematically underrepresented in synthetic output.

Label noise amplification. LLM-generated training data inherits the generating model's biases and errors. Training on synthetic data produced by a biased model reinforces the bias.

Feature correlation artifacts. Synthetic data has correlations because the generator learned them statistically. If the causal structure changes, synthetic data won't update - it's a snapshot of learned statistical relationships.

Practical Synthetic Data Stack

  • Tabular: SDV for general use, Gretel.ai for enterprise with privacy guarantees, CTGAN for complex distributions.
  • Text: GPT-4o with well-engineered generation prompts. Validate with a classifier trained on real data.
  • Images: Stable Diffusion fine-tuned on domain images; specialized medical imaging GANs for clinical images.
  • Time series: TimeGAN or TimeVAE for complex temporal data; bootstrap or Gaussian noise for simpler cases.

The decision to use synthetic data should be driven by a clear problem - not a general belief that more data is always better. Synthetic data adds complexity and failure modes. Use it for specific problems, measure the impact carefully, and always validate on real held-out data before claiming success.

Related reading

Further Reading