Use Cases
Synthetic data covers four production-grade use cases in 2026. AI feature development on realistic distributions without PHI or PII risk. Load testing with realistic relational distributions (a customer should have a plausible number of orders, support tickets, and contacts; uniform random data tests nothing). Edge-case generation for rare intents — the agent that handles 1-in-10,000 dispute cases needs a thousand of those generated to be eval-able. Regulatory audits where real data cannot leave the prod environment but auditors need to see the system work. The HIPAA Safe Harbor and GDPR Article 4(5) pseudonymization provisions both reference synthetic data as a recognized de-identification technique when generation methods are auditable.
Tool Landscape
Tonic.ai, Gretel, MOSTLY AI, and YData lead the enterprise-grade synthetic data market. Tonic excels at relational data and ships connectors for Postgres, MySQL, Snowflake, Databricks, and MongoDB. Gretel emphasizes generative models for unstructured text and tabular data with differential-privacy guarantees. MOSTLY AI focuses on banking and insurance use cases. Salesforce Data Mask remains the on-platform option for sandboxes and produces deterministically masked data rather than fully synthetic — useful for UAT but limited for AI training. The Synthetic Data Vault (SDV) is the strongest open-source option for relational synthesis. Pick on use case: relational depth, generative quality, or platform-native simplicity.
Tool selection
Use case First choice
Relational tabular for warehouse load Tonic, SDV
Generative text for agent conversations Gretel, custom LLM
Salesforce sandbox for UAT Data Mask
Banking/insurance with strict DP MOSTLY AI
Open-source self-hosted SDVPrivacy Posture
Synthetic data is not automatically private. Poorly trained generators leak training examples — the 2023 NeurIPS paper “Are Synthetic Data Private?” demonstrated reconstruction attacks against GANs trained without differential privacy. Differential privacy techniques (epsilon-delta budgets, DP-SGD training) provide measurable guarantees but reduce fidelity; the privacy-utility tradeoff is real. Verify your provider’s privacy guarantees in writing; “we use synthetic data so it is GDPR-safe” is the answer of someone who has not read GDPR Article 4(5). For high-sensitivity use cases, run a membership inference attack against your synthetic dataset before relying on its privacy claims.
AI Testing Specifics
For agent eval, you need synthetic conversations, not just records. Realistic intent distributions (top 20 intents covering 80% of volume; long-tail intents with at least 30 examples each), realistic message patterns (typos, multi-turn clarifications, frustrated tone, off-topic excursions), and realistic edge cases (mixed language, ambiguous pronouns, contradictory user statements). Generate thousands, not hundreds — low-volume synthetic eval sets test rare behavior poorly and produce overconfident pass rates. The 2026 best-practice eval set sits at 2,000-10,000 synthetic conversations per agent, refreshed quarterly, version-controlled in git, and locked to a specific eval prompt set.
What Changed in 2026
Three shifts: regulators began accepting synthetic data as a recognized risk-reduction technique under EU AI Act Article 10 data-governance obligations; LLM-based conversation synthesis matured to the point that human evaluators can no longer reliably distinguish synthetic from real customer messages on routine queries (agent eval became one of the few legitimate uses where this is a feature, not a problem); and on-platform options improved (Salesforce expanded Data Mask in Spring ‘26).
Common Failure Modes
The recurring failures: synthetic data with unrealistic distributions (uniform random when reality is power-law) producing eval pass rates that do not predict production behavior; reusing the same synthetic eval set for too long so the agent overfits; failing to refresh synthetic data when the customer base shifts; and trusting “this is synthetic, so it cannot leak” without verifying the generator’s privacy properties.
Cost Considerations
Tonic and Gretel sit in the $50-300K range for enterprise; MOSTLY AI similar. SDV is free but requires data engineering capacity. Salesforce Data Mask is included in most editions. Budget for the regeneration cadence — quarterly refresh on tier-1 use cases, annually on others.
What to do this week
Pull your largest agent eval set. Check whether it contains real customer data or synthetic data; if real, the privacy review is overdue. If synthetic, ask whether anyone has measured its distributional fidelity against production traffic.
