How Is Synthetic Market Research Validated?

Synthetic market research is validated by replication: you ask the synthetic audience the questions a real survey asked comparable people, then compare the answer distributions. The gap, usually the mean absolute difference in percentage points, shows how close the simulation came on those questions. A useful validation also reports an uncertainty range, a simple baseline and which questions were left out, because a single accuracy number without that context cannot be checked.

Who this validation guide is for

This page is for insights leads, methodologists and analysts who need to decide how far to trust a synthetic audience before using it. It explains what a replication measures, how Minds runs one for each Audience, and what a validation score does and does not tell you. For the wider method, read what synthetic market research is.

How to think about the validation problem

A synthetic audience produces estimates of how a group of people would answer. To know whether those estimates are any good, you need answers from real people in the same group to the same questions. Three things decide whether the comparison is fair:

1. The right reference. The survey's respondents should match the audience you simulated. The general public's eating-out habits, for example, are no target for an audience of site workers.
2. The right questions. A question should only be compared if the audience can answer it and the published answers are a fair target for that audience.
3. The right yardstick. Report the error with an uncertainty range, and next to a simple baseline such as equal probability across answer options, so that an easy question does not look like a strong result.

Minds' Audience Validation follows this pattern. It ranks published surveys by how closely their respondents match the Audience, checks every question for fit before any Mind answers, and lists each question it leaves out with the reason. It asks at most 60 questions from up to 12 surveys in one run and needs at least 8 fitting questions to produce a score. You can also validate against survey files you upload, such as a questionnaire with its results as PDF, Excel, CSV, Word or text. Each survey gets a score out of 100, defined as 100 minus the mean absolute percentage-point difference between the Minds' answers and the published answer shares, with a 95% range. The details are in the Audiences guide.

Minds has also published replications against five public datasets: GSS 2024, ANES 2024, the 2024 Canadian Election Study, PISA 2022 and Food and You 2. Mean errors ranged from 4.72 to 8.33 percentage points, and on Food and You an equal-probability baseline reached 7.00 points against 6.01 for the Audience, with baselines that were not rerun at the same time (full results). The studies use different instruments and should not be pooled into one accuracy score.

Comparing the realistic research options

Teams that want evidence about an audience usually choose between three approaches.

The first is a recruited survey or panel. It gives real answers and, with good sampling and weighting, can approximate a population. It takes days to weeks and costs per complete, and quality depends on screening out inattentive and fraudulent respondents.

The second is an unvalidated AI tool: a chatbot asked to role-play customers. It is fast and nearly free, but there is no way to know how close its answers are to real ones, and published research shows that persona-prompted models can match averages while showing too little variation and shifting with prompt wording (Bisbee et al., Political Analysis, 2024).

The third is a synthetic audience that is checked against real data. It keeps the speed of the second option and adds evidence about how far to trust it for a given audience and type of question. It still produces estimates, not a representative sample.

When Minds is and isn't the right answer

Minds fits when you need a fast, directional read and want evidence about how far that read can be trusted: screening concepts and claims, pre-testing questionnaires, comparing segments, or exploring audiences that are hard to recruit.

It is not the right tool when you need representative estimates such as market size or exact willingness to pay, evidence for regulated or audited claims, physical or sensory product tests, or a decision that is expensive to reverse and has not been checked with real people. For those, use a recruited sample or observed behaviour, and use synthetic research to make that study smaller and better aimed.

Read how Minds research panels are built

Related commercial guides

• What Is Synthetic Market Research? Definition and Limits