How Synthetic Survey Panels and AI Respondents Work

A synthetic respondent is a research participant simulated by a language model. The model is conditioned on a profile of the kind of person you want to study (age, region, job, habits, attitudes, and ideally real survey or interview data about people like them) and then answers survey questions, reacts to a concept or takes part in an interview in that role.

A synthetic panel is a group of synthetic respondents built to represent one audience. Because every member can be asked the same questions, their answers can be aggregated into distributions and compared across segments, much like a survey of a recruited panel.

Both are tools for fast, directional research. They let a team screen ideas and questionnaires before fieldwork, but their answers are estimates produced by a model, not measurements of real people, and they need to be checked against real data before they carry a decision that is expensive to reverse.

Synthetic respondents compared with related terms

The vocabulary in this market overlaps. These are the distinctions that matter when you compare tools or read a methods section:

For the wider method that uses these respondents, read what synthetic market research is.

How synthetic respondents are built

A useful synthetic respondent has three layers.

1. A base language model. It supplies fluency and general knowledge, and also brings its own tendencies: agreeableness, verbosity and the opinion skews of its training data.
2. A profile and the evidence behind it. The profile sets demographic, behavioural and attitudinal attributes. Stronger systems ground each profile in sources: published statistics, public research, and the team's own survey results, interview transcripts or customer files. A respondent built only from a demographic label tends to answer like a stereotype.
3. A response protocol. The respondent answers within a defined instrument (single choice, multiselect, scales, rankings, forced-choice tasks such as MaxDiff, or open questions) so that answers can be compared and aggregated.

How closed answers are collected matters. In Minds' published replications, asking synthetic respondents for probabilities across answer options instead of one forced choice each reduced distribution error by 12.47 to 19.72 percentage points across four public datasets (Minds research).

What a synthetic panel is, and how it differs from a traditional panel

A synthetic panel is the group version: dozens to thousands of synthetic respondents that vary within one audience definition, for example "UK grocery shoppers aged 25 to 40 who buy plant-based milk weekly". The panel answers a study, and the results are read as distributions, rankings and themes.

The two are not substitutes. A synthetic panel is a way to decide what is worth asking a real panel, and to make that real study smaller and better aimed. For more on the comparison, see synthetic panels vs traditional panels.

What the accuracy evidence shows

No single accuracy number applies to synthetic respondents. Results depend on the audience, the question and how answers are collected, and the published evidence points both ways.

• Argyle and colleagues conditioned GPT-3 on socio-demographic backstories of real US survey participants and found it could reproduce the response distributions of many subgroups, which they called "algorithmic fidelity" (Political Analysis, 2023).
• A Stanford-led team built agents for 1,052 Americans from two-hour interviews, surveys or both. On held-out General Social Survey items they reached 83%, 82% and 86% of the participants' own two-week test-retest consistency, against 74% for agents given demographics only (arXiv, 2024). Grounding in real self-reports beat demographic labels.
• Bisbee and colleagues found that persona-prompted ChatGPT matched average feeling-thermometer scores from the American National Election Study, but with less variation than real respondents, different regression relationships, sensitivity to small wording changes and drift over time (Political Analysis, 2024).
• Santurkar and colleagues found substantial misalignment between the opinions language models express and those of 60 US demographic groups, with some groups, such as people over 65, reflected poorly even when the model was steered towards them (ICML 2023).

For a longer review of where synthetic and human answers agree and diverge, read synthetic vs real respondents: accuracy.

Failure modes to check for

Synthetic respondents fail in ways that differ from human panels:

• Too little variance. Answers cluster around the most typical view, so distributions look tidier than real ones and minorities of opinion disappear.
• Correlated error. Every respondent shares the same model, so a blind spot affects all of them at once. Conventional margins of error assume independent respondents and do not apply.
• Prompt and model dependence. Small changes in wording, order or model version can move results.
• Agreeableness. Respondents may like everything they are shown unless the instrument forces trade-offs.
• Thin-data audiences. Accuracy drops for low-incidence groups, specialist roles and new categories that are poorly described in available data.

How Minds builds and checks synthetic panels

In Minds, each synthetic respondent is a Mind and a synthetic panel is an Audience. Minds are grounded in sources and in files you add, such as survey results, interview transcripts or customer research, and a Study asks an Audience open and closed questions, MaxDiff and other methods, or shows it stimuli such as concepts, images, websites and copy.

Audience Validation checks an Audience against real surveys rather than relying on a general accuracy claim. Minds finds published surveys whose respondents match the Audience, or uses survey files you upload, keeps only questions whose published answers are a fair target for that Audience, and lists every question it left out with the reason. Each survey gets a score out of 100, defined as 100 minus the mean absolute percentage-point difference from the published answer shares, with a 95% range. Read how Audience Validation works for the details.

Minds' published replications against five public datasets found mean distribution errors from 4.72 to 8.33 percentage points (full results). Those studies use different instruments and should not be pooled into one accuracy score, and a good result on one survey does not transfer automatically to a different audience or question.

When to use synthetic respondents, and when not to

Good uses:

• Screening many concepts, claims or messages down to a shortlist before fieldwork.
• Pre-testing a questionnaire for confusing wording, missing answer options and leading questions.
• A first read on audiences that are hard or expensive to recruit, before targeted interviews.
• Forming hypotheses and choosing which segments a human study should sample.

Use recruited people or observed behaviour instead when:

• You need representative estimates such as market size, share or the percentage who would buy.
• You need willingness to pay or price elasticity at specific price points.
• The claim will be published, regulated or audited, including advertising substantiation and health claims.
• The product must be touched, tasted or used physically.
• The audience or category is new, so little reliable data describes it.

How to evaluate a synthetic respondent provider

Ask for evidence you can check:

1. Which public surveys have you replicated, what was the error, and what baseline did you compare against?
2. Can I validate my own panel against real survey data, and see which questions were excluded and why?
3. How is each respondent grounded: a demographic label, or sources and my own research files?
4. How are closed answers collected, and can respondents disagree or reject a concept?
5. What do you say the tool should not be used for?

A comparison of providers is in the synthetic respondent platforms guide.

Sources

• Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C. and Wingate, D. (2023). Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis, 31(3), 337-351.
• Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B. and Larson, J. M. (2024). Synthetic Replacements for Human Survey Data? The Perils of Large Language Models. Political Analysis, 32(4), 401-416.
• Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P. and Hashimoto, T. (2023). Whose Opinions Do Language Models Reflect? Proceedings of the 40th International Conference on Machine Learning.
• Park, J. S. and colleagues (2024). LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals. arXiv 2411.10109.
• Minds (2026). We Tested Synthetic Audiences Against Reality.