AI can help research teams explore questions, test instruments, summarize material, and simulate how a defined audience might respond. The outputs can be useful, but fluent language is not evidence that a simulation represents real people.
The responsible question is not “Are synthetic respondents accurate?” It is: for this population, task, setup, and decision, how was performance tested—and where did it fail?
This guide is a source synthesis. It is not an original Minds benchmark and does not report proprietary customer results.
Rich grounding can improve individual simulation
Park and colleagues built agents for 1,052 people and tested whether the agents could reproduce those individuals' answers and experimental behavior. In the current paper revision, agents grounded in both interviews and surveys reached 86% of participants' own two-week test-retest consistency benchmark on held-out General Social Survey items; interview-only agents reached 83%. These are normalized, study-specific results—not accuracy guarantees for unrelated customer-research tasks.
Source: Park et al., “LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals”.
Demographic conditioning can reproduce some distributions
Argyle and colleagues created “silicon samples” by conditioning GPT-3 on sociodemographic backstories from US surveys. Their work showed that model outputs could reflect relationships between demographic characteristics and political attitudes in the evaluated settings.
That finding supports further testing. It does not establish universal validity across countries, minority populations, novel products, purchasing behavior, or later model versions.
Source: Argyle et al., “Out of One, Many”.
Similar averages can hide important failures
Follow-up research has warned against treating synthetic survey data as a drop-in replacement for human responses. Models can compress variance, miss subgroup differences, reproduce stereotypes, and generate distributions that look plausible while failing at individual or tail behavior.
Source: Bisbee et al., “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models”.
Research bodies require transparency and human judgment
The Market Research Society's Delphi report treats synthetic participants as a method requiring disclosure, validation, and ethical scrutiny. It warns that empirical similarity alone does not make synthetic and human respondents equivalent.
Source: MRS Delphi Report: Using Synthetic Participants for Market Research.
Synthetic participants can be useful for:
- Generating hypotheses and possible objections.
- Piloting discussion guides and questionnaires.
- Screening early concepts or message variants before fieldwork.
- Exploring how explicit segment assumptions change responses.
- Stress-testing the clarity of stimuli and answer options.
- Preparing follow-up questions for recruited interviews.
These are exploratory uses. They should narrow the search space, not manufacture certainty.
Use recruited participants, observed behavior, or another appropriate real-world source when:
- The decision is legal, medical, political, regulated, safety-critical, or otherwise high-stakes.
- You need population estimates, confidence intervals, incidence rates, or claims of representativeness.
- The question concerns actual purchase, renewal, churn, voting, adherence, or other behavior.
- The stimulus depends on taste, touch, physical context, accessibility, or lived experience.
- The population is poorly represented in the underlying data.
- The category or event is new enough that historical patterns may be a weak guide.
- Real customer quotations or participant provenance are required.
1. Define the decision
Write down what the study is allowed to influence. “Explore possible objections” requires a different evidentiary standard than “select the final launch claim.”
2. Predefine the comparison
Before running the synthetic study, specify:
- Target population and inclusion criteria.
- Questions and stimuli.
- Human or behavioral comparison source.
- Sample sizes.
- Primary metric and acceptable error.
- Subgroups that must be evaluated separately.
- Conditions that will invalidate the result.
3. Keep a held-out reference set
Do not tune prompts, personas, or scoring on the same human responses later presented as independent validation. Separate development evidence from evaluation evidence.
4. Report disagreement, not only the headline
Show where synthetic and human evidence align and where they diverge. Include item-level results, subgroup results, missing tails, variance, and failed cases where the data permits.
5. Match the metric to the task
Concept ranking, open-text themes, rating distributions, and observed conversion are different targets. A strong result on one cannot be transferred automatically to another.
6. Revalidate after material changes
Model versions, prompts, source material, audience definitions, languages, and product categories can change performance. Record the configuration and date, then rerun the comparison after material changes.
| Field | What to report |
|---|
| Intended population | Who the simulation is meant to approximate, and who it excludes |
| Source material | Descriptions, files, links, research notes, or other inputs used |
| Model configuration | Provider/model identifier where permitted, run date, and material settings |
| Prompting procedure | Persona construction, questions, order effects, and sampling procedure |
| Synthetic sample | Number of simulated respondents/runs and how they were generated |
| Human comparison | Recruitment/source, field dates, sample size, and provenance |
| Metrics | Predefined scoring rule and uncertainty or tolerance |
| Results | Agreement and disagreement at item and subgroup level where possible |
| Limitations | Poorly represented groups, novel contexts, sensory/behavioral gaps, and other known boundaries |
| Decision rule | What the study may influence and what still requires human validation |
Minds supports creating reusable AI personas from descriptions, profiles, links, files, or research notes; organizing them into target groups; and collecting parallel synthetic responses. Available inputs and creation modes depend on the workspace configuration.
The product does not remove the researcher's responsibility to document sources, review group construction, choose suitable questions, protect confidential material, and validate consequential findings.
Read How Minds Builds Synthetic Research Panels for the product methodology, or use the Target Audience Research Template Library to structure a study brief and decision memo.
Minds. “AI Customer Research in 2026: Evidence, Limits, and Validation.” July 31, 2026. https://getminds.ai/blog/synthetic-research-evidence-review
