PRISM Alignment: Testing Participant Fit in Held-Out Answers
A 299-person, 869-item comparison measures whether grounded Minds better reflect participants' values and reasons, with a blinded judge and clear task limits.
In a confirmatory comparison using the external PRISM Alignment Dataset, full Minds won 32.6% of blinded participant-fit evaluations, compared with 25.7% for demographic prompts and 20.5% for generic prompts. The strongest gains concerned values expression, specificity and lower genericness.
This PRISM dataset is an external benchmark. It is distinct from the proprietary Minds PRISM source-modeling engine.
Participants, targets and conditions
The study evaluated 299 UK participants and 869 participant-domain items. Three conditions produced 2,607 scored outputs: full Minds through the production path, demographic-only prompts, and generic prompts.
Selection was outcome blind and real participant answers were held out from generation. A single blinded LLM judge evaluated condition-anonymous, length-matched comparisons. This is automated participant-fit assessment, not independent human-rater validation.
What the judge preferred
Share of evaluated items
Reported results for this study. The table and discussion retain the study scope, baselines and uncertainty.
- Full Minds winner32.5662%
- Demographic prompt winner25.6617%
- Generic prompt winner20.4833%
- Tie21.2888%
| Outcome | Share of evaluated items |
|---|---|
| Full Minds winner | 32.5662% |
| Demographic prompt winner | 25.6617% |
| Generic prompt winner | 20.4833% |
| Tie | 21.2888% |
The registered overall comparison reported a 12.04-percentage-point Minds advantage over generic prompting and 6.69 points over demographic prompting. These are the registered comparison estimates; they should not be replaced with subtraction of separately summarized displayed rates.
Winner rate is not survey approximation. It means the judge preferred that candidate under the participant-fit rubric, not that the same proportion of people's answers was reproduced exactly. A substantial tie share also remains part of the result.
The task boundary
Values-guided and controversy-guided tasks supplied the clearest advantage. These questions can depend on a participant's reasons and previously expressed viewpoint, making richer grounding directly relevant.
The advantage did not extend to every task. On short everyday prompts, Minds won 13.8%, compared with 30.5% for generic prompts and 34.0% for demographic prompts. Unguided prompts were also weaker. Overall semantic similarity did not significantly improve, and length-matched lexical similarity was lower than both baselines.
These findings make the overall result partial, not a universal response-fidelity win. They also distinguish participant fit from copying wording. A response can reflect a person's reasons without closely matching their phrasing, and a fluent everyday answer need not benefit from more profile detail.
What this supports
The supported claim is narrower and useful: grounded Minds improved judged participant fit in the evaluated cohort overall, particularly for values and specificity. Independent human judges and additional domains remain important for stronger fidelity claims.
The held-out interview confirmation examines earlier evidence and later answers in two other corpora. The foundation-model comparison asks how that grounded workflow compares with generic model responses. The evidence overview separates these qualitative metrics from survey approximation.
