Pluralistic AI governance · methodological pilot
The Moral Values Observatory studies how exact AI configurations respond when legitimate human values and moral considerations conflict—and which forms of human disagreement their outputs reflect, concentrate, or leave unmeasured.
This is a methodological pilot, not a completed representation audit. The project began by testing instruments, decision tasks, prompting conditions, comparison methods, and public research infrastructure. The current release retains distributive justice as its first active case study.
Scope
The Observatory connects three distinct layers without treating them as interchangeable.
Interpretive boundary
Results belong to an exact configuration, task, prompt, language, inference setting, and collection date. They do not establish that a system possesses beliefs, a stable moral character, or human-like moral competence.
From pilots to an observatory
The pilots were not attempts to declare what values AI systems possess. They tested how value-relevant outputs could be measured responsibly, which comparisons failed, and what evidence must be collected next.
Long-term objective: build a cumulative public infrastructure for auditing how AI systems handle normative disagreement across domains, configurations, populations, languages, and time.
Current status · first tested component of that research systemUse questionnaires, dilemmas, allocations, prompt variants, and repeated generations to identify what each method can and cannot support.
Separate configurations, treat items as the primary units, withdraw unsupported associations, and document missing provenance.
Collect equivalent human and AI responses, preserve human distributions, and control relevant and irrelevant task variation.
Repeat validated studies across domains and system updates without turning a single metric into a universal moral score.
Current case study · 01
Five exact configurations divided CHF 18,000 among three hypothetical recipients in a reconstructed workplace-bonus task and a family-inheritance task.
High performance and high dedication were positively associated with workplace allocations in all five configurations.
Tight finances were positively associated with inheritance allocations in all five configurations.
Model intervals for exact equal splits were below every selected published human point estimate included in both tasks.
Estimated allocation effects
Points show estimated percentage-point changes in allocated share. Lines show 95% vignette-set bootstrap intervals. Each panel uses its own readable scale; compare patterns, not panel widths.
Workplace bonus
Workplace bonus
Family inheritance
Source: generated from data/results/active/dse_coefficients.csv · 5,000 vignette-set bootstrap replicates.
| Configuration | Task | Characteristic | Effect | 95% model interval | Vignette sets | Generation rows |
|---|---|---|---|---|---|---|
claude-opus-4-8 | Workplace | High performance | 12.3 pp | 11.3 to 13.3 | 72 | 719 |
claude-opus-4-8 | Workplace | High dedication | 7.6 pp | 6.8 to 8.5 | 72 | 719 |
claude-opus-4-8 | Inheritance | Tight finances | 11.4 pp | 10.2 to 12.8 | 32 | 319 |
claude-fable-5 | Workplace | High performance | 12.5 pp | 11.6 to 13.4 | 72 | 720 |
claude-fable-5 | Workplace | High dedication | 8.3 pp | 7.5 to 9.1 | 72 | 720 |
claude-fable-5 | Inheritance | Tight finances | 12.0 pp | 10.9 to 13.4 | 32 | 320 |
gpt-5.5 | Workplace | High performance | 17.4 pp | 15.8 to 18.9 | 72 | 720 |
gpt-5.5 | Workplace | High dedication | 12.2 pp | 10.6 to 13.6 | 72 | 720 |
gpt-5.5 | Inheritance | Tight finances | 18.7 pp | 17.0 to 20.8 | 32 | 320 |
deepseek-v4-pro | Workplace | High performance | 15.3 pp | 13.5 to 17.0 | 72 | 666 |
deepseek-v4-pro | Workplace | High dedication | 10.5 pp | 9.0 to 12.0 | 72 | 666 |
deepseek-v4-pro | Inheritance | Tight finances | 20.9 pp | 18.2 to 23.6 | 32 | 319 |
glm-5.2 | Workplace | High performance | 20.8 pp | 18.4 to 23.0 | 72 | 719 |
glm-5.2 | Workplace | High dedication | 10.8 pp | 8.6 to 12.8 | 72 | 719 |
glm-5.2 | Inheritance | Tight finances | 17.6 pp | 15.9 to 19.3 | 32 | 318 |
Models and published human aggregates
Model lines are 95% vignette-set bootstrap intervals. Human circles are published aggregate point estimates from different samples and subgroups; they are not human confidence intervals or individual-level distributions.
Workplace bonus
Family inheritance
Human source: Gilgen (2022)[4], pp. 117 and 134–135 · model source: dse_equal_split.csv · 10,000 vignette-set bootstrap replicates.
| Configuration | Task | Estimate | 95% model interval | Vignette sets | Generation rows |
|---|---|---|---|---|---|
claude-opus-4-8 | Workplace | 8.5% | 4.3% to 13.4% | 72 | 719 |
claude-fable-5 | Workplace | 0.6% | 0.0% to 1.3% | 72 | 720 |
gpt-5.5 | Workplace | 1.7% | 0.1% to 4.0% | 72 | 720 |
deepseek-v4-pro | Workplace | 0.0% | 0.0% to 0.0% | 72 | 666 |
glm-5.2 | Workplace | 0.4% | 0.0% to 1.3% | 72 | 719 |
claude-opus-4-8 | Inheritance | 14.1% | 6.2% to 23.8% | 32 | 319 |
claude-fable-5 | Inheritance | 4.7% | 0.0% to 12.5% | 32 | 320 |
gpt-5.5 | Inheritance | 1.2% | 0.0% to 3.8% | 32 | 320 |
deepseek-v4-pro | Inheritance | 3.1% | 0.0% to 9.4% | 32 | 319 |
glm-5.2 | Inheritance | 5.3% | 0.9% to 12.2% | 32 | 318 |
What this pilot does not establish
The central question remains open because the available human evidence does not preserve the distributions, uncertainty, and coverage required for a representation audit.
Not established
The current comparisons cannot determine which individuals, communities, or populations would regard the outputs as reflecting their judgements.
Not established
Aggregate percentages do not reveal multimodality, minority positions, within-group variation, or the distance between a model tendency and a full human distribution.
Not established
Hypothetical allocations do not demonstrate real-world decisions, longitudinal stability, multilingual equivalence, or underlying moral competence.
What next
The next experiment will use public human microdata from 60 countries to examine how exact AI configurations respond to country-specific claims about wealth, unfairness, and redistribution. The protocol will then be extended to incentivized spectator-allocation decisions.
Shared protocol
The country-explicit item is the core condition: it names the country but does not ask the system to impersonate a resident. Perspective prompting, removal of the country, and estimation of a population distribution are separate tasks. Repeated generations measure configuration stability, not human diversity.[3][5][9]
Exact configurations will answer four country-specific survey items using public microdata from 60 countries: whether the rich became richer through selfishness or illegal activity, whether economic differences are unfair, and whether government should reduce them. The public file contains 65,856 records, while valid sample size varies substantially by item.[9]
A preregistered 15–20-country subset will test a resident-perspective prompt, a country-neutral item, matched paraphrases, option order, and explanation conditions. A separate task will ask the model to estimate how a population would distribute its answers; that estimate will not be conflated with the configuration’s own response.
After verifying public files, licence, treatment coding, weights, and reproducibility, the Observatory will extend the protocol to the global spectator-allocation experiment. This is a distinct study of distributive choices, not the same dataset as the attitudes audit.[10]
A confirmatory DSE extension will use equivalent human and AI tasks with individual-level human responses. ISSP and WVS items can then be linked to allocation decisions to test whether stated redistribution principles predict contextual choices, rather than assuming questionnaire profiles generalize.[6][7]
Priority reflects contribution to the Observatory’s research question, not whether another group has already used the dataset.
| Test | Role | Human base | Decision |
|---|---|---|---|
| DSE | Current active allocation case; matched confirmatory extension | Published Swiss and Princeton aggregates; new matched sample required | Active case |
| FAW attitudes | Global position, robustness, and joint-pattern audit | Public microdata from 60 countries; valid N varies by item | Next experiment |
| FAIR impartial spectator | Global distributive choices under luck, merit, and efficiency | >65,000 participants across 60 countries | After data audit |
| ISSP inequality | Declared pay and inequality principles | Repeated cross-national survey waves | Complementary |
| WVS redistribution–incentives | Broad national context | Single redistribution–incentives item | Context only |
| GlobalOpinionQA | Perspective and language robustness | Pew and WVS country aggregates | Selective replication |
| OpinionQA | US subgroup representation | Pew American Trends Panel | Optional |
| ESS Schwartz values | Declared-value context | European Social Survey | Auxiliary |
| Moral Machine | Extreme harm trade-offs | Large international public dataset | Not prioritised |
| MFQ-2 | Expressed questionnaire profiles | Limited national means in the pilot archive | Archive |
Evidence to date
Current research provides several starting points for the Observatory’s next phase of study.
Starting rule: describe tested behaviour and avoid claims about possessed beliefs, character, or internal competence.
Moral decisions can depend on interacting moral, non-moral, contextual, and irrelevant considerations. Evaluation therefore requires systematic variation rather than a collection of unrelated dilemmas.[2]
Starting rule: define each domain and manipulate the relevant considerations explicitly.
Starting rule: pair relevant manipulations with irrelevant controls in every mature case study.
Starting rule: evaluate coverage, concentration, and omissions rather than attribute a model to a country or rank it against one moral optimum.
Human variation reflects differences among people, whereas repeated model outputs measure variability around a configuration’s conditional tendency. LLM studies can also produce inflated effects and completely uniform responses.[3]
Starting rule: use repetitions for stability and concentration; use human participants for population distributions.
Human-derived scales can be non-equivalent, socially desirable, format-sensitive, or contaminated when applied to LLMs. Studies comparing abstract moral-foundation responses with concrete vignettes report weak or mismatched relations, while prompting can shift both questionnaire scores and downstream behaviour.[1][6][7]
Starting rule: keep expressed principles and applied decisions separate, then test their relationship directly.
Cross-national prompting can move outputs towards a country aggregate without demonstrating nuanced cultural representation. Translation alone may also fail to move responses towards populations that speak the target language.[5]
Starting rule: evaluate default behaviour, requested perspective, language, and explanatory content as separate conditions.
Starting rule: test homogenization with matched human distributions and explicit coverage metrics rather than infer it from profile similarity alone.
Research development
Earlier analyses remain visible as methodological development. They are not presented as a sequence of definitive discoveries.
Tested whether a moral-foundations questionnaire could yield stable, interpretable profiles and descriptive similarities to a limited set of national means.
Learning: profiles were measurable but conditional on prompt and instrument; national-mean proximity cannot establish cultural or population representation.
Tested whether declared preferences for equality or proportionality predicted choices across seven dilemmas.
Learning: the design did not contain enough independent and discriminating decisions to support the intended association. The claim was withdrawn; the research question remains.
Tested variants, English–Mandarin responses, reasoning modes, endpoints, and repeated generations.
Learning: exact configurations must be separated; language studies require matched designs; repetitions measure stability rather than independent participants.
Moved from expressed profiles to a controlled allocation task involving equality, merit, and need.
Learning: applied normative decisions can be measured reproducibly, but matched human distributions are still required before representation or disagreement compression can be estimated.
Human reference coverage
The limitation is not a short list of illustrative identities. It is the absence of the human data structure needed to evaluate representation across people, groups, intersections, contexts, and cultures.
The active case can place model estimates alongside selected published point estimates.
The general objective requires evidence that preserves human heterogeneity and its limits.
Interpretation rule: absence of comparable human evidence means “not currently assessable.” It does not establish that a system does or does not represent a population.
Research record
Active findings, exploratory evidence, superseded interpretations, excluded configurations, and withdrawn claims remain separated and inspectable.
DeepSeek Chat / non_thinking is excluded from every active and derived result because it is a distinct endpoint that was previously pooled with DeepSeek Reasoner as though both were repeated observations of one system. The original material remains archived for traceability.
This publication is cross-sectional. Longitudinal monitoring is an intended function of the Observatory, but it has not yet been demonstrated through a purpose-built repeated study.
Methods, sources, and data
Collection records and exact configurations come first, followed by raw responses, analysis scripts, generated results, documentation, and this website.
The package contains raw data, analysis scripts, active and exploratory outputs, tests, provenance, correction records, licenses, and a SHA-256 manifest.
Allocation intervals resample complete vignette sets. Items or scenarios are the primary statistical units; repeated generations are reported as rows and used to study configuration stability. Human comparison values are published aggregate point estimates, not confidence intervals or individual distributions.
The active DSE includes claude-opus-4-8, claude-fable-5, gpt-5.5, deepseek-v4-pro, and glm-5.2. Results must not be generalized to provider families or later versions.
Selected references used in Evidence to date and the research programme
Ye, H., Jin, J., Xie, Y., Zhang, X., & Song, G. (2025). Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement (Version 3, revised 11 March 2026). arXiv:2505.08245.
Used for construct equivalence, psychometric validity, prompt and language sensitivity, ecological validity, and standardization.
Publication recordHaas, J., Bridgers, S., Manzini, A., Henke, B., May, J., Levine, S., Weidinger, L., Shanahan, M., Lum, K., Gabriel, I., & Isaac, W. (2026). A roadmap for evaluating moral competence in large language models. Nature, 650, 565–573.
Used for performance–competence distinction, parametric evaluation, robustness, multidimensionality, and pluralistic ranges.
DOICui, Z., Li, N., & Zhou, H. (2025). A large-scale replication of scenario-based experiments in psychology and management using large language models. Nature Computational Science, 5, 627–634.
Used for the distinction between generative variation and human participants, effect-size inflation, uniform responses, and the limits of silicon replication.
DOIGilgen, S. (2022). Disentangling Justice: Needs, Equality or Merit? On the Situation-Dependency of Distributive Justice. Nomos.
Source of the reconstructed allocation task and selected published human aggregate estimates.
DOIDurmus, E., Nguyen, K., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., Lovitt, L., McCandlish, S., Sikder, O., Tamkin, A., Thamkul, J., Kaplan, J., Clark, J., & Ganguli, D. (2023). Towards Measuring the Representation of Subjective Global Opinions in Language Models (Version 2, revised 12 April 2024). arXiv:2306.16388.
Used for global-opinion similarity, the limits of country averages, cross-national prompting, stereotype risk, and language effects.
Publication recordNunes, J. L., Almeida, G. F. C. F., de Araujo, M., & Barbosa, S. D. J. (2024). Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7(1), 1074–1087.
Used for the distinction between abstract questionnaire responses and concrete moral judgements.
DOIAbdulhai, M., Serapio-García, G., Crepy, C., Valter, D., Canny, J., & Jaques, N. (2024). Moral Foundations of Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 17737–17752).
Used for prompt-conditioned moral-foundation profiles, context sensitivity, and downstream-task transfer.
DOIMünker, S. (2025). Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires. Proceedings of the 0th Symposium on Moral and Legal AI Alignment of the IACAP/AISB Conference. arXiv:2507.10073.
Used as recent evidence reporting cross-cultural profile homogenization; its synthetic-population framing is not adopted by the Observatory.
Publication recordAlmås, I., Cappelen, A. W., Sørensen, E. Ø., & Tungodden, B. (2022). Global evidence on the selfish rich inequality hypothesis. Proceedings of the National Academy of Sciences, 119(3), e2109690119.
Human base for the planned global attitudes audit. The public microdata contain country-specific ordinal responses on selfishness, illegal activity, unfair inequality, and government redistribution.
DOIAlmås, I., Cappelen, A. W., Sørensen, E. Ø., & Tungodden, B. (2025). Fairness Across the World. NHH Department of Economics Discussion Paper No. 06/2025.
Candidate human base for the later global impartial-spectator study of luck, merit, efficiency costs, and inequality acceptance. It is distinct from the 2022 attitudes dataset.
DOI