Andreas Haupt

AI Institute Fellow-in-Residence, Schmidt Sciences

Digital Fellow, Stanford Digital Economy Lab

Technology and Human Rights Fellow, Harvard Kennedy School

Andreas Haupt is an AI Institute Fellow-in-Residence at Schmidt Sciences, a Digital Fellow at the Stanford Digital Economy Lab, and a Technology and Human Rights Fellow at the Harvard Kennedy School. He studies how algorithmic systems represent and are evaluated against diverse human preferences, using methods from microeconomic theory, structural econometrics, and reinforcement learning. His work has been published in AI venues such as NeurIPS and ICML, in computer science venues such as ACM EC, ACM RecSys, and ACM FAccT, and as a lead article in the American Economic Review. He has served as an area chair at ICLR, ICML, and AAAI. He earned a Ph.D. in Engineering-Economic Systems from MIT in February 2025 with a committee evenly split between Economics and Computer Science. Prior to that, he completed two master’s degrees at the University of Bonn—first in Mathematics (2017) and then in Economics (2018), with distinction. He has worked on competition enforcement for the European Commission’s Directorate-General for Competition and the U.S. Federal Trade Commission, and taught high school mathematics and computer science in Germany before his Ph.D. He remains committed to education and scholarship, most recently as a co-author of a textbook on Machine Learning from Human Preferences, forthcoming with Princeton University Press.

280-character bio Andreas Haupt is an AI Institute Fellow-in-Residence at Schmidt Sciences and a fellow at Stanford's Digital Economy Lab and Harvard Kennedy School. He studies how AI systems represent and are evaluated against diverse human preferences, publishing in NeurIPS, ACM EC, and the AER.
Tagline Use AI to Learn about People's Preferences
Media Assets BANNER FORMAL HEADSHOT INFORMAL HEADSHOT PANORAMA WHITEBOARD HEADSHOT

Publications

A full publication list is available as pdf and BibTeX, and a more complete list on Google Scholar. ‡ indicates equal contribution or alphabetic author listing, † equal supervision. Only published work is shown by default; select another type, or All types, to see the rest.

Welfare, Improvability, and Variance: A Principal-Agent Approach to Optimal Benchmark Item Aggregation

A. Haupt‡, J. Hartenstein‡, A. Reuel, M. Kochenderfer, S. Koyejo

Advances in Neural Information Processing Systems (Evaluations and Datasets Track), 2026.

Published AI Evaluation Aggregation Market Design PDF HTML
In brief
The optimal item weights depend on welfare, improvability and variance; improvability is estimated from the cost of a 10% improvement by pretraining, SFT or RL, shown for one item per quantile. v* = (M + rΣ)⁻¹ M w optimal weight of each benchmark item w welfare: how much the item matters to workers Σ variance: how noisy the item is; r: the lab's risk aversion M improvability: how cheaply effort moves the item. Estimated from what a 10% gain costs on the EvoLM 4B suite (Table 1): improvability benchmark topic cheapest axis cost (10% gain) 10% quantile HellaSwag performing tricks with trained dogs SFT $385 50% quantile MathQA finding a number from a ratio and LCM RL $4.2k 90% quantile MMLU (accounting) modifying report for financial stmts. PT $37k Cheapest axis across all items: supervised fine-tuning (SFT) for 50%, reinforcement learning (RL) for 35%, pretraining (PT) for 14%. Cheaper to move ⇒ larger M ⇒ larger optimal weight; uniform weights steer labs toward SFT-tractable items.
The optimal weight of an item rises with its welfare relevance and improvability and falls with its variance. Improvability is estimated from what a 10% improvement costs on the EvoLM 4B suite; the table shows one OLMES item per improvability quantile from the paper's Table 1.

A benchmark is reported as a measurement but works as an incentive: once labs optimize against it, the weight on each item steers effort. We model benchmark aggregation as a multitask principal–agent problem. The optimal item weights have a closed form, v* = (M + rΣ)−1Mw—higher for items that matter for welfare (w) and that effort can improve (M), lower for noisy items (Σ) when the lab is risk-averse (r). On the OLMES suite, with welfare weights from workers' own task ratings, all 52 mathematics items are Pareto-dominated by general-knowledge items under a pro-worker welfare definition, and only ~6% of the suite's "automation loading" falls on activities workers want automated. Evaluation designers should publish why each item carries the weight it does.

Reward Bias Substitution: Single-Axis Bias Mitigations Redirect Optimization Pressure

M. Lamparth, D. Fein, A. Haupt, M. Hussing, M.J. Kochenderfer

Advances in Neural Information Processing Systems, 2026.

Published Preferences PDF
In brief
Penalizing length lowers the length bias but raises overconfidence: optimization pressure is redirected rather than removed. before mitigation length bias over- confidence after a length penalty length bias over- confidence pressure moves A single-axis fix redirects optimization pressure. On the usual audit distribution, success, substitution and overcorrection look the same. Evaluate under the policy-induced distribution; track several biases.
A length penalty lowers the length bias, but the optimization pressure resurfaces as overconfidence (schematic).

Fixing one bias in a reward model (length, sycophancy, style) can simply redirect optimization pressure elsewhere. We prove that success, substitution, and overcorrection are indistinguishable under the audit distributions on which mitigations are usually reported—even with an oracle true reward. Empirically, a length penalty under GRPO shifts pressure into overconfidence with lower factual accuracy, and a published length-debiasing operator's bias reappears under best-of-n sampling for three of four state-of-the-art reward models. Prescription: evaluate under the policy-induced distribution and track several biases at once.

Position: AI Development Should Prioritize Cognitive Security

B. El, S. Su, A. Pappu, P. Yin, J. Heng, E. Heng, R.Z. Wang, A. Haupt†, J. Zou†

Advances in Neural Information Processing Systems (Position Paper), 2026.

Published Preferences PDF

Contextually Private Mechanisms

A. Haupt‡, Z. Hitzig‡

American Economic Review 116 (9), p. 3223-3262 (Lead Article), 2026.

Published Privacy Market Design PDF HTML
In brief
The corners lemma: when two agents are jointly but not individually pivotal, whichever agent is asked first is told apart from a same-outcome profile, so every protocol violates contextual privacy. x x x y agent j reports θⱼ θⱼ′ θᵢ θᵢ′ agent i a corner: outcome y only if both i and j report the primed type; neither is pivotal alone ask i first x x x y i answers θᵢ′, then j answers θⱼ outcome x told apart, same outcome: a contextual privacy violation ask j first x x x y j answers θⱼ′, then i answers θᵢ outcome x told apart, same outcome: a contextual privacy violation Corners lemma: someone has to be asked first, so every protocol implementing a rule with a corner violates contextual privacy. Corners are exactly where agents are collectively, not individually, pivotal.
The corners lemma: when two agents are jointly but not individually pivotal, whoever is asked first is told apart from a profile with the same outcome, so no protocol implementing the rule is contextually private.

Mechanism design usually asks what to implement and models the mechanism as a one-shot report of types. This paper studies the extensive form instead: a protocol asks participants questions one at a time, and what it learns depends on which question it asks, of whom, and when. A protocol violates contextual privacy when it tells two people apart who would have received the same outcome anyway. Violations are unavoidable exactly when agents are collectively, but not individually, pivotal. Beyond that, the order of questions is a design choice: delaying a question until its answer is needed for the outcome means fewer people are told apart for no reason, so delay improves privacy, and different extensive forms of the same allocation rule can differ sharply in how much they learn. The paper connects auction theory to Helen Nissenbaum's contextual integrity: "information not needed for the outcome" becomes a theorem about protocols.

Machine Learning from Human Preferences

S. Truong‡, A. Haupt‡, S. Koyejo‡

Princeton University Press (forthcoming)

Published Preferences HTML

Don't Walk the Line: Boundary Guidance for Filtered Generation

S. Ball, A. Haupt

International Conference on Machine Learning, 2026.

Published Multi-Agent PDF

Bruno: An AI Product Manager for Scientists

A. Haupt‡, P. Ranganathan‡

ICML Workshop on AI for Science (Best AI Scientist Award), 2026.

Published Work HTML VIDEO

The Collapse of Heterogeneity in Silicon Philosophers

Y. Shi, A. Haupt

ACM Conference on Fairness, Accountability, and Transparency, 2026.

Published AI Evaluation Preferences PDF CODE
In brief
Philosophers' positions across two questions are dispersed; language models' positions line up, manufacturing consensus. 277 philosophers: disagreement position on question 1 question 2 7 models: manufactured consensus position on question 1 question 2 Models reproduce individual positions tolerably, but over- correlate judgments across questions: heterogeneity collapses.
Philosophers' positions across questions are dispersed; the models' positions line up, manufacturing consensus where the profession disagrees (schematic).

Can a language model stand in for an expert panel? We tested seven models against 277 professional philosophers and the PhilPapers survey (N = 1,785). Models reproduce individual positions tolerably, but they over-correlate judgments across questions, manufacturing consensus where the profession disagrees—partly a "specialist effect": the model assumes specialists in a domain share views. For anyone using "silicon samples" as proxies for publics, the heterogeneity collapses exactly where it matters.

Discussion of "The ICML 2023 Ranking Experiment" by Su et al.

A. Haupt, S. Koyejo

Journal of the American Statistical Association 121 (554), p. 853-854, 2026.

Published AI Evaluation Aggregation

Position: ML Conferences Should Establish a Refutations and Critiques Track

R. Schaeffer, J. Kazdan, Y. Denisov-Blanch, B. Miranda, M. Gerstgrasser, S. Zhang, A. Haupt, I. Gupta, E. Obbad, J. Dodge, et al.

Advances in Neural Information Processing Systems (Position Paper), 2025.

Published AI Evaluation PDF

Scaling Human Judgment in Community Notes with LLMs

H. Li, S. De, M. Revel, A. Haupt, B. Miller, K. Coleman, J. Baxter, M. Saveski, M.A. Bakker

Journal of Online Trust and Safety 3 (1), 2025.

Published AI Evaluation Aggregation Platforms PDF

AI should not be an imitation game: Centaur evaluations

A. Haupt, E. Brynjolfsson

International Conference on Machine Learning (Position Paper), 2025.

Published AI Evaluation Work PDF
In brief
A centaur evaluation scores the uplift of a human-AI team over the human alone and the AI alone. human alone AI alone human + AI team uplift what the evaluation scores Standalone benchmark scores the AI alone, so it rewards replacing people. Centaur evaluation scores what the team achieves over either alone, so interpretability, helpfulness and complementarity count.
A centaur evaluation scores the uplift of a human–AI team over either alone, instead of the AI by itself.

Standalone benchmarks reward AI for replacing people. Centaur evaluations instead measure what a human–AI team achieves—the uplift over either alone—and thereby make human-centred properties (interpretability, helpfulness, complementarity) first-class objectives rather than afterthoughts. The paper argues that such evaluations are both more demanding and more realistic than current leaderboards, and sets out a research program: who the humans are, how to randomize access, how to score teams. It is the conceptual basis for AlignmentBench (interview-then-predict), SimulacraBench, and the AI-oral-assessment study.

Convex Markov Games: A New Frontier for Multi-Agent Reinforcement Learning

I. Gemp, A. Haupt, L. Marris, S. Liu, G. Piliouras

International Conference on Machine Learning, 2025.

Published Multi-Agent PDF
In brief
In a Markov game, preferences over occupancy measures are linear; in a convex Markov game they can be any convex function, which allows objectives such as diversity, imitation, fairness and safety. Markov game linear reward: parallel level sets convex Markov game convex preference: curved level sets generalizes diversity, imitation, fairness, safety
Preferences over occupancy measures: linear in a Markov game, any convex function in a convex Markov game.

Multi-agent reinforcement learning assumes rewards add up over time. Many objectives do not: diversity, imitation, fairness, safety. Convex Markov games let players hold general convex preferences over occupancy measures, strictly generalizing Markov games (including infinite horizon). Pure-strategy Nash equilibria exist, and we approximate them by gradient descent on an exploitability bound. In the prisoner's dilemma, "transient imitation" recovers a profile close to human play with higher utility and roughly a thousand-fold lower exploitability; experiments also show fairer coordination and safer robot-warehouse behaviour.

Risk Preferences of Learning Algorithms

A. Haupt‡, A. Narayanan‡

Games and Economic Behavior 148, p. 415-426, 2024.

Published Multi-Agent Preferences PDF
In brief
Two actions with the same expected payoff: epsilon-greedy learning ends up choosing the lower-variance one. payoff same expected payoff low-variance action chosen with probability → 1 high-variance action ε-greedy bandits are emergently risk-averse: unlucky early draws make a risky action look bad, and it is then rarely tried again. Reweighting data by choice frequency, or optimism for rarely-chosen actions, restores risk neutrality.
Two actions with the same expected payoff: ε-greedy learning ends up choosing the low-variance one.

Standard bandit algorithms are not risk-neutral. ε-greedy learning is emergently risk-averse: between two actions with equal expected payoff it picks the lower-variance one with probability approaching one, and transiently even when the riskier action is strictly better on average. In a recommender or marketplace this favours measurable, homogeneous options and produces inequity and homogenization. Two corrections—reweighting data by how often an action was chosen, or optimistic estimates for rarely-chosen actions—restore risk neutrality.

Formal Contracts Mitigate Social Dilemmas in Multi-Agent Reinforcement Learning

A. Haupt‡, P. Christoffersen‡, M. Damani, D. Hadfield-Menell

Autonomous Agents and Multi-Agent Systems 38, Article 51, 2024.

Published Multi-Agent Market Design PDF CODE
In brief
A contracting stage before a Markov game: agents propose, accept and then play with reward transfers. 1. Propose a contract “whoever pollutes pays t to the others” 2. Accept or decline binding only if every agent agrees 3. Play the Markov game transfers apply to the rewards Rich enough contract space ⇒ every subgame-perfect equilibrium is socially optimal (fully observable games); richer spaces weakly raise welfare in general. MOCA learns the stage; tested in traffic, pollution and common-pool-resource environments. social dilemma → cooperation
A contracting stage before the game: agents propose and accept reward transfers, then play.

Add a contracting stage to a Markov game—agents can voluntarily bind themselves to reward transfers—and, with a rich enough contract space, every subgame-perfect equilibrium of a fully observable Markov game is socially optimal; richer spaces weakly raise welfare in general, including under partial observability. The MOCA algorithm makes this learnable, validated in traffic, pollution, and common-pool-resource environments.

Steering No-Regret Learners to a Desired Equilibrium

B.H. Zhang, G. Farina, I. Anagnostides, F. Cacciamani, S.M. McAleer, A. Haupt, A. Celli, N. Gatti, V. Conitzer, T. Sandholm

ACM Conference on Economics and Computation, 2024.

Published Multi-Agent Market Design PDF

Certification Design for a Competitive Market

A. Haupt‡, N. Immorlica‡, B. Lucier‡

ACM Conference on Economics and Computation, 2024.

Published Market Design PDF
In brief
A certifier partitions unobservable seller quality into priced certificate tiers; sellers sort into them. seller quality (unobservable to buyers) no certificate certificate 1 price p₁ certificate 2 price p₂ certificate 3 price p₃ The certifier chooses the tiers and their prices; sellers choose quality and a certificate; prices form competitively. Under single crossing, each seller's choice is pinned down. • sellers (choose quality and certificate) Optimal menus can be arbitrarily large.
The certifier partitions unobservable quality into priced certificate tiers; sellers choose both their quality and their certificate (schematic).

Certification is the market's answer to unobservable quality, from carbon credits to educational tests to AI models. We study a certifier that designs and prices certificates while sellers choose both quality and certificate and prices form competitively. Under single crossing, sellers' certification choices are uniquely pinned down; the revenue-maximizing certifier's problem reduces to monopoly pricing with non-linear valuations, for which we give an FPTAS; and both welfare- and revenue-optimal certificate menus can be arbitrarily large.

Black-Box Access is Insufficient for Rigorous AI Audits

S. Casper, C. Ezell, C. Siegmann, N. Kolt, T.L. Curtis, B. Bucknall, A. Haupt, K. Wei, J. Scheurer, M. Hobbhahn, et al.

ACM Conference on Fairness, Accountability, and Transparency, 2024.

Published AI Evaluation PDF

Computing Optimal Equilibria and Mechanisms via Learning in Zero-Sum Extensive-Form Games

B. Zhang, G. Farina, I. Anagnostides, F. Cacciamani, S. McAleer, A. Haupt, A. Celli, N. Gatti, V. Conitzer, T. Sandholm

Advances in Neural Information Processing Systems, 2023.

Published Multi-Agent Market Design PDF

Understanding Multi-Homing and Switching by Platform Drivers

X. Guo‡, A. Haupt‡, H. Wang, R. Qadri, J. Zhao

Transportation Research Part C: Emerging Technologies 154, 2023.

Published Platforms Work PDF

Towards Psychologically-Grounded Dynamic Preference Models

M. Curmei‡, A. Haupt‡, B. Recht, D. Hadfield-Menell

ACM Conference on Recommender Systems, 2022.

Published Preferences PDF

The Optimality of Upgrade Pricing

D. Bergemann‡, A. Bonatti‡, A. Haupt‡, A. Smolin‡

Web and Internet Economics, 2021.

Published Market Design PDF

Ongoing Work

TaskNet task bags from AI-guided interviews Work AI Evaluation

Economic Evaluations measurement science of economically relevant tasks Work AI Evaluation

AlignmentBench alignment as a centaur evaluation Work AI Evaluation Preferences

SituatedEvals competitions in predicting real-world microdata AI Evaluation Preferences

Design for Disobedience algorithmic monoculture as a benchmark AI Evaluation Preferences

VZBVal consumer protection evaluations in the German context Platforms Market Design AI Evaluation

The Role of Roles in contextual integrity Privacy Preferences

Vita

A Resume, a Computer Science-style CV and an Economics-style CV are available as pdf; their LaTeX source uses this site’s style.