FairStress: Ai2’s 13M-item benchmark finds LLMs abandon correct answers 9.3 points more often for majority-coded options

What FairStress actually tests

Most fairness benchmarks check whether a model picks the "right" answer when a demographic detail is added. FairStress, released by Ai2 on 20 October 2026, asks a harder question: when a model has a correct answer (or a genuine tie), how easily can a persuasive argument push it off that answer, and does the push work equally well for every group?

The corpus is large but carefully structured. It starts from 300 base decision scenarios across 29 domains, crosses them with 31 demographic groups, 33 minority-majority contrasts, four degrees of identity signalling (none, name, implied, explicit label) and 78 adversarial prefixes in 15 families. The result is 13,032,104 items. Because few teams can run that, Ai2 also ships FairStress-Core, a 50,012-item subset chosen with item response theory; the paper reports it tracks the full corpus within 3.0 points on ground-truth accuracy and 1.3 points on tie lean.

The headline finding: the asymmetry only appears once identity is signalled

With no identity information, the paper reports no model shows an accuracy or robustness gap. Add identity and the picture changes. Across 15 models, the average gap in abandoning a correct answer was 6.9 percentage points with implied identity and 9.3 points with an explicit label, with models more willing to be talked out of correct answers that favoured the majority-coded candidate. On ties, the frontier models Ai2 tested, including Claude Opus 5 and GPT-5.5, picked the minority-coded candidate roughly 60 to 65 percent of the time where no candidate was objectively stronger.

That direction is the surprise. The older BBQ benchmark shows a lean toward stereotype-consistent answers; Ai2 reports that FairStress finds the opposite sign, and that moving BBQ-style tests to a chat template, yes/no wording and generated answers flipped Qwen2.5-7B, Gemma-3-27B and Llama-3.1-70B to minority-favouring too. The lesson for practitioners is that "unbiased on benchmark X" does not transfer across prompt formats.

Why it matters if you ship decision-support

If you use an LLM to screen applications, rank candidates, triage tickets or draft recommendations, the risk is not only a systematic lean. It is that the lean is *conditional on how the case is argued*. A model that holds its ground on a neutral prompt but folds under a persuasive framing is a different failure mode from one that is uniformly skewed, and it will not show up in a static test set.

Ai2 also looked at explanations: in 11 open-weight models, minority-coded choices were justified with confident, positive words such as "resilience" and "diversity", while majority-coded choices drew procedural words such as "evidence" and "criteria". That is a useful audit trick even without running the benchmark: read the reasons, not just the verdicts.

Fixes are not free

Ai2 tested nine post-hoc interventions. Most did not reliably reduce the asymmetry, and several cost accuracy: one activation-steering method cut Qwen2.5-7B-Instruct's accuracy on clear-cut decisions from 98% to 47%. Counterfactual chain-of-thought came closest, preserving accuracy across three model families, though ties still leaned minority. In a training-data audit of Olmo 3, the team found about 99,000 refusal-training examples in the SFT corpus, and of 28,000 topic clusters only nine concerned merit-based decisions, all of them refusals. The authors are explicit that this may not cause the behavioural gaps.

Caveats before you cite it

Ai2 built the benchmark and also evaluates its own Olmo family on it, so independent replication matters. The scenarios and prefixes are partly model-generated and author-reviewed. The benchmark uses an equality-centred definition of fairness and does not capture equity-based views. The tech report is hosted by Ai2, not peer-reviewed, and no arXiv version is linked. The dataset card's load snippet points at a personal namespace rather than allenai/fairstress, so check you are loading the right copy. The 9.3-point figure is described slightly differently in the blog and paper conclusion; treat the exact metric definition as something to confirm in the paper's tables.

Axion take

FairStress is the most practical fairness release of the month because it is licensed for use (dataset CC BY 4.0, code MIT), runs a manageable 50K-item core, and measures susceptibility to argument rather than a single score. Run FairStress-Core against any model you put in a decision loop, and treat a clean neutral-prompt result as the start of the audit, not the end.