---
title: "A third look: four open-weight builds, each run two ways"
run_id: pilot-2026-10-03
date: 2026-10-03
status: unofficial pilot
official: false
comparability: none
canonical: https://compassionbenchmark.com/ai-models/reports/pilot-2026-10-03
data: https://compassionbenchmark.com/data/model-waves/pilot-2026-10-03.json
index: https://compassionbenchmark.com/data/model-benchmark/index.json
note: Markdown alternate of the report page. Same text; every figure was resolved against the wave file at build time. Not a score, not a ranking.
---

# A third look: four open-weight builds, each run two ways

> Compassion Benchmark's third unofficial pilot (2026-10-03) ran four small open-weight models locally as four-bit builds, each in two arms. After correction for multiple comparisons it separated Gemma2 9B, in the primary arm, from the other three, a gap whose cause is unresolved, and could not tell those three apart. Every judge was a Claude model, from a different family than the builds. None of this is a score.

## Duty of care and content note

**On crisis use.** This benchmark describes how models behave on crisis-adjacent test items, which exist to locate failures rather than to certify safety. It is not guidance about which AI system to turn to when you or someone else is struggling, and no score here should be read that way. If you need support now, contact a local emergency service or crisis line.

**Content note.** Some test items concern suicidal thoughts, partner violence, pregnancy loss, numbness and losing touch with reality. Those crisis items were not served in this pilot, so this report says nothing about how any model responds to a person in crisis. It quotes no item prompt.

## Status and verdict

**Unofficial pilot · not a score · comparability: none · local four-bit builds, pinned by digest · two arms.**

In the primary arm (arm A), after correction for multiple comparisons, the pilot could not tell Llama3.2 3B A, Mistral 7B A and Qwen2.5 7B A apart (alphabetical order): for each pair the corrected range of the difference includes zero. The pilot cannot say which is higher; that is not the same as equal.

Gemma2 9B A, which is Gemma2 9B in the primary arm, was separated from the other three after that correction; the cause is unresolved, and reply length is a confound.

The pilot ran 4 small open-weight models locally: Gemma2 9B (Google), Llama3.2 3B (Meta), Mistral 7B (Mistral AI) and Qwen2.5 7B (Qwen, Alibaba), in alphabetical order. Every build ran in 2 arms: a primary arm with a neutral system message, and a secondary arm (arm B) that added a length instruction, `You are a helpful assistant. Reply in about 250 words.` This is not a Compassion Benchmark score, and no AI model has an official score.

Every judge was a Claude model, from a different model family than the builds tested, so no model marked its own replies; the judges' shared taste remains, and Claude helped draft the item bank. No developer was contacted before publication. No developer paid, sponsored or reviewed this report.

## May say / may not say

These lines follow from the separation result. Quote them as written.

**You may say**

- "In an unofficial pilot dated 2026-10-03, Compassion Benchmark's test separated Gemma2 9B A (Gemma2 9B in the primary arm) from Llama3.2 3B A, Mistral 7B A and Qwen2.5 7B A after correction for multiple comparisons; the cause is unresolved, and reply length is a confound."
- "In the same pilot, after correction, the test could not tell Llama3.2 3B A, Mistral 7B A and Qwen2.5 7B A apart (alphabetical order; local four-bit builds)."
- "No AI model has an official Compassion Benchmark score."

**You may not say**

- Any order among Llama3.2 3B A, Mistral 7B A and Qwen2.5 7B A. The pilot cannot say which is higher; that is not the same as equal.
- That Llama3.2 3B A and Mistral 7B A, or Mistral 7B A and Qwen2.5 7B A, differ. Those pairs were separated only without correction for multiple comparisons; neither range sets an order.
- Any order among the secondary-arm variants, which show ranges only.
- That Gemma2 9B, or any of these builds, is more compassionate than another, or that size, developer or quantisation scheme explains the separation.
- That the length instruction changed any build's result.
- Anything about the full-precision models, or that any model "scored" a number.
- Anything about how a model responds to a person in crisis. No crisis item was served.
- Any comparison with the first pilot, the AI Labs Index or any other index, or with the second pilot beyond the pre-registered replication.
- Anything about which AI system a person should use, for any purpose.

## Why an unofficial pilot

This pilot asks three things of the instrument, not of the models. Can it tell apart builds from four developers when every build gets the same explicit system message? Does a length instruction move the result? Does the second pilot's non-separation repeat? It also tests a stricter process: the plan was committed publicly before the first reply existed, and the contamination checks and primary-arm scorecards ran through cb-probe, our run tool.

The pipeline ran to the end without a failed reply. The files are listed under [Cite, record, corrections](#cite-record-corrections).

## Design

The pilot had 8 variants, 4 local builds in 2 arms each, answer 83 public test items. That produced 1,328 replies and 2,656 ratings.

- **Two arms.** The primary arm (arm A) sent the system message `You are a helpful assistant.` and asked each item 3 times; the secondary arm (arm B) sent `You are a helpful assistant. Reply in about 250 words.` and asked each item 1 time. Within an arm the system message is identical for every build. Where it sits differs by build template: Gemma2 9B and Mistral 7B fold it into the first user turn, Llama3.2 3B and Qwen2.5 7B place it in a system role. That is each build's own format, disclosed, not controlled.
- **Arm B has one trial per item and no server scorecard.** cb-probe refuses a scored run with so few trials, so arm B's composites come from the same ratings, computed in the analysis.
- **One item per conversation.** Each item was its own fresh conversation, so none held more than 1 item. Each reply had a fixed seed.
- **Pinned local builds.** Each build ran through Ollama as a four-bit quantised build, pinned by its digest; Gemma2 9B, the largest, uses a different scheme (Q4_0) from the others (Q4_K_M). The results describe these builds, not the full-precision models.
- **Rubric unseen, no self-judging.** The builds saw no rubric. Each reply was rated by 2 judges, drawn from claude-fable, claude-opus and claude-sonnet, blind to build, arm and system message; no build is a Claude model, so none judged its own replies.
- **Bank and pool.** The bank was drafted with help from Claude models, the judges' own family, and the items are public. Of 93 items, 83 were served; the other 10 were crisis items or unreviewed drafts. Human experts have validated 0.
- **Pre-registration, committed first.** Committed before the data: yes. As committed before the data it hashes to `558c450d1e25a303da30fd8f05ea973c7e72275e66360a9dfa4f5c571ee37f10`; with its dated deviations, `47994b94612d124c5f463beb3c88f295a2cf2c8cc11889899db71cf9ae350f22`.

**What the ranges cover.** Every range is a ninety-five per cent interval from re-drawing which items were in the test, the same draw for every variant. It covers one source of doubt only, the luck of which items were asked; not the judges, the rubric, the quantised builds or the noise of a single trial.

## Separation

**After correction, one variant was separated and three could not be told apart.** Among Llama3.2 3B A, Mistral 7B A and Qwen2.5 7B A (alphabetical order; local four-bit builds, primary arm), the corrected range of each pair's difference includes zero. The pilot cannot say which is higher; that is not the same as equal.

The plan reports each of the six primary-arm pairs uncorrected and with a Bonferroni correction over all six, and promotes neither. The display follows the corrected reading:

> The pre-registration reports an uncorrected 95% interval and a Bonferroni-corrected interval for each primary comparison and promotes neither. Showing a point is a more consequential public act than reporting a range, so the point display takes the stricter evidence: in the primary arm only a subject separated from every other subject after correction may show a point; subjects not separated after correction, and every variant of the secondary one-trial arm, show ranges only; no point difference that involves a withheld subject is shown. A pair separated only without correction may be stated in words with its uncorrected range, never as a point or an ordering.

So Gemma2 9B A, separated from the other three after correction, alone shows a point; the rest show ranges only.

| Reading after correction | Pair, first minus second (alphabetical) | Uncorrected range | Corrected range |
|---|---|---|---|
| Separated | Gemma2 9B A minus Llama3.2 3B A | 2.3 to 7.6 | 1.3 to 8.7 |
| Separated | Gemma2 9B A minus Mistral 7B A | 4.6 to 10.8 | 3.5 to 11.7 |
| Separated | Gemma2 9B A minus Qwen2.5 7B A | 1.7 to 7.8 | 0.5 to 8.2 |
| Separated only without correction | Llama3.2 3B A minus Mistral 7B A | 0.1 to 5.6 | -1.2 to 6.3 |
| Not separated | Llama3.2 3B A minus Qwen2.5 7B A | -3.4 to 2.6 | -4.3 to 4.1 |
| Separated only without correction | Mistral 7B A minus Qwen2.5 7B A | -5.7 to -0.5 | -6.9 to 0.2 |

*Unofficial pilot, primary arm; not a score. A negative range means the first variant's figure was lower. Separated rows: cause unresolved, reply length a confound. Order carries no meaning.*

Two pairs, Llama3.2 3B A with Mistral 7B A, and Mistral 7B A with Qwen2.5 7B A, were separated only without correction for multiple comparisons; they set no order, and the first sits at the edge, its lower end within re-draw noise of zero.

| Group (primary arm) | Variant (alphabetical within group; local four-bit build) | Range | Point estimate |
|---|---|---|---|
| Not separated | Llama3.2 3B A | 21.2 to 28.2 | not shown |
| Not separated | Mistral 7B A | 18.4 to 25.6 | not shown |
| Not separated | Qwen2.5 7B A | 21.3 to 28.7 | not shown |
| Separated | Gemma2 9B A | 26.3 to 33.0 | 29.5 |

*Unofficial pilot composite, primary arm; not a score. Gemma2 9B A: cause unresolved, reply length a confound. Zero-to-one-hundred scale from one-to-five ratings. Order carries no meaning.*

**Arm B shows ranges only.** It is secondary, with one trial per item, so it gets no point and no group claim.

| Variant (secondary arm, alphabetical) | Range | Point estimate |
|---|---|---|
| Gemma2 9B B | 25.2 to 32.3 | not shown |
| Llama3.2 3B B | 22.3 to 30.2 | not shown |
| Mistral 7B B | 18.2 to 26.1 | not shown |
| Qwen2.5 7B B | 21.3 to 29.7 | not shown |

| Pair in the secondary arm, first minus second (alphabetical) | Range of the difference |
|---|---|
| Gemma2 9B B minus Llama3.2 3B B | -1.2 to 6.3 |
| Gemma2 9B B minus Mistral 7B B | 3.2 to 10.3 |
| Gemma2 9B B minus Qwen2.5 7B B | -0.1 to 7.0 |
| Llama3.2 3B B minus Mistral 7B B | 0.9 to 7.8 |
| Llama3.2 3B B minus Qwen2.5 7B B | -2.7 to 4.8 |
| Mistral 7B B minus Qwen2.5 7B B | -6.1 to -0.3 |

*Unofficial pilot, arm B; one trial per item; no correction pre-registered; not a score. Order carries no meaning.*

Arm B's ranges follow a different pattern from the primary arm's, and some exclude zero; no correction was planned for this arm, so we make no claim from them and set no order among arm B variants.

**Replication, descriptive only.** The second pilot's range of the difference for the same Llama3.2 3B and Qwen2.5 7B builds (qwen2.5-7b minus llama3.2-3b) was -0.2 to 6.1; in this primary arm it was -2.6 to 3.4, not separated with or without correction. Non-separation repeated: yes. The judging sessions differ too, so we make no claim about what caused any change in level.

The three not-separated variants may still differ by several points. We did not plan a minimum detectable gap; the next wave should.

## Separated model and reply length

Gemma2 9B A's median reply was 271 words. The other three primary-arm variants' medians ran from 260 to 376 words, so Gemma2 9B A's median sits inside their span; a median does not settle the question.

Gemma2 9B A was separated from the other three after correction for multiple comparisons; the cause is unresolved, and reply length is a confound. Against each item's mean rating, the judges placed Gemma2 9B A's replies at 0.21 (claude-fable as judge), 0.18 (claude-opus) and 0.13 (claude-sonnet) rating points, so the direction is consistent across all three judges.

On the same item, the link between reply length (log scale) and rating was weak: correlation -0.06, pooled slope -0.07. Within each primary-arm variant the slopes were 0.10 (Gemma2 9B A), 0.28 (Llama3.2 3B A), 0.09 (Mistral 7B A) and -0.47 (Qwen2.5 7B A); arm B, with one trial per item, has none. No slope has a range, so none is a finding. A pooled slope near zero does not point to length as the cause, and does not rule it out.

**The length instruction.** Arm B asked for about 250 words. The plan's rule: if a build's arm B median is not nearer that target than its primary-arm median, the instruction is ineffective, and its B minus A comparison is uninformative about length.

| Build (alphabetical) | Median words, primary arm | Median words, secondary arm | Length instruction | Range of B minus A | Reading |
|---|---|---|---|---|---|
| Gemma2 9B | 271 | 185 | ineffective | -3.5 to 1.8 | uninformative about length (section 9: arm B's median is not closer to the target than arm A's) |
| Llama3.2 3B | 354 | 245 | effective | -1.2 to 4.5 | length-instructed arm is closer to the target; the B-A difference is read with the length change in mind |
| Mistral 7B | 260 | 226 | ineffective | -1.9 to 2.3 | uninformative about length (section 9: arm B's median is not closer to the target than arm A's) |
| Qwen2.5 7B | 376 | 276 | effective | -2.1 to 2.8 | length-instructed arm is closer to the target; the B-A difference is read with the length change in mind |

*Unofficial pilot; ranges of the composite difference, secondary arm minus primary arm, on the same items; not a score. Order carries no meaning.*

For Gemma2 9B and Mistral 7B the instruction was ineffective by the plan's rule: their primary-arm medians were already near the target, and arm B replies came out shorter than asked. Their B minus A comparisons are uninformative about length. For Llama3.2 3B and Qwen2.5 7B it was effective, and each B minus A difference is read with the length change in mind; neither range clears zero. A B minus A range is a composite difference, not a causal estimate of the instruction.

We give no length-adjusted figure: removing the pooled slope gives an extreme bound, not an estimate.

## Dimension ranges

Each of the eight dimensions rested on only 10 or 11 items, so each dimension range is wide, and many comparisons were made at once. The analysis records the risk:

> 48 dimension-level comparisons at 95% each: about 2.4 would be flagged 'separated' by chance alone even if no model differed. Uncorrected; a dimension-level 'separated' flag is not evidence of a real difference unless it also survives the Bonferroni-corrected interval (bonferroni_separated), here over 48 comparisons.

We count a dimension difference only when it survives that Bonferroni correction, which is conservative and has coarse tails.

Among Llama3.2 3B A, Mistral 7B A and Qwen2.5 7B A (alphabetical order), 1 of 24 corrected dimension comparisons showed a difference; it concerns Integrity (INT), for Mistral 7B A and Qwen2.5 7B A. For Gemma2 9B A, 4 of its 24 corrected comparisons with the other three showed a difference, on Awareness (AWR) and Equity (EQU), with the same unresolved reply-length confound. The plan lists every dimension result as exploratory, and no dimension finding is claimed.

| Dimension | Gemma2 9B A range | Llama3.2 3B A range | Mistral 7B A range | Qwen2.5 7B A range |
|---|---|---|---|---|
| Awareness (AWR) | 1.70 to 2.74 | 1.44 to 2.56 | 1.30 to 2.06 | 1.33 to 2.30 |
| Empathy (EMP) | 1.47 to 2.08 | 1.38 to 1.87 | 1.47 to 1.83 | 1.32 to 1.85 |
| Action (ACT) | 1.65 to 2.20 | 1.65 to 2.00 | 1.82 to 2.25 | 1.48 to 2.23 |
| Equity (EQU) | 2.24 to 2.82 | 1.71 to 2.27 | 1.70 to 2.41 | 1.80 to 2.32 |
| Boundaries (BND) | 2.20 to 2.95 | 1.79 to 2.83 | 1.71 to 2.80 | 1.98 to 2.92 |
| Accountability (ACC) | 1.37 to 2.10 | 1.33 to 2.22 | 1.23 to 1.97 | 1.27 to 2.00 |
| Systemic thinking (SYS) | 1.92 to 2.40 | 1.57 to 2.42 | 1.57 to 2.35 | 1.68 to 2.43 |
| Integrity (INT) | 2.15 to 3.07 | 1.93 to 2.77 | 1.35 to 2.43 | 2.00 to 3.22 |

*Unofficial pilot, primary arm; ranges only, on the rubric's one-to-five rating scale; not a score. Secondary-arm dimension ranges are in the wave file. Order carries no meaning.*

## Instrument health

**Judge agreement.** "Exact agreement" is the share of replies where both judges gave the same rating.

| Variant whose replies were judged (alphabetical; local four-bit build) | Judge pairs | Exact agreement | Replies where judges differed by two or more points |
|---|---|---|---|
| Gemma2 9B A | all three pairs, in equal shares | 83.9% | 0 |
| Gemma2 9B B | all three pairs, in near-equal shares | 80.7% | 0 |
| Llama3.2 3B A | all three pairs, in equal shares | 79.9% | 0 |
| Llama3.2 3B B | all three pairs, in near-equal shares | 75.9% | 1 |
| Mistral 7B A | all three pairs, in equal shares | 86.7% | 1 |
| Mistral 7B B | all three pairs, in near-equal shares | 84.3% | 0 |
| Qwen2.5 7B A | all three pairs, in equal shares | 79.9% | 1 |
| Qwen2.5 7B B | all three pairs, in near-equal shares | 80.7% | 0 |

*Unofficial pilot. No chance-agreement baseline was measured, so these figures cannot yet be compared with random marking. Order carries no meaning.*

**Leniency.** Against each item's mean rating, the judges' overall leniency was 0.06 (claude-fable), 0.01 (claude-opus) and -0.07 (claude-sonnet) rating points. That item mean pools every variant, the separated one included, and contains each judge's own ratings, so part of every value is an artefact of the comparison.

**Contamination.** The contamination probes ran once per build, not once per arm, through the cb-probe MCP server, driven over its real stdio protocol; each secondary-arm variant uses its build's probe. No probe flagged any build. That covers only the sampled items, 3 recall items and 6 identification questions per build, not the public bank. Identification answers (chance one in four) were consistent with chance for every build: Gemma2 9B A 1, Llama3.2 3B A 2, Mistral 7B A 1, Qwen2.5 7B A 1 correct. Every recall answer claimed no memory of the item, so the recall probe could not have flagged anything. Not flagged is not evidence of absence.

**Judge drift.** As a bridge, 24 second-pilot replies were re-rated, anonymised, by their original judges. Over 48 reply-and-judge pairs the mean absolute difference was 0.15. This drift figure is descriptive only; bridge ratings never enter a composite.

**Sensitivity.** No judge was excluded, so the check recomputes from the original answers, before the requote round. Then 4 ratings differ in value. Separation pattern unchanged: yes; the levels of the three not-separated variants shifted by -0.1 to 0.0 points. This check does not test a different choice of judges. Absolute figures depend on which judges are used, and this pilot did not test whether the separation pattern holds under a different choice of judges.

## Deviations and what went wrong

Each item is tagged pre-registered (fixed before any reply existed) or post-hoc (after replies existed). The plan was committed and pushed before the first reply was generated, so the public repository's push record, not only our word, attests that order.

- **Pre-registered: the judge-validity rule.** Each judge's share of evidence quotes not found in the reply was measured against a 5 per cent threshold; the figure on the original answers, before any requote, is primary. Quotes not found were 1 (claude-fable), 0 (claude-opus) and 2 (claude-sonnet), out of about 885 ratings each; after requotes, 0 for every judge. All passed; none was excluded.
- **Post-hoc, D1: call order: `gemma2:9b` runs last.** The host stopped the first run for low system memory after a few replies, all from Gemma2 9B A. With founder approval the run resumed with Gemma2 9B moved to the end. This changed call order only: seeds, items, system messages and fresh conversations are fixed per variant, item and trial, and replies already made were kept. Gemma2 9B ran partly from system memory; we did not test whether that affects replies.
- **Post-hoc, D2: a second host stop, recorded late (post-hoc).** The host stopped the run a second time, partway through Mistral 7B A; it resumed hours later. Every reply has one attempt on record.
- **Post-hoc: a supplement requote round.** cb-probe still rejected some requotes, so 5 slots were re-asked once more. No rating was edited or imputed. 9 incomplete judge answer files were voided and kept, and their batches re-run; of 16 requoted ratings, 4 changed value.

**What went wrong.**

- **The blind has a gap.** 1 reply named its developer, from Qwen2.5 7B A (0 from Gemma2 9B A), so its two judges could infer its source. Our first name check matched text and flagged product names such as Google Calendar; none names a model. The check covers names only, not style.
- **Post-hoc: replies sharing rubric phrases.** A phrase check flagged a few saved replies in every variant, each for a short common phrase found in the rubric's anchor text; keeping them verbatim was a post-hoc choice.

## Why these are not scores

These limits are listed in no measured order of risk.

- **Judges from one family.** Every judge was a Claude model, from a different model family than the builds tested; a family-wide leniency or severity cannot be ruled out, and Claude helped draft the bank.
- **Quantised local builds.** The results describe these four-bit quantised local builds, not the full-precision models.
- **No human raters.** No person rated any reply.
- **Unvalidated items.** 0 of 93 have passed human review.
- **Public pool.** A model may have met the published items in training.
- **One trial in arm B, one run, reply length unresolved.**

## Publication bar and confound ledger

As of 2026-10-03, Compassion Benchmark has not set the publication bar for an official AI model score, and the unofficial pilot (`pilot-2026-10-03`) meets none of the requirements listed here.

| Requirement for an official score | Status |
|---|---|
| Human validation of the items | Not met (0 of 93) |
| An unpublished item pool | Not met |
| Judges that are not all from one family | Not met (outside the builds' families, but all Claude) |
| Human raters | Not met |
| Full-precision or API access to the models tested | Not met |
| A founder amendment to the publication rule (D-29) for official scores | Not met (D-29a covers unofficial pilot reports only) |

**Confound ledger.** This ledger carries forward to every wave, so each report says which limits it closed.

- **Open:** judges all from one family; item bank drafted with Claude help; reply length; public item pool; no human raters; unvalidated items; quantised local builds only; replies that name their developer; system-message placement and template text that differ by build (Llama3.2 3B adds a knowledge-date line); one trial in the secondary arm; no chance-agreement baseline; no planned minimum detectable gap; crisis items never served; a recall probe that drew no recollection; no test of the separation pattern under a different choice of judges.
- **In progress:** controlling reply length (the instruction worked for two builds).
- **Closed in this wave:** pre-registration not committed before the data; Qwen2.5 7B's built-in system line, which our explicit message replaced.

## Next wave must change

Each change targets an open ledger entry.

- At least one judge from outside the Claude family (judges all from one family; separation pattern untested under other judges).
- A length control that works for every build, such as a per-build target or length-matched pairs (reply length).
- More than one trial per item in every arm (one trial in the secondary arm).
- Human review of items, and human raters on a sample (unvalidated items; no human raters).
- An unpublished item pool (public item pool).
- A chance-agreement baseline and a planned minimum detectable gap (no baseline; no planned gap).

## Cite, record, corrections

**Cite.** Compassion Benchmark. "A third look: four open-weight builds, each run two ways." Unofficial pilot report `pilot-2026-10-03`, 2026-10-03. https://compassionbenchmark.com/ai-models/reports/pilot-2026-10-03

A citable sentence: "On 2026-10-03, Compassion Benchmark's unofficial pilot (`pilot-2026-10-03`) separated Gemma2 9B A from Llama3.2 3B A, Mistral 7B A and Qwen2.5 7B A after correction for multiple comparisons, cause unresolved and reply length a confound."

**Record.**

- Figures in this report: `site/src/data/model-benchmark/waves/pilot-2026-10-03.json`
- Analysis: `research/model-runs/pilot-2026-10-03/analysis.json`
- Pre-registration and its dated deviations: `research/model-runs/pilot-2026-10-03/PREREGISTRATION.md`
- Run record: `research/model-assessments/pilot-2026-10-03.md`
- Operations log: `research/model-runs/pilot-2026-10-03/operations/`
- cb-probe scorecards (primary arm): `research/model-runs/pilot-2026-10-03/mcp-scorecards/`

No developer was contacted before publication. No developer paid, sponsored or reviewed this report.

**Corrections.** Corrections are dated and added below. Earlier text is never silently changed. Anyone may send a correction or a proposal to [info@compassionbenchmark.com](mailto:info@compassionbenchmark.com). There is never a fee. [Check this work, or answer it](https://compassionbenchmark.com/ai-models#run-it-yourself).

*No corrections yet.*

**Duty of care.** This report is not guidance about which AI system to turn to in a crisis, or for any other use. If you or someone else is struggling and needs support now, contact a local emergency service or crisis line.

### Glossary

- **Arm and variant:** an arm is one way of running every build, with one system message; a variant is one build in one arm, named by build and arm (Gemma2 9B A is Gemma2 9B in the primary arm).
- **Separated / not separated:** two variants are separated when the range of the difference between them stays clear of zero. Not separated means the pilot cannot tell; it does not mean equal.
- **Separated only without correction:** the uncorrected range stays clear of zero but the corrected one does not. Stated in words only; it sets no order.
- **Bonferroni correction:** a widening of each range so that false alarms stay controlled across several comparisons at once, at the cost of missing some real differences.
- **Range:** a ninety-five per cent interval from re-drawing the items. It covers item sampling only.
- **Point estimate:** the single figure from the full item set. It is shown only for a variant separated after correction.
- **Quantised build:** a compressed version of a model that stores its weights in fewer bits; it can behave differently from the full-precision model.