# The evaluation-results feed

`results.json` in this directory is the published evaluation feed the
[/evaluation](https://fuaran-ui.io/evaluation) page renders. The page shows
*only* what the feed reports – nothing is hand-typed, and a schema drift fails
the site build rather than mis-rendering.

The committed default is the honest pending seed:

```json
{ "status": "pending" }
```

A published feed carries `"status": "published"` plus:

- Provenance: `generatedAt` (ISO timestamp), `commit` (harness short SHA),
  `suiteVersion`, `sourceUrl` (the public "read the numbers yourself" link),
  and the `cohort` + `promptSha` stamps.
- `summary` – `totalPrompts` (judged runs, the pass-rate denominator), `tasks`
  (distinct corpus tasks), `passed`, `passRate`, `passRateCi95` (95% CI
  half-width, clustered by task), `providersEvaluated`, and the optional
  `parsePassRate`, `passRateWithRepair`, `reasoningShare`,
  `costPerCorrectUsd`, `costPerCorrectCachedUsd`.
- `providers[]` – per (provider, model): `name`, `model`, `passed`, `total`,
  `passRate`, `passRateCi95`, `meanOutputTokens`, and the optional
  `parsePassRate`, `costUsd`, `passRateWithRepair`, `reasoningShare` (+
  `reasoningSplitSource`), `costPerCorrectUsd`, `costPerCorrectCachedUsd`.
- `compactness[]` – emission size vs each baseline over (task, provider) pairs
  where both sides passed: `complexity`, `baseline`, `baselineId`,
  `fuaranTokens` / `baselineTokens` / `ratio` (text – the stored artefact),
  `fuaranBilledTokens` / `baselineBilledTokens` / `billedRatio`, the optional
  cold/cached per-call USD fields, `pairs`, `tasks`.
- `costPerCorrect[]` – per condition: `condition`, `conditionId`, `runs`,
  `passRate`, and the optional `meanColdCallUsd`, `meanCachedTurnUsd`,
  `coldUsdPerCorrect`, `cachedUsdPerCorrect`; with the accompanying
  `costPerCorrectNote` and `costPerCorrectCaveat` strings.
- Optional `sessionEconomics` – the multi-turn refinement section, published
  under its own `cohort` stamp: `note`, `excludedTasks[]` (`id`, `reason`),
  and `arms[]` – one row per (condition, model pin): `condition`, `modelPin`,
  `cells`, cumulative billed `usdAtTurn3` / `usdAtTurn5` with their
  `cellsAtTurn3` / `cellsAtTurn5` denominators, `identityMean` /
  `identityCells` (prompt-named ids preserved first → final emission), and the
  `pricingRetrieved` snapshot date. Multi-turn cells publish only here – the
  single-shot rates above never pool them.
- Optional `excludedTasks[]` (`id`, `reason`) + `exclusionCriterion` – the
  zero-discrimination disclosure.

Honesty rules the producer holds and the page preserves: an unmeasured
optional metric is *omitted* (the page renders –, never a fabricated 0); the
pending seed is preserved between runs; every figure carries its provenance.