Evaluation

The claim on the landing page – a UI language AIs can write, and you can trust – is a measurable one, and this page is where the measurement lands. Two questions under test: given a natural-language prompt, how reliably does a model produce a correct, conformant interface – and once the measurement follows a whole refinement session rather than a single call, what does that session cost?

What is measured

Each evaluated model receives the same task set under controlled conditions: emitting Fuaran wire JSON, and emitting conventional UI code against baseline stacks. A Fuaran emission passes only when it clears two gates in sequence – the strict canonical decode (does it parse as valid Fuaran at all?) and an independent judge of task correctness. The code baselines pass a judged render gate over their own output. Each condition passes its own gate, and the comparison is conservative toward the baselines: the Fuaran gate is the stricter one.

Two measurement tiers feed the sections below. The single-turn tier asks for one emission per task and yields the pass, conformance, compactness and cost-per-correct figures. The multi-turn tier refines one artefact across successive prompts under the same per-condition gates, and measures the session rather than the call: the cumulative billed spend through each refinement turn, and whether the elements named in earlier turns keep their identity through to the final emission. The two tiers never pool – a single-shot rate is never diluted (or flattered) by multi-turn cells, and the session figures publish in their own section under their own cohort stamp.

Three honesty rules govern everything below. Every figure is emitted by the harness into this page's feed – nothing is hand-typed, and the page fails its build rather than render a drifted feed. An unmeasured metric is omitted at source and shown here as –, never as a fabricated zero. Confidence intervals are clustered by task, because runs of the same task are not independent – the honest unit of replication.

Results

Best per provider

Each provider's best-performing published arm on this corpus – the model the playground ships, at the measured recommended settings.

Budget postures

The remaining published arms – including each provider's cost-optimal posture, disclosed beside the best rather than replaced by it. A best arm that is also its provider's cost pick appears only above.

gemini-3.6-flash, grok-4.5 double as the provider's cost recommendation.

An unmeasured metric shows – rather than a fabricated figure.

Compactness vs the baselines

Fuaran emission size ÷ baseline size over (task, provider) pairs where both sides produced a passing UI — below 1.00 the Fuaran artefact is smaller. Text = the stored artefact (reasoning excluded); billed = the one-time generation bill's output side.

Session economics vs the baselines – multi-turn refinement

What a whole refinement session costs, not one emission: the same artefact is refined across successive prompts, and each spend figure is the cumulative billed USD through that turn, averaged over the sessions that reached it (denominator beside each figure). Identity preserved = the share of prompt-named element ids present in both the first and the final emission – whether the artefact survives refinement as itself rather than being regenerated.

Tier-B multi-turn session economics: cumulative billed USD through turn k, mean over sessions that reached turn k (denominator beside each figure); identity = fraction of prompt-named element ids present in both the first and final emission, mean over measured cells. Multi-turn cells publish here only — parsePassRate remains a single-shot claim.

Excluded from this section: tier-b-001 — pre-v3 probe scaffold: identity figures structurally unfair (authoring protocol v3 note).

Session-economics cohort: 618-stage2-20260807.

Expected spend per correct artifact (USD)

Broken out per model arm, not only per condition. The conditions did not run the same model mix, so the pooled row compares mixes as much as languages — read the arm rows, where each line is the same model on both sides.

Expected spend per correct artifact: mean per-call USD over ALL primary cells (failures included; repair cells excluded) / pass rate. Each condition passes its OWN gate - fuaran: strict wire parse AND judge; code baselines: a judged render gate over code never compiled or executed in-harness - so cross-condition comparison is conservative toward the baselines.

spend totals undercount whenever attempts were abandoned client-side (timeouts bill the provider but return no usage payload) — see the harness cost-undercount caveat

Provenance

Every figure above is emitted by the harness into this page's feed — nothing is hand-typed, and a number cannot travel without its stamp.

• Generated: 2026-08-09T07:03:49Z
• Cohort: since-20260730T003649Z
• Harness commit: 3e065a78
• Suite version: 1.0
• Read the numbers at the public source