The meatiest part of #164 is the Judge. Playtests showed "masterful" is almost unreachable (about 1 in 98 expert messages), and the bake-off flagged Haiku grading the middle rungs a notch harsh. The owner-approved direction is to bump the Judge to Sonnet and nudge the rubric so the top grades are reachable on pure writing craft.
That change is held back on purpose. The Judge sits on the dice-tilt and gains-multiplier path the run economy rides on, and the repo's rule is that the bake-off decides routes — shipping it unvalidated would break that contract. So the route is unchanged and a breadcrumb points to the plan.
The judge route is untouched; the breadcrumb explains the deferral and the Sonnet-needs-effort gotcha.
server/utils/ai-routing.ts · 212 lines server/utils/ai-routing.ts 212 lines · TypeScript expand all
⋯ 78 lines hidden (lines 1–78) 2 * The per-task model routing table — the provider seam the go-to-market plan 3 * calls for (dispatch 1's bake-off lives behind this file). Every AI task names 4 * its lane and its full parameter payload here, because valid parameter sets 5 * differ by model family: 7 * - Haiku 4.5 rejects `effort` and has no adaptive thinking — omit both. 8 * - Sonnet 4.6 defaults `effort` to HIGH — always set it explicitly (this game 9 * wants reflexes, not deliberation). `temperature` is accepted. 10 * - Opus 4.8 rejects `temperature`/`top_p`/`top_k` — steer tone via prompts. 11 * Thinking is adaptive-only; omitting `thinking` runs without thinking. 13 * Lane defaults were decided by the two-lane bake-off plus the prompt-tuning 14 * campaign (evals/bakeoff.eval.ts, evals/prompt-tune.eval.ts; snapshots + 15 * verdict in evals/results/) — quality-first, with cost a close second. All 16 * thirteen tasks run Anthropic: the prose tellings initially lost the blind 17 * pairwise 19-0 under the shared prompt, and the Claude-voice rewrite 18 * (buildChroniclePromptClaude) won them back on Sonnet. The OpenAI lane stays 19 * fully alive behind EVERWHEN_AI_LANE so a forced reverse migration is a 20 * config change, not a rewrite (the GTM's platform-risk counter). 38 /** 'fixture' is the dev-only deterministic lane (issue #105): a scripted structured 39 * output, no model call. Never selected by `routeFor` (the table is anthropic/openai 40 * only) — the gateway short-circuits to it when a dev fixture is installed, behind 41 * the dev gate. It exists in the union so telemetry can label a fixtured call. */ 42 export type AiLane = 'anthropic' | 'openai' | 'fixture' 44 export interface AnthropicParams { 47 /** Omit on Opus-tier models — the API rejects sampling params there. */ 49 /** Omit on Haiku 4.5 — the API rejects the effort parameter there. 50 * 'xhigh' exists on Opus 4.7+ and Fable 5 only. */ 51 effort ? : 'low' | 'medium' | 'high' | 'xhigh' 52 /** 'adaptive' turns thinking on (billed as output); omitted = no thinking. */ 56 export interface OpenAiParams { 58 /** gpt-5.5 bills hidden reasoning inside this cap — keep generous headroom. */ 60 reasoningEffort ? : 'low' | 'medium' | 'high' 63 export interface TaskRoute { 65 anthropic : AnthropicParams 69 /** The OpenAI lane's flagship — one place, like the old exported MODEL constant. */ 71 const OPENAI_DEFAULTS : OpenAiParams = { model : GPT , maxTokens : 1200 , reasoningEffort : 'low' } 74 * With thinking off, Anthropic's max_tokens is TRUE output budget — no hidden 75 * reasoning billed inside it — so caps are sized to real output shapes instead 76 * of the old padded 1200. 78 export const AI_ROUTES : Record < AiTask , TaskRoute > = { 79 // A rubric classifier in the critical path of every dispatch (it runs before 80 // the die). Haiku for speed; the calibration eval is its gate. 81 // #164 (DEFERRED, needs keys): now that #162 narrowed this to pure writing craft, 82 // re-tune to make sharp/masterful reachable (playtests saw masterful ~1/98) and 83 // unstick Haiku's harsh middle anchor — planned change is a bump to Sonnet PLUS a 84 // rubric nudge. NOT applied here: it's a balance change to a load-bearing rater and 85 // must be confirmed against fresh judge-tune.eval.ts + eval:bakeoff data first. A 86 // Sonnet route must also pin `effort` explicitly (see ai-routing.spec.ts). Full 87 // plan + commands in evals/results/2026-06-24-retune-notes.md. 90 anthropic : { model : 'claude-haiku-4-5' , maxTokens : 300 , temperature : 0 } , 91 openai : OPENAI_DEFAULTS ⋯ 121 lines hidden (lines 92–212) 93 // The player converses with this output every turn — prose is product. 96 anthropic : { model : 'claude-sonnet-4-6' , maxTokens : 700 , temperature : 0.9 , effort : 'low' } , 97 openai : OPENAI_DEFAULTS 99 // The rules-critical judgment layer; code clamps swings into the rolled band, 100 // the model places the swing within it (anachronism is rated separately, by the 101 // reach-rater, and amplifies the result deterministically). Output is four small 102 // fields now (#163 moved anachronism out): a ~9-word headline, a 1-2 sentence 103 // detail, an integer swing, and a valence enum — well under 150 output tokens. 104 // maxTokens trimmed 600 -> 300 (#164) to fit that real shape; with thinking off 105 // it's a true output ceiling, and 300 still leaves >2x headroom. temp/effort 106 // unchanged — the last bake-off held them at 4/4 band ordering and a 7.0 neutral 107 // corridor, and no fresh signal moved them. 110 anthropic : { model : 'claude-sonnet-4-6' , maxTokens : 300 , temperature : 0.3 , effort : 'medium' } , 111 openai : OPENAI_DEFAULTS 113 // The anachronism reach-rater (issue #163): rates how far beyond the figure's 114 // own era a message reaches — message × era only, BLIND to goal, dice, progress, 115 // and ledger, so the wager tier can't flex with the outcome it amplifies. A cheap 116 // temp-0 Haiku classifier off the critical generation path (it runs alongside the 117 // Judge, before the roll); its tier feeds the deterministic anachronism amplifier. 118 // Re-confirmed post-split (#164): Haiku/temp-0 is the cheapest tier that rates 119 // reach reliably (the canonical classifier setting), and the output is two small 120 // fields — a one-phrase reach gloss plus the tier enum, ~40 output tokens — so the 121 // 200 cap stays (a true ceiling well clear of the real shape; trimming it saves no 122 // cost, only risks clipping the tier-anchoring reach phrase). Held as-is. 125 anthropic : { model : 'claude-haiku-4-5' , maxTokens : 200 , temperature : 0 } , 126 openai : OPENAI_DEFAULTS 128 // Re-narrates the whole timeline each turn. The original bake-off lost this 129 // slot 9-0 under the shared prompt; the Claude-voice rewrite (V2's 130 // transmission-chain bar, buildChroniclePromptClaude) won it back 8-5 in 131 // blind pairwise on Sonnet at ~1/4 the prior cost-per-telling — and the 132 // incumbent lane's quota outage made the flip availability-driven too. 133 // Higher-N confirmation pending (evals/results/2026-06-11-verdict.md). 136 anthropic : { model : 'claude-sonnet-4-6' , maxTokens : 2000 , temperature : 1 , effort : 'high' } , 137 openai : OPENAI_DEFAULTS 139 // The final telling — the share artifact and the thing people pay for. The 140 // Claude-voice V1 prompt on SONNET beat the incumbent 16-3 across two 141 // dual-graded rounds (7-3, then 9-0 at N=12) — decisively confirmed, and 142 // cheaper than the Opus/Fable candidates it outscored. Sonnet over Opus is 143 // what the data said, twice; Fable won too but at 4x Sonnet's price for no 144 // measured gain over it. 147 anthropic : { model : 'claude-sonnet-4-6' , maxTokens : 2000 , temperature : 1 , effort : 'high' } , 148 openai : OPENAI_DEFAULTS 150 // Feeds the player's wager calibration — accuracy matters. 153 anthropic : { model : 'claude-sonnet-4-6' , maxTokens : 800 , temperature : 0.5 , effort : 'low' } , 154 openai : OPENAI_DEFAULTS 156 // Factual micro-answers with a known-since stamp. 159 anthropic : { model : 'claude-haiku-4-5' , maxTokens : 500 , temperature : 0.3 } , 160 openai : OPENAI_DEFAULTS 162 // 0-1 calls per run; the objective steers every layer's valence. 165 anthropic : { model : 'claude-sonnet-4-6' , maxTokens : 600 , temperature : 1 , effort : 'low' } , 166 openai : OPENAI_DEFAULTS 168 // Gates a freshly composed objective on the safely-historical bounds (#71): a 169 // cheap, temp-0 classifier (the figure-floor pattern) that catches the 170 // living-people / current-events framings a year bound alone can't. Haiku, 171 // off the critical generation path — at most twice per composition. 174 anthropic : { model : 'claude-haiku-4-5' , maxTokens : 200 , temperature : 0 } , 175 openai : OPENAI_DEFAULTS 177 // Tool-using agent verified against free grounding, curated floor beneath it. 178 // maxTokens here is per ROUND of the agent loop. 181 anthropic : { model : 'claude-haiku-4-5' , maxTokens : 1500 , temperature : 0.7 } , 182 openai : { model : GPT , maxTokens : 2500 } 184 // A dates classifier with hard validation guards behind it (issue #31 bridge). 187 anthropic : { model : 'claude-haiku-4-5' , maxTokens : 300 , temperature : 0 } , 188 openai : OPENAI_DEFAULTS 190 // The safety Sentinel — classifies a whole exchange against the verbatim 191 // Usage Policy (server/utils/usage-policy.ts). Haiku at temp 0 for a 192 // consistent, cheap classifier, exactly the tier Anthropic's content- 193 // moderation guide recommends. Runs once per user action, never on the 194 // critical generation path's tokens. 197 anthropic : { model : 'claude-haiku-4-5' , maxTokens : 400 , temperature : 0 } , 198 openai : OPENAI_DEFAULTS 203 * Resolves a task to its lane + parameters. EVERWHEN_AI_LANE overrides the 204 * whole table ('openai' | 'anthropic') — the emergency lever, and how the eval 205 * matrix runs identical fixtures down each lane. 207 export function routeFor ( task : AiTask ) : { lane : AiLane ; route : TaskRoute } { 208 const route = AI_ROUTES [ task ] 209 const override = process . env . EVERWHEN_AI_LANE 210 const lane = override === 'openai' || override === 'anthropic' ? override : route . lane 211 return { lane , route }
The full plan lives in a results note: the rationale, the exact change to make (including that a Sonnet route must pin effort explicitly, which the routing spec enforces), and the commands that gate it — judge-tune.eval.ts for absolute calibration, eval:bakeoff for the re-pin evidence, and npm run eval to keep the standing "sharp ≥ vague" behavior green.
The deferred-Judge section: decision, why, and the change to make.
evals/results/2026-06-24-retune-notes.md · 121 lines evals/results/2026-06-24-retune-notes.md 121 lines · Markdown expand all
⋯ 49 lines hidden (lines 1–49) 1 # Per-task re-tune after the Judge + reach-rater splits — 2026-06-24 3 Issue #164 . Follow-up to the separation-of-concerns splits (#162 made the Judge 4 objective-blind / pure writing craft; #163 extracted the anachronism reach-rater 5 out of the Timeline Engine). Those splits reshaped the AI task set, so the 6 per-task settings in `server/utils/ai-routing.ts` get a re-tune. 8 The bake-off harness already tracks the new shape: `bakeoff.eval.ts` and 9 `behavior.eval.ts` call `callReachRater` , score anachronism under its own10 `Reach·` layer, and the Timeline block no longer rates anachronism. So this pass11 is settings, not harness. 13 ## Scope this pass: mechanical only 15 No fresh bake-off was run. `eval:bakeoff` + `judge-tune.eval.ts` make real model 16 calls and need `ANTHROPIC_API_KEY` + `OPENAI_API_KEY` (~$1-3), which weren't 17 available here. So this pass lands only the changes that don't depend on a live 18 run, and the Judge recalibration — which is a balance change to a load-bearing 19 rater and must be confirmed against fresh data — is captured below and deferred 20 to a keyed session. The 2026-06-11 snapshot (`bakeoff-2026-06-11.json` ) still 21 stands as the last live evidence. 25 ### Timeline — `maxTokens` 600 → 300 27 #163 moved the anachronism tier out of the Timeline Engine's output. The model 28 now returns four small fields: a ~9-word `headline` , a 1-2 sentence `detail` , an 29 integer `progressChange` , and a `valence` enum — under ~150 output tokens in the 30 worst case. The 600 cap was sized for the old, padded shape (and predates the 31 "caps sized to real output shapes" convention at the top of `ai-routing.ts` ). 32 Trimmed to 300: with thinking off it's a true output ceiling, so 300 still leaves 33 better than 2x headroom over the real shape while dropping the dead padding. 35 `temperature` (0.3) and `effort` (medium) held. The last bake-off had Timeline at36 4/4 band ordering and a 7.0 neutral-EV corridor on those settings, and nothing 37 this pass moved them; re-confirm them in the next live run alongside the Judge. 39 ### Reach-rater — confirmed `haiku` / temp-0 / `maxTokens` 200, held 41 The issue asks for the cheapest model that rates reach reliably. #163 already 42 pinned the canonical cheap-classifier tier (Haiku, temp 0), and the output is two 43 small fields — a one-phrase reach gloss plus the tier enum, ~40 output tokens. The 44 200 cap is a true ceiling well clear of that, so trimming it saves no cost (you're 45 billed for real output, not the cap) and only risks clipping the reach phrase that 46 anchors the tier. Held as-is; this is the re-tune outcome, not a skipped task. The 47 `Reach·` layer in `bakeoff.eval.ts` / `behavior.eval.ts` (≥75% era-reach accuracy)50 ## Deferred — Judge recalibration (needs keys) 52 **Decision (owner-approved): bump the Judge to Sonnet AND nudge the rubric.** Not53 applied this pass — it changes the dice tilt + gains multiplier the run economy 54 rides on, and the repo's rule is that the bake-off decides routes. Apply and 55 confirm it in a keyed session. 57 **Why.** Two findings converge on the Judge:59 1. **Masterful is effectively unreachable** — ~1 in 98 expert messages in60 playtesting. The rubric in `buildJudgePrompt` pins masterful at "the top few 61 percent" and sharp at "uncommon," so that rarity is the prompt working as 62 written. Net effect: the +2 `CRAFT_MODIFIER` and 1.45x `CRAFT_GAIN_FACTOR` (the 63 top of the craft economy) almost never pay out — dead reward surface. 64 2. **Haiku anchors the middle harsh** — the 2026-06-11 bake-off had Haiku65 ordinally perfect (36/36) but grading the authored-"sound" rung a notch low 66 (read vague in 4/6 ladders). `judge-tune.eval.ts` measured Sonnet modestly ⋯ 55 lines hidden (lines 67–121) 67 better on absolute anchoring (mean deviation 0.22; middle-rung exact 21/36). 69 #162 narrowed the Judge from a three-axis composite (sharpness + period-fit + 70 leverage) to writing craft alone. The "top few percent" language was calibrated 71 for the old composite; on a single writing-quality axis the top grades should be 72 reachable for genuinely excellent writing, and the harsh middle is now pure 73 downward miscalibration. 75 **The change to make + validate:** 77 1. **Route → Sonnet.** In `ai-routing.ts` , set the `judge` route to78 `claude-sonnet-4-6` . A Sonnet route MUST pin `effort` explicitly — the server 79 defaults Sonnet to HIGH and `ai-routing.spec.ts` enforces it; this game wants 80 reflexes, so start at `effort: 'low'` (it's a fast pre-roll classifier). Keep 81 `temperature: 0` . Right-size `maxTokens` to the real output (craft enum + 82 <90-char reason, plus the optional event/timing/continuity fields ≈ ~90 tokens 83 worst case): ~220 is ample. Cost note: this moves a per-dispatch, pre-roll 84 critical-path call from Haiku to Sonnet — price it in the cost row. 85 2. **Rubric nudge** in `server/utils/prompt-builder.ts` `buildJudgePrompt` :86 - Pull the "sound" anchor back so honest, competent asks read sound, not vague87 (fix the harsh middle). 88 - Make sharp/masterful reachable for genuinely excellent pure-craft writing —89 reframe "the top few percent" / "uncommon" so the top grades reward writing 90 quality on its own, without a quota mindset, while keeping masterful 91 genuinely rare (a real ceiling, not dead content). 92 - Keep the fairness floors intact: bottom rungs never read masterful, top rung93 never reads reckless (the `hardViolations` gate in `bakeoff.eval.ts` ). 95 **How to validate (the gate):** 98 # both keys required; ~$1-3 total 99 export ANTHROPIC_API_KEY = ... OPENAI_API_KEY = ...101 # 1. Anchor sweep — Haiku vs Sonnet absolute calibration against the authored ladders 102 npx vitest run --config vitest.eval.config.ts evals/judge-tune.eval.ts 103 # want: Sonnet mean anchor deviation down vs the 0.22 baseline; middle-rung 104 # exact up from 21/36; the authored-"sound" rung reads sound, not vague. 106 # 2. Full two-lane bake-off — re-pin evidence + a fresh snapshot 108 # want: Judge ladder ordinal agreement stays at/near 36/36; hard violations 0; 109 # masterful now actually appears on the top rungs; cost row reflects the Sonnet bump. 111 # 3. Standing behavior gate must stay green 113 # want: "Message Judge — pairwise: sharp ≥ vague" stays ok. 116 Iterate the rubric wording until judge-tune shows the middle anchored correctly 117 and the top reachable without breaking ordering or the fairness floors, then 118 re-pin the route and write a fresh `bakeoff-YYYY-MM-DD.json` + verdict. If the 119 Sonnet bump's critical-path cost doesn't earn its keep over a Haiku-plus-rubric 120 nudge, that comparison is the bake-off's to settle — keep both candidates in the
The validation gate — the exact commands a keyed session runs.
evals/results/2026-06-24-retune-notes.md · 121 lines evals/results/2026-06-24-retune-notes.md 121 lines · Markdown expand all
⋯ 93 lines hidden (lines 1–93) 1 # Per-task re-tune after the Judge + reach-rater splits — 2026-06-24 3 Issue #164 . Follow-up to the separation-of-concerns splits (#162 made the Judge 4 objective-blind / pure writing craft; #163 extracted the anachronism reach-rater 5 out of the Timeline Engine). Those splits reshaped the AI task set, so the 6 per-task settings in `server/utils/ai-routing.ts` get a re-tune. 8 The bake-off harness already tracks the new shape: `bakeoff.eval.ts` and 9 `behavior.eval.ts` call `callReachRater` , score anachronism under its own10 `Reach·` layer, and the Timeline block no longer rates anachronism. So this pass11 is settings, not harness. 13 ## Scope this pass: mechanical only 15 No fresh bake-off was run. `eval:bakeoff` + `judge-tune.eval.ts` make real model 16 calls and need `ANTHROPIC_API_KEY` + `OPENAI_API_KEY` (~$1-3), which weren't 17 available here. So this pass lands only the changes that don't depend on a live 18 run, and the Judge recalibration — which is a balance change to a load-bearing 19 rater and must be confirmed against fresh data — is captured below and deferred 20 to a keyed session. The 2026-06-11 snapshot (`bakeoff-2026-06-11.json` ) still 21 stands as the last live evidence. 25 ### Timeline — `maxTokens` 600 → 300 27 #163 moved the anachronism tier out of the Timeline Engine's output. The model 28 now returns four small fields: a ~9-word `headline` , a 1-2 sentence `detail` , an 29 integer `progressChange` , and a `valence` enum — under ~150 output tokens in the 30 worst case. The 600 cap was sized for the old, padded shape (and predates the 31 "caps sized to real output shapes" convention at the top of `ai-routing.ts` ). 32 Trimmed to 300: with thinking off it's a true output ceiling, so 300 still leaves 33 better than 2x headroom over the real shape while dropping the dead padding. 35 `temperature` (0.3) and `effort` (medium) held. The last bake-off had Timeline at36 4/4 band ordering and a 7.0 neutral-EV corridor on those settings, and nothing 37 this pass moved them; re-confirm them in the next live run alongside the Judge. 39 ### Reach-rater — confirmed `haiku` / temp-0 / `maxTokens` 200, held 41 The issue asks for the cheapest model that rates reach reliably. #163 already 42 pinned the canonical cheap-classifier tier (Haiku, temp 0), and the output is two 43 small fields — a one-phrase reach gloss plus the tier enum, ~40 output tokens. The 44 200 cap is a true ceiling well clear of that, so trimming it saves no cost (you're 45 billed for real output, not the cap) and only risks clipping the reach phrase that 46 anchors the tier. Held as-is; this is the re-tune outcome, not a skipped task. The 47 `Reach·` layer in `bakeoff.eval.ts` / `behavior.eval.ts` (≥75% era-reach accuracy)50 ## Deferred — Judge recalibration (needs keys) 52 **Decision (owner-approved): bump the Judge to Sonnet AND nudge the rubric.** Not53 applied this pass — it changes the dice tilt + gains multiplier the run economy 54 rides on, and the repo's rule is that the bake-off decides routes. Apply and 55 confirm it in a keyed session. 57 **Why.** Two findings converge on the Judge:59 1. **Masterful is effectively unreachable** — ~1 in 98 expert messages in60 playtesting. The rubric in `buildJudgePrompt` pins masterful at "the top few 61 percent" and sharp at "uncommon," so that rarity is the prompt working as 62 written. Net effect: the +2 `CRAFT_MODIFIER` and 1.45x `CRAFT_GAIN_FACTOR` (the 63 top of the craft economy) almost never pay out — dead reward surface. 64 2. **Haiku anchors the middle harsh** — the 2026-06-11 bake-off had Haiku65 ordinally perfect (36/36) but grading the authored-"sound" rung a notch low 66 (read vague in 4/6 ladders). `judge-tune.eval.ts` measured Sonnet modestly 67 better on absolute anchoring (mean deviation 0.22; middle-rung exact 21/36). 69 #162 narrowed the Judge from a three-axis composite (sharpness + period-fit + 70 leverage) to writing craft alone. The "top few percent" language was calibrated 71 for the old composite; on a single writing-quality axis the top grades should be 72 reachable for genuinely excellent writing, and the harsh middle is now pure 73 downward miscalibration. 75 **The change to make + validate:** 77 1. **Route → Sonnet.** In `ai-routing.ts` , set the `judge` route to78 `claude-sonnet-4-6` . A Sonnet route MUST pin `effort` explicitly — the server 79 defaults Sonnet to HIGH and `ai-routing.spec.ts` enforces it; this game wants 80 reflexes, so start at `effort: 'low'` (it's a fast pre-roll classifier). Keep 81 `temperature: 0` . Right-size `maxTokens` to the real output (craft enum + 82 <90-char reason, plus the optional event/timing/continuity fields ≈ ~90 tokens 83 worst case): ~220 is ample. Cost note: this moves a per-dispatch, pre-roll 84 critical-path call from Haiku to Sonnet — price it in the cost row. 85 2. **Rubric nudge** in `server/utils/prompt-builder.ts` `buildJudgePrompt` :86 - Pull the "sound" anchor back so honest, competent asks read sound, not vague87 (fix the harsh middle). 88 - Make sharp/masterful reachable for genuinely excellent pure-craft writing —89 reframe "the top few percent" / "uncommon" so the top grades reward writing 90 quality on its own, without a quota mindset, while keeping masterful 91 genuinely rare (a real ceiling, not dead content). 92 - Keep the fairness floors intact: bottom rungs never read masterful, top rung93 never reads reckless (the `hardViolations` gate in `bakeoff.eval.ts` ). 95 **How to validate (the gate):** 98 # both keys required; ~$1-3 total 99 export ANTHROPIC_API_KEY = ... OPENAI_API_KEY = ...101 # 1. Anchor sweep — Haiku vs Sonnet absolute calibration against the authored ladders 102 npx vitest run --config vitest.eval.config.ts evals/judge-tune.eval.ts 103 # want: Sonnet mean anchor deviation down vs the 0.22 baseline; middle-rung 104 # exact up from 21/36; the authored-"sound" rung reads sound, not vague. 106 # 2. Full two-lane bake-off — re-pin evidence + a fresh snapshot 108 # want: Judge ladder ordinal agreement stays at/near 36/36; hard violations 0; 109 # masterful now actually appears on the top rungs; cost row reflects the Sonnet bump. 111 # 3. Standing behavior gate must stay green 113 # want: "Message Judge — pairwise: sharp ≥ vague" stays ok. ⋯ 8 lines hidden (lines 114–121) 116 Iterate the rubric wording until judge-tune shows the middle anchored correctly 117 and the top reachable without breaking ordering or the fairness floors, then 118 re-pin the route and write a fresh `bakeoff-YYYY-MM-DD.json` + verdict. If the 119 Sonnet bump's critical-path cost doesn't earn its keep over a Haiku-plus-rubric 120 nudge, that comparison is the bake-off's to settle — keep both candidates in the