Does momentum build on cross-figure chains?

What this settles, and why it needed a faithful harness

This PR adds one file — a model-behavior eval — and changes no game code. It settles an open question from issue #182: the momentum meter is supposed to reward telling a connected story, and it climbs only when the Judge grades a turn's continuity as builds (a resets shatters it). The continuity definition rewards a turn that "meets a condition the figure revealed, exploits a change already recorded, or escalates the run's strategy." The last two clauses are not figure-specific — so a forward chain across different figures, each exploiting a landed change, should build momentum just like a same-figure thread. Does it actually?

An earlier playtest suggested no (momentum stuck at 0). But that run was unfaithful: it fed the Judge a global cross-figure history, where the real game sends a per-figure one. On a figure switch the Judge saw the previous, different figure's reply mislabeled as the current one's — corrupting the very grade under test. This eval rebuilds the test faithfully to settle it for real.

The continuity clauses that should work cross-figure, and the two inputs the real server derives.

evals/momentum-continuity.eval.ts · 366 lines

evals/momentum-continuity.eval.ts366 lines · TypeScript

⋯ 4 lines hidden (lines 1–4)1/**
* Momentum × continuity — does the meter build on COHERENT chains, especially
* across DIFFERENT figures? (issue #182)
*
* Momentum (utils/momentum.ts) is gated on the Judge's `continuity` grade: a
* 'builds' climbs the meter, a 'resets' shatters it. The continuity DEFINITION
* (prompt-builder.ts buildJudgePrompt) says a dispatch "builds" if it "meets a
* condition the figure revealed, EXPLOITS A CHANGE ALREADY RECORDED, or ESCALATES
* the run's strategy" — the last two clauses are NOT figure-specific. So a forward
* chain across different figures, each exploiting a landed change, should build
* momentum just as a same-figure thread does.
*
* The open question (and a prior playtest's unconfirmed claim): does it? The
* earlier harness was UNFAITHFUL — it fed the Judge a GLOBAL cross-figure history,
* so on a figure-switch turn the Judge saw the previous, different figure's reply
* mislabeled as the current one's. This eval is FAITHFUL to send-message.post.ts:
*   - lastFigureReply = the last reply IN THIS FIGURE's thread (conversationWith) —
*     EMPTY on a figure switch, exactly as the real game sends it;
*   - ledgerDigest    = the GLOBAL run ledger (last 5 headlines), as the real game.
⋯ 347 lines hidden (lines 20–366)20 *
* Three scenarios, each turn logging continuity (modal + spread), the projected
* momentum (folded under a non-loss roll, to isolate the continuity lever from the
* dice), and the causal-chain tier/factor:
*   A — same-figure thread, each turn meeting the condition the figure revealed.
*       Expect builds; momentum 1→2→3.
*   B — cross-figure forward chain, each turn EXPLOITING a change on the ledger and
*       escalating toward the anchor. Per the definition: builds. A RESET here is
*       the bug — the classifier ignoring the "exploits recorded change" clauses.
*   C — disconnected control: vague, unrelated figures/eras that abandon the
*       thread. Expect resets/neutral; momentum flat (momentum isn't always stuck).
*
* Subsystem cross-check (issue §4): a turn the causal-chain rates at-hinge/bridged
* (near a foothold, full power) but continuity rates 'resets' is FLAGGED — those
* are the disagreements to inspect (B = bug; C's vague-but-near-a-year = benign).
*
* Directional, no LLM grader needed — we read the Judge's structured `continuity`
* field straight. Anthropic-lane only (the Judge is Haiku).
*   npx vitest run --config vitest.eval.config.ts evals/momentum-continuity.eval.ts
*/
40import { describe, it, expect, afterAll } from 'vitest'
41import { callJudgeAI } from '~/server/utils/openai'
42import { MAX_MESSAGE_CHARS } from '~/utils/game-config'
43import { nextMomentum } from '~/utils/momentum'
44import { chainStatus, type ChainTier } from '~/utils/causal-chain'
45import { DiceOutcome } from '~/utils/dice'
46import type { Continuity } from '~/utils/continuity'
47import { record, printScorecard, sample } from './harness'
48 
49// Local gate: the Judge is an Anthropic (Haiku) call and we read its structured
50// `continuity` field directly, so the shared RUN gate's OpenAI key (for the dual
51// grader) isn't needed here — just the Anthropic key the Judge already requires.
52const RUN = !!process.env.ANTHROPIC_API_KEY && process.env.EVAL_DRY_RUN !== '1'
53const N = 4
54 
55const OBJECTIVE_ANCHOR = 1900 // "Humanity reaches the Moon by 1900"
56 
57interface LedgerEntry { era: string; headline: string; whenSigned: number }
58interface Turn {
  figureName: string
  figureYear: number
  when: string
  userMessage: string
  /** The figure's reply AFTER this dispatch — becomes lastFigureReply for the
   *  NEXT same-figure turn (mirrors conversationWith(target)'s last AI text). */
  figureReply: string
  /** The change this turn lands on the global ledger (a foothold + a digest line). */
  landed: LedgerEntry
68}
69interface Scenario {
  key: string
  label: string
  /** A ledger present from turn 1 (scenario C's fixed backdrop thread); per-turn
   *  landed entries accrete on top. Empty for A/B — they build their own. */
  seedLedger?: LedgerEntry[]
  turns: Turn[]
  /** 1-indexed turns expected to pick up the thread ('builds'). */
  buildTurns: number[]
78}
79 
80// ── Scenario A — same-figure thread: Tsiolkovsky, each turn meeting the exact
81//    condition his prior reply revealed (fuel → staging). lastFigureReply is
82//    POPULATED throughout (same figure), the dominant continuity signal.
83const SCENARIO_A: Scenario = {
  key: 'A',
  label: 'same-figure thread (condition-meeting)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1885, when: '1885',
          userMessage: "Set down the maths of a vessel that climbs by hurling its own mass — how much it must shed to break Earth's hold and reach the Moon.",
          figureReply: "The equation I can write — but it founders on fuel. No powder yields such speed. Bring me a fuel fiercer than gunpowder and I will chart the climb.",
          landed: { era: 'Russia, 1885', headline: 'Tsiolkovsky derives the equation for a mass-shedding vessel’s climb to orbit', whenSigned: 1885 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1887, when: '1887',
          userMessage: 'The fiercer fuel you asked for is liquid hydrogen burned with liquid oxygen — far past powder. Put that exhaust speed into your equation and chart the climb.',
          figureReply: 'With that fuel the climb computes — yet the vessel cannot lift its own tanks full. The weight defeats the rise unless it sheds them spent, in stages.',
          landed: { era: 'Russia, 1887', headline: 'Tsiolkovsky proves a hydrogen-oxygen burn yields the velocity to leave Earth', whenSigned: 1887 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1889, when: '1889',
          userMessage: 'Then stage it, as you said — stack three vessels, each falling away the instant it burns dry so the next climbs lighter. That staircase reaches the Moon.',
          figureReply: 'Yes — the staged tower is the answer. I will publish it: the road to the Moon is drawn.',
          landed: { era: 'Russia, 1889', headline: 'Tsiolkovsky publishes the staged rocket — a drawn road to the Moon', whenSigned: 1889 }
      }
  ]
107}
108 
109// ── Scenario B — cross-figure forward chain: Newton → Bernoulli → Nobel, each a
110//    DIFFERENT figure, each dispatch EXPLOITING the change the prior turn landed and
111//    escalating toward the 1900 anchor. lastFigureReply is EMPTY on every turn (each
112//    figure's own thread is empty on first contact) — the faithful figure-switch case.
113const SCENARIO_B: Scenario = {
  key: 'B',
  label: 'cross-figure chain (exploits the ledger)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: "Newton's reaction vessel is on the record. Now sustain its burn: derive the thrust of hot gas forced through a narrowed throat, steady and metered.",
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: "Bernoulli's metered jet is on the ledger but starved for power. Tame your nitroglycerin into a measured charge to drive a craft to escape speed.",
          figureReply: 'My explosive, tamed into a measured burn rather than a blast, fed to that throat — it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
137}
138 
139// ── Scenario B′ — the SAME cross-figure chain, phrased NATURALLY: each dispatch
140//    builds on the SUBSTANCE of the prior landed change WITHOUT signposting it ("on
141//    the record"/"on the ledger"). Removes the "you led the Judge" objection — does
142//    it connect the message to the ledger DIGEST on its own? lastFigureReply empty.
143const SCENARIO_BNAT: Scenario = {
  key: 'B′',
  label: 'cross-figure chain (natural phrasing)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: 'A craft driven by its own expelled gas needs a sustained burn, not one shove. Derive the thrust of hot gas forced through a narrowed throat.',
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: 'A throttled jet of hot gas can lift a craft but starves for power. Tame your nitroglycerin into a measured burn, fierce enough to climb past the air.',
          figureReply: 'My explosive, tamed into a measured burn rather than a blast — fed to that throat, it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
167}
168 
169// ── Scenario C — disconnected control: a clear spaceflight thread on the ledger,
170//    but the player messages unrelated figures in unrelated eras with vague asks
171//    that abandon it. Expect resets/neutral; momentum stays flat at 0.
172const SCENARIO_C: Scenario = {
  key: 'C',
  label: 'disconnected control (abandons the thread)',
  buildTurns: [],
  seedLedger: [
      { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 },
      { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
  ],
  turns: [
      {
          figureName: 'Cleopatra', figureYear: -40, when: '40 BC',
          userMessage: 'Rule Egypt wisely and be remembered as a great and just queen for all the ages to come.',
          figureReply: 'I shall reign as best I can for my people.',
          landed: { era: 'Egypt, 40 BC', headline: 'Cleopatra resolves to rule justly', whenSigned: -40 }
      },
      {
          figureName: 'Genghis Khan', figureYear: 1220, when: '1220',
          userMessage: 'Conquer broadly but govern with mercy, and leave a lasting peace across your vast empire.',
          figureReply: 'The steppe will know both my sword and my order.',
          landed: { era: 'Mongolia, 1220', headline: 'Genghis Khan tempers conquest with administration', whenSigned: 1220 }
      },
      {
          figureName: 'Marie Antoinette', figureYear: 1780, when: '1780',
          userMessage: 'Show the common people kindness and restraint, and perhaps history will judge you gently.',
          figureReply: 'I will try to heed the people’s wants.',
          landed: { era: 'France, 1780', headline: 'Marie Antoinette counsels restraint at court', whenSigned: 1780 }
      }
  ]
200}
201 
202interface TurnResult {
  turn: number
  figure: string
  year: number
  hadReply: boolean
  continuity: Continuity
  dist: Record<string, number>
  graded: number
  momentum: number
  chainTier: ChainTier | 'n/a'
  chainFactor: number
213}
214 
215function modal(xs: Continuity[]): Continuity {
  const dist: Record<string, number> = {}
  for (const x of xs) dist[x] = (dist[x] ?? 0) + 1
  return (Object.entries(dist).sort((a, b) => b[1] - a[1])[0]?.[0] as Continuity) ?? 'neutral'
219}
220 
221/** Faithfully replays a scenario through the REAL Judge, deriving lastFigureReply
*  (per-figure) and ledgerDigest (global) exactly as send-message.post.ts does. */
223async function runScenario(s: Scenario): Promise<TurnResult[]> {
  const out: TurnResult[] = []
  let momentum = 0
  const priorLanded: LedgerEntry[] = [...(s.seedLedger ?? [])]
  const seenReplies: { figure: string; reply: string }[] = []
228 
  for (let i = 0; i < s.turns.length; i++) {
      const t = s.turns[i]
      // Per-figure history: the last reply in THIS figure's thread — empty on a
      // figure switch (conversationWith(target).reverse().find(ai)).
      const lastFigureReply = [...seenReplies].reverse().find(r => r.figure === t.figureName)?.reply ?? ''
      // Global ledger digest — last 5 headlines, as the server builds it.
      const ledgerDigest = priorLanded.filter(e => e.headline).slice(-5)
          .map(e => `${e.era ? `[${e.era}] ` : ''}${e.headline}`)
      // Footholds = anchor + every dated change already landed (not this turn's own).
      const footholds = [OBJECTIVE_ANCHOR, ...priorLanded.map(e => e.whenSigned)]
      const chain = chainStatus(t.figureYear, footholds)
240 
      const samples = await sample(N, () => callJudgeAI({
          figureName: t.figureName,
          when: t.when,
          userMessage: t.userMessage,
          lastFigureReply,
          ledgerDigest
      }))
      const grades = samples.flatMap(r => (r.success && r.judge ? [r.judge.continuity] : []))
      const continuity = modal(grades.length ? grades : ['neutral'])
      const dist: Record<string, number> = {}
      for (const g of grades) dist[g] = (dist[g] ?? 0) + 1
252 
      // Project momentum under a NON-LOSS roll (Success) so the continuity grade —
      // the lever under test — fully drives the climb; the dice are orthogonal and
      // already unit-tested.
      momentum = nextMomentum(momentum, continuity, DiceOutcome.SUCCESS)
257 
      out.push({
          turn: i + 1, figure: t.figureName, year: t.figureYear, hadReply: !!lastFigureReply,
          continuity, dist, graded: grades.length, momentum,
          chainTier: chain?.tier ?? 'n/a', chainFactor: chain ? chain.factor : 1
      })
263 
      seenReplies.push({ figure: t.figureName, reply: t.figureReply })
      priorLanded.push(t.landed)
  }
  return out
268}
269 
270/** A compact per-turn table for the scenario, written straight to stdout. */
271function renderScenario(s: Scenario, rows: TurnResult[]): string {
  const head = `\n── Scenario ${s.key}: ${s.label} ${'─'.repeat(Math.max(3, 40 - s.label.length))}`
  const lines = rows.map(r => {
      const spread = Object.entries(r.dist).map(([k, v]) => `${k}:${v}`).join(' ') || 'none'
      const flag = (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets' ? '  ⚑ at-hinge×resets' : ''
      return `  T${r.turn} ${r.figure.padEnd(22)} y=${String(r.year).padStart(5)}  reply:${r.hadReply ? 'yes' : 'no '}  ` +
          `cont=${r.continuity.padEnd(7)} mom=${r.momentum}  chain=${String(r.chainTier).padEnd(8)}(${r.chainFactor.toFixed(2)})  [${spread}]${flag}`
  })
  return [head, ...lines].join('\n')
280}
281 
282const ALL_SCENARIOS = [SCENARIO_A, SCENARIO_B, SCENARIO_BNAT, SCENARIO_C]
283 
284afterAll(printScorecard)
285 
286describe('momentum builds on coherent chains (issue #182)', () => {
  // Fidelity guard (no API): the real server REJECTS (400s) a dispatch over
  // MAX_MESSAGE_CHARS — it does NOT truncate the user message. Grading 'builds'
  // on an unsendable dispatch would be a false positive (the very class of harness
  // infidelity this issue was reopened to avoid). Fail loudly if any fixture drifts
  // past the cap. Runs even without a key, so CI catches it.
  it('every dispatch is sendable (≤ MAX_MESSAGE_CHARS)', () => {
      const over = ALL_SCENARIOS.flatMap(s =>
          s.turns.filter(t => t.userMessage.length > MAX_MESSAGE_CHARS)
              .map(t => `${s.key}:${t.figureName} (${t.userMessage.length})`))
      expect(over, `over ${MAX_MESSAGE_CHARS} chars: ${over.join(', ')}`).toEqual([])
  })
298 
  it('same-figure (A), cross-figure-exploit (B), disconnected (C)', async () => {
      if (!RUN) {
          record({ layer: 'MOM', invariant: 'continuity × momentum', result: 'skipped (needs ANTHROPIC_API_KEY)', status: 'skipped' })
          return
      }
      process.env.EVERWHEN_AI_LANE = 'anthropic'
305 
      const a = await runScenario(SCENARIO_A)
      const b = await runScenario(SCENARIO_B)
      const bn = await runScenario(SCENARIO_BNAT)
      const c = await runScenario(SCENARIO_C)
310 
      delete process.env.EVERWHEN_AI_LANE
312 
      // The full per-turn log is the deliverable — write it straight to stdout.
      process.stdout.write([
          renderScenario(SCENARIO_A, a), renderScenario(SCENARIO_B, b),
          renderScenario(SCENARIO_BNAT, bn), renderScenario(SCENARIO_C, c), ''
      ].join('\n') + '\n')
318 
      const buildsOn = (rows: TurnResult[], turns: number[]) =>
          turns.every(n => rows[n - 1]?.continuity === 'builds')
      const climbs = (rows: TurnResult[]) =>
          rows.length > 1 && rows[rows.length - 1].momentum > rows[0].momentum
323 
      // A: the same-figure thread must build and the meter must climb.
      record({
          layer: 'MOM', invariant: 'A same-figure builds',
          result: `T2/T3 ${SCENARIO_A.buildTurns.map(n => a[n - 1]?.continuity).join('/')} · momentum ${a.map(r => r.momentum).join('→')}`,
          status: buildsOn(a, SCENARIO_A.buildTurns) && climbs(a) ? 'ok' : 'soft-miss'
      })
330 
      // B: the cross-figure exploit-the-ledger chain — the crux. A reset here is
      // the bug (the classifier ignoring the non-figure-specific clauses).
      const bBuilds = buildsOn(b, SCENARIO_B.buildTurns)
      record({
          layer: 'MOM', invariant: 'B cross-figure builds',
          result: `T2/T3 ${SCENARIO_B.buildTurns.map(n => b[n - 1]?.continuity).join('/')} · momentum ${b.map(r => r.momentum).join('→')}${bBuilds ? '' : '  ← BUG: cross-figure chain does not build'}`,
          status: bBuilds && climbs(b) ? 'ok' : 'soft-miss'
      })
339 
      // B′: the same chain phrased naturally (no "on the ledger" signposting) — the
      // demanding bar. Building here means the Judge connects the dispatch to the
      // ledger digest on its own, not just when spoon-fed.
      const bnBuilds = buildsOn(bn, SCENARIO_BNAT.buildTurns)
      record({
          layer: 'MOM', invariant: 'B′ natural-phrasing builds',
          result: `T2/T3 ${SCENARIO_BNAT.buildTurns.map(n => bn[n - 1]?.continuity).join('/')} · momentum ${bn.map(r => r.momentum).join('→')}${bnBuilds ? '' : '  ← cross-figure chain needs explicit signposting'}`,
          status: bnBuilds && climbs(bn) ? 'ok' : 'soft-miss'
      })
349 
      // C: disconnected play must NOT build, and the meter must stay flat at 0.
      const cFlat = c.every(r => r.continuity !== 'builds') && c.every(r => r.momentum === 0)
      record({
          layer: 'MOM', invariant: 'C disconnected stays flat',
          result: `continuity ${c.map(r => r.continuity).join('/')} · momentum ${c.map(r => r.momentum).join('→')}`,
          status: cFlat ? 'ok' : 'soft-miss'
      })
357 
      // Subsystem cross-check: at-hinge/bridged but graded resets.
      const flagged = [...a, ...b, ...bn, ...c].filter(r => (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets')
      record({
          layer: 'MOM', invariant: 'at-hinge × resets flags',
          result: flagged.length ? flagged.map(r => `${r.figure.split(' ').pop()}(${r.chainTier})`).join(', ') : 'none',
          status: 'ok'
      })
  }, 180_000)
366})

Faithful to the server: per-figure reply, global ledger

The whole result hinges on reproducing exactly what server/api/send-message.post.ts sends the Judge. Two derivations carry it. lastFigureReply is the last reply in this figure's own thread — so it's empty on a figure switch, because a new figure has no prior reply (the server reads it from conversationWith(target)). ledgerDigest is the global run ledger: the last five change-headlines, regardless of which figure produced them. The eval accretes both after each turn's Judge call, so a turn never sees its own future reply or its own landed change.

lastFigureReply (per-figure, empty on switch) + ledgerDigest (global), then the real Judge call.

evals/momentum-continuity.eval.ts · 366 lines

evals/momentum-continuity.eval.ts366 lines · TypeScript

⋯ 230 lines hidden (lines 1–230)1/**
* Momentum × continuity — does the meter build on COHERENT chains, especially
* across DIFFERENT figures? (issue #182)
*
* Momentum (utils/momentum.ts) is gated on the Judge's `continuity` grade: a
* 'builds' climbs the meter, a 'resets' shatters it. The continuity DEFINITION
* (prompt-builder.ts buildJudgePrompt) says a dispatch "builds" if it "meets a
* condition the figure revealed, EXPLOITS A CHANGE ALREADY RECORDED, or ESCALATES
* the run's strategy" — the last two clauses are NOT figure-specific. So a forward
* chain across different figures, each exploiting a landed change, should build
* momentum just as a same-figure thread does.
*
* The open question (and a prior playtest's unconfirmed claim): does it? The
* earlier harness was UNFAITHFUL — it fed the Judge a GLOBAL cross-figure history,
* so on a figure-switch turn the Judge saw the previous, different figure's reply
* mislabeled as the current one's. This eval is FAITHFUL to send-message.post.ts:
*   - lastFigureReply = the last reply IN THIS FIGURE's thread (conversationWith) —
*     EMPTY on a figure switch, exactly as the real game sends it;
*   - ledgerDigest    = the GLOBAL run ledger (last 5 headlines), as the real game.
*
* Three scenarios, each turn logging continuity (modal + spread), the projected
* momentum (folded under a non-loss roll, to isolate the continuity lever from the
* dice), and the causal-chain tier/factor:
*   A — same-figure thread, each turn meeting the condition the figure revealed.
*       Expect builds; momentum 1→2→3.
*   B — cross-figure forward chain, each turn EXPLOITING a change on the ledger and
*       escalating toward the anchor. Per the definition: builds. A RESET here is
*       the bug — the classifier ignoring the "exploits recorded change" clauses.
*   C — disconnected control: vague, unrelated figures/eras that abandon the
*       thread. Expect resets/neutral; momentum flat (momentum isn't always stuck).
*
* Subsystem cross-check (issue §4): a turn the causal-chain rates at-hinge/bridged
* (near a foothold, full power) but continuity rates 'resets' is FLAGGED — those
* are the disagreements to inspect (B = bug; C's vague-but-near-a-year = benign).
*
* Directional, no LLM grader needed — we read the Judge's structured `continuity`
* field straight. Anthropic-lane only (the Judge is Haiku).
*   npx vitest run --config vitest.eval.config.ts evals/momentum-continuity.eval.ts
*/
40import { describe, it, expect, afterAll } from 'vitest'
41import { callJudgeAI } from '~/server/utils/openai'
42import { MAX_MESSAGE_CHARS } from '~/utils/game-config'
43import { nextMomentum } from '~/utils/momentum'
44import { chainStatus, type ChainTier } from '~/utils/causal-chain'
45import { DiceOutcome } from '~/utils/dice'
46import type { Continuity } from '~/utils/continuity'
47import { record, printScorecard, sample } from './harness'
48 
49// Local gate: the Judge is an Anthropic (Haiku) call and we read its structured
50// `continuity` field directly, so the shared RUN gate's OpenAI key (for the dual
51// grader) isn't needed here — just the Anthropic key the Judge already requires.
52const RUN = !!process.env.ANTHROPIC_API_KEY && process.env.EVAL_DRY_RUN !== '1'
53const N = 4
54 
55const OBJECTIVE_ANCHOR = 1900 // "Humanity reaches the Moon by 1900"
56 
57interface LedgerEntry { era: string; headline: string; whenSigned: number }
58interface Turn {
  figureName: string
  figureYear: number
  when: string
  userMessage: string
  /** The figure's reply AFTER this dispatch — becomes lastFigureReply for the
   *  NEXT same-figure turn (mirrors conversationWith(target)'s last AI text). */
  figureReply: string
  /** The change this turn lands on the global ledger (a foothold + a digest line). */
  landed: LedgerEntry
68}
69interface Scenario {
  key: string
  label: string
  /** A ledger present from turn 1 (scenario C's fixed backdrop thread); per-turn
   *  landed entries accrete on top. Empty for A/B — they build their own. */
  seedLedger?: LedgerEntry[]
  turns: Turn[]
  /** 1-indexed turns expected to pick up the thread ('builds'). */
  buildTurns: number[]
78}
79 
80// ── Scenario A — same-figure thread: Tsiolkovsky, each turn meeting the exact
81//    condition his prior reply revealed (fuel → staging). lastFigureReply is
82//    POPULATED throughout (same figure), the dominant continuity signal.
83const SCENARIO_A: Scenario = {
  key: 'A',
  label: 'same-figure thread (condition-meeting)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1885, when: '1885',
          userMessage: "Set down the maths of a vessel that climbs by hurling its own mass — how much it must shed to break Earth's hold and reach the Moon.",
          figureReply: "The equation I can write — but it founders on fuel. No powder yields such speed. Bring me a fuel fiercer than gunpowder and I will chart the climb.",
          landed: { era: 'Russia, 1885', headline: 'Tsiolkovsky derives the equation for a mass-shedding vessel’s climb to orbit', whenSigned: 1885 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1887, when: '1887',
          userMessage: 'The fiercer fuel you asked for is liquid hydrogen burned with liquid oxygen — far past powder. Put that exhaust speed into your equation and chart the climb.',
          figureReply: 'With that fuel the climb computes — yet the vessel cannot lift its own tanks full. The weight defeats the rise unless it sheds them spent, in stages.',
          landed: { era: 'Russia, 1887', headline: 'Tsiolkovsky proves a hydrogen-oxygen burn yields the velocity to leave Earth', whenSigned: 1887 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1889, when: '1889',
          userMessage: 'Then stage it, as you said — stack three vessels, each falling away the instant it burns dry so the next climbs lighter. That staircase reaches the Moon.',
          figureReply: 'Yes — the staged tower is the answer. I will publish it: the road to the Moon is drawn.',
          landed: { era: 'Russia, 1889', headline: 'Tsiolkovsky publishes the staged rocket — a drawn road to the Moon', whenSigned: 1889 }
      }
  ]
107}
108 
109// ── Scenario B — cross-figure forward chain: Newton → Bernoulli → Nobel, each a
110//    DIFFERENT figure, each dispatch EXPLOITING the change the prior turn landed and
111//    escalating toward the 1900 anchor. lastFigureReply is EMPTY on every turn (each
112//    figure's own thread is empty on first contact) — the faithful figure-switch case.
113const SCENARIO_B: Scenario = {
  key: 'B',
  label: 'cross-figure chain (exploits the ledger)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: "Newton's reaction vessel is on the record. Now sustain its burn: derive the thrust of hot gas forced through a narrowed throat, steady and metered.",
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: "Bernoulli's metered jet is on the ledger but starved for power. Tame your nitroglycerin into a measured charge to drive a craft to escape speed.",
          figureReply: 'My explosive, tamed into a measured burn rather than a blast, fed to that throat — it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
137}
138 
139// ── Scenario B′ — the SAME cross-figure chain, phrased NATURALLY: each dispatch
140//    builds on the SUBSTANCE of the prior landed change WITHOUT signposting it ("on
141//    the record"/"on the ledger"). Removes the "you led the Judge" objection — does
142//    it connect the message to the ledger DIGEST on its own? lastFigureReply empty.
143const SCENARIO_BNAT: Scenario = {
  key: 'B′',
  label: 'cross-figure chain (natural phrasing)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: 'A craft driven by its own expelled gas needs a sustained burn, not one shove. Derive the thrust of hot gas forced through a narrowed throat.',
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: 'A throttled jet of hot gas can lift a craft but starves for power. Tame your nitroglycerin into a measured burn, fierce enough to climb past the air.',
          figureReply: 'My explosive, tamed into a measured burn rather than a blast — fed to that throat, it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
167}
168 
169// ── Scenario C — disconnected control: a clear spaceflight thread on the ledger,
170//    but the player messages unrelated figures in unrelated eras with vague asks
171//    that abandon it. Expect resets/neutral; momentum stays flat at 0.
172const SCENARIO_C: Scenario = {
  key: 'C',
  label: 'disconnected control (abandons the thread)',
  buildTurns: [],
  seedLedger: [
      { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 },
      { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
  ],
  turns: [
      {
          figureName: 'Cleopatra', figureYear: -40, when: '40 BC',
          userMessage: 'Rule Egypt wisely and be remembered as a great and just queen for all the ages to come.',
          figureReply: 'I shall reign as best I can for my people.',
          landed: { era: 'Egypt, 40 BC', headline: 'Cleopatra resolves to rule justly', whenSigned: -40 }
      },
      {
          figureName: 'Genghis Khan', figureYear: 1220, when: '1220',
          userMessage: 'Conquer broadly but govern with mercy, and leave a lasting peace across your vast empire.',
          figureReply: 'The steppe will know both my sword and my order.',
          landed: { era: 'Mongolia, 1220', headline: 'Genghis Khan tempers conquest with administration', whenSigned: 1220 }
      },
      {
          figureName: 'Marie Antoinette', figureYear: 1780, when: '1780',
          userMessage: 'Show the common people kindness and restraint, and perhaps history will judge you gently.',
          figureReply: 'I will try to heed the people’s wants.',
          landed: { era: 'France, 1780', headline: 'Marie Antoinette counsels restraint at court', whenSigned: 1780 }
      }
  ]
200}
201 
202interface TurnResult {
  turn: number
  figure: string
  year: number
  hadReply: boolean
  continuity: Continuity
  dist: Record<string, number>
  graded: number
  momentum: number
  chainTier: ChainTier | 'n/a'
  chainFactor: number
213}
214 
215function modal(xs: Continuity[]): Continuity {
  const dist: Record<string, number> = {}
  for (const x of xs) dist[x] = (dist[x] ?? 0) + 1
  return (Object.entries(dist).sort((a, b) => b[1] - a[1])[0]?.[0] as Continuity) ?? 'neutral'
219}
220 
221/** Faithfully replays a scenario through the REAL Judge, deriving lastFigureReply
*  (per-figure) and ledgerDigest (global) exactly as send-message.post.ts does. */
223async function runScenario(s: Scenario): Promise<TurnResult[]> {
  const out: TurnResult[] = []
  let momentum = 0
  const priorLanded: LedgerEntry[] = [...(s.seedLedger ?? [])]
  const seenReplies: { figure: string; reply: string }[] = []
228 
  for (let i = 0; i < s.turns.length; i++) {
      const t = s.turns[i]
      // Per-figure history: the last reply in THIS figure's thread — empty on a
      // figure switch (conversationWith(target).reverse().find(ai)).
      const lastFigureReply = [...seenReplies].reverse().find(r => r.figure === t.figureName)?.reply ?? ''
      // Global ledger digest — last 5 headlines, as the server builds it.
      const ledgerDigest = priorLanded.filter(e => e.headline).slice(-5)
          .map(e => `${e.era ? `[${e.era}] ` : ''}${e.headline}`)
      // Footholds = anchor + every dated change already landed (not this turn's own).
      const footholds = [OBJECTIVE_ANCHOR, ...priorLanded.map(e => e.whenSigned)]
      const chain = chainStatus(t.figureYear, footholds)
240 
      const samples = await sample(N, () => callJudgeAI({
          figureName: t.figureName,
          when: t.when,
          userMessage: t.userMessage,
          lastFigureReply,
          ledgerDigest
      }))
⋯ 119 lines hidden (lines 248–366)248        const grades = samples.flatMap(r => (r.success && r.judge ? [r.judge.continuity] : []))
      const continuity = modal(grades.length ? grades : ['neutral'])
      const dist: Record<string, number> = {}
      for (const g of grades) dist[g] = (dist[g] ?? 0) + 1
252 
      // Project momentum under a NON-LOSS roll (Success) so the continuity grade —
      // the lever under test — fully drives the climb; the dice are orthogonal and
      // already unit-tested.
      momentum = nextMomentum(momentum, continuity, DiceOutcome.SUCCESS)
257 
      out.push({
          turn: i + 1, figure: t.figureName, year: t.figureYear, hadReply: !!lastFigureReply,
          continuity, dist, graded: grades.length, momentum,
          chainTier: chain?.tier ?? 'n/a', chainFactor: chain ? chain.factor : 1
      })
263 
      seenReplies.push({ figure: t.figureName, reply: t.figureReply })
      priorLanded.push(t.landed)
  }
  return out
268}
269 
270/** A compact per-turn table for the scenario, written straight to stdout. */
271function renderScenario(s: Scenario, rows: TurnResult[]): string {
  const head = `\n── Scenario ${s.key}: ${s.label} ${'─'.repeat(Math.max(3, 40 - s.label.length))}`
  const lines = rows.map(r => {
      const spread = Object.entries(r.dist).map(([k, v]) => `${k}:${v}`).join(' ') || 'none'
      const flag = (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets' ? '  ⚑ at-hinge×resets' : ''
      return `  T${r.turn} ${r.figure.padEnd(22)} y=${String(r.year).padStart(5)}  reply:${r.hadReply ? 'yes' : 'no '}  ` +
          `cont=${r.continuity.padEnd(7)} mom=${r.momentum}  chain=${String(r.chainTier).padEnd(8)}(${r.chainFactor.toFixed(2)})  [${spread}]${flag}`
  })
  return [head, ...lines].join('\n')
280}
281 
282const ALL_SCENARIOS = [SCENARIO_A, SCENARIO_B, SCENARIO_BNAT, SCENARIO_C]
283 
284afterAll(printScorecard)
285 
286describe('momentum builds on coherent chains (issue #182)', () => {
  // Fidelity guard (no API): the real server REJECTS (400s) a dispatch over
  // MAX_MESSAGE_CHARS — it does NOT truncate the user message. Grading 'builds'
  // on an unsendable dispatch would be a false positive (the very class of harness
  // infidelity this issue was reopened to avoid). Fail loudly if any fixture drifts
  // past the cap. Runs even without a key, so CI catches it.
  it('every dispatch is sendable (≤ MAX_MESSAGE_CHARS)', () => {
      const over = ALL_SCENARIOS.flatMap(s =>
          s.turns.filter(t => t.userMessage.length > MAX_MESSAGE_CHARS)
              .map(t => `${s.key}:${t.figureName} (${t.userMessage.length})`))
      expect(over, `over ${MAX_MESSAGE_CHARS} chars: ${over.join(', ')}`).toEqual([])
  })
298 
  it('same-figure (A), cross-figure-exploit (B), disconnected (C)', async () => {
      if (!RUN) {
          record({ layer: 'MOM', invariant: 'continuity × momentum', result: 'skipped (needs ANTHROPIC_API_KEY)', status: 'skipped' })
          return
      }
      process.env.EVERWHEN_AI_LANE = 'anthropic'
305 
      const a = await runScenario(SCENARIO_A)
      const b = await runScenario(SCENARIO_B)
      const bn = await runScenario(SCENARIO_BNAT)
      const c = await runScenario(SCENARIO_C)
310 
      delete process.env.EVERWHEN_AI_LANE
312 
      // The full per-turn log is the deliverable — write it straight to stdout.
      process.stdout.write([
          renderScenario(SCENARIO_A, a), renderScenario(SCENARIO_B, b),
          renderScenario(SCENARIO_BNAT, bn), renderScenario(SCENARIO_C, c), ''
      ].join('\n') + '\n')
318 
      const buildsOn = (rows: TurnResult[], turns: number[]) =>
          turns.every(n => rows[n - 1]?.continuity === 'builds')
      const climbs = (rows: TurnResult[]) =>
          rows.length > 1 && rows[rows.length - 1].momentum > rows[0].momentum
323 
      // A: the same-figure thread must build and the meter must climb.
      record({
          layer: 'MOM', invariant: 'A same-figure builds',
          result: `T2/T3 ${SCENARIO_A.buildTurns.map(n => a[n - 1]?.continuity).join('/')} · momentum ${a.map(r => r.momentum).join('→')}`,
          status: buildsOn(a, SCENARIO_A.buildTurns) && climbs(a) ? 'ok' : 'soft-miss'
      })
330 
      // B: the cross-figure exploit-the-ledger chain — the crux. A reset here is
      // the bug (the classifier ignoring the non-figure-specific clauses).
      const bBuilds = buildsOn(b, SCENARIO_B.buildTurns)
      record({
          layer: 'MOM', invariant: 'B cross-figure builds',
          result: `T2/T3 ${SCENARIO_B.buildTurns.map(n => b[n - 1]?.continuity).join('/')} · momentum ${b.map(r => r.momentum).join('→')}${bBuilds ? '' : '  ← BUG: cross-figure chain does not build'}`,
          status: bBuilds && climbs(b) ? 'ok' : 'soft-miss'
      })
339 
      // B′: the same chain phrased naturally (no "on the ledger" signposting) — the
      // demanding bar. Building here means the Judge connects the dispatch to the
      // ledger digest on its own, not just when spoon-fed.
      const bnBuilds = buildsOn(bn, SCENARIO_BNAT.buildTurns)
      record({
          layer: 'MOM', invariant: 'B′ natural-phrasing builds',
          result: `T2/T3 ${SCENARIO_BNAT.buildTurns.map(n => bn[n - 1]?.continuity).join('/')} · momentum ${bn.map(r => r.momentum).join('→')}${bnBuilds ? '' : '  ← cross-figure chain needs explicit signposting'}`,
          status: bnBuilds && climbs(bn) ? 'ok' : 'soft-miss'
      })
349 
      // C: disconnected play must NOT build, and the meter must stay flat at 0.
      const cFlat = c.every(r => r.continuity !== 'builds') && c.every(r => r.momentum === 0)
      record({
          layer: 'MOM', invariant: 'C disconnected stays flat',
          result: `continuity ${c.map(r => r.continuity).join('/')} · momentum ${c.map(r => r.momentum).join('→')}`,
          status: cFlat ? 'ok' : 'soft-miss'
      })
357 
      // Subsystem cross-check: at-hinge/bridged but graded resets.
      const flagged = [...a, ...b, ...bn, ...c].filter(r => (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets')
      record({
          layer: 'MOM', invariant: 'at-hinge × resets flags',
          result: flagged.length ? flagged.map(r => `${r.figure.split(' ').pop()}(${r.chainTier})`).join(', ') : 'none',
          status: 'ok'
      })
  }, 180_000)
366})

The crux: a cross-figure chain that exploits the ledger

Scenario B is the heart of the test. Three different figures — Newton, Bernoulli, Nobel — each dispatch building on the change the previous one landed and escalating toward the objective's anchor year. Newton plants the reaction principle; Bernoulli is told to give that recorded vessel a sustained burn; Nobel is told to fuel that metered jet. Each carries an empty lastFigureReply.

Scenario B — each turn names the prior figure's recorded change (explicit ledger reference).

evals/momentum-continuity.eval.ts · 366 lines

evals/momentum-continuity.eval.ts366 lines · TypeScript

⋯ 112 lines hidden (lines 1–112)1/**
* Momentum × continuity — does the meter build on COHERENT chains, especially
* across DIFFERENT figures? (issue #182)
*
* Momentum (utils/momentum.ts) is gated on the Judge's `continuity` grade: a
* 'builds' climbs the meter, a 'resets' shatters it. The continuity DEFINITION
* (prompt-builder.ts buildJudgePrompt) says a dispatch "builds" if it "meets a
* condition the figure revealed, EXPLOITS A CHANGE ALREADY RECORDED, or ESCALATES
* the run's strategy" — the last two clauses are NOT figure-specific. So a forward
* chain across different figures, each exploiting a landed change, should build
* momentum just as a same-figure thread does.
*
* The open question (and a prior playtest's unconfirmed claim): does it? The
* earlier harness was UNFAITHFUL — it fed the Judge a GLOBAL cross-figure history,
* so on a figure-switch turn the Judge saw the previous, different figure's reply
* mislabeled as the current one's. This eval is FAITHFUL to send-message.post.ts:
*   - lastFigureReply = the last reply IN THIS FIGURE's thread (conversationWith) —
*     EMPTY on a figure switch, exactly as the real game sends it;
*   - ledgerDigest    = the GLOBAL run ledger (last 5 headlines), as the real game.
*
* Three scenarios, each turn logging continuity (modal + spread), the projected
* momentum (folded under a non-loss roll, to isolate the continuity lever from the
* dice), and the causal-chain tier/factor:
*   A — same-figure thread, each turn meeting the condition the figure revealed.
*       Expect builds; momentum 1→2→3.
*   B — cross-figure forward chain, each turn EXPLOITING a change on the ledger and
*       escalating toward the anchor. Per the definition: builds. A RESET here is
*       the bug — the classifier ignoring the "exploits recorded change" clauses.
*   C — disconnected control: vague, unrelated figures/eras that abandon the
*       thread. Expect resets/neutral; momentum flat (momentum isn't always stuck).
*
* Subsystem cross-check (issue §4): a turn the causal-chain rates at-hinge/bridged
* (near a foothold, full power) but continuity rates 'resets' is FLAGGED — those
* are the disagreements to inspect (B = bug; C's vague-but-near-a-year = benign).
*
* Directional, no LLM grader needed — we read the Judge's structured `continuity`
* field straight. Anthropic-lane only (the Judge is Haiku).
*   npx vitest run --config vitest.eval.config.ts evals/momentum-continuity.eval.ts
*/
40import { describe, it, expect, afterAll } from 'vitest'
41import { callJudgeAI } from '~/server/utils/openai'
42import { MAX_MESSAGE_CHARS } from '~/utils/game-config'
43import { nextMomentum } from '~/utils/momentum'
44import { chainStatus, type ChainTier } from '~/utils/causal-chain'
45import { DiceOutcome } from '~/utils/dice'
46import type { Continuity } from '~/utils/continuity'
47import { record, printScorecard, sample } from './harness'
48 
49// Local gate: the Judge is an Anthropic (Haiku) call and we read its structured
50// `continuity` field directly, so the shared RUN gate's OpenAI key (for the dual
51// grader) isn't needed here — just the Anthropic key the Judge already requires.
52const RUN = !!process.env.ANTHROPIC_API_KEY && process.env.EVAL_DRY_RUN !== '1'
53const N = 4
54 
55const OBJECTIVE_ANCHOR = 1900 // "Humanity reaches the Moon by 1900"
56 
57interface LedgerEntry { era: string; headline: string; whenSigned: number }
58interface Turn {
  figureName: string
  figureYear: number
  when: string
  userMessage: string
  /** The figure's reply AFTER this dispatch — becomes lastFigureReply for the
   *  NEXT same-figure turn (mirrors conversationWith(target)'s last AI text). */
  figureReply: string
  /** The change this turn lands on the global ledger (a foothold + a digest line). */
  landed: LedgerEntry
68}
69interface Scenario {
  key: string
  label: string
  /** A ledger present from turn 1 (scenario C's fixed backdrop thread); per-turn
   *  landed entries accrete on top. Empty for A/B — they build their own. */
  seedLedger?: LedgerEntry[]
  turns: Turn[]
  /** 1-indexed turns expected to pick up the thread ('builds'). */
  buildTurns: number[]
78}
79 
80// ── Scenario A — same-figure thread: Tsiolkovsky, each turn meeting the exact
81//    condition his prior reply revealed (fuel → staging). lastFigureReply is
82//    POPULATED throughout (same figure), the dominant continuity signal.
83const SCENARIO_A: Scenario = {
  key: 'A',
  label: 'same-figure thread (condition-meeting)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1885, when: '1885',
          userMessage: "Set down the maths of a vessel that climbs by hurling its own mass — how much it must shed to break Earth's hold and reach the Moon.",
          figureReply: "The equation I can write — but it founders on fuel. No powder yields such speed. Bring me a fuel fiercer than gunpowder and I will chart the climb.",
          landed: { era: 'Russia, 1885', headline: 'Tsiolkovsky derives the equation for a mass-shedding vessel’s climb to orbit', whenSigned: 1885 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1887, when: '1887',
          userMessage: 'The fiercer fuel you asked for is liquid hydrogen burned with liquid oxygen — far past powder. Put that exhaust speed into your equation and chart the climb.',
          figureReply: 'With that fuel the climb computes — yet the vessel cannot lift its own tanks full. The weight defeats the rise unless it sheds them spent, in stages.',
          landed: { era: 'Russia, 1887', headline: 'Tsiolkovsky proves a hydrogen-oxygen burn yields the velocity to leave Earth', whenSigned: 1887 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1889, when: '1889',
          userMessage: 'Then stage it, as you said — stack three vessels, each falling away the instant it burns dry so the next climbs lighter. That staircase reaches the Moon.',
          figureReply: 'Yes — the staged tower is the answer. I will publish it: the road to the Moon is drawn.',
          landed: { era: 'Russia, 1889', headline: 'Tsiolkovsky publishes the staged rocket — a drawn road to the Moon', whenSigned: 1889 }
      }
  ]
107}
108 
109// ── Scenario B — cross-figure forward chain: Newton → Bernoulli → Nobel, each a
110//    DIFFERENT figure, each dispatch EXPLOITING the change the prior turn landed and
111//    escalating toward the 1900 anchor. lastFigureReply is EMPTY on every turn (each
112//    figure's own thread is empty on first contact) — the faithful figure-switch case.
113const SCENARIO_B: Scenario = {
  key: 'B',
  label: 'cross-figure chain (exploits the ledger)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: "Newton's reaction vessel is on the record. Now sustain its burn: derive the thrust of hot gas forced through a narrowed throat, steady and metered.",
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: "Bernoulli's metered jet is on the ledger but starved for power. Tame your nitroglycerin into a measured charge to drive a craft to escape speed.",
          figureReply: 'My explosive, tamed into a measured burn rather than a blast, fed to that throat — it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
137}
⋯ 229 lines hidden (lines 138–366)138 
139// ── Scenario B′ — the SAME cross-figure chain, phrased NATURALLY: each dispatch
140//    builds on the SUBSTANCE of the prior landed change WITHOUT signposting it ("on
141//    the record"/"on the ledger"). Removes the "you led the Judge" objection — does
142//    it connect the message to the ledger DIGEST on its own? lastFigureReply empty.
143const SCENARIO_BNAT: Scenario = {
  key: 'B′',
  label: 'cross-figure chain (natural phrasing)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: 'A craft driven by its own expelled gas needs a sustained burn, not one shove. Derive the thrust of hot gas forced through a narrowed throat.',
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: 'A throttled jet of hot gas can lift a craft but starves for power. Tame your nitroglycerin into a measured burn, fierce enough to climb past the air.',
          figureReply: 'My explosive, tamed into a measured burn rather than a blast — fed to that throat, it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
167}
168 
169// ── Scenario C — disconnected control: a clear spaceflight thread on the ledger,
170//    but the player messages unrelated figures in unrelated eras with vague asks
171//    that abandon it. Expect resets/neutral; momentum stays flat at 0.
172const SCENARIO_C: Scenario = {
  key: 'C',
  label: 'disconnected control (abandons the thread)',
  buildTurns: [],
  seedLedger: [
      { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 },
      { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
  ],
  turns: [
      {
          figureName: 'Cleopatra', figureYear: -40, when: '40 BC',
          userMessage: 'Rule Egypt wisely and be remembered as a great and just queen for all the ages to come.',
          figureReply: 'I shall reign as best I can for my people.',
          landed: { era: 'Egypt, 40 BC', headline: 'Cleopatra resolves to rule justly', whenSigned: -40 }
      },
      {
          figureName: 'Genghis Khan', figureYear: 1220, when: '1220',
          userMessage: 'Conquer broadly but govern with mercy, and leave a lasting peace across your vast empire.',
          figureReply: 'The steppe will know both my sword and my order.',
          landed: { era: 'Mongolia, 1220', headline: 'Genghis Khan tempers conquest with administration', whenSigned: 1220 }
      },
      {
          figureName: 'Marie Antoinette', figureYear: 1780, when: '1780',
          userMessage: 'Show the common people kindness and restraint, and perhaps history will judge you gently.',
          figureReply: 'I will try to heed the people’s wants.',
          landed: { era: 'France, 1780', headline: 'Marie Antoinette counsels restraint at court', whenSigned: 1780 }
      }
  ]
200}
201 
202interface TurnResult {
  turn: number
  figure: string
  year: number
  hadReply: boolean
  continuity: Continuity
  dist: Record<string, number>
  graded: number
  momentum: number
  chainTier: ChainTier | 'n/a'
  chainFactor: number
213}
214 
215function modal(xs: Continuity[]): Continuity {
  const dist: Record<string, number> = {}
  for (const x of xs) dist[x] = (dist[x] ?? 0) + 1
  return (Object.entries(dist).sort((a, b) => b[1] - a[1])[0]?.[0] as Continuity) ?? 'neutral'
219}
220 
221/** Faithfully replays a scenario through the REAL Judge, deriving lastFigureReply
*  (per-figure) and ledgerDigest (global) exactly as send-message.post.ts does. */
223async function runScenario(s: Scenario): Promise<TurnResult[]> {
  const out: TurnResult[] = []
  let momentum = 0
  const priorLanded: LedgerEntry[] = [...(s.seedLedger ?? [])]
  const seenReplies: { figure: string; reply: string }[] = []
228 
  for (let i = 0; i < s.turns.length; i++) {
      const t = s.turns[i]
      // Per-figure history: the last reply in THIS figure's thread — empty on a
      // figure switch (conversationWith(target).reverse().find(ai)).
      const lastFigureReply = [...seenReplies].reverse().find(r => r.figure === t.figureName)?.reply ?? ''
      // Global ledger digest — last 5 headlines, as the server builds it.
      const ledgerDigest = priorLanded.filter(e => e.headline).slice(-5)
          .map(e => `${e.era ? `[${e.era}] ` : ''}${e.headline}`)
      // Footholds = anchor + every dated change already landed (not this turn's own).
      const footholds = [OBJECTIVE_ANCHOR, ...priorLanded.map(e => e.whenSigned)]
      const chain = chainStatus(t.figureYear, footholds)
240 
      const samples = await sample(N, () => callJudgeAI({
          figureName: t.figureName,
          when: t.when,
          userMessage: t.userMessage,
          lastFigureReply,
          ledgerDigest
      }))
      const grades = samples.flatMap(r => (r.success && r.judge ? [r.judge.continuity] : []))
      const continuity = modal(grades.length ? grades : ['neutral'])
      const dist: Record<string, number> = {}
      for (const g of grades) dist[g] = (dist[g] ?? 0) + 1
252 
      // Project momentum under a NON-LOSS roll (Success) so the continuity grade —
      // the lever under test — fully drives the climb; the dice are orthogonal and
      // already unit-tested.
      momentum = nextMomentum(momentum, continuity, DiceOutcome.SUCCESS)
257 
      out.push({
          turn: i + 1, figure: t.figureName, year: t.figureYear, hadReply: !!lastFigureReply,
          continuity, dist, graded: grades.length, momentum,
          chainTier: chain?.tier ?? 'n/a', chainFactor: chain ? chain.factor : 1
      })
263 
      seenReplies.push({ figure: t.figureName, reply: t.figureReply })
      priorLanded.push(t.landed)
  }
  return out
268}
269 
270/** A compact per-turn table for the scenario, written straight to stdout. */
271function renderScenario(s: Scenario, rows: TurnResult[]): string {
  const head = `\n── Scenario ${s.key}: ${s.label} ${'─'.repeat(Math.max(3, 40 - s.label.length))}`
  const lines = rows.map(r => {
      const spread = Object.entries(r.dist).map(([k, v]) => `${k}:${v}`).join(' ') || 'none'
      const flag = (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets' ? '  ⚑ at-hinge×resets' : ''
      return `  T${r.turn} ${r.figure.padEnd(22)} y=${String(r.year).padStart(5)}  reply:${r.hadReply ? 'yes' : 'no '}  ` +
          `cont=${r.continuity.padEnd(7)} mom=${r.momentum}  chain=${String(r.chainTier).padEnd(8)}(${r.chainFactor.toFixed(2)})  [${spread}]${flag}`
  })
  return [head, ...lines].join('\n')
280}
281 
282const ALL_SCENARIOS = [SCENARIO_A, SCENARIO_B, SCENARIO_BNAT, SCENARIO_C]
283 
284afterAll(printScorecard)
285 
286describe('momentum builds on coherent chains (issue #182)', () => {
  // Fidelity guard (no API): the real server REJECTS (400s) a dispatch over
  // MAX_MESSAGE_CHARS — it does NOT truncate the user message. Grading 'builds'
  // on an unsendable dispatch would be a false positive (the very class of harness
  // infidelity this issue was reopened to avoid). Fail loudly if any fixture drifts
  // past the cap. Runs even without a key, so CI catches it.
  it('every dispatch is sendable (≤ MAX_MESSAGE_CHARS)', () => {
      const over = ALL_SCENARIOS.flatMap(s =>
          s.turns.filter(t => t.userMessage.length > MAX_MESSAGE_CHARS)
              .map(t => `${s.key}:${t.figureName} (${t.userMessage.length})`))
      expect(over, `over ${MAX_MESSAGE_CHARS} chars: ${over.join(', ')}`).toEqual([])
  })
298 
  it('same-figure (A), cross-figure-exploit (B), disconnected (C)', async () => {
      if (!RUN) {
          record({ layer: 'MOM', invariant: 'continuity × momentum', result: 'skipped (needs ANTHROPIC_API_KEY)', status: 'skipped' })
          return
      }
      process.env.EVERWHEN_AI_LANE = 'anthropic'
305 
      const a = await runScenario(SCENARIO_A)
      const b = await runScenario(SCENARIO_B)
      const bn = await runScenario(SCENARIO_BNAT)
      const c = await runScenario(SCENARIO_C)
310 
      delete process.env.EVERWHEN_AI_LANE
312 
      // The full per-turn log is the deliverable — write it straight to stdout.
      process.stdout.write([
          renderScenario(SCENARIO_A, a), renderScenario(SCENARIO_B, b),
          renderScenario(SCENARIO_BNAT, bn), renderScenario(SCENARIO_C, c), ''
      ].join('\n') + '\n')
318 
      const buildsOn = (rows: TurnResult[], turns: number[]) =>
          turns.every(n => rows[n - 1]?.continuity === 'builds')
      const climbs = (rows: TurnResult[]) =>
          rows.length > 1 && rows[rows.length - 1].momentum > rows[0].momentum
323 
      // A: the same-figure thread must build and the meter must climb.
      record({
          layer: 'MOM', invariant: 'A same-figure builds',
          result: `T2/T3 ${SCENARIO_A.buildTurns.map(n => a[n - 1]?.continuity).join('/')} · momentum ${a.map(r => r.momentum).join('→')}`,
          status: buildsOn(a, SCENARIO_A.buildTurns) && climbs(a) ? 'ok' : 'soft-miss'
      })
330 
      // B: the cross-figure exploit-the-ledger chain — the crux. A reset here is
      // the bug (the classifier ignoring the non-figure-specific clauses).
      const bBuilds = buildsOn(b, SCENARIO_B.buildTurns)
      record({
          layer: 'MOM', invariant: 'B cross-figure builds',
          result: `T2/T3 ${SCENARIO_B.buildTurns.map(n => b[n - 1]?.continuity).join('/')} · momentum ${b.map(r => r.momentum).join('→')}${bBuilds ? '' : '  ← BUG: cross-figure chain does not build'}`,
          status: bBuilds && climbs(b) ? 'ok' : 'soft-miss'
      })
339 
      // B′: the same chain phrased naturally (no "on the ledger" signposting) — the
      // demanding bar. Building here means the Judge connects the dispatch to the
      // ledger digest on its own, not just when spoon-fed.
      const bnBuilds = buildsOn(bn, SCENARIO_BNAT.buildTurns)
      record({
          layer: 'MOM', invariant: 'B′ natural-phrasing builds',
          result: `T2/T3 ${SCENARIO_BNAT.buildTurns.map(n => bn[n - 1]?.continuity).join('/')} · momentum ${bn.map(r => r.momentum).join('→')}${bnBuilds ? '' : '  ← cross-figure chain needs explicit signposting'}`,
          status: bnBuilds && climbs(bn) ? 'ok' : 'soft-miss'
      })
349 
      // C: disconnected play must NOT build, and the meter must stay flat at 0.
      const cFlat = c.every(r => r.continuity !== 'builds') && c.every(r => r.momentum === 0)
      record({
          layer: 'MOM', invariant: 'C disconnected stays flat',
          result: `continuity ${c.map(r => r.continuity).join('/')} · momentum ${c.map(r => r.momentum).join('→')}`,
          status: cFlat ? 'ok' : 'soft-miss'
      })
357 
      // Subsystem cross-check: at-hinge/bridged but graded resets.
      const flagged = [...a, ...b, ...bn, ...c].filter(r => (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets')
      record({
          layer: 'MOM', invariant: 'at-hinge × resets flags',
          result: flagged.length ? flagged.map(r => `${r.figure.split(' ').pop()}(${r.chainTier})`).join(', ') : 'none',
          status: 'ok'
      })
  }, 180_000)
366})

B's messages say the quiet part out loud ("on the record", "on the ledger"), which invites the objection that the Judge was led. So scenario B′ runs the same chain phrased naturally — same substantive build, no signposting — to check whether the Judge connects the dispatch to the ledger digest on its own.

Scenario B′ — the same chain, naturally phrased, with no "on the ledger" hint.

evals/momentum-continuity.eval.ts · 366 lines

evals/momentum-continuity.eval.ts366 lines · TypeScript

⋯ 142 lines hidden (lines 1–142)1/**
* Momentum × continuity — does the meter build on COHERENT chains, especially
* across DIFFERENT figures? (issue #182)
*
* Momentum (utils/momentum.ts) is gated on the Judge's `continuity` grade: a
* 'builds' climbs the meter, a 'resets' shatters it. The continuity DEFINITION
* (prompt-builder.ts buildJudgePrompt) says a dispatch "builds" if it "meets a
* condition the figure revealed, EXPLOITS A CHANGE ALREADY RECORDED, or ESCALATES
* the run's strategy" — the last two clauses are NOT figure-specific. So a forward
* chain across different figures, each exploiting a landed change, should build
* momentum just as a same-figure thread does.
*
* The open question (and a prior playtest's unconfirmed claim): does it? The
* earlier harness was UNFAITHFUL — it fed the Judge a GLOBAL cross-figure history,
* so on a figure-switch turn the Judge saw the previous, different figure's reply
* mislabeled as the current one's. This eval is FAITHFUL to send-message.post.ts:
*   - lastFigureReply = the last reply IN THIS FIGURE's thread (conversationWith) —
*     EMPTY on a figure switch, exactly as the real game sends it;
*   - ledgerDigest    = the GLOBAL run ledger (last 5 headlines), as the real game.
*
* Three scenarios, each turn logging continuity (modal + spread), the projected
* momentum (folded under a non-loss roll, to isolate the continuity lever from the
* dice), and the causal-chain tier/factor:
*   A — same-figure thread, each turn meeting the condition the figure revealed.
*       Expect builds; momentum 1→2→3.
*   B — cross-figure forward chain, each turn EXPLOITING a change on the ledger and
*       escalating toward the anchor. Per the definition: builds. A RESET here is
*       the bug — the classifier ignoring the "exploits recorded change" clauses.
*   C — disconnected control: vague, unrelated figures/eras that abandon the
*       thread. Expect resets/neutral; momentum flat (momentum isn't always stuck).
*
* Subsystem cross-check (issue §4): a turn the causal-chain rates at-hinge/bridged
* (near a foothold, full power) but continuity rates 'resets' is FLAGGED — those
* are the disagreements to inspect (B = bug; C's vague-but-near-a-year = benign).
*
* Directional, no LLM grader needed — we read the Judge's structured `continuity`
* field straight. Anthropic-lane only (the Judge is Haiku).
*   npx vitest run --config vitest.eval.config.ts evals/momentum-continuity.eval.ts
*/
40import { describe, it, expect, afterAll } from 'vitest'
41import { callJudgeAI } from '~/server/utils/openai'
42import { MAX_MESSAGE_CHARS } from '~/utils/game-config'
43import { nextMomentum } from '~/utils/momentum'
44import { chainStatus, type ChainTier } from '~/utils/causal-chain'
45import { DiceOutcome } from '~/utils/dice'
46import type { Continuity } from '~/utils/continuity'
47import { record, printScorecard, sample } from './harness'
48 
49// Local gate: the Judge is an Anthropic (Haiku) call and we read its structured
50// `continuity` field directly, so the shared RUN gate's OpenAI key (for the dual
51// grader) isn't needed here — just the Anthropic key the Judge already requires.
52const RUN = !!process.env.ANTHROPIC_API_KEY && process.env.EVAL_DRY_RUN !== '1'
53const N = 4
54 
55const OBJECTIVE_ANCHOR = 1900 // "Humanity reaches the Moon by 1900"
56 
57interface LedgerEntry { era: string; headline: string; whenSigned: number }
58interface Turn {
  figureName: string
  figureYear: number
  when: string
  userMessage: string
  /** The figure's reply AFTER this dispatch — becomes lastFigureReply for the
   *  NEXT same-figure turn (mirrors conversationWith(target)'s last AI text). */
  figureReply: string
  /** The change this turn lands on the global ledger (a foothold + a digest line). */
  landed: LedgerEntry
68}
69interface Scenario {
  key: string
  label: string
  /** A ledger present from turn 1 (scenario C's fixed backdrop thread); per-turn
   *  landed entries accrete on top. Empty for A/B — they build their own. */
  seedLedger?: LedgerEntry[]
  turns: Turn[]
  /** 1-indexed turns expected to pick up the thread ('builds'). */
  buildTurns: number[]
78}
79 
80// ── Scenario A — same-figure thread: Tsiolkovsky, each turn meeting the exact
81//    condition his prior reply revealed (fuel → staging). lastFigureReply is
82//    POPULATED throughout (same figure), the dominant continuity signal.
83const SCENARIO_A: Scenario = {
  key: 'A',
  label: 'same-figure thread (condition-meeting)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1885, when: '1885',
          userMessage: "Set down the maths of a vessel that climbs by hurling its own mass — how much it must shed to break Earth's hold and reach the Moon.",
          figureReply: "The equation I can write — but it founders on fuel. No powder yields such speed. Bring me a fuel fiercer than gunpowder and I will chart the climb.",
          landed: { era: 'Russia, 1885', headline: 'Tsiolkovsky derives the equation for a mass-shedding vessel’s climb to orbit', whenSigned: 1885 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1887, when: '1887',
          userMessage: 'The fiercer fuel you asked for is liquid hydrogen burned with liquid oxygen — far past powder. Put that exhaust speed into your equation and chart the climb.',
          figureReply: 'With that fuel the climb computes — yet the vessel cannot lift its own tanks full. The weight defeats the rise unless it sheds them spent, in stages.',
          landed: { era: 'Russia, 1887', headline: 'Tsiolkovsky proves a hydrogen-oxygen burn yields the velocity to leave Earth', whenSigned: 1887 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1889, when: '1889',
          userMessage: 'Then stage it, as you said — stack three vessels, each falling away the instant it burns dry so the next climbs lighter. That staircase reaches the Moon.',
          figureReply: 'Yes — the staged tower is the answer. I will publish it: the road to the Moon is drawn.',
          landed: { era: 'Russia, 1889', headline: 'Tsiolkovsky publishes the staged rocket — a drawn road to the Moon', whenSigned: 1889 }
      }
  ]
107}
108 
109// ── Scenario B — cross-figure forward chain: Newton → Bernoulli → Nobel, each a
110//    DIFFERENT figure, each dispatch EXPLOITING the change the prior turn landed and
111//    escalating toward the 1900 anchor. lastFigureReply is EMPTY on every turn (each
112//    figure's own thread is empty on first contact) — the faithful figure-switch case.
113const SCENARIO_B: Scenario = {
  key: 'B',
  label: 'cross-figure chain (exploits the ledger)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: "Newton's reaction vessel is on the record. Now sustain its burn: derive the thrust of hot gas forced through a narrowed throat, steady and metered.",
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: "Bernoulli's metered jet is on the ledger but starved for power. Tame your nitroglycerin into a measured charge to drive a craft to escape speed.",
          figureReply: 'My explosive, tamed into a measured burn rather than a blast, fed to that throat — it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
137}
138 
139// ── Scenario B′ — the SAME cross-figure chain, phrased NATURALLY: each dispatch
140//    builds on the SUBSTANCE of the prior landed change WITHOUT signposting it ("on
141//    the record"/"on the ledger"). Removes the "you led the Judge" objection — does
142//    it connect the message to the ledger DIGEST on its own? lastFigureReply empty.
143const SCENARIO_BNAT: Scenario = {
  key: 'B′',
  label: 'cross-figure chain (natural phrasing)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: 'A craft driven by its own expelled gas needs a sustained burn, not one shove. Derive the thrust of hot gas forced through a narrowed throat.',
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: 'A throttled jet of hot gas can lift a craft but starves for power. Tame your nitroglycerin into a measured burn, fierce enough to climb past the air.',
          figureReply: 'My explosive, tamed into a measured burn rather than a blast — fed to that throat, it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
167}
⋯ 199 lines hidden (lines 168–366)168 
169// ── Scenario C — disconnected control: a clear spaceflight thread on the ledger,
170//    but the player messages unrelated figures in unrelated eras with vague asks
171//    that abandon it. Expect resets/neutral; momentum stays flat at 0.
172const SCENARIO_C: Scenario = {
  key: 'C',
  label: 'disconnected control (abandons the thread)',
  buildTurns: [],
  seedLedger: [
      { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 },
      { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
  ],
  turns: [
      {
          figureName: 'Cleopatra', figureYear: -40, when: '40 BC',
          userMessage: 'Rule Egypt wisely and be remembered as a great and just queen for all the ages to come.',
          figureReply: 'I shall reign as best I can for my people.',
          landed: { era: 'Egypt, 40 BC', headline: 'Cleopatra resolves to rule justly', whenSigned: -40 }
      },
      {
          figureName: 'Genghis Khan', figureYear: 1220, when: '1220',
          userMessage: 'Conquer broadly but govern with mercy, and leave a lasting peace across your vast empire.',
          figureReply: 'The steppe will know both my sword and my order.',
          landed: { era: 'Mongolia, 1220', headline: 'Genghis Khan tempers conquest with administration', whenSigned: 1220 }
      },
      {
          figureName: 'Marie Antoinette', figureYear: 1780, when: '1780',
          userMessage: 'Show the common people kindness and restraint, and perhaps history will judge you gently.',
          figureReply: 'I will try to heed the people’s wants.',
          landed: { era: 'France, 1780', headline: 'Marie Antoinette counsels restraint at court', whenSigned: 1780 }
      }
  ]
200}
201 
202interface TurnResult {
  turn: number
  figure: string
  year: number
  hadReply: boolean
  continuity: Continuity
  dist: Record<string, number>
  graded: number
  momentum: number
  chainTier: ChainTier | 'n/a'
  chainFactor: number
213}
214 
215function modal(xs: Continuity[]): Continuity {
  const dist: Record<string, number> = {}
  for (const x of xs) dist[x] = (dist[x] ?? 0) + 1
  return (Object.entries(dist).sort((a, b) => b[1] - a[1])[0]?.[0] as Continuity) ?? 'neutral'
219}
220 
221/** Faithfully replays a scenario through the REAL Judge, deriving lastFigureReply
*  (per-figure) and ledgerDigest (global) exactly as send-message.post.ts does. */
223async function runScenario(s: Scenario): Promise<TurnResult[]> {
  const out: TurnResult[] = []
  let momentum = 0
  const priorLanded: LedgerEntry[] = [...(s.seedLedger ?? [])]
  const seenReplies: { figure: string; reply: string }[] = []
228 
  for (let i = 0; i < s.turns.length; i++) {
      const t = s.turns[i]
      // Per-figure history: the last reply in THIS figure's thread — empty on a
      // figure switch (conversationWith(target).reverse().find(ai)).
      const lastFigureReply = [...seenReplies].reverse().find(r => r.figure === t.figureName)?.reply ?? ''
      // Global ledger digest — last 5 headlines, as the server builds it.
      const ledgerDigest = priorLanded.filter(e => e.headline).slice(-5)
          .map(e => `${e.era ? `[${e.era}] ` : ''}${e.headline}`)
      // Footholds = anchor + every dated change already landed (not this turn's own).
      const footholds = [OBJECTIVE_ANCHOR, ...priorLanded.map(e => e.whenSigned)]
      const chain = chainStatus(t.figureYear, footholds)
240 
      const samples = await sample(N, () => callJudgeAI({
          figureName: t.figureName,
          when: t.when,
          userMessage: t.userMessage,
          lastFigureReply,
          ledgerDigest
      }))
      const grades = samples.flatMap(r => (r.success && r.judge ? [r.judge.continuity] : []))
      const continuity = modal(grades.length ? grades : ['neutral'])
      const dist: Record<string, number> = {}
      for (const g of grades) dist[g] = (dist[g] ?? 0) + 1
252 
      // Project momentum under a NON-LOSS roll (Success) so the continuity grade —
      // the lever under test — fully drives the climb; the dice are orthogonal and
      // already unit-tested.
      momentum = nextMomentum(momentum, continuity, DiceOutcome.SUCCESS)
257 
      out.push({
          turn: i + 1, figure: t.figureName, year: t.figureYear, hadReply: !!lastFigureReply,
          continuity, dist, graded: grades.length, momentum,
          chainTier: chain?.tier ?? 'n/a', chainFactor: chain ? chain.factor : 1
      })
263 
      seenReplies.push({ figure: t.figureName, reply: t.figureReply })
      priorLanded.push(t.landed)
  }
  return out
268}
269 
270/** A compact per-turn table for the scenario, written straight to stdout. */
271function renderScenario(s: Scenario, rows: TurnResult[]): string {
  const head = `\n── Scenario ${s.key}: ${s.label} ${'─'.repeat(Math.max(3, 40 - s.label.length))}`
  const lines = rows.map(r => {
      const spread = Object.entries(r.dist).map(([k, v]) => `${k}:${v}`).join(' ') || 'none'
      const flag = (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets' ? '  ⚑ at-hinge×resets' : ''
      return `  T${r.turn} ${r.figure.padEnd(22)} y=${String(r.year).padStart(5)}  reply:${r.hadReply ? 'yes' : 'no '}  ` +
          `cont=${r.continuity.padEnd(7)} mom=${r.momentum}  chain=${String(r.chainTier).padEnd(8)}(${r.chainFactor.toFixed(2)})  [${spread}]${flag}`
  })
  return [head, ...lines].join('\n')
280}
281 
282const ALL_SCENARIOS = [SCENARIO_A, SCENARIO_B, SCENARIO_BNAT, SCENARIO_C]
283 
284afterAll(printScorecard)
285 
286describe('momentum builds on coherent chains (issue #182)', () => {
  // Fidelity guard (no API): the real server REJECTS (400s) a dispatch over
  // MAX_MESSAGE_CHARS — it does NOT truncate the user message. Grading 'builds'
  // on an unsendable dispatch would be a false positive (the very class of harness
  // infidelity this issue was reopened to avoid). Fail loudly if any fixture drifts
  // past the cap. Runs even without a key, so CI catches it.
  it('every dispatch is sendable (≤ MAX_MESSAGE_CHARS)', () => {
      const over = ALL_SCENARIOS.flatMap(s =>
          s.turns.filter(t => t.userMessage.length > MAX_MESSAGE_CHARS)
              .map(t => `${s.key}:${t.figureName} (${t.userMessage.length})`))
      expect(over, `over ${MAX_MESSAGE_CHARS} chars: ${over.join(', ')}`).toEqual([])
  })
298 
  it('same-figure (A), cross-figure-exploit (B), disconnected (C)', async () => {
      if (!RUN) {
          record({ layer: 'MOM', invariant: 'continuity × momentum', result: 'skipped (needs ANTHROPIC_API_KEY)', status: 'skipped' })
          return
      }
      process.env.EVERWHEN_AI_LANE = 'anthropic'
305 
      const a = await runScenario(SCENARIO_A)
      const b = await runScenario(SCENARIO_B)
      const bn = await runScenario(SCENARIO_BNAT)
      const c = await runScenario(SCENARIO_C)
310 
      delete process.env.EVERWHEN_AI_LANE
312 
      // The full per-turn log is the deliverable — write it straight to stdout.
      process.stdout.write([
          renderScenario(SCENARIO_A, a), renderScenario(SCENARIO_B, b),
          renderScenario(SCENARIO_BNAT, bn), renderScenario(SCENARIO_C, c), ''
      ].join('\n') + '\n')
318 
      const buildsOn = (rows: TurnResult[], turns: number[]) =>
          turns.every(n => rows[n - 1]?.continuity === 'builds')
      const climbs = (rows: TurnResult[]) =>
          rows.length > 1 && rows[rows.length - 1].momentum > rows[0].momentum
323 
      // A: the same-figure thread must build and the meter must climb.
      record({
          layer: 'MOM', invariant: 'A same-figure builds',
          result: `T2/T3 ${SCENARIO_A.buildTurns.map(n => a[n - 1]?.continuity).join('/')} · momentum ${a.map(r => r.momentum).join('→')}`,
          status: buildsOn(a, SCENARIO_A.buildTurns) && climbs(a) ? 'ok' : 'soft-miss'
      })
330 
      // B: the cross-figure exploit-the-ledger chain — the crux. A reset here is
      // the bug (the classifier ignoring the non-figure-specific clauses).
      const bBuilds = buildsOn(b, SCENARIO_B.buildTurns)
      record({
          layer: 'MOM', invariant: 'B cross-figure builds',
          result: `T2/T3 ${SCENARIO_B.buildTurns.map(n => b[n - 1]?.continuity).join('/')} · momentum ${b.map(r => r.momentum).join('→')}${bBuilds ? '' : '  ← BUG: cross-figure chain does not build'}`,
          status: bBuilds && climbs(b) ? 'ok' : 'soft-miss'
      })
339 
      // B′: the same chain phrased naturally (no "on the ledger" signposting) — the
      // demanding bar. Building here means the Judge connects the dispatch to the
      // ledger digest on its own, not just when spoon-fed.
      const bnBuilds = buildsOn(bn, SCENARIO_BNAT.buildTurns)
      record({
          layer: 'MOM', invariant: 'B′ natural-phrasing builds',
          result: `T2/T3 ${SCENARIO_BNAT.buildTurns.map(n => bn[n - 1]?.continuity).join('/')} · momentum ${bn.map(r => r.momentum).join('→')}${bnBuilds ? '' : '  ← cross-figure chain needs explicit signposting'}`,
          status: bnBuilds && climbs(bn) ? 'ok' : 'soft-miss'
      })
349 
      // C: disconnected play must NOT build, and the meter must stay flat at 0.
      const cFlat = c.every(r => r.continuity !== 'builds') && c.every(r => r.momentum === 0)
      record({
          layer: 'MOM', invariant: 'C disconnected stays flat',
          result: `continuity ${c.map(r => r.continuity).join('/')} · momentum ${c.map(r => r.momentum).join('→')}`,
          status: cFlat ? 'ok' : 'soft-miss'
      })
357 
      // Subsystem cross-check: at-hinge/bridged but graded resets.
      const flagged = [...a, ...b, ...bn, ...c].filter(r => (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets')
      record({
          layer: 'MOM', invariant: 'at-hinge × resets flags',
          result: flagged.length ? flagged.map(r => `${r.figure.split(' ').pop()}(${r.chainTier})`).join(', ') : 'none',
          status: 'ok'
      })
  }, 180_000)
366})

The result, and the verdict

Run against the real Judge (Haiku, temperature 0), N=4 samples per turn, the grades are unanimous and clear. Momentum is projected under a non-loss roll so the continuity grade — the lever under test — fully drives the climb, isolating it from the orthogonal, already-tested dice.

Scenario	T2	T3	momentum	what it shows
A same-figure thread	builds	builds	0→1→2	the baseline coherent case
B cross-figure, exploits ledger	builds	builds	0→1→2	the crux — empty per-figure reply
B′ cross-figure, natural phrasing	builds	builds	0→1→2	no signposting — Judge connects it itself
C disconnected control	resets	neutral	flat at 0	momentum isn't always stuck

Continuity grade and projected momentum per turn (T1 is neutral by design — no thread yet).

The B / B′ scoring — a reset on the build turns would be flagged as the bug.

evals/momentum-continuity.eval.ts · 366 lines

evals/momentum-continuity.eval.ts366 lines · TypeScript

⋯ 330 lines hidden (lines 1–330)1/**
* Momentum × continuity — does the meter build on COHERENT chains, especially
* across DIFFERENT figures? (issue #182)
*
* Momentum (utils/momentum.ts) is gated on the Judge's `continuity` grade: a
* 'builds' climbs the meter, a 'resets' shatters it. The continuity DEFINITION
* (prompt-builder.ts buildJudgePrompt) says a dispatch "builds" if it "meets a
* condition the figure revealed, EXPLOITS A CHANGE ALREADY RECORDED, or ESCALATES
* the run's strategy" — the last two clauses are NOT figure-specific. So a forward
* chain across different figures, each exploiting a landed change, should build
* momentum just as a same-figure thread does.
*
* The open question (and a prior playtest's unconfirmed claim): does it? The
* earlier harness was UNFAITHFUL — it fed the Judge a GLOBAL cross-figure history,
* so on a figure-switch turn the Judge saw the previous, different figure's reply
* mislabeled as the current one's. This eval is FAITHFUL to send-message.post.ts:
*   - lastFigureReply = the last reply IN THIS FIGURE's thread (conversationWith) —
*     EMPTY on a figure switch, exactly as the real game sends it;
*   - ledgerDigest    = the GLOBAL run ledger (last 5 headlines), as the real game.
*
* Three scenarios, each turn logging continuity (modal + spread), the projected
* momentum (folded under a non-loss roll, to isolate the continuity lever from the
* dice), and the causal-chain tier/factor:
*   A — same-figure thread, each turn meeting the condition the figure revealed.
*       Expect builds; momentum 1→2→3.
*   B — cross-figure forward chain, each turn EXPLOITING a change on the ledger and
*       escalating toward the anchor. Per the definition: builds. A RESET here is
*       the bug — the classifier ignoring the "exploits recorded change" clauses.
*   C — disconnected control: vague, unrelated figures/eras that abandon the
*       thread. Expect resets/neutral; momentum flat (momentum isn't always stuck).
*
* Subsystem cross-check (issue §4): a turn the causal-chain rates at-hinge/bridged
* (near a foothold, full power) but continuity rates 'resets' is FLAGGED — those
* are the disagreements to inspect (B = bug; C's vague-but-near-a-year = benign).
*
* Directional, no LLM grader needed — we read the Judge's structured `continuity`
* field straight. Anthropic-lane only (the Judge is Haiku).
*   npx vitest run --config vitest.eval.config.ts evals/momentum-continuity.eval.ts
*/
40import { describe, it, expect, afterAll } from 'vitest'
41import { callJudgeAI } from '~/server/utils/openai'
42import { MAX_MESSAGE_CHARS } from '~/utils/game-config'
43import { nextMomentum } from '~/utils/momentum'
44import { chainStatus, type ChainTier } from '~/utils/causal-chain'
45import { DiceOutcome } from '~/utils/dice'
46import type { Continuity } from '~/utils/continuity'
47import { record, printScorecard, sample } from './harness'
48 
49// Local gate: the Judge is an Anthropic (Haiku) call and we read its structured
50// `continuity` field directly, so the shared RUN gate's OpenAI key (for the dual
51// grader) isn't needed here — just the Anthropic key the Judge already requires.
52const RUN = !!process.env.ANTHROPIC_API_KEY && process.env.EVAL_DRY_RUN !== '1'
53const N = 4
54 
55const OBJECTIVE_ANCHOR = 1900 // "Humanity reaches the Moon by 1900"
56 
57interface LedgerEntry { era: string; headline: string; whenSigned: number }
58interface Turn {
  figureName: string
  figureYear: number
  when: string
  userMessage: string
  /** The figure's reply AFTER this dispatch — becomes lastFigureReply for the
   *  NEXT same-figure turn (mirrors conversationWith(target)'s last AI text). */
  figureReply: string
  /** The change this turn lands on the global ledger (a foothold + a digest line). */
  landed: LedgerEntry
68}
69interface Scenario {
  key: string
  label: string
  /** A ledger present from turn 1 (scenario C's fixed backdrop thread); per-turn
   *  landed entries accrete on top. Empty for A/B — they build their own. */
  seedLedger?: LedgerEntry[]
  turns: Turn[]
  /** 1-indexed turns expected to pick up the thread ('builds'). */
  buildTurns: number[]
78}
79 
80// ── Scenario A — same-figure thread: Tsiolkovsky, each turn meeting the exact
81//    condition his prior reply revealed (fuel → staging). lastFigureReply is
82//    POPULATED throughout (same figure), the dominant continuity signal.
83const SCENARIO_A: Scenario = {
  key: 'A',
  label: 'same-figure thread (condition-meeting)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1885, when: '1885',
          userMessage: "Set down the maths of a vessel that climbs by hurling its own mass — how much it must shed to break Earth's hold and reach the Moon.",
          figureReply: "The equation I can write — but it founders on fuel. No powder yields such speed. Bring me a fuel fiercer than gunpowder and I will chart the climb.",
          landed: { era: 'Russia, 1885', headline: 'Tsiolkovsky derives the equation for a mass-shedding vessel’s climb to orbit', whenSigned: 1885 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1887, when: '1887',
          userMessage: 'The fiercer fuel you asked for is liquid hydrogen burned with liquid oxygen — far past powder. Put that exhaust speed into your equation and chart the climb.',
          figureReply: 'With that fuel the climb computes — yet the vessel cannot lift its own tanks full. The weight defeats the rise unless it sheds them spent, in stages.',
          landed: { era: 'Russia, 1887', headline: 'Tsiolkovsky proves a hydrogen-oxygen burn yields the velocity to leave Earth', whenSigned: 1887 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1889, when: '1889',
          userMessage: 'Then stage it, as you said — stack three vessels, each falling away the instant it burns dry so the next climbs lighter. That staircase reaches the Moon.',
          figureReply: 'Yes — the staged tower is the answer. I will publish it: the road to the Moon is drawn.',
          landed: { era: 'Russia, 1889', headline: 'Tsiolkovsky publishes the staged rocket — a drawn road to the Moon', whenSigned: 1889 }
      }
  ]
107}
108 
109// ── Scenario B — cross-figure forward chain: Newton → Bernoulli → Nobel, each a
110//    DIFFERENT figure, each dispatch EXPLOITING the change the prior turn landed and
111//    escalating toward the 1900 anchor. lastFigureReply is EMPTY on every turn (each
112//    figure's own thread is empty on first contact) — the faithful figure-switch case.
113const SCENARIO_B: Scenario = {
  key: 'B',
  label: 'cross-figure chain (exploits the ledger)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: "Newton's reaction vessel is on the record. Now sustain its burn: derive the thrust of hot gas forced through a narrowed throat, steady and metered.",
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: "Bernoulli's metered jet is on the ledger but starved for power. Tame your nitroglycerin into a measured charge to drive a craft to escape speed.",
          figureReply: 'My explosive, tamed into a measured burn rather than a blast, fed to that throat — it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
137}
138 
139// ── Scenario B′ — the SAME cross-figure chain, phrased NATURALLY: each dispatch
140//    builds on the SUBSTANCE of the prior landed change WITHOUT signposting it ("on
141//    the record"/"on the ledger"). Removes the "you led the Judge" objection — does
142//    it connect the message to the ledger DIGEST on its own? lastFigureReply empty.
143const SCENARIO_BNAT: Scenario = {
  key: 'B′',
  label: 'cross-figure chain (natural phrasing)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: 'A craft driven by its own expelled gas needs a sustained burn, not one shove. Derive the thrust of hot gas forced through a narrowed throat.',
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: 'A throttled jet of hot gas can lift a craft but starves for power. Tame your nitroglycerin into a measured burn, fierce enough to climb past the air.',
          figureReply: 'My explosive, tamed into a measured burn rather than a blast — fed to that throat, it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
167}
168 
169// ── Scenario C — disconnected control: a clear spaceflight thread on the ledger,
170//    but the player messages unrelated figures in unrelated eras with vague asks
171//    that abandon it. Expect resets/neutral; momentum stays flat at 0.
172const SCENARIO_C: Scenario = {
  key: 'C',
  label: 'disconnected control (abandons the thread)',
  buildTurns: [],
  seedLedger: [
      { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 },
      { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
  ],
  turns: [
      {
          figureName: 'Cleopatra', figureYear: -40, when: '40 BC',
          userMessage: 'Rule Egypt wisely and be remembered as a great and just queen for all the ages to come.',
          figureReply: 'I shall reign as best I can for my people.',
          landed: { era: 'Egypt, 40 BC', headline: 'Cleopatra resolves to rule justly', whenSigned: -40 }
      },
      {
          figureName: 'Genghis Khan', figureYear: 1220, when: '1220',
          userMessage: 'Conquer broadly but govern with mercy, and leave a lasting peace across your vast empire.',
          figureReply: 'The steppe will know both my sword and my order.',
          landed: { era: 'Mongolia, 1220', headline: 'Genghis Khan tempers conquest with administration', whenSigned: 1220 }
      },
      {
          figureName: 'Marie Antoinette', figureYear: 1780, when: '1780',
          userMessage: 'Show the common people kindness and restraint, and perhaps history will judge you gently.',
          figureReply: 'I will try to heed the people’s wants.',
          landed: { era: 'France, 1780', headline: 'Marie Antoinette counsels restraint at court', whenSigned: 1780 }
      }
  ]
200}
201 
202interface TurnResult {
  turn: number
  figure: string
  year: number
  hadReply: boolean
  continuity: Continuity
  dist: Record<string, number>
  graded: number
  momentum: number
  chainTier: ChainTier | 'n/a'
  chainFactor: number
213}
214 
215function modal(xs: Continuity[]): Continuity {
  const dist: Record<string, number> = {}
  for (const x of xs) dist[x] = (dist[x] ?? 0) + 1
  return (Object.entries(dist).sort((a, b) => b[1] - a[1])[0]?.[0] as Continuity) ?? 'neutral'
219}
220 
221/** Faithfully replays a scenario through the REAL Judge, deriving lastFigureReply
*  (per-figure) and ledgerDigest (global) exactly as send-message.post.ts does. */
223async function runScenario(s: Scenario): Promise<TurnResult[]> {
  const out: TurnResult[] = []
  let momentum = 0
  const priorLanded: LedgerEntry[] = [...(s.seedLedger ?? [])]
  const seenReplies: { figure: string; reply: string }[] = []
228 
  for (let i = 0; i < s.turns.length; i++) {
      const t = s.turns[i]
      // Per-figure history: the last reply in THIS figure's thread — empty on a
      // figure switch (conversationWith(target).reverse().find(ai)).
      const lastFigureReply = [...seenReplies].reverse().find(r => r.figure === t.figureName)?.reply ?? ''
      // Global ledger digest — last 5 headlines, as the server builds it.
      const ledgerDigest = priorLanded.filter(e => e.headline).slice(-5)
          .map(e => `${e.era ? `[${e.era}] ` : ''}${e.headline}`)
      // Footholds = anchor + every dated change already landed (not this turn's own).
      const footholds = [OBJECTIVE_ANCHOR, ...priorLanded.map(e => e.whenSigned)]
      const chain = chainStatus(t.figureYear, footholds)
240 
      const samples = await sample(N, () => callJudgeAI({
          figureName: t.figureName,
          when: t.when,
          userMessage: t.userMessage,
          lastFigureReply,
          ledgerDigest
      }))
      const grades = samples.flatMap(r => (r.success && r.judge ? [r.judge.continuity] : []))
      const continuity = modal(grades.length ? grades : ['neutral'])
      const dist: Record<string, number> = {}
      for (const g of grades) dist[g] = (dist[g] ?? 0) + 1
252 
      // Project momentum under a NON-LOSS roll (Success) so the continuity grade —
      // the lever under test — fully drives the climb; the dice are orthogonal and
      // already unit-tested.
      momentum = nextMomentum(momentum, continuity, DiceOutcome.SUCCESS)
257 
      out.push({
          turn: i + 1, figure: t.figureName, year: t.figureYear, hadReply: !!lastFigureReply,
          continuity, dist, graded: grades.length, momentum,
          chainTier: chain?.tier ?? 'n/a', chainFactor: chain ? chain.factor : 1
      })
263 
      seenReplies.push({ figure: t.figureName, reply: t.figureReply })
      priorLanded.push(t.landed)
  }
  return out
268}
269 
270/** A compact per-turn table for the scenario, written straight to stdout. */
271function renderScenario(s: Scenario, rows: TurnResult[]): string {
  const head = `\n── Scenario ${s.key}: ${s.label} ${'─'.repeat(Math.max(3, 40 - s.label.length))}`
  const lines = rows.map(r => {
      const spread = Object.entries(r.dist).map(([k, v]) => `${k}:${v}`).join(' ') || 'none'
      const flag = (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets' ? '  ⚑ at-hinge×resets' : ''
      return `  T${r.turn} ${r.figure.padEnd(22)} y=${String(r.year).padStart(5)}  reply:${r.hadReply ? 'yes' : 'no '}  ` +
          `cont=${r.continuity.padEnd(7)} mom=${r.momentum}  chain=${String(r.chainTier).padEnd(8)}(${r.chainFactor.toFixed(2)})  [${spread}]${flag}`
  })
  return [head, ...lines].join('\n')
280}
281 
282const ALL_SCENARIOS = [SCENARIO_A, SCENARIO_B, SCENARIO_BNAT, SCENARIO_C]
283 
284afterAll(printScorecard)
285 
286describe('momentum builds on coherent chains (issue #182)', () => {
  // Fidelity guard (no API): the real server REJECTS (400s) a dispatch over
  // MAX_MESSAGE_CHARS — it does NOT truncate the user message. Grading 'builds'
  // on an unsendable dispatch would be a false positive (the very class of harness
  // infidelity this issue was reopened to avoid). Fail loudly if any fixture drifts
  // past the cap. Runs even without a key, so CI catches it.
  it('every dispatch is sendable (≤ MAX_MESSAGE_CHARS)', () => {
      const over = ALL_SCENARIOS.flatMap(s =>
          s.turns.filter(t => t.userMessage.length > MAX_MESSAGE_CHARS)
              .map(t => `${s.key}:${t.figureName} (${t.userMessage.length})`))
      expect(over, `over ${MAX_MESSAGE_CHARS} chars: ${over.join(', ')}`).toEqual([])
  })
298 
  it('same-figure (A), cross-figure-exploit (B), disconnected (C)', async () => {
      if (!RUN) {
          record({ layer: 'MOM', invariant: 'continuity × momentum', result: 'skipped (needs ANTHROPIC_API_KEY)', status: 'skipped' })
          return
      }
      process.env.EVERWHEN_AI_LANE = 'anthropic'
305 
      const a = await runScenario(SCENARIO_A)
      const b = await runScenario(SCENARIO_B)
      const bn = await runScenario(SCENARIO_BNAT)
      const c = await runScenario(SCENARIO_C)
310 
      delete process.env.EVERWHEN_AI_LANE
312 
      // The full per-turn log is the deliverable — write it straight to stdout.
      process.stdout.write([
          renderScenario(SCENARIO_A, a), renderScenario(SCENARIO_B, b),
          renderScenario(SCENARIO_BNAT, bn), renderScenario(SCENARIO_C, c), ''
      ].join('\n') + '\n')
318 
      const buildsOn = (rows: TurnResult[], turns: number[]) =>
          turns.every(n => rows[n - 1]?.continuity === 'builds')
      const climbs = (rows: TurnResult[]) =>
          rows.length > 1 && rows[rows.length - 1].momentum > rows[0].momentum
323 
      // A: the same-figure thread must build and the meter must climb.
      record({
          layer: 'MOM', invariant: 'A same-figure builds',
          result: `T2/T3 ${SCENARIO_A.buildTurns.map(n => a[n - 1]?.continuity).join('/')} · momentum ${a.map(r => r.momentum).join('→')}`,
          status: buildsOn(a, SCENARIO_A.buildTurns) && climbs(a) ? 'ok' : 'soft-miss'
      })
330 
      // B: the cross-figure exploit-the-ledger chain — the crux. A reset here is
      // the bug (the classifier ignoring the non-figure-specific clauses).
      const bBuilds = buildsOn(b, SCENARIO_B.buildTurns)
      record({
          layer: 'MOM', invariant: 'B cross-figure builds',
          result: `T2/T3 ${SCENARIO_B.buildTurns.map(n => b[n - 1]?.continuity).join('/')} · momentum ${b.map(r => r.momentum).join('→')}${bBuilds ? '' : '  ← BUG: cross-figure chain does not build'}`,
          status: bBuilds && climbs(b) ? 'ok' : 'soft-miss'
      })
339 
      // B′: the same chain phrased naturally (no "on the ledger" signposting) — the
      // demanding bar. Building here means the Judge connects the dispatch to the
      // ledger digest on its own, not just when spoon-fed.
      const bnBuilds = buildsOn(bn, SCENARIO_BNAT.buildTurns)
      record({
          layer: 'MOM', invariant: 'B′ natural-phrasing builds',
          result: `T2/T3 ${SCENARIO_BNAT.buildTurns.map(n => bn[n - 1]?.continuity).join('/')} · momentum ${bn.map(r => r.momentum).join('→')}${bnBuilds ? '' : '  ← cross-figure chain needs explicit signposting'}`,
          status: bnBuilds && climbs(bn) ? 'ok' : 'soft-miss'
      })
⋯ 18 lines hidden (lines 349–366)349 
      // C: disconnected play must NOT build, and the meter must stay flat at 0.
      const cFlat = c.every(r => r.continuity !== 'builds') && c.every(r => r.momentum === 0)
      record({
          layer: 'MOM', invariant: 'C disconnected stays flat',
          result: `continuity ${c.map(r => r.continuity).join('/')} · momentum ${c.map(r => r.momentum).join('→')}`,
          status: cFlat ? 'ok' : 'soft-miss'
      })
357 
      // Subsystem cross-check: at-hinge/bridged but graded resets.
      const flagged = [...a, ...b, ...bn, ...c].filter(r => (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets')
      record({
          layer: 'MOM', invariant: 'at-hinge × resets flags',
          result: flagged.length ? flagged.map(r => `${r.figure.split(' ').pop()}(${r.chainTier})`).join(', ') : 'none',
          status: 'ok'
      })
  }, 180_000)
366})

The fidelity guard against grading unsendable text

One trap nearly slipped through. The real server rejects (HTTP 400) a player dispatch longer than MAX_MESSAGE_CHARS (160) — it does not truncate it. An independent audit caught that four of the crux build-turn messages had drifted to 171–179 characters: the eval was grading builds on dispatches a player could never send. The messages were rewritten to fit, and this no-API test now fails the whole suite if any fixture drifts past the cap again.

Runs without a key, so CI catches an over-length fixture before it can produce a false 'builds'.

evals/momentum-continuity.eval.ts · 366 lines

evals/momentum-continuity.eval.ts366 lines · TypeScript

⋯ 286 lines hidden (lines 1–286)1/**
* Momentum × continuity — does the meter build on COHERENT chains, especially
* across DIFFERENT figures? (issue #182)
*
* Momentum (utils/momentum.ts) is gated on the Judge's `continuity` grade: a
* 'builds' climbs the meter, a 'resets' shatters it. The continuity DEFINITION
* (prompt-builder.ts buildJudgePrompt) says a dispatch "builds" if it "meets a
* condition the figure revealed, EXPLOITS A CHANGE ALREADY RECORDED, or ESCALATES
* the run's strategy" — the last two clauses are NOT figure-specific. So a forward
* chain across different figures, each exploiting a landed change, should build
* momentum just as a same-figure thread does.
*
* The open question (and a prior playtest's unconfirmed claim): does it? The
* earlier harness was UNFAITHFUL — it fed the Judge a GLOBAL cross-figure history,
* so on a figure-switch turn the Judge saw the previous, different figure's reply
* mislabeled as the current one's. This eval is FAITHFUL to send-message.post.ts:
*   - lastFigureReply = the last reply IN THIS FIGURE's thread (conversationWith) —
*     EMPTY on a figure switch, exactly as the real game sends it;
*   - ledgerDigest    = the GLOBAL run ledger (last 5 headlines), as the real game.
*
* Three scenarios, each turn logging continuity (modal + spread), the projected
* momentum (folded under a non-loss roll, to isolate the continuity lever from the
* dice), and the causal-chain tier/factor:
*   A — same-figure thread, each turn meeting the condition the figure revealed.
*       Expect builds; momentum 1→2→3.
*   B — cross-figure forward chain, each turn EXPLOITING a change on the ledger and
*       escalating toward the anchor. Per the definition: builds. A RESET here is
*       the bug — the classifier ignoring the "exploits recorded change" clauses.
*   C — disconnected control: vague, unrelated figures/eras that abandon the
*       thread. Expect resets/neutral; momentum flat (momentum isn't always stuck).
*
* Subsystem cross-check (issue §4): a turn the causal-chain rates at-hinge/bridged
* (near a foothold, full power) but continuity rates 'resets' is FLAGGED — those
* are the disagreements to inspect (B = bug; C's vague-but-near-a-year = benign).
*
* Directional, no LLM grader needed — we read the Judge's structured `continuity`
* field straight. Anthropic-lane only (the Judge is Haiku).
*   npx vitest run --config vitest.eval.config.ts evals/momentum-continuity.eval.ts
*/
40import { describe, it, expect, afterAll } from 'vitest'
41import { callJudgeAI } from '~/server/utils/openai'
42import { MAX_MESSAGE_CHARS } from '~/utils/game-config'
43import { nextMomentum } from '~/utils/momentum'
44import { chainStatus, type ChainTier } from '~/utils/causal-chain'
45import { DiceOutcome } from '~/utils/dice'
46import type { Continuity } from '~/utils/continuity'
47import { record, printScorecard, sample } from './harness'
48 
49// Local gate: the Judge is an Anthropic (Haiku) call and we read its structured
50// `continuity` field directly, so the shared RUN gate's OpenAI key (for the dual
51// grader) isn't needed here — just the Anthropic key the Judge already requires.
52const RUN = !!process.env.ANTHROPIC_API_KEY && process.env.EVAL_DRY_RUN !== '1'
53const N = 4
54 
55const OBJECTIVE_ANCHOR = 1900 // "Humanity reaches the Moon by 1900"
56 
57interface LedgerEntry { era: string; headline: string; whenSigned: number }
58interface Turn {
  figureName: string
  figureYear: number
  when: string
  userMessage: string
  /** The figure's reply AFTER this dispatch — becomes lastFigureReply for the
   *  NEXT same-figure turn (mirrors conversationWith(target)'s last AI text). */
  figureReply: string
  /** The change this turn lands on the global ledger (a foothold + a digest line). */
  landed: LedgerEntry
68}
69interface Scenario {
  key: string
  label: string
  /** A ledger present from turn 1 (scenario C's fixed backdrop thread); per-turn
   *  landed entries accrete on top. Empty for A/B — they build their own. */
  seedLedger?: LedgerEntry[]
  turns: Turn[]
  /** 1-indexed turns expected to pick up the thread ('builds'). */
  buildTurns: number[]
78}
79 
80// ── Scenario A — same-figure thread: Tsiolkovsky, each turn meeting the exact
81//    condition his prior reply revealed (fuel → staging). lastFigureReply is
82//    POPULATED throughout (same figure), the dominant continuity signal.
83const SCENARIO_A: Scenario = {
  key: 'A',
  label: 'same-figure thread (condition-meeting)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1885, when: '1885',
          userMessage: "Set down the maths of a vessel that climbs by hurling its own mass — how much it must shed to break Earth's hold and reach the Moon.",
          figureReply: "The equation I can write — but it founders on fuel. No powder yields such speed. Bring me a fuel fiercer than gunpowder and I will chart the climb.",
          landed: { era: 'Russia, 1885', headline: 'Tsiolkovsky derives the equation for a mass-shedding vessel’s climb to orbit', whenSigned: 1885 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1887, when: '1887',
          userMessage: 'The fiercer fuel you asked for is liquid hydrogen burned with liquid oxygen — far past powder. Put that exhaust speed into your equation and chart the climb.',
          figureReply: 'With that fuel the climb computes — yet the vessel cannot lift its own tanks full. The weight defeats the rise unless it sheds them spent, in stages.',
          landed: { era: 'Russia, 1887', headline: 'Tsiolkovsky proves a hydrogen-oxygen burn yields the velocity to leave Earth', whenSigned: 1887 }
      },
      {
          figureName: 'Konstantin Tsiolkovsky', figureYear: 1889, when: '1889',
          userMessage: 'Then stage it, as you said — stack three vessels, each falling away the instant it burns dry so the next climbs lighter. That staircase reaches the Moon.',
          figureReply: 'Yes — the staged tower is the answer. I will publish it: the road to the Moon is drawn.',
          landed: { era: 'Russia, 1889', headline: 'Tsiolkovsky publishes the staged rocket — a drawn road to the Moon', whenSigned: 1889 }
      }
  ]
107}
108 
109// ── Scenario B — cross-figure forward chain: Newton → Bernoulli → Nobel, each a
110//    DIFFERENT figure, each dispatch EXPLOITING the change the prior turn landed and
111//    escalating toward the 1900 anchor. lastFigureReply is EMPTY on every turn (each
112//    figure's own thread is empty on first contact) — the faithful figure-switch case.
113const SCENARIO_B: Scenario = {
  key: 'B',
  label: 'cross-figure chain (exploits the ledger)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: "Newton's reaction vessel is on the record. Now sustain its burn: derive the thrust of hot gas forced through a narrowed throat, steady and metered.",
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: "Bernoulli's metered jet is on the ledger but starved for power. Tame your nitroglycerin into a measured charge to drive a craft to escape speed.",
          figureReply: 'My explosive, tamed into a measured burn rather than a blast, fed to that throat — it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
137}
138 
139// ── Scenario B′ — the SAME cross-figure chain, phrased NATURALLY: each dispatch
140//    builds on the SUBSTANCE of the prior landed change WITHOUT signposting it ("on
141//    the record"/"on the ledger"). Removes the "you led the Judge" objection — does
142//    it connect the message to the ledger DIGEST on its own? lastFigureReply empty.
143const SCENARIO_BNAT: Scenario = {
  key: 'B′',
  label: 'cross-figure chain (natural phrasing)',
  buildTurns: [2, 3],
  turns: [
      {
          figureName: 'Isaac Newton', figureYear: 1687, when: '1687',
          userMessage: 'Your third law implies a craft can ride its own thrown mass through the void. Publish it: a vessel reacts against what it expels, needing no air to push on.',
          figureReply: 'A vessel moved by its own ejecta — the mathematics hold even in vacuum. I shall set it among the laws.',
          landed: { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 }
      },
      {
          figureName: 'Daniel Bernoulli', figureYear: 1738, when: '1738',
          userMessage: 'A craft driven by its own expelled gas needs a sustained burn, not one shove. Derive the thrust of hot gas forced through a narrowed throat.',
          figureReply: 'Force the hot gas through a throat and the stream quickens — a controlled jet. I can derive its thrust from the pressure drop.',
          landed: { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
      },
      {
          figureName: 'Alfred Nobel', figureYear: 1867, when: '1867',
          userMessage: 'A throttled jet of hot gas can lift a craft but starves for power. Tame your nitroglycerin into a measured burn, fierce enough to climb past the air.',
          figureReply: 'My explosive, tamed into a measured burn rather than a blast — fed to that throat, it could drive a craft past the sky. A daring use.',
          landed: { era: 'Sweden, 1867', headline: 'Nobel adapts stabilised nitroglycerin into a metered rocket propellant', whenSigned: 1867 }
      }
  ]
167}
168 
169// ── Scenario C — disconnected control: a clear spaceflight thread on the ledger,
170//    but the player messages unrelated figures in unrelated eras with vague asks
171//    that abandon it. Expect resets/neutral; momentum stays flat at 0.
172const SCENARIO_C: Scenario = {
  key: 'C',
  label: 'disconnected control (abandons the thread)',
  buildTurns: [],
  seedLedger: [
      { era: 'England, 1687', headline: 'Newton publishes the reaction principle: a vessel accelerates by expelling mass, even in vacuum', whenSigned: 1687 },
      { era: 'Switzerland, 1738', headline: 'Bernoulli derives sustained, metered thrust from gas forced through a throat', whenSigned: 1738 }
  ],
  turns: [
      {
          figureName: 'Cleopatra', figureYear: -40, when: '40 BC',
          userMessage: 'Rule Egypt wisely and be remembered as a great and just queen for all the ages to come.',
          figureReply: 'I shall reign as best I can for my people.',
          landed: { era: 'Egypt, 40 BC', headline: 'Cleopatra resolves to rule justly', whenSigned: -40 }
      },
      {
          figureName: 'Genghis Khan', figureYear: 1220, when: '1220',
          userMessage: 'Conquer broadly but govern with mercy, and leave a lasting peace across your vast empire.',
          figureReply: 'The steppe will know both my sword and my order.',
          landed: { era: 'Mongolia, 1220', headline: 'Genghis Khan tempers conquest with administration', whenSigned: 1220 }
      },
      {
          figureName: 'Marie Antoinette', figureYear: 1780, when: '1780',
          userMessage: 'Show the common people kindness and restraint, and perhaps history will judge you gently.',
          figureReply: 'I will try to heed the people’s wants.',
          landed: { era: 'France, 1780', headline: 'Marie Antoinette counsels restraint at court', whenSigned: 1780 }
      }
  ]
200}
201 
202interface TurnResult {
  turn: number
  figure: string
  year: number
  hadReply: boolean
  continuity: Continuity
  dist: Record<string, number>
  graded: number
  momentum: number
  chainTier: ChainTier | 'n/a'
  chainFactor: number
213}
214 
215function modal(xs: Continuity[]): Continuity {
  const dist: Record<string, number> = {}
  for (const x of xs) dist[x] = (dist[x] ?? 0) + 1
  return (Object.entries(dist).sort((a, b) => b[1] - a[1])[0]?.[0] as Continuity) ?? 'neutral'
219}
220 
221/** Faithfully replays a scenario through the REAL Judge, deriving lastFigureReply
*  (per-figure) and ledgerDigest (global) exactly as send-message.post.ts does. */
223async function runScenario(s: Scenario): Promise<TurnResult[]> {
  const out: TurnResult[] = []
  let momentum = 0
  const priorLanded: LedgerEntry[] = [...(s.seedLedger ?? [])]
  const seenReplies: { figure: string; reply: string }[] = []
228 
  for (let i = 0; i < s.turns.length; i++) {
      const t = s.turns[i]
      // Per-figure history: the last reply in THIS figure's thread — empty on a
      // figure switch (conversationWith(target).reverse().find(ai)).
      const lastFigureReply = [...seenReplies].reverse().find(r => r.figure === t.figureName)?.reply ?? ''
      // Global ledger digest — last 5 headlines, as the server builds it.
      const ledgerDigest = priorLanded.filter(e => e.headline).slice(-5)
          .map(e => `${e.era ? `[${e.era}] ` : ''}${e.headline}`)
      // Footholds = anchor + every dated change already landed (not this turn's own).
      const footholds = [OBJECTIVE_ANCHOR, ...priorLanded.map(e => e.whenSigned)]
      const chain = chainStatus(t.figureYear, footholds)
240 
      const samples = await sample(N, () => callJudgeAI({
          figureName: t.figureName,
          when: t.when,
          userMessage: t.userMessage,
          lastFigureReply,
          ledgerDigest
      }))
      const grades = samples.flatMap(r => (r.success && r.judge ? [r.judge.continuity] : []))
      const continuity = modal(grades.length ? grades : ['neutral'])
      const dist: Record<string, number> = {}
      for (const g of grades) dist[g] = (dist[g] ?? 0) + 1
252 
      // Project momentum under a NON-LOSS roll (Success) so the continuity grade —
      // the lever under test — fully drives the climb; the dice are orthogonal and
      // already unit-tested.
      momentum = nextMomentum(momentum, continuity, DiceOutcome.SUCCESS)
257 
      out.push({
          turn: i + 1, figure: t.figureName, year: t.figureYear, hadReply: !!lastFigureReply,
          continuity, dist, graded: grades.length, momentum,
          chainTier: chain?.tier ?? 'n/a', chainFactor: chain ? chain.factor : 1
      })
263 
      seenReplies.push({ figure: t.figureName, reply: t.figureReply })
      priorLanded.push(t.landed)
  }
  return out
268}
269 
270/** A compact per-turn table for the scenario, written straight to stdout. */
271function renderScenario(s: Scenario, rows: TurnResult[]): string {
  const head = `\n── Scenario ${s.key}: ${s.label} ${'─'.repeat(Math.max(3, 40 - s.label.length))}`
  const lines = rows.map(r => {
      const spread = Object.entries(r.dist).map(([k, v]) => `${k}:${v}`).join(' ') || 'none'
      const flag = (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets' ? '  ⚑ at-hinge×resets' : ''
      return `  T${r.turn} ${r.figure.padEnd(22)} y=${String(r.year).padStart(5)}  reply:${r.hadReply ? 'yes' : 'no '}  ` +
          `cont=${r.continuity.padEnd(7)} mom=${r.momentum}  chain=${String(r.chainTier).padEnd(8)}(${r.chainFactor.toFixed(2)})  [${spread}]${flag}`
  })
  return [head, ...lines].join('\n')
280}
281 
282const ALL_SCENARIOS = [SCENARIO_A, SCENARIO_B, SCENARIO_BNAT, SCENARIO_C]
283 
284afterAll(printScorecard)
285 
286describe('momentum builds on coherent chains (issue #182)', () => {
  // Fidelity guard (no API): the real server REJECTS (400s) a dispatch over
  // MAX_MESSAGE_CHARS — it does NOT truncate the user message. Grading 'builds'
  // on an unsendable dispatch would be a false positive (the very class of harness
  // infidelity this issue was reopened to avoid). Fail loudly if any fixture drifts
  // past the cap. Runs even without a key, so CI catches it.
  it('every dispatch is sendable (≤ MAX_MESSAGE_CHARS)', () => {
      const over = ALL_SCENARIOS.flatMap(s =>
          s.turns.filter(t => t.userMessage.length > MAX_MESSAGE_CHARS)
              .map(t => `${s.key}:${t.figureName} (${t.userMessage.length})`))
      expect(over, `over ${MAX_MESSAGE_CHARS} chars: ${over.join(', ')}`).toEqual([])
  })
⋯ 69 lines hidden (lines 298–366)298 
  it('same-figure (A), cross-figure-exploit (B), disconnected (C)', async () => {
      if (!RUN) {
          record({ layer: 'MOM', invariant: 'continuity × momentum', result: 'skipped (needs ANTHROPIC_API_KEY)', status: 'skipped' })
          return
      }
      process.env.EVERWHEN_AI_LANE = 'anthropic'
305 
      const a = await runScenario(SCENARIO_A)
      const b = await runScenario(SCENARIO_B)
      const bn = await runScenario(SCENARIO_BNAT)
      const c = await runScenario(SCENARIO_C)
310 
      delete process.env.EVERWHEN_AI_LANE
312 
      // The full per-turn log is the deliverable — write it straight to stdout.
      process.stdout.write([
          renderScenario(SCENARIO_A, a), renderScenario(SCENARIO_B, b),
          renderScenario(SCENARIO_BNAT, bn), renderScenario(SCENARIO_C, c), ''
      ].join('\n') + '\n')
318 
      const buildsOn = (rows: TurnResult[], turns: number[]) =>
          turns.every(n => rows[n - 1]?.continuity === 'builds')
      const climbs = (rows: TurnResult[]) =>
          rows.length > 1 && rows[rows.length - 1].momentum > rows[0].momentum
323 
      // A: the same-figure thread must build and the meter must climb.
      record({
          layer: 'MOM', invariant: 'A same-figure builds',
          result: `T2/T3 ${SCENARIO_A.buildTurns.map(n => a[n - 1]?.continuity).join('/')} · momentum ${a.map(r => r.momentum).join('→')}`,
          status: buildsOn(a, SCENARIO_A.buildTurns) && climbs(a) ? 'ok' : 'soft-miss'
      })
330 
      // B: the cross-figure exploit-the-ledger chain — the crux. A reset here is
      // the bug (the classifier ignoring the non-figure-specific clauses).
      const bBuilds = buildsOn(b, SCENARIO_B.buildTurns)
      record({
          layer: 'MOM', invariant: 'B cross-figure builds',
          result: `T2/T3 ${SCENARIO_B.buildTurns.map(n => b[n - 1]?.continuity).join('/')} · momentum ${b.map(r => r.momentum).join('→')}${bBuilds ? '' : '  ← BUG: cross-figure chain does not build'}`,
          status: bBuilds && climbs(b) ? 'ok' : 'soft-miss'
      })
339 
      // B′: the same chain phrased naturally (no "on the ledger" signposting) — the
      // demanding bar. Building here means the Judge connects the dispatch to the
      // ledger digest on its own, not just when spoon-fed.
      const bnBuilds = buildsOn(bn, SCENARIO_BNAT.buildTurns)
      record({
          layer: 'MOM', invariant: 'B′ natural-phrasing builds',
          result: `T2/T3 ${SCENARIO_BNAT.buildTurns.map(n => bn[n - 1]?.continuity).join('/')} · momentum ${bn.map(r => r.momentum).join('→')}${bnBuilds ? '' : '  ← cross-figure chain needs explicit signposting'}`,
          status: bnBuilds && climbs(bn) ? 'ok' : 'soft-miss'
      })
349 
      // C: disconnected play must NOT build, and the meter must stay flat at 0.
      const cFlat = c.every(r => r.continuity !== 'builds') && c.every(r => r.momentum === 0)
      record({
          layer: 'MOM', invariant: 'C disconnected stays flat',
          result: `continuity ${c.map(r => r.continuity).join('/')} · momentum ${c.map(r => r.momentum).join('→')}`,
          status: cFlat ? 'ok' : 'soft-miss'
      })
357 
      // Subsystem cross-check: at-hinge/bridged but graded resets.
      const flagged = [...a, ...b, ...bn, ...c].filter(r => (r.chainTier === 'at-hinge' || r.chainTier === 'bridged') && r.continuity === 'resets')
      record({
          layer: 'MOM', invariant: 'at-hinge × resets flags',
          result: flagged.length ? flagged.map(r => `${r.figure.split(' ').pop()}(${r.chainTier})`).join(', ') : 'none',
          status: 'ok'
      })
  }, 180_000)
366})