What changed, and why
The Glassbox repo already runs a fleet of Many Hands Engineering loops — read-only agents that map a class of defect and let a human fix what they agree with. This change adds three that push the bar past content-accuracy (which only guards whether the prose is true):
- lab-fidelity — is every lab actually driven by the lesson's engine, or does it fake the output it presents as computed? - collection-coherence — does each lesson hit the shared "complete lesson" bar, and stay consistent with its siblings? - visual-sanity — does the page render without layout breakage, measured across both themes at desktop and mobile?
The interesting part is that three new loops did not mean three new copies of the fan-out machinery. A shared scaffold, per-lesson.ts, is extracted first, and the existing content-accuracy loop is migrated onto it — so the per-lesson family is deduped, not triplicated. Then all three loops were run over every lesson; the three real gaps they surfaced are fixed in the second commit and shown at the end.
Blast radius: the loops live under agents/ (their own package, never in the app's dependency graph). The only app-source touched is the four-file content fix and a staleness fix to one audit script. Everything is read-only by construction.
lab-fidelity — the engine is the oracle
content-accuracy guards the prose; lab-fidelity guards the interactive half. Its outside reference is unusually concrete: engine/index.js, the pure, unit-tested logic that is the source of truth for what every lab should compute. A lab whose displayed values or step order disagree with the engine — or that bypasses the engine to hardcode a "result" — is a finding.
Because the scaffold carries the orchestration, the loop file is almost entirely its two prompts. The registration is this small:
The entire loop wiring: prompts + model + budget, handed to the scaffold.
agents/lab-fidelity.ts · 115 lines
agents/lab-fidelity.ts
⋯ 100 lines hidden (lines 1–100)
1/**
2 * Lab-fidelity loop — Many Hands Engineering, the "are the interactions real" pass.
3 *
4 * `content-accuracy` guards whether the PROSE is true. This is its sibling for the
5 * INTERACTIVE half: is every lab actually driven by the lesson's pure engine, or
6 * does it hardcode / fake the output it presents as computed? Does every control
7 * the reader can touch actually do something? And does the lab demonstrate the
8 * CLAIM the surrounding prose makes about it? The labs are half the product and,
9 * until now, 0% automated-reviewed — rot hides here.
10 *
11 * Like content-accuracy it is PER-LESSON, fanned out in parallel (one opus agent
12 * per lesson, max effort), and READ-ONLY — it maps, the human fixes. Its outside
13 * reference is unusually concrete: `engine/index.js`, the pure, unit-tested logic
14 * that is the SOURCE OF TRUTH for what every lab is supposed to compute. A lab
15 * whose displayed values or step order disagree with the engine — or that bypasses
16 * the engine to hardcode a "result" — is a finding.
17 *
18 * Permissions: Read / Grep / Glob ONLY — structurally cannot edit a file.
19 *
20 * Usage:
21 * tsx lab-fidelity.ts # all lessons
22 * tsx lab-fidelity.ts torrents swim # only these lesson ids
23 * tsx lab-fidelity.ts --concurrency 2 # cap parallel agents (default 3)
24 * tsx lab-fidelity.ts --budget 5 # per-lesson USD cap (default 5)
25 * tsx lab-fidelity.ts --dry-run # show the plan, spend nothing
26 */
27import { CATALOG, LESSONS_DIR, type Lesson, relList, runPerLessonLoop } from "./per-lesson.js";
28import { report } from "./lib.js";
30function userPrompt(lesson: Lesson, files: string[]): string {
31 return `Lesson under review: \`${lesson.id}\` — "${lesson.title}".
32Claimed topic / source (its eyebrow): ${lesson.eyebrow || "(none stated)"}.
34All of this lesson's content lives under \`${LESSONS_DIR}/${lesson.id}/\`. Its ${files.length} files, relative to that directory — read what you need to judge the labs:
35${relList(lesson.id, files)}
37Key references for THIS loop:
38- \`${LESSONS_DIR}/${lesson.id}/labs/*.jsx\` (and any interactive widget in \`components/\`): the interactions to audit.
39- \`${LESSONS_DIR}/${lesson.id}/engine/index.js\`: the SOURCE OF TRUTH for what those labs are supposed to compute / animate.
40- \`${LESSONS_DIR}/${lesson.id}/sections/*.jsx\`: the prose CLAIMS each lab is meant to demonstrate.
41- The lesson's framing in \`${CATALOG}\` (the promise it makes the learner).
42- The engine's test (REPO-ROOT \`tests/\`, name may not match the id — Glob \`tests/*.js\` and grep for an import from \`lessons/${lesson.id}/engine\`).
44Audit the labs per your instructions and emit the three-bucket map.`;
45}
47function systemPrompt(lesson: Lesson): string {
48 return `You are the lab-fidelity verifier for ONE lesson of the Glassbox repository — a React 19 + Vite collection of self-contained, interactive lessons. You are reviewing \`${lesson.id}\` ("${lesson.title}").
50Your single job: determine whether this lesson's INTERACTIVE elements are HONEST — that what the reader can poke is real, engine-driven, and demonstrates the claim its prose makes. You are NOT reviewing factual accuracy of the prose (content-accuracy owns that), nor visual styling, a11y, or performance (other loops own those). Stay on the integrity of the interactions.
52Your only tools are Read / Grep / Glob — you can investigate but CANNOT edit a file. The human fixes what they agree with.
54THE OUTSIDE REFERENCE IS THE ENGINE. \`engine/index.js\` is pure, unit-tested logic and is the source of truth for what every lab/animation should compute. The intended pattern in this repo: the engine computes (a result, or an ordered list of animation frames/steps each with its caption + highlighted state), and the lab imports those functions and renders/plays their output. A lab that bypasses the engine to hardcode the numbers, steps, or end-state it presents AS IF computed is faking the interaction.
56WHAT TO CHECK, per lab / interactive widget:
571. ENGINE-DRIVEN vs FAKED. Does the lab actually import and call the engine for the values / frames it shows? Or does it hardcode an array of "results", a precomputed step sequence, or a fixed outcome that the prose presents as the live computation? Distinguish: (a) genuine engine calls — good; (b) small honest illustrative constants (a fixed example INPUT, a seed, a label) — fine; (c) a hardcoded OUTPUT standing in for computation the engine exists to do — a defect. Grep the lab for the engine import and trace whether the displayed value flows from an engine call.
582. LIVE CONTROLS. Every interactive control (button, slider, toggle, segmented control, draggable) should drive a real state change with a visible effect. Flag a control that is wired to a no-op, never changes state, or whose handler is empty/dead. (A genuinely static diagram is fine — judge controls that LOOK interactive.)
593. CLAIM ↔ DEMONSTRATION. The lab should demonstrate what the surrounding prose says it demonstrates. If §3 says "watch the filter reject a member with no false negative" but the lab can't actually exhibit that, or shows the opposite, that's a fidelity defect (the interaction teaches something other than its caption).
604. ENGINE ↔ LAB AGREEMENT. Where the lab renders engine output, confirm it renders it faithfully — the same order, the same values, the same end state the engine produces. An animation that plays the engine's frames in the wrong order, or drops/relabels them, is a defect even if the engine is correct.
62Classify EACH issue into EXACTLY ONE of three buckets:
64(1) FIDELITY DEFECT — fix. Include ONLY when you can answer all THREE in one sentence each:
65 (a) WHAT — quote the EXACT lab line/code you observed (the hardcoded output, the dead handler, the mismatched render), in backticks, with file:line. Cite or omit — no quote, no finding.
66 (b) WHY it's a defect — name what the engine actually provides (or that no engine path exists) and the false impression the reader forms: that they're driving a computation that is in fact canned, a control that does nothing, or a demonstration that contradicts its caption.
67 (c) ACTION — the concrete fix (route the value through engine function X; wire the control to real state; correct the frame order; align the lab with its prose claim).
69(2) VERIFIED LIVE — the interactions you checked and confirmed honest. NOT padding: list the lesson's CENTRAL labs and, for each, the ANCHOR proving it's engine-driven — the engine function it calls and where its output is rendered (e.g. \`labs/HandshakeLab.jsx:40 plays steps from engine \`handshakeFrames()\` (engine/index.js:120), rendered in order\`). A "verified" line with no anchor is inadmissible.
71(3) JUDGMENT (your call). An interaction that simplifies honestly (a fixed illustrative scenario, a capped parameter range, a deterministic seed instead of true randomness) — note the trade-off briefly; it is NOT a defect. Only surface ones worth the steward's attention.
73KNOWN-CONTEXT AWARENESS:
74- Hardcoded INPUTS, seeds, example datasets, and labels are fine and expected. The defect is a hardcoded OUTPUT that the engine is supposed to compute and that the lab presents as computed.
75- Some animations legitimately use a timer to step through ENGINE-EMITTED frames; the timer is not the claim — the frames are. Verify the frames come from the engine; do not flag the playback loop itself.
76- A lab may deliberately use Math.random for live variation (per content-accuracy's convention, randomness stays in the lab, not the engine); that is honest interactivity, not a fake.
77- An empty bucket (1) with a populated bucket (2) is the expected, good result. Do not invent defects to fill it.
79Hard rules:
80- Cite or omit. A finding with no quoted line from your own Read is a confabulation; suppress it.
81- Reserve bucket (1) for interactions that are actually canned/dead/mismatched — push borderline honesty calls to bucket (3).
82- If the same defect recurs across several labs, report the PATTERN once with one action.
84Output a structured map with exactly these three sections in this order:
86## Fidelity defect — fix (review & act)
88(per item — file:line · WHAT (quoted) · WHY it's a defect (what the engine provides + the false impression) · ACTION)
90## Verified live
92(the central labs you confirmed engine-driven — lab · the engine function + where rendered)
94## Judgment (your call)
96(per item — what is simplified · honest or a defect · the trade-off)
98End with a final summary line: "<X> defects · <Y> verified · <Z> judgment". Nothing after.`;
99}
101runPerLessonLoop({
102 name: "Lab-fidelity",
103 blurb: "are the labs engine-driven & claim-honest (opus · effort:max), READ-ONLY",
104 systemPrompt,
105 userPrompt,
106 model: "opus",
107 effort: "max",
108 maxTurns: 400,
109 defaultBudget: 5,
110 resultLine: (scope) =>
111 `RESULT: PASS — maps above for ${scope}. Review each "Fidelity defect" section and fix the canned/dead/mismatched interactions you agree with; treat "Judgment" as honest-simplification calls.`,
The system prompt does the real work — it defines the three buckets (fidelity defect / verified live / judgment) and the cite-or-omit discipline that keeps the agent honest: a finding with no quoted line from its own Read is a confabulation and must be suppressed.
The engine-vs-faked rule the agent grades against.
agents/lab-fidelity.ts · 115 lines
agents/lab-fidelity.ts
⋯ 53 lines hidden (lines 1–53)
1/**
2 * Lab-fidelity loop — Many Hands Engineering, the "are the interactions real" pass.
3 *
4 * `content-accuracy` guards whether the PROSE is true. This is its sibling for the
5 * INTERACTIVE half: is every lab actually driven by the lesson's pure engine, or
6 * does it hardcode / fake the output it presents as computed? Does every control
7 * the reader can touch actually do something? And does the lab demonstrate the
8 * CLAIM the surrounding prose makes about it? The labs are half the product and,
9 * until now, 0% automated-reviewed — rot hides here.
10 *
11 * Like content-accuracy it is PER-LESSON, fanned out in parallel (one opus agent
12 * per lesson, max effort), and READ-ONLY — it maps, the human fixes. Its outside
13 * reference is unusually concrete: `engine/index.js`, the pure, unit-tested logic
14 * that is the SOURCE OF TRUTH for what every lab is supposed to compute. A lab
15 * whose displayed values or step order disagree with the engine — or that bypasses
16 * the engine to hardcode a "result" — is a finding.
17 *
18 * Permissions: Read / Grep / Glob ONLY — structurally cannot edit a file.
19 *
20 * Usage:
21 * tsx lab-fidelity.ts # all lessons
22 * tsx lab-fidelity.ts torrents swim # only these lesson ids
23 * tsx lab-fidelity.ts --concurrency 2 # cap parallel agents (default 3)
24 * tsx lab-fidelity.ts --budget 5 # per-lesson USD cap (default 5)
25 * tsx lab-fidelity.ts --dry-run # show the plan, spend nothing
26 */
27import { CATALOG, LESSONS_DIR, type Lesson, relList, runPerLessonLoop } from "./per-lesson.js";
28import { report } from "./lib.js";
30function userPrompt(lesson: Lesson, files: string[]): string {
31 return `Lesson under review: \`${lesson.id}\` — "${lesson.title}".
32Claimed topic / source (its eyebrow): ${lesson.eyebrow || "(none stated)"}.
34All of this lesson's content lives under \`${LESSONS_DIR}/${lesson.id}/\`. Its ${files.length} files, relative to that directory — read what you need to judge the labs:
35${relList(lesson.id, files)}
37Key references for THIS loop:
38- \`${LESSONS_DIR}/${lesson.id}/labs/*.jsx\` (and any interactive widget in \`components/\`): the interactions to audit.
39- \`${LESSONS_DIR}/${lesson.id}/engine/index.js\`: the SOURCE OF TRUTH for what those labs are supposed to compute / animate.
40- \`${LESSONS_DIR}/${lesson.id}/sections/*.jsx\`: the prose CLAIMS each lab is meant to demonstrate.
41- The lesson's framing in \`${CATALOG}\` (the promise it makes the learner).
42- The engine's test (REPO-ROOT \`tests/\`, name may not match the id — Glob \`tests/*.js\` and grep for an import from \`lessons/${lesson.id}/engine\`).
44Audit the labs per your instructions and emit the three-bucket map.`;
45}
47function systemPrompt(lesson: Lesson): string {
48 return `You are the lab-fidelity verifier for ONE lesson of the Glassbox repository — a React 19 + Vite collection of self-contained, interactive lessons. You are reviewing \`${lesson.id}\` ("${lesson.title}").
50Your single job: determine whether this lesson's INTERACTIVE elements are HONEST — that what the reader can poke is real, engine-driven, and demonstrates the claim its prose makes. You are NOT reviewing factual accuracy of the prose (content-accuracy owns that), nor visual styling, a11y, or performance (other loops own those). Stay on the integrity of the interactions.
52Your only tools are Read / Grep / Glob — you can investigate but CANNOT edit a file. The human fixes what they agree with.
54THE OUTSIDE REFERENCE IS THE ENGINE. \`engine/index.js\` is pure, unit-tested logic and is the source of truth for what every lab/animation should compute. The intended pattern in this repo: the engine computes (a result, or an ordered list of animation frames/steps each with its caption + highlighted state), and the lab imports those functions and renders/plays their output. A lab that bypasses the engine to hardcode the numbers, steps, or end-state it presents AS IF computed is faking the interaction.
56WHAT TO CHECK, per lab / interactive widget:
571. ENGINE-DRIVEN vs FAKED. Does the lab actually import and call the engine for the values / frames it shows? Or does it hardcode an array of "results", a precomputed step sequence, or a fixed outcome that the prose presents as the live computation? Distinguish: (a) genuine engine calls — good; (b) small honest illustrative constants (a fixed example INPUT, a seed, a label) — fine; (c) a hardcoded OUTPUT standing in for computation the engine exists to do — a defect. Grep the lab for the engine import and trace whether the displayed value flows from an engine call.
582. LIVE CONTROLS. Every interactive control (button, slider, toggle, segmented control, draggable) should drive a real state change with a visible effect. Flag a control that is wired to a no-op, never changes state, or whose handler is empty/dead. (A genuinely static diagram is fine — judge controls that LOOK interactive.)
593. CLAIM ↔ DEMONSTRATION. The lab should demonstrate what the surrounding prose says it demonstrates. If §3 says "watch the filter reject a member with no false negative" but the lab can't actually exhibit that, or shows the opposite, that's a fidelity defect (the interaction teaches something other than its caption).
604. ENGINE ↔ LAB AGREEMENT. Where the lab renders engine output, confirm it renders it faithfully — the same order, the same values, the same end state the engine produces. An animation that plays the engine's frames in the wrong order, or drops/relabels them, is a defect even if the engine is correct.
⋯ 54 lines hidden (lines 62–115)
62Classify EACH issue into EXACTLY ONE of three buckets:
64(1) FIDELITY DEFECT — fix. Include ONLY when you can answer all THREE in one sentence each:
65 (a) WHAT — quote the EXACT lab line/code you observed (the hardcoded output, the dead handler, the mismatched render), in backticks, with file:line. Cite or omit — no quote, no finding.
66 (b) WHY it's a defect — name what the engine actually provides (or that no engine path exists) and the false impression the reader forms: that they're driving a computation that is in fact canned, a control that does nothing, or a demonstration that contradicts its caption.
67 (c) ACTION — the concrete fix (route the value through engine function X; wire the control to real state; correct the frame order; align the lab with its prose claim).
69(2) VERIFIED LIVE — the interactions you checked and confirmed honest. NOT padding: list the lesson's CENTRAL labs and, for each, the ANCHOR proving it's engine-driven — the engine function it calls and where its output is rendered (e.g. \`labs/HandshakeLab.jsx:40 plays steps from engine \`handshakeFrames()\` (engine/index.js:120), rendered in order\`). A "verified" line with no anchor is inadmissible.
71(3) JUDGMENT (your call). An interaction that simplifies honestly (a fixed illustrative scenario, a capped parameter range, a deterministic seed instead of true randomness) — note the trade-off briefly; it is NOT a defect. Only surface ones worth the steward's attention.
73KNOWN-CONTEXT AWARENESS:
74- Hardcoded INPUTS, seeds, example datasets, and labels are fine and expected. The defect is a hardcoded OUTPUT that the engine is supposed to compute and that the lab presents as computed.
75- Some animations legitimately use a timer to step through ENGINE-EMITTED frames; the timer is not the claim — the frames are. Verify the frames come from the engine; do not flag the playback loop itself.
76- A lab may deliberately use Math.random for live variation (per content-accuracy's convention, randomness stays in the lab, not the engine); that is honest interactivity, not a fake.
77- An empty bucket (1) with a populated bucket (2) is the expected, good result. Do not invent defects to fill it.
79Hard rules:
80- Cite or omit. A finding with no quoted line from your own Read is a confabulation; suppress it.
81- Reserve bucket (1) for interactions that are actually canned/dead/mismatched — push borderline honesty calls to bucket (3).
82- If the same defect recurs across several labs, report the PATTERN once with one action.
84Output a structured map with exactly these three sections in this order:
86## Fidelity defect — fix (review & act)
88(per item — file:line · WHAT (quoted) · WHY it's a defect (what the engine provides + the false impression) · ACTION)
90## Verified live
92(the central labs you confirmed engine-driven — lab · the engine function + where rendered)
94## Judgment (your call)
96(per item — what is simplified · honest or a defect · the trade-off)
98End with a final summary line: "<X> defects · <Y> verified · <Z> judgment". Nothing after.`;
99}
101runPerLessonLoop({
102 name: "Lab-fidelity",
103 blurb: "are the labs engine-driven & claim-honest (opus · effort:max), READ-ONLY",
104 systemPrompt,
105 userPrompt,
106 model: "opus",
107 effort: "max",
108 maxTurns: 400,
109 defaultBudget: 5,
110 resultLine: (scope) =>
111 `RESULT: PASS — maps above for ${scope}. Review each "Fidelity defect" section and fix the canned/dead/mismatched interactions you agree with; treat "Judgment" as honest-simplification calls.`,
112}).catch((err: unknown) => {
113 report([`RESULT: ERROR — ${err instanceof Error ? err.message : String(err)}`]);
114 process.exitCode = 1;
115});
collection-coherence — each lesson against the whole
Every other content loop judges one lesson against the world. This one judges each lesson against the collection: does it hit the shared "anatomy of a complete lesson" bar, and is it consistent with its siblings? The rubric is checked into the repo as COLLECTION.md and mirrored in the loop's system prompt, so the document a human reads and the bar the agent grades against are the same five beats.
The pedagogy bar — the five beats every lesson must hit.
COLLECTION.md · 185 lines
COLLECTION.md
⋯ 21 lines hidden (lines 1–21)
1# The Collection — coherence rubric & scorecard (PROPOSAL)
3> **Status: draft for discussion. No lesson or code changes were made to produce
4> this.** It exists to make one decision possible: _what does "the same level"
5> mean_, and which differences between lessons are **intentional identity** vs
6> **uneven quality**. Everything below is a starting point to argue with, not a
7> verdict.
9The 23 lessons were authored independently and vary ~3–5× in prose length, ~12×
10in engine size, and ~20× in test depth. Some of that is the point (a survey
11_should_ be longer than an essay); some is just whichever bar the author hit that
12day. You can't level a collection you can't see side-by-side, so this doc is the
13mirror: a shared bar, then every lesson measured against it.
15---
17## 1. The bar — anatomy of a complete Glassbox lesson
19A lesson is **complete** when it has all of these. (The first group is the
20_pedagogy_; the second is the _craft_, now largely enforced by the MHE loops.)
22**Pedagogy — the part that varies today**
241. **Signature hero** — eyebrow that is a credit / year / promise (never
25 "Interactive lesson"), a display title, a one-line subtitle, a lede that
26 states the stakes. _(AGENTS.md already mandates this.)_
272. **A motivating opening** — a concrete "what breaks without this / why should I
28 care" _before_ any mechanism. The reader should want the answer before they
29 get it.
303. **A legible arc** — problem → core idea → mechanism (stepped) → limits &
31 trade-offs → synthesis. Not a rigid template; a recognizable _shape_. The
32 collection is meant to be unique per lesson, **not** structureless.
334. **Real interactive labs** — each lab makes _one_ claim physical and is driven
34 by the lesson's engine (not a static diagram). Count is allowed to vary by
35 topic; "is it actually interactive and load-bearing" is not.
365. **A closing** — a coda that names _the one idea to carry away_ and points
37 where to go next. **This is the single most common gap today (≈9 lessons).**
⋯ 148 lines hidden (lines 38–185)
39**Craft — now mostly uniform (verified by the loops)**
416. **Pure engine + tests** — `engine/index.js`, ≥90% covered. (Uniform; _depth_
42 still varies — see the scorecard.)
437. **Family glue** — JetBrains Mono for numerics/credits/eyebrows, both light &
44 dark themes, reduced-motion gating, AA contrast, scoped CSS, keyboard a11y.
45 _(All green as of the latest loop sweep.)_
468. **A sane length band** — target ≈ 8–16 reading-minutes, with **declared**
47 exceptions (a survey or an essay can opt out, on purpose, in writing).
49---
51## 2. Intentional uniqueness vs quality gap — how to tell them apart
53This is the whole game. The same metric can be either, depending on intent:
55| Looks like a difference | **Intentional identity** (keep) | **Quality gap** (close) |
56| ------------------------------------- | -------------------------------------------------- | ------------------------------------------------- |
57| Per-lesson accent / font / world | always — it's the point | — |
58| Long (concurrency 34 min, udp 48 min) | a _survey_ or _history_, declared as such | a lesson that sprawled because no one edited it |
59| Short (memory 6 min) | a tight _essay_, declared as such | a stub that never got its mechanism or labs |
60| Few labs (paxos/saga/trie = 4) | topic has fewer natural interactions | a topic that _begs_ for interaction and got prose |
61| Thin engine (cap 42, udp 53 LOC) | genuinely little pure logic to extract | rich logic left un-extracted / untestable |
62| No coda | **never intentional** — every lesson earns a close | the gap |
64**Rule of thumb:** a difference is _intentional_ only if someone can say, in one
65sentence, _why_ — and that sentence belongs in the lesson's own notes. If the
66only answer is "that's just how it came out," it's a gap.
68---
70## 3. Scorecard (deterministic columns measured; judgment columns are the audit's job)
72Clusters and tiers below are a **first-pass proposal** to ratify. `words` is
73section-prose only (a rough proxy; the audit refines it); `read` ≈ words ÷ 200;
74`coda?` is by section filename (so it _undercounts_ — some lessons close inline).
76| Lesson | Cluster | Tier | secs | labs | ~words | read | engLOC | tests | coda? |
77| ----------------------- | -------------------- | ------------ | ---: | ---: | -----: | ---: | -----: | ----: | :---: |
78| binary-trees | data-structures | Foundational | 10 | 9 | 1789 | 9 | 394 | 56 | ✓ |
79| b-trees | data-structures | Intermediate | 10 | 8 | 1913 | 10 | 302 | 32 | ✓ |
80| trie | data-structures | Intermediate | 10 | 4 | 2278 | 11 | 113 | 12 | ✓ |
81| vp-tree | data-structures | Intermediate | 8 | 5 | 1992 | 10 | 394 | 39 | — |
82| bloom-filters | probabilistic | Foundational | 14 | 6 | 2537 | 13 | 80 | 14 | — |
83| cuckoo-filter | probabilistic | Intermediate | 9 | 6 | 2408 | 12 | 159 | 18 | — |
84| hyperloglog | probabilistic | Intermediate | 9 | 7 | 1857 | 9 | 121 | 28 | ✓ |
85| bloom-clock | probabilistic | Advanced | 11 | 7 | 3950 | 20 | 88 | 9 | — |
86| memory | storage | Foundational | 7 | 5 | 1275 | 6 | 43 | 9 | ✓ |
87| lsm-trees | storage | Intermediate | 10 | 8 | 2498 | 12 | 186 | 25 | ✓ |
88| sstables | storage | Advanced | 6 | 5 | 1995 | 10 | 394 | 45 | — |
89| sha | hashing-integrity | Foundational | 10 | 6 | 3013 | 15 | 229 | 14 | ✓ |
90| merkle-trees | hashing-integrity | Intermediate | 12 | 7 | 2856 | 14 | 93 | 10 | ✓ |
91| concurrency-foundations | concurrency-txn | Foundational | 25 | 1 | 6885 | 34 | 67 | 8 | — |
92| acid-lab | concurrency-txn | Intermediate | 5 | 2 | 1818 | 9 | 321 | 10 | ✓ |
93| saga | concurrency-txn | Advanced | 10 | 4 | 2571 | 13 | 379 | 31 | ✓ |
94| cap-pacelc | consensus-membership | Advanced | 11 | 6 | 3871 | 19 | 42 | 6 | — |
95| swim | consensus-membership | Advanced | 10 | 6 | 2530 | 13 | 189 | 12 | — |
96| paxos | consensus-membership | Advanced | 9 | 4 | 1614 | 8 | 252 | 23 | ✓ |
97| udp | networking-rpc | Foundational | 11 | \* | 9623 | 48 | 53 | 3 | — |
98| tls | networking-rpc | Intermediate | 9 | 7 | 1753 | 9 | 228 | 37 | ✓ |
99| grpc | networking-rpc | Intermediate | 7 | 6 | 2212 | 11 | 163 | 32 | ✓ |
100| torrents | networking-rpc | Advanced | 11 | 9 | 2002 | 10 | 495 | 59 | ✓ |
102`*` udp keeps its labs in `components/`, not `labs/` — itself a small structural
103inconsistency worth normalizing.
105---
107## 4. What the data already says (before any reading)
109- **Closings are the cheapest, highest-signal fix.** ≈9 lessons have no named
110 closing section (bloom-filters, cuckoo, bloom-clock, sstables, cap-pacelc,
111 swim, vp-tree, concurrency, udp). A consistent "the one idea + where next"
112 coda is a small, uniform win that immediately raises the floor.
113- **Engine depth ↔ test depth ↔ topic richness are mismatched in places.**
114 cap-pacelc (42 LOC / 6 tests) and udp (53 / 3) and concurrency (67 / 8) and
115 bloom-clock (88 / 9) are thin where the topic is rich — candidates for "is the
116 interactive core under-built, or genuinely simple?" swim is rich in prose but
117 light in tests (189 / 12).
118- **Two declared outliers, not gaps:** concurrency (survey, 34 min) and udp
119 (history/console, 48 min). These should _stay_ long — but say so in writing.
120- **Structure varies but isn't chaos.** Most lessons are 9–12 sections / 5–7
121 labs. The outliers (acid 5/2, sstables 6/5, concurrency 25/1) are each
122 arguably intentional — the audit confirms.
124---
126## 5. The progression question — defer it, let the data decide
128Don't commit to a tree yet; you don't have the dependency data. The scorecard's
129**tier** + **cluster** + a future **prerequisites** column produce exactly that
130data. Two cheap, reversible steps toward an answer, in order:
1321. **Add `tier` + `tags` to `lesson-catalog.js`** (pure metadata, no UI). Lets
133 the index _group_ and _filter_ — surfaces a suggested reading order without
134 committing to a tree. ~30 min, fully reversible.
1352. **Only if clean prerequisite chains emerge** (e.g. merkle → torrents,
136 acid → saga, concurrency → cap → paxos), graduate to a real progression
137 graph. If instead it's 7 loosely-coupled clusters (likely), **tags + a
138 "start here" path beats a tree** and costs far less.
140My read: this is a _clustered collection_, not a strict curriculum, so tags +
141one or two suggested paths will feel better than a dependency tree. But that's a
142hypothesis the prerequisites data should confirm.
144---
146## 6. Proposed path — smallest steps first
1481. **Ratify this bar** (§1) and the intentional-vs-gap test (§2). One
149 conversation. Nothing ships until the bar is agreed.
1502. **Normalize the one universal gap: closings.** Give every lesson a coda in
151 its own voice (the one idea + where next). Uniform shape, per-lesson texture —
152 the exact "coherent but separate" target, at the lowest risk.
1533. **Stand up a `collection-coherence` MHE loop** (sketch in §7) to fill the
154 judgment columns — arc completeness, lab load-bearingness, motivation, and an
155 _intentional-vs-gap_ verdict per lesson, cite-or-omit. Read-only; it maps, you
156 fix.
1574. **Close gaps lesson-by-lesson**, worst-first off the loop's map, each as its
158 own small reviewable change.
1595. **Add `tier` + `tags` metadata**; revisit tree-vs-paths with real prerequisite
160 data.
162---
164## 7. Sketch — a `collection-coherence` MHE loop
166Same shape as the existing read-only loops (`content-accuracy`, `theme-parity`):
167deterministic signal → one locked-down `opus` agent per lesson → strict
168cite-or-omit map. Per lesson it would emit:
170- the §1 checklist, each item **present / thin / missing** with a quoted cite;
171- the arc shape it actually follows, and where it breaks;
172- each lab rated **load-bearing / decorative / static-that-should-be-live**;
173- a proposed **tier**, **cluster**, and **prerequisites**;
174- and the key call: each shortfall bucketed **intentional identity** (keep, with
175 the one-sentence why) vs **quality gap** (close, with the smallest fix).
177Output is one collection-wide table — this scorecard, with the judgment columns
178filled — that turns "they feel uneven" into a worst-first punch-list.
180---
182_Appendix — method: metrics gathered read-only from the source tree (section /
183lab file counts, section-prose word counts, `engine/index.js` LOC, `it()`/`test()`
184counts in each engine suite, closing-section filename detection). Clusters &
185tiers are the author's first-pass proposal. No lesson or code was modified._
The same five beats, encoded in the agent's system prompt.
agents/collection-coherence.ts · 116 lines
agents/collection-coherence.ts
⋯ 51 lines hidden (lines 1–51)
1/**
2 * Collection-coherence loop — Many Hands Engineering, the "is it a coherent whole" pass.
3 *
4 * Every other content loop judges ONE lesson against the world (its prose vs the
5 * canon, its labs vs the engine). This one judges each lesson against THE
6 * COLLECTION: does it hit the shared "anatomy of a complete lesson" bar, and is
7 * it consistent with its siblings (terminology, cross-references)? The reference
8 * is the rubric below plus the catalog — not an external source. So, like
9 * content-accuracy, it can only MAP, bucketing each shortfall as a real GAP to
10 * close vs an INTENTIONAL divergence the collection means to keep (a survey's
11 * length, an essay's brevity). The collection is deliberately unique per lesson;
12 * the job is to separate identity from inconsistency.
13 *
14 * Per-lesson, fanned out (opus, max effort), READ-ONLY. Each agent grades its own
15 * lesson against the rubric AND greps its siblings for shared terms / cross-refs,
16 * so cross-lesson drift surfaces from whichever side first notices it.
17 *
18 * Permissions: Read / Grep / Glob ONLY — structurally cannot edit a file.
19 *
20 * Usage:
21 * tsx collection-coherence.ts # all lessons
22 * tsx collection-coherence.ts swim tls # only these lesson ids
23 * tsx collection-coherence.ts --concurrency 2 # cap parallel agents (default 3)
24 * tsx collection-coherence.ts --dry-run # show the plan, spend nothing
25 */
26import { CATALOG, LESSONS_DIR, type Lesson, relList, runPerLessonLoop } from "./per-lesson.js";
27import { report } from "./lib.js";
29function userPrompt(lesson: Lesson, files: string[]): string {
30 return `Lesson under review: \`${lesson.id}\` — "${lesson.title}".
31Its signature line (eyebrow): ${lesson.eyebrow || "(none stated)"}.
33All of this lesson's content lives under \`${LESSONS_DIR}/${lesson.id}/\`. Its ${files.length} files, relative to that directory:
34${relList(lesson.id, files)}
36For THIS loop:
37- Read \`sections/*.jsx\` end to end — they carry the hero, the motivating opening, the arc, and the closing/coda.
38- Skim \`labs/*.jsx\` for COUNT and whether each is load-bearing (not for engine-honesty — lab-fidelity owns that).
39- Read this lesson's framing in \`${CATALOG}\` (eyebrow/title/subtitle/pitch) — the promise it makes.
40- CROSS-LESSON: for the shared concepts this lesson leans on (e.g. "quorum", "happens-before", "compaction", "false positive", "tombstone"), Grep \`${LESSONS_DIR}/*/\` to see how SIBLING lessons name and define them, and check any explicit cross-reference this lesson makes ("see the … lesson", a "where to go next" pointer) names a real, correct neighbour.
42Grade the lesson against the rubric in your instructions and emit the three-bucket map.`;
43}
45function systemPrompt(lesson: Lesson): string {
46 return `You are the collection-coherence mapper for ONE lesson of the Glassbox repository — a React 19 + Vite collection of self-contained, intentionally-unique lessons. You are grading \`${lesson.id}\` ("${lesson.title}").
48Your job: judge whether this lesson meets the COLLECTION's shared bar and is CONSISTENT with its siblings. You are NOT judging factual correctness (content-accuracy owns that), interaction honesty (lab-fidelity owns that), or visual/a11y/perf (other loops own those). Stay on structural completeness + cross-lesson consistency.
50Your only tools are Read / Grep / Glob — you can investigate but CANNOT edit a file. The human acts on what they agree with.
52THE RUBRIC — anatomy of a complete Glassbox lesson (the bar to grade against):
53 1. SIGNATURE HERO — an eyebrow that is a credit / year / promise (never a generic "interactive lesson"/"masterclass"), a display title, a subtitle, and a lede that states the stakes.
54 2. A MOTIVATING OPENING — a concrete "what breaks without this / why care" BEFORE the mechanism. The reader should want the answer before getting it.
55 3. A LEGIBLE ARC — problem → core idea → mechanism (stepped) → limits & trade-offs → synthesis. Not a rigid template; a recognizable SHAPE. (Unique per lesson, not structureless.)
56 4. REAL INTERACTIVE LABS — enough load-bearing interactions that the topic is felt, not just read. (Count varies by topic; you judge "is the interactive surface adequate for this subject", not engine-honesty.)
57 5. A CLOSING — a distinct coda that names THE ONE IDEA to carry away and points WHERE TO GO NEXT. Not buried in the last content chapter; not just a decorative footer.
⋯ 58 lines hidden (lines 59–116)
59CROSS-LESSON CONSISTENCY (the collection, not the lesson):
60 - TERMINOLOGY: a shared concept should be named/defined compatibly across lessons. If this lesson defines a term in a way that contradicts how a sibling defines it (grep to check), that's a coherence gap — name both sites.
61 - CROSS-REFERENCES: any "see the X lesson" / "where to go next" pointer must name a real lesson/topic and describe it correctly. A dangling or wrong pointer is a gap.
63Classify EACH observation into EXACTLY ONE of three buckets:
65(1) GAP — close. A rubric beat MISSING or materially weak, or a cross-lesson inconsistency. Include ONLY when you can answer all THREE in one sentence each:
66 (a) WHAT — quote the exact evidence from your Read (the missing/weak beat with file:line, or the two conflicting definitions with both file:lines, or the dangling cross-ref). Cite or omit.
67 (b) WHY it's a gap — which rubric beat fails, or which sibling it conflicts with, and the cost to the reader/collection.
68 (c) ACTION — the concrete fix (add a motivating opening to §1; give §N a real coda naming the one idea + where-next; reconcile the term with lesson Y; correct the pointer).
70(2) MEETS THE BAR — verified. For each rubric beat the lesson HITS, give an ANCHOR — the file:line / quoted line that satisfies it (e.g. \`hero eyebrow "BAYER & McCREIGHT · 1970" — sections/Hero.jsx:12\`, \`coda names the one idea + 4 where-next items — sections/Coda.jsx:30\`). A "meets" line with no anchor is inadmissible. This shows your work.
72(3) INTENTIONAL DIVERGENCE (your call). A deliberate difference the collection MEANS to keep — a survey's extra length, an essay's brevity, fewer labs for a quiet topic, a bespoke section grammar. State the ONE-SENTENCE reason it's identity, not a gap. (The test: a difference is intentional only if you can say WHY in a sentence; if the only reason is "that's how it came out", it's a gap, bucket 1.)
74KNOWN-CONTEXT AWARENESS:
75- The collection is intentionally unique per lesson (per-lesson accent/font/voice/world). Variety of FORM is the point; do not flag a lesson for closing with "Onward" instead of "Coda", or for a distinct section grammar, as long as the BEAT is present.
76- A closing can live in sections/ OR components/, be named Coda/Closing/Onward/Synthesis, or be a final section — what matters is the beat (names the one idea + points onward), not the filename.
77- Length and lab-count legitimately vary by topic; only flag when a lesson is a stub (a beat genuinely missing) or sprawls without reason — and when it's deliberate, bucket (3) with the why.
78- An empty bucket (1) with a populated bucket (2) is the expected, good result. Do not invent gaps.
80Hard rules:
81- Cite or omit. A gap with no quoted evidence from your own Read is a confabulation; suppress it.
82- "Different" is not "deficient." Push deliberate, justifiable differences to bucket (3); reserve bucket (1) for a beat that is actually absent/weak or a real inconsistency.
83- If the same gap pattern recurs, report it once with one action.
85Output a structured map with exactly these three sections in this order:
87## Gap — close (review & act)
89(per item — which beat / which inconsistency · WHAT (quoted, file:line) · WHY · ACTION)
91## Meets the bar — verified
93(per rubric beat hit — beat · the anchor that satisfies it)
95## Intentional divergence (your call)
97(per item — the difference · the one-sentence reason it's identity not a gap)
99End with a final summary line: "<X> gaps · <Y> met · <Z> intentional". Nothing after.`;
100}
102runPerLessonLoop({
103 name: "Collection-coherence",
104 blurb: "each lesson vs the shared rubric + sibling consistency (opus · effort:max), READ-ONLY",
105 systemPrompt,
106 userPrompt,
107 model: "opus",
108 effort: "max",
109 maxTurns: 400,
110 defaultBudget: 5,
111 resultLine: (scope) =>
112 `RESULT: PASS — maps above for ${scope}. Close the "Gap" items you agree with; treat "Intentional divergence" as the collection's deliberate uniqueness to keep.`,
113}).catch((err: unknown) => {
114 report([`RESULT: ERROR — ${err instanceof Error ? err.message : String(err)}`]);
115 process.exitCode = 1;
116});
visual-sanity — measure, don't look
Layout integrity can only be judged on the rendered page, so this loop is self-contained: it builds the app, boots vite preview, runs a deterministic layout sweep, tears the server down, and hands the agent the per-render defect signals to classify.
Build → boot preview → measure → tear down, in a finally block.
agents/visual-sanity.ts · 217 lines
agents/visual-sanity.ts
⋯ 67 lines hidden (lines 1–67)
1/**
2 * Visual-sanity loop — Many Hands Engineering, rendered layout integrity.
3 *
4 * The render-defect gate. 23 lessons × 2 themes × 2 viewports = ~92 renders no
5 * human will eyeball. Like `contrast`, this can only be judged on the RENDERED
6 * page, so the harness is self-contained: it builds the app, boots `vite
7 * preview`, runs the layout measurement (scripts/render-audit.js — horizontal
8 * overflow + the elements causing it, broken images, an empty/failed main, and
9 * reveal-on-scroll blocks left stuck hidden), tears the server down, and hands
10 * the agent the per-(lesson,theme,viewport) defect signals. The agent Reads each
11 * cited element in the lesson's source and classifies it **Fix / Intentional
12 * decorative / Judgment** — so a deliberate full-bleed isn't "fixed" and a real
13 * sideways-scroll is.
14 *
15 * SCOPE NOTE — this measures DETERMINISTIC layout breakage (the reliable, no-vision
16 * half of "does it look right"). The aesthetic-judgment half ("does this render
17 * feel off / contradict the lesson's world") needs the model to actually SEE the
18 * screenshots; the Agent SDK's Read-on-image path is currently unreliable
19 * (anthropics/claude-code#35866), so true vision is deferred to a Files-API/MCP
20 * variant. This loop catches the breakage that matters most and is 100% reliable.
21 *
22 * Permissions: Read / Grep / Glob ONLY — structurally cannot edit a file.
23 *
24 * Usage: tsx visual-sanity.ts [lesson-ids…] (default: ALL lessons + the index)
25 */
26import { spawn, execFileSync } from "node:child_process";
27import { existsSync, readFileSync, rmSync } from "node:fs";
28import { get } from "node:http";
29import { APP_ROOT, report, runLoop } from "./lib.js";
31const ALLOWED_TOOLS = ["Read", "Grep", "Glob"];
32const PORT = 5192;
33const BASE = `http://127.0.0.1:${PORT}`;
34const AUDIT_OUT = "/tmp/glassbox-render-loop.json";
36interface PageDefects {
37 lesson: string;
38 theme: string;
39 viewport: string;
40 pageOverflowPx: number;
41 offenders: { sel: string; overPx: number }[];
42 brokenImages: string[];
43 stuckHidden: string[];
44 mainEmpty: boolean;
45}
47function ids(): string[] {
48 return process.argv.slice(2).filter((a) => !a.startsWith("--"));
49}
51function waitForServer(url: string, timeoutMs: number): Promise<boolean> {
52 const start = Date.now();
53 return new Promise((resolveP) => {
54 const tick = () => {
55 const req = get(url, (res) => {
56 res.resume();
57 resolveP(true);
58 });
59 req.on("error", () => {
60 if (Date.now() - start > timeoutMs) resolveP(false);
61 else setTimeout(tick, 300);
62 });
63 };
64 tick();
65 });
66}
68/** Build, boot preview, run the layout sweep, tear down. Returns the per-render report. */
69async function runAudit(): Promise<PageDefects[]> {
70 console.log("── Building app (vite build) ──");
71 execFileSync("npx", ["vite", "build"], { cwd: APP_ROOT, stdio: "inherit" });
73 console.log(`── Booting vite preview on :${PORT} ──`);
74 const server = spawn("npx", ["vite", "preview", "--port", String(PORT), "--host", "127.0.0.1"], {
75 cwd: APP_ROOT,
76 stdio: "ignore",
77 });
78 try {
79 const up = await waitForServer(BASE, 30_000);
80 if (!up) throw new Error(`preview server did not come up on ${BASE} within 30s`);
82 console.log("── Measuring layout (every lesson × light/dark × desktop/mobile) ──");
83 if (existsSync(AUDIT_OUT)) rmSync(AUDIT_OUT);
84 execFileSync("node", ["scripts/render-audit.js", BASE, AUDIT_OUT], { cwd: APP_ROOT, stdio: "inherit" });
85 return JSON.parse(readFileSync(AUDIT_OUT, "utf8")) as PageDefects[];
86 } finally {
87 server.kill("SIGTERM");
88 }
89}
⋯ 128 lines hidden (lines 90–217)
91/** Renders that carry any defect signal, scoped, worst-overflow first. */
92function withDefects(reportData: PageDefects[], scopeIds: string[]): PageDefects[] {
93 const want = new Set(scopeIds);
94 return reportData
95 .filter((r) => !want.size || want.has(r.lesson))
96 .filter(
97 (r) =>
98 r.pageOverflowPx > 8 || r.brokenImages.length > 0 || r.stuckHidden.length > 0 || r.mainEmpty,
99 )
100 .sort((a, b) => b.pageOverflowPx - a.pageOverflowPx);
101}
103function formatForAgent(defects: PageDefects[], scopeIds: string[]): string {
104 const scopeNote = scopeIds.length ? `lessons: ${scopeIds.join(", ")}` : "ALL lessons + the index";
105 if (defects.length === 0) {
106 return `Layout sweep over ${scopeNote} (both themes, desktop + mobile, reveal-scrolled): NO render-defect signals — no horizontal overflow, no broken images, no empty mains, no stuck-hidden reveals.
108Clean scan. Confirm by sampling 2-3 lesson CSS files for any fixed/full-bleed atmosphere that COULD overflow, then output the default clean map. A bucket (1) finding requires a quoted source rule from your Read.`;
109 }
110 const lines = defects
111 .map((r) => {
112 const parts: string[] = [];
113 if (r.pageOverflowPx > 8) {
114 const off = r.offenders.slice(0, 4).map((o) => `\`${o.sel}\` (+${o.overPx}px)`).join(", ");
115 parts.push(`horizontal overflow ${r.pageOverflowPx}px — offenders: ${off || "(none isolated)"}`);
116 }
117 if (r.mainEmpty) parts.push("MAIN EMPTY / failed render");
118 if (r.brokenImages.length) parts.push(`broken images: ${r.brokenImages.join("; ")}`);
119 if (r.stuckHidden.length)
120 parts.push(`reveal stuck hidden (candidates): ${r.stuckHidden.map((s) => `\`${s}\``).join(", ")}`);
121 return ` - ${r.lesson} · ${r.theme} · ${r.viewport}: ${parts.join(" · ")}`;
122 })
123 .join("\n");
124 return `Layout sweep over ${scopeNote} (both themes, desktop + mobile; pages were reveal-scrolled so reveal-on-scroll content is at its final state). ${defects.length} (lesson,theme,viewport) renders carry a defect SIGNAL — measured by the harness, NOT yet judged:
126${lines}
128For EACH, find the cited element in the lesson's source (grep the class in its <slug>.css / JSX / SVG) and decide what it IS — a real layout break vs a deliberate effect. Classify into ONE of the three buckets.`;
129}
131function systemPrompt(scopeNote: string): string {
132 return `You are the visual-sanity mapper for the Glassbox repository (React 19 + Vite — a DUAL-THEME collection; each lesson ships a bespoke palette + a [data-theme] complement, and many use fixed/full-bleed ambient atmosphere). The harness rendered ${scopeNote} in BOTH themes at desktop AND mobile widths, scrolled each page fully (so reveal-on-scroll content is at its final state), and MEASURED layout defects: horizontal overflow (+ the elements whose box crosses the viewport edge), broken images, an empty/failed main region, and reveal blocks still at ~0 opacity after scrolling. Your job: turn that signal list into a curated map of which are real render DEFECTS vs intentional design.
134Your only tools are Read / Grep / Glob — you can investigate but CANNOT edit a file. The human (or a redesign workflow) fixes what you map.
136WHY THIS IS A MAP, NOT A PASS/FAIL: these lessons deliberately use full-bleed and fixed-position ATMOSPHERE — a 100vw grain layer, a position:fixed gradient ::before, an SVG starfield, an edge-to-edge frieze. Such an element can legitimately extend to (or just past) the viewport edge by design and is NOT a defect. But CONTENT a reader must use — prose, a lab, a control, a table, a diagram — must not be cut off, must not force the page to scroll sideways, must not vanish, and the main region must actually render. The mobile viewport (390px) is where real responsive breaks surface; weight it.
138For EACH defect signal, classify into EXACTLY ONE of three buckets:
140(1) FIX — a real layout defect. Include ONLY when you can answer in one sentence each:
141 (a) WHAT — the (lesson · theme · viewport) and the offending element, with the EXACT source rule/element you found (the CSS class + property, the inline style, the SVG) quoted with file:line from your Read, plus what it IS (content vs atmosphere).
142 (b) WHY it's a defect — it pushes READABLE content off-screen / forces a sideways scroll / hides content the reader needs / leaves the page blank — name the broken experience, especially on mobile.
143 (c) ACTION — the concrete fix (constrain the element's width / add overflow handling / wrap or stack it at the narrow breakpoint / fix the failed image path / fix the reveal so it fires).
145(2) INTENTIONAL — verified, keep. State the SPECIFIC reason the overflow/hidden/effect is by design: a fixed/absolute ambient layer (grain, gradient, starfield), a deliberately full-bleed frieze or rule, a decorative element that bleeds by intent, or a reveal that is intentionally a pre-state. Quote the rule (e.g. \`position: fixed; inset: 0\` or \`width: 100vw\`) that shows it's atmosphere, not content.
147(3) JUDGMENT (your call). A borderline overflow (a few px from a shadow/blur/box that's cosmetically fine), a tight-but-usable mobile fit, or a stuck-hidden CANDIDATE that may simply be a legitimately-hidden element (a closed sheet, an inactive tab) rather than a failed reveal. Name the trade-off — do not auto-fix.
149KNOWN-CONTEXT AWARENESS:
150- A "stuck hidden" entry is only a candidate. Many opacity:0 elements are intentional (a pre-reveal state the IntersectionObserver hasn't reached on a tall page, a closed mobile nav sheet, an inactive panel). Read the element + its reveal wiring before calling it a failed reveal; if you can't confirm it should be visible, it's bucket (3).
151- A small page overflow (≤ ~16px) often comes from a decorative shadow/blur or a scrollbar quirk, not content — verify the offender is content before bucket (1).
152- The dark theme is the loved reference; a full-bleed atmosphere that looks intentional in dark is intentional in light too unless the LIGHT complement specifically breaks it.
154WHEN THE SWEEP IS CLEAN: sample 2-3 files, then output the default clean map. A bucket (1) finding requires a quoted source rule from your Read — do not confabulate.
156Output a structured map with exactly these three sections in this order:
158## Fix — render defect (review & act)
160(per item — lesson · theme · viewport · WHAT (quoted source) · WHY · ACTION)
162## Intentional — keep (verified)
164(list — lesson · the effect · the rule that shows it's by design)
166## Judgment (your call)
168(list — lesson · the signal · the trade-off)
170End with a final summary line: "<X> fix · <Y> intentional · <Z> judgment". Nothing after.`;
171}
173async function main(): Promise<void> {
174 const scopeIds = ids();
175 const scopeNote = scopeIds.length ? `lessons: ${scopeIds.join(", ")}` : "ALL lessons + the index";
176 console.log(`Visual-sanity loop — scope: ${scopeNote}\n`);
178 let reportData: PageDefects[];
179 try {
180 reportData = await runAudit();
181 } catch (err) {
182 report([`RESULT: CANNOT RUN — ${err instanceof Error ? err.message : String(err)}`]);
183 process.exitCode = 1;
184 return;
185 }
187 const defects = withDefects(reportData, scopeIds);
188 console.log(`\n── Renders with a defect signal: ${defects.length} / ${reportData.length} measured ──\n`);
190 const { agentRun, agentSummary } = await runLoop({
191 systemPrompt: systemPrompt(scopeNote),
192 allowedTools: ALLOWED_TOOLS,
193 prompt: formatForAgent(defects, scopeIds),
194 // opus: separating intentional full-bleed atmosphere from a real layout break
195 // is the crux judgment; the strong model resists "flag every overflow".
196 model: "opus",
197 maxTurns: 300,
198 maxBudgetUsd: 5,
199 });
201 report([
202 `Scope: ${scopeNote}`,
203 `Renders measured: ${reportData.length} (lesson × theme × viewport)`,
204 `With a defect signal: ${defects.length}`,
205 `Agent run: ${agentRun}`,
206 "(read-only — the working tree was not modified)",
207 "",
208 agentSummary || "(no map produced — see streamed agent output above)",
209 "",
210 `RESULT: PASS — map above for ${scopeNote}; review the Fix section and repair the layout breaks you choose (atmosphere stays).`,
211 ]);
212}
214main().catch((err) => {
215 report([`RESULT: ERROR — ${err instanceof Error ? err.message : String(err)}`]);
216 process.exitCode = 1;
217});
The sweep itself is plain Playwright measurement, not vision. For each lesson × theme × viewport it walks the DOM and records horizontal overflow (and the elements causing it), broken images, an empty main, and reveal-on-scroll blocks left stuck hidden.
The overflow oracle: which elements cross the viewport edge, and by how much.
scripts/render-audit.js · 159 lines
scripts/render-audit.js
⋯ 93 lines hidden (lines 1–93)
1/**
2 * Rendered layout-defect auditor — the objective signal for the visual-sanity loop.
3 *
4 * Loads every lesson + the index in BOTH themes at TWO viewports (desktop + mobile)
5 * and MEASURES render breakage against the live DOM — the only reliable oracle for
6 * layout: horizontal overflow (and the elements causing it), broken images, an
7 * empty/failed main region, and reveal-on-scroll blocks left stuck hidden after a
8 * full scroll. Emits a per-(lesson,theme,viewport) JSON report so the loop's agent
9 * classifies real defect vs intentional from data, not vibes.
10 *
11 * Lesson ids are parsed from src/lesson-catalog.js (NOT hardcoded), so the sweep
12 * never goes stale as lessons are added.
13 *
14 * Usage: node scripts/render-audit.js [baseURL] [outfile]
15 * defaults: http://127.0.0.1:5192 /tmp/gb-render.json
16 */
17import { chromium } from '@playwright/test';
18import { readFileSync, writeFileSync } from 'node:fs';
19import { fileURLToPath } from 'node:url';
20import { dirname, resolve } from 'node:path';
22const HERE = dirname(fileURLToPath(import.meta.url));
23const BASE = process.argv[2] || 'http://127.0.0.1:5192';
24const OUT = process.argv[3] || '/tmp/gb-render.json';
25const THEMES = ['dark', 'light'];
26const VIEWPORTS = [
27 { name: 'desktop', width: 1280, height: 1400 },
28 { name: 'mobile', width: 390, height: 844 },
29];
31// Parse lesson ids straight from the catalog (regex — no React import in node).
32function lessonIds() {
33 const text = readFileSync(resolve(HERE, '..', 'src', 'lesson-catalog.js'), 'utf8');
34 const ids = [];
35 const re = /\bid:\s*(['"])([^'"]+)\1/g;
36 let m;
37 while ((m = re.exec(text)) !== null) if (m[2] !== 'index') ids.push(m[2]);
38 return ids;
39}
41const pages = [
42 { id: 'index', path: '/' },
43 ...lessonIds().map((id) => ({ id, path: `/?lesson=${id}` })),
44];
46const browser = await chromium.launch();
47const report = [];
49for (const vp of VIEWPORTS) {
50 for (const theme of THEMES) {
51 const ctx = await browser.newContext({
52 viewport: { width: vp.width, height: vp.height },
53 deviceScaleFactor: 1,
54 reducedMotion: 'reduce', // reveal snaps to its final state, no mid-fade
55 });
56 await ctx.addInitScript((t) => localStorage.setItem('glassbox-theme', t), theme);
57 const page = await ctx.newPage();
58 for (const p of pages) {
59 await page.goto(`${BASE}${p.path}`, { waitUntil: 'load' });
60 await page.waitForLoadState('networkidle').catch(() => {});
61 await page.waitForTimeout(250);
62 // Full scroll so every reveal-on-scroll IntersectionObserver fires.
63 await page.evaluate(async () => {
64 const h = document.body.scrollHeight;
65 for (let y = 0; y < h; y += Math.round(window.innerHeight * 0.8)) {
66 window.scrollTo(0, y);
67 await new Promise((r) => setTimeout(r, 60));
68 }
69 window.scrollTo(0, 0);
70 await new Promise((r) => setTimeout(r, 200));
71 });
73 const d = await page.evaluate((vpWidth) => {
74 const out = {
75 pageOverflowPx: 0,
76 offenders: [],
77 brokenImages: [],
78 stuckHidden: [],
79 mainEmpty: false,
80 };
81 const sel = (el) => {
82 if (!el || el.nodeType !== 1) return '';
83 if (el.id) return `#${el.id}`;
84 const cls = (el.getAttribute('class') || '')
85 .trim()
86 .split(/\s+/)
87 .filter(Boolean)
88 .slice(0, 2);
89 return el.tagName.toLowerCase() + (cls.length ? '.' + cls.join('.') : '');
90 };
91 const doc = document.scrollingElement || document.documentElement;
92 out.pageOverflowPx = Math.max(0, Math.round(doc.scrollWidth - vpWidth));
94 // Elements whose box extends past the right edge (the overflow culprits).
95 const seen = new Set();
96 for (const el of Array.from(document.body.querySelectorAll('*'))) {
97 const r = el.getBoundingClientRect();
98 if (r.width === 0 || r.height === 0) continue;
99 const over = Math.round(r.right - vpWidth);
100 if (over > 8 || r.left < -8) {
101 const s = sel(el);
102 if (seen.has(s)) continue;
103 seen.add(s);
104 out.offenders.push({ sel: s, overPx: Math.max(over, Math.round(-r.left)) });
105 if (out.offenders.length >= 8) break;
106 }
107 }
108 out.offenders.sort((a, b) => b.overPx - a.overPx);
⋯ 51 lines hidden (lines 109–159)
110 // Broken images (loaded but zero natural size).
111 for (const img of Array.from(document.images)) {
112 if (img.complete && img.naturalWidth === 0) {
113 out.brokenImages.push(sel(img) + (img.src ? ` (${img.src.slice(0, 60)})` : ''));
114 if (out.brokenImages.length >= 6) break;
115 }
116 }
118 // Reveal-on-scroll blocks left hidden after a full scroll (candidates:
119 // the agent confirms whether each is a stuck reveal or intentional).
120 const revRe = /(^|[\s-])(reveal|rv|rev)([\s-]|$)/i;
121 let hiddenCount = 0;
122 for (const el of Array.from(document.body.querySelectorAll('[class]'))) {
123 const cls = el.getAttribute('class') || '';
124 if (!revRe.test(cls)) continue;
125 const cs = getComputedStyle(el);
126 if (cs.display === 'none' || cs.visibility === 'hidden') continue;
127 if (parseFloat(cs.opacity) < 0.08 && (el.textContent || '').trim().length > 0) {
128 if (hiddenCount < 6) out.stuckHidden.push(sel(el));
129 hiddenCount++;
130 }
131 }
133 // Did the lesson render any content at all?
134 const main = document.querySelector('main') || document.body;
135 out.mainEmpty =
136 (main.textContent || '').trim().length < 40 || main.getBoundingClientRect().height < 80;
137 return out;
138 }, vp.width);
140 report.push({ lesson: p.id, theme, viewport: vp.name, ...d });
141 const flags =
142 (d.pageOverflowPx ? `overflow ${d.pageOverflowPx}px` : '') +
143 (d.brokenImages.length ? ` ${d.brokenImages.length} broken-img` : '') +
144 (d.stuckHidden.length ? ` ${d.stuckHidden.length} stuck-hidden` : '') +
145 (d.mainEmpty ? ' EMPTY' : '');
146 console.log(`${vp.name.padEnd(7)} ${theme.padEnd(5)} ${p.id.padEnd(24)} ${flags || 'ok'}`);
147 }
148 await ctx.close();
149 }
150}
151await browser.close();
153writeFileSync(OUT, JSON.stringify(report, null, 2));
154const defectPages = report.filter(
155 (r) => r.pageOverflowPx > 8 || r.brokenImages.length || r.stuckHidden.length || r.mainEmpty,
156).length;
157console.log(
158 `\n${report.length} page-renders measured · ${defectPages} with a defect signal → ${OUT}`,
159);
Proof the dedup is real — and a staleness bug fixed in passing
The claim "deduped, not triplicated" is only true if the existing loop actually moved onto the scaffold. It did: content-accuracy now imports from per-lesson.ts and ends in the same runPerLessonLoop call the new loops use — its ~200 lines of bespoke orchestration deleted, its two long prompts kept verbatim.
content-accuracy, after migration — the same one-call shape.
agents/content-accuracy.ts · 139 lines
agents/content-accuracy.ts
⋯ 124 lines hidden (lines 1–124)
1/**
2 * Content-accuracy loop — Many Hands Engineering, the deep-honesty pass.
3 *
4 * Every other loop guards the *machinery* (deps, lint, motion gates, a11y, dead
5 * code, comment/console/promise hygiene). This one guards the *truth of the
6 * teaching*: is what each lesson asserts — in its prose, its numbers, the
7 * algorithm its engine implements, and what its labs and animations VISUALLY
8 * claim — actually correct for the topic it promises, and faithful to the
9 * canonical source named in its eyebrow (RFC 8446, Bayer & McCreight 1970,
10 * Flajolet et al. 2007, …)?
11 *
12 * It is PER-LESSON, fanned out in parallel on the per-lesson.ts scaffold: one
13 * agent per lesson, each on the most capable model at the highest reasoning
14 * effort (`opus` + `effort: "max"`), reading the lesson's whole content corpus
15 * and emitting a strict three-bucket map under a hard cite-or-omit rule. It is
16 * READ-ONLY — it can investigate but never edits; the human corrects what they
17 * agree with.
18 *
19 * This is the HEAVIEST loop by far (N max-effort Opus agents, each reading a
20 * full lesson). NOT routine — run it occasionally, or scope it to the lesson(s)
21 * you just touched, and bound it with `--limit`, `--concurrency`, `--budget`.
22 *
23 * The outside reference is not a test suite — accuracy is not something vitest /
24 * eslint / vite can assert. It is the canonical literature of each topic, brought
25 * by a domain-expert reviewer. So this loop can only MAP, with every finding
26 * anchored to a quoted line; that is why it is read-only and the human stays the
27 * steward.
28 *
29 * Permissions: Read / Grep / Glob ONLY — structurally cannot edit a file.
30 *
31 * Usage:
32 * tsx content-accuracy.ts # all lessons (heavy!)
33 * tsx content-accuracy.ts swim tls # only these lesson ids
34 * tsx content-accuracy.ts --concurrency 2 # cap parallel agents (default 3)
35 * tsx content-accuracy.ts --limit 3 # only the first 3 catalog lessons
36 * tsx content-accuracy.ts --budget 8 # per-lesson USD cap (default 6)
37 * tsx content-accuracy.ts --dry-run # show the plan, spend nothing
38 */
39import { CATALOG, LESSONS_DIR, type Lesson, relList, runPerLessonLoop } from "./per-lesson.js";
40import { report } from "./lib.js";
42// ── Per-lesson context handed to the agent ────────────────────────────────────
43function userPrompt(lesson: Lesson, files: string[]): string {
44 return `Lesson under review: \`${lesson.id}\` — "${lesson.title}".
45Claimed topic / canonical source (the lesson's eyebrow in ${CATALOG}): ${lesson.eyebrow || "(none stated)"}.
47All of this lesson's content lives under \`${LESSONS_DIR}/${lesson.id}/\`. Its ${files.length} files, relative to that directory — read EVERY one before you judge:
48${relList(lesson.id, files)}
50Also:
51- Read this lesson's framing in \`${CATALOG}\` (its title, subtitle, and pitch — the promises it makes the learner).
52- \`${LESSONS_DIR}/${lesson.id}/engine/index.js\` is the SOURCE OF TRUTH for what every lab, animation, and visualization actually computes — any disagreement between the prose and the engine is a finding.
53- Find this lesson's engine test in the REPO-ROOT \`tests/\` directory (its name may not match the id — e.g. bloom-filters → \`tests/bloom-math.test.js\`): Glob \`tests/*.js\` and grep for an import from \`lessons/${lesson.id}/engine\`. It encodes the intended behavior and is a strong cross-check.
55Verify the lesson per your instructions and emit the three-bucket map.`;
56}
58// ── Agent's job (the quality bar lives here) ──────────────────────────────────
59function systemPrompt(lesson: Lesson): string {
60 return `You are the content-accuracy verifier for ONE lesson of the Interactive Lessons repository — a React 19 + Vite collection of self-contained, animation-heavy lessons that teach computer-science, distributed-systems, data-structure, and networking topics. You are reviewing \`${lesson.id}\` ("${lesson.title}")${
61 lesson.eyebrow ? `, which claims the topic/source: ${lesson.eyebrow}` : ""
62 }.
64You are a DOMAIN EXPERT on this lesson's subject. Your single job: determine whether everything the lesson teaches is FACTUALLY AND CONCEPTUALLY CORRECT for that subject and FAITHFUL to its canonical source — across every medium the lesson uses, not just the prose.
66Your only tools are Read / Grep / Glob — you can investigate but you CANNOT edit any file. The human corrects what they agree with.
68WHAT TO REVIEW (read all of it before judging — deep coverage is the whole point of this loop):
69- Prose — \`sections/*.jsx\` and explanatory text in \`components/*.jsx\` / \`components/data.js\` (often rendered as markdown). Definitions, claims, history, names, dates, and any formula stated in words.
70- The engine — \`engine/index.js\`. This pure-logic module is the SOURCE OF TRUTH for what every lab and animation computes. Verify the ALGORITHM it implements is the real one (correct steps, correct invariants, correct complexity if a complexity is claimed) and that it matches what the prose says it does.
71- Interactive labs — \`labs/*.jsx\`. Is the cause→effect the lab lets the learner explore TRUE to the real system? A lab that lets you "delete from a standard Bloom filter" or shows a B-tree splitting the wrong way teaches a falsehood.
72- Animations — the step / timeline logic. An animation is a CLAIM about sequence and causality. In this repo the animation is usually ENGINE-DRIVEN: the engine emits an ordered list of frames / steps (each with a caption, a highlighted node, an invariant) and the lab just plays them on a timer. So verify the step ORDER and the per-step CAPTIONS where the claim actually originates — the engine's frame-building code — then confirm the lab / visual renders those frames faithfully; do not stop at the playback timer. The depicted order, the highlighted invariant, and the end state must all be what really happens. A polished animation that shows the wrong step order is a lie the learner will remember.
73- Visuals / diagrams — hand-coded or engine-driven SVG / Canvas (trees, grids, curves, registers). Verify the geometry, labels, and values they render are correct (e.g. a HyperLogLog error curve with the wrong asymptote, a Merkle proof drawn against the wrong sibling).
74- Numbers & formulas everywhere — constants, thresholds, big-O, probabilities, bit-widths, RFC field sizes. Recompute the ones the lesson leans on to confirm the MAGNITUDE and the TREND — not the last decimal. Numbers here are frequently rounded for teaching ("9.6 bits/element" for 9.585…, "~10 bits", "k ≈ 7"); honest rounding is NOT an error (see the numbers rule below).
76Cross-checks available to you (use them — they are your strongest IN-REPO ground truth): read the lesson's framing in ${CATALOG} (title / subtitle / pitch — the promises made), and find this lesson's engine test. The tests live in the REPO-ROOT \`tests/\` directory (NOT under the lesson folder), and the filename does not always match the lesson id (e.g. bloom-filters → \`tests/bloom-math.test.js\`, lsm-trees → \`tests/lsm-engine.test.js\`). Glob \`tests/*.js\` and grep the candidates for an import from \`lessons/${lesson.id}/engine\` to pick the right one, then read it — it encodes the intended engine behavior.
78Classify EACH issue into EXACTLY ONE of three buckets:
80(1) INACCURACY / ERROR — must fix. Include ONLY when you can answer all THREE in one sentence each:
81 (a) WHAT — quote the EXACT offending text or code line as you observed it in your Read, in backticks, with file:line. No quote pasted from a Read = no finding. Cite or omit — a hard binary.
82 (b) WHY it is wrong — state the CORRECT fact/behavior (cite the algorithm / RFC / paper reality where it matters) and name the consequence for a learner: the wrong mental model they would form.
83 (c) ACTION — the concrete correction (the right value, the corrected sentence, the fixed step order, the engine change that makes the visualization honest).
84 Kinds that belong here: a flat factual error (wrong author / year / value / definition); a conceptual error (misstates how the thing works); an algorithmic error (the engine computes the wrong thing, or a stated complexity the code contradicts); an animation or visual that depicts a step / order / invariant that does not occur; a wrong formula or miscomputed number; prose and engine that contradict each other.
86(2) VERIFIED CORRECT — the load-bearing claims you checked and confirmed. NOT padding: list the CENTRAL assertions of the lesson (the headline formula, the core invariant, the named author / year, the key complexity, the one thing the marquee animation claims) and confirm each holds. For each, give an ANCHOR — the exact file:line / quoted formula / engine function you checked AND the recomputed value or the test that confirms it (e.g. \`engine/index.js:66 falsePositiveRate = (1 - e^(-kn/m))^k — the classic closed form, asserted in tests/bloom-math.test.js\`). A "verified" line with no anchor is as inadmissible as an uncited bucket-(1) finding. This shows your work and bounds the credibility of bucket (1).
88(3) PEDAGOGICAL SIMPLIFICATION — judgment calls. Teaching REQUIRES simplifying; that is craft, not error. For each place the lesson abstracts, omits, or uses a friendly metaphor, decide:
89 - HONEST simplification (a fair first-order model that does not claim to be the whole truth) — note it briefly; it is NOT a defect.
90 - MISLEADING simplification (will plant a wrong mental model the learner must later unlearn, or states the simplified case as if it were the general truth) — describe precisely what breaks and when.
91 Only surface simplifications worth the steward's attention. Do not list every reasonable omission.
93KNOWN-CONTEXT AWARENESS:
94- These lessons are intentionally INTUITIVE and metaphor-rich (a card catalog for B-trees, a postal service for UDP). A metaphor is not wrong for being a metaphor — judge whether it leads to a CORRECT mental model, not whether it is literally precise.
95- Lessons legitimately scope down (TLS 1.3 only, one hash variant, a small parameter range). Teaching a subset is not an error UNLESS the lesson states the subset as the whole.
96- NUMBERS ARE OFTEN ROUNDED FOR TEACHING. A rounded or order-of-magnitude figure (9.6 vs 9.585 bits, "~10 bits", "k ≈ 7", a curve's stated asymptote) is bucket (3) AT MOST — never bucket (1). Flag a number as an error only when it is wrong beyond any reasonable rounding (wrong leading digit, wrong exponent, wrong sign of the trend) OR when the stated figure contradicts what THIS lesson's engine actually computes for the stated inputs. Confirm magnitude and trend; do not police the last decimal.
97- YOU HAVE NO WEB ACCESS (Read / Grep / Glob only). Any claim you make about an EXTERNAL source — an RFC field size, a paper's author / year, a published constant — rests on your own memory, which can be wrong. Calibrate by how well-established the fact is and whether the repo can corroborate it: contradicting a landmark, textbook-level fact you are genuinely certain of (a famous paper's author / year, a canonical complexity) is fair for bucket (1) with the correction stated; contradicting a precise external detail you are NOT certain of belongs in bucket (3), explicitly flagged "verify against the source." When the in-repo engine / tests / prose independently corroborate the discrepancy, it is bucket (1) regardless. Never present a shaky recollection as established fact.
98- The engine is pure and unit-tested; if a test asserts a behavior, that behavior is intentional — verify the behavior is CORRECT, do not flag it merely for existing.
99- You are judging ACCURACY AND CORRECTNESS ONLY. Do NOT review prose style, tone, pacing, accessibility, performance, or code quality — other loops own those. Stay on truth.
101Hard rules:
102- Cite or omit. A finding with no quoted line from your own Read is a confabulation; suppress it. Do not pattern-match on "what an error usually looks like" — verify the actual content.
103- Be a tough but fair expert: contradict the lesson only with a correct, specific fact, and still quote the exact claim you are contradicting.
104- "Simplified" is not "wrong." Push borderline cases to bucket (3); reserve bucket (1) for things that are actually false or that actually contradict the engine / source.
105- If the same error recurs across sections, report the PATTERN once with one action covering all sites.
106- An empty bucket (1) with a populated bucket (2) is a perfectly good, expected result. Do not invent findings to fill it.
108Output a structured map with exactly these three sections in this order:
110## Inaccuracy / error — fix (review & act)
112(per item — file:line · WHAT (quoted) · WHY it's wrong (correct fact + learner consequence) · ACTION)
114## Verified correct
116(the central claims you checked and confirmed — claim · what you checked)
118## Pedagogical simplification (your call)
120(per item — what is simplified · honest or misleading · what breaks and when)
122End with a final summary line: "<X> errors · <Y> verified · <Z> simplifications". Nothing after.`;
123}
125runPerLessonLoop({
126 name: "Content-accuracy",
127 blurb: "deep per-lesson truth review (opus · effort:max), READ-ONLY",
128 systemPrompt,
129 userPrompt,
130 model: "opus",
131 effort: "max",
132 maxTurns: 400,
133 defaultBudget: 6,
134 resultLine: (scope) =>
135 `RESULT: PASS — maps above for ${scope}. Review each lesson's "Inaccuracy / error" section and correct what you agree with; treat "Pedagogical simplification" as judgment calls.`,
Wiring up the render sweep also turned up a real bug in a neighbouring script. contrast-audit.js had hardcoded an 18-lesson id list, so it had been silently skipping the five newest lessons every run. It now parses the catalog the way the scaffold does — the exact staleness class these loops exist to kill.
Lesson ids parsed from the catalog, so the sweep can't go stale.
scripts/contrast-audit.js · 113 lines
scripts/contrast-audit.js
⋯ 29 lines hidden (lines 1–29)
1/**
2 * Rendered color-contrast auditor — the objective gate for the dual-theme work.
3 *
4 * Loads every lesson + the index in BOTH themes and runs axe-core's
5 * `color-contrast` rule against the live DOM (the only reliable oracle, since it
6 * resolves the real cascade + computed colors). Emits a per-lesson JSON report of
7 * every failing text/background pair (selector, fg, bg, ratio, required) so the
8 * artisan pass works from data, not vibes.
9 *
10 * Usage: node scripts/contrast-audit.mjs [baseURL] [outfile]
11 * defaults: http://127.0.0.1:5180 /tmp/gb-contrast.json
12 *
13 * Note: axe color-contrast also catches INTENTIONAL decorative low-contrast
14 * (faint eyebrows, ghost captions). The report is candidates; the human/agent
15 * judges decorative-vs-defect. We rank by how far below threshold each pair is.
16 */
17import AxeBuilder from '@axe-core/playwright';
18import { chromium } from '@playwright/test';
19import { readFileSync, writeFileSync } from 'node:fs';
20import { fileURLToPath } from 'node:url';
21import { dirname, resolve } from 'node:path';
23const HERE = dirname(fileURLToPath(import.meta.url));
24const BASE = process.argv[2] || 'http://127.0.0.1:5180';
25const OUT = process.argv[3] || '/tmp/gb-contrast.json';
26const THEMES = ['dark', 'light'];
28// Parse lesson ids straight from the catalog (regex — no React import in node),
29// so the sweep never goes stale as lessons are added.
30const LESSON_IDS = (() => {
31 const text = readFileSync(resolve(HERE, '..', 'src', 'lesson-catalog.js'), 'utf8');
32 const ids = [];
33 const re = /\bid:\s*(['"])([^'"]+)\1/g;
34 let m;
35 while ((m = re.exec(text)) !== null) if (m[2] !== 'index') ids.push(m[2]);
36 return ids;
37})();
⋯ 76 lines hidden (lines 38–113)
38const pages = [
39 { id: 'index', title: 'index', path: '/' },
40 ...LESSON_IDS.map((id) => ({ id, title: id, path: `/?lesson=${id}` })),
41];
43const browser = await chromium.launch();
44const report = [];
46for (const theme of THEMES) {
47 // reducedMotion: reveal-on-scroll snaps to opacity:1 instantly (no mid-fade),
48 // so once an element scrolls into view axe sees its TRUE color.
49 const ctx = await browser.newContext({
50 viewport: { width: 1280, height: 1400 },
51 deviceScaleFactor: 1,
52 reducedMotion: 'reduce',
53 });
54 await ctx.addInitScript((t) => localStorage.setItem('glassbox-theme', t), theme);
55 const page = await ctx.newPage();
56 for (const p of pages) {
57 await page.goto(`${BASE}${p.path}`, { waitUntil: 'load' });
58 await page.waitForLoadState('networkidle').catch(() => {});
59 await page.waitForTimeout(300);
60 // Scroll the whole page so every reveal-on-scroll section fires its
61 // IntersectionObserver (→ opacity:1), then return to top. Without this, axe
62 // measures below-fold content mid-reveal and reports phantom low-contrast.
63 await page.evaluate(async () => {
64 const h = document.body.scrollHeight;
65 for (let y = 0; y < h; y += Math.round(window.innerHeight * 0.8)) {
66 window.scrollTo(0, y);
67 await new Promise((r) => setTimeout(r, 70));
68 }
69 window.scrollTo(0, 0);
70 await new Promise((r) => setTimeout(r, 250));
71 });
72 const resolved = await page.evaluate(() => document.documentElement.getAttribute('data-theme'));
73 const results = await new AxeBuilder({ page }).withRules(['color-contrast']).analyze();
75 const fails = [];
76 for (const v of results.violations) {
77 for (const node of v.nodes) {
78 const d = (node.any && node.any[0] && node.any[0].data) || {};
79 fails.push({
80 selector: Array.isArray(node.target) ? node.target.join(' ') : String(node.target),
81 fg: d.fgColor,
82 bg: d.bgColor,
83 ratio: d.contrastRatio,
84 required: d.expectedContrastRatio,
85 fontSize: d.fontSize,
86 fontWeight: d.fontWeight,
87 });
88 }
89 }
90 fails.sort((a, b) => (a.ratio ?? 99) - (b.ratio ?? 99)); // worst first
91 report.push({ lesson: p.id, title: p.title, theme, resolved, fails });
92 const worst = fails[0] ? ` worst ${fails[0].ratio}:1 (need ${fails[0].required})` : '';
93 console.log(
94 `${theme.padEnd(5)} ${p.id.padEnd(24)} ${String(fails.length).padStart(3)} fail${worst}`,
95 );
96 }
97 await ctx.close();
98}
99await browser.close();
101writeFileSync(OUT, JSON.stringify(report, null, 2));
103// Per-lesson rollup across both themes
104const byLesson = {};
105for (const r of report) {
106 byLesson[r.lesson] = byLesson[r.lesson] || { dark: 0, light: 0 };
107 byLesson[r.lesson][r.theme] = r.fails.length;
108}
109const total = report.reduce((n, r) => n + r.fails.length, 0);
110console.log('\n── per-lesson (dark / light) ──');
111for (const [id, c] of Object.entries(byLesson))
112 console.log(` ${id.padEnd(24)} ${String(c.dark).padStart(3)} / ${String(c.light).padStart(3)}`);
113console.log(`\nTOTAL contrast failures: ${total} → ${OUT}`);
What the loops caught — and the fixes
All three loops were run over every one of the 23 lessons. lab-fidelity (0 defects) and visual-sanity (0 of 96 renders flagged) came back clean. collection-coherence surfaced three real gaps — all fixed in the second commit, and confirmed closed by re-running the loop on those lessons.
| Lesson | Gap the loop found | Fix |
|---|---|---|
| bloom-filters | Hero claims 7 interactive labs; only 6 render, and badges skip from Lab 05 to Lab 07 | Renumber to Lab 06, correct the Hero count to 6 |
| bloom-filters | Ch08 puts HyperLogLog at ~1.5 KB, contradicting the hyperloglog lesson's ~12 KB (stated 4×) | Reconcile to the owner's ~12 KB |
| b-trees | Coda asks why "filing 80" made the tree wider not taller — but 80 is never staged and the question is never posed | Retarget the callback to the split lab the reader actually used |
The b-trees one is the subtle one. The §IX coda was a rhetorical "the question from before" callback — but it referenced a key (80) the split lab never files and a question the lesson never asks (§V scripts filing 50 into a full root). The fix drops the fabricated setup and points the reflection at the lab the reader did use, keeping the pedagogy while making the callback resolve.
The retargeted coda — no fabricated key, references the real split lab.
src/lessons/b-trees/sections/Onward.jsx · 67 lines
src/lessons/b-trees/sections/Onward.jsx
⋯ 57 lines hidden (lines 1–57)
1import { Reveal } from '../../../shared/reveal.jsx';
2import Section from '../components/Section.jsx';
3import Callout from '../components/Callout.jsx';
5// §IX — the one idea to carry away, then where to go next. The synthesis box
6// names what a B-tree *is*; the numbered list points at three real directions
7// (concurrency, write-optimized variants, deletion merge). Pure prose, no lab.
8const NEXT = [
9 [
10 'i',
11 'Concurrency',
12 'Many threads splitting and merging one tree without corrupting it. The trick is latch crabbing: you release a parent’s lock only once you’re sure the child won’t split into it.',
13 ],
14 [
15 'ii',
16 'Write-optimized variants',
17 'Bε-trees and fractal trees buffer writes inside the nodes, clawing back the LSM-tree’s write advantage without giving up the B-tree’s reads.',
18 ],
19 [
20 'iii',
21 'Deletion, in full',
22 'The symmetric move to splitting: when a node falls below half-full, it borrows a key from a sibling, or pulls one down from the parent and merges. Trees shrink from the top, too.',
23 ],
24];
26export default function Onward() {
27 return (
28 <Section roman="IX" kicker="Onward" title="Where this goes next">
29 <Reveal base="bt-rev">
30 <div className="bt-coda">
31 <p className="bt-kicker">The one idea</p>
32 <p className="bt-p bt-coda-lead">
33 A B-tree is one decision held to its limit:{' '}
34 <span className="bt-em">fill a node until it fills a disk page, then split</span>. Match
35 the branching factor to the hardware, and everything else follows — the few short
36 hops to any record, the ordered leaves that make a range a single walk, the tree that
37 stays balanced no matter the order things arrive. One forklift trip per node, and you
38 always load it full.
39 </p>
40 </div>
41 </Reveal>
43 <Reveal base="bt-rev">
44 <p className="bt-kicker bt-coda-next-label">Where to go next</p>
45 <div className="bt-next">
46 {NEXT.map(([n, t, d]) => (
47 <div className="bt-next-item" key={n}>
48 <span className="bt-next-n">{n}</span>
49 <div>
50 <span className="bt-next-t">{t}</span>
51 <p className="bt-next-d">{d}</p>
52 </div>
53 </div>
54 ))}
55 </div>
56 </Reveal>
58 <Callout title="Why most inserts widen but don't deepen">
59 Back in the split lab, only the key that overflowed a{' '}
60 <span className="bt-em">full root</span> ever added a floor. Every other insert split a leaf
61 and sent its median up, but the root still had room to absorb it — so the tree grew{' '}
62 <span className="bt-em">wider</span>, not <span className="bt-em">taller</span>. A B-tree
63 gains height only at the top, and only when the top itself is full.
64 </Callout>