What changed, and why
froot's dead-code loop knew one shape of dead weight: an unused dependency. It dropped the dep from the manifest and let the repo's own CI confirm the build no longer needed it. This change finishes the loop out to dead code, the rest of the MHE dead-code catalog, with no new loop, gate, or schedule. The same dead-code loop now also deletes whole unused files and un-exports symbols no other module imports.
The interesting part is where the new lines are not. The state machine, the workflows, the merge gate, and the durable schedule did not move. The change adds two work-item kinds the spine carries, one pure source transform, and a few lines of knip parsing the adapter was already discarding. These are froot's first edits to source rather than a manifest, so they double as the proof that a source-editing work item fits the existing chassis.
Blast radius stays small by construction: every change is still a PR a human approves (trust is propose-only by default), CI is still the only oracle, and a dead export is un-exported, not deleted. That last choice is the safe response to what the analyzer actually claims, and the rest of this guide explains why.
Two new work-item kinds
A bump and a removal were the whole work-item union. Now there are four. DeadFile is a module nothing imports; DeadExport is a symbol exported but imported by no other module. They are separate kinds, not one shape with an optional symbol, so each is valid by construction: a file deletion has no symbol, an un-export must name exactly one.
Each kind is frozen and self-describing; subject is its human identifier (a path, a symbol).
src/froot/domain/dead_source.py · 97 lines
src/froot/domain/dead_source.py
⋯ 33 lines hidden (lines 1–33)
1"""Dead source: the work-item kinds the dead-code loop deletes from the source.
3The dead-code loop began on one shape of dead weight — an unused *dependency*
4(:class:`~froot.domain.removal.Removal`). These two kinds finish it out to the
5rest of MHE's dead-code catalog: dead *code*. A static analyzer (knip) that
6already maps the import graph reports two further shapes of deadness:
8* :class:`DeadFile` — a whole module nothing imports. The action deletes the
9 file; CI is the oracle (a build that still needs it goes red).
10* :class:`DeadExport` — a symbol exported but imported by no other module. knip
11 flags "does not need to be exported", not "dead everywhere", so the safe,
12 reversible action is to *un-export* it (strip the ``export`` modifier),
13 leaving it module-private. Cross-module use the analyzer missed surfaces as a
14 red CI; in-file use is untouched.
16Both are *signal-judged* like a removal — a safe-to-remove veto runs at the
17signal, there is no changelog to read — and neither carries a version, so like a
18removal they set ``reconciles=False`` and never touch the forward-stable
19invariant a candidate must satisfy. They are separate kinds (not one shape with
20an optional symbol) so each is valid by construction: a file deletion has no
21symbol, an un-export must name one.
22"""
24from __future__ import annotations
26from typing import Literal
28from pydantic import Field
30from froot.domain.base import Frozen
31from froot.domain.ecosystem import Ecosystem
34class DeadFile(Frozen):
35 """A proposed deletion of a single unused source file.
37 Attributes:
38 kind: The work-item discriminator — always ``"dead_file"``.
39 path: The file to delete, relative to the manifest directory the
40 analyzer ran in (where the action resolves it against the checkout).
41 ecosystem: The ecosystem whose analyzer flagged it (namespacing only;
42 the deletion itself is language-agnostic).
43 justification: A short "why" — the analyzer's finding plus the
44 safe-to-remove veto's reasoning; carried to the PR body.
45 """
47 kind: Literal["dead_file"] = "dead_file"
48 path: str = Field(min_length=1)
49 ecosystem: Ecosystem
50 justification: str | None = None
52 @property
53 def subject(self) -> str:
54 """The work item's human-readable identifier (its path)."""
55 return self.path
57 def __str__(self) -> str:
58 """Render as ``delete path (unused)``."""
59 return f"delete {self.path} (unused)"
⋯ 22 lines hidden (lines 61–82)
62class DeadExport(Frozen):
63 """A proposed un-export of a single unused exported symbol.
65 The action strips the ``export`` modifier on ``file`` at ``line`` — a
66 one-line edit that leaves the symbol in place but module-private. The signal
67 only ever emits the un-exportable inline declaration forms (``export
68 function/const/class/…``); clause re-exports, ``export default``, and
69 ``export *`` are dropped at the signal because un-exporting them is not a
70 single-line edit (see :func:`froot.policy.dead_source.unexport_line`).
72 Attributes:
73 kind: The work-item discriminator — always ``"dead_export"``.
74 file: The source file holding the export, relative to the manifest dir.
75 symbol: The exported name the analyzer flagged as unused cross-module.
76 line: The 1-based line of the export declaration (the analyzer's
77 position; the action re-checks it names ``symbol`` before acting).
78 ecosystem: The ecosystem whose analyzer flagged it (namespacing).
79 justification: A short "why" — the analyzer's finding plus the veto's
80 reasoning; carried to the PR body.
81 """
83 kind: Literal["dead_export"] = "dead_export"
84 file: str = Field(min_length=1)
85 symbol: str = Field(min_length=1)
86 line: int = Field(gt=0)
87 ecosystem: Ecosystem
88 justification: str | None = None
90 @property
91 def subject(self) -> str:
92 """The work item's human-readable identifier (its symbol)."""
93 return self.symbol
95 def __str__(self) -> str:
96 """Render as ``un-export symbol in file (unused)``."""
97 return f"un-export {self.symbol} in {self.file} (unused)"
That subject property is small but load-bearing. The spine's logs and labels used to read .package off every work item; a file or an export has no package. Giving every kind a subject lets the spine stay generic instead of growing a branch per kind. The union itself just widens, and kind routes it.
The discriminated union widens a third and fourth time.
src/froot/domain/work.py · 31 lines
src/froot/domain/work.py
⋯ 22 lines hidden (lines 1–22)
1"""The work item: the bounded unit of work the chassis carries, for any loop.
3A loop proposes one work item at a time; the spine shuttles it through the same
4states and effects (judge → open PR → await CI → record → gate) regardless of
5*what* it is. The kinds are heterogeneous on purpose — a bump moves a version, a
6removal deletes dead weight — so they are a discriminated union, not a single
7forced shape. Activities (the impure boundary) dispatch on ``kind`` to the right
8signal, judge, and action; the pure spine never inspects the payload.
10This is the first widening of the chassis past "a bump" — the seam that, taken
11to its conclusion (an open loop registry), is froot's north star (see VISION).
12"""
14from __future__ import annotations
16from typing import Annotated
18from pydantic import Field
20from froot.domain.candidate import Candidate
21from froot.domain.dead_source import DeadExport, DeadFile
22from froot.domain.removal import Removal
24# One bounded unit of work, of any loop's kind. ``kind`` discriminates so the
25# Temporal data converter and the activities both route without guesswork. A
26# bump moves a version; a removal deletes a dependency; a dead file is deleted
27# whole; a dead export is stripped of its ``export``. The non-bump kinds are all
28# signal-judged (a veto at the signal, no changelog) and carry no version.
29WorkItem = Annotated[
30 Candidate | Removal | DeadFile | DeadExport, Field(discriminator="kind")
31]
The action: a file unlinked, an export stripped
The open-PR activity already branched per kind to apply a bump or a removal. Two arms join it. A dead file is deleted from the checkout; a dead export has its line rewritten. Neither goes through the package manager — deleting a file is language-agnostic, and the un-export is a pure text edit — so both are small helpers the activity calls directly. The push that follows stages whatever changed (git add -A), and CI runs against the result.
The two source actions. The export edit re-validates the line before writing.
src/froot/workflow/activities.py · 921 lines
src/froot/workflow/activities.py
⋯ 136 lines hidden (lines 1–136)
1"""Activities: the effect interpreters that wrap the adapters.
3Each activity is the impure boundary for one effect. Adapter imports are LAZY
4(inside the bodies) so the model and HTTP stacks never enter a workflow sandbox;
5the pure policy and domain imports stay at module level. Activities return
6domain values — the workflow wraps them into events and feeds the pure state
7machine. The activity signatures' types are evaluated at runtime by Temporal's
8data converter, so the domain imports here are deliberately not deferred.
9"""
11from __future__ import annotations
13import json
14import logging
15import tempfile
16from pathlib import Path
17from typing import TYPE_CHECKING, assert_never
19from temporalio import activity
21from froot.domain.a11y import A11yAnalysis, A11yVerdict
22from froot.domain.candidate import Candidate
23from froot.domain.changelog import (
24 ChangelogVerdict,
25 CleanVerdict,
26 UnknownVerdict,
27)
28from froot.domain.ci import CIStatus
29from froot.domain.dead_source import DeadExport, DeadFile
30from froot.domain.determinism import AnalysisResult, FrontierVerdict
31from froot.domain.pull_request import PullRequestDraft, PullRequestRef
32from froot.domain.removal import Removal
33from froot.domain.repo import TargetRepo
34from froot.domain.work import WorkItem
35from froot.policy.a11y_comment import (
36 A11Y_MARKER,
37 render_a11y_comment,
38 should_post,
39)
40from froot.policy.a11y_scan import scan_sources
41from froot.policy.compose import (
42 dead_export_pull_request_draft,
43 dead_file_pull_request_draft,
44 pr_labels,
45 pull_request_draft,
46 removal_pull_request_draft,
47)
48from froot.policy.dead_source import unexport_line
49from froot.policy.determinism import analyze_workflow_surface
50from froot.policy.naming import (
51 branch_name,
52 bump_workflow_id,
53 pr_a11y_review_workflow_id,
54 pr_review_workflow_id,
55)
56from froot.policy.review_comment import (
57 REVIEW_MARKER,
58 render_review_comment,
59)
60from froot.policy.review_comment import (
61 should_post as should_post_review,
62)
63from froot.workflow.types import (
64 AdjudicateA11yInput,
65 AdjudicateInput,
66 AutoMergeInput,
67 CiCheckInput,
68 CloseInput,
69 DispatchA11yInput,
70 DispatchInput,
71 DispatchReviewInput,
72 GateReviewInput,
73 GateSelfTestInput,
74 JudgeInput,
75 MergeInput,
76 OpenPrInput,
77 PostA11yInput,
78 PostReviewInput,
79 PrA11yReviewParams,
80 PrReviewParams,
81 ReconcileInput,
82 RecordInput,
83 ScanCandidatesInput,
84)
86if TYPE_CHECKING:
87 from froot.domain.loop import Loop
88 from froot.ports.protocols import PackageManager
90_log = logging.getLogger("froot.outcome")
91_review_log = logging.getLogger("froot.review")
92_a11y_log = logging.getLogger("froot.a11y")
93_reconcile_log = logging.getLogger("froot.reconcile")
94_scan_log = logging.getLogger("froot.scan")
95_gate_log = logging.getLogger("froot.gate")
98def _manifest_dir(target: TargetRepo, workspace: Path) -> Path:
99 """The directory the manifest lives in (a monorepo subdir, or the root)."""
100 return workspace / target.manifest_dir if target.manifest_dir else workspace
103def _draft_for(params: OpenPrInput) -> PullRequestDraft:
104 """Build the PR draft for the work item — bump vs removal compose.
106 The PR-title verb is the loop's, resolved from its registered spec here (the
107 impure boundary) and passed into the pure draft builders.
108 """
109 from froot.loops import registry
111 item = params.candidate
112 title_prefix = registry.commit_tail(params.loop).title_prefix
113 match item:
114 case Candidate():
115 return pull_request_draft(
116 params.target,
117 item,
118 params.verdict,
119 params.loop,
120 title_prefix=title_prefix,
121 )
122 case Removal():
123 return removal_pull_request_draft(
124 params.target, item, params.loop, title_prefix=title_prefix
125 )
126 case DeadFile():
127 return dead_file_pull_request_draft(
128 params.target, item, params.loop, title_prefix=title_prefix
129 )
130 case DeadExport():
131 return dead_export_pull_request_draft(
132 params.target, item, params.loop, title_prefix=title_prefix
133 )
134 assert_never(item)
137def _delete_dead_file(base: Path, item: DeadFile) -> None:
138 """Delete the unused file from the checkout (``push_branch`` stages it).
140 Resolved against ``base`` — the manifest dir the analyzer ran in, so the
141 path matches what the signal flagged. A missing file raises (the same
142 fail-loud as the bump action), never a silent no-op PR.
143 """
144 (base / item.path).unlink()
147def _apply_dead_export(base: Path, item: DeadExport) -> None:
148 """Strip the unused ``export`` from the file at the analyzer's line.
150 Re-validates the line still declares ``symbol`` via the same pure transform
151 the signal narrowed with (:func:`~froot.policy.dead_source.unexport_line`);
152 a mismatch — the source drifted under the signal — raises rather than push
153 an empty diff, so the loop never opens a no-op PR.
154 """
155 path = base / item.file
156 lines = path.read_text().split("\n")
157 index = item.line - 1
158 if not 0 <= index < len(lines):
159 msg = f"{item.file}:{item.line} out of range for {item.symbol!r}"
160 raise RuntimeError(msg)
161 rewritten = unexport_line(lines[index], item.symbol)
162 if rewritten is None:
163 msg = (
164 f"{item.file}:{item.line} is no longer an un-exportable "
165 f"declaration of {item.symbol!r}"
166 )
167 raise RuntimeError(msg)
168 lines[index] = rewritten
169 path.write_text("\n".join(lines))
⋯ 751 lines hidden (lines 171–921)
172async def _select_candidates(
173 loop: Loop,
174 target: TargetRepo,
175 package_manager: PackageManager,
176 manifest_dir: Path,
177) -> tuple[int, tuple[WorkItem, ...]]:
178 """Gather this loop's signal from the checkout and select its work items.
180 The one genuinely per-loop seam: dependency-patch reads the available
181 upgrades and picks the highest patch; security-patch reads the installed,
182 asks OSV for advisories, and picks the lowest version clearing each;
183 dead-code reads the unused dependencies a static analyzer flags and vetoes
184 each with the safe-to-remove judge. The impure sources are lazy-imported per
185 arm so none drags another's stack into a sandbox. Each feeds a pure (or, for
186 dead-code, model-vetoed) selection.
188 Returns ``(considered, items)`` — ``considered`` is the size of the upstream
189 signal (available upgrades / advisories found / unused deps flagged) so the
190 scan can make its selectivity legible (how much was seen versus kept).
192 The one genuinely per-loop body now lives in each loop's spec ``observe``
193 (see :mod:`froot.loops`); the spine looks the loop up and runs it, so this
194 selection seam needs no per-loop arm.
195 """
196 from froot.loops import registry
198 return await registry.commit_tail(loop).observe(
199 target, package_manager, manifest_dir
200 )
203@activity.defn
204async def scan_candidates(
205 params: ScanCandidatesInput,
206) -> tuple[WorkItem, ...]:
207 """Check out the repo and select this loop's candidates.
209 Emits the tick's selectivity — how much upstream signal was considered
210 versus how much was kept — as span attributes (when ``FROOT_OTEL`` is on)
211 and as a structured ``scan_tick`` log, so the signal stage is legible in the
212 run ledger even on a tick that proposes nothing.
213 """
214 from froot.adapters.github import GitHubForge
215 from froot.adapters.registry import package_manager_for
216 from froot.adapters.telemetry import set_span_attributes
218 forge = GitHubForge()
219 package_manager = package_manager_for(params.target.ecosystem)
220 with tempfile.TemporaryDirectory() as tmp:
221 workspace = Path(tmp)
222 await forge.checkout(params.target, workspace)
223 considered, candidates = await _select_candidates(
224 params.loop,
225 params.target,
226 package_manager,
227 _manifest_dir(params.target, workspace),
228 )
229 selected = len(candidates)
230 dropped = max(considered - selected, 0)
231 set_span_attributes(
232 scan_loop=params.loop.value,
233 scan_repo=params.target.repo.slug,
234 scan_considered=considered,
235 scan_selected=selected,
236 scan_dropped=dropped,
237 )
238 _scan_log.info(
239 json.dumps(
240 {
241 "event": "scan_tick",
242 "loop": params.loop.value,
243 "repo": params.target.repo.slug,
244 "considered": considered,
245 "selected": selected,
246 "dropped": dropped,
247 }
248 )
249 )
250 return candidates
253@activity.defn
254async def judge_changelog(params: JudgeInput) -> ChangelogVerdict:
255 """Fetch the candidate's changelog and get the model's typed verdict.
257 The model is froot's one thin, non-load-bearing judgment: a clean verdict
258 never *gates* a PR (CI is the oracle), so a model that is down, slow, or
259 erroring must not stall the spine. A judge failure degrades to
260 ``UnknownVerdict`` — the bump proceeds, the human still gets the PR, and the
261 dashboard records the verdict as unknown — rather than failing (and then
262 retrying) the activity. Only the model call is guarded; the fetch is already
263 best-effort (returns ``None``, not an exception). The loop selects what the
264 model is asked (clean-patch vs breaking-change-on-a-security-bump).
265 """
266 from froot.adapters.changelog_http import HttpChangelogSource
267 from froot.adapters.model_judge import PydanticAiJudge
269 candidate = params.candidate
270 if isinstance(candidate, Removal | DeadFile | DeadExport):
271 # A signal-judged kind (removal / dead file / dead export) already
272 # cleared its safe-to-remove veto at scan; there is no changelog to
273 # read, so carry that conclusion straight into the PR framing (the
274 # rationale the veto recorded on the work item).
275 return CleanVerdict(
276 rationale=candidate.justification or "unused; safe to remove"
277 )
278 changelog = await HttpChangelogSource().fetch(candidate)
279 if changelog is None:
280 return UnknownVerdict(rationale="No changelog could be fetched.")
281 try:
282 return await PydanticAiJudge().judge(changelog, params.loop)
283 except Exception as exc:
284 activity.logger.warning(
285 "changelog judge unavailable for %s; degrading to unknown: %r",
286 params.candidate.subject,
287 exc,
288 )
289 return UnknownVerdict(
290 rationale=f"Changelog judge unavailable ({type(exc).__name__})."
291 )
294@activity.defn
295async def gate_review(params: GateReviewInput) -> ChangelogVerdict:
296 """Independently deep-review a bump at the gate; fail-closed to a hold.
298 The fourth trust leg (§3.7): a second, adversarial model pass over the
299 changelog, run only when a bump is about to auto-merge. ``clean`` approves
300 the merge; ``risky``/``unknown`` hold the PR for the human. Fail-CLOSED — a
301 missing changelog or a model error returns a non-clean verdict, so an
302 unreviewable bump never merges unattended. This is the opposite disposition
303 from :func:`judge_changelog` (which degrades-to-proceed): here the safe
304 direction is to hold, and a non-clean verdict already means hold.
305 """
306 from froot.adapters.changelog_http import HttpChangelogSource
307 from froot.adapters.model_judge import PydanticAiJudge
309 candidate = params.candidate
310 verdict: ChangelogVerdict
311 if isinstance(candidate, Removal):
312 # The fourth leg for a removal: independently re-judge that it is safe
313 # to remove (the same check the scan veto ran). No changelog to read;
314 # fail-CLOSED to a hold on a model error, like the bump path.
315 try:
316 verdict = await PydanticAiJudge().judge_removal(candidate)
317 except Exception as exc:
318 activity.logger.warning(
319 "safe-to-remove gate review unavailable for %s; holding: %r",
320 candidate.subject,
321 exc,
322 )
323 verdict = UnknownVerdict(
324 rationale=(
325 f"Removal reviewer unavailable ({type(exc).__name__})."
326 )
327 )
328 elif isinstance(candidate, DeadFile | DeadExport):
329 # The fourth leg for a dead file / unused export: independently re-judge
330 # the delete / un-export is safe (the scan veto's check). No changelog
331 # to read; fail-CLOSED to a hold on a model error, like the bump path.
332 try:
333 verdict = await PydanticAiJudge().judge_dead_source(candidate)
334 except Exception as exc:
335 activity.logger.warning(
336 "dead-source gate review unavailable for %s; holding: %r",
337 candidate.subject,
338 exc,
339 )
340 verdict = UnknownVerdict(
341 rationale=(
342 f"Dead-source reviewer unavailable ({type(exc).__name__})."
343 )
344 )
345 else:
346 changelog = await HttpChangelogSource().fetch(candidate)
347 if changelog is None:
348 verdict = UnknownVerdict(
349 rationale="No changelog to review; holding (fail-closed)."
350 )
351 else:
352 try:
353 verdict = await PydanticAiJudge().gate_review(
354 changelog, params.loop
355 )
356 except Exception as exc:
357 activity.logger.warning(
358 "gate reviewer unavailable for %s; holding: %r",
359 candidate.package,
360 exc,
361 )
362 verdict = UnknownVerdict(
363 rationale=(
364 f"Gate reviewer unavailable ({type(exc).__name__})."
365 )
366 )
367 _gate_log.info(
368 json.dumps(
369 {
370 "event": "gate_review",
371 "loop": params.loop.value,
372 "pr": params.pr.number,
373 "pr_url": params.pr.url,
374 "package": params.candidate.subject,
375 "verdict": verdict.kind,
376 "approved": verdict.kind == "clean",
377 }
378 )
379 )
380 return verdict
383@activity.defn
384async def open_pull_request(params: OpenPrInput) -> PullRequestRef:
385 """Apply the work item's edit and open (idempotently) its PR.
387 The one place the action differs by work-item kind: a bump regenerates the
388 lockfile at the target version; a removal deletes a dependency (both
389 lockfile-only, no install or scripts); a dead file is deleted whole; a dead
390 export is stripped of its ``export``. Everything else (the dedup branch, the
391 checkout, the push, the open) is the same chassis, and CI is the oracle for
392 every kind.
393 """
394 from froot.adapters.github import GitHubForge
395 from froot.adapters.registry import package_manager_for
397 forge = GitHubForge()
398 package_manager = package_manager_for(params.target.ecosystem)
399 branch = branch_name(params.candidate, params.loop)
400 existing = await forge.find_open_pull_request(params.target, branch)
401 if existing is not None:
402 return existing
403 draft = _draft_for(params)
404 item = params.candidate
405 with tempfile.TemporaryDirectory() as tmp:
406 workspace = Path(tmp)
407 await forge.checkout(params.target, workspace)
408 manifest_dir = _manifest_dir(params.target, workspace)
409 match item:
410 case Candidate():
411 await package_manager.apply_patch_bump(item, manifest_dir)
412 case Removal():
413 await package_manager.remove_dependency(item, manifest_dir)
414 case DeadFile():
415 _delete_dead_file(manifest_dir, item)
416 case DeadExport():
417 _apply_dead_export(manifest_dir, item)
418 await forge.push_branch(workspace, branch, draft.title)
419 return await forge.open_pull_request(params.target, draft)
422@activity.defn
423async def check_ci(params: CiCheckInput) -> CIStatus:
424 """Read the repo's CI status for the PR's head commit (the oracle)."""
425 from froot.adapters.github import GitHubForge
427 return await GitHubForge().ci_status(params.target, params.head_sha)
430@activity.defn
431async def record_outcome(params: RecordInput) -> None:
432 """Label the PR and log the run telemetry — the signal-update.
434 The labels carry the loop *and* the judgment environment (the judge model)
435 the PR was opened under, so the gate can count only the track record earned
436 under the current environment and reset it when the model changes (§3.7).
437 """
438 from froot.adapters.github import GitHubForge
439 from froot.config.settings import ModelSettings
440 from froot.policy.environment import env_label
442 outcome = params.outcome
443 item = outcome.candidate
444 labels = (
445 *pr_labels(params.loop),
446 env_label(ModelSettings().ollama_model),
447 )
448 await GitHubForge().add_labels(params.target, outcome.pr.number, labels)
449 record = {
450 "event": "loop_outcome",
451 "loop": params.loop.value,
452 "repo": params.target.repo.slug,
453 "package": item.subject,
454 "changelog": outcome.verdict.kind,
455 "ci": outcome.ci.kind,
456 "ci_passed": outcome.ci_passed,
457 "pr": outcome.pr.number,
458 "pr_url": outcome.pr.url,
459 }
460 match item:
461 case Candidate():
462 record |= {
463 "action": "bump",
464 "from": str(item.current),
465 "to": str(item.target),
466 }
467 case Removal():
468 record |= {"action": "remove", "dev": item.dev}
469 case DeadFile():
470 record |= {"action": "remove_file", "path": item.path}
471 case DeadExport():
472 record |= {
473 "action": "unexport",
474 "file": item.file,
475 "symbol": item.symbol,
476 }
477 _log.info(json.dumps(record))
480@activity.defn
481async def dispatch_bump(params: DispatchInput) -> None:
482 """Start the bump loop for a candidate (idempotent per bump identity).
484 Reads the close-on-red toggle here, at the impure boundary, and pins it onto
485 the bump's params — so the running workflow never reads config itself and an
486 in-flight bump keeps the value it was dispatched with.
487 """
488 from temporalio.common import WorkflowIDReusePolicy
489 from temporalio.exceptions import WorkflowAlreadyStartedError
491 from froot.config.settings import BehaviorSettings
492 from froot.workflow.bump_workflow import BumpWorkflow
493 from froot.workflow.temporal_client import client, task_queue
494 from froot.workflow.types import BumpParams
496 temporal = await client()
497 try:
498 await temporal.start_workflow(
499 BumpWorkflow.run,
500 BumpParams(
501 target=params.target,
502 candidate=params.candidate,
503 close_on_red=BehaviorSettings().close_on_red,
504 loop=params.loop,
505 ),
506 id=bump_workflow_id(params.target, params.candidate, params.loop),
507 task_queue=task_queue(),
508 id_reuse_policy=WorkflowIDReusePolicy.REJECT_DUPLICATE,
509 )
510 except WorkflowAlreadyStartedError:
511 # This bump already has a loop (running or completed) — a no-op, so
512 # re-scanning never opens a second PR for the same bump.
513 return
516@activity.defn
517async def auto_merge_eligible(params: AutoMergeInput) -> bool:
518 """Whether this (repo, loop) class has earned the auto-merge grant.
520 Short-circuits to ``False`` for any repo a steward has not allowlisted (the
521 default), so the common case costs nothing. Otherwise it re-derives the
522 class's standing from the live GitHub history — the same triangulated,
523 windowed, environment-scoped computation the dashboard's shadow gate shows
524 (``read_model.earned_now``) — so the acting gate and the advisory panel can
525 never disagree. Best-effort: any read failure degrades to ``False`` (hold),
526 never an auto-merge.
527 """
528 from datetime import UTC, datetime
530 from froot.config.settings import AutonomySettings, ModelSettings
531 from froot.dashboard import github_source, read_model
532 from froot.policy.environment import environment_slug
534 policy = AutonomySettings().policy()
535 repo = params.target.repo.slug
536 if repo not in policy.allowlisted_repos:
537 return False
538 now = datetime.now(UTC)
539 prs, prs_error = await github_source.fetch((repo,))
540 if prs_error is not None:
541 return False # can't confirm the record -> hold, never merge blind
542 outcomes, _ = await github_source.fetch_outcomes(
543 (repo,), prs, now=now, window_days=policy.window_days
544 )
545 return read_model.earned_now(
546 now,
547 prs,
548 outcomes,
549 repo,
550 params.loop,
551 policy,
552 environment_slug(ModelSettings().ollama_model),
553 )
556@activity.defn
557async def merge_pull_request(params: MergeInput) -> None:
558 """Auto-merge an earned, clean+green bump's PR (the acting gate's write).
560 Reached only after the pure machine confirmed clean+green and the class
561 earned the grant on an allowlisted repo. Passes the head SHA so GitHub
562 refuses the merge if the head moved since the gate decided.
563 """
564 from froot.adapters.github import GitHubForge
566 await GitHubForge().merge_pull_request(
567 params.target, params.pr.number, head_sha=params.pr.head_sha
568 )
569 _log.info(
570 json.dumps(
571 {
572 "event": "pr_merged",
573 "loop": params.loop.value,
574 "reason": "auto_merge",
575 "repo": params.target.repo.slug,
576 "pr": params.pr.number,
577 "pr_url": params.pr.url,
578 }
579 )
580 )
583@activity.defn
584async def close_pull_request(params: CloseInput) -> None:
585 """Comment why, then close a red bump's PR and delete its branch.
587 The note goes through the idempotent ``upsert_issue_comment`` and the close
588 itself is idempotent, so a retried close edits its comment in place and
589 never double-posts. The bump's record step still runs after this, so the red
590 outcome is logged either way.
591 """
592 from froot.adapters.github import GitHubForge
593 from froot.policy.compose import CLOSE_MARKER, closed_on_red_comment
595 forge = GitHubForge()
596 body = closed_on_red_comment(params.failing)
597 await forge.upsert_issue_comment(
598 params.target, params.pr.number, CLOSE_MARKER, body
599 )
600 await forge.close_pull_request(
601 params.target, params.pr.number, params.pr.branch
602 )
603 _log.info(
604 json.dumps(
605 {
606 "event": "pr_closed",
607 "loop": params.loop.value,
608 "reason": "ci_red",
609 "repo": params.target.repo.slug,
610 "pr": params.pr.number,
611 "pr_url": params.pr.url,
612 "failing": list(params.failing),
613 }
614 )
615 )
618@activity.defn
619async def reconcile_open_prs(params: ReconcileInput) -> int:
620 """Close this loop's PRs that a newer candidate or the base has overtaken.
622 Self-contained: lists the repo's open PRs, re-derives this loop's live
623 candidates (a fresh checkout + the loop's signal, derive-never-store), asks
624 the pure :func:`~froot.policy.reconcile.reconciliations` policy: which of
625 loop's PRs to close, and closes each (deleting its branch). Scoped to the
626 loop's own branch namespace, so the two loops never reconcile each other's
627 PRs. A no-op when reconcile is off (``FROOT_RECONCILE``). Returns the count.
628 """
629 from froot.adapters.github import GitHubForge
630 from froot.adapters.registry import package_manager_for
631 from froot.config.settings import BehaviorSettings
632 from froot.policy.compose import CLOSE_MARKER
633 from froot.policy.reconcile import reconciliations
635 if not BehaviorSettings().reconcile:
636 return 0
637 # Reconcile is version-supersession cleanup, which only bump loops have: a
638 # removal carries no version to be overtaken. A loop that does not reconcile
639 # (dead-code) is skipped rather than re-running its signal (knip + the veto
640 # judge) every tick only to close nothing. (A removal-specific reconcile —
641 # close when no longer unused — is future work.) The trait is on the spec.
642 from froot.loops import registry
644 if not registry.commit_tail(params.loop).reconciles:
645 return 0
647 target, loop = params.target, params.loop
648 forge = GitHubForge()
649 package_manager = package_manager_for(target.ecosystem)
650 open_prs = await forge.list_open_pull_requests(target)
651 with tempfile.TemporaryDirectory() as tmp:
652 workspace = Path(tmp)
653 await forge.checkout(target, workspace)
654 _considered, candidates = await _select_candidates(
655 loop, target, package_manager, _manifest_dir(target, workspace)
656 )
657 closures = reconciliations(open_prs, candidates, loop)
658 for closure in closures:
659 await forge.upsert_issue_comment(
660 target, closure.pr.number, CLOSE_MARKER, closure.comment
661 )
662 await forge.close_pull_request(
663 target, closure.pr.number, closure.pr.branch
664 )
665 if closures:
666 _reconcile_log.info(
667 json.dumps(
668 {
669 "event": "reconcile",
670 "loop": loop.value,
671 "repo": target.repo.slug,
672 "closed": len(closures),
673 "prs": [closure.pr.number for closure in closures],
674 }
675 )
676 )
677 return len(closures)
680@activity.defn
681async def gate_selftest(params: GateSelfTestInput) -> tuple[str, ...]:
682 """Run the adversarial gate probe against the live policy; alarm on escape.
684 The §2.11 deliberate disturbance for the acting gate: a battery of synthetic
685 known-bad class histories a healthy gate must refuse, scored against the
686 policy froot is *actually running* (config and all). Any escape — a bad
687 class the live gate would grant — is logged at ERROR (the alarm) so it
688 surfaces in telemetry the moment config drifts; a clean pass logs a
689 heartbeat at INFO. Returns the escaped scenario names. Pure compute,
690 repo-independent (the ``target``/``loop`` are only the log's context).
691 """
692 from froot.config.settings import AutonomySettings
693 from froot.policy.gate_probe import gate_escapes
695 escaped = gate_escapes(AutonomySettings().policy())
696 record = json.dumps(
697 {
698 "event": "gate_selftest",
699 "loop": params.loop.value,
700 "repo": params.target.repo.slug,
701 "healthy": not escaped,
702 "escaped": list(escaped),
703 }
704 )
705 if escaped:
706 _gate_log.error(record) # an alarm: the gate would trust a bad class
707 else:
708 _gate_log.info(record)
709 return escaped
712# ── The determinism reviewer (the transitive ring) ──────────────────────────
713@activity.defn
714async def list_review_prs(target: TargetRepo) -> tuple[PullRequestRef, ...]:
715 """List the repo's open PRs for the determinism reviewer to consider."""
716 from froot.adapters.github import GitHubForge
718 return await GitHubForge().list_open_pull_requests(target)
721@activity.defn
722async def dispatch_pr_review(params: DispatchReviewInput) -> None:
723 """Start a PR's determinism review (idempotent per PR + head SHA)."""
724 from temporalio.common import WorkflowIDReusePolicy
725 from temporalio.exceptions import WorkflowAlreadyStartedError
727 from froot.workflow.pr_review_workflow import PrReviewWorkflow
728 from froot.workflow.temporal_client import client, task_queue
730 temporal = await client()
731 try:
732 await temporal.start_workflow(
733 PrReviewWorkflow.run,
734 PrReviewParams(target=params.target, pr=params.pr),
735 id=pr_review_workflow_id(
736 params.target, params.pr.number, params.pr.head_sha
737 ),
738 task_queue=task_queue(),
739 id_reuse_policy=WorkflowIDReusePolicy.REJECT_DUPLICATE,
740 )
741 except WorkflowAlreadyStartedError:
742 # This (PR, head SHA) already has a review — a no-op, so re-polling
743 # never double-reviews the same commit.
744 return
747@activity.defn
748async def analyze_pr(params: PrReviewParams) -> AnalysisResult:
749 """Check out the PR head and analyze the workflow surface for hazards."""
750 from froot.adapters.github import GitHubForge
751 from froot.adapters.source_tree import load_modules
752 from froot.config.settings import ReviewSettings
754 forge = GitHubForge()
755 with tempfile.TemporaryDirectory() as tmp:
756 workspace = Path(tmp)
757 await forge.checkout_pull_request(
758 params.target, workspace, params.pr.number
759 )
760 # The ASTs and source lines are read into memory here, so the analysis
761 # below is unaffected by the workspace being cleaned up.
762 modules = load_modules(workspace)
763 return analyze_workflow_surface(modules, max_depth=ReviewSettings().depth)
766@activity.defn
767async def adjudicate_frontier(
768 params: AdjudicateInput,
769) -> tuple[FrontierVerdict, ...]:
770 """Run the model over each frontier item; return aligned verdicts."""
771 from froot.adapters.determinism_judge import DeterminismFrontierJudge
773 judge = DeterminismFrontierJudge()
774 verdicts: list[FrontierVerdict] = []
775 for item in params.frontier:
776 verdicts.append(await judge.adjudicate(item))
777 return tuple(verdicts)
780@activity.defn
781async def post_review(params: PostReviewInput) -> str | None:
782 """Upsert (or clear) the advisory comment; log the ledger row.
784 Posts when there are findings, or when a prior comment must be cleared to
785 "all clear" (true decay — a PR whose hazards were fixed never keeps a stale
786 finding list). A clean PR with no prior comment stays silent.
787 """
788 from froot.adapters.github import GitHubForge
790 forge = GitHubForge()
791 exists = await forge.find_marked_comment(
792 params.target, params.pr.number, REVIEW_MARKER
793 )
794 url: str | None = None
795 if should_post_review(
796 has_findings=bool(params.findings), comment_exists=exists
797 ):
798 body = render_review_comment(params.findings, params.pr.head_sha)
799 url = await forge.upsert_issue_comment(
800 params.target, params.pr.number, REVIEW_MARKER, body
801 )
802 _review_log.info(
803 json.dumps(
804 {
805 "event": "loop_outcome",
806 "loop": "determinism-review",
807 "repo": params.target.repo.slug,
808 "pr": params.pr.number,
809 "head_sha": params.pr.head_sha,
810 "findings": len(params.findings),
811 "rules": sorted({f.rule for f in params.findings}),
812 "comment_url": url,
813 }
814 )
815 )
816 return url
819# ── The a11y reviewer (the source-level design-system ring) ──────────────────
820@activity.defn
821async def scan_pr_a11y(params: PrA11yReviewParams) -> A11yAnalysis:
822 """Check out the PR head and scan its changed templates for a11y risks.
824 Scoped to the PR's changed Vue/JSX templates (shift-left: review what came
825 in), so the advisory stays bounded and high-signal. The source lines are
826 read into memory in the activity, so the pure scan is unaffected by the
827 workspace being cleaned up.
828 """
829 from froot.adapters.github import GitHubForge
830 from froot.adapters.web_source import load_web_sources
832 forge = GitHubForge()
833 changed = await forge.list_pull_request_files(
834 params.target, params.pr.number
835 )
836 with tempfile.TemporaryDirectory() as tmp:
837 workspace = Path(tmp)
838 await forge.checkout_pull_request(
839 params.target, workspace, params.pr.number
840 )
841 sources = load_web_sources(workspace, changed)
842 candidates = scan_sources(sources)
843 return A11yAnalysis(candidates=candidates, scanned_files=len(sources))
846@activity.defn
847async def adjudicate_a11y(
848 params: AdjudicateA11yInput,
849) -> tuple[A11yVerdict, ...]:
850 """Run the model over each flagged candidate; return aligned verdicts."""
851 from froot.adapters.a11y_judge import A11ySourceJudge
853 judge = A11ySourceJudge()
854 verdicts: list[A11yVerdict] = []
855 for candidate in params.candidates:
856 verdicts.append(await judge.adjudicate(candidate))
857 return tuple(verdicts)
860@activity.defn
861async def post_a11y_review(params: PostA11yInput) -> str | None:
862 """Upsert (or clear) the advisory comment; log the ledger row.
864 Posts when there are findings, or when a prior comment must be cleared to
865 "all clear" (true decay — a PR whose gaps were fixed never keeps a stale
866 finding list, the bug the determinism reviewer leaves). A clean PR with
867 no prior comment stays silent, so a clean PR is never spammed.
868 """
869 from froot.adapters.github import GitHubForge
871 forge = GitHubForge()
872 exists = await forge.find_marked_comment(
873 params.target, params.pr.number, A11Y_MARKER
874 )
875 url: str | None = None
876 if should_post(has_findings=bool(params.findings), comment_exists=exists):
877 body = render_a11y_comment(params.findings, params.pr.head_sha)
878 url = await forge.upsert_issue_comment(
879 params.target, params.pr.number, A11Y_MARKER, body
880 )
881 _a11y_log.info(
882 json.dumps(
883 {
884 "event": "loop_outcome",
885 "loop": "a11y-review",
886 "repo": params.target.repo.slug,
887 "pr": params.pr.number,
888 "head_sha": params.pr.head_sha,
889 "findings": len(params.findings),
890 "kinds": sorted({f.kind for f in params.findings}),
891 "comment_url": url,
892 }
893 )
894 )
895 return url
898@activity.defn
899async def dispatch_pr_a11y_review(params: DispatchA11yInput) -> None:
900 """Start a PR's a11y review (idempotent per PR + head SHA)."""
901 from temporalio.common import WorkflowIDReusePolicy
902 from temporalio.exceptions import WorkflowAlreadyStartedError
904 from froot.workflow.pr_a11y_review_workflow import PrA11yReviewWorkflow
905 from froot.workflow.temporal_client import client, task_queue
907 temporal = await client()
908 try:
909 await temporal.start_workflow(
910 PrA11yReviewWorkflow.run,
911 PrA11yReviewParams(target=params.target, pr=params.pr),
912 id=pr_a11y_review_workflow_id(
913 params.target, params.pr.number, params.pr.head_sha
914 ),
915 task_queue=task_queue(),
916 id_reuse_policy=WorkflowIDReusePolicy.REJECT_DUPLICATE,
917 )
918 except WorkflowAlreadyStartedError:
919 # This (PR, head SHA) already has an a11y review — a no-op, so
920 # re-polling never double-reviews the same commit.
921 return
The action dispatch: four kinds, one push, CI the oracle for all of them.
src/froot/workflow/activities.py · 921 lines
src/froot/workflow/activities.py
⋯ 459 lines hidden (lines 1–459)
1"""Activities: the effect interpreters that wrap the adapters.
3Each activity is the impure boundary for one effect. Adapter imports are LAZY
4(inside the bodies) so the model and HTTP stacks never enter a workflow sandbox;
5the pure policy and domain imports stay at module level. Activities return
6domain values — the workflow wraps them into events and feeds the pure state
7machine. The activity signatures' types are evaluated at runtime by Temporal's
8data converter, so the domain imports here are deliberately not deferred.
9"""
11from __future__ import annotations
13import json
14import logging
15import tempfile
16from pathlib import Path
17from typing import TYPE_CHECKING, assert_never
19from temporalio import activity
21from froot.domain.a11y import A11yAnalysis, A11yVerdict
22from froot.domain.candidate import Candidate
23from froot.domain.changelog import (
24 ChangelogVerdict,
25 CleanVerdict,
26 UnknownVerdict,
27)
28from froot.domain.ci import CIStatus
29from froot.domain.dead_source import DeadExport, DeadFile
30from froot.domain.determinism import AnalysisResult, FrontierVerdict
31from froot.domain.pull_request import PullRequestDraft, PullRequestRef
32from froot.domain.removal import Removal
33from froot.domain.repo import TargetRepo
34from froot.domain.work import WorkItem
35from froot.policy.a11y_comment import (
36 A11Y_MARKER,
37 render_a11y_comment,
38 should_post,
39)
40from froot.policy.a11y_scan import scan_sources
41from froot.policy.compose import (
42 dead_export_pull_request_draft,
43 dead_file_pull_request_draft,
44 pr_labels,
45 pull_request_draft,
46 removal_pull_request_draft,
47)
48from froot.policy.dead_source import unexport_line
49from froot.policy.determinism import analyze_workflow_surface
50from froot.policy.naming import (
51 branch_name,
52 bump_workflow_id,
53 pr_a11y_review_workflow_id,
54 pr_review_workflow_id,
55)
56from froot.policy.review_comment import (
57 REVIEW_MARKER,
58 render_review_comment,
59)
60from froot.policy.review_comment import (
61 should_post as should_post_review,
62)
63from froot.workflow.types import (
64 AdjudicateA11yInput,
65 AdjudicateInput,
66 AutoMergeInput,
67 CiCheckInput,
68 CloseInput,
69 DispatchA11yInput,
70 DispatchInput,
71 DispatchReviewInput,
72 GateReviewInput,
73 GateSelfTestInput,
74 JudgeInput,
75 MergeInput,
76 OpenPrInput,
77 PostA11yInput,
78 PostReviewInput,
79 PrA11yReviewParams,
80 PrReviewParams,
81 ReconcileInput,
82 RecordInput,
83 ScanCandidatesInput,
84)
86if TYPE_CHECKING:
87 from froot.domain.loop import Loop
88 from froot.ports.protocols import PackageManager
90_log = logging.getLogger("froot.outcome")
91_review_log = logging.getLogger("froot.review")
92_a11y_log = logging.getLogger("froot.a11y")
93_reconcile_log = logging.getLogger("froot.reconcile")
94_scan_log = logging.getLogger("froot.scan")
95_gate_log = logging.getLogger("froot.gate")
98def _manifest_dir(target: TargetRepo, workspace: Path) -> Path:
99 """The directory the manifest lives in (a monorepo subdir, or the root)."""
100 return workspace / target.manifest_dir if target.manifest_dir else workspace
103def _draft_for(params: OpenPrInput) -> PullRequestDraft:
104 """Build the PR draft for the work item — bump vs removal compose.
106 The PR-title verb is the loop's, resolved from its registered spec here (the
107 impure boundary) and passed into the pure draft builders.
108 """
109 from froot.loops import registry
111 item = params.candidate
112 title_prefix = registry.commit_tail(params.loop).title_prefix
113 match item:
114 case Candidate():
115 return pull_request_draft(
116 params.target,
117 item,
118 params.verdict,
119 params.loop,
120 title_prefix=title_prefix,
121 )
122 case Removal():
123 return removal_pull_request_draft(
124 params.target, item, params.loop, title_prefix=title_prefix
125 )
126 case DeadFile():
127 return dead_file_pull_request_draft(
128 params.target, item, params.loop, title_prefix=title_prefix
129 )
130 case DeadExport():
131 return dead_export_pull_request_draft(
132 params.target, item, params.loop, title_prefix=title_prefix
133 )
134 assert_never(item)
137def _delete_dead_file(base: Path, item: DeadFile) -> None:
138 """Delete the unused file from the checkout (``push_branch`` stages it).
140 Resolved against ``base`` — the manifest dir the analyzer ran in, so the
141 path matches what the signal flagged. A missing file raises (the same
142 fail-loud as the bump action), never a silent no-op PR.
143 """
144 (base / item.path).unlink()
147def _apply_dead_export(base: Path, item: DeadExport) -> None:
148 """Strip the unused ``export`` from the file at the analyzer's line.
150 Re-validates the line still declares ``symbol`` via the same pure transform
151 the signal narrowed with (:func:`~froot.policy.dead_source.unexport_line`);
152 a mismatch — the source drifted under the signal — raises rather than push
153 an empty diff, so the loop never opens a no-op PR.
154 """
155 path = base / item.file
156 lines = path.read_text().split("\n")
157 index = item.line - 1
158 if not 0 <= index < len(lines):
159 msg = f"{item.file}:{item.line} out of range for {item.symbol!r}"
160 raise RuntimeError(msg)
161 rewritten = unexport_line(lines[index], item.symbol)
162 if rewritten is None:
163 msg = (
164 f"{item.file}:{item.line} is no longer an un-exportable "
165 f"declaration of {item.symbol!r}"
166 )
167 raise RuntimeError(msg)
168 lines[index] = rewritten
169 path.write_text("\n".join(lines))
172async def _select_candidates(
173 loop: Loop,
174 target: TargetRepo,
175 package_manager: PackageManager,
176 manifest_dir: Path,
177) -> tuple[int, tuple[WorkItem, ...]]:
178 """Gather this loop's signal from the checkout and select its work items.
180 The one genuinely per-loop seam: dependency-patch reads the available
181 upgrades and picks the highest patch; security-patch reads the installed,
182 asks OSV for advisories, and picks the lowest version clearing each;
183 dead-code reads the unused dependencies a static analyzer flags and vetoes
184 each with the safe-to-remove judge. The impure sources are lazy-imported per
185 arm so none drags another's stack into a sandbox. Each feeds a pure (or, for
186 dead-code, model-vetoed) selection.
188 Returns ``(considered, items)`` — ``considered`` is the size of the upstream
189 signal (available upgrades / advisories found / unused deps flagged) so the
190 scan can make its selectivity legible (how much was seen versus kept).
192 The one genuinely per-loop body now lives in each loop's spec ``observe``
193 (see :mod:`froot.loops`); the spine looks the loop up and runs it, so this
194 selection seam needs no per-loop arm.
195 """
196 from froot.loops import registry
198 return await registry.commit_tail(loop).observe(
199 target, package_manager, manifest_dir
200 )
203@activity.defn
204async def scan_candidates(
205 params: ScanCandidatesInput,
206) -> tuple[WorkItem, ...]:
207 """Check out the repo and select this loop's candidates.
209 Emits the tick's selectivity — how much upstream signal was considered
210 versus how much was kept — as span attributes (when ``FROOT_OTEL`` is on)
211 and as a structured ``scan_tick`` log, so the signal stage is legible in the
212 run ledger even on a tick that proposes nothing.
213 """
214 from froot.adapters.github import GitHubForge
215 from froot.adapters.registry import package_manager_for
216 from froot.adapters.telemetry import set_span_attributes
218 forge = GitHubForge()
219 package_manager = package_manager_for(params.target.ecosystem)
220 with tempfile.TemporaryDirectory() as tmp:
221 workspace = Path(tmp)
222 await forge.checkout(params.target, workspace)
223 considered, candidates = await _select_candidates(
224 params.loop,
225 params.target,
226 package_manager,
227 _manifest_dir(params.target, workspace),
228 )
229 selected = len(candidates)
230 dropped = max(considered - selected, 0)
231 set_span_attributes(
232 scan_loop=params.loop.value,
233 scan_repo=params.target.repo.slug,
234 scan_considered=considered,
235 scan_selected=selected,
236 scan_dropped=dropped,
237 )
238 _scan_log.info(
239 json.dumps(
240 {
241 "event": "scan_tick",
242 "loop": params.loop.value,
243 "repo": params.target.repo.slug,
244 "considered": considered,
245 "selected": selected,
246 "dropped": dropped,
247 }
248 )
249 )
250 return candidates
253@activity.defn
254async def judge_changelog(params: JudgeInput) -> ChangelogVerdict:
255 """Fetch the candidate's changelog and get the model's typed verdict.
257 The model is froot's one thin, non-load-bearing judgment: a clean verdict
258 never *gates* a PR (CI is the oracle), so a model that is down, slow, or
259 erroring must not stall the spine. A judge failure degrades to
260 ``UnknownVerdict`` — the bump proceeds, the human still gets the PR, and the
261 dashboard records the verdict as unknown — rather than failing (and then
262 retrying) the activity. Only the model call is guarded; the fetch is already
263 best-effort (returns ``None``, not an exception). The loop selects what the
264 model is asked (clean-patch vs breaking-change-on-a-security-bump).
265 """
266 from froot.adapters.changelog_http import HttpChangelogSource
267 from froot.adapters.model_judge import PydanticAiJudge
269 candidate = params.candidate
270 if isinstance(candidate, Removal | DeadFile | DeadExport):
271 # A signal-judged kind (removal / dead file / dead export) already
272 # cleared its safe-to-remove veto at scan; there is no changelog to
273 # read, so carry that conclusion straight into the PR framing (the
274 # rationale the veto recorded on the work item).
275 return CleanVerdict(
276 rationale=candidate.justification or "unused; safe to remove"
277 )
278 changelog = await HttpChangelogSource().fetch(candidate)
279 if changelog is None:
280 return UnknownVerdict(rationale="No changelog could be fetched.")
281 try:
282 return await PydanticAiJudge().judge(changelog, params.loop)
283 except Exception as exc:
284 activity.logger.warning(
285 "changelog judge unavailable for %s; degrading to unknown: %r",
286 params.candidate.subject,
287 exc,
288 )
289 return UnknownVerdict(
290 rationale=f"Changelog judge unavailable ({type(exc).__name__})."
291 )
294@activity.defn
295async def gate_review(params: GateReviewInput) -> ChangelogVerdict:
296 """Independently deep-review a bump at the gate; fail-closed to a hold.
298 The fourth trust leg (§3.7): a second, adversarial model pass over the
299 changelog, run only when a bump is about to auto-merge. ``clean`` approves
300 the merge; ``risky``/``unknown`` hold the PR for the human. Fail-CLOSED — a
301 missing changelog or a model error returns a non-clean verdict, so an
302 unreviewable bump never merges unattended. This is the opposite disposition
303 from :func:`judge_changelog` (which degrades-to-proceed): here the safe
304 direction is to hold, and a non-clean verdict already means hold.
305 """
306 from froot.adapters.changelog_http import HttpChangelogSource
307 from froot.adapters.model_judge import PydanticAiJudge
309 candidate = params.candidate
310 verdict: ChangelogVerdict
311 if isinstance(candidate, Removal):
312 # The fourth leg for a removal: independently re-judge that it is safe
313 # to remove (the same check the scan veto ran). No changelog to read;
314 # fail-CLOSED to a hold on a model error, like the bump path.
315 try:
316 verdict = await PydanticAiJudge().judge_removal(candidate)
317 except Exception as exc:
318 activity.logger.warning(
319 "safe-to-remove gate review unavailable for %s; holding: %r",
320 candidate.subject,
321 exc,
322 )
323 verdict = UnknownVerdict(
324 rationale=(
325 f"Removal reviewer unavailable ({type(exc).__name__})."
326 )
327 )
328 elif isinstance(candidate, DeadFile | DeadExport):
329 # The fourth leg for a dead file / unused export: independently re-judge
330 # the delete / un-export is safe (the scan veto's check). No changelog
331 # to read; fail-CLOSED to a hold on a model error, like the bump path.
332 try:
333 verdict = await PydanticAiJudge().judge_dead_source(candidate)
334 except Exception as exc:
335 activity.logger.warning(
336 "dead-source gate review unavailable for %s; holding: %r",
337 candidate.subject,
338 exc,
339 )
340 verdict = UnknownVerdict(
341 rationale=(
342 f"Dead-source reviewer unavailable ({type(exc).__name__})."
343 )
344 )
345 else:
346 changelog = await HttpChangelogSource().fetch(candidate)
347 if changelog is None:
348 verdict = UnknownVerdict(
349 rationale="No changelog to review; holding (fail-closed)."
350 )
351 else:
352 try:
353 verdict = await PydanticAiJudge().gate_review(
354 changelog, params.loop
355 )
356 except Exception as exc:
357 activity.logger.warning(
358 "gate reviewer unavailable for %s; holding: %r",
359 candidate.package,
360 exc,
361 )
362 verdict = UnknownVerdict(
363 rationale=(
364 f"Gate reviewer unavailable ({type(exc).__name__})."
365 )
366 )
367 _gate_log.info(
368 json.dumps(
369 {
370 "event": "gate_review",
371 "loop": params.loop.value,
372 "pr": params.pr.number,
373 "pr_url": params.pr.url,
374 "package": params.candidate.subject,
375 "verdict": verdict.kind,
376 "approved": verdict.kind == "clean",
377 }
378 )
379 )
380 return verdict
383@activity.defn
384async def open_pull_request(params: OpenPrInput) -> PullRequestRef:
385 """Apply the work item's edit and open (idempotently) its PR.
387 The one place the action differs by work-item kind: a bump regenerates the
388 lockfile at the target version; a removal deletes a dependency (both
389 lockfile-only, no install or scripts); a dead file is deleted whole; a dead
390 export is stripped of its ``export``. Everything else (the dedup branch, the
391 checkout, the push, the open) is the same chassis, and CI is the oracle for
392 every kind.
393 """
394 from froot.adapters.github import GitHubForge
395 from froot.adapters.registry import package_manager_for
397 forge = GitHubForge()
398 package_manager = package_manager_for(params.target.ecosystem)
399 branch = branch_name(params.candidate, params.loop)
400 existing = await forge.find_open_pull_request(params.target, branch)
401 if existing is not None:
402 return existing
403 draft = _draft_for(params)
404 item = params.candidate
405 with tempfile.TemporaryDirectory() as tmp:
406 workspace = Path(tmp)
407 await forge.checkout(params.target, workspace)
408 manifest_dir = _manifest_dir(params.target, workspace)
409 match item:
410 case Candidate():
411 await package_manager.apply_patch_bump(item, manifest_dir)
412 case Removal():
413 await package_manager.remove_dependency(item, manifest_dir)
414 case DeadFile():
415 _delete_dead_file(manifest_dir, item)
416 case DeadExport():
417 _apply_dead_export(manifest_dir, item)
418 await forge.push_branch(workspace, branch, draft.title)
419 return await forge.open_pull_request(params.target, draft)
422@activity.defn
423async def check_ci(params: CiCheckInput) -> CIStatus:
424 """Read the repo's CI status for the PR's head commit (the oracle)."""
425 from froot.adapters.github import GitHubForge
427 return await GitHubForge().ci_status(params.target, params.head_sha)
430@activity.defn
431async def record_outcome(params: RecordInput) -> None:
432 """Label the PR and log the run telemetry — the signal-update.
434 The labels carry the loop *and* the judgment environment (the judge model)
435 the PR was opened under, so the gate can count only the track record earned
436 under the current environment and reset it when the model changes (§3.7).
437 """
438 from froot.adapters.github import GitHubForge
439 from froot.config.settings import ModelSettings
440 from froot.policy.environment import env_label
442 outcome = params.outcome
443 item = outcome.candidate
444 labels = (
445 *pr_labels(params.loop),
446 env_label(ModelSettings().ollama_model),
447 )
448 await GitHubForge().add_labels(params.target, outcome.pr.number, labels)
449 record = {
450 "event": "loop_outcome",
451 "loop": params.loop.value,
452 "repo": params.target.repo.slug,
453 "package": item.subject,
454 "changelog": outcome.verdict.kind,
455 "ci": outcome.ci.kind,
456 "ci_passed": outcome.ci_passed,
457 "pr": outcome.pr.number,
458 "pr_url": outcome.pr.url,
459 }
460 match item:
461 case Candidate():
462 record |= {
463 "action": "bump",
464 "from": str(item.current),
465 "to": str(item.target),
466 }
467 case Removal():
468 record |= {"action": "remove", "dev": item.dev}
469 case DeadFile():
470 record |= {"action": "remove_file", "path": item.path}
471 case DeadExport():
472 record |= {
473 "action": "unexport",
⋯ 448 lines hidden (lines 474–921)
474 "file": item.file,
475 "symbol": item.symbol,
476 }
477 _log.info(json.dumps(record))
480@activity.defn
481async def dispatch_bump(params: DispatchInput) -> None:
482 """Start the bump loop for a candidate (idempotent per bump identity).
484 Reads the close-on-red toggle here, at the impure boundary, and pins it onto
485 the bump's params — so the running workflow never reads config itself and an
486 in-flight bump keeps the value it was dispatched with.
487 """
488 from temporalio.common import WorkflowIDReusePolicy
489 from temporalio.exceptions import WorkflowAlreadyStartedError
491 from froot.config.settings import BehaviorSettings
492 from froot.workflow.bump_workflow import BumpWorkflow
493 from froot.workflow.temporal_client import client, task_queue
494 from froot.workflow.types import BumpParams
496 temporal = await client()
497 try:
498 await temporal.start_workflow(
499 BumpWorkflow.run,
500 BumpParams(
501 target=params.target,
502 candidate=params.candidate,
503 close_on_red=BehaviorSettings().close_on_red,
504 loop=params.loop,
505 ),
506 id=bump_workflow_id(params.target, params.candidate, params.loop),
507 task_queue=task_queue(),
508 id_reuse_policy=WorkflowIDReusePolicy.REJECT_DUPLICATE,
509 )
510 except WorkflowAlreadyStartedError:
511 # This bump already has a loop (running or completed) — a no-op, so
512 # re-scanning never opens a second PR for the same bump.
513 return
516@activity.defn
517async def auto_merge_eligible(params: AutoMergeInput) -> bool:
518 """Whether this (repo, loop) class has earned the auto-merge grant.
520 Short-circuits to ``False`` for any repo a steward has not allowlisted (the
521 default), so the common case costs nothing. Otherwise it re-derives the
522 class's standing from the live GitHub history — the same triangulated,
523 windowed, environment-scoped computation the dashboard's shadow gate shows
524 (``read_model.earned_now``) — so the acting gate and the advisory panel can
525 never disagree. Best-effort: any read failure degrades to ``False`` (hold),
526 never an auto-merge.
527 """
528 from datetime import UTC, datetime
530 from froot.config.settings import AutonomySettings, ModelSettings
531 from froot.dashboard import github_source, read_model
532 from froot.policy.environment import environment_slug
534 policy = AutonomySettings().policy()
535 repo = params.target.repo.slug
536 if repo not in policy.allowlisted_repos:
537 return False
538 now = datetime.now(UTC)
539 prs, prs_error = await github_source.fetch((repo,))
540 if prs_error is not None:
541 return False # can't confirm the record -> hold, never merge blind
542 outcomes, _ = await github_source.fetch_outcomes(
543 (repo,), prs, now=now, window_days=policy.window_days
544 )
545 return read_model.earned_now(
546 now,
547 prs,
548 outcomes,
549 repo,
550 params.loop,
551 policy,
552 environment_slug(ModelSettings().ollama_model),
553 )
556@activity.defn
557async def merge_pull_request(params: MergeInput) -> None:
558 """Auto-merge an earned, clean+green bump's PR (the acting gate's write).
560 Reached only after the pure machine confirmed clean+green and the class
561 earned the grant on an allowlisted repo. Passes the head SHA so GitHub
562 refuses the merge if the head moved since the gate decided.
563 """
564 from froot.adapters.github import GitHubForge
566 await GitHubForge().merge_pull_request(
567 params.target, params.pr.number, head_sha=params.pr.head_sha
568 )
569 _log.info(
570 json.dumps(
571 {
572 "event": "pr_merged",
573 "loop": params.loop.value,
574 "reason": "auto_merge",
575 "repo": params.target.repo.slug,
576 "pr": params.pr.number,
577 "pr_url": params.pr.url,
578 }
579 )
580 )
583@activity.defn
584async def close_pull_request(params: CloseInput) -> None:
585 """Comment why, then close a red bump's PR and delete its branch.
587 The note goes through the idempotent ``upsert_issue_comment`` and the close
588 itself is idempotent, so a retried close edits its comment in place and
589 never double-posts. The bump's record step still runs after this, so the red
590 outcome is logged either way.
591 """
592 from froot.adapters.github import GitHubForge
593 from froot.policy.compose import CLOSE_MARKER, closed_on_red_comment
595 forge = GitHubForge()
596 body = closed_on_red_comment(params.failing)
597 await forge.upsert_issue_comment(
598 params.target, params.pr.number, CLOSE_MARKER, body
599 )
600 await forge.close_pull_request(
601 params.target, params.pr.number, params.pr.branch
602 )
603 _log.info(
604 json.dumps(
605 {
606 "event": "pr_closed",
607 "loop": params.loop.value,
608 "reason": "ci_red",
609 "repo": params.target.repo.slug,
610 "pr": params.pr.number,
611 "pr_url": params.pr.url,
612 "failing": list(params.failing),
613 }
614 )
615 )
618@activity.defn
619async def reconcile_open_prs(params: ReconcileInput) -> int:
620 """Close this loop's PRs that a newer candidate or the base has overtaken.
622 Self-contained: lists the repo's open PRs, re-derives this loop's live
623 candidates (a fresh checkout + the loop's signal, derive-never-store), asks
624 the pure :func:`~froot.policy.reconcile.reconciliations` policy: which of
625 loop's PRs to close, and closes each (deleting its branch). Scoped to the
626 loop's own branch namespace, so the two loops never reconcile each other's
627 PRs. A no-op when reconcile is off (``FROOT_RECONCILE``). Returns the count.
628 """
629 from froot.adapters.github import GitHubForge
630 from froot.adapters.registry import package_manager_for
631 from froot.config.settings import BehaviorSettings
632 from froot.policy.compose import CLOSE_MARKER
633 from froot.policy.reconcile import reconciliations
635 if not BehaviorSettings().reconcile:
636 return 0
637 # Reconcile is version-supersession cleanup, which only bump loops have: a
638 # removal carries no version to be overtaken. A loop that does not reconcile
639 # (dead-code) is skipped rather than re-running its signal (knip + the veto
640 # judge) every tick only to close nothing. (A removal-specific reconcile —
641 # close when no longer unused — is future work.) The trait is on the spec.
642 from froot.loops import registry
644 if not registry.commit_tail(params.loop).reconciles:
645 return 0
647 target, loop = params.target, params.loop
648 forge = GitHubForge()
649 package_manager = package_manager_for(target.ecosystem)
650 open_prs = await forge.list_open_pull_requests(target)
651 with tempfile.TemporaryDirectory() as tmp:
652 workspace = Path(tmp)
653 await forge.checkout(target, workspace)
654 _considered, candidates = await _select_candidates(
655 loop, target, package_manager, _manifest_dir(target, workspace)
656 )
657 closures = reconciliations(open_prs, candidates, loop)
658 for closure in closures:
659 await forge.upsert_issue_comment(
660 target, closure.pr.number, CLOSE_MARKER, closure.comment
661 )
662 await forge.close_pull_request(
663 target, closure.pr.number, closure.pr.branch
664 )
665 if closures:
666 _reconcile_log.info(
667 json.dumps(
668 {
669 "event": "reconcile",
670 "loop": loop.value,
671 "repo": target.repo.slug,
672 "closed": len(closures),
673 "prs": [closure.pr.number for closure in closures],
674 }
675 )
676 )
677 return len(closures)
680@activity.defn
681async def gate_selftest(params: GateSelfTestInput) -> tuple[str, ...]:
682 """Run the adversarial gate probe against the live policy; alarm on escape.
684 The §2.11 deliberate disturbance for the acting gate: a battery of synthetic
685 known-bad class histories a healthy gate must refuse, scored against the
686 policy froot is *actually running* (config and all). Any escape — a bad
687 class the live gate would grant — is logged at ERROR (the alarm) so it
688 surfaces in telemetry the moment config drifts; a clean pass logs a
689 heartbeat at INFO. Returns the escaped scenario names. Pure compute,
690 repo-independent (the ``target``/``loop`` are only the log's context).
691 """
692 from froot.config.settings import AutonomySettings
693 from froot.policy.gate_probe import gate_escapes
695 escaped = gate_escapes(AutonomySettings().policy())
696 record = json.dumps(
697 {
698 "event": "gate_selftest",
699 "loop": params.loop.value,
700 "repo": params.target.repo.slug,
701 "healthy": not escaped,
702 "escaped": list(escaped),
703 }
704 )
705 if escaped:
706 _gate_log.error(record) # an alarm: the gate would trust a bad class
707 else:
708 _gate_log.info(record)
709 return escaped
712# ── The determinism reviewer (the transitive ring) ──────────────────────────
713@activity.defn
714async def list_review_prs(target: TargetRepo) -> tuple[PullRequestRef, ...]:
715 """List the repo's open PRs for the determinism reviewer to consider."""
716 from froot.adapters.github import GitHubForge
718 return await GitHubForge().list_open_pull_requests(target)
721@activity.defn
722async def dispatch_pr_review(params: DispatchReviewInput) -> None:
723 """Start a PR's determinism review (idempotent per PR + head SHA)."""
724 from temporalio.common import WorkflowIDReusePolicy
725 from temporalio.exceptions import WorkflowAlreadyStartedError
727 from froot.workflow.pr_review_workflow import PrReviewWorkflow
728 from froot.workflow.temporal_client import client, task_queue
730 temporal = await client()
731 try:
732 await temporal.start_workflow(
733 PrReviewWorkflow.run,
734 PrReviewParams(target=params.target, pr=params.pr),
735 id=pr_review_workflow_id(
736 params.target, params.pr.number, params.pr.head_sha
737 ),
738 task_queue=task_queue(),
739 id_reuse_policy=WorkflowIDReusePolicy.REJECT_DUPLICATE,
740 )
741 except WorkflowAlreadyStartedError:
742 # This (PR, head SHA) already has a review — a no-op, so re-polling
743 # never double-reviews the same commit.
744 return
747@activity.defn
748async def analyze_pr(params: PrReviewParams) -> AnalysisResult:
749 """Check out the PR head and analyze the workflow surface for hazards."""
750 from froot.adapters.github import GitHubForge
751 from froot.adapters.source_tree import load_modules
752 from froot.config.settings import ReviewSettings
754 forge = GitHubForge()
755 with tempfile.TemporaryDirectory() as tmp:
756 workspace = Path(tmp)
757 await forge.checkout_pull_request(
758 params.target, workspace, params.pr.number
759 )
760 # The ASTs and source lines are read into memory here, so the analysis
761 # below is unaffected by the workspace being cleaned up.
762 modules = load_modules(workspace)
763 return analyze_workflow_surface(modules, max_depth=ReviewSettings().depth)
766@activity.defn
767async def adjudicate_frontier(
768 params: AdjudicateInput,
769) -> tuple[FrontierVerdict, ...]:
770 """Run the model over each frontier item; return aligned verdicts."""
771 from froot.adapters.determinism_judge import DeterminismFrontierJudge
773 judge = DeterminismFrontierJudge()
774 verdicts: list[FrontierVerdict] = []
775 for item in params.frontier:
776 verdicts.append(await judge.adjudicate(item))
777 return tuple(verdicts)
780@activity.defn
781async def post_review(params: PostReviewInput) -> str | None:
782 """Upsert (or clear) the advisory comment; log the ledger row.
784 Posts when there are findings, or when a prior comment must be cleared to
785 "all clear" (true decay — a PR whose hazards were fixed never keeps a stale
786 finding list). A clean PR with no prior comment stays silent.
787 """
788 from froot.adapters.github import GitHubForge
790 forge = GitHubForge()
791 exists = await forge.find_marked_comment(
792 params.target, params.pr.number, REVIEW_MARKER
793 )
794 url: str | None = None
795 if should_post_review(
796 has_findings=bool(params.findings), comment_exists=exists
797 ):
798 body = render_review_comment(params.findings, params.pr.head_sha)
799 url = await forge.upsert_issue_comment(
800 params.target, params.pr.number, REVIEW_MARKER, body
801 )
802 _review_log.info(
803 json.dumps(
804 {
805 "event": "loop_outcome",
806 "loop": "determinism-review",
807 "repo": params.target.repo.slug,
808 "pr": params.pr.number,
809 "head_sha": params.pr.head_sha,
810 "findings": len(params.findings),
811 "rules": sorted({f.rule for f in params.findings}),
812 "comment_url": url,
813 }
814 )
815 )
816 return url
819# ── The a11y reviewer (the source-level design-system ring) ──────────────────
820@activity.defn
821async def scan_pr_a11y(params: PrA11yReviewParams) -> A11yAnalysis:
822 """Check out the PR head and scan its changed templates for a11y risks.
824 Scoped to the PR's changed Vue/JSX templates (shift-left: review what came
825 in), so the advisory stays bounded and high-signal. The source lines are
826 read into memory in the activity, so the pure scan is unaffected by the
827 workspace being cleaned up.
828 """
829 from froot.adapters.github import GitHubForge
830 from froot.adapters.web_source import load_web_sources
832 forge = GitHubForge()
833 changed = await forge.list_pull_request_files(
834 params.target, params.pr.number
835 )
836 with tempfile.TemporaryDirectory() as tmp:
837 workspace = Path(tmp)
838 await forge.checkout_pull_request(
839 params.target, workspace, params.pr.number
840 )
841 sources = load_web_sources(workspace, changed)
842 candidates = scan_sources(sources)
843 return A11yAnalysis(candidates=candidates, scanned_files=len(sources))
846@activity.defn
847async def adjudicate_a11y(
848 params: AdjudicateA11yInput,
849) -> tuple[A11yVerdict, ...]:
850 """Run the model over each flagged candidate; return aligned verdicts."""
851 from froot.adapters.a11y_judge import A11ySourceJudge
853 judge = A11ySourceJudge()
854 verdicts: list[A11yVerdict] = []
855 for candidate in params.candidates:
856 verdicts.append(await judge.adjudicate(candidate))
857 return tuple(verdicts)
860@activity.defn
861async def post_a11y_review(params: PostA11yInput) -> str | None:
862 """Upsert (or clear) the advisory comment; log the ledger row.
864 Posts when there are findings, or when a prior comment must be cleared to
865 "all clear" (true decay — a PR whose gaps were fixed never keeps a stale
866 finding list, the bug the determinism reviewer leaves). A clean PR with
867 no prior comment stays silent, so a clean PR is never spammed.
868 """
869 from froot.adapters.github import GitHubForge
871 forge = GitHubForge()
872 exists = await forge.find_marked_comment(
873 params.target, params.pr.number, A11Y_MARKER
874 )
875 url: str | None = None
876 if should_post(has_findings=bool(params.findings), comment_exists=exists):
877 body = render_a11y_comment(params.findings, params.pr.head_sha)
878 url = await forge.upsert_issue_comment(
879 params.target, params.pr.number, A11Y_MARKER, body
880 )
881 _a11y_log.info(
882 json.dumps(
883 {
884 "event": "loop_outcome",
885 "loop": "a11y-review",
886 "repo": params.target.repo.slug,
887 "pr": params.pr.number,
888 "head_sha": params.pr.head_sha,
889 "findings": len(params.findings),
890 "kinds": sorted({f.kind for f in params.findings}),
891 "comment_url": url,
892 }
893 )
894 )
895 return url
898@activity.defn
899async def dispatch_pr_a11y_review(params: DispatchA11yInput) -> None:
900 """Start a PR's a11y review (idempotent per PR + head SHA)."""
901 from temporalio.common import WorkflowIDReusePolicy
902 from temporalio.exceptions import WorkflowAlreadyStartedError
904 from froot.workflow.pr_a11y_review_workflow import PrA11yReviewWorkflow
905 from froot.workflow.temporal_client import client, task_queue
907 temporal = await client()
908 try:
909 await temporal.start_workflow(
910 PrA11yReviewWorkflow.run,
911 PrA11yReviewParams(target=params.target, pr=params.pr),
912 id=pr_a11y_review_workflow_id(
913 params.target, params.pr.number, params.pr.head_sha
914 ),
915 task_queue=task_queue(),
916 id_reuse_policy=WorkflowIDReusePolicy.REJECT_DUPLICATE,
917 )
918 except WorkflowAlreadyStartedError:
919 # This (PR, head SHA) already has an a11y review — a no-op, so
920 # re-polling never double-reviews the same commit.
921 return
The signal knip already had
The cheapest part of the change. knip maps the whole import graph and reports unused files and exports in the same JSON run the adapter was already making for unused dependencies — it just parsed the dependency section and dropped the rest. Two small parsers read the files array and the per-file exports (and types, which un-export identically). Then the exports are narrowed against the real source in the checkout, and only the editable ones survive.
Two parsers over the same knip JSON; defensive, never a raise.
src/froot/adapters/npm.py · 431 lines
src/froot/adapters/npm.py
⋯ 169 lines hidden (lines 1–169)
1"""The npm package-manager adapter.
3Reads upgrade availability from the committed manifest + lockfile (no install
4needed) plus ``npm view <pkg> versions``, and regenerates the manifest +
5lockfile with ``npm install <pkg>@<target> --package-lock-only
6--ignore-scripts`` — lockfile-only, no install scripts, so no third-party
7dependency code ever runs in the worker (the real install + tests happen in the
8repo's CI, the oracle).
10The installed baseline comes from ``package-lock.json``, *not* ``npm
11outdated``: ``npm outdated``'s ``current`` field is absent without a
12``node_modules`` tree, and froot only does a clone (never an install). The
13parsing is split into pure module functions so it is unit-tested with fixtures,
14away from the subprocess and the network.
15"""
17from __future__ import annotations
19import json
20import logging
21from typing import TYPE_CHECKING
23from froot.adapters._proc import run_text
24from froot.domain.candidate import AvailableUpgrade, InstalledPackage
25from froot.domain.dead_source import DeadExport, DeadFile
26from froot.domain.removal import Removal
27from froot.domain.version import Version
28from froot.policy.dead_source import unexport_line
29from froot.result import Ok
31if TYPE_CHECKING:
32 from pathlib import Path
34 from froot.domain.candidate import Candidate
35 from froot.domain.ecosystem import Ecosystem
36 from froot.domain.repo import TargetRepo
38_NODE_MODULES = "node_modules/"
40_log = logging.getLogger("froot.scan")
43def parse_direct_dependencies(package_json: str) -> frozenset[str]:
44 """The direct dependency names from a ``package.json`` (deps + devDeps).
46 Only direct dependencies are bumped: ``npm install <pkg>`` on a transitive
47 dependency would promote it to a direct one, which is not a patch.
48 """
49 data = json.loads(package_json)
50 names: set[str] = set()
51 if isinstance(data, dict):
52 for field in ("dependencies", "devDependencies"):
53 section = data.get(field)
54 if isinstance(section, dict):
55 names.update(name for name in section if isinstance(name, str))
56 return frozenset(names)
59def parse_locked_versions(package_lock: str) -> dict[str, str]:
60 """Resolved version per top-level dependency from a ``package-lock.json``.
62 Reads the lockfileVersion 2/3 ``packages`` map (keys
63 ``node_modules/<name>``, skipping nested ``.../node_modules/...`` transitive
64 entries), falling back to the legacy v1 top-level ``dependencies`` map.
65 """
66 data = json.loads(package_lock)
67 if not isinstance(data, dict):
68 return {}
69 versions: dict[str, str] = {}
70 packages = data.get("packages")
71 if isinstance(packages, dict):
72 for key, info in packages.items():
73 if not (isinstance(key, str) and key.startswith(_NODE_MODULES)):
74 continue
75 name = key[len(_NODE_MODULES) :]
76 if "/node_modules/" in name: # a nested (transitive) entry
77 continue
78 version = info.get("version") if isinstance(info, dict) else None
79 if isinstance(version, str):
80 versions[name] = version
81 if versions:
82 return versions
83 legacy = data.get("dependencies") # lockfileVersion 1
84 if isinstance(legacy, dict):
85 for name, info in legacy.items():
86 version = info.get("version") if isinstance(info, dict) else None
87 if isinstance(name, str) and isinstance(version, str):
88 versions[name] = version
89 return versions
92def parse_versions(stdout: str) -> tuple[Version, ...]:
93 """Parse ``npm view <pkg> versions --json`` into domain versions.
95 Accepts a JSON array (the usual case) or a bare JSON string (a single
96 version); empty or non-JSON output (e.g. a failed lookup) yields ``()``.
97 Unparseable entries are dropped.
98 """
99 if not stdout.strip():
100 return ()
101 try:
102 raw = json.loads(stdout)
103 except json.JSONDecodeError:
104 return ()
105 items = raw if isinstance(raw, list) else [raw]
106 versions: list[Version] = []
107 for item in items:
108 if isinstance(item, str):
109 match Version.parse(item):
110 case Ok(version):
111 versions.append(version)
112 case _:
113 continue
114 return tuple(versions)
117def _loads_embedded_json(stdout: str) -> object:
118 """``json.loads`` ``stdout``, tolerating leading plugin chatter.
120 knip's plugins may print progress lines to stdout *before* the JSON report,
121 so a plain parse can fail; this retries from the first ``{``. Empty or
122 unparseable input yields ``None``.
123 """
124 text = stdout.strip()
125 if not text:
126 return None
127 try:
128 return json.loads(text)
129 except json.JSONDecodeError:
130 pass
131 brace = text.find("{")
132 if brace < 0:
133 return None
134 try:
135 return json.loads(text[brace:])
136 except json.JSONDecodeError:
137 return None
140def parse_knip_unused(stdout: str) -> tuple[tuple[str, bool], ...]:
141 """Parse ``knip --reporter json`` into ``(package, dev)`` unused flags.
143 knip groups issues by file; each issue's ``dependencies`` lists unused
144 production deps and ``devDependencies`` unused dev deps, every entry an
145 object with a ``name``. Returns one ``(name, dev)`` pair per unused
146 dependency (``dev`` True for a devDependency). Empty or unparseable output
147 yields ``()`` — conservative: no flags, never a raise.
148 """
149 data = _loads_embedded_json(stdout)
150 if not isinstance(data, dict):
151 return ()
152 issues = data.get("issues")
153 if not isinstance(issues, list):
154 return ()
155 flags: list[tuple[str, bool]] = []
156 for issue in issues:
157 if not isinstance(issue, dict):
158 continue
159 for field, dev in (("dependencies", False), ("devDependencies", True)):
160 section = issue.get(field)
161 if not isinstance(section, list):
162 continue
163 for entry in section:
164 name = entry.get("name") if isinstance(entry, dict) else None
165 if isinstance(name, str) and name:
166 flags.append((name, dev))
167 return tuple(flags)
170def parse_knip_files(stdout: str) -> tuple[str, ...]:
171 """Parse ``knip``'s top-level ``files`` — whole modules nothing imports.
173 knip reports unused files as a top-level array of repo-relative paths.
174 Empty or unparseable output yields ``()`` — conservative, never a raise.
175 """
176 data = _loads_embedded_json(stdout)
177 if not isinstance(data, dict):
178 return ()
179 files = data.get("files")
180 if not isinstance(files, list):
181 return ()
182 return tuple(path for path in files if isinstance(path, str) and path)
185def parse_knip_exports(stdout: str) -> tuple[tuple[str, str, int], ...]:
186 """Parse ``knip`` unused ``exports`` + ``types`` to ``(file, name, line)``.
188 knip groups issues by file; ``exports`` (values) and ``types`` (type-only)
189 both list symbols exported but imported by no other module, each an object
190 with a ``name`` and a 1-based ``line``. The two are un-exported identically,
191 so they fold together. Entries missing a usable name+line are skipped.
192 """
193 data = _loads_embedded_json(stdout)
194 if not isinstance(data, dict):
195 return ()
196 issues = data.get("issues")
197 if not isinstance(issues, list):
198 return ()
199 found: list[tuple[str, str, int]] = []
200 for issue in issues:
201 if not isinstance(issue, dict):
202 continue
203 file = issue.get("file")
204 if not isinstance(file, str) or not file:
205 continue
206 for field in ("exports", "types"):
207 section = issue.get(field)
208 if not isinstance(section, list):
209 continue
210 for entry in section:
211 if not isinstance(entry, dict):
212 continue
213 name = entry.get("name")
214 line = entry.get("line")
215 if (
216 isinstance(name, str)
217 and name
218 and isinstance(line, int)
219 and line > 0
220 ):
221 found.append((file, name, line))
222 return tuple(found)
⋯ 209 lines hidden (lines 223–431)
225def narrow_unexportable(
226 workspace: Path,
227 exports: tuple[tuple[str, str, int], ...],
228 ecosystem: Ecosystem,
229) -> tuple[tuple[DeadExport, ...], int]:
230 """Keep only exports the action can un-export; count what was dropped.
232 Reads the flagged source line and keeps the export iff it is an inline named
233 declaration the pure transform handles (:func:`unexport_line` returns the
234 stripped line). Clause re-exports, ``export default``, and ``export *`` are
235 dropped — they need an AST, not a one-line edit — as are entries whose file
236 or line cannot be read. Returns ``(kept, dropped)`` so the caller can make
237 the drop legible rather than silently swallow it.
238 """
239 kept: list[DeadExport] = []
240 dropped = 0
241 for file, symbol, line in exports:
242 try:
243 source = (workspace / file).read_text()
244 except OSError:
245 dropped += 1
246 continue
247 lines = source.split("\n")
248 index = line - 1
249 if not (0 <= index < len(lines)) or (
250 unexport_line(lines[index], symbol) is None
251 ):
252 dropped += 1
253 continue
254 kept.append(
255 DeadExport(
256 file=file,
257 symbol=symbol,
258 line=line,
259 ecosystem=ecosystem,
260 justification="unused export (knip)",
261 )
262 )
263 return tuple(kept), dropped
266class NpmPackageManager:
267 """A :class:`~froot.ports.protocols.PackageManager` backed by ``npm``."""
269 async def list_upgrades(
270 self, target: TargetRepo, workspace: Path
271 ) -> tuple[AvailableUpgrade, ...]:
272 """Report each direct dependency and the versions available to it."""
273 direct = parse_direct_dependencies(
274 (workspace / "package.json").read_text()
275 )
276 lock_path = workspace / "package-lock.json"
277 locked = (
278 parse_locked_versions(lock_path.read_text())
279 if lock_path.exists()
280 else {}
281 )
282 upgrades: list[AvailableUpgrade] = []
283 for name in sorted(direct):
284 current_text = locked.get(name)
285 if current_text is None:
286 continue
287 match Version.parse(current_text):
288 case Ok(current):
289 pass
290 case _:
291 continue
292 _, versions_out, _ = await run_text(
293 "npm", "view", name, "versions", "--json", cwd=workspace
294 )
295 upgrades.append(
296 AvailableUpgrade(
297 package=name,
298 ecosystem=target.ecosystem,
299 current=current,
300 available=parse_versions(versions_out),
301 )
302 )
303 return tuple(upgrades)
305 async def list_installed(
306 self, target: TargetRepo, workspace: Path
307 ) -> tuple[InstalledPackage, ...]:
308 """Report each direct dependency and its locked version (no network)."""
309 direct = parse_direct_dependencies(
310 (workspace / "package.json").read_text()
311 )
312 lock_path = workspace / "package-lock.json"
313 locked = (
314 parse_locked_versions(lock_path.read_text())
315 if lock_path.exists()
316 else {}
317 )
318 installed: list[InstalledPackage] = []
319 for name in sorted(direct):
320 current_text = locked.get(name)
321 if current_text is None:
322 continue
323 match Version.parse(current_text):
324 case Ok(version):
325 installed.append(
326 InstalledPackage(
327 package=name,
328 ecosystem=target.ecosystem,
329 version=version,
330 )
331 )
332 case _:
333 continue
334 return tuple(installed)
336 async def apply_patch_bump(
337 self, candidate: Candidate, workspace: Path
338 ) -> None:
339 """Rewrite the manifest + lockfile to the target (lockfile-only)."""
340 code, out, err = await run_text(
341 "npm",
342 "install",
343 f"{candidate.package}@{candidate.target}",
344 "--package-lock-only",
345 "--ignore-scripts",
346 cwd=workspace,
347 )
348 if code != 0:
349 raise RuntimeError(f"npm install failed ({code}): {err or out}")
351 async def list_unused(
352 self, target: TargetRepo, workspace: Path
353 ) -> tuple[Removal | DeadFile | DeadExport, ...]:
354 """Report the dead code ``knip`` flags: deps, files, and exports.
356 One ``knip`` run surfaces three shapes of dead weight: unused direct
357 dependencies (a :class:`Removal`), whole unused files (a
358 :class:`DeadFile`), and exports no other module imports (a
359 :class:`DeadExport`, un-exported in place). Exports are narrowed to the
360 inline forms the action can edit (the rest are dropped, the count
361 logged, so a skipped export is never mistaken for "none found").
363 Best-effort: ``knip`` exits non-zero precisely *because* it found
364 issues, so the exit code is ignored and stdout is parsed regardless;
365 crashed or empty output simply yields nothing. ``knip`` is baked into
366 the worker image and on ``PATH``; if it is absent (e.g. local dev), the
367 signal degrades to nothing rather than raising. ``justification``
368 records the detector so the judge and PR body name it.
369 """
370 try:
371 _, out, _ = await run_text(
372 "knip",
373 "--reporter",
374 "json",
375 "--no-progress",
376 cwd=workspace,
377 )
378 except FileNotFoundError:
379 return ()
380 items: list[Removal | DeadFile | DeadExport] = [
381 Removal(
382 package=name,
383 ecosystem=target.ecosystem,
384 dev=dev,
385 justification="unused (knip)",
386 )
387 for name, dev in parse_knip_unused(out)
388 ]
389 items.extend(
390 DeadFile(
391 path=path,
392 ecosystem=target.ecosystem,
393 justification="unused file (knip)",
394 )
395 for path in parse_knip_files(out)
396 )
397 exports, dropped = narrow_unexportable(
398 workspace, parse_knip_exports(out), target.ecosystem
399 )
400 items.extend(exports)
401 if dropped:
402 _log.info(
403 json.dumps(
404 {
405 "event": "knip_exports_dropped",
406 "repo": target.repo.slug,
407 "dropped": dropped,
408 }
409 )
410 )
411 return tuple(items)
413 async def remove_dependency(
414 self, removal: Removal, workspace: Path
415 ) -> None:
416 """Remove the dependency from package.json + lockfile (lockfile-only).
418 ``npm uninstall`` finds the dependency's section itself, so ``dev`` need
419 not be passed; ``--package-lock-only --ignore-scripts`` keeps it to a
420 manifest+lock rewrite with no ``node_modules`` and no scripts.
421 """
422 code, out, err = await run_text(
423 "npm",
424 "uninstall",
425 removal.package,
426 "--package-lock-only",
427 "--ignore-scripts",
428 cwd=workspace,
429 )
430 if code != 0:
431 raise RuntimeError(f"npm uninstall failed ({code}): {err or out}")
Narrowing reads the flagged line and keeps only what the transform can edit; it counts the drops.
src/froot/adapters/npm.py · 431 lines
src/froot/adapters/npm.py
⋯ 224 lines hidden (lines 1–224)
1"""The npm package-manager adapter.
3Reads upgrade availability from the committed manifest + lockfile (no install
4needed) plus ``npm view <pkg> versions``, and regenerates the manifest +
5lockfile with ``npm install <pkg>@<target> --package-lock-only
6--ignore-scripts`` — lockfile-only, no install scripts, so no third-party
7dependency code ever runs in the worker (the real install + tests happen in the
8repo's CI, the oracle).
10The installed baseline comes from ``package-lock.json``, *not* ``npm
11outdated``: ``npm outdated``'s ``current`` field is absent without a
12``node_modules`` tree, and froot only does a clone (never an install). The
13parsing is split into pure module functions so it is unit-tested with fixtures,
14away from the subprocess and the network.
15"""
17from __future__ import annotations
19import json
20import logging
21from typing import TYPE_CHECKING
23from froot.adapters._proc import run_text
24from froot.domain.candidate import AvailableUpgrade, InstalledPackage
25from froot.domain.dead_source import DeadExport, DeadFile
26from froot.domain.removal import Removal
27from froot.domain.version import Version
28from froot.policy.dead_source import unexport_line
29from froot.result import Ok
31if TYPE_CHECKING:
32 from pathlib import Path
34 from froot.domain.candidate import Candidate
35 from froot.domain.ecosystem import Ecosystem
36 from froot.domain.repo import TargetRepo
38_NODE_MODULES = "node_modules/"
40_log = logging.getLogger("froot.scan")
43def parse_direct_dependencies(package_json: str) -> frozenset[str]:
44 """The direct dependency names from a ``package.json`` (deps + devDeps).
46 Only direct dependencies are bumped: ``npm install <pkg>`` on a transitive
47 dependency would promote it to a direct one, which is not a patch.
48 """
49 data = json.loads(package_json)
50 names: set[str] = set()
51 if isinstance(data, dict):
52 for field in ("dependencies", "devDependencies"):
53 section = data.get(field)
54 if isinstance(section, dict):
55 names.update(name for name in section if isinstance(name, str))
56 return frozenset(names)
59def parse_locked_versions(package_lock: str) -> dict[str, str]:
60 """Resolved version per top-level dependency from a ``package-lock.json``.
62 Reads the lockfileVersion 2/3 ``packages`` map (keys
63 ``node_modules/<name>``, skipping nested ``.../node_modules/...`` transitive
64 entries), falling back to the legacy v1 top-level ``dependencies`` map.
65 """
66 data = json.loads(package_lock)
67 if not isinstance(data, dict):
68 return {}
69 versions: dict[str, str] = {}
70 packages = data.get("packages")
71 if isinstance(packages, dict):
72 for key, info in packages.items():
73 if not (isinstance(key, str) and key.startswith(_NODE_MODULES)):
74 continue
75 name = key[len(_NODE_MODULES) :]
76 if "/node_modules/" in name: # a nested (transitive) entry
77 continue
78 version = info.get("version") if isinstance(info, dict) else None
79 if isinstance(version, str):
80 versions[name] = version
81 if versions:
82 return versions
83 legacy = data.get("dependencies") # lockfileVersion 1
84 if isinstance(legacy, dict):
85 for name, info in legacy.items():
86 version = info.get("version") if isinstance(info, dict) else None
87 if isinstance(name, str) and isinstance(version, str):
88 versions[name] = version
89 return versions
92def parse_versions(stdout: str) -> tuple[Version, ...]:
93 """Parse ``npm view <pkg> versions --json`` into domain versions.
95 Accepts a JSON array (the usual case) or a bare JSON string (a single
96 version); empty or non-JSON output (e.g. a failed lookup) yields ``()``.
97 Unparseable entries are dropped.
98 """
99 if not stdout.strip():
100 return ()
101 try:
102 raw = json.loads(stdout)
103 except json.JSONDecodeError:
104 return ()
105 items = raw if isinstance(raw, list) else [raw]
106 versions: list[Version] = []
107 for item in items:
108 if isinstance(item, str):
109 match Version.parse(item):
110 case Ok(version):
111 versions.append(version)
112 case _:
113 continue
114 return tuple(versions)
117def _loads_embedded_json(stdout: str) -> object:
118 """``json.loads`` ``stdout``, tolerating leading plugin chatter.
120 knip's plugins may print progress lines to stdout *before* the JSON report,
121 so a plain parse can fail; this retries from the first ``{``. Empty or
122 unparseable input yields ``None``.
123 """
124 text = stdout.strip()
125 if not text:
126 return None
127 try:
128 return json.loads(text)
129 except json.JSONDecodeError:
130 pass
131 brace = text.find("{")
132 if brace < 0:
133 return None
134 try:
135 return json.loads(text[brace:])
136 except json.JSONDecodeError:
137 return None
140def parse_knip_unused(stdout: str) -> tuple[tuple[str, bool], ...]:
141 """Parse ``knip --reporter json`` into ``(package, dev)`` unused flags.
143 knip groups issues by file; each issue's ``dependencies`` lists unused
144 production deps and ``devDependencies`` unused dev deps, every entry an
145 object with a ``name``. Returns one ``(name, dev)`` pair per unused
146 dependency (``dev`` True for a devDependency). Empty or unparseable output
147 yields ``()`` — conservative: no flags, never a raise.
148 """
149 data = _loads_embedded_json(stdout)
150 if not isinstance(data, dict):
151 return ()
152 issues = data.get("issues")
153 if not isinstance(issues, list):
154 return ()
155 flags: list[tuple[str, bool]] = []
156 for issue in issues:
157 if not isinstance(issue, dict):
158 continue
159 for field, dev in (("dependencies", False), ("devDependencies", True)):
160 section = issue.get(field)
161 if not isinstance(section, list):
162 continue
163 for entry in section:
164 name = entry.get("name") if isinstance(entry, dict) else None
165 if isinstance(name, str) and name:
166 flags.append((name, dev))
167 return tuple(flags)
170def parse_knip_files(stdout: str) -> tuple[str, ...]:
171 """Parse ``knip``'s top-level ``files`` — whole modules nothing imports.
173 knip reports unused files as a top-level array of repo-relative paths.
174 Empty or unparseable output yields ``()`` — conservative, never a raise.
175 """
176 data = _loads_embedded_json(stdout)
177 if not isinstance(data, dict):
178 return ()
179 files = data.get("files")
180 if not isinstance(files, list):
181 return ()
182 return tuple(path for path in files if isinstance(path, str) and path)
185def parse_knip_exports(stdout: str) -> tuple[tuple[str, str, int], ...]:
186 """Parse ``knip`` unused ``exports`` + ``types`` to ``(file, name, line)``.
188 knip groups issues by file; ``exports`` (values) and ``types`` (type-only)
189 both list symbols exported but imported by no other module, each an object
190 with a ``name`` and a 1-based ``line``. The two are un-exported identically,
191 so they fold together. Entries missing a usable name+line are skipped.
192 """
193 data = _loads_embedded_json(stdout)
194 if not isinstance(data, dict):
195 return ()
196 issues = data.get("issues")
197 if not isinstance(issues, list):
198 return ()
199 found: list[tuple[str, str, int]] = []
200 for issue in issues:
201 if not isinstance(issue, dict):
202 continue
203 file = issue.get("file")
204 if not isinstance(file, str) or not file:
205 continue
206 for field in ("exports", "types"):
207 section = issue.get(field)
208 if not isinstance(section, list):
209 continue
210 for entry in section:
211 if not isinstance(entry, dict):
212 continue
213 name = entry.get("name")
214 line = entry.get("line")
215 if (
216 isinstance(name, str)
217 and name
218 and isinstance(line, int)
219 and line > 0
220 ):
221 found.append((file, name, line))
222 return tuple(found)
225def narrow_unexportable(
226 workspace: Path,
227 exports: tuple[tuple[str, str, int], ...],
228 ecosystem: Ecosystem,
229) -> tuple[tuple[DeadExport, ...], int]:
230 """Keep only exports the action can un-export; count what was dropped.
232 Reads the flagged source line and keeps the export iff it is an inline named
233 declaration the pure transform handles (:func:`unexport_line` returns the
234 stripped line). Clause re-exports, ``export default``, and ``export *`` are
235 dropped — they need an AST, not a one-line edit — as are entries whose file
236 or line cannot be read. Returns ``(kept, dropped)`` so the caller can make
237 the drop legible rather than silently swallow it.
238 """
239 kept: list[DeadExport] = []
240 dropped = 0
241 for file, symbol, line in exports:
242 try:
243 source = (workspace / file).read_text()
244 except OSError:
245 dropped += 1
246 continue
247 lines = source.split("\n")
248 index = line - 1
249 if not (0 <= index < len(lines)) or (
250 unexport_line(lines[index], symbol) is None
251 ):
252 dropped += 1
253 continue
254 kept.append(
255 DeadExport(
256 file=file,
257 symbol=symbol,
258 line=line,
259 ecosystem=ecosystem,
260 justification="unused export (knip)",
261 )
262 )
263 return tuple(kept), dropped
⋯ 168 lines hidden (lines 264–431)
266class NpmPackageManager:
267 """A :class:`~froot.ports.protocols.PackageManager` backed by ``npm``."""
269 async def list_upgrades(
270 self, target: TargetRepo, workspace: Path
271 ) -> tuple[AvailableUpgrade, ...]:
272 """Report each direct dependency and the versions available to it."""
273 direct = parse_direct_dependencies(
274 (workspace / "package.json").read_text()
275 )
276 lock_path = workspace / "package-lock.json"
277 locked = (
278 parse_locked_versions(lock_path.read_text())
279 if lock_path.exists()
280 else {}
281 )
282 upgrades: list[AvailableUpgrade] = []
283 for name in sorted(direct):
284 current_text = locked.get(name)
285 if current_text is None:
286 continue
287 match Version.parse(current_text):
288 case Ok(current):
289 pass
290 case _:
291 continue
292 _, versions_out, _ = await run_text(
293 "npm", "view", name, "versions", "--json", cwd=workspace
294 )
295 upgrades.append(
296 AvailableUpgrade(
297 package=name,
298 ecosystem=target.ecosystem,
299 current=current,
300 available=parse_versions(versions_out),
301 )
302 )
303 return tuple(upgrades)
305 async def list_installed(
306 self, target: TargetRepo, workspace: Path
307 ) -> tuple[InstalledPackage, ...]:
308 """Report each direct dependency and its locked version (no network)."""
309 direct = parse_direct_dependencies(
310 (workspace / "package.json").read_text()
311 )
312 lock_path = workspace / "package-lock.json"
313 locked = (
314 parse_locked_versions(lock_path.read_text())
315 if lock_path.exists()
316 else {}
317 )
318 installed: list[InstalledPackage] = []
319 for name in sorted(direct):
320 current_text = locked.get(name)
321 if current_text is None:
322 continue
323 match Version.parse(current_text):
324 case Ok(version):
325 installed.append(
326 InstalledPackage(
327 package=name,
328 ecosystem=target.ecosystem,
329 version=version,
330 )
331 )
332 case _:
333 continue
334 return tuple(installed)
336 async def apply_patch_bump(
337 self, candidate: Candidate, workspace: Path
338 ) -> None:
339 """Rewrite the manifest + lockfile to the target (lockfile-only)."""
340 code, out, err = await run_text(
341 "npm",
342 "install",
343 f"{candidate.package}@{candidate.target}",
344 "--package-lock-only",
345 "--ignore-scripts",
346 cwd=workspace,
347 )
348 if code != 0:
349 raise RuntimeError(f"npm install failed ({code}): {err or out}")
351 async def list_unused(
352 self, target: TargetRepo, workspace: Path
353 ) -> tuple[Removal | DeadFile | DeadExport, ...]:
354 """Report the dead code ``knip`` flags: deps, files, and exports.
356 One ``knip`` run surfaces three shapes of dead weight: unused direct
357 dependencies (a :class:`Removal`), whole unused files (a
358 :class:`DeadFile`), and exports no other module imports (a
359 :class:`DeadExport`, un-exported in place). Exports are narrowed to the
360 inline forms the action can edit (the rest are dropped, the count
361 logged, so a skipped export is never mistaken for "none found").
363 Best-effort: ``knip`` exits non-zero precisely *because* it found
364 issues, so the exit code is ignored and stdout is parsed regardless;
365 crashed or empty output simply yields nothing. ``knip`` is baked into
366 the worker image and on ``PATH``; if it is absent (e.g. local dev), the
367 signal degrades to nothing rather than raising. ``justification``
368 records the detector so the judge and PR body name it.
369 """
370 try:
371 _, out, _ = await run_text(
372 "knip",
373 "--reporter",
374 "json",
375 "--no-progress",
376 cwd=workspace,
377 )
378 except FileNotFoundError:
379 return ()
380 items: list[Removal | DeadFile | DeadExport] = [
381 Removal(
382 package=name,
383 ecosystem=target.ecosystem,
384 dev=dev,
385 justification="unused (knip)",
386 )
387 for name, dev in parse_knip_unused(out)
388 ]
389 items.extend(
390 DeadFile(
391 path=path,
392 ecosystem=target.ecosystem,
393 justification="unused file (knip)",
394 )
395 for path in parse_knip_files(out)
396 )
397 exports, dropped = narrow_unexportable(
398 workspace, parse_knip_exports(out), target.ecosystem
399 )
400 items.extend(exports)
401 if dropped:
402 _log.info(
403 json.dumps(
404 {
405 "event": "knip_exports_dropped",
406 "repo": target.repo.slug,
407 "dropped": dropped,
408 }
409 )
410 )
411 return tuple(items)
413 async def remove_dependency(
414 self, removal: Removal, workspace: Path
415 ) -> None:
416 """Remove the dependency from package.json + lockfile (lockfile-only).
418 ``npm uninstall`` finds the dependency's section itself, so ``dev`` need
419 not be passed; ``--package-lock-only --ignore-scripts`` keeps it to a
420 manifest+lock rewrite with no ``node_modules`` and no scripts.
421 """
422 code, out, err = await run_text(
423 "npm",
424 "uninstall",
425 removal.package,
426 "--package-lock-only",
427 "--ignore-scripts",
428 cwd=workspace,
429 )
430 if code != 0:
431 raise RuntimeError(f"npm uninstall failed ({code}): {err or out}")
One run, three kinds. The dropped-export count is logged, not swallowed.
src/froot/adapters/npm.py · 431 lines
src/froot/adapters/npm.py
⋯ 350 lines hidden (lines 1–350)
1"""The npm package-manager adapter.
3Reads upgrade availability from the committed manifest + lockfile (no install
4needed) plus ``npm view <pkg> versions``, and regenerates the manifest +
5lockfile with ``npm install <pkg>@<target> --package-lock-only
6--ignore-scripts`` — lockfile-only, no install scripts, so no third-party
7dependency code ever runs in the worker (the real install + tests happen in the
8repo's CI, the oracle).
10The installed baseline comes from ``package-lock.json``, *not* ``npm
11outdated``: ``npm outdated``'s ``current`` field is absent without a
12``node_modules`` tree, and froot only does a clone (never an install). The
13parsing is split into pure module functions so it is unit-tested with fixtures,
14away from the subprocess and the network.
15"""
17from __future__ import annotations
19import json
20import logging
21from typing import TYPE_CHECKING
23from froot.adapters._proc import run_text
24from froot.domain.candidate import AvailableUpgrade, InstalledPackage
25from froot.domain.dead_source import DeadExport, DeadFile
26from froot.domain.removal import Removal
27from froot.domain.version import Version
28from froot.policy.dead_source import unexport_line
29from froot.result import Ok
31if TYPE_CHECKING:
32 from pathlib import Path
34 from froot.domain.candidate import Candidate
35 from froot.domain.ecosystem import Ecosystem
36 from froot.domain.repo import TargetRepo
38_NODE_MODULES = "node_modules/"
40_log = logging.getLogger("froot.scan")
43def parse_direct_dependencies(package_json: str) -> frozenset[str]:
44 """The direct dependency names from a ``package.json`` (deps + devDeps).
46 Only direct dependencies are bumped: ``npm install <pkg>`` on a transitive
47 dependency would promote it to a direct one, which is not a patch.
48 """
49 data = json.loads(package_json)
50 names: set[str] = set()
51 if isinstance(data, dict):
52 for field in ("dependencies", "devDependencies"):
53 section = data.get(field)
54 if isinstance(section, dict):
55 names.update(name for name in section if isinstance(name, str))
56 return frozenset(names)
59def parse_locked_versions(package_lock: str) -> dict[str, str]:
60 """Resolved version per top-level dependency from a ``package-lock.json``.
62 Reads the lockfileVersion 2/3 ``packages`` map (keys
63 ``node_modules/<name>``, skipping nested ``.../node_modules/...`` transitive
64 entries), falling back to the legacy v1 top-level ``dependencies`` map.
65 """
66 data = json.loads(package_lock)
67 if not isinstance(data, dict):
68 return {}
69 versions: dict[str, str] = {}
70 packages = data.get("packages")
71 if isinstance(packages, dict):
72 for key, info in packages.items():
73 if not (isinstance(key, str) and key.startswith(_NODE_MODULES)):
74 continue
75 name = key[len(_NODE_MODULES) :]
76 if "/node_modules/" in name: # a nested (transitive) entry
77 continue
78 version = info.get("version") if isinstance(info, dict) else None
79 if isinstance(version, str):
80 versions[name] = version
81 if versions:
82 return versions
83 legacy = data.get("dependencies") # lockfileVersion 1
84 if isinstance(legacy, dict):
85 for name, info in legacy.items():
86 version = info.get("version") if isinstance(info, dict) else None
87 if isinstance(name, str) and isinstance(version, str):
88 versions[name] = version
89 return versions
92def parse_versions(stdout: str) -> tuple[Version, ...]:
93 """Parse ``npm view <pkg> versions --json`` into domain versions.
95 Accepts a JSON array (the usual case) or a bare JSON string (a single
96 version); empty or non-JSON output (e.g. a failed lookup) yields ``()``.
97 Unparseable entries are dropped.
98 """
99 if not stdout.strip():
100 return ()
101 try:
102 raw = json.loads(stdout)
103 except json.JSONDecodeError:
104 return ()
105 items = raw if isinstance(raw, list) else [raw]
106 versions: list[Version] = []
107 for item in items:
108 if isinstance(item, str):
109 match Version.parse(item):
110 case Ok(version):
111 versions.append(version)
112 case _:
113 continue
114 return tuple(versions)
117def _loads_embedded_json(stdout: str) -> object:
118 """``json.loads`` ``stdout``, tolerating leading plugin chatter.
120 knip's plugins may print progress lines to stdout *before* the JSON report,
121 so a plain parse can fail; this retries from the first ``{``. Empty or
122 unparseable input yields ``None``.
123 """
124 text = stdout.strip()
125 if not text:
126 return None
127 try:
128 return json.loads(text)
129 except json.JSONDecodeError:
130 pass
131 brace = text.find("{")
132 if brace < 0:
133 return None
134 try:
135 return json.loads(text[brace:])
136 except json.JSONDecodeError:
137 return None
140def parse_knip_unused(stdout: str) -> tuple[tuple[str, bool], ...]:
141 """Parse ``knip --reporter json`` into ``(package, dev)`` unused flags.
143 knip groups issues by file; each issue's ``dependencies`` lists unused
144 production deps and ``devDependencies`` unused dev deps, every entry an
145 object with a ``name``. Returns one ``(name, dev)`` pair per unused
146 dependency (``dev`` True for a devDependency). Empty or unparseable output
147 yields ``()`` — conservative: no flags, never a raise.
148 """
149 data = _loads_embedded_json(stdout)
150 if not isinstance(data, dict):
151 return ()
152 issues = data.get("issues")
153 if not isinstance(issues, list):
154 return ()
155 flags: list[tuple[str, bool]] = []
156 for issue in issues:
157 if not isinstance(issue, dict):
158 continue
159 for field, dev in (("dependencies", False), ("devDependencies", True)):
160 section = issue.get(field)
161 if not isinstance(section, list):
162 continue
163 for entry in section:
164 name = entry.get("name") if isinstance(entry, dict) else None
165 if isinstance(name, str) and name:
166 flags.append((name, dev))
167 return tuple(flags)
170def parse_knip_files(stdout: str) -> tuple[str, ...]:
171 """Parse ``knip``'s top-level ``files`` — whole modules nothing imports.
173 knip reports unused files as a top-level array of repo-relative paths.
174 Empty or unparseable output yields ``()`` — conservative, never a raise.
175 """
176 data = _loads_embedded_json(stdout)
177 if not isinstance(data, dict):
178 return ()
179 files = data.get("files")
180 if not isinstance(files, list):
181 return ()
182 return tuple(path for path in files if isinstance(path, str) and path)
185def parse_knip_exports(stdout: str) -> tuple[tuple[str, str, int], ...]:
186 """Parse ``knip`` unused ``exports`` + ``types`` to ``(file, name, line)``.
188 knip groups issues by file; ``exports`` (values) and ``types`` (type-only)
189 both list symbols exported but imported by no other module, each an object
190 with a ``name`` and a 1-based ``line``. The two are un-exported identically,
191 so they fold together. Entries missing a usable name+line are skipped.
192 """
193 data = _loads_embedded_json(stdout)
194 if not isinstance(data, dict):
195 return ()
196 issues = data.get("issues")
197 if not isinstance(issues, list):
198 return ()
199 found: list[tuple[str, str, int]] = []
200 for issue in issues:
201 if not isinstance(issue, dict):
202 continue
203 file = issue.get("file")
204 if not isinstance(file, str) or not file:
205 continue
206 for field in ("exports", "types"):
207 section = issue.get(field)
208 if not isinstance(section, list):
209 continue
210 for entry in section:
211 if not isinstance(entry, dict):
212 continue
213 name = entry.get("name")
214 line = entry.get("line")
215 if (
216 isinstance(name, str)
217 and name
218 and isinstance(line, int)
219 and line > 0
220 ):
221 found.append((file, name, line))
222 return tuple(found)
225def narrow_unexportable(
226 workspace: Path,
227 exports: tuple[tuple[str, str, int], ...],
228 ecosystem: Ecosystem,
229) -> tuple[tuple[DeadExport, ...], int]:
230 """Keep only exports the action can un-export; count what was dropped.
232 Reads the flagged source line and keeps the export iff it is an inline named
233 declaration the pure transform handles (:func:`unexport_line` returns the
234 stripped line). Clause re-exports, ``export default``, and ``export *`` are
235 dropped — they need an AST, not a one-line edit — as are entries whose file
236 or line cannot be read. Returns ``(kept, dropped)`` so the caller can make
237 the drop legible rather than silently swallow it.
238 """
239 kept: list[DeadExport] = []
240 dropped = 0
241 for file, symbol, line in exports:
242 try:
243 source = (workspace / file).read_text()
244 except OSError:
245 dropped += 1
246 continue
247 lines = source.split("\n")
248 index = line - 1
249 if not (0 <= index < len(lines)) or (
250 unexport_line(lines[index], symbol) is None
251 ):
252 dropped += 1
253 continue
254 kept.append(
255 DeadExport(
256 file=file,
257 symbol=symbol,
258 line=line,
259 ecosystem=ecosystem,
260 justification="unused export (knip)",
261 )
262 )
263 return tuple(kept), dropped
266class NpmPackageManager:
267 """A :class:`~froot.ports.protocols.PackageManager` backed by ``npm``."""
269 async def list_upgrades(
270 self, target: TargetRepo, workspace: Path
271 ) -> tuple[AvailableUpgrade, ...]:
272 """Report each direct dependency and the versions available to it."""
273 direct = parse_direct_dependencies(
274 (workspace / "package.json").read_text()
275 )
276 lock_path = workspace / "package-lock.json"
277 locked = (
278 parse_locked_versions(lock_path.read_text())
279 if lock_path.exists()
280 else {}
281 )
282 upgrades: list[AvailableUpgrade] = []
283 for name in sorted(direct):
284 current_text = locked.get(name)
285 if current_text is None:
286 continue
287 match Version.parse(current_text):
288 case Ok(current):
289 pass
290 case _:
291 continue
292 _, versions_out, _ = await run_text(
293 "npm", "view", name, "versions", "--json", cwd=workspace
294 )
295 upgrades.append(
296 AvailableUpgrade(
297 package=name,
298 ecosystem=target.ecosystem,
299 current=current,
300 available=parse_versions(versions_out),
301 )
302 )
303 return tuple(upgrades)
305 async def list_installed(
306 self, target: TargetRepo, workspace: Path
307 ) -> tuple[InstalledPackage, ...]:
308 """Report each direct dependency and its locked version (no network)."""
309 direct = parse_direct_dependencies(
310 (workspace / "package.json").read_text()
311 )
312 lock_path = workspace / "package-lock.json"
313 locked = (
314 parse_locked_versions(lock_path.read_text())
315 if lock_path.exists()
316 else {}
317 )
318 installed: list[InstalledPackage] = []
319 for name in sorted(direct):
320 current_text = locked.get(name)
321 if current_text is None:
322 continue
323 match Version.parse(current_text):
324 case Ok(version):
325 installed.append(
326 InstalledPackage(
327 package=name,
328 ecosystem=target.ecosystem,
329 version=version,
330 )
331 )
332 case _:
333 continue
334 return tuple(installed)
336 async def apply_patch_bump(
337 self, candidate: Candidate, workspace: Path
338 ) -> None:
339 """Rewrite the manifest + lockfile to the target (lockfile-only)."""
340 code, out, err = await run_text(
341 "npm",
342 "install",
343 f"{candidate.package}@{candidate.target}",
344 "--package-lock-only",
345 "--ignore-scripts",
346 cwd=workspace,
347 )
348 if code != 0:
349 raise RuntimeError(f"npm install failed ({code}): {err or out}")
351 async def list_unused(
352 self, target: TargetRepo, workspace: Path
353 ) -> tuple[Removal | DeadFile | DeadExport, ...]:
354 """Report the dead code ``knip`` flags: deps, files, and exports.
356 One ``knip`` run surfaces three shapes of dead weight: unused direct
357 dependencies (a :class:`Removal`), whole unused files (a
358 :class:`DeadFile`), and exports no other module imports (a
359 :class:`DeadExport`, un-exported in place). Exports are narrowed to the
360 inline forms the action can edit (the rest are dropped, the count
361 logged, so a skipped export is never mistaken for "none found").
363 Best-effort: ``knip`` exits non-zero precisely *because* it found
364 issues, so the exit code is ignored and stdout is parsed regardless;
365 crashed or empty output simply yields nothing. ``knip`` is baked into
366 the worker image and on ``PATH``; if it is absent (e.g. local dev), the
367 signal degrades to nothing rather than raising. ``justification``
368 records the detector so the judge and PR body name it.
369 """
370 try:
371 _, out, _ = await run_text(
372 "knip",
373 "--reporter",
374 "json",
375 "--no-progress",
376 cwd=workspace,
377 )
378 except FileNotFoundError:
379 return ()
380 items: list[Removal | DeadFile | DeadExport] = [
381 Removal(
382 package=name,
383 ecosystem=target.ecosystem,
384 dev=dev,
385 justification="unused (knip)",
386 )
387 for name, dev in parse_knip_unused(out)
388 ]
389 items.extend(
390 DeadFile(
391 path=path,
392 ecosystem=target.ecosystem,
393 justification="unused file (knip)",
394 )
395 for path in parse_knip_files(out)
396 )
397 exports, dropped = narrow_unexportable(
398 workspace, parse_knip_exports(out), target.ecosystem
399 )
400 items.extend(exports)
401 if dropped:
402 _log.info(
403 json.dumps(
404 {
405 "event": "knip_exports_dropped",
406 "repo": target.repo.slug,
407 "dropped": dropped,
408 }
409 )
410 )
411 return tuple(items)
⋯ 20 lines hidden (lines 412–431)
413 async def remove_dependency(
414 self, removal: Removal, workspace: Path
415 ) -> None:
416 """Remove the dependency from package.json + lockfile (lockfile-only).
418 ``npm uninstall`` finds the dependency's section itself, so ``dev`` need
419 not be passed; ``--package-lock-only --ignore-scripts`` keeps it to a
420 manifest+lock rewrite with no ``node_modules`` and no scripts.
421 """
422 code, out, err = await run_text(
423 "npm",
424 "uninstall",
425 removal.package,
426 "--package-lock-only",
427 "--ignore-scripts",
428 cwd=workspace,
429 )
430 if code != 0:
431 raise RuntimeError(f"npm uninstall failed ({code}): {err or out}")
The veto routes by kind
Every flagged item still passes a safe-to-remove veto at the signal, before a workflow ever starts — "not imported" is not "dead" for a dependency (think pytest, eslint) and it is not "dead" for source either (a framework entry loaded by convention, a route or plugin file). The loop now routes a dependency to the dependency judge and a file or export to the source judge; only a clean verdict becomes a PR.
One loop, two vetoes, dispatched by kind; survivors carry the judge's reasoning into the PR body.
src/froot/loops/dead_code.py · 92 lines
src/froot/loops/dead_code.py
⋯ 43 lines hidden (lines 1–43)
1"""The dead-code loop: remove the dead weight a static analyzer flags.
3Signal: a static analyzer (npm via knip) flags three shapes of dead code — an
4unused dependency, a whole unused file, and an export no other module imports.
5Judgment: the safe-to-remove judge vetoes each *at the signal* (a tool used
6without an import, a framework entry loaded by convention — these are dropped
7before a workflow ever starts), so this loop's one thin model call lives inside
8``observe``, not as an in-loop changelog judge (``judge_context`` is therefore
9``None``). Work item: the survivors (a removal, a dead file, or an un-export).
10Disposition: commit — the spine opens a PR and gates the merge (CI is oracle).
11"""
13from __future__ import annotations
15from typing import TYPE_CHECKING
17from temporalio import activity
19from froot.domain.loop import Loop
20from froot.domain.removal import Removal
21from froot.loops.registry import CommitTail, LoopSpec, register
23if TYPE_CHECKING:
24 from pathlib import Path
26 from froot.domain.repo import TargetRepo
27 from froot.domain.work import WorkItem
28 from froot.ports.protocols import PackageManager
31async def observe(
32 target: TargetRepo,
33 package_manager: PackageManager,
34 manifest_dir: Path,
35) -> tuple[int, tuple[WorkItem, ...]]:
36 """Dead code flagged, then vetoed safe-to-remove (the veto is the judge).
38 The static analyzer flags every shape of dead code; the safe-to-remove judge
39 then vetoes each — a dependency through the dependency veto, a file/export
40 through the source veto — and only a ``clean`` survives to become a PR. A
41 judge error drops that item (fail-safe: never propose what was not vetted).
42 ``considered`` is the flagged count; the kept count is the survivors, so the
43 scan tick shows how much the veto filtered.
44 """
45 from froot.adapters.model_judge import PydanticAiJudge
47 flagged = await package_manager.list_unused(target, manifest_dir)
48 if not flagged:
49 return 0, ()
50 judge = PydanticAiJudge()
51 kept: list[WorkItem] = []
52 for item in flagged:
53 try:
54 if isinstance(item, Removal):
55 verdict = await judge.judge_removal(item)
56 else:
57 verdict = await judge.judge_dead_source(item)
58 except Exception as exc:
59 activity.logger.warning(
60 "safe-to-remove judge unavailable for %s; skipping: %r",
61 item.subject,
62 exc,
63 )
64 continue
65 if verdict.kind != "clean":
66 continue
67 # Carry the judge's reasoning into the work item so the PR body explains
68 # why the change is safe, beside the detector note.
69 enriched = (
70 f"{item.justification}; {verdict.rationale}"
71 if item.justification
72 else verdict.rationale
73 )
74 kept.append(item.model_copy(update={"justification": enriched}))
75 return len(flagged), tuple(kept)
⋯ 17 lines hidden (lines 76–92)
78register(
79 LoopSpec(
80 loop=Loop.DEAD_CODE,
81 dashboard_icon="scissors",
82 tail=CommitTail(
83 observe=observe,
84 title_prefix="dead-code",
85 # Dead-code judges at the signal (the veto), not the changelog.
86 judge_context=None,
87 # No dead-code item carries a version to be overtaken — nothing to
88 # reconcile.
89 reconciles=False,
90 ),
91 )
92)
The source veto is a fourth thin agent over the shared assessment shape, a distinct prompt.
src/froot/adapters/model_judge.py · 282 lines
src/froot/adapters/model_judge.py
⋯ 271 lines hidden (lines 1–271)
1"""The thin model judgment: how risky is this dependency bump's changelog?
3froot's one model call. A Pydantic AI agent reads the changelog and returns a
4typed :class:`_Assessment`, which :func:`assessment_to_verdict` maps to the
5domain :data:`~froot.domain.changelog.ChangelogVerdict`. The verdict is framing,
6not a gate — the spine proposes the bump regardless — so even a "risky" reading
7just shapes the PR. The *loop* shapes what the model is asked: a patch bump asks
8"is this clean?", a security bump (often a minor/major) asks "what breaks?". The
9model is injected, so tests run it offline with a ``TestModel`` /
10``FunctionModel``; the mapping is a pure function, tested apart from any model.
11"""
13from __future__ import annotations
15from typing import TYPE_CHECKING, Literal, assert_never
17from pydantic import BaseModel, Field
18from pydantic_ai import Agent
20from froot.adapters.model import build_model
21from froot.domain.changelog import (
22 CleanVerdict,
23 RiskyVerdict,
24 UnknownVerdict,
25)
26from froot.domain.loop import Loop
28if TYPE_CHECKING:
29 from pydantic_ai.models import Model
31 from froot.domain.changelog import Changelog, ChangelogVerdict
32 from froot.domain.dead_source import DeadExport, DeadFile
33 from froot.domain.removal import Removal
35_SYSTEM_PROMPT = (
36 "You assess the changelog of a dependency upgrade for a code-maintenance "
37 "bot. The bot proposes the bump either way; your job is only to frame the "
38 "risk for the human reviewer.\n"
39 "Return one verdict:\n"
40 "- clean: the notes describe only fixes / docs / internal changes with no "
41 "behavioral or API impact.\n"
42 "- risky: the notes hint at behavior change, deprecations, or anything a "
43 "reviewer should look at closely; list the concerns.\n"
44 "- unknown: the notes are empty or uninformative.\n"
45 "Quote-or-omit: base 'risky' concerns on what the text actually says; do "
46 "not speculate. Keep the rationale to one or two sentences."
47)
49# The gate reviewer's stance is adversarial and asymmetric: this bump is about
50# to merge with NO human in the loop, so the burden is on the changelog to prove
51# it is safe, not on the reviewer to prove it is dangerous. The same _Assessment
52# shape, read as approve/hold (clean = approve).
53_GATE_SYSTEM_PROMPT = (
54 "You are the LAST line of defense before a dependency upgrade auto-merges "
55 "with no human review. Re-read the changelog adversarially and decide "
56 "whether it is safe to merge unattended.\n"
57 "Return one verdict:\n"
58 "- clean: ONLY if the notes clearly describe nothing beyond fixes / docs / "
59 "internal changes — no behavioral change, no API change, no deprecation, "
60 "no ambiguity. This approves the merge.\n"
61 "- risky: any hint of behavior change, removal, deprecation, or surface a "
62 "reviewer should see; list the concerns. This holds the PR.\n"
63 "- unknown: the notes are empty, uninformative, or you are unsure. This "
64 "holds the PR.\n"
65 "The burden is on the changelog to prove safety: when in doubt, do NOT "
66 "return clean. Base concerns on what the text says; keep it to one or two "
67 "sentences."
68)
71# The dead-code loop's thin judgment: a static analyzer flagged a dependency as
72# never imported, but "not imported" is not "unused" — CLI/test/build tools
73# (pytest, eslint, tsc), framework plugins, type-only/peer deps, and dynamically
74# loaded packages are all used without an import. This judge vetoes those so the
75# loop never opens a noisy "remove pytest" PR; CI is still the final oracle, but
76# a wrong removal wastes a human's time, so the burden is on "safe".
77_REMOVAL_SYSTEM_PROMPT = (
78 "You decide whether an UNUSED dependency is safe to remove from a project, "
79 "for a code-maintenance bot. A static analyzer flagged it as never "
80 "imported in the source. But 'not imported' does not always mean "
81 "'unused'.\n"
82 "Return one verdict:\n"
83 "- clean: a normal library that, if genuinely never imported, the build no "
84 "longer needs — safe to remove. This proposes the removal.\n"
85 "- risky: plausibly used WITHOUT an import — a CLI/test/build tool run as "
86 "a command (e.g. pytest, eslint, prettier, tsc, ruff), a framework or "
87 "bundler plugin, a type-only or peer dependency, or something loaded "
88 "dynamically or via config. Say which. This holds it back (no PR).\n"
89 "- unknown: you cannot tell. This holds it back.\n"
90 "CI is the final check, but a wrong removal wastes a human's time, so when "
91 "in doubt do NOT return clean. Keep the rationale to one or two sentences."
92)
95# The dead-code loop's source arm: a static analyzer flagged a file as imported
96# by nothing, or an export as imported by no other module. As with deps, "not
97# imported" is not "dead": frameworks load files by convention (route/page/
98# plugin files), dynamic imports and config reference code by string, and a
99# published library's exports are its public API even when nothing in-repo uses
100# them. This judge vetoes those; CI is still the final oracle.
101_DEAD_SOURCE_SYSTEM_PROMPT = (
102 "You decide whether a piece of dead source is safe to remove, for a "
103 "code-maintenance bot. A static analyzer flagged either a FILE that no "
104 "module imports, or an EXPORT that no other module imports (which the bot "
105 "would un-export, leaving it module-private). 'Not imported' is not always "
106 "'dead'.\n"
107 "Return one verdict:\n"
108 "- clean: ordinary internal code that, if genuinely unreferenced, the "
109 "project no longer needs — safe to delete the file / un-export the symbol. "
110 "This proposes the change.\n"
111 "- risky: plausibly used WITHOUT a static import: a framework entry "
112 "loaded by convention (a route, page, or plugin file), code imported "
113 "dynamically or named in config, a test fixture, or the public API of a "
114 "published library. Say which. This holds it back (no PR).\n"
115 "- unknown: you cannot tell. This holds it back.\n"
116 "CI is the final check, but a wrong change wastes a human's time, so when "
117 "in doubt do NOT return clean. Keep the rationale to one or two sentences."
118)
121def _loop_context(loop: Loop) -> str:
122 """The framing line telling the model what kind of bump it is judging.
124 Sourced from the loop's registered spec (see :mod:`froot.loops`). Only
125 changelog-judging loops reach this; a loop without a changelog judge
126 (dead-code, whose judgment is a signal-time safe-to-remove veto) has a
127 ``judge_context`` of ``None`` and never gets here — the empty fallback just
128 keeps the call total.
129 """
130 from froot.loops import registry
132 return registry.commit_tail(loop).judge_context or ""
135class _Assessment(BaseModel):
136 """The model's structured output, mapped to a domain verdict."""
138 verdict: Literal["clean", "risky", "unknown"]
139 rationale: str
140 concerns: list[str] = Field(default_factory=list)
143def assessment_to_verdict(assessment: _Assessment) -> ChangelogVerdict:
144 """Map the model's structured assessment to a domain verdict (pure)."""
145 match assessment.verdict:
146 case "clean":
147 return CleanVerdict(rationale=assessment.rationale)
148 case "risky":
149 return RiskyVerdict(
150 rationale=assessment.rationale,
151 concerns=tuple(assessment.concerns),
152 )
153 case "unknown":
154 return UnknownVerdict(rationale=assessment.rationale)
155 assert_never(assessment.verdict)
158def _gate_model() -> Model:
159 """Build the gate reviewer's model: the override, else the judge model."""
160 from froot.config.settings import ModelSettings
162 return build_model(model_name=ModelSettings().gate_review_model or None)
165def _changelog_prompt(changelog: Changelog, loop: Loop) -> str:
166 """The shared user prompt: the bump's context and its changelog text."""
167 return (
168 f"{_loop_context(loop)}\n"
169 f"Package: {changelog.package}\n"
170 f"Target version: {changelog.version}\n"
171 f"Changelog:\n{changelog.text}"
172 )
175def _removal_prompt(removal: Removal) -> str:
176 """The user prompt for the safe-to-remove judge: the flagged dependency."""
177 declared = "a dev dependency" if removal.dev else "a runtime dependency"
178 return (
179 f"Package: {removal.package}\n"
180 f"Ecosystem: {removal.ecosystem.value}\n"
181 f"Declared as: {declared}\n"
182 f"Detector note: {removal.justification or 'flagged as unused'}\n"
183 "Is this safe to remove?"
184 )
187def _dead_source_prompt(item: DeadFile | DeadExport) -> str:
188 """The user prompt for the dead-source judge: the flagged file/export."""
189 note = item.justification or "flagged as unused"
190 if item.kind == "dead_file":
191 return (
192 f"Unused FILE: {item.path}\n"
193 f"Ecosystem: {item.ecosystem.value}\n"
194 f"Detector note: {note}\n"
195 "Is it safe to delete this file?"
196 )
197 return (
198 f"Unused EXPORT: {item.symbol} (in {item.file})\n"
199 f"Ecosystem: {item.ecosystem.value}\n"
200 f"Detector note: {note}\n"
201 "Is it safe to un-export this symbol (leave it module-private)?"
202 )
205class PydanticAiJudge:
206 """A :class:`~froot.ports.protocols.ModelJudge` backed by Pydantic AI.
208 Two agents share the structured ``_Assessment`` output but differ in stance:
209 the neutral framing judge (:meth:`judge`) and the adversarial gate reviewer
210 (:meth:`gate_review`), each its own model so a steward can make the deep
211 review independent in capability, not just prompt.
212 """
214 def __init__(
215 self, model: Model | None = None, gate_model: Model | None = None
216 ) -> None:
217 """Build both agents.
219 ``model`` defaults to the configured Ollama; ``gate_model`` to the
220 gate-review override, else ``model``, else the gate-review model.
221 """
222 self._agent: Agent[None, _Assessment] = Agent(
223 model or build_model(),
224 output_type=_Assessment,
225 system_prompt=_SYSTEM_PROMPT,
226 )
227 self._gate_agent: Agent[None, _Assessment] = Agent(
228 gate_model or model or _gate_model(),
229 output_type=_Assessment,
230 system_prompt=_GATE_SYSTEM_PROMPT,
231 )
232 # The dead-code loop's safe-to-remove judge; the neutral model, its own
233 # prompt. It is a veto (clean = propose), so it carries real weight —
234 # but CI remains the oracle.
235 self._removal_agent: Agent[None, _Assessment] = Agent(
236 model or build_model(),
237 output_type=_Assessment,
238 system_prompt=_REMOVAL_SYSTEM_PROMPT,
239 )
240 # The dead-code loop's source veto (dead files + unused exports); the
241 # neutral model, its own prompt. Also a veto (clean = propose).
242 self._dead_source_agent: Agent[None, _Assessment] = Agent(
243 model or build_model(),
244 output_type=_Assessment,
245 system_prompt=_DEAD_SOURCE_SYSTEM_PROMPT,
246 )
248 async def judge(
249 self, changelog: Changelog, loop: Loop = Loop.DEPENDENCY_PATCH
250 ) -> ChangelogVerdict:
251 """Assess a changelog into a verdict, framed by the loop."""
252 result = await self._agent.run(_changelog_prompt(changelog, loop))
253 return assessment_to_verdict(result.output)
255 async def gate_review(
256 self, changelog: Changelog, loop: Loop = Loop.DEPENDENCY_PATCH
257 ) -> ChangelogVerdict:
258 """Adversarially re-review a bump at the gate (clean = approve)."""
259 result = await self._gate_agent.run(_changelog_prompt(changelog, loop))
260 return assessment_to_verdict(result.output)
262 async def judge_removal(self, removal: Removal) -> ChangelogVerdict:
263 """Assess whether an unused dependency is safe to remove (clean = yes).
265 The dead-code loop's veto: ``clean`` lets the removal become a PR; any
266 other verdict holds it back. Same ``_Assessment`` shape as the changelog
267 judge, a different prompt.
268 """
269 result = await self._removal_agent.run(_removal_prompt(removal))
270 return assessment_to_verdict(result.output)
272 async def judge_dead_source(
273 self, item: DeadFile | DeadExport
274 ) -> ChangelogVerdict:
275 """Assess whether a dead file / unused export is safe to act on.
277 The dead-code loop's source veto: ``clean`` lets the deletion (file) or
278 un-export (symbol) become a PR; any other verdict holds it. Same
279 ``_Assessment`` shape as :meth:`judge_removal`, a different prompt.
280 """
281 result = await self._dead_source_agent.run(_dead_source_prompt(item))
282 return assessment_to_verdict(result.output)
How it's verified
The transform is the riskiest piece, so it carries a table of cases: every form it must edit, and every form it must refuse. The action is proven end to end on a real temp checkout — the export is stripped, its body and the still-exported sibling untouched — and the signal is proven to surface all three kinds from one knip body.
The un-export contract, kept and refused forms side by side.
tests/test_dead_source.py · 69 lines
tests/test_dead_source.py
⋯ 41 lines hidden (lines 1–41)
1from __future__ import annotations
3import pytest
4from pydantic import ValidationError
6from froot.domain.dead_source import DeadExport, DeadFile
7from froot.domain.ecosystem import Ecosystem
8from froot.policy.dead_source import unexport_line
11def test_dead_file_subject_and_str():
12 item = DeadFile(path="src/old/util.ts", ecosystem=Ecosystem.NPM)
13 assert item.kind == "dead_file"
14 assert item.subject == "src/old/util.ts" # the path is its identifier
15 assert str(item) == "delete src/old/util.ts (unused)"
18def test_dead_export_subject_and_str():
19 item = DeadExport(
20 file="src/util.ts", symbol="helper", line=12, ecosystem=Ecosystem.NPM
21 )
22 assert item.kind == "dead_export"
23 assert item.subject == "helper" # the symbol is its identifier
24 assert str(item) == "un-export helper in src/util.ts (unused)"
27def test_dead_export_line_must_be_positive():
28 # The analyzer's position is 1-based; a non-positive line can't exist.
29 with pytest.raises(ValidationError):
30 DeadExport(
31 file="src/util.ts", symbol="x", line=0, ecosystem=Ecosystem.NPM
32 )
35def test_dead_file_path_must_be_nonempty():
36 with pytest.raises(ValidationError):
37 DeadFile(path="", ecosystem=Ecosystem.NPM)
40# (line, symbol, expected) — the inline declaration forms the action handles,
41# plus the forms it must refuse (returns None so the signal drops them).
42_UNEXPORT_CASES = [
43 ("export function foo() {", "foo", "function foo() {"),
44 ("export const foo = 1", "foo", "const foo = 1"),
45 ("export let foo = 1", "foo", "let foo = 1"),
46 ("export class Foo {", "Foo", "class Foo {"),
47 ("export interface I {", "I", "interface I {"),
48 ("export type Foo = number", "Foo", "type Foo = number"),
49 ("export enum E {", "E", "enum E {"),
50 ("export async function foo() {", "foo", "async function foo() {"),
51 ("export abstract class Base {", "Base", "abstract class Base {"),
52 (" export const inner = 1", "inner", " const inner = 1"),
53 # Refused: the named symbol is not the one this line declares.
54 ("export const foo = 1", "bar", None),
55 # Refused: a default export has no plain un-exported form.
56 ("export default function foo() {", "foo", None),
57 # Refused: clause re-exports / star re-exports need an AST, not a line edit.
58 ("export { foo, bar }", "foo", None),
59 ("export * from './x'", "foo", None),
60 # Refused: a destructuring export declares no single identifier here.
61 ("export const { a } = obj", "a", None),
62 # Refused: the line is not an export at all.
63 ("const foo = 1", "foo", None),
64]
67@pytest.mark.parametrize(("line", "symbol", "expected"), _UNEXPORT_CASES)
68def test_unexport_line(line: str, symbol: str, expected: str | None):
69 assert unexport_line(line, symbol) == expected
The action edits the real file in the checkout, nothing else.
tests/test_activities.py · 934 lines
tests/test_activities.py
⋯ 460 lines hidden (lines 1–460)
1from __future__ import annotations
3import json
5import pytest
6from temporalio.common import WorkflowIDReusePolicy
7from temporalio.exceptions import WorkflowAlreadyStartedError
9import froot.adapters.changelog_http as changelog_mod
10import froot.adapters.github as github_mod
11import froot.adapters.model_judge as model_mod
12import froot.adapters.osv as osv_mod
13import froot.adapters.registry as registry_mod
14import froot.dashboard.github_source as github_source_mod
15import froot.dashboard.read_model as read_model_mod
16import froot.workflow.temporal_client as temporal_client
17from froot.domain.candidate import AvailableUpgrade, Candidate
18from froot.domain.changelog import (
19 Changelog,
20 CleanVerdict,
21 RiskyVerdict,
22 UnknownVerdict,
23)
24from froot.domain.ci import CIPassed
25from froot.domain.ecosystem import Ecosystem
26from froot.domain.loop import Loop
27from froot.domain.outcome import LoopOutcome
28from froot.domain.removal import Removal
29from froot.policy.naming import bump_workflow_id
30from froot.workflow import activities
31from froot.workflow.types import (
32 AutoMergeInput,
33 BumpParams,
34 CiCheckInput,
35 CloseInput,
36 DispatchInput,
37 GateReviewInput,
38 GateSelfTestInput,
39 JudgeInput,
40 MergeInput,
41 OpenPrInput,
42 ReconcileInput,
43 RecordInput,
44 ScanCandidatesInput,
45)
46from tests.support import (
47 FakeAdvisorySource,
48 FakeChangelogSource,
49 FakeForge,
50 FakeJudge,
51 FakePackageManager,
52 make_advisory,
53 make_candidate,
54 make_dead_export,
55 make_dead_file,
56 make_installed,
57 make_pr,
58 make_removal,
59 make_repo,
60 make_upgrade,
61 ver,
62)
65async def test_scan_candidates_selects_patches(
66 monkeypatch: pytest.MonkeyPatch,
67):
68 upgrades = (
69 AvailableUpgrade(
70 package="left-pad",
71 ecosystem=Ecosystem.NPM,
72 current=ver("1.4.2"),
73 available=(ver("1.4.3"), ver("1.5.0")),
74 ),
75 )
76 monkeypatch.setattr(github_mod, "GitHubForge", FakeForge)
77 monkeypatch.setattr(
78 registry_mod,
79 "package_manager_for",
80 lambda ecosystem: FakePackageManager(upgrades),
81 )
82 result = await activities.scan_candidates(
83 ScanCandidatesInput(target=make_repo())
84 )
85 # The patch loop yields bumps; narrow to read the version it targets.
86 assert [c.target for c in result if isinstance(c, Candidate)] == [
87 ver("1.4.3")
88 ]
91async def test_scan_candidates_security_loop(
92 monkeypatch: pytest.MonkeyPatch,
93):
94 # The security arm: installed set -> OSV advisories -> clearing targets.
95 installed = (make_installed("left-pad", "1.4.2"),)
96 advisories = (
97 make_advisory("left-pad", "GHSA-1", ranges=(("0", "1.4.3"),)),
98 )
99 monkeypatch.setattr(github_mod, "GitHubForge", FakeForge)
100 monkeypatch.setattr(
101 registry_mod,
102 "package_manager_for",
103 lambda ecosystem: FakePackageManager(installed=installed),
104 )
105 monkeypatch.setattr(
106 osv_mod, "OsvAdvisorySource", lambda: FakeAdvisorySource(advisories)
107 )
108 result = await activities.scan_candidates(
109 ScanCandidatesInput(target=make_repo(), loop=Loop.SECURITY_PATCH)
110 )
111 assert [c.target for c in result if isinstance(c, Candidate)] == [
112 ver("1.4.3")
113 ]
114 assert result[0].justification is not None
115 assert "GHSA-1" in result[0].justification
118async def test_scan_candidates_dead_code_keeps_judge_approved_removals(
119 monkeypatch: pytest.MonkeyPatch,
120):
121 # The dead-code arm: knip flags unused deps, the safe-to-remove judge vetoes
122 # each, survivors carry the judge's rationale into their justification.
123 unused = (make_removal(package="left-pad"),)
124 monkeypatch.setattr(github_mod, "GitHubForge", FakeForge)
125 monkeypatch.setattr(
126 registry_mod,
127 "package_manager_for",
128 lambda ecosystem: FakePackageManager(unused=unused),
129 )
130 monkeypatch.setattr(
131 model_mod,
132 "PydanticAiJudge",
133 lambda: FakeJudge(removal_verdict=CleanVerdict(rationale="not used")),
134 )
135 result = await activities.scan_candidates(
136 ScanCandidatesInput(target=make_repo(), loop=Loop.DEAD_CODE)
137 )
138 assert [r.package for r in result if isinstance(r, Removal)] == ["left-pad"]
139 assert result[0].justification == "unused (knip); not used"
142async def test_scan_candidates_dead_code_vetoes_unsafe_removals(
143 monkeypatch: pytest.MonkeyPatch,
144):
145 # A tool used without an import (pytest) is flagged unused but the judge
146 # holds it (risky) — the veto at the signal drops it before any PR.
147 unused = (make_removal(package="pytest", dev=True),)
148 monkeypatch.setattr(github_mod, "GitHubForge", FakeForge)
149 monkeypatch.setattr(
150 registry_mod,
151 "package_manager_for",
152 lambda ecosystem: FakePackageManager(unused=unused),
153 )
154 monkeypatch.setattr(
155 model_mod,
156 "PydanticAiJudge",
157 lambda: FakeJudge(
158 removal_verdict=RiskyVerdict(
159 rationale="test runner", concerns=("used via CLI",)
160 )
161 ),
162 )
163 result = await activities.scan_candidates(
164 ScanCandidatesInput(target=make_repo(), loop=Loop.DEAD_CODE)
165 )
166 assert result == ()
169async def test_scan_candidates_dead_code_keeps_source_dead_code(
170 monkeypatch: pytest.MonkeyPatch,
171):
172 # The source arm: a dead file + dead export go through the dead-source veto
173 # (judge_dead_source, not judge_removal); survivors carry its rationale.
174 unused = (
175 make_dead_file(path="src/orphan.ts"),
176 make_dead_export(file="src/util.ts", symbol="gone", line=2),
177 )
178 judge = FakeJudge(dead_source_verdict=CleanVerdict(rationale="truly dead"))
179 monkeypatch.setattr(github_mod, "GitHubForge", FakeForge)
180 monkeypatch.setattr(
181 registry_mod,
182 "package_manager_for",
183 lambda ecosystem: FakePackageManager(unused=unused),
184 )
185 monkeypatch.setattr(model_mod, "PydanticAiJudge", lambda: judge)
186 result = await activities.scan_candidates(
187 ScanCandidatesInput(target=make_repo(), loop=Loop.DEAD_CODE)
188 )
189 assert [item.kind for item in result] == ["dead_file", "dead_export"]
190 assert all(
191 item.justification == "unused (knip); truly dead" for item in result
192 )
193 # The source veto ran, never the dependency one.
194 assert judge.dead_sources == list(unused)
195 assert judge.removals == []
198async def test_reconcile_skips_dead_code(monkeypatch: pytest.MonkeyPatch):
199 # Reconcile is version-supersession cleanup; a removal has no version, so
200 # dead-code reconcile must return 0 *without* re-running its signal (which
201 # would cost a checkout + the veto judge every tick).
202 def boom(ecosystem: object) -> object:
203 raise AssertionError("dead-code reconcile must not re-scan")
205 monkeypatch.setattr(github_mod, "GitHubForge", FakeForge)
206 monkeypatch.setattr(registry_mod, "package_manager_for", boom)
207 closed = await activities.reconcile_open_prs(
208 ReconcileInput(target=make_repo(), loop=Loop.DEAD_CODE)
209 )
210 assert closed == 0
213async def test_scan_candidates_logs_considered_and_selected(
214 monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
215):
216 # Two upgrades available, but only one yields a patch candidate (the other
217 # is a major bump) — so the tick considered 2 and selected 1, dropped 1.
218 upgrades = (
219 AvailableUpgrade(
220 package="left-pad",
221 ecosystem=Ecosystem.NPM,
222 current=ver("1.4.2"),
223 available=(ver("1.4.3"),), # a patch -> kept
224 ),
225 AvailableUpgrade(
226 package="lodash",
227 ecosystem=Ecosystem.NPM,
228 current=ver("4.0.0"),
229 available=(ver("5.0.0"),), # only a major -> dropped
230 ),
231 )
232 monkeypatch.setattr(github_mod, "GitHubForge", FakeForge)
233 monkeypatch.setattr(
234 registry_mod,
235 "package_manager_for",
236 lambda ecosystem: FakePackageManager(upgrades),
237 )
238 with caplog.at_level("INFO", logger="froot.scan"):
239 result = await activities.scan_candidates(
240 ScanCandidatesInput(target=make_repo())
241 )
242 assert len(result) == 1 # only left-pad's patch
243 ticks = [r for r in caplog.records if r.name == "froot.scan"]
244 assert len(ticks) == 1
245 record = json.loads(ticks[0].getMessage())
246 assert record["event"] == "scan_tick"
247 assert record["loop"] == "dependency-patch"
248 assert record["considered"] == 2
249 assert record["selected"] == 1
250 assert record["dropped"] == 1
253async def test_scan_candidates_selects_package_manager_by_ecosystem(
254 monkeypatch: pytest.MonkeyPatch,
255):
256 seen: dict[str, Ecosystem] = {}
258 def factory(ecosystem: Ecosystem) -> FakePackageManager:
259 seen["ecosystem"] = ecosystem
260 return FakePackageManager(())
262 monkeypatch.setattr(github_mod, "GitHubForge", FakeForge)
263 monkeypatch.setattr(registry_mod, "package_manager_for", factory)
264 await activities.scan_candidates(
265 ScanCandidatesInput(target=make_repo(ecosystem=Ecosystem.UV))
266 )
267 assert seen["ecosystem"] is Ecosystem.UV
270async def test_judge_changelog_unknown_when_missing(
271 monkeypatch: pytest.MonkeyPatch,
272):
273 monkeypatch.setattr(
274 changelog_mod, "HttpChangelogSource", lambda: FakeChangelogSource(None)
275 )
276 verdict = await activities.judge_changelog(
277 JudgeInput(candidate=make_candidate())
278 )
279 assert isinstance(verdict, UnknownVerdict)
282async def test_judge_changelog_for_removal_is_clean_from_justification():
283 # A removal already cleared the safe-to-remove veto at scan; the in-workflow
284 # judge step carries that conclusion into the PR framing without fetching a
285 # changelog (there is none) or calling the model.
286 verdict = await activities.judge_changelog(
287 JudgeInput(
288 candidate=make_removal(
289 package="left-pad", justification="unused (knip); not imported"
290 )
291 )
292 )
293 assert isinstance(verdict, CleanVerdict)
294 assert verdict.rationale == "unused (knip); not imported"
297async def test_judge_changelog_uses_model(monkeypatch: pytest.MonkeyPatch):
298 changelog = Changelog(
299 package="left-pad", version=ver("1.4.3"), text="fixes"
300 )
301 monkeypatch.setattr(
302 changelog_mod,
303 "HttpChangelogSource",
304 lambda: FakeChangelogSource(changelog),
305 )
306 monkeypatch.setattr(
307 model_mod,
308 "PydanticAiJudge",
309 lambda: FakeJudge(CleanVerdict(rationale="clean")),
310 )
311 verdict = await activities.judge_changelog(
312 JudgeInput(candidate=make_candidate())
313 )
314 assert isinstance(verdict, CleanVerdict)
317async def test_open_pull_request_idempotent_short_circuit(
318 monkeypatch: pytest.MonkeyPatch,
319):
320 fake = FakeForge(existing_pr=make_pr(number=42))
321 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
322 monkeypatch.setattr(
323 registry_mod,
324 "package_manager_for",
325 lambda ecosystem: FakePackageManager(),
326 )
327 params = OpenPrInput(
328 target=make_repo(),
329 candidate=make_candidate(),
330 verdict=CleanVerdict(rationale="x"),
331 )
332 pr = await activities.open_pull_request(params)
333 assert pr.number == 42
334 assert fake.checked_out is False # short-circuited: no checkout/apply/push
337async def test_open_pull_request_full_path(monkeypatch: pytest.MonkeyPatch):
338 fake = FakeForge(existing_pr=None, opened_pr=make_pr(number=7))
339 package_manager = FakePackageManager()
340 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
341 monkeypatch.setattr(
342 registry_mod, "package_manager_for", lambda ecosystem: package_manager
343 )
344 params = OpenPrInput(
345 target=make_repo(),
346 candidate=make_candidate(),
347 verdict=CleanVerdict(rationale="x"),
348 )
349 pr = await activities.open_pull_request(params)
350 assert pr.number == 7
351 assert fake.checked_out is True
352 assert package_manager.applied == make_candidate()
353 assert fake.pushed is not None
356async def test_open_pull_request_removal_runs_remove_dependency(
357 monkeypatch: pytest.MonkeyPatch,
358):
359 # The one place the action differs by kind: a removal calls
360 # remove_dependency (not apply_patch_bump) and uses the removal PR draft.
361 fake = FakeForge(existing_pr=None, opened_pr=make_pr(number=8))
362 package_manager = FakePackageManager()
363 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
364 monkeypatch.setattr(
365 registry_mod, "package_manager_for", lambda ecosystem: package_manager
366 )
367 removal = make_removal(package="left-pad")
368 params = OpenPrInput(
369 target=make_repo(),
370 candidate=removal,
371 verdict=CleanVerdict(rationale="safe to remove"),
372 )
373 pr = await activities.open_pull_request(params)
374 assert pr.number == 8
375 assert package_manager.applied is None # no bump
376 assert package_manager.removed == [removal] # the removal action ran
377 assert fake.pushed is not None
380async def test_gate_review_for_removal_uses_safe_to_remove_judge(
381 monkeypatch: pytest.MonkeyPatch,
382):
383 # The fourth leg for a removal re-runs the safe-to-remove judge (no
384 # changelog); a clean verdict approves the auto-merge.
385 judge = FakeJudge(removal_verdict=CleanVerdict(rationale="safe"))
386 monkeypatch.setattr(model_mod, "PydanticAiJudge", lambda: judge)
387 removal = make_removal(package="left-pad")
388 # gate_review dispatches on the candidate kind, so the loop is irrelevant
389 # here (the dead-code loop lands in a later slice).
390 verdict = await activities.gate_review(
391 GateReviewInput(candidate=removal, pr=make_pr())
392 )
393 assert isinstance(verdict, CleanVerdict)
394 assert judge.removals == [removal]
397async def test_record_outcome_removal_logs_remove_action(
398 monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
399):
400 fake = FakeForge()
401 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
402 outcome = LoopOutcome(
403 candidate=make_removal(package="left-pad", dev=True),
404 verdict=CleanVerdict(rationale="safe to remove"),
405 pr=make_pr(),
406 ci=CIPassed(),
407 )
408 with caplog.at_level("INFO", logger="froot.outcome"):
409 await activities.record_outcome(
410 RecordInput(target=make_repo(), outcome=outcome)
411 )
412 records = [r for r in caplog.records if r.name == "froot.outcome"]
413 assert len(records) == 1
414 logged = json.loads(records[0].getMessage())
415 assert logged["action"] == "remove"
416 assert logged["package"] == "left-pad"
417 assert logged["dev"] is True
418 assert "from" not in logged # no version fields for a removal
421async def test_judge_changelog_for_dead_source_is_clean_from_justification():
422 # A dead file / export already cleared its safe-to-remove veto at scan, like
423 # a removal: the in-workflow judge carries that into the PR framing with no
424 # changelog and no model call.
425 for item in (
426 make_dead_file(justification="unused file (knip)"),
427 make_dead_export(justification="unused export (knip)"),
428 ):
429 verdict = await activities.judge_changelog(JudgeInput(candidate=item))
430 assert isinstance(verdict, CleanVerdict)
431 assert verdict.rationale == item.justification
434async def test_open_pull_request_dead_file_deletes_the_file(
435 monkeypatch: pytest.MonkeyPatch,
436):
437 # A dead file's action deletes the file (no package-manager call); push
438 # then stages the deletion. CI is still the oracle.
439 fake = FakeForge(
440 existing_pr=None,
441 opened_pr=make_pr(number=9),
442 checkout_files={"src/unused.ts": "export const x = 1\n"},
443 )
444 package_manager = FakePackageManager()
445 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
446 monkeypatch.setattr(
447 registry_mod, "package_manager_for", lambda ecosystem: package_manager
448 )
449 params = OpenPrInput(
450 target=make_repo(),
451 candidate=make_dead_file(path="src/unused.ts"),
452 verdict=CleanVerdict(rationale="dead"),
453 loop=Loop.DEAD_CODE,
454 )
455 pr = await activities.open_pull_request(params)
456 assert pr.number == 9
457 assert package_manager.applied is None and package_manager.removed == []
458 assert fake.pushed_tree["src/unused.ts"] is None # the file was deleted
461async def test_open_pull_request_dead_export_strips_the_export(
462 monkeypatch: pytest.MonkeyPatch,
463):
464 # A dead export's action strips the `export` on the analyzer's line, leaving
465 # the symbol module-private; the rest of the file stays untouched.
466 source = "const keep = 1\nexport function gone() {\n return 2\n}\n"
467 fake = FakeForge(
468 existing_pr=None,
469 opened_pr=make_pr(number=10),
470 checkout_files={"src/util.ts": source},
471 )
472 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
473 monkeypatch.setattr(
474 registry_mod,
475 "package_manager_for",
476 lambda ecosystem: FakePackageManager(),
477 )
478 params = OpenPrInput(
479 target=make_repo(),
480 candidate=make_dead_export(file="src/util.ts", symbol="gone", line=2),
481 verdict=CleanVerdict(rationale="dead"),
482 loop=Loop.DEAD_CODE,
483 )
484 await activities.open_pull_request(params)
485 assert fake.pushed_tree["src/util.ts"] == (
486 "const keep = 1\nfunction gone() {\n return 2\n}\n"
⋯ 448 lines hidden (lines 487–934)
487 )
490async def test_gate_review_for_dead_source_uses_dead_source_judge(
491 monkeypatch: pytest.MonkeyPatch,
492):
493 # The fourth leg for a dead file / export re-runs the dead-source veto (not
494 # the dependency one); a clean verdict approves the auto-merge.
495 judge = FakeJudge(dead_source_verdict=CleanVerdict(rationale="safe"))
496 monkeypatch.setattr(model_mod, "PydanticAiJudge", lambda: judge)
497 item = make_dead_file(path="src/unused.ts")
498 verdict = await activities.gate_review(
499 GateReviewInput(candidate=item, pr=make_pr(), loop=Loop.DEAD_CODE)
500 )
501 assert isinstance(verdict, CleanVerdict)
502 assert judge.dead_sources == [item]
503 assert judge.removals == [] # the dependency judge was not touched
506async def test_record_outcome_dead_source_logs_source_actions(
507 monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
508):
509 # Each source kind records its own action + detail in the run-ledger row.
510 fake = FakeForge()
511 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
512 cases = (
513 (
514 make_dead_file(path="src/unused.ts"),
515 {"action": "remove_file", "path": "src/unused.ts"},
516 ),
517 (
518 make_dead_export(file="src/util.ts", symbol="gone"),
519 {"action": "unexport", "file": "src/util.ts", "symbol": "gone"},
520 ),
521 )
522 for item, expected in cases:
523 outcome = LoopOutcome(
524 candidate=item,
525 verdict=CleanVerdict(rationale="dead"),
526 pr=make_pr(),
527 ci=CIPassed(),
528 )
529 with caplog.at_level("INFO", logger="froot.outcome"):
530 await activities.record_outcome(
531 RecordInput(
532 target=make_repo(), outcome=outcome, loop=Loop.DEAD_CODE
533 )
534 )
535 logged = json.loads(
536 [r for r in caplog.records if r.name == "froot.outcome"][
537 -1
538 ].getMessage()
539 )
540 assert logged["package"] == item.subject
541 for key, value in expected.items():
542 assert logged[key] == value
545async def test_check_ci(monkeypatch: pytest.MonkeyPatch):
546 monkeypatch.setattr(
547 github_mod, "GitHubForge", lambda: FakeForge(ci=CIPassed())
548 )
549 status = await activities.check_ci(
550 CiCheckInput(target=make_repo(), head_sha="abc1234")
551 )
552 assert isinstance(status, CIPassed)
555async def test_record_outcome_sets_labels(monkeypatch: pytest.MonkeyPatch):
556 fake = FakeForge()
557 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
558 outcome = LoopOutcome(
559 candidate=make_candidate(),
560 verdict=CleanVerdict(rationale="x"),
561 pr=make_pr(),
562 ci=CIPassed(),
563 )
564 await activities.record_outcome(
565 RecordInput(target=make_repo(), outcome=outcome)
566 )
567 assert fake.labeled is not None
568 labeled = set(fake.labeled)
569 # The loop labels (no changelog/CI labels, regardless of outcome) plus the
570 # environment stamp the gate uses to scope trust to the current judge model.
571 assert {"froot", "dependency-patch"} <= labeled
572 assert any(name.startswith("froot-env:") for name in labeled)
575class _FakeClient:
576 def __init__(self, *, already_started: bool = False) -> None:
577 self.already_started = already_started
578 self.started: list[dict[str, object]] = []
580 async def start_workflow(
581 self, run, arg, *, id, task_queue, id_reuse_policy
582 ):
583 if self.already_started:
584 raise WorkflowAlreadyStartedError(
585 workflow_id=id, workflow_type="BumpWorkflow"
586 )
587 self.started.append({"id": id, "policy": id_reuse_policy, "arg": arg})
590async def test_dispatch_bump_starts_with_reject_duplicate(
591 monkeypatch: pytest.MonkeyPatch,
592):
593 fake = _FakeClient()
595 async def _client() -> _FakeClient:
596 return fake
598 monkeypatch.setattr(temporal_client, "client", _client)
599 repo, candidate = make_repo(), make_candidate()
600 await activities.dispatch_bump(
601 DispatchInput(target=repo, candidate=candidate)
602 )
603 assert len(fake.started) == 1
604 assert fake.started[0]["id"] == bump_workflow_id(repo, candidate)
605 assert fake.started[0]["policy"] == WorkflowIDReusePolicy.REJECT_DUPLICATE
608async def test_dispatch_bump_is_noop_when_already_started(
609 monkeypatch: pytest.MonkeyPatch,
610):
611 fake = _FakeClient(already_started=True)
613 async def _client() -> _FakeClient:
614 return fake
616 monkeypatch.setattr(temporal_client, "client", _client)
617 # Must not raise: re-dispatching an in-flight bump is a no-op (the
618 # workflow-id + REJECT_DUPLICATE keep it to one PR per bump).
619 await activities.dispatch_bump(
620 DispatchInput(target=make_repo(), candidate=make_candidate())
621 )
624async def test_dispatch_bump_pins_close_on_red(
625 monkeypatch: pytest.MonkeyPatch,
626):
627 # The toggle is read at dispatch and pinned onto the bump's params, so the
628 # workflow never reads config itself.
629 monkeypatch.setenv("FROOT_CLOSE_ON_RED", "0")
630 fake = _FakeClient()
632 async def _client() -> _FakeClient:
633 return fake
635 monkeypatch.setattr(temporal_client, "client", _client)
636 await activities.dispatch_bump(
637 DispatchInput(target=make_repo(), candidate=make_candidate())
638 )
639 arg = fake.started[0]["arg"]
640 assert isinstance(arg, BumpParams)
641 assert arg.close_on_red is False
644async def test_close_pull_request_comments_then_closes_and_deletes(
645 monkeypatch: pytest.MonkeyPatch,
646):
647 fake = FakeForge()
648 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
649 pr = make_pr(number=7)
650 await activities.close_pull_request(
651 CloseInput(target=make_repo(), pr=pr, failing=("build",))
652 )
653 assert fake.closed == [7]
654 assert fake.deleted_branches == [pr.branch]
655 # The "why" is posted (idempotent comment path) and names the failing check.
656 assert fake.comments and fake.comments[-1][0] == 7
657 assert "build" in fake.comments[-1][1]
660async def test_merge_pull_request_merges_via_forge(
661 monkeypatch: pytest.MonkeyPatch,
662):
663 fake = FakeForge()
664 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
665 pr = make_pr(number=7)
666 await activities.merge_pull_request(
667 MergeInput(target=make_repo(), pr=pr, loop=Loop.DEPENDENCY_PATCH)
668 )
669 # Auto-merged through the forge (the head SHA is pinned for concurrency).
670 assert fake.merged == [7]
673async def test_auto_merge_eligible_false_when_not_allowlisted(
674 monkeypatch: pytest.MonkeyPatch,
675):
676 # The default: empty allowlist -> hold, and not even a network read.
677 monkeypatch.delenv("FROOT_AUTOMERGE_ALLOWLIST", raising=False)
679 async def _boom(*args: object, **kwargs: object) -> object:
680 raise AssertionError("fetch must not run when the repo is not allowed")
682 monkeypatch.setattr(github_source_mod, "fetch", _boom)
683 eligible = await activities.auto_merge_eligible(
684 AutoMergeInput(target=make_repo(), loop=Loop.DEPENDENCY_PATCH)
685 )
686 assert eligible is False
689async def test_auto_merge_eligible_true_when_class_earned(
690 monkeypatch: pytest.MonkeyPatch,
691):
692 # Allowlisted + the triangulated read says earned -> the grant is live.
693 monkeypatch.setenv("FROOT_AUTOMERGE_ALLOWLIST", "acme/widgets")
695 async def _fetch(repos: tuple[str, ...]) -> tuple[tuple[object, ...], None]:
696 return (), None
698 async def _fetch_outcomes(
699 repos: object, prs: object, **kwargs: object
700 ) -> tuple[dict[object, object], None]:
701 return {}, None
703 monkeypatch.setattr(github_source_mod, "fetch", _fetch)
704 monkeypatch.setattr(github_source_mod, "fetch_outcomes", _fetch_outcomes)
705 monkeypatch.setattr(read_model_mod, "earned_now", lambda *a, **k: True)
706 eligible = await activities.auto_merge_eligible(
707 AutoMergeInput(target=make_repo(), loop=Loop.DEPENDENCY_PATCH)
708 )
709 assert eligible is True
712async def test_auto_merge_eligible_false_on_github_error(
713 monkeypatch: pytest.MonkeyPatch,
714):
715 # Allowlisted but the record can't be read -> hold, never merge blind.
716 monkeypatch.setenv("FROOT_AUTOMERGE_ALLOWLIST", "acme/widgets")
718 async def _fetch(repos: tuple[str, ...]) -> tuple[tuple[object, ...], str]:
719 return (), "rate limited"
721 monkeypatch.setattr(github_source_mod, "fetch", _fetch)
722 eligible = await activities.auto_merge_eligible(
723 AutoMergeInput(target=make_repo(), loop=Loop.DEPENDENCY_PATCH)
724 )
725 assert eligible is False
728async def test_reconcile_open_prs_closes_superseded(
729 monkeypatch: pytest.MonkeyPatch,
730):
731 monkeypatch.setenv("FROOT_RECONCILE", "1")
732 upgrades = (
733 make_upgrade("left-pad", current="1.4.2", available=("1.4.3", "1.4.4")),
734 )
735 stale = make_pr(number=5, branch="froot/dependency-patch/left-pad-1.4.3")
736 fake = FakeForge(open_prs=(stale,))
737 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
738 monkeypatch.setattr(
739 registry_mod,
740 "package_manager_for",
741 lambda ecosystem: FakePackageManager(upgrades),
742 )
743 closed = await activities.reconcile_open_prs(
744 ReconcileInput(target=make_repo())
745 )
746 assert closed == 1
747 assert fake.closed == [5]
748 assert fake.deleted_branches == [stale.branch]
751async def test_reconcile_open_prs_noop_when_disabled(
752 monkeypatch: pytest.MonkeyPatch,
753):
754 monkeypatch.setenv("FROOT_RECONCILE", "0")
755 fake = FakeForge(open_prs=(make_pr(),))
756 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
757 closed = await activities.reconcile_open_prs(
758 ReconcileInput(target=make_repo())
759 )
760 assert closed == 0
761 assert fake.closed == []
764async def test_gate_selftest_healthy_under_default_policy(
765 monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
766):
767 # The default live policy refuses every known-bad class -> nothing escapes,
768 # and the heartbeat logs healthy=True at INFO.
769 for var in (
770 "FROOT_AUTOMERGE_MIN_RATE",
771 "FROOT_AUTOMERGE_MIN_DECIDED",
772 "FROOT_AUTOMERGE_MAX_DEFECT_RATE",
773 ):
774 monkeypatch.delenv(var, raising=False)
775 with caplog.at_level("INFO", logger="froot.gate"):
776 escaped = await activities.gate_selftest(
777 GateSelfTestInput(target=make_repo())
778 )
779 assert escaped == ()
780 record = json.loads(caplog.records[-1].getMessage())
781 assert record["event"] == "gate_selftest"
782 assert record["healthy"] is True
783 assert record["escaped"] == []
786async def test_gate_selftest_alarms_when_config_loosens_the_gate(
787 monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
788):
789 # Drift: a steward raises the defect ceiling in config. The probe runs
790 # against the *live* policy, so a known-bad class now escapes and the alarm
791 # is logged at ERROR.
792 monkeypatch.setenv("FROOT_AUTOMERGE_MAX_DEFECT_RATE", "1.0")
793 with caplog.at_level("ERROR", logger="froot.gate"):
794 escaped = await activities.gate_selftest(
795 GateSelfTestInput(target=make_repo())
796 )
797 assert "a defect on record" in escaped
798 record = json.loads(caplog.records[-1].getMessage())
799 assert record["healthy"] is False
800 assert caplog.records[-1].levelname == "ERROR"
803async def test_judge_changelog_degrades_to_unknown_on_model_error(
804 monkeypatch: pytest.MonkeyPatch,
805):
806 changelog = Changelog(
807 package="left-pad", version=ver("1.4.3"), text="fixes"
808 )
809 monkeypatch.setattr(
810 changelog_mod,
811 "HttpChangelogSource",
812 lambda: FakeChangelogSource(changelog),
813 )
815 class _BoomJudge:
816 async def judge(
817 self, changelog: Changelog, loop: object = None
818 ) -> object:
819 raise RuntimeError("ollama unreachable")
821 monkeypatch.setattr(model_mod, "PydanticAiJudge", lambda: _BoomJudge())
822 # The model is non-load-bearing: its failure degrades to unknown, never
823 # failing the activity (which would stall the spine on a flaky model).
824 verdict = await activities.judge_changelog(
825 JudgeInput(candidate=make_candidate())
826 )
827 assert isinstance(verdict, UnknownVerdict)
828 assert "unavailable" in verdict.rationale.lower()
831async def test_judge_changelog_passes_loop_to_model(
832 monkeypatch: pytest.MonkeyPatch,
833):
834 fake = FakeJudge(CleanVerdict(rationale="ok"))
835 changelog = Changelog(package="x", version=ver("1.4.3"), text="notes")
836 monkeypatch.setattr(
837 changelog_mod,
838 "HttpChangelogSource",
839 lambda: FakeChangelogSource(changelog),
840 )
841 monkeypatch.setattr(model_mod, "PydanticAiJudge", lambda: fake)
842 await activities.judge_changelog(
843 JudgeInput(candidate=make_candidate(), loop=Loop.SECURITY_PATCH)
844 )
845 assert fake.loops == [Loop.SECURITY_PATCH]
848async def test_gate_review_approves_a_clean_re_read(
849 monkeypatch: pytest.MonkeyPatch,
850):
851 changelog = Changelog(package="left-pad", version=ver("1.4.3"), text="fix")
852 monkeypatch.setattr(
853 changelog_mod,
854 "HttpChangelogSource",
855 lambda: FakeChangelogSource(changelog),
856 )
857 fake = FakeJudge(gate_verdict=CleanVerdict(rationale="re-read clean"))
858 monkeypatch.setattr(model_mod, "PydanticAiJudge", lambda: fake)
859 verdict = await activities.gate_review(
860 GateReviewInput(candidate=make_candidate(), pr=make_pr(number=7))
861 )
862 assert isinstance(verdict, CleanVerdict) # clean = approve the merge
863 assert fake.gate_loops == [Loop.DEPENDENCY_PATCH]
866async def test_gate_review_holds_when_no_changelog(
867 monkeypatch: pytest.MonkeyPatch,
868):
869 # Fail-closed: nothing to review -> a non-clean verdict, so the bump never
870 # merges unattended.
871 monkeypatch.setattr(
872 changelog_mod, "HttpChangelogSource", lambda: FakeChangelogSource(None)
873 )
874 verdict = await activities.gate_review(
875 GateReviewInput(candidate=make_candidate(), pr=make_pr(number=7))
876 )
877 # UnknownVerdict is a hold — never "clean", so the bump never merges.
878 assert isinstance(verdict, UnknownVerdict)
881async def test_gate_review_holds_on_model_error(
882 monkeypatch: pytest.MonkeyPatch,
883):
884 # Fail-closed the other way: a reviewer that errors holds, never approves.
885 changelog = Changelog(package="left-pad", version=ver("1.4.3"), text="fix")
886 monkeypatch.setattr(
887 changelog_mod,
888 "HttpChangelogSource",
889 lambda: FakeChangelogSource(changelog),
890 )
892 class _BoomReviewer:
893 async def gate_review(
894 self, changelog: Changelog, loop: object = None
895 ) -> object:
896 raise RuntimeError("ollama unreachable")
898 monkeypatch.setattr(model_mod, "PydanticAiJudge", lambda: _BoomReviewer())
899 verdict = await activities.gate_review(
900 GateReviewInput(candidate=make_candidate(), pr=make_pr(number=7))
901 )
902 # A reviewer that errors holds (unknown), never approves.
903 assert isinstance(verdict, UnknownVerdict)
906async def test_open_pull_request_security_loop_namespaces_branch(
907 monkeypatch: pytest.MonkeyPatch,
908):
909 fake = FakeForge(
910 existing_pr=None,
911 opened_pr=make_pr(
912 number=8, branch="froot/security-patch/left-pad-1.5.0"
913 ),
914 )
915 monkeypatch.setattr(github_mod, "GitHubForge", lambda: fake)
916 monkeypatch.setattr(
917 registry_mod,
918 "package_manager_for",
919 lambda ecosystem: FakePackageManager(),
920 )
921 # A security bump is often a minor — the generalized Candidate allows it.
922 candidate = make_candidate(
923 package="left-pad", current="1.4.2", target="1.5.0"
924 )
925 await activities.open_pull_request(
926 OpenPrInput(
927 target=make_repo(),
928 candidate=candidate,
929 verdict=CleanVerdict(rationale="x"),
930 loop=Loop.SECURITY_PATCH,
931 )
932 )
933 assert fake.pushed is not None
934 assert fake.pushed.value.startswith("froot/security-patch/")
The full suite is green: 503 tests (34 new), both determinism gates clean, ruff and mypy clean. Run against a throwaway git repo, the loop produces exactly the diff a reviewer would want: the dead file deleted whole, the unused export's export stripped, its body and the still-used sibling left alone.