Files
msd-core/docs/issueevidence/1192-adr-test-audit-2026-06-13.md
Tom Boucher ffd5370464 fix(#2903): use the command form that actually works in reader-facing docs (#3047)
* fix(#2903): use the command form that actually works in reader-facing docs

Docs told readers to type the colon form, which no runtime registers -- 18 of
19 runtimes use slash-hyphen and the 19th uses shell-var -- so anyone copying an
example got an unrecognized command. Swept 178 occurrences across 53 files,
locale mirrors included so they do not re-diverge from English.

The colon form is a source-authoring token, not a user-facing one: install-time
converters key on it to produce the hyphen form runtimes actually register. So
the sweep is scoped, and three things are deliberately left alone:

- ADRs, which are a historical record; editing their prose falsifies what was
  written at the time.
- The legacy release-notes archive, pending a maintainer decision on whether it
  follows the same historical carve-out. Excluding it keeps a later reversal
  additive rather than a revert.
- Source artifacts under commands, workflows and agents, where the colon form is
  load-bearing. Rewriting those would break the installed-skill guarantee across
  every runtime -- the single largest hazard here.

The plugin namespace form is a real, separate token and survives untouched.

Adds a lint enforcing exactly that boundary, since the correct form genuinely
differs by directory and nothing previously caught the drift.

Also fixes a hardcoded colon form in the capability-matrix generator. The sweep
alone would have left the generated matrix disagreeing with the template that
produces it, so the fix is at the source and the output regenerated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#2903): stop the sweep misquoting source frontmatter

Adversarial review caught three lines where the sweep rewrote a citation of the
literal YAML name: key from a source command file. That key genuinely is the
colon form -- this change's own carve-out logic says source-authoring tokens keep
it -- so the docs ended up misquoting the real files. One of the three is an
acceptance-checklist assertion, which the sweep turned into a false statement.

Restored the three citations to match their sources verbatim, surgically: where a
line carried both a name: citation and a real reader-facing slash command, only
the citation reverted and the command stayed corrected.

The guard needed the same distinction, or it would have flagged the restoration
and reddened the build: a gsd:<cmd> token preceded by name: is a citation of a
source token and is now permitted. The exemption is deliberately narrow -- a bare
gsd:<cmd> anywhere else still fails -- with a test pinning that narrowness.

Also makes the detection case-insensitive. Review found /GSD:next slipped through
silently; no such casing exists in the tree today, so this closes a latent gap
rather than fixing a live one.

Swept the whole tree for further corrupted citations: none beyond the three.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#2903): retire the stale-next invariant and sweep next like every other command

Maintainer decision on a genuine conflict between two contracts.

Invariant #3054 banned the literal /gsd-next from user-facing docs because it
named a retired workflow-advance command. But commands/gsd/next.md is a live
command -- the state-aware smart-entry launcher -- and this issue requires docs
to use the hyphen form every runtime actually registers. Both could not hold for
this one command, so docs had been sidestepping the ban by keeping the colon
form, which is exactly the defect this issue exists to remove.

FEATURES.md already recorded the reassignment: the hyphen form "is not the
retired workflow-advance command; it is reserved for the state-aware smart-entry
launcher. Workflow advancement remains under /gsd-progress --next." With that
reassignment the invariant's premise is obsolete and the guard now contradicts
the documented command form, so it is retired with a comment recording why
rather than deleted silently.

next is now swept like every other command, and the earlier exemption added to
the new guard is removed so nothing is special-cased.

Four citations of the literal name: frontmatter key stay in colon form, because
the source file really does carry name: gsd:next and a doc quoting it must
reproduce it verbatim. Two of those lines were reworded to say which side is the
frontmatter key and which is the slash command, since they previously conflated
the two.

Verified the retired scan would now genuinely fail against this tree -- the
conflict was real and resolved, not dodged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#2903): backfill changeset pr number

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 13:23:44 -04:00

35 KiB
Raw Blame History

ADR Test-Coverage Audit — 2026-06-13

Lead QA architect final report. Scope: 38 ADRs in docs/adr/, evaluated with the qa-test-architect methodology, plus 4 platform-capability lenses over the test infrastructure. Every RETIRE verdict was independently re-checked by an adversarial verification pass.

Scope note. ADR-857 (capability system core) was intentionally excluded from this audit — it is being worked by another instance. The capability cluster (ADR-894, ADR-959, ADR-1016, ADR-1143) is coupled to ADR-857, so all findings for those four ADRs are provisional and may shift once ADR-857 lands.


Executive summary

The test portfolio is, on the whole, healthier than its volume suggests: the deterministic seams (dispatch hub, model catalog, planning-path resolution, installer migration, shell-command projection) are genuinely behavioral and well-covered, and the project has built real test-rigor infrastructure (an injectable clock seam, fast-check with a pinned seed, Stryker mutation testing wired into CI, and four custom ESLint rules). The dominant problem is not absent tests — it is enforcement gaps that let the rigor infrastructure run as theater, and a long tail of weak tests concentrated in three families: source-text-grep assertions, tautological assert.ok(true) placeholders, and architecture-retirement guards that can never go red.

Five headline problems demand attention first: (1) Stryker mutation testing covers only ~4–6% of the lib surface and excludes every high-risk module (state, phase, verify, init, commands), with the break threshold set to 50% instead of the ADR-456-mandated 80% — the mutation gate trivially passes for PRs touching those modules. (2) 332 of 335 allow-test-rule exemptions carry no tracking issue, directly violating ADR-456's delete-bad-tests policy. (3) A subdirectory blind spot in scripts/run-tests.cjs silently excludes all of tests/observability/ and tests/dispatch/ from npm test — meaning ADR-227's entire UUID-validation seam is unguarded by any continuously-executed test. (4) Multiple ADRs marked Accepted or Proposed have zero attributed tests or test files that cannot run because their source modules are gitignored build artifacts absent from a clean worktree (ADR-218, ADR-0011 family). (5) A scatter of vacuous and self-referential tests (verify-test-quality.test.cjs, assert.ok(true) placeholders, environment-dependent path-guard branches) that provide false confidence on security-relevant code.

Do first: raise the Stryker break threshold and bring core.cts/config-loader.cts into the mutation-covered set (P0); fix the run-tests.cjs non-recursive file scan so observability/dispatch tests actually execute (P0); retire the 5 verified-worthless tests and redesign the 6 downgraded retire-candidates (P0/P1).


Testing-platform robustness

Synthesis of the four platform lenses: adr-456-test-rigor, coverage-mutation, anti-pattern-sweep, determinism-flake.

(a) What the platform CAN do today

  • Deterministic clock seam (tests/helpers/clock.cjs, makeFakeClock) — well-designed and correctly used by lock/state tests (clock-seam.test.cjs, perf-407, bug-474). Eliminates wall-clock races in the lock subsystem.
  • Property-based testing via fast-check with a pinned seed (tests/helpers/fast-check-setup.cjs, seed=42) — 9 *.property.test.cjs files exist and run reproducibly.
  • Mutation testing — Stryker is configured (stryker.config.mjs) and wired into a PR-gating workflow (.github/workflows/mutation.yml) that shards per changed module.
  • Custom ESLint rules — no-source-grep, no-magic-sleep-in-tests, no-elapsed-assertion, no-raw-rmsync-in-tests, plus no-only-tests — exist and are unit-tested via RuleTester (tests/eslint-rules.test.cjs).
  • Env hermeticity — scripts/run-tests.cjs deletes GSD_PROJECT/GSD_WORKSTREAM before spawning child test processes (mitigates the ambient-env hazard recorded in MEMORY.md).
  • Line-coverage gate — c8 --lines 70 (lib) enforced in CI as part of the unit lane.

(b) ENFORCED vs aspirational

Mechanism Status Note
Clock seam in lock/state tests Enforced Used correctly; but not mandated by lint — bug-3707 and phase.test.cjs:4362 still use real Date.now()
fast-check seed pinned Enforced seed=42; but no lint rule requires *.property.test.cjs to import the setup, and context-utilization.property.test.cjs uses Math.random() inside fc.property, breaking reproducibility
Stryker mutation gate Aspirational Runs in CI, but break=50% (not 80%) and covers 6/109 modules; trivially passes for excluded modules
Line coverage 70% Enforced But satisfiable via CLI integration tests that execute lines without asserting sub-behaviors; no branch-coverage floor
no-magic-sleep-in-tests Enforced (error) Promoted to error
no-raw-rmsync-in-tests Enforced (error)
no-only-tests Aspirational Wired at error but has zero RuleTester coverage and no integration test
no-source-grep Aspirational At warn only, and NOT applied to tests/** at all — the file-level allow-test-rule bypass blanket-exempts source-grep in test files
no-elapsed-assertion Aspirational At warn; only matches .elapsed/.duration/.took/.ms properties, missing Date.now()-start and computed-variable patterns (e.g. elapsedMs)
allow-test-rule tracking-issue requirement Not enforced 332/335 exemptions lack an issue number; no CI gate prevents new untracked exemptions
Subdirectory test discovery Broken run-tests.cjs uses flat readdirSync(testDir); tests/observability/ and tests/dispatch/ are silently never run by npm test
Fail-on-zero-executed Not enforced run-tests.cjs exits 0 when a suite resolves to zero files

(c) Prioritized capability upgrades

Capability Current state Recommendation Priority
Recursive test discovery Flat readdirSync skips tests/observability/, tests/dispatch/ Replace with the existing findTestFilesRecursive (already in the same file) P0
Stryker break threshold break: 50 in stryker.config.mjs Raise to 80 per ADR-456; move sub-80% modules to UNMUTATED with documented reasons P0
Mutation coverage breadth 6/109 modules covered; 12 critical modules excluded Graduate one module/release cycle starting with core.cts, config-loader.cts; require any module with a *.unit/property.test.cjs to be in COVERED or UNMUTATED P0
allow-test-rule audit 332/335 untracked exemptions Lint gate requiring either an approved category tag or #NNN; one-time tracking-issue sweep P1
no-source-grep in test files Not applied to tests/**; warn-only Apply at error to tests/**; split rule to allow product-file (.md/.json) reads but block source-file (.cjs/.js/.ts) grep assertions P1
Branch-coverage floor Only line coverage gated Add --branches 60; per-file branch floor for the UNMUTATED critical modules P1
no-elapsed-assertion reach Property-name match only Extend to Date.now()-start binary expressions and identifiers ending Ms/Elapsed/Duration in assert args P1
no-only-tests coverage Wired error, untested Add RuleTester valid/invalid cases + an integration assertion it fires P1
no-tautological-assert rule None New ~30-line local rule flagging assert.ok(true)/assert.ok(1) in tests/** P1
Fail-on-zero-executed Exits 0 on empty suite Exit 1 by default; gate the empty-suite allowance behind RUN_TESTS_ALLOW_EMPTY_SUITE=1 P2
Shared-GSD_TEMP_DIR isolation reap tests write to global os.tmpdir()/gsd Redirect via the existing dir option to per-test tmpdirs; lint against writing to the shared path without cleanup P2
Build-artifact preflight in artifact-dependent tests Top-level require() of gitignored .cjs hard-crashes on clean worktree before() existsSync guard with a clear "run npm run build:lib" message P2

ADR-456 compliance — per mandate

ADR-456 defines four test-rigor mandates. Compliance is partial on all four:

  • (a) Concurrency via injectable clock seam, not real OS races — Substantially met. The clock seam exists and the lock subsystem uses it. Gaps: bug-3707-locked-worktree-cleanup.test.cjs computes stale-lock mtimes from real Date.now() (7 sites); phase.test.cjs:4362 asserts a 60-second wall-clock window; core.test.cjs timeAgo boundary tests use real clock at the 5s bucket edge. No lint rule mandates the seam.
  • (b) fast-check + Stryker antagonistic tier, 80% kill threshold — Largely aspirational. fast-check is wired with a pinned seed, but Stryker covers ~4–6% of source lines, excludes all high-risk modules, and the break threshold is 50%, not the mandated 80%. The antagonistic tier exists in name but does not guard the dangerous surface.
  • (c) Assert on typed --json fields and exported registries, not rendered text or source literals — Partially met, weakly enforced. New tests largely follow it, but no-source-grep is warn-only and not applied to test files, so source-grep assertions persist (e.g. install.test.cjs .includes() on bin/install.js; sh-hook-paths.test.cjs variable-name scans). Several CLI tests still assert on stdout.includes() substrings rather than parsed JSON.
  • (d) Delete pass-always / vacuous / source-grep / elapsed / real-race / permanent-allow-rule tests — Not met. Concrete pass-always tests survive (research-cli.test.cjs:423, worktree-baseref-install.test.cjs:444, 24 assert.ok(true) in eslint-rules.test.cjs); 332/335 allow-test-rule exemptions are effectively permanent and untracked, directly contradicting the policy clause requiring // allow-test-rule: see #NNN on post-ADR exemptions.

Per-ADR coverage

ADR Status #Tests Happy Boundary Negative Independence Verdict
0001 Dispatch Policy Module Accepted 12 covered partial covered covered Core protected; SDK bridge / exit-code mapping unverified
0002 Command Contract Validation Accepted 6 covered partial missing covered Integration-strong; helper unit tests + lint negatives missing
0003 Model Catalog Module Accepted 13 covered partial covered covered CJS solid; SDK-parity invariant untested
0004 Worktree/Workstream Seam Accepted 13 covered partial covered covered Substantially protected; cross-path naming + .planning/.lock 2-proc gaps
0005 SDK Architecture Seam Map Superseded (0174) 13 partial partial partial covered Retirement guards OK; thin-adapter enforcement absent
0006 Planning-Path Projection Accepted 6 partial partial covered covered Critical gap: init-handler planningPaths() consumption untested
0007 SDK Package Seam Superseded (0174) 10 covered partial partial covered Retirement guarded; classifyPromptUserAction boundary untested
0008 Installer Migration Module Accepted 16 covered partial covered covered Rich suite; rewrite-json apply + local-scope gaps
0009 Shell Command Projection Accepted 21 covered partial covered covered Strong; 3 ADR-listed test files absent; source-grep in sh-hook-paths
0010a Skill Surface Budget Proposed (impl w/ gaps) 8 covered partial partial covered ~75% protected; stageCommandsForRuntimeFlat + migration one-shot untested
0010b File Operation Engine Superseded (0009) 10 covered partial covered covered Canonical seam solid; divergent atomicWriteFileSync + path-containment gaps
0011a Skill Surface Budget Accepted 12 covered partial partial covered Broad; source modules absent from worktree → MODULE_NOT_FOUND; cyclic-requires untested
0011b Review Default Reviewers (PRD) Draft 5 partial partial partial covered infos-path + --all+flags edge untested; build-artifact dependency
0011c Review Default Reviewers Proposed 4 covered partial partial covered Mid-tier risks (infos, empty-selection) missing; build-artifact dependency
0012 Command Routing Hub Superseded (0174) 5 covered partial covered covered Hub well-covered; uat-passed adapter arg-shape gap; 3 redundant tests
15 Autonomous Cross-AI Convergence Proposed 8 partial partial partial partial Primary surface (/gsd-progress --next --auto --converge) has zero coverage
22 Plan Drift Guard Proposed 6 partial partial covered covered Severity mapping, authority auto-upgrade, rung≥3 hard-block untested
58 Runtime Install Policy Accepted 7 covered partial partial covered Strong projection coverage; purity guarantee + sandboxTier-invalid untested
0174 Retire gsd-sdk Package Accepted 12 covered covered covered covered Strong & behavioral; init-composer trace + tsc-output invariant gaps
218 Release Version Validation Accepted 0 missing missing missing na ZERO TESTS — entirely unprotected (regex lives only in release.yml bash)
227 Input Validation Shape Accepted 5 covered covered partial partial All 5 test files silently excluded from npm test (subdir blind spot)
230 Next Integration Branch Proposed 5 partial partial partial covered Most critical seams untested; 2 files misattributed (unrelated features)
415 Stale-Base Token Reintro Accepted 12 covered partial partial covered Token reintro guarded; propagator + ruleset-strict-mode untested
443 Unified Effort / Fast-Mode Proposed 12 covered covered covered partial Resolver exceptionally covered; end-to-end effort propagation untested
452 ESLint Lint Harness Accepted 2 partial partial partial covered Local rules covered; no-only-tests + plugin-n + script-retirement untested
456 Test Rigor Architecture Accepted 19 covered partial partial covered Clock seam strong; mutation breadth + warn→error + allow-rule tracking gaps
457 Generated CJS Single Source Accepted 8 covered partial partial covered Value-baking solid; .gitignore↔eslint sync + emit-path gaps
550 Spec-Phase Probe Contract Accepted 6 covered covered covered covered Core seam strong; Decisions 4(b–d)/6 deferred to #644 (untested by design)
0656 Research Module Seam Accepted 9 covered covered covered covered Strong; slopcheck "cannot-lower" + scrape-dispatch gaps
660 Release From Next Head Proposed 13 partial partial covered covered Scripts covered; orchestration (release.yml/auto-backmerge) largely untested
766 Claude Plugin Manifest Accepted 3 covered partial partial partial agents-absence + post-#770 event drift untested; CI-skipped schema validate
894 Capability Decl Format Proposed (coupled 857) 5 covered partial covered partial Provisional. Strong; when-ownership, tier-monotone std→full gaps
959 Capability Command Contribution Proposed (coupled 857) 6 covered partial covered covered Provisional. Strong; proto-pollution family-name + case-removal gaps
1016 Runtime Capability Descriptor Proposed (coupled 857) 9 covered partial covered covered Provisional. Dense; sandboxTier fail-loud + 5f required-axes gaps
1143 Claude Orchestration Proposed (coupled 857) 9 missing missing missing na Provisional. No impl; primary surface entirely untested
3524 CJS-SDK Hard Seam Superseded (0174) 12 partial partial covered covered Drift-bugs guarded; all subjects gitignored artifacts (crash on clean worktree)
3660 Runtime Artifact Layout Accepted 6 covered partial partial covered Core protected; findInstallSourceRoot fail-path + agent tools:-strip gaps

ADRs with zero tests or notable gaps

ADR-218 (Release Version Validation) — ZERO attributed tests. This is the most acute hole. Both ADR decisions — the leading-zero-rejecting format regex (^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.0$) and the npm duplicate-version precheck — live exclusively in a bash run: block in .github/workflows/release.yml. No test exercises either. The 1.01.0 incident that motivated the ADR caused an irrecoverable production divergence, and nothing prevents a reversion to the old broken regex. issue-version-gate.test.cjs is a superficially-related but wholly separate feature. This is a critical gap in a high-blast-radius zone.

ADR-227 (Input Validation Shape) — 5 tests that never run. All five attributed files live in tests/observability/ and tests/dispatch/, which scripts/run-tests.cjs:269 never enters (flat readdirSync). The tests are individually high quality (8 boundary/negative UUID-v4 cases, real file I/O, per-test isolation) but are silently excluded from npm test, CI, and gsd-test. ADR-227's two-layer validation guarantee — the correlation-poisoning and log-blowup classes it was written to prevent — is effectively unguarded until the runner is fixed.

ADR-0006 (Planning-Path Projection) — critical handler-consumption gap. The ADR's single most important invariant — that cmdInitExecutePhase/PlanPhase/PhaseOp/MilestoneOp route through planningPaths().planning rather than flat path.join(cwd, '.planning') — has no regression test. The banned .planning/projects/<project> form also has no guard.

ADR-15 (Autonomous Cross-AI Convergence) — primary surface untested. The ADR's designated primary operator surface, /gsd-progress --next --auto --converge, is both unimplemented and untested; the existing cross-ai-execution.test.cjs tests a different feature (execute-phase delegation) and is misattributed.

ADR-0011a / 0011b / 0011c (Skill Surface / Review Default Reviewers). Source modules (install-profiles.cjs, surface.cjs, clusters.cjs, review-reviewer-selection.cjs, config-schema.cjs) are gitignored build artifacts absent from a clean worktree; all attributed tests throw MODULE_NOT_FOUND without a prior npm run build:lib. CI's pretest hook covers this, but isolated runs and fresh worktrees fail confusingly.

ADR-1143 (Claude Orchestration) — provisional, no implementation. Proposed and blocked on ADR-857; the capability directory does not exist, so the primary surface has zero coverage. Findings are provisional pending ADR-857.


Weak tests

Retire

Only entries whose adversarial verification confirmed finalVerdict === "retire". 5 tests survived verification as truly worthless.

File Location Why worthless Confirmed by verification
tests/enh-2790-skill-consolidation.test.cjs Lines 127–157 — 4 "has a name: field" spot-checks Assert only fm.name.length > 0; command-contract.test.cjs already enforces the stricter /^gsd[:-]/ prefix on all 67 files. Renaming gsd-capture→capture (the exact regression) passes these but is caught by command-contract. Cannot go red for the bug class the ADR exists to prevent. Yes — verified command-contract.test.cjs:44-54 covers these 4 files more strictly; deletion creates no coverage gap.
tests/command-routing-hub.test.cjs Lines 134–137 — "constructs successfully with only cjsRegistry" Duplicate of the construction test at 71–75; asserts only typeof hub.dispatch === 'function' with no dispatch. createHub does zero registry-dependent branching at construction. Yes — verified identical assertion; populated-registry dispatch path covered by the happy-path block at line 142.
tests/command-routing-hub.test.cjs Lines 515–519 — "ERROR_KINDS.UnknownCommand === result.kind" The identical assertion appears at 9 other sites (222, 234, 245, 256, 360, 393, 410, 422, 518). Same code path covered at 414–424. No unique setup/edge/field. Yes — verified superset coverage by lines 384–424 + the constant-stability test at 521–526.
tests/no-cjs-sdk-handsync-tooling.test.cjs All 3 tests (lines 26–53) Assert absence of lint-shared-module-handsync.cjs / allowlist JSON that never existed on main (CONTEXT.md:693 confirms). Can only go red if someone manually drops those exact files. No behavioral coverage. Yes — verified the real retired artifacts (cjs-sdk-bridge.cjs, sdk/) are covered by bug-190; these guard ADR-text-only artifacts.
tests/runtime-artifact-layout.test.cjs Lines 319–323 — "cline has one skills kind (#782)" Default-scope call is behaviorally identical to the explicit-global test at 249–265, which asserts a strict superset (kinds.length, kind, destSubpath, prefix, stage). Yes — verified default scope = global (source line 430); also independently covered by bug-782-cline-skills-emission.test.cjs:629-636.

Redesign / improve

All redesign + keep-but-improve verdicts, plus the 6 retire-candidates that verification downgraded to redesign. (Representative selection across ADRs; the full set is large — these are the highest-signal entries and every downgraded retire-candidate.)

Downgraded retire-candidates (verification overruled the retire verdict — do NOT delete)

File Location Problem What the rewritten test should assert
tests/bug-3135-capture-backlog-workflow.test.cjs Lines 136–182 broad @-ref regression The block IS redundant (only checks /workflows/ refs, misses 24 files with references//templates/ refs that command-contract Rule 4 catches) — delete ONLY that block. But the FILE's lines 1–132 test unique add-backlog.md content invariants with no equivalent elsewhere. Delete lines 134–182 only. Retain content-contract assertions: add-backlog.md uses phase.next-decimal, writes ROADMAP before mkdir, 999.x scheme; capture.md routes to add-backlog.md.
tests/review-reviewer-selection.test.cjs Lines 115–122 result-shape test Shape-only on { detected: [] }. But the empty-detected call exercises the no_config_all_detected else-branch that no other test asserts (existing test uses ['gemini']). Replace shape assertions with behavioral: source === 'no_config_all_detected', selected/errors/warnings === [] for the empty-detected fallback.
tests/review-reviewer-selection.test.cjs Lines 17–27 KNOWN_REVIEWER_SLUGS Tests a compiled-CJS constant's shape. Verification found the slug membership DOES drive resolveReviewerSelection's warn/drop logic at runtime — it's the only guard against silent slug-list drift. Collapse to one behavioral test: resolveReviewerSelection with a config'd known slug yields no warning + slug in selected; an unknown slug yields a warning.
tests/enh-191-retire-sdk-package.test.cjs Lines 52–58 AGENTS.md absence Off-topic for SDK retirement, but NOT valueless: bin/install.js:~10963 writes a root AGENTS.md on local Copilot install; this is the only guard against that artifact committing. Extract to a repo-layout governance test (e.g. tests/repo-layout.test.cjs); rename to state the real invariant (no ad-hoc root instruction file vs CONTEXT.md/ADRs); link the copilot install path.
tests/capability-registry.test.cjs Lines 558–584 drift test Steps 2–3 are a file-I/O tautology and skip the real --check path (stripGeneratedComment + normalizeLineEndings). But it is the only unit-level coverage of the drift-comparison pipeline. Call the actual comparison expression: assert a stale version survives both stripGeneratedComment and normalizeLineEndings and is detected; assert a comment-only-timestamp change is NOT flagged after stripping.
tests/verify-test-quality.test.cjs 3 describes (disabled / circular / assertion-strength detection) Self-referential: define a regex inline, write a matching fixture, assert the regex matches — cannot go red for any production change. The detector lives as prose in verify-phase.md. Replace with a structural guard reading gsd-core/workflows/verify-phase.md: assert the audit_test_quality step tag exists and contains the skip-pattern, circular-detection, and assertion-strength-table markers.

Other redesign / keep-but-improve (high-signal selection)

File Location Problem What the rewritten test should assert
tests/research-cli.test.cjs Line 423 Pass-always assert.ok(true) placeholder; the 2-package arg-parse regression is undefended. Extract the flag parser; call with ['--bad-flag','pkgA','pkgB'] and assert unknown-flag error + that pkgA/pkgB are not consumed as flag values. (No network.)
tests/worktree-baseref-install.test.cjs Line 444 Pass-always assert.ok(true); the null-settings #683 guard is never exercised. assert.doesNotThrow(() => applyWorktreeBaseRef(null, tmpDir)) and assert no worktree.baseRef is written.
tests/bug-260-worktree-path-guard.test.cjs Line 338 Environment-dependent branch silently passes via assert.ok(true) when the traversal path resolves inside the worktree — a security guard goes untested on macOS/private-symlink hosts. Construct the traversal target in a separate tmpdir outside the worktree; remove the if/else; unconditionally assert result.status === 2 / decision block.
tests/eslint-rules.test.cjs 24 trailing assert.ok(true) calls RuleTester already throws on failure; the trailing literal asserts nothing and inflates count. Delete the assert.ok(true) lines; optionally add assert.strictEqual(typeof rule.create, 'function') per describe to confirm the module loaded.
tests/phase.test.cjs Lines 4343–4368 (last_updated refresh) Math.abs(new Date() - updatedAt) < 60_000 wall-clock window — flaky under loaded CI. Assert last_updated is a valid ISO timestamp ≠ the stale seed value AND its date portion equals today's UTC date.
tests/bug-3707-locked-worktree-cleanup.test.cjs 7 Date.now() stale-mtime sites Real-clock coupling for stale-lock boundary; violates the ADR-456 clock-seam mandate. Pin a constant epoch (PINNED_NOW_MS) for mtime fixtures; pass it to the SUT as a clock parameter where supported.
tests/context-utilization.property.test.cjs Line 187 Math.floor(Math.random()*…) inside fc.property breaks reproducibility under the pinned seed. Replace with fc.integer({ min: 0, max: contextWindow }); split a shape test from a value-correctness test at known ratios.
tests/research-provider.property.test.cjs Single test Only "never throws + valid type"; inverting HIGH/LOW classification passes. Keep robustness property; add example-based unit tests for classification boundaries; add module to mutation COVERED.
tests/research-store.property.test.cjs All 3 tests Shape/determinism only; cache-key collisions or over-normalization invisible. Add a collision property (distinct inputs → distinct keys) + boundary examples (npm/lodash ≠ npm/react).
tests/install.test.cjs Lines 683–712 (Kilo source-grep) Reads bin/install.js as a string and .includes() internal symbol names — source-grep, breaks on rename, passes on dead code. Invoke install.js with --kilo in a tmpdir; assert on the installed config/manifest content (or export+call the kilo-detection logic).
tests/sh-hook-paths.test.cjs Tests 2–4 (lines 119–194) Source-grep variable-name scans of install.js/runtime-hooks-surface.cts; behavior already covered by Test 1's behavioral call. Call buildHookCommand per .sh hook for linux+win32; assert non-empty, bash runner, absolute quoted configDir prefix.
tests/clusters.test.cjs Lines 50–58 `assert.ok(utilitySize >= skills.length
tests/model-profiles.test.cjs Lines 115–122 Test titled "resolution order: override > profile > default" exercises no override and asserts nothing about override behavior. Use a temp project with config.model_overrides, call resolveModelInternal, assert override > profile > default explicitly.
tests/active-workstream-store.unit.test.cjs Lines 361–364 `assert.ok(key === null
tests/feat-488-effort-sync.test.cjs Lines 177–219 Does not redirect GSD_HOME; resolved effort depends on the dev's real ~/.gsd; degrades to a type-only check. Set GSD_HOME=tmpHome; assert result.synced === 1 and changes[0].to === 'low'.
tests/bug-492-effort-manifest-fallback.test.cjs Lines 20–48 Mutates the module-level CANONICAL_CONFIG_DEFAULTS singleton — independence violation; races under parallel runs. Drive the same path via a tmpHome ~/.gsd/defaults.json rather than mutating shared module state.
tests/issue-766-plugin-manifest.test.cjs Section C, "claude plugin validate --strict" Permanently skipped when the claude binary is absent (i.e. on CI) — the only full-schema gate never runs. Install the claude CLI in a CI step, OR validate plugin.json against a snapshotted JSON schema unconditionally.
tests/cross-ai-execution.test.cjs Entire file Misattributed to ADR-15; tests a different feature (workflow.cross_ai_execution). Also a shared-mutable let content independence issue. Leave the file for its real feature; create tests/adr-15-plan-strategy-seam.test.cjs for the actual converge seam; hoist content to a single module-level read.
tests/issue-844-manifest-version-sync.test.cjs Describe A (33–127) Order-dependent shared tmpRoot across setup/assert/cleanup tests. Refactor to a single test with per-test before()/after() tmpRoot allocation.
tests/plan-review-convergence.test.cjs Lines 688–692 Four workflow.includes(verdict) whole-file substring checks — pass even if verdicts move to dead/comment text. Extract the source-grounding section slice; assert each verdict + its severity mapping (AMBIGUOUS→MEDIUM, UNCHECKABLE→INFO) inside that slice.
tests/feat-3594-parser-property-style.test.cjs Lines 119–135 assert.ok(elapsedMs < 2000) wall-clock; escapes no-elapsed-assertion via the variable name. Replace with a call-count metric (instrument char comparisons, assert O(n)); or raise bound to 10000ms as a stopgap.
tests/core.test.cjs Lines 1834–1879 (reapStaleTempFiles) Writes to the global shared os.tmpdir()/gsd; races with temp-subdir.test.cjs under concurrency. Use a per-test isolated tmpdir via the existing { dir } option; clean up in afterEach.

Prioritized action plan

P0 — do first (blast radius / false-green)

  1. Fix recursive test discovery in scripts/run-tests.cjs (replace flat readdirSync with the already-present findTestFilesRecursive). This alone reactivates ADR-227's entire UUID-validation suite (tests/observability/, tests/dispatch/) — currently silently excluded from npm test and CI.
  2. Raise Stryker break from 50 → 80 in stryker.config.mjs (ADR-456 mandate). Move any module that cannot meet 80% into UNMUTATED with a documented per-entry reason rather than diluting the global floor.
  3. Add tests for ADR-218 (zero coverage today): extract the release-version regex and npm-duplicate precheck into a testable .cjs helper (or a source-text-is-the-product workflow assertion on release.yml), and assert 1.01.0/01.0.0 are rejected, valid versions pass, and the duplicate-version step exists and is wired before the publish jobs.
  4. Retire the 5 verified-worthless tests (see Retire table): enh-2790 name spot-checks (127–157), command-routing-hub lines 134–137 and 515–519, no-cjs-sdk-handsync-tooling (all 3), runtime-artifact-layout cline edge-case (319–323).
  5. Redesign the 6 downgraded retire-candidates (do NOT delete): bug-3135 (delete only the broad block, keep content tests), both review-reviewer-selection tests, enh-191 AGENTS.md (extract to governance test), capability-registry drift test, verify-test-quality (replace with structural workflow guard).
  6. Close the ADR-0006 critical gap: add a regression test that init execute-phase/plan-phase/phase-op/milestone-op return workstream-scoped paths under GSD_WORKSTREAM, plus a guard against the banned .planning/projects/<project> form.

P1 — high priority (rigor enforcement + concentrated weakness)

  1. Bring core.cts and config-loader.cts into the Stryker COVERED set (smallest high-value modules first); write their *.unit.test.cjs as a prerequisite.
  2. Promote no-source-grep to error and apply it to tests/**, splitting product-file reads (.md/.json, allowed) from source-file grep (.cjs/.js/.ts, blocked). Then fix the surfaced violations (install.test.cjs Kilo block, sh-hook-paths.test.cjs Tests 2–4).
  3. Add the allow-test-rule tracking-issue lint gate + run the one-time sweep to attach #NNN (or an approved category tag) to the 332 untracked exemptions.
  4. Eliminate the pass-always placeholders: redesign research-cli.test.cjs:423, worktree-baseref-install.test.cjs:444, the bug-260 environment-dependent branch, and strip the 24 assert.ok(true) lines in eslint-rules.test.cjs. Add a no-tautological-assert ESLint rule to prevent recurrence.
  5. Add no-only-tests RuleTester coverage and extend no-elapsed-assertion to catch Date.now()-start and computed-variable (elapsedMs) patterns.
  6. Add a branch-coverage floor (--branches 60) and a per-file branch floor for the UNMUTATED critical modules.
  7. De-flake the clock-coupled tests: phase.test.cjs:4362 (60s window → ISO-date assertion), bug-3707 (pin epoch), context-utilization.property.test.cjs:187 (fc.integer not Math.random).
  8. Add the ADR-15 primary-surface tests (/gsd-progress --next --auto --converge) and re-attribute cross-ai-execution.test.cjs to its real feature.

P2 — follow-up (robustness & hygiene)

  1. Fail-on-zero-executed in run-tests.cjs (exit 1 on empty suite by default; gate the allowance behind an env flag).
  2. Isolate the shared GSD_TEMP_DIR reap tests (core.test.cjs:1834-1879, temp-subdir.test.cjs) via the existing { dir } option + afterEach cleanup.
  3. Add build-artifact before() guards to artifact-dependent suites (ADR-3524 subjects, ADR-0011 family) so a clean worktree fails with a clear "run npm run build:lib" message instead of a top-level crash.
  4. Strengthen the strong-but-incomplete ADRs: ADR-443 end-to-end effort propagation, ADR-0656 slopcheck "cannot-lower" + scrape-dispatch, ADR-0009's 3 absent ADR-listed test files, ADR-766 agents-absence assertion + post-#770 hook drift.
  5. Add .gitignore ↔ eslint.config.mjs ignore-list parity test (ADR-457) so a migration that updates one list but not the other is caught.

Capability-cluster items (ADR-894/959/1016/1143) are deliberately deferred — they are coupled to ADR-857 (worked elsewhere) and their findings are provisional. Re-run this audit slice once ADR-857 lands.