Files
msd-core/docs/issueevidence/1192-adr-test-audit-2026-06-13.md
Tom Boucher 5fa4dcd78c fix: recover silently-excluded test dirs + test-architecture audit hardening (#1195)
* fix: recurse test discovery so subdir test suites actually run

scripts/run-tests.cjs discovered tests with a flat readdirSync(testDir),
silently excluding tests/observability/ (4 files), tests/dispatch/ (1) and
tests/installer-migrations/ (1) — 94 passing tests — from `npm test` and all
CI lanes. Walk the tree recursively (relative subpaths preserved), classify
suites by basename, and add a fail-on-zero-executed guard for suite/default
runs (escape hatch GSD_ALLOW_EMPTY_SUITE=1) while preserving the empty
--files/--files-from path the CI inert lane relies on.

Unit suite 735 -> 741 files; surfaces ADR-227's observability/dispatch seam.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: retire 5 verified-worthless tests

Adversarial verification confirmed these 5 prove nothing — their coverage is
provided more strictly elsewhere:
- enh-2790 'has a name: field' spot-checks (command-contract enforces /^gsd[:-]/)
- command-routing-hub duplicate construct + duplicate ERROR_KINDS assertions
- no-cjs-sdk-handsync-tooling (guarded files that never existed on main; bug-190
  covers the real retired SDK artifacts)
- runtime-artifact-layout cline edge case (subsumed by the explicit-global test
  and bug-782-cline-skills-emission)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: add ADR-218 release version-validation coverage

ADR-218 (reject leading-zero versions like 1.01.0; npm duplicate pre-check) had
zero tests — the logic lived only in release.yml bash. Add a test that extracts
the actual rejection regexes from the workflow and exercises them against a
boundary table (leading-zero/malformed rejected, valid accepted) plus structural
wiring assertions. Goes red if the regex is reverted to [0-9]+.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: redesign weak tests into behavioral, deterministic assertions

Per the ADR test audit, rewrite 27 weak test files (test-only, no source
changes) so each can go red for the defect it guards:
- kill pass-always assert.ok(true) placeholders (research-cli, worktree-baseref,
  bug-260 security guard, eslint-rules x24, clusters '|| true')
- replace source-text grep with behavioral calls (install Kilo, sh-hook-paths,
  plan-review-convergence) and add a repo-layout governance test
- de-flake real-clock/Math.random coupling (phase last_updated, bug-3707 mtime,
  context-utilization property, feat-3594)
- fix independence/shared-state violations (bug-492 singleton, issue-844 tmpRoot,
  core reapStaleTempFiles, active-workstream TTY, feat-488 GSD_HOME)
- strengthen property/shape-only tests (research-provider/store classification +
  collision) and unconditional plugin.json schema validation (issue-766)

Verified: all 28 files run together 1220 pass / 0 fail / 1 skip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: add no-tautological-assert lint rule, error in test suite

New custom ESLint rule (eslint-rules/no-tautological-assert.cjs) bans asserts
that can never fail: assert(true)/assert.ok(<always-truthy literal>),
'cond || true' inside an assert, and equality asserts comparing two identical
literals. Wired as error on tests/**; full sweep confirmed zero existing
violations so the suite stays green. Prevents the placeholder-assert regressions
the audit redesigns just removed. RuleTester coverage added (6 valid, 8 invalid).

Note: no-only-tests was already enforced via eslint-plugin-no-only-tests, so no
duplicate rule was added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: gate new allow-test-rule exemptions to require an issue ref

ADR-456 requires any allow-test-rule exemption added after the ADR to carry a
tracking issue number, but nothing enforced it. New ratchet gate
(scripts/lint-allow-test-rule-refs.cjs, wired into lint:ci) fails when a NEW
allow-test-rule comment lacks a #NNN/URL reference; the 323 existing untracked
exemptions are grandfathered in an allowlist that ratchets down as they gain
refs. Red-green verified (novel untracked offender fails; compliant passes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs: add ADR test-audit evidence report (#1192)

Full risk-first qa-test-architect audit of the ADR portfolio (37 ADRs + 4
platform lenses, adversarial verification of retire verdicts) that drove the
P0 discovery fix, ADR-218 coverage, 5 retires, 27 redesigns, and the two new
lint gates. Filed as point-in-time evidence under docs/issueevidence/, named
for tracking issue #1192.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: replace pre-existing raw NUL byte with escape in feat-3594 fixture

feat-3594's null-byte parser fixture contained a literal NUL byte (pre-existing
on next at b10e5681 — confirmed: base blob has 1 NUL, this fix has 0), which
made git treat the file as binary and would break grep/editors. Switch to the
\x00 escape; the runtime string value (a real NUL in the parser input) is
unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: address adversarial-review findings

Codex adversarial pass over the branch:
- capability-registry drift test no longer mutates the committed generated
  capability-registry.cjs in place (concurrency hazard) — uses in-memory
  checkPipeline comparison instead.
- allow-test-rule ratchet now detects exemptions in ALL comment forms (block
  /* */ too, matching no-source-grep) so a block comment can't bypass it;
  one newly-surfaced pre-existing offender grandfathered (323->324).
- install.test Kilo case asserts on what install(false,'kilo') actually writes
  rather than manually calling configureKiloPermissions (masked the call site).
- issue-766 drops the undeclared transitive ajv dep for explicit structural
  assertions from the schema fixture.
- adr-218 test notes the hotfix leading-zero gap is tracked in #1186.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: address code-review findings (subdir discovery, rule + test gaps)

xhigh code review surfaced 15 confirmed issues, all fixed:
- run-tests.cjs --files now resolves subdir tests by bare basename + handles
  Windows backslash paths (ambiguous basenames error clearly).
- affected-tests-lib.cjs listTestFiles made recursive — the targeted CI lane was
  silently dropping changed subdir tests (same false-green class the audit fixed).
- no-tautological-assert now catches 'true || cond' and empty []/{}  equality.
- verify-test-quality: restore provenance-classification coverage, tighten the
  writeFile circular-detection check, guard the module-level file read.
- sh-hook-paths: cover the global-install .sh delegation branch (#2045 guard).
- active-workstream null-guard runs deterministically (no longer skipped on TTY).
- adr-218 structural guards tightened (major/minor leading-zero; needs: membership).
- repo-layout AGENTS.md guard no longer false-alarms on equivalent refactors.
- cross-ai ordering guard fails red when the step is missing.
- issue-766 parses required fields from the schema fixture (auto-enforced).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: stub USERPROFILE alongside HOME in feat-488 (Windows parity)

The feat-488 redesign stubbed process.env.HOME but not USERPROFILE; os.homedir()
resolves from USERPROFILE on Windows, so the home stub was not hermetic there —
caught by windows-test-parity-guard (stubsHomeNoUserProfile). Save/set/restore
USERPROFILE symmetrically with HOME (delete-if-originally-undefined).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: reconcile allow-test-rule allowlist after rebase onto next

Rebasing onto current next pulled in merged PR #1170, which added
inventory-headings-countfree.test.cjs (a baseline allow-test-rule exemption) and
deleted inventory-counts.test.cjs. Grandfather the former and prune the latter so
the ratchet matches the merged tree. No new debt from this PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 23:35:08 -04:00

35 KiB
Raw Blame History

ADR Test-Coverage Audit — 2026-06-13

Lead QA architect final report. Scope: 38 ADRs in docs/adr/, evaluated with the qa-test-architect methodology, plus 4 platform-capability lenses over the test infrastructure. Every RETIRE verdict was independently re-checked by an adversarial verification pass.

Scope note. ADR-857 (capability system core) was intentionally excluded from this audit — it is being worked by another instance. The capability cluster (ADR-894, ADR-959, ADR-1016, ADR-1143) is coupled to ADR-857, so all findings for those four ADRs are provisional and may shift once ADR-857 lands.


Executive summary

The test portfolio is, on the whole, healthier than its volume suggests: the deterministic seams (dispatch hub, model catalog, planning-path resolution, installer migration, shell-command projection) are genuinely behavioral and well-covered, and the project has built real test-rigor infrastructure (an injectable clock seam, fast-check with a pinned seed, Stryker mutation testing wired into CI, and four custom ESLint rules). The dominant problem is not absent tests — it is enforcement gaps that let the rigor infrastructure run as theater, and a long tail of weak tests concentrated in three families: source-text-grep assertions, tautological assert.ok(true) placeholders, and architecture-retirement guards that can never go red.

Five headline problems demand attention first: (1) Stryker mutation testing covers only ~4–6% of the lib surface and excludes every high-risk module (state, phase, verify, init, commands), with the break threshold set to 50% instead of the ADR-456-mandated 80% — the mutation gate trivially passes for PRs touching those modules. (2) 332 of 335 allow-test-rule exemptions carry no tracking issue, directly violating ADR-456's delete-bad-tests policy. (3) A subdirectory blind spot in scripts/run-tests.cjs silently excludes all of tests/observability/ and tests/dispatch/ from npm test — meaning ADR-227's entire UUID-validation seam is unguarded by any continuously-executed test. (4) Multiple ADRs marked Accepted or Proposed have zero attributed tests or test files that cannot run because their source modules are gitignored build artifacts absent from a clean worktree (ADR-218, ADR-0011 family). (5) A scatter of vacuous and self-referential tests (verify-test-quality.test.cjs, assert.ok(true) placeholders, environment-dependent path-guard branches) that provide false confidence on security-relevant code.

Do first: raise the Stryker break threshold and bring core.cts/config-loader.cts into the mutation-covered set (P0); fix the run-tests.cjs non-recursive file scan so observability/dispatch tests actually execute (P0); retire the 5 verified-worthless tests and redesign the 6 downgraded retire-candidates (P0/P1).


Testing-platform robustness

Synthesis of the four platform lenses: adr-456-test-rigor, coverage-mutation, anti-pattern-sweep, determinism-flake.

(a) What the platform CAN do today

  • Deterministic clock seam (tests/helpers/clock.cjs, makeFakeClock) — well-designed and correctly used by lock/state tests (clock-seam.test.cjs, perf-407, bug-474). Eliminates wall-clock races in the lock subsystem.
  • Property-based testing via fast-check with a pinned seed (tests/helpers/fast-check-setup.cjs, seed=42) — 9 *.property.test.cjs files exist and run reproducibly.
  • Mutation testing — Stryker is configured (stryker.config.mjs) and wired into a PR-gating workflow (.github/workflows/mutation.yml) that shards per changed module.
  • Custom ESLint rules — no-source-grep, no-magic-sleep-in-tests, no-elapsed-assertion, no-raw-rmsync-in-tests, plus no-only-tests — exist and are unit-tested via RuleTester (tests/eslint-rules.test.cjs).
  • Env hermeticity — scripts/run-tests.cjs deletes GSD_PROJECT/GSD_WORKSTREAM before spawning child test processes (mitigates the ambient-env hazard recorded in MEMORY.md).
  • Line-coverage gate — c8 --lines 70 (lib) enforced in CI as part of the unit lane.

(b) ENFORCED vs aspirational

Mechanism Status Note
Clock seam in lock/state tests Enforced Used correctly; but not mandated by lint — bug-3707 and phase.test.cjs:4362 still use real Date.now()
fast-check seed pinned Enforced seed=42; but no lint rule requires *.property.test.cjs to import the setup, and context-utilization.property.test.cjs uses Math.random() inside fc.property, breaking reproducibility
Stryker mutation gate Aspirational Runs in CI, but break=50% (not 80%) and covers 6/109 modules; trivially passes for excluded modules
Line coverage 70% Enforced But satisfiable via CLI integration tests that execute lines without asserting sub-behaviors; no branch-coverage floor
no-magic-sleep-in-tests Enforced (error) Promoted to error
no-raw-rmsync-in-tests Enforced (error)
no-only-tests Aspirational Wired at error but has zero RuleTester coverage and no integration test
no-source-grep Aspirational At warn only, and NOT applied to tests/** at all — the file-level allow-test-rule bypass blanket-exempts source-grep in test files
no-elapsed-assertion Aspirational At warn; only matches .elapsed/.duration/.took/.ms properties, missing Date.now()-start and computed-variable patterns (e.g. elapsedMs)
allow-test-rule tracking-issue requirement Not enforced 332/335 exemptions lack an issue number; no CI gate prevents new untracked exemptions
Subdirectory test discovery Broken run-tests.cjs uses flat readdirSync(testDir); tests/observability/ and tests/dispatch/ are silently never run by npm test
Fail-on-zero-executed Not enforced run-tests.cjs exits 0 when a suite resolves to zero files

(c) Prioritized capability upgrades

Capability Current state Recommendation Priority
Recursive test discovery Flat readdirSync skips tests/observability/, tests/dispatch/ Replace with the existing findTestFilesRecursive (already in the same file) P0
Stryker break threshold break: 50 in stryker.config.mjs Raise to 80 per ADR-456; move sub-80% modules to UNMUTATED with documented reasons P0
Mutation coverage breadth 6/109 modules covered; 12 critical modules excluded Graduate one module/release cycle starting with core.cts, config-loader.cts; require any module with a *.unit/property.test.cjs to be in COVERED or UNMUTATED P0
allow-test-rule audit 332/335 untracked exemptions Lint gate requiring either an approved category tag or #NNN; one-time tracking-issue sweep P1
no-source-grep in test files Not applied to tests/**; warn-only Apply at error to tests/**; split rule to allow product-file (.md/.json) reads but block source-file (.cjs/.js/.ts) grep assertions P1
Branch-coverage floor Only line coverage gated Add --branches 60; per-file branch floor for the UNMUTATED critical modules P1
no-elapsed-assertion reach Property-name match only Extend to Date.now()-start binary expressions and identifiers ending Ms/Elapsed/Duration in assert args P1
no-only-tests coverage Wired error, untested Add RuleTester valid/invalid cases + an integration assertion it fires P1
no-tautological-assert rule None New ~30-line local rule flagging assert.ok(true)/assert.ok(1) in tests/** P1
Fail-on-zero-executed Exits 0 on empty suite Exit 1 by default; gate the empty-suite allowance behind RUN_TESTS_ALLOW_EMPTY_SUITE=1 P2
Shared-GSD_TEMP_DIR isolation reap tests write to global os.tmpdir()/gsd Redirect via the existing dir option to per-test tmpdirs; lint against writing to the shared path without cleanup P2
Build-artifact preflight in artifact-dependent tests Top-level require() of gitignored .cjs hard-crashes on clean worktree before() existsSync guard with a clear "run npm run build:lib" message P2

ADR-456 compliance — per mandate

ADR-456 defines four test-rigor mandates. Compliance is partial on all four:

  • (a) Concurrency via injectable clock seam, not real OS races — Substantially met. The clock seam exists and the lock subsystem uses it. Gaps: bug-3707-locked-worktree-cleanup.test.cjs computes stale-lock mtimes from real Date.now() (7 sites); phase.test.cjs:4362 asserts a 60-second wall-clock window; core.test.cjs timeAgo boundary tests use real clock at the 5s bucket edge. No lint rule mandates the seam.
  • (b) fast-check + Stryker antagonistic tier, 80% kill threshold — Largely aspirational. fast-check is wired with a pinned seed, but Stryker covers ~4–6% of source lines, excludes all high-risk modules, and the break threshold is 50%, not the mandated 80%. The antagonistic tier exists in name but does not guard the dangerous surface.
  • (c) Assert on typed --json fields and exported registries, not rendered text or source literals — Partially met, weakly enforced. New tests largely follow it, but no-source-grep is warn-only and not applied to test files, so source-grep assertions persist (e.g. install.test.cjs .includes() on bin/install.js; sh-hook-paths.test.cjs variable-name scans). Several CLI tests still assert on stdout.includes() substrings rather than parsed JSON.
  • (d) Delete pass-always / vacuous / source-grep / elapsed / real-race / permanent-allow-rule tests — Not met. Concrete pass-always tests survive (research-cli.test.cjs:423, worktree-baseref-install.test.cjs:444, 24 assert.ok(true) in eslint-rules.test.cjs); 332/335 allow-test-rule exemptions are effectively permanent and untracked, directly contradicting the policy clause requiring // allow-test-rule: see #NNN on post-ADR exemptions.

Per-ADR coverage

ADR Status #Tests Happy Boundary Negative Independence Verdict
0001 Dispatch Policy Module Accepted 12 covered partial covered covered Core protected; SDK bridge / exit-code mapping unverified
0002 Command Contract Validation Accepted 6 covered partial missing covered Integration-strong; helper unit tests + lint negatives missing
0003 Model Catalog Module Accepted 13 covered partial covered covered CJS solid; SDK-parity invariant untested
0004 Worktree/Workstream Seam Accepted 13 covered partial covered covered Substantially protected; cross-path naming + .planning/.lock 2-proc gaps
0005 SDK Architecture Seam Map Superseded (0174) 13 partial partial partial covered Retirement guards OK; thin-adapter enforcement absent
0006 Planning-Path Projection Accepted 6 partial partial covered covered Critical gap: init-handler planningPaths() consumption untested
0007 SDK Package Seam Superseded (0174) 10 covered partial partial covered Retirement guarded; classifyPromptUserAction boundary untested
0008 Installer Migration Module Accepted 16 covered partial covered covered Rich suite; rewrite-json apply + local-scope gaps
0009 Shell Command Projection Accepted 21 covered partial covered covered Strong; 3 ADR-listed test files absent; source-grep in sh-hook-paths
0010a Skill Surface Budget Proposed (impl w/ gaps) 8 covered partial partial covered ~75% protected; stageCommandsForRuntimeFlat + migration one-shot untested
0010b File Operation Engine Superseded (0009) 10 covered partial covered covered Canonical seam solid; divergent atomicWriteFileSync + path-containment gaps
0011a Skill Surface Budget Accepted 12 covered partial partial covered Broad; source modules absent from worktree → MODULE_NOT_FOUND; cyclic-requires untested
0011b Review Default Reviewers (PRD) Draft 5 partial partial partial covered infos-path + --all+flags edge untested; build-artifact dependency
0011c Review Default Reviewers Proposed 4 covered partial partial covered Mid-tier risks (infos, empty-selection) missing; build-artifact dependency
0012 Command Routing Hub Superseded (0174) 5 covered partial covered covered Hub well-covered; uat-passed adapter arg-shape gap; 3 redundant tests
15 Autonomous Cross-AI Convergence Proposed 8 partial partial partial partial Primary surface (/gsd-progress --next --auto --converge) has zero coverage
22 Plan Drift Guard Proposed 6 partial partial covered covered Severity mapping, authority auto-upgrade, rung≥3 hard-block untested
58 Runtime Install Policy Accepted 7 covered partial partial covered Strong projection coverage; purity guarantee + sandboxTier-invalid untested
0174 Retire gsd-sdk Package Accepted 12 covered covered covered covered Strong & behavioral; init-composer trace + tsc-output invariant gaps
218 Release Version Validation Accepted 0 missing missing missing na ZERO TESTS — entirely unprotected (regex lives only in release.yml bash)
227 Input Validation Shape Accepted 5 covered covered partial partial All 5 test files silently excluded from npm test (subdir blind spot)
230 Next Integration Branch Proposed 5 partial partial partial covered Most critical seams untested; 2 files misattributed (unrelated features)
415 Stale-Base Token Reintro Accepted 12 covered partial partial covered Token reintro guarded; propagator + ruleset-strict-mode untested
443 Unified Effort / Fast-Mode Proposed 12 covered covered covered partial Resolver exceptionally covered; end-to-end effort propagation untested
452 ESLint Lint Harness Accepted 2 partial partial partial covered Local rules covered; no-only-tests + plugin-n + script-retirement untested
456 Test Rigor Architecture Accepted 19 covered partial partial covered Clock seam strong; mutation breadth + warn→error + allow-rule tracking gaps
457 Generated CJS Single Source Accepted 8 covered partial partial covered Value-baking solid; .gitignore↔eslint sync + emit-path gaps
550 Spec-Phase Probe Contract Accepted 6 covered covered covered covered Core seam strong; Decisions 4(b–d)/6 deferred to #644 (untested by design)
0656 Research Module Seam Accepted 9 covered covered covered covered Strong; slopcheck "cannot-lower" + scrape-dispatch gaps
660 Release From Next Head Proposed 13 partial partial covered covered Scripts covered; orchestration (release.yml/auto-backmerge) largely untested
766 Claude Plugin Manifest Accepted 3 covered partial partial partial agents-absence + post-#770 event drift untested; CI-skipped schema validate
894 Capability Decl Format Proposed (coupled 857) 5 covered partial covered partial Provisional. Strong; when-ownership, tier-monotone std→full gaps
959 Capability Command Contribution Proposed (coupled 857) 6 covered partial covered covered Provisional. Strong; proto-pollution family-name + case-removal gaps
1016 Runtime Capability Descriptor Proposed (coupled 857) 9 covered partial covered covered Provisional. Dense; sandboxTier fail-loud + 5f required-axes gaps
1143 Claude Orchestration Proposed (coupled 857) 9 missing missing missing na Provisional. No impl; primary surface entirely untested
3524 CJS-SDK Hard Seam Superseded (0174) 12 partial partial covered covered Drift-bugs guarded; all subjects gitignored artifacts (crash on clean worktree)
3660 Runtime Artifact Layout Accepted 6 covered partial partial covered Core protected; findInstallSourceRoot fail-path + agent tools:-strip gaps

ADRs with zero tests or notable gaps

ADR-218 (Release Version Validation) — ZERO attributed tests. This is the most acute hole. Both ADR decisions — the leading-zero-rejecting format regex (^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.0$) and the npm duplicate-version precheck — live exclusively in a bash run: block in .github/workflows/release.yml. No test exercises either. The 1.01.0 incident that motivated the ADR caused an irrecoverable production divergence, and nothing prevents a reversion to the old broken regex. issue-version-gate.test.cjs is a superficially-related but wholly separate feature. This is a critical gap in a high-blast-radius zone.

ADR-227 (Input Validation Shape) — 5 tests that never run. All five attributed files live in tests/observability/ and tests/dispatch/, which scripts/run-tests.cjs:269 never enters (flat readdirSync). The tests are individually high quality (8 boundary/negative UUID-v4 cases, real file I/O, per-test isolation) but are silently excluded from npm test, CI, and gsd-test. ADR-227's two-layer validation guarantee — the correlation-poisoning and log-blowup classes it was written to prevent — is effectively unguarded until the runner is fixed.

ADR-0006 (Planning-Path Projection) — critical handler-consumption gap. The ADR's single most important invariant — that cmdInitExecutePhase/PlanPhase/PhaseOp/MilestoneOp route through planningPaths().planning rather than flat path.join(cwd, '.planning') — has no regression test. The banned .planning/projects/<project> form also has no guard.

ADR-15 (Autonomous Cross-AI Convergence) — primary surface untested. The ADR's designated primary operator surface, /gsd-progress --next --auto --converge, is both unimplemented and untested; the existing cross-ai-execution.test.cjs tests a different feature (execute-phase delegation) and is misattributed.

ADR-0011a / 0011b / 0011c (Skill Surface / Review Default Reviewers). Source modules (install-profiles.cjs, surface.cjs, clusters.cjs, review-reviewer-selection.cjs, config-schema.cjs) are gitignored build artifacts absent from a clean worktree; all attributed tests throw MODULE_NOT_FOUND without a prior npm run build:lib. CI's pretest hook covers this, but isolated runs and fresh worktrees fail confusingly.

ADR-1143 (Claude Orchestration) — provisional, no implementation. Proposed and blocked on ADR-857; the capability directory does not exist, so the primary surface has zero coverage. Findings are provisional pending ADR-857.


Weak tests

Retire

Only entries whose adversarial verification confirmed finalVerdict === "retire". 5 tests survived verification as truly worthless.

File Location Why worthless Confirmed by verification
tests/enh-2790-skill-consolidation.test.cjs Lines 127–157 — 4 "has a name: field" spot-checks Assert only fm.name.length > 0; command-contract.test.cjs already enforces the stricter /^gsd[:-]/ prefix on all 67 files. Renaming gsd:capture→capture (the exact regression) passes these but is caught by command-contract. Cannot go red for the bug class the ADR exists to prevent. Yes — verified command-contract.test.cjs:44-54 covers these 4 files more strictly; deletion creates no coverage gap.
tests/command-routing-hub.test.cjs Lines 134–137 — "constructs successfully with only cjsRegistry" Duplicate of the construction test at 71–75; asserts only typeof hub.dispatch === 'function' with no dispatch. createHub does zero registry-dependent branching at construction. Yes — verified identical assertion; populated-registry dispatch path covered by the happy-path block at line 142.
tests/command-routing-hub.test.cjs Lines 515–519 — "ERROR_KINDS.UnknownCommand === result.kind" The identical assertion appears at 9 other sites (222, 234, 245, 256, 360, 393, 410, 422, 518). Same code path covered at 414–424. No unique setup/edge/field. Yes — verified superset coverage by lines 384–424 + the constant-stability test at 521–526.
tests/no-cjs-sdk-handsync-tooling.test.cjs All 3 tests (lines 26–53) Assert absence of lint-shared-module-handsync.cjs / allowlist JSON that never existed on main (CONTEXT.md:693 confirms). Can only go red if someone manually drops those exact files. No behavioral coverage. Yes — verified the real retired artifacts (cjs-sdk-bridge.cjs, sdk/) are covered by bug-190; these guard ADR-text-only artifacts.
tests/runtime-artifact-layout.test.cjs Lines 319–323 — "cline has one skills kind (#782)" Default-scope call is behaviorally identical to the explicit-global test at 249–265, which asserts a strict superset (kinds.length, kind, destSubpath, prefix, stage). Yes — verified default scope = global (source line 430); also independently covered by bug-782-cline-skills-emission.test.cjs:629-636.

Redesign / improve

All redesign + keep-but-improve verdicts, plus the 6 retire-candidates that verification downgraded to redesign. (Representative selection across ADRs; the full set is large — these are the highest-signal entries and every downgraded retire-candidate.)

Downgraded retire-candidates (verification overruled the retire verdict — do NOT delete)

File Location Problem What the rewritten test should assert
tests/bug-3135-capture-backlog-workflow.test.cjs Lines 136–182 broad @-ref regression The block IS redundant (only checks /workflows/ refs, misses 24 files with references//templates/ refs that command-contract Rule 4 catches) — delete ONLY that block. But the FILE's lines 1–132 test unique add-backlog.md content invariants with no equivalent elsewhere. Delete lines 134–182 only. Retain content-contract assertions: add-backlog.md uses phase.next-decimal, writes ROADMAP before mkdir, 999.x scheme; capture.md routes to add-backlog.md.
tests/review-reviewer-selection.test.cjs Lines 115–122 result-shape test Shape-only on { detected: [] }. But the empty-detected call exercises the no_config_all_detected else-branch that no other test asserts (existing test uses ['gemini']). Replace shape assertions with behavioral: source === 'no_config_all_detected', selected/errors/warnings === [] for the empty-detected fallback.
tests/review-reviewer-selection.test.cjs Lines 17–27 KNOWN_REVIEWER_SLUGS Tests a compiled-CJS constant's shape. Verification found the slug membership DOES drive resolveReviewerSelection's warn/drop logic at runtime — it's the only guard against silent slug-list drift. Collapse to one behavioral test: resolveReviewerSelection with a config'd known slug yields no warning + slug in selected; an unknown slug yields a warning.
tests/enh-191-retire-sdk-package.test.cjs Lines 52–58 AGENTS.md absence Off-topic for SDK retirement, but NOT valueless: bin/install.js:~10963 writes a root AGENTS.md on local Copilot install; this is the only guard against that artifact committing. Extract to a repo-layout governance test (e.g. tests/repo-layout.test.cjs); rename to state the real invariant (no ad-hoc root instruction file vs CONTEXT.md/ADRs); link the copilot install path.
tests/capability-registry.test.cjs Lines 558–584 drift test Steps 2–3 are a file-I/O tautology and skip the real --check path (stripGeneratedComment + normalizeLineEndings). But it is the only unit-level coverage of the drift-comparison pipeline. Call the actual comparison expression: assert a stale version survives both stripGeneratedComment and normalizeLineEndings and is detected; assert a comment-only-timestamp change is NOT flagged after stripping.
tests/verify-test-quality.test.cjs 3 describes (disabled / circular / assertion-strength detection) Self-referential: define a regex inline, write a matching fixture, assert the regex matches — cannot go red for any production change. The detector lives as prose in verify-phase.md. Replace with a structural guard reading gsd-core/workflows/verify-phase.md: assert the audit_test_quality step tag exists and contains the skip-pattern, circular-detection, and assertion-strength-table markers.

Other redesign / keep-but-improve (high-signal selection)

File Location Problem What the rewritten test should assert
tests/research-cli.test.cjs Line 423 Pass-always assert.ok(true) placeholder; the 2-package arg-parse regression is undefended. Extract the flag parser; call with ['--bad-flag','pkgA','pkgB'] and assert unknown-flag error + that pkgA/pkgB are not consumed as flag values. (No network.)
tests/worktree-baseref-install.test.cjs Line 444 Pass-always assert.ok(true); the null-settings #683 guard is never exercised. assert.doesNotThrow(() => applyWorktreeBaseRef(null, tmpDir)) and assert no worktree.baseRef is written.
tests/bug-260-worktree-path-guard.test.cjs Line 338 Environment-dependent branch silently passes via assert.ok(true) when the traversal path resolves inside the worktree — a security guard goes untested on macOS/private-symlink hosts. Construct the traversal target in a separate tmpdir outside the worktree; remove the if/else; unconditionally assert result.status === 2 / decision block.
tests/eslint-rules.test.cjs 24 trailing assert.ok(true) calls RuleTester already throws on failure; the trailing literal asserts nothing and inflates count. Delete the assert.ok(true) lines; optionally add assert.strictEqual(typeof rule.create, 'function') per describe to confirm the module loaded.
tests/phase.test.cjs Lines 4343–4368 (last_updated refresh) Math.abs(new Date() - updatedAt) < 60_000 wall-clock window — flaky under loaded CI. Assert last_updated is a valid ISO timestamp ≠ the stale seed value AND its date portion equals today's UTC date.
tests/bug-3707-locked-worktree-cleanup.test.cjs 7 Date.now() stale-mtime sites Real-clock coupling for stale-lock boundary; violates the ADR-456 clock-seam mandate. Pin a constant epoch (PINNED_NOW_MS) for mtime fixtures; pass it to the SUT as a clock parameter where supported.
tests/context-utilization.property.test.cjs Line 187 Math.floor(Math.random()*…) inside fc.property breaks reproducibility under the pinned seed. Replace with fc.integer({ min: 0, max: contextWindow }); split a shape test from a value-correctness test at known ratios.
tests/research-provider.property.test.cjs Single test Only "never throws + valid type"; inverting HIGH/LOW classification passes. Keep robustness property; add example-based unit tests for classification boundaries; add module to mutation COVERED.
tests/research-store.property.test.cjs All 3 tests Shape/determinism only; cache-key collisions or over-normalization invisible. Add a collision property (distinct inputs → distinct keys) + boundary examples (npm/lodash ≠ npm/react).
tests/install.test.cjs Lines 683–712 (Kilo source-grep) Reads bin/install.js as a string and .includes() internal symbol names — source-grep, breaks on rename, passes on dead code. Invoke install.js with --kilo in a tmpdir; assert on the installed config/manifest content (or export+call the kilo-detection logic).
tests/sh-hook-paths.test.cjs Tests 2–4 (lines 119–194) Source-grep variable-name scans of install.js/runtime-hooks-surface.cts; behavior already covered by Test 1's behavioral call. Call buildHookCommand per .sh hook for linux+win32; assert non-empty, bash runner, absolute quoted configDir prefix.
tests/clusters.test.cjs Lines 50–58 `assert.ok(utilitySize >= skills.length
tests/model-profiles.test.cjs Lines 115–122 Test titled "resolution order: override > profile > default" exercises no override and asserts nothing about override behavior. Use a temp project with config.model_overrides, call resolveModelInternal, assert override > profile > default explicitly.
tests/active-workstream-store.unit.test.cjs Lines 361–364 `assert.ok(key === null
tests/feat-488-effort-sync.test.cjs Lines 177–219 Does not redirect GSD_HOME; resolved effort depends on the dev's real ~/.gsd; degrades to a type-only check. Set GSD_HOME=tmpHome; assert result.synced === 1 and changes[0].to === 'low'.
tests/bug-492-effort-manifest-fallback.test.cjs Lines 20–48 Mutates the module-level CANONICAL_CONFIG_DEFAULTS singleton — independence violation; races under parallel runs. Drive the same path via a tmpHome ~/.gsd/defaults.json rather than mutating shared module state.
tests/issue-766-plugin-manifest.test.cjs Section C, "claude plugin validate --strict" Permanently skipped when the claude binary is absent (i.e. on CI) — the only full-schema gate never runs. Install the claude CLI in a CI step, OR validate plugin.json against a snapshotted JSON schema unconditionally.
tests/cross-ai-execution.test.cjs Entire file Misattributed to ADR-15; tests a different feature (workflow.cross_ai_execution). Also a shared-mutable let content independence issue. Leave the file for its real feature; create tests/adr-15-plan-strategy-seam.test.cjs for the actual converge seam; hoist content to a single module-level read.
tests/issue-844-manifest-version-sync.test.cjs Describe A (33–127) Order-dependent shared tmpRoot across setup/assert/cleanup tests. Refactor to a single test with per-test before()/after() tmpRoot allocation.
tests/plan-review-convergence.test.cjs Lines 688–692 Four workflow.includes(verdict) whole-file substring checks — pass even if verdicts move to dead/comment text. Extract the source-grounding section slice; assert each verdict + its severity mapping (AMBIGUOUS→MEDIUM, UNCHECKABLE→INFO) inside that slice.
tests/feat-3594-parser-property-style.test.cjs Lines 119–135 assert.ok(elapsedMs < 2000) wall-clock; escapes no-elapsed-assertion via the variable name. Replace with a call-count metric (instrument char comparisons, assert O(n)); or raise bound to 10000ms as a stopgap.
tests/core.test.cjs Lines 1834–1879 (reapStaleTempFiles) Writes to the global shared os.tmpdir()/gsd; races with temp-subdir.test.cjs under concurrency. Use a per-test isolated tmpdir via the existing { dir } option; clean up in afterEach.

Prioritized action plan

P0 — do first (blast radius / false-green)

  1. Fix recursive test discovery in scripts/run-tests.cjs (replace flat readdirSync with the already-present findTestFilesRecursive). This alone reactivates ADR-227's entire UUID-validation suite (tests/observability/, tests/dispatch/) — currently silently excluded from npm test and CI.
  2. Raise Stryker break from 50 → 80 in stryker.config.mjs (ADR-456 mandate). Move any module that cannot meet 80% into UNMUTATED with a documented per-entry reason rather than diluting the global floor.
  3. Add tests for ADR-218 (zero coverage today): extract the release-version regex and npm-duplicate precheck into a testable .cjs helper (or a source-text-is-the-product workflow assertion on release.yml), and assert 1.01.0/01.0.0 are rejected, valid versions pass, and the duplicate-version step exists and is wired before the publish jobs.
  4. Retire the 5 verified-worthless tests (see Retire table): enh-2790 name spot-checks (127–157), command-routing-hub lines 134–137 and 515–519, no-cjs-sdk-handsync-tooling (all 3), runtime-artifact-layout cline edge-case (319–323).
  5. Redesign the 6 downgraded retire-candidates (do NOT delete): bug-3135 (delete only the broad block, keep content tests), both review-reviewer-selection tests, enh-191 AGENTS.md (extract to governance test), capability-registry drift test, verify-test-quality (replace with structural workflow guard).
  6. Close the ADR-0006 critical gap: add a regression test that init execute-phase/plan-phase/phase-op/milestone-op return workstream-scoped paths under GSD_WORKSTREAM, plus a guard against the banned .planning/projects/<project> form.

P1 — high priority (rigor enforcement + concentrated weakness)

  1. Bring core.cts and config-loader.cts into the Stryker COVERED set (smallest high-value modules first); write their *.unit.test.cjs as a prerequisite.
  2. Promote no-source-grep to error and apply it to tests/**, splitting product-file reads (.md/.json, allowed) from source-file grep (.cjs/.js/.ts, blocked). Then fix the surfaced violations (install.test.cjs Kilo block, sh-hook-paths.test.cjs Tests 2–4).
  3. Add the allow-test-rule tracking-issue lint gate + run the one-time sweep to attach #NNN (or an approved category tag) to the 332 untracked exemptions.
  4. Eliminate the pass-always placeholders: redesign research-cli.test.cjs:423, worktree-baseref-install.test.cjs:444, the bug-260 environment-dependent branch, and strip the 24 assert.ok(true) lines in eslint-rules.test.cjs. Add a no-tautological-assert ESLint rule to prevent recurrence.
  5. Add no-only-tests RuleTester coverage and extend no-elapsed-assertion to catch Date.now()-start and computed-variable (elapsedMs) patterns.
  6. Add a branch-coverage floor (--branches 60) and a per-file branch floor for the UNMUTATED critical modules.
  7. De-flake the clock-coupled tests: phase.test.cjs:4362 (60s window → ISO-date assertion), bug-3707 (pin epoch), context-utilization.property.test.cjs:187 (fc.integer not Math.random).
  8. Add the ADR-15 primary-surface tests (/gsd-progress --next --auto --converge) and re-attribute cross-ai-execution.test.cjs to its real feature.

P2 — follow-up (robustness & hygiene)

  1. Fail-on-zero-executed in run-tests.cjs (exit 1 on empty suite by default; gate the allowance behind an env flag).
  2. Isolate the shared GSD_TEMP_DIR reap tests (core.test.cjs:1834-1879, temp-subdir.test.cjs) via the existing { dir } option + afterEach cleanup.
  3. Add build-artifact before() guards to artifact-dependent suites (ADR-3524 subjects, ADR-0011 family) so a clean worktree fails with a clear "run npm run build:lib" message instead of a top-level crash.
  4. Strengthen the strong-but-incomplete ADRs: ADR-443 end-to-end effort propagation, ADR-0656 slopcheck "cannot-lower" + scrape-dispatch, ADR-0009's 3 absent ADR-listed test files, ADR-766 agents-absence assertion + post-#770 hook drift.
  5. Add .gitignore ↔ eslint.config.mjs ignore-list parity test (ADR-457) so a migration that updates one list but not the other is caught.

Capability-cluster items (ADR-894/959/1016/1143) are deliberately deferred — they are coupled to ADR-857 (worked elsewhere) and their findings are provisional. Re-run this audit slice once ADR-857 lands.