msd-core/docs/issueevidence/1192-adr-test-audit-2026-06-13.md

# ADR Test-Coverage Audit — 2026-06-13

Lead QA architect final report. Scope: 38 ADRs in `docs/adr/`, evaluated with the qa-test-architect methodology, plus 4 platform-capability lenses over the test infrastructure. Every RETIRE verdict was independently re-checked by an adversarial verification pass.

> **Scope note.** ADR-857 (capability system core) was intentionally excluded from this audit — it is being worked by another instance. The capability cluster (ADR-894, ADR-959, ADR-1016, ADR-1143) is *coupled to* ADR-857, so all findings for those four ADRs are **provisional** and may shift once ADR-857 lands.

---

## Executive summary

The test portfolio is, on the whole, healthier than its volume suggests: the deterministic seams (dispatch hub, model catalog, planning-path resolution, installer migration, shell-command projection) are genuinely behavioral and well-covered, and the project has built real test-rigor infrastructure (an injectable clock seam, fast-check with a pinned seed, Stryker mutation testing wired into CI, and four custom ESLint rules). The dominant problem is not absent tests — it is **enforcement gaps that let the rigor infrastructure run as theater**, and a long tail of **weak tests concentrated in three families**: source-text-grep assertions, tautological `assert.ok(true)` placeholders, and architecture-retirement guards that can never go red.

Five headline problems demand attention first: (1) **Stryker mutation testing covers only ~4–6% of the lib surface** and excludes every high-risk module (state, phase, verify, init, commands), with the break threshold set to 50% instead of the ADR-456-mandated 80% — the mutation gate trivially passes for PRs touching those modules. (2) **332 of 335 `allow-test-rule` exemptions carry no tracking issue**, directly violating ADR-456's delete-bad-tests policy. (3) **A subdirectory blind spot in `scripts/run-tests.cjs` silently excludes all of `tests/observability/` and `tests/dispatch/` from `npm test`** — meaning ADR-227's entire UUID-validation seam is unguarded by any continuously-executed test. (4) **Multiple ADRs marked Accepted or Proposed have zero attributed tests or test files that cannot run** because their source modules are gitignored build artifacts absent from a clean worktree (ADR-218, ADR-0011 family). (5) A scatter of **vacuous and self-referential tests** (`verify-test-quality.test.cjs`, `assert.ok(true)` placeholders, environment-dependent path-guard branches) that provide false confidence on security-relevant code.

**Do first:** raise the Stryker break threshold and bring `core.cts`/`config-loader.cts` into the mutation-covered set (P0); fix the `run-tests.cjs` non-recursive file scan so observability/dispatch tests actually execute (P0); retire the 5 verified-worthless tests and redesign the 6 downgraded retire-candidates (P0/P1).

---

## Testing-platform robustness

Synthesis of the four platform lenses: `adr-456-test-rigor`, `coverage-mutation`, `anti-pattern-sweep`, `determinism-flake`.

### (a) What the platform CAN do today

- **Deterministic clock seam** (`tests/helpers/clock.cjs`, `makeFakeClock`) — well-designed and correctly used by lock/state tests (`clock-seam.test.cjs`, `perf-407`, `bug-474`). Eliminates wall-clock races in the lock subsystem.
- **Property-based testing** via fast-check with a pinned seed (`tests/helpers/fast-check-setup.cjs`, seed=42) — 9 `*.property.test.cjs` files exist and run reproducibly.
- **Mutation testing** — Stryker is configured (`stryker.config.mjs`) and wired into a PR-gating workflow (`.github/workflows/mutation.yml`) that shards per changed module.
- **Custom ESLint rules** — `no-source-grep`, `no-magic-sleep-in-tests`, `no-elapsed-assertion`, `no-raw-rmsync-in-tests`, plus `no-only-tests` — exist and are unit-tested via RuleTester (`tests/eslint-rules.test.cjs`).
- **Env hermeticity** — `scripts/run-tests.cjs` deletes `GSD_PROJECT`/`GSD_WORKSTREAM` before spawning child test processes (mitigates the ambient-env hazard recorded in MEMORY.md).
- **Line-coverage gate** — `c8 --lines 70` (lib) enforced in CI as part of the unit lane.

### (b) ENFORCED vs aspirational

| Mechanism | Status | Note |
|---|---|---|
| Clock seam in lock/state tests | **Enforced** | Used correctly; but not mandated by lint — `bug-3707` and `phase.test.cjs:4362` still use real `Date.now()` |
| fast-check seed pinned | **Enforced** | seed=42; but no lint rule requires `*.property.test.cjs` to import the setup, and `context-utilization.property.test.cjs` uses `Math.random()` inside `fc.property`, breaking reproducibility |
| Stryker mutation gate | **Aspirational** | Runs in CI, but break=50% (not 80%) and covers 6/109 modules; trivially passes for excluded modules |
| Line coverage 70% | **Enforced** | But satisfiable via CLI integration tests that execute lines without asserting sub-behaviors; no branch-coverage floor |
| `no-magic-sleep-in-tests` | **Enforced** (`error`) | Promoted to error |
| `no-raw-rmsync-in-tests` | **Enforced** (`error`) | |
| `no-only-tests` | **Aspirational** | Wired at `error` but has zero RuleTester coverage and no integration test |
| `no-source-grep` | **Aspirational** | At `warn` only, and NOT applied to `tests/**` at all — the file-level `allow-test-rule` bypass blanket-exempts source-grep in test files |
| `no-elapsed-assertion` | **Aspirational** | At `warn`; only matches `.elapsed/.duration/.took/.ms` properties, missing `Date.now()-start` and computed-variable patterns (e.g. `elapsedMs`) |
| `allow-test-rule` tracking-issue requirement | **Not enforced** | 332/335 exemptions lack an issue number; no CI gate prevents new untracked exemptions |
| Subdirectory test discovery | **Broken** | `run-tests.cjs` uses flat `readdirSync(testDir)`; `tests/observability/` and `tests/dispatch/` are silently never run by `npm test` |
| Fail-on-zero-executed | **Not enforced** | `run-tests.cjs` exits 0 when a suite resolves to zero files |

### (c) Prioritized capability upgrades

| Capability | Current state | Recommendation | Priority |
|---|---|---|---|
| Recursive test discovery | Flat `readdirSync` skips `tests/observability/`, `tests/dispatch/` | Replace with the existing `findTestFilesRecursive` (already in the same file) | **P0** |
| Stryker break threshold | `break: 50` in `stryker.config.mjs` | Raise to `80` per ADR-456; move sub-80% modules to UNMUTATED with documented reasons | **P0** |
| Mutation coverage breadth | 6/109 modules covered; 12 critical modules excluded | Graduate one module/release cycle starting with `core.cts`, `config-loader.cts`; require any module with a `*.unit/property.test.cjs` to be in COVERED or UNMUTATED | **P0** |
| `allow-test-rule` audit | 332/335 untracked exemptions | Lint gate requiring either an approved category tag or `#NNN`; one-time tracking-issue sweep | **P1** |
| `no-source-grep` in test files | Not applied to `tests/**`; warn-only | Apply at `error` to `tests/**`; split rule to allow product-file (`.md/.json`) reads but block source-file (`.cjs/.js/.ts`) grep assertions | **P1** |
| Branch-coverage floor | Only line coverage gated | Add `--branches 60`; per-file branch floor for the UNMUTATED critical modules | **P1** |
| `no-elapsed-assertion` reach | Property-name match only | Extend to `Date.now()-start` binary expressions and identifiers ending `Ms/Elapsed/Duration` in assert args | **P1** |
| `no-only-tests` coverage | Wired `error`, untested | Add RuleTester valid/invalid cases + an integration assertion it fires | **P1** |
| `no-tautological-assert` rule | None | New ~30-line local rule flagging `assert.ok(true)`/`assert.ok(1)` in `tests/**` | **P1** |
| Fail-on-zero-executed | Exits 0 on empty suite | Exit 1 by default; gate the empty-suite allowance behind `RUN_TESTS_ALLOW_EMPTY_SUITE=1` | **P2** |
| Shared-`GSD_TEMP_DIR` isolation | reap tests write to global `os.tmpdir()/gsd` | Redirect via the existing `dir` option to per-test tmpdirs; lint against writing to the shared path without cleanup | **P2** |
| Build-artifact preflight in artifact-dependent tests | Top-level `require()` of gitignored `.cjs` hard-crashes on clean worktree | `before()` `existsSync` guard with a clear "run npm run build:lib" message | **P2** |

### ADR-456 compliance — per mandate

ADR-456 defines four test-rigor mandates. Compliance is **partial on all four**:

- **(a) Concurrency via injectable clock seam, not real OS races** — *Substantially met.* The clock seam exists and the lock subsystem uses it. **Gaps:** `bug-3707-locked-worktree-cleanup.test.cjs` computes stale-lock mtimes from real `Date.now()` (7 sites); `phase.test.cjs:4362` asserts a 60-second wall-clock window; `core.test.cjs` `timeAgo` boundary tests use real clock at the 5s bucket edge. No lint rule mandates the seam.
- **(b) fast-check + Stryker antagonistic tier, 80% kill threshold** — *Largely aspirational.* fast-check is wired with a pinned seed, but Stryker covers ~4–6% of source lines, excludes all high-risk modules, and the break threshold is **50%, not the mandated 80%**. The antagonistic tier exists in name but does not guard the dangerous surface.
- **(c) Assert on typed `--json` fields and exported registries, not rendered text or source literals** — *Partially met, weakly enforced.* New tests largely follow it, but `no-source-grep` is `warn`-only and not applied to test files, so source-grep assertions persist (e.g. `install.test.cjs` `.includes()` on `bin/install.js`; `sh-hook-paths.test.cjs` variable-name scans). Several CLI tests still assert on `stdout.includes()` substrings rather than parsed JSON.
- **(d) Delete pass-always / vacuous / source-grep / elapsed / real-race / permanent-allow-rule tests** — *Not met.* Concrete pass-always tests survive (`research-cli.test.cjs:423`, `worktree-baseref-install.test.cjs:444`, 24 `assert.ok(true)` in `eslint-rules.test.cjs`); 332/335 `allow-test-rule` exemptions are effectively permanent and untracked, directly contradicting the policy clause requiring `// allow-test-rule: see #NNN` on post-ADR exemptions.

---

## Per-ADR coverage

| ADR | Status | #Tests | Happy | Boundary | Negative | Independence | Verdict |
|---|---|---|---|---|---|---|---|
| 0001 Dispatch Policy Module | Accepted | 12 | covered | partial | covered | covered | Core protected; SDK bridge / exit-code mapping unverified |
| 0002 Command Contract Validation | Accepted | 6 | covered | partial | missing | covered | Integration-strong; helper unit tests + lint negatives missing |
| 0003 Model Catalog Module | Accepted | 13 | covered | partial | covered | covered | CJS solid; SDK-parity invariant untested |
| 0004 Worktree/Workstream Seam | Accepted | 13 | covered | partial | covered | covered | Substantially protected; cross-path naming + .planning/.lock 2-proc gaps |
| 0005 SDK Architecture Seam Map | Superseded (0174) | 13 | partial | partial | partial | covered | Retirement guards OK; thin-adapter enforcement absent |
| 0006 Planning-Path Projection | Accepted | 6 | partial | partial | covered | covered | **Critical gap:** init-handler `planningPaths()` consumption untested |
| 0007 SDK Package Seam | Superseded (0174) | 10 | covered | partial | partial | covered | Retirement guarded; `classifyPromptUserAction` boundary untested |
| 0008 Installer Migration Module | Accepted | 16 | covered | partial | covered | covered | Rich suite; rewrite-json apply + local-scope gaps |
| 0009 Shell Command Projection | Accepted | 21 | covered | partial | covered | covered | Strong; 3 ADR-listed test files absent; source-grep in `sh-hook-paths` |
| 0010a Skill Surface Budget | Proposed (impl w/ gaps) | 8 | covered | partial | partial | covered | ~75% protected; `stageCommandsForRuntimeFlat` + migration one-shot untested |
| 0010b File Operation Engine | Superseded (0009) | 10 | covered | partial | covered | covered | Canonical seam solid; divergent `atomicWriteFileSync` + path-containment gaps |
| 0011a Skill Surface Budget | Accepted | 12 | covered | partial | partial | covered | Broad; **source modules absent from worktree → MODULE_NOT_FOUND**; cyclic-requires untested |
| 0011b Review Default Reviewers (PRD) | Draft | 5 | partial | partial | partial | covered | infos-path + `--all`+flags edge untested; build-artifact dependency |
| 0011c Review Default Reviewers | Proposed | 4 | covered | partial | partial | covered | Mid-tier risks (infos, empty-selection) missing; build-artifact dependency |
| 0012 Command Routing Hub | Superseded (0174) | 5 | covered | partial | covered | covered | Hub well-covered; `uat-passed` adapter arg-shape gap; 3 redundant tests |
| 15 Autonomous Cross-AI Convergence | Proposed | 8 | partial | partial | partial | partial | **Primary surface (`/gsd-progress --next --auto --converge`) has zero coverage** |
| 22 Plan Drift Guard | Proposed | 6 | partial | partial | covered | covered | Severity mapping, authority auto-upgrade, rung≥3 hard-block untested |
| 58 Runtime Install Policy | Accepted | 7 | covered | partial | partial | covered | Strong projection coverage; purity guarantee + sandboxTier-invalid untested |
| 0174 Retire gsd-sdk Package | Accepted | 12 | covered | covered | covered | covered | Strong & behavioral; init-composer trace + tsc-output invariant gaps |
| 218 Release Version Validation | Accepted | **0** | missing | missing | missing | na | **ZERO TESTS — entirely unprotected (regex lives only in release.yml bash)** |
| 227 Input Validation Shape | Accepted | 5 | covered | covered | partial | partial | **All 5 test files silently excluded from npm test (subdir blind spot)** |
| 230 Next Integration Branch | Proposed | 5 | partial | partial | partial | covered | Most critical seams untested; 2 files misattributed (unrelated features) |
| 415 Stale-Base Token Reintro | Accepted | 12 | covered | partial | partial | covered | Token reintro guarded; propagator + ruleset-strict-mode untested |
| 443 Unified Effort / Fast-Mode | Proposed | 12 | covered | covered | covered | partial | Resolver exceptionally covered; **end-to-end effort propagation untested** |
| 452 ESLint Lint Harness | Accepted | 2 | partial | partial | partial | covered | Local rules covered; `no-only-tests` + plugin-n + script-retirement untested |
| 456 Test Rigor Architecture | Accepted | 19 | covered | partial | partial | covered | Clock seam strong; mutation breadth + warn→error + allow-rule tracking gaps |
| 457 Generated CJS Single Source | Accepted | 8 | covered | partial | partial | covered | Value-baking solid; .gitignore↔eslint sync + emit-path gaps |
| 550 Spec-Phase Probe Contract | Accepted | 6 | covered | covered | covered | covered | Core seam strong; Decisions 4(b–d)/6 deferred to #644 (untested by design) |
| 0656 Research Module Seam | Accepted | 9 | covered | covered | covered | covered | Strong; slopcheck "cannot-lower" + scrape-dispatch gaps |
| 660 Release From Next Head | Proposed | 13 | partial | partial | covered | covered | Scripts covered; orchestration (release.yml/auto-backmerge) largely untested |
| 766 Claude Plugin Manifest | Accepted | 3 | covered | partial | partial | partial | `agents`-absence + post-#770 event drift untested; CI-skipped schema validate |
| 894 Capability Decl Format | Proposed (coupled 857) | 5 | covered | partial | covered | partial | *Provisional.* Strong; `when`-ownership, tier-monotone std→full gaps |
| 959 Capability Command Contribution | Proposed (coupled 857) | 6 | covered | partial | covered | covered | *Provisional.* Strong; proto-pollution family-name + case-removal gaps |
| 1016 Runtime Capability Descriptor | Proposed (coupled 857) | 9 | covered | partial | covered | covered | *Provisional.* Dense; sandboxTier fail-loud + 5f required-axes gaps |
| 1143 Claude Orchestration | Proposed (coupled 857) | 9 | missing | missing | missing | na | *Provisional.* No impl; primary surface entirely untested |
| 3524 CJS-SDK Hard Seam | Superseded (0174) | 12 | partial | partial | covered | covered | Drift-bugs guarded; all subjects gitignored artifacts (crash on clean worktree) |
| 3660 Runtime Artifact Layout | Accepted | 6 | covered | partial | partial | covered | Core protected; `findInstallSourceRoot` fail-path + agent `tools:`-strip gaps |

### ADRs with zero tests or notable gaps

**ADR-218 (Release Version Validation) — ZERO attributed tests.** This is the most acute hole. Both ADR decisions — the leading-zero-rejecting format regex (`^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.0$`) and the npm duplicate-version precheck — live exclusively in a bash `run:` block in `.github/workflows/release.yml`. No test exercises either. The `1.01.0` incident that motivated the ADR caused an irrecoverable production divergence, and nothing prevents a reversion to the old broken regex. `issue-version-gate.test.cjs` is a superficially-related but wholly separate feature. **This is a critical gap in a high-blast-radius zone.**

**ADR-227 (Input Validation Shape) — 5 tests that never run.** All five attributed files live in `tests/observability/` and `tests/dispatch/`, which `scripts/run-tests.cjs:269` never enters (flat `readdirSync`). The tests are individually high quality (8 boundary/negative UUID-v4 cases, real file I/O, per-test isolation) but are silently excluded from `npm test`, CI, and gsd-test. ADR-227's two-layer validation guarantee — the correlation-poisoning and log-blowup classes it was written to prevent — is effectively unguarded until the runner is fixed.

**ADR-0006 (Planning-Path Projection) — critical handler-consumption gap.** The ADR's single most important invariant — that `cmdInitExecutePhase/PlanPhase/PhaseOp/MilestoneOp` route through `planningPaths().planning` rather than flat `path.join(cwd, '.planning')` — has no regression test. The banned `.planning/projects/<project>` form also has no guard.

**ADR-15 (Autonomous Cross-AI Convergence) — primary surface untested.** The ADR's designated primary operator surface, `/gsd-progress --next --auto --converge`, is both unimplemented and untested; the existing `cross-ai-execution.test.cjs` tests a *different* feature (execute-phase delegation) and is misattributed.

**ADR-0011a / 0011b / 0011c (Skill Surface / Review Default Reviewers).** Source modules (`install-profiles.cjs`, `surface.cjs`, `clusters.cjs`, `review-reviewer-selection.cjs`, `config-schema.cjs`) are gitignored build artifacts absent from a clean worktree; all attributed tests throw `MODULE_NOT_FOUND` without a prior `npm run build:lib`. CI's pretest hook covers this, but isolated runs and fresh worktrees fail confusingly.

**ADR-1143 (Claude Orchestration) — provisional, no implementation.** Proposed and blocked on ADR-857; the capability directory does not exist, so the primary surface has zero coverage. Findings are provisional pending ADR-857.

---

## Weak tests

### Retire

Only entries whose adversarial verification confirmed `finalVerdict === "retire"`. **5 tests survived verification as truly worthless.**

| File | Location | Why worthless | Confirmed by verification |
|---|---|---|---|
| `tests/enh-2790-skill-consolidation.test.cjs` | Lines 127–157 — 4 "has a name: field" spot-checks | Assert only `fm.name.length > 0`; `command-contract.test.cjs` already enforces the stricter `/^gsd[:-]/` prefix on all 67 files. Renaming `gsd:capture`→`capture` (the exact regression) passes these but is caught by command-contract. Cannot go red for the bug class the ADR exists to prevent. | **Yes** — verified `command-contract.test.cjs:44-54` covers these 4 files more strictly; deletion creates no coverage gap. |
| `tests/command-routing-hub.test.cjs` | Lines 134–137 — "constructs successfully with only cjsRegistry" | Duplicate of the construction test at 71–75; asserts only `typeof hub.dispatch === 'function'` with no dispatch. `createHub` does zero registry-dependent branching at construction. | **Yes** — verified identical assertion; populated-registry dispatch path covered by the happy-path block at line 142. |
| `tests/command-routing-hub.test.cjs` | Lines 515–519 — "ERROR_KINDS.UnknownCommand === result.kind" | The identical assertion appears at 9 other sites (222, 234, 245, 256, 360, 393, 410, 422, 518). Same code path covered at 414–424. No unique setup/edge/field. | **Yes** — verified superset coverage by lines 384–424 + the constant-stability test at 521–526. |
| `tests/no-cjs-sdk-handsync-tooling.test.cjs` | All 3 tests (lines 26–53) | Assert absence of `lint-shared-module-handsync.cjs` / allowlist JSON that **never existed on main** (CONTEXT.md:693 confirms). Can only go red if someone manually drops those exact files. No behavioral coverage. | **Yes** — verified the real retired artifacts (`cjs-sdk-bridge.cjs`, `sdk/`) are covered by `bug-190`; these guard ADR-text-only artifacts. |
| `tests/runtime-artifact-layout.test.cjs` | Lines 319–323 — "cline has one skills kind (#782)" | Default-scope call is behaviorally identical to the explicit-global test at 249–265, which asserts a strict superset (kinds.length, kind, destSubpath, prefix, stage). | **Yes** — verified default scope = global (source line 430); also independently covered by `bug-782-cline-skills-emission.test.cjs:629-636`. |

### Redesign / improve

All `redesign` + `keep-but-improve` verdicts, **plus the 6 retire-candidates that verification downgraded** to redesign. (Representative selection across ADRs; the full set is large — these are the highest-signal entries and every downgraded retire-candidate.)

#### Downgraded retire-candidates (verification overruled the retire verdict — do NOT delete)

| File | Location | Problem | What the rewritten test should assert |
|---|---|---|---|
| `tests/bug-3135-capture-backlog-workflow.test.cjs` | Lines 136–182 broad @-ref regression | The block IS redundant (only checks `/workflows/` refs, misses 24 files with `references/`/`templates/` refs that command-contract Rule 4 catches) — delete ONLY that block. But the FILE's lines 1–132 test unique `add-backlog.md` content invariants with no equivalent elsewhere. | Delete lines 134–182 only. Retain content-contract assertions: `add-backlog.md` uses `phase.next-decimal`, writes ROADMAP before mkdir, 999.x scheme; `capture.md` routes to `add-backlog.md`. |
| `tests/review-reviewer-selection.test.cjs` | Lines 115–122 result-shape test | Shape-only on `{ detected: [] }`. But the empty-detected call exercises the `no_config_all_detected` else-branch that no other test asserts (existing test uses `['gemini']`). | Replace shape assertions with behavioral: `source === 'no_config_all_detected'`, `selected/errors/warnings === []` for the empty-detected fallback. |
| `tests/review-reviewer-selection.test.cjs` | Lines 17–27 KNOWN_REVIEWER_SLUGS | Tests a compiled-CJS constant's shape. Verification found the slug membership DOES drive `resolveReviewerSelection`'s warn/drop logic at runtime — it's the only guard against silent slug-list drift. | Collapse to one behavioral test: `resolveReviewerSelection` with a config'd known slug yields no warning + slug in `selected`; an unknown slug yields a warning. |
| `tests/enh-191-retire-sdk-package.test.cjs` | Lines 52–58 AGENTS.md absence | Off-topic for SDK retirement, but NOT valueless: `bin/install.js:~10963` writes a root `AGENTS.md` on local Copilot install; this is the only guard against that artifact committing. | Extract to a repo-layout governance test (e.g. `tests/repo-layout.test.cjs`); rename to state the real invariant (no ad-hoc root instruction file vs CONTEXT.md/ADRs); link the copilot install path. |
| `tests/capability-registry.test.cjs` | Lines 558–584 drift test | Steps 2–3 are a file-I/O tautology and skip the real `--check` path (`stripGeneratedComment` + `normalizeLineEndings`). But it is the only unit-level coverage of the drift-comparison pipeline. | Call the actual comparison expression: assert a stale version survives both `stripGeneratedComment` and `normalizeLineEndings` and is detected; assert a comment-only-timestamp change is NOT flagged after stripping. |
| `tests/verify-test-quality.test.cjs` | 3 describes (disabled / circular / assertion-strength detection) | Self-referential: define a regex inline, write a matching fixture, assert the regex matches — cannot go red for any production change. The detector lives as prose in `verify-phase.md`. | Replace with a structural guard reading `gsd-core/workflows/verify-phase.md`: assert the `audit_test_quality` step tag exists and contains the skip-pattern, circular-detection, and assertion-strength-table markers. |

#### Other redesign / keep-but-improve (high-signal selection)

| File | Location | Problem | What the rewritten test should assert |
|---|---|---|---|
| `tests/research-cli.test.cjs` | Line 423 | Pass-always `assert.ok(true)` placeholder; the 2-package arg-parse regression is undefended. | Extract the flag parser; call with `['--bad-flag','pkgA','pkgB']` and assert unknown-flag error + that `pkgA/pkgB` are not consumed as flag values. (No network.) |
| `tests/worktree-baseref-install.test.cjs` | Line 444 | Pass-always `assert.ok(true)`; the null-settings #683 guard is never exercised. | `assert.doesNotThrow(() => applyWorktreeBaseRef(null, tmpDir))` and assert no `worktree.baseRef` is written. |
| `tests/bug-260-worktree-path-guard.test.cjs` | Line 338 | Environment-dependent branch silently passes via `assert.ok(true)` when the traversal path resolves inside the worktree — a security guard goes untested on macOS/private-symlink hosts. | Construct the traversal target in a separate tmpdir outside the worktree; remove the if/else; unconditionally assert `result.status === 2` / decision `block`. |
| `tests/eslint-rules.test.cjs` | 24 trailing `assert.ok(true)` calls | RuleTester already throws on failure; the trailing literal asserts nothing and inflates count. | Delete the `assert.ok(true)` lines; optionally add `assert.strictEqual(typeof rule.create, 'function')` per describe to confirm the module loaded. |
| `tests/phase.test.cjs` | Lines 4343–4368 (`last_updated` refresh) | `Math.abs(new Date() - updatedAt) < 60_000` wall-clock window — flaky under loaded CI. | Assert `last_updated` is a valid ISO timestamp ≠ the stale seed value AND its date portion equals today's UTC date. |
| `tests/bug-3707-locked-worktree-cleanup.test.cjs` | 7 `Date.now()` stale-mtime sites | Real-clock coupling for stale-lock boundary; violates the ADR-456 clock-seam mandate. | Pin a constant epoch (`PINNED_NOW_MS`) for mtime fixtures; pass it to the SUT as a clock parameter where supported. |
| `tests/context-utilization.property.test.cjs` | Line 187 | `Math.floor(Math.random()*…)` inside `fc.property` breaks reproducibility under the pinned seed. | Replace with `fc.integer({ min: 0, max: contextWindow })`; split a shape test from a value-correctness test at known ratios. |
| `tests/research-provider.property.test.cjs` | Single test | Only "never throws + valid type"; inverting HIGH/LOW classification passes. | Keep robustness property; add example-based unit tests for classification boundaries; add module to mutation COVERED. |
| `tests/research-store.property.test.cjs` | All 3 tests | Shape/determinism only; cache-key collisions or over-normalization invisible. | Add a collision property (distinct inputs → distinct keys) + boundary examples (`npm/lodash` ≠ `npm/react`). |
| `tests/install.test.cjs` | Lines 683–712 (Kilo source-grep) | Reads `bin/install.js` as a string and `.includes()` internal symbol names — source-grep, breaks on rename, passes on dead code. | Invoke `install.js` with `--kilo` in a tmpdir; assert on the installed config/manifest content (or export+call the kilo-detection logic). |
| `tests/sh-hook-paths.test.cjs` | Tests 2–4 (lines 119–194) | Source-grep variable-name scans of `install.js`/`runtime-hooks-surface.cts`; behavior already covered by Test 1's behavioral call. | Call `buildHookCommand` per `.sh` hook for linux+win32; assert non-empty, `bash` runner, absolute quoted configDir prefix. |
| `tests/clusters.test.cjs` | Lines 50–58 | `assert.ok(utilitySize >= skills.length || true)` — `|| true` makes it vacuous. | Remove `|| true`; assert a meaningful lower bound (`utilitySize >= 5`). |
| `tests/model-profiles.test.cjs` | Lines 115–122 | Test titled "resolution order: override > profile > default" exercises no override and asserts nothing about override behavior. | Use a temp project with `config.model_overrides`, call `resolveModelInternal`, assert override > profile > default explicitly. |
| `tests/active-workstream-store.unit.test.cjs` | Lines 361–364 | `assert.ok(key === null || typeof key === 'string')` — disjunction always passes; env-dependent. | Clear `TTY`/`SSH_TTY` in `beforeEach`; assert `strictEqual(key, null)`, or skip when a real TTY is present. |
| `tests/feat-488-effort-sync.test.cjs` | Lines 177–219 | Does not redirect `GSD_HOME`; resolved effort depends on the dev's real `~/.gsd`; degrades to a type-only check. | Set `GSD_HOME=tmpHome`; assert `result.synced === 1` and `changes[0].to === 'low'`. |
| `tests/bug-492-effort-manifest-fallback.test.cjs` | Lines 20–48 | Mutates the module-level `CANONICAL_CONFIG_DEFAULTS` singleton — independence violation; races under parallel runs. | Drive the same path via a tmpHome `~/.gsd/defaults.json` rather than mutating shared module state. |
| `tests/issue-766-plugin-manifest.test.cjs` | Section C, "claude plugin validate --strict" | Permanently skipped when the `claude` binary is absent (i.e. on CI) — the only full-schema gate never runs. | Install the claude CLI in a CI step, OR validate `plugin.json` against a snapshotted JSON schema unconditionally. |
| `tests/cross-ai-execution.test.cjs` | Entire file | Misattributed to ADR-15; tests a different feature (`workflow.cross_ai_execution`). Also a shared-mutable `let content` independence issue. | Leave the file for its real feature; create `tests/adr-15-plan-strategy-seam.test.cjs` for the actual converge seam; hoist `content` to a single module-level read. |
| `tests/issue-844-manifest-version-sync.test.cjs` | Describe A (33–127) | Order-dependent shared `tmpRoot` across setup/assert/cleanup tests. | Refactor to a single test with per-test `before()/after()` tmpRoot allocation. |
| `tests/plan-review-convergence.test.cjs` | Lines 688–692 | Four `workflow.includes(verdict)` whole-file substring checks — pass even if verdicts move to dead/comment text. | Extract the source-grounding section slice; assert each verdict + its severity mapping (AMBIGUOUS→MEDIUM, UNCHECKABLE→INFO) inside that slice. |
| `tests/feat-3594-parser-property-style.test.cjs` | Lines 119–135 | `assert.ok(elapsedMs < 2000)` wall-clock; escapes `no-elapsed-assertion` via the variable name. | Replace with a call-count metric (instrument char comparisons, assert O(n)); or raise bound to 10000ms as a stopgap. |
| `tests/core.test.cjs` | Lines 1834–1879 (`reapStaleTempFiles`) | Writes to the global shared `os.tmpdir()/gsd`; races with `temp-subdir.test.cjs` under concurrency. | Use a per-test isolated tmpdir via the existing `{ dir }` option; clean up in `afterEach`. |

---

## Prioritized action plan

### P0 — do first (blast radius / false-green)

1. **Fix recursive test discovery** in `scripts/run-tests.cjs` (replace flat `readdirSync` with the already-present `findTestFilesRecursive`). This alone reactivates ADR-227's entire UUID-validation suite (`tests/observability/`, `tests/dispatch/`) — currently silently excluded from `npm test` and CI.
2. **Raise Stryker `break` from 50 → 80** in `stryker.config.mjs` (ADR-456 mandate). Move any module that cannot meet 80% into `UNMUTATED` with a documented per-entry reason rather than diluting the global floor.
3. **Add tests for ADR-218** (zero coverage today): extract the release-version regex and npm-duplicate precheck into a testable `.cjs` helper (or a `source-text-is-the-product` workflow assertion on `release.yml`), and assert `1.01.0`/`01.0.0` are rejected, valid versions pass, and the duplicate-version step exists and is wired before the publish jobs.
4. **Retire the 5 verified-worthless tests** (see Retire table): `enh-2790` name spot-checks (127–157), `command-routing-hub` lines 134–137 and 515–519, `no-cjs-sdk-handsync-tooling` (all 3), `runtime-artifact-layout` cline edge-case (319–323).
5. **Redesign the 6 downgraded retire-candidates** (do NOT delete): `bug-3135` (delete only the broad block, keep content tests), both `review-reviewer-selection` tests, `enh-191` AGENTS.md (extract to governance test), `capability-registry` drift test, `verify-test-quality` (replace with structural workflow guard).
6. **Close the ADR-0006 critical gap**: add a regression test that `init execute-phase/plan-phase/phase-op/milestone-op` return workstream-scoped paths under `GSD_WORKSTREAM`, plus a guard against the banned `.planning/projects/<project>` form.

### P1 — high priority (rigor enforcement + concentrated weakness)

7. **Bring `core.cts` and `config-loader.cts` into the Stryker COVERED set** (smallest high-value modules first); write their `*.unit.test.cjs` as a prerequisite.
8. **Promote `no-source-grep` to `error` and apply it to `tests/**`**, splitting product-file reads (`.md/.json`, allowed) from source-file grep (`.cjs/.js/.ts`, blocked). Then fix the surfaced violations (`install.test.cjs` Kilo block, `sh-hook-paths.test.cjs` Tests 2–4).
9. **Add the `allow-test-rule` tracking-issue lint gate** + run the one-time sweep to attach `#NNN` (or an approved category tag) to the 332 untracked exemptions.
10. **Eliminate the pass-always placeholders**: redesign `research-cli.test.cjs:423`, `worktree-baseref-install.test.cjs:444`, the `bug-260` environment-dependent branch, and strip the 24 `assert.ok(true)` lines in `eslint-rules.test.cjs`. Add a `no-tautological-assert` ESLint rule to prevent recurrence.
11. **Add `no-only-tests` RuleTester coverage** and extend `no-elapsed-assertion` to catch `Date.now()-start` and computed-variable (`elapsedMs`) patterns.
12. **Add a branch-coverage floor** (`--branches 60`) and a per-file branch floor for the UNMUTATED critical modules.
13. **De-flake the clock-coupled tests**: `phase.test.cjs:4362` (60s window → ISO-date assertion), `bug-3707` (pin epoch), `context-utilization.property.test.cjs:187` (`fc.integer` not `Math.random`).
14. **Add the ADR-15 primary-surface tests** (`/gsd-progress --next --auto --converge`) and re-attribute `cross-ai-execution.test.cjs` to its real feature.

### P2 — follow-up (robustness & hygiene)

15. **Fail-on-zero-executed** in `run-tests.cjs` (exit 1 on empty suite by default; gate the allowance behind an env flag).
16. **Isolate the shared `GSD_TEMP_DIR` reap tests** (`core.test.cjs:1834-1879`, `temp-subdir.test.cjs`) via the existing `{ dir }` option + `afterEach` cleanup.
17. **Add build-artifact `before()` guards** to artifact-dependent suites (ADR-3524 subjects, ADR-0011 family) so a clean worktree fails with a clear "run npm run build:lib" message instead of a top-level crash.
18. **Strengthen the strong-but-incomplete ADRs**: ADR-443 end-to-end effort propagation, ADR-0656 slopcheck "cannot-lower" + scrape-dispatch, ADR-0009's 3 absent ADR-listed test files, ADR-766 `agents`-absence assertion + post-#770 hook drift.
19. **Add `.gitignore` ↔ `eslint.config.mjs` ignore-list parity test** (ADR-457) so a migration that updates one list but not the other is caught.

> Capability-cluster items (ADR-894/959/1016/1143) are deliberately deferred — they are coupled to ADR-857 (worked elsewhere) and their findings are provisional. Re-run this audit slice once ADR-857 lands.