msd-core/gsd-core/references/verifier-phase-gates.md

# Verifier Phase Gates

> Loaded eagerly by `agents/gsd-verifier.md` (`<required_reading>`). Carries the three
> verification-time gates that lived in the retired `verify-phase` workflow
> (#1892 / epic #1891 F7): decision-coverage validation (#2492), the test-quality audit,
> and infrastructure-phase human-verification scoping (#2504) — plus the backstop-abstention
> reporting contract (#3206). Run each gate at its named
> agent step; `gsd_run` is the launcher shim defined in the agent's own Step 1 block.

## verify_decisions — Decision Coverage Gate (run after Step 6, requirements coverage)

<step name="verify_decisions">
**Decision coverage validation gate (issue #2492).**

After requirements coverage, also check that each trackable CONTEXT.md
`<decisions>` entry shows up somewhere in the shipped artifacts (plans,
SUMMARY.md, files modified by the phase, or recent commit subjects on the
phase branch).

This gate is **non-blocking / warning only** by deliberate asymmetry with
the plan-phase translation gate. The plan-phase gate already blocked at
translation time, so by the time verification runs every decision has
either been translated or explicitly deferred. This gate's job is to
surface decisions that *were* translated but vanished during execution —
that's a soft signal because "honors a decision" is a fuzzy substring
heuristic, and we don't want a paraphrase miss to fail an otherwise good
phase.

**Skip if** `workflow.context_coverage_gate` is explicitly set to `false`
(absent key = enabled). Also skip cleanly when CONTEXT.md is missing or has
no `<decisions>` block.

```bash
GATE_CFG=$(gsd_run query config-get workflow.context_coverage_gate 2>/dev/null || echo "true")
if [ "$GATE_CFG" != "false" ]; then
  CONTEXT_PATH=$(ls "${PHASE_DIR}"/*-CONTEXT.md 2>/dev/null | head -1)  # #2962: not a for-glob (zsh aborts)
  DECISION_RESULT=$(gsd_run query check.decision-coverage-verify "${PHASE_DIR}" "${CONTEXT_PATH}")
fi
```

The handler returns JSON `{ skipped, blocking: false, total, honored,
not_honored: [...], message }`.

**Reporting:** Append the handler's `message` (a `### Decision Coverage`
section) to VERIFICATION.md regardless of outcome — even when all
decisions are honored, recording the count helps reviewers spot drift over
time. Set `decision_coverage` in the verification result to
`{honored, total, not_honored: [...]}` so downstream tooling can read it.

**Status impact:** none. The decision gate does NOT influence the
`gaps_found` / `human_needed` / `passed` decision tree in Step 9. Its
findings are warnings the user reviews and may act on by re-opening the
phase or by acknowledging the decision was abandoned intentionally.
</step>

## audit_test_quality (run after Step 7b, alongside anti-patterns)

<step name="audit_test_quality">
**Verify that tests PROVE what they claim to prove.**

This step catches test-level deceptions that pass all prior checks: files exist, are substantive, are wired, and tests pass — but the tests don't actually validate the requirement.

**1. Identify requirement-linked test files**

From PLAN and SUMMARY files, map each requirement to the test files that are supposed to prove it.

**2. Disabled test scan**

For ALL test files linked to requirements, search for disabled/skipped patterns:

```bash
grep -rn -E "it\.skip|describe\.skip|test\.skip|xit\(|xdescribe\(|xtest\(|@pytest\.mark\.skip|@unittest\.skip|#\[ignore\]|\.pending|it\.todo|test\.todo" "$TEST_FILE"
```

**Rule:** A disabled test linked to a requirement = requirement NOT tested.
- 🛑 BLOCKER if the disabled test is the only test proving that requirement
- ⚠️ WARNING if other active tests also cover the requirement

**3. Circular test detection**

Search for scripts/utilities that generate expected values by running the system under test:

```bash
grep -rn -E "writeFileSync|writeFile|fs\.write|open\(.*w\)" "$TEST_DIRS"
```

For each match, check if it also imports the system/service/module being tested. If a script both imports the system-under-test AND writes expected output values → CIRCULAR.

**Circular test indicators:**
- Script imports a service AND writes to fixture files
- Expected values have comments like "computed from engine", "captured from baseline"
- Script filename contains "capture", "baseline", "generate", "snapshot" in test context
- Expected values were added in the same commit as the test assertions

**Rule:** A test comparing system output against values generated by the same system is circular. It proves consistency, not correctness.

**4. Expected value provenance** (for comparison/parity/migration requirements)

When a requirement demands comparison with an external source ("identical to X", "matches Y", "same output as Z"):

- Is the external source actually invoked or referenced in the test pipeline?
- Do fixture files contain data sourced from the external system?
- Or do all expected values come from the new system itself or from mathematical formulas?

**Provenance classification:**
- VALID: Expected value from external/legacy system output, manual capture, or independent oracle
- PARTIAL: Expected value from mathematical derivation (proves formula, not system match)
- CIRCULAR: Expected value from the system being tested
- UNKNOWN: No provenance information — treat as SUSPECT

**5. Assertion strength**

For each test linked to a requirement, classify the strongest assertion:

| Level | Examples | Proves |
|-------|---------|--------|
| Existence | `toBeDefined()`, `!= null` | Something returned |
| Type | `typeof x === 'number'` | Correct shape |
| Status | `code === 200` | No error |
| Value | `toEqual(expected)`, `toBeCloseTo(x)` | Specific value |
| Behavioral | Multi-step workflow assertions | End-to-end correctness |

If a requirement demands value-level or behavioral-level proof and the test only has existence/type/status assertions → INSUFFICIENT.

**6. Coverage quantity**

If a requirement specifies a quantity of test cases (e.g., "30 calculations"), check if the actual number of active (non-skipped) test cases meets the requirement.

**Reporting — add to VERIFICATION.md:**

```markdown
### Test Quality Audit

| Test File | Linked Req | Active | Skipped | Circular | Assertion Level | Verdict |
|-----------|-----------|--------|---------|----------|-----------------|---------|

**Disabled tests on requirements:** {N} → {BLOCKER if any req has ONLY disabled tests}
**Circular patterns detected:** {N} → {BLOCKER if any}
**Insufficient assertions:** {N} → {WARNING}
```

**Impact on status:** Any BLOCKER from test quality audit → overall status = `gaps_found` (Step 9 rule 1), regardless of other checks passing.
</step>

## identify_human_verification — infrastructure/foundation scoping (apply at Step 8)

**First: determine if this is an infrastructure/foundation phase.**

Infrastructure and foundation phases — code foundations, database schema, internal APIs, data models, build tooling, CI/CD, internal service integrations — have no user-facing elements by definition. For these phases:

- Do NOT invent artificial manual steps (e.g., "manually run git commits", "manually invoke methods", "manually check database state").
- Mark human verification as **N/A** with rationale: "Infrastructure/foundation phase — no user-facing elements to test manually."
- Set `human_verification: []` and do **not** produce a `human_needed` status solely due to lack of user-facing features.
- Only add human verification items if the phase goal or success criteria explicitly describe something a user would interact with (UI, CLI command output visible to end users, external service UX).
- **Exception — behavior-unverified truths still count.** A truth marked ⚠️ PRESENT_BEHAVIOR_UNVERIFIED (a state transition or a cancellation/cleanup/ordering invariant with no test exercising it) is a behavioral-evidence gap, not an artificial user-facing step. Record it in `behavior_unverified_items` and emit a human-verification item for it **even on an infrastructure/foundation phase** — these invariants are exactly where infra phases hide runtime state leaks. Such a truth drives `human_needed`; the auto-pass-UAT shortcut applies only to the absence of user-facing UX, never to a behavior-unverified invariant. The same carve-out covers an **abstained non-inferable truth** (⚠️ `insufficient_spec`, § Backstop abstention below) — an insufficient-spec gap is an evidence gap, not a user-facing step, so it too still emits its human-verification item and drives `human_needed` on an infrastructure phase.

**How to determine if a phase is infrastructure/foundation:**
- Phase goal or name contains: "foundation", "infrastructure", "schema", "database", "internal API", "data model", "scaffolding", "pipeline", "tooling", "CI", "migrations", "service layer", "backend", "core library"
- Phase success criteria describe only technical artifacts (files exist, tests pass, schema is valid) with no user interaction required
- There is no UI, CLI output visible to end users, or real-time behavior to observe

**If the phase IS infrastructure/foundation:** auto-pass UAT — skip the human verification items list entirely, **except any ⚠️ PRESENT_BEHAVIOR_UNVERIFIED or abstained ⚠️ `insufficient_spec` truth (see exception above), which still emits a human-verification item and drives `human_needed`.** Only when no such excepted truth exists, log:

```markdown
## Human Verification

N/A — Infrastructure/foundation phase with no user-facing elements.
All acceptance criteria are verifiable programmatically.
```

**If the phase IS user-facing:** only flag items that genuinely require a human — per the Step 8 always/uncertain lists already in the agent. Do not invent steps.

## Backstop abstention — reporting contract (#3206, companion to agent Step 3 item 5b)

When a non-inferable (`verification: backstop`) truth abstains for lack of explicit evidence:

- **Never silent, never a hard halt.** *Interactive:* the abstained item routes to the end-of-phase
  human checkpoint. *Autonomous (AFK):* it produces a prominent `unverified — held-out test
  recommended` flag and the completion line reads "complete with N unverified non-inferable checks";
  the run neither silently passes the blind spot nor hard-halts.
- **Distinguishable reason.** The abstain disposition carries `reason: insufficient_spec` so its
  `human_needed` outcome is never conflated with an ordinary manual-UAT `human_needed`.
- **Infrastructure phases included.** This rides the same carve-out as ⚠️ PRESENT_BEHAVIOR_UNVERIFIED
  in the infrastructure-phase gate above: an abstention is an evidence gap, not a user-facing step,
  so the infra auto-pass-UAT shortcut never absorbs it.

Full protocol and rationale: `gsd-core/references/honest-verifier.md`.

## Lazy references

- **Per-stack verification patterns:** before Step 4 (artifact verification) on an unfamiliar stack, Read `~/.claude/gsd-core/references/verification-patterns.md` — the grep catalog for React/Next.js components, API routes, database schema, and the universal stub patterns. Read it lazily (only the sections for the stack under verification); it is too large to load wholesale on every run.
- **Canonical report shape:** the emitted VERIFICATION.md follows `@~/.claude/gsd-core/templates/verification-report.md` — the template whose Guidelines and row shapes `src/uat.cts` treats as canonical when consuming verification output.