From c933184b97ee78f2114d3988e48e64aebdd2bfb2 Mon Sep 17 00:00:00 2001 From: Tom Boucher Date: Mon, 24 Aug 2026 19:05:11 -0400 Subject: [PATCH] enhance(#3172): require a stated failing direction for every automated acceptance command (#3825) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test(#3172): failing-first suite for the stated failing-direction probe Pins the pairing walk, placeholder denylist, MISSING sentinel exemption, degraded-read contract, CLI arm and the plan-authoring contract text. RED by construction: the module exports it requires do not exist yet. Executed on the remote runner. * feat(#3172): require a stated failing direction for every automated acceptance command Every runnable command now carries a sibling naming what output constitutes failure. A command with no expressible failure mode is not an acceptance test: it reads as rigour and is not falsifiable. - verify-command-grounding gains a failing-direction probe sharing the existing grammar, MISSING sentinel and walk guard rather than copying them - gsd-tools check verify-failure-directions backs it; plan-phase dispatches it and hands the JSON to gsd-plan-checker check 8f - Dimension 8 detail extracted to references to stay under the agent size cap Verified on the remote runner. * fix(#3172): close four review findings in the failing-direction probe - MISSING_SENTINEL_RE matched an env-var assignment prefix (MISSING=1 cmd), so a real command was exempted from the new blocking gate. Tightened the SHARED constant rather than adding a second copy. - Both token regexes scanned to EOF on unclosed openers (O(n^2), 1562ms at 40k). Bodies are now non-crossing; 1ms, byte-identical on well-formed input. The pre-existing AUTOMATED_BLOCK_RE carried the same defect and is fixed here too. - probePhaseFailingDirections reported status 'ok' when one plan was unreadable, conflating 'could not look' with 'nothing to report'. - Extracted the phase-resolution block both check arms had copied verbatim. Also corrects a docs/AGENTS.md dimension list stale since #2401. Verified on the remote runner. * fix(#3172): project the planner rule onto the spawn contract, settle emitted bookkeeping The remote runner refuted the planner-side edit. agents/gsd-planner.md is frozen under a 49152-LF-char cap asserted by four suites and sat at 49,146 — six chars of headroom — so the +537 of authoring rule blew it. #3297/#3645 already settled where such a rule goes: the planner spawn contract in plan-phase.md, beside . The agent file is reverted to origin/next verbatim. - plan-phase.md gains ; tests row 30 now asserts the contract there and row 30b guards the freeze in both directions - plan-phase.md growth acknowledged by APPENDING to the 3409 fragment, per the precedent that two ack sources may never name the same path - install-tree fixtures regenerated for the three new reference files Verified on the remote runner. * chore(#3172): backfill PR number into the changeset fragment pr:0 -> pr:3825 now that the PR exists. --------- Co-authored-by: sim --- .changeset/kind-ravens-dart.md | 5 + CONTEXT.md | 2 +- agents/gsd-plan-checker.md | 59 +- docs/AGENTS.md | 7 +- docs/COMMANDS.md | 49 ++ docs/FEATURES.md | 32 + docs/INVENTORY-MANIFEST.json | 3 + docs/INVENTORY.md | 5 +- docs/README.md | 1 + docs/how-to/state-a-failing-direction.md | 94 +++ gsd-core/references/failing-direction.md | 78 +++ gsd-core/references/nyquist-compliance.md | 74 ++ .../references/planner-failing-direction.md | 53 ++ gsd-core/workflows/plan-phase.md | 49 +- src/check-command-router.cts | 103 ++- src/verify-command-grounding.cts | 288 +++++++- .../3172-stated-failing-direction.json | 6 + tests/failing-direction.test.cjs | 657 ++++++++++++++++++ tests/fixtures/install-tree/antigravity.json | 3 + tests/fixtures/install-tree/augment.json | 3 + tests/fixtures/install-tree/claude-local.json | 3 + tests/fixtures/install-tree/claude.json | 3 + tests/fixtures/install-tree/cline.json | 3 + tests/fixtures/install-tree/codebuddy.json | 3 + tests/fixtures/install-tree/codex.json | 3 + tests/fixtures/install-tree/copilot.json | 3 + tests/fixtures/install-tree/cursor.json | 3 + tests/fixtures/install-tree/hermes.json | 3 + tests/fixtures/install-tree/kilo.json | 3 + tests/fixtures/install-tree/kimi-code.json | 3 + tests/fixtures/install-tree/kimi.json | 3 + tests/fixtures/install-tree/opencode.json | 3 + tests/fixtures/install-tree/pi.json | 3 + tests/fixtures/install-tree/qwen.json | 3 + tests/fixtures/install-tree/trae.json | 3 + tests/fixtures/install-tree/windsurf.json | 3 + tests/fixtures/install-tree/zcode.json | 3 + 37 files changed, 1537 insertions(+), 85 deletions(-) create mode 100644 .changeset/kind-ravens-dart.md create mode 100644 docs/how-to/state-a-failing-direction.md create mode 100644 gsd-core/references/failing-direction.md create mode 100644 gsd-core/references/nyquist-compliance.md create mode 100644 gsd-core/references/planner-failing-direction.md create mode 100644 tests/emitted-drift-acks/3172-stated-failing-direction.json create mode 100644 tests/failing-direction.test.cjs diff --git a/.changeset/kind-ravens-dart.md b/.changeset/kind-ravens-dart.md new file mode 100644 index 000000000..23718177e --- /dev/null +++ b/.changeset/kind-ravens-dart.md @@ -0,0 +1,5 @@ +--- +type: Changed +pr: 3825 +--- +**Plans must now say what output constitutes failure** — every runnable `` acceptance command needs a `` sibling naming an observable failure signal, and `/gsd-plan-phase` blocks a plan that omits one. A command with no expressible failure mode is not an acceptance test. Breaking for phases planned before this release: re-check reports one blocker per unstated command until statements are added or the phase is re-planned. (#3172) diff --git a/CONTEXT.md b/CONTEXT.md index a70a6adff..fab006c70 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -27,7 +27,7 @@ Module owning phase-effort estimation and its calibration against measured reali Module owning the canonical phase-verification status projection shared by phase transition, progress, manager, autonomous, and closeout readiness paths. `readVerificationStatus(phaseDir, opts?)` reads the first `*-VERIFICATION.md` frontmatter `status`, maps it through `VERIFICATION_ROUTING_TABLE`, and fail-closes — only `{passed}` satisfies the canonical gate; `missing`/`unknown`/`gaps_found`/`human_needed`/`stale` all route away from "complete" (#1522). `findStaleVerificationSummary` flags a SUMMARY newer than the VERIFICATION file (status `stale`). Both honor a no-throw, degrade-to-safe contract (any FS error → `missing` / not-stale) and an injectable `opts.fs` seam. `isPhaseComplete(phaseDir, deps?)` is the single canonical owner of "is phase P complete?" (ADR-3180 §7.4, issue #3186, disk-strict per #2957): it wraps `readVerificationStatus`, calling it UNCONDITIONALLY — plan count is never a precondition, so a zero-plan phase with a passing `*-VERIFICATION.md` is complete (#3168) — and returns `{ value: { complete, verification }, scope }`; `complete` is exactly `verification.status === 'passed'`. A ROADMAP checkbox carries no machine authority and is never consulted. `cmdPhaseComplete`, `buildPhaseCompletionProjection`, and `buildStateFrontmatter` all route through it. Source of truth: `gsd-core/bin/lib/verification.cjs` (generated from `src/verification.cts`). ### Verify Command Grounding Module -Module owning the deterministic resolvability probe over PLAN.md `` verify commands (#2401), plus the prior-phase command harvest that feeds the planner. `extractAutomatedCommands(planText)` pulls every `…` body with its owning ``, in document order, via a ReDoS-safe stop-at-next-open task pattern (shape mirrors `PLAN_TASK_BLOCK_RE` in `verify.cjs`) and a monotonic span pointer; non-string input yields `[]`. `resolveVerifyCommandTarget(command, {projectRoot, declaredPaths})` is a **RECOGNIZER, not a shell interpreter** (deliberate, per Greenspun): it grounds exactly two forms — a folded leading `cd ` chain and `npm --prefix ` — and any path carrying `$`, a backtick, `*`, `?`, `~`, or a newline returns `unresolvable`/`dynamic_path` at WARNING severity, never BLOCKER. Status is a closed 5-atom enum (`ok`/`broken`/`unresolvable`/`not_applicable`/`pending_creation`) and severity a closed 3-atom enum (`blocker`/`warning`/`none`); `broken` is only ever `missing_dir` or `no_manifest`, while `script_missing`/`manifest_unreadable`/`outside_root` stay advisory on an `ok` status. A target an earlier task in the same phase declares (`` or the `## Artifacts this phase produces` section) is `pending_creation`, never a blocker — without that, every greenfield phase would red. A bare ancestor climb (`cd ../..`, every segment `..`) short-circuits to `outside_root` without touching the filesystem, because the checker's root and a parallel executor worktree's root differ; a climb naming a concrete sibling (`cd ../../frontend` — the exact #2401 shape) still names something checkable and is probed normally. **The module never executes command text** (`fs.statSync`/`existsSync`/`readFileSync`/`readdirSync` only — PLAN.md is model-authored untrusted input) and deliberately exposes **no `suggestion` field**: prescribing a replacement path is the failure being fixed, not the fix. `probePhaseVerifyCommands({phaseDir, projectRoot})` backs `gsd-tools check verify-command-paths ` (routed in `check-command-router.cjs`), degrading to a populated `readError` rather than throwing — an empty `commands` with a non-empty `readError` means *could not look*, not *nothing to report*. `harvestPriorVerifyCommands({planningDir, beforePhase, limit=20, lookback=3})` walks descending phase dirs for the nearest prior phase with any command, deduped first-seen and capped, and is emitted as `init.plan-phase`'s `prior_verify_commands` **ungated by `context_window`** — the `>= 500000` enrichment gate is exactly what starved the planner at 200k. Source of truth: `gsd-core/bin/lib/verify-command-grounding.cjs` (generated from `src/verify-command-grounding.cts`). +Module owning the deterministic resolvability probe over PLAN.md `` verify commands (#2401), plus the prior-phase command harvest that feeds the planner. `extractAutomatedCommands(planText)` pulls every `…` body with its owning ``, in document order, via a ReDoS-safe stop-at-next-open task pattern (shape mirrors `PLAN_TASK_BLOCK_RE` in `verify.cjs`) and a monotonic span pointer; non-string input yields `[]`. `resolveVerifyCommandTarget(command, {projectRoot, declaredPaths})` is a **RECOGNIZER, not a shell interpreter** (deliberate, per Greenspun): it grounds exactly two forms — a folded leading `cd ` chain and `npm --prefix ` — and any path carrying `$`, a backtick, `*`, `?`, `~`, or a newline returns `unresolvable`/`dynamic_path` at WARNING severity, never BLOCKER. Status is a closed 5-atom enum (`ok`/`broken`/`unresolvable`/`not_applicable`/`pending_creation`) and severity a closed 3-atom enum (`blocker`/`warning`/`none`); `broken` is only ever `missing_dir` or `no_manifest`, while `script_missing`/`manifest_unreadable`/`outside_root` stay advisory on an `ok` status. A target an earlier task in the same phase declares (`` or the `## Artifacts this phase produces` section) is `pending_creation`, never a blocker — without that, every greenfield phase would red. A bare ancestor climb (`cd ../..`, every segment `..`) short-circuits to `outside_root` without touching the filesystem, because the checker's root and a parallel executor worktree's root differ; a climb naming a concrete sibling (`cd ../../frontend` — the exact #2401 shape) still names something checkable and is probed normally. **The module never executes command text** (`fs.statSync`/`existsSync`/`readFileSync`/`readdirSync` only — PLAN.md is model-authored untrusted input) and deliberately exposes **no `suggestion` field**: prescribing a replacement path is the failure being fixed, not the fix. `probePhaseVerifyCommands({phaseDir, projectRoot})` backs `gsd-tools check verify-command-paths ` (routed in `check-command-router.cjs`), degrading to a populated `readError` rather than throwing — an empty `commands` with a non-empty `readError` means *could not look*, not *nothing to report*. `harvestPriorVerifyCommands({planningDir, beforePhase, limit=20, lookback=3})` walks descending phase dirs for the nearest prior phase with any command, deduped first-seen and capped, and is emitted as `init.plan-phase`'s `prior_verify_commands` **ungated by `context_window`** — the `>= 500000` enrichment gate is exactly what starved the planner at 200k. **Failing-direction probe (#3172), the module's second concern.** Every runnable `` must carry a `` sibling naming what output constitutes failure; a command with no expressible failure mode is not an acceptance test. `extractFailingDirections(planText)` recovers `` and `` in ONE document-order pass (a single backreferenced alternation — two independent scans would discard the relative positions the pairing walk needs) over the SAME text units as `extractAutomatedCommands` (each `` body, then the task-stripped remainder), sharing that function's `MAX_BLOCK_WALK` guard, `MISSING_SENTINEL_RE` and task grammar rather than copying them (`DEFECT.GENERATIVE-FIX-DIVERGENCE`); pairing never crosses a task boundary. Each `` binds to the nearest PRECEDING ``, FIRST-WINS — a redundant second statement for one command is ignored and adds no row, and a statement preceding every command is an `orphan` WARNING, never a blocker. `resolveFailingDirection(command, statement)` is the single verdict implementation: status is a closed 6-atom enum (`ok`/`missing`/`empty`/`placeholder`/`sentinel`/`orphan`) over the same 3-atom severity enum, and check ORDER is load-bearing — empty command first, then the `MISSING` Wave-0 sentinel (exempt at `none` even when a statement IS present, because it is not runnable and without that exemption every greenfield phase would red), then missing/empty/placeholder. The placeholder set (`tbd`/`todo`/`n/a`/`na`/`none`/`unknown`/`tba`/`?`/`-`) matches the WHOLE trimmed value case-insensitively, never a substring: `"TBD in the harness output"` is real prose and passes. **PRESENCE only** — whether a statement names the RIGHT signal stays plan-checker judgment at WARNING, so every BLOCKER is deterministic and reproducible. The probe never prescribes a statement, for the same reason it never prescribes a path: a prescribed one is copied verbatim and carries zero information. `probePhaseFailingDirections({phaseDir})` backs `gsd-tools check verify-failure-directions `, degrading to a populated `readError` with top-level `status: 'unresolvable'` — an empty `commands` with a non-empty `readError` means *could not look*, not *nothing to report*. The #2401 path-probe surface is deliberately NOT overloaded (its 5-atom status and counts are unchanged by a missing statement) so `docs/how-to/resolve-verify-command-path-findings.md` stays true. Source of truth: `gsd-core/bin/lib/verify-command-grounding.cjs` (generated from `src/verify-command-grounding.cts`). Design: `.gsd/phase/feat-3172-stated-failing-direction/40-design.md`. ### Phase Locator Module Module owning phase-directory search and location: active-phase discovery against the `.planning/phases/` tree (`searchPhaseInDir`, `findPhaseInternal`) and archived-phase-dir enumeration (`getArchivedPhaseDirs`), matching phase ids/tokens against the filesystem. Depends only on leaf modules (`phase-id` for token/name matching, `core-utils` for fs-scan/path helpers, `planning-workspace` for `planningDir`) — no `loadConfig`, no other core dependency. Extracted from the Core module per ADR-857 rollout phase 2d (#881); the `core.cjs` re-export spine was retired in epic #1267, so callers import this leaf directly. Source of truth: `gsd-core/bin/lib/phase-locator.cjs` (generated from `src/phase-locator.cts`). Since #2830, `searchPhaseInDir` also parses each plan's `depends_on` and each completed plan's SUMMARY `status` and calls Plan Dependency Graph Module's `computeHaltPropagation` to populate `halted_plans`/`blocked_by`/`runnable_plans` — additive fields; `incomplete_plans` keeps its pre-#2830 meaning unchanged. Since #3185 (ADR-3180 Decision 1, Phase 3), the module also owns `listMilestonePhaseDirs(phasesDir, { cwd, ws, versionOverride, phaseIdConvention })`, the single canonical owner of milestone-scoped phase-directory enumeration: it applies the current milestone's `ROADMAP.md` window (via `getMilestonePhaseFilter`) and then the canonical `isSentinelPhaseId` sentinel filter, in that order, over the raw `phasesDir` directory listing. It returns `{ value: string[], scope }`, where `scope` is the `SCOPE` enum from `src/planning-scope.cts` (`complete`/`truncated`/`unscoped`/`unreadable`), so a caller can distinguish a genuinely empty milestone from an enumeration that could not be scoped. Consumed by `query progress`, `stats`, and the bare `phases list`, all of which need "which phases belong to this milestone." `phases list --phase` and `--include-archived` (lookup/archive questions) read the unscoped physical directory set and do not call this owner. `phases clear` and `milestone complete`'s phase-archival move call `isSentinelPhaseId` directly instead — they must sweep every non-sentinel phase directory regardless of milestone window, so they take the sentinel filter without this owner's window scoping. diff --git a/agents/gsd-plan-checker.md b/agents/gsd-plan-checker.md index 1be809339..6455173d3 100644 --- a/agents/gsd-plan-checker.md +++ b/agents/gsd-plan-checker.md @@ -497,61 +497,16 @@ issue: ## Dimension 8: Nyquist Compliance -Skip if: `workflow.nyquist_validation` is explicitly set to `false` in config.json (absent key = enabled), phase has no RESEARCH.md, or RESEARCH.md has no "Validation Architecture" section. Output: "Dimension 8: SKIPPED (nyquist_validation disabled or not applicable)" +**Question:** Is every task's completion decided by an automated check that can actually fail? -### Check 8e — VALIDATION.md Existence (Gate) +Checks 8a-8e (presence, latency, sampling continuity, Wave 0 completeness, VALIDATION.md gate), +their skip condition and the Dimension 8 output table: @gsd-core/references/nyquist-compliance.md -Before running checks 8a-8d, verify VALIDATION.md exists: +### Check 8f - Stated Failing Direction (#3172) -```bash -ls "${PHASE_DIR}"/*-VALIDATION.md 2>/dev/null -``` - -**If missing:** **BLOCKING FAIL** — "VALIDATION.md not found for phase {N}. Re-run `/gsd:plan-phase {N} --research` to regenerate." -Skip checks 8a-8d entirely. Report Dimension 8 as FAIL with this single issue. - -**If exists:** Proceed to checks 8a-8d. - -### Check 8a — Automated Verify Presence - -For each `` in each plan: -- `` must contain `` command, OR a Wave 0 dependency that creates the test first -- If `` is absent with no Wave 0 dependency → **BLOCKING FAIL** -- If `` says "MISSING", a Wave 0 task must reference the same test file path → **BLOCKING FAIL** if link broken - -### Check 8b — Feedback Latency Assessment - -For each `` command: -- Full E2E suite (playwright, cypress, selenium) → **WARNING** — suggest faster unit/smoke test -- Watch mode flags (`--watchAll`) → **BLOCKING FAIL** -- Delays > 30 seconds → **WARNING** - -### Check 8c — Sampling Continuity - -Map tasks to waves. Per wave, any consecutive window of 3 implementation tasks must have ≥2 with `` verify. 3 consecutive without → **BLOCKING FAIL**. - -### Check 8d — Wave 0 Completeness - -For each `MISSING` reference: -- Wave 0 task must exist with matching `` path -- Wave 0 plan must execute before dependent task -- Missing match → **BLOCKING FAIL** - -### Dimension 8 Output - -``` -## Dimension 8: Nyquist Compliance - -| Task | Plan | Wave | Automated Command | Status | -|------|------|------|-------------------|--------| -| {task} | {plan} | {wave} | `{command}` | ✅ / ❌ | - -Sampling: Wave {N}: {X}/{Y} verified → ✅ / ❌ -Wave 0: {test file} → ✅ present / ❌ MISSING -Overall: ✅ PASS / ❌ FAIL -``` - -If FAIL: return to planner with specific fixes. Same revision loop as other dimensions (max 3 loops). +Each runnable `` command needs a `` sibling naming what output constitutes +failure. Consume the supplied `{FAILING_DIRECTIONS}` probe, never re-derive it: +@gsd-core/references/failing-direction.md ## Dimension 9: Cross-Plan Data Contracts diff --git a/docs/AGENTS.md b/docs/AGENTS.md index baea495e3..6afd0915b 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -175,6 +175,7 @@ GSD uses a multi-agent architecture where thin orchestrators (workflow files) sp - Reads PROJECT.md, REQUIREMENTS.md, CONTEXT.md, RESEARCH.md - Creates 2-3 atomic task plans sized for single context windows - Uses XML structure with `` elements +- Emits a `` sibling for every runnable `` verify command, naming what output constitutes failure (#3172) - Includes `read_first` and `acceptance_criteria` sections - Groups plans into dependency waves - Performs reachability check to validate plan steps reference accessible files and APIs (v1.32) @@ -255,14 +256,14 @@ GSD uses a multi-agent architecture where thin orchestrators (workflow files) sp | 7 | Context compliance (when CONTEXT.md exists) | | 7b | Scope reduction detection | | 7c | Architectural tier compliance (when RESEARCH.md defines a responsibility map) | -| 8 | Nyquist compliance (when enabled) | +| 8 | Nyquist compliance (when enabled) — checks 8a-8e cover automated-verify presence, feedback latency, sampling continuity and Wave 0 completeness; check 8f blocks a runnable `` command with no stated `` failing direction (#3172) | | 9 | Cross-plan data contracts | | 10 | CLAUDE.md compliance | | 11 | Research resolution | | 12 | Pattern compliance | -Two further dimensions carry no number: **Verify Command Format Sanity** and -**Numeric/Factual Claim Authority**. +Three further dimensions carry no number: **Verify Command Format Sanity**, +**Verify Command Path Resolvability**, and **Numeric/Factual Claim Authority**. --- diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md index b0d2719cd..1115fb7e5 100644 --- a/docs/COMMANDS.md +++ b/docs/COMMANDS.md @@ -1269,6 +1269,55 @@ gsd-tools check verify-command-paths 3 --raw # probe phase 3's verify command See [Resolve verify-command path findings](how-to/resolve-verify-command-path-findings.md). +### `gsd-tools check verify-failure-directions` + +Deterministic presence probe over a phase's stated failing directions (#3172). Run automatically +by `/gsd-plan-phase` before the plan-check pass and handed to `gsd-plan-checker`; runnable by +hand to see what the checker saw. + +Every runnable `` command must carry a `` sibling naming what output +constitutes failure. A command with no expressible failure mode is not an acceptance test. + +| Argument | Required | Description | +|----------|----------|-------------| +| `N` | **Yes** | Phase number whose `-PLAN.md` files are probed | + +| Flag | Description | +|------|-------------| +| `--raw` | Emit the JSON payload with no surrounding prose | + +**Prerequisites:** none — an unresolvable phase degrades to a JSON payload with `readError` set +rather than failing. +**Produces:** JSON on stdout. Nothing is written to disk. + +**It never executes command text**, and it never authors a statement for the planner — a +prescribed failure signal would be copied verbatim and carry no information. + +**Pairing.** Within one ``, each `` binds to the nearest **preceding** +``; the first statement after a command is the binding one. N runnable commands need +N statements. A redundant second statement for the same command is ignored. + +Each row of `commands` carries `command`, `statement`, `plan`, `task`, `status`, and `severity`. + +| `status` | `severity` | Meaning | +|---|---|---| +| `ok` | `none` | A non-empty, non-placeholder statement is bound to this command | +| `missing` | `blocker` | The command has no `` at all | +| `empty` | `blocker` | A `` is present but blank | +| `placeholder` | `blocker` | The whole statement is `TBD`, `TODO`, `N/A`, `NA`, `none`, `unknown`, `TBA`, `?`, or `-` (case-insensitive, whole value only) | +| `orphan` | `warning` | A `` that follows no command — it satisfies nothing | +| `sentinel` | `none` | A Nyquist `MISSING — Wave 0 …` placeholder; not runnable, so exempt | + +The top-level `status` is `blocked` when any row is a blocker, `unresolvable` when the probe +could not look, and `ok` otherwise. A non-empty `readError` means the probe **could not look** — +distinct from finding nothing. + +```bash +gsd-tools check verify-failure-directions 3 --raw # probe phase 3's failing directions +``` + +See [State a failing direction](how-to/state-a-failing-direction.md). + --- ## Workstream Management diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 8d29526c2..10c12dd45 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -3574,3 +3574,35 @@ See [Resolve verify-command path findings](how-to/resolve-verify-command-path-fi **Known limits:** an empty `phases: []` cannot be told apart from "no `ROADMAP.md`" or "roadmap unreadable" — the `1.0` schema carries no diagnostic channel, and `planning inspect` is the surface that does. A roadmap phase marked `Deferred` is reported as `pending`, because the roadmap vocabulary has four values and this contract has three; inventing a fourth wire value would break every existing reader. `phases[]` is not milestone-scoped, so a long-running project lists every phase it has ever had. **Reference:** [Consume the state contract](how-to/consume-the-state-contract.md) · [Consume the planning snapshot](how-to/consume-the-planning-snapshot.md) + +--- + +### 167. Stated Failing Direction + +**Command:** `/gsd-plan-phase` (automatic), `gsd-tools check verify-failure-directions ` (#3172) + +**Behavior:** A plan's `` block is the thing that decides whether work is done, and nothing checked that the command inside it could fail. In the motivating case six plans shipped 21 commands that could not run at all — `cargo test -p --lib` against a package with no library target. They read as rigour and were not falsifiable, so three separate executors each rediscovered the defect and improvised a substitute at execution time. Every runnable `` command now needs a `` sibling naming what output constitutes failure: + +```xml + + npm --prefix apps/api test -- auth.spec.ts + non-zero exit, or "0 passed" in the summary line + +``` + +`gsd-planner` emits it; `gsd-tools check verify-failure-directions ` verifies it deterministically; `/gsd-plan-phase` runs the probe before the plan-check pass and hands the JSON to `gsd-plan-checker`, whose check 8f blocks on `severity`. + +**Why this shape and not the two obvious alternatives.** *Validating command shape* — teaching the checker Cargo's `--lib`/`--bin` target resolution, then pytest's node-ids, then the next one — always trails the newest toolchain. *Executing each command at plan time* is the strongest signal but means running planner-invented commands, with whatever side effects they carry, during planning. Requiring a stated failing direction needs no toolchain knowledge at all, and it is the only one of the three that catches the dangerous case: the motivating command exited non-zero, so it failed loudly, but the same class of error with a command that exits 0 on a no-op passes green and silently. Naming the failure signal is what makes that visible. + +**Presence, not quality — deliberately split.** The probe is deterministic and owns the blockers: a statement is missing, blank, or a whole-value placeholder (`TBD`, `TODO`, `N/A`, `NA`, `none`, `unknown`, `TBA`, `?`, `-`). Whether the statement names the *right* signal is prose judgment, so `gsd-plan-checker` raises a vacuous statement (*"the command fails"*) as a WARNING only. Every BLOCKER stays reproducible; judgment stays advisory. + +**It reports, it never prescribes.** The payload names the command with no stated failure mode and stops there. A prescribed statement would be copied verbatim and carry zero information — reproducing the original defect one level up. + +**Not findings:** the Nyquist `MISSING — Wave 0 …` sentinel (not runnable, so it has no failure mode to state), an empty `` body (check 8a owns command presence), and a `` with no `` at all. + +**Known limits:** +- Presence only. A statement that is present and specific can still name the wrong signal; that is caught, if at all, by judgment rather than by the probe. +- **Breaking for plans authored before this shipped.** A phase planned earlier has no `` anywhere and blocks on re-check until statements are added or the phase is re-planned. +- The adjacent **vacuous pass** — a command that runs successfully and asserts nothing, such as a test-name filter matching zero tests and exiting 0 — is a distinct problem and is explicitly out of scope. + +See [State a failing direction](how-to/state-a-failing-direction.md) and [`gsd-tools check verify-failure-directions`](COMMANDS.md#gsd-tools-check-verify-failure-directions). diff --git a/docs/INVENTORY-MANIFEST.json b/docs/INVENTORY-MANIFEST.json index 06b4279fd..5f7ce29fa 100644 --- a/docs/INVENTORY-MANIFEST.json +++ b/docs/INVENTORY-MANIFEST.json @@ -235,6 +235,7 @@ "execute-phase-response-language.md", "execute-phase-wave-guard.md", "executor-examples.md", + "failing-direction.md", "gate-prompts.md", "gates.md", "git-integration.md", @@ -247,10 +248,12 @@ "model-profile-resolution.md", "model-profiles.md", "mvp-concepts.md", + "nyquist-compliance.md", "offer-next.md", "phase-argument-parsing.md", "planner-antipatterns.md", "planner-chunked.md", + "planner-failing-direction.md", "planner-gap-closure.md", "planner-graphify-auto-update.md", "planner-guidance.md", diff --git a/docs/INVENTORY.md b/docs/INVENTORY.md index c8a0ef4ae..403061bfe 100644 --- a/docs/INVENTORY.md +++ b/docs/INVENTORY.md @@ -332,6 +332,8 @@ Full roster at `gsd-core/references/*.md`. References are shared knowledge docum | `research-philosophy.md` | Shared research philosophy (training-as-hypothesis, honest reporting, investigation-not-confirmation) injected into researcher agents. | | `research-verification-protocol.md` | Shared research verification protocol (4 pitfalls + pre-submission checklist) injected into researcher agents. | | `verify-command-path-resolvability.md` | Verify Command Path Resolvability dimension (#2401) loaded by `gsd-plan-checker`: how to consume the `{VERIFY_PATHS}` probe result (never re-run or hand-reason the filesystem), the severity/reason table, and report-never-prescribe rules. | +| `nyquist-compliance.md` | Dimension 8 checks 8a-8e (#3172) loaded by `gsd-plan-checker`: the VALIDATION.md gate, automated-verify presence, feedback-latency assessment, sampling continuity, Wave 0 completeness, and the Dimension 8 output table — extracted from the agent to stay under its LARGE size cap. | +| `failing-direction.md` | Check 8f, Stated Failing Direction (#3172), loaded by `gsd-plan-checker`: how to consume the `{FAILING_DIRECTIONS}` probe result, the status/severity table, the sentinel exemption, and the split between deterministic blockers and advisory vacuous-statement warnings. | ### Workflow References @@ -425,6 +427,7 @@ The `gsd-planner` agent is decomposed into a core agent plus reference modules t | `planner-interface-context.md` | Interface context rules for executors — how to extract key interfaces/types/exports from existing code and document new interfaces that downstream plans will consume. | | `planner-load-graph-context.md` | Planner's load_graph_context step: knowledge-graph freshness + dependency-context query via the gsd_run launcher (extracted from gsd-planner.md). | | `planner-verify-command-grounding.md` | Verify Command Grounding rules (#2401): inherit `prior_verify_commands` verbatim when the story repeats, prefer `npm --prefix run