golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

enhance(#3172): require a stated failing direction for every automated acceptance command (#3825)

Browse Source 
* test(#3172): failing-first suite for the stated failing-direction probe

Pins the <fails_when> pairing walk, placeholder denylist, MISSING sentinel
exemption, degraded-read contract, CLI arm and the plan-authoring contract text.
RED by construction: the module exports it requires do not exist yet.
Executed on the remote runner.

* feat(#3172): require a stated failing direction for every automated acceptance command

Every runnable <automated> command now carries a <fails_when> sibling naming
what output constitutes failure. A command with no expressible failure mode is
not an acceptance test: it reads as rigour and is not falsifiable.

- verify-command-grounding gains a failing-direction probe sharing the existing
  <automated> grammar, MISSING sentinel and walk guard rather than copying them
- gsd-tools check verify-failure-directions <N> backs it; plan-phase dispatches
  it and hands the JSON to gsd-plan-checker check 8f
- Dimension 8 detail extracted to references to stay under the agent size cap

Verified on the remote runner.

* fix(#3172): close four review findings in the failing-direction probe

- MISSING_SENTINEL_RE matched an env-var assignment prefix (MISSING=1 cmd), so
  a real command was exempted from the new blocking gate. Tightened the SHARED
  constant rather than adding a second copy.
- Both token regexes scanned to EOF on unclosed openers (O(n^2), 1562ms at 40k).
  Bodies are now non-crossing; 1ms, byte-identical on well-formed input. The
  pre-existing AUTOMATED_BLOCK_RE carried the same defect and is fixed here too.
- probePhaseFailingDirections reported status 'ok' when one plan was unreadable,
  conflating 'could not look' with 'nothing to report'.
- Extracted the phase-resolution block both check arms had copied verbatim.

Also corrects a docs/AGENTS.md dimension list stale since #2401.
Verified on the remote runner.

* fix(#3172): project the planner rule onto the spawn contract, settle emitted bookkeeping

The remote runner refuted the planner-side edit. agents/gsd-planner.md is frozen
under a 49152-LF-char cap asserted by four suites and sat at 49,146 — six chars
of headroom — so the +537 of authoring rule blew it. #3297/#3645 already settled
where such a rule goes: the planner spawn contract in plan-phase.md, beside
<tracked_source_paths>. The agent file is reverted to origin/next verbatim.

- plan-phase.md gains <failing_direction_contract>; tests row 30 now asserts the
  contract there and row 30b guards the freeze in both directions
- plan-phase.md growth acknowledged by APPENDING to the 3409 fragment, per the
  precedent that two ack sources may never name the same path
- install-tree fixtures regenerated for the three new reference files

Verified on the remote runner.

* chore(#3172): backfill PR number into the changeset fragment

pr:0 -> pr:3825 now that the PR exists.

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-24 19:05:11 -04:00
committed by GitHub
parent 7a41248c4f
commit c933184b97
37 changed files with 1537 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/AGENTS.md
View File

@@ -175,6 +175,7 @@ GSD uses a multi-agent architecture where thin orchestrators (workflow files) sp
- Reads PROJECT.md, REQUIREMENTS.md, CONTEXT.md, RESEARCH.md
- Creates 2-3 atomic task plans sized for single context windows
- Uses XML structure with `<task>` elements
- Emits a `<fails_when>` sibling for every runnable `<automated>` verify command, naming what output constitutes failure (#3172)
- Includes `read_first` and `acceptance_criteria` sections
- Groups plans into dependency waves
- Performs reachability check to validate plan steps reference accessible files and APIs (v1.32)
@@ -255,14 +256,14 @@ GSD uses a multi-agent architecture where thin orchestrators (workflow files) sp
| 7 | Context compliance (when CONTEXT.md exists) |
| 7b | Scope reduction detection |
| 7c | Architectural tier compliance (when RESEARCH.md defines a responsibility map) |
| 8 | Nyquist compliance (when enabled) |
| 8 | Nyquist compliance (when enabled) — checks 8a-8e cover automated-verify presence, feedback latency, sampling continuity and Wave 0 completeness; check 8f blocks a runnable `<automated>` command with no stated `<fails_when>` failing direction (#3172) |
| 9 | Cross-plan data contracts |
| 10 | CLAUDE.md compliance |
| 11 | Research resolution |
| 12 | Pattern compliance |
Two further dimensions carry no number: **Verify Command Format Sanity** and
**Numeric/Factual Claim Authority**.
Three further dimensions carry no number: **Verify Command Format Sanity**,
**Verify Command Path Resolvability**, and **Numeric/Factual Claim Authority**.
---

49
docs/COMMANDS.md
View File

@@ -1269,6 +1269,55 @@ gsd-tools check verify-command-paths 3 --raw # probe phase 3's verify command
See [Resolve verify-command path findings](how-to/resolve-verify-command-path-findings.md).
### `gsd-tools check verify-failure-directions`
Deterministic presence probe over a phase's stated failing directions (#3172). Run automatically
by `/gsd-plan-phase` before the plan-check pass and handed to `gsd-plan-checker`; runnable by
hand to see what the checker saw.
Every runnable `<automated>` command must carry a `<fails_when>` sibling naming what output
constitutes failure. A command with no expressible failure mode is not an acceptance test.
| Argument | Required | Description |
|----------|----------|-------------|
| `N` | **Yes** | Phase number whose `-PLAN.md` files are probed |
| Flag | Description |
|------|-------------|
| `--raw` | Emit the JSON payload with no surrounding prose |
**Prerequisites:** none — an unresolvable phase degrades to a JSON payload with `readError` set
rather than failing.
**Produces:** JSON on stdout. Nothing is written to disk.
**It never executes command text**, and it never authors a statement for the planner — a
prescribed failure signal would be copied verbatim and carry no information.
**Pairing.** Within one `<task>`, each `<fails_when>` binds to the nearest **preceding**
`<automated>`; the first statement after a command is the binding one. N runnable commands need
N statements. A redundant second statement for the same command is ignored.
Each row of `commands` carries `command`, `statement`, `plan`, `task`, `status`, and `severity`.
| `status` | `severity` | Meaning |
|---|---|---|
| `ok` | `none` | A non-empty, non-placeholder statement is bound to this command |
| `missing` | `blocker` | The command has no `<fails_when>` at all |
| `empty` | `blocker` | A `<fails_when>` is present but blank |
| `placeholder` | `blocker` | The whole statement is `TBD`, `TODO`, `N/A`, `NA`, `none`, `unknown`, `TBA`, `?`, or `-` (case-insensitive, whole value only) |
| `orphan` | `warning` | A `<fails_when>` that follows no command — it satisfies nothing |
| `sentinel` | `none` | A Nyquist `MISSING — Wave 0 …` placeholder; not runnable, so exempt |
The top-level `status` is `blocked` when any row is a blocker, `unresolvable` when the probe
could not look, and `ok` otherwise. A non-empty `readError` means the probe **could not look** —
distinct from finding nothing.
```bash
gsd-tools check verify-failure-directions 3 --raw # probe phase 3's failing directions
```
See [State a failing direction](how-to/state-a-failing-direction.md).
---
## Workstream Management

32
docs/FEATURES.md
View File

@@ -3574,3 +3574,35 @@ See [Resolve verify-command path findings](how-to/resolve-verify-command-path-fi
**Known limits:** an empty `phases: []` cannot be told apart from "no `ROADMAP.md`" or "roadmap unreadable" — the `1.0` schema carries no diagnostic channel, and `planning inspect` is the surface that does. A roadmap phase marked `Deferred` is reported as `pending`, because the roadmap vocabulary has four values and this contract has three; inventing a fourth wire value would break every existing reader. `phases[]` is not milestone-scoped, so a long-running project lists every phase it has ever had.
**Reference:** [Consume the state contract](how-to/consume-the-state-contract.md) · [Consume the planning snapshot](how-to/consume-the-planning-snapshot.md)
---
### 167. Stated Failing Direction
**Command:** `/gsd-plan-phase` (automatic), `gsd-tools check verify-failure-directions <N>` (#3172)
**Behavior:** A plan's `<automated>` block is the thing that decides whether work is done, and nothing checked that the command inside it could fail. In the motivating case six plans shipped 21 commands that could not run at all — `cargo test -p <pkg> --lib` against a package with no library target. They read as rigour and were not falsifiable, so three separate executors each rediscovered the defect and improvised a substitute at execution time. Every runnable `<automated>` command now needs a `<fails_when>` sibling naming what output constitutes failure:
```xml
<verify>
<automated>npm --prefix apps/api test -- auth.spec.ts</automated>
<fails_when>non-zero exit, or "0 passed" in the summary line</fails_when>
</verify>
```
`gsd-planner` emits it; `gsd-tools check verify-failure-directions <N>` verifies it deterministically; `/gsd-plan-phase` runs the probe before the plan-check pass and hands the JSON to `gsd-plan-checker`, whose check 8f blocks on `severity`.
**Why this shape and not the two obvious alternatives.** *Validating command shape* — teaching the checker Cargo's `--lib`/`--bin` target resolution, then pytest's node-ids, then the next one — always trails the newest toolchain. *Executing each command at plan time* is the strongest signal but means running planner-invented commands, with whatever side effects they carry, during planning. Requiring a stated failing direction needs no toolchain knowledge at all, and it is the only one of the three that catches the dangerous case: the motivating command exited non-zero, so it failed loudly, but the same class of error with a command that exits 0 on a no-op passes green and silently. Naming the failure signal is what makes that visible.
**Presence, not quality — deliberately split.** The probe is deterministic and owns the blockers: a statement is missing, blank, or a whole-value placeholder (`TBD`, `TODO`, `N/A`, `NA`, `none`, `unknown`, `TBA`, `?`, `-`). Whether the statement names the *right* signal is prose judgment, so `gsd-plan-checker` raises a vacuous statement (*"the command fails"*) as a WARNING only. Every BLOCKER stays reproducible; judgment stays advisory.
**It reports, it never prescribes.** The payload names the command with no stated failure mode and stops there. A prescribed statement would be copied verbatim and carry zero information — reproducing the original defect one level up.
**Not findings:** the Nyquist `MISSING — Wave 0 …` sentinel (not runnable, so it has no failure mode to state), an empty `<automated>` body (check 8a owns command presence), and a `<verify>` with no `<automated>` at all.
**Known limits:**
- Presence only. A statement that is present and specific can still name the wrong signal; that is caught, if at all, by judgment rather than by the probe.
- **Breaking for plans authored before this shipped.** A phase planned earlier has no `<fails_when>` anywhere and blocks on re-check until statements are added or the phase is re-planned.
- The adjacent **vacuous pass** — a command that runs successfully and asserts nothing, such as a test-name filter matching zero tests and exiting 0 — is a distinct problem and is explicitly out of scope.
See [State a failing direction](how-to/state-a-failing-direction.md) and [`gsd-tools check verify-failure-directions`](COMMANDS.md#gsd-tools-check-verify-failure-directions).

3
docs/INVENTORY-MANIFEST.json
View File

@@ -235,6 +235,7 @@
"execute-phase-response-language.md",
"execute-phase-wave-guard.md",
"executor-examples.md",
"failing-direction.md",
"gate-prompts.md",
"gates.md",
"git-integration.md",
@@ -247,10 +248,12 @@
"model-profile-resolution.md",
"model-profiles.md",
"mvp-concepts.md",
"nyquist-compliance.md",
"offer-next.md",
"phase-argument-parsing.md",
"planner-antipatterns.md",
"planner-chunked.md",
"planner-failing-direction.md",
"planner-gap-closure.md",
"planner-graphify-auto-update.md",
"planner-guidance.md",

5
docs/INVENTORY.md
View File

@@ -332,6 +332,8 @@ Full roster at `gsd-core/references/*.md`. References are shared knowledge docum
| `research-philosophy.md` | Shared research philosophy (training-as-hypothesis, honest reporting, investigation-not-confirmation) injected into researcher agents. |
| `research-verification-protocol.md` | Shared research verification protocol (4 pitfalls + pre-submission checklist) injected into researcher agents. |
| `verify-command-path-resolvability.md` | Verify Command Path Resolvability dimension (#2401) loaded by `gsd-plan-checker`: how to consume the `{VERIFY_PATHS}` probe result (never re-run or hand-reason the filesystem), the severity/reason table, and report-never-prescribe rules. |
| `nyquist-compliance.md` | Dimension 8 checks 8a-8e (#3172) loaded by `gsd-plan-checker`: the VALIDATION.md gate, automated-verify presence, feedback-latency assessment, sampling continuity, Wave 0 completeness, and the Dimension 8 output table — extracted from the agent to stay under its LARGE size cap. |
| `failing-direction.md` | Check 8f, Stated Failing Direction (#3172), loaded by `gsd-plan-checker`: how to consume the `{FAILING_DIRECTIONS}` probe result, the status/severity table, the sentinel exemption, and the split between deterministic blockers and advisory vacuous-statement warnings. |
### Workflow References
@@ -425,6 +427,7 @@ The `gsd-planner` agent is decomposed into a core agent plus reference modules t
| `planner-interface-context.md` | Interface context rules for executors — how to extract key interfaces/types/exports from existing code and document new interfaces that downstream plans will consume. |
| `planner-load-graph-context.md` | Planner's load_graph_context step: knowledge-graph freshness + dependency-context query via the gsd_run launcher (extracted from gsd-planner.md). |
| `planner-verify-command-grounding.md` | Verify Command Grounding rules (#2401): inherit `prior_verify_commands` verbatim when the story repeats, prefer `npm --prefix <dir> run <script>` over `cd <dir> && npm run <script>`, and ground every authored path. |
| `planner-failing-direction.md` | Stated Failing Direction rules (#3172): every runnable `<automated>` carries a `<fails_when>` sibling naming an observable failure signal, the pairing and placeholder rules, the `MISSING` sentinel exemption, and the authoring test ("if this command were silently doing nothing, what would tell me?"). |
| `skeleton-template.md` | SKELETON.md template emitted for new-project Walking Skeleton (Phase 1 + `--mvp`). |
| `user-story-template.md` | User story format for MVP planning — "As a / I want to / So that" structured fields. |
| `specless-probe-fallback.md` | Spec-less probe fallback protocol — gate (toggle + per-section absence via the shared `spec-section` helper), the deterministic edge probe (mirrors spec-phase 5.5), the in-planner prohibition recall, and the `must_haves` authoring lift; consumed by plan-phase step 7.95 when a phase SPEC omits `## Edge Coverage` / `## Prohibitions` (ADR-857 Phase 6). |
@@ -657,7 +660,7 @@ Full listing: `gsd-core/bin/lib/*.cjs`.
| `validate.cjs` | Pure phase variant normalization helpers (`phaseVariants`, `buildRoadmapPhaseVariants`, `buildNotStartedPhaseVariants`) used by `verify.cjs` for W006/W007 checks; no I/O, no async |
| `verification-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools verification` |
| `verification.cjs` | Verification-status routing — consolidates pass/gaps_found/human_needed status from phase verifier-emitted VERIFICATION.md frontmatter (#651) |
| `verify-command-grounding.cjs` | Verify-command path-resolvability probe (#2401) — pure `extractAutomatedCommands` (`<automated>` blocks + owning task, ReDoS-safe), `resolveVerifyCommandTarget` (grounds a leading `cd <literal>` chain or `npm --prefix <literal>` against the project root; three-state `ok`/`broken`/`unresolvable` plus `not_applicable`/`pending_creation`), `probePhaseVerifyCommands` (per-phase report backing `gsd-tools check verify-command-paths`), and `harvestPriorVerifyCommands` (nearest prior phase's commands, surfaced to the planner ungated by `context_window`). Never executes command text — PLAN.md is model-authored — and never prescribes a replacement path. Compiled from `src/verify-command-grounding.cts` |
| `verify-command-grounding.cjs` | Verify-command path-resolvability probe (#2401) — pure `extractAutomatedCommands` (`<automated>` blocks + owning task, ReDoS-safe), `resolveVerifyCommandTarget` (grounds a leading `cd <literal>` chain or `npm --prefix <literal>` against the project root; three-state `ok`/`broken`/`unresolvable` plus `not_applicable`/`pending_creation`), `probePhaseVerifyCommands` (per-phase report backing `gsd-tools check verify-command-paths`), `harvestPriorVerifyCommands` (nearest prior phase's commands, surfaced to the planner ungated by `context_window`), and the failing-direction probe (#3172) — `extractFailingDirections` (pairs each `<fails_when>` to the nearest preceding `<automated>` in one document-order pass, first-wins), `resolveFailingDirection` (closed 6-atom status `ok`/`missing`/`empty`/`placeholder`/`sentinel`/`orphan`), and `probePhaseFailingDirections` (per-phase report backing `gsd-tools check verify-failure-directions`). Never executes command text — PLAN.md is model-authored — and never prescribes a replacement path or a failure statement. Compiled from `src/verify-command-grounding.cts` |
| `verify-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools verify` |
| `verify.cjs` | Plan structure, phase completeness, reference, commit validation |
| `workflow-fragments.cjs` | In-file `<!-- gsd:section id= when= -->` marker parser/composer for GSD workflow markdown (ADR-1671, #2930) — `parseWorkflowSections` (fence/HTML-comment-aware document partition into explicit/gap sections, fail-closed on malformed/unclosed/nested/duplicate markers or an unknown `when=`), `toFragments` (maps sections to `context-composer.cjs` `verbatim` fragments — non-lossy by construction), and `renderFragments`/`composeWorkflow` (compose-within-budget then join, run BEFORE per-runtime converters so a marker attribute never reaches a path-rewrite regex). `WHEN_VOCABULARY` is a frozen 4-atom applicability set (`always`, `flag:--wave`, `state:gap-closure-phase`, `state:has-prior-phases`); widening it is an ADR amendment, not an organic edit. Compiled from `src/workflow-fragments.cts` |

1
docs/README.md
View File

@@ -26,6 +26,7 @@ Language versions: [English](README.md) · [Português (pt-BR)](pt-BR/README.md)
- [Resolve prohibition findings](how-to/resolve-prohibition-findings.md) — turn the spec phase's surfaced must-NOT constraints into resolved, dismissed, or deferred spec decisions
- [Resolve an unreachable-workflow finding](how-to/resolve-unreachable-workflow-findings.md) — wire or fully sweep a shipped workflow that no command, agent, or skill references
- [Resolve verify-command path findings](how-to/resolve-verify-command-path-findings.md) — fix an `<automated>` verify command whose target directory does not resolve from the executor's cwd
- [State a failing direction](how-to/state-a-failing-direction.md) — say what output constitutes failure for an `<automated>` verify command, and migrate a phase planned before the rule
- [Resolve a contract-drift finding](how-to/resolve-contract-drift-findings.md) — bring an agent's completion contract, read-tag gate, or deleted-file test reference back into agreement with the registry
- [Resolve unreachable-guard findings](how-to/resolve-unreachable-guard-findings.md) — fix shell guards whose fallback arm cannot run, and tell "nothing to report" apart from "could not look"
- [Resolve an ESLint glob-coverage finding](how-to/resolve-eslint-coverage-findings.md) — bring a source file that matches no lint rule under coverage, or record a reasoned exemption

94
docs/how-to/state-a-failing-direction.md Normal file
View File

@@ -0,0 +1,94 @@
# State a failing direction
`/gsd-plan-phase` blocks when a plan's `<automated>` acceptance command does not say what output
constitutes failure. This page is what to do about it — how to write the statement, how to read
the probe's verdicts, and how to migrate a phase planned before the rule existed.
## The rule
Every runnable `<automated>` command needs a `<fails_when>` sibling naming an observable failure
signal:
```xml
<verify>
<automated>npm --prefix apps/api test -- auth.spec.ts</automated>
<fails_when>non-zero exit, or "0 passed" in the summary line</fails_when>
</verify>
```
Within one `<task>`, each `<fails_when>` binds to the nearest **preceding** `<automated>`, and
the first statement after a command is the binding one. Two commands need two statements.
## Write the statement
Ask one question: **if this command were silently doing nothing, what in its output would tell
me?** The answer is the statement. If there is no answer, you do not have an acceptance command
— fix the command rather than inventing a statement for it.
| Good — names a signal | Rejected — restates "failure" |
|---|---|
| `non-zero exit` | `the command fails` |
| `"0 passed" appears in the summary` | `it doesn't work` |
| `the coverage line is absent from stdout` | `an error occurs` |
| `stderr contains "ECONNREFUSED"` | `TBD` |
| `exit code > 0, or fewer than 12 tests report` | `N/A` |
Short is fine — `non-zero exit` is complete. There is no minimum length and no required keyword.
Any characters are safe: `exit code > 0` and `stderr contains "FAIL" && exit != 0` are ordinary
prose here.
## Run the probe yourself
```bash
gsd-tools check verify-failure-directions 3 --raw
```
Same JSON the plan-checker acts on. Each row carries `command`, `statement`, `plan`, `task`,
`status`, and `severity`.
## Read the verdicts
| `status` | `severity` | What happened | What to do |
|---|---|---|---|
| `ok` | `none` | A real statement is bound to this command | nothing |
| `missing` | `blocker` | The command has no `<fails_when>` | add one after the command |
| `empty` | `blocker` | The element is present but blank | fill it in, or delete the command |
| `placeholder` | `blocker` | The whole value is `TBD`, `TODO`, `N/A`, `NA`, `none`, `unknown`, `TBA`, `?`, or `-` | write the real signal; a statement you cannot write is a command you should not ship |
| `orphan` | `warning` | A `<fails_when>` that follows no command | move it after the command it describes |
| `sentinel` | `none` | A Nyquist `MISSING — Wave 0 …` placeholder — not runnable, so exempt | nothing; check 8a/8d own it |
The placeholder match is **whole-value and case-insensitive**. `TBD in the harness output` is real
prose and passes; a bare `TBD` does not.
## Tell "nothing to report" from "could not look"
The top-level `status` is `blocked` when any row is a blocker and `ok` when none are. A third
state matters: `unresolvable` with a populated `readError` means the probe **could not read the
plans** — an unreadable file, or a phase directory that does not resolve. An empty `commands`
list with a non-empty `readError` is not a clean bill of health, and neither the probe nor the
plan-checker will report it as one.
| Top-level `status` | `readError` | Meaning |
|---|---|---|
| `ok` | `null` | Every runnable command has a stated failing direction |
| `blocked` | `null` | At least one blocker — fix the rows above |
| `unresolvable` | populated | Could not look. Check the phase number and the plans' readability |
## Migrate a phase planned before this rule
A phase planned earlier has no `<fails_when>` anywhere, so re-checking it reports one `missing`
blocker per runnable command. Two ways forward:
1. **Add the statements by hand.** Run the probe with `--raw`, and for each `missing` row open
the named `plan` and `task` and add a `<fails_when>` after that command. This preserves the
plan as reviewed.
2. **Re-plan the phase** with `/gsd-plan-phase <N>`, which emits statements from the start. Do
this when the plan needs revising anyway; it discards any hand edits.
Sentinel commands need no migration — they were never runnable.
## Related
- [`gsd-tools check verify-failure-directions`](../COMMANDS.md#gsd-tools-check-verify-failure-directions) — the command reference
- [Resolve verify-command path findings](resolve-verify-command-path-findings.md) — the sibling probe, for a command whose *target* does not resolve
- [Stated Failing Direction](../FEATURES.md#167-stated-failing-direction) — why the check is shaped this way