enhance(#3172): require a stated failing direction for every automated acceptance command (#3825)

* test(#3172): failing-first suite for the stated failing-direction probe

Pins the <fails_when> pairing walk, placeholder denylist, MISSING sentinel
exemption, degraded-read contract, CLI arm and the plan-authoring contract text.
RED by construction: the module exports it requires do not exist yet.
Executed on the remote runner.

* feat(#3172): require a stated failing direction for every automated acceptance command

Every runnable <automated> command now carries a <fails_when> sibling naming
what output constitutes failure. A command with no expressible failure mode is
not an acceptance test: it reads as rigour and is not falsifiable.

- verify-command-grounding gains a failing-direction probe sharing the existing
  <automated> grammar, MISSING sentinel and walk guard rather than copying them
- gsd-tools check verify-failure-directions <N> backs it; plan-phase dispatches
  it and hands the JSON to gsd-plan-checker check 8f
- Dimension 8 detail extracted to references to stay under the agent size cap

Verified on the remote runner.

* fix(#3172): close four review findings in the failing-direction probe

- MISSING_SENTINEL_RE matched an env-var assignment prefix (MISSING=1 cmd), so
  a real command was exempted from the new blocking gate. Tightened the SHARED
  constant rather than adding a second copy.
- Both token regexes scanned to EOF on unclosed openers (O(n^2), 1562ms at 40k).
  Bodies are now non-crossing; 1ms, byte-identical on well-formed input. The
  pre-existing AUTOMATED_BLOCK_RE carried the same defect and is fixed here too.
- probePhaseFailingDirections reported status 'ok' when one plan was unreadable,
  conflating 'could not look' with 'nothing to report'.
- Extracted the phase-resolution block both check arms had copied verbatim.

Also corrects a docs/AGENTS.md dimension list stale since #2401.
Verified on the remote runner.

* fix(#3172): project the planner rule onto the spawn contract, settle emitted bookkeeping

The remote runner refuted the planner-side edit. agents/gsd-planner.md is frozen
under a 49152-LF-char cap asserted by four suites and sat at 49,146 — six chars
of headroom — so the +537 of authoring rule blew it. #3297/#3645 already settled
where such a rule goes: the planner spawn contract in plan-phase.md, beside
<tracked_source_paths>. The agent file is reverted to origin/next verbatim.

- plan-phase.md gains <failing_direction_contract>; tests row 30 now asserts the
  contract there and row 30b guards the freeze in both directions
- plan-phase.md growth acknowledged by APPENDING to the 3409 fragment, per the
  precedent that two ack sources may never name the same path
- install-tree fixtures regenerated for the three new reference files

Verified on the remote runner.

* chore(#3172): backfill PR number into the changeset fragment

pr:0 -> pr:3825 now that the PR exists.

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-24 19:05:11 -04:00
committed by GitHub
parent 7a41248c4f
commit c933184b97
37 changed files with 1537 additions and 85 deletions

View File

@@ -766,6 +766,31 @@ inherited paths: fix a mirror path, never inherit. Submodule files: check
from within the submodule.
</tracked_source_paths>
<failing_direction_contract>
**Stated failing direction (#3172):** Every runnable `<automated>` verify command
you write MUST be followed by a `<fails_when>` sibling naming what output
constitutes failure — an exit code, a string in the output, a missing line. A
command with no expressible failure mode is not an acceptance test.
```xml
<verify>
<automated>npm --prefix apps/api test -- auth.spec.ts</automated>
<fails_when>non-zero exit, or "0 passed" in the summary line</fails_when>
</verify>
```
One statement per runnable command, placed immediately after it: within a task
each `<fails_when>` binds to the nearest preceding `<automated>`, and the first
statement after a command is the binding one. Name an OBSERVABLE signal, never
the word "failure" — `non-zero exit` is complete, `the command fails` is a
restatement. `TBD`/`TODO`/`N/A`/`none`/`unknown`/`?`/`-` are rejected outright as
whole values. The `MISSING — Wave 0 …` sentinel is exempt: it is not runnable, so
it has no failure mode to state. Ask yourself: if this command were silently
doing nothing, what in its output would tell me? If you cannot answer, fix the
command — do not invent a statement for it.
Rules + worked examples: @gsd-core/references/planner-failing-direction.md
</failing_direction_contract>
**Project instructions:** Read ./CLAUDE.md or ./.claude/CLAUDE.md if either exists — follow project-specific guidelines
**Project skills:** Check .claude/skills/ or .agents/skills/ directory (if either exists) — read SKILL.md files, plans should account for project skill rules
@@ -991,13 +1016,17 @@ Display banner:
◆ Spawning plan checker... (runs in a subagent — no output until it returns, ~1–5 min; expected, not a freeze)
```
**Verify-command path probe (#2401).** Before spawning, run the deterministic resolvability
probe and hand its JSON to the checker. It never executes command text and never prescribes a
replacement path — it reports which `<automated>` targets resolve, which do not, and which it
refused to guess at. Handing it over is what stops the checker hand-reasoning the filesystem.
**Verify-command probes (#2401, #3172).** Before spawning, run both deterministic probes and
hand their JSON to the checker. The first resolves each `<automated>` command's target; the
second reports which runnable commands carry a `<fails_when>` statement naming their failure
signal. Neither executes command text, and neither prescribes a replacement — the first reports
which `<automated>` targets resolve, which do not, and which it refused to guess at; the second
reports which commands state a failure signal and never authors one. Handing both over is what
stops the checker hand-reasoning the filesystem or the plans.
```bash
VERIFY_PATHS=$(gsd_run check verify-command-paths "${PHASE}" --raw)
FAILING_DIRECTIONS=$(gsd_run check verify-failure-directions "${PHASE}" --raw)
```
Checker prompt:
@@ -1032,6 +1061,18 @@ replacement path.
```
</verify_command_path_probe>
<failing_direction_probe>
**Deterministic failing-direction probe (#3172)** — already run; do NOT re-derive these verdicts
by re-reading the plans yourself. Act on `severity` per check 8f: `blocker` → BLOCKER,
`warning` → WARNING, `none` → silent. `status: sentinel` is a Wave-0 `MISSING` placeholder and is
not a finding. A non-empty `readError` means the probe could not look — a WARNING, not a pass.
Quote the command that has no stated failure mode; never author the statement for the planner.
```json
{FAILING_DIRECTIONS}
```
</failing_direction_probe>
<review_incorporation_verification>
**If Mode is reviews:** Read REVIEWS.md and verify each current actionable review finding is visible in executable PLAN.md content or explicitly deferred/rejected in the relevant PLAN.md. A finding remains actionable if it requires a concrete plan task, `<action>`, `<acceptance_criteria>`, `<verify>`, `must_haves`, threat-model item, stale-path correction, or execution contract change before /gsd:execute-phase runs.