enhance(#3172): require a stated failing direction for every automated acceptance command (#3825)

* test(#3172): failing-first suite for the stated failing-direction probe

Pins the <fails_when> pairing walk, placeholder denylist, MISSING sentinel
exemption, degraded-read contract, CLI arm and the plan-authoring contract text.
RED by construction: the module exports it requires do not exist yet.
Executed on the remote runner.

* feat(#3172): require a stated failing direction for every automated acceptance command

Every runnable <automated> command now carries a <fails_when> sibling naming
what output constitutes failure. A command with no expressible failure mode is
not an acceptance test: it reads as rigour and is not falsifiable.

- verify-command-grounding gains a failing-direction probe sharing the existing
  <automated> grammar, MISSING sentinel and walk guard rather than copying them
- gsd-tools check verify-failure-directions <N> backs it; plan-phase dispatches
  it and hands the JSON to gsd-plan-checker check 8f
- Dimension 8 detail extracted to references to stay under the agent size cap

Verified on the remote runner.

* fix(#3172): close four review findings in the failing-direction probe

- MISSING_SENTINEL_RE matched an env-var assignment prefix (MISSING=1 cmd), so
  a real command was exempted from the new blocking gate. Tightened the SHARED
  constant rather than adding a second copy.
- Both token regexes scanned to EOF on unclosed openers (O(n^2), 1562ms at 40k).
  Bodies are now non-crossing; 1ms, byte-identical on well-formed input. The
  pre-existing AUTOMATED_BLOCK_RE carried the same defect and is fixed here too.
- probePhaseFailingDirections reported status 'ok' when one plan was unreadable,
  conflating 'could not look' with 'nothing to report'.
- Extracted the phase-resolution block both check arms had copied verbatim.

Also corrects a docs/AGENTS.md dimension list stale since #2401.
Verified on the remote runner.

* fix(#3172): project the planner rule onto the spawn contract, settle emitted bookkeeping

The remote runner refuted the planner-side edit. agents/gsd-planner.md is frozen
under a 49152-LF-char cap asserted by four suites and sat at 49,146 — six chars
of headroom — so the +537 of authoring rule blew it. #3297/#3645 already settled
where such a rule goes: the planner spawn contract in plan-phase.md, beside
<tracked_source_paths>. The agent file is reverted to origin/next verbatim.

- plan-phase.md gains <failing_direction_contract>; tests row 30 now asserts the
  contract there and row 30b guards the freeze in both directions
- plan-phase.md growth acknowledged by APPENDING to the 3409 fragment, per the
  precedent that two ack sources may never name the same path
- install-tree fixtures regenerated for the three new reference files

Verified on the remote runner.

* chore(#3172): backfill PR number into the changeset fragment

pr:0 -> pr:3825 now that the PR exists.

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-24 19:05:11 -04:00
committed by GitHub
parent 7a41248c4f
commit c933184b97
37 changed files with 1537 additions and 85 deletions

View File

@@ -332,6 +332,8 @@ Full roster at `gsd-core/references/*.md`. References are shared knowledge docum
| `research-philosophy.md` | Shared research philosophy (training-as-hypothesis, honest reporting, investigation-not-confirmation) injected into researcher agents. |
| `research-verification-protocol.md` | Shared research verification protocol (4 pitfalls + pre-submission checklist) injected into researcher agents. |
| `verify-command-path-resolvability.md` | Verify Command Path Resolvability dimension (#2401) loaded by `gsd-plan-checker`: how to consume the `{VERIFY_PATHS}` probe result (never re-run or hand-reason the filesystem), the severity/reason table, and report-never-prescribe rules. |
| `nyquist-compliance.md` | Dimension 8 checks 8a-8e (#3172) loaded by `gsd-plan-checker`: the VALIDATION.md gate, automated-verify presence, feedback-latency assessment, sampling continuity, Wave 0 completeness, and the Dimension 8 output table — extracted from the agent to stay under its LARGE size cap. |
| `failing-direction.md` | Check 8f, Stated Failing Direction (#3172), loaded by `gsd-plan-checker`: how to consume the `{FAILING_DIRECTIONS}` probe result, the status/severity table, the sentinel exemption, and the split between deterministic blockers and advisory vacuous-statement warnings. |
### Workflow References
@@ -425,6 +427,7 @@ The `gsd-planner` agent is decomposed into a core agent plus reference modules t
| `planner-interface-context.md` | Interface context rules for executors — how to extract key interfaces/types/exports from existing code and document new interfaces that downstream plans will consume. |
| `planner-load-graph-context.md` | Planner's load_graph_context step: knowledge-graph freshness + dependency-context query via the gsd_run launcher (extracted from gsd-planner.md). |
| `planner-verify-command-grounding.md` | Verify Command Grounding rules (#2401): inherit `prior_verify_commands` verbatim when the story repeats, prefer `npm --prefix <dir> run <script>` over `cd <dir> && npm run <script>`, and ground every authored path. |
| `planner-failing-direction.md` | Stated Failing Direction rules (#3172): every runnable `<automated>` carries a `<fails_when>` sibling naming an observable failure signal, the pairing and placeholder rules, the `MISSING` sentinel exemption, and the authoring test ("if this command were silently doing nothing, what would tell me?"). |
| `skeleton-template.md` | SKELETON.md template emitted for new-project Walking Skeleton (Phase 1 + `--mvp`). |
| `user-story-template.md` | User story format for MVP planning — "As a / I want to / So that" structured fields. |
| `specless-probe-fallback.md` | Spec-less probe fallback protocol — gate (toggle + per-section absence via the shared `spec-section` helper), the deterministic edge probe (mirrors spec-phase 5.5), the in-planner prohibition recall, and the `must_haves` authoring lift; consumed by plan-phase step 7.95 when a phase SPEC omits `## Edge Coverage` / `## Prohibitions` (ADR-857 Phase 6). |
@@ -657,7 +660,7 @@ Full listing: `gsd-core/bin/lib/*.cjs`.
| `validate.cjs` | Pure phase variant normalization helpers (`phaseVariants`, `buildRoadmapPhaseVariants`, `buildNotStartedPhaseVariants`) used by `verify.cjs` for W006/W007 checks; no I/O, no async |
| `verification-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools verification` |
| `verification.cjs` | Verification-status routing — consolidates pass/gaps_found/human_needed status from phase verifier-emitted VERIFICATION.md frontmatter (#651) |
| `verify-command-grounding.cjs` | Verify-command path-resolvability probe (#2401) — pure `extractAutomatedCommands` (`<automated>` blocks + owning task, ReDoS-safe), `resolveVerifyCommandTarget` (grounds a leading `cd <literal>` chain or `npm --prefix <literal>` against the project root; three-state `ok`/`broken`/`unresolvable` plus `not_applicable`/`pending_creation`), `probePhaseVerifyCommands` (per-phase report backing `gsd-tools check verify-command-paths`), and `harvestPriorVerifyCommands` (nearest prior phase's commands, surfaced to the planner ungated by `context_window`). Never executes command text — PLAN.md is model-authored — and never prescribes a replacement path. Compiled from `src/verify-command-grounding.cts` |
| `verify-command-grounding.cjs` | Verify-command path-resolvability probe (#2401) — pure `extractAutomatedCommands` (`<automated>` blocks + owning task, ReDoS-safe), `resolveVerifyCommandTarget` (grounds a leading `cd <literal>` chain or `npm --prefix <literal>` against the project root; three-state `ok`/`broken`/`unresolvable` plus `not_applicable`/`pending_creation`), `probePhaseVerifyCommands` (per-phase report backing `gsd-tools check verify-command-paths`), `harvestPriorVerifyCommands` (nearest prior phase's commands, surfaced to the planner ungated by `context_window`), and the failing-direction probe (#3172) — `extractFailingDirections` (pairs each `<fails_when>` to the nearest preceding `<automated>` in one document-order pass, first-wins), `resolveFailingDirection` (closed 6-atom status `ok`/`missing`/`empty`/`placeholder`/`sentinel`/`orphan`), and `probePhaseFailingDirections` (per-phase report backing `gsd-tools check verify-failure-directions`). Never executes command text — PLAN.md is model-authored — and never prescribes a replacement path or a failure statement. Compiled from `src/verify-command-grounding.cts` |
| `verify-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools verify` |
| `verify.cjs` | Plan structure, phase completeness, reference, commit validation |
| `workflow-fragments.cjs` | In-file `<!-- gsd:section id= when= -->` marker parser/composer for GSD workflow markdown (ADR-1671, #2930) — `parseWorkflowSections` (fence/HTML-comment-aware document partition into explicit/gap sections, fail-closed on malformed/unclosed/nested/duplicate markers or an unknown `when=`), `toFragments` (maps sections to `context-composer.cjs` `verbatim` fragments — non-lossy by construction), and `renderFragments`/`composeWorkflow` (compose-within-budget then join, run BEFORE per-runtime converters so a marker attribute never reaches a path-rewrite regex). `WHEN_VOCABULARY` is a frozen 4-atom applicability set (`always`, `flag:--wave`, `state:gap-closure-phase`, `state:has-prior-phases`); widening it is an ADR amendment, not an organic edit. Compiled from `src/workflow-fragments.cts` |