enhance(#3172): require a stated failing direction for every automated acceptance command (#3825)

* test(#3172): failing-first suite for the stated failing-direction probe

Pins the <fails_when> pairing walk, placeholder denylist, MISSING sentinel
exemption, degraded-read contract, CLI arm and the plan-authoring contract text.
RED by construction: the module exports it requires do not exist yet.
Executed on the remote runner.

* feat(#3172): require a stated failing direction for every automated acceptance command

Every runnable <automated> command now carries a <fails_when> sibling naming
what output constitutes failure. A command with no expressible failure mode is
not an acceptance test: it reads as rigour and is not falsifiable.

- verify-command-grounding gains a failing-direction probe sharing the existing
  <automated> grammar, MISSING sentinel and walk guard rather than copying them
- gsd-tools check verify-failure-directions <N> backs it; plan-phase dispatches
  it and hands the JSON to gsd-plan-checker check 8f
- Dimension 8 detail extracted to references to stay under the agent size cap

Verified on the remote runner.

* fix(#3172): close four review findings in the failing-direction probe

- MISSING_SENTINEL_RE matched an env-var assignment prefix (MISSING=1 cmd), so
  a real command was exempted from the new blocking gate. Tightened the SHARED
  constant rather than adding a second copy.
- Both token regexes scanned to EOF on unclosed openers (O(n^2), 1562ms at 40k).
  Bodies are now non-crossing; 1ms, byte-identical on well-formed input. The
  pre-existing AUTOMATED_BLOCK_RE carried the same defect and is fixed here too.
- probePhaseFailingDirections reported status 'ok' when one plan was unreadable,
  conflating 'could not look' with 'nothing to report'.
- Extracted the phase-resolution block both check arms had copied verbatim.

Also corrects a docs/AGENTS.md dimension list stale since #2401.
Verified on the remote runner.

* fix(#3172): project the planner rule onto the spawn contract, settle emitted bookkeeping

The remote runner refuted the planner-side edit. agents/gsd-planner.md is frozen
under a 49152-LF-char cap asserted by four suites and sat at 49,146 — six chars
of headroom — so the +537 of authoring rule blew it. #3297/#3645 already settled
where such a rule goes: the planner spawn contract in plan-phase.md, beside
<tracked_source_paths>. The agent file is reverted to origin/next verbatim.

- plan-phase.md gains <failing_direction_contract>; tests row 30 now asserts the
  contract there and row 30b guards the freeze in both directions
- plan-phase.md growth acknowledged by APPENDING to the 3409 fragment, per the
  precedent that two ack sources may never name the same path
- install-tree fixtures regenerated for the three new reference files

Verified on the remote runner.

* chore(#3172): backfill PR number into the changeset fragment

pr:0 -> pr:3825 now that the PR exists.

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-24 19:05:11 -04:00
committed by GitHub
parent 7a41248c4f
commit c933184b97
37 changed files with 1537 additions and 85 deletions

View File

@@ -497,61 +497,16 @@ issue:
## Dimension 8: Nyquist Compliance
Skip if: `workflow.nyquist_validation` is explicitly set to `false` in config.json (absent key = enabled), phase has no RESEARCH.md, or RESEARCH.md has no "Validation Architecture" section. Output: "Dimension 8: SKIPPED (nyquist_validation disabled or not applicable)"
**Question:** Is every task's completion decided by an automated check that can actually fail?
### Check 8e — VALIDATION.md Existence (Gate)
Checks 8a-8e (presence, latency, sampling continuity, Wave 0 completeness, VALIDATION.md gate),
their skip condition and the Dimension 8 output table: @gsd-core/references/nyquist-compliance.md
Before running checks 8a-8d, verify VALIDATION.md exists:
### Check 8f - Stated Failing Direction (#3172)
```bash
ls "${PHASE_DIR}"/*-VALIDATION.md 2>/dev/null
```
**If missing:** **BLOCKING FAIL** — "VALIDATION.md not found for phase {N}. Re-run `/gsd:plan-phase {N} --research` to regenerate."
Skip checks 8a-8d entirely. Report Dimension 8 as FAIL with this single issue.
**If exists:** Proceed to checks 8a-8d.
### Check 8a — Automated Verify Presence
For each `<task>` in each plan:
- `<verify>` must contain `<automated>` command, OR a Wave 0 dependency that creates the test first
- If `<automated>` is absent with no Wave 0 dependency → **BLOCKING FAIL**
- If `<automated>` says "MISSING", a Wave 0 task must reference the same test file path → **BLOCKING FAIL** if link broken
### Check 8b — Feedback Latency Assessment
For each `<automated>` command:
- Full E2E suite (playwright, cypress, selenium) → **WARNING** — suggest faster unit/smoke test
- Watch mode flags (`--watchAll`) → **BLOCKING FAIL**
- Delays > 30 seconds → **WARNING**
### Check 8c — Sampling Continuity
Map tasks to waves. Per wave, any consecutive window of 3 implementation tasks must have ≥2 with `<automated>` verify. 3 consecutive without → **BLOCKING FAIL**.
### Check 8d — Wave 0 Completeness
For each `<automated>MISSING</automated>` reference:
- Wave 0 task must exist with matching `<files>` path
- Wave 0 plan must execute before dependent task
- Missing match → **BLOCKING FAIL**
### Dimension 8 Output
```
## Dimension 8: Nyquist Compliance
| Task | Plan | Wave | Automated Command | Status |
|------|------|------|-------------------|--------|
| {task} | {plan} | {wave} | `{command}` | ✅ / ❌ |
Sampling: Wave {N}: {X}/{Y} verified → ✅ / ❌
Wave 0: {test file} → ✅ present / ❌ MISSING
Overall: ✅ PASS / ❌ FAIL
```
If FAIL: return to planner with specific fixes. Same revision loop as other dimensions (max 3 loops).
Each runnable `<automated>` command needs a `<fails_when>` sibling naming what output constitutes
failure. Consume the supplied `{FAILING_DIRECTIONS}` probe, never re-derive it:
@gsd-core/references/failing-direction.md
## Dimension 9: Cross-Plan Data Contracts