feat: add Nyquist validation layer to plan-phase pipeline

Adds automated feedback architecture research to gsd-phase-researcher, enforces it as Dimension 8 in gsd-plan-checker, and introduces {phase}-VALIDATION.md as the per-phase validation contract. Ensures every phase plan includes automated verify commands before execution begins. Opt-out via workflow.nyquist_validation: false. Closes #122 (partial), related #117 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 13:46:39 +02:00
parent 3cf26d69ee
commit e0f9c738c9
9 changed files with 352 additions and 13 deletions
--- a/agents/gsd-phase-researcher.md
+++ b/agents/gsd-phase-researcher.md
@@ -296,6 +296,37 @@ Verified patterns from official sources:
   - What's unclear: [the gap]
   - Recommendation: [how to handle]

+## Validation Architecture
+
+> Skip this section entirely if workflow.nyquist_validation is false in .planning/config.json
+
+### Test Framework
+| Property | Value |
+|----------|-------|
+| Framework | {framework name + version} |
+| Config file | {path or "none — see Wave 0"} |
+| Quick run command | `{command}` |
+| Full suite command | `{command}` |
+| Estimated runtime | ~{N} seconds |
+
+### Phase Requirements → Test Map
+| Req ID | Behavior | Test Type | Automated Command | File Exists? |
+|--------|----------|-----------|-------------------|-------------|
+| REQ-XX | {behavior description} | unit | `pytest tests/test_{module}.py::test_{name} -x` | ✅ yes / ❌ Wave 0 gap |
+
+### Nyquist Sampling Rate
+- **Minimum sample interval:** After every committed task → run: `{quick run command}`
+- **Full suite trigger:** Before merging final task of any plan wave
+- **Phase-complete gate:** Full suite green before `/gsd:verify-work` runs
+- **Estimated feedback latency per task:** ~{N} seconds
+
+### Wave 0 Gaps (must be created before implementation)
+- [ ] `{tests/test_file.py}` — covers REQ-{XX}
+- [ ] `{tests/conftest.py}` — shared fixtures for phase {N}
+- [ ] Framework install: `{command}` — if no framework detected
+
+*(If no gaps: "None — existing test infrastructure covers all phase requirements")*
+
 ## Sources

 ### Primary (HIGH confidence)
@@ -335,6 +366,8 @@ INIT=$(node ~/.claude/get-shit-done/bin/gsd-tools.cjs init phase-op "${PHASE}")

 Extract from init JSON: `phase_dir`, `padded_phase`, `phase_number`, `commit_docs`.

+Also check Nyquist validation config — read `.planning/config.json` and check if `workflow.nyquist_validation` is `true`. If `true`, include the Validation Architecture section in RESEARCH.md output (scan for test frameworks, map requirements to test types, identify Wave 0 gaps). If `false`, skip the Validation Architecture section entirely and omit it from output.
+
 Then read CONTEXT.md if exists:
 ```bash
 cat "$phase_dir"/*-CONTEXT.md 2>/dev/null
@@ -367,7 +400,33 @@ Based on phase description, identify what needs investigating:

 For each domain: Context7 first → Official docs → WebSearch → Cross-verify. Document findings with confidence levels as you go.

-## Step 4: Quality Check
+## Step 4: Validation Architecture Research (if nyquist_validation enabled)
+
+**Skip this step if** workflow.nyquist_validation is false in config.
+
+This step answers: "How will Claude's executor know, within seconds of committing each task, whether the output is correct?"
+
+### Detect Test Infrastructure
+Scan the codebase for test configuration:
+- Look for test config files: pytest.ini, pyproject.toml, jest.config.*, vitest.config.*, etc.
+- Look for test directories: test/, tests/, __tests__/
+- Look for test files: *.test.*, *.spec.*
+- Check package.json scripts for test commands
+
+### Map Requirements to Tests
+For each requirement in <phase_requirements>:
+- Identify the behavior to verify
+- Determine test type: unit / integration / contract / smoke / e2e / manual-only
+- Specify the automated command to run that test in < 30 seconds
+- Flag if only verifiable manually (justify why)
+
+### Identify Wave 0 Gaps
+List test files, fixtures, or utilities that must be created BEFORE implementation:
+- Missing test files for phase requirements
+- Missing test framework configuration
+- Missing shared fixtures or test utilities
+
+## Step 5: Quality Check

 - [ ] All domains investigated
 - [ ] Negative claims verified
@@ -375,7 +434,7 @@ For each domain: Context7 first → Official docs → WebSearch → Cross-verify
 - [ ] Confidence levels assigned honestly
 - [ ] "What might I have missed?" review

-## Step 5: Write RESEARCH.md
+## Step 6: Write RESEARCH.md

 **ALWAYS use Write tool to persist to disk** — mandatory regardless of `commit_docs` setting.

@@ -414,13 +473,13 @@ Write to: `$PHASE_DIR/$PADDED_PHASE-RESEARCH.md`

 ⚠️ `commit_docs` controls git only, NOT file writing. Always write first.

-## Step 6: Commit Research (optional)
+## Step 7: Commit Research (optional)

 ```bash
 node ~/.claude/get-shit-done/bin/gsd-tools.cjs commit "docs($PHASE): research phase domain" --files "$PHASE_DIR/$PADDED_PHASE-RESEARCH.md"
 ```

-## Step 7: Return Structured Result
+## Step 8: Return Structured Result

 </execution_flow>

--- a/agents/gsd-plan-checker.md
+++ b/agents/gsd-plan-checker.md
@@ -312,6 +312,105 @@ issue:
  fix_hint: "Remove search task - belongs in future phase per user decision"
 ```

+## Dimension 8: Nyquist Compliance
+
+<dimension_8_skip_condition>
+Skip this entire dimension if:
+- workflow.nyquist_validation is false in .planning/config.json
+- The phase being checked has no RESEARCH.md (researcher was skipped)
+- The RESEARCH.md has no "Validation Architecture" section (researcher ran without Nyquist)
+
+If skipped, output: "Dimension 8: SKIPPED (nyquist_validation disabled or not applicable)"
+</dimension_8_skip_condition>
+
+<dimension_8_context>
+This dimension enforces the Nyquist-Shannon Sampling Theorem for AI code generation:
+if Claude's executor produces output at high frequency (one task per commit), feedback
+must run at equally high frequency. A plan that produces code without pre-defined
+automated verification is under-sampled — errors will be statistically missed.
+
+The gsd-phase-researcher already determined WHAT to test. This dimension verifies
+that the planner correctly incorporated that information into the actual task plans.
+</dimension_8_context>
+
+### Check 8a — Automated Verify Presence
+
+For EACH `<task>` element in EACH plan file for this phase:
+
+1. Does `<verify>` contain an `<automated>` command (or structured equivalent)?
+2. If `<automated>` is absent or empty:
+   - Is there a Wave 0 dependency that creates the test before this task runs?
+   - If no Wave 0 dependency exists → **BLOCKING FAIL**
+3. If `<automated>` says "MISSING":
+   - A Wave 0 task must reference the same test file path → verify this link is present
+   - If the link is broken → **BLOCKING FAIL**
+
+**PASS criteria:** Every task either has an `<automated>` verify command, OR explicitly
+references a Wave 0 task that creates the test scaffold it depends on.
+
+### Check 8b — Feedback Latency Assessment
+
+Review each `<automated>` command in the plans:
+
+1. Does the command appear to be a full E2E suite (playwright, cypress, selenium)?
+   - If yes: **WARNING** (non-blocking) — suggest adding a faster unit/smoke test as primary verify
+2. Does the command include `--watchAll` or equivalent watch mode flags?
+   - If yes: **BLOCKING FAIL** — watch mode is not suitable for CI/post-commit sampling
+3. Does the command include `sleep`, `wait`, or arbitrary delays > 30 seconds?
+   - If yes: **WARNING** — flag as latency risk
+
+### Check 8c — Sampling Continuity
+
+Review ALL tasks across ALL plans for this phase in wave order:
+
+1. Map each task to its wave number
+2. For each consecutive window of 3 tasks in the same wave: at least 2 must have
+   an `<automated>` verify command (not just Wave 0 scaffolding)
+3. If any 3 consecutive implementation tasks all lack automated verify: **BLOCKING FAIL**
+
+### Check 8d — Wave 0 Completeness
+
+If any plan contains `<automated>MISSING</automated>` or references Wave 0:
+
+1. Does a Wave 0 task exist for every MISSING reference?
+2. Does the Wave 0 task's `<files>` match the path referenced in the MISSING automated command?
+3. Is the Wave 0 task in a plan that executes BEFORE the dependent task?
+
+**FAIL condition:** Any MISSING automated verify without a matching Wave 0 task.
+
+### Dimension 8 Output Block
+
+Include this block in the plan-checker report:
+
+```
+## Dimension 8: Nyquist Compliance
+
+### Automated Verify Coverage
+| Task | Plan | Wave | Automated Command | Latency | Status |
+|------|------|------|-------------------|---------|--------|
+| {task name} | {plan} | {wave} | `{command}` | ~{N}s | ✅ PASS / ❌ FAIL |
+
+### Sampling Continuity Check
+Wave {N}: {X}/{Y} tasks verified → ✅ PASS / ❌ FAIL
+
+### Wave 0 Completeness
+- {test file} → Wave 0 task present ✅ / MISSING ❌
+
+### Overall Nyquist Status: ✅ PASS / ❌ FAIL
+
+### Revision Instructions (if FAIL)
+Return to planner with the following required changes:
+{list of specific fixes needed}
+```
+
+### Revision Loop Behavior
+
+If Dimension 8 FAILS:
+- Return to `gsd-planner` with the specific revision instructions above
+- The planner must address ALL failing checks before returning
+- This follows the same loop behavior as existing dimensions
+- Maximum 3 revision loops for Dimension 8 before escalating to user
+
 </verification_dimensions>

 <verification_process>
@@ -329,6 +428,8 @@ Orchestrator provides CONTEXT.md content in the verification prompt. If provided

 ```bash
 ls "$phase_dir"/*-PLAN.md 2>/dev/null
+# Read research for Nyquist validation data
+cat "$phase_dir"/*-RESEARCH.md 2>/dev/null
 node ~/.claude/get-shit-done/bin/gsd-tools.cjs roadmap get-phase "$phase_number"
 ls "$phase_dir"/*-BRIEF.md 2>/dev/null
 ```
--- a/agents/gsd-planner.md
+++ b/agents/gsd-planner.md
@@ -157,9 +157,21 @@ Every task has four required fields:
 - Good: "Create POST endpoint accepting {email, password}, validates using bcrypt against User table, returns JWT in httpOnly cookie with 15-min expiry. Use jose library (not jsonwebtoken - CommonJS issues with Edge runtime)."
 - Bad: "Add authentication", "Make login work"

-**<verify>:** How to prove the task is complete.
- Good: `npm test` passes, `curl -X POST /api/auth/login` returns 200 with Set-Cookie header
- Bad: "It works", "Looks good"
+**<verify>:** How to prove the task is complete. Supports structured format:
+
+```xml
+<verify>
+  <automated>pytest tests/test_module.py::test_behavior -x</automated>
+  <manual>Optional: human-readable description of what to check</manual>
+  <sampling_rate>run after this task commits, before next task begins</sampling_rate>
+</verify>
+```
+
+- Good: Specific automated command that runs in < 60 seconds
+- Bad: "It works", "Looks good", manual-only verification
+- Simple format also accepted: `npm test` passes, `curl -X POST /api/auth/login` returns 200 with Set-Cookie header
+
+**Nyquist Rule:** Every `<verify>` must include an `<automated>` command. If no test exists yet for this behavior, set `<automated>MISSING — Wave 0 must create {test_file} first</automated>` and create a Wave 0 task that generates the test scaffold.

 **<done>:** Acceptance criteria - measurable state of completion.
 - Good: "Valid credentials return 200 + JWT cookie, invalid credentials return 401"