Files
msd-core/gsd-core/workflows/validate-phase.md
Tom Boucher 1c93df04db fix(#2711): propagate the #2517 omit-on-inherit rule to all 15 unguarded workflows (#2713)
* test(#2711): derive the omit-rule guarded set from the corpus instead of a hand list

The GUARDED array was a Goodhart metric: it reported green across 15
non-compliant workflows for no better reason than that nobody had added them to
it. The guard now derives its set — every workflow emitting a model="{…}"
dispatch site must state the omit-on-inherit/empty rule — and asserts the
derivation is non-empty so a broken scan fails rather than passes.

Rule detection stays a PROPERTY check, not a template match: plan-phase.md and
execute-phase.md state it in different words and both are correct.

RED expected on 15 workflows: audit-milestone, code-review, code-review-fix,
debug, discuss-phase-assumptions, docs-update, map-codebase, new-milestone,
new-project, quick, secure-phase, ui-phase, ui-review, validate-phase,
verify-work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

* fix(#2711): propagate the #2517 omit-on-inherit rule to all 15 unguarded workflows

15 of the 19 model=-dispatching workflows carried no omit-on-inherit/empty
guidance — 43 unguarded dispatch sites. Each would emit model="" whenever the
bound *_model resolved empty, which is the DEFAULT state on non-Claude runtimes:
the installer writes resolve_model_ids:"omit" into ~/.gsd/defaults.json for every
one of them (references/model-profiles.md:101), and resolveModelInternal returns
"" for that case (src/model-resolver.cts:383-386) and "inherit" for opus-tier
agents and the inherit profile (:395). Both 404 on runtimes without native tier
aliases — the failure #2517 documented and fixed in one file.

Each file now carries a `<!-- #2517 model-omit-on-inherit -->` blockquote naming
its own bound placeholders and linking the canonical statement in
references/model-profile-resolution.md, mirroring the `<!-- #2508
runtime-aware-dispatch -->` block already present in all 15. The rule text lives
in the reference; the workflows carry a pointer plus the one-line instruction, so
the next revision edits one file rather than fifteen.

plan-phase.md and execute-phase.md are deliberately untouched — they already
state the rule in their own wording, and the guard checks the property rather
than a template string.

No dispatch site is edited and no placeholder renamed: the #2684 binding guard
reports the same 19 files / 60 placeholders / 0 findings before and after, which
is the independence proof that this change is additive prose only. There is no
Hyrum's-Law routing change to disclose.

Placement is span-aware. An initial pass anchored to the #2508 marker, but in six
files that marker sits INSIDE the Agent(prompt="…") string, so the new paragraph's
literal model= landed in a dispatch call span and tripped the #2284 fail-closed
Hermes projection guard (bin/install.js:3704), refusing the install. Blocks are
now anchored before the opening Agent( of the span owning the first dispatch, and
verified to fall inside no span. gen:golden exits 0 across all 19 runtimes.

Fixes #2711

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

* fix(#2711): cite the issue number in the changeset body and tidy block placement

Review findings from the two orthogonal passes:

- The changeset body ended (#0). Repo convention across every prior fragment
  (e.g. #2617/#2693, #2608, #2605) is that the trailing (#NNN) is the ISSUE
  number, known at authoring time; only the frontmatter pr: field carries the 0
  placeholder pending backfill. (#0) would have rendered a dead link in the
  published release notes.
- new-milestone.md glued the inserted block directly under the preceding
  paragraph with no blank line, inconsistent with the other 14 insertions.
- The derived-guard non-vacuity floor was >=17 against an actual derived count
  of 19, tolerating a silent two-file regression. Tightened to >=19.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

* fix(#2711): reword the omit block so it survives Hermes projection, and exempt quick.md by size

The first block wording regressed two suites on the full matrix (4 failures on
both linux-node22 and linux-node24). gen:golden passing was not sufficient
evidence — it exercises the installer's own fail-closed guard, which is
narrower than the dedicated tests.

1. tests/fix-2284-hermes-agent-delegate-task-projection.test.cjs asserts that
   the INSTALLED code-review-fix.md contains no `model=` anywhere outside a
   string literal — masked whole-file, not merely inside call spans. The block's
   backticked `model=` survived the mask. The assertion is right: on Hermes the
   projection strips the parameter because delegate_task has no per-call model
   at all, so instructing the orchestrator to "omit the model= parameter" is
   advice about a parameter that does not exist there. The block now says "the
   `model` parameter" and carries no bare `model=` token.

2. tests/prompt-injection-scan.security.test.cjs flagged quick.md at 50,164
   normalized chars against a 50,000 prompt-stuffing threshold. quick.md sits
   just under the line on next, so any insertion trips it — the situation
   review.md is already documented for in SIZE_ONLY_WORKFLOWS ("sat at 49,971
   chars — 29 below the threshold — so it was going to trip on whatever was
   added to it next"). quick.md joins it with the same justification. This is a
   size-finding exemption only: the file is still fully injection scanned, and
   every other security check still runs on it.

Because the canonical block can no longer carry a literal `model=`, the guard's
detector now accepts the `<!-- #2517 model-omit-on-inherit -->` marker as the
canonical signal, falling back to the inline-prose property for the four files
that predate it (plan-phase, execute-phase, scan, ship — all four match the
legacy branch). That is strictly stronger than word-proximity matching, and it
keeps the guard a property check rather than a template match.

Verified: derived guard 19/19 with 0 missing; the #2684 binding guard unchanged
at 19 files / 60 placeholders / 0 findings; no inserted block contains a bare
model= token; the masked-projection assertion passes for code-review-fix.md;
gen:golden exits 0 across all 19 runtimes; lint:ci exits 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

* chore(#2711): backfill changeset PR number

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 15:02:26 -04:00

12 KiB
Raw Blame History

Audit Nyquist validation gaps for a completed phase. Generate missing tests. Update VALIDATION.md.

<required_reading> @~/.claude/gsd-core/references/ui-brand.md </required_reading>

<available_agent_types> Valid GSD subagent types (use exact names — do not fall back to 'general-purpose'):

  • gsd-nyquist-auditor — Validates verification coverage </available_agent_types>

0. Initialize

_GSD_SHIM_NAME="gsd-tools.cjs"; _GSD_RUNTIME_ROOT="${RUNTIME_DIR:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}"; GSD_TOOLS="${_GSD_RUNTIME_ROOT}/gsd-core/bin/${_GSD_SHIM_NAME}"; if [ -f "$GSD_TOOLS" ]; then gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${_GSD_RUNTIME_ROOT}/.claude/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${_GSD_RUNTIME_ROOT}/.claude/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${_GSD_RUNTIME_ROOT}/.codex/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${_GSD_RUNTIME_ROOT}/.codex/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif command -v gsd-tools >/dev/null 2>&1; then GSD_TOOLS="$(command -v gsd-tools)"; gsd_run() { "$GSD_TOOLS" "$@"; }; elif [ -f "${CLAUDE_CONFIG_DIR:-$HOME/.claude}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${CLAUDE_CONFIG_DIR:-$HOME/.claude}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${HERMES_HOME:-$HOME/.hermes}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${HERMES_HOME:-$HOME/.hermes}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${CURSOR_CONFIG_DIR:-$HOME/.cursor}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${CURSOR_CONFIG_DIR:-$HOME/.cursor}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${CODEX_HOME:-$HOME/.codex}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${CODEX_HOME:-$HOME/.codex}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${GEMINI_CONFIG_DIR:-$HOME/.gemini}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${GEMINI_CONFIG_DIR:-$HOME/.gemini}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${COPILOT_CONFIG_DIR:-$HOME/.copilot}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${COPILOT_CONFIG_DIR:-$HOME/.copilot}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${WINDSURF_CONFIG_DIR:-$HOME/.codeium/windsurf}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${WINDSURF_CONFIG_DIR:-$HOME/.codeium/windsurf}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${AUGMENT_CONFIG_DIR:-$HOME/.augment}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${AUGMENT_CONFIG_DIR:-$HOME/.augment}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${TRAE_CONFIG_DIR:-$HOME/.trae}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${TRAE_CONFIG_DIR:-$HOME/.trae}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${QWEN_CONFIG_DIR:-$HOME/.qwen}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${QWEN_CONFIG_DIR:-$HOME/.qwen}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${CODEBUDDY_CONFIG_DIR:-$HOME/.codebuddy}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${CODEBUDDY_CONFIG_DIR:-$HOME/.codebuddy}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${CLINE_CONFIG_DIR:-$HOME/.cline}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${CLINE_CONFIG_DIR:-$HOME/.cline}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${GROK_AGENTS_HOME:-$HOME/.agents}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${GROK_AGENTS_HOME:-$HOME/.agents}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${ANTIGRAVITY_CONFIG_DIR:-$HOME/.gemini/antigravity}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${ANTIGRAVITY_CONFIG_DIR:-$HOME/.gemini/antigravity}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${OPENCODE_CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/opencode}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${OPENCODE_CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/opencode}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; elif [ -f "${KILO_CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/kilo}/gsd-core/bin/${_GSD_SHIM_NAME}" ]; then GSD_TOOLS="${KILO_CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/kilo}/gsd-core/bin/${_GSD_SHIM_NAME}"; gsd_run() { node "$GSD_TOOLS" "$@"; }; else echo "ERROR: gsd-tools.cjs not found at $GSD_TOOLS and gsd-tools is not on PATH. Run: npx -y @opengsd/gsd-core@latest --claude --local" >&2; exit 1; fi; if [ -n "${CLAUDE_ENV_FILE:-}" ] && [ -n "${GSD_TOOLS:-}" ]; then printf "export PATH='%s':\"\$PATH\"\n" "${GSD_TOOLS%/*}" >> "$CLAUDE_ENV_FILE" 2>/dev/null || true; fi
RESPONSE_LANGUAGE=$(gsd_run query config-get response_language --default "" 2>/dev/null || echo "")
INIT=$(gsd_run query init.phase-op "${PHASE_ARG}")
if [[ "$INIT" == @file:* ]]; then INIT=$(cat "${INIT#@file:}"); fi
AGENT_SKILLS_AUDITOR=$(gsd_run query agent-skills gsd-nyquist-auditor)

If response_language is set: All user-facing questions, prompts, and explanations in this workflow MUST be presented in {response_language}. Technical terms, code, file paths, and subagent prompts stay in English — only user-facing output is translated.

Parse: phase_dir, phase_number, phase_name, phase_slug, padded_phase.

AUDITOR_MODEL=$(gsd_run query resolve-model gsd-nyquist-auditor --raw)
VERIFY_POST_HOOKS_JSON=$(gsd_run loop render-hooks verify:post --raw)

Resolve active step hooks from VERIFY_POST_HOOKS_JSON where kind == "step" and ref.skill == "validate-phase".

If no active validate-phase step hook exists: exit with "Nyquist validation is disabled. Enable via /gsd:settings."

Display banner: GSD > VALIDATE PHASE {N}: {name}

1. Detect Input State

VALIDATION_FILE=$(ls "${PHASE_DIR}"/*-VALIDATION.md 2>/dev/null | head -1)
SUMMARY_FILES=$(ls "${PHASE_DIR}"/*-SUMMARY.md 2>/dev/null)
  • State A (VALIDATION_FILE non-empty): Audit existing
  • State B (VALIDATION_FILE empty, SUMMARY_FILES non-empty): Reconstruct from artifacts
  • State C (SUMMARY_FILES empty): Exit — "Phase {N} not executed. Run /gsd:execute-phase {N} ${GSD_WS} first."

2. Discovery

2a. Read Phase Artifacts

Read all PLAN and SUMMARY files. Extract: task lists, requirement IDs, key-files changed, verify blocks.

2b. Build Requirement-to-Task Map

Per task: { task_id, plan_id, wave, requirement_ids, has_automated_command }

2c. Detect Test Infrastructure

State A: Parse from existing VALIDATION.md Test Infrastructure table. State B: Filesystem scan:

find . -name "pytest.ini" -o -name "jest.config.*" -o -name "vitest.config.*" -o -name "pyproject.toml" 2>/dev/null | head -10
find . \( -name "*.test.*" -o -name "*.spec.*" -o -name "test_*" \) -not -path "*/node_modules/*" 2>/dev/null | head -40

2d. Cross-Reference

Match each requirement to existing tests by filename, imports, test descriptions. Record: requirement → test_file → status.

3. Gap Analysis

Classify each requirement:

Status Criteria
COVERED Test exists, targets behavior, runs green
PARTIAL Test exists, failing or incomplete
MISSING No test found

Build: { task_id, requirement, gap_type, suggested_test_path, suggested_command }

No gaps → skip to Step 6, set nyquist_compliant: true.

4. Present Gap Plan

Text mode (workflow.text_mode: true in config or --text flag): Set TEXT_MODE=true if --text is present in $ARGUMENTS OR text_mode from init JSON is true. When TEXT_MODE is active, replace every AskUserQuestion call with a plain-text numbered list and ask the user to type their choice number. This is required for non-Claude runtimes (OpenAI Codex, Gemini CLI, etc.) where AskUserQuestion is not available. Call AskUserQuestion with gap table and options:

  1. "Fix all gaps" → Step 5
  2. "Skip — mark manual-only" → add to Manual-Only, Step 6
  3. "Cancel" → exit

5. Spawn gsd-nyquist-auditor

Print: ◆ Spawning nyquist auditor... (runs in a subagent — no output until it returns, ~1–5 min; expected, not a freeze)

Runtime-aware dispatch (#2508 Phase 4). GSD workflows dispatch specialized subagents by role. Before dispatching on a built-in-only runtime (kimi-code — three built-ins only), resolve the role to a built-in via gsd_run query resolve-dispatch-type --requested <role> --raw. On named-dispatch runtimes (Claude/OpenCode/…) the role is returned unchanged; on kimi-code it maps to coder/explore/plan by role-suffix. The persona rides ${AGENT_SKILLS_<ROLE>} (Phase 3) regardless. See @gsd-core/references/runtime-aware-dispatch.md.

Model omission (#2517). Omit the model parameter entirely when the value it would carry (AUDITOR_MODEL) is "inherit" or empty. An empty value 404s on runtimes without native tier aliases — the default on non-Claude runtimes. Omitting it inherits the orchestrator's model. See @gsd-core/references/model-profile-resolution.md.

Agent(
  prompt="Read ~/.claude/agents/gsd-nyquist-auditor.md for instructions.\n\n" +
    "<files_to_read>{PLAN, SUMMARY, impl files, VALIDATION.md}</files_to_read>" +
    "<gaps>{gap list}</gaps>" +
    "<test_infrastructure>{framework, config, commands}</test_infrastructure>" +
    "<constraints>Never modify impl files. Max 3 debug iterations. Escalate impl bugs.</constraints>" +
    "${AGENT_SKILLS_AUDITOR}",
  subagent_type="gsd-nyquist-auditor",
  model="{AUDITOR_MODEL}",
  description="Fill validation gaps for Phase {N}"
)

ORCHESTRATOR RULE — CODEX RUNTIME: After calling Agent() above, stop working on this task immediately. Do not read more files, edit code, or run tests related to this task while the subagent is active. Wait for the subagent to return its result. This prevents duplicate work, conflicting edits, and wasted context. Only resume when the subagent result is available.

Handle return:

  • ## GAPS FILLED → record tests + map updates, Step 6
  • ## PARTIAL → record resolved, move escalated to manual-only, Step 6
  • ## ESCALATE → move all to manual-only, Step 6

6. Generate/Update VALIDATION.md

State B (create):

  1. Read template from ~/.claude/gsd-core/templates/VALIDATION.md
  2. Fill: frontmatter (set status: validated), Test Infrastructure, Per-Task Map, Manual-Only, Sign-Off
  3. Write to ${PHASE_DIR}/${PADDED_PHASE}-VALIDATION.md

State A (update):

  1. Update Per-Task Map statuses, add escalated to Manual-Only, update frontmatter (set status: validated)
  2. Append audit trail:
## Validation Audit {date}
| Metric | Count |
|--------|-------|
| Gaps found | {N} |
| Resolved | {M} |
| Escalated | {K} |

7. Commit

git add {test_files}
git commit -m "test(phase-${PHASE}): add Nyquist validation tests"

gsd_run query commit "docs(phase-${PHASE}): add/update validation strategy" \
  --files "${PHASE_DIR}/${PADDED_PHASE}-VALIDATION.md"

8. Results + Routing

Compliant:

GSD > PHASE {N} IS NYQUIST-COMPLIANT
All requirements have automated verification.
▶ Next: /gsd:audit-milestone ${GSD_WS}

Partial:

GSD > PHASE {N} VALIDATED (PARTIAL)
{M} automated, {K} manual-only.
▶ Retry: /gsd:validate-phase {N} ${GSD_WS}

Display /clear reminder.

<success_criteria>

  • Nyquist config checked (exit if disabled)
  • Input state detected (A/B/C)
  • State C exits cleanly
  • PLAN/SUMMARY files read, requirement map built
  • Test infrastructure detected
  • Gaps classified (COVERED/PARTIAL/MISSING)
  • User gate with gap table
  • Auditor spawned with complete context
  • All three return formats handled
  • VALIDATION.md created or updated
  • Test files committed separately
  • Results with routing presented </success_criteria>