Files
msd-core/gsd-core/workflows/add-tests.md
Tom Boucher fb2d122d7f feat(#3841): assert gsd-tools identity on every state-mutating verb (#3848)
* feat(#3841): assert gsd-tools identity before any state-mutating verb

only this package publishes. The path-based branches — a project-local install,
a runtime config directory — had no such guarantee; they trusted their
configured location. This closes them.

Mechanism: once resolution finishes, and before any verb runs, the preamble
probes the tool it picked with `runtime-identity --raw` and matches the answer
with a shell `case` pattern ANCHORED to the start of the compact payload
(`{"packageName":"@opengsd/gsd-core"`). An unanchored substring match accepts
the decoy `{"packageName":"get-shit-done-cc","note":"@opengsd/gsd-core"}`, which
any colliding package could publish. The outcome is exported as the two-valued
`GSD_IDENTITY_STATUS` (`ok`/`unverified`), so the gate is asserted on a VALUE
rather than on warning prose. Rollout is warn-then-fail per the #3146 ruling:
`unverified` prints one line naming BOTH causes and continues, because
`no_identity_verb` cannot tell a foreign package from an `@opengsd/gsd-core`
older than the verb, and at rollout the old-version case is the common one.

The blocker was byte budget, not design. The preamble is inlined into 112
shipped files and several sat within single-digit bytes of frozen ceilings
(`gsd-verifier.md` 16 bytes, `gsd-executor.md` 33, `execute-phase.md` 234); a
first attempt broke five of them. What made room was collapsing the resolver's
twenty near-identical `elif [ -f … ]` arms into one candidate-list helper
(`_gsd_at`), which buys far more than the assertion costs. The preamble is now
2,624 bytes against 4,500 — a net 1,876 bytes SMALLER per inlined file, so every
capped file moved away from its ceiling rather than toward it. No cap raised, no
size-budget exception added, no override token emitted.

Resolution order, every runtime-home probe, the `unset -f gsd_run` re-source
fix, the fail-closed `exit 1`, and the `CLAUDE_ENV_FILE` persistence are all
preserved byte-for-byte in substring terms; the snippet still begins with
`_GSD_SHIM_NAME=` and still ends with `fi`, which the parity extractors anchor
on. `gsd-core/references/gsd-run-resolver.md` is re-synced byte-equal.

Also fixes two stale claims found in passing: CONTEXT.md and FEATURES.md both
described an `[ -x ]` guard as the load-bearing re-source defense. That guard
was tried and REMOVED in #3831 — it rejected the bare function name, fell
through every branch, and hit `exit 1`, which kills a sourced caller's shell.
`unset -f gsd_run` is the actual mechanism.

Refs #3841

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3841): pair the anchor's brace by requiring a closed identity payload

The matrix went red on `tests/new-project-mvp-prompt.test.cjs` — "new-project.md
has unbalanced braces: net depth 2" — plus a knock-on report from its parent
`bug #1516` describe, which is the same failure counted once at the child and
once at the block.

Root cause: that guard (:182-189, mirroring #3784 bd53925f) walks characters and
increments on `{`, decrements on `}`, with no awareness of shell quoting. It
scans `new-project.md` PLUS every `new-project/steps/*.md`, and both
`new-project.md` and `steps/auto-mode-config.md` carry one inlined preamble copy
— hence net 2 from a snippet that was off by exactly one. The unpaired brace was
the `{` inside the single-quoted `case` pattern of the identity anchor, which is
correct shell and invisible to a text scanner.

Fix in the snippet, not the guard. The pattern now anchors at BOTH ends:
`'{"packageName":"@opengsd/gsd-core"'*'}'`. That balances 51/51 with a brace that
does real work rather than a cosmetic pair — a truncated payload whose prefix
matches now fails too, where before it verified. Safe for any future additive
field: a JSON object's own closing brace is always the last character, whatever
type the last value has, which is pinned by two negative-space tests (a nested
object and an array-valued last key must both still verify). Cost: +3 bytes,
against the 1,873 the resolver fold already gave back.

The alternative considered and rejected was dropping the literal `{` for a `?`
glob. It balances too, but weakens the anchor from "must be an opening brace" to
"must be any one character", and the anchor is the entire point.

Two guards added so this cannot recur silently:
- runtime-launcher-parity (F0) pins brace balance at the SNIPPET, so the next
  edit to that pattern fails on the file it broke instead of surfacing three
  files downstream in a test whose name mentions neither the launcher nor this
  issue. It also asserts depth never goes negative, since a `}` preceding its
  `{` nets to zero while being unbalanced at every prefix.
- runtime-identity gains behavioral truncated-payload and trailing-garbage
  fixtures, so the added `}` is proven load-bearing rather than merely present.

Verified: snippet 51/51 braces; new-project combined net depth 0; the seven
other preamble-bearing files with nonzero depth are unchanged from merged next
(their own prose, not the preamble, and not in any guard's scan set); all 112
inlined copies and the resolver reference re-synced byte-equal; sync:launcher
idempotent on the second run.

Refs #3841

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#3841): backfill changeset PR number

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 01:05:53 -04:00

14 KiB

Generate unit and E2E tests for a completed phase based on its SUMMARY.md, CONTEXT.md, and implementation. Classifies each changed file into TDD (unit), E2E (browser), or Skip categories, presents a test plan for user approval, then generates tests following RED-GREEN conventions.

Users currently hand-craft /gsd:quick prompts for test generation after each phase. This workflow standardizes the process with proper classification, quality gates, and gap reporting.

<required_reading> Read all files referenced by the invoking prompt's execution_context before starting. </required_reading>

Parse `$ARGUMENTS` for: - Phase number (integer, decimal, or letter-suffix) → store as `$PHASE_ARG` - Remaining text after phase number → store as `$EXTRA_INSTRUCTIONS` (optional)

Example: /gsd:add-tests 12 focus on edge cases → $PHASE_ARG=12, $EXTRA_INSTRUCTIONS="focus on edge cases"

If no phase argument provided:

ERROR: Phase number required
Usage: /gsd:add-tests <phase> [additional instructions]
Example: /gsd:add-tests 12
Example: /gsd:add-tests 12 focus on edge cases in the pricing module

Exit.

Load phase operation context:
_GSD_SHIM_NAME="gsd-tools.cjs"; _GSD_RUNTIME_ROOT="${RUNTIME_DIR:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}"; GSD_TOOLS="${_GSD_RUNTIME_ROOT}/gsd-core/bin/${_GSD_SHIM_NAME}"; _gsd_at() { for _p; do if [ -f "$_p" ]; then GSD_TOOLS="$_p"; return 0; fi; done; return 1; }; if _gsd_at "${_GSD_RUNTIME_ROOT}/gsd-core/bin/${_GSD_SHIM_NAME}" "${_GSD_RUNTIME_ROOT}/.claude/gsd-core/bin/${_GSD_SHIM_NAME}" "${_GSD_RUNTIME_ROOT}/.codex/gsd-core/bin/${_GSD_SHIM_NAME}"; then gsd_run() { node "$GSD_TOOLS" "$@"; }; elif unset -f gsd_run; _G="$(command -v gsd_run)"; then GSD_TOOLS="$_G"; gsd_run() { "$GSD_TOOLS" "$@"; }; elif _gsd_at "${CLAUDE_CONFIG_DIR:-$HOME/.claude}/gsd-core/bin/${_GSD_SHIM_NAME}" "${HERMES_HOME:-$HOME/.hermes}/gsd-core/bin/${_GSD_SHIM_NAME}" "${CURSOR_CONFIG_DIR:-$HOME/.cursor}/gsd-core/bin/${_GSD_SHIM_NAME}" "${CODEX_HOME:-$HOME/.codex}/gsd-core/bin/${_GSD_SHIM_NAME}" "${GEMINI_CONFIG_DIR:-$HOME/.gemini}/gsd-core/bin/${_GSD_SHIM_NAME}" "${COPILOT_CONFIG_DIR:-$HOME/.copilot}/gsd-core/bin/${_GSD_SHIM_NAME}" "${WINDSURF_CONFIG_DIR:-$HOME/.codeium/windsurf}/gsd-core/bin/${_GSD_SHIM_NAME}" "${AUGMENT_CONFIG_DIR:-$HOME/.augment}/gsd-core/bin/${_GSD_SHIM_NAME}" "${TRAE_CONFIG_DIR:-$HOME/.trae}/gsd-core/bin/${_GSD_SHIM_NAME}" "${QWEN_CONFIG_DIR:-$HOME/.qwen}/gsd-core/bin/${_GSD_SHIM_NAME}" "${CODEBUDDY_CONFIG_DIR:-$HOME/.codebuddy}/gsd-core/bin/${_GSD_SHIM_NAME}" "${CLINE_CONFIG_DIR:-$HOME/.cline}/gsd-core/bin/${_GSD_SHIM_NAME}" "${GROK_AGENTS_HOME:-$HOME/.agents}/gsd-core/bin/${_GSD_SHIM_NAME}" "${ANTIGRAVITY_CONFIG_DIR:-$HOME/.gemini/antigravity}/gsd-core/bin/${_GSD_SHIM_NAME}" "${OPENCODE_CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/opencode}/gsd-core/bin/${_GSD_SHIM_NAME}" "${KILO_CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/kilo}/gsd-core/bin/${_GSD_SHIM_NAME}"; then gsd_run() { node "$GSD_TOOLS" "$@"; }; else echo "ERROR: gsd-tools.cjs not found at $GSD_TOOLS and gsd_run is not on PATH. Run: npx -y @opengsd/gsd-core@latest --claude --local" >&2; exit 1; fi; GSD_IDENTITY_STATUS=unverified; case "$(gsd_run runtime-identity --raw 2>/dev/null || true)" in '{"packageName":"@opengsd/gsd-core"'*'}') GSD_IDENTITY_STATUS=ok;; esac; export GSD_IDENTITY_STATUS; [ "$GSD_IDENTITY_STATUS" = ok ] || echo "WARNING: \"$GSD_TOOLS\" did not prove it is @opengsd/gsd-core - it is either a different package or an @opengsd/gsd-core older than the runtime-identity verb. See docs/how-to/diagnose-a-foreign-gsd-tools.md" >&2; if [ -n "${CLAUDE_ENV_FILE:-}" ] && [ -n "${GSD_TOOLS:-}" ]; then printf "export PATH='%s':\"\$PATH\"\n" "${GSD_TOOLS%/*}" >> "$CLAUDE_ENV_FILE" 2>/dev/null || true; fi
INIT=$(gsd_run query init.phase-op "${PHASE_ARG}")
if [[ "$INIT" == @file:* ]]; then INIT=$(cat "${INIT#@file:}"); fi

Extract from init JSON: phase_dir, phase_number, phase_name, response_language.

If response_language is set: All user-facing questions, prompts, and explanations in this workflow MUST be presented in {response_language}. Technical terms, code, file paths, and subagent prompts stay in English — only user-facing output is translated.

Verify the phase directory exists. If not:

ERROR: Phase directory not found for phase ${PHASE_ARG}
Ensure the phase exists in .planning/phases/

Exit.

Read the phase artifacts (in order of priority):

  1. ${phase_dir}/*-SUMMARY.md — what was implemented, files changed
  2. ${phase_dir}/CONTEXT.md — acceptance criteria, decisions
  3. ${phase_dir}/*-VERIFICATION.md — user-verified scenarios (if UAT was done)

If no SUMMARY.md exists:

ERROR: No SUMMARY.md found for phase ${PHASE_ARG}
This command works on completed phases. Run /gsd:execute-phase first.

Exit.

Present banner:

### GSD ► ADD TESTS — Phase ${phase_number}: ${phase_name}
Extract the list of files modified by the phase from SUMMARY.md ("Files Changed" or equivalent section).

For each file, classify into one of three categories:

Category Criteria Test Type
TDD Pure functions where expect(fn(input)).toBe(output) is writable Unit tests
E2E UI behavior verifiable by browser automation Playwright/E2E tests
Skip Not meaningfully testable or already covered None

TDD classification — apply when:

  • Business logic: calculations, pricing, tax rules, validation
  • Data transformations: mapping, filtering, aggregation, formatting
  • Parsers: CSV, JSON, XML, custom format parsing
  • Validators: input validation, schema validation, business rules
  • State machines: status transitions, workflow steps
  • Utilities: string manipulation, date handling, number formatting

E2E classification — apply when:

  • Keyboard shortcuts: key bindings, modifier keys, chord sequences
  • Navigation: page transitions, routing, breadcrumbs, back/forward
  • Form interactions: submit, validation errors, field focus, autocomplete
  • Selection: row selection, multi-select, shift-click ranges
  • Drag and drop: reordering, moving between containers
  • Modal dialogs: open, close, confirm, cancel
  • Data grids: sorting, filtering, inline editing, column resize

Skip classification — apply when:

  • UI layout/styling: CSS classes, visual appearance, responsive breakpoints
  • Configuration: config files, environment variables, feature flags
  • Glue code: dependency injection setup, middleware registration, routing tables
  • Migrations: database migrations, schema changes
  • Simple CRUD: basic create/read/update/delete with no business logic
  • Type definitions: records, DTOs, interfaces with no logic

Read each file to verify classification. Don't classify based on filename alone.

Present the classification to the user for confirmation before proceeding:

Text mode (workflow.text_mode: true in config or --text flag): Set TEXT_MODE=true if --text is present in $ARGUMENTS OR text_mode from init JSON is true. When TEXT_MODE is active, replace every AskUserQuestion call with a plain-text numbered list and ask the user to type their choice number. This is required for non-Claude runtimes (OpenAI Codex, Gemini CLI, etc.) where AskUserQuestion is not available.

AskUserQuestion(
  header: "Test Classification",
  question: |
    ## Files classified for testing

    ### TDD (Unit Tests) — {N} files
    {list of files with brief reason}

    ### E2E (Browser Tests) — {M} files
    {list of files with brief reason}

    ### Skip — {K} files
    {list of files with brief reason}

    {if $EXTRA_INSTRUCTIONS: "Additional instructions: ${EXTRA_INSTRUCTIONS}"}

    How would you like to proceed?
  options:
    - "Approve and generate test plan"
    - "Adjust classification (I'll specify changes)"
    - "Cancel"
)

If user selects "Adjust classification": apply their changes and re-present. If user selects "Cancel": exit gracefully.

Before generating the test plan, discover the project's existing test structure:
# Find existing test directories
find . -type d -name "*test*" -o -name "*spec*" -o -name "*__tests__*" 2>/dev/null | head -20
# Find existing test files for convention matching
find . -type f \( -name "*.test.*" -o -name "*.spec.*" -o -name "*Tests.fs" -o -name "*Test.fs" \) 2>/dev/null | head -20
# Check for test runners
ls package.json *.sln 2>/dev/null || true

Identify:

  • Test directory structure (where unit tests live, where E2E tests live)
  • Naming conventions (.test.ts, .spec.ts, *Tests.fs, etc.)
  • Test runner commands (how to execute unit tests, how to execute E2E tests)
  • Test framework (xUnit, NUnit, Jest, Playwright, etc.)

If test structure is ambiguous, ask the user:

AskUserQuestion(
  header: "Test Structure",
  question: "I found multiple test locations. Where should I create tests?",
  options: [list discovered locations]
)
For each approved file, create a detailed test plan.

For TDD files, plan tests following RED-GREEN-REFACTOR:

  1. Identify testable functions/methods in the file
  2. For each function: list input scenarios, expected outputs, edge cases
  3. Note: since code already exists, tests may pass immediately — that's OK, but verify they test the RIGHT behavior

For E2E files, plan tests following RED-GREEN gates:

  1. Identify user scenarios from CONTEXT.md/VERIFICATION.md
  2. For each scenario: describe the user action, expected outcome, assertions
  3. Note: RED gate means confirming the test would fail if the feature were broken

Present the complete test plan:

AskUserQuestion(
  header: "Test Plan",
  question: |
    ## Test Generation Plan

    ### Unit Tests ({N} tests across {M} files)
    {for each file: test file path, list of test cases}

    ### E2E Tests ({P} tests across {Q} files)
    {for each file: test file path, list of test scenarios}

    ### Test Commands
    - Unit: {discovered test command}
    - E2E: {discovered e2e command}

    Ready to generate?
  options:
    - "Generate all"
    - "Cherry-pick (I'll specify which)"
    - "Adjust plan"
)

If "Cherry-pick": ask user which tests to include. If "Adjust plan": apply changes and re-present.

For each approved TDD test:
  1. Create test file following discovered project conventions (directory, naming, imports)

  2. Write test with clear arrange/act/assert structure:

    // Arrange — set up inputs and expected outputs
    // Act — call the function under test
    // Assert — verify the output matches expectations
    
  3. Run the test:

    {discovered test command}
    
  4. Evaluate result:

    • Test passes: Good — the implementation satisfies the test. Verify the test checks meaningful behavior (not just that it compiles).
    • Test fails with assertion error: This may be a genuine bug discovered by the test. Flag it:
      ⚠️ Potential bug found: {test name}
      Expected: {expected}
      Actual: {actual}
      File: {implementation file}
      
      Do NOT fix the implementation — this is a test-generation command, not a fix command. Record the finding.
    • Test fails with error (import, syntax, etc.): This is a test error. Fix the test and re-run.
For each approved E2E test:
  1. Check for existing tests covering the same scenario:

    grep -r "{scenario keyword}" {e2e test directory} 2>/dev/null || true
    

    If found, extend rather than duplicate.

  2. Create test file targeting the user scenario from CONTEXT.md/VERIFICATION.md

  3. Run the E2E test:

    {discovered e2e command}
    
  4. Evaluate result:

    • GREEN (passes): Record success
    • RED (fails): Determine if it's a test issue or a genuine application bug. Flag bugs:
      ⚠️ E2E failure: {test name}
      Scenario: {description}
      Error: {error message}
      
    • Cannot run: Report blocker. Do NOT mark as complete.
      🛑 E2E blocker: {reason tests cannot run}
      

No-skip rule: If E2E tests cannot execute (missing dependencies, environment issues), report the blocker and mark the test as incomplete. Never mark success without actually running the test.

Create a test coverage report and present to user:
### GSD ► TEST GENERATION COMPLETE

## Results

| Category | Generated | Passing | Failing | Blocked |
|----------|-----------|---------|---------|---------|
| Unit     | {N}       | {n1}    | {n2}    | {n3}    |
| E2E      | {M}       | {m1}    | {m2}    | {m3}    |

## Files Created/Modified
{list of test files with paths}

## Coverage Gaps
{areas that couldn't be tested and why}

## Bugs Discovered
{any assertion failures that indicate implementation bugs}

Record test generation in project state:

gsd_run query state-snapshot

If there are passing tests to commit:

git add {test files}
git commit -m "test(phase-${phase_number}): add unit and E2E tests from add-tests command" -- {test files}

Present next steps:

---

## ▶ Next Up — [${PROJECT_CODE}] ${PROJECT_TITLE}

{if bugs discovered:}
**Fix discovered bugs:** `/gsd:quick fix the {N} test failures discovered in phase ${phase_number}`

{if blocked tests:}
**Resolve test blockers:** {description of what's needed}

{otherwise:}
**All tests passing!** Phase ${phase_number} is fully tested.

---

**Also available:**
- `/gsd:add-tests {next_phase}` — test another phase
- `/gsd:verify-work {phase_number}` — run UAT verification

---

<success_criteria>

  • Phase artifacts loaded (SUMMARY.md, CONTEXT.md, optionally VERIFICATION.md)
  • All changed files classified into TDD/E2E/Skip categories
  • Classification presented to user and approved
  • Project test structure discovered (directories, conventions, runners)
  • Test plan presented to user and approved
  • TDD tests generated with arrange/act/assert structure
  • E2E tests generated targeting user scenarios
  • All tests executed — no untested tests marked as passing
  • Bugs discovered by tests flagged (not fixed)
  • Test files committed with proper message
  • Coverage gaps documented
  • Next steps presented to user </success_criteria>