Commit Graph

8 Commits

Author SHA1 Message Date
Tom Boucher
46ba9ed464 fix(#2444): branch plan-structure validation on task type=checkpoint:* (#2473)
* test(#2444): failing-first regression for checkpoint:* plan-structure validation

Add acceptance-criteria tests covering the three canonical checkpoint task
types (human-verify, decision, human-action) plus an unknown-subtype
forward-compat case. Each canonical type must pass verify plan-structure
when it carries its type-specific required fields (per
gsd-core/references/checkpoints.md), and must be flagged when those
fields are missing. Non-checkpoint tasks keep the existing
<action>/<verify>/<done>/<files> requirements unchanged (AC3 regression
guards).

The existing 'errors when checkpoint task but autonomous is true' fixture
is updated to use the canonical checkpoint:human-verify triple
(<what-built>/<how-to-verify>/<resume-signal>) so it does not collide
with the new per-type validator; the assertion (autonomous is not false)
is unchanged.

* fix(#2444): branch plan-structure validation on task type=checkpoint:*

cmdVerifyPlanStructure unconditionally required <action>/<verify>/<done>/
<files> on every task, so every checkpoint:* task — which uses the
checkpoint convention's type-specific fields instead — was reported as a
structural error. Checkpoint-heavy phases produced walls of false findings.

The fix introduces two pure helpers in verify.cts:

  - extractPlanTaskInfos(content): single ReDoS-safe pass over
    <task ...>...</task> blocks that captures BOTH the opening-tag
    attribute string (so the type= selector is not lost, as it is with
    extractTaggedBlocks) and the body, returning a typed PlanTaskInfo.

  - validatePlanTaskStructure(task): branches on the task's type.
    checkpoint:human-verify requires <what-built>/<how-to-verify>/
    <resume-signal> (the canonical triple).
    checkpoint:decision requires <decision>/<options>/<resume-signal>.
    checkpoint:human-action requires <action>/<instructions>/
    <verification>/<resume-signal>.
    Unknown checkpoint:* subtypes require only the universal
    <resume-signal> (forward-compat). All other types keep the historical
    <action>/<verify>/<done>/<files> requirements unchanged.

Canonical reference: gsd-core/references/checkpoints.md. Per-type field
sets validated against the documented templates in
agents/gsd-planner.md and gsd-core/templates/phase-prompt.md.

* fix(#2444): re-resolve body-parser to 2.3.0 in lockfile (GHSA-v422-hmwv-36x6)

GHSA-v422-hmwv-36x6 (body-parser DoS via invalid limit value, low severity,
published 2026-07-20T23:23:26Z) made tests/npm-integrity-gate.test.cjs
(#3588: root workspace production tree has no advisories) fail any subsequent
npm audit --omit=dev. The advisory affects body-parser >=2.0.0 <2.3.0 pulled
transitively via @anthropic-ai/claude-agent-sdk -> @modelcontextprotocol/sdk
-> express -> body-parser@2.2.2.

express@5.2.1 already declares body-parser as ^2.2.1, so 2.3.0 is a valid
re-resolution within express's own compatibility range — no override needed.
Regenerated the lockfile via 'npm audit fix --omit=dev' which re-resolves
transitive deps within their declared ranges; package.json is unchanged.

Verified: npm audit --omit=dev reports 0/0/0/0/0 advisories; body-parser
now reads as 2.3.0 in 'npm ls body-parser --omit=dev'.

* test(#2444): close review gap-closure tests + harden type-attr charset

Orthogonal review (code-review + security-review subagents) returned APPROVE
on Standards and Spec. Per the playbook's zero-tolerance policy, address
every Low finding:

Spec gap-closures:
- AC3 verbatim: add explicit <done> and <files> regression tests for
  non-checkpoint tasks (pre-existing tests only covered <action> and
  <verify>).
- AC2: add checkpoint:decision missing <decision>, checkpoint:human-action
  missing <action>, checkpoint:human-action missing <verification> cases
  (the implementation enforces all of these; only one missing-field case
  per type was previously tested).
- Remove the duplicate 'returns error for nonexistent file' test that
  leaked into the new describe block from the insertion edit.

Security hardening (Low-sev, defense-in-depth):
- Tighten the task type= attribute extractor in src/verify.cts from
  [^"'>\s]+ to [\w:-]+ so a hostile type= attribute cannot carry
  markup fragments (e.g. type=evil<fragment) into the verifier's typed
  JSON output. All legitimate type values (auto, tracer, manual,
  checkpoint:human-verify, checkpoint:decision, checkpoint:human-action,
  checkpoint:tdd-review) match the tighter charset.
- Add adversarial regression test asserting type=evil<fragment surfaces
  as 'evil' (capture stops at '<'), with no markup chars (< > ( ) &)
  in the surfaced type field.

* docs(changeset): add Fixed fragments for #2444 PR

Two fragments:
- sturdy-jays-tumble.md: the verify plan-structure checkpoint fix
- witty-badgers-hum.md: the body-parser 2.3.0 re-resolution

PR number backfilled to 0 placeholder per CLAUDE.md 'PR Number Handling';
will backfill to the real PR number immediately after gh pr create returns.

* docs(changeset): backfill PR number to 2473

Per CLAUDE.md 'PR Number Handling': backfill the placeholder pr:0 with the
real PR number returned by gh pr create.
2026-07-21 08:12:52 -04:00
Tom Boucher
85ed50cc4f test(#1972): consolidate 94 command/module regression tests into subject suites
Fold 94 issue-named command/module regression files into the canonical test file
that owns each subject-under-test, across 52 existing suites (state, config, frontmatter,
roadmap-parser, capability-registry, shell-command-projection-dispatch, plan-phase-drift-guard,
health-validation, runtime-converters, commands, etc.). Verbatim block-scoped describe
wrappers; 881 subtests conserved 1:1. No new test files.

Host-env pre-check (per B2): the only GSD_WORKSTREAM/GSD_PROJECT-touching destinations
(intel, planning-workspace) clear those vars hermetically, so folded CLI tests are safe.

Regenerates regression-name allowlist (222->162), ratchets file-count allowlist across
8 buckets (validate entry removed after dropping <=2), makes 34 relocated allow-test-rule
exemptions issue-ref-compliant (ADR-456; prunes 34 stale ids). Repoints CONTEXT.md +
ADR-0002/443/1235/3524 test-file references. lint:ci green.

Part of epic #1969. Closes #1972.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 08:59:23 -04:00
Tom Boucher
de3ba45d00 test(#1971): consolidate 48 gsd-tools CLI regression tests into subcommand suites
Fold 48 issue-named gsd-tools CLI regression files into the canonical test file
that owns each subcommand subject (state, roadmap, phase, milestone, audit, config,
router/dispatch, stats, verify, health, etc.), preserving every assertion and its
origin issue number as provenance (block-scoped describe wrappers, 299 subtests
conserved 1:1). No monolithic gsd-tools.test.cjs created — routes into 18 existing
per-subject suites.

Removes 48 tests/ files. Regenerates regression-name allowlist (271->231), ratchets
the file-count allowlist across 6 buckets (audit/milestone/phase/roadmap/state/verify),
and makes 10 relocated allow-test-rule exemptions issue-ref-compliant (ADR-456; prunes
10 stale ids). Repoints one CONTEXT.md symptom ref and ADR-3524's parity-test ref.
lint:ci green.

Part of epic #1969. Closes #1971.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 01:55:48 -04:00
Tom Boucher
2703422be8 refactor(tests): standardize to node:assert/strict and t.after() per CONTRIBUTING.md (#1675)
* refactor(tests): standardize to node:assert/strict and t.after() per CONTRIBUTING.md

- Replace require('node:assert') with require('node:assert/strict') across
  all 73 test files to enforce strict equality (no type coercion)
- Replace try/finally cleanup blocks with t.after() hooks in core.test.cjs
  and hooks-opt-in.test.cjs per the test lifecycle standards
- Utility functions in codex-config and security-scan retain try/finally
  as that is appropriate for per-function resource guards, not lifecycle hooks

Closes #1674

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(tests): add --test-concurrency=4 to test runner for parallel file execution

Node.js --test-concurrency controls how many test files run as parallel child
processes. Set to 4 by default, configurable via TEST_CONCURRENCY env var.
Fixes tests at a known level rather than inheriting os.availableParallelism()
which varies across CI environments.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): allowlist verify.test.cjs in prompt-injection scanner

tests/verify.test.cjs uses <human>...</human> as GSD phase task-type
XML (meaning "a human should verify this step"), which matches the
scanner's fake-message-boundary pattern for LLM APIs. This is a
false positive — add it to the allowlist alongside the other test files
that legitimately contain injection-adjacent patterns.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 14:29:03 -04:00
Ethan Hurst
e8e497bc9d test(03-03): add 20 tests for verify references, commits, artifacts, key-links
- verify references (5): valid refs, missing refs, backtick paths, template skip, not found
- verify commits (3): valid hash, invalid hash, mixed valid+invalid
- verify artifacts (6): all criteria pass, missing file, line count, pattern, export, no artifacts
- verify key-links (6): pattern in source, pattern in target, pattern not found, no-pattern
  string inclusion, source not found, no key_links in frontmatter

Notes:
- parseMustHavesBlock requires 4-space indent for block name, 6-space for items,
  8-space for sub-keys; helpers use this format explicitly
- @https:// refs are NOT skipped by verify references (only backtick http refs are);
  test reflects actual behavior (only template expressions are skipped)
2026-02-26 05:47:19 +10:00
Ethan Hurst
ebe96fe9ee test(03-03): add 8 verify-summary tests including REG-03 and search(-1) regression
- verify-summary returns not found for nonexistent summary
- verify-summary passes for valid summary with real files and commits
- verify-summary reports missing files mentioned in backticks
- verify-summary detects self-check pass and fail indicators
- REG-03: returns self_check 'not_found' when no section exists (not a failure)
- search(-1) regression: guard on line 79 prevents -1 from content.search()
- respects --check-count parameter to limit file checking
2026-02-26 05:47:19 +10:00
Ethan Hurst
9f87ba215b test(03-01): add verify plan-structure and phase-completeness tests
- Add validPlanContent() helper for building valid PLAN.md fixtures
- 7 tests for verify plan-structure: missing frontmatter, valid plan,
  missing name, missing action, wave/depends_on, checkpoint/autonomous,
  and file-not-found
- 4 tests for verify phase-completeness: complete phase, missing summary,
  orphan summary, nonexistent phase
- All 14 tests in verify.test.cjs pass (3 existing + 11 new)
2026-02-26 05:47:19 +10:00
Tyler Satre
fa2e156887 refactor: split gsd-tools.test.cjs into domain test files
Move 81 tests (18 describe blocks) from single monolithic test file
into 7 domain-specific test files under tests/ with shared helpers.
Test parity verified: 81/81 pass before and after split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 10:30:54 -05:00