Commit Graph

11 Commits

Author SHA1 Message Date
Tom Boucher
42f4f184c0 fix(#2844): verify-summary ignores future/prose path mentions; resolves project root (#2910)
* fix(#2844): verify-summary binds file-claim extraction to a creation-claim context

verify-summary's Pattern 1 matched any backticked path-like token with no
context check, so a prose mention of a future deliverable (`shared/types.ts`
in a 'next phase will add…' sentence) was checked for existence and its absence
failed the verdict on a healthy phase. #2685 added shape filtering but no
context check.

- src/verify.cts: both extraction patterns now require a claim label on the line
  (Created/Modified/Added/Updated/Edited/key-files). A bare prose mention no
  longer matches; genuine labeled claims still do.
- gsd-core/bin/gsd-tools.cjs: remove 'verify-summary' from SKIP_ROOT_RESOLUTION
  so relative claim paths resolve against the project root, not the raw cwd
  (subdirectory invocation no longer manufactures missing files).

Regression tests: prose mention not treated as a claim; prose-only SUMMARY
passes; absent claimed file still fails.

* chore(#2844): backfill changeset PR 2910

---------

Co-authored-by: Test <test@example.com>
2026-07-31 03:16:15 -04:00
Tom Boucher
d626dbc6e3 fix(#1883): distinguish a permission/IO error from genuine emptiness in dir scans (#2802)
* test(#1883): failing-first regression for findContextMdIn / listMilestoneArchiveDirs swallowing EACCES

Adds failing-first regression tests proving an unreadable dir is currently
swallowed as empty/null instead of surfacing the permission error. Covers
EACCES, EIO, the unchanged ENOENT empty path, the array fast-path, and both
CONTEXT.md forms. listMilestoneArchiveDirs is exercised in-process via a new
_listMilestoneArchiveDirs test seam (the validate command runs in a subprocess,
so an fs monkeypatch in the test process cannot reach it).

* fix(#1883): distinguish a permission/IO error from genuine emptiness in dir scans

findContextMdIn (src/planning-workspace.cts) and listMilestoneArchiveDirs
(src/verify.cts) catch-alled every readdirSync error into the empty marker
(null / []), conflating a genuine ENOENT ('nothing there') with an EACCES/EIO
failure ('can't read this'). An unreadable phase dir was silently reported as
'no CONTEXT.md' (discuss/plan gates wrongly skipped context) and an unreadable
milestones/ dir as 'no archives' (active-milestone resolution / archived-phase
filtering misbehaved).

Narrow each catch to ENOENT only — keep the long-standing null/[] contract for
genuine absence (Hyrum: empty path unchanged) and re-throw every other error so
it propagates to the caller's existing try/catch. All six findContextMdIn
callers either pass a pre-read string[] (no readdir) or sit inside a try block
that already handles readdir failures; the two listMilestoneArchiveDirs callers
live in the validate command path where errors reach the command error handler.

Exposes a _listMilestoneArchiveDirs test seam so the permission-error path can
be unit-tested in-process (the validate command runs in a subprocess, so an fs
monkeypatch in the test process cannot reach the private helper).

* fix(tests): delete stale emitted-drift ack for gsd-phase-researcher.md

Pre-existing base-branch defect, not part of #1883: commit 6932fb16d
(enhance(#1699): require read-and-cite provenance, #2768) landed the
gsd-phase-researcher.md growth onto next WITHOUT removing its now-obsolete
emitted-drift-ack.json entry. An ack explains a ripple in a PR diff; once the
change merges to next the ripple becomes part of the baseline, so the ack is
permanently stale for every subsequent PR — the emitted-attribution gate fails
on every branch off next with '1 stale acknowledgment(s) — the ripple they
explained is gone'.

The gate's own error message prescribes the fix: delete the stale entry, and
since gsd-phase-researcher.md was the only entry, delete the file (an empty ack
file signals nothing; its presence is the alarm). This unblocks the red base for
all in-flight PRs, not just this one. Folded inline per the bug-fixer directive
(one-line test-prescribed fix, does not bury the #1883 fix).

* docs(changeset): #1883 permission-error-not-empty

* test(#1883): use t.mock.method + t.after per CONTRIBUTING test conventions

Address code-review finding: replace inline try/finally in test bodies and
manual fs.readdirSync reassignment with t.mock.method (auto-restored) and
t.after for tmp-dir cleanup, matching CONTRIBUTING.md test patterns and the
existing t.mock.method idiom in tests/phase.test.cjs.

* docs(changeset): backfill #1883 PR number to 2802
2026-07-28 21:11:48 -04:00
Rezolv
7e8f6a6d7d enhance(#2572): run the verify-summary artifact check against phase SUMMARYs (#2685)
* enhance(#2572): run the verify-summary artifact check against phase SUMMARYs (W025)

The artifact<->git check has existed since the beginning but was only ever
pointed at .planning/research/SUMMARY.md (new-project.md:1145,
new-milestone.md:425). Phase summaries -- the ones that actually claim
"I created these files" -- were never checked.

- extract verifySummaryCore from cmdVerifySummary: same checks, lifted out of
  the output() wrapper so callers consume {passed, checks, errors} directly
  instead of shelling out and re-parsing JSON; cmdVerifySummary is now a thin
  adapter over it
- validate.health gains advisory W025 per phase SUMMARY with missing files

Advisory only: appends to warnings[], never touches status escalation beyond
the channel's own warning semantics, the repair set, or readVerificationStatus.

Resolves both open questions from triage: (a) commits_exist is deliberately NOT
surfaced -- its hash pattern matches any hex-shaped token in prose, too loose to
show a user; (b) a phase carries N per-plan summaries plus a legacy bare
SUMMARY.md, so all of them are checked via the repo-wide filter.

* chore(#2572): add changeset fragment

* feat(#2572): move the SUMMARY artifact check to phase completion

Responds to the #2685 review. Three substantive changes.

Seam (Blocker 2). The check now runs in cmdPhaseComplete, the seam the
issue body cited (src/phase.cts:~1745), not validate.health. That channel
does exist: cmdPhaseComplete declares warnings[], populates it from the
UAT/VERIFICATION pre-scan, and emits it. The cycle objection raised
against the earlier deviation holds for state.cts only -- verify.cts has
no transitive import path to phase.cts, so phase.cts -> verify.cjs adds
no cycle (verified over every src/*.cts). Moving it also retires the
retroactive firing across all historical phases: this fires once, at
completion, for the phase being completed.

Extraction (Blocker 1). Pattern 2 now excludes [ and ] from its path
class. The SUMMARY templates prescribe a YAML flow sequence
(key-files.created: [a.ts, b.ts]) and the label matches case-insensitively,
so the class previously captured the literal [ and produced a candidate
that can never exist on disk -- firing on healthy projects built from the
templates GSD itself ships. Stripping frontmatter was the other offered
remedy; measured across all three shipped templates it is a no-op on top
of the exclusion, so it is not carried. Consequence named in-code: the
key-files block still is not read, which needs a real frontmatter parse.

Also narrowed to the noise classes confirmed in review -- globs, bare
hostnames, and paths resolving outside the project are skipped rather
than reported, and the containment guard the old comment claimed now
actually exists.

Budget (Majors 1 and 3). verifySummaryCore takes a checkCommits option;
phase completion passes false, so the discarded git cat-file probes are
not spawned at all. It also passes Infinity, so every referenced file is
reported instead of the first two -- a summary listing twelve files of
which nine are missing now says nine, not zero. The verb keeps its
historical 2-file default.

Tests (Major 2). The vacuous fixtures are gone with the health block.
The replacements use /-bearing paths that genuinely extract, and each
fix was mutation-checked: un-anchoring pattern 2, dropping the glob,
hostname or containment filter, forcing commit checking on, and
re-capping at 2 each fail at least one test.

* docs(#2572): describe the phase-completion SUMMARY artifact check

The W025 text under /gsd-health is withdrawn with the health seam; the
check is documented where it now runs, under `phase complete` in
docs/CLI-TOOLS.md.

Both the docs and the changeset previously overclaimed: they said a
referenced file not on disk is warned about, while at most two candidates
per SUMMARY were ever examined. The cap is gone at this seam, so the
claim now holds -- and the text states the limits that remain, rather
than leaving them to be discovered: the key-files frontmatter block is
not read, commit hashes are not resolved, and globs, URLs, bare
hostnames and out-of-project paths are skipped rather than reported.

---------

Co-authored-by: CI Rebase Check <ci@gsd-redux>
2026-07-28 18:31:33 -04:00
Tom Boucher
46ba9ed464 fix(#2444): branch plan-structure validation on task type=checkpoint:* (#2473)
* test(#2444): failing-first regression for checkpoint:* plan-structure validation

Add acceptance-criteria tests covering the three canonical checkpoint task
types (human-verify, decision, human-action) plus an unknown-subtype
forward-compat case. Each canonical type must pass verify plan-structure
when it carries its type-specific required fields (per
gsd-core/references/checkpoints.md), and must be flagged when those
fields are missing. Non-checkpoint tasks keep the existing
<action>/<verify>/<done>/<files> requirements unchanged (AC3 regression
guards).

The existing 'errors when checkpoint task but autonomous is true' fixture
is updated to use the canonical checkpoint:human-verify triple
(<what-built>/<how-to-verify>/<resume-signal>) so it does not collide
with the new per-type validator; the assertion (autonomous is not false)
is unchanged.

* fix(#2444): branch plan-structure validation on task type=checkpoint:*

cmdVerifyPlanStructure unconditionally required <action>/<verify>/<done>/
<files> on every task, so every checkpoint:* task — which uses the
checkpoint convention's type-specific fields instead — was reported as a
structural error. Checkpoint-heavy phases produced walls of false findings.

The fix introduces two pure helpers in verify.cts:

  - extractPlanTaskInfos(content): single ReDoS-safe pass over
    <task ...>...</task> blocks that captures BOTH the opening-tag
    attribute string (so the type= selector is not lost, as it is with
    extractTaggedBlocks) and the body, returning a typed PlanTaskInfo.

  - validatePlanTaskStructure(task): branches on the task's type.
    checkpoint:human-verify requires <what-built>/<how-to-verify>/
    <resume-signal> (the canonical triple).
    checkpoint:decision requires <decision>/<options>/<resume-signal>.
    checkpoint:human-action requires <action>/<instructions>/
    <verification>/<resume-signal>.
    Unknown checkpoint:* subtypes require only the universal
    <resume-signal> (forward-compat). All other types keep the historical
    <action>/<verify>/<done>/<files> requirements unchanged.

Canonical reference: gsd-core/references/checkpoints.md. Per-type field
sets validated against the documented templates in
agents/gsd-planner.md and gsd-core/templates/phase-prompt.md.

* fix(#2444): re-resolve body-parser to 2.3.0 in lockfile (GHSA-v422-hmwv-36x6)

GHSA-v422-hmwv-36x6 (body-parser DoS via invalid limit value, low severity,
published 2026-07-20T23:23:26Z) made tests/npm-integrity-gate.test.cjs
(#3588: root workspace production tree has no advisories) fail any subsequent
npm audit --omit=dev. The advisory affects body-parser >=2.0.0 <2.3.0 pulled
transitively via @anthropic-ai/claude-agent-sdk -> @modelcontextprotocol/sdk
-> express -> body-parser@2.2.2.

express@5.2.1 already declares body-parser as ^2.2.1, so 2.3.0 is a valid
re-resolution within express's own compatibility range — no override needed.
Regenerated the lockfile via 'npm audit fix --omit=dev' which re-resolves
transitive deps within their declared ranges; package.json is unchanged.

Verified: npm audit --omit=dev reports 0/0/0/0/0 advisories; body-parser
now reads as 2.3.0 in 'npm ls body-parser --omit=dev'.

* test(#2444): close review gap-closure tests + harden type-attr charset

Orthogonal review (code-review + security-review subagents) returned APPROVE
on Standards and Spec. Per the playbook's zero-tolerance policy, address
every Low finding:

Spec gap-closures:
- AC3 verbatim: add explicit <done> and <files> regression tests for
  non-checkpoint tasks (pre-existing tests only covered <action> and
  <verify>).
- AC2: add checkpoint:decision missing <decision>, checkpoint:human-action
  missing <action>, checkpoint:human-action missing <verification> cases
  (the implementation enforces all of these; only one missing-field case
  per type was previously tested).
- Remove the duplicate 'returns error for nonexistent file' test that
  leaked into the new describe block from the insertion edit.

Security hardening (Low-sev, defense-in-depth):
- Tighten the task type= attribute extractor in src/verify.cts from
  [^"'>\s]+ to [\w:-]+ so a hostile type= attribute cannot carry
  markup fragments (e.g. type=evil<fragment) into the verifier's typed
  JSON output. All legitimate type values (auto, tracer, manual,
  checkpoint:human-verify, checkpoint:decision, checkpoint:human-action,
  checkpoint:tdd-review) match the tighter charset.
- Add adversarial regression test asserting type=evil<fragment surfaces
  as 'evil' (capture stops at '<'), with no markup chars (< > ( ) &)
  in the surfaced type field.

* docs(changeset): add Fixed fragments for #2444 PR

Two fragments:
- sturdy-jays-tumble.md: the verify plan-structure checkpoint fix
- witty-badgers-hum.md: the body-parser 2.3.0 re-resolution

PR number backfilled to 0 placeholder per CLAUDE.md 'PR Number Handling';
will backfill to the real PR number immediately after gh pr create returns.

* docs(changeset): backfill PR number to 2473

Per CLAUDE.md 'PR Number Handling': backfill the placeholder pr:0 with the
real PR number returned by gh pr create.
2026-07-21 08:12:52 -04:00
Tom Boucher
85ed50cc4f test(#1972): consolidate 94 command/module regression tests into subject suites
Fold 94 issue-named command/module regression files into the canonical test file
that owns each subject-under-test, across 52 existing suites (state, config, frontmatter,
roadmap-parser, capability-registry, shell-command-projection-dispatch, plan-phase-drift-guard,
health-validation, runtime-converters, commands, etc.). Verbatim block-scoped describe
wrappers; 881 subtests conserved 1:1. No new test files.

Host-env pre-check (per B2): the only GSD_WORKSTREAM/GSD_PROJECT-touching destinations
(intel, planning-workspace) clear those vars hermetically, so folded CLI tests are safe.

Regenerates regression-name allowlist (222->162), ratchets file-count allowlist across
8 buckets (validate entry removed after dropping <=2), makes 34 relocated allow-test-rule
exemptions issue-ref-compliant (ADR-456; prunes 34 stale ids). Repoints CONTEXT.md +
ADR-0002/443/1235/3524 test-file references. lint:ci green.

Part of epic #1969. Closes #1972.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 08:59:23 -04:00
Tom Boucher
de3ba45d00 test(#1971): consolidate 48 gsd-tools CLI regression tests into subcommand suites
Fold 48 issue-named gsd-tools CLI regression files into the canonical test file
that owns each subcommand subject (state, roadmap, phase, milestone, audit, config,
router/dispatch, stats, verify, health, etc.), preserving every assertion and its
origin issue number as provenance (block-scoped describe wrappers, 299 subtests
conserved 1:1). No monolithic gsd-tools.test.cjs created — routes into 18 existing
per-subject suites.

Removes 48 tests/ files. Regenerates regression-name allowlist (271->231), ratchets
the file-count allowlist across 6 buckets (audit/milestone/phase/roadmap/state/verify),
and makes 10 relocated allow-test-rule exemptions issue-ref-compliant (ADR-456; prunes
10 stale ids). Repoints one CONTEXT.md symptom ref and ADR-3524's parity-test ref.
lint:ci green.

Part of epic #1969. Closes #1971.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 01:55:48 -04:00
Tom Boucher
2703422be8 refactor(tests): standardize to node:assert/strict and t.after() per CONTRIBUTING.md (#1675)
* refactor(tests): standardize to node:assert/strict and t.after() per CONTRIBUTING.md

- Replace require('node:assert') with require('node:assert/strict') across
  all 73 test files to enforce strict equality (no type coercion)
- Replace try/finally cleanup blocks with t.after() hooks in core.test.cjs
  and hooks-opt-in.test.cjs per the test lifecycle standards
- Utility functions in codex-config and security-scan retain try/finally
  as that is appropriate for per-function resource guards, not lifecycle hooks

Closes #1674

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(tests): add --test-concurrency=4 to test runner for parallel file execution

Node.js --test-concurrency controls how many test files run as parallel child
processes. Set to 4 by default, configurable via TEST_CONCURRENCY env var.
Fixes tests at a known level rather than inheriting os.availableParallelism()
which varies across CI environments.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): allowlist verify.test.cjs in prompt-injection scanner

tests/verify.test.cjs uses <human>...</human> as GSD phase task-type
XML (meaning "a human should verify this step"), which matches the
scanner's fake-message-boundary pattern for LLM APIs. This is a
false positive — add it to the allowlist alongside the other test files
that legitimately contain injection-adjacent patterns.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 14:29:03 -04:00
Ethan Hurst
e8e497bc9d test(03-03): add 20 tests for verify references, commits, artifacts, key-links
- verify references (5): valid refs, missing refs, backtick paths, template skip, not found
- verify commits (3): valid hash, invalid hash, mixed valid+invalid
- verify artifacts (6): all criteria pass, missing file, line count, pattern, export, no artifacts
- verify key-links (6): pattern in source, pattern in target, pattern not found, no-pattern
  string inclusion, source not found, no key_links in frontmatter

Notes:
- parseMustHavesBlock requires 4-space indent for block name, 6-space for items,
  8-space for sub-keys; helpers use this format explicitly
- @https:// refs are NOT skipped by verify references (only backtick http refs are);
  test reflects actual behavior (only template expressions are skipped)
2026-02-26 05:47:19 +10:00
Ethan Hurst
ebe96fe9ee test(03-03): add 8 verify-summary tests including REG-03 and search(-1) regression
- verify-summary returns not found for nonexistent summary
- verify-summary passes for valid summary with real files and commits
- verify-summary reports missing files mentioned in backticks
- verify-summary detects self-check pass and fail indicators
- REG-03: returns self_check 'not_found' when no section exists (not a failure)
- search(-1) regression: guard on line 79 prevents -1 from content.search()
- respects --check-count parameter to limit file checking
2026-02-26 05:47:19 +10:00
Ethan Hurst
9f87ba215b test(03-01): add verify plan-structure and phase-completeness tests
- Add validPlanContent() helper for building valid PLAN.md fixtures
- 7 tests for verify plan-structure: missing frontmatter, valid plan,
  missing name, missing action, wave/depends_on, checkpoint/autonomous,
  and file-not-found
- 4 tests for verify phase-completeness: complete phase, missing summary,
  orphan summary, nonexistent phase
- All 14 tests in verify.test.cjs pass (3 existing + 11 new)
2026-02-26 05:47:19 +10:00
Tyler Satre
fa2e156887 refactor: split gsd-tools.test.cjs into domain test files
Move 81 tests (18 describe blocks) from single monolithic test file
into 7 domain-specific test files under tests/ with shared helpers.
Test parity verified: 81/81 pass before and after split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 10:30:54 -05:00