Files
msd-core/tests/code-review-pipeline-regression.test.cjs
Tom Boucher 107eb8c1d9 feat(#3753): run docs guards on the PR that changes the docs they read (#3787)
A PR whose diff is entirely under docs/ runs zero tests, so a guard whose INPUT
is shipped prose cannot protect the PR lane of the diffs it exists to check. Its
only firing opportunity is after merge, on the shared branch -- which is how next
went red on dacae9273 while the PR that caused it (#3746) was green on every
check.

The docs-lint job in .github/workflows/docs-required.yml -- an ALREADY-REQUIRED
context -- now selects and runs the docs guards that read the specific docs files
the PR changed.

  scripts/docs-guard-registry.cjs    test file -> the docs paths it reads (63)
  scripts/select-docs-guards.cjs     pure (changedPaths, registry) -> test files
  scripts/lint-docs-guard-registration.cjs   drift guard, wired into lint:ci

scripts/ci-test-scope.cjs is NOT touched -- `git diff origin/next --` on it is
empty -- so #764's saving stands and its 21 pinning tests are untouched.

Selection: exact path; trailing-slash directory prefix (boundary-checked --
docs/adrenaline.md does NOT match docs/adr/, which a naive startsWith gets
wrong); and '*' for the 6 entries that walk docs/ generally or read a computed
path. Unknown maps to '*' -- guessing narrow is how a guard silently stops
running. Measured: a typo fix selects 6 of 63; docs/AGENTS.md selects 12;
docs/COMMANDS.md selects 18.

Four things this got wrong first, each found by an independent reviewer or by
probe, and each having been asserted safe in a comment:

1. The registry started as a RULE in ci-test-scope.cjs's RULES, on the theory
   that classify()'s !codeChanged normalization made it inert. True for
   docs-ONLY diffs; false for MIXED docs+code diffs, where codeChanged is true
   and the normalization never runs:

     node scripts/ci-test-scope.cjs --files "docs/a.md src/semver.cts"
       with the RULE:  25 targeted_tests
       origin/next:     3 targeted_tests

   Category error: RULES is the scoped lane's input; a docs-guard registry is a
   lane manifest for a consumer that never calls classify(). Extracted; pinned
   by value.

2. The second attempt was a dedicated workflow with paths: [docs/**]. Such a
   workflow never reports on a non-docs PR, so it can never be a required
   context without hanging every non-docs PR -- and a non-required check does not
   block a merge, so the guard would have been advisory and #3753 unfixed.
   docs-required.yml already has no paths: filter, already supplies the required
   docs-lint context, already computes docs_changed, and already ran one docs
   guard gated on it. Generalizing that step needs no ruleset edit at all.

3. The registry and the drift lint were built from ONE path-segment heuristic, so
   both were blind identically -- and blind at the guard that motivated the issue.
   The reader-call regex required a character BEFORE its keyword, so a callee
   named exactly read( / load( / parse( / doc( / file( / content( could never
   match; and only an INLINE path.join(ROOT,'docs','X.md') argument was caught,
   missing the two-step-via-variable form -- the MAJORITY spelling -- plus
   template literals and concatenation. Detector 1 fired on 14 of ~450 files, so
   35 genuine guards sat unregistered while the lint reported 0 violations,
   including cursor-reviewer (reads docs/COMMANDS.md, asserts
   .includes('--cursor')) and inventory-headings-countfree. The "accepted blind
   spot" this shipped with was the common case, not a fringe.

4. With detection fixed the true population is 115 files: 63 genuine guards, 52
   incidental. Running all 63 in a REQUIRED check on a one-line typo fix is the
   cost #764 exists to avoid -- install.test.cjs is 7840 lines and reads exactly
   one docs file, docs/AGENTS.md, for its frontmatter. Dropping it reproduces the
   bug; running it for a typo elsewhere is waste. Hence the map.

Then a second review round found six more, all fixed here:

- fragment-single-edit-propagation.install.test.cjs was EXEMPTED as
  "overlay fixture only". False: it reads the real docs/registries/eos.json and
  asserts on a registry entry name, and reads the real ADR-0001 and asserts its
  H1. A docs-only PR touching either would have gone green and red next -- #3753
  shipping again, from inside the fix for it. Now registered against both paths,
  and all 52 remaining exemptions were re-audited one by one.
- The SUITES-collision guard compared RAW registry keys, but run-tests.cjs strips
  a leading `tests/` BEFORE its suite check. So it caught 'all' and missed
  'tests/all' -- the only spelling that can actually occur, since every key
  carries the prefix. One typo would have run all 824 test files inside the
  required job. Now normalized the same way run-tests.cjs normalizes.
- The lint failed OPEN on an unreadable tests dir or candidate file: 0 violations,
  ok:true. A guard that cannot read its input must never report success.
- The exemption ratchet gated identity only, so a baselined file that later
  STARTED asserting on shipped docs stayed exempt silently -- 52 permanently blind
  files. The baseline now fingerprints the docs paths each exempted file
  references and fails when that set changes, naming what changed.
- The exemption marker was still honored inside a multi-line template literal in
  the header window. The scanner now tracks template-literal and block-comment
  state.
- `git diff --name-only | grep '^docs/'` silently dropped C-quoted non-ASCII docs
  paths, making docs_changed=false a green zero-guard check. Both call sites now
  pass -c core.quotepath=false.
- The run step was gated on hashFiles(), which a force-committed
  .docs-guard-tests.txt would satisfy. The step now rm -f's both scratch files
  first and gates on an output it sets itself.

Three empty states, deliberately distinct, because conflating them rebuilds
#3753: an empty or malformed registry HARD-FAILS; docs changed with no guard
covering them logs and skips; no docs change is already gated. The middle state
must never be expressed as an empty --files-from, which prints `no tests in suite
"all"` and exits 0 -- a green check that guarded nothing. With the current
registry that state is unreachable, because the six '*' entries always match;
the branch is kept as defensive handling for a future registry and says so.

timeout-minutes: 15 bounds the required job against a hanging fork-supplied test;
it had none. npm ci was added because the job never installed dependencies -- the
previous single-file step got away without it, the registry does not.

docs/contributing/docs-guard-registration.md documents the rule, following its
sibling cross-platform-portability-rules.md, and CONTRIBUTING.md's CI Test
Quality Checks table links to it. It is also load-bearing: without a docs/ file
in the diff this PR would not have triggered its own lane, shipping an
unexercised change to a required check.

One unrelated fix, included because this PR surfaced it and CLAUDE.md forbids
deferring a defect found while working. On this branch's first CI run,
`full test (windows-latest, 24, shard 3/3)` was CANCELLED at exactly 30 minutes;
tests were still passing 0.8s before the cancel, so it is a wall-clock timeout,
not a hang, and a cancelled job reddens `Required tests`.

The cause is not this PR's test file, which costs ~60ms. Shard composition is
unstable: adding ONE file to the unit suite reshuffled 115 of 268 files between
shards, and shard 3 drew a heavier mix. Underneath that is a real pre-existing
defect. tests/ci-test-job-timeout-budget.test.cjs requires every lane's budget to
be >= 1.5x its MEASURED cost -- "a lane that got slower must be re-budgeted, not
excused" -- and its test-full entry recorded 19m from a windows-22 shard. That is
stale. Measured on `next` with none of this PR's changes present: 26m18s (run
32614439702, windows-latest/24 shard 3/3), 23m36s and 23m17s on shard 2/3. So the
lane costs ~26m and the 30-minute cap carried 1.14x headroom, not 1.5x. The gate
had been out of compliance with its own rule; this PR was merely the file
addition that reshuffled shard 3 past the cliff.

Fixed as that file prescribes: measuredMinutes 19 -> 27 with fresh evidence, and
test-full timeout-minutes 30 -> 45. The rule's minimum for 27m is 41; 45 is
deliberately above it because the reshuffle means per-shard worst case moves run
to run, and a budget pinned to the exact minimum would be re-breached by the next
test file anyone adds. Only that one job's timeout changed; test.yml's scope,
matrix and steps are untouched, so #764's saving is unaffected.

Raising that cap let the Windows shard finish (28m45s, inside 45) and uncovered
a real failure the 30-minute cancel had been masking:
`new quick-task branch branches off origin/main (#2916)` died with
`outcome=timed_out exitCode=null`, SIGTERM, at the 15000ms bound.

tests/quick-branching.test.cjs:149 `runStep` runs a `#!/usr/bin/env bash` script
executing MULTIPLE git commands, but was bound to GIT_TIMEOUT_MS (15000) -- the
norm for a SINGLE git plumbing call. tests/helpers/timeouts.cjs already documents
this exact failure and exists to fix it: HOOK_FANOUT_TIMEOUT_MS was created after
PR #3285 recorded "outcome=timed_out exitCode=null at exactly the 15000ms probe
bound while every other lane passed the same commit", and calls that "a bound
sized for the wrong class, not a slow machine". Our failure is that case
verbatim, so both sites move to the class norm rather than to a bigger number.

The same class also failed on `next` itself 21 hours earlier -- run 32608945654,
windows-latest/24 shard 1/3, `plan touching only src/ in a submodule project
keeps worktree isolation ENABLED` -- where tests/worktree-safety.test.cjs:5845
`runGate` fans out to `git config --file .gitmodules` under a hardcoded 30000.
Fixed too, since it is a defect in the tree regardless of which branch surfaced
it.

A survey of the whole tests/ tree found the same class-mismatch at further
bash fan-out sites bound under 60000ms, and the maintainer approved sweeping
them rather than leaving them latent to surface the same way one at a time. 16
fan-out sites across 16 files now use the class norm.

The sweep is class-correctness, not raising numbers until things pass. Sites
were moved ONLY where the bash body demonstrably spawns something (git, node,
npm, a CLI); self-contained shell snippets were left where they are, and are
listed as deliberately unchanged: pure if/printf bodies (copilot-install), pure
array/case builtins (code-review-pipeline-regression:638), a documented
pure-shell gsd_run stub (host-integration), single-process hook calls
(workflow-guard:222/271/302), and a deliberately tight 5000ms fast-check hook
(gsd-write-guard.property). Nothing was lowered. process-seam.test.cjs:513
(literal 300) is untouched on purpose -- it tests timeout BEHAVIOR, so raising
it would destroy what it asserts.

Shared file-level constants were the trap here, and were handled per file rather
than by redefinition: GIT_TIMEOUT_MS has ~15 users in git-base-branch and only 1
is a fan-out; WORKTREE_TIMEOUT_MS has 16 users in worktree.test.cjs and 3 are;
PROBE_TIMEOUT_MS has several in three more files. In each the CALL SITE was
changed and the constant left alone, so no single-plumbing-call site silently
inherited a 60s bound. The one exception is hooks-opt-in.test.cjs, where
HOOK_TIMEOUT_MS has exactly one consumer -- spawnHook, the fan-out itself -- so
redefining it is identical in effect and reads better.

Only two of these sites have actually been observed failing. The rest cite that
shared class and those two run ids rather than inventing evidence of their own.

Co-authored-by: sim <sim@local>
2026-08-23 21:21:21 -04:00

1233 lines
56 KiB
JavaScript

// docs-guard-exempt: 'docs/DEVELOPMENT.md' is a synthetic files-list fixture entry, not read as content.
// allow-test-rule: source-text-is-the-product
// The workflow and agent .md files ARE the product: their text is loaded and
// executed/interpreted at runtime by the agent host. Testing that specific
// strings exist within these files tests the deployed contract, not an
// implementation detail. No runtime API exists to enumerate the label accept-
// list or filter-set definitions — the text IS the specification.
//
// Bug 1 (compute_file_scope) — The inline Node.js script embedded in the
// workflow .md is the parser. The test implements the identical parse logic as
// a pure JS function (mirroring lines 172-184 of code-review.md exactly) and
// asserts on its structured output. A separate docs-parity assertion checks
// that the workflow .md contains the hyphen-aware boundary regex and the
// em-dash/parenthetical stripping — both of which are the deployed contract.
//
// Bug 2 (present_results) — Tested both behaviourally (pure JS helper that
// mimics the grep|cut pipeline) and via docs-parity on the workflow .md text.
//
// Bugs 3 and reviewer contract — docs-parity only on agents/*.md: the filter-
// set definition and label-equivalence contract exist only as text in those
// files; there is no runtime enumeration API.
'use strict';
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { runHook } = require('./helpers/process-seam.cjs');
const { toLegacyResult, gitOrThrow } = require('./helpers/git-fixture.cjs');
const { PROBE_TIMEOUT_MS, GIT_TIMEOUT_MS, HOOK_FANOUT_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
const { createTempDir, createTempGitProject, cleanup, readFileNormalized } = require('./helpers.cjs');
const ROOT = path.resolve(__dirname, '..');
const WORKFLOW_PATH = path.join(ROOT, 'gsd-core', 'workflows', 'code-review.md');
const PRE_PASS_STEP_PATH = path.join(ROOT, 'gsd-core', 'workflows', 'code-review', 'steps', 'structural-pre-pass.md');
const FIXER_PATH = path.join(ROOT, 'agents', 'gsd-code-fixer.md');
const REVIEWER_PATH = path.join(ROOT, 'agents', 'gsd-code-reviewer.md');
// ---------------------------------------------------------------------------
// Pure-function implementation of the compute_file_scope Node script body.
// This mirrors the logic in code-review.md lines 172-184 exactly.
// If those lines change, this function must be updated in tandem (and the
// docs-parity assertions below will catch a mismatch at the regex level).
//
// #2666: the acceptance predicate accepts root-level paths (no `/`) and known
// extensionless build files (Dockerfile/Makefile/etc.), not only nested paths
// with a trailing extension. Prose bullets are rejected by the known-filename /
// has-extension distinction (plus the post-processing existence check backstop
// in the shipped workflow).
const KNOWN_EXTENSIONLESS_BUILD_FILES = new Set([
'dockerfile', 'containerfile', 'makefile', 'justfile', 'procfile',
]);
function isAcceptablePath(raw) {
// A trailing `.`+alphanumerics extension qualifies (root-level OR nested):
// package.json, renovate.json, .gitlab-ci.yml, AGENTS.md, app/foo.tsx
if (/\.[A-Za-z0-9]+$/.test(raw)) return true;
// Known extensionless build filename (basename, case-insensitive): Dockerfile, Makefile, …
const base = raw.split('/').pop();
if (KNOWN_EXTENSIONLESS_BUILD_FILES.has(base.toLowerCase())) return true;
return false;
}
function parseKeyFiles(yaml) {
const files = [];
let inSection = null;
for (const line of yaml.split('\n')) {
if (/^\s+created:/.test(line)) { inSection = 'created'; continue; }
if (/^\s+modified:/.test(line)) { inSection = 'modified'; continue; }
// Hyphen-aware boundary: reset inSection for ANY key: line (including key-decisions:, etc.)
if (/^\s*[\w-]+:/.test(line) && !/^\s*-/.test(line)) { inSection = null; continue; }
if (inSection && /^\s+-\s+(.+)/.test(line)) {
let raw = line.match(/^\s+-\s+(.+)/)[1].trim();
raw = raw.replace(/^['"]|['"]$/g, '');
// Order matters: parens BEFORE em-dash because em-dashes can appear inside parens
raw = raw.replace(/\s+\([^)]*\)\s*$/, '');
raw = raw.split(/\s+—\s/)[0].trim();
if (isAcceptablePath(raw)) {
files.push(raw);
}
}
}
return files;
}
// ---------------------------------------------------------------------------
// Pure-function implementation of the present_results severity-label parser.
// Mirrors the grep -E "^\s*(critical|blocker):" | head -1 | cut -d: -f2 | xargs
// pipeline from code-review.md.
// ---------------------------------------------------------------------------
function parseFrontmatterCritical(frontmatter) {
const lines = frontmatter.split('\n');
const match = lines.find((l) => /^\s*(critical|blocker):/.test(l));
if (!match) return { critical: 0 };
const value = match.split(':').slice(1).join(':').trim();
return { critical: parseInt(value, 10) || 0 };
}
// ---------------------------------------------------------------------------
// BUG 1 — SUMMARY parser: compute_file_scope must not bleed prose from
// hyphenated sections (key-decisions:, patterns-established:, etc.) into the
// file list, and must strip em-dash descriptions and parentheticals.
// ---------------------------------------------------------------------------
describe('Bug 1 — compute_file_scope SUMMARY parser', () => {
test('extracts only key-files.created and key-files.modified entries', () => {
const yaml = [
'key-files:',
' created:',
' - app/foo.tsx',
' modified:',
' - lib/bar.ts',
'key-decisions:',
' - We chose RSC for performance reasons',
'patterns-established:',
' - Always validate at the boundary',
'requirements-completed:',
' - REQ-01 done',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(files.sort(), ['app/foo.tsx', 'lib/bar.ts'].sort());
});
test('strips em-dash narrative from bullet: "app/foo.tsx — RSC catalogue with filters"', () => {
const yaml = [
'key-files:',
' created:',
' - app/foo.tsx — RSC catalogue with topic/mode/date filters',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(files, ['app/foo.tsx']);
});
test('strips parenthetical from bullet: "tests/bar.test.ts (122 lines — 17 assertions)"', () => {
const yaml = [
'key-files:',
' created:',
' - tests/bar.test.ts (122 lines — 17 assertions)',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(files, ['tests/bar.test.ts']);
});
test('hyphenated sections in any order produce identical results', () => {
const yamlA = [
'key-decisions:',
' - Some decision',
'key-files:',
' created:',
' - src/index.ts',
'patterns-established:',
' - Some pattern',
].join('\n');
const yamlB = [
'patterns-established:',
' - Some pattern',
'key-files:',
' created:',
' - src/index.ts',
'key-decisions:',
' - Some decision',
].join('\n');
assert.deepStrictEqual(parseKeyFiles(yamlA), parseKeyFiles(yamlB));
assert.deepStrictEqual(parseKeyFiles(yamlA), ['src/index.ts']);
});
test('prose-only bullets from key-decisions are never included in file list', () => {
const yaml = [
'key-decisions:',
' - We chose RSC for performance reasons',
' - Deferred auth to Phase 3',
'key-files:',
' created:',
' - app/page.tsx',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(files, ['app/page.tsx']);
});
// #2666 — the Tier-2 extractor must NOT drop repository-root files (no `/`)
// or known extensionless build files. Pre-fix the buggy predicate
// `/\//.test(raw) && /\.[A-Za-z0-9]+$/.test(raw)` dropped every root-level
// path and every extensionless build file anywhere in the tree.
test('#2666 RED: root-level files with extensions are accepted (package.json, renovate.json, .gitlab-ci.yml, AGENTS.md)', () => {
const yaml = [
'key-files:',
' modified:',
' - package.json',
' - renovate.json',
' - .gitlab-ci.yml',
' - AGENTS.md',
' - CLAUDE.md',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(
files.sort(),
['.gitlab-ci.yml', 'AGENTS.md', 'CLAUDE.md', 'package.json', 'renovate.json'],
'root-level files with extensions must not be dropped for lacking a directory separator',
);
});
test('#2666: nested extensionless build files are accepted (docker/Dockerfile, web/Makefile)', () => {
const yaml = [
'key-files:',
' modified:',
' - docker/Dockerfile',
' - web/Makefile',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(files.sort(), ['docker/Dockerfile', 'web/Makefile']);
});
test('#2666: root-level extensionless build files are accepted (Dockerfile, Makefile, Justfile, Containerfile, Procfile)', () => {
const yaml = [
'key-files:',
' created:',
' - Dockerfile',
' - Makefile',
' - Justfile',
' - Containerfile',
' - Procfile',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(
files.sort(),
['Containerfile', 'Dockerfile', 'Justfile', 'Makefile', 'Procfile'],
);
});
test('#2666 acceptance #1: the reporter 10-file Docker+CI phase yields all 10 paths', () => {
const yaml = [
'key-files:',
' created:',
' - Dockerfile',
' - .gitlab-ci.yml',
' - renovate.json',
' - AGENTS.md',
' - CLAUDE.md',
' - docs/DEVELOPMENT.md',
' - scripts/version-consistency-gate.mjs',
' - web/package.json',
' - web/version_management.md',
' - web/update-version.cjs',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(
files.sort(),
[
'.gitlab-ci.yml', 'AGENTS.md', 'CLAUDE.md', 'Dockerfile',
'docs/DEVELOPMENT.md', 'renovate.json', 'scripts/version-consistency-gate.mjs',
'web/package.json', 'web/update-version.cjs', 'web/version_management.md',
],
'the full reporter phase must scope all 10 files, including Dockerfile + root files',
);
});
test('#2666 negative-space: a path-like prose bullet with no extension and unknown basename is rejected', () => {
// `topic/mode/date filters` has a `/` but no extension and an unknown basename —
// the pre-fix predicate dropped it (good), the relaxed predicate must STILL drop it.
const yaml = [
'key-decisions:',
' - topic/mode/date filters',
'key-files:',
' created:',
' - app/page.tsx',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(files, ['app/page.tsx']);
});
test('#2666 negative-space: em-dash/parenthetical stripping still works on an accepted root file', () => {
const yaml = [
'key-files:',
' modified:',
' - Dockerfile — multi-stage build',
].join('\n');
const files = parseKeyFiles(yaml);
assert.deepStrictEqual(files, ['Dockerfile']);
});
// Docs-parity: the workflow .md must contain the hyphen-aware boundary regex
// so what we tested above is actually what is deployed.
test('code-review.md contains hyphen-aware boundary regex [\\w-]+', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
// Locate the Node script block in the compute_file_scope step
const scriptStart = src.indexOf('const files = [];');
assert.ok(scriptStart !== -1, 'compute_file_scope script must contain "const files = [];"');
const scriptEnd = src.indexOf('if (files.length)', scriptStart);
const scriptSection = src.slice(scriptStart, scriptEnd);
// Must use [\\w-]+ (hyphen-aware) not \\w+ only
const hasHyphenAwareRegex = scriptSection.includes('[\\\\w-]') || scriptSection.includes('[\\w-]');
assert.ok(
hasHyphenAwareRegex,
'compute_file_scope boundary regex must be hyphen-aware ([\\w-]+), found section:\n' + scriptSection
);
});
// Docs-parity: the workflow .md must contain the em-dash and parenthetical stripping.
test('code-review.md contains em-dash split and parenthetical strip in script body', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
const scriptStart = src.indexOf('const files = [];');
const scriptEnd = src.indexOf('if (files.length)', scriptStart);
const scriptSection = src.slice(scriptStart, scriptEnd);
assert.ok(
scriptSection.includes('replace(/\\s+\\([^)]*\\)\\s*$/, \'\')'),
'Script must strip parentheticals with replace(/\\s+\\([^)]*\\)\\s*$/, \'\')'
);
assert.ok(
scriptSection.includes('split(/\\s+—\\s'),
'Script must split on em-dash to strip narrative'
);
});
// #2666 docs-parity: the shipped workflow must NOT still carry the buggy
// AND-joined predicate that required BOTH a `/` and a trailing extension —
// that predicate dropped every root-level file and every extensionless build
// file. Catches a revert of the #2666 fix.
test('#2666 docs-parity: compute_file_scope does not contain the buggy slash-and-extension predicate', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
const scriptStart = src.indexOf('const files = [];');
const scriptEnd = src.indexOf('if (files.length)', scriptStart);
const scriptSection = src.slice(scriptStart, scriptEnd);
assert.ok(
!scriptSection.includes('/\\//.test(raw) && /\\.[A-Za-z0-9]+$/.test(raw)'),
'compute_file_scope must not use the buggy AND-joined /\\//.test(raw) && /\\.[A-Za-z0-9]+$/.test(raw) ' +
'predicate (#2666) — it drops every root-level and extensionless build file. Found section:\n' +
scriptSection
);
});
// #2666 docs-parity: the shipped workflow must reference the known
// extensionless build filenames so Dockerfile/Makefile/etc. are accepted.
test('#2666 docs-parity: compute_file_scope accepts known extensionless build files (Dockerfile)', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
const scriptStart = src.indexOf('const files = [];');
const scriptEnd = src.indexOf('if (files.length)', scriptStart);
const scriptSection = src.slice(scriptStart, scriptEnd);
assert.ok(
/dockerfile/i.test(scriptSection),
'compute_file_scope must reference known extensionless build filenames (e.g. Dockerfile) ' +
'so they are not dropped (#2666). Found section:\n' + scriptSection
);
});
// #2666 docs-parity: the Tier-3 git-diff fallback must intersect with the
// SUMMARY scope and warn on dropped files (not only fire on zero Tier-2 hits).
test('#2666 docs-parity: Tier-3 intersects/warns against git diff --name-only', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
// The shipped workflow must compute git diff --name-only AND emit a warning
// when the diff contains files the SUMMARY extractor did not surface.
assert.ok(
src.includes('git diff --name-only'),
'code-review.md must run `git diff --name-only` to cross-check the SUMMARY scope (#2666)'
);
assert.ok(
/warn|missing|not surfaced|did not|not in/i.test(src),
'code-review.md must warn when git diff contains files the SUMMARY extractor dropped (#2666)'
);
});
// #2666 docs-parity: the membership test must be EXACT whole-line matching
// (grep -Fxq), not an unanchored `case` substring match — otherwise a short
// basename in the diff (root `Dockerfile`) substring-matches a longer scoped
// path (`docker/Dockerfile`) and is silently skipped, reintroducing the bug.
test('#2666 docs-parity: Tier-3 cross-check uses exact whole-line matching (grep -Fxq), not substring case', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
assert.ok(
src.includes('grep -Fxq'),
'code-review.md Tier-3 cross-check must use grep -Fxq (exact whole-line match) for membership ' +
'testing, not an unanchored `case` substring match that would skip a root `Dockerfile` ' +
'whose name appears as a suffix of an already-scoped `docker/Dockerfile` (#2666)'
);
// The unanchored substring `case "$IN_SCOPE" in` membership test must NOT be
// present — it would false-match a basename suffix. Plain substring check (no
// regex, so no CRLF-fragility): the grep -Fxq positive guard above proves the
// correct mechanism; this negative guard catches a revert to the `case` form.
assert.ok(
!src.includes('case "$IN_SCOPE"'),
'code-review.md Tier-3 must not use the unanchored `case "$IN_SCOPE"` substring membership ' +
'test (#2666) — use grep -Fxq for exact whole-line matching'
);
});
});
// ---------------------------------------------------------------------------
// BUG 2 — severity-label parser: present_results must accept both `critical:`
// and `blocker:` as Critical-tier frontmatter keys.
// ---------------------------------------------------------------------------
describe('Bug 2 — present_results severity-label parser', () => {
test('frontmatter with blocker: 8 is parsed as critical: 8', () => {
const frontmatter = [
'phase: 03-courses',
'reviewed: 2025-01-01T00:00:00Z',
'findings:',
' blocker: 8',
' warning: 2',
' info: 0',
' total: 10',
'status: issues_found',
].join('\n');
const result = parseFrontmatterCritical(frontmatter);
assert.strictEqual(result.critical, 8);
});
test('frontmatter with critical: 5 is parsed as critical: 5', () => {
const frontmatter = [
'phase: 03-courses',
'reviewed: 2025-01-01T00:00:00Z',
'findings:',
' critical: 5',
' warning: 1',
' info: 0',
' total: 6',
'status: issues_found',
].join('\n');
const result = parseFrontmatterCritical(frontmatter);
assert.strictEqual(result.critical, 5);
});
test('frontmatter with neither critical nor blocker returns 0', () => {
const frontmatter = [
'phase: 03-courses',
'findings:',
' warning: 3',
' info: 1',
' total: 4',
'status: issues_found',
].join('\n');
const result = parseFrontmatterCritical(frontmatter);
assert.strictEqual(result.critical, 0);
});
// Docs-parity: the workflow .md must contain the updated grep pattern.
test('code-review.md present_results grep accepts both critical and blocker labels', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
assert.ok(
src.includes('grep -E "^[[:space:]]*(critical|blocker):"'),
'code-review.md present_results must grep for both critical: and blocker: labels'
);
});
// Docs-parity: the workflow .md must contain the updated grep for BL- headings.
test('code-review.md present_results grep includes BL- headings alongside CR- and WR-', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
assert.ok(
src.includes('### BL-') && src.includes('### CR-') && src.includes('### WR-'),
'code-review.md present_results must grep for BL- alongside CR- and WR- headings'
);
});
});
// ---------------------------------------------------------------------------
// BUG 3 — fixer agent ID alphabet and filter sets must include BL-* alongside CR-*.
// ---------------------------------------------------------------------------
describe('Bug 3 — gsd-code-fixer BL-* inclusion in filter sets', () => {
test('finding_parser documents BL-\\d+ as Critical-tier-equivalent', () => {
const src = fs.readFileSync(FIXER_PATH, 'utf8');
const parserStart = src.indexOf('<finding_parser>');
const parserEnd = src.indexOf('</finding_parser>');
assert.ok(parserStart !== -1, 'gsd-code-fixer.md must have a <finding_parser> block');
const parserSection = src.slice(parserStart, parserEnd);
assert.ok(
parserSection.includes('BL-'),
'finding_parser block must document BL-* as a Critical-tier-equivalent ID prefix'
);
});
test('parse_findings step documents severity as "Critical (CR-* or BL-*)"', () => {
const src = fs.readFileSync(FIXER_PATH, 'utf8');
const stepStart = src.indexOf('<step name="parse_findings">');
const stepEnd = src.indexOf('</step>', stepStart);
assert.ok(stepStart !== -1, 'gsd-code-fixer.md must have a parse_findings step');
const stepSection = src.slice(stepStart, stepEnd);
assert.ok(
stepSection.includes('CR-* or BL-*') || stepSection.includes('CR-* and BL-*'),
'parse_findings step must describe Critical severity as "CR-* or BL-*"'
);
});
test('critical_warning filter set includes BL-* alongside CR-* and WR-*', () => {
const src = fs.readFileSync(FIXER_PATH, 'utf8');
const stepStart = src.indexOf('<step name="parse_findings">');
const stepEnd = src.indexOf('</step>', stepStart);
const stepSection = src.slice(stepStart, stepEnd);
const critWarningIdx = stepSection.indexOf('critical_warning');
assert.ok(critWarningIdx !== -1, 'parse_findings must define critical_warning filter');
const lineStart = stepSection.lastIndexOf('\n', critWarningIdx);
const lineEnd = stepSection.indexOf('\n', critWarningIdx);
const filterLine = stepSection.slice(lineStart, lineEnd);
assert.ok(
filterLine.includes('BL-'),
'critical_warning filter line must include BL-*: ' + filterLine.trim()
);
});
test('sort order description mentions both CR-* and BL-* for Critical tier', () => {
const src = fs.readFileSync(FIXER_PATH, 'utf8');
const stepStart = src.indexOf('<step name="parse_findings">');
const stepEnd = src.indexOf('</step>', stepStart);
const stepSection = src.slice(stepStart, stepEnd);
assert.ok(
stepSection.includes('BL-'),
'parse_findings sort-order description must mention BL-* as Critical-tier alongside CR-*'
);
});
});
// ---------------------------------------------------------------------------
// REVIEWER CONTRACT — gsd-code-reviewer.md must acknowledge BL-/blocker: as
// an accepted alternative to CR-/critical: (tier-equivalent).
// ---------------------------------------------------------------------------
describe('Reviewer contract — gsd-code-reviewer.md label-equivalence', () => {
test('write_review step documents blocker: as accepted alternative to critical:', () => {
const src = fs.readFileSync(REVIEWER_PATH, 'utf8');
const stepStart = src.indexOf('<step name="write_review">');
const stepEnd = src.indexOf('</step>', stepStart);
assert.ok(stepStart !== -1, 'gsd-code-reviewer.md must have a write_review step');
const stepSection = src.slice(stepStart, stepEnd);
assert.ok(
stepSection.includes('blocker'),
'write_review step must acknowledge blocker: as a tier-equivalent alternative to critical:'
);
});
test('write_review step acknowledges BL- finding ID prefix as Critical-tier-equivalent', () => {
const src = fs.readFileSync(REVIEWER_PATH, 'utf8');
const stepStart = src.indexOf('<step name="write_review">');
const stepEnd = src.indexOf('</step>', stepStart);
const stepSection = src.slice(stepStart, stepEnd);
assert.ok(
stepSection.includes('BL-'),
'write_review step must acknowledge BL- as a Critical-tier-equivalent finding ID prefix'
);
});
});
// ---------------------------------------------------------------------------
// BUG 4 (#2352) — compute_file_scope must tilde-expand `~/...`-prefixed
// SUMMARY.md key-files entries BEFORE the "Filter deleted files" existence
// check. Bash only tilde-expands a literal `~` written in source text, never
// one arriving as the value of an already-expanded variable — so a real file
// recorded as `~/.claude/gsd-core/workflows/verify-phase.md` was silently
// misclassified as deleted and dropped from REVIEW_FILES, and a phase whose
// every recorded file used a `~/...` path hit the empty-scope skip
// ("No source files changed ... Skipping review.") as a false negative.
//
// Tested both ways: a docs-parity assertion (cross-platform, pure fs read)
// that the normalization block exists in the deployed workflow text, and a
// behavioral test that extracts the actual "Expand tilde paths" +
// "Filter deleted files" bash blocks from code-review.md and executes them
// via a real bash subprocess against planted files under a fresh HOME.
// ---------------------------------------------------------------------------
describe('Bug 4 (#2352) — compute_file_scope tilde-path expansion', () => {
// Docs-parity: the workflow .md must contain the tilde-normalization block
// as step 1 of "Post-processing (all tiers)", ahead of the deleted-file
// filter, so what we behaviorally test below is what is actually deployed.
test('code-review.md contains a tilde-expansion block ahead of the deleted-file filter', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf8');
const postProcessingIdx = src.indexOf('**Post-processing (all tiers):**');
assert.ok(postProcessingIdx !== -1, 'code-review.md must have a "Post-processing (all tiers)" section');
const expandIdx = src.indexOf('EXPANDED_FILES=()', postProcessingIdx);
assert.ok(expandIdx !== -1, 'Post-processing must contain an EXPANDED_FILES=() tilde-expansion loop');
const caseIdx = src.indexOf('case "$file" in', postProcessingIdx);
assert.ok(caseIdx !== -1 && caseIdx < expandIdx + 400, 'tilde-expansion loop must use a case "$file" in match');
assert.ok(
src.slice(caseIdx, caseIdx + 200).includes('"~/"*)') &&
src.slice(caseIdx, caseIdx + 200).includes('${HOME}${file#\\~}'),
'tilde-expansion loop must rewrite a leading ~/ to ${HOME}/... via ${file#\\~}'
);
const deletedFilterIdx = src.indexOf('DELETED_COUNT=0', postProcessingIdx);
assert.ok(deletedFilterIdx !== -1, 'Post-processing must still contain the deleted-file filter');
assert.ok(
expandIdx < deletedFilterIdx,
'tilde-expansion loop must run BEFORE the deleted-file filter, not after'
);
});
// Extract the tilde-expansion fence and the (non-adjacent — the exclusions
// filter sits between them) deleted-file-filter fence from the
// "Post-processing (all tiers)" section of code-review.md — the exact
// snippets the runtime executes, located by content anchor rather than
// position so an intervening step doesn't silently swap in the wrong
// block — and glue them behind a synthetic REVIEW_FILES=("$@") seed for
// direct execution. The exclusions filter itself is intentionally skipped
// here: it only matches relative planning-artifact paths and is orthogonal
// to tilde expansion (see code-review.md step 2, "Apply exclusions").
function extractPostProcessingScript() {
// readFileNormalized() strips \r\n -> \n before either fence below is
// sliced out and later spawned via spawnSync('bash', ...) in
// runPostProcessing() — an un-normalized read on a Windows checkout would
// break bash mid-script (DEFECT.TEST-SHELL-PIPELINE-NONPORTABLE, #2650).
const src = readFileNormalized(WORKFLOW_PATH);
const postProcessingIdx = src.indexOf('**Post-processing (all tiers):**');
assert.ok(postProcessingIdx !== -1, 'code-review.md must have a "Post-processing (all tiers)" section');
function fenceContaining(marker) {
const markerIdx = src.indexOf(marker, postProcessingIdx);
assert.ok(markerIdx !== -1, `expected to find "${marker}" in the Post-processing section`);
const fenceStart = src.lastIndexOf('```bash', markerIdx);
assert.ok(fenceStart !== -1 && fenceStart > postProcessingIdx, `no \`\`\`bash fence before "${marker}"`);
const bodyStart = src.indexOf('\n', fenceStart) + 1;
const fenceEnd = src.indexOf('\n```', bodyStart);
assert.ok(fenceEnd !== -1, `unterminated \`\`\`bash fence containing "${marker}"`);
return src.slice(bodyStart, fenceEnd);
}
const tildeBlock = fenceContaining('EXPANDED_FILES=()');
const deletedBlock = fenceContaining('DELETED_COUNT=0');
return [
'REVIEW_FILES=("$@")',
tildeBlock,
deletedBlock,
'printf "%s\\n" "${REVIEW_FILES[@]}"',
'echo "REVIEW_FILES_COUNT=${#REVIEW_FILES[@]}"',
'echo "DELETED_COUNT=$DELETED_COUNT"',
].join('\n');
}
function runPostProcessing(homeDir, files) {
const script = extractPostProcessingScript();
// "bash" as $0 so the real REVIEW_FILES entries land in "$@" from $1.
return toLegacyResult(
runHook('-c', [script, 'bash', ...files], {
interpreter: 'bash',
env: { ...process.env, HOME: homeDir },
timeoutMs: PROBE_TIMEOUT_MS,
})
);
}
let tmpHome;
test('setup: plant a fresh HOME with a real file', { skip: process.platform === 'win32' }, () => {
tmpHome = createTempDir('gsd-2352-home-');
fs.mkdirSync(path.join(tmpHome, '.claude', 'gsd-core', 'workflows'), { recursive: true });
fs.writeFileSync(
path.join(tmpHome, '.claude', 'gsd-core', 'workflows', 'verify-phase.md'),
'# real file\n',
'utf8'
);
});
test(
'AC1: a ~/-prefixed path to a real file survives and is not counted deleted',
{ skip: process.platform === 'win32' },
() => {
const result = runPostProcessing(tmpHome, ['~/.claude/gsd-core/workflows/verify-phase.md']);
assert.equal(result.status, 0, `snippet exited ${result.status}; stderr=${result.stderr}`);
assert.match(
result.stdout,
new RegExp(path.join(tmpHome, '.claude', 'gsd-core', 'workflows', 'verify-phase.md').replace(/[/\\.]/g, '\\$&')),
`expected expanded absolute path in surviving REVIEW_FILES; got: ${JSON.stringify(result.stdout)}`
);
assert.match(result.stdout, /DELETED_COUNT=0/, `expected DELETED_COUNT=0; got: ${JSON.stringify(result.stdout)}`);
assert.match(
result.stdout,
/REVIEW_FILES_COUNT=1/,
`expected the tilde path to survive into REVIEW_FILES; got: ${JSON.stringify(result.stdout)}`
);
}
);
test(
'AC2: a ~/-prefixed path to a non-existent file is still correctly excluded as deleted',
{ skip: process.platform === 'win32' },
() => {
const result = runPostProcessing(tmpHome, ['~/.claude/gsd-core/workflows/does-not-exist.md']);
assert.equal(result.status, 0, `snippet exited ${result.status}; stderr=${result.stderr}`);
assert.match(result.stdout, /DELETED_COUNT=1/, `expected DELETED_COUNT=1; got: ${JSON.stringify(result.stdout)}`);
assert.match(
result.stdout,
/REVIEW_FILES_COUNT=0/,
`expected the missing tilde path to be dropped; got: ${JSON.stringify(result.stdout)}`
);
}
);
test(
'AC3: a phase where every recorded file is a real ~/-prefixed path does not empty the scope',
{ skip: process.platform === 'win32' },
() => {
const result = runPostProcessing(tmpHome, ['~/.claude/gsd-core/workflows/verify-phase.md']);
assert.equal(result.status, 0, `snippet exited ${result.status}; stderr=${result.stderr}`);
const countMatch = result.stdout.match(/REVIEW_FILES_COUNT=(\d+)/);
assert.ok(countMatch, `expected a REVIEW_FILES_COUNT line; got: ${JSON.stringify(result.stdout)}`);
assert.ok(
Number(countMatch[1]) > 0,
'an all-tilde real-file scope must not reduce to zero (would trigger the empty-scope skip)'
);
}
);
test(
'AC4: mixed tilde + missing ordinary relative path resolve independently',
{ skip: process.platform === 'win32' },
() => {
const result = runPostProcessing(tmpHome, [
'~/.claude/gsd-core/workflows/verify-phase.md',
'this/relative/path/does-not-exist.md',
]);
assert.equal(result.status, 0, `snippet exited ${result.status}; stderr=${result.stderr}`);
assert.match(result.stdout, /DELETED_COUNT=1/, `expected exactly 1 deleted; got: ${JSON.stringify(result.stdout)}`);
assert.match(
result.stdout,
/REVIEW_FILES_COUNT=1/,
`expected only the tilde path to survive; got: ${JSON.stringify(result.stdout)}`
);
assert.doesNotMatch(
result.stdout,
/this\/relative\/path\/does-not-exist\.md/,
'the missing ordinary relative path must not survive into REVIEW_FILES'
);
}
);
test('teardown: remove the temp HOME', { skip: process.platform === 'win32' }, () => {
cleanup(tmpHome);
});
});
// ---------------------------------------------------------------------------
// Shared diff-base extraction/execution helpers (Bug 5 #3191, Bug 6 #3503).
//
// The workflow computes "the phase's base commit" in three independent bash
// invocations (each <step> is its own shell): the Tier-3 file-scope fallback
// (compute_file_scope), the agent-context DIFF_BASE (spawn_reviewer), and the
// fallow pre-pass's --changed-since base (structural-pre-pass.md).
//
// Behavioral style follows Bug 4: extract the SHIPPED bash from the workflow
// .md files by content anchor and execute it via a real bash subprocess
// against a git fixture — so the assertion binds the deployed text, not a
// JS reimplementation. Running the real `git log` (not a regex shim) is what
// makes platform-level regex holes (the #3191 macOS `\b` no-op) visible.
// ---------------------------------------------------------------------------
// The ```bash fence containing `marker`, located after `fromIdx`.
function fenceContaining(src, marker, fromIdx = 0) {
const markerIdx = src.indexOf(marker, fromIdx);
assert.ok(markerIdx !== -1, `expected to find "${marker}" in workflow source`);
const fenceStart = src.lastIndexOf('```bash', markerIdx);
assert.ok(fenceStart !== -1, `no \`\`\`bash fence before "${marker}"`);
const bodyStart = src.indexOf('\n', fenceStart) + 1;
const fenceEnd = src.indexOf('\n```', bodyStart);
assert.ok(fenceEnd !== -1, `unterminated \`\`\`bash fence containing "${marker}"`);
return src.slice(bodyStart, fenceEnd);
}
// The Tier-3 derivation prefix: fence start up to the REVIEW_FILES branch.
function extractTier3Derivation() {
const src = readFileNormalized(WORKFLOW_PATH);
const fence = fenceContaining(src, '# Compute diff base from phase commits');
const cut = fence.indexOf('if [ ${#REVIEW_FILES[@]} -eq 0 ]');
assert.ok(cut !== -1, 'Tier-3 fence must contain the REVIEW_FILES empty-scope branch');
return fence.slice(0, cut);
}
// spawn_reviewer's whole DIFF_BASE fence.
function extractSpawnReviewerDerivation() {
const src = readFileNormalized(WORKFLOW_PATH);
const spawnIdx = src.indexOf('<step name="spawn_reviewer">');
assert.ok(spawnIdx !== -1, 'code-review.md must have a spawn_reviewer step');
return fenceContaining(src, 'PHASE_COMMITS=$(git log', spawnIdx);
}
// The fallow phase-scope derivation, from the step fragment. The fragment
// carries markdown-escaped quotes (\") in this fence — an authoring
// artifact that survived #2994 fragmentization verbatim; the runtime agent
// normalizes them when transcribing, so the test does the same before
// executing. Sliced from FALLOW_SCOPE_ARGS=() (skipping the gsd-tools
// runtime resolver line above it, which exits 1 on machines without an
// installed gsd-tools and is orthogonal to the base-derivation under test)
// to just before the gsd_run invocation (which needs the real binary).
function extractFallowDerivation() {
const src = readFileNormalized(PRE_PASS_STEP_PATH);
const fence = fenceContaining(src, 'FALLOW_PHASE_COMMITS=$(git log');
const scopeStart = fence.indexOf('FALLOW_SCOPE_ARGS=()');
assert.ok(scopeStart !== -1, 'fallow fence must define FALLOW_SCOPE_ARGS=()');
const cut = fence.indexOf('gsd_run run-with-timeout');
assert.ok(cut !== -1, 'fallow fence must contain the gsd_run run-with-timeout call');
assert.ok(scopeStart < cut, 'FALLOW_SCOPE_ARGS must precede the gsd_run invocation');
return fence.slice(scopeStart, cut).replace(/\\"/g, '"');
}
// Execute a derivation snippet with PADDED_PHASE (and the fallow scope gate)
// set, echoing the values it computes between sentinels so multi-line
// PHASE_COMMITS parse cleanly.
function runDerivation(repo, snippet, phase) {
const script = [
`PADDED_PHASE=${phase}`,
'FALLOW_SCOPE=phase',
snippet,
'echo "===PHASE_COMMITS==="',
'printf \'%s\\n\' "$PHASE_COMMITS"',
'echo "===DIFF_BASE==="',
'printf \'%s\\n\' "$DIFF_BASE"',
'echo "===FALLOW_BASE==="',
'printf \'%s\\n\' "$FALLOW_BASE"',
'echo "===END==="',
].join('\n');
// Bash FAN-OUT: the extracted snippet runs `git log` plus an `echo | tail`
// pipe — the wrong class for `PROBE_TIMEOUT_MS` (a single short CLI
// probe). Same class as the observed CI failures in
// tests/quick-branching.test.cjs (PR #3787 run 32668773524) and
// tests/worktree-safety.test.cjs (`next` run 32608945654). See
// HOOK_FANOUT_TIMEOUT_MS in ./helpers/timeouts.cjs for the class
// rationale.
return toLegacyResult(
runHook('-c', [script, 'bash'], {
interpreter: 'bash',
cwd: repo,
timeoutMs: HOOK_FANOUT_TIMEOUT_MS,
})
);
}
function parseSentinel(stdout, name) {
const m = stdout.match(new RegExp(`===${name}===\\n([\\s\\S]*?)\\n===`));
if (!m) return null;
return m[1].split('\n').map((l) => l.trim()).filter((l) => l.length > 0);
}
// ---------------------------------------------------------------------------
// Bug 5 (#3191) — EVERY diff-base derivation must use the same anchored,
// portable derivation.
//
// The workflow computes "the phase's base commit" in three independent bash
// invocations (each <step> is its own shell): the Tier-3 file-scope fallback
// (compute_file_scope), the agent-context DIFF_BASE (spawn_reviewer), and the
// fallow pre-pass's --changed-since base (structural-pre-pass.md). #2989
// anchored only the Tier-3 copy — and did so with `\b`, which is not a POSIX
// ERE token, so on macOS (regex(3)) that grep matches NOTHING and Tier 3
// always fails closed. The other two sites kept the original unanchored
// `--grep="${PADDED_PHASE}"`, whose oldest substring match is routinely a
// version-string/date commit from months before the phase existed.
// (#3503 later replaced the anchor itself — a subject-line conventional-
// commit scope match instead of the "[Pp]hase N" prose phrase, which GSD's
// own commits never contain; see Bug 6. The lockstep + portability +
// fail-closed contract THIS block verifies is unchanged.)
//
// Behavioral style follows Bug 4: extract the SHIPPED bash from the workflow
// .md files by content anchor and execute it via a real bash subprocess
// against a git fixture — so the assertion binds the deployed text, not a
// JS reimplementation. Running the real `git log` (not a regex shim) is what
// keeps platform-level regex holes (the #3191 macOS `\b` no-op) visible.
// ---------------------------------------------------------------------------
describe('Bug 5 (#3191) — same anchored, portable phase-scope grep at all three diff-base sites', () => {
const SKIP_WIN32 = { skip: process.platform === 'win32' };
// Fixture: five commits whose messages exercise every false-match class
// from the issue — version string + date, another phase's plan whose scope
// number is a digit-superset, a prose "Phase N" mention in another phase's
// subject — plus the phase's real first scope commit and an unrelated HEAD.
function buildFixture(prefix, phaseCommitMessage) {
const repo = createTempGitProject(prefix);
const commits = [
['c1.txt', 'chore: bump to v2.06.0 on 2026-01-05'],
['c2.txt', 'docs(60-01): unrelated phase-plan work'],
['c3.txt', phaseCommitMessage],
['c4.txt', 'chore: Phase 60 cleanup'],
['c5.txt', 'docs: touch README'],
];
const hashes = {};
for (const [file, message] of commits) {
fs.writeFileSync(path.join(repo, file), `${message}\n`);
gitOrThrow(['add', file], { cwd: repo, timeoutMs: GIT_TIMEOUT_MS });
gitOrThrow(['commit', '-m', message], { cwd: repo, timeoutMs: GIT_TIMEOUT_MS });
hashes[file] = gitOrThrow(['rev-parse', 'HEAD'], { cwd: repo, timeoutMs: GIT_TIMEOUT_MS }).trim();
}
return { repo, hashes };
}
test(
'T1 + T4: Tier-3 derivation matches ONLY the phase\'s real scope commit — never a digit-substring or superset hit',
SKIP_WIN32,
() => {
const { repo, hashes } = buildFixture('gsd-3191-tier3-', 'docs(06): capture phase context');
try {
const result = runDerivation(repo, extractTier3Derivation(), '06');
assert.equal(result.status, 0, `snippet exited ${result.status}; stderr=${result.stderr}`);
const phaseCommits = parseSentinel(result.stdout, 'PHASE_COMMITS');
const diffBase = parseSentinel(result.stdout, 'DIFF_BASE');
// AC: the phase's real commits are a small minority of digit-containing
// commits; the derivation must resolve to an ancestor near the phase's
// actual first commit (c3^) — never the older v2.06.0/docs(06-01) hits.
assert.deepStrictEqual(
phaseCommits,
[hashes['c3.txt']],
`Tier-3 grep must match only the phase's real scope commit; got: ${JSON.stringify(phaseCommits)}`
);
assert.deepStrictEqual(
diffBase,
[`${hashes['c3.txt']}^`],
'Tier-3 DIFF_BASE must be the phase first-commit parent'
);
} finally {
cleanup(repo);
}
}
);
test(
'T2: spawn_reviewer DIFF_BASE derivation uses the same anchored grep (not the bare digit)',
SKIP_WIN32,
() => {
const { repo, hashes } = buildFixture('gsd-3191-spawn-', 'docs(06): capture phase context');
try {
const result = runDerivation(repo, extractSpawnReviewerDerivation(), '06');
assert.equal(result.status, 0, `snippet exited ${result.status}; stderr=${result.stderr}`);
const phaseCommits = parseSentinel(result.stdout, 'PHASE_COMMITS');
const diffBase = parseSentinel(result.stdout, 'DIFF_BASE');
// Pre-fix this matches c1 and c2 as well and tail -1 picks c1 — the
// oldest unrelated match — feeding a bogus diff_base to the reviewer
// agent exactly when files: is empty (the fail-closed scenario).
assert.deepStrictEqual(
phaseCommits,
[hashes['c3.txt']],
`spawn_reviewer grep must match only the phase's real scope commit; got: ${JSON.stringify(phaseCommits)}`
);
assert.deepStrictEqual(
diffBase,
[`${hashes['c3.txt']}^`],
'spawn_reviewer DIFF_BASE must be the phase first-commit parent'
);
} finally {
cleanup(repo);
}
}
);
test(
'T3: fallow phase scope derives --changed-since from the anchored grep, never an old substring match',
SKIP_WIN32,
() => {
const { repo, hashes } = buildFixture('gsd-3191-fallow-', 'docs(06): capture phase context');
try {
const result = runDerivation(repo, extractFallowDerivation(), '06');
assert.equal(result.status, 0, `snippet exited ${result.status}; stderr=${result.stderr}`);
const fallowBase = parseSentinel(result.stdout, 'FALLOW_BASE');
// Pre-fix the unanchored grep's oldest match is the v2.06.0 commit, so
// FALLOW_SCOPE_ARGS resolves to --changed-since <old-unrelated-commit>
// and widens the structural pre-pass far beyond the phase.
assert.deepStrictEqual(
fallowBase,
[`${hashes['c3.txt']}^`],
`FALLOW_BASE must be the phase first-commit parent, got: ${JSON.stringify(fallowBase)}`
);
} finally {
cleanup(repo);
}
}
);
test(
'T5: with no genuine phase scope commit, every derivation yields NO base (fail-closed preserved)',
SKIP_WIN32,
() => {
const { repo } = buildFixture('gsd-3191-closed-', 'feat: scanner core'); // no phase-06 scope commit anywhere
try {
for (const [label, snippet] of [
['tier3', extractTier3Derivation()],
['spawn_reviewer', extractSpawnReviewerDerivation()],
['fallow', extractFallowDerivation()],
]) {
const result = runDerivation(repo, snippet, '06');
assert.equal(result.status, 0, `${label} exited ${result.status}; stderr=${result.stderr}`);
const phaseCommits = parseSentinel(result.stdout, 'PHASE_COMMITS');
const diffBase = parseSentinel(result.stdout, 'DIFF_BASE');
const fallowBase = parseSentinel(result.stdout, 'FALLOW_BASE');
assert.deepStrictEqual(phaseCommits, [], `${label}: no substring-only matches may survive`);
assert.deepStrictEqual(diffBase, [], `${label}: DIFF_BASE must stay empty (no bogus base)`);
assert.deepStrictEqual(fallowBase, [], `${label}: FALLOW_BASE must stay unset`);
}
} finally {
cleanup(repo);
}
}
);
// T6 docs-parity anti-revert: every `git log --grep` derivation in both
// files must use the SAME (#3191 lockstep) #3503 scope-anchored pattern — a
// subject-line conventional-commit phase scope, both padded and unpadded
// spellings via PHASE_SCOPE_NUM — and must not use `\b` under
// --extended-regexp (which silently no-ops on macOS regex(3)).
test('T6 docs-parity: all git-log grep derivations use the identical scope-anchored, POSIX-portable pattern', () => {
const sources = [
readFileNormalized(WORKFLOW_PATH),
readFileNormalized(PRE_PASS_STEP_PATH).replace(/\\"/g, '"'),
];
const grepLines = [];
for (const src of sources) {
for (const m of src.matchAll(/^\s*[A-Z_]+=\$\(git log[^\n]*--grep=[^\n]*$/gm)) {
grepLines.push(m[0]);
}
}
assert.ok(
grepLines.length >= 3,
`expected at least 3 git-log grep derivation sites (Tier 3, spawn_reviewer, fallow); found ${grepLines.length}`
);
const SCOPE_GREP = '--grep="^[[:alpha:]]+!?\\((phase-)?(${PHASE_SCOPE_NUM})(-[0-9]+)?\\)!?:"';
for (const line of grepLines) {
assert.ok(
line.includes(SCOPE_GREP),
`grep derivation must be anchored to GSD's own conventional-commit phase scope (#3503), not free prose:\n${line}`
);
assert.ok(
line.includes('--extended-regexp'),
`grep derivation must pass --extended-regexp:\n${line}`
);
assert.ok(
!line.includes('\\b'),
`grep derivation must not use \\b under --extended-regexp — it is not POSIX ERE and silently matches nothing on macOS (#3191):\n${line}`
);
}
// Lockstep (#3191): all three sites must carry byte-identical grep text —
// and the padded/unpadded PHASE_SCOPE_NUM prep that feeds it.
for (const src of sources) {
assert.ok(
src.includes('PHASE_SCOPE_NUM="${PADDED_PHASE}"'),
'each file must derive PHASE_SCOPE_NUM from PADDED_PHASE (padded/unpadded alternation)'
);
assert.ok(
src.includes('0[0-9]*) PHASE_SCOPE_NUM="${PADDED_PHASE#0}|${PADDED_PHASE}"'),
'each file must accept the UNPADDED phase spelling GSD workflows emit (docs(phase-6):)'
);
}
});
});
// ---------------------------------------------------------------------------
// Bug 6 (#3503) — the diff-base grep must key on GSD's own commit scopes,
// not free prose.
//
// The #2989/#3191 anchor ("[Pp]hase N" + POSIX boundary) still resolves the
// base ~4 phases early on real repos: `git log --grep` searches FULL commit
// bodies, and `tail -1` deliberately keeps the OLDEST match — so a single
// prose mention of the phase anywhere in history (a planning commit that
// forward-references it: "deferred to Phase N per D-09"; a doc commit that
// uses "### Phase N" as a format EXAMPLE) silently captures the base, while
// GSD's own commits — which never contain the literal "Phase N", they use
// conventional-commit scopes: docs(phase-6): from execute-phase.md,
// feat(6-01):/test(6-01): from references/tdd.md, docs(6): plan commits —
// are matched by nothing. The wrong base silently inflates the reviewer's
// reading list (the #2666 SUMMARY/diff union) and widens fallow's
// --changed-since with no warning.
//
// Same behavioral style as Bug 5: the SHIPPED bash is extracted from the
// workflow .md files by content anchor and executed against a real git
// fixture whose history contains every false-positive class from the issue,
// in commit BODIES (which is where the old pattern's damage lives).
// ---------------------------------------------------------------------------
describe('Bug 6 (#3503) — diff base keys on GSD commit scopes, not prose mentions', () => {
const SKIP_WIN32 = { skip: process.platform === 'win32' };
// Commit [file, subject, body?] tuples; bodies use a second -m so they are
// real commit bodies (what `git log --grep` searches beyond the subject).
function buildHistory(prefix, commits) {
const repo = createTempGitProject(prefix);
const hashes = {};
for (const [file, subject, body] of commits) {
fs.writeFileSync(path.join(repo, file), `${subject}\n`);
gitOrThrow(['add', file], { cwd: repo, timeoutMs: GIT_TIMEOUT_MS });
const args = body === undefined
? ['commit', '-m', subject]
: ['commit', '-m', subject, '-m', body];
gitOrThrow(args, { cwd: repo, timeoutMs: GIT_TIMEOUT_MS });
hashes[file] = gitOrThrow(['rev-parse', 'HEAD'], { cwd: repo, timeoutMs: GIT_TIMEOUT_MS }).trim();
}
return { repo, hashes };
}
// The #3503 repro history: every prose false-positive class from the issue
// — a version-string digit substring, a planning commit whose BODY
// forward-references the phase, a doc commit whose BODY uses "### Phase N"
// as a format example — followed by the phase's GENUINE scope-style commits
// in all three spellings GSD emits (padded docs(06):, plan feat(06-01):,
// and the UNPADDED docs(phase-6): that execute-phase.md actually writes,
// since workflows interpolate the unpadded roadmap number while
// PADDED_PHASE is zero-padded).
const REPRO_HISTORY = [
['c1.txt', 'chore: bump to v2.06.0 on 2026-01-05'],
['c2.txt', 'feat(60-01): probe wiring', 'The EF path still uses it, fenced to Phase 06 per D-09.'],
['c3.txt', 'docs: commit message format', 'Phase headers use the form:\n\n### Phase 06 (Cluster B): Title\n\nin ROADMAP detail sections.'],
['c4.txt', 'docs(06): capture phase context'],
['c5.txt', 'feat(06-01): implement scanner core'],
['c6.txt', 'docs(phase-6): update tracking after wave 1'],
['c7.txt', 'docs: touch README'],
];
test(
'T1: prose forward-references and doc-format examples never capture the base — it resolves to the phase first scope commit at all three sites',
SKIP_WIN32,
() => {
const { repo, hashes } = buildHistory('gsd-3503-scope-', REPRO_HISTORY);
try {
const sites = [
['tier3', extractTier3Derivation()],
['spawn_reviewer', extractSpawnReviewerDerivation()],
['fallow', extractFallowDerivation()],
];
for (const [label, snippet] of sites) {
const result = runDerivation(repo, snippet, '06');
assert.equal(result.status, 0, `${label} exited ${result.status}; stderr=${result.stderr}`);
const phaseCommits = parseSentinel(result.stdout, 'PHASE_COMMITS');
const diffBase = parseSentinel(result.stdout, 'DIFF_BASE');
const fallowBase = parseSentinel(result.stdout, 'FALLOW_BASE');
// Pre-fix (#3503): the prose matches in c2/c3 bodies are older than
// the phase and tail -1 keeps the oldest, so DIFF_BASE resolves to
// c2^ — unboundedly before the phase — at every site. (The fallow
// snippet computes FALLOW_PHASE_COMMITS, not PHASE_COMMITS; its
// matched-set is asserted via FALLOW_BASE below.)
if (label !== 'fallow') {
assert.deepStrictEqual(
new Set(phaseCommits || []),
new Set([hashes['c4.txt'], hashes['c5.txt'], hashes['c6.txt']]),
`${label}: grep must match exactly the phase's three scope commits; got: ${JSON.stringify(phaseCommits)}`
);
}
const expected = [`${hashes['c4.txt']}^`];
if (label === 'fallow') {
assert.deepStrictEqual(
fallowBase,
expected,
`${label}: base must be the FIRST (oldest) scope commit's parent`
);
} else {
assert.deepStrictEqual(
diffBase,
expected,
`${label}: base must be the FIRST (oldest) scope commit's parent`
);
}
}
} finally {
cleanup(repo);
}
}
);
test(
'T2: unpadded scope spellings (docs(phase-6):, feat(6-01):) resolve identically — PADDED_PHASE is zero-padded but GSD emits the unpadded number',
SKIP_WIN32,
() => {
const { repo, hashes } = buildHistory('gsd-3503-unpadded-', [
['c1.txt', 'feat(60-01): probe wiring', 'Deferred to Phase 06 per D-09.'],
['c2.txt', 'docs(phase-6): capture phase context'],
['c3.txt', 'feat(6-01): implement scanner core'],
['c4.txt', 'test(6): persist human verification items as UAT'],
['c5.txt', 'docs: touch README'],
]);
try {
for (const [label, snippet] of [
['tier3', extractTier3Derivation()],
['spawn_reviewer', extractSpawnReviewerDerivation()],
['fallow', extractFallowDerivation()],
]) {
const result = runDerivation(repo, snippet, '06');
assert.equal(result.status, 0, `${label} exited ${result.status}; stderr=${result.stderr}`);
const phaseCommits = parseSentinel(result.stdout, 'PHASE_COMMITS');
const diffBase = parseSentinel(result.stdout, 'DIFF_BASE');
const fallowBase = parseSentinel(result.stdout, 'FALLOW_BASE');
// (The fallow snippet computes FALLOW_PHASE_COMMITS, not
// PHASE_COMMITS; its matched set is asserted via FALLOW_BASE below.)
if (label !== 'fallow') {
assert.deepStrictEqual(
new Set(phaseCommits || []),
new Set([hashes['c2.txt'], hashes['c3.txt'], hashes['c4.txt']]),
`${label}: unpadded scope spellings must all match; got: ${JSON.stringify(phaseCommits)}`
);
}
const expected = [`${hashes['c2.txt']}^`];
if (label === 'fallow') {
assert.deepStrictEqual(
fallowBase,
expected,
`${label}: base must be the first unpadded scope commit's parent`
);
} else {
assert.deepStrictEqual(
diffBase,
expected,
`${label}: base must be the first unpadded scope commit's parent`
);
}
}
} finally {
cleanup(repo);
}
}
);
test(
'T3: prose mentions WITHOUT any scope-style commit fail closed (no silent arbitrary base)',
SKIP_WIN32,
() => {
const { repo } = buildHistory('gsd-3503-closed-', REPRO_HISTORY.slice(0, 3).concat([
['c4.txt', 'docs: touch README'],
]));
try {
for (const [label, snippet] of [
['tier3', extractTier3Derivation()],
['spawn_reviewer', extractSpawnReviewerDerivation()],
['fallow', extractFallowDerivation()],
]) {
const result = runDerivation(repo, snippet, '06');
assert.equal(result.status, 0, `${label} exited ${result.status}; stderr=${result.stderr}`);
const phaseCommits = parseSentinel(result.stdout, 'PHASE_COMMITS');
const diffBase = parseSentinel(result.stdout, 'DIFF_BASE');
const fallowBase = parseSentinel(result.stdout, 'FALLOW_BASE');
// Pre-fix (#3503): the prose bodies match, so the derivation picks a
// bogus base instead of failing closed behind the workflow warning.
assert.deepStrictEqual(phaseCommits, [], `${label}: prose mentions may not match`);
assert.deepStrictEqual(diffBase, [], `${label}: DIFF_BASE must stay empty`);
assert.deepStrictEqual(fallowBase, [], `${label}: FALLOW_BASE must stay unset`);
}
} finally {
cleanup(repo);
}
}
);
});