A PR whose diff is entirely under docs/ runs zero tests, so a guard whose INPUT
is shipped prose cannot protect the PR lane of the diffs it exists to check. Its
only firing opportunity is after merge, on the shared branch -- which is how next
went red on dacae9273 while the PR that caused it (#3746) was green on every
check.
The docs-lint job in .github/workflows/docs-required.yml -- an ALREADY-REQUIRED
context -- now selects and runs the docs guards that read the specific docs files
the PR changed.
scripts/docs-guard-registry.cjs test file -> the docs paths it reads (63)
scripts/select-docs-guards.cjs pure (changedPaths, registry) -> test files
scripts/lint-docs-guard-registration.cjs drift guard, wired into lint:ci
scripts/ci-test-scope.cjs is NOT touched -- `git diff origin/next --` on it is
empty -- so #764's saving stands and its 21 pinning tests are untouched.
Selection: exact path; trailing-slash directory prefix (boundary-checked --
docs/adrenaline.md does NOT match docs/adr/, which a naive startsWith gets
wrong); and '*' for the 6 entries that walk docs/ generally or read a computed
path. Unknown maps to '*' -- guessing narrow is how a guard silently stops
running. Measured: a typo fix selects 6 of 63; docs/AGENTS.md selects 12;
docs/COMMANDS.md selects 18.
Four things this got wrong first, each found by an independent reviewer or by
probe, and each having been asserted safe in a comment:
1. The registry started as a RULE in ci-test-scope.cjs's RULES, on the theory
that classify()'s !codeChanged normalization made it inert. True for
docs-ONLY diffs; false for MIXED docs+code diffs, where codeChanged is true
and the normalization never runs:
node scripts/ci-test-scope.cjs --files "docs/a.md src/semver.cts"
with the RULE: 25 targeted_tests
origin/next: 3 targeted_tests
Category error: RULES is the scoped lane's input; a docs-guard registry is a
lane manifest for a consumer that never calls classify(). Extracted; pinned
by value.
2. The second attempt was a dedicated workflow with paths: [docs/**]. Such a
workflow never reports on a non-docs PR, so it can never be a required
context without hanging every non-docs PR -- and a non-required check does not
block a merge, so the guard would have been advisory and #3753 unfixed.
docs-required.yml already has no paths: filter, already supplies the required
docs-lint context, already computes docs_changed, and already ran one docs
guard gated on it. Generalizing that step needs no ruleset edit at all.
3. The registry and the drift lint were built from ONE path-segment heuristic, so
both were blind identically -- and blind at the guard that motivated the issue.
The reader-call regex required a character BEFORE its keyword, so a callee
named exactly read( / load( / parse( / doc( / file( / content( could never
match; and only an INLINE path.join(ROOT,'docs','X.md') argument was caught,
missing the two-step-via-variable form -- the MAJORITY spelling -- plus
template literals and concatenation. Detector 1 fired on 14 of ~450 files, so
35 genuine guards sat unregistered while the lint reported 0 violations,
including cursor-reviewer (reads docs/COMMANDS.md, asserts
.includes('--cursor')) and inventory-headings-countfree. The "accepted blind
spot" this shipped with was the common case, not a fringe.
4. With detection fixed the true population is 115 files: 63 genuine guards, 52
incidental. Running all 63 in a REQUIRED check on a one-line typo fix is the
cost #764 exists to avoid -- install.test.cjs is 7840 lines and reads exactly
one docs file, docs/AGENTS.md, for its frontmatter. Dropping it reproduces the
bug; running it for a typo elsewhere is waste. Hence the map.
Then a second review round found six more, all fixed here:
- fragment-single-edit-propagation.install.test.cjs was EXEMPTED as
"overlay fixture only". False: it reads the real docs/registries/eos.json and
asserts on a registry entry name, and reads the real ADR-0001 and asserts its
H1. A docs-only PR touching either would have gone green and red next -- #3753
shipping again, from inside the fix for it. Now registered against both paths,
and all 52 remaining exemptions were re-audited one by one.
- The SUITES-collision guard compared RAW registry keys, but run-tests.cjs strips
a leading `tests/` BEFORE its suite check. So it caught 'all' and missed
'tests/all' -- the only spelling that can actually occur, since every key
carries the prefix. One typo would have run all 824 test files inside the
required job. Now normalized the same way run-tests.cjs normalizes.
- The lint failed OPEN on an unreadable tests dir or candidate file: 0 violations,
ok:true. A guard that cannot read its input must never report success.
- The exemption ratchet gated identity only, so a baselined file that later
STARTED asserting on shipped docs stayed exempt silently -- 52 permanently blind
files. The baseline now fingerprints the docs paths each exempted file
references and fails when that set changes, naming what changed.
- The exemption marker was still honored inside a multi-line template literal in
the header window. The scanner now tracks template-literal and block-comment
state.
- `git diff --name-only | grep '^docs/'` silently dropped C-quoted non-ASCII docs
paths, making docs_changed=false a green zero-guard check. Both call sites now
pass -c core.quotepath=false.
- The run step was gated on hashFiles(), which a force-committed
.docs-guard-tests.txt would satisfy. The step now rm -f's both scratch files
first and gates on an output it sets itself.
Three empty states, deliberately distinct, because conflating them rebuilds
#3753: an empty or malformed registry HARD-FAILS; docs changed with no guard
covering them logs and skips; no docs change is already gated. The middle state
must never be expressed as an empty --files-from, which prints `no tests in suite
"all"` and exits 0 -- a green check that guarded nothing. With the current
registry that state is unreachable, because the six '*' entries always match;
the branch is kept as defensive handling for a future registry and says so.
timeout-minutes: 15 bounds the required job against a hanging fork-supplied test;
it had none. npm ci was added because the job never installed dependencies -- the
previous single-file step got away without it, the registry does not.
docs/contributing/docs-guard-registration.md documents the rule, following its
sibling cross-platform-portability-rules.md, and CONTRIBUTING.md's CI Test
Quality Checks table links to it. It is also load-bearing: without a docs/ file
in the diff this PR would not have triggered its own lane, shipping an
unexercised change to a required check.
One unrelated fix, included because this PR surfaced it and CLAUDE.md forbids
deferring a defect found while working. On this branch's first CI run,
`full test (windows-latest, 24, shard 3/3)` was CANCELLED at exactly 30 minutes;
tests were still passing 0.8s before the cancel, so it is a wall-clock timeout,
not a hang, and a cancelled job reddens `Required tests`.
The cause is not this PR's test file, which costs ~60ms. Shard composition is
unstable: adding ONE file to the unit suite reshuffled 115 of 268 files between
shards, and shard 3 drew a heavier mix. Underneath that is a real pre-existing
defect. tests/ci-test-job-timeout-budget.test.cjs requires every lane's budget to
be >= 1.5x its MEASURED cost -- "a lane that got slower must be re-budgeted, not
excused" -- and its test-full entry recorded 19m from a windows-22 shard. That is
stale. Measured on `next` with none of this PR's changes present: 26m18s (run
32614439702, windows-latest/24 shard 3/3), 23m36s and 23m17s on shard 2/3. So the
lane costs ~26m and the 30-minute cap carried 1.14x headroom, not 1.5x. The gate
had been out of compliance with its own rule; this PR was merely the file
addition that reshuffled shard 3 past the cliff.
Fixed as that file prescribes: measuredMinutes 19 -> 27 with fresh evidence, and
test-full timeout-minutes 30 -> 45. The rule's minimum for 27m is 41; 45 is
deliberately above it because the reshuffle means per-shard worst case moves run
to run, and a budget pinned to the exact minimum would be re-breached by the next
test file anyone adds. Only that one job's timeout changed; test.yml's scope,
matrix and steps are untouched, so #764's saving is unaffected.
Raising that cap let the Windows shard finish (28m45s, inside 45) and uncovered
a real failure the 30-minute cancel had been masking:
`new quick-task branch branches off origin/main (#2916)` died with
`outcome=timed_out exitCode=null`, SIGTERM, at the 15000ms bound.
tests/quick-branching.test.cjs:149 `runStep` runs a `#!/usr/bin/env bash` script
executing MULTIPLE git commands, but was bound to GIT_TIMEOUT_MS (15000) -- the
norm for a SINGLE git plumbing call. tests/helpers/timeouts.cjs already documents
this exact failure and exists to fix it: HOOK_FANOUT_TIMEOUT_MS was created after
PR #3285 recorded "outcome=timed_out exitCode=null at exactly the 15000ms probe
bound while every other lane passed the same commit", and calls that "a bound
sized for the wrong class, not a slow machine". Our failure is that case
verbatim, so both sites move to the class norm rather than to a bigger number.
The same class also failed on `next` itself 21 hours earlier -- run 32608945654,
windows-latest/24 shard 1/3, `plan touching only src/ in a submodule project
keeps worktree isolation ENABLED` -- where tests/worktree-safety.test.cjs:5845
`runGate` fans out to `git config --file .gitmodules` under a hardcoded 30000.
Fixed too, since it is a defect in the tree regardless of which branch surfaced
it.
A survey of the whole tests/ tree found the same class-mismatch at further
bash fan-out sites bound under 60000ms, and the maintainer approved sweeping
them rather than leaving them latent to surface the same way one at a time. 16
fan-out sites across 16 files now use the class norm.
The sweep is class-correctness, not raising numbers until things pass. Sites
were moved ONLY where the bash body demonstrably spawns something (git, node,
npm, a CLI); self-contained shell snippets were left where they are, and are
listed as deliberately unchanged: pure if/printf bodies (copilot-install), pure
array/case builtins (code-review-pipeline-regression:638), a documented
pure-shell gsd_run stub (host-integration), single-process hook calls
(workflow-guard:222/271/302), and a deliberately tight 5000ms fast-check hook
(gsd-write-guard.property). Nothing was lowered. process-seam.test.cjs:513
(literal 300) is untouched on purpose -- it tests timeout BEHAVIOR, so raising
it would destroy what it asserts.
Shared file-level constants were the trap here, and were handled per file rather
than by redefinition: GIT_TIMEOUT_MS has ~15 users in git-base-branch and only 1
is a fan-out; WORKTREE_TIMEOUT_MS has 16 users in worktree.test.cjs and 3 are;
PROBE_TIMEOUT_MS has several in three more files. In each the CALL SITE was
changed and the constant left alone, so no single-plumbing-call site silently
inherited a 60s bound. The one exception is hooks-opt-in.test.cjs, where
HOOK_TIMEOUT_MS has exactly one consumer -- spawnHook, the fan-out itself -- so
redefining it is identical in effect and reads better.
Only two of these sites have actually been observed failing. The rest cite that
shared class and those two run ids rather than inventing evidence of their own.
Co-authored-by: sim <sim@local>
698 lines
32 KiB
JavaScript
698 lines
32 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* Behavioral tests for the `check ui-safety-gate` subcommand (#1168).
|
|
*
|
|
* Tests the `computeUiSafetyGate` pure function exported from check-command-router.cjs.
|
|
* Uses in-memory tmpdir fixtures — no real CLI subprocess needed.
|
|
*
|
|
* Return shape: { frontend: bool, hasUiFiles: bool, hasUiSpec: bool, block: bool, message?: string }
|
|
* Invariant: block = frontend && hasUiFiles && !hasUiSpec
|
|
*
|
|
* Per RULESET.TESTS.boundary-coverage: exercises all branches:
|
|
* (a) frontend + UI files changed + no spec → block:true
|
|
* (b) frontend + UI files changed + spec exists → block:false
|
|
* (c) non-frontend → block:false
|
|
* (d) frontend + no UI files changed → block:false
|
|
*
|
|
* Per RULESET.TESTS.coderabbit-fix-prefer: calls the exported function and asserts typed fields.
|
|
*/
|
|
|
|
const { describe, test, before, after } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('node:fs');
|
|
const os = require('node:os');
|
|
const path = require('node:path');
|
|
|
|
const { cleanup } = require('./helpers.cjs');
|
|
const { computeUiSafetyGate } = require('../gsd-core/bin/lib/check-command-router.cjs');
|
|
|
|
// ─── Helpers ──────────────────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* Create a minimal project dir with:
|
|
* .planning/ROADMAP.md — one phase section with `phaseSection` body
|
|
* .planning/phases/01-test-phase/ — phase directory
|
|
* (optionally) a *-UI-SPEC.md inside the phase dir
|
|
*/
|
|
function makeProject({ phaseSection = '', hasUiSpec = false } = {}) {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'ui-safety-gate-test-'));
|
|
const planningDir = path.join(tmpDir, '.planning');
|
|
const phasesDir = path.join(planningDir, 'phases');
|
|
const phaseDir = path.join(phasesDir, '01-test-phase');
|
|
|
|
fs.mkdirSync(phaseDir, { recursive: true });
|
|
fs.writeFileSync(path.join(planningDir, 'config.json'), JSON.stringify({}), 'utf8');
|
|
|
|
// Minimal ROADMAP.md with one phase section
|
|
const roadmapContent = [
|
|
'# Project Roadmap',
|
|
'',
|
|
'## Phase 1: Test Phase',
|
|
'',
|
|
phaseSection,
|
|
'',
|
|
].join('\n');
|
|
fs.writeFileSync(path.join(planningDir, 'ROADMAP.md'), roadmapContent, 'utf8');
|
|
|
|
if (hasUiSpec) {
|
|
fs.writeFileSync(path.join(phaseDir, '01-UI-SPEC.md'), '# UI Design Contract\n', 'utf8');
|
|
}
|
|
|
|
return { tmpDir, phaseDir };
|
|
}
|
|
|
|
// ─── Tests ─────────────────────────────────────────────────────────────────────
|
|
|
|
describe('computeUiSafetyGate — ui.safety-gate check logic (#1168)', () => {
|
|
let frontendNoSpec, frontendWithSpec, nonFrontend;
|
|
|
|
before(() => {
|
|
// Branch (a): frontend + no UI-SPEC → tests block behavior
|
|
frontendNoSpec = makeProject({
|
|
phaseSection: 'Build the user interface and dashboard components for the frontend.',
|
|
hasUiSpec: false,
|
|
});
|
|
// Branch (b): frontend + UI-SPEC exists → block:false
|
|
frontendWithSpec = makeProject({
|
|
phaseSection: 'Build the frontend dashboard with React components and UI forms.',
|
|
hasUiSpec: true,
|
|
});
|
|
// Branch (c): no frontend indicators → block:false
|
|
nonFrontend = makeProject({
|
|
phaseSection: 'Add a REST API endpoint and database migration for the user table.',
|
|
hasUiSpec: false,
|
|
});
|
|
});
|
|
|
|
after(() => {
|
|
for (const { tmpDir } of [frontendNoSpec, frontendWithSpec, nonFrontend]) {
|
|
try { cleanup(tmpDir); } catch { /* ignore */ }
|
|
}
|
|
});
|
|
|
|
describe('return shape', () => {
|
|
test('result has required keys: frontend, hasUiFiles, hasUiSpec, block', () => {
|
|
const result = computeUiSafetyGate(nonFrontend.tmpDir, '1');
|
|
assert.ok(typeof result === 'object' && result !== null, 'result must be an object');
|
|
assert.ok(typeof result.frontend === 'boolean', 'frontend must be boolean');
|
|
assert.ok(typeof result.hasUiFiles === 'boolean', 'hasUiFiles must be boolean');
|
|
assert.ok(typeof result.hasUiSpec === 'boolean', 'hasUiSpec must be boolean');
|
|
assert.ok(typeof result.block === 'boolean', 'block must be boolean');
|
|
});
|
|
|
|
test('block invariant: block === frontend && hasUiFiles && !hasUiSpec for all scenarios', () => {
|
|
for (const [label, { tmpDir }] of [
|
|
['frontendNoSpec', frontendNoSpec],
|
|
['frontendWithSpec', frontendWithSpec],
|
|
['nonFrontend', nonFrontend],
|
|
]) {
|
|
const r = computeUiSafetyGate(tmpDir, '1');
|
|
assert.strictEqual(
|
|
r.block,
|
|
r.frontend && r.hasUiFiles && !r.hasUiSpec,
|
|
`${label}: block invariant violated — frontend=${r.frontend} hasUiFiles=${r.hasUiFiles} hasUiSpec=${r.hasUiSpec} block=${r.block}`,
|
|
);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('branch (a) — frontend + no UI-SPEC → gate fires when hasUiFiles', () => {
|
|
test('detects frontend indicators in phase section', () => {
|
|
const r = computeUiSafetyGate(frontendNoSpec.tmpDir, '1');
|
|
assert.strictEqual(r.frontend, true, 'should detect frontend indicators');
|
|
});
|
|
|
|
test('hasUiSpec is false when no *-UI-SPEC.md exists', () => {
|
|
const r = computeUiSafetyGate(frontendNoSpec.tmpDir, '1');
|
|
assert.strictEqual(r.hasUiSpec, false, 'hasUiSpec must be false');
|
|
});
|
|
|
|
test('block is true when frontend + hasUiFiles + no UI-SPEC', () => {
|
|
// hasUiFiles depends on git state; when false, block must also be false (invariant).
|
|
// We verify the invariant holds rather than hardcoding the git state.
|
|
const r = computeUiSafetyGate(frontendNoSpec.tmpDir, '1');
|
|
assert.strictEqual(r.block, r.frontend && r.hasUiFiles && !r.hasUiSpec,
|
|
'block invariant: frontend && hasUiFiles && !hasUiSpec');
|
|
});
|
|
|
|
test('message is present when block is true', () => {
|
|
const r = computeUiSafetyGate(frontendNoSpec.tmpDir, '1');
|
|
if (r.block) {
|
|
assert.ok(typeof r.message === 'string' && r.message.length > 0,
|
|
'message must be a non-empty string when block is true');
|
|
assert.ok(r.message.includes('UI-SPEC'), 'message must reference UI-SPEC');
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('branch (b) — frontend + UI-SPEC exists → block:false', () => {
|
|
test('detects frontend indicators in phase section', () => {
|
|
const r = computeUiSafetyGate(frontendWithSpec.tmpDir, '1');
|
|
assert.strictEqual(r.frontend, true, 'should detect frontend indicators');
|
|
});
|
|
|
|
test('hasUiSpec is true when *-UI-SPEC.md exists', () => {
|
|
const r = computeUiSafetyGate(frontendWithSpec.tmpDir, '1');
|
|
assert.strictEqual(r.hasUiSpec, true, 'hasUiSpec must be true');
|
|
});
|
|
|
|
test('block is false when UI-SPEC exists (regardless of hasUiFiles)', () => {
|
|
const r = computeUiSafetyGate(frontendWithSpec.tmpDir, '1');
|
|
assert.strictEqual(r.block, false, 'block must be false when spec exists');
|
|
});
|
|
|
|
test('message is absent when block is false', () => {
|
|
const r = computeUiSafetyGate(frontendWithSpec.tmpDir, '1');
|
|
assert.ok(!r.message || r.message === undefined,
|
|
'message must be absent when block is false');
|
|
});
|
|
});
|
|
|
|
describe('branch (c) — non-frontend phase → block:false', () => {
|
|
test('frontend is false for non-UI phase section', () => {
|
|
const r = computeUiSafetyGate(nonFrontend.tmpDir, '1');
|
|
assert.strictEqual(r.frontend, false, 'should NOT detect frontend indicators');
|
|
});
|
|
|
|
test('block is false for non-frontend phases', () => {
|
|
const r = computeUiSafetyGate(nonFrontend.tmpDir, '1');
|
|
assert.strictEqual(r.block, false, 'block must be false');
|
|
});
|
|
});
|
|
|
|
describe('graceful degradation', () => {
|
|
test('non-existent project dir returns frontend:false, block:false (no crash)', () => {
|
|
const r = computeUiSafetyGate('/tmp/nonexistent-gsd-test-dir-xyz', '1');
|
|
assert.strictEqual(typeof r.frontend, 'boolean', 'frontend must be boolean');
|
|
assert.strictEqual(r.frontend, false, 'missing roadmap → no frontend indicators');
|
|
assert.strictEqual(r.block, false, 'missing roadmap → block false');
|
|
assert.strictEqual(typeof r.hasUiFiles, 'boolean', 'hasUiFiles must be boolean');
|
|
assert.strictEqual(typeof r.hasUiSpec, 'boolean', 'hasUiSpec must be boolean');
|
|
});
|
|
|
|
test('missing ROADMAP.md returns frontend:false gracefully (no phaseLookupFailed)', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'ui-safety-nomap-'));
|
|
try {
|
|
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-phase'), { recursive: true });
|
|
const r = computeUiSafetyGate(tmpDir, '1');
|
|
assert.strictEqual(r.frontend, false, 'no ROADMAP → no frontend indicators');
|
|
assert.strictEqual(r.block, false, 'no ROADMAP → no block');
|
|
assert.ok(
|
|
!r.phaseLookupFailed,
|
|
'phaseLookupFailed must NOT be set when ROADMAP.md is absent (no-roadmap project is not a lookup failure)',
|
|
);
|
|
} finally {
|
|
try { cleanup(tmpDir); } catch { /* ignore */ }
|
|
}
|
|
});
|
|
|
|
test('ROADMAP.md present but phase not found → phaseLookupFailed:true (not silent false)', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'ui-safety-noPhase-'));
|
|
try {
|
|
const planningDir = path.join(tmpDir, '.planning');
|
|
const phasesDir = path.join(planningDir, 'phases');
|
|
fs.mkdirSync(path.join(phasesDir, '01-test-phase'), { recursive: true });
|
|
fs.writeFileSync(path.join(planningDir, 'ROADMAP.md'), [
|
|
'# Project Roadmap',
|
|
'',
|
|
'## Phase 1: Test Phase',
|
|
'',
|
|
'Build the frontend dashboard with React components.',
|
|
'',
|
|
].join('\n'), 'utf8');
|
|
// Phase 99 is not in the roadmap
|
|
const r = computeUiSafetyGate(tmpDir, '99');
|
|
assert.strictEqual(r.phaseLookupFailed, true,
|
|
'phaseLookupFailed must be true when ROADMAP.md exists but phase is not found');
|
|
assert.strictEqual(r.frontend, false, 'empty section → no frontend indicators');
|
|
} finally {
|
|
try { cleanup(tmpDir); } catch { /* ignore */ }
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('routing — ui-safety-gate is routable via check-command-router', () => {
|
|
test('routeCheckCommand routes ui-safety-gate (hyphen form)', () => {
|
|
const { routeCheckCommand } = require('../gsd-core/bin/lib/check-command-router.cjs');
|
|
// Should not throw; just verify routing works (output goes to stdout)
|
|
let threw = false;
|
|
try {
|
|
routeCheckCommand({ args: ['check', 'ui-safety-gate', '1'], cwd: nonFrontend.tmpDir, raw: true });
|
|
} catch (err) {
|
|
threw = true;
|
|
}
|
|
assert.strictEqual(threw, false, 'routeCheckCommand must not throw for ui-safety-gate');
|
|
});
|
|
|
|
test('routeCheckCommand routes ui.safety-gate (dot form — normalized to hyphens)', () => {
|
|
const { routeCheckCommand } = require('../gsd-core/bin/lib/check-command-router.cjs');
|
|
let threw = false;
|
|
try {
|
|
routeCheckCommand({ args: ['check', 'ui.safety-gate', '1'], cwd: nonFrontend.tmpDir, raw: true });
|
|
} catch (err) {
|
|
threw = true;
|
|
}
|
|
assert.strictEqual(threw, false, 'routeCheckCommand must not throw for ui.safety-gate (dot form)');
|
|
});
|
|
});
|
|
});
|
|
|
|
|
|
// ────────────────────────────────────────────────────────────────────────
|
|
// Folded from tests/bug-3706-ui-safety-gate-false-positives.test.cjs — consolidation epic #1969 (B3 #1972)
|
|
// ────────────────────────────────────────────────────────────────────────
|
|
{
|
|
const { describe: __foldDescribe } = require('node:test');
|
|
__foldDescribe("folded:bug-3706-ui-safety-gate-false-positives (consolidation epic #1969 B3 #1972)", () => {
|
|
'use strict';
|
|
|
|
/**
|
|
* Tests for bug #3706 (word-boundary anchoring) and #3718 (cross-shell portability).
|
|
*
|
|
* Root cause (#3706): grep -iE "UI|..." had no word-boundary anchoring, causing
|
|
* false-positives on "Requirements" (contains "ui"), "overview" ("view"), etc.
|
|
*
|
|
* Root cause (#3718): shell-based invocation (with locale env-var prefix) silently
|
|
* degrades on Windows PowerShell — the prefix is not recognised by pwsh.
|
|
*
|
|
* Fix (#3718, Approach A): gate logic moved to `bin/lib/ui-safety-gate.cjs` (Node.js).
|
|
* Reads phase text from STDIN (not argv) to avoid OS ARG_MAX limits.
|
|
* Invoked as: printf '%s' "$PHASE_SECTION" | node "${GSD_REPO_ROOT}/bin/lib/ui-safety-gate.cjs"
|
|
* Path anchored to repo root via `git rev-parse --show-toplevel`.
|
|
*
|
|
* Test strategy:
|
|
* 1. Import the helper module directly and assert correct results on the full fixture
|
|
* matrix (exercises the production code path).
|
|
* 2. Spawn the helper as a child process with shell:false and input on stdin to prove
|
|
* cross-shell portability — spawnSync with shell:false bypasses any host shell.
|
|
* 3. Assert the workflow .md files now invoke `node ... ui-safety-gate.cjs` via stdin
|
|
* rather than the old shell-based invocation (structural guard against regression).
|
|
*/
|
|
|
|
const { describe, test } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const os = require('node:os');
|
|
const { cleanup } = require('./helpers.cjs');
|
|
const { runNode, runHook } = require('./helpers/process-seam.cjs');
|
|
const { toLegacyResult } = require('./helpers/git-fixture.cjs');
|
|
const { PROBE_TIMEOUT_MS, HOOK_FANOUT_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
|
|
|
|
const HELPER_PATH = path.join(__dirname, '..', 'bin', 'lib', 'ui-safety-gate.cjs');
|
|
const PLAN_PHASE_PATH = path.join(__dirname, '..', 'gsd-core', 'workflows', 'plan-phase.md');
|
|
const AUTONOMOUS_PATH = path.join(__dirname, '..', 'gsd-core', 'workflows', 'autonomous.md');
|
|
const AUTONOMOUS_UI_DESIGN_CONTRACT_REF_PATH = path.join(
|
|
__dirname, '..', 'gsd-core', 'references', 'autonomous-ui-design-contract.md'
|
|
);
|
|
|
|
const { checkUiPresence } = require(HELPER_PATH);
|
|
|
|
/**
|
|
* #2994 fragmentization moved §3a.5's body out of autonomous.md into
|
|
* gsd-core/references/autonomous-ui-design-contract.md, leaving an
|
|
* unconditional `Read and execute:` pointer in the host. Read host +
|
|
* reference file combined so the structural guards below still see the
|
|
* real §3a.5 body (and its absence of the retired ui-safety-gate.cjs /
|
|
* RUNTIME_DIR / LC_ALL=C grep patterns).
|
|
*/
|
|
function readAutonomousCombined() {
|
|
return fs.readFileSync(AUTONOMOUS_PATH, 'utf-8') +
|
|
'\n' + fs.readFileSync(AUTONOMOUS_UI_DESIGN_CONTRACT_REF_PATH, 'utf-8');
|
|
}
|
|
|
|
// ── Helper ────────────────────────────────────────────────────────────────────
|
|
|
|
/**
|
|
* Mirrors the old `hasUiGate` helper for backwards-compatible assertions.
|
|
* Returns 0 (UI found) or 1 (not found), matching grep exit code semantics.
|
|
*/
|
|
function hasUiGate(text) {
|
|
return checkUiPresence(text).hasUI ? 0 : 1;
|
|
}
|
|
|
|
/**
|
|
* Spawn the helper with shell:false, passing input via stdin.
|
|
* Returns the spawnSync result object.
|
|
*/
|
|
function spawnGate(input) {
|
|
const result = runNode([HELPER_PATH], { input, timeoutMs: PROBE_TIMEOUT_MS });
|
|
return toLegacyResult(result);
|
|
}
|
|
|
|
// ── Structural guard — workflow files now invoke Node via stdin ───────────────
|
|
|
|
describe('Workflow .md structural guard (#3718)', () => {
|
|
// allow-test-rule: source-text-is-the-product (see #3706)
|
|
// The UI safety gate invocation is embedded in workflow prose-as-code.
|
|
// These structural tests guard against regression where someone re-introduces
|
|
// the shell-based locale-prefix invocation that silently breaks on Windows PowerShell.
|
|
|
|
// plan-phase.md (post-#1026): §5.6 now delegates UI gate evaluation to
|
|
// `gsd_run check ui-plan-gate` (a CLI command that internally calls checkUiPresence
|
|
// from ui-safety-gate.cjs). The direct shell invocation of ui-safety-gate.cjs was
|
|
// intentionally removed from the workflow — cross-shell portability is now provided
|
|
// by the CLI command layer, not inline shell code.
|
|
test('plan-phase.md must invoke check ui-plan-gate (capability-driven UI gate — #1026)', () => {
|
|
const content = fs.readFileSync(PLAN_PHASE_PATH, 'utf-8');
|
|
assert.ok(
|
|
content.includes('check ui-plan-gate'),
|
|
'plan-phase.md: §5.6 must delegate to `check ui-plan-gate` (capability-driven, #1026)'
|
|
);
|
|
assert.ok(
|
|
!content.includes('LC_ALL=C grep'),
|
|
'plan-phase.md: must NOT contain LC_ALL=C grep — that silently fails on Windows PowerShell (#3718)'
|
|
);
|
|
// Must NOT reintroduce the old shell-based path-search loop for ui-safety-gate.cjs
|
|
assert.ok(
|
|
!content.includes('UI_GATE_JS=$(for _c in'),
|
|
'plan-phase.md: must NOT contain the old shell-based ui-safety-gate.cjs path-search (old §5.6 pattern)'
|
|
);
|
|
});
|
|
|
|
// autonomous.md §3a.5 (post-#1031 cutover): §3a.5 now delegates to
|
|
// `loop render-hooks plan:pre` + `check ui-plan-gate` (capability-driven pattern,
|
|
// matching plan-phase.md §5.6). The direct shell invocation of ui-safety-gate.cjs
|
|
// was intentionally removed — cross-shell portability is provided by the CLI layer.
|
|
test('autonomous.md must invoke check ui-plan-gate (capability-driven UI gate — #1031)', () => {
|
|
const label = 'autonomous.md';
|
|
const content = readAutonomousCombined();
|
|
|
|
// §3a.5 must delegate to the capability-driven gate, not inline shell code
|
|
assert.ok(
|
|
content.includes('check ui-plan-gate'),
|
|
`${label}: §3a.5 must delegate to \`check ui-plan-gate\` (capability-driven, #1031)`
|
|
);
|
|
// §3a.5 must dispatch render-hooks plan:pre
|
|
assert.ok(
|
|
content.includes('loop render-hooks plan:pre'),
|
|
`${label}: §3a.5 must dispatch \`loop render-hooks plan:pre\` (capability hook resolution)`
|
|
);
|
|
// Must NOT reintroduce the old shell-based path-search loop for ui-safety-gate.cjs
|
|
assert.ok(
|
|
!content.includes('ui-safety-gate.cjs'),
|
|
`${label}: must NOT inline ui-safety-gate.cjs invocation — replaced by check ui-plan-gate (#1031)`
|
|
);
|
|
assert.ok(
|
|
!content.includes('UI_GATE_JS=$(for _c in'),
|
|
`${label}: must NOT contain the old shell-based ui-safety-gate.cjs path-search (old §3a.5 pattern)`
|
|
);
|
|
assert.ok(
|
|
!content.includes('LC_ALL=C grep'),
|
|
`${label}: must NOT contain LC_ALL=C grep — that silently fails on Windows PowerShell (#3718)`
|
|
);
|
|
});
|
|
});
|
|
|
|
// ── Cross-shell spawn test (#3718) ────────────────────────────────────────────
|
|
|
|
describe('Cross-shell portability — spawnSync with shell:false, stdin (#3718)', () => {
|
|
test('spawn with shell:false + stdin: UI input exits 0', () => {
|
|
const result = spawnGate('UI Refactor: migrate all screens');
|
|
assert.strictEqual(result.status, 0, `Expected exit 0 (UI found), got ${result.status}. stderr: ${result.stderr}`);
|
|
});
|
|
|
|
test('spawn with shell:false + stdin: non-UI input exits 1', () => {
|
|
const result = spawnGate('Requirements: backend REST API only');
|
|
assert.strictEqual(result.status, 1, `Expected exit 1 (no UI), got ${result.status}. stderr: ${result.stderr}`);
|
|
});
|
|
|
|
test('spawn with shell:false + stdin: empty input exits 1', () => {
|
|
const result = spawnGate('');
|
|
assert.strictEqual(result.status, 1, `Expected exit 1 (no UI for empty input), got ${result.status}. stderr: ${result.stderr}`);
|
|
});
|
|
|
|
test('spawn with shell:false + stdin: CRLF line endings handled correctly', () => {
|
|
const result = spawnGate('Deploy to the cloud platform\r\nand configure CI/CD.');
|
|
assert.strictEqual(result.status, 1, `CRLF "platform" must not trigger gate. stderr: ${result.stderr}`);
|
|
});
|
|
|
|
test('spawn with shell:false + stdin: multi-line input with UI token exits 0', () => {
|
|
const multiLine = 'Backend setup\nBuild the analytics dashboard\nand navigation component.';
|
|
const result = spawnGate(multiLine);
|
|
assert.strictEqual(result.status, 0, `Multi-line input with "dashboard" must exit 0. stderr: ${result.stderr}`);
|
|
});
|
|
|
|
test('spawn with shell:false + stdin: very large input (>100KB) is handled without error', () => {
|
|
// Verifies stdin transport does not hit ARG_MAX (argv transport fails above ~1MB on macOS).
|
|
// Fixture uses pure backend prose — no standalone UI tokens.
|
|
const largeInput = 'Backend service deployment and database migration step.\n'.repeat(3000);
|
|
const result = spawnGate(largeInput);
|
|
assert.strictEqual(result.status, 1, `Large non-UI input must exit 1, not crash. status: ${result.status}, stderr: ${result.stderr}`);
|
|
});
|
|
|
|
test('spawn with shell:false + stdin: large input with UI token exits 0', () => {
|
|
const largeUiInput = 'Backend infrastructure setup.\n'.repeat(2999) + 'Build the analytics dashboard.\n';
|
|
const result = spawnGate(largeUiInput);
|
|
assert.strictEqual(result.status, 0, `Large input ending with UI token must exit 0. stderr: ${result.stderr}`);
|
|
});
|
|
});
|
|
|
|
// ── Behavioral test matrix (via module import) ────────────────────────────────
|
|
|
|
/**
|
|
* Full fixture matrix — exercises the production checkUiPresence() function directly.
|
|
*/
|
|
function runBehavioralTests(label) {
|
|
describe(`${label} — UI gate behavioral matrix`, () => {
|
|
|
|
// ── False-positive tests (must NOT match) ──────────────────────────────
|
|
|
|
test('"Requirements" must NOT trigger UI gate (bug #3706 — "ui" substring)', () => {
|
|
const phaseSection =
|
|
'**Requirements**: The service must expose REST endpoints for authentication.\n' +
|
|
'All work is server-side. Database migrations and API contract only.';
|
|
assert.strictEqual(hasUiGate(phaseSection), 1, '"Requirements" must not match the UI gate');
|
|
});
|
|
|
|
test('"overview" must NOT trigger UI gate ("view" is a substring)', () => {
|
|
assert.strictEqual(hasUiGate('Overview of the data pipeline architecture and backend services.'), 1);
|
|
});
|
|
|
|
test('"performance" must NOT trigger UI gate ("form" is a substring)', () => {
|
|
assert.strictEqual(hasUiGate('Performance testing and benchmark analysis for the API layer.'), 1);
|
|
});
|
|
|
|
test('"platform" must NOT trigger UI gate ("form" is a substring)', () => {
|
|
assert.strictEqual(hasUiGate('Deploy to the cloud platform and configure CI/CD.'), 1);
|
|
});
|
|
|
|
test('"transform" must NOT trigger UI gate ("form" is a substring)', () => {
|
|
assert.strictEqual(hasUiGate('Transform raw event data and write to the warehouse.'), 1);
|
|
});
|
|
|
|
test('"review" must NOT trigger UI gate ("view" is a substring)', () => {
|
|
assert.strictEqual(hasUiGate('Code review checklist and PR approval workflow for the API.'), 1);
|
|
});
|
|
|
|
test('"build" must NOT trigger UI gate ("ui" at positions 2-3 of "build")', () => {
|
|
assert.strictEqual(hasUiGate('Build the backend service and run integration tests.'), 1);
|
|
});
|
|
|
|
test('"screening" must NOT trigger UI gate ("screen" is a substring)', () => {
|
|
assert.strictEqual(hasUiGate('Implement candidate screening criteria for the hiring pipeline.'), 1);
|
|
});
|
|
|
|
test('empty input does NOT trigger UI gate', () => {
|
|
assert.strictEqual(hasUiGate(''), 1);
|
|
});
|
|
|
|
test('whitespace-only input does NOT trigger UI gate', () => {
|
|
assert.strictEqual(hasUiGate(' \n \t '), 1);
|
|
});
|
|
|
|
test('compound "microfrontend" (no separator) does NOT trigger gate — documented behavior', () => {
|
|
assert.strictEqual(
|
|
hasUiGate('Build the microfrontend shell application.'), 1,
|
|
'"microfrontend" compound must NOT trigger gate'
|
|
);
|
|
});
|
|
|
|
// ── True-positive tests (must match) ──────────────────────────────────
|
|
|
|
test('standalone "UI" token DOES trigger UI gate', () => {
|
|
assert.strictEqual(hasUiGate('UI Refactor: migrate all screens to the new design system.'), 0);
|
|
});
|
|
|
|
test('standalone "view" token DOES trigger UI gate', () => {
|
|
assert.strictEqual(hasUiGate('Implement the user profile view controller and associated screen.'), 0);
|
|
});
|
|
|
|
test('standalone "form" token DOES trigger UI gate', () => {
|
|
assert.strictEqual(hasUiGate('Build a sign-up form with client-side validation.'), 0);
|
|
});
|
|
|
|
test('"dashboard" DOES trigger UI gate', () => {
|
|
assert.strictEqual(hasUiGate('Build the analytics dashboard and navigation component.'), 0);
|
|
});
|
|
|
|
test('lowercase "ui" token DOES trigger UI gate (case-insensitive)', () => {
|
|
assert.strictEqual(hasUiGate('Redesign the ui for mobile responsiveness.'), 0);
|
|
});
|
|
|
|
test('"non-UI" hyphenated form DOES trigger UI gate (hyphen is a word boundary)', () => {
|
|
assert.strictEqual(hasUiGate('This is a non-UI backend service with no visual elements.'), 0);
|
|
});
|
|
|
|
test('standalone "screen" token DOES trigger UI gate', () => {
|
|
assert.strictEqual(hasUiGate('Implement the loading screen and splash animation.'), 0);
|
|
});
|
|
|
|
test('hyphenated "micro-frontend" DOES trigger gate (word boundary on hyphen)', () => {
|
|
assert.strictEqual(hasUiGate('Build the micro-frontend shell application.'), 0);
|
|
});
|
|
|
|
// ── CRLF / edge case tests ─────────────────────────────────────────────
|
|
|
|
test('CRLF line endings: non-UI text does not trigger gate', () => {
|
|
assert.strictEqual(hasUiGate('Deploy to the cloud platform\r\nand configure CI/CD.'), 1);
|
|
});
|
|
|
|
test('CRLF line endings: UI token on second line triggers gate', () => {
|
|
assert.strictEqual(hasUiGate('Backend setup complete.\r\nBuild the analytics dashboard.'), 0);
|
|
});
|
|
|
|
test('leading/trailing whitespace does not affect token detection', () => {
|
|
assert.strictEqual(hasUiGate(' UI refactor phase '), 0);
|
|
});
|
|
});
|
|
}
|
|
|
|
runBehavioralTests('ui-safety-gate.cjs');
|
|
|
|
// ── Install-dir resolution from a consuming project (#448) ────────────────────
|
|
|
|
describe('UI gate resolves the helper against RUNTIME_DIR, not the consuming repo (#448)', () => {
|
|
const REPO_ROOT = path.join(__dirname, '..');
|
|
|
|
// Mirrors the §5.6 / §3a.5 resolution. The structural guard above forces the
|
|
// workflows to keep using this RUNTIME_DIR-anchored form; this proves the
|
|
// candidate path is correct and the helper is actually found + executed when
|
|
// the CWD is a consuming project that has no bin/lib of its own.
|
|
const GATE_SNIPPET = [
|
|
'_GSD_RT="${RUNTIME_DIR:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}"',
|
|
'UI_GATE_JS=$(for _c in "$_GSD_RT/gsd-core/bin/lib/ui-safety-gate.cjs" "$_GSD_RT/bin/lib/ui-safety-gate.cjs" "$_GSD_RT/.claude/bin/lib/ui-safety-gate.cjs" "$HOME/.claude/gsd-core/bin/lib/ui-safety-gate.cjs" "$HOME/.claude/bin/lib/ui-safety-gate.cjs"; do [ -f "$_c" ] && { echo "$_c"; break; }; done)',
|
|
'if [ -n "$UI_GATE_JS" ]; then printf \'%s\' "$PHASE_SECTION" | node "$UI_GATE_JS" >/dev/null 2>&1; HAS_UI=$?; else HAS_UI=0; fi',
|
|
'echo "$HAS_UI"',
|
|
].join('\n');
|
|
|
|
function runGateFrom(consumingDir, phaseSection) {
|
|
// Bash FAN-OUT: the snippet runs `git rev-parse`, a `for` loop probing
|
|
// multiple candidate paths, and `node` — the wrong class for
|
|
// `PROBE_TIMEOUT_MS` (a single short CLI probe). Same class as the
|
|
// observed CI failures in tests/quick-branching.test.cjs (PR #3787 run
|
|
// 32668773524) and tests/worktree-safety.test.cjs (`next` run
|
|
// 32608945654). See HOOK_FANOUT_TIMEOUT_MS in ./helpers/timeouts.cjs for
|
|
// the class rationale.
|
|
const result = runHook('-c', [GATE_SNIPPET], {
|
|
interpreter: 'bash',
|
|
cwd: consumingDir,
|
|
env: { ...process.env, RUNTIME_DIR: REPO_ROOT, PHASE_SECTION: phaseSection },
|
|
timeoutMs: HOOK_FANOUT_TIMEOUT_MS,
|
|
});
|
|
return toLegacyResult(result);
|
|
}
|
|
|
|
test('UI text is detected (HAS_UI=0) from a project without bin/lib', () => {
|
|
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-consuming-'));
|
|
try {
|
|
const res = runGateFrom(tmp, 'UI Refactor: migrate all screens');
|
|
assert.strictEqual(res.status, 0, `bash failed: ${res.stderr}`);
|
|
assert.strictEqual(res.stdout.trim(), '0',
|
|
'helper must be found via RUNTIME_DIR and report UI present — not silently no-op');
|
|
} finally {
|
|
cleanup(tmp);
|
|
}
|
|
});
|
|
|
|
test('non-UI text returns HAS_UI=1 via the RUNTIME_DIR-resolved helper', () => {
|
|
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-consuming-'));
|
|
try {
|
|
const res = runGateFrom(tmp, 'Requirements: backend REST API only');
|
|
assert.strictEqual(res.stdout.trim(), '1');
|
|
} finally {
|
|
cleanup(tmp);
|
|
}
|
|
});
|
|
|
|
test('UI gate found via gsd-core/bin/lib/ in installed layout (no root bin/lib/)', () => {
|
|
// Regression for #448: installed RUNTIME_DIR has gsd-core/bin/lib/ but NOT root bin/lib/.
|
|
// The probe must find the helper at the installed path, not silently no-op to HAS_UI=0.
|
|
const fakeRuntime = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-installed-rt-'));
|
|
const consumingProject = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-consuming-'));
|
|
try {
|
|
const installedLibDir = path.join(fakeRuntime, 'gsd-core', 'bin', 'lib');
|
|
fs.mkdirSync(installedLibDir, { recursive: true });
|
|
fs.copyFileSync(
|
|
path.join(REPO_ROOT, 'gsd-core', 'bin', 'lib', 'ui-safety-gate.cjs'),
|
|
path.join(installedLibDir, 'ui-safety-gate.cjs')
|
|
);
|
|
|
|
// Same bash FAN-OUT class as runGateFrom above (git rev-parse + a
|
|
// candidate-path probe loop + node) — see HOOK_FANOUT_TIMEOUT_MS in
|
|
// ./helpers/timeouts.cjs.
|
|
const res = toLegacyResult(
|
|
runHook('-c', [GATE_SNIPPET], {
|
|
interpreter: 'bash',
|
|
cwd: consumingProject,
|
|
env: { ...process.env, RUNTIME_DIR: fakeRuntime, PHASE_SECTION: 'Build the analytics dashboard' },
|
|
timeoutMs: HOOK_FANOUT_TIMEOUT_MS,
|
|
})
|
|
);
|
|
assert.strictEqual(res.status, 0, `bash failed: ${res.stderr}`);
|
|
assert.strictEqual(res.stdout.trim(), '0',
|
|
'helper must be found via gsd-core/bin/lib/ in installed layout and report UI present');
|
|
} finally {
|
|
cleanup(fakeRuntime);
|
|
cleanup(consumingProject);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ── checkUiPresence() return value API ───────────────────────────────────────
|
|
|
|
describe('checkUiPresence() return value API', () => {
|
|
test('returns { hasUI: true, tokens: [...] } when UI found', () => {
|
|
const result = checkUiPresence('Build the dashboard and UI form');
|
|
assert.strictEqual(result.hasUI, true);
|
|
assert.ok(Array.isArray(result.tokens), 'tokens must be an array');
|
|
assert.ok(result.tokens.length > 0, 'tokens must be non-empty when hasUI is true');
|
|
assert.ok(result.tokens.includes('dashboard') || result.tokens.includes('ui') || result.tokens.includes('form'));
|
|
});
|
|
|
|
test('returns { hasUI: false, tokens: [] } when no UI found', () => {
|
|
const result = checkUiPresence('Requirements: backend REST API only');
|
|
assert.strictEqual(result.hasUI, false);
|
|
assert.deepStrictEqual(result.tokens, []);
|
|
});
|
|
|
|
test('tokens are lowercased', () => {
|
|
const result = checkUiPresence('UI Refactor');
|
|
assert.ok(result.tokens.every(t => t === t.toLowerCase()), 'All tokens must be lowercase');
|
|
});
|
|
|
|
test('tokens are deduplicated', () => {
|
|
const result = checkUiPresence('UI redesign and ui cleanup');
|
|
const uiCount = result.tokens.filter(t => t === 'ui').length;
|
|
assert.strictEqual(uiCount, 1, 'Duplicate tokens must be deduplicated');
|
|
});
|
|
|
|
test('non-string input returns { hasUI: false, tokens: [] }', () => {
|
|
assert.deepStrictEqual(checkUiPresence(null), { hasUI: false, tokens: [] });
|
|
assert.deepStrictEqual(checkUiPresence(undefined), { hasUI: false, tokens: [] });
|
|
assert.deepStrictEqual(checkUiPresence(42), { hasUI: false, tokens: [] });
|
|
});
|
|
|
|
test('multiple distinct UI tokens on same line are ALL captured', () => {
|
|
// "form" and "view" both appear as standalone words on one line.
|
|
// Prior exec()-based impl only captured the first match per line.
|
|
const result = checkUiPresence('Build a sign-up form with a view controller');
|
|
assert.strictEqual(result.hasUI, true, 'hasUI must be true');
|
|
assert.ok(result.tokens.includes('form'), `Expected "form" in tokens, got: ${JSON.stringify(result.tokens)}`);
|
|
assert.ok(result.tokens.includes('view'), `Expected "view" in tokens, got: ${JSON.stringify(result.tokens)}`);
|
|
});
|
|
});
|
|
});
|
|
}
|