A PR whose diff is entirely under docs/ runs zero tests, so a guard whose INPUT
is shipped prose cannot protect the PR lane of the diffs it exists to check. Its
only firing opportunity is after merge, on the shared branch -- which is how next
went red on dacae9273 while the PR that caused it (#3746) was green on every
check.
The docs-lint job in .github/workflows/docs-required.yml -- an ALREADY-REQUIRED
context -- now selects and runs the docs guards that read the specific docs files
the PR changed.
scripts/docs-guard-registry.cjs test file -> the docs paths it reads (63)
scripts/select-docs-guards.cjs pure (changedPaths, registry) -> test files
scripts/lint-docs-guard-registration.cjs drift guard, wired into lint:ci
scripts/ci-test-scope.cjs is NOT touched -- `git diff origin/next --` on it is
empty -- so #764's saving stands and its 21 pinning tests are untouched.
Selection: exact path; trailing-slash directory prefix (boundary-checked --
docs/adrenaline.md does NOT match docs/adr/, which a naive startsWith gets
wrong); and '*' for the 6 entries that walk docs/ generally or read a computed
path. Unknown maps to '*' -- guessing narrow is how a guard silently stops
running. Measured: a typo fix selects 6 of 63; docs/AGENTS.md selects 12;
docs/COMMANDS.md selects 18.
Four things this got wrong first, each found by an independent reviewer or by
probe, and each having been asserted safe in a comment:
1. The registry started as a RULE in ci-test-scope.cjs's RULES, on the theory
that classify()'s !codeChanged normalization made it inert. True for
docs-ONLY diffs; false for MIXED docs+code diffs, where codeChanged is true
and the normalization never runs:
node scripts/ci-test-scope.cjs --files "docs/a.md src/semver.cts"
with the RULE: 25 targeted_tests
origin/next: 3 targeted_tests
Category error: RULES is the scoped lane's input; a docs-guard registry is a
lane manifest for a consumer that never calls classify(). Extracted; pinned
by value.
2. The second attempt was a dedicated workflow with paths: [docs/**]. Such a
workflow never reports on a non-docs PR, so it can never be a required
context without hanging every non-docs PR -- and a non-required check does not
block a merge, so the guard would have been advisory and #3753 unfixed.
docs-required.yml already has no paths: filter, already supplies the required
docs-lint context, already computes docs_changed, and already ran one docs
guard gated on it. Generalizing that step needs no ruleset edit at all.
3. The registry and the drift lint were built from ONE path-segment heuristic, so
both were blind identically -- and blind at the guard that motivated the issue.
The reader-call regex required a character BEFORE its keyword, so a callee
named exactly read( / load( / parse( / doc( / file( / content( could never
match; and only an INLINE path.join(ROOT,'docs','X.md') argument was caught,
missing the two-step-via-variable form -- the MAJORITY spelling -- plus
template literals and concatenation. Detector 1 fired on 14 of ~450 files, so
35 genuine guards sat unregistered while the lint reported 0 violations,
including cursor-reviewer (reads docs/COMMANDS.md, asserts
.includes('--cursor')) and inventory-headings-countfree. The "accepted blind
spot" this shipped with was the common case, not a fringe.
4. With detection fixed the true population is 115 files: 63 genuine guards, 52
incidental. Running all 63 in a REQUIRED check on a one-line typo fix is the
cost #764 exists to avoid -- install.test.cjs is 7840 lines and reads exactly
one docs file, docs/AGENTS.md, for its frontmatter. Dropping it reproduces the
bug; running it for a typo elsewhere is waste. Hence the map.
Then a second review round found six more, all fixed here:
- fragment-single-edit-propagation.install.test.cjs was EXEMPTED as
"overlay fixture only". False: it reads the real docs/registries/eos.json and
asserts on a registry entry name, and reads the real ADR-0001 and asserts its
H1. A docs-only PR touching either would have gone green and red next -- #3753
shipping again, from inside the fix for it. Now registered against both paths,
and all 52 remaining exemptions were re-audited one by one.
- The SUITES-collision guard compared RAW registry keys, but run-tests.cjs strips
a leading `tests/` BEFORE its suite check. So it caught 'all' and missed
'tests/all' -- the only spelling that can actually occur, since every key
carries the prefix. One typo would have run all 824 test files inside the
required job. Now normalized the same way run-tests.cjs normalizes.
- The lint failed OPEN on an unreadable tests dir or candidate file: 0 violations,
ok:true. A guard that cannot read its input must never report success.
- The exemption ratchet gated identity only, so a baselined file that later
STARTED asserting on shipped docs stayed exempt silently -- 52 permanently blind
files. The baseline now fingerprints the docs paths each exempted file
references and fails when that set changes, naming what changed.
- The exemption marker was still honored inside a multi-line template literal in
the header window. The scanner now tracks template-literal and block-comment
state.
- `git diff --name-only | grep '^docs/'` silently dropped C-quoted non-ASCII docs
paths, making docs_changed=false a green zero-guard check. Both call sites now
pass -c core.quotepath=false.
- The run step was gated on hashFiles(), which a force-committed
.docs-guard-tests.txt would satisfy. The step now rm -f's both scratch files
first and gates on an output it sets itself.
Three empty states, deliberately distinct, because conflating them rebuilds
#3753: an empty or malformed registry HARD-FAILS; docs changed with no guard
covering them logs and skips; no docs change is already gated. The middle state
must never be expressed as an empty --files-from, which prints `no tests in suite
"all"` and exits 0 -- a green check that guarded nothing. With the current
registry that state is unreachable, because the six '*' entries always match;
the branch is kept as defensive handling for a future registry and says so.
timeout-minutes: 15 bounds the required job against a hanging fork-supplied test;
it had none. npm ci was added because the job never installed dependencies -- the
previous single-file step got away without it, the registry does not.
docs/contributing/docs-guard-registration.md documents the rule, following its
sibling cross-platform-portability-rules.md, and CONTRIBUTING.md's CI Test
Quality Checks table links to it. It is also load-bearing: without a docs/ file
in the diff this PR would not have triggered its own lane, shipping an
unexercised change to a required check.
One unrelated fix, included because this PR surfaced it and CLAUDE.md forbids
deferring a defect found while working. On this branch's first CI run,
`full test (windows-latest, 24, shard 3/3)` was CANCELLED at exactly 30 minutes;
tests were still passing 0.8s before the cancel, so it is a wall-clock timeout,
not a hang, and a cancelled job reddens `Required tests`.
The cause is not this PR's test file, which costs ~60ms. Shard composition is
unstable: adding ONE file to the unit suite reshuffled 115 of 268 files between
shards, and shard 3 drew a heavier mix. Underneath that is a real pre-existing
defect. tests/ci-test-job-timeout-budget.test.cjs requires every lane's budget to
be >= 1.5x its MEASURED cost -- "a lane that got slower must be re-budgeted, not
excused" -- and its test-full entry recorded 19m from a windows-22 shard. That is
stale. Measured on `next` with none of this PR's changes present: 26m18s (run
32614439702, windows-latest/24 shard 3/3), 23m36s and 23m17s on shard 2/3. So the
lane costs ~26m and the 30-minute cap carried 1.14x headroom, not 1.5x. The gate
had been out of compliance with its own rule; this PR was merely the file
addition that reshuffled shard 3 past the cliff.
Fixed as that file prescribes: measuredMinutes 19 -> 27 with fresh evidence, and
test-full timeout-minutes 30 -> 45. The rule's minimum for 27m is 41; 45 is
deliberately above it because the reshuffle means per-shard worst case moves run
to run, and a budget pinned to the exact minimum would be re-breached by the next
test file anyone adds. Only that one job's timeout changed; test.yml's scope,
matrix and steps are untouched, so #764's saving is unaffected.
Raising that cap let the Windows shard finish (28m45s, inside 45) and uncovered
a real failure the 30-minute cancel had been masking:
`new quick-task branch branches off origin/main (#2916)` died with
`outcome=timed_out exitCode=null`, SIGTERM, at the 15000ms bound.
tests/quick-branching.test.cjs:149 `runStep` runs a `#!/usr/bin/env bash` script
executing MULTIPLE git commands, but was bound to GIT_TIMEOUT_MS (15000) -- the
norm for a SINGLE git plumbing call. tests/helpers/timeouts.cjs already documents
this exact failure and exists to fix it: HOOK_FANOUT_TIMEOUT_MS was created after
PR #3285 recorded "outcome=timed_out exitCode=null at exactly the 15000ms probe
bound while every other lane passed the same commit", and calls that "a bound
sized for the wrong class, not a slow machine". Our failure is that case
verbatim, so both sites move to the class norm rather than to a bigger number.
The same class also failed on `next` itself 21 hours earlier -- run 32608945654,
windows-latest/24 shard 1/3, `plan touching only src/ in a submodule project
keeps worktree isolation ENABLED` -- where tests/worktree-safety.test.cjs:5845
`runGate` fans out to `git config --file .gitmodules` under a hardcoded 30000.
Fixed too, since it is a defect in the tree regardless of which branch surfaced
it.
A survey of the whole tests/ tree found the same class-mismatch at further
bash fan-out sites bound under 60000ms, and the maintainer approved sweeping
them rather than leaving them latent to surface the same way one at a time. 16
fan-out sites across 16 files now use the class norm.
The sweep is class-correctness, not raising numbers until things pass. Sites
were moved ONLY where the bash body demonstrably spawns something (git, node,
npm, a CLI); self-contained shell snippets were left where they are, and are
listed as deliberately unchanged: pure if/printf bodies (copilot-install), pure
array/case builtins (code-review-pipeline-regression:638), a documented
pure-shell gsd_run stub (host-integration), single-process hook calls
(workflow-guard:222/271/302), and a deliberately tight 5000ms fast-check hook
(gsd-write-guard.property). Nothing was lowered. process-seam.test.cjs:513
(literal 300) is untouched on purpose -- it tests timeout BEHAVIOR, so raising
it would destroy what it asserts.
Shared file-level constants were the trap here, and were handled per file rather
than by redefinition: GIT_TIMEOUT_MS has ~15 users in git-base-branch and only 1
is a fan-out; WORKTREE_TIMEOUT_MS has 16 users in worktree.test.cjs and 3 are;
PROBE_TIMEOUT_MS has several in three more files. In each the CALL SITE was
changed and the constant left alone, so no single-plumbing-call site silently
inherited a 60s bound. The one exception is hooks-opt-in.test.cjs, where
HOOK_TIMEOUT_MS has exactly one consumer -- spawnHook, the fan-out itself -- so
redefining it is identical in effect and reads better.
Only two of these sites have actually been observed failing. The rest cite that
shared class and those two run ids rather than inventing evidence of their own.
Co-authored-by: sim <sim@local>
This commit is contained in:
339
scripts/docs-guard-registry.cjs
Normal file
339
scripts/docs-guard-registry.cjs
Normal file
@@ -0,0 +1,339 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* docs-guard-registry.cjs — the sole source of truth for which doc-reading
|
||||
* test files the docs-guard lane must run.
|
||||
*
|
||||
* ## Why this is its own module, not a `scripts/ci-test-scope.cjs` RULE
|
||||
*
|
||||
* A prior version of this PR added a `docs guards` RULE to `RULES` in
|
||||
* scripts/ci-test-scope.cjs, on the theory that classify()'s `!codeChanged`
|
||||
* normalization (which zeroes fullMatrix/targeted_tests/windows_tests for
|
||||
* docs-only diffs) made the RULE inert to classify()'s scope decision.
|
||||
*
|
||||
* That is true for docs-ONLY diffs and FALSE for MIXED docs+code diffs:
|
||||
* `codeChanged` is true whenever ANY changed file is product/pipeline code,
|
||||
* so the normalization never runs, and every one of this registry's test
|
||||
* files joined `targeted_tests` for every mixed PR. Probed on that version:
|
||||
* `node scripts/ci-test-scope.cjs --files "docs/a.md src/semver.cts"`
|
||||
* returned 25 targeted_tests, vs. 3 on `origin/next` — an unstated blowup
|
||||
* of the scoped/targeted lane on every mixed docs+code PR.
|
||||
*
|
||||
* Root cause: `RULES` answers "given these changed files, what should
|
||||
* test.yml run" for the MAIN scoped-lane pipeline. The docs-guard registry
|
||||
* is a LANE MANIFEST for a completely different consumer (the `docs-lint`
|
||||
* job in .github/workflows/docs-required.yml, gated on a `docs_changed`
|
||||
* step output). Putting a lane manifest inside a scope-classification rule
|
||||
* set was the defect; extracting it here removes any path by which it can
|
||||
* influence classify() at all.
|
||||
*
|
||||
* ## Who reads this
|
||||
*
|
||||
* - .github/workflows/docs-required.yml (`docs-lint` job) — derives the
|
||||
* PER-PR-SELECTED subset of this registry from scripts/select-docs-guards.cjs,
|
||||
* which is itself driven by this module's DOCS_GUARD_TESTS map. This job has
|
||||
* no `paths:` filter, so it always reports a status and can supply the
|
||||
* already-required `docs-lint` context; a dedicated `paths:`-filtered
|
||||
* workflow cannot be made required without hanging non-docs PRs forever.
|
||||
* - scripts/lint-docs-guard-registration.cjs — derives its registration
|
||||
* lint's comparison set from this module (so a docs-reading test file
|
||||
* that is neither registered here nor exempted fails the lint).
|
||||
* - scripts/select-docs-guards.cjs — the pure selector that maps a PR's
|
||||
* changed docs/ paths to the subset of this registry that actually needs
|
||||
* to run (#3753 follow-up: a flat "run everything" list is disproportionate
|
||||
* for a single-file docs typo fix).
|
||||
*
|
||||
* ## Registry shape (#3753 follow-up)
|
||||
*
|
||||
* `DOCS_GUARD_TESTS` is a MAP from test file path to the array of docs/
|
||||
* paths it actually reads, so a changed-docs-file can be resolved to the
|
||||
* narrow subset of guards that read it, instead of always running the
|
||||
* entire registry. Each value is a non-empty array of PATTERNS:
|
||||
*
|
||||
* - a plain path (e.g. `'docs/AGENTS.md'`) matches that exact file only;
|
||||
* - a trailing-slash path (e.g. `'docs/adr/'`) matches any changed path
|
||||
* under that directory prefix — use this for a test that walks or
|
||||
* `readdirSync`s a whole docs/ subdirectory;
|
||||
* - the sentinel `'*'` means "run on ANY docs/ change" — reserved for a
|
||||
* test that cannot be resolved to a narrower set of paths (the path is
|
||||
* computed, looped over an unresolvable variable, or the test walks
|
||||
* docs/ generally). Conservative fallback: when in doubt, use `'*'`,
|
||||
* never a guessed-narrow path — a false negative here (a guard that
|
||||
* silently stops running) is exactly the #3753 defect class this
|
||||
* registry exists to prevent.
|
||||
*
|
||||
* `DOCS_GUARD_TEST_FILES` (derived: `Object.keys(DOCS_GUARD_TESTS)`) is kept
|
||||
* as a flat array export so the registration lint and its parity test
|
||||
* continue to consume a plain file list without needing to know about the
|
||||
* map shape.
|
||||
*
|
||||
* ## Adding a docs guard
|
||||
*
|
||||
* Add the test file's path (relative to the repo root, `tests/<file>`) as a
|
||||
* key in DOCS_GUARD_TESTS below, with the docs/ paths it reads as the value
|
||||
* (or `['*']` if that cannot be resolved narrowly). Do not duplicate this
|
||||
* list anywhere else — a second, independently maintained list is exactly
|
||||
* the #3753 defect class (10 registered-but-not-run guards, silently
|
||||
* drifted) this registry exists to prevent from recurring.
|
||||
*
|
||||
* Sorted alphabetically by basename for unambiguous diffs.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Duplicate of scripts/run-tests.cjs:50's `SUITES` array. That array is not
|
||||
* exported by run-tests.cjs (module.exports there is deliberately narrow;
|
||||
* run-tests.cjs's own behavior is intentional and out of scope for this
|
||||
* registry to alter), so it cannot be imported directly without changing a
|
||||
* shared, behavior-locked file.
|
||||
*
|
||||
* Why this must exist at all: `run-tests.cjs`'s `selectExplicitFiles`
|
||||
* (scripts/run-tests.cjs:651) treats any registry entry that EQUALS a
|
||||
* SUITES member (e.g. `all`, `unit`) as a suite selector, not a filename —
|
||||
* so a typo in DOCS_GUARD_TESTS matching a suite name would silently run
|
||||
* the ENTIRE suite inside the required `docs-lint` job instead of erroring.
|
||||
* `assertNoSuiteCollision` below rejects that at the registry boundary
|
||||
* instead.
|
||||
*
|
||||
* Divergence risk: if run-tests.cjs's real SUITES list ever changes and this
|
||||
* copy is not updated, this check could under- or over-reject. That risk is
|
||||
* covered by a parity test (tests/ci-docs-guard-registry.test.cjs) that
|
||||
* drives run-tests.cjs's own exported `selectExplicitFiles` behaviorally —
|
||||
* for every token here it asserts run-tests.cjs treats it as a suite
|
||||
* selector (never "file not found"), and for a control non-member it
|
||||
* asserts the opposite — so a real divergence fails that test rather than
|
||||
* silently drifting.
|
||||
*/
|
||||
const RUN_TESTS_SUITES = ['all', 'unit', 'integration', 'install', 'security', 'slow', 'qa'];
|
||||
|
||||
/**
|
||||
* Normalize a registry entry EXACTLY the way scripts/run-tests.cjs's
|
||||
* `splitFileList` (:617-625) normalizes a requested token before its SUITES
|
||||
* check (:651): strip a leading `tests/` and normalize `\`->`/`. Without this,
|
||||
* comparing RAW registry keys (which all carry the `tests/` prefix by
|
||||
* convention) against RUN_TESTS_SUITES misses the realistic typo `'tests/all'`
|
||||
* entirely — proven by probe: `assertNoSuiteCollision(['tests/all'])` did not
|
||||
* throw, and `selectExplicitFiles(allFiles, 'tests/all')` selected all 824
|
||||
* files (the whole suite) inside the required docs-lint job.
|
||||
*
|
||||
* @param {string} t
|
||||
* @returns {string}
|
||||
*/
|
||||
function normalizeForSuiteCheck(t) {
|
||||
return t.replace(/\\/g, '/').replace(/^tests\//, '');
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject any docs-guard registry entry that collides with a run-tests.cjs
|
||||
* suite token (see RUN_TESTS_SUITES doc comment above). Throws with every
|
||||
* offending entry named, rather than failing on only the first. Compares
|
||||
* entries AFTER normalizeForSuiteCheck, mirroring run-tests.cjs's own
|
||||
* splitFileList normalization, so a `tests/`-prefixed or backslash-spelled
|
||||
* entry that would collide post-normalization is caught here too.
|
||||
*
|
||||
* @param {string[]} tests
|
||||
*/
|
||||
function assertNoSuiteCollision(tests) {
|
||||
const collisions = tests.filter((t) => RUN_TESTS_SUITES.includes(normalizeForSuiteCheck(t)));
|
||||
if (collisions.length > 0) {
|
||||
throw new Error(
|
||||
'docs-guard-registry: DOCS_GUARD_TESTS entry collides with a run-tests.cjs SUITES token ' +
|
||||
`(${collisions.join(', ')}) — scripts/run-tests.cjs:651's selectExplicitFiles treats a ` +
|
||||
'registry entry that equals a suite name as a suite selector, not a filename, so this ' +
|
||||
'would silently run the entire suite instead of the intended file(s). Fix the registry entry.',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Map from docs-guard test file to the docs/ path patterns it reads. See
|
||||
* this module's header comment for pattern semantics (exact / trailing-slash
|
||||
* dir-prefix / `'*'` sentinel). Every value here was derived by reading the
|
||||
* test file's actual read call(s) — not guessed — per #3753's own lesson: an
|
||||
* unresolvable read is recorded as `'*'`, never a narrowed guess.
|
||||
*/
|
||||
const DOCS_GUARD_TESTS = {
|
||||
'tests/adr-15-progress-converge.test.cjs': [
|
||||
'docs/COMMANDS.md',
|
||||
'docs/how-to/run-phases-autonomously.md',
|
||||
],
|
||||
// Walks docs/adr/ as a directory (builds/reads docs/adr/README.md and
|
||||
// fixture ADRs throughout) and separately reads docs/contributor-standards.md.
|
||||
'tests/adr-index-gate.test.cjs': ['docs/adr/', 'docs/contributor-standards.md'],
|
||||
'tests/agent-classification-parity.test.cjs': ['docs/AGENTS.md', 'docs/INVENTORY.md'],
|
||||
'tests/analyze-dependencies.test.cjs': ['docs/COMMANDS.md'],
|
||||
'tests/autonomous-converge.test.cjs': [
|
||||
'docs/COMMANDS.md',
|
||||
'docs/how-to/run-phases-autonomously.md',
|
||||
],
|
||||
'tests/capability-matrix-sync.test.cjs': ['docs/reference/capability-matrix.md'],
|
||||
'tests/capability-registry.test.cjs': [
|
||||
'docs/tutorials/build-your-first-capability.md',
|
||||
'docs/tutorials/install-your-first-capability.md',
|
||||
'docs/reference/capability-manifest.md',
|
||||
],
|
||||
'tests/claude-md.test.cjs': ['docs/COMMANDS.md'],
|
||||
'tests/claude-orchestration.test.cjs': ['docs/explanation/claude-orchestration-capability.md'],
|
||||
'tests/command-contract.test.cjs': [
|
||||
'docs/INVENTORY.md',
|
||||
'docs/ja-JP/INVENTORY.md',
|
||||
'docs/ko-KR/INVENTORY.md',
|
||||
'docs/zh-CN/INVENTORY.md',
|
||||
'docs/pt-BR/INVENTORY.md',
|
||||
],
|
||||
// uncoveredFiles(...) scans 'docs' as a generic coverage-scan root
|
||||
// (commit-files-pathspec.test.cjs:1618) — cannot be resolved to specific
|
||||
// files without re-deriving the scan's own file-discovery logic.
|
||||
'tests/commit-files-pathspec.test.cjs': ['*'],
|
||||
'tests/config-field-docs.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
'tests/config.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
'tests/context-index-sync.test.cjs': ['docs/CONTEXT-INDEX.json'],
|
||||
'tests/context-predicates-query.test.cjs': ['docs/contributor-standards.md'],
|
||||
// SCAN_DIRS includes 'docs' and recursively walks every .md file under it
|
||||
// (context7-tool-name-parity.test.cjs:31-38) — a generic tree walk, not a
|
||||
// fixed file set.
|
||||
'tests/context7-tool-name-parity.test.cjs': ['*'],
|
||||
'tests/contributor-standards.test.cjs': ['docs/contributor-standards.md'],
|
||||
'tests/cursor-reviewer.test.cjs': [
|
||||
'docs/COMMANDS.md',
|
||||
'docs/FEATURES.md',
|
||||
'docs/ja-JP/COMMANDS.md',
|
||||
'docs/ja-JP/FEATURES.md',
|
||||
'docs/ko-KR/COMMANDS.md',
|
||||
'docs/ko-KR/FEATURES.md',
|
||||
],
|
||||
'tests/discuss-all-flag.test.cjs': ['docs/COMMANDS.md'],
|
||||
'tests/discuss-mode.test.cjs': ['docs/workflow-discuss-mode.md'],
|
||||
// Walks docs/*.md and every docs/<locale>/*.md dir dynamically
|
||||
// (docs-parity-live-registry.test.cjs:42, 428) — deliberately generic.
|
||||
'tests/docs-parity-live-registry.test.cjs': ['*'],
|
||||
'tests/drift-detection.test.cjs': ['docs/CONFIGURATION.md', 'docs/AGENTS.md'],
|
||||
'tests/edge-probe-docs-fixtures.test.cjs': ['docs/adr/550-spec-phase-probe-contract.md'],
|
||||
'tests/edit-phase.test.cjs': [
|
||||
'docs/INVENTORY.md',
|
||||
'docs/INVENTORY-MANIFEST.json',
|
||||
'docs/COMMANDS.md',
|
||||
],
|
||||
'tests/effort-surface-axis.test.cjs': ['docs/reference/host-integration-capability-matrix.md'],
|
||||
'tests/execute-phase-active-flags.test.cjs': [
|
||||
'docs/reference/host-integration-capability-matrix.md',
|
||||
],
|
||||
'tests/execute-phase-wave.test.cjs': ['docs/COMMANDS.md'],
|
||||
'tests/external-job-waiting.test.cjs': ['docs/reference/planning-artifacts.md'],
|
||||
// Rows 8/9 (negative controls) read real docs/registries/eos.json and
|
||||
// docs/adr/0001-dispatch-policy-module.md and assert on their EXACT
|
||||
// committed content (an entry's name field, ADR-0001's H1 title) as a
|
||||
// sanity check before mutating an overlay fixture — a content edit to
|
||||
// either file changes the string this test asserts on.
|
||||
'tests/fragment-single-edit-propagation.install.test.cjs': [
|
||||
'docs/registries/eos.json',
|
||||
'docs/adr/0001-dispatch-policy-module.md',
|
||||
],
|
||||
'tests/gsd-write-guard.test.cjs': ['docs/USER-GUIDE.md'],
|
||||
'tests/host-integration-descriptors.test.cjs': [
|
||||
'docs/reference/host-integration-capability-matrix.md',
|
||||
],
|
||||
'tests/install.test.cjs': ['docs/AGENTS.md', 'docs/INVENTORY.md', 'docs/INVENTORY-MANIFEST.json'],
|
||||
// SOURCE_DIRS includes 'docs' and walks it recursively for every .md file
|
||||
// (intel.test.cjs:1223-1228) — a generic tree walk.
|
||||
'tests/intel.test.cjs': ['*'],
|
||||
'tests/inventory-headings-countfree.test.cjs': ['docs/INVENTORY.md'],
|
||||
'tests/inventory-manifest-sync.test.cjs': ['docs/INVENTORY.md', 'docs/INVENTORY-MANIFEST.json'],
|
||||
'tests/kilo-upgrades.test.cjs': ['docs/how-to/connect-gsd-mcp-server.md'],
|
||||
'tests/live-config-guard.test.cjs': ['docs/TESTING-SUITES.md'],
|
||||
'tests/model-catalog-runtime-defaults.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
// Scans every git-tracked file in the whole repo via `git ls-files`
|
||||
// (no-pending-3212-markers.test.cjs:37-46), which includes all of docs/ —
|
||||
// cannot be resolved to a fixed docs/ path set.
|
||||
'tests/no-pending-3212-markers.test.cjs': ['*'],
|
||||
'tests/phase6-capability-docs.test.cjs': ['docs/how-to/develop-a-capability.md', 'docs/README.md'],
|
||||
'tests/phase6-review-capabilities.test.cjs': [
|
||||
'docs/reference/review-verification-capabilities.md',
|
||||
'docs/how-to/develop-a-capability.md',
|
||||
'docs/README.md',
|
||||
],
|
||||
'tests/plan-checker-coupling.test.cjs': ['docs/AGENTS.md'],
|
||||
'tests/plan-phase-drift-guard.test.cjs': [
|
||||
'docs/CONFIGURATION.md',
|
||||
'docs/COMMANDS.md',
|
||||
'docs/USER-GUIDE.md',
|
||||
'docs/ARCHITECTURE.md',
|
||||
'docs/INVENTORY.md',
|
||||
'docs/INVENTORY-MANIFEST.json',
|
||||
],
|
||||
'tests/plan-phase-stall-detection.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
'tests/plan-review-convergence.test.cjs': [
|
||||
'docs/CONFIGURATION.md',
|
||||
'docs/USER-GUIDE.md',
|
||||
'docs/ARCHITECTURE.md',
|
||||
],
|
||||
'tests/planner-estimate-emission.test.cjs': ['docs/reference/plan-md.md', 'docs/CONFIGURATION.md'],
|
||||
'tests/precondition-element.test.cjs': ['docs/reference/plan-md.md'],
|
||||
'tests/product-name-purity.test.cjs': [
|
||||
'docs/README.md',
|
||||
'docs/zh-CN/README.md',
|
||||
'docs/ko-KR/README.md',
|
||||
'docs/ja-JP/README.md',
|
||||
'docs/pt-BR/README.md',
|
||||
],
|
||||
'tests/progress-forensic.test.cjs': ['docs/COMMANDS.md'],
|
||||
'tests/repo-layout.test.cjs': ['docs/contributing/bootstrap.md'],
|
||||
'tests/reversibility-tagging.test.cjs': ['docs/reference/plan-md.md'],
|
||||
'tests/reviewer-docs-parity.test.cjs': [
|
||||
'docs/COMMANDS.md',
|
||||
'docs/ja-JP/COMMANDS.md',
|
||||
'docs/ko-KR/COMMANDS.md',
|
||||
'docs/pt-BR/COMMANDS.md',
|
||||
'docs/zh-CN/COMMANDS.md',
|
||||
'docs/FEATURES.md',
|
||||
'docs/ja-JP/FEATURES.md',
|
||||
'docs/ko-KR/FEATURES.md',
|
||||
'docs/pt-BR/FEATURES.md',
|
||||
'docs/zh-CN/FEATURES.md',
|
||||
],
|
||||
'tests/runtime-converters.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
'tests/secure-phase.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
'tests/security-dead-exports.regression.test.cjs': ['docs/FEATURES.md'],
|
||||
'tests/security.test.cjs': ['docs/INVENTORY-MANIFEST.json'],
|
||||
// Walks docs/ recursively (todos-done-rename-guard.test.cjs:19-22
|
||||
// SCAN_DIRS) looking for stale references — a generic tree walk.
|
||||
'tests/todos-done-rename-guard.test.cjs': ['*'],
|
||||
'tests/tracer-bullet.test.cjs': [
|
||||
'docs/COMMANDS.md',
|
||||
'docs/reference/plan-md.md',
|
||||
'docs/how-to/plan-a-phase.md',
|
||||
'docs/AGENTS.md',
|
||||
],
|
||||
'tests/ui-spec-inventory-provenance.test.cjs': [
|
||||
'docs/FEATURES.md',
|
||||
'docs/how-to/design-a-ui-phase.md',
|
||||
'docs/ja-JP/FEATURES.md',
|
||||
'docs/ja-JP/how-to/design-a-ui-phase.md',
|
||||
'docs/zh-CN/FEATURES.md',
|
||||
'docs/zh-CN/how-to/design-a-ui-phase.md',
|
||||
'docs/ko-KR/FEATURES.md',
|
||||
'docs/ko-KR/how-to/design-a-ui-phase.md',
|
||||
'docs/pt-BR/how-to/design-a-ui-phase.md',
|
||||
],
|
||||
'tests/verifier-behavior-unverified.test.cjs': ['docs/reference/planning-artifacts.md'],
|
||||
'tests/verifier-coincidental-reliance.test.cjs': ['docs/AGENTS.md'],
|
||||
'tests/verify.test.cjs': ['docs/reference/plan-md.md'],
|
||||
'tests/workflow-fragments.test.cjs': ['docs/reference/workflow-fragments.md'],
|
||||
};
|
||||
|
||||
/**
|
||||
* Flat file-list view of DOCS_GUARD_TESTS, kept for consumers (the
|
||||
* registration lint and its parity test) that only need "which test files
|
||||
* are registered", not their per-file docs path patterns.
|
||||
*/
|
||||
const DOCS_GUARD_TEST_FILES = Object.keys(DOCS_GUARD_TESTS);
|
||||
|
||||
assertNoSuiteCollision(DOCS_GUARD_TEST_FILES);
|
||||
|
||||
module.exports = {
|
||||
DOCS_GUARD_TESTS,
|
||||
DOCS_GUARD_TEST_FILES,
|
||||
RUN_TESTS_SUITES,
|
||||
assertNoSuiteCollision,
|
||||
normalizeForSuiteCheck,
|
||||
};
|
||||
@@ -1,5 +1,11 @@
|
||||
'use strict';
|
||||
|
||||
// Renamed (was `test-failure-reasons.cjs`): the old basename matched Node's
|
||||
// `test-*.EXT` test-collection pattern, so the remote push-gate runner
|
||||
// collected and executed this SOURCE file as a test while GitHub CI (which
|
||||
// globs only tests/**/*.test.cjs) never saw it — see
|
||||
// scripts/lint-source-test-name-collision.cjs.
|
||||
|
||||
const TEST_GATE_REASON = Object.freeze({
|
||||
PASS: 'pass',
|
||||
TEST_FAILURE: 'test_failure',
|
||||
495
scripts/lint-docs-guard-registration.cjs
Normal file
495
scripts/lint-docs-guard-registration.cjs
Normal file
@@ -0,0 +1,495 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* lint-docs-guard-registration.cjs — every test file that READS a docs/ path
|
||||
* (via a real filesystem read call, not merely a string mention) must either
|
||||
* be named in the docs-guard lane registry, or carry an explicit
|
||||
* `// docs-guard-exempt: <reason>` marker.
|
||||
*
|
||||
* Exported pure function `checkDocsGuardRegistration({ testsDir, registry })`
|
||||
* so tests can drive it against a synthetic fixture directory; also runnable
|
||||
* as a CLI against the real tree (`node scripts/lint-docs-guard-registration.cjs`).
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { assertWithinAllowlist } = require('./lib/allowlist-ratchet.cjs');
|
||||
const { assertNoSuiteCollision } = require('./docs-guard-registry.cjs');
|
||||
const {
|
||||
DOCS_GUARD_EXEMPT_BASELINE,
|
||||
DOCS_GUARD_EXEMPT_DOCS_PATHS,
|
||||
} = require('./lint-docs-guard-registration.exempt-baseline.cjs');
|
||||
|
||||
/**
|
||||
* The lint's registry MUST derive from scripts/docs-guard-registry.cjs's
|
||||
* `DOCS_GUARD_TESTS` export — not from a second, hand-maintained list.
|
||||
* #3753 found exactly that split: this file used to carry its own
|
||||
* DOCS_GUARD_REGISTRY literal (20 entries) while the list that actually
|
||||
* drives the CI lane only had 10, so the lint reported "registered and
|
||||
* fine" for ten guards the lane never ran — the silent gap #3753 exists to
|
||||
* close, rebuilt one level down. Two lists holding one shared fact are free
|
||||
* to drift; one list read from two places cannot.
|
||||
*
|
||||
* Returns basenames (test files all live flat under tests/), matching the
|
||||
* shape `checkDocsGuardRegistration` expects.
|
||||
*/
|
||||
function deriveDocsGuardRegistry() {
|
||||
const { DOCS_GUARD_TEST_FILES } = require('./docs-guard-registry.cjs');
|
||||
if (!Array.isArray(DOCS_GUARD_TEST_FILES) || DOCS_GUARD_TEST_FILES.length === 0) {
|
||||
throw new Error(
|
||||
'lint-docs-guard-registration: scripts/docs-guard-registry.cjs exported an empty or ' +
|
||||
'missing DOCS_GUARD_TEST_FILES — cannot derive a registry. An empty registry would either flag ' +
|
||||
'every docs-reading test as unregistered, or (with an empty comparison set) silently pass ' +
|
||||
'with zero coverage. Fix the registry rather than defaulting this to [].',
|
||||
);
|
||||
}
|
||||
// Redundant with docs-guard-registry.cjs's own module-load-time self-check
|
||||
// (both consumers of that module — this lint and the docs-required.yml
|
||||
// derivation step — must independently reject a suite-token collision per
|
||||
// #3753's security follow-up FIX 3), but kept explicit here rather than
|
||||
// relying solely on the shared module's require()-time throw: this call
|
||||
// makes the guard visible and independently testable from this file's own
|
||||
// exports, instead of depending on an implicit side effect of another
|
||||
// module's load.
|
||||
assertNoSuiteCollision(DOCS_GUARD_TEST_FILES);
|
||||
return DOCS_GUARD_TEST_FILES.map(t => path.basename(t));
|
||||
}
|
||||
|
||||
// Real filesystem read calls we treat as "this file reads a path". A docs/
|
||||
// path appearing only inside a string literal handed to something else
|
||||
// (e.g. assert.equal(msg, 'see docs/foo.md')) must NOT trip this.
|
||||
//
|
||||
// Detector 1 (segment-shaped): `fs.readFileSync(path.join(ROOT, 'docs', 'x.md'))`.
|
||||
// The docs/ path is built from separate path-segment arguments to a known
|
||||
// Node fs read function, so it looks for a bare `'docs'` (or `"docs"`/`` `docs` ``)
|
||||
// segment, or a `'docs/...'` literal, inside the parenthesized argument list
|
||||
// of one of READ_FN_NAMES.
|
||||
const READ_FN_NAMES = ['readFileSync', 'readFile', 'readdirSync', 'readdir', 'createReadStream'];
|
||||
const READ_CALL_RE = new RegExp(
|
||||
`\\b(?:${READ_FN_NAMES.join('|')})\\s*\\(([^()]*(?:\\([^()]*\\)[^()]*)*)\\)`,
|
||||
'g',
|
||||
);
|
||||
const DOCS_QUOTED_PATH_RE = /(['"`])\/?docs\/[^'"`]*\1/;
|
||||
const DOCS_QUOTED_SEGMENT_RE = /(['"`])docs\1/;
|
||||
|
||||
// Detector 2 (single-string-shaped, #3753): `readShipped('docs/how-to/x.md')`.
|
||||
// Detector 1 is blind to this spelling — the docs/ path is a single string
|
||||
// literal, not a `path.join('docs', ...)` segment, and it is not always
|
||||
// handed straight to a bare `fs.read*` call. This is exactly how
|
||||
// tests/ui-spec-inventory-provenance.test.cjs reads — the guard whose
|
||||
// unregistered drift broke `next` on dacae9273 and motivated #3753 in the
|
||||
// first place — so a lint that cannot see its own motivating case's spelling
|
||||
// is not a fix.
|
||||
//
|
||||
// A naive "flag any 'docs/...' string literal" rule produces 59 hits on the
|
||||
// real tree (most are mention-only, e.g. assert messages), which is enough
|
||||
// false-positive noise that the lint gets disabled rather than obeyed. This
|
||||
// heuristic instead requires the docs/ literal to be an argument to a call
|
||||
// whose CALLEE NAME looks like a reader (contains read/load/parse/shipped/
|
||||
// content/file/doc, case-insensitively) — e.g. `readShipped`, `readRepoFile`,
|
||||
// `loadDoc`, `parseContent`. That name-shape restriction is what keeps the
|
||||
// count at 6 new hits instead of 59, at the cost of also matching a few
|
||||
// call sites (e.g. `groupFilesBySubrepo('docs/x.md', ...)`) whose name
|
||||
// happens to contain one of those substrings without actually reading a
|
||||
// file — those get `// docs-guard-exempt:` markers instead of registration.
|
||||
//
|
||||
// Detector 3 (co-occurrence, #3753 correctness follow-up — DEFECT B): the
|
||||
// two detectors above only catch a docs/ PATH EXPRESSION passed INLINE, as
|
||||
// an argument, to a call in the same statement. The dominant real idiom in
|
||||
// this repo builds the path first (`const P = path.join(ROOT, 'docs',
|
||||
// 'X.md');`) and reads it later (`fs.readFileSync(P);`) — a two-step form
|
||||
// neither detector above can see, along with template-literal
|
||||
// (`` `${ROOT}/docs/X.md` ``) and string-concat (`ROOT + '/docs/X.md'`)
|
||||
// paths. This detector decouples "does the file build a docs/ path" from
|
||||
// "does the file perform a read call" and flags the file when BOTH are true
|
||||
// anywhere in it, regardless of whether they share a call site. This trades
|
||||
// precision for recall deliberately: a false positive costs one
|
||||
// `// docs-guard-exempt:` marker with a reason; a false negative is the
|
||||
// #3753 bug shipping again.
|
||||
//
|
||||
// A docs/ path EXPRESSION is: a quoted 'docs/...' / "docs/..." literal
|
||||
// (optionally with a leading slash, for the `ROOT + '/docs/x.md'`
|
||||
// string-concat form), a backtick template literal containing `docs/`
|
||||
// anywhere inside it (covers both a bare `` `docs/x.md` `` literal and an
|
||||
// interpolated `` `${ROOT}/docs/x.md` `` prefix), or a bare `'docs'` segment
|
||||
// passed as one of the arguments to a `path.join(...)` call anywhere in the
|
||||
// file (not just when that call is itself an argument to a read function).
|
||||
const DOCS_TEMPLATE_LITERAL_RE = /`[^`]*\bdocs\/[^`]*`/;
|
||||
const PATH_JOIN_CALL_RE = /\bpath\s*\.\s*join\s*\(([^()]*(?:\([^()]*\)[^()]*)*)\)/g;
|
||||
|
||||
function pathJoinHasDocsSegment(content) {
|
||||
let match;
|
||||
PATH_JOIN_CALL_RE.lastIndex = 0;
|
||||
while ((match = PATH_JOIN_CALL_RE.exec(content)) !== null) {
|
||||
if (DOCS_QUOTED_SEGMENT_RE.test(match[1]) || DOCS_QUOTED_PATH_RE.test(match[1])) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function hasDocsPathExpression(content) {
|
||||
return DOCS_QUOTED_PATH_RE.test(content) ||
|
||||
DOCS_TEMPLATE_LITERAL_RE.test(content) ||
|
||||
pathJoinHasDocsSegment(content);
|
||||
}
|
||||
|
||||
// A "real read" for the co-occurrence detector: any call to a known Node fs
|
||||
// read function, ANYWHERE in the file — deliberately not requiring its
|
||||
// argument to look like a docs/ path here (that pairing is what
|
||||
// hasDocsPathExpression establishes separately). Does not include
|
||||
// `existsSync`: a pure existence check with no content read is exactly the
|
||||
// "incidental" case this lint's exemption path exists for, not a guard.
|
||||
const READ_CALL_PRESENT_RE = new RegExp(`\\b(?:${READ_FN_NAMES.join('|')})\\s*\\(`);
|
||||
|
||||
// Detector 2 (single-string-shaped, #3753): `readShipped('docs/how-to/x.md')`.
|
||||
// Detector 1 is blind to this spelling — the docs/ path is a single string
|
||||
// literal, not a `path.join('docs', ...)` segment, and it is not always
|
||||
// handed straight to a bare `fs.read*` call. This is exactly how
|
||||
// tests/ui-spec-inventory-provenance.test.cjs reads — the guard whose
|
||||
// unregistered drift broke `next` on dacae9273 and motivated #3753 in the
|
||||
// first place — so a lint that cannot see its own motivating case's spelling
|
||||
// is not a fix.
|
||||
//
|
||||
// A naive "flag any 'docs/...' string literal" rule produces 59 hits on the
|
||||
// real tree (most are mention-only, e.g. assert messages), which is enough
|
||||
// false-positive noise that the lint gets disabled rather than obeyed. This
|
||||
// heuristic instead requires the docs/ literal to be an argument to a call
|
||||
// whose CALLEE NAME looks like a reader (contains read/load/parse/shipped/
|
||||
// content/file/doc, case-insensitively) — e.g. `readShipped`, `readRepoFile`,
|
||||
// `loadDoc`, `parseContent`. The callee-name test is applied to the WHOLE
|
||||
// captured identifier (never requiring a non-keyword prefix before it — see
|
||||
// DEFECT A, #3753 correctness follow-up: a prior version of this regex wove
|
||||
// the keyword alternation into the SAME character class as a mandatory
|
||||
// leading identifier-start character, which made it structurally impossible
|
||||
// for the keyword to start at index 0 and silently missed every bare `read(`,
|
||||
// `load(`, `parse(`, `content(`, `file(`, and `doc(` callee — this repo's
|
||||
// most common reader-helper name among them), at the cost of also matching a
|
||||
// few call sites (e.g. `groupFilesBySubrepo('docs/x.md', ...)`) whose name
|
||||
// happens to contain one of those substrings without actually reading a
|
||||
// file — those get `// docs-guard-exempt:` markers instead of registration.
|
||||
const READER_NAME_KEYWORDS_RE = /read|load|parse|shipped|content|file|doc/i;
|
||||
const READER_CALL_RE = /\b([A-Za-z_$][\w$]*)\s*\(\s*[^)]{0,60}?["'`]docs\//gi;
|
||||
|
||||
function readerNameCallHasDocsPath(content) {
|
||||
let match;
|
||||
READER_CALL_RE.lastIndex = 0;
|
||||
while ((match = READER_CALL_RE.exec(content)) !== null) {
|
||||
if (READER_NAME_KEYWORDS_RE.test(match[1])) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// Neither detector can see every spelling a docs read could take (e.g. a
|
||||
// path built through an indirect helper with a read-agnostic name whose call
|
||||
// site never contains a real fs read call in the same file). This registry
|
||||
// is a curated, best-effort net with known holes, not an exhaustive static
|
||||
// analysis.
|
||||
function argsReadDocsPath(args) {
|
||||
return DOCS_QUOTED_PATH_RE.test(args) || DOCS_QUOTED_SEGMENT_RE.test(args);
|
||||
}
|
||||
|
||||
function readsDocsPath(content) {
|
||||
let match;
|
||||
READ_CALL_RE.lastIndex = 0;
|
||||
while ((match = READ_CALL_RE.exec(content)) !== null) {
|
||||
if (argsReadDocsPath(match[1])) return true;
|
||||
}
|
||||
if (readerNameCallHasDocsPath(content)) return true;
|
||||
return hasDocsPathExpression(content) && READ_CALL_PRESENT_RE.test(content);
|
||||
}
|
||||
|
||||
// Only scan the file's HEADER — the first EXEMPTION_SCAN_LINES lines. Scanning
|
||||
// the whole file lets a `// docs-guard-exempt:`-shaped string embedded in a
|
||||
// fixture/template literal (this lint's own test file writes exactly such
|
||||
// strings to synthesize fixtures) exempt the entire real file it appears in.
|
||||
// A header marker convention closes that hole while still finding every
|
||||
// genuine exemption comment, which by convention sits near the top of the
|
||||
// file next to its module docstring.
|
||||
const EXEMPTION_SCAN_LINES = 20;
|
||||
|
||||
// DEFECT C (#3753 correctness follow-up): a marker is only honored when the
|
||||
// line it appears on is an actual COMMENT, not merely a line whose text
|
||||
// contains the marker shape. Probed pre-fix: a string literal like
|
||||
// const s = "// docs-guard-exempt: whatever";
|
||||
// self-exempted the file with zero signal — the header-window narrowing
|
||||
// (EXEMPTION_SCAN_LINES) constrains WHERE the marker may appear but never
|
||||
// constrained WHAT KIND of line it must be, so relocating the marker inside
|
||||
// a string literal anywhere in the header window still worked. Requiring
|
||||
// the line, after trimming leading whitespace, to actually START with a
|
||||
// comment token (`//`, `/*`, or a JSDoc-block `*` continuation line) closes
|
||||
// this without narrowing the legitimate cases: every real marker in this
|
||||
// repo's tests/ sits at column 0 as a full-line `//` comment (verified
|
||||
// against every current `docs-guard-exempt:` occurrence in tests/).
|
||||
const EXEMPTION_LINE_IS_COMMENT_RE = /^(?:\/\/|\/\*|\*)/;
|
||||
|
||||
// Security follow-up FIX 4: a marker line inside a multi-line template
|
||||
// literal in the header window (e.g. `const F = \`\n// docs-guard-exempt: x\n\`;`)
|
||||
// still exempted the file pre-fix — the line, taken on its own, starts with
|
||||
// `//` and so passed EXEMPTION_LINE_IS_COMMENT_RE even though it is actually
|
||||
// backtick-string CONTENT, not a real comment. Track backtick parity across
|
||||
// lines: a line whose entire span is inside an open template literal (i.e.
|
||||
// the literal was already open when the line STARTED) is never honored as a
|
||||
// comment, no matter what its own text looks like.
|
||||
function findExemption(content) {
|
||||
const lines = content.split(/\r?\n/).slice(0, EXEMPTION_SCAN_LINES);
|
||||
let inTemplateLiteral = false;
|
||||
let inBlockComment = false;
|
||||
for (const line of lines) {
|
||||
const lineStartedInTemplateLiteral = inTemplateLiteral;
|
||||
// Count unescaped backticks on this line to toggle template-literal state.
|
||||
const backtickCount = (line.match(/\\`|`/g) || []).filter((tok) => tok === '`').length;
|
||||
if (backtickCount % 2 === 1) inTemplateLiteral = !inTemplateLiteral;
|
||||
if (lineStartedInTemplateLiteral) continue;
|
||||
|
||||
// Track (real) /* ... */ block-comment state across lines: a line that
|
||||
// is genuinely CONTENT inside an open, unterminated block comment is
|
||||
// still a real comment (JS syntax), so it stays eligible — this state is
|
||||
// tracked only so a future check can distinguish "inside a real block
|
||||
// comment" from "inside a template literal" rather than conflating them.
|
||||
const opensBlockComment = /\/\*/.test(line);
|
||||
const closesBlockComment = /\*\//.test(line);
|
||||
if (!inBlockComment && opensBlockComment && !closesBlockComment) {
|
||||
inBlockComment = true;
|
||||
} else if (inBlockComment && closesBlockComment) {
|
||||
inBlockComment = false;
|
||||
}
|
||||
|
||||
if (!EXEMPTION_LINE_IS_COMMENT_RE.test(line.trimStart())) continue;
|
||||
const m = /\/\/\s*docs-guard-exempt:(.*)$/.exec(line);
|
||||
if (m) return { present: true, reason: m[1].trim() };
|
||||
}
|
||||
return { present: false, reason: '' };
|
||||
}
|
||||
|
||||
/**
|
||||
* Security follow-up FIX 3: the exempt ratchet gated on file IDENTITY only —
|
||||
* a baselined file that later STARTS genuinely reading shipped docs/ content
|
||||
* stayed exempt with zero signal (probed: a baselined file doing
|
||||
* `fs.readFileSync('docs/foo.md')` still reported `ok=true, violations=[]`).
|
||||
* This extracts every distinct `docs/...` path TOKEN referenced anywhere in
|
||||
* an exempted file's content, so the baseline can pin a per-file fingerprint
|
||||
* of what it references and fail loudly when that set changes — forcing a
|
||||
* human to re-confirm the exemption still holds. Deliberately broader than
|
||||
* "only tokens passed to a read call" (this module's readsDocsPath heuristics
|
||||
* above): the fingerprint's job is to catch ANY drift in what docs/ paths an
|
||||
* exempted file mentions, not just the ones already read-call-shaped, since
|
||||
* a mention today can become a read call tomorrow without changing the
|
||||
* mention text at all.
|
||||
*/
|
||||
const DOCS_PATH_TOKEN_RE = /\/?docs\/[A-Za-z0-9_./-]*[A-Za-z0-9_-]/g;
|
||||
|
||||
/**
|
||||
* @param {string} content
|
||||
* @returns {string[]} sorted, deduped list of docs/ path tokens referenced —
|
||||
* a readable, diffable fingerprint (never an opaque hash) so a reviewer can
|
||||
* see exactly what changed.
|
||||
*/
|
||||
function extractDocsPathReferences(content) {
|
||||
const set = new Set();
|
||||
let m;
|
||||
DOCS_PATH_TOKEN_RE.lastIndex = 0;
|
||||
while ((m = DOCS_PATH_TOKEN_RE.exec(content)) !== null) {
|
||||
set.add(m[0].replace(/^\//, ''));
|
||||
}
|
||||
return [...set].sort();
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare each currently-exempted file's live docs-path fingerprint against
|
||||
* its pinned baseline fingerprint. A file present in `current` whose sorted
|
||||
* path list differs from the baseline's (paths added OR removed) is a
|
||||
* violation: the exemption's premise ("this file doesn't really guard
|
||||
* shipped docs content") may no longer hold and a human must re-confirm it.
|
||||
*
|
||||
* @param {Record<string, string[]>} current - file -> live sorted docs/ path list.
|
||||
* @param {Record<string, string[]>} baseline - file -> pinned sorted docs/ path list.
|
||||
* @returns {Array<{ file: string, reason: string }>}
|
||||
*/
|
||||
function checkExemptFingerprints(current, baseline) {
|
||||
const violations = [];
|
||||
for (const [file, paths] of Object.entries(current)) {
|
||||
const baselinePaths = baseline[file];
|
||||
if (!baselinePaths) continue; // identity ratchet (checkExemptBaseline) already flags a novel exemption
|
||||
const currentJoined = paths.join('\n');
|
||||
const baselineJoined = [...baselinePaths].sort().join('\n');
|
||||
if (currentJoined !== baselineJoined) {
|
||||
violations.push({
|
||||
file,
|
||||
reason:
|
||||
`the docs paths referenced by ${file} changed; re-confirm the exemption still holds and ` +
|
||||
`update the baseline in ${EXEMPT_BASELINE_FILE}:${EXEMPT_BASELINE_DOCS_PATHS_CONST}. ` +
|
||||
`was: [${baselinePaths.join(', ') || '(none)'}], now: [${paths.join(', ') || '(none)'}]`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return violations;
|
||||
}
|
||||
|
||||
/**
|
||||
* File referenced in this module's own remedy messages below (kept as a
|
||||
* named constant so the CLI path and the message text cannot drift).
|
||||
*/
|
||||
const EXEMPT_BASELINE_FILE = 'scripts/lint-docs-guard-registration.exempt-baseline.cjs';
|
||||
const EXEMPT_BASELINE_CONST = 'DOCS_GUARD_EXEMPT_BASELINE';
|
||||
const EXEMPT_BASELINE_DOCS_PATHS_CONST = 'DOCS_GUARD_EXEMPT_DOCS_PATHS';
|
||||
|
||||
/**
|
||||
* Ratchet the `// docs-guard-exempt:` marker on file identity, mirroring
|
||||
* scripts/lint-allow-test-rule-refs.cjs's `allow-test-rule` identity ratchet
|
||||
* (scripts/lib/allowlist-ratchet.cjs, ADR-456's pattern): a NEW exemption not
|
||||
* already in the pinned baseline fails the lint, and a baseline entry whose
|
||||
* file no longer carries a real marker (removed, renamed, or the marker was
|
||||
* deleted) is reported STALE and must be pruned — the baseline only ever
|
||||
* moves by deliberate edit, never silently grows or goes stale unnoticed.
|
||||
*
|
||||
* Unlike the sibling gate's generic "do not just add to the allowlist"
|
||||
* framing (which fits an offender-tracking allowlist), the correct remedy
|
||||
* for a genuinely-warranted new docs-guard exemption really is to add it to
|
||||
* the baseline — so the novel-entry case gets its own explicit remedy line
|
||||
* here rather than reusing that wording verbatim.
|
||||
*
|
||||
* @param {string[]} exemptedFiles - basenames with a present, non-empty
|
||||
* `docs-guard-exempt:` marker, found in the current tree.
|
||||
* @param {string[]} baseline - the pinned baseline (DOCS_GUARD_EXEMPT_BASELINE).
|
||||
* @returns {Array<{ file: string, reason: string }>}
|
||||
*/
|
||||
function checkExemptBaseline(exemptedFiles, baseline) {
|
||||
const violations = [];
|
||||
const { novel } = assertWithinAllowlist({
|
||||
label: 'docs-guard-exempt',
|
||||
current: exemptedFiles,
|
||||
known: baseline,
|
||||
fail: (msg) => violations.push({ file: '(docs-guard-exempt baseline)', reason: msg }),
|
||||
pruneHint:
|
||||
`its file was removed, renamed, or no longer carries a docs-guard-exempt marker — ` +
|
||||
`prune it from the baseline in ${EXEMPT_BASELINE_FILE}:${EXEMPT_BASELINE_CONST}`,
|
||||
});
|
||||
if (novel.length > 0) {
|
||||
violations.push({
|
||||
file: '(docs-guard-exempt baseline)',
|
||||
reason:
|
||||
`${novel.length} NEW docs-guard-exempt marker(s) not in the pinned baseline: ` +
|
||||
`${novel.join(', ')} — if this exemption is correct, add it to the baseline in ` +
|
||||
`${EXEMPT_BASELINE_FILE}:${EXEMPT_BASELINE_CONST}`,
|
||||
});
|
||||
}
|
||||
return violations;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{ testsDir: string, registry: string[], exemptBaseline?: string[] }} opts
|
||||
* `exemptBaseline`, when provided, ratchets the docs-guard-exempt marker on
|
||||
* file identity against that pinned list (see `checkExemptBaseline`).
|
||||
* Omitted entirely by callers (e.g. isolated fixture-only tests) that do
|
||||
* not want the ratchet applied.
|
||||
* @returns {{ ok: boolean, violations: Array<{ file: string, reason: string }>, exemptedFiles: string[] }}
|
||||
*/
|
||||
function checkDocsGuardRegistration({ testsDir, registry, exemptBaseline, exemptDocsPathsBaseline }) {
|
||||
const violations = [];
|
||||
const exemptedFiles = [];
|
||||
const exemptedDocsPaths = {};
|
||||
|
||||
for (const entry of registry) {
|
||||
const full = path.join(testsDir, entry);
|
||||
if (!fs.existsSync(full)) {
|
||||
violations.push({ file: entry, reason: `registry entry does not exist on disk: ${full}` });
|
||||
}
|
||||
}
|
||||
|
||||
const registrySet = new Set(registry);
|
||||
let files;
|
||||
try {
|
||||
files = fs.readdirSync(testsDir).filter(f => f.endsWith('.test.cjs'));
|
||||
} catch (err) {
|
||||
// A guard that cannot read its own input must never report success.
|
||||
// Probed pre-fix: checkDocsGuardRegistration({testsDir:'/nonexistent',
|
||||
// registry:[]}) returned {ok:true, violations:[]} — a green check that
|
||||
// guarded nothing. Treat an unreadable testsDir as a HARD VIOLATION.
|
||||
return {
|
||||
ok: false,
|
||||
violations: [
|
||||
{
|
||||
file: '(testsDir)',
|
||||
reason: `cannot read testsDir ${testsDir}: ${err.message} — a docs-guard registration ` +
|
||||
'lint that cannot read its own input must fail, never silently report zero violations',
|
||||
},
|
||||
],
|
||||
exemptedFiles: [],
|
||||
};
|
||||
}
|
||||
|
||||
for (const file of files) {
|
||||
const full = path.join(testsDir, file);
|
||||
let content;
|
||||
try {
|
||||
content = fs.readFileSync(full, 'utf8');
|
||||
} catch (err) {
|
||||
// Same class: a directory or broken symlink named `*.test.cjs` (or any
|
||||
// other read failure) must be a hard violation, not a silent skip.
|
||||
violations.push({
|
||||
file,
|
||||
reason: `cannot read candidate test file ${full}: ${err.message} — an unreadable ` +
|
||||
'docs-guard candidate must fail the lint, not be silently skipped',
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const exemption = findExemption(content);
|
||||
if (exemption.present) {
|
||||
if (exemption.reason.length === 0) {
|
||||
violations.push({ file, reason: 'docs-guard-exempt marker present with no reason' });
|
||||
} else {
|
||||
exemptedFiles.push(file);
|
||||
exemptedDocsPaths[file] = extractDocsPathReferences(content);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (readsDocsPath(content) && !registrySet.has(file)) {
|
||||
violations.push({
|
||||
file,
|
||||
reason: 'reads a docs/ path but is not registered in the docs-guard lane and carries no docs-guard-exempt marker',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (exemptBaseline) {
|
||||
violations.push(...checkExemptBaseline(exemptedFiles, exemptBaseline));
|
||||
}
|
||||
|
||||
if (exemptDocsPathsBaseline) {
|
||||
violations.push(...checkExemptFingerprints(exemptedDocsPaths, exemptDocsPathsBaseline));
|
||||
}
|
||||
|
||||
return { ok: violations.length === 0, violations, exemptedFiles, exemptedDocsPaths };
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
checkDocsGuardRegistration,
|
||||
checkExemptBaseline,
|
||||
checkExemptFingerprints,
|
||||
extractDocsPathReferences,
|
||||
deriveDocsGuardRegistry,
|
||||
EXEMPT_BASELINE_FILE,
|
||||
EXEMPT_BASELINE_CONST,
|
||||
EXEMPT_BASELINE_DOCS_PATHS_CONST,
|
||||
};
|
||||
|
||||
if (require.main === module) {
|
||||
const ROOT = path.join(__dirname, '..');
|
||||
const result = checkDocsGuardRegistration({
|
||||
testsDir: path.join(ROOT, 'tests'),
|
||||
registry: deriveDocsGuardRegistry(),
|
||||
exemptBaseline: DOCS_GUARD_EXEMPT_BASELINE,
|
||||
exemptDocsPathsBaseline: DOCS_GUARD_EXEMPT_DOCS_PATHS,
|
||||
});
|
||||
if (!result.ok) {
|
||||
process.stderr.write(`lint-docs-guard-registration: ${result.violations.length} violation(s)\n`);
|
||||
for (const v of result.violations) {
|
||||
process.stderr.write(` ${v.file}: ${v.reason}\n`);
|
||||
}
|
||||
process.exitCode = 1;
|
||||
} else {
|
||||
console.log('ok lint-docs-guard-registration: 0 violations');
|
||||
}
|
||||
}
|
||||
174
scripts/lint-docs-guard-registration.exempt-baseline.cjs
Normal file
174
scripts/lint-docs-guard-registration.exempt-baseline.cjs
Normal file
@@ -0,0 +1,174 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Pinned baseline for scripts/lint-docs-guard-registration.cjs's
|
||||
* `docs-guard-exempt` identity ratchet — mirrors
|
||||
* scripts/lint-allow-test-rule-refs.cjs's `allow-test-rule` ratchet
|
||||
* (scripts/lib/allowlist-ratchet.cjs, ADR-456's pattern).
|
||||
*
|
||||
* Every test file basename here is a KNOWN, already-reviewed
|
||||
* `// docs-guard-exempt: <reason>` marker. Nothing caps how many files may
|
||||
* carry the marker in the abstract, but ADDING a new one anywhere in
|
||||
* tests/*.test.cjs fails `lint-docs-guard-registration` until this baseline
|
||||
* is deliberately updated — so a new exemption always shows up as a
|
||||
* reviewable diff here, instead of silently opting a file out of the
|
||||
* docs-guard registration requirement with zero signal.
|
||||
*
|
||||
* A file listed here that no longer exists, or no longer carries a real
|
||||
* `docs-guard-exempt:` marker in its first 20 lines, is reported STALE and
|
||||
* must be pruned (ratchet-down, same as the sibling gate).
|
||||
*
|
||||
* Seeded 2026-08-23 (#3753 security follow-up FIX 2) from every real marker
|
||||
* found via scripts/lint-docs-guard-registration.cjs's own `findExemption`
|
||||
* scan of tests/*.test.cjs — derived, not retyped from memory.
|
||||
*
|
||||
* Extended 2026-08-23 (#3753 correctness follow-up): the reader-detection
|
||||
* fixes (Defects A/B/C) surfaced 86 previously-undetected docs/ readers. 40
|
||||
* were genuine docs-content guards (added to DOCS_GUARD_TESTS in
|
||||
* scripts/docs-guard-registry.cjs); the 46 added here touch a docs/ path
|
||||
* only incidentally (fixture data, external URL citations, comment-only
|
||||
* mentions, metadata labels, or scan-exclusion lists) — see each file's own
|
||||
* `docs-guard-exempt:` marker for its specific reason.
|
||||
*/
|
||||
const DOCS_GUARD_EXEMPT_BASELINE = [
|
||||
'adr-parser.test.cjs',
|
||||
'adr-parser.unit.test.cjs',
|
||||
'agent-marker-documentation-guard.test.cjs',
|
||||
'antigravity-upgrades.test.cjs',
|
||||
'capability-cli.test.cjs',
|
||||
'ci-docs-guard-registry.test.cjs',
|
||||
'ci-test-scope.test.cjs',
|
||||
'cline-install.test.cjs',
|
||||
'code-review-depth.test.cjs',
|
||||
'code-review-pipeline-regression.test.cjs',
|
||||
'codebuddy-upgrades.test.cjs',
|
||||
'commands.test.cjs',
|
||||
'commit-docs-bypass.test.cjs',
|
||||
'complexity-trigger.test.cjs',
|
||||
'cursor-imperative-reference.test.cjs',
|
||||
'declarative-reference-antigravity.test.cjs',
|
||||
'declarative-reference-zcode.test.cjs',
|
||||
'emitted-attribution.test.cjs',
|
||||
'eslint-rules.test.cjs',
|
||||
'estimate-calibrate.test.cjs',
|
||||
'gen-context-index.test.cjs',
|
||||
'gen-registry.test.cjs',
|
||||
'gsd-agent-isolation-guard.test.cjs',
|
||||
'hermes-dispatch-upgrade.test.cjs',
|
||||
'install-minimal-hooks.test.cjs',
|
||||
'install-runtime-artifacts.test.cjs',
|
||||
'installer-migration-config-root-marker.test.cjs',
|
||||
'installer-migration-pi-extension-ext.test.cjs',
|
||||
'installer-migrations.test.cjs',
|
||||
'kimi-upgrades.test.cjs',
|
||||
'lint-allow-test-rule-refs.test.cjs',
|
||||
'lint-docs-command-form.test.cjs',
|
||||
'lint-docs-required.test.cjs',
|
||||
'manifest-version-sync.test.cjs',
|
||||
'milestone-archive.test.cjs',
|
||||
'model-resolver.test.cjs',
|
||||
'new-project-mvp-prompt.test.cjs',
|
||||
'onboard-command.test.cjs',
|
||||
'opencode-command-dir-plural.test.cjs',
|
||||
'phase.test.cjs',
|
||||
'pr-branch-planning-filter.test.cjs',
|
||||
'precommit-alias-drift-hook.test.cjs',
|
||||
'removed-but-needed-lint.test.cjs',
|
||||
'repo-invariants.test.cjs',
|
||||
'require-issue-link-policy.test.cjs',
|
||||
'reviewer-manifest-body.test.cjs',
|
||||
'run-tests-harness.test.cjs',
|
||||
'runtime-name-policy.test.cjs',
|
||||
'security-prompt-injection.security.test.cjs',
|
||||
'shipped-reference-cites.test.cjs',
|
||||
'state.test.cjs',
|
||||
'worktree-safety.test.cjs',
|
||||
];
|
||||
|
||||
/**
|
||||
* Security follow-up FIX 3: a per-file fingerprint of every distinct
|
||||
* `docs/...` path TOKEN referenced by each baselined file (derived via
|
||||
* scripts/lint-docs-guard-registration.cjs's `extractDocsPathReferences`,
|
||||
* never retyped from memory). A baselined file whose live fingerprint
|
||||
* DIFFERS from the recorded one here fails the lint — the exemption's
|
||||
* premise ("this file doesn't really guard shipped docs content") may no
|
||||
* longer hold and a human must re-confirm it before the entry is updated.
|
||||
* Keeping this a plain, sorted, diffable path list (not a hash) is
|
||||
* deliberate: a reviewer can see exactly WHAT changed from the PR diff
|
||||
* alone.
|
||||
*
|
||||
* Seeded 2026-08-23 alongside DOCS_GUARD_EXEMPT_BASELINE above, from the
|
||||
* exact same scan.
|
||||
*/
|
||||
const DOCS_GUARD_EXEMPT_DOCS_PATHS = {
|
||||
'adr-parser.test.cjs': ['docs/adr/0001.md', 'docs/adr/0002.md', 'docs/adr/0010.md', 'docs/adr/NNNN.md'],
|
||||
'adr-parser.unit.test.cjs': ['docs/adr/0001.md', 'docs/adr/0099.md', 'docs/adr/NNNN.md'],
|
||||
'agent-marker-documentation-guard.test.cjs': ['docs/reference', 'docs/reference/workflow-fragments.md'],
|
||||
'antigravity-upgrades.test.cjs': ['docs/cli', 'docs/cli/gcli-migration', 'docs/cli/permissions'],
|
||||
'capability-cli.test.cjs': ['docs/reference/gsd-capability-command.md'],
|
||||
'ci-docs-guard-registry.test.cjs': [
|
||||
'docs/AGENTS.md', 'docs/COMMANDS.md', 'docs/INVENTORY.md', 'docs/a.md', 'docs/adr',
|
||||
'docs/adr/0001-example.md', 'docs/adrenaline.md', 'docs/bar.md', 'docs/foo.md', 'docs/how-to/foo.md',
|
||||
'docs/how-to/some-unrelated-guide.md', 'docs/how-to/x.md', 'docs/some-unrelated-file.md',
|
||||
'docs/totally-unrelated.md',
|
||||
],
|
||||
'ci-test-scope.test.cjs': [
|
||||
'docs/a.md', 'docs/adr', 'docs/adr/22-plan-drift-guard.md', 'docs/how-to/configure-model-profiles.md',
|
||||
'docs/installer-migrations.md', 'docs/ja-JP', 'docs/ja-JP/USAGE.md', 'docs/usage.md', 'docs/x.md',
|
||||
],
|
||||
'cline-install.test.cjs': ['docs/guide.md'],
|
||||
'code-review-depth.test.cjs': ['docs/src/auth/x.ts'],
|
||||
'code-review-pipeline-regression.test.cjs': ['docs/DEVELOPMENT.md'],
|
||||
'codebuddy-upgrades.test.cjs': ['docs/cli/sub-agents'],
|
||||
'commands.test.cjs': ['docs/x.md'],
|
||||
'commit-docs-bypass.test.cjs': [
|
||||
'docs/40-design.md', 'docs/CONFIGURATION.md', 'docs/readme.md', 'docs/tracked-var-mentioning',
|
||||
],
|
||||
'complexity-trigger.test.cjs': ['docs/readme.md'],
|
||||
'cursor-imperative-reference.test.cjs': ['docs/sdk/typescript'],
|
||||
'declarative-reference-antigravity.test.cjs': ['docs/cli/features'],
|
||||
'declarative-reference-zcode.test.cjs': ['docs/reference/host-integration-capability-matrix.md'],
|
||||
'emitted-attribution.test.cjs': ['docs/README.md', 'docs/tests', 'docs/tests/helpers/install-shared.cjs'],
|
||||
'eslint-rules.test.cjs': ['docs/readme.md'],
|
||||
'estimate-calibrate.test.cjs': [
|
||||
'docs/adr', 'docs/adr/2629-phase-effort-estimation-calibration.md', 'docs/reference',
|
||||
'docs/reference/planning-artifacts.md',
|
||||
],
|
||||
'gen-context-index.test.cjs': ['docs/CONTEXT-INDEX.json', 'docs/INVENTORY-MANIFEST.json'],
|
||||
'gen-registry.test.cjs': ['docs/registries', 'docs/registries/reviewers.json'],
|
||||
'gsd-agent-isolation-guard.test.cjs': ['docs/adr/1239-...md', 'docs/adr/1239-gsd-embeddable-orchestration-engine.md'],
|
||||
'hermes-dispatch-upgrade.test.cjs': ['docs/guides/delegation-patterns.md'],
|
||||
'install-minimal-hooks.test.cjs': ['docs/en/hooks', 'docs/en/users/features/hooks'],
|
||||
'install-runtime-artifacts.test.cjs': ['docs/adr/58-...md', 'docs/adr/58-runtime-install-policy-module.md', 'docs/cli/slash-commands'],
|
||||
'installer-migration-config-root-marker.test.cjs': ['docs/installer-migrations.md'],
|
||||
'installer-migration-pi-extension-ext.test.cjs': ['docs/installer-migrations.md'],
|
||||
'installer-migrations.test.cjs': ['docs/installer-migrations.md'],
|
||||
'kimi-upgrades.test.cjs': ['docs/reference/host-integration-capability-matrix.md'],
|
||||
'lint-allow-test-rule-refs.test.cjs': ['docs/readme.md'],
|
||||
'lint-docs-command-form.test.cjs': ['docs/adr', 'docs/adr/999-example.md', 'docs/how-to/example.md'],
|
||||
'lint-docs-required.test.cjs': [
|
||||
'docs/COMMANDS.md', 'docs/USER-GUIDE.md', 'docs/adr', 'docs/adr/0001-foo.md', 'docs/adr/0099-new.md',
|
||||
'docs/agents', 'docs/agents/triage-labels.md',
|
||||
],
|
||||
'manifest-version-sync.test.cjs': [],
|
||||
'milestone-archive.test.cjs': ['docs/TESTING-SUITES.md'],
|
||||
'model-resolver.test.cjs': ['docs/TESTING-SUITES.md'],
|
||||
'new-project-mvp-prompt.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
'onboard-command.test.cjs': ['docs/adr/0001-runtime.md'],
|
||||
'opencode-command-dir-plural.test.cjs': ['docs/commands'],
|
||||
'phase.test.cjs': ['docs/adr/3524-...md', 'docs/adr/3524-cjs-sdk-hard-seam.md'],
|
||||
'pr-branch-planning-filter.test.cjs': ['docs/readme.md'],
|
||||
'precommit-alias-drift-hook.test.cjs': ['docs/adr/0174-...md', 'docs/adr/0174-retire-gsd-sdk-package-boundary.md'],
|
||||
'removed-but-needed-lint.test.cjs': ['docs/getting-started.md', 'docs/gsd-new-workspace.md', 'docs/setup.md'],
|
||||
'repo-invariants.test.cjs': ['docs/FEATURES.md', 'docs/workflows/README'],
|
||||
'require-issue-link-policy.test.cjs': ['docs/-prefixed', 'docs/CONFIGURATION.md', 'docs/a.md', 'docs/b.md', 'docs/guide.md'],
|
||||
'reviewer-manifest-body.test.cjs': ['docs/how-to/ship-a-reviewer-lane.md'],
|
||||
'run-tests-harness.test.cjs': ['docs/TESTING-SUITES.md'],
|
||||
'runtime-name-policy.test.cjs': ['docs/customize/skills'],
|
||||
'security-prompt-injection.security.test.cjs': ['docs/notes.md'],
|
||||
'shipped-reference-cites.test.cjs': [],
|
||||
'state.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
'worktree-safety.test.cjs': ['docs/SUMMARY.md'],
|
||||
};
|
||||
|
||||
module.exports = { DOCS_GUARD_EXEMPT_BASELINE, DOCS_GUARD_EXEMPT_DOCS_PATHS };
|
||||
@@ -2,10 +2,16 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* lint-fix-has-regression-test.cjs — gate: every fix(#NNNN) commit must
|
||||
* lint-fix-has-regression-tests.cjs — gate: every fix(#NNNN) commit must
|
||||
* include at least one behavioral test file (tests/*.test.cjs) that is NOT
|
||||
* an auto-generated fixture/baseline.
|
||||
*
|
||||
* Renamed (was `lint-fix-has-regression-test.cjs`, singular): the old
|
||||
* basename matched Node's `*-test.EXT` test-collection pattern, so the
|
||||
* remote push-gate runner collected and executed this SOURCE file as a
|
||||
* test while GitHub CI (which globs only tests/**\/*.test.cjs) never saw
|
||||
* it — see scripts/lint-source-test-name-collision.cjs.
|
||||
*
|
||||
* ## Why
|
||||
*
|
||||
* CONTRIBUTING.md:47: "Fix it. Write a test that would have caught the bug."
|
||||
@@ -82,7 +88,7 @@ function getChangedTestFiles(baseRef) {
|
||||
|
||||
function main() {
|
||||
if (process.env.GSD_SKIP_REGRESSION_TEST_GATE === '1') {
|
||||
console.log('lint-fix-has-regression-test: SKIPPED (GSD_SKIP_REGRESSION_TEST_GATE=1)');
|
||||
console.log('lint-fix-has-regression-tests: SKIPPED (GSD_SKIP_REGRESSION_TEST_GATE=1)');
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -92,12 +98,12 @@ function main() {
|
||||
try {
|
||||
fixCommits = getFixCommits(baseRef);
|
||||
} catch {
|
||||
console.log(`lint-fix-has-regression-test: no fix/feat commits found vs ${baseRef}, skipping`);
|
||||
console.log(`lint-fix-has-regression-tests: no fix/feat commits found vs ${baseRef}, skipping`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (fixCommits.length === 0) {
|
||||
console.log('lint-fix-has-regression-test: no fix/feat commits, passing');
|
||||
console.log('lint-fix-has-regression-tests: no fix/feat commits, passing');
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -113,7 +119,7 @@ function main() {
|
||||
.map((c) => ` ${c.sha} ${c.subject}`)
|
||||
.join('\n');
|
||||
throw new ExitError(1,
|
||||
`lint-fix-has-regression-test: ${fixCommits.length} fix/feat commit(s) but ZERO behavioral test files (*.test.cjs) in the diff.\n` +
|
||||
`lint-fix-has-regression-tests: ${fixCommits.length} fix/feat commit(s) but ZERO behavioral test files (*.test.cjs) in the diff.\n` +
|
||||
`Auto-generated fixtures (tests/fixtures/, *-baseline.json) do NOT count.\n\n` +
|
||||
`Fix commits:\n${commitList}\n\n` +
|
||||
`CONTRIBUTING.md:47: "Write a test that would have caught the bug."\n` +
|
||||
@@ -123,7 +129,7 @@ function main() {
|
||||
}
|
||||
|
||||
console.log(
|
||||
`lint-fix-has-regression-test: PASS — ${fixCommits.length} fix/feat commit(s), ` +
|
||||
`lint-fix-has-regression-tests: PASS — ${fixCommits.length} fix/feat commit(s), ` +
|
||||
`${testFiles.length} behavioral test file(s): ${testFiles.join(', ')}`
|
||||
);
|
||||
}
|
||||
@@ -58,7 +58,7 @@
|
||||
* real, non-mocked `buildPlanningSnapshot(tmpCwd)` call (see
|
||||
* `tests/health-diagnostic-rules/root-existence.test.cjs`). This guard
|
||||
* therefore verifies the fixture-proof invariant STATICALLY against the
|
||||
* test files' own text — mirroring `scripts/lint-fix-has-regression-test.cjs`'s
|
||||
* test files' own text — mirroring `scripts/lint-fix-has-regression-tests.cjs`'s
|
||||
* house style — rather than dynamically re-running fixture-building code
|
||||
* this guard does not own.
|
||||
*/
|
||||
|
||||
@@ -215,7 +215,7 @@ function getDeletedFiles(root, baseRef) {
|
||||
// Deliberately let a git failure (unresolvable ref, no merge base, etc.)
|
||||
// propagate as a plain Error — main() treats ANY scan() failure as "cannot
|
||||
// resolve this base ref in this environment" and degrades to a skip,
|
||||
// matching lint-fix-has-regression-test.cjs. There is no failure mode here
|
||||
// matching lint-fix-has-regression-tests.cjs. There is no failure mode here
|
||||
// that should hard-exit non-zero; a real drift is only ever reported once
|
||||
// the diff succeeds and findSurvivingReferences finds a violation.
|
||||
const out = cp.execFileSync('git', ['diff', '--name-status', `${baseRef}...HEAD`], {
|
||||
@@ -284,7 +284,7 @@ function main() {
|
||||
} catch (e) {
|
||||
// origin/<base> unreachable in this environment (e.g. a shallow local
|
||||
// clone with no matching remote-tracking ref) — degrade to a skip rather
|
||||
// than a false failure, matching lint-fix-has-regression-test.cjs.
|
||||
// than a false failure, matching lint-fix-has-regression-tests.cjs.
|
||||
console.log(`lint-removed-but-needed: could not resolve ${baseRef}, skipping (${e.message})`);
|
||||
return;
|
||||
}
|
||||
|
||||
241
scripts/lint-source-test-name-collision.cjs
Normal file
241
scripts/lint-source-test-name-collision.cjs
Normal file
@@ -0,0 +1,241 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* lint-source-test-name-collision.cjs — no SOURCE file may carry a basename
|
||||
* that matches Node's built-in test-file collection patterns.
|
||||
*
|
||||
* ## Why (the incident)
|
||||
*
|
||||
* `src/test-home-guard.cts` was a SOURCE module (a runtime guard, not a
|
||||
* test) whose filename happened to match Node's `test-*` collection
|
||||
* convention. `scripts/run-tests.cjs` — what local `npm test` and GitHub CI
|
||||
* use — globs only `tests/**\/*.test.cjs`, so CI never saw it. But the
|
||||
* REMOTE test runner (the push gate; see CLAUDE.md's `gsd-test` section)
|
||||
* collects test files the way `node --test` does by default, across the
|
||||
* whole tree, so it picked the file up and executed it AS a test, where it
|
||||
* exited 1. Net effect: `next` was green on GitHub CI and red on the push
|
||||
* gate, blocking every push repo-wide. Proven by a control run on the
|
||||
* unmodified `next` tip 622f43353: `37199 passed / 1 failed`,
|
||||
* `throw · src/test-home-guard.cts`. The file has since been renamed to
|
||||
* `src/real-home-guard.cts` in this branch; this lint exists so a source
|
||||
* file can never silently re-acquire a collectable name again.
|
||||
*
|
||||
* ## What "collectable" means
|
||||
*
|
||||
* Node's test runner (`node --test`, and by extension the remote runner
|
||||
* that dispatches this repo's push-gate suite) collects any file whose path
|
||||
* matches, by default:
|
||||
*
|
||||
* **\/*.test.?(c|m)js **\/*-test.?(c|m)js **\/*_test.?(c|m)js
|
||||
* **\/test-*.?(c|m)js **\/test.?(c|m)js **\/test/**
|
||||
*
|
||||
* Node also resolves `.ts`/`.cts`/`.mts` through the same collector once
|
||||
* type-stripping is active (unflagged since Node 23.6; this repo's
|
||||
* `engines.node` floor is >=24) — which is exactly how a `.cts` file ended
|
||||
* up collected in the incident above. This lint therefore checks each of
|
||||
* the six extensions `js`, `cjs`, `mjs`, `ts`, `cts`, `mts` against each
|
||||
* basename-shaped pattern, plus a path-based check for any file living
|
||||
* inside a directory literally named `test` (not `tests` — this repo's own
|
||||
* test directory is deliberately outside the scanned source dirs, see
|
||||
* below).
|
||||
*
|
||||
* ## Scanned (source/shipped) directories
|
||||
*
|
||||
* src/, scripts/, hooks/, bin/, gsd-core/bin/ (excluding
|
||||
* gsd-core/bin/lib/**, see below), eslint-rules/
|
||||
*
|
||||
* ## Exempted
|
||||
*
|
||||
* - tests/ — files there are SUPPOSED to match; that is the point.
|
||||
* - node_modules/, .git/ — never source we own.
|
||||
* - gsd-core/bin/lib/** — build output generated from src/*.cts by
|
||||
* `npm run build:lib` (tsc), and gitignored (verified: every file under
|
||||
* it, including the incident's own post-fix
|
||||
* `gsd-core/bin/lib/real-home-guard.cjs`, is listed in .gitignore). A
|
||||
* generated file inherits its source's basename 1:1, so scanning it
|
||||
* would double-report the exact same defect `src/` already caught —
|
||||
* noise, not signal. `gsd-core/bin/shared/*.json` is data, not code,
|
||||
* but is harmlessly included since it never matches a JS/TS extension.
|
||||
*
|
||||
* Exported pure(ish) function `checkSourceTestNameCollisions({ dirs, root })`
|
||||
* so tests can drive it against synthetic fixture directories; also runnable
|
||||
* as a CLI against the real tree
|
||||
* (`node scripts/lint-source-test-name-collision.cjs`).
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { ExitError, runMain } = require('./lib/cli-exit.cjs');
|
||||
|
||||
// The six extensions Node's test runner collects, per the incident: the
|
||||
// documented `?(c|m)js` set (js, cjs, mjs) plus the TypeScript-loader
|
||||
// equivalents (ts, cts, mts) that the same collector resolves once
|
||||
// type-stripping is active (unflagged since Node 23.6; this repo's
|
||||
// engines.node floor is >=24, per package.json).
|
||||
const COLLECTED_EXTENSIONS = ['js', 'cjs', 'mjs', 'ts', 'cts', 'mts'];
|
||||
const EXT_ALT = COLLECTED_EXTENSIONS.join('|');
|
||||
|
||||
// Basename-shaped patterns, translated 1:1 from Node's documented defaults:
|
||||
// **/*.test.?(c|m)js **/*-test.?(c|m)js **/*_test.?(c|m)js
|
||||
// **/test-*.?(c|m)js **/test.?(c|m)js
|
||||
// (the sixth default, **/test/**, is a path-shaped check — see
|
||||
// isUnderLiteralTestDir below, not a basename regex.)
|
||||
const BASENAME_PATTERNS = [
|
||||
{ name: '*.test.EXT', re: new RegExp(`\\.test\\.(?:${EXT_ALT})$`) },
|
||||
{ name: '*-test.EXT', re: new RegExp(`-test\\.(?:${EXT_ALT})$`) },
|
||||
{ name: '*_test.EXT', re: new RegExp(`_test\\.(?:${EXT_ALT})$`) },
|
||||
{ name: 'test-*.EXT', re: new RegExp(`^test-.*\\.(?:${EXT_ALT})$`) },
|
||||
{ name: 'test.EXT', re: new RegExp(`^test\\.(?:${EXT_ALT})$`) },
|
||||
];
|
||||
|
||||
/**
|
||||
* Source/shipped directories this guard checks, relative to repo root.
|
||||
* Confirmed against the repo layout: src/, scripts/, hooks/, bin/,
|
||||
* gsd-core/bin/, eslint-rules/ all ship first-party source or shipped
|
||||
* tooling; nothing else at the top level carries executable source outside
|
||||
* tests/.
|
||||
*/
|
||||
const DEFAULT_SCAN_DIRS = ['src', 'scripts', 'hooks', 'bin', 'gsd-core/bin', 'eslint-rules'];
|
||||
|
||||
// Directories to never descend into anywhere in the tree.
|
||||
const ALWAYS_EXCLUDE_DIR_NAMES = new Set(['node_modules', '.git']);
|
||||
|
||||
// Relative dir prefixes (POSIX-joined, relative to repo root) that are
|
||||
// generated build output and must not be scanned — see the module doc for
|
||||
// why gsd-core/bin/lib is excluded (it 1:1-inherits src/*.cts basenames, so
|
||||
// scanning it double-reports the same defect src/ already catches).
|
||||
const GENERATED_OUTPUT_PREFIXES = ['gsd-core/bin/lib'];
|
||||
|
||||
function toPosix(p) {
|
||||
return p.split(path.sep).join('/');
|
||||
}
|
||||
|
||||
function isGeneratedOutput(relPath) {
|
||||
const posixRel = toPosix(relPath);
|
||||
return GENERATED_OUTPUT_PREFIXES.some(
|
||||
(prefix) => posixRel === prefix || posixRel.startsWith(`${prefix}/`)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Node's **\/test/** default: any file living inside a directory literally
|
||||
* named `test` (singular) anywhere in its path. Deliberately does NOT match
|
||||
* `tests/` (plural) — this repo's real test directory is a sibling of the
|
||||
* scanned source dirs, not nested inside one, and is never itself scanned.
|
||||
*/
|
||||
function isUnderLiteralTestDir(relPath) {
|
||||
return toPosix(relPath).split('/').slice(0, -1).includes('test');
|
||||
}
|
||||
|
||||
function matchingBasenamePatterns(basename) {
|
||||
return BASENAME_PATTERNS.filter((p) => p.re.test(basename)).map((p) => p.name);
|
||||
}
|
||||
|
||||
function walk(dir, root, out) {
|
||||
let entries;
|
||||
try {
|
||||
entries = fs.readdirSync(dir, { withFileTypes: true });
|
||||
} catch (err) {
|
||||
// A scan dir that cannot be read must never be silently treated as
|
||||
// "zero files, zero violations" — that would be a green check that
|
||||
// guarded nothing (same class of bug as an empty registry elsewhere in
|
||||
// this repo's lints).
|
||||
out.unreadable.push({ dir: path.relative(root, dir) || dir, error: err.message });
|
||||
return;
|
||||
}
|
||||
for (const entry of entries) {
|
||||
const full = path.join(dir, entry.name);
|
||||
const rel = path.relative(root, full);
|
||||
if (entry.isDirectory()) {
|
||||
if (ALWAYS_EXCLUDE_DIR_NAMES.has(entry.name)) continue;
|
||||
if (isGeneratedOutput(rel)) continue;
|
||||
walk(full, root, out);
|
||||
} else if (entry.isFile()) {
|
||||
out.scanned.push(rel);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{ dirs?: string[], root: string }} opts
|
||||
* `dirs` — scan dirs relative to `root` (defaults to DEFAULT_SCAN_DIRS).
|
||||
* `root` — the directory `dirs` are resolved against (repo root for the
|
||||
* real CLI run; a synthetic fixture root in tests).
|
||||
* @returns {{ ok: boolean, violations: Array<{file:string, patterns:string[], reason:string}>, scanned: string[] }}
|
||||
*/
|
||||
function checkSourceTestNameCollisions({ dirs = DEFAULT_SCAN_DIRS, root }) {
|
||||
const violations = [];
|
||||
const out = { scanned: [], unreadable: [] };
|
||||
|
||||
for (const dir of dirs) {
|
||||
const full = path.join(root, dir);
|
||||
if (!fs.existsSync(full)) continue; // a configured dir that doesn't exist is not this lint's problem
|
||||
walk(full, root, out);
|
||||
}
|
||||
|
||||
if (out.unreadable.length > 0) {
|
||||
return {
|
||||
ok: false,
|
||||
violations: out.unreadable.map((u) => ({
|
||||
file: u.dir,
|
||||
patterns: [],
|
||||
reason: `cannot read scan directory ${u.dir}: ${u.error} — a collision guard that cannot ` +
|
||||
'read its own input must fail, never silently report zero violations',
|
||||
})),
|
||||
scanned: out.scanned,
|
||||
};
|
||||
}
|
||||
|
||||
for (const rel of out.scanned) {
|
||||
const basename = path.basename(rel);
|
||||
const patterns = matchingBasenamePatterns(basename);
|
||||
const underTestDir = isUnderLiteralTestDir(rel);
|
||||
if (patterns.length === 0 && !underTestDir) continue;
|
||||
|
||||
const matched = underTestDir ? [...patterns, '**/test/**'] : patterns;
|
||||
violations.push({
|
||||
file: toPosix(rel),
|
||||
patterns: matched,
|
||||
reason:
|
||||
`basename matches Node's test-collection pattern(s) [${matched.join(', ')}] — the remote ` +
|
||||
'test runner (the push gate) collects files by this convention and executes them AS ' +
|
||||
'tests, while GitHub CI (scripts/run-tests.cjs) globs only tests/**/*.test.cjs and never ' +
|
||||
'sees it; the failure then appears ONLY at the push gate (incident: src/test-home-guard.cts, ' +
|
||||
'control run on next tip 622f43353: 37199 passed / 1 failed, throw · src/test-home-guard.cts). ' +
|
||||
'Remedy: rename the file out of the pattern; do NOT add a runner-side exclusion.',
|
||||
});
|
||||
}
|
||||
|
||||
violations.sort((a, b) => a.file.localeCompare(b.file));
|
||||
|
||||
return { ok: violations.length === 0, violations, scanned: out.scanned };
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
checkSourceTestNameCollisions,
|
||||
COLLECTED_EXTENSIONS,
|
||||
DEFAULT_SCAN_DIRS,
|
||||
GENERATED_OUTPUT_PREFIXES,
|
||||
isUnderLiteralTestDir,
|
||||
matchingBasenamePatterns,
|
||||
};
|
||||
|
||||
function main() {
|
||||
const ROOT = path.join(__dirname, '..');
|
||||
const result = checkSourceTestNameCollisions({ root: ROOT });
|
||||
|
||||
if (!result.ok) {
|
||||
process.stderr.write(
|
||||
`lint-source-test-name-collision: ${result.violations.length} violation(s) among ${result.scanned.length} scanned file(s)\n\n`
|
||||
);
|
||||
for (const v of result.violations) {
|
||||
process.stderr.write(` ${v.file}\n ${v.reason}\n`);
|
||||
}
|
||||
throw new ExitError(1);
|
||||
}
|
||||
|
||||
console.log(`ok lint-source-test-name-collision: ${result.scanned.length} file(s) scanned, 0 violations`);
|
||||
}
|
||||
|
||||
if (require.main === module) runMain(main);
|
||||
56
scripts/select-docs-guards.cjs
Normal file
56
scripts/select-docs-guards.cjs
Normal file
@@ -0,0 +1,56 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* select-docs-guards.cjs — pure selector mapping a PR's changed docs/ paths
|
||||
* to the subset of scripts/docs-guard-registry.cjs's DOCS_GUARD_TESTS that
|
||||
* actually reads any of them (#3753 follow-up).
|
||||
*
|
||||
* Deliberately dependency-free: no fs, no git, no process. Callers (the
|
||||
* docs-required.yml workflow, tests) are responsible for producing the
|
||||
* changed-paths list and reading the registry; this module only implements
|
||||
* the matching semantics so they are independently unit-testable.
|
||||
*
|
||||
* Pattern semantics (mirrors scripts/docs-guard-registry.cjs's header doc):
|
||||
* - a plain path matches an exact changed path;
|
||||
* - a trailing-slash path is a DIRECTORY PREFIX match — 'docs/adr/'
|
||||
* matches 'docs/adr/README.md' but must NOT match 'docs/adrenaline.md'
|
||||
* (a naive `startsWith('docs/adr')` without the trailing slash would
|
||||
* wrongly match the latter; matching against the full prefix INCLUDING
|
||||
* the trailing slash is what keeps this boundary correct);
|
||||
* - the sentinel '*' matches any non-empty changedDocsPaths.
|
||||
*/
|
||||
|
||||
/**
|
||||
* @param {string} changedPath - a single changed docs/ path (e.g. 'docs/AGENTS.md').
|
||||
* @param {string} pattern - one entry from a registry value array.
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function patternMatches(changedPath, pattern) {
|
||||
if (pattern === '*') return true;
|
||||
if (pattern.endsWith('/')) return changedPath.startsWith(pattern);
|
||||
return changedPath === pattern;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string[]} changedDocsPaths - repo-relative paths under docs/ that
|
||||
* changed in this PR. An empty array always yields an empty selection.
|
||||
* @param {Record<string, string[]>} registry - DOCS_GUARD_TESTS shape: test
|
||||
* file -> array of patterns it reads.
|
||||
* @returns {string[]} sorted, deduped list of selected test file paths.
|
||||
*/
|
||||
function selectDocsGuards(changedDocsPaths, registry) {
|
||||
if (!Array.isArray(changedDocsPaths) || changedDocsPaths.length === 0) return [];
|
||||
|
||||
const selected = new Set();
|
||||
for (const [testFile, patterns] of Object.entries(registry)) {
|
||||
const hit = patterns.some((pattern) =>
|
||||
changedDocsPaths.some((changedPath) => patternMatches(changedPath, pattern)),
|
||||
);
|
||||
if (hit) selected.add(testFile);
|
||||
}
|
||||
|
||||
return [...selected].sort();
|
||||
}
|
||||
|
||||
module.exports = { selectDocsGuards, patternMatches };
|
||||
Reference in New Issue
Block a user