* test(#4641): failing-first tests for the tier ceiling and a single Windows selector Tests only, committed ahead of the implementation so the RED run is real. - tests/platform-conformance-tier.test.cjs: tier-size ceiling asserted as a ratio against a live denominator (Windows 33%, macOS 25%); per-helper negative cases proving seam calls and path-call-plus-slash-literal are not platform signals; positive pins that genuine platform content, seam-bypassing spawns, chmod and symlink still classify in; macOS signal set and generated list unchanged. - tests/ci-full-lane-sharding.test.cjs: the test job has zero windows-latest rows and test-conformance still has 3 windows + 1 macOS. - tests/ci-test-scope.test.cjs: windows_tests is absent rather than empty, a non-tier test file no longer forces full_matrix, a RULE-pulled windows-hint test does, and resolveSelection rejects the retired windows scope. Refs #4589, #4591, #4592, #4593, #4603 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4641): delete the second Windows selector and narrow the conformance tier Epic #4589's goal — the OS-agnostic bulk on Linux, a small explicitly-scoped conformance tier on real Windows/macOS — was not met. Measured on PR #4640 (run 34618834118): 7 non-Linux jobs, a 546/930 (58.7%) "tier", and 5 of 7 changed test files running on a real Windows runner twice. Two selectors, only one in the epic's scope. The test job's three scope:windows shards predate the epic (#494, sharded #3057) and gate on product_changed, not full_matrix, so they fire on every product PR whatever Phase 3's classifier decides. They are deleted; test-conformance becomes the sole Windows selector, as it already was for macOS. Non-Linux jobs 7 -> 4. Gating the lane instead was rejected as provably redundant: for a test file reachesConformanceTierOrSeam is literally CONFORMANCE_TIER_FILES.includes(file), and that same predicate sets full_matrix, which turns test-conformance on. Every file a gated lane would run is already covered in the same run. The lane's one non-redundant residue -- RULE-pulled tests matched by the isWindowsHint filename heuristic -- is ported into reachesConformanceTierOrSeam so it sets full_matrix instead of feeding a parallel lane. Two detectors matched the repo's own test idiom rather than any platform signal and carried 226 of the tier's sole-signal membership against 41 for the other eight: process-seam-subprocess (335 files, 118 unique) matches the tests/helpers.cjs entry points nearly every CLI test uses, and going through the seam is the opposite of a platform signal since shell-command-projection takes platform as an injected parameter; hardcoded-path-vs-path-call (328, 108) needs only a path call anywhere plus a slash literal anywhere, and that class is already enforced by ADR-1703's Linux-runnable ESLint rules. Both are removed. Tier 546 -> 254 (27.3%). src/ reachability is unchanged at 28 files, measured. Adds the size gate Phase 2 never had, as a ratio against a live denominator so it cannot stop binding as the suite grows. 292 files leave real-OS Windows execution. The drop-out set was audited: 14 have a platform-suggestive filename and all 14 are static source-text analyses or seam-mediated CLI tests. raw-child-process was investigated as a suspected false negative and left unchanged -- relaxing it adds 13 files, all false positives. macOS is untouched: MACOS_CATEGORIES is a separate array and the regenerated macos-conformance-tier.generated.cjs is byte-identical at 196 files. Fixes #4641 Refs #4589, #4591, #4592, #4593, #4603 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4641): register the new ADR path in the docs-guard exempt baseline tests/ci-test-scope.test.cjs references docs/adr/4641-windows-selector-consolidation.md in a comment justifying the retired windows scope; lint-docs-guard-registration tracks that reference set, so the baseline needs the new path. Verified the exemption still holds: the path is prose, not a filesystem read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4641): make the escalation tier-backed and drop every hardcoded count Three follow-ups from measuring the first pass rather than trusting it. The windows-hint escalation now requires tier membership as well as the filename hint. Setting full_matrix runs test-conformance, which runs only the tier; escalating on a test that is NOT in the tier costs four jobs and still never runs that test on Windows. Measured over the 16 RULES entries the narrowed predicate fires on exactly the same rules today, so this is correct-by-construction rather than a behavior change. The broader variant -- escalate on any tier member a rule pulls in, ignoring the hint -- was measured at 14/16 rules and rejected as over-broad. Removes the hardcoded counts. A hardcoded macOS tier length of 196 broke as soon as the rebase pulled in one new test file from #4253, which is the whole argument against them: the ceilings are ratios against a live denominator, the committed lists are pinned by comparison against a fresh classification of the live tree, and the three named probe files now assert on their SIGNAL rather than on membership in a literal list -- asserting by filename is the exact error this PR fixes in the classifier. Regenerates both lists against the rebased tree. Same-tree figures are now 547 -> 255 of 931 eligible (58.8% -> 27.4%), 292 entries removed and none added; macOS is unchanged at 197 with a zero-line diff. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4641): restore real-shell-spawn coverage and repair assertions the narrowing broke An isolated adversarial review found a real false negative. Removing the blanket process-seam-subprocess detector also removed the only coverage for tests that spawn a REAL shell: tests/helpers/process-seam.cjs's runHook spawns options.interpreter via real spawnSync, so runHook('-c', [script], { interpreter: 'bash' }) runs a real bash binary executing a shell script extracted from workflow markdown. The seam argument holds for src/shell-command-projection.cts, which takes platform as an injected parameter; it does NOT hold for the test helpers, which spawn real binaries. Conflating the two is what made the blanket detector look purely noisy -- it was 99% noise wrapping a real signal. Adds a narrow shell-interpreter-spawn category keyed on a real interpreter option. Measured 2026-09-11: 33 files match, 9 were outside the tier and are added back, taking it 255 -> 264 of 931 (27.4% -> 28.4%), still under the 33% ceiling. All 9 confirmed by reading the matching source line, zero comment or fixture matches. runGit-alone and non-node-spawnSeam alternatives were measured and rejected -- each adds 9 files but misses the counterexample entirely. Fixes a real bug the suite caught: jobs.test is ubuntu-only now that its scope:windows rows are gone, so it must wire GSD_STRICT_LIVE_CONFIG_GUARD strictly rather than carrying the Windows report-only carve-out. The carve-out now lives solely on test-conformance, whose matrix does include windows. Repairs seven pre-existing assertions the category removal invalidated, preserving each case's purpose rather than deleting coverage, and converts the last hardcoded tier bounds to live-derived ratios -- including the macOS sanity range that was still a magic [100, 350]. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4641): keep the confinement test on a real OS via a documented allowlist A security review found tests/external-descriptor-confinement.test.cjs had dropped out of the Windows tier. It must stay in, and no content signal can express why: it exercises isPathConfined (src/external-descriptor-trust.cts), which uses the AMBIENT path module -- path.resolve(root, target) and path.sep -- with no injection. Its win32 semantics (drive letters, UNC, separator) are only reachable by actually running on Windows, and it is a security-relevant write-confinement gate. A content classifier cannot see 'this module reads the ambient path module', so no regex belongs here. Adds ALWAYS_REAL_OS, a Map of path -> recorded reason, unioned into the Windows tier only. A Map rather than a list so an entry without a reason is impossible by construction, and tests assert every entry names a file that exists on disk so a stale entry fails loudly instead of rotting. This is the centrally- enumerated single source of truth epic #4589 Phase 2 asked for and ADR-1703's portability-vocab.cjs already models -- deliberately not a heuristic. Windows tier 264 -> 265 of 931 (28.5%), still under the 33% ceiling. macOS is untouched and byte-identical: the win32 concern does not apply to a POSIX runner, and a test asserts the allowlist does not leak into that tier. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4641): inject the path impl into isPathConfined and correct the ADR count Two review findings, both fixed rather than dispositioned. A security review found tests/external-descriptor-confinement.test.cjs had left real-OS execution. The allowlist pinned it back, but that only restored INCIDENTAL coverage: isPathConfined used the ambient path module, and its test carried POSIX-only literals, so a win32 confinement escape was unverified on every platform including Windows. isPathConfined now takes an optional third parameter carrying the path implementation, defaulting to the ambient module. Blast radius is CRITICAL -- 53 affected symbols across 19 files -- so the change is purely additive and every existing two-argument caller is byte-identical. Tests now inject path.win32 and path.posix, covering a different drive letter, a cross-drive absolute, backslash and forward-slash traversal, UNC, and the startsWith prefix-boundary bug (.gsdEVIL against root .gsd) on both separators. Proved load-bearing: dropping the + p.sep from the prefix check fails exactly the two boundary cases and nothing else. Callers' suites 149/149. The spec review caught an off-by-one: the ADR narrated a 264-file tier while the committed list holds 265. The ADR now records the full chain 547 -> 255 -> 264 -> 265 (28.5%). Also corrects a stale comment in scripts/docs-guard-registry.cjs that narrated classify() as zeroing windows_tests, a key this change removes -- kept as historical narration but labelled as such. Refs #4641 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#131): make the unwritable-HOME test actually test something Found by sweeping for the root-bypass class after fixing commit-files-deletion. This one is the silent variant, and it was broken twice over. First, the condition: the test made a fake HOME unwritable with chmod 0o500. The gsd-test Docker bench runs as root, root bypasses mode bits, so HOME stayed writable and the hostile condition never existed. Replaced with a HOME whose PARENT is a regular file, so every write under it fails ENOTDIR at the VFS layer for every uid -- no permission check is involved at all. Second, and more fundamental: the probe was npm --version, which on npm 11.19.0 performs zero filesystem I/O against HOME. Proven rather than assumed -- neutralizing runNpm()'s isolation turned the sibling test red while this one stayed green, so its assertion could never detect the regression it guards, on any uid, with or without the condition fix. npm config get cache was tried next and proved vacuous the same way (it only string-resolves the path). The probe is now npm cache verify, which really does mkdir _cacache under HOME. Re-proved load-bearing after the change: with isolation neutralized the test now fails with ENOTDIR on <blocker>/home/.npm/_cacache. tests/helpers.cjs was restored and verified diff-clean; suite 13/13. Refs #4641 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4641): correct the net drop-out figure in ADR-4641 The Consequences section still said 292 files leave real-OS Windows execution. That was the count before the narrow shell-interpreter-spawn replacement restored 9 and ALWAYS_REAL_OS pinned 1. Net is 282. Also names both real-binary categories rather than only raw-child-process, and clarifies that the 14-file filename audit was against the 292 initially dropped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4641): record the rejected concentration ceiling and its measurement Applying Goodhart's own question to the new ceiling -- how would you make this metric look good without improving what it represents -- surfaces a real weakness: a ratio can be satisfied by inflating the denominator, so adding OS-agnostic tests loosens it without narrowing the tier. The obvious companion gate was a sole-signal concentration ceiling, since the original defect was one detector carrying half the tier. Measured and rejected: peak concentration post-fix is raw-child-process at 53/265 = 20.0%, against the historic offenders at 21.6% and 19.8%. Any threshold above 20% misses the original defect; any threshold below it fails on a legitimate category. The discriminator is whether a signal is platform-meaningful, which no threshold encodes. Weakness disclosed rather than covered by a gate that does not bind. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(#4641): add the changeset fragment for the confinement-check change changeset-lint failed on PR #4643: the PR touches user-facing paths and carried no fragment. The earlier no-changeset call matched #4604's CI-only precedent and was correct then; it was not revisited once the PR grew a src/ change, which is my miss. The fragment describes the real user-visible improvement: the external-descriptor write-confinement check's Windows semantics are now verified deterministically rather than only when the suite happened to run on Windows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4641): correct the tier count in TESTING-SUITES.md Said the tier narrowed from 546 to 254. The final committed list is 265 of 931 eligible (58.8% -> 28.5%) after the shell-interpreter-spawn replacement restored 9 files and ALWAYS_REAL_OS pinned 1. Same error class the spec review caught in the ADR, in a live reference page rather than a dated record, so it states the current truth rather than carrying an amendment note. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4641): record the measured aggregate from real CI job lists Epic #4589's closeout asserted its reduction from a static count; #4641's acceptance criterion asks for a figure read off a real run. Recorded here: test.yml job count 21 -> 15 and non-Linux 7 -> 4, comparing PR #4640's run against this PR's own. Against the true pre-epic baseline of 9, that is 9 -> 4. Also states the caveat that a PR's total CHECK count is not a clean before/after comparison, since many gates are path-scoped and this change touches a broader path set -- the like-for-like figure is the test.yml job count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4641): compare job totals the same way on both sides The measured-aggregate table put #4640's COMPLETED run total (21) against this run's count at matrix-expansion time (15). Those are not the same measurement: the completed total includes the post-test Coverage gate and baseline-publisher jobs. Counted identically, it is 21 -> 17. The load-bearing figure, non-Linux jobs 7 -> 4, was correct and is unchanged. Called out in the table rather than silently corrected -- comparing two differently-derived numbers is exactly the error class this ADR is about. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4641): record measured conformance wall-clock and date the stale counterfactual Adds the per-job durations from both runs. The honest read is that this is a correctness win more than a speed one: file count fell 52% but wall-clock only 9-29%, because what was removed were the cheap static tests and what remains is concentrated in expensive spawn-heavy work. Stated explicitly so nobody expects a future narrowing to buy time proportional to file count. The load-bearing figure is windows shard 3/3: 40m24s against a 45-minute cap on the 547-file tier -- 90% of the cliff #869 and #3057 were both filed about -- pulled back to 31m27s. macOS moved the wrong way (17m48s -> 21m02s) while its tier was UNCHANGED at 197 files, which fixes that as runner variance and is noted as a caution against reading a single duration as signal. Also dates the symlink-keyword counterfactual, which cited a 254-file tier from before the replacement category and allowlist took it to its final 265. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4641): re-measure against the rebased tree and disclose the allowlist's zero next gained #4644 mid-flight, so every absolute count shifted. Re-measured on the tree this actually ships against (932 eligible): 548 -> 257 by detector removal, 257 -> 266 once shell-interpreter-spawn restores 9. Net 282 removed, 9 restored. macOS 198, unchanged by this PR. The percentages did not move across three rebases (58.8% -> 28.5%), which is the whole argument for expressing the ceilings as ratios rather than counts -- noted in the ADR since it is now evidence rather than assertion. Also discloses that ALWAYS_REAL_OS now contributes ZERO files: this PR's own win32 test cases introduced the literal win32 into the pinned file, so it classifies in on content via win32-darwin-literal. The entry stays and the reason is written down, because the file's real-OS need is a property of the code under test (isPathConfined reads the ambient path module), not of the test's text -- the text that currently saves it is incidental and could be refactored away silently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
412 lines
19 KiB
JavaScript
412 lines
19 KiB
JavaScript
#!/usr/bin/env node
|
|
'use strict';
|
|
|
|
/**
|
|
* docs-guard-registry.cjs — the sole source of truth for which doc-reading
|
|
* test files the docs-guard lane must run.
|
|
*
|
|
* ## Why this is its own module, not a `scripts/ci-test-scope.cjs` RULE
|
|
*
|
|
* A prior version of this PR added a `docs guards` RULE to `RULES` in
|
|
* scripts/ci-test-scope.cjs, on the theory that classify()'s `!codeChanged`
|
|
* normalization (which at the time zeroed fullMatrix/targeted_tests/
|
|
* windows_tests for docs-only diffs) made the RULE inert to classify()'s
|
|
* scope decision. (#4641 later removed `windows_tests` from classify()'s
|
|
* output entirely; the normalization today only zeroes fullMatrix and
|
|
* targeted_tests. This paragraph is historical narration of a rejected
|
|
* design, not a description of current behaviour.)
|
|
*
|
|
* That is true for docs-ONLY diffs and FALSE for MIXED docs+code diffs:
|
|
* `codeChanged` is true whenever ANY changed file is product/pipeline code,
|
|
* so the normalization never runs, and every one of this registry's test
|
|
* files joined `targeted_tests` for every mixed PR. Probed on that version:
|
|
* `node scripts/ci-test-scope.cjs --files "docs/a.md src/semver.cts"`
|
|
* returned 25 targeted_tests, vs. 3 on `origin/next` — an unstated blowup
|
|
* of the scoped/targeted lane on every mixed docs+code PR.
|
|
*
|
|
* Root cause: `RULES` answers "given these changed files, what should
|
|
* test.yml run" for the MAIN scoped-lane pipeline. The docs-guard registry
|
|
* is a LANE MANIFEST for a completely different consumer (the `docs-lint`
|
|
* job in .github/workflows/docs-required.yml, gated on a `docs_changed`
|
|
* step output). Putting a lane manifest inside a scope-classification rule
|
|
* set was the defect; extracting it here removes any path by which it can
|
|
* influence classify() at all.
|
|
*
|
|
* ## Who reads this
|
|
*
|
|
* - .github/workflows/docs-required.yml (`docs-lint` job) — derives the
|
|
* PER-PR-SELECTED subset of this registry from scripts/select-docs-guards.cjs,
|
|
* which is itself driven by this module's DOCS_GUARD_TESTS map. This job has
|
|
* no `paths:` filter, so it always reports a status and can supply the
|
|
* already-required `docs-lint` context; a dedicated `paths:`-filtered
|
|
* workflow cannot be made required without hanging non-docs PRs forever.
|
|
* - scripts/lint-docs-guard-registration.cjs — derives its registration
|
|
* lint's comparison set from this module (so a docs-reading test file
|
|
* that is neither registered here nor exempted fails the lint).
|
|
* - scripts/select-docs-guards.cjs — the pure selector that maps a PR's
|
|
* changed docs/ paths to the subset of this registry that actually needs
|
|
* to run (#3753 follow-up: a flat "run everything" list is disproportionate
|
|
* for a single-file docs typo fix).
|
|
*
|
|
* ## Registry shape (#3753 follow-up)
|
|
*
|
|
* `DOCS_GUARD_TESTS` is a MAP from test file path to the array of docs/
|
|
* paths it actually reads, so a changed-docs-file can be resolved to the
|
|
* narrow subset of guards that read it, instead of always running the
|
|
* entire registry. Each value is a non-empty array of PATTERNS:
|
|
*
|
|
* - a plain path (e.g. `'docs/AGENTS.md'`) matches that exact file only;
|
|
* - a trailing-slash path (e.g. `'docs/adr/'`) matches any changed path
|
|
* under that directory prefix — use this for a test that walks or
|
|
* `readdirSync`s a whole docs/ subdirectory;
|
|
* - the sentinel `'*'` means "run on ANY docs/ change" — reserved for a
|
|
* test that cannot be resolved to a narrower set of paths (the path is
|
|
* computed, looped over an unresolvable variable, or the test walks
|
|
* docs/ generally). Conservative fallback: when in doubt, use `'*'`,
|
|
* never a guessed-narrow path — a false negative here (a guard that
|
|
* silently stops running) is exactly the #3753 defect class this
|
|
* registry exists to prevent.
|
|
*
|
|
* `DOCS_GUARD_TEST_FILES` (derived: `Object.keys(DOCS_GUARD_TESTS)`) is kept
|
|
* as a flat array export so the registration lint and its parity test
|
|
* continue to consume a plain file list without needing to know about the
|
|
* map shape.
|
|
*
|
|
* ## Adding a docs guard
|
|
*
|
|
* Add the test file's path (relative to the repo root, `tests/<file>`) as a
|
|
* key in DOCS_GUARD_TESTS below, with the docs/ paths it reads as the value
|
|
* (or `['*']` if that cannot be resolved narrowly). Do not duplicate this
|
|
* list anywhere else — a second, independently maintained list is exactly
|
|
* the #3753 defect class (10 registered-but-not-run guards, silently
|
|
* drifted) this registry exists to prevent from recurring.
|
|
*
|
|
* Sorted alphabetically by basename for unambiguous diffs.
|
|
*/
|
|
|
|
/**
|
|
* Duplicate of scripts/run-tests.cjs:50's `SUITES` array. That array is not
|
|
* exported by run-tests.cjs (module.exports there is deliberately narrow;
|
|
* run-tests.cjs's own behavior is intentional and out of scope for this
|
|
* registry to alter), so it cannot be imported directly without changing a
|
|
* shared, behavior-locked file.
|
|
*
|
|
* Why this must exist at all: `run-tests.cjs`'s `selectExplicitFiles`
|
|
* (scripts/run-tests.cjs:651) treats any registry entry that EQUALS a
|
|
* SUITES member (e.g. `all`, `unit`) as a suite selector, not a filename —
|
|
* so a typo in DOCS_GUARD_TESTS matching a suite name would silently run
|
|
* the ENTIRE suite inside the required `docs-lint` job instead of erroring.
|
|
* `assertNoSuiteCollision` below rejects that at the registry boundary
|
|
* instead.
|
|
*
|
|
* Divergence risk: if run-tests.cjs's real SUITES list ever changes and this
|
|
* copy is not updated, this check could under- or over-reject. That risk is
|
|
* covered by a parity test (tests/ci-docs-guard-registry.test.cjs) that
|
|
* drives run-tests.cjs's own exported `selectExplicitFiles` behaviorally —
|
|
* for every token here it asserts run-tests.cjs treats it as a suite
|
|
* selector (never "file not found"), and for a control non-member it
|
|
* asserts the opposite — so a real divergence fails that test rather than
|
|
* silently drifting.
|
|
*/
|
|
const RUN_TESTS_SUITES = ['all', 'unit', 'integration', 'install', 'security', 'slow', 'qa'];
|
|
|
|
/**
|
|
* Normalize a registry entry EXACTLY the way scripts/run-tests.cjs's
|
|
* `splitFileList` (:617-625) normalizes a requested token before its SUITES
|
|
* check (:651): strip a leading `tests/` and normalize `\`->`/`. Without this,
|
|
* comparing RAW registry keys (which all carry the `tests/` prefix by
|
|
* convention) against RUN_TESTS_SUITES misses the realistic typo `'tests/all'`
|
|
* entirely — proven by probe: `assertNoSuiteCollision(['tests/all'])` did not
|
|
* throw, and `selectExplicitFiles(allFiles, 'tests/all')` selected all 824
|
|
* files (the whole suite) inside the required docs-lint job.
|
|
*
|
|
* @param {string} t
|
|
* @returns {string}
|
|
*/
|
|
function normalizeForSuiteCheck(t) {
|
|
return t.replace(/\\/g, '/').replace(/^tests\//, '');
|
|
}
|
|
|
|
/**
|
|
* Reject any docs-guard registry entry that collides with a run-tests.cjs
|
|
* suite token (see RUN_TESTS_SUITES doc comment above). Throws with every
|
|
* offending entry named, rather than failing on only the first. Compares
|
|
* entries AFTER normalizeForSuiteCheck, mirroring run-tests.cjs's own
|
|
* splitFileList normalization, so a `tests/`-prefixed or backslash-spelled
|
|
* entry that would collide post-normalization is caught here too.
|
|
*
|
|
* @param {string[]} tests
|
|
*/
|
|
function assertNoSuiteCollision(tests) {
|
|
const collisions = tests.filter((t) => RUN_TESTS_SUITES.includes(normalizeForSuiteCheck(t)));
|
|
if (collisions.length > 0) {
|
|
throw new Error(
|
|
'docs-guard-registry: DOCS_GUARD_TESTS entry collides with a run-tests.cjs SUITES token ' +
|
|
`(${collisions.join(', ')}) — scripts/run-tests.cjs:651's selectExplicitFiles treats a ` +
|
|
'registry entry that equals a suite name as a suite selector, not a filename, so this ' +
|
|
'would silently run the entire suite instead of the intended file(s). Fix the registry entry.',
|
|
);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Map from docs-guard test file to the docs/ path patterns it reads. See
|
|
* this module's header comment for pattern semantics (exact / trailing-slash
|
|
* dir-prefix / `'*'` sentinel). Every value here was derived by reading the
|
|
* test file's actual read call(s) — not guessed — per #3753's own lesson: an
|
|
* unresolvable read is recorded as `'*'`, never a narrowed guess.
|
|
*/
|
|
const DOCS_GUARD_TESTS = {
|
|
'tests/adr-15-progress-converge.test.cjs': [
|
|
'docs/COMMANDS.md',
|
|
'docs/how-to/run-phases-autonomously.md',
|
|
],
|
|
// Walks docs/adr/ as a directory (builds/reads docs/adr/README.md and
|
|
// fixture ADRs throughout) and separately reads docs/contributor-standards.md.
|
|
'tests/adr-index-gate.test.cjs': ['docs/adr/', 'docs/contributor-standards.md'],
|
|
|
|
// Walks docs/features/ as a directory (the fragment corpus) and asserts the
|
|
// committed docs/FEATURES.md equals the generated projection of it (#3840).
|
|
'tests/features-index-gate.test.cjs': ['docs/features/', 'docs/FEATURES.md'],
|
|
'tests/agent-classification-parity.test.cjs': ['docs/AGENTS.md', 'docs/INVENTORY.md'],
|
|
// #3683: pins the learnings feature section's agreement with the canonical
|
|
// artifact registry (reads docs/FEATURES.md around the extract-learnings
|
|
// entry).
|
|
'tests/learnings.test.cjs': ['docs/FEATURES.md'],
|
|
'tests/analyze-dependencies.test.cjs': ['docs/COMMANDS.md'],
|
|
'tests/autonomous-converge.test.cjs': [
|
|
'docs/COMMANDS.md',
|
|
'docs/how-to/run-phases-autonomously.md',
|
|
],
|
|
'tests/capability-matrix-sync.test.cjs': ['docs/reference/capability-matrix.md'],
|
|
'tests/capability-registry.test.cjs': [
|
|
'docs/tutorials/build-your-first-capability.md',
|
|
'docs/tutorials/install-your-first-capability.md',
|
|
'docs/reference/capability-manifest.md',
|
|
],
|
|
'tests/claude-md.test.cjs': ['docs/COMMANDS.md'],
|
|
'tests/claude-orchestration.test.cjs': ['docs/explanation/claude-orchestration-capability.md'],
|
|
'tests/command-contract.test.cjs': [
|
|
'docs/INVENTORY.md',
|
|
'docs/ja-JP/INVENTORY.md',
|
|
'docs/ko-KR/INVENTORY.md',
|
|
'docs/zh-CN/INVENTORY.md',
|
|
'docs/pt-BR/INVENTORY.md',
|
|
],
|
|
// uncoveredFiles(...) scans 'docs' as a generic coverage-scan root
|
|
// (commit-files-pathspec.test.cjs:1618) — cannot be resolved to specific
|
|
// files without re-deriving the scan's own file-discovery logic.
|
|
'tests/commit-files-pathspec.test.cjs': ['*'],
|
|
'tests/compact-content-4139.test.cjs': ['docs/CONFIGURATION.md'],
|
|
'tests/config-field-docs.test.cjs': ['docs/CONFIGURATION.md'],
|
|
'tests/config.test.cjs': ['docs/CONFIGURATION.md'],
|
|
'tests/context-index-sync.test.cjs': ['docs/CONTEXT-INDEX.json'],
|
|
'tests/context-predicates-query.test.cjs': ['docs/contributor-standards.md'],
|
|
// SCAN_DIRS includes 'docs' and recursively walks every .md file under it
|
|
// (context7-tool-name-parity.test.cjs:31-38) — a generic tree walk, not a
|
|
// fixed file set.
|
|
'tests/context7-tool-name-parity.test.cjs': ['*'],
|
|
'tests/contributor-standards.test.cjs': ['docs/contributor-standards.md'],
|
|
'tests/cursor-reviewer.test.cjs': [
|
|
'docs/COMMANDS.md',
|
|
'docs/FEATURES.md',
|
|
'docs/ja-JP/COMMANDS.md',
|
|
'docs/ja-JP/FEATURES.md',
|
|
'docs/ko-KR/COMMANDS.md',
|
|
'docs/ko-KR/FEATURES.md',
|
|
],
|
|
'tests/discuss-all-flag.test.cjs': ['docs/COMMANDS.md'],
|
|
'tests/discuss-mode.test.cjs': ['docs/workflow-discuss-mode.md'],
|
|
// Walks docs/*.md and every docs/<locale>/*.md dir dynamically
|
|
// (docs-parity-live-registry.test.cjs:42, 428) — deliberately generic.
|
|
'tests/docs-parity-live-registry.test.cjs': ['*'],
|
|
// #3839: pins every hook-table Event cell in the five locales' ARCHITECTURE
|
|
// and INVENTORY files against src/runtime-hooks-surface.cts registrations.
|
|
'tests/docs-hooks-table-parity.test.cjs': [
|
|
'docs/ARCHITECTURE.md',
|
|
'docs/INVENTORY.md',
|
|
'docs/ja-JP/ARCHITECTURE.md',
|
|
'docs/ja-JP/INVENTORY.md',
|
|
'docs/zh-CN/ARCHITECTURE.md',
|
|
'docs/zh-CN/INVENTORY.md',
|
|
'docs/ko-KR/ARCHITECTURE.md',
|
|
'docs/ko-KR/INVENTORY.md',
|
|
'docs/pt-BR/ARCHITECTURE.md',
|
|
'docs/pt-BR/INVENTORY.md',
|
|
],
|
|
'tests/docs-state-md-locale-parity.test.cjs': [
|
|
'docs/reference/state-md.md',
|
|
'docs/ja-JP/reference/state-md.md',
|
|
'docs/zh-CN/reference/state-md.md',
|
|
'docs/ko-KR/reference/state-md.md',
|
|
'docs/pt-BR/reference/state-md.md',
|
|
],
|
|
'tests/drift-detection.test.cjs': ['docs/CONFIGURATION.md', 'docs/AGENTS.md'],
|
|
'tests/edge-probe-docs-fixtures.test.cjs': ['docs/adr/550-spec-phase-probe-contract.md'],
|
|
'tests/edit-phase.test.cjs': [
|
|
'docs/INVENTORY.md',
|
|
'docs/INVENTORY-MANIFEST.json',
|
|
'docs/COMMANDS.md',
|
|
],
|
|
'tests/effort-surface-axis.test.cjs': ['docs/reference/host-integration-capability-matrix.md'],
|
|
'tests/execute-phase-active-flags.test.cjs': [
|
|
'docs/reference/host-integration-capability-matrix.md',
|
|
],
|
|
'tests/execute-phase-wave.test.cjs': ['docs/COMMANDS.md'],
|
|
// #3913 (ADR-3889 terminal phase): reads the generated docs/reference/exit-codes.md
|
|
// (content invariants, F1/F3) and docs/README.md (F4, the index link).
|
|
'tests/exit-code-registry.test.cjs': ['docs/reference/exit-codes.md', 'docs/README.md'],
|
|
'tests/external-job-waiting.test.cjs': ['docs/reference/planning-artifacts.md'],
|
|
// Rows 8/9 (negative controls) read real docs/registries/eos.json and
|
|
// docs/adr/0001-dispatch-policy-module.md and assert on their EXACT
|
|
// committed content (an entry's name field, ADR-0001's H1 title) as a
|
|
// sanity check before mutating an overlay fixture — a content edit to
|
|
// either file changes the string this test asserts on.
|
|
'tests/fragment-single-edit-propagation.install.test.cjs': [
|
|
'docs/registries/eos.json',
|
|
'docs/adr/0001-dispatch-policy-module.md',
|
|
],
|
|
// Seeds a temp fixture copy of these five files (never mutates the real
|
|
// tree) to exercise scripts/gen-state-md-docs.cjs's marked-region splicing
|
|
// against them — #3873 (ADR-3473 §8.8), rows 10-22/27. Read for fixture
|
|
// seeding, so a content edit to any of them (e.g. renaming a landmark
|
|
// heading/string a hostile-input test targets) can change this test's
|
|
// fixture assumptions.
|
|
'tests/gen-state-md-docs.test.cjs': [
|
|
'docs/reference/state-md.md',
|
|
'docs/ja-JP/reference/state-md.md',
|
|
'docs/zh-CN/reference/state-md.md',
|
|
'docs/ko-KR/reference/state-md.md',
|
|
'docs/pt-BR/reference/state-md.md',
|
|
],
|
|
'tests/gsd-write-guard.test.cjs': ['docs/USER-GUIDE.md'],
|
|
'tests/host-integration-descriptors.test.cjs': [
|
|
'docs/reference/host-integration-capability-matrix.md',
|
|
],
|
|
'tests/install.test.cjs': ['docs/AGENTS.md', 'docs/INVENTORY.md', 'docs/INVENTORY-MANIFEST.json'],
|
|
// SOURCE_DIRS includes 'docs' and walks it recursively for every .md file
|
|
// (intel.test.cjs:1223-1228) — a generic tree walk.
|
|
'tests/intel.test.cjs': ['*'],
|
|
'tests/inventory-headings-countfree.test.cjs': ['docs/INVENTORY.md'],
|
|
'tests/inventory-manifest-sync.test.cjs': ['docs/INVENTORY.md', 'docs/INVENTORY-MANIFEST.json'],
|
|
'tests/kilo-upgrades.test.cjs': ['docs/how-to/connect-gsd-mcp-server.md'],
|
|
'tests/live-config-guard.test.cjs': ['docs/TESTING-SUITES.md'],
|
|
// #3726: pins the `milestone complete` synopsis (English + four localized
|
|
// mirrors), the `--confirm` flag row and the guard-override instructions —
|
|
// `type: Fixed` exempts that PR from the docs-required lint, so this is the
|
|
// only gate on that prose.
|
|
'tests/milestone.test.cjs': [
|
|
'docs/CLI-TOOLS.md',
|
|
'docs/COMMANDS.md',
|
|
'docs/ja-JP/CLI-TOOLS.md',
|
|
'docs/ko-KR/CLI-TOOLS.md',
|
|
'docs/pt-BR/CLI-TOOLS.md',
|
|
'docs/zh-CN/CLI-TOOLS.md',
|
|
],
|
|
'tests/model-catalog-runtime-defaults.test.cjs': ['docs/CONFIGURATION.md'],
|
|
// Scans every git-tracked file in the whole repo via `git ls-files`
|
|
// (no-pending-3212-markers.test.cjs:37-46), which includes all of docs/ —
|
|
// cannot be resolved to a fixed docs/ path set.
|
|
'tests/no-pending-3212-markers.test.cjs': ['*'],
|
|
'tests/phase6-capability-docs.test.cjs': ['docs/how-to/develop-a-capability.md', 'docs/README.md'],
|
|
'tests/phase6-review-capabilities.test.cjs': [
|
|
'docs/reference/review-verification-capabilities.md',
|
|
'docs/how-to/develop-a-capability.md',
|
|
'docs/README.md',
|
|
],
|
|
'tests/plan-checker-coupling.test.cjs': ['docs/AGENTS.md'],
|
|
'tests/plan-phase-drift-guard.test.cjs': [
|
|
'docs/CONFIGURATION.md',
|
|
'docs/COMMANDS.md',
|
|
'docs/USER-GUIDE.md',
|
|
'docs/ARCHITECTURE.md',
|
|
'docs/INVENTORY.md',
|
|
'docs/INVENTORY-MANIFEST.json',
|
|
],
|
|
'tests/plan-phase-stall-detection.test.cjs': ['docs/CONFIGURATION.md'],
|
|
'tests/plan-review-convergence.test.cjs': [
|
|
'docs/CONFIGURATION.md',
|
|
'docs/USER-GUIDE.md',
|
|
'docs/ARCHITECTURE.md',
|
|
],
|
|
'tests/planner-estimate-emission.test.cjs': ['docs/reference/plan-md.md', 'docs/CONFIGURATION.md'],
|
|
'tests/precondition-element.test.cjs': ['docs/reference/plan-md.md'],
|
|
'tests/product-name-purity.test.cjs': [
|
|
'docs/README.md',
|
|
'docs/zh-CN/README.md',
|
|
'docs/ko-KR/README.md',
|
|
'docs/ja-JP/README.md',
|
|
'docs/pt-BR/README.md',
|
|
],
|
|
'tests/progress-forensic.test.cjs': ['docs/COMMANDS.md'],
|
|
'tests/repo-layout.test.cjs': ['docs/contributing/bootstrap.md'],
|
|
// Reads REQ-LANG-04 and runs every form it offers an author through the
|
|
// matcher that enforces it, so a reword of the requirement alone is exactly
|
|
// the change this guard must run on (#2529). Both paths are named because
|
|
// #3840 made docs/FEATURES.md a generated projection: the requirement's
|
|
// source is the fragment, and an edit there that is not regenerated would
|
|
// otherwise reach this guard through neither path.
|
|
'tests/response-language-coverage.test.cjs': [
|
|
'docs/FEATURES.md',
|
|
'docs/features/response-language-config.md',
|
|
],
|
|
'tests/reversibility-tagging.test.cjs': ['docs/reference/plan-md.md'],
|
|
'tests/reviewer-docs-parity.test.cjs': [
|
|
'docs/COMMANDS.md',
|
|
'docs/ja-JP/COMMANDS.md',
|
|
'docs/ko-KR/COMMANDS.md',
|
|
'docs/pt-BR/COMMANDS.md',
|
|
'docs/zh-CN/COMMANDS.md',
|
|
'docs/FEATURES.md',
|
|
'docs/ja-JP/FEATURES.md',
|
|
'docs/ko-KR/FEATURES.md',
|
|
'docs/pt-BR/FEATURES.md',
|
|
'docs/zh-CN/FEATURES.md',
|
|
],
|
|
'tests/runtime-converters.test.cjs': ['docs/CONFIGURATION.md'],
|
|
'tests/secure-phase.test.cjs': ['docs/CONFIGURATION.md'],
|
|
'tests/security-dead-exports.regression.test.cjs': ['docs/FEATURES.md'],
|
|
'tests/security.test.cjs': ['docs/INVENTORY-MANIFEST.json'],
|
|
// Walks docs/ recursively (todos-done-rename-guard.test.cjs:19-22
|
|
// SCAN_DIRS) looking for stale references — a generic tree walk.
|
|
'tests/todos-done-rename-guard.test.cjs': ['*'],
|
|
'tests/tracer-bullet.test.cjs': [
|
|
'docs/COMMANDS.md',
|
|
'docs/reference/plan-md.md',
|
|
'docs/how-to/plan-a-phase.md',
|
|
'docs/AGENTS.md',
|
|
],
|
|
'tests/ui-spec-inventory-provenance.test.cjs': [
|
|
'docs/FEATURES.md',
|
|
'docs/how-to/design-a-ui-phase.md',
|
|
'docs/ja-JP/FEATURES.md',
|
|
'docs/ja-JP/how-to/design-a-ui-phase.md',
|
|
'docs/zh-CN/FEATURES.md',
|
|
'docs/zh-CN/how-to/design-a-ui-phase.md',
|
|
'docs/ko-KR/FEATURES.md',
|
|
'docs/ko-KR/how-to/design-a-ui-phase.md',
|
|
'docs/pt-BR/how-to/design-a-ui-phase.md',
|
|
],
|
|
'tests/verifier-behavior-unverified.test.cjs': ['docs/reference/planning-artifacts.md'],
|
|
'tests/verifier-coincidental-reliance.test.cjs': ['docs/AGENTS.md'],
|
|
'tests/verify.test.cjs': ['docs/reference/plan-md.md'],
|
|
'tests/workflow-fragments.test.cjs': ['docs/reference/workflow-fragments.md'],
|
|
};
|
|
|
|
/**
|
|
* Flat file-list view of DOCS_GUARD_TESTS, kept for consumers (the
|
|
* registration lint and its parity test) that only need "which test files
|
|
* are registered", not their per-file docs path patterns.
|
|
*/
|
|
const DOCS_GUARD_TEST_FILES = Object.keys(DOCS_GUARD_TESTS);
|
|
|
|
assertNoSuiteCollision(DOCS_GUARD_TEST_FILES);
|
|
|
|
module.exports = {
|
|
DOCS_GUARD_TESTS,
|
|
DOCS_GUARD_TEST_FILES,
|
|
RUN_TESTS_SUITES,
|
|
assertNoSuiteCollision,
|
|
normalizeForSuiteCheck,
|
|
};
|