test(suites): seed security/slow/integration suites via measured retags

Renames (git mv) with all references updated (ci-test-scope RULES,
windows-parity allowlist, test-file-count allowlist, docs in 6 locales):

- 5 scanner tests -> *.security.test.cjs — the 'Run security tests' CI step
  ran zero files since the suite taxonomy landed; it is now honest.
- graphify-auto-update -> *.slow.test.cjs (36s, slowest file in the suite;
  e2e gsd-tools spawns) — runs on full-matrix lanes and push to next.
- installer-migration-install-integration -> *.integration.test.cjs
  (13s; an integration test by its own name).

Coverage gate measured after retags: 88.55% lines (gate 70%).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colin
2026-06-09 23:01:00 -04:00
parent a647053dcf
commit cd5db1f8db
22 changed files with 161 additions and 30 deletions

View File

@@ -29,6 +29,23 @@ Examples:
The suite-suffix convention was chosen over a directory layout (`tests/security/`) so the 545+ existing test files don't need to move. Existing files all classify as `unit` until someone explicitly retags them.
## Regression tests
**Do not create new top-level `tests/bug-NNNN-*.test.cjs` files.** Add the
regression case to the owning module's main test file instead (e.g. a
`describe('regressions')` block in `tests/<module>.test.cjs`).
`node --test` spawns one child process per FILE, so file count — not test
count — is the unit of CI overhead, and it is worst on Windows lanes where
every spawn is Defender-scanned. The 2026-06 CI audit found 244 one-off
`bug-*` files (~38% of the suite). That population is grandfathered in
`scripts/lint-regression-test-names.allowlist.json` and enforced by an
identity ratchet (`npm run lint:regression-names`, part of `npm run lint:ci`):
- A **new** `bug-*` file fails CI — fold it into the owning module's file.
- **Deleting/consolidating** a grandfathered file requires pruning its
allowlist entry, so the baseline only ever shrinks.
## Running suites locally
```bash
@@ -53,6 +70,12 @@ node scripts/run-tests.cjs --files "tests/command-contract.test.cjs tests/core.t
node scripts/run-tests.cjs --files-from .ci-selected-tests.txt
```
`npm run test:affected` (scripts/run-affected-tests.cjs) is a **local-only**
convenience that selects tests via the `require()` dependency graph of your
working-tree diff. CI does not use it — CI selection is the rule table in
`scripts/ci-test-scope.cjs`, which is the authoritative mapping. If the two
disagree, trust (and fix) the rule table.
Unknown suites exit non-zero with the list of valid suites. Empty suites (e.g. `--suite security` before any security-tagged file exists) exit `0` with a `no tests in suite "..."` notice on stderr so CI lanes don't go red while a suite is being populated.
## CI matrix
@@ -72,14 +95,27 @@ The `Tests` workflow runs every PR through a scoped gate generated by
smoke set. They are for confidence on the affected surface, not for counting
tests.
The default PR gate runs the broad `unit`, `integration`, and `security` suites
once on Ubuntu / Node 24, scoped smoke on Ubuntu / Node 22, scoped
Windows/path/shell tests on Windows / Node 24, and unit coverage once on Ubuntu /
Node 24. PRs touching workflow, package, test-runner, install, release, or
The default PR gate runs the broad `unit` (under the c8 coverage gate),
`integration`, and `security` suites once on Ubuntu / Node 24, scoped smoke on
Ubuntu / Node 22, and scoped Windows-sensitive tests on Windows / Node 24.
**Every changed test file always joins the Windows scoped lane** (the #494
invariant, narrowed): a modified test is exercised on the divergent OS before
merge at per-file cost, without paying for the three full parity lanes.
PRs touching workflow, package, test-runner, install, release, or
Windows-sensitive surfaces also run the full parity matrix on macOS and the
older Windows runtime, plus `install` and `slow` on the primary Ubuntu lane.
Coverage stays single-lane because multiplying coverage across OS/runtime lanes
adds cost without improving the threshold signal.
Everything (including the full parity matrix) runs on every push to `next`,
which covers the residual macOS / Windows-Node-22 cross-product for scoped PRs.
Coverage runs inside the Ubuntu / Node 24 full lane (not a separate job — that
duplicated the entire unit run) and stays single-lane because multiplying
coverage across OS/runtime lanes adds cost without improving the threshold
signal. Note the gate's deliberate blind spot: it measures
`gsd-core/bin/lib/*.cjs` only — `scripts/`, `hooks/`, and `bin/` are
unenforced, and `stryker.config.mjs` additionally excludes ~48% of lib lines
from mutation testing (see the UNMUTATED list there). Widening either gate is
tracked work, not an accident to "fix" silently by raising thresholds.
To inspect the scope locally: