test(suites): seed security/slow/integration suites via measured retags
Renames (git mv) with all references updated (ci-test-scope RULES, windows-parity allowlist, test-file-count allowlist, docs in 6 locales): - 5 scanner tests -> *.security.test.cjs — the 'Run security tests' CI step ran zero files since the suite taxonomy landed; it is now honest. - graphify-auto-update -> *.slow.test.cjs (36s, slowest file in the suite; e2e gsd-tools spawns) — runs on full-matrix lanes and push to next. - installer-migration-install-integration -> *.integration.test.cjs (13s; an integration test by its own name). Coverage gate measured after retags: 88.55% lines (gate 70%). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -29,6 +29,23 @@ Examples:
|
||||
|
||||
The suite-suffix convention was chosen over a directory layout (`tests/security/`) so the 545+ existing test files don't need to move. Existing files all classify as `unit` until someone explicitly retags them.
|
||||
|
||||
## Regression tests
|
||||
|
||||
**Do not create new top-level `tests/bug-NNNN-*.test.cjs` files.** Add the
|
||||
regression case to the owning module's main test file instead (e.g. a
|
||||
`describe('regressions')` block in `tests/<module>.test.cjs`).
|
||||
|
||||
`node --test` spawns one child process per FILE, so file count — not test
|
||||
count — is the unit of CI overhead, and it is worst on Windows lanes where
|
||||
every spawn is Defender-scanned. The 2026-06 CI audit found 244 one-off
|
||||
`bug-*` files (~38% of the suite). That population is grandfathered in
|
||||
`scripts/lint-regression-test-names.allowlist.json` and enforced by an
|
||||
identity ratchet (`npm run lint:regression-names`, part of `npm run lint:ci`):
|
||||
|
||||
- A **new** `bug-*` file fails CI — fold it into the owning module's file.
|
||||
- **Deleting/consolidating** a grandfathered file requires pruning its
|
||||
allowlist entry, so the baseline only ever shrinks.
|
||||
|
||||
## Running suites locally
|
||||
|
||||
```bash
|
||||
@@ -53,6 +70,12 @@ node scripts/run-tests.cjs --files "tests/command-contract.test.cjs tests/core.t
|
||||
node scripts/run-tests.cjs --files-from .ci-selected-tests.txt
|
||||
```
|
||||
|
||||
`npm run test:affected` (scripts/run-affected-tests.cjs) is a **local-only**
|
||||
convenience that selects tests via the `require()` dependency graph of your
|
||||
working-tree diff. CI does not use it — CI selection is the rule table in
|
||||
`scripts/ci-test-scope.cjs`, which is the authoritative mapping. If the two
|
||||
disagree, trust (and fix) the rule table.
|
||||
|
||||
Unknown suites exit non-zero with the list of valid suites. Empty suites (e.g. `--suite security` before any security-tagged file exists) exit `0` with a `no tests in suite "..."` notice on stderr so CI lanes don't go red while a suite is being populated.
|
||||
|
||||
## CI matrix
|
||||
@@ -72,14 +95,27 @@ The `Tests` workflow runs every PR through a scoped gate generated by
|
||||
smoke set. They are for confidence on the affected surface, not for counting
|
||||
tests.
|
||||
|
||||
The default PR gate runs the broad `unit`, `integration`, and `security` suites
|
||||
once on Ubuntu / Node 24, scoped smoke on Ubuntu / Node 22, scoped
|
||||
Windows/path/shell tests on Windows / Node 24, and unit coverage once on Ubuntu /
|
||||
Node 24. PRs touching workflow, package, test-runner, install, release, or
|
||||
The default PR gate runs the broad `unit` (under the c8 coverage gate),
|
||||
`integration`, and `security` suites once on Ubuntu / Node 24, scoped smoke on
|
||||
Ubuntu / Node 22, and scoped Windows-sensitive tests on Windows / Node 24.
|
||||
**Every changed test file always joins the Windows scoped lane** (the #494
|
||||
invariant, narrowed): a modified test is exercised on the divergent OS before
|
||||
merge at per-file cost, without paying for the three full parity lanes.
|
||||
|
||||
PRs touching workflow, package, test-runner, install, release, or
|
||||
Windows-sensitive surfaces also run the full parity matrix on macOS and the
|
||||
older Windows runtime, plus `install` and `slow` on the primary Ubuntu lane.
|
||||
Coverage stays single-lane because multiplying coverage across OS/runtime lanes
|
||||
adds cost without improving the threshold signal.
|
||||
Everything (including the full parity matrix) runs on every push to `next`,
|
||||
which covers the residual macOS / Windows-Node-22 cross-product for scoped PRs.
|
||||
|
||||
Coverage runs inside the Ubuntu / Node 24 full lane (not a separate job — that
|
||||
duplicated the entire unit run) and stays single-lane because multiplying
|
||||
coverage across OS/runtime lanes adds cost without improving the threshold
|
||||
signal. Note the gate's deliberate blind spot: it measures
|
||||
`gsd-core/bin/lib/*.cjs` only — `scripts/`, `hooks/`, and `bin/` are
|
||||
unenforced, and `stryker.config.mjs` additionally excludes ~48% of lib lines
|
||||
from mutation testing (see the UNMUTATED list there). Widening either gate is
|
||||
tracked work, not an accident to "fix" silently by raising thresholds.
|
||||
|
||||
To inspect the scope locally:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user