test(#3145): bound the installer/runtime cluster onto the process seam (#3176)

* test(#3145): bound the installer/runtime cluster onto the process seam

Migrates 156 unbounded sync spawn sites across 47 files. Allowlist 120 to 73.

Timeouts are sized from evidence already in the tree rather than a house
default, because this wave spawns installers rather than git plumbing and an
undersized bound does not catch a hang -- it manufactures CI flake, which is
worse, since a flake gets re-run instead of investigated. install.test.cjs
records a real spawnSync ETIMEDOUT at a 60000ms cap on a loaded bench while
another lane passed the same commit in 12.7s, so full installs are bound at
120000ms against that recorded incident.

Also adds an auditable escape to the guard's timeout ceiling. The 600000ms
cap was set in #3143 from partial evidence, but fragment-single-edit-
propagation carries a documented, load-tested 900000ms bound on a run that
chains a full build plus eight generators -- the guard would have rejected a
correct timeout the moment that file left the allowlist. A value above the
ceiling is now permitted only with an inline allow-spawn-timeout-ceiling
marker carrying a non-empty reason. It raises the ceiling; it never waives
the requirement for a bound, which is asserted directly.

install-shared.cjs keeps its hand-rolled assert rather than routing through
throwIfFailed: its message embeds both streams, and throwIfFailed carries
only a trimmed stderr. The message now also names the outcome, so a bounded
timeout reads as such across its 38 importers instead of as
expected null to equal 0.

* test(#3145): extract class-norm timeouts and correct the build-hooks sizing

A pre-PR review found 52 copies of four class-norm timeout constants across
this wave. These are not per-suite fixture bindings -- they are shared facts
about how long a class of subprocess takes, derived from a recorded bench
incident. That norm already moved once (60000 to 120000 after a real
ETIMEDOUT), and 52 copies would have drifted the next time it moved.

Extracts tests/helpers/timeouts.cjs, where each norm is justified once, and
converts the copies. A site that genuinely differs -- a real tsc compile, or
regen:derived -- keeps its own local constant with its own justification.

Also corrects a misclassification: scripts/build-hooks.js was sized as a
build at 120000 in twelve places and 60000 in another, but it compiles and
bundles nothing. Its own header says no bundling needed; it copies pre-built
files and syntax-checks them with vm. Three different values bounded one
script; now there is one.

* test(#3145): fix red CI — lint self-match and a Windows chunk overrun

Two failures on PR 3176.

lint-allow-test-rule-refs read a RuleTester fixture as a real exemption. The
fixture exists to prove an unrelated marker does NOT suppress the rule, so it
carries that marker's literal text as test data. Split via concatenation, the
same idiom no-unbounded-spawn-allowlist.test.cjs already uses for its own
self-match problem. The explanatory comment needed the same treatment.

The Windows shard 3/3 chunk was killed at its 600000ms budget. Output stopped
seven minutes before the kill, so this was an overrun rather than a slow
chunk: regenDerivedPropagatesSingleFragmentEditWithNoSecondSourceSurface runs
regen:derived bounded at 900000ms, which is larger than the whole chunk
budget, so the chunk killer always fires first and it can never complete
there. Both the test and that bound predate this change; modifying the file
pulled it into the Windows targeted set and exposed it. Skipped on Windows
with the reason recorded; the Linux lanes cover it. The 900000 bound and its
ceiling marker are unchanged -- they are correct.

* test(#3145): refresh the stale test-timings cost table

The Windows shard was killed at its 600000ms per-chunk budget. run-tests.cjs
packs chunks by measured duration from tests/test-timings.json, and an
unknown file falls back to the table's median weight -- advisory by design,
but it silently underweights exactly the files that matter.

Four of the failing chunk's 22 files were absent from the table, including
the two heaviest: fragment-single-edit-propagation.install.test.cjs at 230s
(it runs regen:derived) and agent-fragments-emission.install.test.cjs at 79s.
Both were weighted as average, so the chunk's total weight read 53.68 against
a budget of 60 and the packer produced a single chunk.

Regenerated from a passing full-suite run, per the remedy the script itself
documents. 700 to 770 entries, 70 added, 0 dropped -- verified, since
gen-test-timings.cjs replaces the table wholesale rather than merging.

Proven against the real packer: the same 22 files now weigh 103.91 and split
into two chunks. No logic, budget, or timeout was changed; raising a budget
to make a red gate pass is not a fix.

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-07 15:18:18 -04:00
committed by GitHub
parent 27aa40f65e
commit 3fac6e629f
68 changed files with 2042 additions and 1388 deletions

View File

@@ -427,7 +427,7 @@ The single subprocess-spawning primitive test code uses (`tests/helpers/process-
The throw-preserving companion to the process seam (`tests/helpers/git-fixture.cjs`, #3143): `gitOrThrow(args, options)` runs `runGit` and returns `stdout` as a string on a clean exit, but throws on any other outcome. It exists because the seam **deliberately never throws** while `execSync` and `execFileSync` — the two forms 237 migrating call sites use — both throw on a non-zero exit. Migrating those mechanically onto `runGit` would convert a loud failure into a silent one: fixture setup that failed would return an empty string and surface as a baffling assertion failure further down. The thrown error carries `status` **and** `exitCode` as deliberate aliases (`status` is what the legacy `execSync` catch idiom reads, e.g. `tests/worktree-safety.test.cjs:1361`), plus `stdout`, `stderr`, `signal`, `timedOut` and `outcome`. Use `runGit` when every outcome is data you branch on; use `gitOrThrow` for fixture setup that must abort loudly. The seam module is **not** modified to add this — a throwing export would falsify the never-throws contract stated in its own header and in the `### Process seam` entry above. The module also exports `throwIfFailed(result, displayName)`, the single implementation of that throw shape: `gitOrThrow` itself is `throwIfFailed` specialized to `runGit`, so it routes through the same code path and the two cannot drift apart. Per-suite wrappers driving non-git targets — a node CLI via `runNode`, a bash snippet via `runHook` — call `throwIfFailed` directly rather than hand-rolling their own copy of this shape, which is exactly how five call sites had drifted from each other before this module exported it (#3144).
### Unbounded-spawn guard
The lint rule enforcing `DEFECT.UNBOUNDED-SUBPROCESS` across the test suite (`eslint-rules/no-unbounded-spawn.cjs`, #3143, wired into the `tests/**/*.cjs` block of `eslint.config.mjs`). Flags `spawnSync` / `execFileSync` / `execSync` whose options carry no usable `timeout`. It resolves renamed destructures (`const { execSync: exec } = require('node:child_process')`) and chained requires (`require('node:child_process').execSync(...)`) rather than matching literal callee names — both forms exist in the suite today and a name-only matcher leaves them permanently invisible. It resolves an options object held in a single-write `const`, which is what keeps `process-seam.cjs` — the bounded reference implementation — from flagging itself. Two values are rejected as *nominally* bounded: `timeout: 0` (Node reads zero as no timeout) and anything above the 600000 ms ceiling (effectively unbounded); a non-literal value is trusted, since the target shape is one named constant with a comment. `no-unbounded-spawn.allowlist.json` grandfathers pre-existing violations and ratchets **down only** — a listed file with zero violations reports its own entry as stale, so the list cannot go quiet while the class survives. Companion tests assert the list never grows, carries no dead entries, and that no `eslint-disable` for this rule exists anywhere under `tests/`.
The lint rule enforcing `DEFECT.UNBOUNDED-SUBPROCESS` across the test suite (`eslint-rules/no-unbounded-spawn.cjs`, #3143, wired into the `tests/**/*.cjs` block of `eslint.config.mjs`). Flags `spawnSync` / `execFileSync` / `execSync` whose options carry no usable `timeout`. It resolves renamed destructures (`const { execSync: exec } = require('node:child_process')`) and chained requires (`require('node:child_process').execSync(...)`) rather than matching literal callee names — both forms exist in the suite today and a name-only matcher leaves them permanently invisible. It resolves an options object held in a single-write `const`, which is what keeps `process-seam.cjs` — the bounded reference implementation — from flagging itself. Two values are rejected as *nominally* bounded: `timeout: 0` (Node reads zero as no timeout) and anything above the 600000 ms ceiling (effectively unbounded); a non-literal value is trusted, since the target shape is one named constant with a comment. `no-unbounded-spawn.allowlist.json` grandfathers pre-existing violations and ratchets **down only** — a listed file with zero violations reports its own entry as stale, so the list cannot go quiet while the class survives. Companion tests assert the list never grows, carries no dead entries, and that no `eslint-disable` for this rule exists anywhere under `tests/`. #3145 added an auditable ceiling escape: a literal timeout over the 600000 ms ceiling is permitted, without `timeoutTooLarge`, only when the call carries an inline `// allow-spawn-timeout-ceiling: <reason>` marker comment (the `// allow-test-rule: <reason>` idiom) with a non-empty reason, on the line immediately above the call or anywhere inside the call's own source range. It binds to that call only — a marker on one call never suppresses a different over-ceiling call elsewhere in the file — and it is deliberately narrow: it raises the ceiling for a call that already has a resolvable numeric timeout, it never waives the requirement for a bound, so a marked call with no `timeout` at all still reports `unboundedSpawn`. The real load-tested exception this exists for is `fragment-single-edit-propagation.install.test.cjs`'s `timeout: 900000` on `npm run regen:derived` (a full build plus eight generators), where 300000 was observed killing a genuinely-completed run near the end on a loaded bench.
### Deterministic scheduler
Test-execution model in which all timing and concurrency outcomes are fully controlled by the test (via clock seam, explicit `await` ordering, or synchronous stepping) rather than by the OS thread scheduler. Opposed to real-race tests, which are non-deterministic on loaded CI runners.