4 Commits

Author	SHA1	Message	Date
sim	ff5c5782ee	test(#4512): migrate process-seam batch to named timeout constants ... Batch 1 of the ad hoc timeout literal migration (epic #4445). Replaces every bare numeric timeout/timeoutMs object-literal property in tests/process-seam.test.cjs, tests/helpers-process-isolation.test.cjs, tests/run-with-timeout.test.cjs, and tests/helpers.cjs with a named constant, per eslint-rules/no-adhoc-timeout-literal.cjs. Removes the 4 files from the rule's allowlist. Adds SEAM_DEFAULT_TIMEOUT_MS to tests/helpers/timeouts.cjs (shared across 2 batch files, mirroring process-seam.cjs's own un-exported default). File-local constants elsewhere for values not shared across files or not a bench-derived class norm. No src/bin file touched, no numeric value changed anywhere. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-08 19:38:41 -04:00
Tom Boucher	c28134ab39	fix(#3271): delete 25 duplicated folded test suites and fix three runner defects found doing it (#3285) ... * test(#3271): guard against a folded suite appearing twice in one host Adds local/no-duplicate-fold-marker, an AST rule that reports the second and every subsequent `folded:<name>` marker in a host file, plus RuleTester cases and a tree-wide regression assertion. Failing-first on purpose: the rule is registered at error and the 25 duplicated regions are still present, so eslint and the new tree-wide test are RED. The deletions land in the next commit. The marker key is the whitespace-delimited token after `folded:` — not the issue's `[a-z0-9-]*` slice, which truncates at `.` and false-positives on tests/model-resolver.test.cjs where feat-443-effort-fast-mode.integration and feat-443-effort-fast-mode are two distinct folded suites. Refs #3271 * fix(#3271): delete 25 duplicated folded suites from three install hosts Three consolidated install suites each carried a verbatim second copy of a contiguous run of #1969 B1 folded blocks. Byte-identical, constant offset, and green — each duplicated block registered and ran twice on every lane. tests/install.test.cjs 5981-9937 (3957 lines, 18 blocks) tests/install-minimal-hooks.test.cjs 2734-4015 (1282 lines, 5 blocks) tests/install-write-confinement.test.cjs 1754-2321 ( 568 lines, 2 blocks) Introduced by 6d072435d (#1975 re-applying #1970's hunks on a tree that already had them, 2026-07-03) — one stale-base re-application, three files, one commit. Verified by marker-count bisect: 1 at 4f779eda4 and 0cc7a1a42, 2 from 6d072435d onward. The later copy is deleted in each case, so every file returns to what its authoring batch produced and blame on the surviving lines stays accurate. local/no-duplicate-fold-marker, red on the previous commit, is now green. tests/model-resolver.test.cjs is untouched: the issue lists it, but its two blocks are folded from two different files and are not identical. It is a false positive of the issue's own grep, whose `[a-z0-9-]*` key truncates at `.`. Fixes #3271 * test(#3271): property-test marker identity and pin the alias non-goal Three review findings, all fixed inline: 1. foldMarkerOf is a parser and carried no fast-check property test. Raised independently by the /code-review standards axis and the isolated adversarial pass; the file already establishes the fc.property-driving-ruleTester idiom for a sibling rule. Added, two arms over markers generated from [a-z0-9-._]: the same marker twice always reports exactly once against firstLine 1, and two distinct markers never collide. The alphabet includes `.` on purpose — an implementation keyed on the issue's [a-z0-9-]* slice passes arm 1 and fails arm 2, which is exactly the model-resolver false positive. 2. meta.docs.category was the novel value 'Test hygiene'; all 16 sibling local rules use 'Best Practices', 'Portability' or 'Reliability'. Now 'Best Practices'. 3. A call through a further alias (const d = __foldDescribe) was unreported and undocumented — accidental rather than deliberate. It is now the fourth entry in the rule's documented non-goals, with the reason, and pinned by a valid RuleTester case so it cannot drift silently. Refs #3271 * test(#3271): name the step and elapsed time when a baseline build fails buildBaselineAtRef runs four bounded steps and, when one exceeded its bound, threw a bare "spawnSync ETIMEDOUT" naming neither the step nor how long anything took. Diagnosing one real failure took four separate experiments to recover information the throw already had. Each step is now timed, and any throw carries the breakdown: which step failed, its elapsed time, the timings of every step that completed before it, all three bounds, and the tail of the child's captured stdout/stderr. The failure message is deliberately the carrier. On the remote runner the captured output field comes back empty in failures.json while error and stack survive verbatim, so the message is the only channel that reaches a reader of a remote verdict. Refs #3271 * fix(#3271): size the baseline generator bound for the machine it runs on Instrumentation from a real remote-runner failure gave the breakdown: git-worktree-add=15.1s npm-run-build-lib=19.8s gen-emitted-baseline=FAILED@300.1s Steps 1 and 2 are comfortable. Only the generator exceeds its bound, and it is not hung — it needs more than 300s there. Measured ladder for that step: ~22s idle in a container, ~39s end-to-end in a clean container, ~142s with 8 CPU burners on 8 cores, and >300s under the real suite. Its cost is 19 sequential installer spawns, and spawn latency is exactly where a container degrades worst (3.9x slower than host, against 1.1x for file IO) — which is why a CPU-only load test did not reproduce it and why four earlier hypotheses (container slowness, network, shallow clone, CPU contention) all measured clean. The 300s bound was sized on an idle machine for a step that never runs on one. Under the remote runner the on-disk baseline cache is structurally absent — CI restores it via actions/cache keyed on github.event.pull_request.base.sha, a key that exists only inside GitHub Actions — so this slow path runs on every remote verification. The result: this gate has passed 0 times in 754 runs, failing 80 times and never once executing successfully. Raised to the 600000ms ceiling that local/no-unbounded-spawn treats as the largest meaningful bound; the other two bounds are untouched. This makes the gate RUN, which is the point: the alternative considered and rejected was degrading the timeout to a skip, and that was measured to turn the suite green with the gate silently not running at all. The real remedy is making the cache reachable from the remote runner so the in-job build returns to being the rare fallback ADR-2719 §5 describes. That is a gsd-test-runner change, not one this repo can make. Refs #3271 * fix(#3271): tolerate an overlay source that vanishes mid-walk Observed on the remote runner, three runs across three different branches: ENOENT: no such file or directory, link '/work/hooks/dist/gsd-config-reload.js' -> '/tmp/gsd-2930-overlay-6nOZay/hooks/dist/gsd-config-reload.js' buildOverlayRepo enumerates names with readdirSync and then acts on each one, so statSync, copyFileSync and linkSync all sit in a TOCTOU window. hooks/dist is regenerated by an ATOMIC REPLACE (scripts/build-hooks.js unlinks and renames), so any concurrently running test that rebuilds hooks retires a just-listed name mid-walk and the overlay dies on it. linkOrCopyFile already tolerated EXDEV and EPERM; ENOENT went straight through. On ENOENT the source is now re-examined ONCE rather than slept on. An atomic rename is a single syscall, so by the time the failure surfaces the successor is either already in place (the retry succeeds) or the path has genuinely left the tree, in which case there is nothing to mirror and the leaf is skipped. No sleep and no spin: a timing-based wait here would be the very flake being fixed. Every other errno still propagates untouched, so a real permission or IO fault stays a hard failure. Five tests hold the boundary: gone-for-good skips without retrying, mid-replace retries exactly once and places the file, EACCES still throws, a real linkSync ENOENT is injected by monkeypatching fs and restoring it in a finally (never a mode-bit trick, which root bypasses), and isMissingPath accepts only ENOENT. Refs #3271 * fix(#3271): order the timeout ladder inward-out and lock it Two review blockers, both real. The generator bound had been raised to 600000ms — exactly the whole-chunk timeout in scripts/run-tests.cjs:973. A step bound equal to the chunk ceiling loses the race: the chunk is killed first and the failure arrives as an opaque "no failed step" kill, so the per-step diagnostic added a commit earlier was built and then made unreachable in the same change. Separately the #2767 test declared a per-test timeout of 300000ms, BELOW the inner bound it was meant to permit, so it could still die at the exact 300s ceiling this was supposed to lift — via node:test's timeout rather than spawnSync's. Its sibling declared 900000ms, above the chunk ceiling, which is the same opaque-kill hazard from the other direction. The three bounds only produce a useful failure if they fire inward-out, so they now do: step 360s, per-test 480s, chunk 600s. 360s is ~3x the passing observation (91.6s / 115.8s) and 20% above the censored 300.1s timeout, while leaving 240s of chunk headroom for every other file sharing it. Four tests lock the ordering, including a drift guard on the exported values — without it, editing a call site's literal timeout would leave the ordering assertions passing while the real ladder inverted. Also from review: - err.gsdBaselineStep and err.gsdBaselineTimings were written and never read anywhere in the tree; only the rewritten message is consumed. Removed rather than kept as speculative surface. - buildOverlayRepo discarded placeVanishableLeaf's boolean at both call sites, so a vanished leaf left the overlay with no accounting at all. It now collects the skipped paths and warns once. Not thrown: a source that left the tree really is not part of the snapshot, and throwing would reintroduce the crash the tolerance removes — but silence would let a dropped leaf resurface later as an unrelated missing-file assertion. - The instrumentation commit shipped no test. One now drives a real failure and asserts the message names the step, its elapsed time, and the bounds. Refs #3271 * chore(#3271): backfill the changeset PR number * fix(#3271): bound a hook fan-out as its own class, not as a bare probe CI failure on PR #3285, job full test (windows-latest, 22, shard 2/3) — every other lane green, including windows-latest node 24 across all three shards: not ok 1 - blocks push when any to-be-pushed commit matches local blocked regex error: bash .githooks\pre-push failed — outcome=timed_out exitCode=null stderr= duration_ms: 15040.2168 A bound, not a hang: the test supplies stdin via input:, so the hook is not blocked reading its ref list, and the duration lands exactly on the 15000ms bound. The site used PROBE_TIMEOUT_MS, which tests/helpers/timeouts.cjs documents as "a single short CLI query or node -e probe against a temp fixture". This is not that. It spawns bash running .githooks/pre-push, and the hook then invokes a MOCK git that is itself a bash script, so one runHook is roughly four Git Bash spawns. On Windows each is Defender-scanned and the first hook test in a file pays cold start on top. That module's own docstring warns against precisely this: a call site that differs from its class must not be forced onto a shared value that does not describe it. HOOK_FANOUT_TIMEOUT_MS is that missing class — 60000ms, 4x the bound that failed and half INSTALL_TIMEOUT_MS, which is the right order: a hook fan-out is much lighter than a full installer run and far heavier than reading back a version string. Two tests lock the ordering against both neighbours, including one asserting real margin over the censored 15040ms observation, since a bound that merely matched what was measured would be the same defect again. Scoped deliberately: the other ~360 runHook sites keep their current bounds. This adds the norm and applies it where a real failure demonstrated the need, rather than sweeping a value across sites with no evidence for any of them. Refs #3271 --------- Co-authored-by: sim <sim@local>	2026-08-09 23:41:44 -04:00
Tom Boucher	9faacc0c15	test(#3148): bound the long tail and delete the unbounded-spawn allowlist (#3192) ... * test(#3148): bound the long tail and delete the allowlist Migrates the final 170 unbounded sync spawn sites across 49 files, then removes the allowlist entirely. local/no-unbounded-spawn now runs with no exemption surface across tests/**: there is no file to add a name to. drift-detection's throw-native git() helper routes to gitOrThrow -- bare runGit would have taken 16 call sites quiet on failure. commands.test.cjs has two independently-scoped runGsdTools/runCli helpers, one already bounded and one not; they are kept distinct rather than unified, the same trap as the two same-named git() helpers in Wave 1. runNpm's bound was erasable. Its options spread callerOptions after the defaults, so an explicit timeout:undefined silently dropped the 180000ms bound -- the rule flagged it and was right; it was not a false positive. Fixed by destructuring with a default, with a test that fails when the default is removed. Two sites stay on a raw spawn with an explicit timeout because the seam cannot express them: one needs shell:true for npm.cmd on Windows, one redirects stdout to a real fd. Both are the rule's own documented second option, not an escape from it. Closure verified rather than asserted: the derivation scan reports 0 unbounded spawn helpers and 0 unbounded direct git call sites, and a temporary file carrying an unbounded spawn still errors with the allowlist gone. Closes #3064. * test(#3148): close a hole in the guard's own eslint-disable ban The ban listed only the top level of tests/, so it was blind to 37 .cjs files under tests/helpers, qa, observability, fixtures and dispatch. With the allowlist deleted this test is the sole remaining way to detect someone silencing the rule inline, so the gap was load-bearing: a nested file could carry an unbounded spawn plus an eslint-disable and pass everything. Proven before and after. A probe planted under tests/helpers with both was invisible to the guard and clean under eslint; after making the listing recursive the guard fails on it. The scanned set goes from 771 files to 808. Pre-existing since the guard shipped, but this wave is what promoted it to sole defense, so it is fixed here rather than filed. Also converts the last hand-rolled throw check to throwIfFailed and the last re-derived legacy shape to compose toLegacyResult, which makes the epic's none-remain claim true rather than nearly true. toLegacyResult itself is not widened -- eight callers depend on its shape and one consumer does not justify changing a shared contract. * fix(#3148): correct seam incoherence at the bound and a slow review-lane error path Two real failures from the remote runner, both fixed at the cause. The seam could return outcome TIMED_OUT together with exitCode 0. At the exact bound spawnSync reports ETIMEDOUT while the child has already exited with a real status, and toSeamResult classified on the error code while passing status straight through -- an incoherent pair its own boundary test was written to catch, and did. A status that is not null is direct evidence the child exited on its own, so it now decides the outcome before the error-code branches run. process-seam.cjs was deliberately untouched by every earlier wave; this is a defect in the module itself, kept surgical, with a unit test that fails against the old logic. review-lane with an unknown subcommand fell through to its usage error only after loading the capability registry and building a per-lane plan, which spawns one child process per lane -- up to twelve. The error path took ~1288ms instead of ~119ms, and under bench load it outran a caller's spawn timeout and was killed before writing anything, which is the empty stdout and stderr CI saw. It now fails fast before any of that work begins. This is the epic's first production change. It is user-facing, so it carries a changeset rather than a no-changelog label. * test(#3148): replace a real-race timeout test with a deterministic one E9 raced git rev-parse against a 1ms bound and assumed git always lost. On a warm container git finishes first, spawnSync returns status 0 with no error at all, the seam correctly classifies EXITED, and gitOrThrow correctly does not throw -- so the test failed on both lanes. A probe confirms a genuine timeout always carries status null, so this was never the seam misbehaving. Raising the bound would only lengthen the odds, which is the same defect with better luck. The test now drives gitOrThrow against a stubbed runGit that returns a synthetic TIMED_OUT result, so it asserts exactly what it always meant to -- that a timeout propagates as a throw -- with no timing dependence. Five consecutive runs are identical where the old one varied. I wrote this test in Wave 0; it is a real-race test by construction and CLAUDE.md says to replace those rather than re-run them. * chore(#3148): backfill changeset PR number 3192 --------- Co-authored-by: sim <sim@local>	2026-08-07 21:03:50 -04:00
Tom Boucher	5fd5c81042	test(#3055): add the process seam so a subprocess timeout is expressible as data (#3066) ... * test(#3055): add the process seam and route runGsdTools through it Adds tests/helpers/process-seam.cjs — runNode/runGit/runHook over spawnSync, each returning a typed discriminated union { outcome, exitCode, stdout, stderr, timedOut, signal, killed, code }. Every call is timeout-bounded; there is no unbounded path. runGsdTools becomes an adapter over the seam. Its legacy { success, output, error, exitCode } shape and retry-once-on-kill behaviour are preserved byte-identically, so none of its 136 caller files change. Outcome discrimination was corrected against probed runtime behaviour rather than assumption: a timeout and a maxBuffer overflow are identical on both status (null) and signal (SIGTERM), and differ only by code (ETIMEDOUT vs ENOBUFS). Overflow is therefore classified before timeout. This fixes a live defect — the previous isKilled() treated an overflow as a kill, retried it for a second full 60s run, and then reported "host OOM or scheduler contention" for a child that had merely printed too much. Also widens the ESLint tests glob from tests/**/*.test.cjs to tests/**/*.cjs, which brought 31 previously unlinted shared helpers under the same rules their sibling test files already obey, and fixes the 5 violations that surfaced — including a bare npm invocation without shell:true in tests/helpers/emitted-runtime.cjs (DEFECT.WINDOWS-TEST-PORTABILITY), now routed through the existing portable runNpm helper. Refs #3051 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(#3055): migrate every local spawn wrapper onto the process seam Replaces the spawn body of all 25 local runHook/runGuard/runGate definitions with a call to tests/helpers/process-seam.cjs. Each wrapper keeps its name, parameter list, return shape and post-processing (JSON parse, ANSI strip, env sanitising, field extraction) — only the spawn mechanism changes, so no test assertion moves. The 4 bash-driven wrappers use the seam's explicit `interpreter` option rather than a fourth primitive; it is explicit rather than inferred from the file extension, because guessing an interpreter from a path fails silently when a script's name does not match its shebang. Seven wrappers were previously unbounded and now carry an explicit timeout sized to what each actually runs, not the seam default. Two of those seven (gsd-write-guard, lint-docs-command-form) were absent from the issue's inventory entirely and were found by scanning after the migration. Adds the CONTEXT.md `### Process seam` glossary entry and a CONTRIBUTING.md reference section covering the three primitives, the discriminated union, and the two rules the seam enforces. Scope disclosure recorded in the phase design notes: the issue scoped three identifier names. A scan for local helpers that spawn AND return the spawn result finds 113 across 82 names, 71 of them unbounded, plus 122 unbounded direct git call sites. This change bounds 25 of those. The remaining surface is the same defect class and is NOT closed by this PR. Refs #3051 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#3055): classify an externally-killed child as KILLED, not EXITED Blocker found in this branch's own diff, independently confirmed by an isolated reviewer. A child killed by an external signal — a genuine bench OOM kill — makes spawnSync return { status: null, signal: 'SIGKILL' } with NO .error field. The seam's "no error implies EXITED" rule therefore classified it as a clean exit, and runGsdTools returned { success: false, exitCode: 1 } without retrying. That silently defeated the #969 kill-discrimination for precisely the case it was built for: the old isKilled() fired on `signal != null`, retried once, then threw a labelled resource-starvation error. A real OOM would have been reported as an ordinary assertion failure. Adds a fifth outcome, KILLED, for "no error but a signal is set", and makes the adapter retry on TIMED_OUT or KILLED — reproducing the old `killed || signal != null || code === 'ETIMEDOUT'` condition exactly. SPAWN_FAILED still does not retry (matching the old behaviour, where signal was null). BUFFER_OVERFLOW still does not retry, which remains a deliberate divergence: the old code retried it because signal was SIGTERM, burning a second 60s run on a child that had merely printed too much. All five outcomes verified against the live runtime rather than assumed: SIGKILL -> killed, exit 0/7 -> exited, timeout -> timed_out (ETIMEDOUT), >1MB stdout -> buffer_overflow (ENOBUFS). Refs #3051 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#3055): address standards-review findings on this branch Three findings from the standards axis of the review, all in this branch's own diff. The CONTEXT.md glossary entry this branch introduced was already stale on the branch's own last commit: it enumerated a 4-member OUTCOME while the code had 5, because the KILLED fix did not update it. That is precisely the drift the "module changes update Domain-terms" gate exists to catch, so the entry now lists all five and explains KILLED. api-coverage-gate-e2e compared an outcome against the raw string 'exited' rather than OUTCOME.EXITED, the only such outlier; the enum is now imported and used. A sweep for the other four outcome literals found no further comparison sites. Three call sites hand the literal bash flag '-c' to the seam's first parameter, which the JSDoc described as an absolute script path. Rather than add a fourth primitive, the contract is corrected to match reality: the parameter is renamed `target` and documented as the first argv element handed to the interpreter — normally a script path, but for an interpreter invoked with an inline program it may be that interpreter's own flag. No behaviour change. Refs #3051 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(#3055): assert the cross-platform timeout contract, not the macOS one The remote runner failed on both Linux lanes (node 22 and node 24, identical) while the same tests passed locally on macOS. Two assertions encoded a platform-specific behaviour as a cross-platform guarantee. When spawnSync times out, macOS preserves the child's partial stdout/stderr; Linux discards it and returns empty strings. Verified on node v26.5.1 both ways. The seam passes through whatever spawnSync hands it and cannot manufacture output that was discarded, so the production code was correct — the tests were wrong. Both tests now assert the guarantee the seam actually makes on every platform: outcome TIMED_OUT, timedOut true, and stdout/stderr always being strings rather than undefined or a Buffer. The partial-content assertions are retained behind an explicit process.platform === 'darwin' guard so the macOS coverage is not lost, and the first test is renamed to say what it now guarantees. This is the failure mode the remote matrix exists to catch: local macOS verification would have shipped it. Refs #3051 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#3055): classify a failed spawn as SPAWN_FAILED, not a timeout Windows CI caught two defects the Linux matrix could not. tests/context-predicates-query.test.cjs passes a 32K-char argv value. On Windows that exceeds the argv limit and spawnSync fails with code ENAMETOOLONG, signal null, status null. The seam's fallback rule — "otherwise, status === null implies TIMED_OUT" — swallowed it, so the adapter retried a spawn that can never succeed and then threw the resource-starvation error. The old isKilled() returned false for that shape and returned an ordinary failure result. TIMED_OUT is now identified positively: code === 'ETIMEDOUT' OR signal is set. Anything else carrying an error is SPAWN_FAILED, which covers ENAMETOOLONG, E2BIG, EACCES and ENOENT alike. The signal clause is what keeps a platform whose timeout errno differs classified correctly, so the greedy catch-all is no longer needed. The second defect is a contract regression I introduced and had claimed otherwise. That same test asserts `typeof r.exitCode === 'number'`, and toLegacyShape was returning null for BUFFER_OVERFLOW and SPAWN_FAILED, so the assertion failed on type. The old code returned `err.status ?? 1` on every non-retried failure path. The adapter now returns 1 again for both, and the comment claiming "never coerced to exitCode:1, unlike the pre-seam helper" is retracted: the seam keeps the richer truth (exitCode null plus a distinct outcome), the legacy adapter keeps the old numeric contract its callers actually depend on. Verified on this host: a 4MB argv yields E2BIG -> SPAWN_FAILED; ENOENT -> SPAWN_FAILED; timeout -> TIMED_OUT; >1MB stdout -> BUFFER_OVERFLOW; SIGKILL -> KILLED; clean exit -> EXITED. Refs #3051 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-08-05 01:06:39 -04:00