fix(#2456): weight test chunks by measured cost and pack with LPT (#2463)

* fix(#2456): weight test chunks by measured cost and pack with LPT

scripts/run-tests.cjs guessed each test file's cost from its filename
(basename matching /^(?:install|codex-)/ scored 12, everything else 1).
Measured durations show that guess is wrong in both directions:
installer-migration-authoring.test.cjs scored 12 while running ~0.1s, and
the two most expensive files in the suite both scored 1 —
run-tests-harness.test.cjs never matched the prefix, and
release-tarball-smoke.install.test.cjs was missed because the regex is
anchored to the START of the basename.

Chunks were therefore balanced by file COUNT, not cost. On the real
shard 2/3 the two heaviest files packed into the SAME chunk, leaving the
slowest chunk 2.8x the lightest and sitting near the 600s per-chunk
timeout while other chunks idled.

Weight each file by its measured duration from a checked-in, regenerable
timings table and pack with LPT (heaviest first, into the lightest
chunk). On the same shard this drops the slowest chunk from 383s to 238s
and the imbalance from 2.79x to 1.00x, and separates the two heavy files.

Timings are advisory, never gated: an unknown file falls back to the
table's median weight, a missing or corrupt table falls back to uniform
weight, and a count-based floor guarantees the packer never produces
fewer chunks than plain count-based packing would.

Closes #2456

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#2456): harden chunk packing against degenerate knobs and table keys

Follow-up hardening found while reviewing the packer, fixed inline.

The chunk knobs are read from the environment with Number(), so a typo
(RUN_TESTS_MAX_FILES_PER_CHUNK=abc) yields NaN and an explicit 0 yields
0. Both flow into the new chunk-count arithmetic: NaN made Math.ceil
return NaN, Array.from({length: NaN}) produce zero bins, and packChunks'
retry loop spin forever — a hung CI job with no output. Zero made the
count Infinity and threw RangeError: Invalid array length. The previous
count-based packer degraded to a single chunk instead, so this was a
regression introduced by the LPT rewrite.

Normalize the knobs at the environment boundary (positiveNumberEnv:
anything not a positive finite number falls back to the default) and
guard packChunks itself, since it is exported and cannot assume its
caller normalized. Non-finite weights from an arbitrary weightOf are
clamped too. RUN_TESTS_CHUNK_TIMEOUT_MS gets the same treatment.

Also resolve timing-table lookups with Object.hasOwn: the table is
JSON-parsed, so a bare index would walk the prototype chain and return a
function for a file named constructor.test.cjs or toString.test.cjs.
The typeof guard already rejected that, but the lookup now resolves
correctly rather than relying on the downstream check.

Refs #2456

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#2456): correct prototype-lookup rationale and guard generator keys

Two findings from independent security review, fixed inline.

The makeFileWeigher comment claimed a bare table lookup "would return a
FUNCTION for a file named constructor.test.cjs". That premise is false:
basename('constructor.test.cjs') is 'constructor.test.cjs', which is not
an Object.prototype key, and walkTestFiles only ever collects *.test.cjs.
The prototype chain was never reachable from a real selection, and the
existing typeof guard already rejected the function it would return, so
Object.hasOwn is defense-in-depth rather than a behavior change. The
comment now says that instead of asserting something untrue.

The accompanying test inherited the same false premise: it fed
constructor.test.cjs and asserted a median fallback that would have held
with or without the guard, so it passed for a reason unrelated to what
it claimed to prove. It now uses BARE keys (constructor, toString,
valueOf, hasOwnProperty, __proto__) — the only inputs that actually
resolve on Object.prototype — and asserts the real exported contract:
any key absent from the table weighs the median, never a function.

gen-test-timings.cjs built its output object by computed-key assignment
from basenames taken out of a reporter stream it does not control — the
js/prototype-polluting-assignment shape, and this repo has a CodeQL
barrier for exactly that pattern. It was not exploitable (the value is
always a rounded number, so the __proto__ setter is a silent no-op), but
it silently DROPPED such an entry rather than reporting it. Validate every
key against a test-basename pattern and fail loudly instead, and build
the table with a null prototype.

Refs #2456

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#2456): replace tautological chunking tests and clamp chunk count

Six findings from independent correctness review, all reproduced and
fixed inline.

The two subprocess tests written to carry the #2088 guarantee forward
were tautological: every seeded file weighed exactly 1, so both passed
under the OLD prefix-heuristic packer and with the timings file deleted
entirely. Neither could fail for the reason it existed. Both are rebuilt
so the old algorithm produces a different packing and the assertion goes
red: the spread test now uses three expensive files named so the old
heuristic scored them 1 alongside three trivial `install-`-prefixed
files it scored 12 — inverted from real cost, giving {2,2,1,1} under the
old packer versus {2,2,2} under measured weights. The companion test
covers the other direction: four trivial `install-` files the old
heuristic split into four single-file chunks now stay in one.

packChunks clamped the chunk count from below but not above, so a
legitimate but tiny budget (RUN_TESTS_MAX_FILES_PER_CHUNK=1e-9, which
positiveNumberEnv accepts) asked for 637,000,000,000 bins and threw
RangeError. More chunks than files is never useful; the count now clamps
at one file per chunk.

The generator's basename-collision guard compared full dirnames, so two
OS lanes reporting the same file under different container roots
(/work/tests vs C:/work/tests) flagged every shared basename as a
collision — on the script's own documented multi-lane usage. Detection is
now scoped per stream, where the root is constant; a genuine same-lane
collision is still caught.

Also: the LPT tie-break compared raw paths, so a path separator (0x2F vs
0x5C) could order a subdir file differently per platform, contradicting
the documented byte-identical guarantee — it now normalizes separators.
loadTestTimings now honors schema_version instead of writing it and
never reading it, falling back to uniform weight on an unknown version.
A comment claiming an all-uniform suite "chunks exactly as it did
before" was false and contradicted by this PR's own test: the chunk
count is preserved, the composition is not. And the missing-table test
created a temp dir it never cleaned up, for a path that only needed to
not exist.

Refs #2456

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

docs/TESTING-SUITES.md

@@ -209,6 +209,55 @@ npm run ci:test-scope -- --files "commands/gsd/plan-phase.md"
 node scripts/ci-test-scope.cjs --base origin/next --head HEAD
 ```
 
+## Chunk packing and the test timing table
+
+`scripts/run-tests.cjs` does not hand the whole selected file list to one
+`node --test` process. It packs the files into **chunks**, each spawned
+separately, because Windows caps a command line at 32,767 characters and because
+each chunk gets its own 600s timeout (`RUN_TESTS_CHUNK_TIMEOUT_MS`) and a fresh
+process, which bounds memory pressure.
+
+How files are distributed across those chunks decides whether the slowest chunk
+sits near that timeout while the others idle. The packer weights each file by its
+**measured duration**, read from `tests/test-timings.json`, and places files with
+LPT (longest-processing-time-first: heaviest file first, each into the currently
+lightest chunk). Before #2456 the weight was guessed from the filename, which
+mis-ranked files badly enough that the slowest chunk ran ~3.9x the lightest.
+
+### Reference
+
+| Knob | Default | Meaning |
+|---|---|---|
+| `RUN_TESTS_MAX_FILES_PER_CHUNK` | `60` | Per-chunk weight budget. Weights are normalized so an **average-cost** file weighs 1, so this still reads as "about 60 average files". |
+| `RUN_TESTS_MAX_CMDLINE_CHARS` | `28000` | argv ceiling per chunk, with headroom under the Windows 32,767 limit. |
+| `RUN_TESTS_TIMINGS_FILE` | `tests/test-timings.json` | Path to the timing table. Tests override it to inject a synthetic cost profile. |
+| `RUN_TESTS_CHUNK_TIMEOUT_MS` | `600000` | Per-chunk timeout. |
+
+The timing table is **advisory and deliberately un-gated**. There is no `--check`
+mode and no CI lint that fails on staleness, because timing data legitimately
+varies run to run. A file missing from the table falls back to the table's median
+weight, and a missing or unparseable table falls back to uniform weight — so
+drift costs chunk *balance*, never a red build. A count-based floor additionally
+guarantees the packer never produces fewer chunks than plain count-based packing
+would, so a badly stale table cannot collapse the suite into a few fat chunks.
+
+### How-to: regenerate the timing table
+
+Regenerate when the suite's cost profile has visibly drifted — after adding or
+removing expensive tests, not on a schedule. The input is a `node:test` reporter
+event stream from a `gsd-test` run:
+
+```bash
+node scripts/gen-test-timings.cjs \
+  ~/.local/state/gsd-test/runs/<run-id>/test-events-linux-node22.jsonl \
+  ~/.local/state/gsd-test/runs/<run-id>/test-events-linux-node24.jsonl
+```
+
+Pass every lane you have. A file's recorded time is the **max** across the
+supplied streams, not the mean: the packer exists to keep the *slowest* lane's
+slowest chunk away from the timeout, so the conservative bound is the right one.
+Keys are sorted so a regeneration diff shows only the files whose cost moved.
+
 ## Best practices for forward-compat (Node 24/26)
 
 - Use `process.execPath` when spawning Node in tests so each matrix lane exercises the lane's Node version.