fix(#4733): derive the win32 chunk cap, isolation bar, and unknown-file weight (#4737)

* test(#4733): pin the cap, unknown-file weight, and isolation rules

Failing-first coverage for the three defects that let a Windows conformance
chunk be killed at the 600s per-chunk backstop with zero failing tests.

The previous boundary rows were VACUOUS: they asserted literal arithmetic
(21 * 18122 <= 400000) that cannot fail, and in doing so masked a shipped
win32 cap of 23 -- a value that violates the very inequality they claimed to
pin. These rows constrain defaultMaxFilesPerChunk itself, from both sides, so
the shipped value is a derived maximum rather than a magic number.

A second vacuous row was caught by review and removed: it recomputed the
isolated set from the function under test using the identical predicate, so it
was empty by construction. It is replaced by an exact deepEqual against the
expected basenames, a cross-platform identity row, dynamism rows in both
directions, an inclusive boundary triplet, and invalid-threshold throw rows.

The cross-platform identity row is the regression guard for a threshold that
was briefly anchored to the per-platform file-COUNT cap; it fails if isolation
ever becomes platform-dependent again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#4733): derive the win32 cap, isolation bar, and unknown weight

A Windows conformance chunk was killed at the 600000ms per-chunk backstop with
no test having failed, taking next red. Three compounding defects.

The win32 cap of 40 permitted 40 * 18122 = 724880ms against a 600000ms
backstop -- 121% of it -- so two rounds of budget-tuning could not hold. The
cap is now derived: 22 is the largest value satisfying cap * 18122 <= 400000.
The budget is 400000, not the raw backstop, because the chunk that died summed
to only ~348328ms of per-file time -- a per-chunk overhead gap of at least
1.72x that no per-file table models.

A file absent from the timings table was priced at medianWeight. The table is
skewed 18.8x, so an unknown weighed 0.0533 -- 19x cheaper than average, and
measured 17.5x under its real cost. Unknowns are now priced at the mean.

ISOLATED_HEAVY_FILES was a static Set, stale by construction. Isolation is now
derived from an absolute ms bar (0.3 * 400000 = 120000ms) converted to weight
units via the live table's mean, so a file that gets heavy is isolated
automatically instead of waiting for someone to edit a list.

Review caught that an earlier cut anchored that bar to the per-platform
file-COUNT cap -- a category error, count vs weight, which silently returned
seven of the historical eight files to the shared pool on linux/darwin. Since
macOS runs the full matrix only after merge, that would have planted a red
next no PR could catch. The bar is absolute and platform-independent.

Also from review: isolation no longer requires unit-suite membership, so
fragment-single-edit-propagation.install.test.cjs -- 575000ms, 96% of the
backstop in one file -- is eligible; partitionIsolatedFiles throws on a
non-finite or non-positive threshold instead of silently isolating nothing;
and stale per-shard figures no test pinned are removed rather than recomputed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#4733): backfill changeset pr number

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-09-14 12:30:51 -04:00
committed by GitHub
parent ca8d9d4459
commit 9c2927bff4
5 changed files with 686 additions and 208 deletions

View File

@@ -515,15 +515,15 @@ mis-ranked files badly enough that the slowest chunk ran ~3.9x the lightest.
| Knob | Default | Meaning |
|---|---|---|
| `RUN_TESTS_MAX_FILES_PER_CHUNK` | `60` | Per-chunk weight budget. Weights are normalized so an **average-cost** file weighs 1, so this still reads as "about 60 average files". |
| `RUN_TESTS_MAX_FILES_PER_CHUNK` | `60` (`22` on win32) | Per-chunk weight budget. Weights are normalized so an **average-cost** file weighs 1, so this still reads as "about 60 average files" (about 22 on win32). Windows gets a lower cap than Linux/macOS because the weight table's calibration does not transfer 1:1 to the Windows runner for install/subprocess-heavy work — see the derivation comment above `DEFAULT_MAX_FILES_PER_CHUNK` in `scripts/run-tests.cjs`. |
| `RUN_TESTS_MAX_CMDLINE_CHARS` | `28000` | argv ceiling per chunk, with headroom under the Windows 32,767 limit. |
| `RUN_TESTS_TIMINGS_FILE` | `tests/test-timings.json` | Path to the timing table. Tests override it to inject a synthetic cost profile. |
| `RUN_TESTS_CHUNK_TIMEOUT_MS` | `600000` | Per-chunk timeout. |
The timing table is **advisory and deliberately un-gated**. There is no `--check`
mode and no CI lint that fails on staleness, because timing data legitimately
varies run to run. A file missing from the table falls back to the table's median
weight, and a missing or unparseable table falls back to uniform weight — so
varies run to run. A file missing from the table falls back to the table's mean
weight (1), and a missing or unparseable table falls back to uniform weight — so
drift costs chunk *balance*, never a red build. A count-based floor additionally
guarantees the packer never produces fewer chunks than plain count-based packing
would, so a badly stale table cannot collapse the suite into a few fat chunks.