Files
msd-core/tests
Tom Boucher 18e5cfff8a fix(#4250, #4260): distinguish a timed-out npm audit from a JSON parse failure, retry with backoff (#4251)
* fix(#4250): distinguish a timed-out npm audit from a JSON parse failure

npm-audit-baseline.cjs's runPackageLockAudit, and the near-identical
auditProductionVulns helper in npm-integrity-gate.test.cjs, both grabbed
e.stdout whenever an npm audit child process exited non-zero -- without
checking whether the process was actually killed by its 180s timeout.
A timeout-killed process's stdout is truncated mid-write, not complete
JSON, so JSON.parse threw a misleading "Unexpected end of JSON input"
instead of naming npm's registry timeout as the real cause.

Root-caused live during a CI investigation: npm's own status page
reported degraded service, and the registry's bulk-advisories endpoint
was returning 503/hanging, causing npm audit to sit until the timeout
fired.

Adds a shared isTimeoutKill(error) predicate (checks execFileSync's
documented killed/signal fields) and checks it first in both catch
blocks, throwing a clear, actionable error before ever reaching
JSON.parse. The pre-existing "non-zero exit with complete JSON"
recovery path is unchanged and still covered by regression tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(#4250): add changeset for npm-audit timeout fix

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#4250): share the timeout-kill error message and cover auditProductionVulns

Two independent review passes (standards + spec) on the first commit found
real gaps: the timeout-kill error message was duplicated verbatim between
runPackageLockAudit and the near-identical auditProductionVulns helper in
tests/npm-integrity-gate.test.cjs (this repo's own Generative Fix Divergence
anti-pattern -- shared logic across parallel surfaces with no parity check),
and auditProductionVulns picked up the same production fix with zero test
coverage of its own.

Extracts buildTimeoutKillError(cwd), used by both callers so the message
cannot independently drift. Gives auditProductionVulns the same injectable
execFileSyncImpl seam runPackageLockAudit already had, and adds the matching
regression tests (timeout-kill throws the clear error; the pre-existing
non-zero-exit-with-complete-JSON path still recovers correctly).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(#4250): backfill changeset PR number to #4251

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* diag(#4250): surface captured stderr in the timeout-kill error

The killed child process's stderr is buffered in-memory by execFileSync
and attached to the thrown error, but nothing surfaced it -- the timeout
message named the timeout but discarded the one piece of data that could
show WHY npm was still running when it fired (DNS stall, TLS handshake
stall, a registry-side retry loop, all look identical without it).

buildTimeoutKillError now takes the killed error and includes its stderr
(or an explicit 'no stderr was captured' note) in the message. This is a
diagnostic improvement for the next CI occurrence, not a behavior fix --
local reproduction has directly ruled out npm version (installed the
exact CI-bundled 11.17.0 and ran it against this repo: 0.49s, clean),
general npm registry reachability (0.4-1.4s locally, repeatedly), and
npm ci speed (2m, succeeded) as explanations for the 180s CI hangs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#4260): bounded retry with backoff for npm audit calls, finish the extraction

The audit backend has real, independent latency variance from the rest of
the npm registry -- measured (see #4260): a bulk-advisories POST took
43.41s vs 0.20s for a plain registry fetch on the same host, and the same
endpoint returned no response at all (000) twice in the same window,
while status.npmjs.org reported fully operational throughout. Against
that, runPackageLockAudit and its near-duplicate auditProductionVulns
each made exactly one attempt with no retry -- any single bad moment
failed a REQUIRED CI gate on a transport hiccup, not a real advisory.

Replaces the single 180s attempt with runNpmAuditWithRetry: up to 3
attempts at 60s each (comfortably above the worst measured working
latency) with exponential backoff between them. Only a confirmed
timeout-kill is retried; a genuine non-timeout failure still fails
immediately, and exhausting all attempts still fails the gate -- per
#4260's own caveat, silently disarming a required security check on a
transport error is worse than occasionally re-running CI.

Also finishes the extraction #4260 flagged as stopped halfway:
auditProductionVulns (tests/npm-integrity-gate.test.cjs) duplicated
runPackageLockAudit's entire candidate loop, recovery branch, and timeout
classification, differing only in npm args and precondition check. It is
now a thin wrapper delegating to the newly-exported runInstalledTreeAudit,
which shares runNpmAuditWithRetry with runPackageLockAudit -- one
implementation instead of two that could independently drift.

buildTimeoutKillError now reports attempt count and still surfaces
captured stderr from the last kill.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(#4260): update changeset for retry/backoff scope

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(#4260): budget for two sequential retry-audit calls, close coverage gaps

Two review passes on the retry/backoff commit found real gaps:

- TEST_TIMEOUT_MS budgeted only one retry-audit call's worst case (210s),
  but checkTreeAgainstBaseline makes two sequential calls (HEAD tree via
  auditProductionVulns, baseline tree via runPackageLockAudit) -- combined
  worst case is ~372s. If both genuinely exhausted retries, node:test's
  own timeout would fire first and mask buildTimeoutKillError's clear
  message, undercutting #4250's own fix in that edge case. Recomputed
  using the same backoff formula the production code uses, so it can't
  independently drift.

- buildTimeoutKillError's default-attempts(1) singular-phrasing branch had
  zero direct test coverage (nothing calls it with a single attempt
  anymore) -- a real mutation-testing risk. Added direct tests for both
  phrasing branches plus the no-error-object case.

- runInstalledTreeAudit's null-guard skip paths (missing package.json,
  missing node_modules) had no tests, unlike runPackageLockAudit's
  matching paths. Added for parity.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-04 10:02:17 -04:00
..