* feat(#4422): block merging into next/main while the base branch's Tests run is red Adds a next-health job to test.yml that checks the base branch's own last push-triggered Tests run via the GitHub API and fails the existing "Required tests" required check when it's red, with a maintainer-applied "fix-next" label as the explicit escape hatch for the fix-forward PR itself. No branch-protection config change needed — it rides the already-required check. The job is deliberately not gated behind preflight, same reasoning as the changes job: a compute-free API read has nothing to save by waiting. Documents the fix-next label in CONTRIBUTING.md and adds a property test locking the CLEAN/RED/INDETERMINATE classification's iff-relationship. This closes the second half of the 2026-09-06 RCA: three unrelated PRs merged on top of an already-broken next before anyone noticed it was red. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: close two zero-margin CI timing gaps found while verifying #4422 Discovered while watching this branch's own CI, root-caused via /diagnose rather than dismissed as Windows flakiness: 1. tests/gsd-check-update-worker-atomic-cache.test.cjs's outer timeout (15000ms) exactly matched the inner npm-view timeout the worker wraps (NPM_VIEW_TIMEOUT_MS, gsd-core/bin/check-latest-version.cjs). A slow registry response raced two SIGKILLs at the same instant, killing the worker before it could catch its own timeout and degrade gracefully. Windows's shell-wrapped npm subprocess made the race lose more often there, but the zero margin was platform-agnostic. Fixed by giving the test real headroom (+10s) beyond the named constant it wraps, plus an invariant test so the two values can't silently collide again. 2. scripts/run-tests.cjs's per-chunk weight budget (MAX_FILES_PER_CHUNK) let a Windows full-matrix chunk that was well under budget by the Linux/macOS-calibrated weight table (~32/60 units) still exceed the 600s wall-clock backstop — codex-config.test.cjs's genuinely-measured weight (17.87) doesn't transfer 1:1 to Windows's slower install/ subprocess overhead. Windows now gets its own lower cap (40 vs 60). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
77
.github/workflows/test.yml
vendored
77
.github/workflows/test.yml
vendored
@@ -45,6 +45,72 @@ jobs:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
|
||||
# #4422: three unrelated PRs merged on top of an already-broken `next` on
|
||||
# 2026-09-06 before anyone noticed — nothing checked the base branch's OWN
|
||||
# health before letting a PR land on it. This job queries the GitHub API for
|
||||
# the base branch's last push-triggered Tests run and blocks the merge if it
|
||||
# is red, with a `fix-next` label escape hatch for the PR that is itself the
|
||||
# fix-forward. See scripts/ci-next-health.cjs's header for the full risk-
|
||||
# asymmetry reasoning (it deliberately does NOT fail open on a definite red
|
||||
# signal, unlike the mergeability preflight above).
|
||||
#
|
||||
# No `if:` guard, for the same reason `preflight` has none: a SKIPPED
|
||||
# dependency skips its dependents exactly like a failed one, so guarding this
|
||||
# job would skip `required-tests` on every push/workflow_dispatch run too.
|
||||
# scripts/ci-next-health.cjs itself no-ops (zero API calls, exit 0) on any
|
||||
# event other than pull_request/merge_group.
|
||||
#
|
||||
# `next-health` is likewise deliberately NOT gated behind `preflight`, same
|
||||
# rationale as `changes` above: it is a ~2-minute, compute-free API read that
|
||||
# runs in parallel with the preflight job, and serializing it behind
|
||||
# `preflight` would add latency to every healthy PR for no saving.
|
||||
next-health:
|
||||
name: Base branch health
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 2
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
steps:
|
||||
- name: Check out the next-health script from the base branch
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
# Same rationale as pr-mergeable-preflight.yml's checkout: pinned to
|
||||
# the BASE sha so the script that decides the gate comes from a
|
||||
# trusted commit, never PR-supplied code. Empty on a non-pull_request
|
||||
# event (e.g. merge_group), where actions/checkout falls back to the
|
||||
# triggering ref — fine, since the script no-ops before reading
|
||||
# anything on those events too.
|
||||
ref: ${{ github.event.pull_request.base.sha }}
|
||||
fetch-depth: 1
|
||||
sparse-checkout: |
|
||||
scripts
|
||||
persist-credentials: false
|
||||
|
||||
- name: Check base branch health
|
||||
id: check
|
||||
env:
|
||||
# Every value arrives through env. CONTRIBUTING.md forbids `${{ }}`
|
||||
# inside a `run:` block.
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
PR_LABELS: ${{ join(github.event.pull_request.labels.*.name, ',') }}
|
||||
MERGE_GROUP_BASE_REF: ${{ github.event.merge_group.base_ref }}
|
||||
run: |
|
||||
# Bootstrap arm, mirroring pr-mergeable-preflight.yml's: the checkout
|
||||
# above is of the BASE sha, so this step runs the script as it exists
|
||||
# on the base branch — which means it is absent on the PR that
|
||||
# INTRODUCES it, and on any PR branched from a base predating it.
|
||||
# Absent is not "red": it is one more thing we cannot determine, so
|
||||
# it takes the same fail-open path as any other unresolved read.
|
||||
# Self-healing — once the script is on the base branch this arm never
|
||||
# fires again.
|
||||
if [ ! -f scripts/ci-next-health.cjs ]; then
|
||||
echo "::warning::scripts/ci-next-health.cjs is not present at the base sha; skipping the base-branch health gate (fail-open). Expected on the PR that introduces it, or on a branch whose base predates it."
|
||||
echo "verdict=INDETERMINATE" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
node scripts/ci-next-health.cjs
|
||||
|
||||
changes:
|
||||
name: Detect test scope
|
||||
runs-on: ubuntu-latest
|
||||
@@ -857,6 +923,7 @@ jobs:
|
||||
name: Required tests
|
||||
needs:
|
||||
- preflight
|
||||
- next-health
|
||||
- changes
|
||||
- lint-tests
|
||||
- test
|
||||
@@ -871,6 +938,7 @@ jobs:
|
||||
- name: Summarize required test gate
|
||||
env:
|
||||
PREFLIGHT_RESULT: ${{ needs.preflight.result }}
|
||||
NEXT_HEALTH_RESULT: ${{ needs.next-health.result }}
|
||||
CODE_CHANGED: ${{ needs.changes.outputs.code_changed }}
|
||||
PRODUCT_CHANGED: ${{ needs.changes.outputs.product_changed }}
|
||||
CHANGES_RESULT: ${{ needs.changes.result }}
|
||||
@@ -883,6 +951,7 @@ jobs:
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "preflight=$PREFLIGHT_RESULT"
|
||||
echo "next-health=$NEXT_HEALTH_RESULT"
|
||||
echo "code_changed=$CODE_CHANGED"
|
||||
echo "product_changed=$PRODUCT_CHANGED"
|
||||
echo "changes=$CHANGES_RESULT"
|
||||
@@ -910,6 +979,14 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# #4422: applies UNCONDITIONALLY, not nested inside the
|
||||
# PRODUCT_CHANGED/CODE_CHANGED branches below — a red base branch
|
||||
# must block every PR, including doc-only ones.
|
||||
if [ "$NEXT_HEALTH_RESULT" != "success" ]; then
|
||||
echo "::error::the base branch's own last Tests run is red — see the 'Base branch health' job above for the failing run. Wait for a fix-forward merge, or if this PR IS the fix, ask a maintainer to apply the 'fix-next' label to override."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$LINT_RESULT" != "success" ]; then
|
||||
echo "::error::lint-tests did not pass"
|
||||
exit 1
|
||||
|
||||
@@ -1285,6 +1285,21 @@ the pipeline runs exactly as it did before, and the per-job
|
||||
including which lanes are deliberately *not* gated, is in
|
||||
[docs/TESTING-SUITES.md → The mergeability preflight](docs/TESTING-SUITES.md#the-mergeability-preflight).
|
||||
|
||||
### A PR cannot merge onto a red base branch
|
||||
|
||||
The `Base branch health` required check queries GitHub for the base branch's
|
||||
own last push-triggered Tests run and blocks your merge if that run is red —
|
||||
independent of whether your own PR's changes pass. This needs no
|
||||
branch-protection reconfiguration: it rides the existing "Required tests"
|
||||
check, the same status GitHub already requires before merge.
|
||||
|
||||
If your PR is itself the fix-forward and you need to land it while the base
|
||||
branch is still red, a maintainer applies the `fix-next` label directly to
|
||||
your PR to explicitly bypass this one check. Applying a label requires
|
||||
GitHub write access to the repo, so a PR author cannot self-apply it to
|
||||
bypass the gate — only a maintainer or another collaborator with label-write
|
||||
permission can. Full decision logic is in `scripts/ci-next-health.cjs`.
|
||||
|
||||
### CI Test Quality Checks
|
||||
|
||||
The following checks run on every PR in addition to the test suite:
|
||||
|
||||
@@ -42,6 +42,12 @@ const SEMVER_RE = /^\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?$/;
|
||||
// `npm view` to an empty or foreign dist-tag.
|
||||
const ALLOWED_TAGS = Object.freeze(['latest', 'next']);
|
||||
|
||||
// Bounded at 15s so a hung registry doesn't block /gsd-update (#2993 CR).
|
||||
// Exported so callers (e.g. the worker's own test harness, #4091) can derive
|
||||
// their own outer timeout with real margin above this inner bound instead of
|
||||
// re-hardcoding 15000 and silently drifting into a zero-margin race.
|
||||
const NPM_VIEW_TIMEOUT_MS = 15_000;
|
||||
|
||||
/**
|
||||
* Build the `npm view` args for a dist-tag. `latest` keeps the bare package
|
||||
* spec so the default invocation is byte-for-byte identical to before tag
|
||||
@@ -92,9 +98,8 @@ function checkLatestVersion(opts = {}) {
|
||||
// Windows shell-flag policy and timeout default). The injection point
|
||||
// remains spawnSync-shaped for test compatibility — the adapter below
|
||||
// translates { exitCode } → { status } so the consumer logic is unchanged.
|
||||
// Bounded at 15s so a hung registry doesn't block /gsd-update (#2993 CR).
|
||||
const defaultSpawn = () => {
|
||||
const r = execNpm(buildViewArgs(tag), { timeout: 15_000 });
|
||||
const r = execNpm(buildViewArgs(tag), { timeout: NPM_VIEW_TIMEOUT_MS });
|
||||
return {
|
||||
status: r.exitCode,
|
||||
stdout: r.stdout,
|
||||
@@ -158,4 +163,4 @@ function main() {
|
||||
|
||||
if (require.main === module) runMain(main);
|
||||
|
||||
module.exports = { checkLatestVersion, CHECK_REASON, PACKAGE_NAME, ALLOWED_TAGS, buildViewArgs, resolveTag };
|
||||
module.exports = { checkLatestVersion, CHECK_REASON, PACKAGE_NAME, ALLOWED_TAGS, NPM_VIEW_TIMEOUT_MS, buildViewArgs, resolveTag };
|
||||
|
||||
271
scripts/ci-next-health.cjs
Normal file
271
scripts/ci-next-health.cjs
Normal file
@@ -0,0 +1,271 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
// ci-next-health.cjs — #4422 base-branch health gate.
|
||||
//
|
||||
// Risk asymmetry drives this design, same shape as scripts/ci-pr-mergeability.cjs
|
||||
// but pointed the other direction: on 2026-09-06 three unrelated PRs merged on
|
||||
// top of an already-broken `next` before anyone noticed, because nothing checked
|
||||
// the base branch's OWN health before letting a PR land on it. A false positive
|
||||
// here (a healthy `next` wrongly reported RED) blocks every PR merge in the repo
|
||||
// until a human notices and applies the `fix-next` bypass label — annoying, but
|
||||
// loud and immediately actionable. A false negative (a broken `next` reported
|
||||
// healthy) reproduces the #4422 incident exactly. So, unlike the mergeability
|
||||
// preflight, this gate does NOT fail open on a definite red signal — it fails
|
||||
// open only when the signal itself is unavailable or inapplicable (wrong event,
|
||||
// no resolvable base ref, an API read that throws). Once GitHub actually answers
|
||||
// with a non-success conclusion for the base branch's last push-triggered Tests
|
||||
// run, that is treated as authoritative and blocks — with one explicit, visible,
|
||||
// human-operated escape hatch (the `fix-next` label) for the PR that is itself
|
||||
// the fix-forward.
|
||||
//
|
||||
// Every non-success conclusion (`failure`, `cancelled`, `timed_out`,
|
||||
// `action_required`, ...) is treated as RED, not just `failure` — a cancelled or
|
||||
// timed-out run on the base branch is not evidence the branch is healthy, it is
|
||||
// evidence nobody knows yet.
|
||||
|
||||
const { ExitError, runMain } = require('./lib/cli-exit.cjs');
|
||||
|
||||
const VERDICT = Object.freeze({
|
||||
CLEAN: 'CLEAN',
|
||||
RED: 'RED',
|
||||
BYPASSED: 'BYPASSED',
|
||||
SKIPPED_NOT_APPLICABLE: 'SKIPPED_NOT_APPLICABLE',
|
||||
INDETERMINATE: 'INDETERMINATE',
|
||||
});
|
||||
|
||||
const APPLICABLE_EVENTS = new Set(['pull_request', 'merge_group']);
|
||||
const BYPASS_LABEL = 'fix-next';
|
||||
const TESTS_WORKFLOW_FILE = 'test.yml';
|
||||
|
||||
/**
|
||||
* Pure, total classifier: the GitHub "list workflow runs" response payload ->
|
||||
* CLEAN | RED | INDETERMINATE. Never throws.
|
||||
*
|
||||
* - Not an object, or `workflow_runs` isn't an array -> INDETERMINATE: the
|
||||
* shape we depend on is not present, so nothing can be concluded.
|
||||
* - No completed push runs found yet (e.g. a brand-new `release/**`/`hotfix/**`
|
||||
* branch with no Tests history) -> CLEAN: absence of evidence of red is not
|
||||
* evidence of red, and a brand-new branch must not be permanently unmergeable.
|
||||
* - The most recent run's `conclusion === 'success'` -> CLEAN.
|
||||
* - Anything else (failure, cancelled, timed_out, action_required, ...) -> RED.
|
||||
*/
|
||||
function classifyRunConclusion(payload) {
|
||||
if (payload === null || typeof payload !== 'object') return VERDICT.INDETERMINATE;
|
||||
if (!Array.isArray(payload.workflow_runs)) return VERDICT.INDETERMINATE;
|
||||
if (payload.workflow_runs.length === 0) return VERDICT.CLEAN;
|
||||
const [latest] = payload.workflow_runs;
|
||||
if (latest && latest.conclusion === 'success') return VERDICT.CLEAN;
|
||||
return VERDICT.RED;
|
||||
}
|
||||
|
||||
/**
|
||||
* Dependency-injected orchestrator. `fetchLatestRun` is an async function
|
||||
* taking the resolved base branch name and returning the parsed API payload
|
||||
* (or throwing) — injected so tests never touch the network.
|
||||
*
|
||||
* @returns {Promise<{verdict:string, payload:*, reason:string}>}
|
||||
*/
|
||||
async function resolveNextHealth({ fetchLatestRun, eventName, baseRef, labels } = {}) {
|
||||
if (!APPLICABLE_EVENTS.has(eventName)) {
|
||||
return { verdict: VERDICT.SKIPPED_NOT_APPLICABLE, payload: null, reason: 'not-applicable-event' };
|
||||
}
|
||||
|
||||
if (typeof baseRef !== 'string' || baseRef.trim() === '') {
|
||||
return { verdict: VERDICT.INDETERMINATE, payload: null, reason: 'no-base-ref' };
|
||||
}
|
||||
|
||||
let payload = null;
|
||||
try {
|
||||
payload = await fetchLatestRun(baseRef);
|
||||
} catch (err) {
|
||||
return { verdict: VERDICT.INDETERMINATE, payload: null, reason: `fetch-failed: ${err.message}` };
|
||||
}
|
||||
|
||||
const classified = classifyRunConclusion(payload);
|
||||
if (classified !== VERDICT.RED) {
|
||||
return { verdict: classified, payload, reason: 'resolved' };
|
||||
}
|
||||
|
||||
// Escape hatch. `labels` is empty/absent for merge_group — that event has no
|
||||
// label surface today, which means a queued merge-group commit cannot use
|
||||
// this bypass. Known gap, not solved here: a maintainer must land the
|
||||
// fix-forward as a direct pull_request merge (where the label IS readable)
|
||||
// rather than through the merge queue, until GitHub exposes an equivalent
|
||||
// signal for merge_group.
|
||||
const labelList = Array.isArray(labels) ? labels : [];
|
||||
if (labelList.includes(BYPASS_LABEL)) {
|
||||
return { verdict: VERDICT.BYPASSED, payload, reason: 'bypass-label' };
|
||||
}
|
||||
|
||||
return { verdict: VERDICT.RED, payload, reason: 'red' };
|
||||
}
|
||||
|
||||
function usage() {
|
||||
return [
|
||||
'Usage:',
|
||||
' node scripts/ci-next-health.cjs',
|
||||
'',
|
||||
'CI-only gate: checks whether the PR/merge-group\'s base branch\'s own last',
|
||||
'push-triggered Tests run is red, and fails the job (exit 1) when it is —',
|
||||
'unless the PR carries the "fix-next" bypass label. Every other case',
|
||||
'(wrong event, no resolvable base ref, an unreadable API read) fails open',
|
||||
'(exit 0).',
|
||||
'',
|
||||
'Environment variables read:',
|
||||
' GITHUB_EVENT_NAME workflow trigger event; only "pull_request" and',
|
||||
' "merge_group" are checked',
|
||||
' GITHUB_REPOSITORY owner/repo',
|
||||
' GITHUB_TOKEN optional bearer token for the API read',
|
||||
' GITHUB_BASE_REF PR base branch name (pull_request events; set',
|
||||
' automatically by GitHub Actions)',
|
||||
' MERGE_GROUP_BASE_REF merge-group base ref (merge_group events; the',
|
||||
' caller workflow must populate this from',
|
||||
' github.event.merge_group.base_ref — a leading',
|
||||
' "refs/heads/" prefix is stripped if present)',
|
||||
' GITHUB_API_URL GitHub API base URL (default: https://api.github.com)',
|
||||
' PR_LABELS comma-separated PR label names (pull_request events;',
|
||||
' the caller workflow must populate this from',
|
||||
' github.event.pull_request.labels.*.name); checked',
|
||||
' for the literal "fix-next" bypass label',
|
||||
' GITHUB_OUTPUT path to append verdict= step output to',
|
||||
' GITHUB_STEP_SUMMARY path to append a human-readable summary to',
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
function parseArgs(argv) {
|
||||
for (const arg of argv) {
|
||||
if (arg === '--help' || arg === '-h') {
|
||||
process.stdout.write(`${usage()}\n`);
|
||||
throw new ExitError(0);
|
||||
}
|
||||
throw new Error(`unknown argument: ${arg}`);
|
||||
}
|
||||
}
|
||||
|
||||
function writeOutput(lines) {
|
||||
const outputPath = process.env.GITHUB_OUTPUT;
|
||||
if (typeof outputPath !== 'string' || outputPath === '') return;
|
||||
try {
|
||||
const fs = require('node:fs');
|
||||
fs.appendFileSync(outputPath, `${lines.join('\n')}\n`);
|
||||
} catch (err) {
|
||||
// A failure while REPORTING the verdict must never invert the gate.
|
||||
process.stderr.write(`::warning::failed to write GITHUB_OUTPUT: ${err.message}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
function writeSummary(text) {
|
||||
const summaryPath = process.env.GITHUB_STEP_SUMMARY;
|
||||
if (typeof summaryPath !== 'string' || summaryPath === '') return;
|
||||
try {
|
||||
const fs = require('node:fs');
|
||||
fs.appendFileSync(summaryPath, `${text}\n`);
|
||||
} catch (err) {
|
||||
process.stderr.write(`::warning::failed to write GITHUB_STEP_SUMMARY: ${err.message}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
function resolveBaseRef(eventName) {
|
||||
if (eventName === 'pull_request') {
|
||||
return process.env.GITHUB_BASE_REF || '';
|
||||
}
|
||||
if (eventName === 'merge_group') {
|
||||
const raw = process.env.MERGE_GROUP_BASE_REF || '';
|
||||
return raw.startsWith('refs/heads/') ? raw.slice('refs/heads/'.length) : raw;
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
function parseLabels(raw) {
|
||||
if (typeof raw !== 'string' || raw.trim() === '') return [];
|
||||
return raw.split(',').map((s) => s.trim()).filter((s) => s !== '');
|
||||
}
|
||||
|
||||
function runUrlOf(payload) {
|
||||
const run = payload && Array.isArray(payload.workflow_runs) ? payload.workflow_runs[0] : undefined;
|
||||
return run && typeof run.html_url === 'string' ? run.html_url : '(unknown run URL)';
|
||||
}
|
||||
|
||||
// `argv` defaults to real CLI argv but is a parameter so tests can call
|
||||
// main() in-process (e.g. main([])) without inheriting the test runner's own
|
||||
// argv, which would otherwise trip parseArgs's "unknown argument" branch.
|
||||
async function main(argv = process.argv.slice(2)) {
|
||||
parseArgs(argv);
|
||||
|
||||
const eventName = process.env.GITHUB_EVENT_NAME;
|
||||
const repo = process.env.GITHUB_REPOSITORY || '';
|
||||
const token = process.env.GITHUB_TOKEN || '';
|
||||
const apiBase = process.env.GITHUB_API_URL || 'https://api.github.com';
|
||||
const baseRef = resolveBaseRef(eventName);
|
||||
const labels = parseLabels(process.env.PR_LABELS);
|
||||
|
||||
const fetchLatestRun = async (branch) => {
|
||||
const url = `${apiBase}/repos/${repo}/actions/workflows/${TESTS_WORKFLOW_FILE}/runs`
|
||||
+ `?branch=${encodeURIComponent(branch)}&event=push&status=completed&per_page=1`;
|
||||
const headers = {
|
||||
accept: 'application/vnd.github+json',
|
||||
'x-github-api-version': '2022-11-28',
|
||||
'user-agent': 'gsd-core-ci-next-health',
|
||||
};
|
||||
if (token) headers.authorization = `Bearer ${token}`;
|
||||
const response = await fetch(url, { headers, signal: AbortSignal.timeout(10000) });
|
||||
if (!response.ok) {
|
||||
throw new Error(`GitHub API responded ${response.status}`);
|
||||
}
|
||||
const text = await response.text();
|
||||
try {
|
||||
return JSON.parse(text);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
const result = await resolveNextHealth({ fetchLatestRun, eventName, baseRef, labels });
|
||||
|
||||
writeOutput([`verdict=${result.verdict}`]);
|
||||
writeSummary(`Base branch health: ${result.verdict}`);
|
||||
|
||||
if (result.verdict === VERDICT.RED) {
|
||||
const runUrl = runUrlOf(result.payload);
|
||||
process.stderr.write(
|
||||
`::error::the base branch "${baseRef}"'s own last Tests run is red: ${runUrl} — `
|
||||
+ 'wait for a fix-forward merge, or if THIS pull request is the fix, ask a '
|
||||
+ `maintainer to apply the "${BYPASS_LABEL}" label to explicitly override this gate.\n`,
|
||||
);
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (result.verdict === VERDICT.BYPASSED) {
|
||||
const runUrl = runUrlOf(result.payload);
|
||||
process.stderr.write(
|
||||
`::warning::the base branch "${baseRef}"'s own last Tests run is red: ${runUrl} — `
|
||||
+ `a human applied the "${BYPASS_LABEL}" label to explicitly override this gate.\n`,
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (result.verdict === VERDICT.INDETERMINATE) {
|
||||
process.stderr.write(
|
||||
`::warning::could not determine "${baseRef || '(no base ref)'}"'s Tests health (${result.reason}); `
|
||||
+ 'proceeding (fail-open).\n',
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
|
||||
process.stdout.write(`Base branch health: ${result.verdict}\n`);
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (require.main === module) {
|
||||
runMain(main);
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
VERDICT,
|
||||
BYPASS_LABEL,
|
||||
TESTS_WORKFLOW_FILE,
|
||||
classifyRunConclusion,
|
||||
resolveNextHealth,
|
||||
main,
|
||||
};
|
||||
@@ -1273,7 +1273,21 @@ function main() {
|
||||
// node process (also relieving per-process memory pressure from 170+ files at once).
|
||||
// Lowered from 90 to 60 after #1575 — macOS Node 22 shard 2/3 chunk 2 (~80 files
|
||||
// including state.test.cjs, perf-*, worktree-cleanup) exceeded 600s with 90.
|
||||
const MAX_FILES_PER_CHUNK = positiveNumberEnv(process.env.RUN_TESTS_MAX_FILES_PER_CHUNK, 60);
|
||||
//
|
||||
// 2026-09-06 (PR #4428 CI): a Windows full-matrix chunk (chunk 3/6, ~32/60
|
||||
// weight-budget units, dominated by codex-config.test.cjs at a genuinely
|
||||
// MEASURED weight of 17.87 — not a stale-table miss) still exceeded the
|
||||
// 600s per-chunk backstop. The weight table's calibration does not
|
||||
// transfer 1:1 to the Windows runner for install/subprocess-heavy work —
|
||||
// it needs a smaller budget than Linux/macOS to stay inside the same
|
||||
// wall-clock ceiling. Windows gets its own, lower cap (~33% reduction,
|
||||
// proportionate to the >30% single-file share codex-config.test.cjs alone
|
||||
// consumed of that chunk's budget); other platforms are unaffected.
|
||||
const DEFAULT_MAX_FILES_PER_CHUNK = process.platform === 'win32' ? 40 : 60;
|
||||
const MAX_FILES_PER_CHUNK = positiveNumberEnv(
|
||||
process.env.RUN_TESTS_MAX_FILES_PER_CHUNK,
|
||||
DEFAULT_MAX_FILES_PER_CHUNK,
|
||||
);
|
||||
// #2088 established that file COUNT is a poor proxy for a chunk's wall-clock:
|
||||
// install-heavy files (real installs) cost ~10x a unit file, and when several
|
||||
// land in the SAME chunk it blows the 600s backstop while unit-only chunks
|
||||
|
||||
552
tests/ci-next-health.test.cjs
Normal file
552
tests/ci-next-health.test.cjs
Normal file
@@ -0,0 +1,552 @@
|
||||
'use strict';
|
||||
|
||||
// #4422 — base-branch health gate.
|
||||
//
|
||||
// Risk asymmetry drives this suite (see scripts/ci-next-health.cjs's header):
|
||||
// false positive (a healthy `next` called RED) => blocks every PR merge
|
||||
// until a human notices and applies the `fix-next` bypass label.
|
||||
// false negative (a broken `next` called healthy) => reproduces the #4422
|
||||
// incident exactly (three unrelated PRs merged on top of an already-broken
|
||||
// `next`).
|
||||
// Unlike the mergeability preflight, this gate does NOT fail open on a
|
||||
// definite RED signal — only on an unavailable/inapplicable one.
|
||||
|
||||
const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const http = require('node:http');
|
||||
const fc = require('fast-check');
|
||||
|
||||
const { createTempDir, cleanup } = require('./helpers.cjs');
|
||||
const { runNode } = require('./helpers/process-seam.cjs');
|
||||
const { PROBE_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
|
||||
|
||||
const ROOT = path.join(__dirname, '..');
|
||||
const SCRIPT = path.join(ROOT, 'scripts', 'ci-next-health.cjs');
|
||||
|
||||
const {
|
||||
VERDICT,
|
||||
BYPASS_LABEL,
|
||||
classifyRunConclusion,
|
||||
resolveNextHealth,
|
||||
main,
|
||||
} = require('../scripts/ci-next-health.cjs');
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fakes. The seams are parameters, not module patches — no global state, so
|
||||
// every case is order-independent.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** A fetchLatestRun fake that replays a scripted response; an Error is thrown. */
|
||||
function scriptedFetch(response) {
|
||||
const calls = [];
|
||||
const fn = async (branch) => {
|
||||
calls.push(branch);
|
||||
if (response instanceof Error) throw response;
|
||||
return response;
|
||||
};
|
||||
fn.calls = calls;
|
||||
return fn;
|
||||
}
|
||||
|
||||
function runPayload(conclusion, htmlUrl = 'https://github.com/open-gsd/gsd-core/actions/runs/1') {
|
||||
return { workflow_runs: [{ conclusion, html_url: htmlUrl }] };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// A. classifyRunConclusion — pure
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('ci-next-health: classifyRunConclusion', () => {
|
||||
test('classifies an undefined payload as INDETERMINATE', () => {
|
||||
assert.equal(classifyRunConclusion(undefined), VERDICT.INDETERMINATE);
|
||||
});
|
||||
|
||||
test('classifies a null payload as INDETERMINATE', () => {
|
||||
assert.equal(classifyRunConclusion(null), VERDICT.INDETERMINATE);
|
||||
});
|
||||
|
||||
test('classifies a non-object payload as INDETERMINATE', () => {
|
||||
for (const value of [0, 'str', true, 42]) {
|
||||
assert.equal(classifyRunConclusion(value), VERDICT.INDETERMINATE);
|
||||
}
|
||||
});
|
||||
|
||||
test('classifies a payload whose workflow_runs is not an array as INDETERMINATE', () => {
|
||||
for (const value of [undefined, null, 'x', {}, 1]) {
|
||||
assert.equal(classifyRunConclusion({ workflow_runs: value }), VERDICT.INDETERMINATE);
|
||||
}
|
||||
});
|
||||
|
||||
test('classifies an empty workflow_runs array as CLEAN', () => {
|
||||
// A brand-new release/**/hotfix/** branch with no Tests history yet must
|
||||
// not be permanently unmergeable.
|
||||
assert.equal(classifyRunConclusion({ workflow_runs: [] }), VERDICT.CLEAN);
|
||||
});
|
||||
|
||||
test('classifies conclusion: success as CLEAN', () => {
|
||||
assert.equal(classifyRunConclusion(runPayload('success')), VERDICT.CLEAN);
|
||||
});
|
||||
|
||||
test('classifies conclusion: failure as RED', () => {
|
||||
assert.equal(classifyRunConclusion(runPayload('failure')), VERDICT.RED);
|
||||
});
|
||||
|
||||
test('classifies conclusion: cancelled as RED', () => {
|
||||
// A cancelled/timed-out run on the base branch is not evidence of health.
|
||||
assert.equal(classifyRunConclusion(runPayload('cancelled')), VERDICT.RED);
|
||||
});
|
||||
|
||||
test('classifies every non-success conclusion as RED', () => {
|
||||
for (const conclusion of ['timed_out', 'action_required', 'neutral', 'skipped', 'stale']) {
|
||||
assert.equal(
|
||||
classifyRunConclusion(runPayload(conclusion)),
|
||||
VERDICT.RED,
|
||||
`conclusion=${conclusion} must be RED`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('only reads the FIRST run in workflow_runs', () => {
|
||||
const payload = { workflow_runs: [{ conclusion: 'success' }, { conclusion: 'failure' }] };
|
||||
assert.equal(classifyRunConclusion(payload), VERDICT.CLEAN);
|
||||
});
|
||||
|
||||
test('VERDICT is frozen and its atom set is locked', () => {
|
||||
assert.ok(Object.isFrozen(VERDICT));
|
||||
assert.deepEqual(
|
||||
Object.keys(VERDICT).sort(),
|
||||
['BYPASSED', 'CLEAN', 'INDETERMINATE', 'RED', 'SKIPPED_NOT_APPLICABLE'],
|
||||
);
|
||||
});
|
||||
|
||||
// A well-formed payload has an ARRAY workflow_runs of length 0 or 1, whose
|
||||
// sole element (if present) carries an arbitrary `conclusion` string. A
|
||||
// malformed payload is anything else: a non-object payload, or an object
|
||||
// whose `workflow_runs` is not an array.
|
||||
const wellFormedPayloadArb = fc.record({
|
||||
workflow_runs: fc.oneof(
|
||||
fc.constant([]),
|
||||
fc.tuple(fc.record({ conclusion: fc.string() })),
|
||||
),
|
||||
});
|
||||
|
||||
const malformedNonArrayRunsArb = fc.oneof(
|
||||
fc.constant(undefined),
|
||||
fc.constant(null),
|
||||
fc.string(),
|
||||
fc.integer(),
|
||||
fc.boolean(),
|
||||
fc.dictionary(fc.string(), fc.string()),
|
||||
);
|
||||
|
||||
const malformedPayloadArb = fc.oneof(
|
||||
fc.constant(null),
|
||||
fc.constant(undefined),
|
||||
fc.string(),
|
||||
fc.integer(),
|
||||
fc.boolean(),
|
||||
fc.record({ workflow_runs: malformedNonArrayRunsArb }),
|
||||
);
|
||||
|
||||
const payloadArb = fc.oneof(wellFormedPayloadArb, malformedPayloadArb);
|
||||
|
||||
test('CLEAN iff (workflow_runs is empty) or (first run succeeded); everything else RED or INDETERMINATE (property)', () => {
|
||||
fc.assert(
|
||||
fc.property(payloadArb, (payload) => {
|
||||
const verdict = classifyRunConclusion(payload);
|
||||
|
||||
const isObject = payload !== null && typeof payload === 'object';
|
||||
const hasArrayRuns = isObject && Array.isArray(payload.workflow_runs);
|
||||
|
||||
if (!hasArrayRuns) {
|
||||
return verdict === VERDICT.INDETERMINATE;
|
||||
}
|
||||
|
||||
const isClean = payload.workflow_runs.length === 0
|
||||
|| payload.workflow_runs[0].conclusion === 'success';
|
||||
|
||||
if (isClean) return verdict === VERDICT.CLEAN;
|
||||
return verdict === VERDICT.RED;
|
||||
}),
|
||||
{ seed: 4422, numRuns: 500, verbose: true },
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// B. resolveNextHealth — dependency-injected orchestrator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('ci-next-health: resolveNextHealth', () => {
|
||||
test('a push event is SKIPPED_NOT_APPLICABLE without calling fetchLatestRun', async () => {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('failure'));
|
||||
const result = await resolveNextHealth({ fetchLatestRun, eventName: 'push', baseRef: 'next', labels: [] });
|
||||
assert.equal(result.verdict, VERDICT.SKIPPED_NOT_APPLICABLE);
|
||||
assert.equal(fetchLatestRun.calls.length, 0);
|
||||
});
|
||||
|
||||
test('workflow_dispatch is SKIPPED_NOT_APPLICABLE without calling fetchLatestRun', async () => {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('failure'));
|
||||
const result = await resolveNextHealth({
|
||||
fetchLatestRun, eventName: 'workflow_dispatch', baseRef: 'next', labels: [],
|
||||
});
|
||||
assert.equal(result.verdict, VERDICT.SKIPPED_NOT_APPLICABLE);
|
||||
assert.equal(fetchLatestRun.calls.length, 0);
|
||||
});
|
||||
|
||||
test('an unknown/absent event name is SKIPPED_NOT_APPLICABLE without calling', async () => {
|
||||
for (const eventName of ['', undefined, 'schedule', 'release', 'issues']) {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('failure'));
|
||||
const result = await resolveNextHealth({ fetchLatestRun, eventName, baseRef: 'next', labels: [] });
|
||||
assert.equal(result.verdict, VERDICT.SKIPPED_NOT_APPLICABLE);
|
||||
assert.equal(fetchLatestRun.calls.length, 0);
|
||||
}
|
||||
});
|
||||
|
||||
test('a pull_request event with no resolvable base ref is INDETERMINATE', async () => {
|
||||
for (const baseRef of [undefined, null, '', ' ']) {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('failure'));
|
||||
const result = await resolveNextHealth({ fetchLatestRun, eventName: 'pull_request', baseRef, labels: [] });
|
||||
assert.equal(result.verdict, VERDICT.INDETERMINATE);
|
||||
assert.equal(fetchLatestRun.calls.length, 0);
|
||||
}
|
||||
});
|
||||
|
||||
test('a merge_group event with no resolvable base ref is INDETERMINATE', async () => {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('failure'));
|
||||
const result = await resolveNextHealth({ fetchLatestRun, eventName: 'merge_group', baseRef: '', labels: [] });
|
||||
assert.equal(result.verdict, VERDICT.INDETERMINATE);
|
||||
assert.equal(fetchLatestRun.calls.length, 0);
|
||||
});
|
||||
|
||||
test('a throwing fetchLatestRun degrades to INDETERMINATE', async () => {
|
||||
const fetchLatestRun = scriptedFetch(new Error('ECONNRESET'));
|
||||
const result = await resolveNextHealth({ fetchLatestRun, eventName: 'pull_request', baseRef: 'next', labels: [] });
|
||||
assert.equal(result.verdict, VERDICT.INDETERMINATE);
|
||||
});
|
||||
|
||||
test('a RED verdict with the fix-next label is BYPASSED', async () => {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('failure'));
|
||||
const result = await resolveNextHealth({
|
||||
fetchLatestRun, eventName: 'pull_request', baseRef: 'next', labels: ['needs-triage', BYPASS_LABEL],
|
||||
});
|
||||
assert.equal(result.verdict, VERDICT.BYPASSED);
|
||||
});
|
||||
|
||||
test('a RED verdict with no matching label stays RED', async () => {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('failure'));
|
||||
const result = await resolveNextHealth({
|
||||
fetchLatestRun, eventName: 'pull_request', baseRef: 'next', labels: ['needs-triage'],
|
||||
});
|
||||
assert.equal(result.verdict, VERDICT.RED);
|
||||
});
|
||||
|
||||
test('a RED verdict with an absent labels array stays RED (merge_group has no label surface)', async () => {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('failure'));
|
||||
const result = await resolveNextHealth({
|
||||
fetchLatestRun, eventName: 'merge_group', baseRef: 'next', labels: undefined,
|
||||
});
|
||||
assert.equal(result.verdict, VERDICT.RED);
|
||||
});
|
||||
|
||||
test('a CLEAN verdict (success conclusion) is not affected by labels', async () => {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('success'));
|
||||
const result = await resolveNextHealth({
|
||||
fetchLatestRun, eventName: 'pull_request', baseRef: 'next', labels: [BYPASS_LABEL],
|
||||
});
|
||||
assert.equal(result.verdict, VERDICT.CLEAN);
|
||||
});
|
||||
|
||||
test('a CLEAN verdict (empty workflow_runs array)', async () => {
|
||||
const fetchLatestRun = scriptedFetch({ workflow_runs: [] });
|
||||
const result = await resolveNextHealth({
|
||||
fetchLatestRun, eventName: 'pull_request', baseRef: 'release/1.0', labels: [],
|
||||
});
|
||||
assert.equal(result.verdict, VERDICT.CLEAN);
|
||||
});
|
||||
|
||||
test('merge_group resolves the same way as pull_request given a base ref', async () => {
|
||||
const fetchLatestRun = scriptedFetch(runPayload('success'));
|
||||
const result = await resolveNextHealth({
|
||||
fetchLatestRun, eventName: 'merge_group', baseRef: 'next', labels: [],
|
||||
});
|
||||
assert.equal(result.verdict, VERDICT.CLEAN);
|
||||
assert.deepEqual(fetchLatestRun.calls, ['next']);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// C. main() — integration through the process seam against a real local API.
|
||||
//
|
||||
// The CLI's real fetch path is exercised by pointing GITHUB_API_URL at a
|
||||
// throwaway localhost server, so no production test-mode branch exists and
|
||||
// nothing reaches api.github.com.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const SENTINEL_TOKEN = 'ghs_sentinel_must_never_be_echoed_4422';
|
||||
|
||||
/** Start a one-shot API stub. `handler(requestCount)` returns { status, body }. */
|
||||
async function startApiStub(handler) {
|
||||
let requestCount = 0;
|
||||
const server = http.createServer((req, res) => {
|
||||
const { status, body } = handler(requestCount++);
|
||||
res.writeHead(status, { 'content-type': 'application/json' });
|
||||
res.end(typeof body === 'string' ? body : JSON.stringify(body));
|
||||
});
|
||||
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve));
|
||||
const { port } = server.address();
|
||||
return {
|
||||
url: `http://127.0.0.1:${port}`,
|
||||
close: () => new Promise((resolve) => server.close(resolve)),
|
||||
get requestCount() { return requestCount; },
|
||||
};
|
||||
}
|
||||
|
||||
function runCli(env, { outputPath } = {}) {
|
||||
return runNode([SCRIPT], {
|
||||
cwd: ROOT,
|
||||
timeoutMs: PROBE_TIMEOUT_MS,
|
||||
env: {
|
||||
...process.env,
|
||||
GITHUB_REPOSITORY: 'open-gsd/gsd-core',
|
||||
GITHUB_TOKEN: SENTINEL_TOKEN,
|
||||
GITHUB_BASE_REF: 'next',
|
||||
...(outputPath ? { GITHUB_OUTPUT: outputPath } : {}),
|
||||
...env,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// `runNode` (tests/helpers/process-seam.cjs) is spawnSync: it blocks this
|
||||
// process's event loop for the whole child lifetime. `startApiStub` is an
|
||||
// http.createServer living in THIS process, so while spawnSync blocks, the
|
||||
// stub can never accept the child's connection. Any test that needs the stub
|
||||
// must instead call main() in-process, which keeps this process's event loop
|
||||
// live so the stub can actually answer. Mirrors tests/ci-pr-mergeability.test.cjs.
|
||||
async function callMain(t, env, { outputPath } = {}) {
|
||||
const overrides = {
|
||||
GITHUB_REPOSITORY: 'open-gsd/gsd-core',
|
||||
GITHUB_TOKEN: SENTINEL_TOKEN,
|
||||
GITHUB_BASE_REF: 'next',
|
||||
...(outputPath ? { GITHUB_OUTPUT: outputPath } : {}),
|
||||
...env,
|
||||
};
|
||||
const saved = new Map();
|
||||
for (const key of Object.keys(overrides)) saved.set(key, process.env[key]);
|
||||
Object.assign(process.env, overrides);
|
||||
t.after(() => {
|
||||
for (const [key, value] of saved) {
|
||||
if (value === undefined) delete process.env[key];
|
||||
else process.env[key] = value;
|
||||
}
|
||||
});
|
||||
|
||||
const stdout = [];
|
||||
const stderr = [];
|
||||
t.mock.method(process.stdout, 'write', (chunk) => { stdout.push(String(chunk)); return true; });
|
||||
t.mock.method(process.stderr, 'write', (chunk) => { stderr.push(String(chunk)); return true; });
|
||||
|
||||
const code = await main([]);
|
||||
return { code, stdout: stdout.join(''), stderr: stderr.join('') };
|
||||
}
|
||||
|
||||
function readOutputs(outputPath) {
|
||||
const raw = fs.readFileSync(outputPath, 'utf8');
|
||||
const outputs = {};
|
||||
for (const line of raw.split(/\r?\n/)) {
|
||||
const index = line.indexOf('=');
|
||||
if (index > 0) outputs[line.slice(0, index)] = line.slice(index + 1);
|
||||
}
|
||||
return outputs;
|
||||
}
|
||||
|
||||
describe('ci-next-health: CLI', () => {
|
||||
test('exits 0 and writes a skip verdict on a push event', (t) => {
|
||||
const dir = createTempDir('next-health-skip-');
|
||||
t.after(() => cleanup(dir));
|
||||
const outputPath = path.join(dir, 'gh-output');
|
||||
|
||||
const result = runCli({ GITHUB_EVENT_NAME: 'push' }, { outputPath });
|
||||
|
||||
assert.equal(result.exitCode, 0, result.stderr);
|
||||
assert.equal(readOutputs(outputPath).verdict, VERDICT.SKIPPED_NOT_APPLICABLE);
|
||||
});
|
||||
|
||||
test('exits 1 and annotates the failing run on a RED base branch', async (t) => {
|
||||
const dir = createTempDir('next-health-red-');
|
||||
const api = await startApiStub(() => ({
|
||||
status: 200,
|
||||
body: { workflow_runs: [{ conclusion: 'failure', html_url: 'https://example.test/runs/99' }] },
|
||||
}));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
const outputPath = path.join(dir, 'gh-output');
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'pull_request', GITHUB_API_URL: api.url, PR_LABELS: '' },
|
||||
{ outputPath },
|
||||
);
|
||||
|
||||
assert.equal(result.code, 1, `stdout: ${result.stdout}\nstderr: ${result.stderr}`);
|
||||
const combined = `${result.stdout}${result.stderr}`;
|
||||
assert.ok(combined.includes('::error::'), 'must emit a workflow error annotation');
|
||||
assert.match(combined, /https:\/\/example\.test\/runs\/99/, 'must name the failing run URL');
|
||||
assert.equal(readOutputs(outputPath).verdict, VERDICT.RED);
|
||||
});
|
||||
|
||||
test('exits 0 with a warning when the fix-next label bypasses a RED base branch', async (t) => {
|
||||
const dir = createTempDir('next-health-bypass-');
|
||||
const api = await startApiStub(() => ({
|
||||
status: 200,
|
||||
body: { workflow_runs: [{ conclusion: 'failure', html_url: 'https://example.test/runs/100' }] },
|
||||
}));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
const outputPath = path.join(dir, 'gh-output');
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'pull_request', GITHUB_API_URL: api.url, PR_LABELS: `needs-triage,${BYPASS_LABEL}` },
|
||||
{ outputPath },
|
||||
);
|
||||
|
||||
assert.equal(result.code, 0, `stdout: ${result.stdout}\nstderr: ${result.stderr}`);
|
||||
const combined = `${result.stdout}${result.stderr}`;
|
||||
assert.ok(combined.includes('::warning::'), 'must warn that a human bypassed the gate');
|
||||
assert.match(combined, /https:\/\/example\.test\/runs\/100/, 'the warning must name the failing run URL');
|
||||
assert.ok(!combined.includes('::error::'), 'a bypassed gate must not also emit an error annotation');
|
||||
assert.equal(readOutputs(outputPath).verdict, VERDICT.BYPASSED);
|
||||
});
|
||||
|
||||
test('exits 0 on a clean base branch', async (t) => {
|
||||
const dir = createTempDir('next-health-clean-');
|
||||
const api = await startApiStub(() => ({ status: 200, body: { workflow_runs: [{ conclusion: 'success' }] } }));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
const outputPath = path.join(dir, 'gh-output');
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'pull_request', GITHUB_API_URL: api.url, PR_LABELS: '' },
|
||||
{ outputPath },
|
||||
);
|
||||
|
||||
assert.equal(result.code, 0, result.stderr);
|
||||
assert.equal(readOutputs(outputPath).verdict, VERDICT.CLEAN);
|
||||
});
|
||||
|
||||
test('resolves the merge_group base ref from MERGE_GROUP_BASE_REF, stripping refs/heads/', async (t) => {
|
||||
const dir = createTempDir('next-health-mergequeue-');
|
||||
const api = await startApiStub(() => ({ status: 200, body: { workflow_runs: [{ conclusion: 'success' }] } }));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
const outputPath = path.join(dir, 'gh-output');
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'merge_group', MERGE_GROUP_BASE_REF: 'refs/heads/next', GITHUB_API_URL: api.url },
|
||||
{ outputPath },
|
||||
);
|
||||
|
||||
assert.equal(result.code, 0, result.stderr);
|
||||
assert.equal(readOutputs(outputPath).verdict, VERDICT.CLEAN);
|
||||
});
|
||||
|
||||
test('fails open when the API rejects the read', async (t) => {
|
||||
const dir = createTempDir('next-health-403-');
|
||||
const api = await startApiStub(() => ({ status: 403, body: { message: 'Resource not accessible' } }));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
const outputPath = path.join(dir, 'gh-output');
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'pull_request', GITHUB_API_URL: api.url },
|
||||
{ outputPath },
|
||||
);
|
||||
|
||||
assert.equal(result.code, 0, 'an unreadable base branch is not a red base branch');
|
||||
assert.equal(readOutputs(outputPath).verdict, VERDICT.INDETERMINATE);
|
||||
});
|
||||
|
||||
test('fails open when the API body is not a JSON object', async (t) => {
|
||||
const dir = createTempDir('next-health-badjson-');
|
||||
const api = await startApiStub(() => ({ status: 200, body: 'not json at all' }));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
const outputPath = path.join(dir, 'gh-output');
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'pull_request', GITHUB_API_URL: api.url },
|
||||
{ outputPath },
|
||||
);
|
||||
|
||||
assert.equal(result.code, 0);
|
||||
assert.equal(readOutputs(outputPath).verdict, VERDICT.INDETERMINATE);
|
||||
});
|
||||
|
||||
test('works when GITHUB_OUTPUT is not set', (t) => {
|
||||
const dir = createTempDir('next-health-nooutput-');
|
||||
t.after(() => cleanup(dir));
|
||||
|
||||
const result = runCli({ GITHUB_EVENT_NAME: 'push', GITHUB_OUTPUT: '' });
|
||||
|
||||
assert.equal(result.exitCode, 0, result.stderr);
|
||||
});
|
||||
|
||||
test('an unwritable GITHUB_OUTPUT does not change the exit code', async (t) => {
|
||||
// A failure while REPORTING the verdict must never invert the gate.
|
||||
// Injected by pointing at a path whose parent does not exist — no mode-bit
|
||||
// tricks, which root Docker/CI silently bypasses.
|
||||
const dir = createTempDir('next-health-badout-');
|
||||
const api = await startApiStub(() => ({ status: 200, body: { workflow_runs: [{ conclusion: 'failure' }] } }));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
const outputPath = path.join(dir, 'no-such-dir', 'gh-output');
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'pull_request', GITHUB_API_URL: api.url, PR_LABELS: '' },
|
||||
{ outputPath },
|
||||
);
|
||||
|
||||
assert.equal(result.code, 1, 'a RED base branch must still exit 1 when the output write fails');
|
||||
});
|
||||
|
||||
test('never echoes the token', async (t) => {
|
||||
const dir = createTempDir('next-health-token-');
|
||||
const api = await startApiStub(() => ({ status: 500, body: { message: 'boom' } }));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'pull_request', GITHUB_API_URL: api.url },
|
||||
{ outputPath: path.join(dir, 'gh-output') },
|
||||
);
|
||||
|
||||
assert.ok(!`${result.stdout}${result.stderr}`.includes(SENTINEL_TOKEN));
|
||||
});
|
||||
|
||||
test('reports a failure without a stack trace', async (t) => {
|
||||
const dir = createTempDir('next-health-nostack-');
|
||||
const api = await startApiStub(() => ({ status: 200, body: { workflow_runs: [{ conclusion: 'failure' }] } }));
|
||||
t.after(async () => { await api.close(); cleanup(dir); });
|
||||
|
||||
const result = await callMain(
|
||||
t,
|
||||
{ GITHUB_EVENT_NAME: 'pull_request', GITHUB_API_URL: api.url, PR_LABELS: '' },
|
||||
{ outputPath: path.join(dir, 'gh-output') },
|
||||
);
|
||||
|
||||
assert.ok(!`${result.stdout}${result.stderr}`.includes(' at '), 'no raw stack frames');
|
||||
});
|
||||
|
||||
test('prints usage', () => {
|
||||
const result = runNode([SCRIPT, '--help'], { cwd: ROOT, timeoutMs: PROBE_TIMEOUT_MS });
|
||||
assert.equal(result.exitCode, 0);
|
||||
assert.ok(/Usage/i.test(result.stdout));
|
||||
});
|
||||
|
||||
test('rejects an unknown argument', () => {
|
||||
const result = runNode([SCRIPT, '--nope'], { cwd: ROOT, timeoutMs: PROBE_TIMEOUT_MS });
|
||||
assert.notEqual(result.exitCode, 0);
|
||||
assert.ok(`${result.stdout}${result.stderr}`.includes('--nope'));
|
||||
});
|
||||
});
|
||||
@@ -30,9 +30,25 @@ const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { runHook: runHookSeam } = require('./helpers/process-seam.cjs');
|
||||
const { createTempDir, cleanup } = require('./helpers.cjs');
|
||||
const { NPM_VIEW_TIMEOUT_MS } = require('../gsd-core/bin/check-latest-version.cjs');
|
||||
|
||||
const WORKER_PATH = path.join(__dirname, '..', 'hooks', 'gsd-check-update-worker.js');
|
||||
|
||||
// #4091/2026-09-06 Windows CI incident: the worker's real `npm view` call
|
||||
// (checkLatestVersion, gsd-core/bin/check-latest-version.cjs) is bounded at
|
||||
// NPM_VIEW_TIMEOUT_MS. This outer test harness timeout used to be hardcoded
|
||||
// to the SAME 15000ms, so a slow registry response raced two SIGKILLs at the
|
||||
// exact same wall-clock instant: the worker's own inner npm-view timeout
|
||||
// fires and it needs real time to catch that failure, build a degraded
|
||||
// result, and atomically publish the cache — but the outer harness could
|
||||
// kill the whole process tree first (exitCode: null, empty stderr) before
|
||||
// the worker ever got the chance. Windows's shell-wrapped npm subprocess
|
||||
// (cmd.exe wrapper, see src/shell-command-projection.cts) adds enough spawn
|
||||
// overhead to make this race lose more often there, but the zero-margin race
|
||||
// itself is platform-agnostic. This margin gives the worker real headroom
|
||||
// beyond the inner timeout it wraps.
|
||||
const WORKER_TEARDOWN_MARGIN_MS = 10_000;
|
||||
|
||||
// allow-test-rule: structural-regression-guard (#4091)
|
||||
// Feeds the real worker source (readFileSync) into the structural assertions
|
||||
// below. The behavior it guards — rename-atomic publish of a file shared
|
||||
@@ -108,11 +124,25 @@ describe('gsd-check-update-worker.js: atomic cache publish (#4091)', () => {
|
||||
GSD_PROJECT_VERSION_FILE: path.join(cacheDir, 'no-such-project', 'VERSION'),
|
||||
GSD_GLOBAL_VERSION_FILE: path.join(cacheDir, 'no-such-global', 'VERSION'),
|
||||
};
|
||||
const r = runHookSeam(WORKER_PATH, [], { env, timeoutMs: 15000 });
|
||||
const r = runHookSeam(WORKER_PATH, [], { env, timeoutMs: NPM_VIEW_TIMEOUT_MS + WORKER_TEARDOWN_MARGIN_MS });
|
||||
assert.equal(r.exitCode, 0, `worker must exit 0; stderr: ${r.stderr}`);
|
||||
const cache = JSON.parse(fs.readFileSync(cacheFile, 'utf8'));
|
||||
assert.equal(cache.installed, '0.0.0', 'worker record replaced the pre-existing one');
|
||||
const residue = fs.readdirSync(cacheDir).filter((f) => f.startsWith('cache.json.tmp') || f.includes('.tmp-'));
|
||||
assert.deepEqual(residue, [], 'no temp stage files may remain after a successful publish');
|
||||
});
|
||||
|
||||
test('outer worker-run timeout keeps real margin beyond the inner npm-view timeout (#4091 exact-tie race)', () => {
|
||||
assert.ok(
|
||||
WORKER_TEARDOWN_MARGIN_MS >= 5000,
|
||||
'the outer test timeout must give the worker real margin beyond the inner npm-view ' +
|
||||
'timeout (NPM_VIEW_TIMEOUT_MS) it wraps, or a slow registry response races the two ' +
|
||||
'SIGKILLs (see #4091/2026-09-06 Windows CI incident: exact-tie timeout killed the worker ' +
|
||||
'before it could degrade gracefully)',
|
||||
);
|
||||
assert.ok(
|
||||
NPM_VIEW_TIMEOUT_MS + WORKER_TEARDOWN_MARGIN_MS > NPM_VIEW_TIMEOUT_MS,
|
||||
'the combined outer timeout must strictly exceed the inner npm-view timeout it wraps',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user