Files
msd-core/tests/check-env.test.cjs
Tom Boucher 48b1e35187 fix(#431): enforce H1 shell policy (linux=bash, macOS=zsh, windows=pwsh) across PR + release gates (#434)
* test(#431): policy-shell-pinning linter — RED baseline (37 violations on origin/next)

Adds scripts/workflow-policy.cjs: H1 shell-policy linter with POLICY map,
VIOLATION enum, matrix expansion, effective-shell resolution order, and
runPolicyLint({ workflowsDir }) entry point.

Adds tests/policy-shell-pinning.test.cjs: 8 tests (baseline + 6 synthetic
counter-tests). Synthetic tests 2–7 pass; baseline test is intentionally RED
(37 violations: 28 in test.yml, 9 in install-smoke.yml — all macos/windows
lanes using shell: bash instead of native zsh/pwsh).

Adds js-yaml@4.1.1 as devDependency for YAML parsing.

* fix(#431): switch ubuntu/windows lanes to native shells; extract bash-isms to Node

Remove all explicit shell: bash pins from ubuntu-only jobs (changes, lint-tests,
coverage, required-tests, smoke-unpacked) — ubuntu runner default is bash, which
is both H1-compliant and the runner default, making the pin redundant.

For the test and test-full mixed-OS jobs (ubuntu+windows, windows+macos):
- Move bash-ism steps to shell-agnostic Node scripts:
    scripts/ci-guard-runner.cjs       — RUNNER_ENVIRONMENT check
    scripts/ci-rebase-check.cjs       — git fetch+merge PR base branch
    scripts/check-npm-integrity.cjs   — Node port of check-npm-integrity.sh
    scripts/ci-prepare-test-scope.cjs — write .ci-selected-tests.txt
    scripts/ci-smoke-skip.cjs         — set skip= output for full-only matrix entries
- Remove shell: bash from simple npm/node command steps (runner default applies)

This brings Windows violations from 19 to 0. Remaining 17 violations are all
MACOS_MISSING_EXPLICIT_ZSH in mixed-OS matrix jobs (test-full: windows+macos,
install-smoke smoke: ubuntu+macos) — these require job splitting to fix; see
BLOCKER in PR description.

* fix(#431): update workflow-shell-pinning test for H1 policy

The old test required all Windows-targeting npm steps to pin shell: bash
(to prevent pwsh stderr-swallow). Under H1, Windows runners must use
pwsh (native, no pin needed) — shell: bash on Windows is now the
violation, not the fix.

Update findViolations() to flag npm steps with effectiveShell === 'bash'
(rather than effectiveShell === null). Update synthetic tests to verify
the H1-inverted semantics: defaults.run.shell: bash on Windows is now 2
violations, not 0. Update test name and assertion messages to describe
the H1 constraint rather than the old missing-pin constraint.

* fix(#431): extend policy linter to resolve matrix.shell expressions

- expandRunsOn now captures all matrix.include row keys as realization
  context (os, node-version, shell, full_only, etc.) instead of only os
- effectiveShell now accepts a realizationContext and resolves
  ${{ matrix.<key> }} expressions against it before checking policy
- Unresolvable matrix key in shell expression emits UNRESOLVABLE_MATRIX
- Add 3 new tests: positive (zsh+pwsh per row → 0 violations),
  counter (bash in macOS row → WRONG_SHELL_FOR_OS), counter (missing
  shell key → UNRESOLVABLE_MATRIX)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#431): apply matrix.shell pattern to test-full and smoke jobs (clears BLOCKER)

test-full job (test.yml):
- Add shell: pwsh/zsh per matrix.include row (windows-latest→pwsh,
  macos-latest→zsh)
- Add job-level defaults.run.shell: ${{ matrix.shell }}
- No step-level shell pins existed to remove

smoke job (install-smoke.yml):
- Add shell: bash/zsh per matrix.include row (ubuntu→bash, macos→zsh)
- Add job-level defaults.run.shell: ${{ matrix.shell }}
- No step-level shell pins existed to remove

Policy linter now reports 0 violations across all workflow files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(#431): migrate .sh check scripts to .cjs; remove .sh originals

- Add scripts/check-env.cjs: Node.js port of check-env.sh with
  identical exit codes (0/1/2), human-readable and --json output,
  --help flag, and all 5 checks (node-version, npm-version,
  lockfile-present, lockfile-sync, version-manager-pin)
- Migrate all callers:
  - package.json check:env → node scripts/check-env.cjs
  - package.json check:integrity → node scripts/check-npm-integrity.cjs
  - scripts/ci-test-scope.cjs path strings → .cjs equivalents
  - .github/workflows/release.yml rc+finalize jobs → node .cjs (drop chmod+x)
  - .github/workflows/security-scan.yml → node .cjs (drop chmod+x)
  - tests/check-env.test.cjs → spawn node process.execPath [.cjs]
  - tests/npm-integrity-gate.test.cjs → spawn node process.execPath [.cjs]
- Delete scripts/check-env.sh and scripts/check-npm-integrity.sh

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(#431): update doc references from .sh to .cjs

Update SECURITY.md and docs/contributing/bootstrap.md to reference the
canonical Node invocation instead of the removed bash scripts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#431): use per-step shell:matrix.shell instead of defaults.run.shell (GHA compat)

GHA does not reliably resolve matrix expressions inside defaults.run.shell.
Per-step shell: always resolves correctly. Removed the defaults.run.shell block
from the test-full job (test.yml) and the smoke job (install-smoke.yml), and
added shell: \${{ matrix.shell }} directly on every run: step in both jobs.

Codex finding: defaults.run.shell with matrix expressions is not a
GHA-supported pattern; per-step shell: is the safe form.

* fix(#431): policy linter validates every matrix.include row independently

Removed runner-label-only dedup from expandRunsOn() in workflow-policy.cjs.
The prior guard (if !realizations.find(r => r.runner === runner)) collapsed
two macos-latest rows with different node-version/shell contexts into one,
hiding the second row's policy violation.

Each matrix.include row is a distinct CI realization with its own context;
validating it twice is harmless but skipping it causes false negatives.

Added counter-test (Test 8) in tests/policy-shell-pinning.test.cjs:
two macos-latest rows (shell:zsh compliant + shell:bash violation) must
produce exactly one WRONG_SHELL_FOR_OS violation on the second row.

* fix(#431): remove dedup-by-runner in Cartesian matrix.<key> expansion (Codex round 3)

The base-list path in expandRunsOn (matrix.<key> arrays, e.g. matrix.os)
previously guarded each push with `if (!realizations.find(r => r.runner === runner))`,
collapsing duplicate runner values into a single realization and hiding policy
violations on later rows of a Cartesian matrix.

Remove the guard unconditionally; each entry in the base-list array now produces
its own realization, matching the same fix already applied to the matrix.include path.

Add counter-test "Cartesian matrix os × shell — dedup must not collapse rows by
runner alone": matrix.os: [macos-latest, macos-latest] + shell: ${{ matrix.shell }}
now yields 2 realizations (not 1). Documents that Cartesian cross-product expansion
(carrying all keys into realization context) is a separate follow-up; current violations
are UNRESOLVABLE_MATRIX pending that work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#431): remove 60s timeout regression on npm ci --dry-run (parity with check-env.sh)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#431): ci-rebase-check.cjs — return truthy sentinel on success (Codex round 4)

run() used execFileSync with stdio:'inherit', which returns null on success.
Caller checked `result !== null`, always false → every successful fetch fell
through to "failed after 3 attempts" exit-1 path.

Fix: run() now returns true on success, false on failure.
Update caller from `result !== null` to `if (result)`.

Adds tests/ci-rebase-check.test.cjs (5 tests) covering the sentinel contract
and a local-bare-remote integration smoke that verifies the full fetch+merge
path exits 0 when fetch succeeds.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: CI Rebase Check <ci@gsd-redux>
2026-05-28 09:23:59 -04:00

235 lines
11 KiB
JavaScript

/**
* Tests for scripts/check-env.cjs (issue #117).
*
* Verifies the environment validator exits correctly and emits
* structured output for every documented check:
* 1. Node version vs engines.node constraint
* 2. npm version vs engines.npm constraint (if present)
* 3. Lockfile presence
* 4. Lockfile sync (npm ci --dry-run)
* 5. Version-manager pin file matches active Node major
* 6. --json flag produces parseable JSON with documented shape
* 7. Integration smoke: exits 0 on the live worktree root
*
* Sources:
* npm engines: https://docs.npmjs.com/cli/v10/configuring-npm/package-json#engines
* npm ci: https://docs.npmjs.com/cli/v10/commands/npm-ci
*/
'use strict';
const { describe, test, before, after } = require('node:test');
const assert = require('node:assert/strict');
const path = require('node:path');
const fs = require('node:fs');
const { spawnSync } = require('node:child_process');
const SCRIPT = path.resolve(__dirname, '..', 'scripts', 'check-env.cjs');
const FIXTURE_ROOT = path.resolve(__dirname, 'fixtures', 'check-env');
const LIVE_ROOT = path.resolve(__dirname, '..');
/**
* Run check-env.cjs synchronously in `cwd` with optional extra args.
* Returns { status, stdout, stderr }.
* @param {string} cwd
* @param {string[]} args
* @param {Record<string,string>} [envOverrides] - optional env vars to overlay
* on process.env. Pass { CI: '' } to suppress GitHub Actions CI detection
* so that version-manager-pin is exercised even inside CI runners.
*/
function runScript(cwd, args = [], envOverrides = {}) {
const result = spawnSync(process.execPath, [SCRIPT, ...args], {
cwd,
encoding: 'utf8',
timeout: 30_000,
env: { ...process.env, ...envOverrides },
});
return {
status: result.status ?? 1,
stdout: result.stdout ?? '',
stderr: result.stderr ?? '',
};
}
describe('check-env.cjs', () => {
// -------------------------------------------------------------------------
// Dynamic .nvmrc setup: write fixture .nvmrc files at test-run time so the
// tests are correct across all Node major versions in the CI matrix (Node 22,
// 24, 26, …). A hardcoded value like "26" passes on Node 26 but fails on
// every other matrix row; using the active major makes the fixture portable.
//
// good/ → .nvmrc = active Node major (should match → exit 0)
// bad-nvmrc/ → .nvmrc = active+99 (guaranteed mismatch → exit 1)
// -------------------------------------------------------------------------
const activeNodeMajor = parseInt(process.version.match(/^v(\d+)/)[1], 10);
const goodNvmrc = path.join(FIXTURE_ROOT, 'good', '.nvmrc');
const badNvmrc = path.join(FIXTURE_ROOT, 'bad-nvmrc', '.nvmrc');
let originalGoodNvmrc;
let originalBadNvmrc;
before(() => {
originalGoodNvmrc = fs.existsSync(goodNvmrc) ? fs.readFileSync(goodNvmrc, 'utf8') : null;
originalBadNvmrc = fs.existsSync(badNvmrc) ? fs.readFileSync(badNvmrc, 'utf8') : null;
fs.writeFileSync(goodNvmrc, `${activeNodeMajor}\n`);
fs.writeFileSync(badNvmrc, `${activeNodeMajor + 99}\n`);
});
after(() => {
if (originalGoodNvmrc !== null) {
fs.writeFileSync(goodNvmrc, originalGoodNvmrc);
}
if (originalBadNvmrc !== null) {
fs.writeFileSync(badNvmrc, originalBadNvmrc);
}
});
// -------------------------------------------------------------------------
// Test 1: Happy path — all checks green
// -------------------------------------------------------------------------
test('exits 0 in a fixture directory with engines, .nvmrc, and matching lockfile', () => {
const cwd = path.join(FIXTURE_ROOT, 'good');
const { status, stdout } = runScript(cwd);
assert.equal(
status, 0,
`Expected exit 0, got ${status}.\nstdout: ${stdout}`
);
});
// -------------------------------------------------------------------------
// Test 2: engines.node constraint not satisfied
// -------------------------------------------------------------------------
test('exits 1 when engines.node constraint is not satisfied by current Node', () => {
const cwd = path.join(FIXTURE_ROOT, 'bad-node-version');
// Fixture has engines.node: "<14.0.0"; current Node is much higher.
const { status, stdout } = runScript(cwd);
assert.equal(
status, 1,
`Expected exit 1 (bad node version), got ${status}.\nstdout: ${stdout}`
);
});
// -------------------------------------------------------------------------
// Test 3: Missing lockfile
// -------------------------------------------------------------------------
test('exits 1 when package-lock.json is missing', () => {
const cwd = path.join(FIXTURE_ROOT, 'missing-lockfile');
const { status, stdout } = runScript(cwd);
assert.equal(
status, 1,
`Expected exit 1 (missing lockfile), got ${status}.\nstdout: ${stdout}`
);
});
// -------------------------------------------------------------------------
// Test 4: .nvmrc major doesn't match active Node major
// -------------------------------------------------------------------------
test('exits 1 when .nvmrc major version does not match active Node major', () => {
const cwd = path.join(FIXTURE_ROOT, 'bad-nvmrc');
// Fixture .nvmrc is set to (activeNodeMajor + 99) by the before() hook above,
// guaranteeing a mismatch regardless of the CI matrix Node version.
// Override CI='' so the version-manager-pin check is not skipped even when
// this test runs inside a CI runner (GitHub Actions sets CI=true, which
// would otherwise turn the pin check into a skip and exit 0).
const { status, stdout } = runScript(cwd, [], { CI: '' });
assert.equal(
status, 1,
`Expected exit 1 (nvmrc mismatch), got ${status}.\nstdout: ${stdout}`
);
});
// -------------------------------------------------------------------------
// Test 5: --json flag produces parseable JSON with documented shape
// -------------------------------------------------------------------------
test('--json emits parseable JSON with pass and checks keys', () => {
const cwd = path.join(FIXTURE_ROOT, 'good');
const { status, stdout } = runScript(cwd, ['--json']);
let parsed;
try {
parsed = JSON.parse(stdout);
} catch (err) {
assert.fail(`--json output was not valid JSON: ${err.message}\nstdout: ${stdout}`);
}
// Top-level shape
assert.equal(typeof parsed.pass, 'boolean', 'JSON must have boolean `pass` key');
assert.ok(Array.isArray(parsed.checks), 'JSON must have array `checks` key');
// The good fixture has engines.node, .nvmrc, and package-lock.json — expect
// at least the node-version, lockfile-present, lockfile-sync, and
// version-manager-pin checks to appear.
const checkNames = parsed.checks.map((c) => c.name);
assert.ok(
checkNames.includes('node-version'),
`Expected 'node-version' check in JSON, got: ${checkNames.join(', ')}`
);
assert.ok(
checkNames.includes('lockfile-present'),
`Expected 'lockfile-present' check in JSON, got: ${checkNames.join(', ')}`
);
// Every check item must have name, status, message fields with expected types
for (const check of parsed.checks) {
assert.equal(typeof check.name, 'string', `check.name must be string in ${JSON.stringify(check)}`);
assert.ok(
['pass', 'fail', 'skip'].includes(check.status),
`check.status must be pass|fail|skip in ${JSON.stringify(check)}`
);
assert.equal(typeof check.message, 'string', `check.message must be string in ${JSON.stringify(check)}`);
}
// Good fixture: overall result must be pass:true
assert.equal(parsed.pass, true, 'good fixture must report pass:true');
assert.equal(
status, 0,
`Expected exit 0 in good fixture with --json, got ${status}`
);
});
// -------------------------------------------------------------------------
// Test 5b: --json reports pass:false on failure fixtures (counter-test for 5)
// -------------------------------------------------------------------------
test('--json reports pass:false when a check fails', () => {
const cwd = path.join(FIXTURE_ROOT, 'bad-node-version');
const { status, stdout } = runScript(cwd, ['--json']);
let parsed;
try {
parsed = JSON.parse(stdout);
} catch (err) {
assert.fail(`--json output was not valid JSON: ${err.message}\nstdout: ${stdout}`);
}
assert.equal(parsed.pass, false, 'failure fixture must report pass:false');
assert.equal(status, 1, `Expected exit 1 with --json on failure fixture, got ${status}`);
// The node-version check must be present and marked fail
const nodeCheck = parsed.checks.find((c) => c.name === 'node-version');
assert.ok(nodeCheck, 'node-version check must appear in JSON output');
assert.equal(nodeCheck.status, 'fail', `Expected node-version status=fail, got ${nodeCheck.status}`);
});
// -------------------------------------------------------------------------
// Test 6: Integration smoke — script runs without tool-error on live root
//
// Verifies the script executes against a real repo without a tool error (exit 2).
// Exit 0 or 1 are acceptable — local Node may differ from the .nvmrc pin (22).
// Uses --json for structured assertion, avoiding raw output-grep.
// -------------------------------------------------------------------------
test('script runs without tool error on the live worktree root (--json)', () => {
const { status, stdout, stderr } = runScript(LIVE_ROOT, ['--json']);
assert.notEqual(
status, 2,
`Expected exit 0 or 1 on live repo, got exit 2 (tool error).\nstdout: ${stdout}\nstderr: ${stderr}`
);
let parsed;
try {
parsed = JSON.parse(stdout);
} catch (err) {
assert.fail(`Live repo --json was not valid JSON: ${err.message}\nstdout: ${stdout}`);
}
assert.equal(typeof parsed.pass, 'boolean', 'Live repo JSON must have boolean pass');
assert.ok(Array.isArray(parsed.checks), 'Live repo JSON must have checks array');
// Node version check must be present and pass (Node >=22 is installed)
const nodeCheck = parsed.checks.find((c) => c.name === 'node-version');
assert.ok(nodeCheck, 'node-version check must be present in live repo output');
assert.equal(nodeCheck.status, 'pass', `node-version should pass on live repo, got: ${nodeCheck.status} — ${nodeCheck.message}`);
// Lockfile checks must pass on the live repo
const lockfileCheck = parsed.checks.find((c) => c.name === 'lockfile-present');
assert.ok(lockfileCheck, 'lockfile-present check must appear in live output');
assert.equal(lockfileCheck.status, 'pass', `lockfile-present should pass on live repo`);
});
});