* test(#431): policy-shell-pinning linter — RED baseline (37 violations on origin/next) Adds scripts/workflow-policy.cjs: H1 shell-policy linter with POLICY map, VIOLATION enum, matrix expansion, effective-shell resolution order, and runPolicyLint({ workflowsDir }) entry point. Adds tests/policy-shell-pinning.test.cjs: 8 tests (baseline + 6 synthetic counter-tests). Synthetic tests 2–7 pass; baseline test is intentionally RED (37 violations: 28 in test.yml, 9 in install-smoke.yml — all macos/windows lanes using shell: bash instead of native zsh/pwsh). Adds js-yaml@4.1.1 as devDependency for YAML parsing. * fix(#431): switch ubuntu/windows lanes to native shells; extract bash-isms to Node Remove all explicit shell: bash pins from ubuntu-only jobs (changes, lint-tests, coverage, required-tests, smoke-unpacked) — ubuntu runner default is bash, which is both H1-compliant and the runner default, making the pin redundant. For the test and test-full mixed-OS jobs (ubuntu+windows, windows+macos): - Move bash-ism steps to shell-agnostic Node scripts: scripts/ci-guard-runner.cjs — RUNNER_ENVIRONMENT check scripts/ci-rebase-check.cjs — git fetch+merge PR base branch scripts/check-npm-integrity.cjs — Node port of check-npm-integrity.sh scripts/ci-prepare-test-scope.cjs — write .ci-selected-tests.txt scripts/ci-smoke-skip.cjs — set skip= output for full-only matrix entries - Remove shell: bash from simple npm/node command steps (runner default applies) This brings Windows violations from 19 to 0. Remaining 17 violations are all MACOS_MISSING_EXPLICIT_ZSH in mixed-OS matrix jobs (test-full: windows+macos, install-smoke smoke: ubuntu+macos) — these require job splitting to fix; see BLOCKER in PR description. * fix(#431): update workflow-shell-pinning test for H1 policy The old test required all Windows-targeting npm steps to pin shell: bash (to prevent pwsh stderr-swallow). Under H1, Windows runners must use pwsh (native, no pin needed) — shell: bash on Windows is now the violation, not the fix. Update findViolations() to flag npm steps with effectiveShell === 'bash' (rather than effectiveShell === null). Update synthetic tests to verify the H1-inverted semantics: defaults.run.shell: bash on Windows is now 2 violations, not 0. Update test name and assertion messages to describe the H1 constraint rather than the old missing-pin constraint. * fix(#431): extend policy linter to resolve matrix.shell expressions - expandRunsOn now captures all matrix.include row keys as realization context (os, node-version, shell, full_only, etc.) instead of only os - effectiveShell now accepts a realizationContext and resolves ${{ matrix.<key> }} expressions against it before checking policy - Unresolvable matrix key in shell expression emits UNRESOLVABLE_MATRIX - Add 3 new tests: positive (zsh+pwsh per row → 0 violations), counter (bash in macOS row → WRONG_SHELL_FOR_OS), counter (missing shell key → UNRESOLVABLE_MATRIX) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): apply matrix.shell pattern to test-full and smoke jobs (clears BLOCKER) test-full job (test.yml): - Add shell: pwsh/zsh per matrix.include row (windows-latest→pwsh, macos-latest→zsh) - Add job-level defaults.run.shell: ${{ matrix.shell }} - No step-level shell pins existed to remove smoke job (install-smoke.yml): - Add shell: bash/zsh per matrix.include row (ubuntu→bash, macos→zsh) - Add job-level defaults.run.shell: ${{ matrix.shell }} - No step-level shell pins existed to remove Policy linter now reports 0 violations across all workflow files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#431): migrate .sh check scripts to .cjs; remove .sh originals - Add scripts/check-env.cjs: Node.js port of check-env.sh with identical exit codes (0/1/2), human-readable and --json output, --help flag, and all 5 checks (node-version, npm-version, lockfile-present, lockfile-sync, version-manager-pin) - Migrate all callers: - package.json check:env → node scripts/check-env.cjs - package.json check:integrity → node scripts/check-npm-integrity.cjs - scripts/ci-test-scope.cjs path strings → .cjs equivalents - .github/workflows/release.yml rc+finalize jobs → node .cjs (drop chmod+x) - .github/workflows/security-scan.yml → node .cjs (drop chmod+x) - tests/check-env.test.cjs → spawn node process.execPath [.cjs] - tests/npm-integrity-gate.test.cjs → spawn node process.execPath [.cjs] - Delete scripts/check-env.sh and scripts/check-npm-integrity.sh Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#431): update doc references from .sh to .cjs Update SECURITY.md and docs/contributing/bootstrap.md to reference the canonical Node invocation instead of the removed bash scripts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): use per-step shell:matrix.shell instead of defaults.run.shell (GHA compat) GHA does not reliably resolve matrix expressions inside defaults.run.shell. Per-step shell: always resolves correctly. Removed the defaults.run.shell block from the test-full job (test.yml) and the smoke job (install-smoke.yml), and added shell: \${{ matrix.shell }} directly on every run: step in both jobs. Codex finding: defaults.run.shell with matrix expressions is not a GHA-supported pattern; per-step shell: is the safe form. * fix(#431): policy linter validates every matrix.include row independently Removed runner-label-only dedup from expandRunsOn() in workflow-policy.cjs. The prior guard (if !realizations.find(r => r.runner === runner)) collapsed two macos-latest rows with different node-version/shell contexts into one, hiding the second row's policy violation. Each matrix.include row is a distinct CI realization with its own context; validating it twice is harmless but skipping it causes false negatives. Added counter-test (Test 8) in tests/policy-shell-pinning.test.cjs: two macos-latest rows (shell:zsh compliant + shell:bash violation) must produce exactly one WRONG_SHELL_FOR_OS violation on the second row. * fix(#431): remove dedup-by-runner in Cartesian matrix.<key> expansion (Codex round 3) The base-list path in expandRunsOn (matrix.<key> arrays, e.g. matrix.os) previously guarded each push with `if (!realizations.find(r => r.runner === runner))`, collapsing duplicate runner values into a single realization and hiding policy violations on later rows of a Cartesian matrix. Remove the guard unconditionally; each entry in the base-list array now produces its own realization, matching the same fix already applied to the matrix.include path. Add counter-test "Cartesian matrix os × shell — dedup must not collapse rows by runner alone": matrix.os: [macos-latest, macos-latest] + shell: ${{ matrix.shell }} now yields 2 realizations (not 1). Documents that Cartesian cross-product expansion (carrying all keys into realization context) is a separate follow-up; current violations are UNRESOLVABLE_MATRIX pending that work. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): remove 60s timeout regression on npm ci --dry-run (parity with check-env.sh) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): ci-rebase-check.cjs — return truthy sentinel on success (Codex round 4) run() used execFileSync with stdio:'inherit', which returns null on success. Caller checked `result !== null`, always false → every successful fetch fell through to "failed after 3 attempts" exit-1 path. Fix: run() now returns true on success, false on failure. Update caller from `result !== null` to `if (result)`. Adds tests/ci-rebase-check.test.cjs (5 tests) covering the sentinel contract and a local-bare-remote integration smoke that verifies the full fetch+merge path exits 0 when fetch succeeds. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: CI Rebase Check <ci@gsd-redux>
235 lines
11 KiB
JavaScript
235 lines
11 KiB
JavaScript
/**
|
|
* Tests for scripts/check-env.cjs (issue #117).
|
|
*
|
|
* Verifies the environment validator exits correctly and emits
|
|
* structured output for every documented check:
|
|
* 1. Node version vs engines.node constraint
|
|
* 2. npm version vs engines.npm constraint (if present)
|
|
* 3. Lockfile presence
|
|
* 4. Lockfile sync (npm ci --dry-run)
|
|
* 5. Version-manager pin file matches active Node major
|
|
* 6. --json flag produces parseable JSON with documented shape
|
|
* 7. Integration smoke: exits 0 on the live worktree root
|
|
*
|
|
* Sources:
|
|
* npm engines: https://docs.npmjs.com/cli/v10/configuring-npm/package-json#engines
|
|
* npm ci: https://docs.npmjs.com/cli/v10/commands/npm-ci
|
|
*/
|
|
|
|
'use strict';
|
|
|
|
const { describe, test, before, after } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const path = require('node:path');
|
|
const fs = require('node:fs');
|
|
const { spawnSync } = require('node:child_process');
|
|
|
|
const SCRIPT = path.resolve(__dirname, '..', 'scripts', 'check-env.cjs');
|
|
const FIXTURE_ROOT = path.resolve(__dirname, 'fixtures', 'check-env');
|
|
const LIVE_ROOT = path.resolve(__dirname, '..');
|
|
|
|
/**
|
|
* Run check-env.cjs synchronously in `cwd` with optional extra args.
|
|
* Returns { status, stdout, stderr }.
|
|
* @param {string} cwd
|
|
* @param {string[]} args
|
|
* @param {Record<string,string>} [envOverrides] - optional env vars to overlay
|
|
* on process.env. Pass { CI: '' } to suppress GitHub Actions CI detection
|
|
* so that version-manager-pin is exercised even inside CI runners.
|
|
*/
|
|
function runScript(cwd, args = [], envOverrides = {}) {
|
|
const result = spawnSync(process.execPath, [SCRIPT, ...args], {
|
|
cwd,
|
|
encoding: 'utf8',
|
|
timeout: 30_000,
|
|
env: { ...process.env, ...envOverrides },
|
|
});
|
|
return {
|
|
status: result.status ?? 1,
|
|
stdout: result.stdout ?? '',
|
|
stderr: result.stderr ?? '',
|
|
};
|
|
}
|
|
|
|
describe('check-env.cjs', () => {
|
|
// -------------------------------------------------------------------------
|
|
// Dynamic .nvmrc setup: write fixture .nvmrc files at test-run time so the
|
|
// tests are correct across all Node major versions in the CI matrix (Node 22,
|
|
// 24, 26, …). A hardcoded value like "26" passes on Node 26 but fails on
|
|
// every other matrix row; using the active major makes the fixture portable.
|
|
//
|
|
// good/ → .nvmrc = active Node major (should match → exit 0)
|
|
// bad-nvmrc/ → .nvmrc = active+99 (guaranteed mismatch → exit 1)
|
|
// -------------------------------------------------------------------------
|
|
const activeNodeMajor = parseInt(process.version.match(/^v(\d+)/)[1], 10);
|
|
const goodNvmrc = path.join(FIXTURE_ROOT, 'good', '.nvmrc');
|
|
const badNvmrc = path.join(FIXTURE_ROOT, 'bad-nvmrc', '.nvmrc');
|
|
let originalGoodNvmrc;
|
|
let originalBadNvmrc;
|
|
|
|
before(() => {
|
|
originalGoodNvmrc = fs.existsSync(goodNvmrc) ? fs.readFileSync(goodNvmrc, 'utf8') : null;
|
|
originalBadNvmrc = fs.existsSync(badNvmrc) ? fs.readFileSync(badNvmrc, 'utf8') : null;
|
|
fs.writeFileSync(goodNvmrc, `${activeNodeMajor}\n`);
|
|
fs.writeFileSync(badNvmrc, `${activeNodeMajor + 99}\n`);
|
|
});
|
|
|
|
after(() => {
|
|
if (originalGoodNvmrc !== null) {
|
|
fs.writeFileSync(goodNvmrc, originalGoodNvmrc);
|
|
}
|
|
if (originalBadNvmrc !== null) {
|
|
fs.writeFileSync(badNvmrc, originalBadNvmrc);
|
|
}
|
|
});
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Test 1: Happy path — all checks green
|
|
// -------------------------------------------------------------------------
|
|
test('exits 0 in a fixture directory with engines, .nvmrc, and matching lockfile', () => {
|
|
const cwd = path.join(FIXTURE_ROOT, 'good');
|
|
const { status, stdout } = runScript(cwd);
|
|
assert.equal(
|
|
status, 0,
|
|
`Expected exit 0, got ${status}.\nstdout: ${stdout}`
|
|
);
|
|
});
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Test 2: engines.node constraint not satisfied
|
|
// -------------------------------------------------------------------------
|
|
test('exits 1 when engines.node constraint is not satisfied by current Node', () => {
|
|
const cwd = path.join(FIXTURE_ROOT, 'bad-node-version');
|
|
// Fixture has engines.node: "<14.0.0"; current Node is much higher.
|
|
const { status, stdout } = runScript(cwd);
|
|
assert.equal(
|
|
status, 1,
|
|
`Expected exit 1 (bad node version), got ${status}.\nstdout: ${stdout}`
|
|
);
|
|
});
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Test 3: Missing lockfile
|
|
// -------------------------------------------------------------------------
|
|
test('exits 1 when package-lock.json is missing', () => {
|
|
const cwd = path.join(FIXTURE_ROOT, 'missing-lockfile');
|
|
const { status, stdout } = runScript(cwd);
|
|
assert.equal(
|
|
status, 1,
|
|
`Expected exit 1 (missing lockfile), got ${status}.\nstdout: ${stdout}`
|
|
);
|
|
});
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Test 4: .nvmrc major doesn't match active Node major
|
|
// -------------------------------------------------------------------------
|
|
test('exits 1 when .nvmrc major version does not match active Node major', () => {
|
|
const cwd = path.join(FIXTURE_ROOT, 'bad-nvmrc');
|
|
// Fixture .nvmrc is set to (activeNodeMajor + 99) by the before() hook above,
|
|
// guaranteeing a mismatch regardless of the CI matrix Node version.
|
|
// Override CI='' so the version-manager-pin check is not skipped even when
|
|
// this test runs inside a CI runner (GitHub Actions sets CI=true, which
|
|
// would otherwise turn the pin check into a skip and exit 0).
|
|
const { status, stdout } = runScript(cwd, [], { CI: '' });
|
|
assert.equal(
|
|
status, 1,
|
|
`Expected exit 1 (nvmrc mismatch), got ${status}.\nstdout: ${stdout}`
|
|
);
|
|
});
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Test 5: --json flag produces parseable JSON with documented shape
|
|
// -------------------------------------------------------------------------
|
|
test('--json emits parseable JSON with pass and checks keys', () => {
|
|
const cwd = path.join(FIXTURE_ROOT, 'good');
|
|
const { status, stdout } = runScript(cwd, ['--json']);
|
|
let parsed;
|
|
try {
|
|
parsed = JSON.parse(stdout);
|
|
} catch (err) {
|
|
assert.fail(`--json output was not valid JSON: ${err.message}\nstdout: ${stdout}`);
|
|
}
|
|
// Top-level shape
|
|
assert.equal(typeof parsed.pass, 'boolean', 'JSON must have boolean `pass` key');
|
|
assert.ok(Array.isArray(parsed.checks), 'JSON must have array `checks` key');
|
|
// The good fixture has engines.node, .nvmrc, and package-lock.json — expect
|
|
// at least the node-version, lockfile-present, lockfile-sync, and
|
|
// version-manager-pin checks to appear.
|
|
const checkNames = parsed.checks.map((c) => c.name);
|
|
assert.ok(
|
|
checkNames.includes('node-version'),
|
|
`Expected 'node-version' check in JSON, got: ${checkNames.join(', ')}`
|
|
);
|
|
assert.ok(
|
|
checkNames.includes('lockfile-present'),
|
|
`Expected 'lockfile-present' check in JSON, got: ${checkNames.join(', ')}`
|
|
);
|
|
// Every check item must have name, status, message fields with expected types
|
|
for (const check of parsed.checks) {
|
|
assert.equal(typeof check.name, 'string', `check.name must be string in ${JSON.stringify(check)}`);
|
|
assert.ok(
|
|
['pass', 'fail', 'skip'].includes(check.status),
|
|
`check.status must be pass|fail|skip in ${JSON.stringify(check)}`
|
|
);
|
|
assert.equal(typeof check.message, 'string', `check.message must be string in ${JSON.stringify(check)}`);
|
|
}
|
|
// Good fixture: overall result must be pass:true
|
|
assert.equal(parsed.pass, true, 'good fixture must report pass:true');
|
|
assert.equal(
|
|
status, 0,
|
|
`Expected exit 0 in good fixture with --json, got ${status}`
|
|
);
|
|
});
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Test 5b: --json reports pass:false on failure fixtures (counter-test for 5)
|
|
// -------------------------------------------------------------------------
|
|
test('--json reports pass:false when a check fails', () => {
|
|
const cwd = path.join(FIXTURE_ROOT, 'bad-node-version');
|
|
const { status, stdout } = runScript(cwd, ['--json']);
|
|
let parsed;
|
|
try {
|
|
parsed = JSON.parse(stdout);
|
|
} catch (err) {
|
|
assert.fail(`--json output was not valid JSON: ${err.message}\nstdout: ${stdout}`);
|
|
}
|
|
assert.equal(parsed.pass, false, 'failure fixture must report pass:false');
|
|
assert.equal(status, 1, `Expected exit 1 with --json on failure fixture, got ${status}`);
|
|
// The node-version check must be present and marked fail
|
|
const nodeCheck = parsed.checks.find((c) => c.name === 'node-version');
|
|
assert.ok(nodeCheck, 'node-version check must appear in JSON output');
|
|
assert.equal(nodeCheck.status, 'fail', `Expected node-version status=fail, got ${nodeCheck.status}`);
|
|
});
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Test 6: Integration smoke — script runs without tool-error on live root
|
|
//
|
|
// Verifies the script executes against a real repo without a tool error (exit 2).
|
|
// Exit 0 or 1 are acceptable — local Node may differ from the .nvmrc pin (22).
|
|
// Uses --json for structured assertion, avoiding raw output-grep.
|
|
// -------------------------------------------------------------------------
|
|
test('script runs without tool error on the live worktree root (--json)', () => {
|
|
const { status, stdout, stderr } = runScript(LIVE_ROOT, ['--json']);
|
|
assert.notEqual(
|
|
status, 2,
|
|
`Expected exit 0 or 1 on live repo, got exit 2 (tool error).\nstdout: ${stdout}\nstderr: ${stderr}`
|
|
);
|
|
let parsed;
|
|
try {
|
|
parsed = JSON.parse(stdout);
|
|
} catch (err) {
|
|
assert.fail(`Live repo --json was not valid JSON: ${err.message}\nstdout: ${stdout}`);
|
|
}
|
|
assert.equal(typeof parsed.pass, 'boolean', 'Live repo JSON must have boolean pass');
|
|
assert.ok(Array.isArray(parsed.checks), 'Live repo JSON must have checks array');
|
|
// Node version check must be present and pass (Node >=22 is installed)
|
|
const nodeCheck = parsed.checks.find((c) => c.name === 'node-version');
|
|
assert.ok(nodeCheck, 'node-version check must be present in live repo output');
|
|
assert.equal(nodeCheck.status, 'pass', `node-version should pass on live repo, got: ${nodeCheck.status} — ${nodeCheck.message}`);
|
|
// Lockfile checks must pass on the live repo
|
|
const lockfileCheck = parsed.checks.find((c) => c.name === 'lockfile-present');
|
|
assert.ok(lockfileCheck, 'lockfile-present check must appear in live output');
|
|
assert.equal(lockfileCheck.status, 'pass', `lockfile-present should pass on live repo`);
|
|
});
|
|
});
|