* test(#431): policy-shell-pinning linter — RED baseline (37 violations on origin/next) Adds scripts/workflow-policy.cjs: H1 shell-policy linter with POLICY map, VIOLATION enum, matrix expansion, effective-shell resolution order, and runPolicyLint({ workflowsDir }) entry point. Adds tests/policy-shell-pinning.test.cjs: 8 tests (baseline + 6 synthetic counter-tests). Synthetic tests 2–7 pass; baseline test is intentionally RED (37 violations: 28 in test.yml, 9 in install-smoke.yml — all macos/windows lanes using shell: bash instead of native zsh/pwsh). Adds js-yaml@4.1.1 as devDependency for YAML parsing. * fix(#431): switch ubuntu/windows lanes to native shells; extract bash-isms to Node Remove all explicit shell: bash pins from ubuntu-only jobs (changes, lint-tests, coverage, required-tests, smoke-unpacked) — ubuntu runner default is bash, which is both H1-compliant and the runner default, making the pin redundant. For the test and test-full mixed-OS jobs (ubuntu+windows, windows+macos): - Move bash-ism steps to shell-agnostic Node scripts: scripts/ci-guard-runner.cjs — RUNNER_ENVIRONMENT check scripts/ci-rebase-check.cjs — git fetch+merge PR base branch scripts/check-npm-integrity.cjs — Node port of check-npm-integrity.sh scripts/ci-prepare-test-scope.cjs — write .ci-selected-tests.txt scripts/ci-smoke-skip.cjs — set skip= output for full-only matrix entries - Remove shell: bash from simple npm/node command steps (runner default applies) This brings Windows violations from 19 to 0. Remaining 17 violations are all MACOS_MISSING_EXPLICIT_ZSH in mixed-OS matrix jobs (test-full: windows+macos, install-smoke smoke: ubuntu+macos) — these require job splitting to fix; see BLOCKER in PR description. * fix(#431): update workflow-shell-pinning test for H1 policy The old test required all Windows-targeting npm steps to pin shell: bash (to prevent pwsh stderr-swallow). Under H1, Windows runners must use pwsh (native, no pin needed) — shell: bash on Windows is now the violation, not the fix. Update findViolations() to flag npm steps with effectiveShell === 'bash' (rather than effectiveShell === null). Update synthetic tests to verify the H1-inverted semantics: defaults.run.shell: bash on Windows is now 2 violations, not 0. Update test name and assertion messages to describe the H1 constraint rather than the old missing-pin constraint. * fix(#431): extend policy linter to resolve matrix.shell expressions - expandRunsOn now captures all matrix.include row keys as realization context (os, node-version, shell, full_only, etc.) instead of only os - effectiveShell now accepts a realizationContext and resolves ${{ matrix.<key> }} expressions against it before checking policy - Unresolvable matrix key in shell expression emits UNRESOLVABLE_MATRIX - Add 3 new tests: positive (zsh+pwsh per row → 0 violations), counter (bash in macOS row → WRONG_SHELL_FOR_OS), counter (missing shell key → UNRESOLVABLE_MATRIX) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): apply matrix.shell pattern to test-full and smoke jobs (clears BLOCKER) test-full job (test.yml): - Add shell: pwsh/zsh per matrix.include row (windows-latest→pwsh, macos-latest→zsh) - Add job-level defaults.run.shell: ${{ matrix.shell }} - No step-level shell pins existed to remove smoke job (install-smoke.yml): - Add shell: bash/zsh per matrix.include row (ubuntu→bash, macos→zsh) - Add job-level defaults.run.shell: ${{ matrix.shell }} - No step-level shell pins existed to remove Policy linter now reports 0 violations across all workflow files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#431): migrate .sh check scripts to .cjs; remove .sh originals - Add scripts/check-env.cjs: Node.js port of check-env.sh with identical exit codes (0/1/2), human-readable and --json output, --help flag, and all 5 checks (node-version, npm-version, lockfile-present, lockfile-sync, version-manager-pin) - Migrate all callers: - package.json check:env → node scripts/check-env.cjs - package.json check:integrity → node scripts/check-npm-integrity.cjs - scripts/ci-test-scope.cjs path strings → .cjs equivalents - .github/workflows/release.yml rc+finalize jobs → node .cjs (drop chmod+x) - .github/workflows/security-scan.yml → node .cjs (drop chmod+x) - tests/check-env.test.cjs → spawn node process.execPath [.cjs] - tests/npm-integrity-gate.test.cjs → spawn node process.execPath [.cjs] - Delete scripts/check-env.sh and scripts/check-npm-integrity.sh Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#431): update doc references from .sh to .cjs Update SECURITY.md and docs/contributing/bootstrap.md to reference the canonical Node invocation instead of the removed bash scripts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): use per-step shell:matrix.shell instead of defaults.run.shell (GHA compat) GHA does not reliably resolve matrix expressions inside defaults.run.shell. Per-step shell: always resolves correctly. Removed the defaults.run.shell block from the test-full job (test.yml) and the smoke job (install-smoke.yml), and added shell: \${{ matrix.shell }} directly on every run: step in both jobs. Codex finding: defaults.run.shell with matrix expressions is not a GHA-supported pattern; per-step shell: is the safe form. * fix(#431): policy linter validates every matrix.include row independently Removed runner-label-only dedup from expandRunsOn() in workflow-policy.cjs. The prior guard (if !realizations.find(r => r.runner === runner)) collapsed two macos-latest rows with different node-version/shell contexts into one, hiding the second row's policy violation. Each matrix.include row is a distinct CI realization with its own context; validating it twice is harmless but skipping it causes false negatives. Added counter-test (Test 8) in tests/policy-shell-pinning.test.cjs: two macos-latest rows (shell:zsh compliant + shell:bash violation) must produce exactly one WRONG_SHELL_FOR_OS violation on the second row. * fix(#431): remove dedup-by-runner in Cartesian matrix.<key> expansion (Codex round 3) The base-list path in expandRunsOn (matrix.<key> arrays, e.g. matrix.os) previously guarded each push with `if (!realizations.find(r => r.runner === runner))`, collapsing duplicate runner values into a single realization and hiding policy violations on later rows of a Cartesian matrix. Remove the guard unconditionally; each entry in the base-list array now produces its own realization, matching the same fix already applied to the matrix.include path. Add counter-test "Cartesian matrix os × shell — dedup must not collapse rows by runner alone": matrix.os: [macos-latest, macos-latest] + shell: ${{ matrix.shell }} now yields 2 realizations (not 1). Documents that Cartesian cross-product expansion (carrying all keys into realization context) is a separate follow-up; current violations are UNRESOLVABLE_MATRIX pending that work. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): remove 60s timeout regression on npm ci --dry-run (parity with check-env.sh) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): ci-rebase-check.cjs — return truthy sentinel on success (Codex round 4) run() used execFileSync with stdio:'inherit', which returns null on success. Caller checked `result !== null`, always false → every successful fetch fell through to "failed after 3 attempts" exit-1 path. Fix: run() now returns true on success, false on failure. Update caller from `result !== null` to `if (result)`. Adds tests/ci-rebase-check.test.cjs (5 tests) covering the sentinel contract and a local-bare-remote integration smoke that verifies the full fetch+merge path exits 0 when fetch succeeds. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: CI Rebase Check <ci@gsd-redux>
170 lines
6.5 KiB
JavaScript
170 lines
6.5 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* Regression test for #114 — npm dependency integrity gate.
|
|
*
|
|
* Verifies that scripts/check-npm-integrity.cjs correctly detects:
|
|
* 1. Clean install — exits 0, no stderr findings
|
|
* 2. Version drift — exits 1, stderr names the offending package + both versions
|
|
* (reproduces the ws 8.20.1 declared vs 8.20.0 installed incident)
|
|
* 3. Extraneous — exits 1 without --ignore-extraneous; exits 0 with it
|
|
* 4. Missing — exits 1 regardless of flags
|
|
*
|
|
* Each fixture lives under tests/fixtures/npm-integrity/<name>/.
|
|
* The test spawns the script as a subprocess — no require/import of internals.
|
|
*
|
|
* Sources:
|
|
* - npm CLI docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
|
|
* - NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
|
|
*/
|
|
|
|
const { describe, test } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const { spawnSync } = require('node:child_process');
|
|
const path = require('node:path');
|
|
|
|
const ROOT = path.resolve(__dirname, '..');
|
|
const SCRIPT = path.join(ROOT, 'scripts', 'check-npm-integrity.cjs');
|
|
const FIXTURES = path.join(__dirname, 'fixtures', 'npm-integrity');
|
|
|
|
/**
|
|
* Run the integrity gate script against a fixture directory.
|
|
*
|
|
* @param {string} fixtureName - subdirectory under tests/fixtures/npm-integrity/
|
|
* @param {string[]} [extraArgs] - additional CLI args passed to the script
|
|
* @returns {{ status: number, stdout: string, stderr: string }}
|
|
*/
|
|
function runGate(fixtureName, extraArgs = []) {
|
|
const fixtureDir = path.join(FIXTURES, fixtureName);
|
|
const result = spawnSync(process.execPath, [SCRIPT, ...extraArgs], {
|
|
cwd: fixtureDir,
|
|
encoding: 'utf-8',
|
|
timeout: 30_000,
|
|
});
|
|
return {
|
|
status: result.status ?? 1,
|
|
stdout: result.stdout ?? '',
|
|
stderr: result.stderr ?? '',
|
|
};
|
|
}
|
|
|
|
// ─── Scenario 1: Clean ───────────────────────────────────────────────────────
|
|
|
|
describe('#114: npm integrity gate — clean fixture', () => {
|
|
test('exits 0 when install matches lockfile', () => {
|
|
const { status } = runGate('clean');
|
|
assert.strictEqual(status, 0, 'expected exit 0 for clean install');
|
|
});
|
|
|
|
test('emits no integrity findings to stderr on clean install', () => {
|
|
const { stderr } = runGate('clean');
|
|
// No "FAIL:" lines expected
|
|
assert.ok(
|
|
!stderr.includes('FAIL:'),
|
|
`expected no FAIL: lines in stderr; got:\n${stderr}`
|
|
);
|
|
});
|
|
});
|
|
|
|
// ─── Scenario 2: Drift (declared vs installed mismatch) ─────────────────────
|
|
// Reproduces: ws 8.20.1 declared in lockfile, 8.20.0 installed in node_modules
|
|
// Fixture uses: stable-dep@8.20.1 (declared) vs stable-dep@8.20.0 (installed)
|
|
|
|
describe('#114: npm integrity gate — drift fixture (declared vs installed mismatch)', () => {
|
|
test('exits 1 on version drift', () => {
|
|
const { status } = runGate('drift');
|
|
assert.strictEqual(status, 1, 'expected exit 1 for version drift');
|
|
});
|
|
|
|
test('stderr names the offending package', () => {
|
|
const { stderr } = runGate('drift');
|
|
assert.ok(
|
|
stderr.includes('stable-dep'),
|
|
`expected stderr to name "stable-dep"; got:\n${stderr}`
|
|
);
|
|
});
|
|
|
|
test('stderr includes both the declared and installed versions', () => {
|
|
const { stderr } = runGate('drift');
|
|
assert.ok(
|
|
stderr.includes('8.20.0'),
|
|
`expected stderr to include installed version "8.20.0"; got:\n${stderr}`
|
|
);
|
|
assert.ok(
|
|
stderr.includes('8.20.1'),
|
|
`expected stderr to include declared version "8.20.1"; got:\n${stderr}`
|
|
);
|
|
});
|
|
});
|
|
|
|
// ─── Scenario 3: Extraneous ──────────────────────────────────────────────────
|
|
|
|
describe('#114: npm integrity gate — extraneous fixture', () => {
|
|
test('exits 1 when extraneous package present (default behavior)', () => {
|
|
const { status } = runGate('extraneous');
|
|
assert.strictEqual(status, 1, 'expected exit 1 for extraneous package without --ignore-extraneous');
|
|
});
|
|
|
|
test('stderr names the extraneous package', () => {
|
|
const { stderr } = runGate('extraneous');
|
|
assert.ok(
|
|
stderr.includes('ghost-pkg'),
|
|
`expected stderr to name "ghost-pkg"; got:\n${stderr}`
|
|
);
|
|
});
|
|
|
|
test('exits 0 with --ignore-extraneous flag', () => {
|
|
const { status } = runGate('extraneous', ['--ignore-extraneous']);
|
|
assert.strictEqual(status, 0, 'expected exit 0 for extraneous package with --ignore-extraneous');
|
|
});
|
|
});
|
|
|
|
// ─── Scenario 4: Missing ─────────────────────────────────────────────────────
|
|
|
|
describe('#114: npm integrity gate — missing fixture', () => {
|
|
test('exits 1 when required package is missing from node_modules', () => {
|
|
const { status } = runGate('missing');
|
|
assert.strictEqual(status, 1, 'expected exit 1 for missing package');
|
|
});
|
|
|
|
test('stderr names the missing package', () => {
|
|
const { stderr } = runGate('missing');
|
|
assert.ok(
|
|
stderr.includes('absent-dep'),
|
|
`expected stderr to name "absent-dep"; got:\n${stderr}`
|
|
);
|
|
});
|
|
|
|
test('exits 1 even with --ignore-extraneous (missing is not extraneous)', () => {
|
|
const { status } = runGate('missing', ['--ignore-extraneous']);
|
|
assert.strictEqual(status, 1, 'expected exit 1 for missing package even with --ignore-extraneous');
|
|
});
|
|
});
|
|
|
|
// ─── Smoke test: --help ───────────────────────────────────────────────────────
|
|
|
|
describe('#114: npm integrity gate — --help output', () => {
|
|
test('exits 0 with --help flag', () => {
|
|
const result = spawnSync(process.execPath, [SCRIPT, '--help'], {
|
|
cwd: ROOT,
|
|
encoding: 'utf-8',
|
|
timeout: 10_000,
|
|
});
|
|
assert.strictEqual(result.status, 0, '--help should exit 0');
|
|
});
|
|
|
|
test('--help output mentions --ignore-extraneous', () => {
|
|
const result = spawnSync(process.execPath, [SCRIPT, '--help'], {
|
|
cwd: ROOT,
|
|
encoding: 'utf-8',
|
|
timeout: 10_000,
|
|
});
|
|
// The .cjs script writes --help to stdout.
|
|
const helpText = (result.stdout ?? '') + (result.stderr ?? '');
|
|
assert.ok(
|
|
helpText.includes('--ignore-extraneous'),
|
|
`expected --help output to document --ignore-extraneous; got:\n${helpText}`
|
|
);
|
|
});
|
|
});
|