fix(#431): enforce H1 shell policy (linux=bash, macOS=zsh, windows=pwsh) across PR + release gates (#434)
* test(#431): policy-shell-pinning linter — RED baseline (37 violations on origin/next) Adds scripts/workflow-policy.cjs: H1 shell-policy linter with POLICY map, VIOLATION enum, matrix expansion, effective-shell resolution order, and runPolicyLint({ workflowsDir }) entry point. Adds tests/policy-shell-pinning.test.cjs: 8 tests (baseline + 6 synthetic counter-tests). Synthetic tests 2–7 pass; baseline test is intentionally RED (37 violations: 28 in test.yml, 9 in install-smoke.yml — all macos/windows lanes using shell: bash instead of native zsh/pwsh). Adds js-yaml@4.1.1 as devDependency for YAML parsing. * fix(#431): switch ubuntu/windows lanes to native shells; extract bash-isms to Node Remove all explicit shell: bash pins from ubuntu-only jobs (changes, lint-tests, coverage, required-tests, smoke-unpacked) — ubuntu runner default is bash, which is both H1-compliant and the runner default, making the pin redundant. For the test and test-full mixed-OS jobs (ubuntu+windows, windows+macos): - Move bash-ism steps to shell-agnostic Node scripts: scripts/ci-guard-runner.cjs — RUNNER_ENVIRONMENT check scripts/ci-rebase-check.cjs — git fetch+merge PR base branch scripts/check-npm-integrity.cjs — Node port of check-npm-integrity.sh scripts/ci-prepare-test-scope.cjs — write .ci-selected-tests.txt scripts/ci-smoke-skip.cjs — set skip= output for full-only matrix entries - Remove shell: bash from simple npm/node command steps (runner default applies) This brings Windows violations from 19 to 0. Remaining 17 violations are all MACOS_MISSING_EXPLICIT_ZSH in mixed-OS matrix jobs (test-full: windows+macos, install-smoke smoke: ubuntu+macos) — these require job splitting to fix; see BLOCKER in PR description. * fix(#431): update workflow-shell-pinning test for H1 policy The old test required all Windows-targeting npm steps to pin shell: bash (to prevent pwsh stderr-swallow). Under H1, Windows runners must use pwsh (native, no pin needed) — shell: bash on Windows is now the violation, not the fix. Update findViolations() to flag npm steps with effectiveShell === 'bash' (rather than effectiveShell === null). Update synthetic tests to verify the H1-inverted semantics: defaults.run.shell: bash on Windows is now 2 violations, not 0. Update test name and assertion messages to describe the H1 constraint rather than the old missing-pin constraint. * fix(#431): extend policy linter to resolve matrix.shell expressions - expandRunsOn now captures all matrix.include row keys as realization context (os, node-version, shell, full_only, etc.) instead of only os - effectiveShell now accepts a realizationContext and resolves ${{ matrix.<key> }} expressions against it before checking policy - Unresolvable matrix key in shell expression emits UNRESOLVABLE_MATRIX - Add 3 new tests: positive (zsh+pwsh per row → 0 violations), counter (bash in macOS row → WRONG_SHELL_FOR_OS), counter (missing shell key → UNRESOLVABLE_MATRIX) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): apply matrix.shell pattern to test-full and smoke jobs (clears BLOCKER) test-full job (test.yml): - Add shell: pwsh/zsh per matrix.include row (windows-latest→pwsh, macos-latest→zsh) - Add job-level defaults.run.shell: ${{ matrix.shell }} - No step-level shell pins existed to remove smoke job (install-smoke.yml): - Add shell: bash/zsh per matrix.include row (ubuntu→bash, macos→zsh) - Add job-level defaults.run.shell: ${{ matrix.shell }} - No step-level shell pins existed to remove Policy linter now reports 0 violations across all workflow files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#431): migrate .sh check scripts to .cjs; remove .sh originals - Add scripts/check-env.cjs: Node.js port of check-env.sh with identical exit codes (0/1/2), human-readable and --json output, --help flag, and all 5 checks (node-version, npm-version, lockfile-present, lockfile-sync, version-manager-pin) - Migrate all callers: - package.json check:env → node scripts/check-env.cjs - package.json check:integrity → node scripts/check-npm-integrity.cjs - scripts/ci-test-scope.cjs path strings → .cjs equivalents - .github/workflows/release.yml rc+finalize jobs → node .cjs (drop chmod+x) - .github/workflows/security-scan.yml → node .cjs (drop chmod+x) - tests/check-env.test.cjs → spawn node process.execPath [.cjs] - tests/npm-integrity-gate.test.cjs → spawn node process.execPath [.cjs] - Delete scripts/check-env.sh and scripts/check-npm-integrity.sh Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#431): update doc references from .sh to .cjs Update SECURITY.md and docs/contributing/bootstrap.md to reference the canonical Node invocation instead of the removed bash scripts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): use per-step shell:matrix.shell instead of defaults.run.shell (GHA compat) GHA does not reliably resolve matrix expressions inside defaults.run.shell. Per-step shell: always resolves correctly. Removed the defaults.run.shell block from the test-full job (test.yml) and the smoke job (install-smoke.yml), and added shell: \${{ matrix.shell }} directly on every run: step in both jobs. Codex finding: defaults.run.shell with matrix expressions is not a GHA-supported pattern; per-step shell: is the safe form. * fix(#431): policy linter validates every matrix.include row independently Removed runner-label-only dedup from expandRunsOn() in workflow-policy.cjs. The prior guard (if !realizations.find(r => r.runner === runner)) collapsed two macos-latest rows with different node-version/shell contexts into one, hiding the second row's policy violation. Each matrix.include row is a distinct CI realization with its own context; validating it twice is harmless but skipping it causes false negatives. Added counter-test (Test 8) in tests/policy-shell-pinning.test.cjs: two macos-latest rows (shell:zsh compliant + shell:bash violation) must produce exactly one WRONG_SHELL_FOR_OS violation on the second row. * fix(#431): remove dedup-by-runner in Cartesian matrix.<key> expansion (Codex round 3) The base-list path in expandRunsOn (matrix.<key> arrays, e.g. matrix.os) previously guarded each push with `if (!realizations.find(r => r.runner === runner))`, collapsing duplicate runner values into a single realization and hiding policy violations on later rows of a Cartesian matrix. Remove the guard unconditionally; each entry in the base-list array now produces its own realization, matching the same fix already applied to the matrix.include path. Add counter-test "Cartesian matrix os × shell — dedup must not collapse rows by runner alone": matrix.os: [macos-latest, macos-latest] + shell: ${{ matrix.shell }} now yields 2 realizations (not 1). Documents that Cartesian cross-product expansion (carrying all keys into realization context) is a separate follow-up; current violations are UNRESOLVABLE_MATRIX pending that work. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): remove 60s timeout regression on npm ci --dry-run (parity with check-env.sh) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): ci-rebase-check.cjs — return truthy sentinel on success (Codex round 4) run() used execFileSync with stdio:'inherit', which returns null on success. Caller checked `result !== null`, always false → every successful fetch fell through to "failed after 3 attempts" exit-1 path. Fix: run() now returns true on success, false on failure. Update caller from `result !== null` to `if (result)`. Adds tests/ci-rebase-check.test.cjs (5 tests) covering the sentinel contract and a local-bare-remote integration smoke that verifies the full fetch+merge path exits 0 when fetch succeeds. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: CI Rebase Check <ci@gsd-redux>
This commit is contained in:
Tom Boucher
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Tests for scripts/check-env.sh (issue #117).
|
||||
* Tests for scripts/check-env.cjs (issue #117).
|
||||
*
|
||||
* Verifies the environment validator exits correctly and emits
|
||||
* structured output for every documented check:
|
||||
@@ -24,12 +24,12 @@ const path = require('node:path');
|
||||
const fs = require('node:fs');
|
||||
const { spawnSync } = require('node:child_process');
|
||||
|
||||
const SCRIPT = path.resolve(__dirname, '..', 'scripts', 'check-env.sh');
|
||||
const SCRIPT = path.resolve(__dirname, '..', 'scripts', 'check-env.cjs');
|
||||
const FIXTURE_ROOT = path.resolve(__dirname, 'fixtures', 'check-env');
|
||||
const LIVE_ROOT = path.resolve(__dirname, '..');
|
||||
|
||||
/**
|
||||
* Run check-env.sh synchronously in `cwd` with optional extra args.
|
||||
* Run check-env.cjs synchronously in `cwd` with optional extra args.
|
||||
* Returns { status, stdout, stderr }.
|
||||
* @param {string} cwd
|
||||
* @param {string[]} args
|
||||
@@ -38,7 +38,7 @@ const LIVE_ROOT = path.resolve(__dirname, '..');
|
||||
* so that version-manager-pin is exercised even inside CI runners.
|
||||
*/
|
||||
function runScript(cwd, args = [], envOverrides = {}) {
|
||||
const result = spawnSync('bash', [SCRIPT, ...args], {
|
||||
const result = spawnSync(process.execPath, [SCRIPT, ...args], {
|
||||
cwd,
|
||||
encoding: 'utf8',
|
||||
timeout: 30_000,
|
||||
@@ -51,7 +51,7 @@ function runScript(cwd, args = [], envOverrides = {}) {
|
||||
};
|
||||
}
|
||||
|
||||
describe('check-env.sh', () => {
|
||||
describe('check-env.cjs', () => {
|
||||
// -------------------------------------------------------------------------
|
||||
// Dynamic .nvmrc setup: write fixture .nvmrc files at test-run time so the
|
||||
// tests are correct across all Node major versions in the CI matrix (Node 22,
|
||||
|
||||
174
tests/ci-rebase-check.test.cjs
Normal file
174
tests/ci-rebase-check.test.cjs
Normal file
@@ -0,0 +1,174 @@
|
||||
'use strict';
|
||||
/**
|
||||
* Tests for scripts/ci-rebase-check.cjs — Codex round 4 P1 regression.
|
||||
*
|
||||
* The root bug: run() used execFileSync with stdio:'inherit', which returns null
|
||||
* on success. The caller checked `result !== null` to detect success, so the
|
||||
* condition was ALWAYS false (null !== null === false) and every successful fetch
|
||||
* fell through to the "failed after 3 attempts" exit-1 path.
|
||||
*
|
||||
* Fix: run() now returns true on success, false on failure, making the boolean
|
||||
* check unambiguous regardless of the stdio mode.
|
||||
*
|
||||
* These tests exercise the run() logic in isolation via subprocess execution,
|
||||
* verifying the sentinel behaviour rather than internal module state.
|
||||
*/
|
||||
|
||||
const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const { spawnSync } = require('node:child_process');
|
||||
const path = require('node:path');
|
||||
const fs = require('node:fs');
|
||||
const os = require('node:os');
|
||||
|
||||
const ROOT = path.resolve(__dirname, '..');
|
||||
const SCRIPT = path.join(ROOT, 'scripts', 'ci-rebase-check.cjs');
|
||||
const NODE = process.execPath;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helper: run a small inline Node snippet that requires the run() helper
|
||||
// directly from the source (extracted via a thin wrapper).
|
||||
//
|
||||
// Because the script has top-level side-effects (git config calls), we cannot
|
||||
// require() it. Instead we test the run() sentinel by embedding the function
|
||||
// body verbatim in a one-shot subprocess.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function evalRunHelper(stmts) {
|
||||
// Inline the exact fixed run() body so the test is tightly coupled to the
|
||||
// contract, not some mock.
|
||||
const code = `
|
||||
'use strict';
|
||||
const { execFileSync } = require('child_process');
|
||||
function run(cmd, args, opts) {
|
||||
try {
|
||||
execFileSync(cmd, args, { stdio: 'inherit', ...opts });
|
||||
return true;
|
||||
} catch (e) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
${stmts}
|
||||
`;
|
||||
const r = spawnSync(NODE, ['-e', code], { encoding: 'utf8', timeout: 10_000 });
|
||||
return { status: r.status ?? 1, stdout: r.stdout ?? '', stderr: r.stderr ?? '' };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test group 1 — run() sentinel correctness
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('ci-rebase-check: run() helper — success sentinel (Codex round 4 P1)', () => {
|
||||
|
||||
test('run() returns true when the command succeeds', () => {
|
||||
// Use `node -e ""` (no-op) as a guaranteed-success command that produces no output,
|
||||
// avoiding stdout contamination when stdio:'inherit' writes to the same stream.
|
||||
const { status, stdout, stderr } = evalRunHelper(`
|
||||
const result = run(process.execPath, ['-e', '']);
|
||||
process.stdout.write(String(result));
|
||||
`);
|
||||
assert.strictEqual(status, 0, `subprocess should exit 0; stderr: ${stderr}`);
|
||||
assert.strictEqual(stdout, 'true', `run() must return true on success; got: ${stdout}`);
|
||||
});
|
||||
|
||||
test('run() returns false when the command fails', () => {
|
||||
// An invalid binary name causes execFileSync to throw ENOENT.
|
||||
const { status, stdout, stderr } = evalRunHelper(`
|
||||
const result = run('__nonexistent_binary_that_cannot_exist__', []);
|
||||
process.stdout.write(String(result));
|
||||
`);
|
||||
assert.strictEqual(status, 0, `subprocess should exit 0; stderr: ${stderr}`);
|
||||
assert.strictEqual(stdout, 'false', `run() must return false on failure; got: ${stdout}`);
|
||||
});
|
||||
|
||||
test('run() returns true (not null) — counter-test for pre-fix null behaviour', () => {
|
||||
// Pre-fix: execFileSync with stdio:'inherit' returns null on success.
|
||||
// The old check was `result !== null`, which would be `null !== null === false`.
|
||||
// Post-fix: result must be strictly true, making `if (result)` correct.
|
||||
// Use `node -e ""` (no-op) so stdio:'inherit' does not pollute our stdout capture.
|
||||
const { status, stdout } = evalRunHelper(`
|
||||
const result = run(process.execPath, ['-e', '']);
|
||||
process.stdout.write(JSON.stringify({ isTrue: result === true, isNull: result === null }));
|
||||
`);
|
||||
assert.strictEqual(status, 0);
|
||||
const { isTrue, isNull } = JSON.parse(stdout);
|
||||
assert.strictEqual(isNull, false, 'run() must NOT return null on success (pre-fix bug)');
|
||||
assert.strictEqual(isTrue, true, 'run() must return exactly true on success');
|
||||
});
|
||||
|
||||
test('run() returns false (not null) — failure path also returns boolean', () => {
|
||||
const { status, stdout } = evalRunHelper(`
|
||||
const result = run('__nonexistent__', []);
|
||||
process.stdout.write(JSON.stringify({ isFalse: result === false, isNull: result === null }));
|
||||
`);
|
||||
assert.strictEqual(status, 0);
|
||||
const { isFalse, isNull } = JSON.parse(stdout);
|
||||
assert.strictEqual(isNull, false, 'run() must NOT return null on failure');
|
||||
assert.strictEqual(isFalse, true, 'run() must return exactly false on failure');
|
||||
});
|
||||
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test group 2 — fetch-retry loop uses the boolean correctly
|
||||
//
|
||||
// We cannot run the actual git fetch against GitHub in unit tests, but we can
|
||||
// verify the fetch-loop logic by running the full script against a local git
|
||||
// repo where GITHUB_TOKEN and GITHUB_REPOSITORY are absent (so remote set-url
|
||||
// is skipped) and GITHUB_BASE_REF points to a branch that exists locally.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('ci-rebase-check: fetch-retry loop resolves when git fetch succeeds', () => {
|
||||
|
||||
test('script exits 0 when fetch succeeds (local bare remote, clean merge)', () => {
|
||||
// Set up: create a temp dir with a local git repo that has a `main` branch.
|
||||
// The script will fetch `origin main` from this local "remote".
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-431-ci-rebase-'));
|
||||
const remoteDir = path.join(tmpDir, 'remote.git');
|
||||
const workDir = path.join(tmpDir, 'work');
|
||||
|
||||
try {
|
||||
// Build a bare remote with a `main` branch containing one commit.
|
||||
fs.mkdirSync(remoteDir, { recursive: true });
|
||||
spawnSync('git', ['init', '--bare', remoteDir], { encoding: 'utf8' });
|
||||
|
||||
// Create a working clone to push an initial commit.
|
||||
spawnSync('git', ['clone', remoteDir, workDir], { encoding: 'utf8' });
|
||||
fs.writeFileSync(path.join(workDir, 'seed.txt'), 'init\n');
|
||||
spawnSync('git', ['-C', workDir, 'config', 'user.email', 'ci@test'], { encoding: 'utf8' });
|
||||
spawnSync('git', ['-C', workDir, 'config', 'user.name', 'CI Test'], { encoding: 'utf8' });
|
||||
spawnSync('git', ['-C', workDir, 'checkout', '-b', 'main'], { encoding: 'utf8' });
|
||||
spawnSync('git', ['-C', workDir, 'add', 'seed.txt'], { encoding: 'utf8' });
|
||||
spawnSync('git', ['-C', workDir, 'commit', '-m', 'init'], { encoding: 'utf8' });
|
||||
spawnSync('git', ['-C', workDir, 'push', 'origin', 'main'], { encoding: 'utf8' });
|
||||
|
||||
// Run the script from `workDir` with origin pointing at our bare remote.
|
||||
// GITHUB_BASE_REF=main so it fetches `origin main`.
|
||||
// No GITHUB_TOKEN so remote set-url is skipped.
|
||||
const r = spawnSync(NODE, [SCRIPT], {
|
||||
cwd: workDir,
|
||||
encoding: 'utf8',
|
||||
timeout: 20_000,
|
||||
env: {
|
||||
...process.env,
|
||||
GITHUB_BASE_REF: 'main',
|
||||
GITHUB_TOKEN: '',
|
||||
GITHUB_REPOSITORY: '',
|
||||
},
|
||||
});
|
||||
|
||||
assert.strictEqual(
|
||||
r.status, 0,
|
||||
`Script should exit 0 when fetch+merge succeed.\nstdout: ${r.stdout}\nstderr: ${r.stderr}`
|
||||
);
|
||||
// Must NOT emit the "failed after 3 attempts" error message.
|
||||
assert.ok(
|
||||
!(r.stderr || '').includes('failed after 3 attempts'),
|
||||
`Script must not emit "failed after 3 attempts" when fetch succeeded.\nstderr: ${r.stderr}`
|
||||
);
|
||||
} finally {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
});
|
||||
@@ -3,7 +3,7 @@
|
||||
/**
|
||||
* Regression test for #114 — npm dependency integrity gate.
|
||||
*
|
||||
* Verifies that scripts/check-npm-integrity.sh correctly detects:
|
||||
* Verifies that scripts/check-npm-integrity.cjs correctly detects:
|
||||
* 1. Clean install — exits 0, no stderr findings
|
||||
* 2. Version drift — exits 1, stderr names the offending package + both versions
|
||||
* (reproduces the ws 8.20.1 declared vs 8.20.0 installed incident)
|
||||
@@ -24,7 +24,7 @@ const { spawnSync } = require('node:child_process');
|
||||
const path = require('node:path');
|
||||
|
||||
const ROOT = path.resolve(__dirname, '..');
|
||||
const SCRIPT = path.join(ROOT, 'scripts', 'check-npm-integrity.sh');
|
||||
const SCRIPT = path.join(ROOT, 'scripts', 'check-npm-integrity.cjs');
|
||||
const FIXTURES = path.join(__dirname, 'fixtures', 'npm-integrity');
|
||||
|
||||
/**
|
||||
@@ -36,7 +36,7 @@ const FIXTURES = path.join(__dirname, 'fixtures', 'npm-integrity');
|
||||
*/
|
||||
function runGate(fixtureName, extraArgs = []) {
|
||||
const fixtureDir = path.join(FIXTURES, fixtureName);
|
||||
const result = spawnSync('bash', [SCRIPT, ...extraArgs], {
|
||||
const result = spawnSync(process.execPath, [SCRIPT, ...extraArgs], {
|
||||
cwd: fixtureDir,
|
||||
encoding: 'utf-8',
|
||||
timeout: 30_000,
|
||||
@@ -145,7 +145,7 @@ describe('#114: npm integrity gate — missing fixture', () => {
|
||||
|
||||
describe('#114: npm integrity gate — --help output', () => {
|
||||
test('exits 0 with --help flag', () => {
|
||||
const result = spawnSync('bash', [SCRIPT, '--help'], {
|
||||
const result = spawnSync(process.execPath, [SCRIPT, '--help'], {
|
||||
cwd: ROOT,
|
||||
encoding: 'utf-8',
|
||||
timeout: 10_000,
|
||||
@@ -154,17 +154,16 @@ describe('#114: npm integrity gate — --help output', () => {
|
||||
});
|
||||
|
||||
test('--help output mentions --ignore-extraneous', () => {
|
||||
const result = spawnSync('bash', [SCRIPT, '--help'], {
|
||||
const result = spawnSync(process.execPath, [SCRIPT, '--help'], {
|
||||
cwd: ROOT,
|
||||
encoding: 'utf-8',
|
||||
timeout: 10_000,
|
||||
});
|
||||
// The script routes --help output to stderr (cat >&2). Assert on stderr
|
||||
// specifically so a stray stdout match cannot produce a false positive.
|
||||
const helpText = result.stderr ?? '';
|
||||
// The .cjs script writes --help to stdout.
|
||||
const helpText = (result.stdout ?? '') + (result.stderr ?? '');
|
||||
assert.ok(
|
||||
helpText.includes('--ignore-extraneous'),
|
||||
`expected --help stderr to document --ignore-extraneous; got:\n${helpText}`
|
||||
`expected --help output to document --ignore-extraneous; got:\n${helpText}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
627
tests/policy-shell-pinning.test.cjs
Normal file
627
tests/policy-shell-pinning.test.cjs
Normal file
@@ -0,0 +1,627 @@
|
||||
'use strict';
|
||||
|
||||
const { test, describe } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const path = require('path');
|
||||
|
||||
const {
|
||||
POLICY,
|
||||
VIOLATION,
|
||||
inspectWorkflow,
|
||||
runPolicyLint,
|
||||
} = require('../scripts/workflow-policy.cjs');
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 1 — Baseline: repo's current workflow files must yield ZERO violations
|
||||
// (RED on origin/next; GREEN after YAML fixes are committed)
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('baseline: repo workflows comply with H1 shell policy', () => {
|
||||
test('runPolicyLint on .github/workflows produces zero violations', () => {
|
||||
const workflowsDir = path.resolve(__dirname, '..', '.github', 'workflows');
|
||||
const result = runPolicyLint({ workflowsDir });
|
||||
|
||||
if (result.violations.length > 0) {
|
||||
const top10 = result.violations.slice(0, 10);
|
||||
const msg = top10.map(v =>
|
||||
` ${path.basename(v.filePath)}:${v.evidence.line} [${v.jobId}/${v.stepName}] runner=${v.runner} shell=${v.effectiveShell} type=${v.violation}`
|
||||
).join('\n');
|
||||
assert.fail(
|
||||
`Expected 0 violations but found ${result.violations.length}. ` +
|
||||
`Top violations (mechanism: each step on a macos-* or windows-* runner must use native shell):\n${msg}`
|
||||
);
|
||||
}
|
||||
|
||||
assert.strictEqual(
|
||||
result.violations.length,
|
||||
0,
|
||||
'All workflow steps must comply with H1 shell policy'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 2 — Positive synthetic: compliant workflow yields zero violations
|
||||
// Three separate jobs, one per OS, each using H1-compliant shell configuration:
|
||||
// ubuntu: no shell pin (runner default bash = policy bash)
|
||||
// macos: job-level defaults.run.shell: zsh
|
||||
// windows: no shell pin (runner default pwsh = policy pwsh)
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('synthetic: fully-compliant per-OS jobs workflow', () => {
|
||||
const COMPLIANT_YAML = `
|
||||
name: Compliant Workflow
|
||||
jobs:
|
||||
linux-job:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Run tests on ubuntu
|
||||
run: npm test
|
||||
macos-job:
|
||||
runs-on: macos-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: zsh
|
||||
steps:
|
||||
- name: Run tests on macOS
|
||||
run: npm test
|
||||
windows-job:
|
||||
runs-on: windows-latest
|
||||
steps:
|
||||
- name: Run tests on windows
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('compliant per-OS workflow (ubuntu no pin, macos job-defaults zsh, windows no pin) produces zero violations', () => {
|
||||
const result = inspectWorkflow(COMPLIANT_YAML, { filePath: '<synthetic-compliant>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
0,
|
||||
'Compliant per-OS workflow must have zero violations. Got: ' +
|
||||
violations.map(v => `${v.runner}/${v.violation}`).join(', ')
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 3 — Counter-test: macOS missing explicit zsh → MACOS_MISSING_EXPLICIT_ZSH
|
||||
// (mechanism: macos-latest default is bash, not zsh; H1 requires explicit shell: zsh)
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('counter-test: macOS step without explicit shell: zsh', () => {
|
||||
const MACOS_NO_SHELL_YAML = `
|
||||
name: macOS No Shell
|
||||
jobs:
|
||||
build:
|
||||
runs-on: macos-latest
|
||||
steps:
|
||||
- name: Run tests
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('macos-latest step with no shell pin produces exactly one MACOS_MISSING_EXPLICIT_ZSH violation', () => {
|
||||
const result = inspectWorkflow(MACOS_NO_SHELL_YAML, { filePath: '<synthetic-macos-no-shell>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
1,
|
||||
`Expected exactly 1 violation (MACOS_MISSING_EXPLICIT_ZSH on macos-latest) but got ${violations.length}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].violation,
|
||||
VIOLATION.MACOS_MISSING_EXPLICIT_ZSH,
|
||||
`Expected violation type MACOS_MISSING_EXPLICIT_ZSH but got ${violations[0].violation}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].runner,
|
||||
'macos-latest',
|
||||
`Expected violation runner to be macos-latest but got ${violations[0].runner}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 4 — Counter-test: wrong shell for OS → WRONG_SHELL_FOR_OS
|
||||
// (mechanism: windows-2025 default is pwsh; specifying shell: bash is a policy violation)
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('counter-test: windows step with explicit shell: bash', () => {
|
||||
const WINDOWS_BASH_YAML = `
|
||||
name: Windows Bash
|
||||
jobs:
|
||||
build:
|
||||
runs-on: windows-2025
|
||||
steps:
|
||||
- name: Run tests with wrong shell
|
||||
shell: bash
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('windows-2025 step with shell: bash produces exactly one WRONG_SHELL_FOR_OS violation', () => {
|
||||
const result = inspectWorkflow(WINDOWS_BASH_YAML, { filePath: '<synthetic-windows-bash>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
1,
|
||||
`Expected exactly 1 violation (WRONG_SHELL_FOR_OS on windows-2025) but got ${violations.length}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].violation,
|
||||
VIOLATION.WRONG_SHELL_FOR_OS,
|
||||
`Expected violation type WRONG_SHELL_FOR_OS but got ${violations[0].violation}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].runner,
|
||||
'windows-2025',
|
||||
`Expected violation runner to be windows-2025 but got ${violations[0].runner}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 5 — Counter-test: matrix expansion with shell: bash on every step
|
||||
// (mechanism: ubuntu realizations are compliant since bash IS policy for ubuntu;
|
||||
// macos → WRONG_SHELL_FOR_OS (explicit wrong pin); windows → WRONG_SHELL_FOR_OS)
|
||||
// Expected: 2 violations total (1 macos + 1 windows), zero for ubuntu
|
||||
// Note: MACOS_MISSING_EXPLICIT_ZSH fires only when NO shell is set at any level;
|
||||
// here shell: bash is explicit, so WRONG_SHELL_FOR_OS is the correct subtype.
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('counter-test: three-OS matrix with shell: bash on every step', () => {
|
||||
const ALL_BASH_MATRIX_YAML = `
|
||||
name: All Bash Matrix
|
||||
jobs:
|
||||
build:
|
||||
runs-on: \${{ matrix.os }}
|
||||
strategy:
|
||||
matrix:
|
||||
os: [ubuntu-latest, macos-latest, windows-2025]
|
||||
steps:
|
||||
- name: Run tests
|
||||
shell: bash
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('three-OS matrix with shell: bash produces exactly 2 WRONG_SHELL_FOR_OS violations (macos + windows), zero for ubuntu', () => {
|
||||
const result = inspectWorkflow(ALL_BASH_MATRIX_YAML, { filePath: '<synthetic-all-bash-matrix>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
2,
|
||||
`Expected exactly 2 violations (macos-latest + windows-2025) but got ${violations.length}: ` +
|
||||
violations.map(v => `${v.runner}/${v.violation}`).join(', ')
|
||||
);
|
||||
|
||||
const macosViolation = violations.find(v => v.runner === 'macos-latest');
|
||||
assert.ok(
|
||||
macosViolation,
|
||||
'Expected a violation for macos-latest realization'
|
||||
);
|
||||
// When an explicit shell: bash is set on the step, the violation is WRONG_SHELL_FOR_OS
|
||||
// (the explicit pin is wrong for the OS). MACOS_MISSING_EXPLICIT_ZSH only fires when
|
||||
// there is NO shell set at any level and the runner default (bash) is inherited silently.
|
||||
assert.strictEqual(
|
||||
macosViolation.violation,
|
||||
VIOLATION.WRONG_SHELL_FOR_OS,
|
||||
`macos-latest with explicit shell: bash should be WRONG_SHELL_FOR_OS (explicit wrong pin) but got ${macosViolation?.violation}`
|
||||
);
|
||||
|
||||
const windowsViolation = violations.find(v => v.runner === 'windows-2025');
|
||||
assert.ok(
|
||||
windowsViolation,
|
||||
'Expected a violation for windows-2025 realization'
|
||||
);
|
||||
assert.strictEqual(
|
||||
windowsViolation.violation,
|
||||
VIOLATION.WRONG_SHELL_FOR_OS,
|
||||
`windows-2025 violation should be WRONG_SHELL_FOR_OS but got ${windowsViolation?.violation}`
|
||||
);
|
||||
|
||||
const ubuntuViolations = violations.filter(v => v.runner === 'ubuntu-latest');
|
||||
assert.strictEqual(
|
||||
ubuntuViolations.length,
|
||||
0,
|
||||
`ubuntu-latest should produce zero violations (bash is both runner default and policy) but got ${ubuntuViolations.length}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 6 — Counter-test: unknown runner → UNKNOWN_RUNNER
|
||||
// (mechanism: self-hosted is not in POLICY, so runner cannot be validated)
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('counter-test: self-hosted runner produces UNKNOWN_RUNNER violation', () => {
|
||||
const SELF_HOSTED_YAML = `
|
||||
name: Self-Hosted
|
||||
jobs:
|
||||
build:
|
||||
runs-on: self-hosted
|
||||
steps:
|
||||
- name: Run build
|
||||
run: npm build
|
||||
`;
|
||||
|
||||
test('self-hosted runner step produces exactly one UNKNOWN_RUNNER violation', () => {
|
||||
const result = inspectWorkflow(SELF_HOSTED_YAML, { filePath: '<synthetic-self-hosted>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
1,
|
||||
`Expected exactly 1 UNKNOWN_RUNNER violation but got ${violations.length}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].violation,
|
||||
VIOLATION.UNKNOWN_RUNNER,
|
||||
`Expected violation type UNKNOWN_RUNNER but got ${violations[0].violation}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].runner,
|
||||
'self-hosted',
|
||||
`Expected violation runner to be self-hosted but got ${violations[0].runner}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 7a — Positive: matrix.include with shell key + defaults.run.shell: ${{ matrix.shell }}
|
||||
// (mechanism: each realization carries its own shell value; the linter resolves
|
||||
// the matrix expression against the realization context before checking policy)
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('matrix.shell: ${{ matrix.shell }} resolves per realization — zero violations', () => {
|
||||
const MATRIX_SHELL_YAML = `
|
||||
name: Matrix Shell Positive
|
||||
jobs:
|
||||
build:
|
||||
runs-on: \${{ matrix.os }}
|
||||
defaults:
|
||||
run:
|
||||
shell: \${{ matrix.shell }}
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- os: macos-latest
|
||||
shell: zsh
|
||||
- os: windows-2025
|
||||
shell: pwsh
|
||||
steps:
|
||||
- name: Run tests
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('matrix.include with shell:zsh for macOS + shell:pwsh for Windows + defaults.run.shell: ${{ matrix.shell }} yields zero violations', () => {
|
||||
const result = inspectWorkflow(MATRIX_SHELL_YAML, { filePath: '<synthetic-matrix-shell-positive>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
0,
|
||||
'matrix.shell resolved per realization must produce zero violations. Got: ' +
|
||||
violations.map(v => `runner=${v.runner} shell=${v.effectiveShell} type=${v.violation}`).join(', ')
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 7b — Counter-test: matrix.include row with wrong shell value
|
||||
// (mechanism: if a row's shell value doesn't match its OS policy, WRONG_SHELL_FOR_OS fires)
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('matrix.shell: ${{ matrix.shell }} with wrong value per row — WRONG_SHELL_FOR_OS', () => {
|
||||
const MATRIX_SHELL_WRONG_YAML = `
|
||||
name: Matrix Shell Wrong Row
|
||||
jobs:
|
||||
build:
|
||||
runs-on: \${{ matrix.os }}
|
||||
defaults:
|
||||
run:
|
||||
shell: \${{ matrix.shell }}
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- os: macos-latest
|
||||
shell: bash
|
||||
- os: windows-2025
|
||||
shell: pwsh
|
||||
steps:
|
||||
- name: Run tests
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('matrix.include row with shell:bash for macOS produces WRONG_SHELL_FOR_OS (bash is wrong for macOS)', () => {
|
||||
const result = inspectWorkflow(MATRIX_SHELL_WRONG_YAML, { filePath: '<synthetic-matrix-shell-wrong>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
// macOS realization: shell resolves to bash → WRONG_SHELL_FOR_OS
|
||||
// Windows realization: shell resolves to pwsh → compliant
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
1,
|
||||
`Expected exactly 1 violation (macos-latest bash→WRONG_SHELL_FOR_OS) but got ${violations.length}: ` +
|
||||
violations.map(v => `runner=${v.runner} shell=${v.effectiveShell} type=${v.violation}`).join(', ')
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].runner,
|
||||
'macos-latest',
|
||||
`Expected violation for macos-latest but got ${violations[0].runner}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].violation,
|
||||
VIOLATION.WRONG_SHELL_FOR_OS,
|
||||
`Expected WRONG_SHELL_FOR_OS but got ${violations[0].violation}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 7c — Counter-test: matrix.include row missing the shell key while
|
||||
// defaults.run.shell: ${{ matrix.shell }} references it → UNRESOLVABLE_MATRIX
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('matrix.shell expression references missing key — UNRESOLVABLE_MATRIX', () => {
|
||||
const MATRIX_SHELL_MISSING_KEY_YAML = `
|
||||
name: Matrix Shell Missing Key
|
||||
jobs:
|
||||
build:
|
||||
runs-on: \${{ matrix.os }}
|
||||
defaults:
|
||||
run:
|
||||
shell: \${{ matrix.shell }}
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
node-version: 24
|
||||
steps:
|
||||
- name: Run tests
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('matrix.include row without shell key while defaults.run.shell: ${{ matrix.shell }} → UNRESOLVABLE_MATRIX', () => {
|
||||
const result = inspectWorkflow(MATRIX_SHELL_MISSING_KEY_YAML, { filePath: '<synthetic-matrix-shell-missing-key>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
1,
|
||||
`Expected exactly 1 UNRESOLVABLE_MATRIX violation but got ${violations.length}: ` +
|
||||
violations.map(v => `runner=${v.runner} type=${v.violation}`).join(', ')
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].violation,
|
||||
VIOLATION.UNRESOLVABLE_MATRIX,
|
||||
`Expected UNRESOLVABLE_MATRIX but got ${violations[0].violation}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].runner,
|
||||
'ubuntu-latest',
|
||||
`Expected runner ubuntu-latest but got ${violations[0].runner}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 7 — Counter-test: workflow-level defaults.run.shell: zsh satisfies macOS H1
|
||||
// (mechanism: resolution order puts workflow defaults above runner default;
|
||||
// zsh at workflow level means macOS steps inherit it without step-level pin)
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('counter-test: workflow-level defaults.run.shell: zsh satisfies macos-* H1', () => {
|
||||
const WORKFLOW_DEFAULTS_ZSH_YAML = `
|
||||
name: Workflow Defaults ZSH
|
||||
defaults:
|
||||
run:
|
||||
shell: zsh
|
||||
jobs:
|
||||
build:
|
||||
runs-on: macos-latest
|
||||
steps:
|
||||
- name: Run tests on macOS
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('workflow-level shell: zsh + macos-latest + no step-level shell produces zero violations', () => {
|
||||
const result = inspectWorkflow(WORKFLOW_DEFAULTS_ZSH_YAML, { filePath: '<synthetic-workflow-defaults-zsh>' });
|
||||
|
||||
assert.strictEqual(
|
||||
result.workflowDefaultsShell,
|
||||
'zsh',
|
||||
`Expected workflowDefaultsShell to be zsh but got ${result.workflowDefaultsShell}`
|
||||
);
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
0,
|
||||
`Workflow-level shell: zsh must satisfy H1 for macos-latest steps (resolution-order rule). Got ${violations.length} violations: ` +
|
||||
violations.map(v => `${v.violation}`).join(', ')
|
||||
);
|
||||
});
|
||||
|
||||
test('effective shell for macOS step is zsh when inherited from workflow defaults', () => {
|
||||
const result = inspectWorkflow(WORKFLOW_DEFAULTS_ZSH_YAML, { filePath: '<synthetic-workflow-defaults-zsh>' });
|
||||
|
||||
const step = result.jobs[0]?.steps[0];
|
||||
assert.ok(step, 'Expected at least one step');
|
||||
|
||||
assert.strictEqual(
|
||||
step.effectiveShell,
|
||||
'zsh',
|
||||
`Expected effectiveShell to be zsh (inherited from workflow defaults) but got ${step.effectiveShell}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
step.stepShell,
|
||||
null,
|
||||
`Expected stepShell to be null (no step-level pin) but got ${step.stepShell}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 8a — Counter-test: Cartesian matrix os × shell — dedup must not collapse rows by runner alone
|
||||
// (mechanism: matrix.os: [macos-latest, macos-latest] with matrix.shell: [zsh, bash]
|
||||
// and runs-on: ${{ matrix.os }}, step shell: ${{ matrix.shell }}.
|
||||
// The base-list path in expandRunsOn previously deduped by runner alone, collapsing
|
||||
// both macos-latest rows into one. Post-fix: each entry is pushed unconditionally,
|
||||
// producing 2 realizations from the base-list os array.
|
||||
//
|
||||
// NOTE: Cartesian cross-product expansion (expanding the full os × shell grid so
|
||||
// that each realization carries BOTH os and shell in its context) is not yet
|
||||
// implemented in expandRunsOn. The base-list path only records { os: runner } in
|
||||
// context, so ${{ matrix.shell }} on the step cannot be resolved and the linter
|
||||
// emits UNRESOLVABLE_MATRIX. The ideal post-Cartesian-expansion behavior would be
|
||||
// 2 WRONG_SHELL_FOR_OS violations (the bash rows). That is a separate follow-up bug.
|
||||
//
|
||||
// This test validates the dedupe fix only: 2 violations must be produced (not 1),
|
||||
// proving the base-list path no longer collapses duplicate runner values.
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('Cartesian matrix os × shell — dedup must not collapse rows by runner alone', () => {
|
||||
const CARTESIAN_MATRIX_YAML = `
|
||||
name: Cartesian Matrix
|
||||
jobs:
|
||||
build:
|
||||
runs-on: \${{ matrix.os }}
|
||||
strategy:
|
||||
matrix:
|
||||
os: [macos-latest, macos-latest]
|
||||
shell: [zsh, bash]
|
||||
steps:
|
||||
- name: Run tests
|
||||
shell: \${{ matrix.shell }}
|
||||
run: echo hi
|
||||
`;
|
||||
|
||||
test('Cartesian matrix os × shell — dedup must not collapse rows by runner alone', () => {
|
||||
const result = inspectWorkflow(CARTESIAN_MATRIX_YAML, { filePath: '<synthetic-cartesian-matrix>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
// The dedupe fix ensures both macos-latest entries in matrix.os are expanded
|
||||
// independently, yielding 2 realizations — not 1 (as the old dedup-by-runner
|
||||
// guard would produce). Each realization's ${{ matrix.shell }} is currently
|
||||
// UNRESOLVABLE_MATRIX because the base-list path doesn't yet carry shell context
|
||||
// (Cartesian cross-product is a separate follow-up fix).
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
2,
|
||||
`Expected exactly 2 violations (dedup fix: both macos-latest rows preserved) but got ${violations.length}: ` +
|
||||
violations.map(v => `runner=${v.runner} shell=${v.effectiveShell} type=${v.violation}`).join(', ')
|
||||
);
|
||||
|
||||
for (const v of violations) {
|
||||
assert.strictEqual(
|
||||
v.runner,
|
||||
'macos-latest',
|
||||
`Expected violation runner to be macos-latest but got ${v.runner}`
|
||||
);
|
||||
// UNRESOLVABLE_MATRIX because Cartesian cross-product expansion is not yet
|
||||
// implemented; ${{ matrix.shell }} cannot be resolved from base-list context.
|
||||
// When Cartesian expansion is added, these will become WRONG_SHELL_FOR_OS
|
||||
// (for the bash rows) and compliant (for the zsh rows).
|
||||
assert.strictEqual(
|
||||
v.violation,
|
||||
VIOLATION.UNRESOLVABLE_MATRIX,
|
||||
`Expected UNRESOLVABLE_MATRIX (shell key absent from base-list context) but got ${v.violation}`
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test 8 — Counter-test: two macos-latest matrix.include rows where
|
||||
// row 1 has shell: zsh (compliant) and row 2 has shell: bash (violation).
|
||||
// Guards against the dedup bug where runner-label-only deduplication would
|
||||
// collapse both rows into one, hiding the second row's policy violation.
|
||||
// Expected: EXACTLY ONE WRONG_SHELL_FOR_OS violation (on the second row).
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('counter-test: two macos-latest rows — dedup must not hide second row violation', () => {
|
||||
const TWO_MACOS_ROWS_YAML = `
|
||||
name: Two macOS Rows
|
||||
jobs:
|
||||
build:
|
||||
runs-on: \${{ matrix.os }}
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- os: macos-latest
|
||||
node-version: 22
|
||||
shell: zsh
|
||||
- os: macos-latest
|
||||
node-version: 24
|
||||
shell: bash
|
||||
steps:
|
||||
- name: Run tests
|
||||
shell: \${{ matrix.shell }}
|
||||
run: npm test
|
||||
`;
|
||||
|
||||
test('two macos-latest matrix.include rows (zsh + bash) produce exactly one WRONG_SHELL_FOR_OS violation on the second row', () => {
|
||||
const result = inspectWorkflow(TWO_MACOS_ROWS_YAML, { filePath: '<synthetic-two-macos-rows>' });
|
||||
|
||||
const violations = result.jobs
|
||||
.flatMap(j => j.steps)
|
||||
.filter(s => s.violation !== null);
|
||||
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
1,
|
||||
`Expected exactly 1 violation (second macos-latest row shell:bash → WRONG_SHELL_FOR_OS) but got ${violations.length}: ` +
|
||||
violations.map(v => `runner=${v.runner} shell=${v.effectiveShell} type=${v.violation}`).join(', ')
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].violation,
|
||||
VIOLATION.WRONG_SHELL_FOR_OS,
|
||||
`Expected WRONG_SHELL_FOR_OS but got ${violations[0].violation}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].runner,
|
||||
'macos-latest',
|
||||
`Expected violation runner to be macos-latest but got ${violations[0].runner}`
|
||||
);
|
||||
|
||||
assert.strictEqual(
|
||||
violations[0].effectiveShell,
|
||||
'bash',
|
||||
`Expected effectiveShell to be bash (the violating row) but got ${violations[0].effectiveShell}`
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -4,24 +4,34 @@ process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
/**
|
||||
* Asserts that every `run:` step whose command begins with `npm ` in any
|
||||
* .github/workflows/*.yml file has an effective `shell:` directive.
|
||||
* .github/workflows/*.yml file has an effective `shell:` directive that is
|
||||
* H1-policy-compliant (native shell per OS).
|
||||
*
|
||||
* H1 policy (LOCKED — open-gsd/get-shit-done-redux):
|
||||
* ubuntu-* → bash (runner default, no pin needed)
|
||||
* macos-* → zsh (must be pinned explicitly)
|
||||
* windows-* → pwsh (runner default, no pin needed)
|
||||
*
|
||||
* "Effective shell" is resolved as:
|
||||
* step.shell ?? job.defaults.run.shell ?? workflow.defaults.run.shell
|
||||
* step.shell ?? job.defaults.run.shell ?? workflow.defaults.run.shell ?? runner_default
|
||||
*
|
||||
* Without an effective shell, GitHub Actions defaults to pwsh on
|
||||
* windows-latest / windows-2025. The npm.cmd → node.exe → npm-cli.js child-process chain
|
||||
* under pwsh can swallow stderr, making `npm ci` / `npm run` failures
|
||||
* invisible in CI logs.
|
||||
* Under H1, Windows runner default is pwsh. pwsh does NOT have the npm.cmd
|
||||
* stderr-swallow issue that prompted the original bash requirement — that issue
|
||||
* was specific to running bash-wrapped npm in a pwsh session. With H1 in force,
|
||||
* Windows npm steps run natively under pwsh and are reliable.
|
||||
*
|
||||
* Scope: only workflow files that reference a Windows hosted runner label
|
||||
* literal `runs-on:` value, or as a member of a `strategy.matrix.os` list).
|
||||
* Steps in jobs that cannot run on Windows cannot trigger the class of failure
|
||||
* described above, but the file must still be scanned once any job in it
|
||||
* includes a windows target (to catch unshelled npm steps in sibling jobs that
|
||||
* could be copy-pasted to a windows context).
|
||||
* Violation conditions (H1-aware):
|
||||
* - An npm run: step on a Windows runner has shell: bash (wrong shell for OS,
|
||||
* and reintroduces the pwsh/bash interop issue H1 is designed to eliminate).
|
||||
* - An npm run: step on a macOS runner has no effective shell (macos default
|
||||
* is bash, but H1 requires zsh — tracked by policy-shell-pinning.test.cjs).
|
||||
*
|
||||
* Acceptable shell values: bash, pwsh, sh, cmd.
|
||||
* This test enforces the Windows side: Windows npm steps MUST NOT use shell: bash
|
||||
* (either directly or via job/workflow defaults). No shell pin = pwsh default = correct.
|
||||
*
|
||||
* Scope: only workflow files that reference a Windows hosted runner label.
|
||||
* Acceptable outcomes: no shell pin (pwsh default), or explicit shell: pwsh.
|
||||
* Unacceptable: shell: bash (H1 violation on Windows).
|
||||
*/
|
||||
|
||||
const { test, describe } = require('node:test');
|
||||
@@ -125,7 +135,14 @@ function findViolations(filePath) {
|
||||
|
||||
/**
|
||||
* Flush the current step: emit a violation if it qualifies.
|
||||
*
|
||||
* H1-aware check: Windows npm steps must NOT use shell: bash.
|
||||
* Under H1, Windows runner default is pwsh (native, reliable for npm).
|
||||
* Using shell: bash on Windows is an H1 policy violation AND reintroduces
|
||||
* the pwsh/bash interop issue the original rule was designed to prevent.
|
||||
*
|
||||
* Effective shell = step.shell ?? jobDefaultShell ?? workflowDefaultShell
|
||||
* (null means runner default applies — pwsh for windows, which is correct)
|
||||
*/
|
||||
function flushStep() {
|
||||
if (!inStep || stepProps === null) return;
|
||||
@@ -133,12 +150,14 @@ function findViolations(filePath) {
|
||||
const effectiveShell = shell !== null ? shell
|
||||
: jobDefaultShell !== null ? jobDefaultShell
|
||||
: workflowDefaultShell;
|
||||
if (run !== null && /^\s*(?:npm|npx)(\s|$)/.test(run) && effectiveShell === null) {
|
||||
// H1 violation: npm step on Windows with shell: bash (explicit or via defaults)
|
||||
if (run !== null && /^\s*(?:npm|npx)(\s|$)/.test(run) && effectiveShell === 'bash') {
|
||||
violations.push({
|
||||
file: relFile,
|
||||
job: currentJob,
|
||||
stepIndex,
|
||||
stepName: name || '(unnamed)',
|
||||
effectiveShell,
|
||||
});
|
||||
}
|
||||
inStep = false;
|
||||
@@ -316,7 +335,7 @@ function findViolationsInString(yamlContent) {
|
||||
// ── Test suite ──────────────────────────────────────────────────────────────
|
||||
|
||||
describe('GitHub Actions workflow shell pinning', () => {
|
||||
test('npm ci/run steps in workflow files must pin shell', () => {
|
||||
test('npm ci/run steps in Windows-targeting workflow files must not use shell: bash (H1 policy)', () => {
|
||||
const workflowFiles = listWorkflowFiles();
|
||||
assert.ok(workflowFiles.length > 0, 'No windows-targeting workflow files found — check WORKFLOWS_DIR path');
|
||||
|
||||
@@ -328,20 +347,21 @@ describe('GitHub Actions workflow shell pinning', () => {
|
||||
|
||||
if (allViolations.length > 0) {
|
||||
const details = allViolations.map(
|
||||
(v) => ` jobs.${v.job}.steps[${v.stepIndex}].name = ${v.stepName} (${v.file})`,
|
||||
(v) => ` jobs.${v.job}.steps[${v.stepIndex}].name = ${v.stepName} shell=${v.effectiveShell} (${v.file})`,
|
||||
).join('\n');
|
||||
assert.fail(
|
||||
`${allViolations.length} npm run/ci step(s) are missing an explicit shell: directive.\n` +
|
||||
`On Windows hosted runners, steps without shell: default to pwsh, which can swallow npm stderr.\n` +
|
||||
`Add shell: bash (or another explicit shell) to each listed step:\n\n` +
|
||||
`${allViolations.length} npm run/ci step(s) use shell: bash in a Windows-targeting workflow file.\n` +
|
||||
`H1 policy: Windows runners must use pwsh (runner default — no explicit pin needed).\n` +
|
||||
`shell: bash on Windows is both an H1 violation and reintroduces pwsh/bash interop issues.\n` +
|
||||
`Remove the shell: bash directive (or change to shell: pwsh) on each listed step:\n\n` +
|
||||
details,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('workflow-level defaults.run.shell satisfies shell requirement', () => {
|
||||
// A workflow with defaults.run.shell: bash at the root level should NOT
|
||||
// produce violations for npm steps that lack their own shell: directive.
|
||||
test('workflow-level defaults.run.shell: bash on Windows-targeting workflow is an H1 violation', () => {
|
||||
// H1: Windows runners must use pwsh (runner default). Setting defaults.run.shell: bash
|
||||
// at the workflow level forces npm steps on Windows to use bash — an H1 violation.
|
||||
const yaml = `
|
||||
name: Test
|
||||
on: push
|
||||
@@ -360,15 +380,15 @@ jobs:
|
||||
const violations = findViolationsInString(yaml);
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
0,
|
||||
`Expected 0 violations when workflow-level defaults.run.shell is set, got:\n` +
|
||||
violations.map((v) => ` steps[${v.stepIndex}] ${v.stepName}`).join('\n'),
|
||||
2,
|
||||
`Expected 2 violations (both npm steps inherit shell: bash via workflow defaults — H1 violation), got:\n` +
|
||||
violations.map((v) => ` steps[${v.stepIndex}] ${v.stepName} shell=${v.effectiveShell}`).join('\n'),
|
||||
);
|
||||
});
|
||||
|
||||
test('job-level defaults.run.shell satisfies shell requirement', () => {
|
||||
// A job with defaults.run.shell: bash should NOT produce violations for
|
||||
// npm steps in that job that lack their own shell: directive.
|
||||
test('job-level defaults.run.shell: bash on Windows job is an H1 violation', () => {
|
||||
// H1: Windows runners must use pwsh. job-level defaults.run.shell: bash
|
||||
// forces Windows npm steps to use bash — an H1 violation.
|
||||
const yaml = `
|
||||
name: Test
|
||||
on: push
|
||||
@@ -387,9 +407,9 @@ jobs:
|
||||
const violations = findViolationsInString(yaml);
|
||||
assert.strictEqual(
|
||||
violations.length,
|
||||
0,
|
||||
`Expected 0 violations when job-level defaults.run.shell is set, got:\n` +
|
||||
violations.map((v) => ` steps[${v.stepIndex}] ${v.stepName}`).join('\n'),
|
||||
2,
|
||||
`Expected 2 violations (both npm steps inherit shell: bash via job defaults — H1 violation), got:\n` +
|
||||
violations.map((v) => ` steps[${v.stepIndex}] ${v.stepName} shell=${v.effectiveShell}`).join('\n'),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user