dbbcb8f736dba30fccdac672fa15578352a3408c
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tom Boucher
|
cec7e704d6 |
feat(#435): expand workflow-policy linter to full Cartesian matrix cross-product (#1050)
`expandRunsOn` enumerated a multi-axis `strategy.matrix` one key at a time,
producing partial realization contexts. A true `os × shell` matrix therefore
left `${{ matrix.shell }}` unresolvable against any `{ os: ... }`-only context,
firing spurious `UNRESOLVABLE_MATRIX` violations and leaving shell-pinning
coverage incomplete on Cartesian jobs.
Enumerate the full GitHub Actions cross-product of all base-list matrix keys
(every `matrix.<k>` array, excluding the `include`/`exclude` control keys) via
a named `cartesianProduct` helper. Each realization's context now carries a
value for every matrix key, so `${{ matrix.<key> }}` resolves per realization.
Single-axis matrices keep byte-for-byte identical output; only multi-axis
matrices change shape. The `include` and `exclude` blocks are unchanged
(full tuple-aware exclude is a documented out-of-scope follow-up).
Tests: updated the `os × shell` test to assert post-fix behavior (4 step
realizations, 2 WRONG_SHELL_FOR_OS, 0 UNRESOLVABLE_MATRIX); added a compliant
`os × node-version` cross-product test; added a fast-check property test that
the realization count equals the product of axis lengths and that no axis key
is dropped from any context.
Closes #435
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Tom Boucher
|
d83e58eea0 |
fix(#437,#439,#440): restore defaults.run.shell + 'zsh {0}' format + Windows .cmd shell:true (PR #434 fallout) (#438)
* fix(#437): restore defaults.run.shell at job level (step-level matrix expr rejected by GHA) Per actions/runner workflow-v1.0.json schema, `jobs.<job_id>.defaults.run.shell` allows `matrix` context (job-defaults-run has context:[matrix,...]); step-level `shell:` does not (run-step's shell field is plain string with no context array). PR #434 used step-level shell:${{matrix.shell}}, which GHA's parser rejects with "Unrecognized named-value: 'matrix'" — blocking every push to next and every release.yml dispatch. This commit: - Removes step-level `shell: ${{ matrix.shell }}` from test-full (test.yml) and smoke (install-smoke.yml) jobs (17 directives). - Adds `defaults.run.shell: ${{ matrix.shell }}` at job level in those two jobs. - Fixes pre-existing shellcheck SC2129 in test.yml (individual >> redirects → grouped brace form) and SC2010 in install-smoke.yml (ls|grep → glob loop). Verified locally with actionlint 1.7.12 (exit 0). Policy linter still 0 violations (matrix.shell now resolves via job.defaults.run.shell which the linter already handles per workflow-policy.cjs:effectiveShell). Refs: actions/runner#444 (open since 2020), GHA contexts page section "Context availability". * fix(#439): inline ci-smoke-skip back to shell (Node port required pre-checkout file resolution) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#440): use platform-correct npm.cmd on Windows for spawn (and surface-check other Node ports) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#437): use 'zsh {0}' format string in matrix.shell for macOS (zsh not in GHA built-ins) Per https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions (jobs.<job_id>.defaults.run.shell section): "You can use built-in shell keywords like bash, pwsh, python, sh, cmd, and powershell, or define a custom set of shell options." zsh is not in the built-ins list. GHA accepts custom shells via a format string containing '{0}', which it replaces with the temporary script file path at runtime (same pattern as the perl {0} example in the docs). Bare `shell: zsh` triggers: "Invalid shell option. Shell must be a valid built-in or a format string containing '{0}'". Precursor: 514cb429 introduced the matrix shell-pinning pattern; this completes it by switching the macOS rows from the bare value to the required format string. Also updates scripts/workflow-policy.cjs to normalise 'zsh {0}' to 'zsh' before the policy comparison, so the repo-baseline test continues to pass (the linter was correctly treating 'zsh {0}' as a distinct value from the policy 'zsh'). Affects: - .github/workflows/test.yml: test-full matrix (node 22 + node 24 macOS rows) - .github/workflows/install-smoke.yml: smoke matrix (macOS node 24 row) - scripts/workflow-policy.cjs: detectViolation strips ' {0}' format suffix * fix(#440): add shell:true to spawnSync on Windows for .cmd files (Node docs requirement) Per https://nodejs.org/docs/latest-v22.x/api/child_process.html: ".bat and .cmd files require a terminal to run and cannot be launched directly with execFile(). To run these scripts on Windows, use child_process.spawn() with the shell option, child_process.exec(), or spawn cmd.exe with the script as an argument." "On Windows, .bat and .cmd files require a shell to execute. Use child_process.exec() or child_process.spawn() with the shell: true option." On Windows, npm is installed as npm.cmd (a batch wrapper). Without shell: true, spawnSync resolves the binary directly and fails with ENOENT / "npm binary not found on PATH" because the OS cannot execute a .cmd file without cmd.exe as the intermediary. The fix uses `shell: process.platform === 'win32'` so the shell spawning is only activated on Windows; macOS/Linux continue to resolve the plain npm binary directly with shell: false, preserving the existing behaviour on non-Windows platforms. Updated both spawnSync(npmCmd, ...) call sites: - npm --version check (line 182) - npm ci --dry-run lockfile-sync check (line 215) * fix(#437): bug-410 defaults test — set USERPROFILE for Windows os.homedir() redirect On Windows, os.homedir() reads USERPROFILE (not HOME), so the test's process.env.HOME = FAKE_HOME redirect was silently ignored. finishInstall's path.join(os.homedir(), '.gsd') resolved to the real user home and the defaults.json write either failed (permissions) or landed outside the temp dir, causing the existsSync assertion to return false. Fix: also set process.env.USERPROFILE = FAKE_HOME so os.homedir() returns the sandboxed directory on Windows. Node.js docs (os.homedir): https://nodejs.org/docs/latest-v22.x/api/os.html#oshomedir Refs: #437 (fix/437-restore-defaults-run-shell), Windows pwsh compat Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#437): precommit-alias-drift hook test — use path.delimiter for PATH Hardcoded ':' PATH separator breaks Windows where process.env.PATH uses ';'. The malformed PATH passed to bash caused the mock git/npm stubs in binDir to be invisible to the hook script; npm was never called and the marker file never written. Fix: replace ':' with path.delimiter in both PATH constructions so the env var is well-formed on Windows (';') and POSIX (':') alike. Refs: #437 (fix/437-restore-defaults-run-shell), Windows pwsh compat Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#437): prepush-enterprise-email hook test — use path.delimiter for PATH Same root cause as precommit-alias-drift: hardcoded ':' PATH separator is invalid on Windows (';' required). The malformed PATH meant bash ran the real git binary instead of the mock stub, which rejected the placeholder SHAs 'refs-local-sha' / 'refs-remote-sha' with a fatal ambiguous-argument error rather than returning the fixture commit list. Fix: replace ':' with path.delimiter in both execFileSync PATH env values. Refs: #437 (fix/437-restore-defaults-run-shell), Windows pwsh compat Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#437): set MSYS2_PATH_TYPE=inherit so mock stubs take precedence in Git Bash PATH Root cause: Git Bash (MSYS2) on Windows prepends its own system directories (/mingw64/bin, /usr/bin, /bin) to the PATH at process startup before the user-supplied Windows PATH entries. This placed the real git/npm binaries ahead of the mock stubs in binDir even though binDir was first in the Windows PATH passed to execFileSync. The path.delimiter fix (0042fe0d) made the PATH syntactically correct for Windows (semicolons) but did not change the MSYS2 system-dir prepend order. The real git rejected placeholder SHAs (refs-local-sha, refs-remote-sha) with "fatal: ambiguous argument", producing the observed Windows CI failure. For the pre-commit test, the real git output nothing (no staged files on a fresh checkout), so the grep match failed and npm was never called. Fix: set MSYS2_PATH_TYPE=inherit in the env passed to both bash spawns. With inherit, MSYS2 uses only the converted Windows PATH without prepending system directories, so binDir (converted from Windows to POSIX) is first in the search path and the mock stubs are found. grep/tr/printf remain available: the GHA Windows runner PATH includes C:\Program Files\Git\usr\bin which contains these utilities; MSYS2 converts that Windows entry to a POSIX path on startup. The /usr/bin/env shebang in mock stubs resolves through MSYS2's virtual filesystem mount (not via PATH) and is always accessible regardless of MSYS2_PATH_TYPE. On macOS/Linux this variable is ignored; no behaviour change on those platforms. Source: https://www.msys2.org/wiki/MSYS2-introduction/#path (MSYS2_PATH_TYPE controls whether system dirs are prepended to converted PATH) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#437): hook test mocks — use cmd-shim pattern for Windows bin resolution On Windows, bash (Git Bash / MSYS2) resolves PATH commands by scanning for extensionless files, but cmd.exe and Win32 process creation resolve via PATHEXT (.CMD, .BAT, .EXE). When execFileSync('bash', [hookPath]) runs a hook that calls `git` or `npm`, both resolution paths may fire. The previous approach set MSYS2_PATH_TYPE=inherit in the child env, but that variable is only read in /etc/profile (login-shell path) — bash launched without --login never sources /etc/profile, so the variable had no effect: https://github.com/msys2/MSYS2-packages/blob/master/filesystem/profile Fix: adopt the cmd-shim three-file pattern used by npm itself: https://github.com/npm/cmd-shim For each mock binary, write: <name> extensionless bash script (bash PATH scan) <name>.cmd batch wrapper delegating to bash (PATHEXT / cmd.exe) <name>.ps1 PowerShell wrapper (completeness) This is the same approach used by stevemao/mock-bin for test mocking with Windows CI green on AppVeyor: https://github.com/stevemao/mock-bin The .cmd and .ps1 files are only written on process.platform === 'win32'. MSYS2_PATH_TYPE is removed from the child env — it was ineffective and is no longer needed with the shim files in place. * fix(#437): tarball-smoke — raise CHILD_TIMEOUT_MS on Windows to 600 s The CI failure showed a test duration of 120003.1812 ms — matching the previous CHILD_TIMEOUT_MS = 120_000 exactly. When spawnSync hits its timeout, it sends SIGTERM and returns { status: null, stdout: '', stderr: '' } per the Node.js docs: https://nodejs.org/docs/latest-v22.x/api/child_process.html "status: <number> | <null> — The exit code of the subprocess, or null if the subprocess terminated due to a signal." The installResult check is `status !== 0`; null !== 0 is true, so the timeout fired the INSTALL_FAILED path with empty stdout/stderr, which made the root cause invisible in CI logs. GitHub-hosted Windows runners are slower than Linux/macOS for filesystem-heavy operations (npm install -g of a 1499-file tarball): https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories Fix: use 600_000 ms (10 min) on Windows, keeping 120_000 ms on POSIX. 600 s matches the SLOW_HOST_TIMEOUT already used in the test before() helper for the pack + install fixture step. Also expose `signal` and `installError` in the INSTALL_FAILED details object so a future timeout (status=null, signal='SIGTERM', stdout='') is immediately diagnosable in CI logs without guesswork. * fix(#437): chmod +x via bash on Windows for hook test mocks (root cause: fs.writeFileSync mode=0o755 no-op on NTFS) Root cause: Node's fs.writeFileSync mode=0o755 is a no-op for the execute bit on Windows NTFS. Per https://nodejs.org/docs/latest-v22.x/api/fs.html: "on Windows only the write permission can be changed." Bash's access(X_OK) therefore skips the mock file; the real git/npm binary is found later in PATH and the hook runs against real state instead of the test double. Fix: after writeFileSync, invoke Git Bash's chmod via the POSIX emulation layer (Cygwin/MSYS2), which sets the NTFS execute ACL that Node cannot reach: const posixPath = filePath.replace(/\\/g, '/'); execFileSync('bash', ['-c', `chmod +x "${posixPath}"`], { stdio: 'pipe' }); execFileSync('bash', ...) works because Git for Windows ships bash on PATH in all GHA Windows runners. Forward-slash conversion is required because MSYS2 bash auto-converts /c/foo paths but not mixed-separator paths. Why prior approaches didn't take effect: - MSYS2_PATH_TYPE=inherit: only read in /etc/profile (login-shell path); execFileSync('bash', ...) launches non-interactively without --login, so /etc/profile is never sourced. Ref: https://github.com/msys2/MSYS2-packages/blob/master/filesystem/profile - .cmd/.ps1 cmd-shim wrappers: bash does POSIX command resolution and does not honor PATHEXT, so wrappers are not found by bash's own PATH scan. They are not wrong (kept for non-bash callers) but do not fix bash's X_OK. Files changed: tests/precommit-alias-drift-hook.test.cjs, tests/prepush-enterprise-email-hook.test.cjs * refactor(#437): hooks use GIT_OVERRIDE/NPM_OVERRIDE env-var DI; tests drop PATH-mocking Four prior rounds (path.delimiter join, MSYS2_PATH_TYPE=inherit, cmd-shim .cmd/.ps1 wrappers, chmod-via-bash post-write) all failed to make MSYS2 bash's PATH-lookup find the mock executables. The root cause is that none of those approaches can reliably override bash's own command-resolution on NTFS without fighting NTFS execute-ACLs or login-shell profile sourcing. The simplest robust solution is to bypass PATH entirely: Hooks: each hook now binds GIT_CMD="${GIT_OVERRIDE:-git}" (and NPM_CMD for pre-commit) at the top. When env vars are unset the hooks invoke bare `git`/`npm` exactly as before — zero behavior change for users. Tests: writeMockBin/binDir/PATH manipulation replaced by writeMock(), which writes a .sh mock to a tmpDir and passes its absolute path via GIT_OVERRIDE / NPM_OVERRIDE in the execFileSync env. Bash inside the hook executes the path directly via the seam — no PATH scan, no NTFS ACL check, no MSYS2 profile dependency. Test-rigor principle: the new seam (env-var injection) is platform- independent and doesn't rely on bash's command-resolution mechanism on the host OS. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: CI Rebase Check <ci@gsd-redux> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Tom Boucher
|
48b1e35187 |
fix(#431): enforce H1 shell policy (linux=bash, macOS=zsh, windows=pwsh) across PR + release gates (#434)
* test(#431): policy-shell-pinning linter — RED baseline (37 violations on origin/next) Adds scripts/workflow-policy.cjs: H1 shell-policy linter with POLICY map, VIOLATION enum, matrix expansion, effective-shell resolution order, and runPolicyLint({ workflowsDir }) entry point. Adds tests/policy-shell-pinning.test.cjs: 8 tests (baseline + 6 synthetic counter-tests). Synthetic tests 2–7 pass; baseline test is intentionally RED (37 violations: 28 in test.yml, 9 in install-smoke.yml — all macos/windows lanes using shell: bash instead of native zsh/pwsh). Adds js-yaml@4.1.1 as devDependency for YAML parsing. * fix(#431): switch ubuntu/windows lanes to native shells; extract bash-isms to Node Remove all explicit shell: bash pins from ubuntu-only jobs (changes, lint-tests, coverage, required-tests, smoke-unpacked) — ubuntu runner default is bash, which is both H1-compliant and the runner default, making the pin redundant. For the test and test-full mixed-OS jobs (ubuntu+windows, windows+macos): - Move bash-ism steps to shell-agnostic Node scripts: scripts/ci-guard-runner.cjs — RUNNER_ENVIRONMENT check scripts/ci-rebase-check.cjs — git fetch+merge PR base branch scripts/check-npm-integrity.cjs — Node port of check-npm-integrity.sh scripts/ci-prepare-test-scope.cjs — write .ci-selected-tests.txt scripts/ci-smoke-skip.cjs — set skip= output for full-only matrix entries - Remove shell: bash from simple npm/node command steps (runner default applies) This brings Windows violations from 19 to 0. Remaining 17 violations are all MACOS_MISSING_EXPLICIT_ZSH in mixed-OS matrix jobs (test-full: windows+macos, install-smoke smoke: ubuntu+macos) — these require job splitting to fix; see BLOCKER in PR description. * fix(#431): update workflow-shell-pinning test for H1 policy The old test required all Windows-targeting npm steps to pin shell: bash (to prevent pwsh stderr-swallow). Under H1, Windows runners must use pwsh (native, no pin needed) — shell: bash on Windows is now the violation, not the fix. Update findViolations() to flag npm steps with effectiveShell === 'bash' (rather than effectiveShell === null). Update synthetic tests to verify the H1-inverted semantics: defaults.run.shell: bash on Windows is now 2 violations, not 0. Update test name and assertion messages to describe the H1 constraint rather than the old missing-pin constraint. * fix(#431): extend policy linter to resolve matrix.shell expressions - expandRunsOn now captures all matrix.include row keys as realization context (os, node-version, shell, full_only, etc.) instead of only os - effectiveShell now accepts a realizationContext and resolves ${{ matrix.<key> }} expressions against it before checking policy - Unresolvable matrix key in shell expression emits UNRESOLVABLE_MATRIX - Add 3 new tests: positive (zsh+pwsh per row → 0 violations), counter (bash in macOS row → WRONG_SHELL_FOR_OS), counter (missing shell key → UNRESOLVABLE_MATRIX) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): apply matrix.shell pattern to test-full and smoke jobs (clears BLOCKER) test-full job (test.yml): - Add shell: pwsh/zsh per matrix.include row (windows-latest→pwsh, macos-latest→zsh) - Add job-level defaults.run.shell: ${{ matrix.shell }} - No step-level shell pins existed to remove smoke job (install-smoke.yml): - Add shell: bash/zsh per matrix.include row (ubuntu→bash, macos→zsh) - Add job-level defaults.run.shell: ${{ matrix.shell }} - No step-level shell pins existed to remove Policy linter now reports 0 violations across all workflow files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#431): migrate .sh check scripts to .cjs; remove .sh originals - Add scripts/check-env.cjs: Node.js port of check-env.sh with identical exit codes (0/1/2), human-readable and --json output, --help flag, and all 5 checks (node-version, npm-version, lockfile-present, lockfile-sync, version-manager-pin) - Migrate all callers: - package.json check:env → node scripts/check-env.cjs - package.json check:integrity → node scripts/check-npm-integrity.cjs - scripts/ci-test-scope.cjs path strings → .cjs equivalents - .github/workflows/release.yml rc+finalize jobs → node .cjs (drop chmod+x) - .github/workflows/security-scan.yml → node .cjs (drop chmod+x) - tests/check-env.test.cjs → spawn node process.execPath [.cjs] - tests/npm-integrity-gate.test.cjs → spawn node process.execPath [.cjs] - Delete scripts/check-env.sh and scripts/check-npm-integrity.sh Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#431): update doc references from .sh to .cjs Update SECURITY.md and docs/contributing/bootstrap.md to reference the canonical Node invocation instead of the removed bash scripts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): use per-step shell:matrix.shell instead of defaults.run.shell (GHA compat) GHA does not reliably resolve matrix expressions inside defaults.run.shell. Per-step shell: always resolves correctly. Removed the defaults.run.shell block from the test-full job (test.yml) and the smoke job (install-smoke.yml), and added shell: \${{ matrix.shell }} directly on every run: step in both jobs. Codex finding: defaults.run.shell with matrix expressions is not a GHA-supported pattern; per-step shell: is the safe form. * fix(#431): policy linter validates every matrix.include row independently Removed runner-label-only dedup from expandRunsOn() in workflow-policy.cjs. The prior guard (if !realizations.find(r => r.runner === runner)) collapsed two macos-latest rows with different node-version/shell contexts into one, hiding the second row's policy violation. Each matrix.include row is a distinct CI realization with its own context; validating it twice is harmless but skipping it causes false negatives. Added counter-test (Test 8) in tests/policy-shell-pinning.test.cjs: two macos-latest rows (shell:zsh compliant + shell:bash violation) must produce exactly one WRONG_SHELL_FOR_OS violation on the second row. * fix(#431): remove dedup-by-runner in Cartesian matrix.<key> expansion (Codex round 3) The base-list path in expandRunsOn (matrix.<key> arrays, e.g. matrix.os) previously guarded each push with `if (!realizations.find(r => r.runner === runner))`, collapsing duplicate runner values into a single realization and hiding policy violations on later rows of a Cartesian matrix. Remove the guard unconditionally; each entry in the base-list array now produces its own realization, matching the same fix already applied to the matrix.include path. Add counter-test "Cartesian matrix os × shell — dedup must not collapse rows by runner alone": matrix.os: [macos-latest, macos-latest] + shell: ${{ matrix.shell }} now yields 2 realizations (not 1). Documents that Cartesian cross-product expansion (carrying all keys into realization context) is a separate follow-up; current violations are UNRESOLVABLE_MATRIX pending that work. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): remove 60s timeout regression on npm ci --dry-run (parity with check-env.sh) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#431): ci-rebase-check.cjs — return truthy sentinel on success (Codex round 4) run() used execFileSync with stdio:'inherit', which returns null on success. Caller checked `result !== null`, always false → every successful fetch fell through to "failed after 3 attempts" exit-1 path. Fix: run() now returns true on success, false on failure. Update caller from `result !== null` to `if (result)`. Adds tests/ci-rebase-check.test.cjs (5 tests) covering the sentinel contract and a local-bare-remote integration smoke that verifies the full fetch+merge path exits 0 when fetch succeeds. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: CI Rebase Check <ci@gsd-redux> |