golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,539 Commits 2 Branches 0 Tags
9e2ef2c94dea3f1267deba570c9a423fb38c2f85
Commit Graph

2 Commits

Author SHA1 Message Date
Tom Boucher
48b1e35187 fix(#431): enforce H1 shell policy (linux=bash, macOS=zsh, windows=pwsh) across PR + release gates (#434) 
* test(#431): policy-shell-pinning linter — RED baseline (37 violations on origin/next)

Adds scripts/workflow-policy.cjs: H1 shell-policy linter with POLICY map,
VIOLATION enum, matrix expansion, effective-shell resolution order, and
runPolicyLint({ workflowsDir }) entry point.

Adds tests/policy-shell-pinning.test.cjs: 8 tests (baseline + 6 synthetic
counter-tests). Synthetic tests 2–7 pass; baseline test is intentionally RED
(37 violations: 28 in test.yml, 9 in install-smoke.yml — all macos/windows
lanes using shell: bash instead of native zsh/pwsh).

Adds js-yaml@4.1.1 as devDependency for YAML parsing.

* fix(#431): switch ubuntu/windows lanes to native shells; extract bash-isms to Node

Remove all explicit shell: bash pins from ubuntu-only jobs (changes, lint-tests,
coverage, required-tests, smoke-unpacked) — ubuntu runner default is bash, which
is both H1-compliant and the runner default, making the pin redundant.

For the test and test-full mixed-OS jobs (ubuntu+windows, windows+macos):
- Move bash-ism steps to shell-agnostic Node scripts:
    scripts/ci-guard-runner.cjs       — RUNNER_ENVIRONMENT check
    scripts/ci-rebase-check.cjs       — git fetch+merge PR base branch
    scripts/check-npm-integrity.cjs   — Node port of check-npm-integrity.sh
    scripts/ci-prepare-test-scope.cjs — write .ci-selected-tests.txt
    scripts/ci-smoke-skip.cjs         — set skip= output for full-only matrix entries
- Remove shell: bash from simple npm/node command steps (runner default applies)

This brings Windows violations from 19 to 0. Remaining 17 violations are all
MACOS_MISSING_EXPLICIT_ZSH in mixed-OS matrix jobs (test-full: windows+macos,
install-smoke smoke: ubuntu+macos) — these require job splitting to fix; see
BLOCKER in PR description.

* fix(#431): update workflow-shell-pinning test for H1 policy

The old test required all Windows-targeting npm steps to pin shell: bash
(to prevent pwsh stderr-swallow). Under H1, Windows runners must use
pwsh (native, no pin needed) — shell: bash on Windows is now the
violation, not the fix.

Update findViolations() to flag npm steps with effectiveShell === 'bash'
(rather than effectiveShell === null). Update synthetic tests to verify
the H1-inverted semantics: defaults.run.shell: bash on Windows is now 2
violations, not 0. Update test name and assertion messages to describe
the H1 constraint rather than the old missing-pin constraint.

* fix(#431): extend policy linter to resolve matrix.shell expressions

- expandRunsOn now captures all matrix.include row keys as realization
  context (os, node-version, shell, full_only, etc.) instead of only os
- effectiveShell now accepts a realizationContext and resolves
  ${{ matrix.<key> }} expressions against it before checking policy
- Unresolvable matrix key in shell expression emits UNRESOLVABLE_MATRIX
- Add 3 new tests: positive (zsh+pwsh per row → 0 violations),
  counter (bash in macOS row → WRONG_SHELL_FOR_OS), counter (missing
  shell key → UNRESOLVABLE_MATRIX)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#431): apply matrix.shell pattern to test-full and smoke jobs (clears BLOCKER)

test-full job (test.yml):
- Add shell: pwsh/zsh per matrix.include row (windows-latest→pwsh,
  macos-latest→zsh)
- Add job-level defaults.run.shell: ${{ matrix.shell }}
- No step-level shell pins existed to remove

smoke job (install-smoke.yml):
- Add shell: bash/zsh per matrix.include row (ubuntu→bash, macos→zsh)
- Add job-level defaults.run.shell: ${{ matrix.shell }}
- No step-level shell pins existed to remove

Policy linter now reports 0 violations across all workflow files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(#431): migrate .sh check scripts to .cjs; remove .sh originals

- Add scripts/check-env.cjs: Node.js port of check-env.sh with
  identical exit codes (0/1/2), human-readable and --json output,
  --help flag, and all 5 checks (node-version, npm-version,
  lockfile-present, lockfile-sync, version-manager-pin)
- Migrate all callers:
  - package.json check:env → node scripts/check-env.cjs
  - package.json check:integrity → node scripts/check-npm-integrity.cjs
  - scripts/ci-test-scope.cjs path strings → .cjs equivalents
  - .github/workflows/release.yml rc+finalize jobs → node .cjs (drop chmod+x)
  - .github/workflows/security-scan.yml → node .cjs (drop chmod+x)
  - tests/check-env.test.cjs → spawn node process.execPath [.cjs]
  - tests/npm-integrity-gate.test.cjs → spawn node process.execPath [.cjs]
- Delete scripts/check-env.sh and scripts/check-npm-integrity.sh

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(#431): update doc references from .sh to .cjs

Update SECURITY.md and docs/contributing/bootstrap.md to reference the
canonical Node invocation instead of the removed bash scripts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#431): use per-step shell:matrix.shell instead of defaults.run.shell (GHA compat)

GHA does not reliably resolve matrix expressions inside defaults.run.shell.
Per-step shell: always resolves correctly. Removed the defaults.run.shell block
from the test-full job (test.yml) and the smoke job (install-smoke.yml), and
added shell: \${{ matrix.shell }} directly on every run: step in both jobs.

Codex finding: defaults.run.shell with matrix expressions is not a
GHA-supported pattern; per-step shell: is the safe form.

* fix(#431): policy linter validates every matrix.include row independently

Removed runner-label-only dedup from expandRunsOn() in workflow-policy.cjs.
The prior guard (if !realizations.find(r => r.runner === runner)) collapsed
two macos-latest rows with different node-version/shell contexts into one,
hiding the second row's policy violation.

Each matrix.include row is a distinct CI realization with its own context;
validating it twice is harmless but skipping it causes false negatives.

Added counter-test (Test 8) in tests/policy-shell-pinning.test.cjs:
two macos-latest rows (shell:zsh compliant + shell:bash violation) must
produce exactly one WRONG_SHELL_FOR_OS violation on the second row.

* fix(#431): remove dedup-by-runner in Cartesian matrix.<key> expansion (Codex round 3)

The base-list path in expandRunsOn (matrix.<key> arrays, e.g. matrix.os)
previously guarded each push with `if (!realizations.find(r => r.runner === runner))`,
collapsing duplicate runner values into a single realization and hiding policy
violations on later rows of a Cartesian matrix.

Remove the guard unconditionally; each entry in the base-list array now produces
its own realization, matching the same fix already applied to the matrix.include path.

Add counter-test "Cartesian matrix os × shell — dedup must not collapse rows by
runner alone": matrix.os: [macos-latest, macos-latest] + shell: ${{ matrix.shell }}
now yields 2 realizations (not 1). Documents that Cartesian cross-product expansion
(carrying all keys into realization context) is a separate follow-up; current violations
are UNRESOLVABLE_MATRIX pending that work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#431): remove 60s timeout regression on npm ci --dry-run (parity with check-env.sh)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#431): ci-rebase-check.cjs — return truthy sentinel on success (Codex round 4)

run() used execFileSync with stdio:'inherit', which returns null on success.
Caller checked `result !== null`, always false → every successful fetch fell
through to "failed after 3 attempts" exit-1 path.

Fix: run() now returns true on success, false on failure.
Update caller from `result !== null` to `if (result)`.

Adds tests/ci-rebase-check.test.cjs (5 tests) covering the sentinel contract
and a local-bare-remote integration smoke that verifies the full fetch+merge
path exits 0 when fetch succeeds.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: CI Rebase Check <ci@gsd-redux>
 2026-05-28 09:23:59 -04:00
Tom Boucher
89886d90b4 feat(114): npm dependency integrity gate (npm ls invalid/extraneous) (#135) 
* test(114): add failing regression tests for npm dependency integrity gate

Adds tests/npm-integrity-gate.test.cjs and four fixture directories under
tests/fixtures/npm-integrity/ covering:
  - clean: matching lockfile and node_modules (expects exit 0)
  - drift: declared vs installed version mismatch (expects exit 1)
    Reproduces the ws 8.20.1 declared / 8.20.0 installed incident shape
    using stable-dep@8.20.1 (package.json) vs stable-dep@8.20.0 (node_modules).
  - extraneous: unlisted package in node_modules (exits 1; exits 0 with --ignore-extraneous)
  - missing: declared package absent from node_modules (exits 1 regardless of flags)

Each test spawns scripts/check-npm-integrity.sh as a subprocess and asserts
on exit code first, then stderr content. Tests are RED at this commit because
the script does not yet exist.

Sources:
  npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
  NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(114): add check-npm-integrity.sh + workspace-aware drift detection

Adds scripts/check-npm-integrity.sh, a Bash script that:
  1. Runs `npm ls --all --json` at the invocation directory
  2. Parses JSON output for invalid, missing, and extraneous package flags
  3. Exits 1 on any finding; emits a structured report to stderr listing offenders
     with both declared and installed versions for invalid packages
  4. Exits 2 on tool error (npm/node not found, JSON parse failure)
  5. Accepts --ignore-extraneous to suppress extraneous-only failures
  6. Documents behaviour in --help output including remediation path

Workspace behaviour: the root package.json in this repo has no "workspaces"
field. npm ls runs at the invocation root and covers that tree only. The sdk/
sub-package is a separate, non-workspace package and is out of scope for a
single invocation. If workspaces are added in future, npm ls will traverse
them automatically (npm >=7).

The drift scenario (ws 8.20.1 declared vs 8.20.0 installed) is reproduced by
using an exact version pin in package.json combined with a mismatched
node_modules/package.json -- npm ls marks this as "invalid" and exits 1.

npm exits 0 for extraneous packages even though they appear in the JSON
"problems" array; this script detects them via JSON parsing regardless of
the npm exit code.

Sources:
  npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
  NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
  OpenSSF Scorecard Pinned-Dependencies:
    https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci(114): wire dependency integrity gate into CI/release/security workflows

Adds a "Dependency integrity gate" step invoking
scripts/check-npm-integrity.sh to three workflows, always after `npm ci`
and before any test or build step:

  .github/workflows/test.yml
    - matrix job: after "Install dependencies" / before "Build SDK dist"
    - coverage job: after "Install dependencies" / before "Build SDK dist"

  .github/workflows/release.yml
    - rc job "Install and test": after npm ci, before npm run test:coverage
    - finalize job "Install and test": after npm ci, before npm run test:coverage

  .github/workflows/security-scan.yml
    - Added setup-node + npm ci + gate before existing source-scan steps
    - Bumped timeout-minutes from 5 to 10 to accommodate the install step

Also adds "check:integrity": "./scripts/check-npm-integrity.sh" to root
package.json scripts for local contributor invocation.

No new workflow files created. All edits extend existing workflows.

Sources:
  npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
  NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(114): document dependency integrity gate in audit runbook

Appends a "Dependency Integrity Verification" section to SECURITY.md
(no docs/runbooks/ directory exists in this repo). Covers:
  - The three detection classes: invalid, missing, extraneous
  - Local invocation: ./scripts/check-npm-integrity.sh + npm run check:integrity
  - Remediation: rm -rf node_modules && npm ci
  - Bypass policy: no flag; commit-message documentation required if skipped
  - Scope: root package only (sdk/ is a non-workspace package, out of scope)
  - CI coverage listing

Sources cited:
  NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
  OpenSSF Scorecard Pinned-Dependencies:
    https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#114): npm integrity gate satisfies its own clean/drift/extraneous fixtures

Replace npm-ls-based analysis with pure package-lock.json parsing so the
script runs correctly in CI and test environments where node_modules is not
installed. Key changes:

- Rewrite check-npm-integrity.sh parser to read package-lock.json directly
  instead of spawning `npm ls --all --json`, which required node_modules on
  disk and incorrectly flagged clean/drift fixtures as MISSING.
- Implement a self-contained semver satisfies() covering exact, caret, tilde,
  comparison-operator, and compound ranges — no external semver package needed.
- Update extraneous fixture package-lock.json to include ghost-pkg with
  "extraneous: true" so the lockfile-based detector can identify it.
- Update missing fixture package-lock.json to omit the node_modules/absent-dep
  entry, making the absent-dep MISSING condition derivable from lockfile alone.

All 13 tests (clean ×2, drift ×3, extraneous ×3, missing ×3, help ×2) pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#114): treat transitive deps as non-extraneous in integrity gate

The extraneous check was comparing all lockfile packages against root
package.json declarations only. This caused every transitive dependency
(e.g. hono, ajv, @anthropic-ai/claude-agent-sdk-darwin-arm64) to be
flagged as EXTRANEOUS, producing false-positive failures in CI.

Only packages that npm itself marks with "extraneous: true" in the
lockfile represent genuinely unwanted packages. Transitive dependencies
installed by parent packages are valid and should be skipped.

All 13 existing tests continue to pass; the extraneous fixture still
works because it uses "extraneous: true" explicitly (npm's own marker).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: retrigger checks after transient git-auth runner failure

The original run for this PR had a single CI job fail with:
"fatal: could not read Username for 'https://github.com': terminal prompts disabled"
That is a hosted-runner infrastructure flake — no code defect. The run
cannot be retried via gh CLI (too old). This empty commit kicks a fresh
full CI cycle.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-23 15:50:17 -04:00