Files
msd-core/tests/npm-integrity-gate.test.cjs
Tom Boucher 89886d90b4 feat(114): npm dependency integrity gate (npm ls invalid/extraneous) (#135)
* test(114): add failing regression tests for npm dependency integrity gate

Adds tests/npm-integrity-gate.test.cjs and four fixture directories under
tests/fixtures/npm-integrity/ covering:
  - clean: matching lockfile and node_modules (expects exit 0)
  - drift: declared vs installed version mismatch (expects exit 1)
    Reproduces the ws 8.20.1 declared / 8.20.0 installed incident shape
    using stable-dep@8.20.1 (package.json) vs stable-dep@8.20.0 (node_modules).
  - extraneous: unlisted package in node_modules (exits 1; exits 0 with --ignore-extraneous)
  - missing: declared package absent from node_modules (exits 1 regardless of flags)

Each test spawns scripts/check-npm-integrity.sh as a subprocess and asserts
on exit code first, then stderr content. Tests are RED at this commit because
the script does not yet exist.

Sources:
  npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
  NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(114): add check-npm-integrity.sh + workspace-aware drift detection

Adds scripts/check-npm-integrity.sh, a Bash script that:
  1. Runs `npm ls --all --json` at the invocation directory
  2. Parses JSON output for invalid, missing, and extraneous package flags
  3. Exits 1 on any finding; emits a structured report to stderr listing offenders
     with both declared and installed versions for invalid packages
  4. Exits 2 on tool error (npm/node not found, JSON parse failure)
  5. Accepts --ignore-extraneous to suppress extraneous-only failures
  6. Documents behaviour in --help output including remediation path

Workspace behaviour: the root package.json in this repo has no "workspaces"
field. npm ls runs at the invocation root and covers that tree only. The sdk/
sub-package is a separate, non-workspace package and is out of scope for a
single invocation. If workspaces are added in future, npm ls will traverse
them automatically (npm >=7).

The drift scenario (ws 8.20.1 declared vs 8.20.0 installed) is reproduced by
using an exact version pin in package.json combined with a mismatched
node_modules/package.json -- npm ls marks this as "invalid" and exits 1.

npm exits 0 for extraneous packages even though they appear in the JSON
"problems" array; this script detects them via JSON parsing regardless of
the npm exit code.

Sources:
  npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
  NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
  OpenSSF Scorecard Pinned-Dependencies:
    https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci(114): wire dependency integrity gate into CI/release/security workflows

Adds a "Dependency integrity gate" step invoking
scripts/check-npm-integrity.sh to three workflows, always after `npm ci`
and before any test or build step:

  .github/workflows/test.yml
    - matrix job: after "Install dependencies" / before "Build SDK dist"
    - coverage job: after "Install dependencies" / before "Build SDK dist"

  .github/workflows/release.yml
    - rc job "Install and test": after npm ci, before npm run test:coverage
    - finalize job "Install and test": after npm ci, before npm run test:coverage

  .github/workflows/security-scan.yml
    - Added setup-node + npm ci + gate before existing source-scan steps
    - Bumped timeout-minutes from 5 to 10 to accommodate the install step

Also adds "check:integrity": "./scripts/check-npm-integrity.sh" to root
package.json scripts for local contributor invocation.

No new workflow files created. All edits extend existing workflows.

Sources:
  npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
  NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(114): document dependency integrity gate in audit runbook

Appends a "Dependency Integrity Verification" section to SECURITY.md
(no docs/runbooks/ directory exists in this repo). Covers:
  - The three detection classes: invalid, missing, extraneous
  - Local invocation: ./scripts/check-npm-integrity.sh + npm run check:integrity
  - Remediation: rm -rf node_modules && npm ci
  - Bypass policy: no flag; commit-message documentation required if skipped
  - Scope: root package only (sdk/ is a non-workspace package, out of scope)
  - CI coverage listing

Sources cited:
  NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
  OpenSSF Scorecard Pinned-Dependencies:
    https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#114): npm integrity gate satisfies its own clean/drift/extraneous fixtures

Replace npm-ls-based analysis with pure package-lock.json parsing so the
script runs correctly in CI and test environments where node_modules is not
installed. Key changes:

- Rewrite check-npm-integrity.sh parser to read package-lock.json directly
  instead of spawning `npm ls --all --json`, which required node_modules on
  disk and incorrectly flagged clean/drift fixtures as MISSING.
- Implement a self-contained semver satisfies() covering exact, caret, tilde,
  comparison-operator, and compound ranges — no external semver package needed.
- Update extraneous fixture package-lock.json to include ghost-pkg with
  "extraneous: true" so the lockfile-based detector can identify it.
- Update missing fixture package-lock.json to omit the node_modules/absent-dep
  entry, making the absent-dep MISSING condition derivable from lockfile alone.

All 13 tests (clean ×2, drift ×3, extraneous ×3, missing ×3, help ×2) pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#114): treat transitive deps as non-extraneous in integrity gate

The extraneous check was comparing all lockfile packages against root
package.json declarations only. This caused every transitive dependency
(e.g. hono, ajv, @anthropic-ai/claude-agent-sdk-darwin-arm64) to be
flagged as EXTRANEOUS, producing false-positive failures in CI.

Only packages that npm itself marks with "extraneous: true" in the
lockfile represent genuinely unwanted packages. Transitive dependencies
installed by parent packages are valid and should be skipped.

All 13 existing tests continue to pass; the extraneous fixture still
works because it uses "extraneous: true" explicitly (npm's own marker).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci: retrigger checks after transient git-auth runner failure

The original run for this PR had a single CI job fail with:
"fatal: could not read Username for 'https://github.com': terminal prompts disabled"
That is a hosted-runner infrastructure flake — no code defect. The run
cannot be retried via gh CLI (too old). This empty commit kicks a fresh
full CI cycle.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 15:50:17 -04:00

171 lines
6.5 KiB
JavaScript

'use strict';
/**
* Regression test for #114 — npm dependency integrity gate.
*
* Verifies that scripts/check-npm-integrity.sh correctly detects:
* 1. Clean install — exits 0, no stderr findings
* 2. Version drift — exits 1, stderr names the offending package + both versions
* (reproduces the ws 8.20.1 declared vs 8.20.0 installed incident)
* 3. Extraneous — exits 1 without --ignore-extraneous; exits 0 with it
* 4. Missing — exits 1 regardless of flags
*
* Each fixture lives under tests/fixtures/npm-integrity/<name>/.
* The test spawns the script as a subprocess — no require/import of internals.
*
* Sources:
* - npm CLI docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
* - NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
*/
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const { spawnSync } = require('node:child_process');
const path = require('node:path');
const ROOT = path.resolve(__dirname, '..');
const SCRIPT = path.join(ROOT, 'scripts', 'check-npm-integrity.sh');
const FIXTURES = path.join(__dirname, 'fixtures', 'npm-integrity');
/**
* Run the integrity gate script against a fixture directory.
*
* @param {string} fixtureName - subdirectory under tests/fixtures/npm-integrity/
* @param {string[]} [extraArgs] - additional CLI args passed to the script
* @returns {{ status: number, stdout: string, stderr: string }}
*/
function runGate(fixtureName, extraArgs = []) {
const fixtureDir = path.join(FIXTURES, fixtureName);
const result = spawnSync('bash', [SCRIPT, ...extraArgs], {
cwd: fixtureDir,
encoding: 'utf-8',
timeout: 30_000,
});
return {
status: result.status ?? 1,
stdout: result.stdout ?? '',
stderr: result.stderr ?? '',
};
}
// ─── Scenario 1: Clean ───────────────────────────────────────────────────────
describe('#114: npm integrity gate — clean fixture', () => {
test('exits 0 when install matches lockfile', () => {
const { status } = runGate('clean');
assert.strictEqual(status, 0, 'expected exit 0 for clean install');
});
test('emits no integrity findings to stderr on clean install', () => {
const { stderr } = runGate('clean');
// No "FAIL:" lines expected
assert.ok(
!stderr.includes('FAIL:'),
`expected no FAIL: lines in stderr; got:\n${stderr}`
);
});
});
// ─── Scenario 2: Drift (declared vs installed mismatch) ─────────────────────
// Reproduces: ws 8.20.1 declared in lockfile, 8.20.0 installed in node_modules
// Fixture uses: stable-dep@8.20.1 (declared) vs stable-dep@8.20.0 (installed)
describe('#114: npm integrity gate — drift fixture (declared vs installed mismatch)', () => {
test('exits 1 on version drift', () => {
const { status } = runGate('drift');
assert.strictEqual(status, 1, 'expected exit 1 for version drift');
});
test('stderr names the offending package', () => {
const { stderr } = runGate('drift');
assert.ok(
stderr.includes('stable-dep'),
`expected stderr to name "stable-dep"; got:\n${stderr}`
);
});
test('stderr includes both the declared and installed versions', () => {
const { stderr } = runGate('drift');
assert.ok(
stderr.includes('8.20.0'),
`expected stderr to include installed version "8.20.0"; got:\n${stderr}`
);
assert.ok(
stderr.includes('8.20.1'),
`expected stderr to include declared version "8.20.1"; got:\n${stderr}`
);
});
});
// ─── Scenario 3: Extraneous ──────────────────────────────────────────────────
describe('#114: npm integrity gate — extraneous fixture', () => {
test('exits 1 when extraneous package present (default behavior)', () => {
const { status } = runGate('extraneous');
assert.strictEqual(status, 1, 'expected exit 1 for extraneous package without --ignore-extraneous');
});
test('stderr names the extraneous package', () => {
const { stderr } = runGate('extraneous');
assert.ok(
stderr.includes('ghost-pkg'),
`expected stderr to name "ghost-pkg"; got:\n${stderr}`
);
});
test('exits 0 with --ignore-extraneous flag', () => {
const { status } = runGate('extraneous', ['--ignore-extraneous']);
assert.strictEqual(status, 0, 'expected exit 0 for extraneous package with --ignore-extraneous');
});
});
// ─── Scenario 4: Missing ─────────────────────────────────────────────────────
describe('#114: npm integrity gate — missing fixture', () => {
test('exits 1 when required package is missing from node_modules', () => {
const { status } = runGate('missing');
assert.strictEqual(status, 1, 'expected exit 1 for missing package');
});
test('stderr names the missing package', () => {
const { stderr } = runGate('missing');
assert.ok(
stderr.includes('absent-dep'),
`expected stderr to name "absent-dep"; got:\n${stderr}`
);
});
test('exits 1 even with --ignore-extraneous (missing is not extraneous)', () => {
const { status } = runGate('missing', ['--ignore-extraneous']);
assert.strictEqual(status, 1, 'expected exit 1 for missing package even with --ignore-extraneous');
});
});
// ─── Smoke test: --help ───────────────────────────────────────────────────────
describe('#114: npm integrity gate — --help output', () => {
test('exits 0 with --help flag', () => {
const result = spawnSync('bash', [SCRIPT, '--help'], {
cwd: ROOT,
encoding: 'utf-8',
timeout: 10_000,
});
assert.strictEqual(result.status, 0, '--help should exit 0');
});
test('--help output mentions --ignore-extraneous', () => {
const result = spawnSync('bash', [SCRIPT, '--help'], {
cwd: ROOT,
encoding: 'utf-8',
timeout: 10_000,
});
// The script routes --help output to stderr (cat >&2). Assert on stderr
// specifically so a stray stdout match cannot produce a false positive.
const helpText = result.stderr ?? '';
assert.ok(
helpText.includes('--ignore-extraneous'),
`expected --help stderr to document --ignore-extraneous; got:\n${helpText}`
);
});
});