* test(114): add failing regression tests for npm dependency integrity gate
Adds tests/npm-integrity-gate.test.cjs and four fixture directories under
tests/fixtures/npm-integrity/ covering:
- clean: matching lockfile and node_modules (expects exit 0)
- drift: declared vs installed version mismatch (expects exit 1)
Reproduces the ws 8.20.1 declared / 8.20.0 installed incident shape
using stable-dep@8.20.1 (package.json) vs stable-dep@8.20.0 (node_modules).
- extraneous: unlisted package in node_modules (exits 1; exits 0 with --ignore-extraneous)
- missing: declared package absent from node_modules (exits 1 regardless of flags)
Each test spawns scripts/check-npm-integrity.sh as a subprocess and asserts
on exit code first, then stderr content. Tests are RED at this commit because
the script does not yet exist.
Sources:
npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(114): add check-npm-integrity.sh + workspace-aware drift detection
Adds scripts/check-npm-integrity.sh, a Bash script that:
1. Runs `npm ls --all --json` at the invocation directory
2. Parses JSON output for invalid, missing, and extraneous package flags
3. Exits 1 on any finding; emits a structured report to stderr listing offenders
with both declared and installed versions for invalid packages
4. Exits 2 on tool error (npm/node not found, JSON parse failure)
5. Accepts --ignore-extraneous to suppress extraneous-only failures
6. Documents behaviour in --help output including remediation path
Workspace behaviour: the root package.json in this repo has no "workspaces"
field. npm ls runs at the invocation root and covers that tree only. The sdk/
sub-package is a separate, non-workspace package and is out of scope for a
single invocation. If workspaces are added in future, npm ls will traverse
them automatically (npm >=7).
The drift scenario (ws 8.20.1 declared vs 8.20.0 installed) is reproduced by
using an exact version pin in package.json combined with a mismatched
node_modules/package.json -- npm ls marks this as "invalid" and exits 1.
npm exits 0 for extraneous packages even though they appear in the JSON
"problems" array; this script detects them via JSON parsing regardless of
the npm exit code.
Sources:
npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
OpenSSF Scorecard Pinned-Dependencies:
https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci(114): wire dependency integrity gate into CI/release/security workflows
Adds a "Dependency integrity gate" step invoking
scripts/check-npm-integrity.sh to three workflows, always after `npm ci`
and before any test or build step:
.github/workflows/test.yml
- matrix job: after "Install dependencies" / before "Build SDK dist"
- coverage job: after "Install dependencies" / before "Build SDK dist"
.github/workflows/release.yml
- rc job "Install and test": after npm ci, before npm run test:coverage
- finalize job "Install and test": after npm ci, before npm run test:coverage
.github/workflows/security-scan.yml
- Added setup-node + npm ci + gate before existing source-scan steps
- Bumped timeout-minutes from 5 to 10 to accommodate the install step
Also adds "check:integrity": "./scripts/check-npm-integrity.sh" to root
package.json scripts for local contributor invocation.
No new workflow files created. All edits extend existing workflows.
Sources:
npm ls docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(114): document dependency integrity gate in audit runbook
Appends a "Dependency Integrity Verification" section to SECURITY.md
(no docs/runbooks/ directory exists in this repo). Covers:
- The three detection classes: invalid, missing, extraneous
- Local invocation: ./scripts/check-npm-integrity.sh + npm run check:integrity
- Remediation: rm -rf node_modules && npm ci
- Bypass policy: no flag; commit-message documentation required if skipped
- Scope: root package only (sdk/ is a non-workspace package, out of scope)
- CI coverage listing
Sources cited:
NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
OpenSSF Scorecard Pinned-Dependencies:
https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(#114): npm integrity gate satisfies its own clean/drift/extraneous fixtures
Replace npm-ls-based analysis with pure package-lock.json parsing so the
script runs correctly in CI and test environments where node_modules is not
installed. Key changes:
- Rewrite check-npm-integrity.sh parser to read package-lock.json directly
instead of spawning `npm ls --all --json`, which required node_modules on
disk and incorrectly flagged clean/drift fixtures as MISSING.
- Implement a self-contained semver satisfies() covering exact, caret, tilde,
comparison-operator, and compound ranges — no external semver package needed.
- Update extraneous fixture package-lock.json to include ghost-pkg with
"extraneous: true" so the lockfile-based detector can identify it.
- Update missing fixture package-lock.json to omit the node_modules/absent-dep
entry, making the absent-dep MISSING condition derivable from lockfile alone.
All 13 tests (clean ×2, drift ×3, extraneous ×3, missing ×3, help ×2) pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(#114): treat transitive deps as non-extraneous in integrity gate
The extraneous check was comparing all lockfile packages against root
package.json declarations only. This caused every transitive dependency
(e.g. hono, ajv, @anthropic-ai/claude-agent-sdk-darwin-arm64) to be
flagged as EXTRANEOUS, producing false-positive failures in CI.
Only packages that npm itself marks with "extraneous: true" in the
lockfile represent genuinely unwanted packages. Transitive dependencies
installed by parent packages are valid and should be skipped.
All 13 existing tests continue to pass; the extraneous fixture still
works because it uses "extraneous: true" explicitly (npm's own marker).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* ci: retrigger checks after transient git-auth runner failure
The original run for this PR had a single CI job fail with:
"fatal: could not read Username for 'https://github.com': terminal prompts disabled"
That is a hosted-runner infrastructure flake — no code defect. The run
cannot be retried via gh CLI (too old). This empty commit kicks a fresh
full CI cycle.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
171 lines
6.5 KiB
JavaScript
171 lines
6.5 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* Regression test for #114 — npm dependency integrity gate.
|
|
*
|
|
* Verifies that scripts/check-npm-integrity.sh correctly detects:
|
|
* 1. Clean install — exits 0, no stderr findings
|
|
* 2. Version drift — exits 1, stderr names the offending package + both versions
|
|
* (reproduces the ws 8.20.1 declared vs 8.20.0 installed incident)
|
|
* 3. Extraneous — exits 1 without --ignore-extraneous; exits 0 with it
|
|
* 4. Missing — exits 1 regardless of flags
|
|
*
|
|
* Each fixture lives under tests/fixtures/npm-integrity/<name>/.
|
|
* The test spawns the script as a subprocess — no require/import of internals.
|
|
*
|
|
* Sources:
|
|
* - npm CLI docs: https://docs.npmjs.com/cli/v10/commands/npm-ls
|
|
* - NIST SSDF PW.4.1: https://csrc.nist.gov/publications/detail/sp/800-218/final
|
|
*/
|
|
|
|
const { describe, test } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const { spawnSync } = require('node:child_process');
|
|
const path = require('node:path');
|
|
|
|
const ROOT = path.resolve(__dirname, '..');
|
|
const SCRIPT = path.join(ROOT, 'scripts', 'check-npm-integrity.sh');
|
|
const FIXTURES = path.join(__dirname, 'fixtures', 'npm-integrity');
|
|
|
|
/**
|
|
* Run the integrity gate script against a fixture directory.
|
|
*
|
|
* @param {string} fixtureName - subdirectory under tests/fixtures/npm-integrity/
|
|
* @param {string[]} [extraArgs] - additional CLI args passed to the script
|
|
* @returns {{ status: number, stdout: string, stderr: string }}
|
|
*/
|
|
function runGate(fixtureName, extraArgs = []) {
|
|
const fixtureDir = path.join(FIXTURES, fixtureName);
|
|
const result = spawnSync('bash', [SCRIPT, ...extraArgs], {
|
|
cwd: fixtureDir,
|
|
encoding: 'utf-8',
|
|
timeout: 30_000,
|
|
});
|
|
return {
|
|
status: result.status ?? 1,
|
|
stdout: result.stdout ?? '',
|
|
stderr: result.stderr ?? '',
|
|
};
|
|
}
|
|
|
|
// ─── Scenario 1: Clean ───────────────────────────────────────────────────────
|
|
|
|
describe('#114: npm integrity gate — clean fixture', () => {
|
|
test('exits 0 when install matches lockfile', () => {
|
|
const { status } = runGate('clean');
|
|
assert.strictEqual(status, 0, 'expected exit 0 for clean install');
|
|
});
|
|
|
|
test('emits no integrity findings to stderr on clean install', () => {
|
|
const { stderr } = runGate('clean');
|
|
// No "FAIL:" lines expected
|
|
assert.ok(
|
|
!stderr.includes('FAIL:'),
|
|
`expected no FAIL: lines in stderr; got:\n${stderr}`
|
|
);
|
|
});
|
|
});
|
|
|
|
// ─── Scenario 2: Drift (declared vs installed mismatch) ─────────────────────
|
|
// Reproduces: ws 8.20.1 declared in lockfile, 8.20.0 installed in node_modules
|
|
// Fixture uses: stable-dep@8.20.1 (declared) vs stable-dep@8.20.0 (installed)
|
|
|
|
describe('#114: npm integrity gate — drift fixture (declared vs installed mismatch)', () => {
|
|
test('exits 1 on version drift', () => {
|
|
const { status } = runGate('drift');
|
|
assert.strictEqual(status, 1, 'expected exit 1 for version drift');
|
|
});
|
|
|
|
test('stderr names the offending package', () => {
|
|
const { stderr } = runGate('drift');
|
|
assert.ok(
|
|
stderr.includes('stable-dep'),
|
|
`expected stderr to name "stable-dep"; got:\n${stderr}`
|
|
);
|
|
});
|
|
|
|
test('stderr includes both the declared and installed versions', () => {
|
|
const { stderr } = runGate('drift');
|
|
assert.ok(
|
|
stderr.includes('8.20.0'),
|
|
`expected stderr to include installed version "8.20.0"; got:\n${stderr}`
|
|
);
|
|
assert.ok(
|
|
stderr.includes('8.20.1'),
|
|
`expected stderr to include declared version "8.20.1"; got:\n${stderr}`
|
|
);
|
|
});
|
|
});
|
|
|
|
// ─── Scenario 3: Extraneous ──────────────────────────────────────────────────
|
|
|
|
describe('#114: npm integrity gate — extraneous fixture', () => {
|
|
test('exits 1 when extraneous package present (default behavior)', () => {
|
|
const { status } = runGate('extraneous');
|
|
assert.strictEqual(status, 1, 'expected exit 1 for extraneous package without --ignore-extraneous');
|
|
});
|
|
|
|
test('stderr names the extraneous package', () => {
|
|
const { stderr } = runGate('extraneous');
|
|
assert.ok(
|
|
stderr.includes('ghost-pkg'),
|
|
`expected stderr to name "ghost-pkg"; got:\n${stderr}`
|
|
);
|
|
});
|
|
|
|
test('exits 0 with --ignore-extraneous flag', () => {
|
|
const { status } = runGate('extraneous', ['--ignore-extraneous']);
|
|
assert.strictEqual(status, 0, 'expected exit 0 for extraneous package with --ignore-extraneous');
|
|
});
|
|
});
|
|
|
|
// ─── Scenario 4: Missing ─────────────────────────────────────────────────────
|
|
|
|
describe('#114: npm integrity gate — missing fixture', () => {
|
|
test('exits 1 when required package is missing from node_modules', () => {
|
|
const { status } = runGate('missing');
|
|
assert.strictEqual(status, 1, 'expected exit 1 for missing package');
|
|
});
|
|
|
|
test('stderr names the missing package', () => {
|
|
const { stderr } = runGate('missing');
|
|
assert.ok(
|
|
stderr.includes('absent-dep'),
|
|
`expected stderr to name "absent-dep"; got:\n${stderr}`
|
|
);
|
|
});
|
|
|
|
test('exits 1 even with --ignore-extraneous (missing is not extraneous)', () => {
|
|
const { status } = runGate('missing', ['--ignore-extraneous']);
|
|
assert.strictEqual(status, 1, 'expected exit 1 for missing package even with --ignore-extraneous');
|
|
});
|
|
});
|
|
|
|
// ─── Smoke test: --help ───────────────────────────────────────────────────────
|
|
|
|
describe('#114: npm integrity gate — --help output', () => {
|
|
test('exits 0 with --help flag', () => {
|
|
const result = spawnSync('bash', [SCRIPT, '--help'], {
|
|
cwd: ROOT,
|
|
encoding: 'utf-8',
|
|
timeout: 10_000,
|
|
});
|
|
assert.strictEqual(result.status, 0, '--help should exit 0');
|
|
});
|
|
|
|
test('--help output mentions --ignore-extraneous', () => {
|
|
const result = spawnSync('bash', [SCRIPT, '--help'], {
|
|
cwd: ROOT,
|
|
encoding: 'utf-8',
|
|
timeout: 10_000,
|
|
});
|
|
// The script routes --help output to stderr (cat >&2). Assert on stderr
|
|
// specifically so a stray stdout match cannot produce a false positive.
|
|
const helpText = result.stderr ?? '';
|
|
assert.ok(
|
|
helpText.includes('--ignore-extraneous'),
|
|
`expected --help stderr to document --ignore-extraneous; got:\n${helpText}`
|
|
);
|
|
});
|
|
});
|