test(#2665): fail the suite when it writes into a live config dir

The recurrence guard, and #2665's own "Optional hardening". This class is silent
by construction: TEST_ENV_BASE cannot see an in-process caller, and CI cannot see
the class at all because CI never has these env vars set. It damages the
developer's machine and reports nothing -- which is how two prior authors each
diagnosed it and fixed only the instance in front of them.

run-tests.cjs snapshots GSD's install footprint in every live runtime config dir
before the suite and re-checks it after, failing the run on a create or a modify.
Roots come from the product's own getGlobalConfigDir, so the guard watches
wherever the product actually points, including through an ambient var.

Scope is ownership-based, not whole-root: the top-level install footprint plus
gsd-prefixed children of dirs GSD shares with the host agent. A config root like
~/.claude is shared, and watching it wholesale would false-positive on the host's
own history.jsonl or settings.json -- a guard that cries wolf gets disabled, and
then catches nothing. The prefix test is load-bearing: the first version watched
only the three top-level entries and MISSED a real leak into skills/gsd-*.

It earned its place immediately -- it is what found the fifth in-process leak in
runtime-artifact-layout.test.cjs, which no amount of reading the review would have
surfaced. Known gap documented in the module: a write to a file GSD does not own
is out of scope by construction.

Lives in scripts/, deliberately NOT scripts/lib/ -- the installer copies that dir
into every user's config dir wholesale while uninstall removes only an allowlist,
so a test-only module there would ship to users and survive uninstall.

Addresses review finding: Minor 8.
This commit is contained in:
0xdhx
2026-07-28 06:05:32 -05:00
parent 36f4f9479a
commit a02462e050
3 changed files with 434 additions and 0 deletions

View File

@@ -39,6 +39,12 @@ const { readdirSync, readFileSync } = require('fs');
const { join, basename } = require('path');
const { execFileSync } = require('child_process');
const { ExitError, runMain } = require('./lib/cli-exit.cjs');
const {
resolveLiveConfigRoots,
snapshotLiveConfig,
diffLiveConfig,
formatViolations,
} = require('./live-config-guard.cjs');
const SUITES = ['all', 'unit', 'integration', 'install', 'security', 'slow', 'qa'];
@@ -965,6 +971,18 @@ function main() {
// job cap. Operator/test override via RUN_TESTS_CHUNK_TIMEOUT_MS.
const chunkTimeoutMs = positiveNumberEnv(process.env.RUN_TESTS_CHUNK_TIMEOUT_MS, 600000);
// #2665: snapshot GSD's install footprint in every LIVE runtime config dir
// before a single test runs. The suite must not write there; the check after
// the chunk loop is what makes a violation loud instead of silent. See
// scripts/lib/live-config-guard.cjs for why the scope is narrow.
const liveConfigGuardEnabled = process.env.GSD_SKIP_LIVE_CONFIG_GUARD !== '1';
let liveConfigRoots = [];
let liveConfigBefore = null;
if (liveConfigGuardEnabled) {
liveConfigRoots = resolveLiveConfigRoots();
if (liveConfigRoots.length > 0) liveConfigBefore = snapshotLiveConfig(liveConfigRoots);
}
let firstFailureExit = 0;
for (let i = 0; i < chunks.length; i++) {
if (chunks.length > 1) {
@@ -1030,6 +1048,17 @@ function main() {
// and the first non-zero exit is reported at the end.
}
}
// #2665: post-suite hermeticity check. Runs even when tests failed — a leaked
// global install is worth reporting alongside the failure that hid it, and
// suppressing it on red would hide it exactly when the suite is least trusted.
if (liveConfigBefore) {
const violations = diffLiveConfig(liveConfigBefore, snapshotLiveConfig(liveConfigRoots));
if (violations.length > 0) {
console.error(formatViolations(violations));
if (firstFailureExit === 0) firstFailureExit = 1;
}
}
if (firstFailureExit !== 0) return firstFailureExit;
}