Phase 2 of #3464, following #3465. Rewrites every assertion that read a shipped .cjs/.js file and text-searched it, so the file no longer needs an allow-test-rule exemption. 6 of the 8 files are now marker-free; the ceiling drops 285 -> 278 against a measured 277. A source-grep passes when a STRING is present, not when the code WORKS. It survives a refactor that keeps the string but breaks the behavior, and breaks on a refactor that keeps the behavior but renames the string. Both failure modes are silent about the thing the test claims to protect. That is the anti-pattern ADR-456 and local/no-source-grep exist to prevent. Rewrites, each against the real exported seam: - discuss-mode: calls cmdInitPlanPhase() against a fixture whose config sets workflow.text_mode, asserts the value propagates to its emitted JSON. - effort-surface-axis: runs review-lane invoke against a real project with a fake `claude` PATH shim, asserts the resolved --effort actually lands in the shim's captured argv. - install-minimal-hooks: calls applySettingsJsonHooks() with a hook source missing, asserts it is neither registered nor silently registers wholesale, with sibling present hooks as the positive control. - install: calls the exported inferPreferredRuntime({fs, env, preferredConfigDir}) via its injected fs seam, asserting 'kilo' from both the config-marker and env-var paths. - opencode-permissions: spawns the real installer with a custom config dir, asserts the written opencode.json permission paths are anchored on it. - repo-layout: spawns the installer for copilot local vs global, asserting AGENTS.md is written only in the local case. - runtime-config-adapter-registry: stubs resolveInstallPlan and force-reloads install.js, asserting the runtime's artifact stops being written -- proving install.js genuinely routes through the registry. - runtime-homes-descriptor-drive: calls buildAgentSkillsBlock() for cursor and claude against real fixture SKILL.md files, asserting each runtime's refs land under its own config dir and never the other's. Every rewrite was mutation-checked before its marker was dropped. Because node --test is not runnable locally in this repo, each assertion was replicated in a standalone probe that requires the same module: run green against the real file, then red against a deliberately broken one (text_mode propagation deleted, existsSync guard removed, config dir hardcoded, !isGlobal guard dropped, kilo branch removed, effort resolution bypassed, skills base hardcoded back to .claude), then the production file restored and confirmed byte-identical. An assertion that could not be made to fail would not have shipped -- a behavioral test that passes regardless of correctness is strictly worse than the source-grep it replaces, because it looks rigorous while asserting nothing. Three claims were checked rather than trusted. All three were wrong: - repo-layout's own marker cited #1188 asserting the `!isGlobal` lexical scope was "unprovable at runtime". It is provable: the guard decides whether AGENTS.md is written, which is directly observable. Both directions verified. - The triage for runtime-config-adapter-registry claimed its source-grep was redundant with the file's EXPECTED_TABLE tests, so deletion would be safe. Those tests only exercise resolveInstallPlan() directly and never load bin/install.js, so they do not cover it. A real behavioral assertion was written instead of deleting coverage. - An earlier revision of this change dropped runtime-config-adapter-registry's two markers on the grounds that ESLint stayed silent without them. An adversarial review caught that this was wrong, and it is restored here. The file still genuinely source-greps bin/install.js at two sites; ESLint is silent only because no-source-grep's TEXT_METHODS omits matchAll. Dropping a marker because the linter cannot see the violation is exploiting the blind spot, not resolving it -- and it would go red the moment the rule is widened. Those two assertions are also irreducible: they assert that EVERY inline `runtime === '...'` branch in install.js names a registry-known runtime, and a branch naming an unregistered runtime would simply never execute, so no runtime observation can prove its absence. The markers now say so explicitly. Two coverage gaps in no-source-grep surfaced while doing this, recorded on #3464 rather than fixed here, since widening the rule is its own change with its own blast radius: - TEXT_METHODS omits matchAll, so a matchAll source-grep never trips the rule (the case above). - The path test requires a literal quoted bin/lib/gsd-core/src segment and tracks the binding one hop, so a read through a dynamic path or an intermediate variable evades it. install-minimal-hooks' genuinely load-bearing read at line 975 is itself unmarked for a related reason. install-minimal-hooks therefore keeps its markers too: its remaining real source-grep of bin/install.js (the Codex legacy gsd-update-check migration check, line 975) is outside this issue's 8 sites. It is the last blocker for that file and is a clean follow-up. Closes #3466 Co-authored-by: sim <sim@local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
@@ -35,6 +35,7 @@ const { resolveRuntimeArtifactLayout } = require(
|
||||
path.join(ROOT, 'gsd-core', 'bin', 'lib', 'runtime-artifact-layout.cjs'),
|
||||
);
|
||||
const { getGlobalConfigDir } = require(path.join(ROOT, 'gsd-core', 'bin', 'lib', 'runtime-homes.cjs'));
|
||||
const { createTempDir, cleanup } = require('./helpers.cjs');
|
||||
|
||||
const sorted = (iterable) => [...iterable].sort();
|
||||
|
||||
@@ -492,7 +493,8 @@ describe('issue-57 AC2 — config-mutation dispatch is closed over the explicit
|
||||
// structural guard over bin/install.js source. Behavioral assertions
|
||||
// cannot observe inline `runtime === '...'` config branching, so this enforces that
|
||||
// every inline per-runtime branch references a runtime the adapter registry knows
|
||||
// about — a NEW branch against an unregistered runtime name fails here.
|
||||
// about — a NEW branch against an unregistered runtime name fails here. ESLint cannot
|
||||
// currently see this grep because no-source-grep's TEXT_METHODS omits matchAll (#3464).
|
||||
test('every inline `runtime === "..."` branch references a registry-known runtime', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'bin', 'install.js'), 'utf8');
|
||||
const literals = new Set(
|
||||
@@ -514,7 +516,8 @@ describe('issue-57 AC2 — config-mutation dispatch is closed over the explicit
|
||||
// allow-test-rule: structural-regression-guard (#2103)
|
||||
// VS Code is a registry runtime but is NEVER CLI-installed (Marketplace/VSIX
|
||||
// extension); it must stay fully descriptor-driven — bin/install.js must
|
||||
// never special-case it by name.
|
||||
// never special-case it by name. ESLint cannot currently see this grep because
|
||||
// no-source-grep's TEXT_METHODS omits matchAll (#3464).
|
||||
test('#2103: bin/install.js has ZERO runtime === "vscode" / isVscode branches (vscode stays fully descriptor-driven)', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'bin', 'install.js'), 'utf8');
|
||||
const runtimeComparisons = [...src.matchAll(/runtime === (?:'vscode'|"vscode")/g)];
|
||||
@@ -534,19 +537,63 @@ describe('issue-57 AC2 — config-mutation dispatch is closed over the explicit
|
||||
);
|
||||
});
|
||||
|
||||
// allow-test-rule: structural-regression-guard (#3336)
|
||||
// delegation-presence guard. Catches wholesale removal of the registry
|
||||
// dispatch (a regression to scattered per-runtime config branching).
|
||||
test('bin/install.js requires the config adapter registry and dispatches through it', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'bin', 'install.js'), 'utf8');
|
||||
assert.ok(
|
||||
src.includes('runtime-config-adapter-registry'),
|
||||
'bin/install.js no longer requires the runtime config adapter registry',
|
||||
);
|
||||
assert.ok(
|
||||
src.includes('resolveInstallPlan('),
|
||||
'bin/install.js no longer dispatches config through resolveInstallPlan',
|
||||
);
|
||||
// Behavioral replacement for the delegation-presence source grep (#3466).
|
||||
//
|
||||
// The EXPECTED_TABLE / resolveInstallPlan projection-contract tests above
|
||||
// (`resolveInstallPlan — descriptor-projection contract`) prove resolveInstallPlan
|
||||
// ITSELF maps every registry descriptor correctly — but they call the registry
|
||||
// function directly and never touch bin/install.js, so they cannot by themselves
|
||||
// prove install.js actually CONSULTS it rather than reimplementing an equivalent
|
||||
// per-runtime branch. This test closes that gap: it stubs the registry's
|
||||
// resolveInstallPlan (the SAME module object bin/install.js requires and
|
||||
// destructures) so it reports a different installSurface for every runtime,
|
||||
// re-requires a fresh bin/install.js so its destructured reference picks up the
|
||||
// stub, and asserts a REAL install(false, 'copilot') run STOPS producing the
|
||||
// copilot-instructions.md artifact that installSurface === 'copilot-instructions'
|
||||
// gates directly inside install() (bin/install.js:~12164 — no finishInstall/CLI
|
||||
// orchestration layer involved, so this is reachable from install() alone). If
|
||||
// install.js ever reverts to a `runtime === 'copilot'` inline branch instead of
|
||||
// reading resolveInstallPlan(runtime).installSurface, the stub has no effect on
|
||||
// that branch and the artifact keeps getting written — which is exactly the
|
||||
// divergence this test would then catch (verified by mutating install.js to that
|
||||
// exact inline form during authoring: the assertion below goes red).
|
||||
test('bin/install.js dispatches config through the REAL resolveInstallPlan (stubbing it changes install() output)', () => {
|
||||
const installPath = require.resolve('../bin/install.js');
|
||||
const registryPath = require.resolve('../gsd-core/bin/lib/runtime-config-adapter-registry.cjs');
|
||||
const registryModule = require(registryPath);
|
||||
const originalResolveInstallPlan = registryModule.resolveInstallPlan;
|
||||
|
||||
registryModule.resolveInstallPlan = (runtime) => ({
|
||||
...originalResolveInstallPlan(runtime),
|
||||
// Force every gate bin/install.js checks against
|
||||
// resolveInstallPlan(runtime).installSurface to read as "nothing special for
|
||||
// this runtime" — including copilot's own surface, which the real descriptor
|
||||
// sets to 'copilot-instructions'.
|
||||
installSurface: 'settings-json',
|
||||
});
|
||||
|
||||
delete require.cache[installPath];
|
||||
const stubbedInstaller = require(installPath);
|
||||
|
||||
const tmpDir = createTempDir('gsd-3466-delegation-guard-');
|
||||
const previousCwd = process.cwd();
|
||||
process.chdir(tmpDir);
|
||||
try {
|
||||
const result = stubbedInstaller.install(false, 'copilot');
|
||||
const instructionsPath = path.join(result.configDir, 'copilot-instructions.md');
|
||||
assert.equal(
|
||||
fs.existsSync(instructionsPath), false,
|
||||
'with resolveInstallPlan stubbed to report installSurface: "settings-json" for every '
|
||||
+ 'runtime, install(\'copilot\') must NOT write copilot-instructions.md — if it still '
|
||||
+ 'does, bin/install.js is not actually gating on resolveInstallPlan(runtime) for this '
|
||||
+ 'decision (a regression to scattered per-runtime branching)',
|
||||
);
|
||||
} finally {
|
||||
process.chdir(previousCwd);
|
||||
cleanup(tmpDir);
|
||||
registryModule.resolveInstallPlan = originalResolveInstallPlan;
|
||||
delete require.cache[installPath];
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user