test(#3466): replace 8 source-grep assertions with behavioral tests (#3500)

Phase 2 of #3464, following #3465. Rewrites every assertion that read a
shipped .cjs/.js file and text-searched it, so the file no longer needs an
allow-test-rule exemption. 6 of the 8 files are now marker-free; the ceiling
drops 285 -> 278 against a measured 277.

A source-grep passes when a STRING is present, not when the code WORKS. It
survives a refactor that keeps the string but breaks the behavior, and breaks
on a refactor that keeps the behavior but renames the string. Both failure
modes are silent about the thing the test claims to protect. That is the
anti-pattern ADR-456 and local/no-source-grep exist to prevent.

Rewrites, each against the real exported seam:

- discuss-mode: calls cmdInitPlanPhase() against a fixture whose config sets
  workflow.text_mode, asserts the value propagates to its emitted JSON.
- effort-surface-axis: runs review-lane invoke against a real project with a
  fake `claude` PATH shim, asserts the resolved --effort actually lands in the
  shim's captured argv.
- install-minimal-hooks: calls applySettingsJsonHooks() with a hook source
  missing, asserts it is neither registered nor silently registers wholesale,
  with sibling present hooks as the positive control.
- install: calls the exported inferPreferredRuntime({fs, env,
  preferredConfigDir}) via its injected fs seam, asserting 'kilo' from both
  the config-marker and env-var paths.
- opencode-permissions: spawns the real installer with a custom config dir,
  asserts the written opencode.json permission paths are anchored on it.
- repo-layout: spawns the installer for copilot local vs global, asserting
  AGENTS.md is written only in the local case.
- runtime-config-adapter-registry: stubs resolveInstallPlan and force-reloads
  install.js, asserting the runtime's artifact stops being written -- proving
  install.js genuinely routes through the registry.
- runtime-homes-descriptor-drive: calls buildAgentSkillsBlock() for cursor and
  claude against real fixture SKILL.md files, asserting each runtime's refs
  land under its own config dir and never the other's.

Every rewrite was mutation-checked before its marker was dropped. Because
node --test is not runnable locally in this repo, each assertion was
replicated in a standalone probe that requires the same module: run green
against the real file, then red against a deliberately broken one (text_mode
propagation deleted, existsSync guard removed, config dir hardcoded, !isGlobal
guard dropped, kilo branch removed, effort resolution bypassed, skills base
hardcoded back to .claude), then the production file restored and confirmed
byte-identical. An assertion that could not be made to fail would not have
shipped -- a behavioral test that passes regardless of correctness is strictly
worse than the source-grep it replaces, because it looks rigorous while
asserting nothing.

Three claims were checked rather than trusted. All three were wrong:

- repo-layout's own marker cited #1188 asserting the `!isGlobal` lexical scope
  was "unprovable at runtime". It is provable: the guard decides whether
  AGENTS.md is written, which is directly observable. Both directions verified.
- The triage for runtime-config-adapter-registry claimed its source-grep was
  redundant with the file's EXPECTED_TABLE tests, so deletion would be safe.
  Those tests only exercise resolveInstallPlan() directly and never load
  bin/install.js, so they do not cover it. A real behavioral assertion was
  written instead of deleting coverage.
- An earlier revision of this change dropped runtime-config-adapter-registry's
  two markers on the grounds that ESLint stayed silent without them. An
  adversarial review caught that this was wrong, and it is restored here. The
  file still genuinely source-greps bin/install.js at two sites; ESLint is
  silent only because no-source-grep's TEXT_METHODS omits matchAll. Dropping a
  marker because the linter cannot see the violation is exploiting the blind
  spot, not resolving it -- and it would go red the moment the rule is
  widened. Those two assertions are also irreducible: they assert that EVERY
  inline `runtime === '...'` branch in install.js names a registry-known
  runtime, and a branch naming an unregistered runtime would simply never
  execute, so no runtime observation can prove its absence. The markers now
  say so explicitly.

Two coverage gaps in no-source-grep surfaced while doing this, recorded on
#3464 rather than fixed here, since widening the rule is its own change with
its own blast radius:

- TEXT_METHODS omits matchAll, so a matchAll source-grep never trips the rule
  (the case above).
- The path test requires a literal quoted bin/lib/gsd-core/src segment and
  tracks the binding one hop, so a read through a dynamic path or an
  intermediate variable evades it. install-minimal-hooks' genuinely
  load-bearing read at line 975 is itself unmarked for a related reason.

install-minimal-hooks therefore keeps its markers too: its remaining real
source-grep of bin/install.js (the Codex legacy gsd-update-check migration
check, line 975) is outside this issue's 8 sites. It is the last blocker for
that file and is a clean follow-up.

Closes #3466

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-08-14 15:57:25 -04:00
committed by GitHub
parent 895d9df96d
commit 8a6d87538f
10 changed files with 588 additions and 311 deletions

View File

@@ -1170,10 +1170,6 @@ describe('descriptor-driven parity: 13 non-probe registry runtimes × no-env-var
const { describe: __foldDescribe } = require('node:test');
__foldDescribe("folded:bug-3126-global-skills-base-runtime-path (consolidation epic #1969 B3 #1972)", () => {
'use strict';
// allow-test-rule: structural-implementation-guard (see #3126)
// Last three tests read init.cjs source to verify delegation contract to
// runtime-homes.cjs — no behavioral IR exposed yet for this wiring point.
// Regression guard for bug #3126.
//
// buildAgentSkillsBlock() in init.cjs hardcoded `globalSkillsBase` to
@@ -1481,39 +1477,93 @@ describe('getGlobalConfigDir — explicitDir override and opencode/kilo file-pat
});
});
describe('bug #3126: init.cjs uses runtime-homes not hardcoded .claude', () => {
test('init.cjs has no hardcoded globalSkillsBase assignment to ~/.claude/skills', () => {
const fs = require('node:fs');
const src = fs.readFileSync(
path.join(ROOT, 'gsd-core', 'bin', 'lib', 'init.cjs'),
'utf8',
);
assert.ok(
!src.includes("const globalSkillsBase = path.join(os.homedir(), '.claude', 'skills')"),
'init.cjs still assigns globalSkillsBase to hardcoded ~/.claude/skills — fix not applied',
);
describe('bug #3126: buildAgentSkillsBlock resolves the agent-skills path per runtime (not hardcoded .claude)', () => {
// Behavioral replacement (#3466) for the three init.cjs source-grep assertions
// ("no hardcoded ~/.claude/skills assignment", "requires runtime-homes",
// "warning message no longer hardcodes ~/.claude/skills"). Those proved a
// STRING was absent/present in init.cjs's text; they would pass even if
// buildAgentSkillsBlock resolved the WRONG path for a non-claude runtime, as
// long as the literal old hardcoded expression didn't reappear verbatim. This
// drives buildAgentSkillsBlock() itself — the real exported function bug
// #3126 fixed — for two DIFFERENT runtimes with real fixture skill files
// under real per-runtime config dirs, and asserts each resolves under ITS
// OWN runtime's skills dir and never falls back to (or leaks into) the
// other's.
const fs = require('node:fs');
const { buildAgentSkillsBlock } = require(path.join(ROOT, 'gsd-core', 'bin', 'lib', 'init.cjs'));
/**
* Creates a temp config dir with a real `skills/<skillName>/SKILL.md` fixture,
* points `configDirEnvKey` at it for the duration of `fn`, and cleans up
* (including restoring the env var) afterward.
*/
function withSkillFixture(configDirEnvKey, skillName, fn) {
const tmpConfigDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-3126-skills-'));
const skillDir = path.join(tmpConfigDir, 'skills', skillName);
fs.mkdirSync(skillDir, { recursive: true });
fs.writeFileSync(path.join(skillDir, 'SKILL.md'), '# fixture skill\n');
const saved = process.env[configDirEnvKey];
process.env[configDirEnvKey] = tmpConfigDir;
try {
return fn(tmpConfigDir);
} finally {
if (saved === undefined) delete process.env[configDirEnvKey];
else process.env[configDirEnvKey] = saved;
cleanup(tmpConfigDir);
}
}
test('cursor: resolves under CURSOR_CONFIG_DIR/skills, never falls back to .claude/skills', () => {
withSkillFixture('CURSOR_CONFIG_DIR', 'gsd-executor', (tmpConfigDir) => {
const diagnostics = { warnings: [] };
const block = buildAgentSkillsBlock(
{ runtime: 'cursor', agent_skills: { 'gsd-executor': 'global:gsd-executor' } },
'gsd-executor',
tmpConfigDir,
diagnostics,
);
const expectedRef = path.join(tmpConfigDir, 'skills', 'gsd-executor', 'SKILL.md').replace(/\\/g, '/');
assert.ok(block.includes(expectedRef), `expected block to include ${expectedRef}, got: ${block}`);
assert.ok(!block.includes('.claude/skills'), `cursor resolution must not fall back to .claude/skills, got: ${block}`);
assert.deepEqual(diagnostics.warnings, [], `expected no warnings, got: ${JSON.stringify(diagnostics.warnings)}`);
});
});
test('init.cjs requires runtime-homes', () => {
const fs = require('node:fs');
const src = fs.readFileSync(
path.join(ROOT, 'gsd-core', 'bin', 'lib', 'init.cjs'),
'utf8',
);
assert.ok(
src.includes('runtime-homes'),
'init.cjs does not require runtime-homes.cjs',
);
test('claude: resolves under CLAUDE_CONFIG_DIR/skills, never leaks into .cursor/skills', () => {
withSkillFixture('CLAUDE_CONFIG_DIR', 'gsd-executor', (tmpConfigDir) => {
const diagnostics = { warnings: [] };
const block = buildAgentSkillsBlock(
{ runtime: 'claude', agent_skills: { 'gsd-executor': 'global:gsd-executor' } },
'gsd-executor',
tmpConfigDir,
diagnostics,
);
const expectedRef = path.join(tmpConfigDir, 'skills', 'gsd-executor', 'SKILL.md').replace(/\\/g, '/');
assert.ok(block.includes(expectedRef), `expected block to include ${expectedRef}, got: ${block}`);
assert.ok(!block.includes('.cursor/skills'), `claude resolution must not use .cursor/skills, got: ${block}`);
assert.deepEqual(diagnostics.warnings, [], `expected no warnings, got: ${JSON.stringify(diagnostics.warnings)}`);
});
});
test('init.cjs warning message no longer hardcodes ~/.claude/skills', () => {
const fs = require('node:fs');
const src = fs.readFileSync(
path.join(ROOT, 'gsd-core', 'bin', 'lib', 'init.cjs'),
'utf8',
);
assert.ok(
!src.includes("~/.claude/skills/${skillName}/SKILL.md"),
'init.cjs warning message still hardcodes ~/.claude/skills path',
);
test('per-runtime resolution: two different runtimes in the same process each resolve into THEIR OWN config dir, never the other\'s', () => {
// Proves this isn't a single special-cased runtime — cursor and claude,
// driven back-to-back, must never cross-resolve into each other's fixture dir.
withSkillFixture('CURSOR_CONFIG_DIR', 'gsd-executor', (cursorDir) => {
withSkillFixture('CLAUDE_CONFIG_DIR', 'gsd-executor', (claudeDir) => {
const cursorBlock = buildAgentSkillsBlock(
{ runtime: 'cursor', agent_skills: { x: 'global:gsd-executor' } }, 'x', cursorDir, { warnings: [] },
);
const claudeBlock = buildAgentSkillsBlock(
{ runtime: 'claude', agent_skills: { x: 'global:gsd-executor' } }, 'x', claudeDir, { warnings: [] },
);
const cursorPosix = cursorDir.replace(/\\/g, '/');
const claudePosix = claudeDir.replace(/\\/g, '/');
assert.ok(cursorBlock.includes(cursorPosix), `cursor block must reference its own config dir, got: ${cursorBlock}`);
assert.ok(!cursorBlock.includes(claudePosix), `cursor block must not reference claude's config dir, got: ${cursorBlock}`);
assert.ok(claudeBlock.includes(claudePosix), `claude block must reference its own config dir, got: ${claudeBlock}`);
assert.ok(!claudeBlock.includes(cursorPosix), `claude block must not reference cursor's config dir, got: ${claudeBlock}`);
});
});
});
});
});