Phase 2 of #3464, following #3465. Rewrites every assertion that read a shipped .cjs/.js file and text-searched it, so the file no longer needs an allow-test-rule exemption. 6 of the 8 files are now marker-free; the ceiling drops 285 -> 278 against a measured 277. A source-grep passes when a STRING is present, not when the code WORKS. It survives a refactor that keeps the string but breaks the behavior, and breaks on a refactor that keeps the behavior but renames the string. Both failure modes are silent about the thing the test claims to protect. That is the anti-pattern ADR-456 and local/no-source-grep exist to prevent. Rewrites, each against the real exported seam: - discuss-mode: calls cmdInitPlanPhase() against a fixture whose config sets workflow.text_mode, asserts the value propagates to its emitted JSON. - effort-surface-axis: runs review-lane invoke against a real project with a fake `claude` PATH shim, asserts the resolved --effort actually lands in the shim's captured argv. - install-minimal-hooks: calls applySettingsJsonHooks() with a hook source missing, asserts it is neither registered nor silently registers wholesale, with sibling present hooks as the positive control. - install: calls the exported inferPreferredRuntime({fs, env, preferredConfigDir}) via its injected fs seam, asserting 'kilo' from both the config-marker and env-var paths. - opencode-permissions: spawns the real installer with a custom config dir, asserts the written opencode.json permission paths are anchored on it. - repo-layout: spawns the installer for copilot local vs global, asserting AGENTS.md is written only in the local case. - runtime-config-adapter-registry: stubs resolveInstallPlan and force-reloads install.js, asserting the runtime's artifact stops being written -- proving install.js genuinely routes through the registry. - runtime-homes-descriptor-drive: calls buildAgentSkillsBlock() for cursor and claude against real fixture SKILL.md files, asserting each runtime's refs land under its own config dir and never the other's. Every rewrite was mutation-checked before its marker was dropped. Because node --test is not runnable locally in this repo, each assertion was replicated in a standalone probe that requires the same module: run green against the real file, then red against a deliberately broken one (text_mode propagation deleted, existsSync guard removed, config dir hardcoded, !isGlobal guard dropped, kilo branch removed, effort resolution bypassed, skills base hardcoded back to .claude), then the production file restored and confirmed byte-identical. An assertion that could not be made to fail would not have shipped -- a behavioral test that passes regardless of correctness is strictly worse than the source-grep it replaces, because it looks rigorous while asserting nothing. Three claims were checked rather than trusted. All three were wrong: - repo-layout's own marker cited #1188 asserting the `!isGlobal` lexical scope was "unprovable at runtime". It is provable: the guard decides whether AGENTS.md is written, which is directly observable. Both directions verified. - The triage for runtime-config-adapter-registry claimed its source-grep was redundant with the file's EXPECTED_TABLE tests, so deletion would be safe. Those tests only exercise resolveInstallPlan() directly and never load bin/install.js, so they do not cover it. A real behavioral assertion was written instead of deleting coverage. - An earlier revision of this change dropped runtime-config-adapter-registry's two markers on the grounds that ESLint stayed silent without them. An adversarial review caught that this was wrong, and it is restored here. The file still genuinely source-greps bin/install.js at two sites; ESLint is silent only because no-source-grep's TEXT_METHODS omits matchAll. Dropping a marker because the linter cannot see the violation is exploiting the blind spot, not resolving it -- and it would go red the moment the rule is widened. Those two assertions are also irreducible: they assert that EVERY inline `runtime === '...'` branch in install.js names a registry-known runtime, and a branch naming an unregistered runtime would simply never execute, so no runtime observation can prove its absence. The markers now say so explicitly. Two coverage gaps in no-source-grep surfaced while doing this, recorded on #3464 rather than fixed here, since widening the rule is its own change with its own blast radius: - TEXT_METHODS omits matchAll, so a matchAll source-grep never trips the rule (the case above). - The path test requires a literal quoted bin/lib/gsd-core/src segment and tracks the binding one hop, so a read through a dynamic path or an intermediate variable evades it. install-minimal-hooks' genuinely load-bearing read at line 975 is itself unmarked for a related reason. install-minimal-hooks therefore keeps its markers too: its remaining real source-grep of bin/install.js (the Codex legacy gsd-update-check migration check, line 975) is outside this issue's 8 sites. It is the last blocker for that file and is a clean follow-up. Closes #3466 Co-authored-by: sim <sim@local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
@@ -1,8 +1,4 @@
|
||||
'use strict';
|
||||
// allow-test-rule: structural-regression-guard (#1188)
|
||||
// Guard-placement verification in bin/install.js requires source-text
|
||||
// analysis; install.js is a non-exportable CLI script and the guard must be
|
||||
// in a specific lexical scope which require()+behavior cannot verify.
|
||||
|
||||
/**
|
||||
* Governance tests for the gsd-core repository root layout.
|
||||
@@ -23,8 +19,10 @@ const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { runGit } = require('./helpers/process-seam.cjs');
|
||||
const { GIT_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
|
||||
const { runGit, runNode } = require('./helpers/process-seam.cjs');
|
||||
const { GIT_TIMEOUT_MS, INSTALL_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
|
||||
const { INSTALL_SCRIPT, installerEnv } = require('./helpers/install-shared.cjs');
|
||||
const { createTempDir, cleanup } = require('./helpers.cjs');
|
||||
|
||||
const ROOT = path.resolve(__dirname, '..');
|
||||
|
||||
@@ -53,70 +51,61 @@ test('repo-layout: root AGENTS.md is not git-tracked — no ad-hoc AI instructio
|
||||
);
|
||||
});
|
||||
|
||||
test('repo-layout: installer writes AGENTS.md only for local Copilot scope (not global), confirming the commit risk is scoped', () => {
|
||||
// Verify the installer source encodes the "!isGlobal" guard that restricts
|
||||
// AGENTS.md emission to local installs. If that guard were removed, the file
|
||||
// could be silently created in any directory the installer runs from,
|
||||
// including the repo root during development. This test is a static read of
|
||||
// the install source — it does not execute the installer.
|
||||
//
|
||||
// Why structural rather than a fixed-window regex: a 200-char sliding-window
|
||||
// regex between `if (!isGlobal)` and the assignment produces false failures
|
||||
// on semantically-equivalent refactors (early-return guards, added comments,
|
||||
// interposed conditions that push the tokens apart). The structural approach
|
||||
// instead verifies that the agentsMdPath assignment appears INSIDE the body
|
||||
// of the `if (!isGlobal)` block in the copilot-instructions surface handler
|
||||
// — which is the invariant that actually matters.
|
||||
const installJs = fs.readFileSync(path.join(ROOT, 'bin', 'install.js'), 'utf8');
|
||||
|
||||
// Step 1: Locate the copilot-instructions surface block.
|
||||
const copilotBlockStart = installJs.indexOf("plan.installSurface === 'copilot-instructions'");
|
||||
assert.ok(
|
||||
copilotBlockStart !== -1,
|
||||
"bin/install.js must contain a 'copilot-instructions' surface handler; " +
|
||||
"the Copilot AGENTS.md guard lives inside it.",
|
||||
);
|
||||
|
||||
// Step 2: Slice to the next installSurface branch so we don't accidentally
|
||||
// match tokens from a sibling surface handler.
|
||||
const nextSurface = installJs.indexOf('plan.installSurface ===', copilotBlockStart + 1);
|
||||
const copilotBlock = installJs.substring(
|
||||
copilotBlockStart,
|
||||
nextSurface > copilotBlockStart ? nextSurface : copilotBlockStart + 5000,
|
||||
);
|
||||
|
||||
// Step 3: Find the `if (!isGlobal)` guard inside the copilot block.
|
||||
const guardIdx = copilotBlock.indexOf('if (!isGlobal)');
|
||||
assert.ok(
|
||||
guardIdx !== -1,
|
||||
'bin/install.js copilot-instructions surface handler must contain an `if (!isGlobal)` guard; ' +
|
||||
'removing that guard would allow a local Copilot install to silently create AGENTS.md ' +
|
||||
'in any working directory, including this repo checkout.',
|
||||
);
|
||||
|
||||
// Step 4: Walk the brace tree to extract the body of the `if (!isGlobal)` block.
|
||||
const openBrace = copilotBlock.indexOf('{', guardIdx);
|
||||
assert.ok(openBrace !== -1, 'if (!isGlobal) guard must have an opening brace');
|
||||
let depth = 0;
|
||||
let i = openBrace;
|
||||
while (i < copilotBlock.length) {
|
||||
if (copilotBlock[i] === '{') depth++;
|
||||
else if (copilotBlock[i] === '}') {
|
||||
depth--;
|
||||
if (depth === 0) break;
|
||||
}
|
||||
i++;
|
||||
test('repo-layout: installer writes AGENTS.md for a LOCAL Copilot install (cwd), confirming the artifact is scoped', () => {
|
||||
// Behavioral replacement for a source-grep/brace-walk assertion (#3466):
|
||||
// runs the REAL installer (`bin/install.js --copilot --local`) with its cwd
|
||||
// pointed at an isolated temp dir, and asserts AGENTS.md is actually written
|
||||
// there. This directly observes the file-output side effect the guard
|
||||
// controls, rather than parsing install.js's lexical structure.
|
||||
const localRoot = createTempDir('gsd-repo-layout-copilot-local-');
|
||||
try {
|
||||
const result = runNode(
|
||||
[INSTALL_SCRIPT, '--copilot', '--local'],
|
||||
{ cwd: localRoot, env: installerEnv({ HOME: localRoot, USERPROFILE: localRoot }), timeoutMs: INSTALL_TIMEOUT_MS },
|
||||
);
|
||||
assert.strictEqual(
|
||||
result.exitCode, 0,
|
||||
`local Copilot install must exit 0\nstdout: ${result.stdout}\nstderr: ${result.stderr}`
|
||||
);
|
||||
assert.ok(
|
||||
fs.existsSync(path.join(localRoot, 'AGENTS.md')),
|
||||
'a LOCAL Copilot install (issue #786) must write AGENTS.md to its cwd — ' +
|
||||
'if this ever stops happening, Copilot CLI loses its primary repo-root instructions file',
|
||||
);
|
||||
} finally {
|
||||
cleanup(localRoot);
|
||||
}
|
||||
const guardBody = copilotBlock.substring(openBrace, i + 1);
|
||||
});
|
||||
|
||||
// Step 5: Assert the repo-root AGENTS.md write site lives inside the guard body.
|
||||
assert.ok(
|
||||
guardBody.includes('agentsMdPath = path.join(process.cwd()'),
|
||||
'bin/install.js must assign `agentsMdPath = path.join(process.cwd(), ...)` INSIDE the ' +
|
||||
'`if (!isGlobal)` block in the copilot-instructions surface handler. ' +
|
||||
'If this assignment moves outside that block the installer would unconditionally create ' +
|
||||
'AGENTS.md in the working directory on every Copilot install, including repo-root runs.',
|
||||
);
|
||||
test('repo-layout: installer does NOT write AGENTS.md for a GLOBAL Copilot install, confirming the commit risk is scoped', () => {
|
||||
// Companion to the local-install test above (#3466): a GLOBAL Copilot
|
||||
// install is already covered by ~/.copilot/copilot-instructions.md (per
|
||||
// the comment at the `if (!isGlobal)` guard's call site in bin/install.js),
|
||||
// so it must NOT also write a repo-root AGENTS.md. Both the cwd AND the
|
||||
// global --config-dir are isolated temp dirs distinct from this checkout,
|
||||
// so even if the guard under test were broken, this run cannot pollute the
|
||||
// real repository root.
|
||||
const globalCwd = createTempDir('gsd-repo-layout-copilot-global-cwd-');
|
||||
const globalConfigDir = createTempDir('gsd-repo-layout-copilot-global-config-');
|
||||
try {
|
||||
const result = runNode(
|
||||
[INSTALL_SCRIPT, '--copilot', '--global', '--config-dir', globalConfigDir],
|
||||
{ cwd: globalCwd, env: installerEnv({ HOME: globalCwd, USERPROFILE: globalCwd }), timeoutMs: INSTALL_TIMEOUT_MS },
|
||||
);
|
||||
assert.strictEqual(
|
||||
result.exitCode, 0,
|
||||
`global Copilot install must exit 0\nstdout: ${result.stdout}\nstderr: ${result.stderr}`
|
||||
);
|
||||
assert.equal(
|
||||
fs.existsSync(path.join(globalCwd, 'AGENTS.md')), false,
|
||||
'a GLOBAL Copilot install must NOT write AGENTS.md to the working directory — ' +
|
||||
'that artifact is scoped to local installs only (issue #786); a global install is ' +
|
||||
'already covered by ~/.copilot/copilot-instructions.md',
|
||||
);
|
||||
} finally {
|
||||
cleanup(globalCwd);
|
||||
cleanup(globalConfigDir);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user