* test(#4709): assert no shipped surface mints a retired runtime id Extends the #1928 removal guard to the surfaces it structurally could not reach. Its own docblock scopes it to the installer CLI contract and the runtime-name-policy exports; it spawns the installer and inspects module exports, and never reads gsd-core/workflows/**, commands/** or skills/**. Four structural assertions, all RED on next: - every RUNTIME= assignment must name a canonical runtime - the runtime->model-tier table must name only model-catalog runtimes - runtime selection menus must offer only canonical runtimes - config-set runtime / model_profile_overrides examples must be canonical Structural, not textual: each asserts the literal is canonical or the runtime exists as a catalog key, never that the string "gemini" is absent. That string is load-bearing across Antigravity's real on-disk contract, so a fifth test pins that contract from the descriptor (not from a resolved path, which would read $ANTIGRAVITY_CONFIG_DIR and the real $HOME -- the #4312 defect class). An over-broad gemini -> antigravity replacement fails there rather than ships. The menu assertion is scoped by the nearest preceding `question:` matching /runtime/i, because the same file carries a provider menu (anthropic, openai) and a budget menu (high, medium, low) whose labels are single lowercase tokens too and name neither a runtime nor anything the policy should judge. Refs #4709 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4709): stop minting the retired gemini runtime id in workflow text #1928 removed the gemini runtime after Google sunset Gemini CLI on 2026-06-18, but the removal stopped at the installer boundary. Runtime-loaded workflow text kept assigning the id, and the name policy's unknown-id fallbacks then applied a default designed for a never-known FUTURE runtime to an id GSD itself retired: getRuntimeLabel('gemini') is 'Claude Code', getProjectInstructionFile('gemini') is 'AGENTS.md', getGlobalConfigDir('gemini') is ~/.claude. A stale id produced a plausible wrong answer instead of an error. Those fallbacks are DELIBERATE and are left untouched here -- four docblocks document them, src/runtime-name-policy.cts:220-222 calls the label default "the always-safe default, fail-closed", and an existing test in this very suite pins getProjectInstructionFile('gemini') === 'AGENTS.md'. This commit removes the REACHABILITY of the retired id instead: - new-project.md, ingest-docs.md: the runtime-detection cascade mapped /.gemini/ and $GEMINI_CONFIG_DIR to RUNTIME=gemini. Both now map /.gemini/antigravity{,-ide,-cli}/ and $ANTIGRAVITY_CONFIG_DIR to RUNTIME=antigravity, the documented successor. ingest-docs.md was not in the original report; the new structural test found it. - settings-advanced.md: dropped the `gemini` row from the runtime->model-tier table. The model catalog has no gemini runtime (runtimeTierDefaults has 18 keys, none of them gemini), so the row advertised built-in defaults for a runtime whose config key is ignored. Its three model IDs were copied from the `google` PROVIDER preset -- a provider axis rendered as a runtime axis. - settings-advanced.md: removed the `gemini` / "Gemini CLI." runtime menu option and its group listing, so no menu offers a runtime GSD cannot install. - settings-advanced.md: repointed the config examples from `runtime gemini` to `runtime antigravity`, which ships no built-in tier defaults and is therefore the case those overrides actually exist for. - reapply-patches.md: $GEMINI_CONFIG_DIR -> $ANTIGRAVITY_CONFIG_DIR, ~/.gemini/gsd-local-patches -> ~/.gemini/antigravity/gsd-local-patches, and the local scan's bare .gemini -> .agents (Antigravity's localConfigDir). This file is hand-written, so `npm run sync:launcher` never reached it. - update.md: bare ~/.gemini and ./.gemini as GSD config dirs -> the real ~/.gemini/antigravity and ./.agents. Antigravity's own Gemini-family surfaces are untouched by design: ~/.gemini as its configHome parent, ~/.gemini/config for global skills/agents (#3738), hookEvents "gemini", GEMINI.md as its projectInstructionFile, the ~/.gemini/antigravity{,-ide,-cli} ambiguity probes (#1441), and every gemini-* model ID. The launcher's own GEMINI_CONFIG_DIR arm is left to #4632, which absorbed #4347 for it. Refs #4709 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4709): record the Gemini -> Antigravity migration research Primary-source research note behind #4709: the sense taxonomy that separates a runtime-axis `gemini` (stale) from Antigravity's on-disk contract, Google's model IDs, and release history (all load-bearing); the PRESERVE table; the guard-gap analysis; and the per-file inventory with file:line citations. Refs #4709 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4709): keep the legacy patches probe, and stop tripping two lint gates Three review findings, fixed inline. 1. Dropping the global ~/.gemini/gsd-local-patches probe was a regression: a pre-#1928 Gemini CLI install put patches there, and a stranded patches dir is still the user's work. Restored as an explicitly-labelled legacy arm probed AFTER Antigravity, so a live install always wins. This is a directory probe, not a runtime home -- it assigns no runtime id, so it does not reintroduce the defect this PR closes. The $GEMINI_CONFIG_DIR env probe is deliberately NOT restored: that names a runtime config home, which tests/declarative-reference-antigravity.test.cjs:307 pins as ignored. 2. The comment added in (1) originally contained the literal string that the new structural test matches, so the test flagged its own fix's comment as a mint. Reworded. The test was right; a comment in shipped workflow text is as readable to a matcher as code is. 3. docs/research/gemini-to-antigravity-migration.md used the colon slash-form inside a quoted manifest description. lint-docs-command-form rejects it: docs are never passed through the install-time converters, so the colon form names a command no runtime registers. Normalised to the hyphen form. Also verified, rather than assumed: the local scan's .agents entry is unambiguous. Antigravity is the ONLY runtime declaring localConfigDir '.agents' across all 19 capability manifests; grok and codex use ~/.agents as a GLOBAL home, and this scan is local (./$dir). Refs #4709 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4709): retire Gemini CLI from the PR templates, and close two review gaps Adversarial review findings, all fixed inline. 1. All three .github/PULL_REQUEST_TEMPLATE/*.md still offered "Gemini CLI" under "Runtimes tested", and none offered Antigravity. #1928's follow-up dropped Gemini CLI from .github/ISSUE_TEMPLATE/*.yml but missed the PR templates, so every contributor opening a fix/feature/enhancement PR has been asked for two releases which runtime they tested and offered a retired one. Now Antigravity. Guarded by a new assertion: runtime checklist labels in the PR templates must appear in the runtime label table. Proven non-vacuous by reverting one template line and watching the probe report the offender. 2. The #4709 scanning corpus excluded agents/, which also ships runtime-loaded markdown including .compact.md variants. Widened: 318 -> 382 files (+64), zero new offenders, so the gap was coverage rather than a live defect. 3. gsd-core/workflows/sync-skills.md said "grok and gemini have no dedicated installer flag — they alias the codex and claude skills roots respectively." The gemini half is wrong twice over: the runtime is retired, and it never aliased claude -- canonicalizeRuntimeName returns null for it and the caller's fail-closed default merely happens to be claude. Describing that as designed aliasing is exactly the confusion this issue is about. Reduced to grok, which genuinely does alias the codex skills root. 4. The changeset said the runtime was removed in 1.11. It shipped in 1.8.0 (CHANGELOG.md:1023 is the enclosing release heading for the #1928 entry at :1124). Corrected. Refs #4709 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(#4709): follow the corrected sync-skills prose, and refresh the compact baseline Three GREEN-run failures, all caused by this PR's own edits. 1. tests/sync-skills-cross-runtime-refuse.test.cjs pinned the literal phrase "grok and gemini have no dedicated installer flag" — a test REQUIRING shipped text to name a runtime retired in 1.8.0, which is the exact class #4709 exists to remove. The assertion and its rationale comment now track the corrected prose ("grok has no dedicated installer flag"), and the docblock's runtime list drops gemini. The remaining assertions in that file — the guard's exit, the installer pointer, the $DEST reference, guard-before-copy ordering — are untouched, so #3025's contract is otherwise intact. 2. tests/fixtures/compact-content-benchmark-baseline.json drifted because the new-project.md edits changed its compacted size (split "new-project": off 14279 -> 14308, on 12335 -> 12364; aggregate off 107411 -> 107440). Refreshed with `node scripts/benchmark-compact-content.cjs --write`, which is that script's own documented remedy. 3. emitted-attribution reported four grown workflow files with no acknowledgment. Acked below as commit trailers per ADR-3942, which moved the acknowledgment out of tests/emitted-drift-acks/*.json fragments and into the PR's own commit range (read with three-dot base...head). Exactly the four files the gate named are acked — settings-advanced.md and sync-skills.md shrank and are deliberately absent, since a trailer no delta consumed is a staleAcks error. Refs #4709 Emitted-Drift-Ack-Growth: ingest-docs.md — the runtime-detection cascade now names Antigravity's three real directories (/.gemini/antigravity{,-ide,-cli}/) and $ANTIGRAVITY_CONFIG_DIR in place of the single retired /.gemini/ arm and $GEMINI_CONFIG_DIR; three correct paths cost more bytes than the one wrong path they replace. Emitted-Drift-Ack-Growth: new-project.md — same runtime-detection correction as ingest-docs.md, plus dropping "gemini/" from the two GEMINI.md instruction-file sentences so the prose stops contradicting getProjectInstructionFile, which returns AGENTS.md for that retired id. Emitted-Drift-Ack-Growth: reapply-patches.md — restores the legacy ~/.gemini/gsd-local-patches probe as an explicitly-labelled arm after an adversarial-review finding that dropping it stranded a pre-#1928 user's patches, and repoints the env/global probes at Antigravity; the four-line comment is load-bearing, since a bare retired-runtime path with no explanation is exactly what the next reader would delete. Emitted-Drift-Ack-Growth: update.md — bare ~/.gemini and ./.gemini as GSD config dirs are replaced by the real ~/.gemini/antigravity and ./.agents, which are longer strings; no content was added beyond the corrected paths. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(#4709): backfill the changeset PR number pr: 0 -> 4711, now that the PR exists. Never guessed ahead of the number. Refs #4709 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
432 lines
21 KiB
JavaScript
432 lines
21 KiB
JavaScript
/**
|
|
* #1928 — Gemini CLI runtime removal + Antigravity redirect.
|
|
*
|
|
* Google sunset Gemini CLI on 2026-06-18; Antigravity CLI is the official
|
|
* successor. GSD removes the `gemini` runtime and turns `--gemini` into an
|
|
* explicit deprecation redirect (NOT a silent alias — Hyrum's Law, per the
|
|
* issue's rejected alternative #2).
|
|
*
|
|
* Coverage:
|
|
* A. CLI redirect contract (spawned installer): the sunset notice, the
|
|
* no-silent-install failure path, clean UX (no stack trace), and that a
|
|
* co-selected valid runtime still installs.
|
|
* B. The `gemini` runtime is gone from every runtime-name-policy surface.
|
|
* C. Antigravity is PRESERVED everywhere it shared surface with gemini
|
|
* (GEMINI.md instruction file + the shared convertGeminiToolName tool
|
|
* vocabulary) — the shared-infra regression this change had to avoid.
|
|
*/
|
|
|
|
'use strict';
|
|
|
|
const { describe, test, before } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const path = require('node:path');
|
|
const fs = require('node:fs');
|
|
const { spawnSync } = require('node:child_process');
|
|
const { runNode } = require('./helpers/process-seam.cjs');
|
|
const { throwIfFailed } = require('./helpers/git-fixture.cjs');
|
|
|
|
const { createTempDir, cleanup } = require('./helpers.cjs');
|
|
const { runMinimalInstall, BUILD_SCRIPT } = require('./helpers/install-shared.cjs');
|
|
|
|
const ROOT = path.join(__dirname, '..');
|
|
const INSTALL_JS = path.join(ROOT, 'bin', 'install.js');
|
|
|
|
// #3145: class-norm timeout, not a per-suite value — see helpers/timeouts.cjs.
|
|
const { BUILD_TIMEOUT_MS: BUILD_HOOKS_TIMEOUT_MS, INSTALL_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
|
|
|
|
// hooks/dist is gitignored + built; build it idempotently so a real install
|
|
// emits hooks (mirrors golden-install-parity / install-minimal-hooks).
|
|
before(() => {
|
|
throwIfFailed(runNode([BUILD_SCRIPT], { timeoutMs: BUILD_HOOKS_TIMEOUT_MS }), `node ${BUILD_SCRIPT}`);
|
|
});
|
|
|
|
const {
|
|
canonicalizeRuntimeName,
|
|
getRuntimeLabel,
|
|
getGlobalConfigHomeFragment,
|
|
getRuntimeNewProjectCommand,
|
|
runtimeFlags,
|
|
getProjectInstructionFile,
|
|
} = require(path.join(ROOT, 'gsd-core', 'bin', 'lib', 'runtime-name-policy.cjs'));
|
|
|
|
const registry = require(path.join(ROOT, 'gsd-core', 'bin', 'lib', 'capability-registry.cjs'));
|
|
const catalog = require(path.join(ROOT, 'gsd-core', 'bin', 'shared', 'model-catalog.json'));
|
|
|
|
const { convertClaudeAgentToAntigravityAgent } = require('../bin/install.js');
|
|
|
|
// Run the installer as a subprocess with an isolated HOME so no install can
|
|
// touch the real machine. Runtime-config env overrides are stripped so the
|
|
// child resolves config dirs strictly under the temp HOME.
|
|
function runInstaller(args, homeDir) {
|
|
const env = { ...process.env, HOME: homeDir, USERPROFILE: homeDir, GSD_TEST_MODE: '1' };
|
|
for (const k of [
|
|
'CLAUDE_CONFIG_DIR', 'GEMINI_CONFIG_DIR', 'ANTIGRAVITY_CONFIG_DIR',
|
|
'XDG_CONFIG_HOME', 'CODEX_CONFIG_DIR', 'OPENCODE_CONFIG_DIR', 'KILO_CONFIG_DIR',
|
|
]) delete env[k];
|
|
return spawnSync(process.execPath, [INSTALL_JS, ...args], {
|
|
cwd: homeDir, env, encoding: 'utf8', timeout: INSTALL_TIMEOUT_MS,
|
|
});
|
|
}
|
|
|
|
describe('#1928 --gemini CLI deprecation redirect', () => {
|
|
test('--gemini alone prints the sunset notice and exits non-zero without installing', (t) => {
|
|
const home = createTempDir('gsd-1928-gemini-only-');
|
|
t.after(() => cleanup(home));
|
|
|
|
const r = runInstaller(['--gemini'], home);
|
|
const out = `${r.stdout || ''}${r.stderr || ''}`;
|
|
|
|
assert.strictEqual(r.status, 1, 'a bare --gemini must exit 1, not silently fall through to a Claude install');
|
|
assert.match(out, /sunset by Google on 2026-06-18/, 'must cite the 2026-06-18 sunset date');
|
|
assert.match(out, /--antigravity/, 'must redirect the user to --antigravity');
|
|
assert.match(out, /Antigravity CLI \(the official successor\)/);
|
|
// No silent install: nothing was written under the isolated HOME.
|
|
assert.ok(!fs.existsSync(path.join(home, '.gemini')), 'must not create a .gemini runtime dir');
|
|
assert.ok(!fs.existsSync(path.join(home, '.claude')), 'bare --gemini must not silently install Claude');
|
|
});
|
|
|
|
test('--gemini --global still exits 1 (removed flag regardless of scope)', (t) => {
|
|
const home = createTempDir('gsd-1928-gemini-global-');
|
|
t.after(() => cleanup(home));
|
|
|
|
const r = runInstaller(['--gemini', '--global'], home);
|
|
assert.strictEqual(r.status, 1);
|
|
assert.match(`${r.stdout || ''}${r.stderr || ''}`, /sunset by Google on 2026-06-18/);
|
|
});
|
|
|
|
test('the redirect is a clean message — no stack trace leaks to the user', (t) => {
|
|
const home = createTempDir('gsd-1928-gemini-clean-');
|
|
t.after(() => cleanup(home));
|
|
|
|
const r = runInstaller(['--gemini'], home);
|
|
const err = r.stderr || '';
|
|
assert.doesNotMatch(err, /^\s+at .+:\d+:\d+/m, 'no V8 stack frame in redirect output');
|
|
assert.doesNotMatch(err, /\bError:|\bTypeError:|\bthrow\b/, 'no thrown-error prose in redirect output');
|
|
});
|
|
|
|
test('--gemini --help still prints usage (the redirect must not suppress help)', (t) => {
|
|
const home = createTempDir('gsd-1928-gemini-help-');
|
|
t.after(() => cleanup(home));
|
|
|
|
const r = runInstaller(['--gemini', '--help'], home);
|
|
assert.strictEqual(r.status, 0, '--help must exit 0, not the redirect error code');
|
|
assert.match(`${r.stdout || ''}`, /Usage:/, 'the usage/help block must still print to stdout');
|
|
assert.match(`${r.stderr || ''}`, /sunset by Google on 2026-06-18/, 'the notice also prints');
|
|
});
|
|
|
|
test('--gemini --uninstall guides manual cleanup and does NOT uninstall Claude', (t) => {
|
|
const home = createTempDir('gsd-1928-gemini-uninstall-');
|
|
t.after(() => cleanup(home));
|
|
|
|
// Sentinel: a pre-existing Claude install that must survive. Run WITHOUT
|
|
// GSD_TEST_MODE so the real uninstall dispatch is active — the redirect must
|
|
// exit before it (the dispatch defaults an empty selection to 'claude').
|
|
fs.mkdirSync(path.join(home, '.claude'), { recursive: true });
|
|
fs.writeFileSync(path.join(home, '.claude', 'sentinel.txt'), 'keep me');
|
|
const env = { ...process.env, HOME: home, USERPROFILE: home };
|
|
delete env.GSD_TEST_MODE;
|
|
delete env.CLAUDE_CONFIG_DIR;
|
|
const r = spawnSync(process.execPath, [INSTALL_JS, '--gemini', '--uninstall', '--global'], {
|
|
cwd: home, env, encoding: 'utf8', timeout: INSTALL_TIMEOUT_MS,
|
|
});
|
|
|
|
assert.strictEqual(r.status, 1, 'must exit 1, not fall through to the uninstall dispatch');
|
|
assert.match(`${r.stderr || ''}`, /`--gemini --uninstall` is no longer available/, 'must guide manual cleanup');
|
|
assert.ok(fs.existsSync(path.join(home, '.claude', 'sentinel.txt')),
|
|
'the Claude install must NOT be uninstalled (the dispatch defaults empty selection to claude)');
|
|
});
|
|
|
|
test('--gemini co-selected with a valid runtime prints the notice AND still installs the other runtime', (t) => {
|
|
// Hermetic install via the repo harness (explicit --config-dir + isolated
|
|
// HOME). `--gemini` is added alongside a valid runtime (codex): the installer
|
|
// prints the notice but does NOT exit 1 (runMinimalInstall asserts status 0
|
|
// internally) and installs codex.
|
|
const { manifest, root, stderr } = runMinimalInstall({ runtime: 'codex', scope: 'global', extraArgs: ['--gemini'] });
|
|
t.after(() => cleanup(root));
|
|
|
|
assert.match(stderr, /sunset by Google on 2026-06-18/, 'the redirect notice still prints alongside the valid install');
|
|
assert.match(stderr, /--antigravity/);
|
|
assert.ok(manifest, 'the co-selected codex runtime must be installed (manifest written)');
|
|
});
|
|
|
|
test('control: an install WITHOUT --gemini does not print the sunset notice', (t) => {
|
|
const { root, stderr } = runMinimalInstall({ runtime: 'codex', scope: 'global' });
|
|
t.after(() => cleanup(root));
|
|
assert.doesNotMatch(stderr, /sunset by Google/, 'the notice must be conditional on --gemini');
|
|
});
|
|
});
|
|
|
|
describe('#1928 gemini removed from every runtime-name-policy surface', () => {
|
|
test('gemini aliases no longer canonicalize', () => {
|
|
for (const alias of ['gemini', 'gemini-cli', 'gemini-code']) {
|
|
assert.strictEqual(canonicalizeRuntimeName(alias), null, `${alias} must not resolve to a known runtime`);
|
|
}
|
|
});
|
|
|
|
test('gemini falls back on label / config-fragment / new-project surfaces', () => {
|
|
assert.strictEqual(getRuntimeLabel('gemini'), 'Claude Code', 'label table entry removed → fail-closed default');
|
|
assert.strictEqual(getGlobalConfigHomeFragment('gemini'), "'.claude'", 'config-home fragment removed → default');
|
|
assert.strictEqual(getRuntimeNewProjectCommand('gemini'), '/gsd-new-project', 'new-project override removed → default');
|
|
});
|
|
|
|
test('runtimeFlags has no isGemini and covers exactly the non-claude, CLI-installable registry runtimes (count-agnostic)', () => {
|
|
const flags = runtimeFlags('claude');
|
|
assert.ok(!('isGemini' in flags), 'isGemini flag must be gone');
|
|
// The flag set tracks the non-claude registry runtimes (one is<Runtime> per
|
|
// id), so adding a runtime updates the count automatically — no hand-pinned
|
|
// number that would break on the next runtime addition.
|
|
// #2103: registry runtimes with installSurface === 'none' (e.g. vscode —
|
|
// Marketplace/VSIX-distributed, never CLI-installed) have no --<rt> flag
|
|
// by design (see tests/runtime-flags.test.cjs's NON_INSTALLABLE_RUNTIMES)
|
|
// and are excluded from this count too.
|
|
const expectedNonClaudeCount = Object.keys(registry.runtimes)
|
|
.filter((id) => id !== 'claude' && registry.runtimes[id].runtime.installSurface !== 'none')
|
|
.length;
|
|
assert.strictEqual(Object.keys(flags).length, expectedNonClaudeCount,
|
|
'flag count must equal the non-claude, CLI-installable registry runtime count');
|
|
});
|
|
|
|
test('gemini no longer maps to GEMINI.md (defaults to AGENTS.md)', () => {
|
|
assert.strictEqual(getProjectInstructionFile('gemini'), 'AGENTS.md');
|
|
});
|
|
});
|
|
|
|
describe('#1928 Antigravity preserved (shared surface with the removed gemini runtime)', () => {
|
|
test('antigravity still resolves and keeps its GEMINI.md instruction file', () => {
|
|
assert.strictEqual(canonicalizeRuntimeName('antigravity'), 'antigravity');
|
|
assert.strictEqual(canonicalizeRuntimeName('antigravity-cli'), 'antigravity');
|
|
assert.strictEqual(getProjectInstructionFile('antigravity'), 'GEMINI.md',
|
|
'Antigravity CLI reads GEMINI.md as its contextFileName — this mapping must survive gemini removal');
|
|
assert.strictEqual(getRuntimeLabel('antigravity'), 'Antigravity');
|
|
});
|
|
|
|
test('the shared Gemini-backend tool vocabulary still powers Antigravity agent conversion', () => {
|
|
const input = ['---', 'name: gsd-x', 'description: d', 'tools: Read, Write, WebFetch, Skill', '---', '', 'body'].join('\n');
|
|
const toolsLine = convertClaudeAgentToAntigravityAgent(input).split('\n').find((l) => l.startsWith('tools:')) || '';
|
|
assert.ok(toolsLine.includes('read_file'), 'Read → read_file via the retained convertGeminiToolName');
|
|
assert.ok(toolsLine.includes('write_file'), 'Write → write_file');
|
|
assert.ok(toolsLine.includes('web_fetch'), 'WebFetch → web_fetch');
|
|
assert.ok(!/\bskill\b/.test(toolsLine), 'Skill is still excluded (would be an invalid backend tool name)');
|
|
});
|
|
});
|
|
|
|
/**
|
|
* #4709 — the #1928 removal reached the installer and the runtime enum, but runtime-loaded
|
|
* workflow text kept MINTING the retired id: `RUNTIME="gemini"` from `$GEMINI_CONFIG_DIR`, a
|
|
* runtime selection menu offering "Gemini CLI.", a runtime->model-tier table row keyed `gemini`,
|
|
* and `config-set runtime gemini` examples.
|
|
*
|
|
* The name policy's unknown-id fallbacks are DELIBERATE and stay unchanged — see the
|
|
* 'gemini no longer maps to GEMINI.md (defaults to AGENTS.md)' test above, and
|
|
* src/runtime-name-policy.cts:220-222, which calls the label default "the always-safe default,
|
|
* fail-closed". This block removes the REACHABILITY instead: nothing shipped may mint an id the
|
|
* policy does not recognize.
|
|
*
|
|
* Every assertion is STRUCTURAL (the literal must be canonical / the runtime must exist as a
|
|
* catalog key), never "the string gemini is absent" — that string is load-bearing across
|
|
* Antigravity's real on-disk contract, which the final test pins.
|
|
*/
|
|
describe('#4709 no shipped surface mints a retired runtime id', () => {
|
|
/** Recursively collect every `.md` file under `dir` (missing dir -> []). */
|
|
function markdownFilesUnder(dir) {
|
|
if (!fs.existsSync(dir)) return [];
|
|
const out = [];
|
|
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
|
|
const full = path.join(dir, entry.name);
|
|
if (entry.isDirectory()) out.push(...markdownFilesUnder(full));
|
|
else if (entry.isFile() && entry.name.endsWith('.md')) out.push(full);
|
|
}
|
|
return out;
|
|
}
|
|
|
|
/**
|
|
* The shipped, runtime-loaded markdown corpus this block governs. `agents/` is included
|
|
* deliberately: it ships runtime-loaded markdown too, including `.compact.md` variants, and
|
|
* leaving it out was a coverage gap an adversarial review caught.
|
|
*/
|
|
function shippedMarkdown() {
|
|
return [
|
|
...markdownFilesUnder(path.join(ROOT, 'gsd-core', 'workflows')),
|
|
...markdownFilesUnder(path.join(ROOT, 'commands')),
|
|
...markdownFilesUnder(path.join(ROOT, 'skills')),
|
|
...markdownFilesUnder(path.join(ROOT, 'agents')),
|
|
];
|
|
}
|
|
|
|
const relPath = (p) => path.relative(ROOT, p).split(path.sep).join('/');
|
|
const linesOf = (file) => fs.readFileSync(file, 'utf8').split(/\r?\n/);
|
|
|
|
const SETTINGS_ADVANCED = path.join(ROOT, 'gsd-core', 'workflows', 'settings-advanced.md');
|
|
|
|
test('every RUNTIME= assignment in workflow text names a canonical runtime', () => {
|
|
// Bare-literal assignments only: RUNTIME=codex / RUNTIME="codex". A `$VAR`, `$(cmd)` or
|
|
// `<placeholder>` assignment is resolved at runtime and carries no id to validate here.
|
|
const ASSIGN = /\bRUNTIME=("?)([a-z][a-z0-9-]*)\1(?![\w-])/g;
|
|
const offenders = [];
|
|
|
|
for (const file of shippedMarkdown()) {
|
|
linesOf(file).forEach((line, i) => {
|
|
for (const m of line.matchAll(ASSIGN)) {
|
|
if (canonicalizeRuntimeName(m[2]) === null) {
|
|
offenders.push(`${relPath(file)}:${i + 1} mints RUNTIME=${m[2]}`);
|
|
}
|
|
}
|
|
});
|
|
}
|
|
|
|
assert.deepStrictEqual(offenders, [],
|
|
'shipped workflow text assigns a runtime id the name policy does not recognize. A '
|
|
+ 'non-canonical id does not fail loudly — it resolves to Claude Code defaults, so the '
|
|
+ `wrong config dir and instruction file are used silently. Offenders:\n ${offenders.join('\n ')}`);
|
|
});
|
|
|
|
test('the runtime tier table names only runtimes the model catalog defines', () => {
|
|
const known = new Set(Object.keys(catalog.runtimeTierDefaults));
|
|
const offenders = [];
|
|
|
|
// Rows of the runtime -> model-tier table: | `<id>` | `<opus>` | `<sonnet>` | `<haiku>` |
|
|
const ROW = /^\|\s*`([a-z][a-z0-9-]*)`\s*\|/;
|
|
linesOf(SETTINGS_ADVANCED).forEach((line, i) => {
|
|
const m = ROW.exec(line);
|
|
if (m && !known.has(m[1])) {
|
|
offenders.push(`${relPath(SETTINGS_ADVANCED)}:${i + 1} tables runtime \`${m[1]}\``);
|
|
}
|
|
});
|
|
|
|
assert.deepStrictEqual(offenders, [],
|
|
'the runtime->model-tier table documents built-in defaults for a runtime the model catalog '
|
|
+ 'has no entry for, so `config-set runtime <id>` would be ignored. The retired `gemini` '
|
|
+ 'row carried the three model IDs of the `google` PROVIDER preset — a provider axis '
|
|
+ `rendered as a runtime axis. Offenders:\n ${offenders.join('\n ')}`);
|
|
});
|
|
|
|
test('the runtime selection menu offers only canonical runtimes', () => {
|
|
const offenders = [];
|
|
|
|
// Scoped to RUNTIME menus by tracking the nearest preceding `question:`. The same file also
|
|
// carries a provider menu (anthropic / openai) and a budget menu (high / medium / low) whose
|
|
// labels are single lowercase tokens too; neither names a runtime, so validating those
|
|
// against the runtime policy would be a false positive, not extra rigor.
|
|
const QUESTION = /^\s*question:\s*"(.*)"\s*,?\s*$/;
|
|
const OPTION = /\{\s*label:\s*"([a-z][a-z0-9-]*)"\s*,\s*description:/;
|
|
let inRuntimeMenu = false;
|
|
|
|
linesOf(SETTINGS_ADVANCED).forEach((line, i) => {
|
|
const q = QUESTION.exec(line);
|
|
if (q) {
|
|
inRuntimeMenu = /runtime/i.test(q[1]);
|
|
return;
|
|
}
|
|
if (!inRuntimeMenu) return;
|
|
const m = OPTION.exec(line);
|
|
if (m && canonicalizeRuntimeName(m[1]) === null) {
|
|
offenders.push(`${relPath(SETTINGS_ADVANCED)}:${i + 1} offers \`${m[1]}\``);
|
|
}
|
|
});
|
|
|
|
assert.deepStrictEqual(offenders, [],
|
|
'a runtime selection menu offers a runtime GSD does not support — selecting it writes a '
|
|
+ `config value that silently resolves to Claude Code. Offenders:\n ${offenders.join('\n ')}`);
|
|
});
|
|
|
|
test('documented config examples name only canonical runtimes', () => {
|
|
const offenders = [];
|
|
const SET_RUNTIME = /config-set\s+runtime\s+([a-z][a-z0-9-]*)/g;
|
|
const OVERRIDE = /model_profile_overrides\.([a-z][a-z0-9-]*)\./g;
|
|
|
|
for (const file of shippedMarkdown()) {
|
|
linesOf(file).forEach((line, i) => {
|
|
for (const m of line.matchAll(SET_RUNTIME)) {
|
|
if (canonicalizeRuntimeName(m[1]) === null) {
|
|
offenders.push(`${relPath(file)}:${i + 1} \`config-set runtime ${m[1]}\``);
|
|
}
|
|
}
|
|
for (const m of line.matchAll(OVERRIDE)) {
|
|
if (canonicalizeRuntimeName(m[1]) === null) {
|
|
offenders.push(`${relPath(file)}:${i + 1} \`model_profile_overrides.${m[1]}\``);
|
|
}
|
|
}
|
|
});
|
|
}
|
|
|
|
assert.deepStrictEqual(offenders, [],
|
|
'a documented example sets a runtime id the name policy does not recognize; a user who '
|
|
+ `copies it lands on Claude Code defaults. Offenders:\n ${offenders.join('\n ')}`);
|
|
});
|
|
|
|
test("Antigravity's Gemini-family descriptor contract is preserved", () => {
|
|
// Negative space for every test above: Antigravity's real on-disk contract IS Google's
|
|
// Gemini surface, so an over-broad gemini -> antigravity replacement must fail HERE rather
|
|
// than ship. Asserted against the DESCRIPTOR, never a resolved path — getGlobalConfigDir()
|
|
// reads $ANTIGRAVITY_CONFIG_DIR and the real $HOME, which is the #4312 defect class.
|
|
const agy = JSON.parse(
|
|
fs.readFileSync(path.join(ROOT, 'capabilities', 'antigravity', 'capability.json'), 'utf8'),
|
|
);
|
|
assert.strictEqual(agy.runtime.configHome.parent, '.gemini',
|
|
"Antigravity's config home is nested under ~/.gemini");
|
|
assert.strictEqual(agy.runtime.configHome.name, 'antigravity');
|
|
assert.strictEqual(agy.runtime.hookEvents, 'gemini',
|
|
'Antigravity speaks the Gemini hook-event dialect');
|
|
assert.strictEqual(agy.runtime.hostBehaviors.projectInstructionFile, 'GEMINI.md');
|
|
for (const kind of agy.runtime.artifactLayout.global) {
|
|
assert.strictEqual(kind.home, '.gemini/config',
|
|
'global skills/agents install to ~/.gemini/config, the dir agy scans (#3738)');
|
|
}
|
|
|
|
assert.ok(Object.prototype.hasOwnProperty.call(catalog.runtimeTierDefaults, 'antigravity'),
|
|
'antigravity must remain a model-catalog runtime');
|
|
|
|
// The three model IDs the stale `gemini` table row carried belong to the google PROVIDER
|
|
// preset and must survive — they name real Google models, not a GSD runtime.
|
|
const google = JSON.stringify(catalog.providerPresets.google);
|
|
for (const model of ['gemini-3.1-pro-preview', 'gemini-3-flash', 'gemini-2.5-flash-lite']) {
|
|
assert.ok(google.includes(model), `google provider preset must still offer ${model}`);
|
|
}
|
|
});
|
|
|
|
test('PR template runtime checklists name only supported runtimes', () => {
|
|
// #1928's follow-up dropped Gemini CLI from .github/ISSUE_TEMPLATE/*.yml but missed the PR
|
|
// templates, which kept offering it under "Runtimes tested" -- a contributor-facing surface
|
|
// still advertising a retired runtime two releases later. Labels here are DISPLAY names
|
|
// ("Claude Code", not "claude"), so they are checked against the label table, not the id set.
|
|
const labels = new Set(
|
|
Object.keys(registry.runtimes).map((id) => getRuntimeLabel(id)),
|
|
);
|
|
// Non-runtime checklist entries that legitimately appear in the same list.
|
|
const NON_RUNTIME = /^(Other:|N\/A\b)/;
|
|
const offenders = [];
|
|
|
|
const templateDir = path.join(ROOT, '.github', 'PULL_REQUEST_TEMPLATE');
|
|
for (const name of fs.readdirSync(templateDir).filter((f) => f.endsWith('.md'))) {
|
|
const file = path.join(templateDir, name);
|
|
const fileLines = fs.readFileSync(file, 'utf8').split(/\r?\n/);
|
|
let inRuntimeSection = false;
|
|
fileLines.forEach((line, i) => {
|
|
if (/^#+\s*Runtimes tested/i.test(line)) {
|
|
inRuntimeSection = true;
|
|
return;
|
|
}
|
|
// The section ends at the next heading or horizontal rule.
|
|
if (inRuntimeSection && /^(#+\s|---\s*$)/.test(line)) {
|
|
inRuntimeSection = false;
|
|
return;
|
|
}
|
|
if (!inRuntimeSection) return;
|
|
const m = /^\s*-\s*\[\s*\]\s*(.+?)\s*$/.exec(line);
|
|
if (!m) return;
|
|
const label = m[1];
|
|
if (NON_RUNTIME.test(label)) return;
|
|
if (!labels.has(label)) {
|
|
offenders.push(`.github/PULL_REQUEST_TEMPLATE/${name}:${i + 1} offers "${label}"`);
|
|
}
|
|
});
|
|
}
|
|
|
|
assert.deepStrictEqual(offenders, [],
|
|
'a PR template asks contributors which runtime they tested and lists one GSD does not '
|
|
+ 'support. Labels must match the runtime label table (src/runtime-name-policy.cts '
|
|
+ `RUNTIME_LABELS), so a retired runtime cannot linger here. Offenders:\n ${offenders.join('\n ')}`);
|
|
});
|
|
});
|