Files
msd-core/tests/adr-15-progress-converge.test.cjs
Tom Boucher b54c1c5848 fix(#4709): retire the Gemini CLI reviewer lane (#4716)
* fix(#4709): retire the Gemini CLI reviewer lane

Google stopped serving Gemini CLI for the free/Pro/Ultra tiers on 2026-06-18 —
the same sunset that removed the gemini RUNTIME in #1928 (shipped 1.8.0). GSD
targets solo developers, so those tiers ARE the user path: the lane spawned
`gemini {{model}} -p -`, a binary that no longer answers for the majority of
users, and five locales documented it as a supported choice.

The lane was re-created after #1928 by the reviewer-lane-as-manifest-data work
(6a9babda69, #2798/#2837, ADR-2782). Per the maintainer that re-creation was an
error in that buildout rather than a considered decision, so this corrects a
mistake and needs no ADR-2782 amendment.

Reviewer roster: 12 lanes / 13 flags -> 11 lanes / 12 flags.

TWO sources of truth had to be removed, not one. Deleting
capabilities/gemini/capability.json left the capability registry at 11 lanes
while src/review-lane-descriptor.cts's hand-maintained REVIEWER_LANES array
still carried its own complete gemini entry at 12 — precisely the disagreement
checkReviewerLaneParity exists to catch. Both are gone; both parity checkers
now run clean against the real tree (lane parity ok/0 violations, docs parity
0 violations).

Surfaces stripped of the dead flag:
- capabilities/gemini/ deleted; registry and capability-matrix regenerated
- src/review-lane-descriptor.cts: REVIEWER_LANES entry, docblock count, and the
  three doc comments that used --gemini as a live example
- commands/gsd/{review,plan-review-convergence,autonomous,progress}.md and the
  four matching skills/*/SKILL.md: argument-hint frontmatter and flag bullets
- gsd-core/workflows/help/modes/{full,full.compact}.md: /gsd-help signatures,
  the detected-CLI list, and the reviewer-title list
- gsd-core/workflows/settings-integrations.md: the integrations wizard no longer
  offers "Gemini" as a model option, and the settable-keys list drops it
- gsd-core/workflows/review.md: the `command -v gemini` probe, the --gemini
  flag, the roster frontmatter, the install pointer to the sunset repo, and the
  jq-less / precedence / self-skip lane lists
- gsd-core/workflows/sync-skills.md: "two runtimes (grok, gemini) resolve to
  ANOTHER runtime's skills root" is now one runtime; gemini never aliased
  anything, it fell through canonicalizeRuntimeName to a fail-closed default
- docs/{CONFIGURATION,COMMANDS,CLI-TOOLS}.md, docs/reference/capability-matrix.md,
  docs/how-to/set-up-cross-ai-review.md — including its `npm install -g
  @google/gemini-cli` instruction and the two rows recommending --gemini
- docs/features/{cross-ai-peer-review,opt-in-parallel-reviewer-lanes}.md as the
  generator inputs behind docs/FEATURES.md, plus the three locale FEATURES.md
  signature lines the docs-parity gate covers (the #2781 class: a flag change
  that never reaches the mirrors)

Counts reconciled against measurement rather than arithmetic: 8 timeout keys of
11 lanes, 11 budget keys, 9 model keys, and four hardcoded literals in
tests/reviewer-lane-declarations.test.cjs (NEW_LANE_ONLY_IDS 5->4, LITERAL_ROSTER
12->11, two roster counts 12->11).

BEHAVIOR CHANGE, accepted deliberately: `gsd config-set review.models.gemini`
now errors with "Unknown config key". An existing key already in
.planning/config.json still parses and is simply never read, so no project fails
to load. This is the repo's own documented policy for exactly this case
(docs/CONFIGURATION.md:327 — "a key left over from a removed reviewer validated
silently and was never read. Such a key is now rejected by config-set"), so no
installer migration ships. Note my first measurement of this was WRONG: I tested
config-get, which reads undeclared keys fine, and generalised. Read and write are
different surfaces and gave different answers.

Antigravity is untouched throughout — its --antigravity/--agy flags,
review.models.agy, ~/.gemini/antigravity configHome, ~/.gemini/config global
skills root (#3738), hookEvents "gemini", GEMINI.md instruction file, and every
gemini-* model id it actually runs on.

Refs #4709

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#4709): changeset for the reviewer-lane retirement

Type Removed: the --gemini flag and its three config keys are user-visible
surface that no longer exists.

Refs #4709

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#4709): close the 24 test failures and the locale-doc gap the gates found

An adversarial review and a full matrix run between them found substantially
more fallout than inspection had. All of it is this PR's own, and all of it is
fixed rather than waved off.

THE MATRIX RUN FOUND 24 FAILURES ACROSS 6 FILES. Inspection had predicted two.
The dominant class was a test helper that looks up a lane by slug and throws
`no declared lane 'gemini'`:

- tests/feat-2483-review-claude-mds-guard.test.cjs (6) — used gemini as the
  "other declared first-party lane" to contrast against claude's env
  suppression. Now qwen, verified from source as a lane that declares no `env`
  (only claude does), so the contrast still holds.
- tests/review-lane-descriptor.test.cjs (6) — the duplicate-flag and
  duplicate-section fixtures deliberately COLLIDED with a real declared lane to
  prove the parity checker reports a duplicate. `--gemini`/`Gemini` no longer
  collide with anything, so the checker reported
  `descriptor_lane_not_in_registry:acme` instead and the tests proved nothing.
  Now collide with `--codex`/`Codex`, reproduced against the real checker.
- tests/review-reviewer-selection.test.cjs (3) — these distinguish KNOWN-but-
  undetected from UNKNOWN. gemini flipped categories, inverting what they
  proved. The known case now uses qwen; `__nope__` stays the unknown fixture.
- tests/review-default-reviewers-resolution.test.cjs (2), and
  tests/settings-integrations.test.cjs (3) — the wizard now offers three
  reviewer CLIs, not four, so the test and its name say three.
- Two count assertions the earlier sweep missed outright:
  reviewer-lane-declarations.test.cjs:359 (`length, 12`) and
  reviewer-docs-parity.test.cjs:681 (`>= 12`).

THE LOCALE-DOC GAP, and why the parity gate stayed green over it. All four
locale mirrors still documented `--gemini` as a live reviewer flag. The
docs-parity checker asserts the PRESENCE of every current flag and never the
ABSENCE of a retired one, so "0 violations" was never evidence those files were
clean — my earlier reading of it as such was wrong. This is the #2781
locale-drift class in the opposite direction. Fixed across 12 locale files:
COMMANDS.md flag lists and table rows, CONFIGURATION.md `review.models.gemini`
rows and reviewer prose, CLI-TOOLS.md config examples, and
set-up-cross-ai-review.md including its install block and its
which-reviewer-to-choose row, which now recommends Antigravity.

ALSO FOUND, and instructive about my own method: docs/CONFIGURATION.md:297 still
carried a `review.models.gemini` row. My sweep had missed it because my grep
excluded lines matching `gemini-[0-9]` to spare Google's model ids — and that
row's example value is `"gemini-2.5-pro"` on the same line. The exclusion built
to avoid false positives created a false negative.

Remaining comment/example sites: src/review-reviewer-selection.cts:309 and
src/config.cts:598 named the dead flag and key as examples;
gsd-core/references/planning-config.md:269 likewise; and
review-reviewer-selection.cts:22 claimed in the PRESENT tense that gemini is a
lane-only reviewer capability. Line 38 of that same docblock says "Before this
phase the five non-runtime reviewers (gemini, ...)" and is left exactly as is —
that is past-tense history, and rewriting it would falsify the record.

Deliberately still deferred to Phase 4, because it is the RUNTIME axis rather
than the reviewer lane: the locale install-on-your-runtime.md `--gemini --global`
instructions, the USER-GUIDE colon-form notes, and the ARCHITECTURE
runtime-detection flag lists.

Both parity checkers green against the real tree; lint:ci exit 0.

Refs #4709

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#4709): backfill the changeset PR number

pr: 0 -> 4716, now that the PR exists. Never guessed ahead of the number.

Refs #4709

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 03:03:44 -04:00

218 lines
8.6 KiB
JavaScript

// allow-test-rule: source-text-is-the-product #1190
// The progress command and next workflow markdown are runtime-loaded contracts.
// Checking their text verifies the shipped slash-command behavior.
'use strict';
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { runNode } = require('./helpers/process-seam.cjs');
const { throwIfFailed } = require('./helpers/git-fixture.cjs');
const { PROBE_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
const REPO_ROOT = path.join(__dirname, '..');
const COMMAND_PATH = path.join(REPO_ROOT, 'commands', 'gsd', 'progress.md');
const WORKFLOW_PATH = path.join(REPO_ROOT, 'gsd-core', 'workflows', 'next.md');
const FULL_MD_PATH = path.join(REPO_ROOT, 'gsd-core', 'workflows', 'help', 'modes', 'full.md');
const COMMANDS_DOC_PATH = path.join(REPO_ROOT, 'docs', 'COMMANDS.md');
const HOW_TO_PATH = path.join(REPO_ROOT, 'docs', 'how-to', 'run-phases-autonomously.md');
const TOOLS = path.join(REPO_ROOT, 'gsd-core', 'bin', 'gsd-tools.cjs');
function read(filePath) {
return fs.readFileSync(filePath, 'utf8');
}
describe('ADR-15: /gsd:progress --next --auto --converge (#1190)', () => {
test('progress command advertises --converge, --auto, and notes --cross-ai alias', () => {
const command = read(COMMAND_PATH);
assert.match(
command,
/^argument-hint:.*--converge/m,
'progress command should advertise --converge in argument-hint',
);
assert.match(
command,
/^argument-hint:.*--auto/m,
'progress command should advertise --auto in argument-hint',
);
assert.match(command, /--cross-ai/, 'progress command should document --cross-ai alias');
assert.match(
command,
/workflow\.plan_review_convergence=true/,
'progress command should mention the convergence feature gate',
);
});
test('next workflow parses converge aliases into a plan strategy', () => {
const workflow = read(WORKFLOW_PATH);
assert.match(workflow, /PLAN_STRATEGY="local"/, 'workflow should default to local planning');
assert.match(workflow, /PLAN_STRATEGY="converge"/, 'workflow should opt into converge planning');
assert.match(workflow, /converge\|cross-ai/, 'workflow should accept --converge and --cross-ai');
});
test('next workflow fails fast when convergence is requested but disabled', () => {
const workflow = read(WORKFLOW_PATH);
assert.match(
workflow,
/config-get workflow\.plan_review_convergence/,
'workflow should check workflow.plan_review_convergence before planning',
);
assert.match(
workflow,
/gsd config-set workflow\.plan_review_convergence true/,
'workflow should print the enable command instead of silently downgrading',
);
});
test('next workflow routes Route 3 through plan-review-convergence when PLAN_STRATEGY=converge', () => {
const workflow = read(WORKFLOW_PATH);
assert.match(
workflow,
/\/gsd:plan-review-convergence/,
'next workflow should reference /gsd:plan-review-convergence for the converge route',
);
assert.match(
workflow,
/PLAN_STRATEGY=converge/,
'next workflow should check PLAN_STRATEGY for the converge override',
);
assert.match(
workflow,
/gsd:plan-phase/,
'local planning path should remain available for default next runs',
);
// Args-forwarding contract: Route 3 invocation must pass ${CONVERGENCE_ARGS} to the convergence
// command — not just route to the command name but actually forward the built args variable.
assert.match(
workflow,
/\/gsd:plan-review-convergence[^\n]*\$\{CONVERGENCE_ARGS\}/,
'Route 3 convergence invocation must include ${CONVERGENCE_ARGS} on the same line as the command',
);
});
test('next workflow forwards reviewer flags and max cycles to convergence', () => {
const workflow = read(WORKFLOW_PATH);
// Non-lane convergence controls remain hand-written literals in the workflow.
const convergenceControls = ['--all', '--text'];
// Reviewer lane flags that were formerly hand-enumerated in the workflow text.
// They must now be DERIVED at runtime via `gsd_run review-lane flags`, not listed.
const formerlyHardcodedLaneFlags = [
'--codex',
'--qwen',
'--claude',
'--opencode',
'--ollama',
'--lm-studio',
'--llama-cpp',
];
// The literal-absence guard below excludes '--claude': the runtime-launcher
// preamble legitimately contains an unrelated "npx ... --claude --local"
// install-runtime flag, so a substring match on '--claude' would false-positive
// against that literal, not against a re-added reviewer-flag list.
const antiParityLaneFlags = formerlyHardcodedLaneFlags.filter((flag) => flag !== '--claude');
assert.match(workflow, /CONVERGENCE_ARGS/, 'workflow should build convergence pass-through args');
assert.match(
workflow,
/gsd_run review-lane flags/,
'workflow should derive reviewer flags from the review-lane roster instead of hand-listing them',
);
for (const flag of convergenceControls) {
assert.ok(workflow.includes(flag), `workflow should pass through ${flag}`);
}
assert.match(workflow, /--max-cycles/, 'workflow should pass through --max-cycles N');
// Anti-parity guard (deliberately inverted polarity): the whole point of the
// review-lane-flags derivation is that reviewer lane flags are declared ONCE
// (in the review-lane roster) and never hand-listed again in workflow prose.
// If a future edit re-adds a hardcoded reviewer-flag list here, that is the
// regression this test exists to catch — so this assertion must FAIL when
// any of these flags reappear as literals in the workflow text.
for (const flag of antiParityLaneFlags) {
assert.ok(
!workflow.includes(flag),
`workflow should NOT hand-enumerate reviewer lane flag ${flag}; it must be derived via review-lane flags`,
);
}
// Behavioral coverage: prove the roster the workflow derives from actually
// yields the flags this test used to hardcode, so the derivation is not vacuous.
const laneFlagsResult = runNode([TOOLS, 'review-lane', 'flags'], { timeoutMs: PROBE_TIMEOUT_MS });
throwIfFailed(laneFlagsResult, `node ${TOOLS} review-lane flags`);
const laneFlags = laneFlagsResult.stdout.split('\n').filter(Boolean);
for (const flag of formerlyHardcodedLaneFlags) {
assert.ok(laneFlags.includes(flag), `review-lane flags should include ${flag}`);
}
});
test('next workflow preserves --auto re-invocation chaining', () => {
const workflow = read(WORKFLOW_PATH);
assert.match(
workflow,
/--auto/,
'workflow should document the --auto chaining behavior',
);
assert.match(
workflow,
/\/gsd:progress --next --auto/,
'workflow should re-invoke /gsd:progress --next --auto for chaining',
);
// Forwarding contract: the --auto re-invocation must explicitly state that --converge/--cross-ai
// and reviewer flags are forwarded — not just re-invoke --auto alone.
assert.match(
workflow,
/\/gsd:progress --next --auto[^\n]*(forwarding|--converge)/,
'workflow --auto re-invocation should document forwarding --converge/--cross-ai and reviewer flags',
);
});
test('full.md documents --converge, --auto, and --cross-ai for /gsd:progress', () => {
const fullMd = read(FULL_MD_PATH);
assert.match(
fullMd,
/\/gsd:progress --next --auto --converge/,
'full.md should show /gsd:progress --next --auto --converge usage',
);
assert.match(
fullMd,
/--auto/,
'full.md should document --auto for progress',
);
assert.match(
fullMd,
/--cross-ai/,
'full.md should mention --cross-ai alias for convergence',
);
});
test('COMMANDS.md documents progress convergence flags and usage', () => {
const commandsDoc = read(COMMANDS_DOC_PATH);
assert.match(
commandsDoc,
/\/gsd-progress --next --auto --converge/,
'COMMANDS.md should show /gsd-progress --next --auto --converge usage example',
);
assert.match(commandsDoc, /--converge/, 'COMMANDS.md should document --converge for progress');
assert.match(commandsDoc, /--cross-ai/, 'COMMANDS.md should document --cross-ai alias for progress');
});
test('how-to shows /gsd-progress --next --auto --converge usage', () => {
const howTo = read(HOW_TO_PATH);
assert.match(
howTo,
/\/gsd-progress --next --auto --converge/,
'how-to should show /gsd-progress --next --auto --converge usage',
);
});
});