Files
msd-core/tests/settings-integrations.test.cjs
Tom Boucher b54c1c5848 fix(#4709): retire the Gemini CLI reviewer lane (#4716)
* fix(#4709): retire the Gemini CLI reviewer lane

Google stopped serving Gemini CLI for the free/Pro/Ultra tiers on 2026-06-18 —
the same sunset that removed the gemini RUNTIME in #1928 (shipped 1.8.0). GSD
targets solo developers, so those tiers ARE the user path: the lane spawned
`gemini {{model}} -p -`, a binary that no longer answers for the majority of
users, and five locales documented it as a supported choice.

The lane was re-created after #1928 by the reviewer-lane-as-manifest-data work
(6a9babda69, #2798/#2837, ADR-2782). Per the maintainer that re-creation was an
error in that buildout rather than a considered decision, so this corrects a
mistake and needs no ADR-2782 amendment.

Reviewer roster: 12 lanes / 13 flags -> 11 lanes / 12 flags.

TWO sources of truth had to be removed, not one. Deleting
capabilities/gemini/capability.json left the capability registry at 11 lanes
while src/review-lane-descriptor.cts's hand-maintained REVIEWER_LANES array
still carried its own complete gemini entry at 12 — precisely the disagreement
checkReviewerLaneParity exists to catch. Both are gone; both parity checkers
now run clean against the real tree (lane parity ok/0 violations, docs parity
0 violations).

Surfaces stripped of the dead flag:
- capabilities/gemini/ deleted; registry and capability-matrix regenerated
- src/review-lane-descriptor.cts: REVIEWER_LANES entry, docblock count, and the
  three doc comments that used --gemini as a live example
- commands/gsd/{review,plan-review-convergence,autonomous,progress}.md and the
  four matching skills/*/SKILL.md: argument-hint frontmatter and flag bullets
- gsd-core/workflows/help/modes/{full,full.compact}.md: /gsd-help signatures,
  the detected-CLI list, and the reviewer-title list
- gsd-core/workflows/settings-integrations.md: the integrations wizard no longer
  offers "Gemini" as a model option, and the settable-keys list drops it
- gsd-core/workflows/review.md: the `command -v gemini` probe, the --gemini
  flag, the roster frontmatter, the install pointer to the sunset repo, and the
  jq-less / precedence / self-skip lane lists
- gsd-core/workflows/sync-skills.md: "two runtimes (grok, gemini) resolve to
  ANOTHER runtime's skills root" is now one runtime; gemini never aliased
  anything, it fell through canonicalizeRuntimeName to a fail-closed default
- docs/{CONFIGURATION,COMMANDS,CLI-TOOLS}.md, docs/reference/capability-matrix.md,
  docs/how-to/set-up-cross-ai-review.md — including its `npm install -g
  @google/gemini-cli` instruction and the two rows recommending --gemini
- docs/features/{cross-ai-peer-review,opt-in-parallel-reviewer-lanes}.md as the
  generator inputs behind docs/FEATURES.md, plus the three locale FEATURES.md
  signature lines the docs-parity gate covers (the #2781 class: a flag change
  that never reaches the mirrors)

Counts reconciled against measurement rather than arithmetic: 8 timeout keys of
11 lanes, 11 budget keys, 9 model keys, and four hardcoded literals in
tests/reviewer-lane-declarations.test.cjs (NEW_LANE_ONLY_IDS 5->4, LITERAL_ROSTER
12->11, two roster counts 12->11).

BEHAVIOR CHANGE, accepted deliberately: `gsd config-set review.models.gemini`
now errors with "Unknown config key". An existing key already in
.planning/config.json still parses and is simply never read, so no project fails
to load. This is the repo's own documented policy for exactly this case
(docs/CONFIGURATION.md:327 — "a key left over from a removed reviewer validated
silently and was never read. Such a key is now rejected by config-set"), so no
installer migration ships. Note my first measurement of this was WRONG: I tested
config-get, which reads undeclared keys fine, and generalised. Read and write are
different surfaces and gave different answers.

Antigravity is untouched throughout — its --antigravity/--agy flags,
review.models.agy, ~/.gemini/antigravity configHome, ~/.gemini/config global
skills root (#3738), hookEvents "gemini", GEMINI.md instruction file, and every
gemini-* model id it actually runs on.

Refs #4709

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#4709): changeset for the reviewer-lane retirement

Type Removed: the --gemini flag and its three config keys are user-visible
surface that no longer exists.

Refs #4709

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#4709): close the 24 test failures and the locale-doc gap the gates found

An adversarial review and a full matrix run between them found substantially
more fallout than inspection had. All of it is this PR's own, and all of it is
fixed rather than waved off.

THE MATRIX RUN FOUND 24 FAILURES ACROSS 6 FILES. Inspection had predicted two.
The dominant class was a test helper that looks up a lane by slug and throws
`no declared lane 'gemini'`:

- tests/feat-2483-review-claude-mds-guard.test.cjs (6) — used gemini as the
  "other declared first-party lane" to contrast against claude's env
  suppression. Now qwen, verified from source as a lane that declares no `env`
  (only claude does), so the contrast still holds.
- tests/review-lane-descriptor.test.cjs (6) — the duplicate-flag and
  duplicate-section fixtures deliberately COLLIDED with a real declared lane to
  prove the parity checker reports a duplicate. `--gemini`/`Gemini` no longer
  collide with anything, so the checker reported
  `descriptor_lane_not_in_registry:acme` instead and the tests proved nothing.
  Now collide with `--codex`/`Codex`, reproduced against the real checker.
- tests/review-reviewer-selection.test.cjs (3) — these distinguish KNOWN-but-
  undetected from UNKNOWN. gemini flipped categories, inverting what they
  proved. The known case now uses qwen; `__nope__` stays the unknown fixture.
- tests/review-default-reviewers-resolution.test.cjs (2), and
  tests/settings-integrations.test.cjs (3) — the wizard now offers three
  reviewer CLIs, not four, so the test and its name say three.
- Two count assertions the earlier sweep missed outright:
  reviewer-lane-declarations.test.cjs:359 (`length, 12`) and
  reviewer-docs-parity.test.cjs:681 (`>= 12`).

THE LOCALE-DOC GAP, and why the parity gate stayed green over it. All four
locale mirrors still documented `--gemini` as a live reviewer flag. The
docs-parity checker asserts the PRESENCE of every current flag and never the
ABSENCE of a retired one, so "0 violations" was never evidence those files were
clean — my earlier reading of it as such was wrong. This is the #2781
locale-drift class in the opposite direction. Fixed across 12 locale files:
COMMANDS.md flag lists and table rows, CONFIGURATION.md `review.models.gemini`
rows and reviewer prose, CLI-TOOLS.md config examples, and
set-up-cross-ai-review.md including its install block and its
which-reviewer-to-choose row, which now recommends Antigravity.

ALSO FOUND, and instructive about my own method: docs/CONFIGURATION.md:297 still
carried a `review.models.gemini` row. My sweep had missed it because my grep
excluded lines matching `gemini-[0-9]` to spare Google's model ids — and that
row's example value is `"gemini-2.5-pro"` on the same line. The exclusion built
to avoid false positives created a false negative.

Remaining comment/example sites: src/review-reviewer-selection.cts:309 and
src/config.cts:598 named the dead flag and key as examples;
gsd-core/references/planning-config.md:269 likewise; and
review-reviewer-selection.cts:22 claimed in the PRESENT tense that gemini is a
lane-only reviewer capability. Line 38 of that same docblock says "Before this
phase the five non-runtime reviewers (gemini, ...)" and is left exactly as is —
that is past-tense history, and rewriting it would falsify the record.

Deliberately still deferred to Phase 4, because it is the RUNTIME axis rather
than the reviewer lane: the locale install-on-your-runtime.md `--gemini --global`
instructions, the USER-GUIDE colon-form notes, and the ARCHITECTURE
runtime-detection flag lists.

Both parity checkers green against the real tree; lint:ci exit 0.

Refs #4709

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#4709): backfill the changeset PR number

pr: 0 -> 4716, now that the PR exists. Never guessed ahead of the number.

Refs #4709

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 03:03:44 -04:00

607 lines
27 KiB
JavaScript

'use strict';
// allow-test-rule: source-text-is-the-product
// Reads .md/.json/.yml product files whose deployed text IS what the
// runtime loads — testing text content tests the deployed contract.
/**
* #2529 — /gsd-settings-integrations: configure third-party search and review integrations.
*
* Covers:
* - Artifacts exist (command, workflow, skill stub) with correct frontmatter
* - Workflow references the four search API key fields
* - Workflow exposes review.models.{claude,codex,opencode} routing
* - Workflow exposes agent_skills.<agent-type> injection input
* - #3651: workflow states the registry-derived review.models settable rule (no
* dynamic-pattern claim) and enumerates exactly the registry's settable lanes
* - #3651: workflow prescribes the JSON array agent_skills write form (never a
* comma-joined string); behavioral pins for array/comma/single shapes
* - Masking convention (****last4) is documented in the workflow and the displayed
* confirmation pattern does not echo plaintext
* - config-set round-trips all integration keys through VALID_CONFIG_KEYS,
* dynamic patterns, and the federated capability registry
* - Config merge preserves unrelated keys
* - /gsd:settings confirmation output mentions /gsd:settings-integrations
* - Negative: invalid agent-type name (path traversal / special char) is rejected
* - Negative: malformed review.models key is rejected
* - Logging: plaintext API keys do not appear in any file written under .planning/
* by the config-set flow other than config.json itself
*/
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { createTempProject, cleanup, runGsdTools } = require('./helpers.cjs');
const {
VALID_CONFIG_KEYS,
isValidConfigKey,
} = require('../gsd-core/bin/lib/config-schema.cjs');
const REPO_ROOT = path.join(__dirname, '..');
// #2790: settings-integrations.md was consolidated into config.md as the --integrations flag.
const COMMAND_PATH = path.join(REPO_ROOT, 'commands', 'gsd', 'config.md');
const WORKFLOW_PATH = path.join(REPO_ROOT, 'gsd-core', 'workflows', 'settings-integrations.md');
const SKILL_PATH = path.join(REPO_ROOT, '.claude', 'skills', 'gsd-settings-integrations.md');
const SETTINGS_WORKFLOW_PATH = path.join(REPO_ROOT, 'gsd-core', 'workflows', 'settings.md');
// ─── Artifacts ───────────────────────────────────────────────────────────────
describe('#2529 artifacts', () => {
test('consolidated config.md command exists (#2790: settings-integrations absorbed)', () => {
// #2790: settings-integrations.md was absorbed into config.md as the --integrations flag.
assert.ok(fs.existsSync(COMMAND_PATH), `missing ${COMMAND_PATH}`);
});
test('config.md frontmatter declares name gsd:config and routes to --integrations', () => {
const src = fs.readFileSync(COMMAND_PATH, 'utf-8');
// #2790: consolidated command uses gsd:config name
assert.match(src, /name:\s*gsd:config/);
assert.match(src, /description:\s*.+/);
assert.match(src, /allowed-tools:/);
assert.match(src, /AskUserQuestion/);
});
test('workflow exists at gsd-core/workflows/settings-integrations.md', () => {
assert.ok(fs.existsSync(WORKFLOW_PATH), `missing ${WORKFLOW_PATH}`);
});
test('skill stub or canonical command surface ships (#2790: via config.md --integrations)', () => {
// #2790: The command surface is now config.md + settings-integrations.md workflow.
const hasStub = fs.existsSync(SKILL_PATH);
const hasCanonical =
fs.existsSync(COMMAND_PATH) && fs.existsSync(WORKFLOW_PATH);
assert.ok(
hasStub || hasCanonical,
`neither ${SKILL_PATH} nor the canonical command/workflow pair exists`
);
});
test('config.md routes --integrations to the settings-integrations workflow', () => {
const src = fs.readFileSync(COMMAND_PATH, 'utf-8');
assert.ok(
src.includes('workflows/settings-integrations.md') || src.includes('--integrations'),
'config.md must reference settings-integrations workflow or --integrations flag'
);
});
});
// ─── Content: search API keys ────────────────────────────────────────────────
describe('#2529 workflow — search integrations', () => {
test('workflow references all four search fields', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
for (const key of ['brave_search', 'firecrawl', 'exa_search', 'search_gitignored']) {
assert.ok(src.includes(key), `workflow must reference ${key}`);
}
});
});
// ─── Content: review.models routing ──────────────────────────────────────────
describe('#2529 workflow — review.models routing', () => {
test('workflow references all three reviewer CLIs', () => {
// #4709: the gemini reviewer lane was retired (Google sunset Gemini CLI), and the
// integrations wizard's AskUserQuestion options dropped its "Gemini" choice along with it —
// the workflow's `AskUserQuestion` block at settings-integrations.md:198-200 now offers
// exactly Claude, Codex, OpenCode.
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
for (const cli of ['claude', 'codex', 'opencode']) {
assert.ok(
src.includes(`review.models.${cli}`),
`workflow must reference review.models.${cli}`
);
}
});
test('review.models.<cli> keys validate via the federated capability registry', () => {
// #3651: these pass because the capability registry federates each lane's
// modelConfigKey into the valid-key set — NOT via a dynamicKeyPatterns regex
// (no such pattern exists; see the #3651 describe below).
// #4709: gemini dropped from this list along with the retired lane — review.models.gemini
// no longer validates because REVIEWER_LANES no longer declares a gemini modelConfigKey.
for (const cli of ['claude', 'codex', 'opencode']) {
assert.ok(
isValidConfigKey(`review.models.${cli}`),
`review.models.${cli} must pass isValidConfigKey`
);
}
});
});
// ─── Content: agent_skills.<agent-type> injection ────────────────────────────
describe('#2529 workflow — agent_skills injection', () => {
test('workflow references agent_skills.<agent-type> injection concept', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
assert.ok(src.includes('agent_skills'), 'workflow must reference agent_skills');
assert.ok(
// eslint-disable-next-line local/no-unbounded-quantifier -- parses maintainer-authored workflow markdown, bounded prose, not adversarial input
/agent_skills\.<[^>]+>|agent_skills\.\w+/.test(src),
'workflow must reference agent_skills.<agent-type> or concrete agent_skills.<slug>'
);
});
test('agent_skills.<valid-slug> passes validator', () => {
assert.ok(isValidConfigKey('agent_skills.gsd-executor'));
assert.ok(isValidConfigKey('agent_skills.gsd-planner'));
assert.ok(isValidConfigKey('agent_skills.my_custom_agent'));
});
});
// ─── #3651: prescribed writes must match the real config-set contract ───────
// The set `config-set` actually accepts for review.models.*: the frozen
// first-party registry's configSchema — the exact map isCapabilityConfigKey
// consults (hasOwnProperty), so this helper cannot drift from the validator.
function collectReviewerModelConfigKeys() {
const registry = require('../gsd-core/bin/lib/capability-registry.cjs');
const schema = registry.configSchema || {};
return new Set(
Object.keys(schema)
.filter((k) => k.startsWith('review.models.'))
.map((k) => k.slice('review.models.'.length))
);
}
// Shared shape for the #3651 behavioral rows: write agent_skills for a slug,
// then read back the resolver's structured diagnostic.
function resolveSkillsCount(tmp, slug) {
const diag = runGsdTools(['agent-skills', slug, '--json'], tmp);
assert.ok(diag.success, `agent-skills failed: ${diag.error}`);
return JSON.parse(diag.output);
}
describe('#3651 workflow — review.models settable-set rule', () => {
test('workflow states the registry-derived review.models rule, not a dynamic-pattern claim (#3651)', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
assert.ok(
!src.includes('^review\\.models\\.'),
'workflow must not claim a review.models dynamic-key pattern — dynamicKeyPatterns has no review.models entry'
);
assert.ok(
/modelConfigKey/.test(src),
'workflow must name the per-lane modelConfigKey rule'
);
assert.ok(
/capability registry/i.test(src),
'workflow must state that the settable set is derived from the capability registry'
);
});
test("workflow enumerates exactly the registry's settable review.models lanes (#3651)", () => {
const registryKeys = collectReviewerModelConfigKeys();
assert.ok(
registryKeys.size >= 9,
`expected the shipped model-bearing lane set from the registry, found ${registryKeys.size}`
);
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
const mentioned = new Set(
[...src.matchAll(/review\.models\.([a-zA-Z0-9_-]+)/g)].map((m) => m[1])
);
for (const key of registryKeys) {
assert.ok(
mentioned.has(key),
`workflow must enumerate settable lane review.models.${key} (registry truth)`
);
}
for (const slug of mentioned) {
assert.ok(
registryKeys.has(slug),
`workflow mentions review.models.${slug} but the registry declares no such settable key`
);
}
});
test('keyless reviewer lanes have no settable review.models key (#3651)', (t) => {
// Lanes whose capability declares modelConfigKey: null — the workflow used
// to walk users into writing these keys, and config-set rejects them.
// `cursor` gained a real modelConfigKey (#3653) and is no longer keyless.
for (const keyless of ['qwen', 'coderabbit']) {
assert.ok(
!isValidConfigKey(`review.models.${keyless}`),
`review.models.${keyless} must not validate (lane declares no modelConfigKey)`
);
}
for (const settable of collectReviewerModelConfigKeys()) {
assert.ok(
isValidConfigKey(`review.models.${settable}`),
`review.models.${settable} must validate`
);
}
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r = runGsdTools(['config-set', 'review.models.qwen', 'qwen-model'], tmp);
assert.ok(
!r.success,
'config-set must reject a keyless lane — the exact error the old workflow steered users into'
);
});
});
describe('#3651 workflow — agent_skills array-form write', () => {
test('workflow prescribes the JSON array agent_skills write, not a comma-joined string (#3651)', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
assert.ok(
!src.includes('"<skill-a,skill-b,skill-c>"'),
'the comma-joined string write prescription must be gone — the resolver never splits it'
);
assert.ok(
/config-set agent_skills\.<slug> '\["[^"]{0,80}"(?:,\s*"[^"]{0,80}"){0,20}\]'/.test(src),
'workflow must show the JSON array write form (config-set agent_skills.<slug> \'["…","…"]\')'
);
assert.ok(
/[Ss]plit/.test(src) && /comma/i.test(src),
'workflow must instruct the driving agent to split comma-separated input before the write'
);
});
test('JSON array agent_skills write round-trips and resolves per-element (#3651)', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r = runGsdTools(
['config-set', 'agent_skills.gsd-planner', '["skills/alpha","skills/beta"]'],
tmp
);
assert.ok(r.success, `array-form set failed: ${r.error}`);
const cfg = JSON.parse(fs.readFileSync(path.join(tmp, '.planning', 'config.json'), 'utf-8'));
assert.deepStrictEqual(
cfg.agent_skills?.['gsd-planner'],
['skills/alpha', 'skills/beta'],
'stored value must be the JSON array, element per skill'
);
const parsed = resolveSkillsCount(tmp, 'gsd-planner');
assert.strictEqual(
parsed.skills_count,
2,
`array form must resolve as 2 skill paths, got ${parsed.skills_count}`
);
});
test('comma-joined agent_skills value resolves as ONE path — the hazard the array form avoids (#3651)', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r = runGsdTools(
['config-set', 'agent_skills.gsd-planner', 'skills/alpha,skills/beta'],
tmp
);
assert.ok(r.success, `comma-string set is accepted by config-set (shape is legal): ${r.error}`);
const parsed = resolveSkillsCount(tmp, 'gsd-planner');
assert.strictEqual(
parsed.skills_count,
1,
`a comma-joined string is ONE path (never split) — got ${parsed.skills_count}; if this changes, the workflow prescription and this pin must change together`
);
});
test('single bare-string agent_skills path keeps working (#3651)', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r = runGsdTools(
['config-set', 'agent_skills.gsd-planner', 'skills/solo'],
tmp
);
assert.ok(r.success, `single-string set failed: ${r.error}`);
const parsed = resolveSkillsCount(tmp, 'gsd-planner');
assert.strictEqual(parsed.configured, true, 'a single string path is a configured entry');
assert.strictEqual(parsed.skills_count, 1, 'one string = one skill path');
});
test('one-element array agent_skills write resolves identically to the bare string (#3651)', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r = runGsdTools(
['config-set', 'agent_skills.gsd-planner', '["skills/solo"]'],
tmp
);
assert.ok(r.success, `one-element-array set failed: ${r.error}`);
const cfg = JSON.parse(fs.readFileSync(path.join(tmp, '.planning', 'config.json'), 'utf-8'));
assert.deepStrictEqual(
cfg.agent_skills?.['gsd-planner'],
['skills/solo'],
'one-element array must persist verbatim'
);
const parsed = resolveSkillsCount(tmp, 'gsd-planner');
assert.strictEqual(parsed.configured, true);
assert.strictEqual(parsed.skills_count, 1, 'one-element array = one skill path, same as the bare string');
});
});
// ─── Content: masking ────────────────────────────────────────────────────────
describe('#2529 workflow — API key masking', () => {
test('workflow documents the **** masking convention', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
// Must reference the **** mask pattern
assert.ok(src.includes('****'), 'workflow must document the **** mask pattern');
// Must explicitly state that plaintext is not displayed
assert.ok(
// eslint-disable-next-line local/no-unbounded-quantifier -- parses maintainer-authored workflow markdown, bounded prose, not adversarial input
/never\s+(echo|display|log|show)[^.]*plaintext|plaintext[^.]*never\s+(echo|display|log|shown)|plaintext[^.]*not\s+(echoed|displayed|logged|shown)|not\s+(echoed|displayed|logged|shown)[^.]*plaintext/i.test(src),
'workflow must explicitly forbid displaying plaintext API keys'
);
});
test('workflow shows masked-value confirmation pattern, not raw secrets', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
// The confirmation table in the workflow must describe the masked display
assert.ok(
/\*\*\*\*\w{0,4}|\*\*\*\* *already set|\*\*\*\*<last.?4>/i.test(src),
'workflow must describe a masked confirmation pattern (e.g. ****last4 or **** already set)'
);
});
test('workflow includes a Leave / Replace / Clear flow for already-set keys', () => {
const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
assert.ok(/Leave/i.test(src) && /Replace/i.test(src) && /Clear/i.test(src),
'workflow must offer Leave / Replace / Clear when a key is already set');
});
});
// ─── config-set round-trip ───────────────────────────────────────────────────
describe('#2529 config-set round-trip', () => {
test('brave_search, firecrawl, exa_search, search_gitignored are valid keys', () => {
for (const k of ['brave_search', 'firecrawl', 'exa_search', 'search_gitignored']) {
assert.ok(VALID_CONFIG_KEYS.has(k), `${k} must be in VALID_CONFIG_KEYS`);
}
});
test('config-set writes brave_search, firecrawl, exa_search values to config.json', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r1 = runGsdTools(['config-set', 'brave_search', 'BSKY-111111112222'], tmp);
assert.ok(r1.success, `brave_search set failed: ${r1.error}`);
const r2 = runGsdTools(['config-set', 'firecrawl', 'fc-aaaaaaaabbbbcccc'], tmp);
assert.ok(r2.success, `firecrawl set failed: ${r2.error}`);
const r3 = runGsdTools(['config-set', 'exa_search', 'ex-000011112222dddd'], tmp);
assert.ok(r3.success, `exa_search set failed: ${r3.error}`);
const r4 = runGsdTools(['config-set', 'search_gitignored', 'true'], tmp);
assert.ok(r4.success, `search_gitignored set failed: ${r4.error}`);
const cfg = JSON.parse(fs.readFileSync(path.join(tmp, '.planning', 'config.json'), 'utf-8'));
assert.strictEqual(cfg.brave_search, 'BSKY-111111112222');
assert.strictEqual(cfg.firecrawl, 'fc-aaaaaaaabbbbcccc');
assert.strictEqual(cfg.exa_search, 'ex-000011112222dddd');
assert.ok(
cfg.search_gitignored === true || cfg.search_gitignored === 'true',
`search_gitignored round-trip mismatch: got ${JSON.stringify(cfg.search_gitignored)}`
);
});
test('config-set round-trips review.models.<cli>', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r = runGsdTools(
['config-set', 'review.models.codex', 'codex exec --model gpt-5'],
tmp
);
assert.ok(r.success, `review.models.codex set failed: ${r.error}`);
const cfg = JSON.parse(fs.readFileSync(path.join(tmp, '.planning', 'config.json'), 'utf-8'));
assert.strictEqual(cfg.review?.models?.codex, 'codex exec --model gpt-5');
});
test('config-set round-trips agent_skills.<agent-type> (array form — the shape the workflow prescribes, #3651)', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r = runGsdTools(
['config-set', 'agent_skills.gsd-executor', '["skill-a","skill-b"]'],
tmp
);
assert.ok(r.success, `agent_skills.gsd-executor set failed: ${r.error}`);
const cfg = JSON.parse(fs.readFileSync(path.join(tmp, '.planning', 'config.json'), 'utf-8'));
// #3651: the prescribed write form must persist as a real JSON array — the
// either-shape acceptance this row used to allow hid the comma-string hazard.
assert.deepStrictEqual(
cfg.agent_skills?.['gsd-executor'],
['skill-a', 'skill-b'],
`expected the array form to persist verbatim, got ${JSON.stringify(cfg.agent_skills?.['gsd-executor'])}`
);
});
});
// ─── Config merge preserves unrelated keys ───────────────────────────────────
describe('#2529 config merge safety', () => {
test('setting brave_search preserves unrelated workflow.research key', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
runGsdTools(['config-set', 'workflow.research', 'false'], tmp);
const r = runGsdTools(['config-set', 'brave_search', 'BSKY-preserve-me-9999'], tmp);
assert.ok(r.success, `set failed: ${r.error}`);
const cfg = JSON.parse(fs.readFileSync(path.join(tmp, '.planning', 'config.json'), 'utf-8'));
assert.strictEqual(cfg.workflow?.research, false, 'unrelated workflow.research must be preserved');
assert.strictEqual(cfg.brave_search, 'BSKY-preserve-me-9999');
});
test('setting agent_skills.gsd-executor preserves unrelated review.models.codex', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
runGsdTools(['config-set', 'review.models.codex', 'codex exec'], tmp);
const r = runGsdTools(['config-set', 'agent_skills.gsd-planner', 'a,b'], tmp);
assert.ok(r.success, `set failed: ${r.error}`);
const cfg = JSON.parse(fs.readFileSync(path.join(tmp, '.planning', 'config.json'), 'utf-8'));
assert.strictEqual(cfg.review?.models?.codex, 'codex exec', 'unrelated review.models.codex must be preserved');
assert.ok(cfg.agent_skills?.['gsd-planner'], 'agent_skills.gsd-planner must be set');
});
});
// ─── /gsd-settings mentions /gsd-settings-integrations ──────────────────────
describe('#2529 /gsd-settings mentions new command', () => {
test('settings workflow mentions canonical /gsd-config --integrations', () => {
const src = fs.readFileSync(SETTINGS_WORKFLOW_PATH, 'utf-8');
assert.ok(
src.includes('/gsd:config --integrations'),
'settings.md must mention /gsd:config --integrations'
);
assert.ok(
!src.includes('/gsd-settings-integrations'),
'settings.md must not mention the legacy /gsd-settings-integrations variant'
);
});
});
// ─── Negative scenarios ──────────────────────────────────────────────────────
describe('#2529 negative — invalid inputs rejected', () => {
test('invalid agent-type with path separators is rejected by validator', () => {
assert.ok(!isValidConfigKey('agent_skills.../etc/passwd'),
'agent_skills.../etc/passwd must be rejected');
assert.ok(!isValidConfigKey('agent_skills./evil'),
'agent_skills./evil must be rejected');
assert.ok(!isValidConfigKey('agent_skills.a b c'),
'agent_skills with spaces must be rejected');
assert.ok(!isValidConfigKey('agent_skills.$(whoami)'),
'agent_skills with shell metacharacters must be rejected');
});
test('config-set rejects agent_skills with path traversal', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const r = runGsdTools(['config-set', 'agent_skills.../etc/passwd', 'x'], tmp);
assert.ok(!r.success, 'config-set must reject path-traversal agent-type slug');
});
test('malformed review.models entry (empty cli) is rejected', () => {
assert.ok(!isValidConfigKey('review.models.'),
'review.models. (empty) must be rejected');
assert.ok(!isValidConfigKey('review.models'),
'review.models (no cli) must be rejected');
assert.ok(!isValidConfigKey('review.models.claude/../../x'),
'review.models with path separators must be rejected');
});
});
// ─── Security: plaintext never leaks to disk outside config.json ─────────────
describe('#2529 security — plaintext containment', () => {
test('after setting brave_search, plaintext appears only in config.json', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
// Build sentinel via concat so secret-scanners do not flag the literal.
const marker = ['MASKCHECK', '9f3a7b2c'].join('-');
const r = runGsdTools(['config-set', 'brave_search', marker], tmp);
assert.ok(r.success, `set failed: ${r.error}`);
const planning = path.join(tmp, '.planning');
const hits = [];
function walk(dir) {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) { walk(full); continue; }
if (!entry.isFile()) continue;
let buf;
try { buf = fs.readFileSync(full, 'utf-8'); } catch { continue; }
if (buf.includes(marker)) hits.push(full);
}
}
walk(planning);
assert.deepStrictEqual(
hits.map(h => path.basename(h)).sort(),
['config.json'],
`plaintext marker leaked outside config.json: found in ${hits.join(', ')}`
);
});
test('config-set does not echo plaintext secret on stdout/stderr', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const marker = ['ECHOCHECK', '77aa33bb'].join('-');
const r = runGsdTools(['config-set', 'brave_search', marker], tmp);
assert.ok(r.success, `set failed: ${r.error}`);
const combined = `${r.output || ''}\n${r.error || ''}`;
assert.ok(
!combined.includes(marker),
`config-set output must not echo the plaintext marker. Got:\n${combined}`
);
});
test('config-get masks secrets and never echoes plaintext for brave_search/firecrawl/exa_search', (t) => {
const tmp = createTempProject();
t.after(() => cleanup(tmp));
runGsdTools(['config-ensure-section'], tmp);
const cases = [
{ key: 'brave_search', marker: ['GETMASK', 'brave', 'aaaa1111'].join('-') },
{ key: 'firecrawl', marker: ['GETMASK', 'fc', 'bbbb2222'].join('-') },
{ key: 'exa_search', marker: ['GETMASK', 'ex', 'cccc3333'].join('-') },
];
for (const { key, marker } of cases) {
const set = runGsdTools(['config-set', key, marker], tmp);
assert.ok(set.success, `${key} set failed: ${set.error}`);
const get = runGsdTools(['config-get', key], tmp);
assert.ok(get.success, `${key} get failed: ${get.error}`);
const combined = `${get.output || ''}\n${get.error || ''}`;
assert.ok(
!combined.includes(marker),
`config-get must not echo plaintext for ${key}. Got:\n${combined}`
);
// Must contain the masked tail (last 4 of marker)
const expectedMask = '****' + marker.slice(-4);
assert.ok(
combined.includes(expectedMask),
`config-get must show masked form (${expectedMask}) for ${key}. Got:\n${combined}`
);
}
});
});