Files
msd-core/tests/feat-443-effort-fast-mode.integration.test.cjs
Tom Boucher 463cffd894 chore(#604): rename get-shit-done/ runtime directory to gsd-core/ (#615)
* chore(#604): rename get-shit-done/ runtime directory to gsd-core/

Renames the installed runtime directory `get-shit-done/` to `gsd-core/` so the
on-disk name matches the package (`@opengsd/gsd-core`), repo, and binary
(`gsd-tools`). The npm package name and binary are unchanged; npx/npm consumers
are unaffected.

Mechanical (bulk, ~90% of the diff):
- `git mv get-shit-done gsd-core`
- Swept path/identifier references across the repo via
  `perl -pe 's/get-shit-done(?!-\w)/gsd-core/g'`. The negative lookahead
  preserves the five legitimate slug variants that are NOT the directory:
  get-shit-done-{OLD,cc,classic,cli,redux} (old package/repo names).
- Build/manifest wiring: package.json (bin, files, coverage globs),
  tsconfig.build.json (outDir), ~86 .gitignore build-output entries,
  stryker.config.mjs, scan-ignore files, install.js path strings.
- Frozen (not rewritten): CHANGELOG.md history; translated docs
  (README.<locale>.md and docs/{ja-JP,ko-KR,pt-BR,zh-CN}/).

New logic (review here):
- src/installer-migrations/003-rename-get-shit-done-to-gsd-core.cts: a proper
  ADR-0008 installer migration. On upgrade it walks the legacy
  `~/.claude/get-shit-done/` tree, classifies each file via the prior install
  manifest, and emits remove-managed / backup-and-remove for managed files
  while PRESERVING unknown user-added files. Symlink-safe (skips a symlinked
  root and symlinked entries; bounds-checks every path under configDir). The
  framework rolls back on install failure. Emptied dirs may remain (framework
  has no recursive dir-removal primitive) — documented.
- scripts/lint-legacy-dir-name.cjs: CI regression guard forbidding the bare
  `get-shit-done` directory token (split token to avoid self-match; case-
  insensitive; `(?!-\w)` lookahead allows the slug variants; allowlists
  CHANGELOG, translated docs, and `gsd-allow-legacy-name` marker lines).
  Wired into the lint-tests CI job.
- Restored scripts/lint-package-identity-drift.cjs detection regexes (the
  mechanical sweep had wrongly rewritten the old-name patterns it exists to
  detect) and marked them as intentional legacy references.
- TDD tests for the migration and the guard; do.md slash-command guard regex
  tightened so a `/gsd-core/bin` path segment is not mistaken for a command;
  changeset + docs/installer-migrations.md row added.

Breaking: the installed runtime path moves `~/.claude/get-shit-done/` ->
`~/.claude/gsd-core/`. Migration 003 removes the stale legacy dir's managed
files (preserving user files) on upgrade. Users with custom hooks/configs
hardcoding the old path must update them.

Closes #604

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): unsweep pending changesets + allowlist injection-example docs

CI fixes for the rename PR:
- Do not sweep pending .changeset/*.md (ephemeral release-note fragments,
  like CHANGELOG); reverted those body edits so 5 pre-existing malformed
  fragments (missing type/pr) no longer enter the PR diff and trip docs-lint.
  Allowlisted .changeset/ in the legacy-name guard accordingly.
- Allowlisted TEST-EXAMPLES.md and docs/explanation/security-model.md in
  prompt-injection-scan.sh: they contain intentional injection examples /
  security-model prose; the path-reference rewrites are kept.

CodeQL alerts on this PR are pre-existing (alert lines unchanged by this PR;
none in the new migration/guard) and are out of scope for the rename.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): resolve CodeQL alerts surfaced on this PR

The rename diff touched files carrying pre-existing CodeQL findings; per the
no-pre-existing-dismissal rule, fixing every surfaced alert rather than waving
them off. All behavior-preserving:

- scripts/ci-test-scope.cjs: build the config-path match from string
  .includes() instead of a RegExp over an arg-derived value (js/regex-injection).
- src/profile-output.cts: escape backslashes before pipe-escaping desc/safeName
  so the table-cell escape is complete (js/incomplete-sanitization).
- tests/{bug-2643,bug-2808,docs-parity-live-registry}: two-pass HTML-comment
  strip so a bare/unclosed `<!--` cannot survive (js/incomplete-multi-character-sanitization).
- tests/inline-plan-threshold: drop the no-op `\s`->`\s` identity replace,
  keep the meaningful POSIX-class conversion (js/identity-replacement).

Verified: build:lib green; the touched test files + ci-test-scope + profile-output
suites pass; lint:legacy-name clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): correctly resolve remaining CodeQL alerts (regex-injection + sanitization)

The prior commit's fixes for two alerts were ineffective:
- ci-test-scope.cjs js/regex-injection: the alert is the CLI-arg-derived `file`
  reaching static regex `.test(file)` calls (not the config rule). Removed ALL
  regex over file/t — startsWith/includes/=== string checks + an isWindowsHint
  helper — so there is no regex sink for the tainted value.
- js/incomplete-multi-character-sanitization (3 test files): a single
  `.replace(/<!--...-->/g,'')` can let `<!--` re-form. Replaced with a fixpoint
  loop (replace until stable) plus a final bare-opener strip.

Verified: no regex over file/t remains; ci-test-scope + the 3 test suites pass;
lint:legacy-name clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): make ci-test-scope + comment-strippers regex-free to clear CodeQL

CodeQL flags the regex PATTERNS syntactically (regex-injection on the
--files arg split; incomplete-multi-character-sanitization on the <!--...-->
replace), so loop fixes do not satisfy it. Made these paths regex-free:
- ci-test-scope.cjs splitFiles: char-by-char separator tokenizer (no /[,\\s]+/).
- 3 test files: indexOf/slice HTML-comment stripper (no .replace(/<!--/)).
Behavior preserved; ci-test-scope + the 3 suites pass; guard clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): unblock security base64 scan on the large rename diff

The security job hit its 10m timeout: base64-scan.sh choked on the binary
test fixture tests/feat-3594-parser-property-style.test.cjs (embedded NUL/
non-UTF8 bytes -> thousands of bogus blobs + "ignored null byte" warnings),
and the ~800-file rename diff is slow to scan regardless.

- scripts/base64-scan.sh: skip binary-by-content files (grep -Iq .) — they
  can't carry base64-obfuscated *text* and feeding NUL bytes through the
  per-line scanner is pathologically slow. collect_files already filtered
  binary *extensions*; this catches binary *content* in text extensions.
- .github/workflows/security-scan.yml: raise the security job timeout 10m->30m
  to accommodate very large diffs (the scan itself is unchanged).

Verified locally: scan skips the fixture, 0 "ignored null byte" warnings,
0 findings, exit 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): sweep get-shit-done refs introduced by merging next

The branch was updated with next (#614/#384/#618 etc.), which reference the
get-shit-done/ dir (still named that on next). Swept the stale references in
the merged files to gsd-core so the rename stays consistent and lint:legacy-name
passes:
- commands/gsd/discuss-phase.md (runtime-launcher shim paths)
- src/core.cts (getAgentsDir layout comments)
- tests/bug-384-agents-runtime-aware.test.cjs (require path to runtime lib)

Verified: guard 0 violations; build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): exclude gsd-core/ path segments from bug-3683 command cross-ref invariant

The #614 runtime-launcher shim added to discuss-phase.md references
`${_GSD_RUNTIME_ROOT}/gsd-core/bin/...`. bug-3683's REF_PATTERN excluded path-y
refs only via lookbehind, but `}` precedes `/gsd-core/` in the shim, so it
mis-read the directory path as a dangling `/gsd-core` command ref (same class as
the #604 bug-2954 fix). Added a trailing `(?![\w-]*\/)` so `/gsd-<x>/...` path
segments are not treated as slash-command references.

Verified locally on BOTH platforms before pushing:
- mac (node 26) full suite: 0 failures
- gsd-test-runner (linux, node22 image) full suite: 0 failures
- bug-3683 + bug-2954 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): lazily resolve findProjectRoot in gsd-tools (harden flaky CI)

CI intermittently failed state.test's gsd-tools subprocess with
"findProjectRoot is not a function" (flip-flopping across legs; not reproducible
on mac full suite, gsd-test linux full suite, test:unit, or state.test x8).
findProjectRoot is a re-export from core.cjs (sourced from project-root.cjs);
binding it via destructure at module-load can be undefined under a load-ordering
edge. Resolve it lazily at call time via a small wrapper so the lookup happens
after core.cjs is fully initialized.

Verified green on BOTH platforms before pushing:
- mac (node 26) full suite: 0 failures
- gsd-test-runner (linux, node22) full suite: 0 failures
- state.test.cjs: 106/106; gsd-tools loads cleanly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#604): allowlist verification-patterns.md placeholder examples in secret scan

The rename git-mv'd references/verification-patterns.md into gsd-core/, pulling
it into the secret-scan diff. It documents stub/placeholder RED-FLAG env-var
examples (illustrative Stripe test-key / database-URL / API-key placeholders) —
not real credentials. Added it to .secretscanignore with the strict annotation,
mirroring the existing gsd-core/workflows/plan-phase.md exception.

Verified locally: secret-scan-lint --strict OK; secret-scan --diff origin/next
exits 0 with 0 findings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 18:35:29 -04:00

719 lines
30 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
'use strict';
/**
* Architecture-level QA for issue #443 — unified effort + fast_mode engine.
*
* Integration suite (*.integration.test.cjs): cross-module flows that exercise
* real CLI invocations via runGsdTools, the full 33-agent registry, and the
* config round-trip through config-set -> resolve-execution.
*
* INVARIANTS tested here (each is also documented in docs/TESTING-SUITES.md):
*
* (a) CROSS-PROVIDER VALIDITY — renderEffortForRuntime never emits a value
* that the real provider API would 400 on. Ground-truth provider enums are
* defined as local constants (not sourced from the implementation).
*
* (b) PARAM/CHANNEL CONTRACT — each runtime exposes a stable parameter name
* and propagation channel.
*
* (c) RESOLVE-EXECUTION JSON CONTRACT — the CLI command emits a stable JSON
* shape with all required keys and correct types.
*
* (d) TOTALITY across the real 33-agent registry — every agent produces a
* valid effort value; none returns undefined/null.
*
* (e) FAST-MODE HONESTY INVARIANT — claude runtime always reports
* fast_mode_supported=false (emitting fast_mode frontmatter is a silent
* no-op for Claude Code subagents).
*
* (f) PRECEDENCE MATRIX — first-valid-wins for both effort and fast_mode
* cascades, including invalid values correctly falling through.
*
* (g) DYNAMIC-ROUTING COMPOSITION — resolveEffortForTier escalates
* independently of model tier logic; clamps at 'max'; respects
* max_escalations; disabled when escalate_on_failure=false.
*
* (h) CONFIG-TOOLING ROUND-TRIP — config-set accepts all new effort/fast_mode
* key paths (schema validation passes); values survive round-trip through
* resolve-execution.
*/
process.env.GSD_TEST_MODE = '1';
const { describe, test, before, after, beforeEach, afterEach } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { createTempProject, cleanup, runGsdTools } = require('./helpers.cjs');
const {
resolveEffortInternal,
resolveFastModeInternal,
resolveEffortForTier,
VALID_EFFORTS,
} = require('../gsd-core/bin/lib/core.cjs');
const {
renderEffortForRuntime,
RUNTIMES_WITH_FAST_MODE,
catalog,
} = require('../gsd-core/bin/lib/model-catalog.cjs');
// ─────────────────────────────────────────────────────────────────────────────
// Ground-truth provider enums (defined HERE, not sourced from the implementation).
// These are the exact values the real APIs accept — using a value outside these
// sets would result in a 400 response from the provider.
//
// Sources:
// Anthropic: output_config.effort — https://docs.anthropic.com (Claude API)
// OpenAI: model_reasoning_effort — https://platform.openai.com/docs (Codex)
// ─────────────────────────────────────────────────────────────────────────────
const PROVIDER_EFFORT_ENUMS = {
claude: new Set(['low', 'medium', 'high', 'xhigh', 'max']),
codex: new Set(['minimal', 'low', 'medium', 'high', 'xhigh']),
};
// Helper: write config.json into a temp project
function writeConfig(dir, config) {
const planningDir = path.join(dir, '.planning');
fs.mkdirSync(planningDir, { recursive: true });
fs.writeFileSync(path.join(planningDir, 'config.json'), JSON.stringify(config, null, 2));
}
// ─── (a) CROSS-PROVIDER VALIDITY INVARIANT ───────────────────────────────────
describe('#443 integration (a): cross-provider validity invariant', () => {
// For every universal effort × every provider runtime, the rendered value
// must be a member of that provider's real API enum.
test('all VALID_EFFORTS render within provider enums for claude and codex', () => {
for (const universalEffort of VALID_EFFORTS) {
for (const [runtime, providerEnum] of Object.entries(PROVIDER_EFFORT_ENUMS)) {
const rendered = renderEffortForRuntime(runtime, universalEffort);
assert.ok(
providerEnum.has(rendered.value),
`render('${runtime}', '${universalEffort}').value = '${rendered.value}' is NOT in the ` +
`${runtime} provider enum ${[...providerEnum].join('|')} — real API would 400`
);
}
}
});
// Documented clamps must hold exactly
test("render('codex','max').value === 'xhigh' (max is Anthropic-only)", () => {
assert.strictEqual(renderEffortForRuntime('codex', 'max').value, 'xhigh');
});
test("render('claude','minimal').value === 'low' (minimal is Codex-only)", () => {
assert.strictEqual(renderEffortForRuntime('claude', 'minimal').value, 'low');
});
// Common levels must pass through unchanged on BOTH providers
test('common levels (low/medium/high/xhigh) pass through unchanged on claude', () => {
for (const level of ['low', 'medium', 'high', 'xhigh']) {
assert.strictEqual(
renderEffortForRuntime('claude', level).value,
level,
`claude: level '${level}' should pass through unchanged`
);
}
});
test('common levels (low/medium/high/xhigh) pass through unchanged on codex', () => {
for (const level of ['low', 'medium', 'high', 'xhigh']) {
assert.strictEqual(
renderEffortForRuntime('codex', level).value,
level,
`codex: level '${level}' should pass through unchanged`
);
}
});
});
// ─── (b) PARAM/CHANNEL CONTRACT ──────────────────────────────────────────────
describe('#443 integration (b): param/channel contract', () => {
test("claude: param is always 'output_config.effort'", () => {
for (const effort of VALID_EFFORTS) {
const r = renderEffortForRuntime('claude', effort);
assert.strictEqual(r.param, 'output_config.effort',
`claude param must be 'output_config.effort' for effort '${effort}'`);
}
});
test("codex: param is always 'model_reasoning_effort'", () => {
for (const effort of VALID_EFFORTS) {
const r = renderEffortForRuntime('codex', effort);
assert.strictEqual(r.param, 'model_reasoning_effort',
`codex param must be 'model_reasoning_effort' for effort '${effort}'`);
}
});
test('claude channel is stable: frontmatter', () => {
for (const effort of VALID_EFFORTS) {
assert.strictEqual(renderEffortForRuntime('claude', effort).channel, 'frontmatter');
}
});
test('codex channel is stable: api', () => {
for (const effort of VALID_EFFORTS) {
assert.strictEqual(renderEffortForRuntime('codex', effort).channel, 'api');
}
});
test("unknown runtimes (gemini, qwen, 'mystery'): param===null, value passes through", () => {
for (const runtime of ['gemini', 'qwen', 'mystery']) {
for (const effort of VALID_EFFORTS) {
const r = renderEffortForRuntime(runtime, effort);
assert.strictEqual(r.param, null, `${runtime}: param must be null`);
assert.strictEqual(r.channel, null, `${runtime}: channel must be null`);
assert.strictEqual(r.value, effort, `${runtime}: value must pass through unchanged`);
}
}
});
});
// ─── (c) RESOLVE-EXECUTION JSON CONTRACT ─────────────────────────────────────
describe('#443 integration (c): resolve-execution JSON contract', () => {
let tmpDir;
beforeEach(() => { tmpDir = createTempProject(); });
afterEach(() => { cleanup(tmpDir); });
function assertFullContract(output, label) {
assert.ok(typeof output.model === 'string' && output.model.length > 0,
`${label}: model must be a non-empty string`);
assert.ok(typeof output.profile === 'string' && output.profile.length > 0,
`${label}: profile must be a non-empty string`);
assert.ok(VALID_EFFORTS.includes(output.effort),
`${label}: effort '${output.effort}' must be a member of VALID_EFFORTS`);
assert.ok(typeof output.effort_rendered === 'string' && output.effort_rendered.length > 0,
`${label}: effort_rendered must be a non-empty string`);
assert.ok(output.effort_param === null || typeof output.effort_param === 'string',
`${label}: effort_param must be string or null`);
assert.ok(output.effort_propagation === null || typeof output.effort_propagation === 'string',
`${label}: effort_propagation must be string or null`);
assert.ok(typeof output.fast_mode === 'boolean',
`${label}: fast_mode must be a boolean`);
assert.ok(typeof output.fast_mode_supported === 'boolean',
`${label}: fast_mode_supported must be a boolean`);
}
test('gsd-planner (default claude runtime): full contract + known-agent shape', () => {
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assertFullContract(output, 'gsd-planner/claude');
assert.strictEqual(output.effort_param, 'output_config.effort');
assert.strictEqual(output.effort_propagation, 'frontmatter');
assert.strictEqual(output.fast_mode_supported, false);
// known agent must NOT have unknown_agent:true
assert.ok(!output.unknown_agent, 'known agent must not have unknown_agent:true');
});
test('codex runtime: full contract + effort_param=model_reasoning_effort', () => {
writeConfig(tmpDir, { runtime: 'codex' });
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assertFullContract(output, 'gsd-planner/codex');
assert.strictEqual(output.effort_param, 'model_reasoning_effort');
assert.strictEqual(output.fast_mode_supported, false);
});
test('gemini runtime: full contract + effort_param===null (no effort wire)', () => {
writeConfig(tmpDir, { runtime: 'gemini' });
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assertFullContract(output, 'gsd-planner/gemini');
assert.strictEqual(output.effort_param, null);
assert.strictEqual(output.effort_propagation, null);
assert.strictEqual(output.fast_mode_supported, false);
});
test('unknown agent: full contract + unknown_agent===true', () => {
const result = runGsdTools(['resolve-execution', 'unknown-agent-xyz'], tmpDir, { HOME: tmpDir });
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assertFullContract(output, 'unknown-agent-xyz');
assert.strictEqual(output.unknown_agent, true, 'unknown agent must have unknown_agent:true');
});
});
// ─── (d) TOTALITY across the real 33-agent registry ──────────────────────────
describe('#443 integration (d): totality across real registry', () => {
let tmpDir;
before(() => { tmpDir = createTempProject(); });
after(() => { cleanup(tmpDir); });
const registeredAgents = Object.keys(catalog.agents);
// Confirm we're covering the full registry — snapshot the count so a
// catalog shrink is caught by this assertion.
test(`registry has at least 33 agents (currently ${registeredAgents.length})`, () => {
assert.ok(registeredAgents.length >= 33,
`Expected at least 33 agents in registry, got ${registeredAgents.length}`);
});
test(`all ${registeredAgents.length} agents: resolveEffortInternal returns a VALID_EFFORTS member`, () => {
const effortSet = new Set(VALID_EFFORTS);
const bad = [];
for (const agent of registeredAgents) {
const effort = resolveEffortInternal(tmpDir, agent);
if (effort === undefined || effort === null || !effortSet.has(effort)) {
bad.push(`${agent}: got ${JSON.stringify(effort)}`);
}
}
assert.strictEqual(bad.length, 0,
`Agents with invalid effort:\n${bad.join('\n')}`);
});
test(`all ${registeredAgents.length} agents: resolveFastModeInternal returns strict boolean`, () => {
const bad = [];
for (const agent of registeredAgents) {
const fm = resolveFastModeInternal(tmpDir, agent);
if (typeof fm !== 'boolean') {
bad.push(`${agent}: got ${JSON.stringify(fm)} (${typeof fm})`);
}
}
assert.strictEqual(bad.length, 0,
`Agents with non-boolean fast_mode:\n${bad.join('\n')}`);
});
test(`all ${registeredAgents.length} agents: renderEffortForRuntime('claude', effort) stays in claude enum`, () => {
const claudeEnum = PROVIDER_EFFORT_ENUMS.claude;
const bad = [];
for (const agent of registeredAgents) {
const effort = resolveEffortInternal(tmpDir, agent);
const rendered = renderEffortForRuntime('claude', effort);
if (!claudeEnum.has(rendered.value)) {
bad.push(`${agent}: effort=${effort} rendered=${rendered.value} not in claude enum`);
}
}
assert.strictEqual(bad.length, 0,
`Agents producing invalid claude effort:\n${bad.join('\n')}`);
});
});
// ─── (e) FAST-MODE HONESTY INVARIANT ─────────────────────────────────────────
describe('#443 integration (e): fast-mode honesty invariant', () => {
let tmpDir;
beforeEach(() => { tmpDir = createTempProject(); });
afterEach(() => { cleanup(tmpDir); });
// Sample of agents across all tiers to prove the invariant is not agent-specific
const testAgents = ['gsd-planner', 'gsd-executor', 'gsd-codebase-mapper', 'gsd-verifier'];
test('claude runtime: fast_mode_supported is ALWAYS false regardless of fast_mode config', () => {
const configs = [
{},
{ fast_mode: { enabled: true } },
{ fast_mode: { routing_tier_defaults: { heavy: true } } },
{ fast_mode: { agent_overrides: { 'gsd-planner': true } } },
];
for (const config of configs) {
writeConfig(tmpDir, config);
for (const agent of testAgents) {
const result = runGsdTools(['resolve-execution', agent], tmpDir, { HOME: tmpDir });
assert.ok(result.success, `Command failed for ${agent}: ${result.error}`);
const output = JSON.parse(result.output);
assert.strictEqual(output.fast_mode_supported, false,
`claude/${agent}: fast_mode_supported must be false (Claude has no per-subagent fast-mode mechanism); config=${JSON.stringify(config)}`);
}
}
});
test("RUNTIMES_WITH_FAST_MODE.has('api') === true (api is the only fast-mode capable runtime)", () => {
assert.ok(RUNTIMES_WITH_FAST_MODE.has('api'),
"RUNTIMES_WITH_FAST_MODE must include 'api' — this is the only runtime with per-call fast_mode support");
});
test("RUNTIMES_WITH_FAST_MODE.has('claude') === false (claude fast-mode is session-level only)", () => {
assert.ok(!RUNTIMES_WITH_FAST_MODE.has('claude'),
"RUNTIMES_WITH_FAST_MODE must NOT include 'claude' — emitting fast_mode frontmatter on a Claude subagent is a silent no-op");
});
test("RUNTIMES_WITH_FAST_MODE.has('codex') === false", () => {
assert.ok(!RUNTIMES_WITH_FAST_MODE.has('codex'),
"codex does not support per-call fast_mode");
});
test("RUNTIMES_WITH_FAST_MODE.has('gemini') === false", () => {
assert.ok(!RUNTIMES_WITH_FAST_MODE.has('gemini'),
"gemini does not support per-call fast_mode");
});
});
// ─── (f) PRECEDENCE MATRIX ───────────────────────────────────────────────────
describe('#443 integration (f): precedence matrix (property/table-driven)', () => {
let tmpDir;
beforeEach(() => { tmpDir = createTempProject(); });
afterEach(() => { cleanup(tmpDir); });
// Effort: first-valid-wins from highest precedence to lowest
// 1. opts.override (invocation)
// 2. effort.agent_overrides.<agent>
// 3. effort.routing_tier_defaults.<tier>
// 4. effort.default
// 5. manifest tier default
// 6. hardcoded 'high'
const effortPrecedenceTable = [
{
label: 'layer 1 (invocation override) beats all',
config: {
effort: {
agent_overrides: { 'gsd-planner': 'low' },
routing_tier_defaults: { heavy: 'medium' },
default: 'xhigh',
},
},
opts: { override: 'minimal' },
expected: 'minimal',
},
{
label: 'layer 2 (agent_override) beats tier default and default',
config: {
effort: {
agent_overrides: { 'gsd-planner': 'low' },
routing_tier_defaults: { heavy: 'medium' },
default: 'xhigh',
},
},
opts: {},
expected: 'low',
},
{
label: 'layer 3 (routing_tier_defaults) beats effort.default',
config: {
effort: {
routing_tier_defaults: { heavy: 'medium' },
default: 'xhigh',
},
},
opts: {},
expected: 'medium',
},
{
label: 'layer 4 (effort.default) when no tier default set',
config: {
effort: { default: 'low' },
},
opts: {},
expected: 'low',
},
{
label: 'invalid layer 1 (turbo) falls through to layer 2 (agent_override)',
config: {
effort: { agent_overrides: { 'gsd-planner': 'medium' } },
},
opts: { override: 'turbo' },
expected: 'medium',
},
{
label: 'invalid layer 2 (agent_override=123 numeric) falls through to tier default',
config: {
effort: {
agent_overrides: { 'gsd-planner': 123 },
routing_tier_defaults: { heavy: 'high' },
},
},
opts: {},
expected: 'high',
},
{
label: 'invalid tier default (turbo) falls through to effort.default',
config: {
effort: {
routing_tier_defaults: { heavy: 'turbo' },
default: 'low',
},
},
opts: {},
expected: 'low',
},
];
for (const row of effortPrecedenceTable) {
test(`effort precedence: ${row.label}`, () => {
writeConfig(tmpDir, row.config);
const result = resolveEffortInternal(tmpDir, 'gsd-planner', row.opts);
assert.strictEqual(result, row.expected,
`Expected '${row.expected}', got '${result}' — config: ${JSON.stringify(row.config)}`);
});
}
// fast_mode precedence:
// 1. opts.override (strict boolean only)
// 2. fast_mode.agent_overrides.<agent> (strict boolean only)
// 3. fast_mode.routing_tier_defaults.<tier> (strict boolean only)
// 4. fast_mode.enabled (strict boolean only)
// 5. false
const fastModePrecedenceTable = [
{
label: 'layer 1 (opts.override=false) beats enabled=true',
config: { fast_mode: { enabled: true } },
opts: { override: false },
expected: false,
},
{
label: 'layer 2 (agent_override=true) beats tier default',
config: {
fast_mode: {
agent_overrides: { 'gsd-planner': true },
routing_tier_defaults: { heavy: false },
enabled: false,
},
},
opts: {},
expected: true,
},
{
label: 'layer 3 (tier default=true) beats enabled=false',
config: {
fast_mode: {
routing_tier_defaults: { heavy: true },
enabled: false,
},
},
opts: {},
expected: true,
},
{
label: 'layer 4 (enabled=true) when no tier/agent overrides',
config: { fast_mode: { enabled: true } },
opts: {},
expected: true,
},
{
label: 'layer 5 (default false) when all absent',
config: {},
opts: {},
expected: false,
},
{
label: 'string "true" in opts.override is NOT accepted (falls through)',
config: { fast_mode: { enabled: true } },
// override must be strict boolean; string falls through to next layer
opts: { override: 'true' },
// 'true' as string is not boolean -> falls through to tier default
// gsd-planner is heavy; no tier default set; falls to enabled=true
expected: true,
},
{
label: 'string "true" in agent_overrides is NOT accepted',
config: {
fast_mode: {
agent_overrides: { 'gsd-planner': 'true' },
enabled: false,
},
},
opts: {},
// string 'true' is not boolean -> fall through to tier default -> enabled=false -> false
expected: false,
},
];
for (const row of fastModePrecedenceTable) {
test(`fast_mode precedence: ${row.label}`, () => {
writeConfig(tmpDir, row.config);
const result = resolveFastModeInternal(tmpDir, 'gsd-planner', row.opts);
assert.strictEqual(result, row.expected,
`Expected ${row.expected}, got ${result} — config: ${JSON.stringify(row.config)}`);
});
}
});
// ─── (g) DYNAMIC-ROUTING COMPOSITION ─────────────────────────────────────────
describe('#443 integration (g): dynamic-routing composition', () => {
let tmpDir;
beforeEach(() => { tmpDir = createTempProject(); });
afterEach(() => { cleanup(tmpDir); });
const dynamicRoutingBase = {
dynamic_routing: {
enabled: true,
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
escalate_on_failure: true,
max_escalations: 4,
},
effort: { routing_tier_defaults: { light: 'low' } },
};
test('resolveEffortForTier escalates independently of model resolution', () => {
writeConfig(tmpDir, dynamicRoutingBase);
const effort0 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 0);
const effort1 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1);
const effort2 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 2);
assert.strictEqual(effort0, 'low');
assert.strictEqual(effort1, 'medium');
assert.strictEqual(effort2, 'high');
// Verify the effort ladder steps up correctly without asserting model value
// (model timing is a separate concern from effort escalation)
assert.notStrictEqual(effort0, effort1, 'effort should escalate at attempt 1');
assert.notStrictEqual(effort1, effort2, 'effort should escalate at attempt 2');
});
test('escalate_on_failure=false: attempt is ignored for effort', () => {
writeConfig(tmpDir, {
...dynamicRoutingBase,
dynamic_routing: {
...dynamicRoutingBase.dynamic_routing,
escalate_on_failure: false,
},
});
const e0 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 0);
const e1 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1);
const e3 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 3);
assert.strictEqual(e0, e1, 'effort must not escalate when escalate_on_failure=false');
assert.strictEqual(e0, e3, 'effort must not escalate when escalate_on_failure=false');
});
test('escalation clamps at "max" regardless of attempt number', () => {
writeConfig(tmpDir, {
dynamic_routing: {
enabled: true,
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
escalate_on_failure: true,
max_escalations: 99,
},
effort: { default: 'max' },
});
// Any large attempt number — result must never exceed 'max'
const r = resolveEffortForTier(tmpDir, 'gsd-planner', 50);
assert.strictEqual(r, 'max', `Effort must clamp at 'max', got '${r}'`);
const EFFORT_LADDER = VALID_EFFORTS;
const maxIdx = EFFORT_LADDER.indexOf('max');
const rIdx = EFFORT_LADDER.indexOf(r);
assert.ok(rIdx <= maxIdx, 'Effort must not exceed the max position in the ladder');
});
test('respects max_escalations cap: attempt beyond cap gives same as cap', () => {
writeConfig(tmpDir, {
dynamic_routing: {
enabled: true,
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
escalate_on_failure: true,
max_escalations: 1,
},
effort: { routing_tier_defaults: { light: 'low' } },
});
const atCap = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1); // 1 escalation
const beyond = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 5); // capped at 1
assert.strictEqual(atCap, beyond,
'Effort beyond max_escalations must be same as at cap');
assert.strictEqual(atCap, 'medium', 'low + 1 escalation = medium');
});
test('dynamic_routing disabled: resolveEffortForTier ignores attempt', () => {
writeConfig(tmpDir, {
effort: { routing_tier_defaults: { light: 'low' } },
});
const e0 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 0);
const e5 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 5);
assert.strictEqual(e0, e5, 'Effort must not change when dynamic_routing is disabled');
assert.strictEqual(e0, 'low');
});
});
// ─── (h) CONFIG-TOOLING ROUND-TRIP ───────────────────────────────────────────
describe('#443 integration (h): config-tooling round-trip', () => {
let tmpDir;
beforeEach(() => { tmpDir = createTempProject(); });
afterEach(() => { cleanup(tmpDir); });
test('config-set effort.default then resolve-execution reflects new value', () => {
const setResult = runGsdTools(['config-set', 'effort.default', 'low'], tmpDir, { HOME: tmpDir });
assert.ok(setResult.success, `config-set effort.default failed: ${setResult.error}`);
const execResult = runGsdTools(['resolve-execution', 'unknown-agent-xyz'], tmpDir, { HOME: tmpDir });
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
const output = JSON.parse(execResult.output);
// unknown agent falls through to effort.default
assert.strictEqual(output.effort, 'low',
`Expected effort='low' after config-set, got '${output.effort}'`);
});
test('config-set effort.routing_tier_defaults.heavy then resolve-execution uses it', () => {
const setResult = runGsdTools(
['config-set', 'effort.routing_tier_defaults.heavy', 'medium'],
tmpDir, { HOME: tmpDir }
);
assert.ok(setResult.success, `config-set failed: ${setResult.error}`);
const execResult = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
const output = JSON.parse(execResult.output);
// gsd-planner is heavy; tier default now overridden to medium
assert.strictEqual(output.effort, 'medium',
`Expected effort='medium' after routing_tier_defaults override, got '${output.effort}'`);
});
test('config-set effort.agent_overrides.<agent> wins over tier default', () => {
// Set tier default first, then per-agent override
runGsdTools(['config-set', 'effort.routing_tier_defaults.heavy', 'medium'], tmpDir, { HOME: tmpDir });
const setResult = runGsdTools(
['config-set', 'effort.agent_overrides.gsd-planner', 'xhigh'],
tmpDir, { HOME: tmpDir }
);
assert.ok(setResult.success, `config-set agent_overrides failed: ${setResult.error}`);
const execResult = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
const output = JSON.parse(execResult.output);
assert.strictEqual(output.effort, 'xhigh',
`Expected agent_overrides to win (xhigh), got '${output.effort}'`);
});
test('config-set fast_mode.enabled true then resolve-execution reflects fast_mode=true', () => {
const setResult = runGsdTools(['config-set', 'fast_mode.enabled', 'true'], tmpDir, { HOME: tmpDir });
assert.ok(setResult.success, `config-set fast_mode.enabled failed: ${setResult.error}`);
const execResult = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
const output = JSON.parse(execResult.output);
assert.strictEqual(output.fast_mode, true,
`Expected fast_mode=true after config-set, got ${output.fast_mode}`);
// fast_mode_supported stays false (claude runtime)
assert.strictEqual(output.fast_mode_supported, false);
});
test('config-set fast_mode.agent_overrides.<agent> true reflects in output', () => {
const setResult = runGsdTools(
['config-set', 'fast_mode.agent_overrides.gsd-codebase-mapper', 'true'],
tmpDir, { HOME: tmpDir }
);
assert.ok(setResult.success, `config-set failed: ${setResult.error}`);
const execResult = runGsdTools(['resolve-execution', 'gsd-codebase-mapper'], tmpDir, { HOME: tmpDir });
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
const output = JSON.parse(execResult.output);
assert.strictEqual(output.fast_mode, true,
`Expected fast_mode=true for agent-specific override`);
});
// Prove the config-set commands accept all the new key namespaces (schema validation)
test('config-set accepts all effort/* and fast_mode/* key namespaces without error', () => {
const keysToTest = [
['effort.default', 'high'],
['effort.routing_tier_defaults.light', 'low'],
['effort.routing_tier_defaults.standard', 'medium'],
['effort.routing_tier_defaults.heavy', 'xhigh'],
['effort.agent_overrides.gsd-executor', 'high'],
['fast_mode.enabled', 'false'],
['fast_mode.routing_tier_defaults.light', 'false'],
['fast_mode.routing_tier_defaults.standard', 'false'],
['fast_mode.routing_tier_defaults.heavy', 'false'],
['fast_mode.agent_overrides.gsd-verifier', 'false'],
];
for (const [key, val] of keysToTest) {
const r = runGsdTools(['config-set', key, val], tmpDir, { HOME: tmpDir });
assert.ok(r.success, `config-set '${key}' '${val}' should succeed, got: ${r.error}`);
}
});
});