* test(#443): RED unified effort + fast_mode + resolve-execution All 68 tests failing as expected — no implementation yet. Covers: effort cascade (tier defaults, overrides, invalid fallthrough), fast_mode cascade (boolean-only, tier defaults), resolveEffortForTier escalation, renderEffortForRuntime clamping, resolve-execution CLI, config schema new keys, QA hostile-input matrix. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(#443): unified cross-provider effort + fast_mode knobs and resolve-execution query Adds config-driven effort control (universal ladder: minimal<low<medium<high<xhigh<max) and fast_mode propagation knobs, with per-runtime rendering that clamps the unique tail values (max=Anthropic-only clamps to xhigh on Codex; minimal=Codex-only clamps to low on Claude). Key changes: - config-schema.manifest.json: add effort.default, fast_mode.enabled as validKeys; add 4 dynamicKeyPatterns for effort.routing_tier_defaults, effort.agent_overrides, fast_mode.routing_tier_defaults, fast_mode.agent_overrides; fix stale _comment - config-defaults.manifest.json: add effort and fast_mode blocks with tier defaults - model-catalog.cjs: add EFFORT_RENDERING map, renderEffortForRuntime(), RUNTIMES_WITH_FAST_MODE - model-profiles.cjs: re-export new catalog exports - core.cjs: add resolveEffortInternal, resolveFastModeInternal, resolveEffortForTier, VALID_EFFORTS, EFFORT_SET, nextEffort; pass effort/fast_mode through loadConfig - commands.cjs: replace reasoning_effort in cmdResolveModel with unified effort; add cmdResolveExecution (superset command with effort_rendered, effort_param, effort_propagation, fast_mode, fast_mode_supported) - gsd-tools.cjs: add resolve-execution case with --effort/--fast-mode/--attempt flags - tests/feat-443: 69 tests covering cascade, rendering, escalation, CLI, schema, QA matrix - tests/commands.test.cjs: convert 3 reasoning_effort assertions to unified effort - docs/CONFIGURATION.md: document effort + fast_mode + resolve-execution sections - settings-advanced.md: list new effort/fast_mode keys in confirmation table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#443): remove dead catalog effort lane; unify codex effort through renderEffortForRuntime - Remove resolveReasoningEffortInternal (catalog-driven effort function) from core.cjs and its export; remove from commands.cjs destructure import - Convert tests/issue-2517-runtime-aware-profiles.test.cjs: all 11 effort assertions now use resolveEffortInternal + renderEffortForRuntime; Claude effort is first-class (output_config.effort); unknown runtimes assert param===null - Convert tests/feat-3023-model-phase-types.test.cjs: replace the entire resolveReasoningEffortInternal describe with unified effort assertions; effort derives from AGENT_DEFAULT_TIERS routing tier, not phase-type tier Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(#443): ADR for unified cross-provider effort + fast-mode routing Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(#443): architecture-level QA invariants + test-strategy doc Add 48-test integration suite (feat-443-effort-fast-mode.integration.test.cjs) covering 8 architectural invariants: cross-provider validity (never emit a value the real API would 400 on), param/channel contract stability, resolve-execution JSON contract (all 8 keys + correct types), totality across the full 33-agent registry, fast-mode honesty (claude always fast_mode_supported=false), precedence first-valid-wins matrix for both effort and fast_mode cascades, dynamic-routing composition (effort escalation independent of model tier), and config-set round-trip for all new effort/* and fast_mode/* key namespaces. Append test-strategy section with invariant rationale and E2E gap documentation to docs/TESTING-SUITES.md. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(#443): add failing install-wiring tests for effort per-runtime injection (RED) TDD RED: 10 failing tests covering: - Claude .md gets effort: injected per tier (planner=xhigh, mapper=low, executor=high) - Gemini .md does NOT get effort: (already passing — Gemini-safe) - Codex .toml gets model_reasoning_effort via unified resolver - Config-driven: effort.agent_overrides drives both Claude .md and Codex .toml - Source purity: agents/*.md have no effort: key (already passing) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(#443): wire effort per-runtime at install (Claude .md frontmatter + Codex .toml unified) - Import AGENT_DEFAULT_TIERS and renderEffortForRuntime from model-catalog.cjs - Add readGsdEffectiveEffortConfig(targetDir): reads merged effort config from .planning/config.json (per-project wins) + ~/.gsd/defaults.json (global fallback), same probe pattern as readGsdRuntimeProfileResolver - Add resolveInstallTimeEffort(effortCfg, agentName): pure function matching resolveEffortInternal() precedence (agent_overrides > routing_tier_defaults > default > 'high') without loadConfig side-effects (no sub-repo detection, no migration writes) - Claude agent copy loop: inject `effort: <value>` into frontmatter ONLY for runtime === 'claude'; all other .md runtimes (Gemini, Qwen, Hermes, etc.) stay effort-free (Gemini-safe source contract preserved in agents/*.md) - generateCodexAgentToml: add effortCfg param; emit model_reasoning_effort from unified resolver (replaces old catalog entry.reasoning_effort); Codex clamps max → xhigh via renderEffortForRuntime('codex', ...) - installCodexConfig: pass readGsdEffectiveEffortConfig(targetDir) to generateCodexAgentToml so per-project config wins for Codex .toml too - Update failing tests to GREEN: 12/12 pass; all 17 install tests pass; 2847/2848 unit tests pass (1 pre-existing failure: policy-shell-pinning) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#443): source install effort defaults from manifest (kill drift) + guard test Replace hardcoded _GSD_EFFORT_MANIFEST_TIER_DEFAULTS and the 'high' fallback in resolveInstallTimeEffort with values read from config-defaults.manifest.json at module init, using the same __dirname-relative path install.js already uses for all shared manifests. Add feat-443-effort-defaults-drift.test.cjs to assert equality between install.js's runtime constants and the manifest on every CI run. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): reconcile Codex TOML tests with unified effort design The #443 unified effort resolver makes generateCodexAgentToml always emit model_reasoning_effort (driven by resolveInstallTimeEffort, not model_profile_overrides). The test 'generated TOML omits reasoning_effort when runtime has none' had an obsolete premise — model_profile_overrides.reasoning_effort:'' no longer suppresses unified effort. Convert it to assert the new invariant: Codex TOML always carries a valid model_reasoning_effort from the agent's routing tier (xhigh for gsd-planner, a heavy-tier agent), while model_profile_overrides model override is still respected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): make install.js effort resolution lazy (no load-time side effects breaking launcher-parity) Replace module-load-time IIFE + hard throw (config-defaults.manifest.json read) and top-level require of model-catalog.cjs with a lazy _getGsdEffortCatalog() getter that initialises on first call from resolveInstallTimeEffort / generateCodexAgentToml / Claude .md effort injection. Requiring install.js in unrelated test contexts (e.g. runtime-launcher-parity) no longer triggers manifest IO or throws, eliminating the load-time side effect that changed subprocess exit codes / stderr on the bench. Drift-guard exports (_GSD_EFFORT_MANIFEST_TIER_DEFAULTS / _GSD_EFFORT_MANIFEST_DEFAULT) preserved as lazy getter properties on module.exports so feat-443-effort-defaults-drift still validates them without forcing eager load. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): isolate install-wiring test HOME to stop \$HOME/.claude pollution breaking launcher-parity runGlobalInstall() now redirects HOME to a per-call isolated tmpdir in addition to the existing runtime-specific env-var redirects (CLAUDE_CONFIG_DIR, GEMINI_CONFIG_DIR, CODEX_HOME). This ensures install.js code that uses os.homedir() directly — including the ~/.cache/gsd update-check deletion, ~/.gsd/defaults.json reads, and any HOME-relative npm subprocess writes — never touches the real \$HOME during the test. Without the HOME isolation the install test (which is new to this branch and is now picked up by Docker's raw \`tests/*.test.cjs\` glob) could write or delete files under the real \$HOME, causing runtime-launcher-parity test (D) to fail: (D) asserts a loud non-zero exit when \$RUNTIME_DIR/gsd-tools.cjs is absent and gsd-tools is not on PATH, but the launcher's \$HOME/.claude fallback arm succeeds if \$HOME/.claude/get-shit-done/bin/gsd-tools.cjs exists. Also sets GSD_SKIP_STALE_SDK_CHECK=1 to suppress the \`npm ls -g\` subprocess that the global installer spawns — irrelevant to effort-wiring assertions, slow, and potentially writes to ~/.npm cache. All 12 feat-443 install-wiring assertions preserved. Drift-guard 5/5. Unit suite 2848/2850 (pre-existing policy-shell-pinning.test.cjs failure on next). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(#443): add changeset fragment for effort + fast-mode routing Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(#443): set GSD_TEST_MODE before requiring install.js in drift-guard test to prevent HOME leak Without GSD_TEST_MODE=1, require('bin/install.js') runs the module's main install block (guarded by !GSD_TEST_MODE), performing a real global Claude install into $HOME/.claude/. On CI ubuntu where node is on standard PATH, the launcher's $HOME/.claude fallback arm then finds gsd-tools.cjs, causing runtime-launcher-parity test (D) to exit zero when it must exit non-zero. Root cause: feat-443-effort-defaults-drift.test.cjs (unit suite) runs alphabetically before runtime-launcher-parity.test.cjs in the same node --test invocation. Each runs in a separate worker process but shares the same HOME. The drift test's install leaks gsd-tools.cjs into that HOME, then the launcher test's bash subprocess finds it via the $HOME/.claude arm. Fix: add process.env.GSD_TEST_MODE = '1' at the top of the drift-guard test, before the require(installPath) call. This matches the pattern used by feat-443-effort-fast-mode.test.cjs and feat-443-effort-install-wiring .install.test.cjs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): deterministic resolve-execution arg parsing + validate install-time effort (Codex adversarial findings) Finding 1: resolve-execution --effort low gsd-planner misrouted 'low' as the agent. Replace find(non-dash) with a proper flag-consuming loop that collects a single positional; validate missing/extra positionals and malformed --attempt values. Finding 2: resolveInstallTimeEffort returned unvalidated effort strings (e.g. "ultra") verbatim. Each precedence layer now checks GSD_EFFORT_SET (imported once from core.cjs) before accepting a value, mirroring resolveEffortInternal exactly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): newline-agnostic effort frontmatter injection (Windows CRLF) + CRLF-safe assertions Extracts injectEffortFrontmatter(content, effortValue) pure helper that detects EOL (LF vs CRLF) from the opening '---' line and inserts 'effort: <value>' before the closing '---' delimiter using the same EOL as the surrounding frontmatter. Regex now uses /^---\r?\n([\s\S]*?)^---\r?$/m instead of the LF-only /^(---\n[\s\S]*?)(---)(\n|$)/ that silently skipped CRLF files on Windows (git core.autocrlf=true checkout). Also adds 7 unit tests covering LF, CRLF, idempotency, no-frontmatter, and complex frontmatter cases. Exports injectEffortFrontmatter from module.exports. Fixes 6 CI failures in tests/feat-443-effort-install-wiring.install.test.cjs on windows-latest runners (lines 138, 145, 152, 261, 345, 356). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: CI Rebase Check <ci@gsd-redux> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
5
.changeset/clever-pumas-frolic.md
Normal file
5
.changeset/clever-pumas-frolic.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Added
|
||||
pr: 463
|
||||
---
|
||||
**Effort and fast-mode routing controls (Opus 4.8)** — GSD resolves a universal `effort` level (minimal–max, default high) and an orthogonal `fast_mode` toggle per agent via config (`effort.default`/`effort.routing_tier_defaults`/`effort.agent_overrides` and the `fast_mode.*` equivalents), rendered to each runtime's native parameter (Claude `output_config.effort` / subagent `effort` frontmatter; Codex `model_reasoning_effort`) with cross-provider clamping. New `query resolve-execution` exposes the resolved execution profile.
|
||||
278
bin/install.js
278
bin/install.js
@@ -145,8 +145,61 @@ const { MODEL_PROFILES: GSD_MODEL_PROFILES } = require(path.join(_gsdLibDir, 'mo
|
||||
const {
|
||||
RUNTIME_PROFILE_MAP: GSD_RUNTIME_PROFILE_MAP,
|
||||
resolveTierEntry: gsdResolveTierEntry,
|
||||
EFFORT_SET: GSD_EFFORT_SET,
|
||||
} = require(path.join(_gsdLibDir, 'core.cjs'));
|
||||
|
||||
// #443 — model-catalog and config-defaults.manifest.json exports needed only
|
||||
// by effort-resolution code paths (resolveInstallTimeEffort /
|
||||
// generateCodexAgentToml / Claude .md effort injection). Loaded lazily the
|
||||
// first time they are needed so that requiring install.js in test contexts that
|
||||
// never trigger an install does NOT produce module-load-time side effects (the
|
||||
// manifest read + hard throw) that could alter subprocess exit codes or stderr.
|
||||
let _gsdEffortCatalogCache = null;
|
||||
function _getGsdEffortCatalog() {
|
||||
if (_gsdEffortCatalogCache) return _gsdEffortCatalogCache;
|
||||
|
||||
const { AGENT_DEFAULT_TIERS, renderEffortForRuntime } = require(path.join(_gsdLibDir, 'model-catalog.cjs'));
|
||||
|
||||
const manifestPath = path.join(
|
||||
__dirname,
|
||||
'..',
|
||||
'get-shit-done',
|
||||
'bin',
|
||||
'shared',
|
||||
'config-defaults.manifest.json'
|
||||
);
|
||||
let manifestData;
|
||||
try {
|
||||
manifestData = JSON.parse(fs.readFileSync(manifestPath, 'utf-8'));
|
||||
} catch (_err) {
|
||||
// Fail loudly — a missing manifest is a broken install, not a soft degradation.
|
||||
throw new Error(
|
||||
`gsd install: cannot load config-defaults.manifest.json at ${manifestPath}: ${_err.message}`
|
||||
);
|
||||
}
|
||||
|
||||
const tierDefaults =
|
||||
(manifestData.effort &&
|
||||
manifestData.effort.routing_tier_defaults &&
|
||||
typeof manifestData.effort.routing_tier_defaults === 'object' &&
|
||||
!Array.isArray(manifestData.effort.routing_tier_defaults))
|
||||
? manifestData.effort.routing_tier_defaults
|
||||
: { light: 'low', standard: 'high', heavy: 'xhigh' }; // guard: unreachable if manifest is valid
|
||||
|
||||
const effortDefault =
|
||||
(manifestData.effort && typeof manifestData.effort.default === 'string')
|
||||
? manifestData.effort.default
|
||||
: 'high'; // guard: unreachable if manifest is valid
|
||||
|
||||
_gsdEffortCatalogCache = {
|
||||
AGENT_DEFAULT_TIERS,
|
||||
renderEffortForRuntime,
|
||||
EFFORT_MANIFEST_TIER_DEFAULTS: tierDefaults,
|
||||
EFFORT_MANIFEST_DEFAULT: effortDefault,
|
||||
};
|
||||
return _gsdEffortCatalogCache;
|
||||
}
|
||||
|
||||
const {
|
||||
MINIMAL_SKILL_ALLOWLIST,
|
||||
isMinimalMode,
|
||||
@@ -1381,6 +1434,185 @@ function readGsdEffectiveModelOverrides(targetDir = null) {
|
||||
return { ...(global || {}), ...(projectOverrides || {}) };
|
||||
}
|
||||
|
||||
/**
|
||||
* #443 — Read the merged `effort` config block for install-time effort resolution.
|
||||
*
|
||||
* Probes the same config sources as readGsdRuntimeProfileResolver (per-project
|
||||
* `.planning/config.json` wins over `~/.gsd/defaults.json`) but extracts the
|
||||
* `effort` object instead of the model-profile fields.
|
||||
*
|
||||
* Returns the merged `effort` object or null when neither source defines one.
|
||||
* The caller can pass this to resolveInstallTimeEffort() which is pure and
|
||||
* requires no filesystem access beyond what this helper already performs.
|
||||
*
|
||||
* @param {string|null} targetDir Runtime install root (walks up to find .planning/).
|
||||
* @returns {object|null}
|
||||
*/
|
||||
function readGsdEffectiveEffortConfig(targetDir = null) {
|
||||
const homeDefaults = _readGsdConfigFile(
|
||||
path.join(os.homedir(), '.gsd', 'defaults.json'),
|
||||
'~/.gsd/defaults.json'
|
||||
);
|
||||
|
||||
let projectConfig = null;
|
||||
if (targetDir) {
|
||||
let probeDir = path.resolve(targetDir);
|
||||
for (let depth = 0; depth < 8; depth += 1) {
|
||||
const candidate = path.join(probeDir, '.planning', 'config.json');
|
||||
if (fs.existsSync(candidate)) {
|
||||
projectConfig = _readGsdConfigFile(candidate, '.planning/config.json');
|
||||
break;
|
||||
}
|
||||
const parent = path.dirname(probeDir);
|
||||
if (parent === probeDir) break;
|
||||
probeDir = parent;
|
||||
}
|
||||
}
|
||||
|
||||
const homeEffort = (homeDefaults && homeDefaults.effort && typeof homeDefaults.effort === 'object' && !Array.isArray(homeDefaults.effort))
|
||||
? homeDefaults.effort
|
||||
: null;
|
||||
const projectEffort = (projectConfig && projectConfig.effort && typeof projectConfig.effort === 'object' && !Array.isArray(projectConfig.effort))
|
||||
? projectConfig.effort
|
||||
: null;
|
||||
|
||||
if (!homeEffort && !projectEffort) return null;
|
||||
|
||||
// Per-project wins on conflict within each sub-field. Merge field-by-field so
|
||||
// a project config that only sets agent_overrides still inherits global
|
||||
// routing_tier_defaults and default.
|
||||
return {
|
||||
...(homeEffort || {}),
|
||||
...(projectEffort || {}),
|
||||
// Deep-merge agent_overrides (project wins per-key)
|
||||
agent_overrides: {
|
||||
...((homeEffort && homeEffort.agent_overrides) || {}),
|
||||
...((projectEffort && projectEffort.agent_overrides) || {}),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* #443 — Resolve install-time effort for a given agent, using the same
|
||||
* precedence chain as resolveEffortInternal() in core.cjs, but operating
|
||||
* on a pre-loaded effortCfg object (no loadConfig side-effects at install).
|
||||
*
|
||||
* Precedence (mirrors resolveEffortInternal):
|
||||
* 1. effortCfg.agent_overrides[agentName]
|
||||
* 2. effortCfg.routing_tier_defaults[agentTier] (if effortCfg present)
|
||||
* — OR manifest tier defaults when effortCfg is null
|
||||
* 3. effortCfg.default
|
||||
* 4. 'high' (hardcoded fallback)
|
||||
*
|
||||
* @param {object|null} effortCfg Result of readGsdEffectiveEffortConfig().
|
||||
* @param {string} agentName e.g. 'gsd-planner'
|
||||
* @returns {string} Universal effort string (low/medium/high/xhigh/max/minimal)
|
||||
*/
|
||||
function resolveInstallTimeEffort(effortCfg, agentName) {
|
||||
// Validates each candidate against the canonical EFFORT_SET (sourced once
|
||||
// from core.cjs) before accepting it, mirroring resolveEffortInternal exactly.
|
||||
// Invalid values fall through to the next precedence layer; final fallback 'high'.
|
||||
|
||||
// Step 1: agent_overrides
|
||||
if (effortCfg) {
|
||||
const ao = effortCfg.agent_overrides;
|
||||
if (ao && typeof ao === 'object' && !Array.isArray(ao)) {
|
||||
const v = ao[agentName];
|
||||
if (typeof v === 'string' && GSD_EFFORT_SET.has(v)) return v;
|
||||
}
|
||||
}
|
||||
|
||||
// Step 2: routing_tier_defaults keyed by the agent's catalog tier
|
||||
const { AGENT_DEFAULT_TIERS, EFFORT_MANIFEST_TIER_DEFAULTS, EFFORT_MANIFEST_DEFAULT } = _getGsdEffortCatalog();
|
||||
const agentTier = AGENT_DEFAULT_TIERS[agentName];
|
||||
if (agentTier) {
|
||||
if (effortCfg && effortCfg.routing_tier_defaults &&
|
||||
typeof effortCfg.routing_tier_defaults === 'object' &&
|
||||
!Array.isArray(effortCfg.routing_tier_defaults)) {
|
||||
const v = effortCfg.routing_tier_defaults[agentTier];
|
||||
if (typeof v === 'string' && GSD_EFFORT_SET.has(v)) return v;
|
||||
} else if (!effortCfg) {
|
||||
// No effort config — use manifest tier defaults
|
||||
const v = EFFORT_MANIFEST_TIER_DEFAULTS[agentTier];
|
||||
if (typeof v === 'string' && GSD_EFFORT_SET.has(v)) return v;
|
||||
}
|
||||
// effortCfg exists but has no routing_tier_defaults — fall through
|
||||
}
|
||||
|
||||
// Step 3: effort.default
|
||||
if (effortCfg) {
|
||||
const d = effortCfg.default;
|
||||
if (typeof d === 'string' && GSD_EFFORT_SET.has(d)) return d;
|
||||
}
|
||||
|
||||
// Step 4: manifest default (sourced from config-defaults.manifest.json effort.default)
|
||||
// If even the manifest default is invalid, fall back to 'high'.
|
||||
if (typeof EFFORT_MANIFEST_DEFAULT === 'string' && GSD_EFFORT_SET.has(EFFORT_MANIFEST_DEFAULT)) {
|
||||
return EFFORT_MANIFEST_DEFAULT;
|
||||
}
|
||||
return 'high';
|
||||
}
|
||||
|
||||
/**
|
||||
* #443 — Inject `effort: <value>` into YAML frontmatter of a Claude .md agent
|
||||
* file in a newline-agnostic way (LF and CRLF source files are both handled).
|
||||
*
|
||||
* The function:
|
||||
* - Detects the file's EOL (CRLF if the first `---` line ends with \r\n,
|
||||
* otherwise LF).
|
||||
* - Skips injection if an `effort:` key already exists in the frontmatter
|
||||
* (idempotent).
|
||||
* - Inserts `effort: <value>` immediately before the closing `---` delimiter,
|
||||
* using the same EOL as the surrounding frontmatter so the output file
|
||||
* stays EOL-consistent.
|
||||
* - Returns the original content unchanged when no YAML frontmatter is found.
|
||||
*
|
||||
* @param {string} content Raw file content (may have LF or CRLF endings).
|
||||
* @param {string} effortValue Rendered effort string, e.g. "xhigh".
|
||||
* @returns {string} Updated content with `effort:` injected, or the
|
||||
* original content when no frontmatter is found.
|
||||
*/
|
||||
function injectEffortFrontmatter(content, effortValue) {
|
||||
// Detect the dominant EOL from the first line (the opening `---`).
|
||||
// If the very first `---` is followed by \r\n, treat the whole file as CRLF.
|
||||
const eol = /^---\r\n/.test(content) ? '\r\n' : '\n';
|
||||
|
||||
// Build a frontmatter-matching regex that tolerates an optional \r before
|
||||
// each \n, so we handle both LF and CRLF files without needing to normalise
|
||||
// the whole content.
|
||||
//
|
||||
// Breakdown:
|
||||
// ^---\r?\n — opening delimiter (with optional \r)
|
||||
// ([\s\S]*?) — frontmatter body (non-greedy)
|
||||
// ^---\r?$ — closing delimiter line (optional \r, $ before \n in
|
||||
// multiline mode)
|
||||
// (\r?\n|$) — newline after closing --- (or end of string)
|
||||
//
|
||||
// The `m` flag makes ^ / $ match at every line boundary.
|
||||
const fmRe = /^---\r?\n([\s\S]*?)^---\r?$/m;
|
||||
const match = fmRe.exec(content);
|
||||
if (!match) return content; // no YAML frontmatter — leave unchanged
|
||||
|
||||
// Idempotency guard: don't insert a second effort: line.
|
||||
const fmBody = match[1]; // content between the two `---` lines
|
||||
if (/^effort:/m.test(fmBody)) return content;
|
||||
|
||||
// Locate the exact position of the closing `---` line so we can insert
|
||||
// before it using a simple string splice (avoids re-running the regex and
|
||||
// avoids any edge-cases with $ matching \r differently per engine).
|
||||
const closeIdx = match.index + 4 + fmBody.length; // 4 = len("---\n") (opening)
|
||||
// Actually compute based on the full match start + captured group length:
|
||||
// match[0] = full frontmatter block; match.index = start of that block.
|
||||
// The closing `---` starts at: match.index + ("---" + eol).length + fmBody.length
|
||||
const openLen = 3 + eol.length; // "---" + eol
|
||||
const closingStart = match.index + openLen + fmBody.length;
|
||||
|
||||
const before = content.slice(0, closingStart);
|
||||
const after = content.slice(closingStart);
|
||||
return `${before}effort: ${effortValue}${eol}${after}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* #2517 — Read a single GSD config file (defaults.json or per-project
|
||||
* config.json) into a plain object, returning null on missing/empty files
|
||||
@@ -2691,8 +2923,14 @@ purpose: ${toSingleLine(description)}
|
||||
* Generate a per-agent .toml config file for Codex.
|
||||
* Sets required agent metadata, sandbox_mode, and developer_instructions
|
||||
* from the agent markdown content.
|
||||
*
|
||||
* @param {string} agentName
|
||||
* @param {string} agentContent
|
||||
* @param {object|null} modelOverrides
|
||||
* @param {object|null} runtimeResolver — runtime-aware tier resolver from readGsdRuntimeProfileResolver
|
||||
* @param {object|null} effortCfg — #443: merged effort config from readGsdEffectiveEffortConfig
|
||||
*/
|
||||
function generateCodexAgentToml(agentName, agentContent, modelOverrides = null, runtimeResolver = null) {
|
||||
function generateCodexAgentToml(agentName, agentContent, modelOverrides = null, runtimeResolver = null, effortCfg = null) {
|
||||
const sandboxMode = CODEX_AGENT_SANDBOX[agentName] || 'read-only';
|
||||
const { frontmatter, body } = extractFrontmatterAndBody(agentContent);
|
||||
const frontmatterText = frontmatter || '';
|
||||
@@ -2721,12 +2959,20 @@ function generateCodexAgentToml(agentName, agentContent, modelOverrides = null,
|
||||
const entry = runtimeResolver.resolve(resolvedName) || runtimeResolver.resolve(agentName);
|
||||
if (entry?.model) {
|
||||
lines.push(`model = ${JSON.stringify(entry.model)}`);
|
||||
if (entry.reasoning_effort) {
|
||||
lines.push(`model_reasoning_effort = ${JSON.stringify(entry.reasoning_effort)}`);
|
||||
}
|
||||
// model is resolved here; reasoning_effort from catalog tier is REPLACED by the
|
||||
// unified effort resolver below (#443). Do NOT emit entry.reasoning_effort here.
|
||||
}
|
||||
}
|
||||
|
||||
// #443 — Unified effort for Codex .toml. Uses the same config-driven precedence chain
|
||||
// as the Claude .md effort injection (resolveInstallTimeEffort), so both runtimes read
|
||||
// from the same effort.agent_overrides / effort.routing_tier_defaults / effort.default
|
||||
// config source. Codex does not support 'max' → clamped to 'xhigh' by
|
||||
// gsdRenderEffortForRuntime('codex', ...).
|
||||
const _universalEffortCodex = resolveInstallTimeEffort(effortCfg, resolvedName !== agentName ? resolvedName : agentName);
|
||||
const _renderedEffortCodex = _getGsdEffortCatalog().renderEffortForRuntime('codex', _universalEffortCodex).value;
|
||||
lines.push(`model_reasoning_effort = ${JSON.stringify(_renderedEffortCodex)}`);
|
||||
|
||||
// Agent prompts contain raw backslashes in regexes and shell snippets.
|
||||
// TOML literal multiline strings preserve them without escape parsing.
|
||||
lines.push(`developer_instructions = '''`);
|
||||
@@ -4992,7 +5238,10 @@ function installCodexConfig(targetDir, agentsSrc) {
|
||||
// setting runtime in the project config reaches the Codex emit path is
|
||||
// false (review finding #1).
|
||||
const runtimeResolver = readGsdRuntimeProfileResolver(targetDir);
|
||||
const tomlContent = generateCodexAgentToml(name, content, modelOverrides, runtimeResolver);
|
||||
// #443 — pass unified effort config so model_reasoning_effort in the .toml
|
||||
// follows the same config-driven precedence as the Claude .md effort key.
|
||||
const effortCfg = readGsdEffectiveEffortConfig(targetDir);
|
||||
const tomlContent = generateCodexAgentToml(name, content, modelOverrides, runtimeResolver, effortCfg);
|
||||
fs.writeFileSync(path.join(agentsTomlDir, `${name}.toml`), tomlContent);
|
||||
}
|
||||
|
||||
@@ -8686,6 +8935,20 @@ function install(isGlobal, runtime = 'claude', options = {}) {
|
||||
content = content.replace(/\bClaude Code\b/g, 'Hermes Agent');
|
||||
content = content.replace(/\.claude\//g, '.hermes/');
|
||||
}
|
||||
// #443 — Inject `effort:` into the Claude .md frontmatter ONLY.
|
||||
// Gemini/OpenCode/Qwen/Hermes also produce .md files but break on
|
||||
// unknown frontmatter keys (the repo bans skills:/permissionMode: for
|
||||
// the same reason — see tests/agent-frontmatter.test.cjs).
|
||||
// Claude Code reads per-subagent `effort:` frontmatter (anthropics/claude-code #31536).
|
||||
// Injection is per-runtime at install time because the canonical source
|
||||
// agents/*.md must stay Gemini-safe (no effort: key in source).
|
||||
if (runtime === 'claude') {
|
||||
const _effortCfg = readGsdEffectiveEffortConfig(targetDir);
|
||||
const _agentName = entry.name.replace(/\.md$/, '');
|
||||
const _universalEffort = resolveInstallTimeEffort(_effortCfg, _agentName);
|
||||
const _renderedEffort = _getGsdEffortCatalog().renderEffortForRuntime('claude', _universalEffort).value;
|
||||
content = injectEffortFrontmatter(content, _renderedEffort);
|
||||
}
|
||||
// #3677 — normalize retired `/gsd:<cmd>` colon refs in the agent body
|
||||
// to the canonical hyphen form `/gsd-<cmd>` for hyphen-`name:`
|
||||
// runtimes (claude / qwen / hermes). Self-converting runtimes and
|
||||
@@ -11419,6 +11682,11 @@ module.exports = {
|
||||
installCodexConfig,
|
||||
readGsdRuntimeProfileResolver,
|
||||
readGsdEffectiveModelOverrides,
|
||||
readGsdEffectiveEffortConfig,
|
||||
resolveInstallTimeEffort,
|
||||
injectEffortFrontmatter,
|
||||
get _GSD_EFFORT_MANIFEST_TIER_DEFAULTS() { return _getGsdEffortCatalog().EFFORT_MANIFEST_TIER_DEFAULTS; },
|
||||
get _GSD_EFFORT_MANIFEST_DEFAULT() { return _getGsdEffortCatalog().EFFORT_MANIFEST_DEFAULT; },
|
||||
install,
|
||||
installAllRuntimes,
|
||||
uninstall,
|
||||
|
||||
@@ -972,6 +972,123 @@ The `dynamic_routing` block is **disabled by default** — `enabled: false` (or
|
||||
|
||||
`dynamic_routing` is structurally a *cost lever*: you pay Opus rates only for the hard cases that warrant Opus. Compose with `model_overrides` for per-agent exceptions (override always wins).
|
||||
|
||||
---
|
||||
|
||||
### Effort Control (`effort`) — added in v1.42
|
||||
|
||||
> Unified cross-provider effort knob. Added in [#443](https://github.com/open-gsd/get-shit-done-redux/issues/443).
|
||||
|
||||
Control the reasoning effort of agent invocations with a single config. The universal ladder is:
|
||||
|
||||
```
|
||||
minimal < low < medium < high < xhigh < max
|
||||
```
|
||||
|
||||
Effort is rendered per-runtime: `output_config.effort` for Claude (Claude Code subagent `effort` frontmatter / `CLAUDE_CODE_EFFORT_LEVEL` env), `model_reasoning_effort` for Codex (Responses API `reasoning.effort`).
|
||||
|
||||
**Cross-provider clamping:** `max` is Anthropic-only — it clamps to `xhigh` on Codex. `minimal` is Codex-only — it clamps to `low` on Claude.
|
||||
|
||||
The model-catalog's `reasoning_effort` per-tier hint is a legacy field kept for reference; effort is now config-driven.
|
||||
|
||||
**Precedence (highest → lowest):**
|
||||
1. Invocation override (e.g. `--effort` flag on `resolve-execution`)
|
||||
2. `effort.agent_overrides[<agent-id>]`
|
||||
3. `effort.routing_tier_defaults[<light|standard|heavy>]`
|
||||
4. `effort.default`
|
||||
5. `"high"` (Anthropic Opus 4.8 universal default)
|
||||
|
||||
```json
|
||||
{
|
||||
"effort": {
|
||||
"default": "high",
|
||||
"routing_tier_defaults": {
|
||||
"light": "low",
|
||||
"standard": "high",
|
||||
"heavy": "xhigh"
|
||||
},
|
||||
"agent_overrides": {
|
||||
"gsd-planner": "max"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Settings
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `effort.default` | enum | `"high"` | Global fallback effort level. Applies when no tier or agent override matches. |
|
||||
| `effort.routing_tier_defaults.light` | enum | `"low"` | Effort for light-tier agents (fast mappers/scanners). |
|
||||
| `effort.routing_tier_defaults.standard` | enum | `"high"` | Effort for standard-tier agents (workhorse agents). |
|
||||
| `effort.routing_tier_defaults.heavy` | enum | `"xhigh"` | Effort for heavy-tier agents (deep reasoning). |
|
||||
| `effort.agent_overrides.<agent-id>` | enum | (none) | Per-agent effort override. Beats tier defaults. |
|
||||
|
||||
Valid effort values: `minimal`, `low`, `medium`, `high`, `xhigh`, `max`.
|
||||
|
||||
---
|
||||
|
||||
### Fast Mode (`fast_mode`) — added in v1.42
|
||||
|
||||
> Per-agent fast_mode propagation knob. Added in [#443](https://github.com/open-gsd/get-shit-done-redux/issues/443).
|
||||
|
||||
Control whether fast_mode is propagated to agent invocations. Only accepts real booleans — string `"true"` is rejected.
|
||||
|
||||
**Note:** `fast_mode` is only propagatable via API runtimes (`api` speed:"fast"). Claude Code has no per-subagent fast-mode mechanism — `/fast` is session-level only, so emitting a `fast_mode` frontmatter key on a Claude subagent is a silent no-op. `fast_mode_supported` in `resolve-execution` output tells you if the configured runtime supports it.
|
||||
|
||||
**Precedence (highest → lowest):**
|
||||
1. Invocation override (e.g. `--fast-mode` flag on `resolve-execution`)
|
||||
2. `fast_mode.agent_overrides[<agent-id>]` (boolean)
|
||||
3. `fast_mode.routing_tier_defaults[<light|standard|heavy>]` (boolean)
|
||||
4. `fast_mode.enabled` (boolean)
|
||||
5. `false`
|
||||
|
||||
```json
|
||||
{
|
||||
"fast_mode": {
|
||||
"enabled": false,
|
||||
"routing_tier_defaults": {
|
||||
"light": true,
|
||||
"standard": false,
|
||||
"heavy": false
|
||||
},
|
||||
"agent_overrides": {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Settings
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `fast_mode.enabled` | boolean | `false` | Global fast_mode flag. Only honored when no tier/agent override matches. |
|
||||
| `fast_mode.routing_tier_defaults.light` | boolean | `true` | Fast mode for light-tier agents. |
|
||||
| `fast_mode.routing_tier_defaults.standard` | boolean | `false` | Fast mode for standard-tier agents. |
|
||||
| `fast_mode.routing_tier_defaults.heavy` | boolean | `false` | Fast mode for heavy-tier agents. |
|
||||
| `fast_mode.agent_overrides.<agent-id>` | boolean | (none) | Per-agent fast_mode override. |
|
||||
|
||||
---
|
||||
|
||||
### Execution Query (`resolve-execution`)
|
||||
|
||||
Use `node gsd-tools.cjs resolve-execution <agent-type> [--effort <level>] [--fast-mode <true|false>] [--attempt <n>]` to get the full resolved execution context for an agent:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "opus",
|
||||
"profile": "balanced",
|
||||
"effort": "xhigh",
|
||||
"effort_rendered": "xhigh",
|
||||
"effort_param": "output_config.effort",
|
||||
"effort_propagation": "frontmatter",
|
||||
"fast_mode": false,
|
||||
"fast_mode_supported": false
|
||||
}
|
||||
```
|
||||
|
||||
`effort_param` tells you which runtime parameter to set. `fast_mode_supported` tells you whether the configured runtime supports per-agent fast_mode propagation.
|
||||
|
||||
---
|
||||
|
||||
### Non-Claude Runtimes (Codex, OpenCode, Gemini CLI, Kilo)
|
||||
|
||||
> **Codex CLI minimum supported version: `0.130.0`** (issue [#3562](https://github.com/open-gsd/get-shit-done-redux/issues/3562)).
|
||||
|
||||
@@ -94,3 +94,139 @@ node scripts/ci-test-scope.cjs --base origin/next --head HEAD
|
||||
- Avoid stack-trace or error-message prose assertions. Assert `err.code`, structured JSON fields, or enums — Node minor releases routinely tweak error wording.
|
||||
- Prefer `node:test`, `node:assert/strict`, and `node:test` mocks. No external test frameworks.
|
||||
- Coverage uses `c8` and propagates `NODE_V8_COVERAGE` through the harness's child process.
|
||||
|
||||
---
|
||||
|
||||
## Test strategy: #443 effort + fast_mode engine
|
||||
|
||||
> Feature: unified cross-provider effort and fast_mode knobs (issue #443).
|
||||
> Test files: `tests/feat-443-effort-fast-mode.test.cjs` (unit),
|
||||
> `tests/feat-443-effort-fast-mode.integration.test.cjs` (integration).
|
||||
|
||||
### Testing pyramid
|
||||
|
||||
| Layer | File | What it covers |
|
||||
|---|---|---|
|
||||
| **Unit** | `feat-443-effort-fast-mode.test.cjs` | Pure logic: cascade rules, clamping, escalation math, malformed config handling, schema key validation. No CLI subprocess. |
|
||||
| **Integration** | `feat-443-effort-fast-mode.integration.test.cjs` | Architecture-level invariants: cross-provider validity, totality across the 33-agent registry, CLI JSON contract, config round-trip, fast-mode honesty. Real subprocesses via `runGsdTools`. |
|
||||
| **E2E** *(pending)* | *(not yet wired)* | Propagation layer: effort frontmatter / `CLAUDE_CODE_EFFORT_LEVEL` env actually reaching a spawned Claude Code subagent. See "Gaps" below. |
|
||||
|
||||
### Architectural invariants
|
||||
|
||||
Each invariant exists to prevent a specific class of production failure.
|
||||
|
||||
#### (a) Cross-provider validity
|
||||
|
||||
**What:** `renderEffortForRuntime(runtime, universalEffort).value` must always
|
||||
be a member of the runtime's real provider enum. Ground-truth enums are defined
|
||||
as local constants in the test — not sourced from the implementation.
|
||||
|
||||
```
|
||||
PROVIDER_EFFORT_ENUMS = {
|
||||
claude: Set { 'low', 'medium', 'high', 'xhigh', 'max' } // Anthropic output_config.effort
|
||||
codex: Set { 'minimal', 'low', 'medium', 'high', 'xhigh' } // OpenAI model_reasoning_effort
|
||||
}
|
||||
```
|
||||
|
||||
**Why:** Passing a value outside these sets results in a 400 from the real API.
|
||||
The clamping logic (`max -> xhigh` for codex; `minimal -> low` for claude) must
|
||||
hold for every cell of the VALID_EFFORTS × runtimes matrix.
|
||||
|
||||
#### (b) Param/channel contract
|
||||
|
||||
**What:** Each runtime exposes a stable `param` string (the native API field
|
||||
name) and `channel` (how the value is propagated). Unknown runtimes return
|
||||
`param: null, channel: null` and pass the effort value through unchanged.
|
||||
|
||||
**Why:** Callers read `.param` to construct the dispatch payload. A regression
|
||||
here would silently drop effort from subagent invocations.
|
||||
|
||||
#### (c) Resolve-execution JSON contract
|
||||
|
||||
**What:** The `gsd-tools resolve-execution <agent>` command emits a JSON object
|
||||
with all eight keys present and typed correctly: `model` (string), `profile`
|
||||
(string), `effort` (VALID_EFFORTS member), `effort_rendered` (string),
|
||||
`effort_param` (string|null), `effort_propagation` (string|null), `fast_mode`
|
||||
(boolean), `fast_mode_supported` (boolean).
|
||||
|
||||
**Why:** Orchestrators and workflow dispatchers parse this JSON. A missing or
|
||||
mistyped field silently breaks downstream consumers.
|
||||
|
||||
#### (d) Totality across the real registry
|
||||
|
||||
**What:** For every agent in the 33-agent registry, `resolveEffortInternal`
|
||||
returns a VALID_EFFORTS member (never undefined/null), `resolveFastModeInternal`
|
||||
returns a strict boolean, and `renderEffortForRuntime('claude', effort)` stays
|
||||
within the claude provider enum.
|
||||
|
||||
**Why:** A catalog addition that introduces a missing `routingTier` mapping
|
||||
would otherwise produce `undefined` and propagate silently.
|
||||
|
||||
#### (e) Fast-mode honesty invariant
|
||||
|
||||
**What:** When the runtime is `claude`, `fast_mode_supported` in
|
||||
resolve-execution output is always `false`, regardless of the fast_mode config.
|
||||
`RUNTIMES_WITH_FAST_MODE` contains only `'api'`.
|
||||
|
||||
**Why:** Claude Code's `/fast` toggle is session-level only. Emitting
|
||||
`fast_mode: true` as frontmatter on a Claude subagent is a silent no-op.
|
||||
Advertising `fast_mode_supported: true` for claude would cause orchestrators to
|
||||
believe the knob was wired when it is not.
|
||||
|
||||
#### (f) Precedence first-valid-wins
|
||||
|
||||
**What:** Both effort and fast_mode use a layered cascade. The test table covers
|
||||
all four effort layers (invocation override → agent_overrides →
|
||||
routing_tier_defaults → default) and all five fast_mode layers, including the
|
||||
case where an invalid value at a higher layer correctly falls through.
|
||||
|
||||
**Why:** Silent precedence bugs (e.g., a numeric value in agent_overrides not
|
||||
being rejected) would override intentional user config.
|
||||
|
||||
#### (g) Dynamic-routing composition
|
||||
|
||||
**What:** `resolveEffortForTier` escalates effort by attempt number
|
||||
independently of the model tier mapping. The test verifies the effort ladder
|
||||
(`low -> medium -> high -> xhigh -> max`), the `max` clamp, the
|
||||
`max_escalations` cap, and that `escalate_on_failure: false` suppresses
|
||||
escalation entirely.
|
||||
|
||||
**Why:** Effort escalation and model escalation share configuration
|
||||
(`dynamic_routing`) but must operate independently; coupling them would cause
|
||||
over-escalation or under-escalation.
|
||||
|
||||
#### (h) Config-tooling round-trip
|
||||
|
||||
**What:** `gsd-tools config-set` accepts all new key namespaces
|
||||
(`effort.default`, `effort.routing_tier_defaults.<tier>`,
|
||||
`effort.agent_overrides.<agent>`, `fast_mode.enabled`,
|
||||
`fast_mode.routing_tier_defaults.<tier>`, `fast_mode.agent_overrides.<agent>`)
|
||||
without an "Unknown config key" error, and values set via `config-set` are
|
||||
reflected in `resolve-execution` output.
|
||||
|
||||
**Why:** The schema validation gate (`VALID_CONFIG_KEYS` + `DYNAMIC_KEY_PATTERNS`)
|
||||
is separate from the resolver logic. A key missing from the schema would produce
|
||||
a silent write failure and appear as a bug only at runtime.
|
||||
|
||||
### Coverage targets
|
||||
|
||||
| Suite | Target |
|
||||
|---|---|
|
||||
| Unit | Every cascade rule, every fallthrough, every clamp. All function branches in `resolveEffortInternal`, `resolveFastModeInternal`, `resolveEffortForTier`, `renderEffortForRuntime`. |
|
||||
| Integration | All 8 architectural invariants. All 33 registered agents. All 6 provider × effort combinations for the valid-enum check. Full config-set key namespace. |
|
||||
|
||||
### Gaps / not yet covered
|
||||
|
||||
**E2E orchestrator-spawn-propagation layer (pending follow-up wiring):**
|
||||
The integration tests verify that GSD resolves and renders effort values
|
||||
correctly. They do NOT verify that the rendered values actually reach a spawned
|
||||
Claude Code or Codex subagent at runtime. Specifically uncovered:
|
||||
|
||||
- `CLAUDE_CODE_EFFORT_LEVEL` env var being set and read by a spawned claude subprocess
|
||||
- `output_config.effort` frontmatter key surviving the AGENTS.md template substitution
|
||||
- `model_reasoning_effort` field surviving serialization into a Codex API request body
|
||||
- Fast-mode `speed: "fast"` field reaching an `api`-runtime request when `fast_mode_supported: true`
|
||||
|
||||
These require spawning real subagents (or stubs thereof) and asserting on the
|
||||
process environment / request payload — a scope that belongs in a future E2E
|
||||
suite under `*.slow.test.cjs` or dedicated fixture-driven integration work.
|
||||
|
||||
93
docs/adr/443-opus48-unified-effort-and-fast-mode-routing.md
Normal file
93
docs/adr/443-opus48-unified-effort-and-fast-mode-routing.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# ADR 443: Unified cross-provider effort controls and fast-mode-aware routing
|
||||
|
||||
- **Status:** Proposed (2026-05-28)
|
||||
- **Date:** 2026-05-28
|
||||
- **Tracking issue:** [#443](https://github.com/open-gsd/get-shit-done-redux/issues/443)
|
||||
|
||||
## Context
|
||||
|
||||
### Effort control and fast mode in Claude Opus 4.8
|
||||
|
||||
Claude Opus 4.8 introduced two orthogonal execution controls relevant to GSD's agent orchestration:
|
||||
|
||||
1. **Effort control** — API request field `output_config.effort` (string enum). Anthropic levels: `low`, `medium`, `high`, `xhigh`, `max`; Opus 4.8 defaults to `high`. In Claude Code it is exposed as `/effort`, the `--effort` CLI flag, the `CLAUDE_CODE_EFFORT_LEVEL` env var, the `effortLevel` settings.json key (accepts `low`/`medium`/`high`/`xhigh`; `max` is session-only), and — critically for orchestration — a per-subagent `effort` frontmatter key (shipped per anthropics/claude-code issue #31536, CLOSED/COMPLETED).
|
||||
|
||||
2. **Fast mode** — API request field `speed` (`standard`|`fast`); `fast` enables high output-tokens-per-second inference. Pricing for Opus 4.8 fast mode is $10/$50 per MTok in/out vs $5/$25 standard. In Claude Code it is the interactive `/fast` toggle ONLY — there is no settings.json key, env var, or subagent-frontmatter mechanism to enable fast mode for a spawned subagent.
|
||||
|
||||
GSD already routes WHICH model runs a task (routingTier `heavy`/`standard`/`light`, model_profile `quality`/`balanced`/`budget`/`adaptive`/`inherit`, `model_overrides`, and dynamic_routing escalation). It had no way to control HOW HARD the model reasons or WHICH speed tier it uses.
|
||||
|
||||
### The "flavor text" problem: issue #2517
|
||||
|
||||
Issue #2517 added `resolveReasoningEffortInternal` and made `query resolve-model` emit a `reasoning_effort` field derived from the Codex runtime's per-tier catalog values (`model-catalog.json` `runtimeTierDefaults.codex.*.reasoning_effort`). However, a codebase audit found that NO orchestrator, workflow, or agent ever consumes that emitted field — it is never passed to an actual Codex invocation. The resolver computed a value and a test asserted the computed JSON, but the value reached no runtime. The feature was inert ("flavor text, no code"): asserting a resolver's return value is not the same as asserting the control reaches the model.
|
||||
|
||||
### Cross-provider effort enum mismatch
|
||||
|
||||
The two providers' effort enums are NOT identical:
|
||||
|
||||
- **Anthropic/Claude (Opus 4.8):** `low`, `medium`, `high`, `xhigh`, `max` (has `max`; no `minimal`)
|
||||
- **OpenAI/Codex** (`model_reasoning_effort` / Responses API `reasoning.effort`; SDK `ReasoningEffort` ranks `none=0`, `minimal=1`, `low=2`, `medium=3`, `high=4`, `xhigh=5`): `minimal`, `low`, `medium`, `high`, `xhigh` (has `minimal`; no `max`)
|
||||
|
||||
Common core: `low`, `medium`, `high`, `xhigh`.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Introduce a single universal `effort` config knob** (and an orthogonal `fast_mode` knob) that compose with model selection rather than replace it. Resolution precedence mirrors the existing model cascade: (1) orchestrator invocation override, (2) `effort.agent_overrides[agent]`, (3) `effort.routing_tier_defaults[routingTier]`, (4) `effort.default`, (5) built-in default `high`. Same cascade for `fast_mode` with built-in default `false`. Invalid enum values at any level are ignored and fall through (mirrors the `VALID_TIERS` gate in `resolveModelInternal`) so a typo never silently breaks resolution.
|
||||
|
||||
2. **The universal effort value is provider-agnostic; a per-runtime renderer maps it to each runtime's wire parameter**, clamping the genuinely-unique tail levels:
|
||||
|
||||
- **Claude / API:** param `output_config.effort` (Claude Code: subagent `effort` frontmatter / `CLAUDE_CODE_EFFORT_LEVEL` env). `minimal` clamps to `low` (Claude has no `minimal`); `low`/`medium`/`high`/`xhigh`/`max` pass through.
|
||||
- **Codex:** param `model_reasoning_effort` (Responses API `reasoning.effort`). `max` clamps to `xhigh` (Codex has no `max`); `minimal`/`low`/`medium`/`high`/`xhigh` pass through.
|
||||
|
||||
| Universal level | Claude rendering | Codex rendering |
|
||||
| --- | --- | --- |
|
||||
| `minimal` | `low` (clamped) | `minimal` |
|
||||
| `low` | `low` | `low` |
|
||||
| `medium` | `medium` | `medium` |
|
||||
| `high` (default) | `high` | `high` |
|
||||
| `xhigh` | `xhigh` | `xhigh` |
|
||||
| `max` | `max` | `xhigh` (clamped) |
|
||||
|
||||
3. **Fold the inert `reasoning_effort` output into this unified model.** `query resolve-model` is preserved for back-compat; a NEW `query resolve-execution` is the superset that emits: `model`, `effort` (universal), the per-runtime rendered effort, the wire param name, the propagation channel, `fast_mode`, and `fast_mode_supported`. Each config key ships help text naming exactly which runtime field/invocation it drives.
|
||||
|
||||
4. **Make effort actually reach the runtime (close the flavor-text gap).** Claude is first-class: the resolved effort propagates to spawned subagents via the `effort` frontmatter / `CLAUDE_CODE_EFFORT_LEVEL` env. Tests assert end-to-end propagation, not just resolver return values.
|
||||
|
||||
5. **Fast mode honesty:** because Claude Code has no per-subagent fast-mode mechanism, `fast_mode` is resolved and surfaced (with a `fast_mode_supported` flag, `false` for the claude runtime's subagents) but is NEVER emitted as a fake frontmatter key — doing so would be a silent no-op. It propagates only where the runtime supports it (API `speed:"fast"`).
|
||||
|
||||
6. **Dynamic-routing integration is additive:** a new effort-escalation path (effort steps up the ladder on a failed attempt BEFORE model-tier escalation) is gated on the same `dynamic_routing.enabled` / `escalate_on_failure` switches and does NOT modify `resolveModelForTier` (so existing feat-3024 behavior is unchanged).
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- One coherent effort policy across all runtimes; Claude effort is first-class and actually wired.
|
||||
- The dead `reasoning_effort` field becomes meaningful; finer-grained cost/quality control (a light-tier scanning agent can run `low` effort; a heavy planning agent `xhigh`) without changing model class.
|
||||
- Effort-first escalation reduces unnecessary model upgrades.
|
||||
- Cross-provider clamping is explicit and documented.
|
||||
|
||||
### Negative
|
||||
|
||||
- The universal enum is the union of two providers' ladders, so two levels (`max`, `minimal`) are runtime-specific and clamp when rendered to the other provider — users must understand the mapping (mitigated by help text and the table above).
|
||||
- Fast mode remains asymmetric: it cannot be forced per-subagent on Claude Code, only at session level or on API-direct runtimes.
|
||||
- Updating issue-2517's tests to assert real wiring is a deliberate behavior/contract change (the old "null on claude" assertion encoded the now-false premise that Claude has no effort control).
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
**(a) Global effort env override** (e.g. a single `CLAUDE_CODE_EFFORT_LEVEL` for the whole session) — rejected: caps cost but starves heavy agents that legitimately need deep reasoning; static global breaks the per-tier design.
|
||||
|
||||
**(b) Model selection alone (status quo)** — rejected: choosing Haiku for light tasks reduces cost, but within one model class there is no way to tune reasoning depth; a quality profile pays full reasoning cost even for scanning.
|
||||
|
||||
**(c) Static per-agent effort only** — rejected: loses context sensitivity; the same agent doing trivial vs complex work should not always get the same effort.
|
||||
|
||||
**(d) A separate `effort` field kept fully parallel to Codex's existing `reasoning_effort` (two independent lanes)** — rejected: produces two overlapping fields that can diverge and confuse; Codex's `reasoning_effort` is better modeled as one rendering of the single universal effort.
|
||||
|
||||
**(e) Overloading the existing `reasoning_effort` field to also carry Claude effort** — rejected: it would conflate a Codex-specific wire name with the universal concept and break the clean per-runtime rendering.
|
||||
|
||||
## References
|
||||
|
||||
- Tracking issue: #443
|
||||
- Prior art (inert reasoning_effort): #2517; `tests/issue-2517-runtime-aware-profiles.test.cjs`
|
||||
- dynamic_routing escalation: #3024; `tests/feat-3024-dynamic-routing.test.cjs`
|
||||
- phase-type tiers: #3023
|
||||
- Anthropic effort API: `output_config.effort` (`low`/`medium`/`high`/`xhigh`/`max`); fast mode: `speed` (`standard`/`fast`)
|
||||
- Claude Code effort: `/effort`, `--effort`, `CLAUDE_CODE_EFFORT_LEVEL`, `effortLevel` setting, subagent `effort` frontmatter (anthropics/claude-code #31536, completed); fast mode: `/fast` (interactive only)
|
||||
- OpenAI Codex effort: `model_reasoning_effort` config key; Responses API `reasoning.effort`; `ReasoningEffort` enum `none<minimal<low<medium<high<xhigh`
|
||||
@@ -538,6 +538,81 @@ async function runCommand(command, args, cwd, raw, defaultValue, originalCommand
|
||||
break;
|
||||
}
|
||||
|
||||
case 'resolve-execution': {
|
||||
// Deterministic flag parsing: consume --flag <value> pairs first,
|
||||
// then the AGENT is the single remaining positional.
|
||||
// Supports both orderings: <agent> --flag val AND --flag val <agent>.
|
||||
// Also supports --flag=value form (same convention as --cwd= above).
|
||||
const execArgs = args.slice(1);
|
||||
let effortOverride;
|
||||
let fastModeOverride;
|
||||
let attempt;
|
||||
const positionals = [];
|
||||
for (let i = 0; i < execArgs.length; i++) {
|
||||
const a = execArgs[i];
|
||||
// --effort=<val> form
|
||||
if (a.startsWith('--effort=')) {
|
||||
effortOverride = a.slice('--effort='.length);
|
||||
continue;
|
||||
}
|
||||
// --fast-mode=<val> form
|
||||
if (a.startsWith('--fast-mode=')) {
|
||||
const v = a.slice('--fast-mode='.length);
|
||||
fastModeOverride = v === 'true' ? true : v === 'false' ? false : undefined;
|
||||
continue;
|
||||
}
|
||||
// --attempt=<val> form
|
||||
if (a.startsWith('--attempt=')) {
|
||||
const v = a.slice('--attempt='.length);
|
||||
const n = parseInt(v, 10);
|
||||
if (!Number.isInteger(n) || n < 0) error('--attempt requires a non-negative integer', ERROR_REASON.USAGE);
|
||||
attempt = n;
|
||||
continue;
|
||||
}
|
||||
// --effort <val>
|
||||
if (a === '--effort') {
|
||||
const val = execArgs[i + 1];
|
||||
if (val === undefined || val.startsWith('--')) error('Missing value for --effort', ERROR_REASON.USAGE);
|
||||
effortOverride = val;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
// --fast-mode <val>
|
||||
if (a === '--fast-mode') {
|
||||
const val = execArgs[i + 1];
|
||||
if (val === undefined || val.startsWith('--')) error('Missing value for --fast-mode', ERROR_REASON.USAGE);
|
||||
fastModeOverride = val === 'true' ? true : val === 'false' ? false : undefined;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
// --attempt <val>
|
||||
if (a === '--attempt') {
|
||||
const val = execArgs[i + 1];
|
||||
if (val === undefined || val.startsWith('--')) error('Missing value for --attempt', ERROR_REASON.USAGE);
|
||||
const n = parseInt(val, 10);
|
||||
if (!Number.isInteger(n) || n < 0) error('--attempt requires a non-negative integer', ERROR_REASON.USAGE);
|
||||
attempt = n;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
// --raw is handled by top-level arg processing; skip it here
|
||||
if (a === '--raw') continue;
|
||||
// Unknown flag
|
||||
if (a.startsWith('-')) error(`Unknown flag for resolve-execution: ${a}`, ERROR_REASON.USAGE);
|
||||
// Positional
|
||||
positionals.push(a);
|
||||
}
|
||||
if (positionals.length === 0) error('agent-type required', ERROR_REASON.USAGE);
|
||||
if (positionals.length > 1) error(`resolve-execution requires exactly one agent-type argument; got: ${positionals.join(', ')}`, ERROR_REASON.USAGE);
|
||||
const agentTypeArg = positionals[0];
|
||||
commands.cmdResolveExecution(cwd, agentTypeArg, raw, {
|
||||
effortOverride,
|
||||
fastModeOverride,
|
||||
attempt,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case 'find-phase': {
|
||||
// Phase 6 (#3575): dispatch via SDK executeForCjs when available.
|
||||
// SDK handler: findPhase in sdk/src/query/phase.ts.
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { execGit, platformWriteSync, platformReadSync, platformEnsureDir } = require('./shell-command-projection.cjs');
|
||||
const { loadConfig, isGitIgnored, normalizePhaseName, comparePhaseNum, getArchivedPhaseDirs, generateSlugInternal, getMilestoneInfo, getMilestonePhaseFilter, resolveModelInternal, resolveReasoningEffortInternal, stripShippedMilestones, extractCurrentMilestone, toPosixPath, output, error, findPhaseInternal, extractOneLinerFromBody, getRoadmapPhaseInternal } = require('./core.cjs');
|
||||
const { loadConfig, isGitIgnored, normalizePhaseName, comparePhaseNum, getArchivedPhaseDirs, generateSlugInternal, getMilestoneInfo, getMilestonePhaseFilter, resolveModelInternal, resolveEffortInternal, resolveFastModeInternal, resolveEffortForTier, stripShippedMilestones, extractCurrentMilestone, toPosixPath, output, error, findPhaseInternal, extractOneLinerFromBody, getRoadmapPhaseInternal } = require('./core.cjs');
|
||||
const { renderEffortForRuntime, RUNTIMES_WITH_FAST_MODE } = require('./model-catalog.cjs');
|
||||
const { planningDir, planningPaths } = require('./planning-workspace.cjs');
|
||||
const { extractFrontmatter } = require('./frontmatter.cjs');
|
||||
const { MODEL_PROFILES } = require('./model-profiles.cjs');
|
||||
@@ -241,16 +242,71 @@ function cmdResolveModel(cwd, agentType, raw) {
|
||||
const config = loadConfig(cwd);
|
||||
const profile = config.model_profile || 'balanced';
|
||||
const model = resolveModelInternal(cwd, agentType);
|
||||
const reasoningEffort = resolveReasoningEffortInternal(cwd, agentType);
|
||||
const effort = resolveEffortInternal(cwd, agentType);
|
||||
|
||||
const agentModels = MODEL_PROFILES[agentType];
|
||||
const result = agentModels
|
||||
? { model, profile }
|
||||
: { model, profile, unknown_agent: true };
|
||||
if (reasoningEffort) result.reasoning_effort = reasoningEffort;
|
||||
? { model, profile, effort }
|
||||
: { model, profile, effort, unknown_agent: true };
|
||||
output(result, raw, model);
|
||||
}
|
||||
|
||||
/**
|
||||
* #443 — Superset execution query: model + unified effort + fast_mode.
|
||||
*
|
||||
* Emits JSON:
|
||||
* { model, profile, effort, effort_rendered, effort_param, effort_propagation,
|
||||
* fast_mode, fast_mode_supported, [unknown_agent] }
|
||||
*
|
||||
* Flags: --effort <level>, --fast-mode <true|false>, --attempt <n>
|
||||
*
|
||||
* @param {string} cwd
|
||||
* @param {string} agentType
|
||||
* @param {boolean} raw
|
||||
* @param {{ effortOverride?: string, fastModeOverride?: boolean, attempt?: number }} [opts]
|
||||
*/
|
||||
function cmdResolveExecution(cwd, agentType, raw, opts) {
|
||||
if (!agentType) {
|
||||
error('agent-type required');
|
||||
}
|
||||
|
||||
opts = opts || {};
|
||||
const config = loadConfig(cwd);
|
||||
const profile = config.model_profile || 'balanced';
|
||||
const model = resolveModelInternal(cwd, agentType);
|
||||
|
||||
const effortOpts = {};
|
||||
if (typeof opts.effortOverride === 'string') effortOpts.override = opts.effortOverride;
|
||||
|
||||
const fastModeOpts = {};
|
||||
if (typeof opts.fastModeOverride === 'boolean') fastModeOpts.override = opts.fastModeOverride;
|
||||
|
||||
const effort = (opts.attempt !== undefined && opts.attempt !== null)
|
||||
? resolveEffortForTier(cwd, agentType, opts.attempt)
|
||||
: resolveEffortInternal(cwd, agentType, effortOpts);
|
||||
|
||||
const fastMode = resolveFastModeInternal(cwd, agentType, fastModeOpts);
|
||||
|
||||
const runtime = config.runtime || 'claude';
|
||||
const rendered = renderEffortForRuntime(runtime, effort);
|
||||
|
||||
const fastModeSupported = RUNTIMES_WITH_FAST_MODE.has(runtime);
|
||||
|
||||
const agentModels = MODEL_PROFILES[agentType];
|
||||
const result = {
|
||||
model,
|
||||
profile,
|
||||
effort,
|
||||
effort_rendered: rendered.value,
|
||||
effort_param: rendered.param,
|
||||
effort_propagation: rendered.channel,
|
||||
fast_mode: fastMode,
|
||||
fast_mode_supported: fastModeSupported,
|
||||
};
|
||||
if (!agentModels) result.unknown_agent = true;
|
||||
output(result, raw, effort);
|
||||
}
|
||||
|
||||
function cmdCommit(cwd, message, files, raw, amend, noVerify) {
|
||||
if (!message && !amend) {
|
||||
error('commit message required');
|
||||
@@ -1117,6 +1173,7 @@ module.exports = {
|
||||
cmdVerifyPathExists,
|
||||
cmdHistoryDigest,
|
||||
cmdResolveModel,
|
||||
cmdResolveExecution,
|
||||
cmdCommit,
|
||||
cmdCommitToSubrepo,
|
||||
cmdSummaryExtract,
|
||||
|
||||
@@ -7,7 +7,7 @@ const os = require('os');
|
||||
const path = require('path');
|
||||
const { execGit, platformWriteSync, platformReadSync, platformEnsureDir } = require('./shell-command-projection.cjs');
|
||||
const { MODEL_PROFILES, AGENT_TO_PHASE_TYPE, VALID_PHASE_TYPES, AGENT_DEFAULT_TIERS, VALID_AGENT_TIERS, nextTier } = require('./model-profiles.cjs');
|
||||
const { MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, KNOWN_RUNTIMES, RUNTIMES_WITH_REASONING_EFFORT } = require('./model-catalog.cjs');
|
||||
const { MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, KNOWN_RUNTIMES, RUNTIMES_WITH_REASONING_EFFORT, renderEffortForRuntime, RUNTIMES_WITH_FAST_MODE } = require('./model-catalog.cjs');
|
||||
const {
|
||||
resolveWorktreeContext,
|
||||
parseWorktreePorcelain: parseWorktreePorcelainPolicy,
|
||||
@@ -410,7 +410,7 @@ function loadConfig(cwd, options = {}) {
|
||||
// Dynamic-pattern top-level containers (e.g. review, model_profile_overrides)
|
||||
...DYNAMIC_KEY_PATTERNS.map(p => p.topLevel),
|
||||
// Internal keys loadConfig reads but config-set doesn't expose
|
||||
'model_overrides', 'context_window', 'resolve_model_ids', 'claude_md_path',
|
||||
'model_overrides', 'context_window', 'resolve_model_ids', 'claude_md_path', 'effort', 'fast_mode',
|
||||
// Deprecated keys (still accepted for migration, not in config-set)
|
||||
// 'branching_strategy' is kept here as a safety net: it is migrated to
|
||||
// git.branching_strategy above (#3523), but on the first read of a root
|
||||
@@ -514,6 +514,10 @@ function loadConfig(cwd, options = {}) {
|
||||
// unknown runtime/tier names at load time, not silently (review finding #10).
|
||||
runtime: parsed.runtime || null,
|
||||
model_profile_overrides: parsed.model_profile_overrides || null,
|
||||
// #443 — effort/fast_mode: pass through from config.json; resolvers handle
|
||||
// defaults + tier lookups internally.
|
||||
effort: parsed.effort || null,
|
||||
fast_mode: parsed.fast_mode || null,
|
||||
agent_skills: parsed.agent_skills || {},
|
||||
manager: parsed.manager || {},
|
||||
response_language: get('response_language') || null,
|
||||
@@ -558,6 +562,8 @@ function loadConfig(cwd, options = {}) {
|
||||
model_overrides: globalDefaults.model_overrides || null,
|
||||
models: globalDefaults.models || null,
|
||||
dynamic_routing: globalDefaults.dynamic_routing || null,
|
||||
effort: globalDefaults.effort || null,
|
||||
fast_mode: globalDefaults.fast_mode || null,
|
||||
agent_skills: globalDefaults.agent_skills || {},
|
||||
response_language: globalDefaults.response_language || null,
|
||||
};
|
||||
@@ -1248,9 +1254,9 @@ function _resetRuntimeWarningCacheForTests() {
|
||||
/**
|
||||
* #2517 — Resolve the runtime-aware tier entry for (runtime, tier).
|
||||
*
|
||||
* Single source of truth shared by core.cjs (resolveModelInternal /
|
||||
* resolveReasoningEffortInternal) and bin/install.js (Codex/OpenCode TOML emit
|
||||
* paths). Always merges built-in defaults with user overrides at the field
|
||||
* Single source of truth shared by core.cjs (resolveModelInternal)
|
||||
* and bin/install.js (Codex/OpenCode TOML emit paths). Always merges
|
||||
* built-in defaults with user overrides at the field
|
||||
* level so partial overrides keep the unspecified fields:
|
||||
*
|
||||
* `{ codex: { opus: "gpt-5-pro" } }` keeps reasoning_effort: 'xhigh'
|
||||
@@ -1287,7 +1293,7 @@ function resolveTierEntry({ runtime, tier, overrides }) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Convenience wrapper used by resolveModelInternal / resolveReasoningEffortInternal.
|
||||
* Convenience wrapper used by resolveModelInternal.
|
||||
* Pulls runtime + overrides out of a loaded config and delegates to resolveTierEntry.
|
||||
*/
|
||||
function _resolveRuntimeTier(config, tier) {
|
||||
@@ -1475,66 +1481,209 @@ function resolveModelForTier(cwd, agentType, attempt) {
|
||||
return alias;
|
||||
}
|
||||
|
||||
// ─── #443 — Unified effort + fast_mode resolvers ─────────────────────────────
|
||||
//
|
||||
// Universal effort ladder (ordered):
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
const EFFORT_SET = new Set(VALID_EFFORTS);
|
||||
|
||||
/**
|
||||
* #2517 — Resolve runtime-specific reasoning_effort for an agent.
|
||||
* Returns null unless:
|
||||
* - `runtime` is explicitly set in config,
|
||||
* - the runtime supports reasoning_effort (currently: codex),
|
||||
* - profile is not 'inherit',
|
||||
* - the resolved tier entry has a `reasoning_effort` value.
|
||||
*
|
||||
* Never returns a value for Claude — keeps reasoning_effort out of Claude spawn paths.
|
||||
* Walk one step up the effort ladder from `e`. Returns the next level, or
|
||||
* the same level if already at the top.
|
||||
*/
|
||||
function resolveReasoningEffortInternal(cwd, agentType) {
|
||||
function nextEffort(e) {
|
||||
const i = VALID_EFFORTS.indexOf(e);
|
||||
if (i < 0) return null;
|
||||
return VALID_EFFORTS[Math.min(i + 1, VALID_EFFORTS.length - 1)];
|
||||
}
|
||||
|
||||
/**
|
||||
* #443 — Resolve a universal effort string for (cwd, agentType).
|
||||
*
|
||||
* Precedence (first valid wins; invalid/wrong-type values are IGNORED and fall
|
||||
* through — mirrors the VALID_TIERS gate pattern in resolveModelInternal):
|
||||
* 1. opts.override (if in EFFORT_SET)
|
||||
* 2. config.effort.agent_overrides[agentType] (if valid)
|
||||
* 3. config.effort.routing_tier_defaults[ AGENT_DEFAULT_TIERS[agentType] ] (agent known + valid)
|
||||
* 4. config.effort.default (if valid)
|
||||
* 5. 'high' (Anthropic Opus 4.8 universal default)
|
||||
*
|
||||
* Handles: config.effort missing; effort.* non-object/malformed; unknown
|
||||
* agentType skips step 3; numeric/boolean garbage ignored.
|
||||
*
|
||||
* @param {string} cwd - Project directory.
|
||||
* @param {string} agentType - Agent name.
|
||||
* @param {{ override?: string }} [opts]
|
||||
* @returns {string} A valid effort string.
|
||||
*/
|
||||
function resolveEffortInternal(cwd, agentType, opts) {
|
||||
// Step 1: invocation override
|
||||
if (opts && typeof opts.override === 'string' && EFFORT_SET.has(opts.override)) {
|
||||
return opts.override;
|
||||
}
|
||||
|
||||
const config = loadConfig(cwd);
|
||||
if (!config.runtime) return null;
|
||||
// Strict allowlist: reasoning_effort only propagates for runtimes whose
|
||||
// install path actually accepts it. Adding a new runtime here is the only
|
||||
// way to enable effort propagation — overrides cannot bypass the gate.
|
||||
// Without this, a typo in `runtime` (e.g. `"codx"`) plus a user override
|
||||
// for that typo would leak `xhigh` into a Claude or unknown install
|
||||
// (review finding #3).
|
||||
if (!RUNTIMES_WITH_REASONING_EFFORT.has(config.runtime)) return null;
|
||||
// Per-agent override means user supplied a fully-qualified ID; reasoning_effort
|
||||
// for that case must be set via per-agent mechanism, not tier inference.
|
||||
if (config.model_overrides?.[agentType]) return null;
|
||||
const effortCfg = (config.effort && typeof config.effort === 'object' && !Array.isArray(config.effort))
|
||||
? config.effort
|
||||
: null;
|
||||
|
||||
const profile = String(config.model_profile || 'balanced').toLowerCase();
|
||||
const agentModels = MODEL_PROFILES[agentType];
|
||||
if (!agentModels) return null;
|
||||
// Step 2: agent_overrides
|
||||
if (effortCfg) {
|
||||
const ao = effortCfg.agent_overrides;
|
||||
if (ao && typeof ao === 'object' && !Array.isArray(ao)) {
|
||||
const v = ao[agentType];
|
||||
if (typeof v === 'string' && EFFORT_SET.has(v)) return v;
|
||||
}
|
||||
}
|
||||
|
||||
// #3023 (CR Major): mirror the phase-type tier lookup from
|
||||
// resolveModelInternal. Without this, `model` and `reasoning_effort`
|
||||
// derive from different tier sources on Codex when models.<phase_type>
|
||||
// overrides the profile.
|
||||
//
|
||||
// #3030 CR follow-up: do NOT short-circuit on profile === 'inherit'
|
||||
// before reading the phase-type tier. A config like
|
||||
// { model_profile: 'inherit', models: { execution: 'opus' } }
|
||||
// must produce the opus runtime effort, not null. Compute tier from
|
||||
// phase-type first; only fall back to profile when there's no valid
|
||||
// phase-type override; only return null when the resolved tier is
|
||||
// 'inherit' or unknown.
|
||||
const phaseType = AGENT_TO_PHASE_TYPE[agentType];
|
||||
const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object')
|
||||
? config.models[phaseType]
|
||||
: undefined;
|
||||
// Explicit phase-type 'inherit' is the user opting out of tier-based
|
||||
// effort for this phase — return null instead of falling through to
|
||||
// profile (which would silently emit the profile's effort and
|
||||
// contradict the user's choice).
|
||||
if (phaseTypeTier === 'inherit') return null;
|
||||
const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku']);
|
||||
const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier))
|
||||
? phaseTypeTier
|
||||
: (profile === 'inherit'
|
||||
? 'inherit'
|
||||
: (agentModels[profile] || agentModels['balanced']));
|
||||
// 'inherit' (from profile fallback) yields no runtime effort.
|
||||
if (!tier || tier === 'inherit') return null;
|
||||
// Step 3: routing_tier_defaults by agent's default tier.
|
||||
// Manifest tier defaults are only used when there is NO effort config block at all
|
||||
// (effortCfg === null). When the user explicitly sets an effort block, we respect
|
||||
// their explicit routing_tier_defaults (if set) and fall through to effort.default
|
||||
// if they didn't set them. This prevents the manifest tier defaults from silently
|
||||
// overriding a user's `effort: { default: "medium" }`.
|
||||
const agentTier = AGENT_DEFAULT_TIERS[agentType];
|
||||
if (agentTier) {
|
||||
if (effortCfg && effortCfg.routing_tier_defaults &&
|
||||
typeof effortCfg.routing_tier_defaults === 'object' &&
|
||||
!Array.isArray(effortCfg.routing_tier_defaults)) {
|
||||
// User provided routing_tier_defaults — honor them
|
||||
const v = effortCfg.routing_tier_defaults[agentTier];
|
||||
if (typeof v === 'string' && EFFORT_SET.has(v)) return v;
|
||||
} else if (!effortCfg) {
|
||||
// No effort config at all — use manifest tier defaults
|
||||
const manifestDefaults = CANONICAL_CONFIG_DEFAULTS.effort?.routing_tier_defaults;
|
||||
if (manifestDefaults && typeof manifestDefaults === 'object') {
|
||||
const v = manifestDefaults[agentTier];
|
||||
if (typeof v === 'string' && EFFORT_SET.has(v)) return v;
|
||||
}
|
||||
}
|
||||
// else: effortCfg exists but no routing_tier_defaults — fall through to effort.default
|
||||
}
|
||||
|
||||
const entry = _resolveRuntimeTier(config, tier);
|
||||
return entry?.reasoning_effort || null;
|
||||
// Step 4: effort.default
|
||||
if (effortCfg) {
|
||||
const d = effortCfg.default;
|
||||
if (typeof d === 'string' && EFFORT_SET.has(d)) return d;
|
||||
}
|
||||
|
||||
// Step 5: hardcoded default
|
||||
return 'high';
|
||||
}
|
||||
|
||||
/**
|
||||
* #443 — Resolve fast_mode boolean for (cwd, agentType).
|
||||
*
|
||||
* Accepts ONLY real booleans at each level. Strings like "true" are NOT accepted
|
||||
* and fall through.
|
||||
*
|
||||
* Precedence:
|
||||
* 1. opts.override (typeof boolean)
|
||||
* 2. config.fast_mode.agent_overrides[agentType] (boolean)
|
||||
* 3. config.fast_mode.routing_tier_defaults[ AGENT_DEFAULT_TIERS[agentType] ] (agent known + boolean)
|
||||
* 4. config.fast_mode.enabled (boolean)
|
||||
* 5. false
|
||||
*
|
||||
* @param {string} cwd
|
||||
* @param {string} agentType
|
||||
* @param {{ override?: boolean }} [opts]
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function resolveFastModeInternal(cwd, agentType, opts) {
|
||||
// Step 1: invocation override
|
||||
if (opts && typeof opts.override === 'boolean') {
|
||||
return opts.override;
|
||||
}
|
||||
|
||||
const config = loadConfig(cwd);
|
||||
const fmCfg = (config.fast_mode && typeof config.fast_mode === 'object' && !Array.isArray(config.fast_mode))
|
||||
? config.fast_mode
|
||||
: null;
|
||||
|
||||
// Step 2: agent_overrides
|
||||
if (fmCfg) {
|
||||
const ao = fmCfg.agent_overrides;
|
||||
if (ao && typeof ao === 'object' && !Array.isArray(ao)) {
|
||||
const v = ao[agentType];
|
||||
if (typeof v === 'boolean') return v;
|
||||
}
|
||||
}
|
||||
|
||||
// Step 3: routing_tier_defaults by agent's default tier.
|
||||
// Manifest tier defaults are only used when there is no fast_mode config block at all
|
||||
// (fmCfg === null). When the user explicitly set a fast_mode block (even with just
|
||||
// `enabled`), manifest routing_tier_defaults do not fire — we fall through to enabled (step 4).
|
||||
// This ensures `fast_mode: { enabled: true }` works intuitively without the user having
|
||||
// to also spell out all three tier defaults.
|
||||
const agentTier = AGENT_DEFAULT_TIERS[agentType];
|
||||
if (agentTier) {
|
||||
if (fmCfg && fmCfg.routing_tier_defaults &&
|
||||
typeof fmCfg.routing_tier_defaults === 'object' &&
|
||||
!Array.isArray(fmCfg.routing_tier_defaults)) {
|
||||
// User provided routing_tier_defaults — honor them
|
||||
const v = fmCfg.routing_tier_defaults[agentTier];
|
||||
if (typeof v === 'boolean') return v;
|
||||
} else if (!fmCfg) {
|
||||
// No fast_mode config at all — use manifest defaults for tier
|
||||
const manifestDefaults = CANONICAL_CONFIG_DEFAULTS.fast_mode?.routing_tier_defaults;
|
||||
if (manifestDefaults && typeof manifestDefaults === 'object') {
|
||||
const v = manifestDefaults[agentTier];
|
||||
if (typeof v === 'boolean') return v;
|
||||
}
|
||||
}
|
||||
// else: fmCfg exists but no routing_tier_defaults — fall through to enabled
|
||||
}
|
||||
|
||||
// Step 4: fast_mode.enabled
|
||||
if (fmCfg && typeof fmCfg.enabled === 'boolean') {
|
||||
return fmCfg.enabled;
|
||||
}
|
||||
|
||||
// Step 5: hardcoded default
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* #443 — Resolve effort for a dynamic-routing attempt (with escalation).
|
||||
*
|
||||
* MUST NOT modify resolveModelForTier behavior.
|
||||
* base = resolveEffortInternal(cwd, agentType).
|
||||
* If config.dynamic_routing missing/enabled!==true OR escalate_on_failure===false
|
||||
* -> return base (attempt ignored).
|
||||
* Else: effectiveAttempt = min(max(0, attempt), max_escalations).
|
||||
* Walk nextEffort effectiveAttempt times from base, clamp at 'max'.
|
||||
*
|
||||
* @param {string} cwd
|
||||
* @param {string} agentType
|
||||
* @param {number} [attempt=0]
|
||||
* @returns {string}
|
||||
*/
|
||||
function resolveEffortForTier(cwd, agentType, attempt) {
|
||||
const base = resolveEffortInternal(cwd, agentType);
|
||||
|
||||
const config = loadConfig(cwd);
|
||||
const dr = config.dynamic_routing;
|
||||
if (!dr || typeof dr !== 'object' || dr.enabled !== true) {
|
||||
return base;
|
||||
}
|
||||
if (dr.escalate_on_failure === false) {
|
||||
return base;
|
||||
}
|
||||
|
||||
const maxEscalations = Number.isInteger(dr.max_escalations) && dr.max_escalations >= 0
|
||||
? dr.max_escalations
|
||||
: 1;
|
||||
|
||||
const attemptN = Number.isInteger(attempt) && attempt > 0 ? attempt : 0;
|
||||
const effectiveAttempt = Math.min(attemptN, maxEscalations);
|
||||
|
||||
let current = base;
|
||||
for (let i = 0; i < effectiveAttempt; i++) {
|
||||
const next = nextEffort(current);
|
||||
if (!next || next === current) break; // already at max
|
||||
current = next;
|
||||
}
|
||||
return current;
|
||||
}
|
||||
|
||||
// ─── Summary body helpers ─────────────────────────────────────────────────
|
||||
@@ -1913,9 +2062,15 @@ module.exports = {
|
||||
getRoadmapPhaseInternal,
|
||||
resolveModelInternal,
|
||||
resolveModelForTier,
|
||||
resolveReasoningEffortInternal,
|
||||
resolveEffortInternal,
|
||||
resolveFastModeInternal,
|
||||
resolveEffortForTier,
|
||||
VALID_EFFORTS,
|
||||
EFFORT_SET,
|
||||
nextEffort,
|
||||
RUNTIME_PROFILE_MAP,
|
||||
RUNTIMES_WITH_REASONING_EFFORT,
|
||||
RUNTIMES_WITH_FAST_MODE,
|
||||
KNOWN_RUNTIMES,
|
||||
RUNTIME_OVERRIDE_TIERS,
|
||||
resolveTierEntry,
|
||||
|
||||
@@ -118,6 +118,79 @@ function getAgentToModelMapForProfile(normalizedProfile) {
|
||||
return out;
|
||||
}
|
||||
|
||||
// ─── Effort rendering ────────────────────────────────────────────────────────
|
||||
//
|
||||
// Universal effort ladder: minimal < low < medium < high < xhigh < max
|
||||
//
|
||||
// Each runtime supports a subset. The unique tails must be clamped when emitting
|
||||
// to a runtime that does not support them:
|
||||
// - 'max' is Anthropic-only: Codex does not support it -> clamp to 'xhigh'
|
||||
// - 'minimal' is Codex-only: Claude does not support it -> clamp to 'low'
|
||||
//
|
||||
// Rendering maps the universal effort string to the runtime's native parameter.
|
||||
|
||||
const EFFORT_RENDERING = {
|
||||
// Claude Code subagent effort: output_config.effort frontmatter key /
|
||||
// CLAUDE_CODE_EFFORT_LEVEL env. Supports: low, medium, high, xhigh, max.
|
||||
// Does NOT support 'minimal' (Codex-only) -> clamp to 'low'.
|
||||
claude: {
|
||||
param: 'output_config.effort',
|
||||
channel: 'frontmatter',
|
||||
supported: new Set(['low', 'medium', 'high', 'xhigh', 'max']),
|
||||
clamp(level) {
|
||||
if (level === 'minimal') return 'low';
|
||||
return level;
|
||||
},
|
||||
},
|
||||
// Codex Responses API reasoning.effort. Supports: minimal, low, medium, high, xhigh.
|
||||
// Does NOT support 'max' (Anthropic-only) -> clamp to 'xhigh'.
|
||||
codex: {
|
||||
param: 'model_reasoning_effort',
|
||||
channel: 'api',
|
||||
supported: new Set(['minimal', 'low', 'medium', 'high', 'xhigh']),
|
||||
clamp(level) {
|
||||
if (level === 'max') return 'xhigh';
|
||||
return level;
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Render a universal effort string for a specific runtime.
|
||||
*
|
||||
* Returns { value (clamped), param, channel } where:
|
||||
* - value: the clamped effort string safe to pass to the runtime
|
||||
* - param: the native parameter name (e.g. 'output_config.effort')
|
||||
* - channel: how the value is propagated ('frontmatter', 'api', null)
|
||||
*
|
||||
* Unknown runtimes return { value: universalEffort, param: null, channel: null }
|
||||
* so callers can always read .value safely.
|
||||
*/
|
||||
function renderEffortForRuntime(runtime, universalEffort) {
|
||||
const spec = EFFORT_RENDERING[runtime];
|
||||
if (!spec) {
|
||||
return { value: universalEffort, param: null, channel: null };
|
||||
}
|
||||
return {
|
||||
value: spec.clamp(universalEffort),
|
||||
param: spec.param,
|
||||
channel: spec.channel,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Fast mode propagation ───────────────────────────────────────────────────
|
||||
//
|
||||
// RUNTIMES_WITH_FAST_MODE is the set of runtimes where fast_mode=true can be
|
||||
// propagated to a SPAWNED SUBAGENT via a native mechanism.
|
||||
//
|
||||
// Claude Code has NO per-subagent fast-mode mechanism — /fast is a session-level
|
||||
// toggle only. Emitting a `fast_mode: true` frontmatter key on a Claude subagent
|
||||
// would be a SILENT NO-OP, which is why 'claude' is deliberately excluded here.
|
||||
//
|
||||
// Only API-direct runtimes ('api') accept a speed:"fast" field in the request.
|
||||
// Codex and other runtimes do not expose per-call fast_mode either.
|
||||
const RUNTIMES_WITH_FAST_MODE = new Set(['api']);
|
||||
|
||||
module.exports = {
|
||||
catalog,
|
||||
MODEL_PROFILES,
|
||||
@@ -133,4 +206,7 @@ module.exports = {
|
||||
nextTier,
|
||||
formatAgentToModelMapAsTable,
|
||||
getAgentToModelMapForProfile,
|
||||
EFFORT_RENDERING,
|
||||
renderEffortForRuntime,
|
||||
RUNTIMES_WITH_FAST_MODE,
|
||||
};
|
||||
|
||||
@@ -10,6 +10,9 @@ const {
|
||||
nextTier,
|
||||
formatAgentToModelMapAsTable,
|
||||
getAgentToModelMapForProfile,
|
||||
EFFORT_RENDERING,
|
||||
renderEffortForRuntime,
|
||||
RUNTIMES_WITH_FAST_MODE,
|
||||
} = require('./model-catalog.cjs');
|
||||
|
||||
module.exports = {
|
||||
@@ -22,4 +25,7 @@ module.exports = {
|
||||
nextTier,
|
||||
formatAgentToModelMapAsTable,
|
||||
getAgentToModelMapForProfile,
|
||||
EFFORT_RENDERING,
|
||||
renderEffortForRuntime,
|
||||
RUNTIMES_WITH_FAST_MODE,
|
||||
};
|
||||
|
||||
@@ -71,5 +71,23 @@
|
||||
"graphify": {
|
||||
"auto_update": false
|
||||
},
|
||||
"agent_skills": {}
|
||||
"agent_skills": {},
|
||||
"effort": {
|
||||
"default": "high",
|
||||
"routing_tier_defaults": {
|
||||
"light": "low",
|
||||
"standard": "high",
|
||||
"heavy": "xhigh"
|
||||
},
|
||||
"agent_overrides": {}
|
||||
},
|
||||
"fast_mode": {
|
||||
"enabled": false,
|
||||
"routing_tier_defaults": {
|
||||
"light": true,
|
||||
"standard": false,
|
||||
"heavy": false
|
||||
},
|
||||
"agent_overrides": {}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"_comment": "Canonical schema manifest for valid config key paths. validKeys is the union of CJS config-schema.cjs and SDK query/config-schema.ts (they are enforced set-equal by tests/config-schema-sdk-parity.test.cjs). runtimeStateKeys mirrors RUNTIME_STATE_KEYS. dynamicKeyPatterns mirrors DYNAMIC_KEY_PATTERNS from SDK (which includes the canonical 'source' regex strings). The .test function is reconstructed at runtime from new RegExp(source).",
|
||||
"_comment": "Canonical schema manifest for valid config key paths. This manifest is the single CJS source of truth for valid config keys; dynamicKeyPatterns source strings are recompiled to RegExp at runtime by config-schema.cjs. runtimeStateKeys mirrors RUNTIME_STATE_KEYS.",
|
||||
"validKeys": [
|
||||
"mode",
|
||||
"granularity",
|
||||
@@ -96,7 +96,9 @@
|
||||
"claude_md_path",
|
||||
"claude_md_assembly.mode",
|
||||
"runtime",
|
||||
"resolve_model_ids"
|
||||
"resolve_model_ids",
|
||||
"effort.default",
|
||||
"fast_mode.enabled"
|
||||
],
|
||||
"runtimeStateKeys": [
|
||||
"workflow._auto_chain_active"
|
||||
@@ -142,6 +144,26 @@
|
||||
"source": "^model_overrides\\.[a-zA-Z0-9_-]+$",
|
||||
"description": "model_overrides.<agent-id>"
|
||||
},
|
||||
{
|
||||
"topLevel": "effort",
|
||||
"source": "^effort\\.routing_tier_defaults\\.(light|standard|heavy)$",
|
||||
"description": "effort.routing_tier_defaults.<light|standard|heavy>"
|
||||
},
|
||||
{
|
||||
"topLevel": "effort",
|
||||
"source": "^effort\\.agent_overrides\\.[a-zA-Z0-9_-]+$",
|
||||
"description": "effort.agent_overrides.<agent-id>"
|
||||
},
|
||||
{
|
||||
"topLevel": "fast_mode",
|
||||
"source": "^fast_mode\\.routing_tier_defaults\\.(light|standard|heavy)$",
|
||||
"description": "fast_mode.routing_tier_defaults.<light|standard|heavy>"
|
||||
},
|
||||
{
|
||||
"topLevel": "fast_mode",
|
||||
"source": "^fast_mode\\.agent_overrides\\.[a-zA-Z0-9_-]+$",
|
||||
"description": "fast_mode.agent_overrides.<agent-id>"
|
||||
},
|
||||
{
|
||||
"topLevel": "review",
|
||||
"source": "^review\\.max_prompt_tokens_per_reviewer\\.[a-zA-Z0-9_-]+$",
|
||||
|
||||
@@ -584,6 +584,16 @@ Display:
|
||||
| model_profile_overrides.<runtime>.opus | {model/built-in/null} |
|
||||
| model_profile_overrides.<runtime>.sonnet | {model/built-in/null} |
|
||||
| model_profile_overrides.<runtime>.haiku | {model/built-in/null} |
|
||||
| effort.default | {low/medium/high/xhigh/max} |
|
||||
| effort.routing_tier_defaults.light | {low/medium/high/xhigh/max} |
|
||||
| effort.routing_tier_defaults.standard | {low/medium/high/xhigh/max} |
|
||||
| effort.routing_tier_defaults.heavy | {low/medium/high/xhigh/max} |
|
||||
| effort.agent_overrides.<agent-id> | {low/medium/high/xhigh/max} |
|
||||
| fast_mode.enabled | {true/false} |
|
||||
| fast_mode.routing_tier_defaults.light | {true/false} |
|
||||
| fast_mode.routing_tier_defaults.standard | {true/false} |
|
||||
| fast_mode.routing_tier_defaults.heavy | {true/false} |
|
||||
| fast_mode.agent_overrides.<agent-id> | {true/false} |
|
||||
|
||||
These settings apply to future /gsd:plan-phase, /gsd:execute-phase, /gsd:discuss-phase,
|
||||
and /gsd:ship runs.
|
||||
|
||||
@@ -1135,7 +1135,10 @@ describe('resolve-model command', () => {
|
||||
assert.ok(output.model, 'should resolve a model');
|
||||
});
|
||||
|
||||
test('includes reasoning_effort when selected runtime supports it', () => {
|
||||
// #443: resolve-model now emits unified `effort` instead of `reasoning_effort`.
|
||||
// reasoning_effort was flavor-text (resolved but consumed by nobody); effort is
|
||||
// the wired, config-driven universal effort string for all runtimes.
|
||||
test('emits unified effort (not reasoning_effort) when runtime supports tiered effort', () => {
|
||||
fs.writeFileSync(path.join(tmpDir, '.planning', 'config.json'), JSON.stringify({
|
||||
model_profile: 'balanced',
|
||||
runtime: 'codex',
|
||||
@@ -1147,10 +1150,14 @@ describe('resolve-model command', () => {
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.model, 'gpt-5.4');
|
||||
assert.strictEqual(output.profile, 'balanced');
|
||||
assert.strictEqual(output.reasoning_effort, 'xhigh');
|
||||
// #443: effort is now the unified field (xhigh for gsd-planner heavy tier default)
|
||||
assert.strictEqual(output.effort, 'xhigh');
|
||||
// reasoning_effort must be absent — replaced by unified effort
|
||||
assert.ok(!Object.prototype.hasOwnProperty.call(output, 'reasoning_effort'),
|
||||
'reasoning_effort must not appear in resolve-model output (replaced by effort)');
|
||||
});
|
||||
|
||||
test('does not include reasoning_effort for unsupported runtime overrides', () => {
|
||||
test('does not include reasoning_effort for unsupported runtime overrides (effort present instead)', () => {
|
||||
fs.writeFileSync(path.join(tmpDir, '.planning', 'config.json'), JSON.stringify({
|
||||
model_profile: 'balanced',
|
||||
runtime: 'opencode',
|
||||
@@ -1167,10 +1174,13 @@ describe('resolve-model command', () => {
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.model, 'openrouter/openai/gpt-5.5');
|
||||
assert.strictEqual(output.profile, 'balanced');
|
||||
assert.ok(!Object.prototype.hasOwnProperty.call(output, 'reasoning_effort'));
|
||||
// #443: effort always present; reasoning_effort never present
|
||||
assert.ok(Object.prototype.hasOwnProperty.call(output, 'effort'), 'effort must be present');
|
||||
assert.ok(!Object.prototype.hasOwnProperty.call(output, 'reasoning_effort'),
|
||||
'reasoning_effort must not appear (replaced by unified effort)');
|
||||
});
|
||||
|
||||
test('does not include reasoning_effort for per-agent model_overrides', () => {
|
||||
test('does not include reasoning_effort for per-agent model_overrides (effort present instead)', () => {
|
||||
fs.writeFileSync(path.join(tmpDir, '.planning', 'config.json'), JSON.stringify({
|
||||
model_profile: 'balanced',
|
||||
runtime: 'codex',
|
||||
@@ -1183,7 +1193,10 @@ describe('resolve-model command', () => {
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.model, 'gpt-5.5');
|
||||
assert.strictEqual(output.profile, 'balanced');
|
||||
assert.ok(!Object.prototype.hasOwnProperty.call(output, 'reasoning_effort'));
|
||||
// #443: effort always present; reasoning_effort never present
|
||||
assert.ok(Object.prototype.hasOwnProperty.call(output, 'effort'), 'effort must be present');
|
||||
assert.ok(!Object.prototype.hasOwnProperty.call(output, 'reasoning_effort'),
|
||||
'reasoning_effort must not appear (replaced by unified effort)');
|
||||
});
|
||||
|
||||
test('fails when no agent-type provided', () => {
|
||||
|
||||
@@ -253,129 +253,100 @@ describe('#3023 resolver: models.<phase_type> overrides profile-based tier', ()
|
||||
});
|
||||
});
|
||||
|
||||
// ─── #3030 CR Major outside-diff: reasoning_effort honors phase-type ───────
|
||||
// ─── #443 Unified effort: resolveEffortInternal + renderEffortForRuntime ────
|
||||
|
||||
const { resolveReasoningEffortInternal } = require('../get-shit-done/bin/lib/core.cjs');
|
||||
const { resolveEffortInternal } = require('../get-shit-done/bin/lib/core.cjs');
|
||||
const { renderEffortForRuntime } = require('../get-shit-done/bin/lib/model-catalog.cjs');
|
||||
|
||||
describe('#3023 + #3030 CR: resolveReasoningEffortInternal honors phase-type tier (Codex)', () => {
|
||||
describe('#3023 + #443: unified effort resolver (resolveEffortInternal) for Codex', () => {
|
||||
let projectDir;
|
||||
beforeEach(() => { projectDir = makeTmp('effort'); });
|
||||
afterEach(() => { rmr(projectDir); });
|
||||
|
||||
test('exported from core.cjs', () => {
|
||||
assert.equal(typeof resolveReasoningEffortInternal, 'function');
|
||||
test('resolveEffortInternal exported from core.cjs', () => {
|
||||
assert.equal(typeof resolveEffortInternal, 'function');
|
||||
});
|
||||
|
||||
test('phase-type override flips both model AND reasoning_effort to the same tier (Codex)', () => {
|
||||
// The CR Major bug: previously the model was resolved from the
|
||||
// phase-type tier (opus → gpt-5.4) but reasoning_effort still came
|
||||
// from the profile-derived sonnet tier (medium) — leading to a
|
||||
// mismatched (model, effort) pair on Codex spawn.
|
||||
test('effort derives from AGENT_DEFAULT_TIERS (routing), not phase-type; gsd-executor is standard → high', () => {
|
||||
// Under unification, effort is config-driven via routing_tier_defaults.
|
||||
// gsd-executor has routing tier 'standard' → default effort 'high', regardless
|
||||
// of models.execution phase-type or model_profile setting.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'balanced',
|
||||
models: { execution: 'opus' },
|
||||
});
|
||||
// gsd-executor's profile tier under balanced is sonnet, so without
|
||||
// the phase-type lookup mirror, model would resolve to opus (xhigh)
|
||||
// but effort to medium. Both must derive from the same tier source.
|
||||
const effort = resolveReasoningEffortInternal(projectDir, 'gsd-executor');
|
||||
// The exact effort value depends on the runtime tier map's opus row;
|
||||
// the test guards the relationship: it must NOT be the sonnet/medium
|
||||
// value when the phase-type forced opus.
|
||||
const sonnetEffort = (() => {
|
||||
// Read the sonnet effort by setting a config that uses the sonnet tier
|
||||
// and reading what comes back, so the assertion is semantic (effort
|
||||
// matches phase-type tier) rather than a hard-coded string.
|
||||
const sonnetDir = makeTmp('effort-sonnet');
|
||||
try {
|
||||
writeConfig(sonnetDir, {
|
||||
runtime: 'codex', model_profile: 'balanced',
|
||||
});
|
||||
return resolveReasoningEffortInternal(sonnetDir, 'gsd-executor');
|
||||
} finally {
|
||||
rmr(sonnetDir);
|
||||
}
|
||||
})();
|
||||
const opusEffort = (() => {
|
||||
const opusDir = makeTmp('effort-opus');
|
||||
try {
|
||||
writeConfig(opusDir, {
|
||||
runtime: 'codex', model_profile: 'quality', // quality → executor=opus
|
||||
});
|
||||
return resolveReasoningEffortInternal(opusDir, 'gsd-executor');
|
||||
} finally {
|
||||
rmr(opusDir);
|
||||
}
|
||||
})();
|
||||
// The phase-type override (models.execution=opus) must produce the
|
||||
// SAME effort as a profile-only opus config.
|
||||
assert.equal(effort, opusEffort,
|
||||
`phase-type override must match opus-tier effort, got ${effort}, expected ${opusEffort}`);
|
||||
// And it must NOT match sonnet effort (proving the override fired).
|
||||
if (opusEffort !== null && sonnetEffort !== null && opusEffort !== sonnetEffort) {
|
||||
assert.notEqual(effort, sonnetEffort,
|
||||
`phase-type override should not silently use sonnet effort: ${effort}`);
|
||||
}
|
||||
const eff = resolveEffortInternal(projectDir, 'gsd-executor');
|
||||
// standard tier → 'high' (not 'xhigh' from opus, not 'medium' from old catalog)
|
||||
assert.equal(eff, 'high');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.equal(rendered.param, 'model_reasoning_effort');
|
||||
assert.equal(rendered.value, 'high');
|
||||
});
|
||||
|
||||
test('inherit phase-type tier returns null effort (no runtime entry maps to inherit)', () => {
|
||||
test('effort resolves universally even when models.execution=inherit', () => {
|
||||
// Under unification, models.execution='inherit' does not affect effort resolution.
|
||||
// Effort always resolves from routing_tier_defaults: gsd-executor (standard) → 'high'.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'balanced',
|
||||
models: { execution: 'inherit' },
|
||||
});
|
||||
// 'inherit' has no runtime-tier entry, so the resolver returns null.
|
||||
const effort = resolveReasoningEffortInternal(projectDir, 'gsd-executor');
|
||||
assert.equal(effort, null);
|
||||
const eff = resolveEffortInternal(projectDir, 'gsd-executor');
|
||||
assert.equal(eff, 'high');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.equal(rendered.param, 'model_reasoning_effort');
|
||||
assert.equal(rendered.value, 'high');
|
||||
});
|
||||
|
||||
test('per-agent override still bypasses phase-type for reasoning_effort', () => {
|
||||
test('per-agent model_overrides does not affect effort (effort is routing-tier-based)', () => {
|
||||
// Under unification, effort does not check model_overrides.
|
||||
// gsd-executor (standard tier) → 'high' regardless.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'balanced',
|
||||
models: { execution: 'opus' },
|
||||
model_overrides: { 'gsd-executor': 'openai/gpt-5' },
|
||||
});
|
||||
// model_overrides[agent] short-circuits resolveReasoningEffortInternal
|
||||
// (the user supplied a fully-qualified ID; effort must be set per-agent).
|
||||
assert.equal(resolveReasoningEffortInternal(projectDir, 'gsd-executor'), null);
|
||||
const eff = resolveEffortInternal(projectDir, 'gsd-executor');
|
||||
assert.equal(eff, 'high');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.equal(rendered.param, 'model_reasoning_effort');
|
||||
assert.equal(rendered.value, 'high');
|
||||
});
|
||||
|
||||
test('claude runtime ignores models.* for reasoning_effort (returns null)', () => {
|
||||
test('Claude runtime: effort is first-class (emits output_config.effort, not null)', () => {
|
||||
// Under unification, Claude effort is first-class via output_config.effort.
|
||||
// No `runtime` set → defaults to claude (no runtime key → undefined runtime).
|
||||
writeConfig(projectDir, {
|
||||
// No `runtime` set → defaults to claude, which has no reasoning_effort.
|
||||
model_profile: 'balanced',
|
||||
models: { execution: 'opus' },
|
||||
});
|
||||
assert.equal(resolveReasoningEffortInternal(projectDir, 'gsd-executor'), null);
|
||||
const eff = resolveEffortInternal(projectDir, 'gsd-executor');
|
||||
// effort resolves universally; claude render gives output_config.effort
|
||||
const rendered = renderEffortForRuntime(undefined, eff);
|
||||
// undefined runtime yields param=null (no runtime key set)
|
||||
assert.equal(rendered.param, null);
|
||||
// But if explicitly set to 'claude':
|
||||
const renderedClaude = renderEffortForRuntime('claude', eff);
|
||||
assert.equal(renderedClaude.param, 'output_config.effort');
|
||||
assert.equal(renderedClaude.value, 'high');
|
||||
});
|
||||
|
||||
test('phase-type override wins over profile=inherit for effort (CR Major #3030)', () => {
|
||||
// Pre-fix bug: profile=inherit short-circuited to null even when
|
||||
// models.execution=opus would have supplied a valid tier.
|
||||
test('profile=inherit does not affect effort; effort resolves from routing tier', () => {
|
||||
// Under unification, effort is completely independent of model_profile.
|
||||
// gsd-executor (standard routing tier) → 'high' even with model_profile='inherit'.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'inherit',
|
||||
models: { execution: 'opus' },
|
||||
});
|
||||
// Compute the expected effort by reading what gsd-executor would
|
||||
// get under a profile-only opus config — the phase-type override
|
||||
// must produce the SAME result.
|
||||
const expected = (() => {
|
||||
const dir = makeTmp('effort-opus2');
|
||||
try {
|
||||
writeConfig(dir, { runtime: 'codex', model_profile: 'quality' });
|
||||
return resolveReasoningEffortInternal(dir, 'gsd-executor');
|
||||
} finally {
|
||||
rmr(dir);
|
||||
}
|
||||
})();
|
||||
const actual = resolveReasoningEffortInternal(projectDir, 'gsd-executor');
|
||||
assert.equal(actual, expected,
|
||||
`phase-type override over profile=inherit must produce the opus-tier effort; got ${actual}, expected ${expected}`);
|
||||
assert.notEqual(actual, null,
|
||||
'phase-type opus must NOT return null effort just because profile=inherit');
|
||||
const eff = resolveEffortInternal(projectDir, 'gsd-executor');
|
||||
assert.equal(eff, 'high',
|
||||
'profile=inherit must not affect effort; standard routing tier → high');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.equal(rendered.param, 'model_reasoning_effort');
|
||||
assert.equal(rendered.value, 'high');
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
83
tests/feat-443-effort-defaults-drift.test.cjs
Normal file
83
tests/feat-443-effort-defaults-drift.test.cjs
Normal file
@@ -0,0 +1,83 @@
|
||||
'use strict';
|
||||
/**
|
||||
* feat-443-effort-defaults-drift.test.cjs
|
||||
*
|
||||
* Drift-guard: asserts that install.js's resolved baseline effort defaults
|
||||
* equal config-defaults.manifest.json's effort block. Any future divergence
|
||||
* (someone edits the manifest without updating install.js or vice-versa) fails
|
||||
* CI immediately rather than silently injecting stale effort values.
|
||||
*
|
||||
* Real assertions on runtime values — no source-grep.
|
||||
*/
|
||||
|
||||
// MUST be set before require('bin/install.js') so the main install block
|
||||
// (guarded by !GSD_TEST_MODE) does not execute and perform a real global
|
||||
// install into $HOME/.claude/ — which would leak gsd-tools.cjs into the
|
||||
// ambient HOME and break runtime-launcher-parity test (D) in the same
|
||||
// node --test run (all unit tests share the same HOME on CI).
|
||||
process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
const assert = require('assert');
|
||||
const path = require('path');
|
||||
|
||||
const { test } = require('node:test');
|
||||
|
||||
// Load the manifest directly (JSON, not a .cjs source file — allowed by lint rule)
|
||||
const manifestPath = path.join(
|
||||
__dirname,
|
||||
'..',
|
||||
'get-shit-done',
|
||||
'bin',
|
||||
'shared',
|
||||
'config-defaults.manifest.json'
|
||||
);
|
||||
const manifest = require(manifestPath);
|
||||
|
||||
// Load install.js exported values (executes the module, not text inspection)
|
||||
const installPath = path.join(__dirname, '..', 'bin', 'install.js');
|
||||
const {
|
||||
_GSD_EFFORT_MANIFEST_TIER_DEFAULTS,
|
||||
_GSD_EFFORT_MANIFEST_DEFAULT,
|
||||
} = require(installPath);
|
||||
|
||||
test('install.js _GSD_EFFORT_MANIFEST_TIER_DEFAULTS.light matches manifest effort.routing_tier_defaults.light', () => {
|
||||
assert.strictEqual(
|
||||
_GSD_EFFORT_MANIFEST_TIER_DEFAULTS.light,
|
||||
manifest.effort.routing_tier_defaults.light,
|
||||
`install.js tier default for "light" (${_GSD_EFFORT_MANIFEST_TIER_DEFAULTS.light}) differs from manifest (${manifest.effort.routing_tier_defaults.light})`
|
||||
);
|
||||
});
|
||||
|
||||
test('install.js _GSD_EFFORT_MANIFEST_TIER_DEFAULTS.standard matches manifest effort.routing_tier_defaults.standard', () => {
|
||||
assert.strictEqual(
|
||||
_GSD_EFFORT_MANIFEST_TIER_DEFAULTS.standard,
|
||||
manifest.effort.routing_tier_defaults.standard,
|
||||
`install.js tier default for "standard" (${_GSD_EFFORT_MANIFEST_TIER_DEFAULTS.standard}) differs from manifest (${manifest.effort.routing_tier_defaults.standard})`
|
||||
);
|
||||
});
|
||||
|
||||
test('install.js _GSD_EFFORT_MANIFEST_TIER_DEFAULTS.heavy matches manifest effort.routing_tier_defaults.heavy', () => {
|
||||
assert.strictEqual(
|
||||
_GSD_EFFORT_MANIFEST_TIER_DEFAULTS.heavy,
|
||||
manifest.effort.routing_tier_defaults.heavy,
|
||||
`install.js tier default for "heavy" (${_GSD_EFFORT_MANIFEST_TIER_DEFAULTS.heavy}) differs from manifest (${manifest.effort.routing_tier_defaults.heavy})`
|
||||
);
|
||||
});
|
||||
|
||||
test('install.js _GSD_EFFORT_MANIFEST_DEFAULT matches manifest effort.default', () => {
|
||||
assert.strictEqual(
|
||||
_GSD_EFFORT_MANIFEST_DEFAULT,
|
||||
manifest.effort.default,
|
||||
`install.js effort default (${_GSD_EFFORT_MANIFEST_DEFAULT}) differs from manifest (${manifest.effort.default})`
|
||||
);
|
||||
});
|
||||
|
||||
test('install.js tier-defaults object has exactly the same keys as manifest effort.routing_tier_defaults', () => {
|
||||
const installKeys = Object.keys(_GSD_EFFORT_MANIFEST_TIER_DEFAULTS).sort();
|
||||
const manifestKeys = Object.keys(manifest.effort.routing_tier_defaults).sort();
|
||||
assert.deepStrictEqual(
|
||||
installKeys,
|
||||
manifestKeys,
|
||||
`Key mismatch — install.js: [${installKeys.join(', ')}], manifest: [${manifestKeys.join(', ')}]`
|
||||
);
|
||||
});
|
||||
718
tests/feat-443-effort-fast-mode.integration.test.cjs
Normal file
718
tests/feat-443-effort-fast-mode.integration.test.cjs
Normal file
@@ -0,0 +1,718 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Architecture-level QA for issue #443 — unified effort + fast_mode engine.
|
||||
*
|
||||
* Integration suite (*.integration.test.cjs): cross-module flows that exercise
|
||||
* real CLI invocations via runGsdTools, the full 33-agent registry, and the
|
||||
* config round-trip through config-set -> resolve-execution.
|
||||
*
|
||||
* INVARIANTS tested here (each is also documented in docs/TESTING-SUITES.md):
|
||||
*
|
||||
* (a) CROSS-PROVIDER VALIDITY — renderEffortForRuntime never emits a value
|
||||
* that the real provider API would 400 on. Ground-truth provider enums are
|
||||
* defined as local constants (not sourced from the implementation).
|
||||
*
|
||||
* (b) PARAM/CHANNEL CONTRACT — each runtime exposes a stable parameter name
|
||||
* and propagation channel.
|
||||
*
|
||||
* (c) RESOLVE-EXECUTION JSON CONTRACT — the CLI command emits a stable JSON
|
||||
* shape with all required keys and correct types.
|
||||
*
|
||||
* (d) TOTALITY across the real 33-agent registry — every agent produces a
|
||||
* valid effort value; none returns undefined/null.
|
||||
*
|
||||
* (e) FAST-MODE HONESTY INVARIANT — claude runtime always reports
|
||||
* fast_mode_supported=false (emitting fast_mode frontmatter is a silent
|
||||
* no-op for Claude Code subagents).
|
||||
*
|
||||
* (f) PRECEDENCE MATRIX — first-valid-wins for both effort and fast_mode
|
||||
* cascades, including invalid values correctly falling through.
|
||||
*
|
||||
* (g) DYNAMIC-ROUTING COMPOSITION — resolveEffortForTier escalates
|
||||
* independently of model tier logic; clamps at 'max'; respects
|
||||
* max_escalations; disabled when escalate_on_failure=false.
|
||||
*
|
||||
* (h) CONFIG-TOOLING ROUND-TRIP — config-set accepts all new effort/fast_mode
|
||||
* key paths (schema validation passes); values survive round-trip through
|
||||
* resolve-execution.
|
||||
*/
|
||||
|
||||
process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
const { describe, test, before, after, beforeEach, afterEach } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
|
||||
const { createTempProject, cleanup, runGsdTools } = require('./helpers.cjs');
|
||||
|
||||
const {
|
||||
resolveEffortInternal,
|
||||
resolveFastModeInternal,
|
||||
resolveEffortForTier,
|
||||
VALID_EFFORTS,
|
||||
} = require('../get-shit-done/bin/lib/core.cjs');
|
||||
|
||||
const {
|
||||
renderEffortForRuntime,
|
||||
RUNTIMES_WITH_FAST_MODE,
|
||||
catalog,
|
||||
} = require('../get-shit-done/bin/lib/model-catalog.cjs');
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// Ground-truth provider enums (defined HERE, not sourced from the implementation).
|
||||
// These are the exact values the real APIs accept — using a value outside these
|
||||
// sets would result in a 400 response from the provider.
|
||||
//
|
||||
// Sources:
|
||||
// Anthropic: output_config.effort — https://docs.anthropic.com (Claude API)
|
||||
// OpenAI: model_reasoning_effort — https://platform.openai.com/docs (Codex)
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
const PROVIDER_EFFORT_ENUMS = {
|
||||
claude: new Set(['low', 'medium', 'high', 'xhigh', 'max']),
|
||||
codex: new Set(['minimal', 'low', 'medium', 'high', 'xhigh']),
|
||||
};
|
||||
|
||||
// Helper: write config.json into a temp project
|
||||
function writeConfig(dir, config) {
|
||||
const planningDir = path.join(dir, '.planning');
|
||||
fs.mkdirSync(planningDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(planningDir, 'config.json'), JSON.stringify(config, null, 2));
|
||||
}
|
||||
|
||||
// ─── (a) CROSS-PROVIDER VALIDITY INVARIANT ───────────────────────────────────
|
||||
|
||||
describe('#443 integration (a): cross-provider validity invariant', () => {
|
||||
// For every universal effort × every provider runtime, the rendered value
|
||||
// must be a member of that provider's real API enum.
|
||||
test('all VALID_EFFORTS render within provider enums for claude and codex', () => {
|
||||
for (const universalEffort of VALID_EFFORTS) {
|
||||
for (const [runtime, providerEnum] of Object.entries(PROVIDER_EFFORT_ENUMS)) {
|
||||
const rendered = renderEffortForRuntime(runtime, universalEffort);
|
||||
assert.ok(
|
||||
providerEnum.has(rendered.value),
|
||||
`render('${runtime}', '${universalEffort}').value = '${rendered.value}' is NOT in the ` +
|
||||
`${runtime} provider enum ${[...providerEnum].join('|')} — real API would 400`
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Documented clamps must hold exactly
|
||||
test("render('codex','max').value === 'xhigh' (max is Anthropic-only)", () => {
|
||||
assert.strictEqual(renderEffortForRuntime('codex', 'max').value, 'xhigh');
|
||||
});
|
||||
|
||||
test("render('claude','minimal').value === 'low' (minimal is Codex-only)", () => {
|
||||
assert.strictEqual(renderEffortForRuntime('claude', 'minimal').value, 'low');
|
||||
});
|
||||
|
||||
// Common levels must pass through unchanged on BOTH providers
|
||||
test('common levels (low/medium/high/xhigh) pass through unchanged on claude', () => {
|
||||
for (const level of ['low', 'medium', 'high', 'xhigh']) {
|
||||
assert.strictEqual(
|
||||
renderEffortForRuntime('claude', level).value,
|
||||
level,
|
||||
`claude: level '${level}' should pass through unchanged`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('common levels (low/medium/high/xhigh) pass through unchanged on codex', () => {
|
||||
for (const level of ['low', 'medium', 'high', 'xhigh']) {
|
||||
assert.strictEqual(
|
||||
renderEffortForRuntime('codex', level).value,
|
||||
level,
|
||||
`codex: level '${level}' should pass through unchanged`
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── (b) PARAM/CHANNEL CONTRACT ──────────────────────────────────────────────
|
||||
|
||||
describe('#443 integration (b): param/channel contract', () => {
|
||||
test("claude: param is always 'output_config.effort'", () => {
|
||||
for (const effort of VALID_EFFORTS) {
|
||||
const r = renderEffortForRuntime('claude', effort);
|
||||
assert.strictEqual(r.param, 'output_config.effort',
|
||||
`claude param must be 'output_config.effort' for effort '${effort}'`);
|
||||
}
|
||||
});
|
||||
|
||||
test("codex: param is always 'model_reasoning_effort'", () => {
|
||||
for (const effort of VALID_EFFORTS) {
|
||||
const r = renderEffortForRuntime('codex', effort);
|
||||
assert.strictEqual(r.param, 'model_reasoning_effort',
|
||||
`codex param must be 'model_reasoning_effort' for effort '${effort}'`);
|
||||
}
|
||||
});
|
||||
|
||||
test('claude channel is stable: frontmatter', () => {
|
||||
for (const effort of VALID_EFFORTS) {
|
||||
assert.strictEqual(renderEffortForRuntime('claude', effort).channel, 'frontmatter');
|
||||
}
|
||||
});
|
||||
|
||||
test('codex channel is stable: api', () => {
|
||||
for (const effort of VALID_EFFORTS) {
|
||||
assert.strictEqual(renderEffortForRuntime('codex', effort).channel, 'api');
|
||||
}
|
||||
});
|
||||
|
||||
test("unknown runtimes (gemini, qwen, 'mystery'): param===null, value passes through", () => {
|
||||
for (const runtime of ['gemini', 'qwen', 'mystery']) {
|
||||
for (const effort of VALID_EFFORTS) {
|
||||
const r = renderEffortForRuntime(runtime, effort);
|
||||
assert.strictEqual(r.param, null, `${runtime}: param must be null`);
|
||||
assert.strictEqual(r.channel, null, `${runtime}: channel must be null`);
|
||||
assert.strictEqual(r.value, effort, `${runtime}: value must pass through unchanged`);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── (c) RESOLVE-EXECUTION JSON CONTRACT ─────────────────────────────────────
|
||||
|
||||
describe('#443 integration (c): resolve-execution JSON contract', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
function assertFullContract(output, label) {
|
||||
assert.ok(typeof output.model === 'string' && output.model.length > 0,
|
||||
`${label}: model must be a non-empty string`);
|
||||
assert.ok(typeof output.profile === 'string' && output.profile.length > 0,
|
||||
`${label}: profile must be a non-empty string`);
|
||||
assert.ok(VALID_EFFORTS.includes(output.effort),
|
||||
`${label}: effort '${output.effort}' must be a member of VALID_EFFORTS`);
|
||||
assert.ok(typeof output.effort_rendered === 'string' && output.effort_rendered.length > 0,
|
||||
`${label}: effort_rendered must be a non-empty string`);
|
||||
assert.ok(output.effort_param === null || typeof output.effort_param === 'string',
|
||||
`${label}: effort_param must be string or null`);
|
||||
assert.ok(output.effort_propagation === null || typeof output.effort_propagation === 'string',
|
||||
`${label}: effort_propagation must be string or null`);
|
||||
assert.ok(typeof output.fast_mode === 'boolean',
|
||||
`${label}: fast_mode must be a boolean`);
|
||||
assert.ok(typeof output.fast_mode_supported === 'boolean',
|
||||
`${label}: fast_mode_supported must be a boolean`);
|
||||
}
|
||||
|
||||
test('gsd-planner (default claude runtime): full contract + known-agent shape', () => {
|
||||
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assertFullContract(output, 'gsd-planner/claude');
|
||||
assert.strictEqual(output.effort_param, 'output_config.effort');
|
||||
assert.strictEqual(output.effort_propagation, 'frontmatter');
|
||||
assert.strictEqual(output.fast_mode_supported, false);
|
||||
// known agent must NOT have unknown_agent:true
|
||||
assert.ok(!output.unknown_agent, 'known agent must not have unknown_agent:true');
|
||||
});
|
||||
|
||||
test('codex runtime: full contract + effort_param=model_reasoning_effort', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex' });
|
||||
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assertFullContract(output, 'gsd-planner/codex');
|
||||
assert.strictEqual(output.effort_param, 'model_reasoning_effort');
|
||||
assert.strictEqual(output.fast_mode_supported, false);
|
||||
});
|
||||
|
||||
test('gemini runtime: full contract + effort_param===null (no effort wire)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'gemini' });
|
||||
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assertFullContract(output, 'gsd-planner/gemini');
|
||||
assert.strictEqual(output.effort_param, null);
|
||||
assert.strictEqual(output.effort_propagation, null);
|
||||
assert.strictEqual(output.fast_mode_supported, false);
|
||||
});
|
||||
|
||||
test('unknown agent: full contract + unknown_agent===true', () => {
|
||||
const result = runGsdTools(['resolve-execution', 'unknown-agent-xyz'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assertFullContract(output, 'unknown-agent-xyz');
|
||||
assert.strictEqual(output.unknown_agent, true, 'unknown agent must have unknown_agent:true');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── (d) TOTALITY across the real 33-agent registry ──────────────────────────
|
||||
|
||||
describe('#443 integration (d): totality across real registry', () => {
|
||||
let tmpDir;
|
||||
before(() => { tmpDir = createTempProject(); });
|
||||
after(() => { cleanup(tmpDir); });
|
||||
|
||||
const registeredAgents = Object.keys(catalog.agents);
|
||||
// Confirm we're covering the full registry — snapshot the count so a
|
||||
// catalog shrink is caught by this assertion.
|
||||
test(`registry has at least 33 agents (currently ${registeredAgents.length})`, () => {
|
||||
assert.ok(registeredAgents.length >= 33,
|
||||
`Expected at least 33 agents in registry, got ${registeredAgents.length}`);
|
||||
});
|
||||
|
||||
test(`all ${registeredAgents.length} agents: resolveEffortInternal returns a VALID_EFFORTS member`, () => {
|
||||
const effortSet = new Set(VALID_EFFORTS);
|
||||
const bad = [];
|
||||
for (const agent of registeredAgents) {
|
||||
const effort = resolveEffortInternal(tmpDir, agent);
|
||||
if (effort === undefined || effort === null || !effortSet.has(effort)) {
|
||||
bad.push(`${agent}: got ${JSON.stringify(effort)}`);
|
||||
}
|
||||
}
|
||||
assert.strictEqual(bad.length, 0,
|
||||
`Agents with invalid effort:\n${bad.join('\n')}`);
|
||||
});
|
||||
|
||||
test(`all ${registeredAgents.length} agents: resolveFastModeInternal returns strict boolean`, () => {
|
||||
const bad = [];
|
||||
for (const agent of registeredAgents) {
|
||||
const fm = resolveFastModeInternal(tmpDir, agent);
|
||||
if (typeof fm !== 'boolean') {
|
||||
bad.push(`${agent}: got ${JSON.stringify(fm)} (${typeof fm})`);
|
||||
}
|
||||
}
|
||||
assert.strictEqual(bad.length, 0,
|
||||
`Agents with non-boolean fast_mode:\n${bad.join('\n')}`);
|
||||
});
|
||||
|
||||
test(`all ${registeredAgents.length} agents: renderEffortForRuntime('claude', effort) stays in claude enum`, () => {
|
||||
const claudeEnum = PROVIDER_EFFORT_ENUMS.claude;
|
||||
const bad = [];
|
||||
for (const agent of registeredAgents) {
|
||||
const effort = resolveEffortInternal(tmpDir, agent);
|
||||
const rendered = renderEffortForRuntime('claude', effort);
|
||||
if (!claudeEnum.has(rendered.value)) {
|
||||
bad.push(`${agent}: effort=${effort} rendered=${rendered.value} not in claude enum`);
|
||||
}
|
||||
}
|
||||
assert.strictEqual(bad.length, 0,
|
||||
`Agents producing invalid claude effort:\n${bad.join('\n')}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── (e) FAST-MODE HONESTY INVARIANT ─────────────────────────────────────────
|
||||
|
||||
describe('#443 integration (e): fast-mode honesty invariant', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
// Sample of agents across all tiers to prove the invariant is not agent-specific
|
||||
const testAgents = ['gsd-planner', 'gsd-executor', 'gsd-codebase-mapper', 'gsd-verifier'];
|
||||
|
||||
test('claude runtime: fast_mode_supported is ALWAYS false regardless of fast_mode config', () => {
|
||||
const configs = [
|
||||
{},
|
||||
{ fast_mode: { enabled: true } },
|
||||
{ fast_mode: { routing_tier_defaults: { heavy: true } } },
|
||||
{ fast_mode: { agent_overrides: { 'gsd-planner': true } } },
|
||||
];
|
||||
for (const config of configs) {
|
||||
writeConfig(tmpDir, config);
|
||||
for (const agent of testAgents) {
|
||||
const result = runGsdTools(['resolve-execution', agent], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed for ${agent}: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.fast_mode_supported, false,
|
||||
`claude/${agent}: fast_mode_supported must be false (Claude has no per-subagent fast-mode mechanism); config=${JSON.stringify(config)}`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("RUNTIMES_WITH_FAST_MODE.has('api') === true (api is the only fast-mode capable runtime)", () => {
|
||||
assert.ok(RUNTIMES_WITH_FAST_MODE.has('api'),
|
||||
"RUNTIMES_WITH_FAST_MODE must include 'api' — this is the only runtime with per-call fast_mode support");
|
||||
});
|
||||
|
||||
test("RUNTIMES_WITH_FAST_MODE.has('claude') === false (claude fast-mode is session-level only)", () => {
|
||||
assert.ok(!RUNTIMES_WITH_FAST_MODE.has('claude'),
|
||||
"RUNTIMES_WITH_FAST_MODE must NOT include 'claude' — emitting fast_mode frontmatter on a Claude subagent is a silent no-op");
|
||||
});
|
||||
|
||||
test("RUNTIMES_WITH_FAST_MODE.has('codex') === false", () => {
|
||||
assert.ok(!RUNTIMES_WITH_FAST_MODE.has('codex'),
|
||||
"codex does not support per-call fast_mode");
|
||||
});
|
||||
|
||||
test("RUNTIMES_WITH_FAST_MODE.has('gemini') === false", () => {
|
||||
assert.ok(!RUNTIMES_WITH_FAST_MODE.has('gemini'),
|
||||
"gemini does not support per-call fast_mode");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── (f) PRECEDENCE MATRIX ───────────────────────────────────────────────────
|
||||
|
||||
describe('#443 integration (f): precedence matrix (property/table-driven)', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
// Effort: first-valid-wins from highest precedence to lowest
|
||||
// 1. opts.override (invocation)
|
||||
// 2. effort.agent_overrides.<agent>
|
||||
// 3. effort.routing_tier_defaults.<tier>
|
||||
// 4. effort.default
|
||||
// 5. manifest tier default
|
||||
// 6. hardcoded 'high'
|
||||
const effortPrecedenceTable = [
|
||||
{
|
||||
label: 'layer 1 (invocation override) beats all',
|
||||
config: {
|
||||
effort: {
|
||||
agent_overrides: { 'gsd-planner': 'low' },
|
||||
routing_tier_defaults: { heavy: 'medium' },
|
||||
default: 'xhigh',
|
||||
},
|
||||
},
|
||||
opts: { override: 'minimal' },
|
||||
expected: 'minimal',
|
||||
},
|
||||
{
|
||||
label: 'layer 2 (agent_override) beats tier default and default',
|
||||
config: {
|
||||
effort: {
|
||||
agent_overrides: { 'gsd-planner': 'low' },
|
||||
routing_tier_defaults: { heavy: 'medium' },
|
||||
default: 'xhigh',
|
||||
},
|
||||
},
|
||||
opts: {},
|
||||
expected: 'low',
|
||||
},
|
||||
{
|
||||
label: 'layer 3 (routing_tier_defaults) beats effort.default',
|
||||
config: {
|
||||
effort: {
|
||||
routing_tier_defaults: { heavy: 'medium' },
|
||||
default: 'xhigh',
|
||||
},
|
||||
},
|
||||
opts: {},
|
||||
expected: 'medium',
|
||||
},
|
||||
{
|
||||
label: 'layer 4 (effort.default) when no tier default set',
|
||||
config: {
|
||||
effort: { default: 'low' },
|
||||
},
|
||||
opts: {},
|
||||
expected: 'low',
|
||||
},
|
||||
{
|
||||
label: 'invalid layer 1 (turbo) falls through to layer 2 (agent_override)',
|
||||
config: {
|
||||
effort: { agent_overrides: { 'gsd-planner': 'medium' } },
|
||||
},
|
||||
opts: { override: 'turbo' },
|
||||
expected: 'medium',
|
||||
},
|
||||
{
|
||||
label: 'invalid layer 2 (agent_override=123 numeric) falls through to tier default',
|
||||
config: {
|
||||
effort: {
|
||||
agent_overrides: { 'gsd-planner': 123 },
|
||||
routing_tier_defaults: { heavy: 'high' },
|
||||
},
|
||||
},
|
||||
opts: {},
|
||||
expected: 'high',
|
||||
},
|
||||
{
|
||||
label: 'invalid tier default (turbo) falls through to effort.default',
|
||||
config: {
|
||||
effort: {
|
||||
routing_tier_defaults: { heavy: 'turbo' },
|
||||
default: 'low',
|
||||
},
|
||||
},
|
||||
opts: {},
|
||||
expected: 'low',
|
||||
},
|
||||
];
|
||||
|
||||
for (const row of effortPrecedenceTable) {
|
||||
test(`effort precedence: ${row.label}`, () => {
|
||||
writeConfig(tmpDir, row.config);
|
||||
const result = resolveEffortInternal(tmpDir, 'gsd-planner', row.opts);
|
||||
assert.strictEqual(result, row.expected,
|
||||
`Expected '${row.expected}', got '${result}' — config: ${JSON.stringify(row.config)}`);
|
||||
});
|
||||
}
|
||||
|
||||
// fast_mode precedence:
|
||||
// 1. opts.override (strict boolean only)
|
||||
// 2. fast_mode.agent_overrides.<agent> (strict boolean only)
|
||||
// 3. fast_mode.routing_tier_defaults.<tier> (strict boolean only)
|
||||
// 4. fast_mode.enabled (strict boolean only)
|
||||
// 5. false
|
||||
const fastModePrecedenceTable = [
|
||||
{
|
||||
label: 'layer 1 (opts.override=false) beats enabled=true',
|
||||
config: { fast_mode: { enabled: true } },
|
||||
opts: { override: false },
|
||||
expected: false,
|
||||
},
|
||||
{
|
||||
label: 'layer 2 (agent_override=true) beats tier default',
|
||||
config: {
|
||||
fast_mode: {
|
||||
agent_overrides: { 'gsd-planner': true },
|
||||
routing_tier_defaults: { heavy: false },
|
||||
enabled: false,
|
||||
},
|
||||
},
|
||||
opts: {},
|
||||
expected: true,
|
||||
},
|
||||
{
|
||||
label: 'layer 3 (tier default=true) beats enabled=false',
|
||||
config: {
|
||||
fast_mode: {
|
||||
routing_tier_defaults: { heavy: true },
|
||||
enabled: false,
|
||||
},
|
||||
},
|
||||
opts: {},
|
||||
expected: true,
|
||||
},
|
||||
{
|
||||
label: 'layer 4 (enabled=true) when no tier/agent overrides',
|
||||
config: { fast_mode: { enabled: true } },
|
||||
opts: {},
|
||||
expected: true,
|
||||
},
|
||||
{
|
||||
label: 'layer 5 (default false) when all absent',
|
||||
config: {},
|
||||
opts: {},
|
||||
expected: false,
|
||||
},
|
||||
{
|
||||
label: 'string "true" in opts.override is NOT accepted (falls through)',
|
||||
config: { fast_mode: { enabled: true } },
|
||||
// override must be strict boolean; string falls through to next layer
|
||||
opts: { override: 'true' },
|
||||
// 'true' as string is not boolean -> falls through to tier default
|
||||
// gsd-planner is heavy; no tier default set; falls to enabled=true
|
||||
expected: true,
|
||||
},
|
||||
{
|
||||
label: 'string "true" in agent_overrides is NOT accepted',
|
||||
config: {
|
||||
fast_mode: {
|
||||
agent_overrides: { 'gsd-planner': 'true' },
|
||||
enabled: false,
|
||||
},
|
||||
},
|
||||
opts: {},
|
||||
// string 'true' is not boolean -> fall through to tier default -> enabled=false -> false
|
||||
expected: false,
|
||||
},
|
||||
];
|
||||
|
||||
for (const row of fastModePrecedenceTable) {
|
||||
test(`fast_mode precedence: ${row.label}`, () => {
|
||||
writeConfig(tmpDir, row.config);
|
||||
const result = resolveFastModeInternal(tmpDir, 'gsd-planner', row.opts);
|
||||
assert.strictEqual(result, row.expected,
|
||||
`Expected ${row.expected}, got ${result} — config: ${JSON.stringify(row.config)}`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ─── (g) DYNAMIC-ROUTING COMPOSITION ─────────────────────────────────────────
|
||||
|
||||
describe('#443 integration (g): dynamic-routing composition', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
const dynamicRoutingBase = {
|
||||
dynamic_routing: {
|
||||
enabled: true,
|
||||
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
|
||||
escalate_on_failure: true,
|
||||
max_escalations: 4,
|
||||
},
|
||||
effort: { routing_tier_defaults: { light: 'low' } },
|
||||
};
|
||||
|
||||
test('resolveEffortForTier escalates independently of model resolution', () => {
|
||||
writeConfig(tmpDir, dynamicRoutingBase);
|
||||
const effort0 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 0);
|
||||
const effort1 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1);
|
||||
const effort2 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 2);
|
||||
assert.strictEqual(effort0, 'low');
|
||||
assert.strictEqual(effort1, 'medium');
|
||||
assert.strictEqual(effort2, 'high');
|
||||
// Verify the effort ladder steps up correctly without asserting model value
|
||||
// (model timing is a separate concern from effort escalation)
|
||||
assert.notStrictEqual(effort0, effort1, 'effort should escalate at attempt 1');
|
||||
assert.notStrictEqual(effort1, effort2, 'effort should escalate at attempt 2');
|
||||
});
|
||||
|
||||
test('escalate_on_failure=false: attempt is ignored for effort', () => {
|
||||
writeConfig(tmpDir, {
|
||||
...dynamicRoutingBase,
|
||||
dynamic_routing: {
|
||||
...dynamicRoutingBase.dynamic_routing,
|
||||
escalate_on_failure: false,
|
||||
},
|
||||
});
|
||||
const e0 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 0);
|
||||
const e1 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1);
|
||||
const e3 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 3);
|
||||
assert.strictEqual(e0, e1, 'effort must not escalate when escalate_on_failure=false');
|
||||
assert.strictEqual(e0, e3, 'effort must not escalate when escalate_on_failure=false');
|
||||
});
|
||||
|
||||
test('escalation clamps at "max" regardless of attempt number', () => {
|
||||
writeConfig(tmpDir, {
|
||||
dynamic_routing: {
|
||||
enabled: true,
|
||||
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
|
||||
escalate_on_failure: true,
|
||||
max_escalations: 99,
|
||||
},
|
||||
effort: { default: 'max' },
|
||||
});
|
||||
// Any large attempt number — result must never exceed 'max'
|
||||
const r = resolveEffortForTier(tmpDir, 'gsd-planner', 50);
|
||||
assert.strictEqual(r, 'max', `Effort must clamp at 'max', got '${r}'`);
|
||||
const EFFORT_LADDER = VALID_EFFORTS;
|
||||
const maxIdx = EFFORT_LADDER.indexOf('max');
|
||||
const rIdx = EFFORT_LADDER.indexOf(r);
|
||||
assert.ok(rIdx <= maxIdx, 'Effort must not exceed the max position in the ladder');
|
||||
});
|
||||
|
||||
test('respects max_escalations cap: attempt beyond cap gives same as cap', () => {
|
||||
writeConfig(tmpDir, {
|
||||
dynamic_routing: {
|
||||
enabled: true,
|
||||
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
|
||||
escalate_on_failure: true,
|
||||
max_escalations: 1,
|
||||
},
|
||||
effort: { routing_tier_defaults: { light: 'low' } },
|
||||
});
|
||||
const atCap = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1); // 1 escalation
|
||||
const beyond = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 5); // capped at 1
|
||||
assert.strictEqual(atCap, beyond,
|
||||
'Effort beyond max_escalations must be same as at cap');
|
||||
assert.strictEqual(atCap, 'medium', 'low + 1 escalation = medium');
|
||||
});
|
||||
|
||||
test('dynamic_routing disabled: resolveEffortForTier ignores attempt', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { routing_tier_defaults: { light: 'low' } },
|
||||
});
|
||||
const e0 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 0);
|
||||
const e5 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 5);
|
||||
assert.strictEqual(e0, e5, 'Effort must not change when dynamic_routing is disabled');
|
||||
assert.strictEqual(e0, 'low');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── (h) CONFIG-TOOLING ROUND-TRIP ───────────────────────────────────────────
|
||||
|
||||
describe('#443 integration (h): config-tooling round-trip', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
test('config-set effort.default then resolve-execution reflects new value', () => {
|
||||
const setResult = runGsdTools(['config-set', 'effort.default', 'low'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(setResult.success, `config-set effort.default failed: ${setResult.error}`);
|
||||
|
||||
const execResult = runGsdTools(['resolve-execution', 'unknown-agent-xyz'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
|
||||
const output = JSON.parse(execResult.output);
|
||||
// unknown agent falls through to effort.default
|
||||
assert.strictEqual(output.effort, 'low',
|
||||
`Expected effort='low' after config-set, got '${output.effort}'`);
|
||||
});
|
||||
|
||||
test('config-set effort.routing_tier_defaults.heavy then resolve-execution uses it', () => {
|
||||
const setResult = runGsdTools(
|
||||
['config-set', 'effort.routing_tier_defaults.heavy', 'medium'],
|
||||
tmpDir, { HOME: tmpDir }
|
||||
);
|
||||
assert.ok(setResult.success, `config-set failed: ${setResult.error}`);
|
||||
|
||||
const execResult = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
|
||||
const output = JSON.parse(execResult.output);
|
||||
// gsd-planner is heavy; tier default now overridden to medium
|
||||
assert.strictEqual(output.effort, 'medium',
|
||||
`Expected effort='medium' after routing_tier_defaults override, got '${output.effort}'`);
|
||||
});
|
||||
|
||||
test('config-set effort.agent_overrides.<agent> wins over tier default', () => {
|
||||
// Set tier default first, then per-agent override
|
||||
runGsdTools(['config-set', 'effort.routing_tier_defaults.heavy', 'medium'], tmpDir, { HOME: tmpDir });
|
||||
const setResult = runGsdTools(
|
||||
['config-set', 'effort.agent_overrides.gsd-planner', 'xhigh'],
|
||||
tmpDir, { HOME: tmpDir }
|
||||
);
|
||||
assert.ok(setResult.success, `config-set agent_overrides failed: ${setResult.error}`);
|
||||
|
||||
const execResult = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
|
||||
const output = JSON.parse(execResult.output);
|
||||
assert.strictEqual(output.effort, 'xhigh',
|
||||
`Expected agent_overrides to win (xhigh), got '${output.effort}'`);
|
||||
});
|
||||
|
||||
test('config-set fast_mode.enabled true then resolve-execution reflects fast_mode=true', () => {
|
||||
const setResult = runGsdTools(['config-set', 'fast_mode.enabled', 'true'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(setResult.success, `config-set fast_mode.enabled failed: ${setResult.error}`);
|
||||
|
||||
const execResult = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
|
||||
const output = JSON.parse(execResult.output);
|
||||
assert.strictEqual(output.fast_mode, true,
|
||||
`Expected fast_mode=true after config-set, got ${output.fast_mode}`);
|
||||
// fast_mode_supported stays false (claude runtime)
|
||||
assert.strictEqual(output.fast_mode_supported, false);
|
||||
});
|
||||
|
||||
test('config-set fast_mode.agent_overrides.<agent> true reflects in output', () => {
|
||||
const setResult = runGsdTools(
|
||||
['config-set', 'fast_mode.agent_overrides.gsd-codebase-mapper', 'true'],
|
||||
tmpDir, { HOME: tmpDir }
|
||||
);
|
||||
assert.ok(setResult.success, `config-set failed: ${setResult.error}`);
|
||||
|
||||
const execResult = runGsdTools(['resolve-execution', 'gsd-codebase-mapper'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(execResult.success, `resolve-execution failed: ${execResult.error}`);
|
||||
const output = JSON.parse(execResult.output);
|
||||
assert.strictEqual(output.fast_mode, true,
|
||||
`Expected fast_mode=true for agent-specific override`);
|
||||
});
|
||||
|
||||
// Prove the config-set commands accept all the new key namespaces (schema validation)
|
||||
test('config-set accepts all effort/* and fast_mode/* key namespaces without error', () => {
|
||||
const keysToTest = [
|
||||
['effort.default', 'high'],
|
||||
['effort.routing_tier_defaults.light', 'low'],
|
||||
['effort.routing_tier_defaults.standard', 'medium'],
|
||||
['effort.routing_tier_defaults.heavy', 'xhigh'],
|
||||
['effort.agent_overrides.gsd-executor', 'high'],
|
||||
['fast_mode.enabled', 'false'],
|
||||
['fast_mode.routing_tier_defaults.light', 'false'],
|
||||
['fast_mode.routing_tier_defaults.standard', 'false'],
|
||||
['fast_mode.routing_tier_defaults.heavy', 'false'],
|
||||
['fast_mode.agent_overrides.gsd-verifier', 'false'],
|
||||
];
|
||||
for (const [key, val] of keysToTest) {
|
||||
const r = runGsdTools(['config-set', key, val], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(r.success, `config-set '${key}' '${val}' should succeed, got: ${r.error}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
862
tests/feat-443-effort-fast-mode.test.cjs
Normal file
862
tests/feat-443-effort-fast-mode.test.cjs
Normal file
@@ -0,0 +1,862 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Feature test for issue #443 — unified cross-provider effort + fast_mode knobs.
|
||||
*
|
||||
* Adds config-driven effort (universal ladder: minimal<low<medium<high<xhigh<max)
|
||||
* and fast_mode knobs. Per-runtime rendering clamps the unique tails:
|
||||
* - Anthropic/Claude: supports {low,medium,high,xhigh,max}, param=output_config.effort
|
||||
* - Codex: supports {minimal,low,medium,high,xhigh}, param=model_reasoning_effort
|
||||
*
|
||||
* Also adds resolve-execution query which is the superset command including
|
||||
* effort rendering and fast_mode propagation metadata.
|
||||
*/
|
||||
|
||||
process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
const { describe, test, beforeEach, afterEach } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
|
||||
const { createTempProject, createTempDir, cleanup, runGsdTools } = require('./helpers.cjs');
|
||||
|
||||
const {
|
||||
resolveEffortInternal,
|
||||
resolveFastModeInternal,
|
||||
resolveEffortForTier,
|
||||
} = require('../get-shit-done/bin/lib/core.cjs');
|
||||
|
||||
const {
|
||||
renderEffortForRuntime,
|
||||
RUNTIMES_WITH_FAST_MODE,
|
||||
} = require('../get-shit-done/bin/lib/model-catalog.cjs');
|
||||
|
||||
const {
|
||||
injectEffortFrontmatter,
|
||||
} = require('../bin/install.js');
|
||||
|
||||
function writeConfig(dir, config) {
|
||||
const planningDir = path.join(dir, '.planning');
|
||||
fs.mkdirSync(planningDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(planningDir, 'config.json'), JSON.stringify(config, null, 2));
|
||||
}
|
||||
|
||||
// ─── Effort cascade ───────────────────────────────────────────────────────────
|
||||
|
||||
describe('#443 effort cascade', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
test('no config -> gsd-planner (heavy) defaults to "xhigh" via tier default', () => {
|
||||
// gsd-planner is heavy tier; manifest default for heavy is xhigh
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
||||
});
|
||||
|
||||
test('routing_tier_defaults: light (gsd-codebase-mapper) -> "low"', () => {
|
||||
// gsd-codebase-mapper routingTier=light, default for light is "low"
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-codebase-mapper'), 'low');
|
||||
});
|
||||
|
||||
test('routing_tier_defaults: standard (gsd-executor) -> "high"', () => {
|
||||
// gsd-executor routingTier=standard, default for standard is "high"
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-executor'), 'high');
|
||||
});
|
||||
|
||||
test('routing_tier_defaults: heavy (gsd-planner) -> "xhigh"', () => {
|
||||
// gsd-planner routingTier=heavy, default for heavy is "xhigh"
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
||||
});
|
||||
|
||||
test('effort.routing_tier_defaults override beats tier default', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { routing_tier_defaults: { heavy: 'medium' } },
|
||||
});
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'medium');
|
||||
});
|
||||
|
||||
test('effort.agent_overrides beats routing_tier_defaults', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: {
|
||||
routing_tier_defaults: { heavy: 'medium' },
|
||||
agent_overrides: { 'gsd-planner': 'low' },
|
||||
},
|
||||
});
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'low');
|
||||
});
|
||||
|
||||
test('opts.override beats agent_overrides', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { agent_overrides: { 'gsd-planner': 'low' } },
|
||||
});
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner', { override: 'minimal' }), 'minimal');
|
||||
});
|
||||
|
||||
test('invalid override falls through to agent_overrides', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { agent_overrides: { 'gsd-planner': 'low' } },
|
||||
});
|
||||
// 'turbo' is not a valid effort — should fall through to agent_overrides
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner', { override: 'turbo' }), 'low');
|
||||
});
|
||||
|
||||
test('invalid agent_overrides value falls through to routing_tier_defaults', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: {
|
||||
agent_overrides: { 'gsd-planner': 123 },
|
||||
routing_tier_defaults: { heavy: 'medium' },
|
||||
},
|
||||
});
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'medium');
|
||||
});
|
||||
|
||||
test('invalid routing_tier_defaults value falls through to effort.default', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: {
|
||||
routing_tier_defaults: { heavy: 'turbo' },
|
||||
default: 'low',
|
||||
},
|
||||
});
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'low');
|
||||
});
|
||||
|
||||
test('invalid effort.default falls through to hardcoded "high" (no routing_tier_defaults set)', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { default: 'turbo' },
|
||||
});
|
||||
// effortCfg set but no routing_tier_defaults; turbo is invalid; fallback = hardcoded 'high'
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'high');
|
||||
});
|
||||
|
||||
test('unknown agent -> uses effort.default', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { default: 'medium' },
|
||||
});
|
||||
// unknown-agent has no routingTier, so step 3 skipped
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'unknown-agent-xyz'), 'medium');
|
||||
});
|
||||
|
||||
test('effort.default numeric value (123) ignored, hardcoded "high" fallback', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { default: 123 },
|
||||
});
|
||||
// effortCfg set, no routing_tier_defaults -> no tier default; numeric ignored -> 'high'
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'high');
|
||||
});
|
||||
|
||||
test('effort block missing entirely -> uses tier default', () => {
|
||||
// No effort key in config at all
|
||||
writeConfig(tmpDir, { model_profile: 'balanced' });
|
||||
// heavy agent: tier default xhigh
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
||||
});
|
||||
|
||||
test('effort block is non-object (string) -> effortCfg=null -> uses manifest tier default xhigh', () => {
|
||||
writeConfig(tmpDir, { effort: 'bad' });
|
||||
// Non-object effort => effortCfg=null; gsd-planner heavy tier manifest default = xhigh
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
||||
});
|
||||
|
||||
test('effort.routing_tier_defaults empty object -> effort.default', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { routing_tier_defaults: {}, default: 'low' },
|
||||
});
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'low');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Fast mode cascade ────────────────────────────────────────────────────────
|
||||
|
||||
describe('#443 fast_mode cascade', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
test('no config -> defaults to false', () => {
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-planner'), false);
|
||||
});
|
||||
|
||||
test('fast_mode.enabled=true -> true when no tier/agent overrides', () => {
|
||||
writeConfig(tmpDir, { fast_mode: { enabled: true } });
|
||||
// heavy agent: tier default is false, but enabled=true is layer 4
|
||||
// tier default for heavy is false (below enabled), so gets enabled=true
|
||||
// Wait — the cascade is: 1.override 2.agent_overrides 3.tier_defaults 4.enabled 5.false
|
||||
// For gsd-planner (heavy), tier default is false — falls through to enabled=true
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-planner'), true);
|
||||
});
|
||||
|
||||
test('fast_mode.routing_tier_defaults.light=true -> light agent gets true', () => {
|
||||
writeConfig(tmpDir, {
|
||||
fast_mode: { routing_tier_defaults: { light: true } },
|
||||
});
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-codebase-mapper'), true);
|
||||
});
|
||||
|
||||
test('fast_mode.routing_tier_defaults.heavy=false -> heavy agent stays false', () => {
|
||||
writeConfig(tmpDir, {
|
||||
fast_mode: { enabled: true, routing_tier_defaults: { heavy: false } },
|
||||
});
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-planner'), false);
|
||||
});
|
||||
|
||||
test('fast_mode.agent_overrides beats routing_tier_defaults', () => {
|
||||
writeConfig(tmpDir, {
|
||||
fast_mode: {
|
||||
routing_tier_defaults: { light: false },
|
||||
agent_overrides: { 'gsd-codebase-mapper': true },
|
||||
},
|
||||
});
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-codebase-mapper'), true);
|
||||
});
|
||||
|
||||
test('opts.override beats agent_overrides', () => {
|
||||
writeConfig(tmpDir, {
|
||||
fast_mode: { agent_overrides: { 'gsd-planner': true } },
|
||||
});
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-planner', { override: false }), false);
|
||||
});
|
||||
|
||||
test('string "true" NOT accepted as fast_mode override', () => {
|
||||
writeConfig(tmpDir, {
|
||||
fast_mode: { agent_overrides: { 'gsd-planner': 'true' } },
|
||||
});
|
||||
// string "true" is not boolean -> fall through to tier default or enabled
|
||||
const result = resolveFastModeInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(typeof result, 'boolean');
|
||||
});
|
||||
|
||||
test('string "true" in opts.override NOT accepted', () => {
|
||||
// opts.override must be strict boolean — string falls through
|
||||
const result = resolveFastModeInternal(tmpDir, 'gsd-planner', { override: 'true' });
|
||||
assert.strictEqual(result, false);
|
||||
});
|
||||
|
||||
test('fast_mode block missing entirely -> defaults to false', () => {
|
||||
writeConfig(tmpDir, { model_profile: 'balanced' });
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-planner'), false);
|
||||
});
|
||||
|
||||
test('fast_mode.enabled="yes" (non-boolean) ignored -> false', () => {
|
||||
writeConfig(tmpDir, { fast_mode: { enabled: 'yes' } });
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-planner'), false);
|
||||
});
|
||||
|
||||
test('unknown agent fast_mode -> uses enabled flag', () => {
|
||||
writeConfig(tmpDir, { fast_mode: { enabled: true } });
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'unknown-agent-xyz'), true);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Effort escalation (resolveEffortForTier) ─────────────────────────────────
|
||||
|
||||
describe('#443 resolveEffortForTier escalation', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
test('dynamic_routing disabled -> attempt ignored, returns base effort', () => {
|
||||
// gsd-planner heavy -> xhigh baseline
|
||||
const base = resolveEffortForTier(tmpDir, 'gsd-planner', 0);
|
||||
const attempt1 = resolveEffortForTier(tmpDir, 'gsd-planner', 1);
|
||||
assert.strictEqual(base, 'xhigh');
|
||||
assert.strictEqual(attempt1, 'xhigh'); // no dynamic_routing -> attempt ignored
|
||||
});
|
||||
|
||||
test('dynamic_routing enabled, escalate_on_failure=false -> attempt ignored', () => {
|
||||
writeConfig(tmpDir, {
|
||||
dynamic_routing: {
|
||||
enabled: true,
|
||||
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
|
||||
escalate_on_failure: false,
|
||||
max_escalations: 2,
|
||||
},
|
||||
});
|
||||
const base = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 0);
|
||||
const attempt1 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1);
|
||||
assert.strictEqual(base, attempt1);
|
||||
});
|
||||
|
||||
test('dynamic_routing enabled, attempt=1 -> one step up from base', () => {
|
||||
writeConfig(tmpDir, {
|
||||
dynamic_routing: {
|
||||
enabled: true,
|
||||
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
|
||||
escalate_on_failure: true,
|
||||
max_escalations: 2,
|
||||
},
|
||||
effort: { routing_tier_defaults: { light: 'low' } },
|
||||
});
|
||||
// gsd-codebase-mapper: light -> effort 'low'; attempt=1 -> 'medium'
|
||||
assert.strictEqual(resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 0), 'low');
|
||||
assert.strictEqual(resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1), 'medium');
|
||||
});
|
||||
|
||||
test('escalation clamps at "max"', () => {
|
||||
writeConfig(tmpDir, {
|
||||
dynamic_routing: {
|
||||
enabled: true,
|
||||
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
|
||||
escalate_on_failure: true,
|
||||
max_escalations: 99,
|
||||
},
|
||||
effort: { default: 'xhigh' },
|
||||
});
|
||||
// xhigh -> max -> max (clamp)
|
||||
const result = resolveEffortForTier(tmpDir, 'gsd-planner', 99);
|
||||
assert.strictEqual(result, 'max');
|
||||
});
|
||||
|
||||
test('respects max_escalations cap', () => {
|
||||
writeConfig(tmpDir, {
|
||||
dynamic_routing: {
|
||||
enabled: true,
|
||||
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
|
||||
escalate_on_failure: true,
|
||||
max_escalations: 1,
|
||||
},
|
||||
effort: { routing_tier_defaults: { light: 'low' } },
|
||||
});
|
||||
// light: low -> attempt=1 -> medium (but max=1 so can only escalate once)
|
||||
const at1 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 1);
|
||||
const at2 = resolveEffortForTier(tmpDir, 'gsd-codebase-mapper', 2);
|
||||
// at2 is capped at 1 escalation, same as at1
|
||||
assert.strictEqual(at1, at2);
|
||||
assert.strictEqual(at1, 'medium');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Rendering / clamping ──────────────────────────────────────────────────────
|
||||
|
||||
describe('#443 renderEffortForRuntime', () => {
|
||||
test('codex: "max" clamps to "xhigh"', () => {
|
||||
const r = renderEffortForRuntime('codex', 'max');
|
||||
assert.strictEqual(r.value, 'xhigh');
|
||||
assert.strictEqual(r.param, 'model_reasoning_effort');
|
||||
});
|
||||
|
||||
test('codex: common levels passthrough', () => {
|
||||
assert.strictEqual(renderEffortForRuntime('codex', 'low').value, 'low');
|
||||
assert.strictEqual(renderEffortForRuntime('codex', 'medium').value, 'medium');
|
||||
assert.strictEqual(renderEffortForRuntime('codex', 'high').value, 'high');
|
||||
assert.strictEqual(renderEffortForRuntime('codex', 'xhigh').value, 'xhigh');
|
||||
});
|
||||
|
||||
test('codex: "minimal" passthrough', () => {
|
||||
assert.strictEqual(renderEffortForRuntime('codex', 'minimal').value, 'minimal');
|
||||
});
|
||||
|
||||
test('claude: "minimal" clamps to "low"', () => {
|
||||
const r = renderEffortForRuntime('claude', 'minimal');
|
||||
assert.strictEqual(r.value, 'low');
|
||||
assert.strictEqual(r.param, 'output_config.effort');
|
||||
});
|
||||
|
||||
test('claude: "max" passthrough (Anthropic-only)', () => {
|
||||
const r = renderEffortForRuntime('claude', 'max');
|
||||
assert.strictEqual(r.value, 'max');
|
||||
assert.strictEqual(r.param, 'output_config.effort');
|
||||
});
|
||||
|
||||
test('claude: common levels passthrough', () => {
|
||||
assert.strictEqual(renderEffortForRuntime('claude', 'low').value, 'low');
|
||||
assert.strictEqual(renderEffortForRuntime('claude', 'medium').value, 'medium');
|
||||
assert.strictEqual(renderEffortForRuntime('claude', 'high').value, 'high');
|
||||
assert.strictEqual(renderEffortForRuntime('claude', 'xhigh').value, 'xhigh');
|
||||
});
|
||||
|
||||
test('unknown runtime: param is null, value passthrough', () => {
|
||||
const r = renderEffortForRuntime('unknown-runtime', 'high');
|
||||
assert.strictEqual(r.param, null);
|
||||
assert.strictEqual(r.value, 'high');
|
||||
});
|
||||
|
||||
test('RUNTIMES_WITH_FAST_MODE does NOT include "claude"', () => {
|
||||
// Claude Code has no per-subagent fast-mode mechanism — session-level only
|
||||
assert.ok(!RUNTIMES_WITH_FAST_MODE.has('claude'),
|
||||
'claude must NOT be in RUNTIMES_WITH_FAST_MODE — emitting fast_mode frontmatter is a silent no-op');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── resolve-execution end-to-end ─────────────────────────────────────────────
|
||||
|
||||
describe('#443 resolve-execution CLI command', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => {
|
||||
tmpDir = createTempProject();
|
||||
// HOME isolation to prevent ~/.gsd/defaults.json bleed
|
||||
process.env._GSD_TEST_HOME_OVERRIDE = tmpDir;
|
||||
});
|
||||
afterEach(() => {
|
||||
cleanup(tmpDir);
|
||||
delete process.env._GSD_TEST_HOME_OVERRIDE;
|
||||
});
|
||||
|
||||
test('default (claude) runtime -> effort present, effort_param=output_config.effort, fast_mode_supported=false', () => {
|
||||
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok(output.effort, 'should have effort field');
|
||||
assert.strictEqual(output.effort_param, 'output_config.effort');
|
||||
assert.strictEqual(output.fast_mode_supported, false);
|
||||
assert.ok('fast_mode' in output, 'should have fast_mode field');
|
||||
assert.ok('model' in output, 'should have model field');
|
||||
assert.ok('profile' in output, 'should have profile field');
|
||||
});
|
||||
|
||||
test('codex runtime -> effort_param=model_reasoning_effort, max clamps to xhigh, fast_mode_supported=false', () => {
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
effort: { default: 'max' },
|
||||
});
|
||||
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.effort_param, 'model_reasoning_effort');
|
||||
assert.strictEqual(output.effort_rendered, 'xhigh');
|
||||
// fast_mode_supported: codex does not support fast mode via subagent
|
||||
assert.strictEqual(output.fast_mode_supported, false);
|
||||
});
|
||||
|
||||
test('--effort flag overrides config effort', () => {
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', 'gsd-planner', '--effort', 'low'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.effort, 'low');
|
||||
});
|
||||
|
||||
test('--fast-mode flag honored', () => {
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', 'gsd-planner', '--fast-mode', 'true'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.fast_mode, true);
|
||||
});
|
||||
|
||||
test('--attempt flag triggers escalation', () => {
|
||||
writeConfig(tmpDir, {
|
||||
dynamic_routing: {
|
||||
enabled: true,
|
||||
tier_models: { light: 'haiku', standard: 'sonnet', heavy: 'opus' },
|
||||
escalate_on_failure: true,
|
||||
max_escalations: 2,
|
||||
},
|
||||
effort: { routing_tier_defaults: { light: 'low' } },
|
||||
});
|
||||
const result0 = runGsdTools(
|
||||
['resolve-execution', 'gsd-codebase-mapper', '--attempt', '0'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
const result1 = runGsdTools(
|
||||
['resolve-execution', 'gsd-codebase-mapper', '--attempt', '1'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(result0.success && result1.success);
|
||||
const out0 = JSON.parse(result0.output);
|
||||
const out1 = JSON.parse(result1.output);
|
||||
assert.strictEqual(out0.effort, 'low');
|
||||
assert.strictEqual(out1.effort, 'medium');
|
||||
});
|
||||
|
||||
test('--raw prints effort string', () => {
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', 'gsd-planner', '--raw'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
// Raw output should be the effort string
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
assert.ok(VALID_EFFORTS.includes(result.output.trim()),
|
||||
`Expected effort string, got: ${result.output}`);
|
||||
});
|
||||
|
||||
test('fails when no agent-type provided', () => {
|
||||
const result = runGsdTools(['resolve-execution'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(!result.success, 'should fail without agent-type');
|
||||
assert.ok(result.error.includes('agent-type required'), `error: ${result.error}`);
|
||||
});
|
||||
|
||||
test('unknown agent -> unknown_agent=true still emits effort', () => {
|
||||
const result = runGsdTools(['resolve-execution', 'unknown-agent-xyz'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.unknown_agent, true);
|
||||
assert.ok(output.effort, 'should have effort even for unknown agent');
|
||||
});
|
||||
|
||||
test('emits effort_propagation (channel) field', () => {
|
||||
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok('effort_propagation' in output, 'should have effort_propagation field');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── resolve-model now emits effort (replaces reasoning_effort) ───────────────
|
||||
|
||||
describe('#443 resolve-model emits effort (unified)', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
test('resolve-model on claude runtime emits effort (not null)', () => {
|
||||
const result = runGsdTools(['resolve-model', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
// effort must be present and valid
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
assert.ok(VALID_EFFORTS.includes(output.effort),
|
||||
`Expected valid effort, got: ${output.effort}`);
|
||||
// reasoning_effort must NOT be present (removed)
|
||||
assert.ok(!Object.prototype.hasOwnProperty.call(output, 'reasoning_effort'),
|
||||
'resolve-model must not emit reasoning_effort (replaced by effort)');
|
||||
});
|
||||
|
||||
test('resolve-model on codex runtime emits unified effort (not reasoning_effort)', () => {
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, '.planning', 'config.json'),
|
||||
JSON.stringify({ runtime: 'codex', model_profile: 'balanced' })
|
||||
);
|
||||
const result = runGsdTools(['resolve-model', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
assert.ok(VALID_EFFORTS.includes(output.effort),
|
||||
`Expected valid effort, got: ${output.effort}`);
|
||||
assert.ok(!Object.prototype.hasOwnProperty.call(output, 'reasoning_effort'),
|
||||
'resolve-model must not emit reasoning_effort');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── QA Matrix — hostile/malformed configs ───────────────────────────────────
|
||||
|
||||
describe('#443 QA matrix — malformed effort/fast_mode configs', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { tmpDir = createTempProject(); });
|
||||
afterEach(() => { cleanup(tmpDir); });
|
||||
|
||||
test('effort.default=123 (numeric) -> gracefully falls through', () => {
|
||||
writeConfig(tmpDir, { effort: { default: 123 } });
|
||||
// gsd-planner is heavy, tier default xhigh is used instead
|
||||
const result = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.ok(typeof result === 'string');
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
assert.ok(VALID_EFFORTS.includes(result));
|
||||
});
|
||||
|
||||
test('fast_mode.enabled="yes" (string) -> ignored, returns false', () => {
|
||||
writeConfig(tmpDir, { fast_mode: { enabled: 'yes' } });
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-planner'), false);
|
||||
});
|
||||
|
||||
test('effort:{} empty block -> uses tier default or hardcoded high', () => {
|
||||
writeConfig(tmpDir, { effort: {} });
|
||||
const result = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
assert.ok(VALID_EFFORTS.includes(result));
|
||||
});
|
||||
|
||||
test('fast_mode:{} empty block -> false', () => {
|
||||
writeConfig(tmpDir, { fast_mode: {} });
|
||||
assert.strictEqual(resolveFastModeInternal(tmpDir, 'gsd-planner'), false);
|
||||
});
|
||||
|
||||
test('effort config is completely absent -> still resolves valid effort', () => {
|
||||
writeConfig(tmpDir, { model_profile: 'quality' });
|
||||
const result = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
assert.ok(VALID_EFFORTS.includes(result));
|
||||
});
|
||||
|
||||
test('effort.routing_tier_defaults has boolean value -> falls through', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: {
|
||||
routing_tier_defaults: { heavy: true },
|
||||
default: 'medium',
|
||||
},
|
||||
});
|
||||
// boolean true is not a valid effort -> falls through to default 'medium'
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'gsd-planner'), 'medium');
|
||||
});
|
||||
|
||||
test('effort.agent_overrides is non-object -> falls through gracefully', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: {
|
||||
agent_overrides: 'not-an-object',
|
||||
default: 'low',
|
||||
},
|
||||
});
|
||||
// non-object agent_overrides -> skip step 2, use tier default (heavy=xhigh)
|
||||
// actually heavy tier default kicks in first if no routing_tier_defaults
|
||||
const result = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
assert.ok(VALID_EFFORTS.includes(result));
|
||||
});
|
||||
|
||||
test('config.json has unknown agent with effort.default set -> uses effort.default', () => {
|
||||
writeConfig(tmpDir, { effort: { default: 'minimal' } });
|
||||
assert.strictEqual(resolveEffortInternal(tmpDir, 'completely-unknown-agent-98765'), 'minimal');
|
||||
});
|
||||
|
||||
test('resolve-execution with malformed config does not crash', () => {
|
||||
writeConfig(tmpDir, {
|
||||
effort: { default: null, routing_tier_defaults: null },
|
||||
fast_mode: { enabled: null, agent_overrides: null },
|
||||
});
|
||||
const result = runGsdTools(['resolve-execution', 'gsd-planner'], tmpDir, { HOME: tmpDir });
|
||||
assert.ok(result.success, `Should not crash with null config values: ${result.error}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Config schema: new keys are valid ───────────────────────────────────────
|
||||
|
||||
describe('#443 config schema: new effort/fast_mode keys valid', () => {
|
||||
const { isValidConfigKey } = require('../get-shit-done/bin/lib/config-schema.cjs');
|
||||
|
||||
test('effort.default is a valid config key', () => {
|
||||
assert.ok(isValidConfigKey('effort.default'), 'effort.default must be valid');
|
||||
});
|
||||
|
||||
test('fast_mode.enabled is a valid config key', () => {
|
||||
assert.ok(isValidConfigKey('fast_mode.enabled'), 'fast_mode.enabled must be valid');
|
||||
});
|
||||
|
||||
test('effort.routing_tier_defaults.light is valid (dynamic pattern)', () => {
|
||||
assert.ok(isValidConfigKey('effort.routing_tier_defaults.light'));
|
||||
});
|
||||
|
||||
test('effort.routing_tier_defaults.standard is valid', () => {
|
||||
assert.ok(isValidConfigKey('effort.routing_tier_defaults.standard'));
|
||||
});
|
||||
|
||||
test('effort.routing_tier_defaults.heavy is valid', () => {
|
||||
assert.ok(isValidConfigKey('effort.routing_tier_defaults.heavy'));
|
||||
});
|
||||
|
||||
test('effort.agent_overrides.<agent-id> is valid (dynamic pattern)', () => {
|
||||
assert.ok(isValidConfigKey('effort.agent_overrides.gsd-planner'));
|
||||
assert.ok(isValidConfigKey('effort.agent_overrides.my-custom-agent'));
|
||||
});
|
||||
|
||||
test('fast_mode.routing_tier_defaults.light is valid', () => {
|
||||
assert.ok(isValidConfigKey('fast_mode.routing_tier_defaults.light'));
|
||||
});
|
||||
|
||||
test('fast_mode.agent_overrides.<agent-id> is valid', () => {
|
||||
assert.ok(isValidConfigKey('fast_mode.agent_overrides.gsd-planner'));
|
||||
});
|
||||
|
||||
test('effort.routing_tier_defaults.invalid-tier is NOT valid', () => {
|
||||
assert.ok(!isValidConfigKey('effort.routing_tier_defaults.super'));
|
||||
});
|
||||
});
|
||||
|
||||
// ─── resolve-execution arg parsing matrix (Codex adversarial finding #1) ──────
|
||||
//
|
||||
// These tests FAIL before the fix: flags-first ordering misroutes the agent.
|
||||
|
||||
describe('#443 resolve-execution: deterministic arg parsing (flags-first ordering)', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => {
|
||||
tmpDir = createTempProject();
|
||||
process.env._GSD_TEST_HOME_OVERRIDE = tmpDir;
|
||||
});
|
||||
afterEach(() => {
|
||||
cleanup(tmpDir);
|
||||
delete process.env._GSD_TEST_HOME_OVERRIDE;
|
||||
});
|
||||
|
||||
test('flags-first: --effort low gsd-planner resolves gsd-planner (NOT "low" as agent)', () => {
|
||||
// BUG: before fix, agentTypeArg = 'low' (first non-dash token) -> unknown_agent:true
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', '--effort', 'low', 'gsd-planner'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok(!output.unknown_agent, `agent must be resolved (not unknown_agent), got: ${JSON.stringify(output)}`);
|
||||
assert.strictEqual(output.effort, 'low', `effort should be low, got: ${output.effort}`);
|
||||
});
|
||||
|
||||
test('flags-first: --attempt 1 gsd-codebase-mapper resolves gsd-codebase-mapper (NOT "1" as agent)', () => {
|
||||
// BUG: before fix, agentTypeArg = '1' -> unknown_agent:true
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', '--attempt', '1', 'gsd-codebase-mapper'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok(!output.unknown_agent, `gsd-codebase-mapper must be resolved, got: ${JSON.stringify(output)}`);
|
||||
});
|
||||
|
||||
test('agent-first parity: gsd-planner --effort low produces same effort as flags-first', () => {
|
||||
const flagsFirst = runGsdTools(
|
||||
['resolve-execution', '--effort', 'low', 'gsd-planner'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
const agentFirst = runGsdTools(
|
||||
['resolve-execution', 'gsd-planner', '--effort', 'low'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(flagsFirst.success && agentFirst.success,
|
||||
`Both orderings must succeed. flags-first err: ${flagsFirst.error} agent-first err: ${agentFirst.error}`);
|
||||
const outFF = JSON.parse(flagsFirst.output);
|
||||
const outAF = JSON.parse(agentFirst.output);
|
||||
assert.strictEqual(outFF.effort, outAF.effort, 'effort must be identical for both orderings');
|
||||
assert.strictEqual(outFF.model, outAF.model, 'model must be identical for both orderings');
|
||||
});
|
||||
|
||||
test('error: missing agent (--effort low with no positional) -> non-zero exit, no stack trace', () => {
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', '--effort', 'low'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(!result.success, 'must exit non-zero when agent is missing');
|
||||
assert.ok(!result.error.includes('at '), `error must not contain stack trace, got: ${result.error}`);
|
||||
assert.ok(result.error.length > 0, 'must emit an error message');
|
||||
});
|
||||
|
||||
test('error: two positional agents -> non-zero exit', () => {
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', 'gsd-planner', 'gsd-executor'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(!result.success, 'must exit non-zero when two agents are given');
|
||||
});
|
||||
|
||||
test('error: --attempt notanumber -> non-zero exit, clear error', () => {
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', '--attempt', 'notanumber', 'gsd-planner'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(!result.success, 'must exit non-zero for non-integer --attempt');
|
||||
assert.ok(result.error.length > 0, 'must emit an error message');
|
||||
});
|
||||
|
||||
test('error: trailing --effort (no value) -> non-zero exit', () => {
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', 'gsd-planner', '--effort'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(!result.success, 'must exit non-zero for trailing --effort with no value');
|
||||
assert.ok(result.error.length > 0, 'must emit an error message');
|
||||
});
|
||||
|
||||
test('unknown agent positional -> unknown_agent:true (preserved behavior)', () => {
|
||||
const result = runGsdTools(
|
||||
['resolve-execution', 'totally-not-an-agent'],
|
||||
tmpDir,
|
||||
{ HOME: tmpDir }
|
||||
);
|
||||
assert.ok(result.success, `Should succeed (unknown agent is valid input): ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.unknown_agent, true, 'unknown agent must emit unknown_agent:true');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── injectEffortFrontmatter: newline-agnostic injection (#443 Windows fix) ──
|
||||
|
||||
describe('#443 injectEffortFrontmatter: newline-agnostic YAML frontmatter injection', () => {
|
||||
// LF source (macOS / Linux git checkout) — baseline
|
||||
test('LF frontmatter: injects effort: before closing ---', () => {
|
||||
const content = '---\nname: gsd-planner\ndescription: Creates plans\ncolor: blue\n---\nBody here\n';
|
||||
const result = injectEffortFrontmatter(content, 'xhigh');
|
||||
assert.notStrictEqual(result, content, 'content should be modified');
|
||||
assert.match(result, /^effort:\s*xhigh$/m, 'effort: xhigh must be present');
|
||||
assert.ok(result.includes('\neffort: xhigh\n---\n'), 'effort: must appear before closing --- with LF');
|
||||
// Closing --- must still be present and intact
|
||||
assert.ok(result.includes('\n---\n'), 'closing --- must remain with LF');
|
||||
});
|
||||
|
||||
// CRLF source (Windows git checkout with core.autocrlf=true) — the actual bug
|
||||
test('CRLF frontmatter: injects effort: with CRLF preserved (Windows fix)', () => {
|
||||
const content = '---\r\nname: gsd-planner\r\ndescription: Creates plans\r\ncolor: blue\r\n---\r\nBody here\r\n';
|
||||
const result = injectEffortFrontmatter(content, 'xhigh');
|
||||
assert.notStrictEqual(result, content, 'content should be modified (CRLF source was silently skipped before fix)');
|
||||
// effort: line must use CRLF, not LF (EOL consistency)
|
||||
assert.ok(result.includes('effort: xhigh\r\n'), 'effort: line must use CRLF to match surrounding frontmatter');
|
||||
// Closing --- must use CRLF and remain intact
|
||||
assert.ok(result.includes('\r\neffort: xhigh\r\n---\r\n'), 'effort: must appear before closing ---\\r\\n with CRLF');
|
||||
// The effort value must be readable via multiline regex (as the install-wiring assertions do)
|
||||
assert.match(result, /^effort:\s*xhigh$/m, '/^effort:\\s*xhigh$/m must match in CRLF output');
|
||||
});
|
||||
|
||||
// Idempotency: don't double-insert if effort: already exists
|
||||
test('idempotent: does NOT insert a second effort: line when already present (LF)', () => {
|
||||
const content = '---\nname: gsd-planner\neffort: high\n---\nBody\n';
|
||||
const result = injectEffortFrontmatter(content, 'xhigh');
|
||||
assert.strictEqual(result, content, 'content must be unchanged when effort: already present');
|
||||
// Confirm no duplicate
|
||||
const matches = [...result.matchAll(/^effort:/mg)];
|
||||
assert.strictEqual(matches.length, 1, 'exactly one effort: key must exist');
|
||||
});
|
||||
|
||||
test('idempotent: does NOT insert a second effort: line when already present (CRLF)', () => {
|
||||
const content = '---\r\nname: gsd-planner\r\neffort: high\r\n---\r\nBody\r\n';
|
||||
const result = injectEffortFrontmatter(content, 'xhigh');
|
||||
assert.strictEqual(result, content, 'content must be unchanged when effort: already present (CRLF)');
|
||||
});
|
||||
|
||||
// No frontmatter — leave unchanged
|
||||
test('no YAML frontmatter: returns content unchanged', () => {
|
||||
const content = 'Just a body\nNo frontmatter here\n';
|
||||
const result = injectEffortFrontmatter(content, 'xhigh');
|
||||
assert.strictEqual(result, content, 'content without frontmatter must be returned unchanged');
|
||||
});
|
||||
|
||||
// Complex frontmatter with comment lines and color: key (mirrors real agent .md files)
|
||||
test('complex LF frontmatter (# comment + color:) still injects effort: before ---', () => {
|
||||
const content = [
|
||||
'---',
|
||||
'name: gsd-executor',
|
||||
'# hooks: see .claude/settings.json',
|
||||
'description: Executes tasks',
|
||||
'color: green',
|
||||
'---',
|
||||
'Body content here',
|
||||
'',
|
||||
].join('\n');
|
||||
const result = injectEffortFrontmatter(content, 'high');
|
||||
assert.match(result, /^effort:\s*high$/m, 'effort: high must be present');
|
||||
assert.ok(result.includes('\neffort: high\n---\n'), 'effort: must appear immediately before closing ---');
|
||||
// Other frontmatter fields must be untouched
|
||||
assert.ok(result.includes('color: green'), 'color: must be preserved');
|
||||
assert.ok(result.includes('# hooks:'), '# comment must be preserved');
|
||||
});
|
||||
|
||||
test('complex CRLF frontmatter (# comment + color:) still injects effort: with CRLF before ---', () => {
|
||||
const lines = [
|
||||
'---',
|
||||
'name: gsd-executor',
|
||||
'# hooks: see .claude/settings.json',
|
||||
'description: Executes tasks',
|
||||
'color: green',
|
||||
'---',
|
||||
'Body content here',
|
||||
'',
|
||||
];
|
||||
const content = lines.join('\r\n');
|
||||
const result = injectEffortFrontmatter(content, 'high');
|
||||
assert.ok(result.includes('effort: high\r\n'), 'effort: must use CRLF in CRLF file');
|
||||
assert.ok(result.includes('\r\neffort: high\r\n---\r\n'), 'effort: must appear before closing ---\\r\\n');
|
||||
assert.ok(result.includes('color: green\r\n'), 'color: must be preserved with CRLF');
|
||||
});
|
||||
});
|
||||
408
tests/feat-443-effort-install-wiring.install.test.cjs
Normal file
408
tests/feat-443-effort-install-wiring.install.test.cjs
Normal file
@@ -0,0 +1,408 @@
|
||||
// allow-test-rule: integration-test-input
|
||||
// Exercises install() + generateCodexAgentToml() as a black-box by inspecting
|
||||
// produced output files in temp dirs. Source agent .md files are inputs whose
|
||||
// installed transformation is asserted — not inspected for string presence.
|
||||
|
||||
/**
|
||||
* #443 — Effort per-runtime wiring at install time.
|
||||
*
|
||||
* Verifies:
|
||||
* 1. Claude global install injects `effort:` into agent .md frontmatter.
|
||||
* 2. Gemini global install does NOT inject `effort:` (Gemini-safe .md).
|
||||
* 3. Codex global install emits `model_reasoning_effort` in .toml via the
|
||||
* unified resolver (not the old catalog-static path).
|
||||
* 4. Config-driven proof: effort.agent_overrides wins over tier defaults
|
||||
* for both Claude .md and Codex .toml.
|
||||
* 5. Source agents/gsd-planner.md has NO effort: key (injection is
|
||||
* install-only, source stays Gemini-safe).
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
|
||||
process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
const { describe, test, beforeEach, afterEach } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const os = require('node:os');
|
||||
|
||||
const { install } = require('../bin/install.js');
|
||||
|
||||
const REPO_ROOT = path.resolve(__dirname, '..');
|
||||
const SOURCE_AGENTS_DIR = path.join(REPO_ROOT, 'agents');
|
||||
|
||||
// ─── helpers ─────────────────────────────────────────────────────────────────
|
||||
|
||||
function makeTmpDir(prefix) {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), prefix));
|
||||
}
|
||||
|
||||
function readFrontmatter(mdPath) {
|
||||
const content = fs.readFileSync(mdPath, 'utf8');
|
||||
if (!content.startsWith('---')) return '';
|
||||
const end = content.indexOf('---', 3);
|
||||
if (end === -1) return '';
|
||||
return content.substring(3, end);
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a global install for the given runtime, redirecting its home dir to
|
||||
* tmpHome. Returns the tmpHome for inspection.
|
||||
*
|
||||
* Env-var redirection:
|
||||
* claude → CLAUDE_CONFIG_DIR
|
||||
* gemini → GEMINI_CONFIG_DIR
|
||||
* codex → CODEX_HOME
|
||||
*
|
||||
* HOME is also redirected to an isolated temp dir for the duration of the
|
||||
* install call. This prevents any install.js code that uses os.homedir()
|
||||
* directly (e.g. ~/.cache/gsd update-check deletion, ~/.gsd/defaults.json
|
||||
* reads, stale-SDK npm subprocess writes to ~/.npm) from touching the real
|
||||
* HOME and polluting the test environment for other concurrently-running
|
||||
* test files (e.g. runtime-launcher-parity test (D) checks that
|
||||
* $HOME/.claude/get-shit-done/bin/gsd-tools.cjs is absent).
|
||||
*
|
||||
* GSD_SKIP_STALE_SDK_CHECK=1 is set to suppress the `npm ls -g` subprocess
|
||||
* that the installer spawns for global installs — that subprocess is slow,
|
||||
* writes to ~/.npm cache, and is irrelevant to effort-wiring assertions.
|
||||
*
|
||||
* The working directory is set to REPO_ROOT so install() can find the source
|
||||
* agents/. For config-driven tests, place tmpHome inside the project dir
|
||||
* so that readGsdEffectiveEffortConfig(targetDir) can walk up from tmpHome
|
||||
* and find .planning/config.json.
|
||||
*/
|
||||
function runGlobalInstall(runtime, tmpHome) {
|
||||
const envVarMap = {
|
||||
claude: 'CLAUDE_CONFIG_DIR',
|
||||
gemini: 'GEMINI_CONFIG_DIR',
|
||||
codex: 'CODEX_HOME',
|
||||
};
|
||||
const envVar = envVarMap[runtime];
|
||||
if (!envVar) throw new Error(`Unsupported runtime in test: ${runtime}`);
|
||||
|
||||
// Isolate HOME to a fresh temp dir so install.js code that calls
|
||||
// os.homedir() (cache deletion, defaults.json reads, npm subprocess)
|
||||
// never touches the real $HOME/.claude / $HOME/.cache / $HOME/.gsd.
|
||||
const isolatedHome = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-443-home-'));
|
||||
|
||||
const prev = process.env[envVar];
|
||||
const prevCwd = process.cwd();
|
||||
const prevHome = process.env.HOME;
|
||||
const prevSkipStale = process.env.GSD_SKIP_STALE_SDK_CHECK;
|
||||
|
||||
process.env[envVar] = tmpHome;
|
||||
process.env.HOME = isolatedHome;
|
||||
process.env.GSD_SKIP_STALE_SDK_CHECK = '1';
|
||||
process.chdir(REPO_ROOT);
|
||||
|
||||
try {
|
||||
install(true, runtime);
|
||||
} finally {
|
||||
process.chdir(prevCwd);
|
||||
if (prev === undefined) delete process.env[envVar];
|
||||
else process.env[envVar] = prev;
|
||||
if (prevHome === undefined) delete process.env.HOME;
|
||||
else process.env.HOME = prevHome;
|
||||
if (prevSkipStale === undefined) delete process.env.GSD_SKIP_STALE_SDK_CHECK;
|
||||
else process.env.GSD_SKIP_STALE_SDK_CHECK = prevSkipStale;
|
||||
// Clean up the isolated HOME dir
|
||||
try { fs.rmSync(isolatedHome, { recursive: true, force: true }); } catch (_) { /* best-effort */ }
|
||||
}
|
||||
|
||||
return tmpHome;
|
||||
}
|
||||
|
||||
// ─── Tier default expectations ────────────────────────────────────────────────
|
||||
// light → low, standard → high, heavy → xhigh (catalog defaults)
|
||||
// gsd-planner: heavy → xhigh
|
||||
// gsd-codebase-mapper: light → low
|
||||
// gsd-executor: standard → high
|
||||
|
||||
// ─── describe 1: Claude install injects effort: ───────────────────────────────
|
||||
|
||||
describe('#443 Claude install: effort: injected into frontmatter', () => {
|
||||
let tmpDir;
|
||||
let claudeHome;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpDir = makeTmpDir('gsd-443-claude-');
|
||||
claudeHome = path.join(tmpDir, 'claude-home');
|
||||
fs.mkdirSync(claudeHome, { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('gsd-planner.md contains effort: xhigh (heavy tier default)', () => {
|
||||
runGlobalInstall('claude', claudeHome);
|
||||
const fm = readFrontmatter(path.join(claudeHome, 'agents', 'gsd-planner.md'));
|
||||
assert.match(fm, /^effort:\s*xhigh$/m,
|
||||
`gsd-planner frontmatter should have effort: xhigh\nActual:\n${fm}`);
|
||||
});
|
||||
|
||||
test('gsd-codebase-mapper.md contains effort: low (light tier default)', () => {
|
||||
runGlobalInstall('claude', claudeHome);
|
||||
const fm = readFrontmatter(path.join(claudeHome, 'agents', 'gsd-codebase-mapper.md'));
|
||||
assert.match(fm, /^effort:\s*low$/m,
|
||||
`gsd-codebase-mapper frontmatter should have effort: low\nActual:\n${fm}`);
|
||||
});
|
||||
|
||||
test('gsd-executor.md contains effort: high (standard tier default)', () => {
|
||||
runGlobalInstall('claude', claudeHome);
|
||||
const fm = readFrontmatter(path.join(claudeHome, 'agents', 'gsd-executor.md'));
|
||||
assert.match(fm, /^effort:\s*high$/m,
|
||||
`gsd-executor frontmatter should have effort: high\nActual:\n${fm}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── describe 2: Gemini install does NOT inject effort: ──────────────────────
|
||||
|
||||
describe('#443 Gemini install: effort: absent (Gemini-safe)', () => {
|
||||
let tmpDir;
|
||||
let geminiHome;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpDir = makeTmpDir('gsd-443-gemini-');
|
||||
geminiHome = path.join(tmpDir, 'gemini-home');
|
||||
fs.mkdirSync(geminiHome, { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('gsd-planner.md does NOT contain effort: (Gemini install)', () => {
|
||||
runGlobalInstall('gemini', geminiHome);
|
||||
const fm = readFrontmatter(path.join(geminiHome, 'agents', 'gsd-planner.md'));
|
||||
assert.doesNotMatch(fm, /^effort:/m,
|
||||
`gsd-planner (Gemini) frontmatter must NOT have effort:\nActual:\n${fm}`);
|
||||
});
|
||||
|
||||
test('gsd-executor.md does NOT contain effort: (Gemini install)', () => {
|
||||
runGlobalInstall('gemini', geminiHome);
|
||||
const fm = readFrontmatter(path.join(geminiHome, 'agents', 'gsd-executor.md'));
|
||||
assert.doesNotMatch(fm, /^effort:/m,
|
||||
`gsd-executor (Gemini) frontmatter must NOT have effort:\nActual:\n${fm}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── describe 3: Codex install emits model_reasoning_effort in .toml ─────────
|
||||
|
||||
describe('#443 Codex install: model_reasoning_effort in .toml (unified resolver)', () => {
|
||||
let tmpDir;
|
||||
let codexHome;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpDir = makeTmpDir('gsd-443-codex-');
|
||||
codexHome = path.join(tmpDir, 'codex-home');
|
||||
fs.mkdirSync(codexHome, { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('gsd-planner.toml contains model_reasoning_effort = "xhigh" (heavy tier)', () => {
|
||||
runGlobalInstall('codex', codexHome);
|
||||
const tomlContent = fs.readFileSync(
|
||||
path.join(codexHome, 'agents', 'gsd-planner.toml'), 'utf8'
|
||||
);
|
||||
assert.match(tomlContent, /^model_reasoning_effort\s*=\s*"xhigh"$/m,
|
||||
`gsd-planner.toml should have model_reasoning_effort = "xhigh"\nActual:\n${tomlContent.slice(0, 500)}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── describe 4: Config-driven proof ─────────────────────────────────────────
|
||||
//
|
||||
// The runtime home dir must be INSIDE (or a sibling of) the project root so
|
||||
// that readGsdEffectiveEffortConfig(targetDir) can walk up from the runtime
|
||||
// home and find .planning/config.json. We put .claude/ and .codex/ as siblings
|
||||
// of .planning/ inside the project dir — this is the natural local-install shape.
|
||||
|
||||
describe('#443 Config-driven: effort.agent_overrides drives install-time effort', () => {
|
||||
let tmpDir;
|
||||
let claudeHome;
|
||||
let codexHome;
|
||||
|
||||
beforeEach(() => {
|
||||
// Layout: tmpDir/project/ <-- project root (cwd for install)
|
||||
// .planning/config.json
|
||||
// .claude/ <-- claudeHome (CLAUDE_CONFIG_DIR)
|
||||
// .codex/ <-- codexHome (CODEX_HOME)
|
||||
tmpDir = makeTmpDir('gsd-443-cfg-');
|
||||
const projectDir = path.join(tmpDir, 'project');
|
||||
claudeHome = path.join(projectDir, '.claude');
|
||||
codexHome = path.join(projectDir, '.codex');
|
||||
|
||||
fs.mkdirSync(claudeHome, { recursive: true });
|
||||
fs.mkdirSync(codexHome, { recursive: true });
|
||||
fs.mkdirSync(path.join(projectDir, '.planning'), { recursive: true });
|
||||
|
||||
// Write a project config with effort.agent_overrides overriding gsd-planner to 'low'
|
||||
const config = {
|
||||
effort: {
|
||||
agent_overrides: {
|
||||
'gsd-planner': 'low',
|
||||
},
|
||||
},
|
||||
};
|
||||
fs.writeFileSync(
|
||||
path.join(projectDir, '.planning', 'config.json'),
|
||||
JSON.stringify(config, null, 2)
|
||||
);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('Claude .md gets effort: low when agent_overrides.gsd-planner=low', () => {
|
||||
// projectDir is the cwd for install — chdir handled inside runGlobalInstall.
|
||||
// claudeHome is inside projectDir, so walking up from claudeHome finds .planning/config.json.
|
||||
runGlobalInstall('claude', claudeHome);
|
||||
const fm = readFrontmatter(path.join(claudeHome, 'agents', 'gsd-planner.md'));
|
||||
assert.match(fm, /^effort:\s*low$/m,
|
||||
`gsd-planner should have effort: low from config override\nActual:\n${fm}`);
|
||||
});
|
||||
|
||||
test('Codex .toml gets model_reasoning_effort = "low" when agent_overrides.gsd-planner=low', () => {
|
||||
runGlobalInstall('codex', codexHome);
|
||||
const tomlContent = fs.readFileSync(
|
||||
path.join(codexHome, 'agents', 'gsd-planner.toml'), 'utf8'
|
||||
);
|
||||
assert.match(tomlContent, /^model_reasoning_effort\s*=\s*"low"$/m,
|
||||
`gsd-planner.toml should have model_reasoning_effort = "low" from config override\nActual:\n${tomlContent.slice(0, 500)}`);
|
||||
});
|
||||
|
||||
test('Codex .toml clamps effort max → xhigh when agent_overrides.gsd-planner=max', () => {
|
||||
const projectDir = path.dirname(codexHome);
|
||||
// Overwrite config with max override
|
||||
const config = {
|
||||
effort: {
|
||||
agent_overrides: {
|
||||
'gsd-planner': 'max',
|
||||
},
|
||||
},
|
||||
};
|
||||
fs.writeFileSync(
|
||||
path.join(projectDir, '.planning', 'config.json'),
|
||||
JSON.stringify(config, null, 2)
|
||||
);
|
||||
|
||||
runGlobalInstall('codex', codexHome);
|
||||
const tomlContent = fs.readFileSync(
|
||||
path.join(codexHome, 'agents', 'gsd-planner.toml'), 'utf8'
|
||||
);
|
||||
// Codex does not support 'max' → clamped to 'xhigh'
|
||||
assert.match(tomlContent, /^model_reasoning_effort\s*=\s*"xhigh"$/m,
|
||||
`gsd-planner.toml should clamp max → xhigh for Codex\nActual:\n${tomlContent.slice(0, 500)}`);
|
||||
assert.doesNotMatch(tomlContent, /model_reasoning_effort\s*=\s*"max"/,
|
||||
'Codex .toml must never contain model_reasoning_effort = "max"');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── describe 5b: Invalid effort tokens fall through (Codex adversarial finding #2) ─
|
||||
//
|
||||
// These tests FAIL before the fix: resolveInstallTimeEffort returns the raw
|
||||
// invalid string without validating it against VALID_EFFORTS.
|
||||
|
||||
describe('#443 resolveInstallTimeEffort: invalid tokens fall through to valid effort', () => {
|
||||
let tmpDir;
|
||||
let claudeHome;
|
||||
let codexHome;
|
||||
|
||||
beforeEach(() => {
|
||||
// Layout: tmpDir/project/ <-- project root
|
||||
// .planning/config.json
|
||||
// .claude/ <-- claudeHome
|
||||
// .codex/ <-- codexHome
|
||||
tmpDir = makeTmpDir('gsd-443-invalid-effort-');
|
||||
const projectDir = path.join(tmpDir, 'project');
|
||||
claudeHome = path.join(projectDir, '.claude');
|
||||
codexHome = path.join(projectDir, '.codex');
|
||||
|
||||
fs.mkdirSync(claudeHome, { recursive: true });
|
||||
fs.mkdirSync(codexHome, { recursive: true });
|
||||
fs.mkdirSync(path.join(projectDir, '.planning'), { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function writeProjectConfig(config) {
|
||||
const projectDir = path.dirname(claudeHome);
|
||||
fs.writeFileSync(
|
||||
path.join(projectDir, '.planning', 'config.json'),
|
||||
JSON.stringify(config, null, 2)
|
||||
);
|
||||
}
|
||||
|
||||
const VALID_EFFORTS = ['minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
||||
|
||||
test('effort.default="ultra" (invalid) -> Claude .md effort: is a VALID value (falls through to high)', () => {
|
||||
// BUG before fix: resolveInstallTimeEffort returns "ultra" verbatim
|
||||
writeProjectConfig({ effort: { default: 'ultra' } });
|
||||
runGlobalInstall('claude', claudeHome);
|
||||
const fm = readFrontmatter(path.join(claudeHome, 'agents', 'gsd-planner.md'));
|
||||
const match = fm.match(/^effort:\s*(\S+)$/m);
|
||||
assert.ok(match, `effort: must be present in frontmatter\nActual:\n${fm}`);
|
||||
assert.ok(VALID_EFFORTS.includes(match[1]),
|
||||
`effort: must be a VALID effort string, got: "${match[1]}"\nActual frontmatter:\n${fm}`);
|
||||
});
|
||||
|
||||
test('effort.agent_overrides.gsd-planner="bogus" (invalid) with valid default -> falls through to valid default', () => {
|
||||
// BUG before fix: "bogus" is returned and written verbatim
|
||||
writeProjectConfig({
|
||||
effort: {
|
||||
agent_overrides: { 'gsd-planner': 'bogus' },
|
||||
default: 'medium',
|
||||
},
|
||||
});
|
||||
runGlobalInstall('claude', claudeHome);
|
||||
const fm = readFrontmatter(path.join(claudeHome, 'agents', 'gsd-planner.md'));
|
||||
const match = fm.match(/^effort:\s*(\S+)$/m);
|
||||
assert.ok(match, `effort: must be present in frontmatter\nActual:\n${fm}`);
|
||||
assert.ok(VALID_EFFORTS.includes(match[1]),
|
||||
`effort: must be a VALID effort string, got: "${match[1]}"\nActual frontmatter:\n${fm}`);
|
||||
// Falls through invalid "bogus" -> valid tier default or "medium" default
|
||||
// "medium" is valid, so it should appear (or tier default if medium is invalid, but medium is valid)
|
||||
});
|
||||
|
||||
test('effort.default="ultra" (invalid) -> Codex .toml model_reasoning_effort is VALID', () => {
|
||||
// BUG before fix: "ultra" written into .toml verbatim
|
||||
writeProjectConfig({ effort: { default: 'ultra' } });
|
||||
runGlobalInstall('codex', codexHome);
|
||||
const tomlContent = fs.readFileSync(
|
||||
path.join(codexHome, 'agents', 'gsd-planner.toml'), 'utf8'
|
||||
);
|
||||
const match = tomlContent.match(/^model_reasoning_effort\s*=\s*"([^"]+)"/m);
|
||||
assert.ok(match, `model_reasoning_effort must be present in .toml\nActual:\n${tomlContent.slice(0, 500)}`);
|
||||
assert.ok(VALID_EFFORTS.includes(match[1]),
|
||||
`model_reasoning_effort must be VALID, got: "${match[1]}"\nActual:\n${tomlContent.slice(0, 500)}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── describe 5: Source stays clean ──────────────────────────────────────────
|
||||
|
||||
describe('#443 Source purity: agents/gsd-planner.md has no effort: key', () => {
|
||||
test('source agents/gsd-planner.md frontmatter does not contain effort:', () => {
|
||||
const fm = readFrontmatter(path.join(SOURCE_AGENTS_DIR, 'gsd-planner.md'));
|
||||
assert.doesNotMatch(fm, /^effort:/m,
|
||||
`Source agents/gsd-planner.md must NOT contain effort: (injection is install-only)`);
|
||||
});
|
||||
|
||||
test('source agents/gsd-executor.md frontmatter does not contain effort:', () => {
|
||||
const fm = readFrontmatter(path.join(SOURCE_AGENTS_DIR, 'gsd-executor.md'));
|
||||
assert.doesNotMatch(fm, /^effort:/m,
|
||||
`Source agents/gsd-executor.md must NOT contain effort: (injection is install-only)`);
|
||||
});
|
||||
|
||||
test('source agents/gsd-codebase-mapper.md frontmatter does not contain effort:', () => {
|
||||
const fm = readFrontmatter(path.join(SOURCE_AGENTS_DIR, 'gsd-codebase-mapper.md'));
|
||||
assert.doesNotMatch(fm, /^effort:/m,
|
||||
`Source agents/gsd-codebase-mapper.md must NOT contain effort: (injection is install-only)`);
|
||||
});
|
||||
});
|
||||
@@ -34,12 +34,13 @@ const { createTempProject, cleanup } = require('./helpers.cjs');
|
||||
|
||||
const {
|
||||
resolveModelInternal,
|
||||
resolveReasoningEffortInternal,
|
||||
resolveEffortInternal,
|
||||
resolveTierEntry,
|
||||
RUNTIME_PROFILE_MAP,
|
||||
KNOWN_RUNTIMES,
|
||||
_resetRuntimeWarningCacheForTests,
|
||||
} = require('../get-shit-done/bin/lib/core.cjs');
|
||||
const { renderEffortForRuntime } = require('../get-shit-done/bin/lib/model-catalog.cjs');
|
||||
const { isValidConfigKey } = require('../get-shit-done/bin/lib/config-schema.cjs');
|
||||
|
||||
function writeConfig(tmpDir, obj) {
|
||||
@@ -98,9 +99,12 @@ describe('issue #2517: backwards compat — no runtime key set', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), '');
|
||||
});
|
||||
|
||||
test('reasoning_effort returns null when runtime absent', () => {
|
||||
test('effort resolves universally but render param is null when runtime absent', () => {
|
||||
writeConfig(tmpDir, { model_profile: 'balanced' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
// Effort always resolves (universal); rendering without a runtime yields no wire param.
|
||||
const rendered = renderEffortForRuntime(undefined, eff);
|
||||
assert.strictEqual(rendered.param, null);
|
||||
});
|
||||
|
||||
test('adaptive profile still works without runtime (#1713/#1806)', () => {
|
||||
@@ -144,9 +148,14 @@ describe('issue #2517: runtime "claude" is a no-op for resolution (finding #4)',
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'claude-opus-4-7');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null on Claude (never leaks)', () => {
|
||||
test('effort is first-class on Claude (emits output_config.effort)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'claude', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
// Under unification, Claude effort is first-class — rendered via output_config.effort.
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('claude', eff);
|
||||
assert.strictEqual(rendered.param, 'output_config.effort');
|
||||
// gsd-planner is heavy tier → default effort 'xhigh'
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -156,23 +165,35 @@ describe('issue #2517: runtime "codex" — Codex tier resolution', () => {
|
||||
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
||||
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
||||
|
||||
test('opus tier -> gpt-5.4 with reasoning_effort xhigh', () => {
|
||||
test('opus tier -> gpt-5.4 model; heavy-tier agent -> xhigh effort on codex', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'quality' });
|
||||
// gsd-planner quality -> opus -> gpt-5.4
|
||||
// gsd-planner quality -> opus -> gpt-5.4 (model unchanged)
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
||||
// gsd-planner is heavy routing tier → effort 'xhigh' → rendered model_reasoning_effort
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('sonnet tier -> gpt-5.3-codex with reasoning_effort medium', () => {
|
||||
test('sonnet tier -> gpt-5.3-codex model; heavy-tier agent -> xhigh effort on codex', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'balanced' });
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-roadmapper'), 'gpt-5.3-codex');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-roadmapper'), 'medium');
|
||||
// gsd-roadmapper is heavy routing tier → effort 'xhigh' (not catalog medium)
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-roadmapper');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('haiku tier -> gpt-5.4-mini with reasoning_effort medium', () => {
|
||||
test('haiku tier -> gpt-5.4-mini model; light-tier agent -> low effort on codex', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'budget' });
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gpt-5.4-mini');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-codebase-mapper'), 'medium');
|
||||
// gsd-codebase-mapper is light routing tier → effort 'low' (not catalog medium)
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-codebase-mapper');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'low');
|
||||
});
|
||||
|
||||
test('adaptive profile resolves on Codex (no #1713/#1806 regression)', () => {
|
||||
@@ -183,11 +204,15 @@ describe('issue #2517: runtime "codex" — Codex tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gpt-5.4-mini');
|
||||
});
|
||||
|
||||
test('inherit profile still returns "inherit" on Codex', () => {
|
||||
test('inherit profile still returns "inherit" on Codex; effort still resolves universally', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'inherit' });
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'inherit');
|
||||
// No reasoning_effort when inherit
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
// Unified effort is config-driven (routing_tier_defaults), independent of model_profile.
|
||||
// gsd-planner (heavy tier) → 'xhigh'; rendered to codex param.
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('runtime:"codex" beats resolve_model_ids:"omit" (explicit non-Claude opt-in wins)', () => {
|
||||
@@ -261,33 +286,39 @@ describe('issue #2517: field-merge of overrides with built-in defaults (finding
|
||||
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
||||
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
||||
|
||||
test('string-shorthand override keeps reasoning_effort from built-in (CONFIGURATION.md example)', () => {
|
||||
// `{ codex: { opus: "gpt-5-pro" } }` is the documented shorthand. Pre-fix,
|
||||
// it silently dropped reasoning_effort. Post-fix, the model is overridden
|
||||
// and reasoning_effort comes from the built-in entry.
|
||||
test('string-shorthand override: model is overridden; unified effort derives from routing tier', () => {
|
||||
// `{ codex: { opus: "gpt-5-pro" } }` is the documented shorthand.
|
||||
// Model is overridden to gpt-5-pro; effort now derives from the universal
|
||||
// config-driven path (gsd-planner heavy tier → 'xhigh'), not from the catalog.
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'quality',
|
||||
model_profile_overrides: { codex: { opus: 'gpt-5-pro' } },
|
||||
});
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5-pro');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('partial-object override (no model) keeps model from built-in', () => {
|
||||
// `{ codex: { opus: { reasoning_effort: "low" } } }` previously dropped
|
||||
// the model entirely (returned undefined and fell through). Post-fix, the
|
||||
// built-in `gpt-5.4` model is preserved and `low` reasoning_effort wins.
|
||||
test('partial-object override (no model) keeps model from built-in; unified effort from routing tier', () => {
|
||||
// `{ codex: { opus: { reasoning_effort: "low" } } }` preserves the built-in model.
|
||||
// Under unification, the catalog reasoning_effort field is not read for effort resolution;
|
||||
// effort comes from routing_tier_defaults (gsd-planner heavy → 'xhigh').
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'quality',
|
||||
model_profile_overrides: { codex: { opus: { reasoning_effort: 'low' } } },
|
||||
});
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'low');
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('full-object override replaces both fields', () => {
|
||||
test('full-object override: model replaced; unified effort from routing tier (not catalog field)', () => {
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'quality',
|
||||
@@ -296,7 +327,11 @@ describe('issue #2517: field-merge of overrides with built-in defaults (finding
|
||||
},
|
||||
});
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'custom-model');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'minimal');
|
||||
// Effort comes from routing_tier_defaults, not the catalog 'minimal' field.
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('resolveTierEntry helper: shorthand merge', () => {
|
||||
@@ -328,15 +363,15 @@ describe('issue #2517: field-merge of overrides with built-in defaults (finding
|
||||
});
|
||||
});
|
||||
|
||||
// ─── reasoning_effort allowlist (review finding #3) ─────────────────────────
|
||||
describe('issue #2517: reasoning_effort allowlist gates regardless of overrides (finding #3)', () => {
|
||||
// ─── Unknown runtime render safety (finding #3 spirit) ──────────────────────
|
||||
describe('issue #2517: unknown runtime render param is null (effort does not leak to install path)', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
||||
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
||||
|
||||
test('unknown runtime with overrides supplying reasoning_effort yields null effort', () => {
|
||||
// Pre-fix: `if (!overrides) return null` left a hole — overrides for an
|
||||
// unknown runtime made effort propagate, defeating the typo guard.
|
||||
test('unknown runtime: model resolves via override; render param is null (no wire param leaked)', () => {
|
||||
// Under unification, effort always resolves (universal), but renderEffortForRuntime
|
||||
// returns param=null for unknown runtimes — no effort leaks to the install path.
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'mystery',
|
||||
model_profile: 'quality',
|
||||
@@ -346,17 +381,21 @@ describe('issue #2517: reasoning_effort allowlist gates regardless of overrides
|
||||
});
|
||||
// Model still resolves (overrides are honored).
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'mystery-opus');
|
||||
// …but reasoning_effort does NOT propagate to a runtime not in the allowlist.
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
// Effort resolves universally but the unknown runtime has no wire param.
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('mystery', eff);
|
||||
assert.strictEqual(rendered.param, null);
|
||||
});
|
||||
|
||||
test('typo runtime "codx" with overrides yields null effort (no leak into install path)', () => {
|
||||
test('typo runtime "codx": render param is null (no leak into install path)', () => {
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codx',
|
||||
model_profile: 'quality',
|
||||
model_profile_overrides: { codx: { opus: { model: 'gpt-5.4', reasoning_effort: 'xhigh' } } },
|
||||
});
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codx', eff);
|
||||
assert.strictEqual(rendered.param, null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -541,10 +580,13 @@ describe('issue #2517: install end-to-end — per-project config reaches Codex T
|
||||
assert.match(toml, /^model_reasoning_effort = "xhigh"$/m);
|
||||
});
|
||||
|
||||
test('generated TOML omits reasoning_effort when runtime has none', () => {
|
||||
// For a known runtime with model but no reasoning_effort, only model is emitted.
|
||||
// Use the user-override path to simulate this with codex (no built-in returns
|
||||
// model alone, so fabricate via override of an unknown-runtime entry).
|
||||
test('generated TOML always includes model_reasoning_effort even when model_profile_overrides sets reasoning_effort to empty (#443 unified)', () => {
|
||||
// Under the unified effort design (#443), model_reasoning_effort in the Codex TOML
|
||||
// is driven by the unified effort resolver (resolveInstallTimeEffort / effortCfg),
|
||||
// NOT by model_profile_overrides.reasoning_effort. Setting reasoning_effort: '' in
|
||||
// model_profile_overrides does NOT suppress the unified effort — the TOML always
|
||||
// carries a valid model_reasoning_effort drawn from the agent's routing tier.
|
||||
// gsd-planner is a heavy-tier agent → unified default resolves to "xhigh".
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'quality',
|
||||
@@ -557,8 +599,12 @@ describe('issue #2517: install end-to-end — per-project config reaches Codex T
|
||||
null,
|
||||
resolver
|
||||
);
|
||||
// Model override (from model_profile_overrides) is still respected.
|
||||
assert.match(toml, /^model = "custom"$/m);
|
||||
assert.doesNotMatch(toml, /model_reasoning_effort/);
|
||||
// Unified effort always fires — model_reasoning_effort is present and valid.
|
||||
assert.match(toml, /^model_reasoning_effort = "(minimal|low|medium|high|xhigh)"$/m);
|
||||
// gsd-planner is heavy-tier, so with no effortCfg the manifest tier default applies → xhigh.
|
||||
assert.match(toml, /^model_reasoning_effort = "xhigh"$/m);
|
||||
});
|
||||
|
||||
test('resolver returns null with no global, no per-project config', () => {
|
||||
@@ -612,9 +658,10 @@ describe('issue #2612: runtime "gemini" — Gemini tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gemini-2.5-flash-lite');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null for gemini (no reasoning_effort in spec)', () => {
|
||||
test('gemini: effort resolves universally but render param is null (no wire param)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'gemini', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(renderEffortForRuntime('gemini', eff).param, null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -639,9 +686,10 @@ describe('issue #2612: runtime "qwen" — Qwen tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'qwen3-coder-next');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null for qwen (no reasoning_effort in spec)', () => {
|
||||
test('qwen: effort resolves universally but render param is null (no wire param)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'qwen', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(renderEffortForRuntime('qwen', eff).param, null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -666,9 +714,10 @@ describe('issue #2612: runtime "opencode" — OpenCode tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'anthropic/claude-haiku-4-5');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null for opencode (no reasoning_effort in spec)', () => {
|
||||
test('opencode: effort resolves universally but render param is null (no wire param)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'opencode', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(renderEffortForRuntime('opencode', eff).param, null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -693,9 +742,10 @@ describe('issue #2612: runtime "copilot" — Copilot tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'claude-haiku-4-5');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null for copilot (no reasoning_effort in spec)', () => {
|
||||
test('copilot: effort resolves universally but render param is null (no wire param)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'copilot', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(renderEffortForRuntime('copilot', eff).param, null);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user