* test(#443): RED unified effort + fast_mode + resolve-execution All 68 tests failing as expected — no implementation yet. Covers: effort cascade (tier defaults, overrides, invalid fallthrough), fast_mode cascade (boolean-only, tier defaults), resolveEffortForTier escalation, renderEffortForRuntime clamping, resolve-execution CLI, config schema new keys, QA hostile-input matrix. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(#443): unified cross-provider effort + fast_mode knobs and resolve-execution query Adds config-driven effort control (universal ladder: minimal<low<medium<high<xhigh<max) and fast_mode propagation knobs, with per-runtime rendering that clamps the unique tail values (max=Anthropic-only clamps to xhigh on Codex; minimal=Codex-only clamps to low on Claude). Key changes: - config-schema.manifest.json: add effort.default, fast_mode.enabled as validKeys; add 4 dynamicKeyPatterns for effort.routing_tier_defaults, effort.agent_overrides, fast_mode.routing_tier_defaults, fast_mode.agent_overrides; fix stale _comment - config-defaults.manifest.json: add effort and fast_mode blocks with tier defaults - model-catalog.cjs: add EFFORT_RENDERING map, renderEffortForRuntime(), RUNTIMES_WITH_FAST_MODE - model-profiles.cjs: re-export new catalog exports - core.cjs: add resolveEffortInternal, resolveFastModeInternal, resolveEffortForTier, VALID_EFFORTS, EFFORT_SET, nextEffort; pass effort/fast_mode through loadConfig - commands.cjs: replace reasoning_effort in cmdResolveModel with unified effort; add cmdResolveExecution (superset command with effort_rendered, effort_param, effort_propagation, fast_mode, fast_mode_supported) - gsd-tools.cjs: add resolve-execution case with --effort/--fast-mode/--attempt flags - tests/feat-443: 69 tests covering cascade, rendering, escalation, CLI, schema, QA matrix - tests/commands.test.cjs: convert 3 reasoning_effort assertions to unified effort - docs/CONFIGURATION.md: document effort + fast_mode + resolve-execution sections - settings-advanced.md: list new effort/fast_mode keys in confirmation table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#443): remove dead catalog effort lane; unify codex effort through renderEffortForRuntime - Remove resolveReasoningEffortInternal (catalog-driven effort function) from core.cjs and its export; remove from commands.cjs destructure import - Convert tests/issue-2517-runtime-aware-profiles.test.cjs: all 11 effort assertions now use resolveEffortInternal + renderEffortForRuntime; Claude effort is first-class (output_config.effort); unknown runtimes assert param===null - Convert tests/feat-3023-model-phase-types.test.cjs: replace the entire resolveReasoningEffortInternal describe with unified effort assertions; effort derives from AGENT_DEFAULT_TIERS routing tier, not phase-type tier Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(#443): ADR for unified cross-provider effort + fast-mode routing Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(#443): architecture-level QA invariants + test-strategy doc Add 48-test integration suite (feat-443-effort-fast-mode.integration.test.cjs) covering 8 architectural invariants: cross-provider validity (never emit a value the real API would 400 on), param/channel contract stability, resolve-execution JSON contract (all 8 keys + correct types), totality across the full 33-agent registry, fast-mode honesty (claude always fast_mode_supported=false), precedence first-valid-wins matrix for both effort and fast_mode cascades, dynamic-routing composition (effort escalation independent of model tier), and config-set round-trip for all new effort/* and fast_mode/* key namespaces. Append test-strategy section with invariant rationale and E2E gap documentation to docs/TESTING-SUITES.md. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(#443): add failing install-wiring tests for effort per-runtime injection (RED) TDD RED: 10 failing tests covering: - Claude .md gets effort: injected per tier (planner=xhigh, mapper=low, executor=high) - Gemini .md does NOT get effort: (already passing — Gemini-safe) - Codex .toml gets model_reasoning_effort via unified resolver - Config-driven: effort.agent_overrides drives both Claude .md and Codex .toml - Source purity: agents/*.md have no effort: key (already passing) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(#443): wire effort per-runtime at install (Claude .md frontmatter + Codex .toml unified) - Import AGENT_DEFAULT_TIERS and renderEffortForRuntime from model-catalog.cjs - Add readGsdEffectiveEffortConfig(targetDir): reads merged effort config from .planning/config.json (per-project wins) + ~/.gsd/defaults.json (global fallback), same probe pattern as readGsdRuntimeProfileResolver - Add resolveInstallTimeEffort(effortCfg, agentName): pure function matching resolveEffortInternal() precedence (agent_overrides > routing_tier_defaults > default > 'high') without loadConfig side-effects (no sub-repo detection, no migration writes) - Claude agent copy loop: inject `effort: <value>` into frontmatter ONLY for runtime === 'claude'; all other .md runtimes (Gemini, Qwen, Hermes, etc.) stay effort-free (Gemini-safe source contract preserved in agents/*.md) - generateCodexAgentToml: add effortCfg param; emit model_reasoning_effort from unified resolver (replaces old catalog entry.reasoning_effort); Codex clamps max → xhigh via renderEffortForRuntime('codex', ...) - installCodexConfig: pass readGsdEffectiveEffortConfig(targetDir) to generateCodexAgentToml so per-project config wins for Codex .toml too - Update failing tests to GREEN: 12/12 pass; all 17 install tests pass; 2847/2848 unit tests pass (1 pre-existing failure: policy-shell-pinning) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(#443): source install effort defaults from manifest (kill drift) + guard test Replace hardcoded _GSD_EFFORT_MANIFEST_TIER_DEFAULTS and the 'high' fallback in resolveInstallTimeEffort with values read from config-defaults.manifest.json at module init, using the same __dirname-relative path install.js already uses for all shared manifests. Add feat-443-effort-defaults-drift.test.cjs to assert equality between install.js's runtime constants and the manifest on every CI run. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): reconcile Codex TOML tests with unified effort design The #443 unified effort resolver makes generateCodexAgentToml always emit model_reasoning_effort (driven by resolveInstallTimeEffort, not model_profile_overrides). The test 'generated TOML omits reasoning_effort when runtime has none' had an obsolete premise — model_profile_overrides.reasoning_effort:'' no longer suppresses unified effort. Convert it to assert the new invariant: Codex TOML always carries a valid model_reasoning_effort from the agent's routing tier (xhigh for gsd-planner, a heavy-tier agent), while model_profile_overrides model override is still respected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): make install.js effort resolution lazy (no load-time side effects breaking launcher-parity) Replace module-load-time IIFE + hard throw (config-defaults.manifest.json read) and top-level require of model-catalog.cjs with a lazy _getGsdEffortCatalog() getter that initialises on first call from resolveInstallTimeEffort / generateCodexAgentToml / Claude .md effort injection. Requiring install.js in unrelated test contexts (e.g. runtime-launcher-parity) no longer triggers manifest IO or throws, eliminating the load-time side effect that changed subprocess exit codes / stderr on the bench. Drift-guard exports (_GSD_EFFORT_MANIFEST_TIER_DEFAULTS / _GSD_EFFORT_MANIFEST_DEFAULT) preserved as lazy getter properties on module.exports so feat-443-effort-defaults-drift still validates them without forcing eager load. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): isolate install-wiring test HOME to stop \$HOME/.claude pollution breaking launcher-parity runGlobalInstall() now redirects HOME to a per-call isolated tmpdir in addition to the existing runtime-specific env-var redirects (CLAUDE_CONFIG_DIR, GEMINI_CONFIG_DIR, CODEX_HOME). This ensures install.js code that uses os.homedir() directly — including the ~/.cache/gsd update-check deletion, ~/.gsd/defaults.json reads, and any HOME-relative npm subprocess writes — never touches the real \$HOME during the test. Without the HOME isolation the install test (which is new to this branch and is now picked up by Docker's raw \`tests/*.test.cjs\` glob) could write or delete files under the real \$HOME, causing runtime-launcher-parity test (D) to fail: (D) asserts a loud non-zero exit when \$RUNTIME_DIR/gsd-tools.cjs is absent and gsd-tools is not on PATH, but the launcher's \$HOME/.claude fallback arm succeeds if \$HOME/.claude/get-shit-done/bin/gsd-tools.cjs exists. Also sets GSD_SKIP_STALE_SDK_CHECK=1 to suppress the \`npm ls -g\` subprocess that the global installer spawns — irrelevant to effort-wiring assertions, slow, and potentially writes to ~/.npm cache. All 12 feat-443 install-wiring assertions preserved. Drift-guard 5/5. Unit suite 2848/2850 (pre-existing policy-shell-pinning.test.cjs failure on next). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(#443): add changeset fragment for effort + fast-mode routing Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(#443): set GSD_TEST_MODE before requiring install.js in drift-guard test to prevent HOME leak Without GSD_TEST_MODE=1, require('bin/install.js') runs the module's main install block (guarded by !GSD_TEST_MODE), performing a real global Claude install into $HOME/.claude/. On CI ubuntu where node is on standard PATH, the launcher's $HOME/.claude fallback arm then finds gsd-tools.cjs, causing runtime-launcher-parity test (D) to exit zero when it must exit non-zero. Root cause: feat-443-effort-defaults-drift.test.cjs (unit suite) runs alphabetically before runtime-launcher-parity.test.cjs in the same node --test invocation. Each runs in a separate worker process but shares the same HOME. The drift test's install leaks gsd-tools.cjs into that HOME, then the launcher test's bash subprocess finds it via the $HOME/.claude arm. Fix: add process.env.GSD_TEST_MODE = '1' at the top of the drift-guard test, before the require(installPath) call. This matches the pattern used by feat-443-effort-fast-mode.test.cjs and feat-443-effort-install-wiring .install.test.cjs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): deterministic resolve-execution arg parsing + validate install-time effort (Codex adversarial findings) Finding 1: resolve-execution --effort low gsd-planner misrouted 'low' as the agent. Replace find(non-dash) with a proper flag-consuming loop that collects a single positional; validate missing/extra positionals and malformed --attempt values. Finding 2: resolveInstallTimeEffort returned unvalidated effort strings (e.g. "ultra") verbatim. Each precedence layer now checks GSD_EFFORT_SET (imported once from core.cjs) before accepting a value, mirroring resolveEffortInternal exactly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#443): newline-agnostic effort frontmatter injection (Windows CRLF) + CRLF-safe assertions Extracts injectEffortFrontmatter(content, effortValue) pure helper that detects EOL (LF vs CRLF) from the opening '---' line and inserts 'effort: <value>' before the closing '---' delimiter using the same EOL as the surrounding frontmatter. Regex now uses /^---\r?\n([\s\S]*?)^---\r?$/m instead of the LF-only /^(---\n[\s\S]*?)(---)(\n|$)/ that silently skipped CRLF files on Windows (git core.autocrlf=true checkout). Also adds 7 unit tests covering LF, CRLF, idempotency, no-frontmatter, and complex frontmatter cases. Exports injectEffortFrontmatter from module.exports. Fixes 6 CI failures in tests/feat-443-effort-install-wiring.install.test.cjs on windows-latest runners (lines 138, 145, 152, 261, 345, 356). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: CI Rebase Check <ci@gsd-redux> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
@@ -34,12 +34,13 @@ const { createTempProject, cleanup } = require('./helpers.cjs');
|
||||
|
||||
const {
|
||||
resolveModelInternal,
|
||||
resolveReasoningEffortInternal,
|
||||
resolveEffortInternal,
|
||||
resolveTierEntry,
|
||||
RUNTIME_PROFILE_MAP,
|
||||
KNOWN_RUNTIMES,
|
||||
_resetRuntimeWarningCacheForTests,
|
||||
} = require('../get-shit-done/bin/lib/core.cjs');
|
||||
const { renderEffortForRuntime } = require('../get-shit-done/bin/lib/model-catalog.cjs');
|
||||
const { isValidConfigKey } = require('../get-shit-done/bin/lib/config-schema.cjs');
|
||||
|
||||
function writeConfig(tmpDir, obj) {
|
||||
@@ -98,9 +99,12 @@ describe('issue #2517: backwards compat — no runtime key set', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), '');
|
||||
});
|
||||
|
||||
test('reasoning_effort returns null when runtime absent', () => {
|
||||
test('effort resolves universally but render param is null when runtime absent', () => {
|
||||
writeConfig(tmpDir, { model_profile: 'balanced' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
// Effort always resolves (universal); rendering without a runtime yields no wire param.
|
||||
const rendered = renderEffortForRuntime(undefined, eff);
|
||||
assert.strictEqual(rendered.param, null);
|
||||
});
|
||||
|
||||
test('adaptive profile still works without runtime (#1713/#1806)', () => {
|
||||
@@ -144,9 +148,14 @@ describe('issue #2517: runtime "claude" is a no-op for resolution (finding #4)',
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'claude-opus-4-7');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null on Claude (never leaks)', () => {
|
||||
test('effort is first-class on Claude (emits output_config.effort)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'claude', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
// Under unification, Claude effort is first-class — rendered via output_config.effort.
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('claude', eff);
|
||||
assert.strictEqual(rendered.param, 'output_config.effort');
|
||||
// gsd-planner is heavy tier → default effort 'xhigh'
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -156,23 +165,35 @@ describe('issue #2517: runtime "codex" — Codex tier resolution', () => {
|
||||
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
||||
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
||||
|
||||
test('opus tier -> gpt-5.4 with reasoning_effort xhigh', () => {
|
||||
test('opus tier -> gpt-5.4 model; heavy-tier agent -> xhigh effort on codex', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'quality' });
|
||||
// gsd-planner quality -> opus -> gpt-5.4
|
||||
// gsd-planner quality -> opus -> gpt-5.4 (model unchanged)
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
||||
// gsd-planner is heavy routing tier → effort 'xhigh' → rendered model_reasoning_effort
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('sonnet tier -> gpt-5.3-codex with reasoning_effort medium', () => {
|
||||
test('sonnet tier -> gpt-5.3-codex model; heavy-tier agent -> xhigh effort on codex', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'balanced' });
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-roadmapper'), 'gpt-5.3-codex');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-roadmapper'), 'medium');
|
||||
// gsd-roadmapper is heavy routing tier → effort 'xhigh' (not catalog medium)
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-roadmapper');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('haiku tier -> gpt-5.4-mini with reasoning_effort medium', () => {
|
||||
test('haiku tier -> gpt-5.4-mini model; light-tier agent -> low effort on codex', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'budget' });
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gpt-5.4-mini');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-codebase-mapper'), 'medium');
|
||||
// gsd-codebase-mapper is light routing tier → effort 'low' (not catalog medium)
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-codebase-mapper');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'low');
|
||||
});
|
||||
|
||||
test('adaptive profile resolves on Codex (no #1713/#1806 regression)', () => {
|
||||
@@ -183,11 +204,15 @@ describe('issue #2517: runtime "codex" — Codex tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gpt-5.4-mini');
|
||||
});
|
||||
|
||||
test('inherit profile still returns "inherit" on Codex', () => {
|
||||
test('inherit profile still returns "inherit" on Codex; effort still resolves universally', () => {
|
||||
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'inherit' });
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'inherit');
|
||||
// No reasoning_effort when inherit
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
// Unified effort is config-driven (routing_tier_defaults), independent of model_profile.
|
||||
// gsd-planner (heavy tier) → 'xhigh'; rendered to codex param.
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('runtime:"codex" beats resolve_model_ids:"omit" (explicit non-Claude opt-in wins)', () => {
|
||||
@@ -261,33 +286,39 @@ describe('issue #2517: field-merge of overrides with built-in defaults (finding
|
||||
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
||||
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
||||
|
||||
test('string-shorthand override keeps reasoning_effort from built-in (CONFIGURATION.md example)', () => {
|
||||
// `{ codex: { opus: "gpt-5-pro" } }` is the documented shorthand. Pre-fix,
|
||||
// it silently dropped reasoning_effort. Post-fix, the model is overridden
|
||||
// and reasoning_effort comes from the built-in entry.
|
||||
test('string-shorthand override: model is overridden; unified effort derives from routing tier', () => {
|
||||
// `{ codex: { opus: "gpt-5-pro" } }` is the documented shorthand.
|
||||
// Model is overridden to gpt-5-pro; effort now derives from the universal
|
||||
// config-driven path (gsd-planner heavy tier → 'xhigh'), not from the catalog.
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'quality',
|
||||
model_profile_overrides: { codex: { opus: 'gpt-5-pro' } },
|
||||
});
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5-pro');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('partial-object override (no model) keeps model from built-in', () => {
|
||||
// `{ codex: { opus: { reasoning_effort: "low" } } }` previously dropped
|
||||
// the model entirely (returned undefined and fell through). Post-fix, the
|
||||
// built-in `gpt-5.4` model is preserved and `low` reasoning_effort wins.
|
||||
test('partial-object override (no model) keeps model from built-in; unified effort from routing tier', () => {
|
||||
// `{ codex: { opus: { reasoning_effort: "low" } } }` preserves the built-in model.
|
||||
// Under unification, the catalog reasoning_effort field is not read for effort resolution;
|
||||
// effort comes from routing_tier_defaults (gsd-planner heavy → 'xhigh').
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'quality',
|
||||
model_profile_overrides: { codex: { opus: { reasoning_effort: 'low' } } },
|
||||
});
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'low');
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('full-object override replaces both fields', () => {
|
||||
test('full-object override: model replaced; unified effort from routing tier (not catalog field)', () => {
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'quality',
|
||||
@@ -296,7 +327,11 @@ describe('issue #2517: field-merge of overrides with built-in defaults (finding
|
||||
},
|
||||
});
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'custom-model');
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'minimal');
|
||||
// Effort comes from routing_tier_defaults, not the catalog 'minimal' field.
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codex', eff);
|
||||
assert.strictEqual(rendered.param, 'model_reasoning_effort');
|
||||
assert.strictEqual(rendered.value, 'xhigh');
|
||||
});
|
||||
|
||||
test('resolveTierEntry helper: shorthand merge', () => {
|
||||
@@ -328,15 +363,15 @@ describe('issue #2517: field-merge of overrides with built-in defaults (finding
|
||||
});
|
||||
});
|
||||
|
||||
// ─── reasoning_effort allowlist (review finding #3) ─────────────────────────
|
||||
describe('issue #2517: reasoning_effort allowlist gates regardless of overrides (finding #3)', () => {
|
||||
// ─── Unknown runtime render safety (finding #3 spirit) ──────────────────────
|
||||
describe('issue #2517: unknown runtime render param is null (effort does not leak to install path)', () => {
|
||||
let tmpDir;
|
||||
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
||||
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
||||
|
||||
test('unknown runtime with overrides supplying reasoning_effort yields null effort', () => {
|
||||
// Pre-fix: `if (!overrides) return null` left a hole — overrides for an
|
||||
// unknown runtime made effort propagate, defeating the typo guard.
|
||||
test('unknown runtime: model resolves via override; render param is null (no wire param leaked)', () => {
|
||||
// Under unification, effort always resolves (universal), but renderEffortForRuntime
|
||||
// returns param=null for unknown runtimes — no effort leaks to the install path.
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'mystery',
|
||||
model_profile: 'quality',
|
||||
@@ -346,17 +381,21 @@ describe('issue #2517: reasoning_effort allowlist gates regardless of overrides
|
||||
});
|
||||
// Model still resolves (overrides are honored).
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'mystery-opus');
|
||||
// …but reasoning_effort does NOT propagate to a runtime not in the allowlist.
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
// Effort resolves universally but the unknown runtime has no wire param.
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('mystery', eff);
|
||||
assert.strictEqual(rendered.param, null);
|
||||
});
|
||||
|
||||
test('typo runtime "codx" with overrides yields null effort (no leak into install path)', () => {
|
||||
test('typo runtime "codx": render param is null (no leak into install path)', () => {
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codx',
|
||||
model_profile: 'quality',
|
||||
model_profile_overrides: { codx: { opus: { model: 'gpt-5.4', reasoning_effort: 'xhigh' } } },
|
||||
});
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
const rendered = renderEffortForRuntime('codx', eff);
|
||||
assert.strictEqual(rendered.param, null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -541,10 +580,13 @@ describe('issue #2517: install end-to-end — per-project config reaches Codex T
|
||||
assert.match(toml, /^model_reasoning_effort = "xhigh"$/m);
|
||||
});
|
||||
|
||||
test('generated TOML omits reasoning_effort when runtime has none', () => {
|
||||
// For a known runtime with model but no reasoning_effort, only model is emitted.
|
||||
// Use the user-override path to simulate this with codex (no built-in returns
|
||||
// model alone, so fabricate via override of an unknown-runtime entry).
|
||||
test('generated TOML always includes model_reasoning_effort even when model_profile_overrides sets reasoning_effort to empty (#443 unified)', () => {
|
||||
// Under the unified effort design (#443), model_reasoning_effort in the Codex TOML
|
||||
// is driven by the unified effort resolver (resolveInstallTimeEffort / effortCfg),
|
||||
// NOT by model_profile_overrides.reasoning_effort. Setting reasoning_effort: '' in
|
||||
// model_profile_overrides does NOT suppress the unified effort — the TOML always
|
||||
// carries a valid model_reasoning_effort drawn from the agent's routing tier.
|
||||
// gsd-planner is a heavy-tier agent → unified default resolves to "xhigh".
|
||||
writeConfig(tmpDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'quality',
|
||||
@@ -557,8 +599,12 @@ describe('issue #2517: install end-to-end — per-project config reaches Codex T
|
||||
null,
|
||||
resolver
|
||||
);
|
||||
// Model override (from model_profile_overrides) is still respected.
|
||||
assert.match(toml, /^model = "custom"$/m);
|
||||
assert.doesNotMatch(toml, /model_reasoning_effort/);
|
||||
// Unified effort always fires — model_reasoning_effort is present and valid.
|
||||
assert.match(toml, /^model_reasoning_effort = "(minimal|low|medium|high|xhigh)"$/m);
|
||||
// gsd-planner is heavy-tier, so with no effortCfg the manifest tier default applies → xhigh.
|
||||
assert.match(toml, /^model_reasoning_effort = "xhigh"$/m);
|
||||
});
|
||||
|
||||
test('resolver returns null with no global, no per-project config', () => {
|
||||
@@ -612,9 +658,10 @@ describe('issue #2612: runtime "gemini" — Gemini tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gemini-2.5-flash-lite');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null for gemini (no reasoning_effort in spec)', () => {
|
||||
test('gemini: effort resolves universally but render param is null (no wire param)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'gemini', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(renderEffortForRuntime('gemini', eff).param, null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -639,9 +686,10 @@ describe('issue #2612: runtime "qwen" — Qwen tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'qwen3-coder-next');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null for qwen (no reasoning_effort in spec)', () => {
|
||||
test('qwen: effort resolves universally but render param is null (no wire param)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'qwen', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(renderEffortForRuntime('qwen', eff).param, null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -666,9 +714,10 @@ describe('issue #2612: runtime "opencode" — OpenCode tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'anthropic/claude-haiku-4-5');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null for opencode (no reasoning_effort in spec)', () => {
|
||||
test('opencode: effort resolves universally but render param is null (no wire param)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'opencode', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(renderEffortForRuntime('opencode', eff).param, null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -693,9 +742,10 @@ describe('issue #2612: runtime "copilot" — Copilot tier resolution', () => {
|
||||
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'claude-haiku-4-5');
|
||||
});
|
||||
|
||||
test('reasoning_effort is null for copilot (no reasoning_effort in spec)', () => {
|
||||
test('copilot: effort resolves universally but render param is null (no wire param)', () => {
|
||||
writeConfig(tmpDir, { runtime: 'copilot', model_profile: 'quality' });
|
||||
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
||||
const eff = resolveEffortInternal(tmpDir, 'gsd-planner');
|
||||
assert.strictEqual(renderEffortForRuntime('copilot', eff).param, null);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user