diff --git a/.changeset/per-phase-type-models.md b/.changeset/per-phase-type-models.md new file mode 100644 index 000000000..096dd8e42 --- /dev/null +++ b/.changeset/per-phase-type-models.md @@ -0,0 +1,5 @@ +--- +type: Added +pr: 3030 +--- +**`models` block in `.planning/config.json` for per-phase-type model selection (#3023).** A new resolution layer between per-agent `model_overrides` and the `model_profile` tier table. Six named slots (`planning` / `discuss` / `research` / `execution` / `verification` / `completion`) accept tier aliases (`opus` / `sonnet` / `haiku` / `inherit`). Lets you express "Opus for planning, Sonnet for the rest" in two lines without learning the agent taxonomy. Fully backward compatible — configs without `models` behave exactly as today. diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index c080589bc..240524b99 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -16,6 +16,7 @@ GSD stores project settings in `.planning/config.json`. Created during `/gsd-new "granularity": "standard", "model_profile": "balanced", "model_overrides": {}, + "models": {}, "planning": { "commit_docs": true, "search_gitignored": false, @@ -117,6 +118,7 @@ GSD stores project settings in `.planning/config.json`. Created during `/gsd-new | `model_profile` | enum | `quality`, `balanced`, `budget`, `adaptive`, `inherit` | `balanced` | Model tier for each agent (see [Model Profiles](#model-profiles)). `adaptive` was added per [#1713](https://github.com/gsd-build/get-shit-done/issues/1713) / [#1806](https://github.com/gsd-build/get-shit-done/issues/1806) and resolves the same way as the other tiers under runtime-aware profiles. | | `runtime` | string | `claude`, `codex`, or any string | (none) | Active runtime for [runtime-aware profile resolution](#runtime-aware-profiles-2517). When set, profile tiers (opus/sonnet/haiku) resolve to runtime-native model IDs. Today only the Codex install path emits per-agent model IDs from this resolver; other runtimes (`opencode`, `gemini`, `qwen`, `copilot`, …) consume the resolver at spawn time and gain dedicated install-path support in [#2612](https://github.com/gsd-build/get-shit-done/issues/2612). When unset (default), behavior is unchanged from prior versions. Added in v1.39 | | `model_profile_overrides..` | string \| object | per-runtime tier override | (none) | Override the runtime-aware tier mapping for a specific `(runtime, tier)`. Tier is one of `opus`, `sonnet`, `haiku`. Value is either a model ID string (e.g. `"gpt-5-pro"`) or `{ model, reasoning_effort }`. See [Runtime-Aware Profiles](#runtime-aware-profiles-2517). Added in v1.39 | +| `models.` | enum | `opus`, `sonnet`, `haiku`, `inherit` | (none) | Per-phase-type model tier. Six accepted slots: `planning`, `discuss`, `research`, `execution`, `verification`, `completion`. Lets you tune at the phase level ("Opus for planning, Sonnet for the rest") without learning agent names. Resolves between `model_overrides` (higher) and `model_profile` (lower); see [Per-Phase-Type Models](#per-phase-type-models-models--added-in-v140). Added in v1.40 ([#3023](https://github.com/gsd-build/get-shit-done/pull/3030)) | | `project_code` | string | any short string | (none) | Prefix for phase directory names (e.g., `"ABC"` produces `ABC-01-setup/`). Added in v1.31 | | `response_language` | string | language code | (none) | Language for agent responses (e.g., `"pt"`, `"ko"`, `"ja"`). Propagates to all spawned agents for cross-phase language consistency. Added in v1.32 | | `context_window` | number | any integer | `200000` | Context window size in tokens. Set `1000000` for 1M-context models (e.g., `claude-opus-4-7[1m]`). Values `>= 500000` enable adaptive context enrichment (full-body reads of prior SUMMARY.md, deeper anti-pattern reads). Configured via `/gsd-settings-advanced`. | @@ -643,6 +645,89 @@ OpenCode's `task` interface do not accept an inline `model` parameter, so running `gsd install ` after editing `model_overrides` is required for the change to take effect. See issue #2256. +### Per-Phase-Type Models (`models`) — added in v1.40 + +> Express tuning at the **phase** level (planning, research, execution, verification) without learning the agent taxonomy. Added in [#3023](https://github.com/gsd-build/get-shit-done/pull/3030). + +`model_overrides` is per-**agent** (precise but verbose; you have to know that `gsd-codebase-mapper` is research and `gsd-doc-writer` is execution). The `models` block lets you say "Opus for planning and execution, Sonnet for the rest" in two lines: + +```json +{ + "model_profile": "balanced", + "models": { + "planning": "opus", + "discuss": "opus", + "research": "sonnet", + "execution": "opus", + "verification": "sonnet", + "completion": "sonnet" + }, + "model_overrides": { + "gsd-codebase-mapper": "haiku" + } +} +``` + +#### Phase-type → agent mapping + +| Phase type | Agents | +|---|---| +| `planning` | `gsd-planner`, `gsd-roadmapper`, `gsd-pattern-mapper` | +| `discuss` | (reserved — no subagent today) | +| `research` | `gsd-phase-researcher`, `gsd-project-researcher`, `gsd-research-synthesizer`, `gsd-codebase-mapper`, `gsd-ui-researcher` | +| `execution` | `gsd-executor`, `gsd-debugger`, `gsd-doc-writer` | +| `verification` | `gsd-verifier`, `gsd-plan-checker`, `gsd-integration-checker`, `gsd-nyquist-auditor`, `gsd-ui-checker`, `gsd-ui-auditor`, `gsd-doc-verifier` | +| `completion` | (reserved — no subagent today) | + +`discuss` and `completion` are accepted by the schema for forward compatibility; setting them today is a no-op until a subagent maps to them. + +#### Resolution precedence (highest → lowest) + +``` +1. model_overrides[] ← per-agent; full IDs accepted; targeted exception +2. models[] ← coarse phase-level tier +3. model_profile (per-agent col) ← global tier strategy +4. Runtime default ← when nothing else applies +``` + +The three layers compose: `models` defaults a phase, and `model_overrides` carves an exception out of it. In the example above, all five research agents resolve to `sonnet` *except* `gsd-codebase-mapper`, which the per-agent override pins to `haiku`. + +#### Accepted values + +`models.` accepts only tier aliases: + +| Value | Effect | +|---|---| +| `"opus"` / `"sonnet"` / `"haiku"` | Standard tier — runtime resolution maps to the active runtime's model for that tier | +| `"inherit"` | Agents in this phase follow the session model (same semantics as `model_profile: "inherit"`) | + +If you need a fully-qualified model ID (`"openai/gpt-5"`, `"google/gemini-2.5-pro"`), use `model_overrides` per agent instead. `models.*` is intentionally tier-only so the runtime-aware mapping stays correct on Codex / OpenCode / Gemini CLI installs. + +#### When to use which + +| You want | Use | +|---|---| +| One global tier strategy ("balanced everywhere") | `model_profile` | +| Coarse phase-level tuning ("Opus for planning") | `models.` | +| Per-agent precision ("force haiku on the codebase mapper") | `model_overrides[]` | +| Full model ID for a specific agent | `model_overrides[]: "openai/gpt-5"` | + +Mix freely — the precedence rule above resolves any overlap deterministically. + +#### Validation + +`config-set` rejects unknown phase-types: + +```bash +$ gsd config-set models.deployment opus +Error: 'models.deployment' is not a valid config key + +# Valid: +$ gsd config-set models.research sonnet +``` + +Direct edits to `.planning/config.json` are looser — the resolver simply ignores values it doesn't recognize and falls through to the profile tier — so a typo doesn't silently break tier resolution. + ### Non-Claude Runtimes (Codex, OpenCode, Gemini CLI, Kilo) When GSD is installed for a non-Claude runtime, the installer automatically sets `resolve_model_ids: "omit"` in `~/.gsd/defaults.json`. This causes GSD to return an empty model parameter for all agents, so each agent uses whatever model the runtime is configured with. No additional setup is needed for the default case. diff --git a/docs/FEATURES.md b/docs/FEATURES.md index cdccb6e57..83fd3eddc 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -923,6 +923,7 @@ continues. Drift detection cannot fail verification. | `mode` | enum | `interactive` | `interactive` or `yolo` (auto-approve) | | `granularity` | enum | `standard` | `coarse`, `standard`, or `fine` | | `model_profile` | enum | `balanced` | `quality`, `balanced`, `budget`, or `inherit` | +| `models.` | enum | (none) | Per-phase-type tier override (`planning`, `discuss`, `research`, `execution`, `verification`, `completion`). Values: `opus`, `sonnet`, `haiku`, `inherit`. Coarse phase-level tuning that wins over `model_profile` but loses to per-agent `model_overrides`. See [CONFIGURATION.md](CONFIGURATION.md#per-phase-type-models-models--added-in-v140). Added in v1.40 | | `workflow.research` | boolean | `true` | Domain research before planning | | `workflow.plan_check` | boolean | `true` | Plan verification loop | | `workflow.verifier` | boolean | `true` | Post-execution verification | diff --git a/docs/USER-GUIDE.md b/docs/USER-GUIDE.md index 082ea8b66..7afb9185d 100644 --- a/docs/USER-GUIDE.md +++ b/docs/USER-GUIDE.md @@ -1012,6 +1012,41 @@ Do not re-run `/gsd-execute-phase`. Use `/gsd-quick` for targeted fixes, or `/gs Switch to budget profile: `/gsd-set-profile budget`. Disable research and plan-check agents via `/gsd-settings` if the domain is familiar to you (or to Claude). +### Tuning model cost by phase (`models`) — added in v1.40 + +If you've heard "use Opus for planning, Sonnet for verification" and want to apply that without learning the agent taxonomy, add a `models` block to `.planning/config.json`: + +```json +{ + "model_profile": "balanced", + "models": { + "planning": "opus", + "discuss": "opus", + "research": "sonnet", + "execution": "opus", + "verification": "sonnet", + "completion": "sonnet" + } +} +``` + +The six slots (`planning` / `discuss` / `research` / `execution` / `verification` / `completion`) accept tier aliases (`opus`, `sonnet`, `haiku`, `inherit`). Each slot covers a group of agents — for example, setting `models.research = "sonnet"` applies to `gsd-phase-researcher`, `gsd-codebase-mapper`, `gsd-research-synthesizer`, and the other research agents in one shot. + +Need a per-agent exception? Add `model_overrides` alongside — it wins over `models`: + +```json +{ + "models": { "research": "sonnet" }, + "model_overrides": { + "gsd-codebase-mapper": "haiku" + } +} +``` + +That gives sonnet to all research agents *except* the codebase mapper, which runs haiku for the cheap-but-broad fan-out scan. + +For the full mapping table and resolution-precedence rules, see [Per-Phase-Type Models](CONFIGURATION.md#per-phase-type-models-models--added-in-v140) in the configuration reference. + ### Using Non-Claude Runtimes (Codex, OpenCode, Gemini CLI, Kilo) If you installed GSD for a non-Claude runtime, the installer already configured model resolution so all agents use the runtime's default model. No manual setup is needed. Specifically, the installer sets `resolve_model_ids: "omit"` in your config, which tells GSD to skip Anthropic model ID resolution and let the runtime choose its own default model. diff --git a/get-shit-done/bin/lib/config-schema.cjs b/get-shit-done/bin/lib/config-schema.cjs index 423eead92..42e5671f9 100644 --- a/get-shit-done/bin/lib/config-schema.cjs +++ b/get-shit-done/bin/lib/config-schema.cjs @@ -84,6 +84,12 @@ const DYNAMIC_KEY_PATTERNS = [ // is a free string (so users can map non-built-in runtimes); is enum-restricted. { topLevel: 'model_profile_overrides', test: (k) => /^model_profile_overrides\.[a-zA-Z0-9_-]+\.(opus|sonnet|haiku)$/.test(k), description: 'model_profile_overrides..' }, + // #3023 — per-phase-type model map: models. = + // Six named slots (planning/discuss/research/execution/verification/completion); + // unknown phase-types are rejected. Per-agent model_overrides still take + // precedence over phase-type at resolve time. + { topLevel: 'models', test: (k) => /^models\.(planning|discuss|research|execution|verification|completion)$/.test(k), + description: 'models.' }, ]; /** diff --git a/get-shit-done/bin/lib/core.cjs b/get-shit-done/bin/lib/core.cjs index c3cb1862e..1435d03b9 100644 --- a/get-shit-done/bin/lib/core.cjs +++ b/get-shit-done/bin/lib/core.cjs @@ -6,7 +6,7 @@ const fs = require('fs'); const os = require('os'); const path = require('path'); const { execSync, execFileSync, spawnSync } = require('child_process'); -const { MODEL_PROFILES } = require('./model-profiles.cjs'); +const { MODEL_PROFILES, AGENT_TO_PHASE_TYPE, VALID_PHASE_TYPES } = require('./model-profiles.cjs'); // Compatibility shim: new imports should use planning-workspace.cjs directly. const { planningDir, @@ -500,6 +500,12 @@ function loadConfig(cwd) { project_code: get('project_code') ?? defaults.project_code, subagent_timeout: get('subagent_timeout', { section: 'workflow', field: 'subagent_timeout' }) ?? defaults.subagent_timeout, model_overrides: parsed.model_overrides || null, + // #3023 — per-phase-type model map. Six named slots + // (planning/discuss/research/execution/verification/completion). + // Resolves between per-agent override and profile-derived tier in + // resolveModelInternal. Defaults to null so configs without it + // behave exactly as today. + models: parsed.models || null, // #2517 — runtime-aware profiles. `runtime` defaults to null (back-compat). // When null, resolveModelInternal preserves today's Claude-native behavior. // NOTE: `runtime` and `model_profile_overrides` are intentionally read @@ -560,6 +566,7 @@ function loadConfig(cwd) { context_window: globalDefaults.context_window ?? defaults.context_window, subagent_timeout: globalDefaults.subagent_timeout ?? defaults.subagent_timeout, model_overrides: globalDefaults.model_overrides || null, + models: globalDefaults.models || null, agent_skills: globalDefaults.agent_skills || {}, response_language: globalDefaults.response_language || null, }; @@ -1518,10 +1525,37 @@ function resolveModelInternal(cwd, agentType) { return override; } - // 2. Compute the tier (opus/sonnet/haiku) for this agent under the active profile. + // 2. Compute the tier (opus/sonnet/haiku/inherit) for this agent. + // + // #3023: phase-type slot can override the profile-derived tier. + // Precedence: per-agent override (above) > phase-type slot > profile. + // Phase-type values are tier aliases (opus/sonnet/haiku/inherit) — same + // shape as model_profile output — so the runtime-resolution chain + // (step 3), resolve_model_ids handling (step 4), and profile lookup + // (step 5) all stay correct without further branching. const profile = String(config.model_profile || 'balanced').toLowerCase(); const agentModels = MODEL_PROFILES[agentType]; - const tier = agentModels ? (agentModels[profile] || agentModels['balanced']) : null; + const phaseType = AGENT_TO_PHASE_TYPE[agentType]; + const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object') + ? config.models[phaseType] + : undefined; + // Only honor phase-type tier if it's one of the recognized aliases. + // Anything else falls through to profile lookup so a typo doesn't + // silently break tier resolution. + const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku', 'inherit']); + // Resolve tier: phase-type wins when valid; else profile-derived; else + // (when profile === 'inherit') propagate inherit so the later short- + // circuit fires. CR Major (#3030): a config like + // { model_profile: 'inherit', models: { execution: 'opus' } } + // must honor the phase-type opus, not return 'inherit'. Synthesizing + // tier='inherit' only when there's no phase-type override keeps the + // original inherit semantics intact while letting a valid phase-type + // tier win. + const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier)) + ? phaseTypeTier + : (profile === 'inherit' + ? 'inherit' + : (agentModels ? (agentModels[profile] || agentModels['balanced']) : null)); // 3. Runtime-aware resolution (#2517) — only when `runtime` is explicitly set // to a non-Claude runtime. `runtime: "claude"` is the implicit default and is @@ -1529,8 +1563,10 @@ function resolveModelInternal(cwd, agentType) { // "omit"` (review finding #4). Deliberate ordering for non-Claude runtimes: // explicit opt-in beats `resolve_model_ids: "omit"` so users on Codex installs // that auto-set "omit" can still flip on tiered behavior by setting runtime - // alone. inherit profile is preserved verbatim. - if (config.runtime && config.runtime !== 'claude' && profile !== 'inherit' && tier) { + // alone. Gate on tier !== 'inherit' (not profile !== 'inherit') so a + // valid phase-type tier flips runtime resolution on even when the + // profile is inherit. + if (config.runtime && config.runtime !== 'claude' && tier && tier !== 'inherit') { const entry = _resolveRuntimeTier(config, tier); if (entry?.model) return entry.model; // Unknown runtime with no user-supplied overrides — fall through to Claude-safe @@ -1546,7 +1582,9 @@ function resolveModelInternal(cwd, agentType) { // 5. Profile lookup (Claude-native default). if (!agentModels) return 'sonnet'; - if (profile === 'inherit') return 'inherit'; + // Gate on tier (not profile) so a valid phase-type override beats + // profile=inherit (#3030 CR Major). + if (tier === 'inherit') return 'inherit'; // `tier` is guaranteed truthy here: agentModels exists, and MODEL_PROFILES // entries always define `balanced`, so `agentModels[profile] || agentModels.balanced` // resolves to a string. Keep the local for readability — no defensive fallback. @@ -1586,11 +1624,38 @@ function resolveReasoningEffortInternal(cwd, agentType) { if (config.model_overrides?.[agentType]) return null; const profile = String(config.model_profile || 'balanced').toLowerCase(); - if (profile === 'inherit') return null; const agentModels = MODEL_PROFILES[agentType]; if (!agentModels) return null; - const tier = agentModels[profile] || agentModels['balanced']; - if (!tier) return null; + + // #3023 (CR Major): mirror the phase-type tier lookup from + // resolveModelInternal. Without this, `model` and `reasoning_effort` + // derive from different tier sources on Codex when models. + // overrides the profile. + // + // #3030 CR follow-up: do NOT short-circuit on profile === 'inherit' + // before reading the phase-type tier. A config like + // { model_profile: 'inherit', models: { execution: 'opus' } } + // must produce the opus runtime effort, not null. Compute tier from + // phase-type first; only fall back to profile when there's no valid + // phase-type override; only return null when the resolved tier is + // 'inherit' or unknown. + const phaseType = AGENT_TO_PHASE_TYPE[agentType]; + const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object') + ? config.models[phaseType] + : undefined; + // Explicit phase-type 'inherit' is the user opting out of tier-based + // effort for this phase — return null instead of falling through to + // profile (which would silently emit the profile's effort and + // contradict the user's choice). + if (phaseTypeTier === 'inherit') return null; + const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku']); + const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier)) + ? phaseTypeTier + : (profile === 'inherit' + ? 'inherit' + : (agentModels[profile] || agentModels['balanced'])); + // 'inherit' (from profile fallback) yields no runtime effort. + if (!tier || tier === 'inherit') return null; const entry = _resolveRuntimeTier(config, tier); return entry?.reasoning_effort || null; diff --git a/get-shit-done/bin/lib/model-profiles.cjs b/get-shit-done/bin/lib/model-profiles.cjs index ae3472148..a314d30b7 100644 --- a/get-shit-done/bin/lib/model-profiles.cjs +++ b/get-shit-done/bin/lib/model-profiles.cjs @@ -28,6 +28,61 @@ const MODEL_PROFILES = { }; const VALID_PROFILES = [...Object.keys(MODEL_PROFILES['gsd-planner']), 'inherit']; +/** + * #3023 — Phase-type → agent mapping table. + * + * Lets users tune model selection at the *phase-type* level (planning, + * research, execution, verification, ...) instead of per-agent. Maps + * each agent in MODEL_PROFILES to exactly one phase-type so resolution + * is deterministic. + * + * Adding a new agent to MODEL_PROFILES requires adding an entry here too; + * tests/feat-3023-phase-type-models.test.cjs asserts coverage. + * + * Phase-type semantics: + * - planning: produces the plan (PLAN.md, ROADMAP.md, patterns) + * - discuss: collaborative scoping (no subagent today; reserved + * for orchestrators that may spawn one) + * - research: gathers external/codebase context (RESEARCH.md) + * - execution: implements the plan or writes user-facing artifacts + * - verification: checks correctness (VERIFICATION.md, audits) + * - completion: post-execution wrap-up (no subagent today; reserved) + */ +const AGENT_TO_PHASE_TYPE = { + // Planning — produces the plan / roadmap / pattern map + 'gsd-planner': 'planning', + 'gsd-roadmapper': 'planning', + 'gsd-pattern-mapper': 'planning', + // Research — external/codebase information gathering + 'gsd-phase-researcher': 'research', + 'gsd-project-researcher': 'research', + 'gsd-research-synthesizer': 'research', + 'gsd-codebase-mapper': 'research', + 'gsd-ui-researcher': 'research', + // Execution — implementation, debugging, doc writing + 'gsd-executor': 'execution', + 'gsd-debugger': 'execution', + 'gsd-doc-writer': 'execution', + // Verification — correctness checks, audits, gap analysis + 'gsd-verifier': 'verification', + 'gsd-plan-checker': 'verification', + 'gsd-integration-checker': 'verification', + 'gsd-nyquist-auditor': 'verification', + 'gsd-ui-checker': 'verification', + 'gsd-ui-auditor': 'verification', + 'gsd-doc-verifier': 'verification', +}; + +/** + * The six phase-type slots accepted in `.planning/config.json` `models` + * block. `discuss` and `completion` are reserved — no current agent maps + * to them today — so users can pre-configure those slots without + * breaking validation when an orchestrator starts honoring them. + */ +const VALID_PHASE_TYPES = new Set([ + 'planning', 'discuss', 'research', 'execution', 'verification', 'completion', +]); + /** * Formats the agent-to-model mapping as a human-readable table (in string format). * @@ -68,6 +123,8 @@ function getAgentToModelMapForProfile(normalizedProfile) { module.exports = { MODEL_PROFILES, VALID_PROFILES, + AGENT_TO_PHASE_TYPE, + VALID_PHASE_TYPES, formatAgentToModelMapAsTable, getAgentToModelMapForProfile, }; diff --git a/get-shit-done/references/model-profiles.md b/get-shit-done/references/model-profiles.md index 6bd4420ba..0367e46a1 100644 --- a/get-shit-done/references/model-profiles.md +++ b/get-shit-done/references/model-profiles.md @@ -19,6 +19,53 @@ Model profiles control which Claude model each GSD agent uses. This allows balan | gsd-integration-checker | sonnet | sonnet | haiku | haiku | inherit | | gsd-nyquist-auditor | sonnet | sonnet | haiku | haiku | inherit | +## Per-Phase-Type Model Map (#3023) + +`.planning/config.json` accepts a coarse per-**phase-type** map under the `models` key. Use this when you want tuning at the phase level ("Opus for planning and execution, Sonnet for the rest") without learning the agent taxonomy. + +```json +{ + "model_profile": "balanced", + "models": { + "planning": "opus", + "discuss": "opus", + "research": "sonnet", + "execution": "opus", + "verification": "sonnet", + "completion": "sonnet" + }, + "model_overrides": { + "gsd-codebase-mapper": "haiku" + } +} +``` + +### Phase-type → agent mapping + +| Phase type | Agents | +|---|---| +| `planning` | gsd-planner, gsd-roadmapper, gsd-pattern-mapper | +| `discuss` | (reserved — no subagent today) | +| `research` | gsd-phase-researcher, gsd-project-researcher, gsd-research-synthesizer, gsd-codebase-mapper, gsd-ui-researcher | +| `execution` | gsd-executor, gsd-debugger, gsd-doc-writer | +| `verification` | gsd-verifier, gsd-plan-checker, gsd-integration-checker, gsd-nyquist-auditor, gsd-ui-checker, gsd-ui-auditor, gsd-doc-verifier | +| `completion` | (reserved — no subagent today) | + +### Resolution precedence (highest to lowest) + +1. **Per-agent `model_overrides[agent]`** — full IDs accepted; targeted exceptions +2. **Phase-type `models[phase_type]`** — tier alias only (`opus` / `sonnet` / `haiku` / `inherit`) +3. **Profile table** — the per-agent column from the active `model_profile` +4. **Runtime default** — when nothing else applies + +### Why two layers above the profile? + +- **Profile** is a global tier strategy (everyone runs balanced). +- **`models`** is coarse phase-level tuning without learning agent names. +- **`model_overrides`** is per-agent precision (e.g. force `haiku` on `gsd-codebase-mapper` for a fan-out). + +The three layers compose: `models` defaults a phase, `model_overrides` carves an exception out of it. + ## Profile Philosophy **quality** - Maximum reasoning power @@ -93,10 +140,16 @@ Orchestrators resolve model before spawning: ``` 1. Read .planning/config.json 2. Check model_overrides for agent-specific override -3. If no override, look up agent in profile table -4. Pass model parameter to Task call +3. If no override, check models[phase_type] for a phase-type tier + (see §Per-Phase-Type Model Map for the agent → phase-type mapping) +4. If no phase-type slot, look up agent in profile table +5. Pass model parameter to Task call ``` +The same precedence applies to `reasoning_effort` resolution on runtimes +that support it (Codex), so `model` and `reasoning_effort` always derive +from the same tier source — a `models[phase_type]` override flips both. + ## Per-Agent Overrides Override specific agents without changing the entire profile: diff --git a/sdk/src/query/config-schema.ts b/sdk/src/query/config-schema.ts index f1d5e6728..9e6b0d7be 100644 --- a/sdk/src/query/config-schema.ts +++ b/sdk/src/query/config-schema.ts @@ -111,6 +111,12 @@ export const DYNAMIC_KEY_PATTERNS: readonly DynamicKeyPattern[] = [ description: 'model_profile_overrides..', test: (k) => /^model_profile_overrides\.[a-zA-Z0-9_-]+\.(opus|sonnet|haiku)$/.test(k), }, + // #3023 — per-phase-type model map: models. = + { + source: '^models\\.(planning|discuss|research|execution|verification|completion)$', + description: 'models.', + test: (k) => /^models\.(planning|discuss|research|execution|verification|completion)$/.test(k), + }, ]; /** Returns true if keyPath is a valid config key (exact or dynamic pattern). */ diff --git a/tests/feat-3023-phase-type-models.test.cjs b/tests/feat-3023-phase-type-models.test.cjs new file mode 100644 index 000000000..d0b0f5051 --- /dev/null +++ b/tests/feat-3023-phase-type-models.test.cjs @@ -0,0 +1,408 @@ +/** + * Feature test for issue #3023 — per-phase-type model map. + * + * Adds a `models` block to .planning/config.json that accepts phase-type + * keys (planning / discuss / research / execution / verification / + * completion). Resolution precedence: + * + * 1. Per-agent `model_overrides[agent]` (highest) + * 2. Phase-type `models[phase_type]` (NEW) + * 3. Profile table (`model_profile`) + * 4. Runtime default + * + * Tests are typed-IR / structural — assert on the value returned by + * resolveModelInternal, not stdout/grep. Each test seeds a temp project + * with a fixture .planning/config.json and asserts the resolver picks + * the right tier for each agent. + */ + +'use strict'; + +process.env.GSD_TEST_MODE = '1'; + +const { test, describe, beforeEach, afterEach } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const os = require('node:os'); + +const { + resolveModelInternal, +} = require('../get-shit-done/bin/lib/core.cjs'); +const { + AGENT_TO_PHASE_TYPE, + VALID_PHASE_TYPES, + MODEL_PROFILES, +} = require('../get-shit-done/bin/lib/model-profiles.cjs'); +const { isValidConfigKey } = require('../get-shit-done/bin/lib/config-schema.cjs'); + +function makeTmp(prefix) { + return fs.mkdtempSync(path.join(os.tmpdir(), `gsd-3023-${prefix}-`)); +} + +function writeConfig(projectDir, config) { + const planningDir = path.join(projectDir, '.planning'); + fs.mkdirSync(planningDir, { recursive: true }); + fs.writeFileSync(path.join(planningDir, 'config.json'), JSON.stringify(config, null, 2)); +} + +function rmr(p) { + try { fs.rmSync(p, { recursive: true, force: true }); } catch { /* noop */ } +} + +// ─── Schema: AGENT_TO_PHASE_TYPE table + VALID_PHASE_TYPES ────────────────── + +describe('#3023 phase-type schema: every agent has a phase-type assignment', () => { + test('AGENT_TO_PHASE_TYPE is exported as a non-empty object', () => { + assert.equal(typeof AGENT_TO_PHASE_TYPE, 'object'); + assert.ok(AGENT_TO_PHASE_TYPE !== null); + assert.ok(Object.keys(AGENT_TO_PHASE_TYPE).length > 0); + }); + + test('VALID_PHASE_TYPES exposes the six named slots from the issue', () => { + // The issue specified exactly these slots. Adding new slots here is a + // schema change that must coordinate with config-schema's dynamic + // pattern and the docs. + assert.deepStrictEqual( + [...VALID_PHASE_TYPES].sort(), + ['completion', 'discuss', 'execution', 'planning', 'research', 'verification'].sort() + ); + }); + + test('every agent in MODEL_PROFILES has a phase-type assignment', () => { + const missing = Object.keys(MODEL_PROFILES).filter( + (agent) => !AGENT_TO_PHASE_TYPE[agent] + ); + assert.deepStrictEqual(missing, [], + `every agent in MODEL_PROFILES must have a phase-type — missing: ${JSON.stringify(missing)}`); + }); + + test('every assigned phase-type is one of the six valid slots', () => { + const invalid = Object.entries(AGENT_TO_PHASE_TYPE).filter( + ([, phaseType]) => !VALID_PHASE_TYPES.has(phaseType) + ); + assert.deepStrictEqual(invalid, [], + `phase-type assignments must use VALID_PHASE_TYPES — invalid: ${JSON.stringify(invalid)}`); + }); +}); + +// ─── Resolver behavior: phase-type drives tier ────────────────────────────── + +describe('#3023 resolver: models. overrides profile-based tier', () => { + let projectDir; + beforeEach(() => { projectDir = makeTmp('resolver'); }); + afterEach(() => { rmr(projectDir); }); + + test('phase-type alone — research agents get the phase-type tier, planner gets profile default', () => { + writeConfig(projectDir, { + model_profile: 'balanced', + models: { research: 'haiku' }, + }); + // gsd-phase-researcher is a research agent — should pick up 'haiku' + // from the phase-type slot, not 'sonnet' from the balanced profile. + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'haiku'); + // gsd-codebase-mapper is also research → haiku + assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku'); + // gsd-planner is planning, no models.planning set → falls through to + // profile (balanced → opus per MODEL_PROFILES). + assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus'); + }); + + test('per-agent override beats phase-type (acceptance criterion b)', () => { + writeConfig(projectDir, { + model_profile: 'balanced', + models: { research: 'haiku' }, + model_overrides: { 'gsd-phase-researcher': 'opus' }, + }); + // The targeted per-agent override wins for that one agent. + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'opus'); + // Other research agents still pick up the phase-type tier. + assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku'); + assert.equal(resolveModelInternal(projectDir, 'gsd-research-synthesizer'), 'haiku'); + }); + + test('phase-type beats profile (acceptance criterion c)', () => { + // model_profile=quality would normally make research agents 'opus'. + // models.research='haiku' must win. + writeConfig(projectDir, { + model_profile: 'quality', + models: { research: 'haiku' }, + }); + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'haiku'); + assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku'); + // gsd-planner is planning, no slot set, profile=quality → opus. + assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus'); + }); + + test('issue example: opus for planning/discuss/execution, sonnet for research/verification/completion', () => { + writeConfig(projectDir, { + model_profile: 'balanced', + models: { + planning: 'opus', + discuss: 'opus', + execution: 'opus', + research: 'sonnet', + verification: 'sonnet', + completion: 'sonnet', + }, + }); + // Planning agents → opus + assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus'); + // Execution agents → opus + assert.equal(resolveModelInternal(projectDir, 'gsd-executor'), 'opus'); + // Research agents → sonnet + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet'); + // Verification agents → sonnet + assert.equal(resolveModelInternal(projectDir, 'gsd-verifier'), 'sonnet'); + }); + + test('phase-type "inherit" is honored (preserves existing inherit semantics)', () => { + writeConfig(projectDir, { + model_profile: 'balanced', + models: { research: 'inherit' }, + }); + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'inherit'); + }); + + test('empty models block is a no-op (acceptance criterion: backward compat)', () => { + writeConfig(projectDir, { + model_profile: 'balanced', + models: {}, + }); + // Behavior must match no-models config (balanced profile). + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet'); + assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus'); + }); + + test('no models block at all is a no-op (acceptance criterion: backward compat)', () => { + writeConfig(projectDir, { + model_profile: 'balanced', + }); + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet'); + assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus'); + }); + + test('unrecognized tier value falls through to profile (typo safety) — CR follow-up', () => { + // The VALID_TIERS guard in resolveModelInternal must reject any value + // that isn't a known tier alias and fall back to the profile tier. + // Without this guard a typo like "haiku3" would pollute the runtime + // resolution chain. Locks the guard in so a future regression that + // removes it is caught. + writeConfig(projectDir, { + model_profile: 'balanced', + models: { research: 'haiku3' }, // typo; not a valid tier alias + }); + // Falls back to balanced → sonnet for research agents. + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet'); + assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku', + 'gsd-codebase-mapper at balanced is haiku per profile, unaffected by typo'); + }); + + test('full model ID in models. is rejected; falls through to profile — CR follow-up', () => { + // Full IDs are not valid in models.; they belong in + // model_overrides per agent. The guard ensures we don't accidentally + // hand a full ID into the runtime-tier resolution chain. + writeConfig(projectDir, { + model_profile: 'balanced', + models: { research: 'openai/gpt-5' }, + }); + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet'); + }); + + // ─── CR Major: phase-type beats inherit profile ───────────────────────── + // Pre-fix bug: model_profile='inherit' + models.execution='opus' returned + // 'inherit' because the profile short-circuit fired BEFORE the phase-type + // override could win, violating the documented precedence where + // models[phase_type] beats model_profile. + + test('phase-type override wins over profile=inherit (CR Major) — model resolver', () => { + writeConfig(projectDir, { + model_profile: 'inherit', + models: { execution: 'opus' }, + }); + // gsd-executor (execution) must get the phase-type opus, not inherit. + assert.equal(resolveModelInternal(projectDir, 'gsd-executor'), 'opus'); + }); + + test('phase-type "haiku" wins over profile=inherit; agents without a slot still inherit', () => { + writeConfig(projectDir, { + model_profile: 'inherit', + models: { research: 'haiku' }, + }); + // research agents → haiku (phase-type wins) + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'haiku'); + assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku'); + // planning agent has no slot set → falls through to profile=inherit. + assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'inherit'); + }); + + test('profile=inherit with no models block still returns inherit (no regression)', () => { + writeConfig(projectDir, { + model_profile: 'inherit', + }); + assert.equal(resolveModelInternal(projectDir, 'gsd-executor'), 'inherit'); + assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'inherit'); + }); + + test('profile=inherit with models block but agent has no slot → inherit', () => { + writeConfig(projectDir, { + model_profile: 'inherit', + models: { research: 'haiku' }, + }); + // gsd-executor (execution slot) is not set → falls through to inherit. + assert.equal(resolveModelInternal(projectDir, 'gsd-executor'), 'inherit'); + }); +}); + +// ─── #3030 CR Major outside-diff: reasoning_effort honors phase-type ─────── + +const { resolveReasoningEffortInternal } = require('../get-shit-done/bin/lib/core.cjs'); + +describe('#3023 + #3030 CR: resolveReasoningEffortInternal honors phase-type tier (Codex)', () => { + let projectDir; + beforeEach(() => { projectDir = makeTmp('effort'); }); + afterEach(() => { rmr(projectDir); }); + + test('exported from core.cjs', () => { + assert.equal(typeof resolveReasoningEffortInternal, 'function'); + }); + + test('phase-type override flips both model AND reasoning_effort to the same tier (Codex)', () => { + // The CR Major bug: previously the model was resolved from the + // phase-type tier (opus → gpt-5.4) but reasoning_effort still came + // from the profile-derived sonnet tier (medium) — leading to a + // mismatched (model, effort) pair on Codex spawn. + writeConfig(projectDir, { + runtime: 'codex', + model_profile: 'balanced', + models: { execution: 'opus' }, + }); + // gsd-executor's profile tier under balanced is sonnet, so without + // the phase-type lookup mirror, model would resolve to opus (xhigh) + // but effort to medium. Both must derive from the same tier source. + const effort = resolveReasoningEffortInternal(projectDir, 'gsd-executor'); + // The exact effort value depends on the runtime tier map's opus row; + // the test guards the relationship: it must NOT be the sonnet/medium + // value when the phase-type forced opus. + const sonnetEffort = (() => { + // Read the sonnet effort by setting a config that uses the sonnet tier + // and reading what comes back, so the assertion is semantic (effort + // matches phase-type tier) rather than a hard-coded string. + const sonnetDir = makeTmp('effort-sonnet'); + try { + writeConfig(sonnetDir, { + runtime: 'codex', model_profile: 'balanced', + }); + return resolveReasoningEffortInternal(sonnetDir, 'gsd-executor'); + } finally { + rmr(sonnetDir); + } + })(); + const opusEffort = (() => { + const opusDir = makeTmp('effort-opus'); + try { + writeConfig(opusDir, { + runtime: 'codex', model_profile: 'quality', // quality → executor=opus + }); + return resolveReasoningEffortInternal(opusDir, 'gsd-executor'); + } finally { + rmr(opusDir); + } + })(); + // The phase-type override (models.execution=opus) must produce the + // SAME effort as a profile-only opus config. + assert.equal(effort, opusEffort, + `phase-type override must match opus-tier effort, got ${effort}, expected ${opusEffort}`); + // And it must NOT match sonnet effort (proving the override fired). + if (opusEffort !== null && sonnetEffort !== null && opusEffort !== sonnetEffort) { + assert.notEqual(effort, sonnetEffort, + `phase-type override should not silently use sonnet effort: ${effort}`); + } + }); + + test('inherit phase-type tier returns null effort (no runtime entry maps to inherit)', () => { + writeConfig(projectDir, { + runtime: 'codex', + model_profile: 'balanced', + models: { execution: 'inherit' }, + }); + // 'inherit' has no runtime-tier entry, so the resolver returns null. + const effort = resolveReasoningEffortInternal(projectDir, 'gsd-executor'); + assert.equal(effort, null); + }); + + test('per-agent override still bypasses phase-type for reasoning_effort', () => { + writeConfig(projectDir, { + runtime: 'codex', + model_profile: 'balanced', + models: { execution: 'opus' }, + model_overrides: { 'gsd-executor': 'openai/gpt-5' }, + }); + // model_overrides[agent] short-circuits resolveReasoningEffortInternal + // (the user supplied a fully-qualified ID; effort must be set per-agent). + assert.equal(resolveReasoningEffortInternal(projectDir, 'gsd-executor'), null); + }); + + test('claude runtime ignores models.* for reasoning_effort (returns null)', () => { + writeConfig(projectDir, { + // No `runtime` set → defaults to claude, which has no reasoning_effort. + model_profile: 'balanced', + models: { execution: 'opus' }, + }); + assert.equal(resolveReasoningEffortInternal(projectDir, 'gsd-executor'), null); + }); + + test('phase-type override wins over profile=inherit for effort (CR Major #3030)', () => { + // Pre-fix bug: profile=inherit short-circuited to null even when + // models.execution=opus would have supplied a valid tier. + writeConfig(projectDir, { + runtime: 'codex', + model_profile: 'inherit', + models: { execution: 'opus' }, + }); + // Compute the expected effort by reading what gsd-executor would + // get under a profile-only opus config — the phase-type override + // must produce the SAME result. + const expected = (() => { + const dir = makeTmp('effort-opus2'); + try { + writeConfig(dir, { runtime: 'codex', model_profile: 'quality' }); + return resolveReasoningEffortInternal(dir, 'gsd-executor'); + } finally { + rmr(dir); + } + })(); + const actual = resolveReasoningEffortInternal(projectDir, 'gsd-executor'); + assert.equal(actual, expected, + `phase-type override over profile=inherit must produce the opus-tier effort; got ${actual}, expected ${expected}`); + assert.notEqual(actual, null, + 'phase-type opus must NOT return null effort just because profile=inherit'); + }); +}); + +// ─── Schema validation ────────────────────────────────────────────────────── + +describe('#3023 config-schema: models. validation', () => { + test('models.planning is a valid config key', () => { + assert.equal(isValidConfigKey('models.planning'), true); + }); + + test('all six phase-type slots are valid config keys', () => { + for (const slot of ['planning', 'discuss', 'research', 'execution', 'verification', 'completion']) { + assert.equal(isValidConfigKey(`models.${slot}`), true, + `models.${slot} must be a valid config key`); + } + }); + + test('unknown phase-type is rejected (acceptance criterion d)', () => { + assert.equal(isValidConfigKey('models.deployment'), false, + 'unknown phase-type must NOT be accepted'); + assert.equal(isValidConfigKey('models.gsd-planner'), false, + 'agent name in models.* must NOT be accepted (use model_overrides for agents)'); + }); + + test('models alone (without a slot) is not a valid config-set key', () => { + // Setting the whole block isn't a granular set; users edit JSON directly. + assert.equal(isValidConfigKey('models'), false); + }); +});