* feat(#3023): per-phase-type model map in .planning/config.json Adds a new `models` block to .planning/config.json with six phase-type slots (planning / discuss / research / execution / verification / completion). Lets users express coarse tuning ("Opus for planning, Sonnet for the rest") without learning the agent taxonomy. Resolution precedence (highest → lowest): 1. Per-agent `model_overrides[agent]` (full IDs; targeted exception) 2. Phase-type `models[phase_type]` (NEW; tier alias) 3. Profile table (`model_profile`) (per-agent column) 4. Runtime default The three layers compose: `models` defaults a phase, `model_overrides` carves an exception. Phase-type values are tier aliases (opus/sonnet/ haiku/inherit) so the runtime-resolution chain (#2517) stays correct end-to-end without further branching. Implementation: - model-profiles.cjs: new AGENT_TO_PHASE_TYPE map + VALID_PHASE_TYPES set. Each agent in MODEL_PROFILES gets one phase-type assignment; tests assert coverage so adding a new agent without updating the table fails CI. - core.cjs (resolveModelInternal): inserts phase-type tier lookup between per-agent override and profile-derived tier. Skips runtime resolution when the resolved tier is 'inherit' (was previously gated only on profile === 'inherit'; phase-type can now produce inherit independently). - core.cjs (loadConfig): pass `parsed.models` through both code paths so resolveModelInternal can read it. - config-schema.cjs + sdk/src/query/config-schema.ts: dynamic-pattern validator accepts only the six known phase-types; unknown slots rejected at config-set time. Backward compat: configs without `models` behave exactly as today. Tests (15 cases, all structural-IR — no stdout grep): - Schema: AGENT_TO_PHASE_TYPE coverage, VALID_PHASE_TYPES exact match - Resolver: phase-type alone; per-agent override beats phase-type; phase-type beats profile; issue's full example; "inherit"; empty block is no-op; no block is no-op - Validation: each of the 6 slots accepted; unknown slot rejected; bare `models` (no slot) rejected Verification: - 15/15 pass on new regression test - 6808/6808 full suite (5 net new), 0 fail - lint-no-source-grep clean across 375 test files Closes #3023 * docs(#3023): document `models` per-phase-type config in user-facing docs Adds `models` block coverage to the three user-facing docs that ship with each release: 1. docs/CONFIGURATION.md - New "Per-Phase-Type Models" section between "Per-Agent Overrides" and "Non-Claude Runtimes" with: * full example mixing models + model_overrides * phase-type → agent mapping table * resolution-precedence pseudocode * accepted values (tier alias only) * "When to use which" decision matrix * validation behavior + example error - Added `"models": {}` to the Full Schema snippet - Added a row for `models.<phase_type>` to the config keys table (next to model_profile_overrides for adjacency) 2. docs/FEATURES.md - Added a row for models.<phase_type> in the Configurable Settings table (right under model_profile) - Cross-link to CONFIGURATION.md for the full surface 3. docs/USER-GUIDE.md - New task-oriented "Tuning model cost by phase" section above "Using Non-Claude Runtimes" — leads with the concrete config and shows the override pattern (one-shot phase + targeted exception) - Cross-link to CONFIGURATION.md Verification: - 29/29 pass on config-schema-docs-parity + docs-update + new feature test (parity-check passes, so the config-schema entry I added in the feature commit is now matched by a docs row) - 6808/6808 full suite pass - lint-no-source-grep clean Doc style follows the same pattern used by the existing model_profile, model_overrides, and model_profile_overrides sections — example-led, table-backed, cross-referenced. Each doc surfaces the feature at the right depth (reference / settings table / task guide). * fix(#3023): mirror phase-type tier in resolveReasoningEffortInternal (CR Major) CodeRabbit caught a real Codex correctness bug + 3 minor docs/test issues: 1. **Major (outside-diff)** — resolveReasoningEffortInternal in core.cjs derived its tier exclusively from the profile table, ignoring the models.<phase_type> override added in #3023. Failure mode on Codex: Config: model_profile=balanced, models.execution=opus, agent=gsd-executor resolveModelInternal: tier=opus → gpt-5.4 resolveReasoningEffortInternal: tier=sonnet → reasoning_effort=medium ↑ WRONG — should be xhigh (opus tier on Codex) The runtime received a mismatched (model, effort) pair. Mirrored the phase-type lookup from resolveModelInternal so both functions derive from the same tier source. 'inherit' phase-type returns null effort (no runtime entry maps to 'inherit'; let runtime decide). 2. Minor — .changeset/per-phase-type-models.md `pr: TBD` → `pr: 3030`. 3. Minor (outside-diff) — model-profiles.md "Resolution Logic" section omitted the new phase-type tier. Updated the 4-step block to a 5-step block including `models[phase_type]` between override and profile, plus a paragraph noting that `model` and `reasoning_effort` derive from the same tier source. 4. Nitpick — added 2 typo-safety tests: - models.research = "haiku3" (typo) → falls through to profile - models.research = "openai/gpt-5" (full ID) → falls through to profile Plus 5 new reasoning_effort tests covering the Major fix: - exported correctly - phase-type override flips both model AND effort to same tier - inherit phase-type returns null effort - per-agent override still bypasses phase-type for effort - claude runtime ignores models.* (no effort propagation) Verification: - 24/24 pass on regression test (15 original + 2 typo-safety + 5 effort + 2 outside-diff related) - 6815/6815 full suite (7 net new from 6808) - lint-no-source-grep clean The reasoning_effort tests are written semantically (phase-type override must produce the SAME effort as a profile-only opus config) rather than hard-coding tier-specific effort strings, so changes to the runtime tier map don't break them. * fix(#3023): phase-type override beats profile=inherit (CR Major round 2) CodeRabbit caught another precedence inversion: when { model_profile: 'inherit', models: { execution: 'opus' } } both resolvers short-circuited on `profile === 'inherit'` BEFORE the phase-type override could be honored. Result: model returned 'inherit' and reasoning_effort returned null — both contradicting the documented precedence where models[phase_type] wins over model_profile. Fix in resolveModelInternal: - Compute tier from phase-type FIRST. If phase-type is a valid alias, it wins. Otherwise, fall back to profile-derived tier OR 'inherit' (when profile === 'inherit'). - Gate the runtime-resolution branch on `tier !== 'inherit'` (was `profile !== 'inherit'`) so phase-type=opus can flip runtime mapping on even when profile=inherit. - Gate the inherit-return on `tier === 'inherit'` (was `profile === 'inherit'`). Fix in resolveReasoningEffortInternal: - Remove the `if (profile === 'inherit') return null;` early-return. - Compute tier from phase-type first, fall back to profile. If phase-type is explicitly 'inherit' OR the resolved tier is 'inherit', return null (no runtime entry maps to inherit). Tests added (5 new): - model: phase-type wins over profile=inherit (with explicit opus, with haiku for one phase + planner-without-slot still inheriting) - model: profile=inherit + no models block → all agents inherit (no regression on existing inherit semantics) - model: profile=inherit + models block but agent has no slot → that agent inherits, agents with slots get phase-type tier - effort: phase-type opus + profile=inherit → produces opus-tier effort, NOT null (the original bug) Verification: - 27/27 pass on regression test (24 prior + 3 model + 1 effort) - 6820/6820 full suite (5 net new) - lint-no-source-grep clean The effort test reads the expected value by running a profile-only opus config and comparing — semantic check, not hard-coded effort string. So runtime tier map changes don't break the test.
This commit is contained in:
@@ -84,6 +84,12 @@ const DYNAMIC_KEY_PATTERNS = [
|
||||
// <runtime> is a free string (so users can map non-built-in runtimes); <tier> is enum-restricted.
|
||||
{ topLevel: 'model_profile_overrides', test: (k) => /^model_profile_overrides\.[a-zA-Z0-9_-]+\.(opus|sonnet|haiku)$/.test(k),
|
||||
description: 'model_profile_overrides.<runtime>.<opus|sonnet|haiku>' },
|
||||
// #3023 — per-phase-type model map: models.<phase_type> = <tier>
|
||||
// Six named slots (planning/discuss/research/execution/verification/completion);
|
||||
// unknown phase-types are rejected. Per-agent model_overrides still take
|
||||
// precedence over phase-type at resolve time.
|
||||
{ topLevel: 'models', test: (k) => /^models\.(planning|discuss|research|execution|verification|completion)$/.test(k),
|
||||
description: 'models.<planning|discuss|research|execution|verification|completion>' },
|
||||
];
|
||||
|
||||
/**
|
||||
|
||||
@@ -6,7 +6,7 @@ const fs = require('fs');
|
||||
const os = require('os');
|
||||
const path = require('path');
|
||||
const { execSync, execFileSync, spawnSync } = require('child_process');
|
||||
const { MODEL_PROFILES } = require('./model-profiles.cjs');
|
||||
const { MODEL_PROFILES, AGENT_TO_PHASE_TYPE, VALID_PHASE_TYPES } = require('./model-profiles.cjs');
|
||||
// Compatibility shim: new imports should use planning-workspace.cjs directly.
|
||||
const {
|
||||
planningDir,
|
||||
@@ -500,6 +500,12 @@ function loadConfig(cwd) {
|
||||
project_code: get('project_code') ?? defaults.project_code,
|
||||
subagent_timeout: get('subagent_timeout', { section: 'workflow', field: 'subagent_timeout' }) ?? defaults.subagent_timeout,
|
||||
model_overrides: parsed.model_overrides || null,
|
||||
// #3023 — per-phase-type model map. Six named slots
|
||||
// (planning/discuss/research/execution/verification/completion).
|
||||
// Resolves between per-agent override and profile-derived tier in
|
||||
// resolveModelInternal. Defaults to null so configs without it
|
||||
// behave exactly as today.
|
||||
models: parsed.models || null,
|
||||
// #2517 — runtime-aware profiles. `runtime` defaults to null (back-compat).
|
||||
// When null, resolveModelInternal preserves today's Claude-native behavior.
|
||||
// NOTE: `runtime` and `model_profile_overrides` are intentionally read
|
||||
@@ -560,6 +566,7 @@ function loadConfig(cwd) {
|
||||
context_window: globalDefaults.context_window ?? defaults.context_window,
|
||||
subagent_timeout: globalDefaults.subagent_timeout ?? defaults.subagent_timeout,
|
||||
model_overrides: globalDefaults.model_overrides || null,
|
||||
models: globalDefaults.models || null,
|
||||
agent_skills: globalDefaults.agent_skills || {},
|
||||
response_language: globalDefaults.response_language || null,
|
||||
};
|
||||
@@ -1518,10 +1525,37 @@ function resolveModelInternal(cwd, agentType) {
|
||||
return override;
|
||||
}
|
||||
|
||||
// 2. Compute the tier (opus/sonnet/haiku) for this agent under the active profile.
|
||||
// 2. Compute the tier (opus/sonnet/haiku/inherit) for this agent.
|
||||
//
|
||||
// #3023: phase-type slot can override the profile-derived tier.
|
||||
// Precedence: per-agent override (above) > phase-type slot > profile.
|
||||
// Phase-type values are tier aliases (opus/sonnet/haiku/inherit) — same
|
||||
// shape as model_profile output — so the runtime-resolution chain
|
||||
// (step 3), resolve_model_ids handling (step 4), and profile lookup
|
||||
// (step 5) all stay correct without further branching.
|
||||
const profile = String(config.model_profile || 'balanced').toLowerCase();
|
||||
const agentModels = MODEL_PROFILES[agentType];
|
||||
const tier = agentModels ? (agentModels[profile] || agentModels['balanced']) : null;
|
||||
const phaseType = AGENT_TO_PHASE_TYPE[agentType];
|
||||
const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object')
|
||||
? config.models[phaseType]
|
||||
: undefined;
|
||||
// Only honor phase-type tier if it's one of the recognized aliases.
|
||||
// Anything else falls through to profile lookup so a typo doesn't
|
||||
// silently break tier resolution.
|
||||
const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku', 'inherit']);
|
||||
// Resolve tier: phase-type wins when valid; else profile-derived; else
|
||||
// (when profile === 'inherit') propagate inherit so the later short-
|
||||
// circuit fires. CR Major (#3030): a config like
|
||||
// { model_profile: 'inherit', models: { execution: 'opus' } }
|
||||
// must honor the phase-type opus, not return 'inherit'. Synthesizing
|
||||
// tier='inherit' only when there's no phase-type override keeps the
|
||||
// original inherit semantics intact while letting a valid phase-type
|
||||
// tier win.
|
||||
const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier))
|
||||
? phaseTypeTier
|
||||
: (profile === 'inherit'
|
||||
? 'inherit'
|
||||
: (agentModels ? (agentModels[profile] || agentModels['balanced']) : null));
|
||||
|
||||
// 3. Runtime-aware resolution (#2517) — only when `runtime` is explicitly set
|
||||
// to a non-Claude runtime. `runtime: "claude"` is the implicit default and is
|
||||
@@ -1529,8 +1563,10 @@ function resolveModelInternal(cwd, agentType) {
|
||||
// "omit"` (review finding #4). Deliberate ordering for non-Claude runtimes:
|
||||
// explicit opt-in beats `resolve_model_ids: "omit"` so users on Codex installs
|
||||
// that auto-set "omit" can still flip on tiered behavior by setting runtime
|
||||
// alone. inherit profile is preserved verbatim.
|
||||
if (config.runtime && config.runtime !== 'claude' && profile !== 'inherit' && tier) {
|
||||
// alone. Gate on tier !== 'inherit' (not profile !== 'inherit') so a
|
||||
// valid phase-type tier flips runtime resolution on even when the
|
||||
// profile is inherit.
|
||||
if (config.runtime && config.runtime !== 'claude' && tier && tier !== 'inherit') {
|
||||
const entry = _resolveRuntimeTier(config, tier);
|
||||
if (entry?.model) return entry.model;
|
||||
// Unknown runtime with no user-supplied overrides — fall through to Claude-safe
|
||||
@@ -1546,7 +1582,9 @@ function resolveModelInternal(cwd, agentType) {
|
||||
|
||||
// 5. Profile lookup (Claude-native default).
|
||||
if (!agentModels) return 'sonnet';
|
||||
if (profile === 'inherit') return 'inherit';
|
||||
// Gate on tier (not profile) so a valid phase-type override beats
|
||||
// profile=inherit (#3030 CR Major).
|
||||
if (tier === 'inherit') return 'inherit';
|
||||
// `tier` is guaranteed truthy here: agentModels exists, and MODEL_PROFILES
|
||||
// entries always define `balanced`, so `agentModels[profile] || agentModels.balanced`
|
||||
// resolves to a string. Keep the local for readability — no defensive fallback.
|
||||
@@ -1586,11 +1624,38 @@ function resolveReasoningEffortInternal(cwd, agentType) {
|
||||
if (config.model_overrides?.[agentType]) return null;
|
||||
|
||||
const profile = String(config.model_profile || 'balanced').toLowerCase();
|
||||
if (profile === 'inherit') return null;
|
||||
const agentModels = MODEL_PROFILES[agentType];
|
||||
if (!agentModels) return null;
|
||||
const tier = agentModels[profile] || agentModels['balanced'];
|
||||
if (!tier) return null;
|
||||
|
||||
// #3023 (CR Major): mirror the phase-type tier lookup from
|
||||
// resolveModelInternal. Without this, `model` and `reasoning_effort`
|
||||
// derive from different tier sources on Codex when models.<phase_type>
|
||||
// overrides the profile.
|
||||
//
|
||||
// #3030 CR follow-up: do NOT short-circuit on profile === 'inherit'
|
||||
// before reading the phase-type tier. A config like
|
||||
// { model_profile: 'inherit', models: { execution: 'opus' } }
|
||||
// must produce the opus runtime effort, not null. Compute tier from
|
||||
// phase-type first; only fall back to profile when there's no valid
|
||||
// phase-type override; only return null when the resolved tier is
|
||||
// 'inherit' or unknown.
|
||||
const phaseType = AGENT_TO_PHASE_TYPE[agentType];
|
||||
const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object')
|
||||
? config.models[phaseType]
|
||||
: undefined;
|
||||
// Explicit phase-type 'inherit' is the user opting out of tier-based
|
||||
// effort for this phase — return null instead of falling through to
|
||||
// profile (which would silently emit the profile's effort and
|
||||
// contradict the user's choice).
|
||||
if (phaseTypeTier === 'inherit') return null;
|
||||
const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku']);
|
||||
const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier))
|
||||
? phaseTypeTier
|
||||
: (profile === 'inherit'
|
||||
? 'inherit'
|
||||
: (agentModels[profile] || agentModels['balanced']));
|
||||
// 'inherit' (from profile fallback) yields no runtime effort.
|
||||
if (!tier || tier === 'inherit') return null;
|
||||
|
||||
const entry = _resolveRuntimeTier(config, tier);
|
||||
return entry?.reasoning_effort || null;
|
||||
|
||||
@@ -28,6 +28,61 @@ const MODEL_PROFILES = {
|
||||
};
|
||||
const VALID_PROFILES = [...Object.keys(MODEL_PROFILES['gsd-planner']), 'inherit'];
|
||||
|
||||
/**
|
||||
* #3023 — Phase-type → agent mapping table.
|
||||
*
|
||||
* Lets users tune model selection at the *phase-type* level (planning,
|
||||
* research, execution, verification, ...) instead of per-agent. Maps
|
||||
* each agent in MODEL_PROFILES to exactly one phase-type so resolution
|
||||
* is deterministic.
|
||||
*
|
||||
* Adding a new agent to MODEL_PROFILES requires adding an entry here too;
|
||||
* tests/feat-3023-phase-type-models.test.cjs asserts coverage.
|
||||
*
|
||||
* Phase-type semantics:
|
||||
* - planning: produces the plan (PLAN.md, ROADMAP.md, patterns)
|
||||
* - discuss: collaborative scoping (no subagent today; reserved
|
||||
* for orchestrators that may spawn one)
|
||||
* - research: gathers external/codebase context (RESEARCH.md)
|
||||
* - execution: implements the plan or writes user-facing artifacts
|
||||
* - verification: checks correctness (VERIFICATION.md, audits)
|
||||
* - completion: post-execution wrap-up (no subagent today; reserved)
|
||||
*/
|
||||
const AGENT_TO_PHASE_TYPE = {
|
||||
// Planning — produces the plan / roadmap / pattern map
|
||||
'gsd-planner': 'planning',
|
||||
'gsd-roadmapper': 'planning',
|
||||
'gsd-pattern-mapper': 'planning',
|
||||
// Research — external/codebase information gathering
|
||||
'gsd-phase-researcher': 'research',
|
||||
'gsd-project-researcher': 'research',
|
||||
'gsd-research-synthesizer': 'research',
|
||||
'gsd-codebase-mapper': 'research',
|
||||
'gsd-ui-researcher': 'research',
|
||||
// Execution — implementation, debugging, doc writing
|
||||
'gsd-executor': 'execution',
|
||||
'gsd-debugger': 'execution',
|
||||
'gsd-doc-writer': 'execution',
|
||||
// Verification — correctness checks, audits, gap analysis
|
||||
'gsd-verifier': 'verification',
|
||||
'gsd-plan-checker': 'verification',
|
||||
'gsd-integration-checker': 'verification',
|
||||
'gsd-nyquist-auditor': 'verification',
|
||||
'gsd-ui-checker': 'verification',
|
||||
'gsd-ui-auditor': 'verification',
|
||||
'gsd-doc-verifier': 'verification',
|
||||
};
|
||||
|
||||
/**
|
||||
* The six phase-type slots accepted in `.planning/config.json` `models`
|
||||
* block. `discuss` and `completion` are reserved — no current agent maps
|
||||
* to them today — so users can pre-configure those slots without
|
||||
* breaking validation when an orchestrator starts honoring them.
|
||||
*/
|
||||
const VALID_PHASE_TYPES = new Set([
|
||||
'planning', 'discuss', 'research', 'execution', 'verification', 'completion',
|
||||
]);
|
||||
|
||||
/**
|
||||
* Formats the agent-to-model mapping as a human-readable table (in string format).
|
||||
*
|
||||
@@ -68,6 +123,8 @@ function getAgentToModelMapForProfile(normalizedProfile) {
|
||||
module.exports = {
|
||||
MODEL_PROFILES,
|
||||
VALID_PROFILES,
|
||||
AGENT_TO_PHASE_TYPE,
|
||||
VALID_PHASE_TYPES,
|
||||
formatAgentToModelMapAsTable,
|
||||
getAgentToModelMapForProfile,
|
||||
};
|
||||
|
||||
@@ -19,6 +19,53 @@ Model profiles control which Claude model each GSD agent uses. This allows balan
|
||||
| gsd-integration-checker | sonnet | sonnet | haiku | haiku | inherit |
|
||||
| gsd-nyquist-auditor | sonnet | sonnet | haiku | haiku | inherit |
|
||||
|
||||
## Per-Phase-Type Model Map (#3023)
|
||||
|
||||
`.planning/config.json` accepts a coarse per-**phase-type** map under the `models` key. Use this when you want tuning at the phase level ("Opus for planning and execution, Sonnet for the rest") without learning the agent taxonomy.
|
||||
|
||||
```json
|
||||
{
|
||||
"model_profile": "balanced",
|
||||
"models": {
|
||||
"planning": "opus",
|
||||
"discuss": "opus",
|
||||
"research": "sonnet",
|
||||
"execution": "opus",
|
||||
"verification": "sonnet",
|
||||
"completion": "sonnet"
|
||||
},
|
||||
"model_overrides": {
|
||||
"gsd-codebase-mapper": "haiku"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Phase-type → agent mapping
|
||||
|
||||
| Phase type | Agents |
|
||||
|---|---|
|
||||
| `planning` | gsd-planner, gsd-roadmapper, gsd-pattern-mapper |
|
||||
| `discuss` | (reserved — no subagent today) |
|
||||
| `research` | gsd-phase-researcher, gsd-project-researcher, gsd-research-synthesizer, gsd-codebase-mapper, gsd-ui-researcher |
|
||||
| `execution` | gsd-executor, gsd-debugger, gsd-doc-writer |
|
||||
| `verification` | gsd-verifier, gsd-plan-checker, gsd-integration-checker, gsd-nyquist-auditor, gsd-ui-checker, gsd-ui-auditor, gsd-doc-verifier |
|
||||
| `completion` | (reserved — no subagent today) |
|
||||
|
||||
### Resolution precedence (highest to lowest)
|
||||
|
||||
1. **Per-agent `model_overrides[agent]`** — full IDs accepted; targeted exceptions
|
||||
2. **Phase-type `models[phase_type]`** — tier alias only (`opus` / `sonnet` / `haiku` / `inherit`)
|
||||
3. **Profile table** — the per-agent column from the active `model_profile`
|
||||
4. **Runtime default** — when nothing else applies
|
||||
|
||||
### Why two layers above the profile?
|
||||
|
||||
- **Profile** is a global tier strategy (everyone runs balanced).
|
||||
- **`models`** is coarse phase-level tuning without learning agent names.
|
||||
- **`model_overrides`** is per-agent precision (e.g. force `haiku` on `gsd-codebase-mapper` for a fan-out).
|
||||
|
||||
The three layers compose: `models` defaults a phase, `model_overrides` carves an exception out of it.
|
||||
|
||||
## Profile Philosophy
|
||||
|
||||
**quality** - Maximum reasoning power
|
||||
@@ -93,10 +140,16 @@ Orchestrators resolve model before spawning:
|
||||
```
|
||||
1. Read .planning/config.json
|
||||
2. Check model_overrides for agent-specific override
|
||||
3. If no override, look up agent in profile table
|
||||
4. Pass model parameter to Task call
|
||||
3. If no override, check models[phase_type] for a phase-type tier
|
||||
(see §Per-Phase-Type Model Map for the agent → phase-type mapping)
|
||||
4. If no phase-type slot, look up agent in profile table
|
||||
5. Pass model parameter to Task call
|
||||
```
|
||||
|
||||
The same precedence applies to `reasoning_effort` resolution on runtimes
|
||||
that support it (Codex), so `model` and `reasoning_effort` always derive
|
||||
from the same tier source — a `models[phase_type]` override flips both.
|
||||
|
||||
## Per-Agent Overrides
|
||||
|
||||
Override specific agents without changing the entire profile:
|
||||
|
||||
Reference in New Issue
Block a user