feat(#3023): per-phase-type model map in .planning/config.json (#3030)

* feat(#3023): per-phase-type model map in .planning/config.json

Adds a new `models` block to .planning/config.json with six phase-type
slots (planning / discuss / research / execution / verification /
completion). Lets users express coarse tuning ("Opus for planning,
Sonnet for the rest") without learning the agent taxonomy.

Resolution precedence (highest → lowest):
  1. Per-agent `model_overrides[agent]`     (full IDs; targeted exception)
  2. Phase-type `models[phase_type]`         (NEW; tier alias)
  3. Profile table (`model_profile`)          (per-agent column)
  4. Runtime default

The three layers compose: `models` defaults a phase, `model_overrides`
carves an exception. Phase-type values are tier aliases (opus/sonnet/
haiku/inherit) so the runtime-resolution chain (#2517) stays correct
end-to-end without further branching.

Implementation:
- model-profiles.cjs: new AGENT_TO_PHASE_TYPE map + VALID_PHASE_TYPES
  set. Each agent in MODEL_PROFILES gets one phase-type assignment;
  tests assert coverage so adding a new agent without updating the
  table fails CI.
- core.cjs (resolveModelInternal): inserts phase-type tier lookup
  between per-agent override and profile-derived tier. Skips runtime
  resolution when the resolved tier is 'inherit' (was previously gated
  only on profile === 'inherit'; phase-type can now produce inherit
  independently).
- core.cjs (loadConfig): pass `parsed.models` through both code paths
  so resolveModelInternal can read it.
- config-schema.cjs + sdk/src/query/config-schema.ts: dynamic-pattern
  validator accepts only the six known phase-types; unknown slots
  rejected at config-set time.

Backward compat: configs without `models` behave exactly as today.

Tests (15 cases, all structural-IR — no stdout grep):
- Schema: AGENT_TO_PHASE_TYPE coverage, VALID_PHASE_TYPES exact match
- Resolver: phase-type alone; per-agent override beats phase-type;
  phase-type beats profile; issue's full example; "inherit"; empty
  block is no-op; no block is no-op
- Validation: each of the 6 slots accepted; unknown slot rejected;
  bare `models` (no slot) rejected

Verification:
- 15/15 pass on new regression test
- 6808/6808 full suite (5 net new), 0 fail
- lint-no-source-grep clean across 375 test files

Closes #3023

* docs(#3023): document `models` per-phase-type config in user-facing docs

Adds `models` block coverage to the three user-facing docs that ship
with each release:

1. docs/CONFIGURATION.md
   - New "Per-Phase-Type Models" section between "Per-Agent Overrides"
     and "Non-Claude Runtimes" with:
       * full example mixing models + model_overrides
       * phase-type → agent mapping table
       * resolution-precedence pseudocode
       * accepted values (tier alias only)
       * "When to use which" decision matrix
       * validation behavior + example error
   - Added `"models": {}` to the Full Schema snippet
   - Added a row for `models.<phase_type>` to the config keys table
     (next to model_profile_overrides for adjacency)

2. docs/FEATURES.md
   - Added a row for models.<phase_type> in the Configurable Settings
     table (right under model_profile)
   - Cross-link to CONFIGURATION.md for the full surface

3. docs/USER-GUIDE.md
   - New task-oriented "Tuning model cost by phase" section above
     "Using Non-Claude Runtimes" — leads with the concrete config
     and shows the override pattern (one-shot phase + targeted exception)
   - Cross-link to CONFIGURATION.md

Verification:
- 29/29 pass on config-schema-docs-parity + docs-update + new feature
  test (parity-check passes, so the config-schema entry I added in the
  feature commit is now matched by a docs row)
- 6808/6808 full suite pass
- lint-no-source-grep clean

Doc style follows the same pattern used by the existing model_profile,
model_overrides, and model_profile_overrides sections — example-led,
table-backed, cross-referenced. Each doc surfaces the feature at the
right depth (reference / settings table / task guide).

* fix(#3023): mirror phase-type tier in resolveReasoningEffortInternal (CR Major)

CodeRabbit caught a real Codex correctness bug + 3 minor docs/test issues:

1. **Major (outside-diff)** — resolveReasoningEffortInternal in core.cjs
   derived its tier exclusively from the profile table, ignoring the
   models.<phase_type> override added in #3023. Failure mode on Codex:

     Config: model_profile=balanced, models.execution=opus, agent=gsd-executor
     resolveModelInternal:           tier=opus    → gpt-5.4
     resolveReasoningEffortInternal: tier=sonnet  → reasoning_effort=medium
                                                     ↑
                                                  WRONG — should be xhigh
                                                  (opus tier on Codex)

   The runtime received a mismatched (model, effort) pair. Mirrored the
   phase-type lookup from resolveModelInternal so both functions derive
   from the same tier source. 'inherit' phase-type returns null effort
   (no runtime entry maps to 'inherit'; let runtime decide).

2. Minor — .changeset/per-phase-type-models.md `pr: TBD` → `pr: 3030`.

3. Minor (outside-diff) — model-profiles.md "Resolution Logic" section
   omitted the new phase-type tier. Updated the 4-step block to a 5-step
   block including `models[phase_type]` between override and profile,
   plus a paragraph noting that `model` and `reasoning_effort` derive
   from the same tier source.

4. Nitpick — added 2 typo-safety tests:
   - models.research = "haiku3" (typo) → falls through to profile
   - models.research = "openai/gpt-5" (full ID) → falls through to profile
   Plus 5 new reasoning_effort tests covering the Major fix:
   - exported correctly
   - phase-type override flips both model AND effort to same tier
   - inherit phase-type returns null effort
   - per-agent override still bypasses phase-type for effort
   - claude runtime ignores models.* (no effort propagation)

Verification:
- 24/24 pass on regression test (15 original + 2 typo-safety + 5 effort + 2 outside-diff related)
- 6815/6815 full suite (7 net new from 6808)
- lint-no-source-grep clean

The reasoning_effort tests are written semantically (phase-type override
must produce the SAME effort as a profile-only opus config) rather than
hard-coding tier-specific effort strings, so changes to the runtime tier
map don't break them.

* fix(#3023): phase-type override beats profile=inherit (CR Major round 2)

CodeRabbit caught another precedence inversion: when
  { model_profile: 'inherit', models: { execution: 'opus' } }
both resolvers short-circuited on `profile === 'inherit'` BEFORE the
phase-type override could be honored. Result: model returned 'inherit'
and reasoning_effort returned null — both contradicting the documented
precedence where models[phase_type] wins over model_profile.

Fix in resolveModelInternal:
- Compute tier from phase-type FIRST. If phase-type is a valid alias,
  it wins. Otherwise, fall back to profile-derived tier OR 'inherit'
  (when profile === 'inherit').
- Gate the runtime-resolution branch on `tier !== 'inherit'` (was
  `profile !== 'inherit'`) so phase-type=opus can flip runtime mapping
  on even when profile=inherit.
- Gate the inherit-return on `tier === 'inherit'` (was
  `profile === 'inherit'`).

Fix in resolveReasoningEffortInternal:
- Remove the `if (profile === 'inherit') return null;` early-return.
- Compute tier from phase-type first, fall back to profile. If
  phase-type is explicitly 'inherit' OR the resolved tier is 'inherit',
  return null (no runtime entry maps to inherit).

Tests added (5 new):
- model: phase-type wins over profile=inherit (with explicit opus, with
  haiku for one phase + planner-without-slot still inheriting)
- model: profile=inherit + no models block → all agents inherit (no
  regression on existing inherit semantics)
- model: profile=inherit + models block but agent has no slot → that
  agent inherits, agents with slots get phase-type tier
- effort: phase-type opus + profile=inherit → produces opus-tier
  effort, NOT null (the original bug)

Verification:
- 27/27 pass on regression test (24 prior + 3 model + 1 effort)
- 6820/6820 full suite (5 net new)
- lint-no-source-grep clean

The effort test reads the expected value by running a profile-only opus
config and comparing — semantic check, not hard-coded effort string. So
runtime tier map changes don't break the test.
This commit is contained in:
Tom Boucher
2026-05-02 13:19:15 -04:00
committed by GitHub
parent c9f5b7daac
commit d812c66020
10 changed files with 732 additions and 11 deletions

View File

@@ -84,6 +84,12 @@ const DYNAMIC_KEY_PATTERNS = [
// <runtime> is a free string (so users can map non-built-in runtimes); <tier> is enum-restricted.
{ topLevel: 'model_profile_overrides', test: (k) => /^model_profile_overrides\.[a-zA-Z0-9_-]+\.(opus|sonnet|haiku)$/.test(k),
description: 'model_profile_overrides.<runtime>.<opus|sonnet|haiku>' },
// #3023 — per-phase-type model map: models.<phase_type> = <tier>
// Six named slots (planning/discuss/research/execution/verification/completion);
// unknown phase-types are rejected. Per-agent model_overrides still take
// precedence over phase-type at resolve time.
{ topLevel: 'models', test: (k) => /^models\.(planning|discuss|research|execution|verification|completion)$/.test(k),
description: 'models.<planning|discuss|research|execution|verification|completion>' },
];
/**

View File

@@ -6,7 +6,7 @@ const fs = require('fs');
const os = require('os');
const path = require('path');
const { execSync, execFileSync, spawnSync } = require('child_process');
const { MODEL_PROFILES } = require('./model-profiles.cjs');
const { MODEL_PROFILES, AGENT_TO_PHASE_TYPE, VALID_PHASE_TYPES } = require('./model-profiles.cjs');
// Compatibility shim: new imports should use planning-workspace.cjs directly.
const {
planningDir,
@@ -500,6 +500,12 @@ function loadConfig(cwd) {
project_code: get('project_code') ?? defaults.project_code,
subagent_timeout: get('subagent_timeout', { section: 'workflow', field: 'subagent_timeout' }) ?? defaults.subagent_timeout,
model_overrides: parsed.model_overrides || null,
// #3023 — per-phase-type model map. Six named slots
// (planning/discuss/research/execution/verification/completion).
// Resolves between per-agent override and profile-derived tier in
// resolveModelInternal. Defaults to null so configs without it
// behave exactly as today.
models: parsed.models || null,
// #2517 — runtime-aware profiles. `runtime` defaults to null (back-compat).
// When null, resolveModelInternal preserves today's Claude-native behavior.
// NOTE: `runtime` and `model_profile_overrides` are intentionally read
@@ -560,6 +566,7 @@ function loadConfig(cwd) {
context_window: globalDefaults.context_window ?? defaults.context_window,
subagent_timeout: globalDefaults.subagent_timeout ?? defaults.subagent_timeout,
model_overrides: globalDefaults.model_overrides || null,
models: globalDefaults.models || null,
agent_skills: globalDefaults.agent_skills || {},
response_language: globalDefaults.response_language || null,
};
@@ -1518,10 +1525,37 @@ function resolveModelInternal(cwd, agentType) {
return override;
}
// 2. Compute the tier (opus/sonnet/haiku) for this agent under the active profile.
// 2. Compute the tier (opus/sonnet/haiku/inherit) for this agent.
//
// #3023: phase-type slot can override the profile-derived tier.
// Precedence: per-agent override (above) > phase-type slot > profile.
// Phase-type values are tier aliases (opus/sonnet/haiku/inherit) — same
// shape as model_profile output — so the runtime-resolution chain
// (step 3), resolve_model_ids handling (step 4), and profile lookup
// (step 5) all stay correct without further branching.
const profile = String(config.model_profile || 'balanced').toLowerCase();
const agentModels = MODEL_PROFILES[agentType];
const tier = agentModels ? (agentModels[profile] || agentModels['balanced']) : null;
const phaseType = AGENT_TO_PHASE_TYPE[agentType];
const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object')
? config.models[phaseType]
: undefined;
// Only honor phase-type tier if it's one of the recognized aliases.
// Anything else falls through to profile lookup so a typo doesn't
// silently break tier resolution.
const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku', 'inherit']);
// Resolve tier: phase-type wins when valid; else profile-derived; else
// (when profile === 'inherit') propagate inherit so the later short-
// circuit fires. CR Major (#3030): a config like
// { model_profile: 'inherit', models: { execution: 'opus' } }
// must honor the phase-type opus, not return 'inherit'. Synthesizing
// tier='inherit' only when there's no phase-type override keeps the
// original inherit semantics intact while letting a valid phase-type
// tier win.
const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier))
? phaseTypeTier
: (profile === 'inherit'
? 'inherit'
: (agentModels ? (agentModels[profile] || agentModels['balanced']) : null));
// 3. Runtime-aware resolution (#2517) — only when `runtime` is explicitly set
// to a non-Claude runtime. `runtime: "claude"` is the implicit default and is
@@ -1529,8 +1563,10 @@ function resolveModelInternal(cwd, agentType) {
// "omit"` (review finding #4). Deliberate ordering for non-Claude runtimes:
// explicit opt-in beats `resolve_model_ids: "omit"` so users on Codex installs
// that auto-set "omit" can still flip on tiered behavior by setting runtime
// alone. inherit profile is preserved verbatim.
if (config.runtime && config.runtime !== 'claude' && profile !== 'inherit' && tier) {
// alone. Gate on tier !== 'inherit' (not profile !== 'inherit') so a
// valid phase-type tier flips runtime resolution on even when the
// profile is inherit.
if (config.runtime && config.runtime !== 'claude' && tier && tier !== 'inherit') {
const entry = _resolveRuntimeTier(config, tier);
if (entry?.model) return entry.model;
// Unknown runtime with no user-supplied overrides — fall through to Claude-safe
@@ -1546,7 +1582,9 @@ function resolveModelInternal(cwd, agentType) {
// 5. Profile lookup (Claude-native default).
if (!agentModels) return 'sonnet';
if (profile === 'inherit') return 'inherit';
// Gate on tier (not profile) so a valid phase-type override beats
// profile=inherit (#3030 CR Major).
if (tier === 'inherit') return 'inherit';
// `tier` is guaranteed truthy here: agentModels exists, and MODEL_PROFILES
// entries always define `balanced`, so `agentModels[profile] || agentModels.balanced`
// resolves to a string. Keep the local for readability — no defensive fallback.
@@ -1586,11 +1624,38 @@ function resolveReasoningEffortInternal(cwd, agentType) {
if (config.model_overrides?.[agentType]) return null;
const profile = String(config.model_profile || 'balanced').toLowerCase();
if (profile === 'inherit') return null;
const agentModels = MODEL_PROFILES[agentType];
if (!agentModels) return null;
const tier = agentModels[profile] || agentModels['balanced'];
if (!tier) return null;
// #3023 (CR Major): mirror the phase-type tier lookup from
// resolveModelInternal. Without this, `model` and `reasoning_effort`
// derive from different tier sources on Codex when models.<phase_type>
// overrides the profile.
//
// #3030 CR follow-up: do NOT short-circuit on profile === 'inherit'
// before reading the phase-type tier. A config like
// { model_profile: 'inherit', models: { execution: 'opus' } }
// must produce the opus runtime effort, not null. Compute tier from
// phase-type first; only fall back to profile when there's no valid
// phase-type override; only return null when the resolved tier is
// 'inherit' or unknown.
const phaseType = AGENT_TO_PHASE_TYPE[agentType];
const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object')
? config.models[phaseType]
: undefined;
// Explicit phase-type 'inherit' is the user opting out of tier-based
// effort for this phase — return null instead of falling through to
// profile (which would silently emit the profile's effort and
// contradict the user's choice).
if (phaseTypeTier === 'inherit') return null;
const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku']);
const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier))
? phaseTypeTier
: (profile === 'inherit'
? 'inherit'
: (agentModels[profile] || agentModels['balanced']));
// 'inherit' (from profile fallback) yields no runtime effort.
if (!tier || tier === 'inherit') return null;
const entry = _resolveRuntimeTier(config, tier);
return entry?.reasoning_effort || null;

View File

@@ -28,6 +28,61 @@ const MODEL_PROFILES = {
};
const VALID_PROFILES = [...Object.keys(MODEL_PROFILES['gsd-planner']), 'inherit'];
/**
* #3023 — Phase-type → agent mapping table.
*
* Lets users tune model selection at the *phase-type* level (planning,
* research, execution, verification, ...) instead of per-agent. Maps
* each agent in MODEL_PROFILES to exactly one phase-type so resolution
* is deterministic.
*
* Adding a new agent to MODEL_PROFILES requires adding an entry here too;
* tests/feat-3023-phase-type-models.test.cjs asserts coverage.
*
* Phase-type semantics:
* - planning: produces the plan (PLAN.md, ROADMAP.md, patterns)
* - discuss: collaborative scoping (no subagent today; reserved
* for orchestrators that may spawn one)
* - research: gathers external/codebase context (RESEARCH.md)
* - execution: implements the plan or writes user-facing artifacts
* - verification: checks correctness (VERIFICATION.md, audits)
* - completion: post-execution wrap-up (no subagent today; reserved)
*/
const AGENT_TO_PHASE_TYPE = {
// Planning — produces the plan / roadmap / pattern map
'gsd-planner': 'planning',
'gsd-roadmapper': 'planning',
'gsd-pattern-mapper': 'planning',
// Research — external/codebase information gathering
'gsd-phase-researcher': 'research',
'gsd-project-researcher': 'research',
'gsd-research-synthesizer': 'research',
'gsd-codebase-mapper': 'research',
'gsd-ui-researcher': 'research',
// Execution — implementation, debugging, doc writing
'gsd-executor': 'execution',
'gsd-debugger': 'execution',
'gsd-doc-writer': 'execution',
// Verification — correctness checks, audits, gap analysis
'gsd-verifier': 'verification',
'gsd-plan-checker': 'verification',
'gsd-integration-checker': 'verification',
'gsd-nyquist-auditor': 'verification',
'gsd-ui-checker': 'verification',
'gsd-ui-auditor': 'verification',
'gsd-doc-verifier': 'verification',
};
/**
* The six phase-type slots accepted in `.planning/config.json` `models`
* block. `discuss` and `completion` are reserved — no current agent maps
* to them today — so users can pre-configure those slots without
* breaking validation when an orchestrator starts honoring them.
*/
const VALID_PHASE_TYPES = new Set([
'planning', 'discuss', 'research', 'execution', 'verification', 'completion',
]);
/**
* Formats the agent-to-model mapping as a human-readable table (in string format).
*
@@ -68,6 +123,8 @@ function getAgentToModelMapForProfile(normalizedProfile) {
module.exports = {
MODEL_PROFILES,
VALID_PROFILES,
AGENT_TO_PHASE_TYPE,
VALID_PHASE_TYPES,
formatAgentToModelMapAsTable,
getAgentToModelMapForProfile,
};

View File

@@ -19,6 +19,53 @@ Model profiles control which Claude model each GSD agent uses. This allows balan
| gsd-integration-checker | sonnet | sonnet | haiku | haiku | inherit |
| gsd-nyquist-auditor | sonnet | sonnet | haiku | haiku | inherit |
## Per-Phase-Type Model Map (#3023)
`.planning/config.json` accepts a coarse per-**phase-type** map under the `models` key. Use this when you want tuning at the phase level ("Opus for planning and execution, Sonnet for the rest") without learning the agent taxonomy.
```json
{
"model_profile": "balanced",
"models": {
"planning": "opus",
"discuss": "opus",
"research": "sonnet",
"execution": "opus",
"verification": "sonnet",
"completion": "sonnet"
},
"model_overrides": {
"gsd-codebase-mapper": "haiku"
}
}
```
### Phase-type → agent mapping
| Phase type | Agents |
|---|---|
| `planning` | gsd-planner, gsd-roadmapper, gsd-pattern-mapper |
| `discuss` | (reserved — no subagent today) |
| `research` | gsd-phase-researcher, gsd-project-researcher, gsd-research-synthesizer, gsd-codebase-mapper, gsd-ui-researcher |
| `execution` | gsd-executor, gsd-debugger, gsd-doc-writer |
| `verification` | gsd-verifier, gsd-plan-checker, gsd-integration-checker, gsd-nyquist-auditor, gsd-ui-checker, gsd-ui-auditor, gsd-doc-verifier |
| `completion` | (reserved — no subagent today) |
### Resolution precedence (highest to lowest)
1. **Per-agent `model_overrides[agent]`** — full IDs accepted; targeted exceptions
2. **Phase-type `models[phase_type]`** — tier alias only (`opus` / `sonnet` / `haiku` / `inherit`)
3. **Profile table** — the per-agent column from the active `model_profile`
4. **Runtime default** — when nothing else applies
### Why two layers above the profile?
- **Profile** is a global tier strategy (everyone runs balanced).
- **`models`** is coarse phase-level tuning without learning agent names.
- **`model_overrides`** is per-agent precision (e.g. force `haiku` on `gsd-codebase-mapper` for a fan-out).
The three layers compose: `models` defaults a phase, `model_overrides` carves an exception out of it.
## Profile Philosophy
**quality** - Maximum reasoning power
@@ -93,10 +140,16 @@ Orchestrators resolve model before spawning:
```
1. Read .planning/config.json
2. Check model_overrides for agent-specific override
3. If no override, look up agent in profile table
4. Pass model parameter to Task call
3. If no override, check models[phase_type] for a phase-type tier
(see §Per-Phase-Type Model Map for the agent → phase-type mapping)
4. If no phase-type slot, look up agent in profile table
5. Pass model parameter to Task call
```
The same precedence applies to `reasoning_effort` resolution on runtimes
that support it (Codex), so `model` and `reasoning_effort` always derive
from the same tier source — a `models[phase_type]` override flips both.
## Per-Agent Overrides
Override specific agents without changing the entire profile: