* feat(#3023): per-phase-type model map in .planning/config.json Adds a new `models` block to .planning/config.json with six phase-type slots (planning / discuss / research / execution / verification / completion). Lets users express coarse tuning ("Opus for planning, Sonnet for the rest") without learning the agent taxonomy. Resolution precedence (highest → lowest): 1. Per-agent `model_overrides[agent]` (full IDs; targeted exception) 2. Phase-type `models[phase_type]` (NEW; tier alias) 3. Profile table (`model_profile`) (per-agent column) 4. Runtime default The three layers compose: `models` defaults a phase, `model_overrides` carves an exception. Phase-type values are tier aliases (opus/sonnet/ haiku/inherit) so the runtime-resolution chain (#2517) stays correct end-to-end without further branching. Implementation: - model-profiles.cjs: new AGENT_TO_PHASE_TYPE map + VALID_PHASE_TYPES set. Each agent in MODEL_PROFILES gets one phase-type assignment; tests assert coverage so adding a new agent without updating the table fails CI. - core.cjs (resolveModelInternal): inserts phase-type tier lookup between per-agent override and profile-derived tier. Skips runtime resolution when the resolved tier is 'inherit' (was previously gated only on profile === 'inherit'; phase-type can now produce inherit independently). - core.cjs (loadConfig): pass `parsed.models` through both code paths so resolveModelInternal can read it. - config-schema.cjs + sdk/src/query/config-schema.ts: dynamic-pattern validator accepts only the six known phase-types; unknown slots rejected at config-set time. Backward compat: configs without `models` behave exactly as today. Tests (15 cases, all structural-IR — no stdout grep): - Schema: AGENT_TO_PHASE_TYPE coverage, VALID_PHASE_TYPES exact match - Resolver: phase-type alone; per-agent override beats phase-type; phase-type beats profile; issue's full example; "inherit"; empty block is no-op; no block is no-op - Validation: each of the 6 slots accepted; unknown slot rejected; bare `models` (no slot) rejected Verification: - 15/15 pass on new regression test - 6808/6808 full suite (5 net new), 0 fail - lint-no-source-grep clean across 375 test files Closes #3023 * docs(#3023): document `models` per-phase-type config in user-facing docs Adds `models` block coverage to the three user-facing docs that ship with each release: 1. docs/CONFIGURATION.md - New "Per-Phase-Type Models" section between "Per-Agent Overrides" and "Non-Claude Runtimes" with: * full example mixing models + model_overrides * phase-type → agent mapping table * resolution-precedence pseudocode * accepted values (tier alias only) * "When to use which" decision matrix * validation behavior + example error - Added `"models": {}` to the Full Schema snippet - Added a row for `models.<phase_type>` to the config keys table (next to model_profile_overrides for adjacency) 2. docs/FEATURES.md - Added a row for models.<phase_type> in the Configurable Settings table (right under model_profile) - Cross-link to CONFIGURATION.md for the full surface 3. docs/USER-GUIDE.md - New task-oriented "Tuning model cost by phase" section above "Using Non-Claude Runtimes" — leads with the concrete config and shows the override pattern (one-shot phase + targeted exception) - Cross-link to CONFIGURATION.md Verification: - 29/29 pass on config-schema-docs-parity + docs-update + new feature test (parity-check passes, so the config-schema entry I added in the feature commit is now matched by a docs row) - 6808/6808 full suite pass - lint-no-source-grep clean Doc style follows the same pattern used by the existing model_profile, model_overrides, and model_profile_overrides sections — example-led, table-backed, cross-referenced. Each doc surfaces the feature at the right depth (reference / settings table / task guide). * fix(#3023): mirror phase-type tier in resolveReasoningEffortInternal (CR Major) CodeRabbit caught a real Codex correctness bug + 3 minor docs/test issues: 1. **Major (outside-diff)** — resolveReasoningEffortInternal in core.cjs derived its tier exclusively from the profile table, ignoring the models.<phase_type> override added in #3023. Failure mode on Codex: Config: model_profile=balanced, models.execution=opus, agent=gsd-executor resolveModelInternal: tier=opus → gpt-5.4 resolveReasoningEffortInternal: tier=sonnet → reasoning_effort=medium ↑ WRONG — should be xhigh (opus tier on Codex) The runtime received a mismatched (model, effort) pair. Mirrored the phase-type lookup from resolveModelInternal so both functions derive from the same tier source. 'inherit' phase-type returns null effort (no runtime entry maps to 'inherit'; let runtime decide). 2. Minor — .changeset/per-phase-type-models.md `pr: TBD` → `pr: 3030`. 3. Minor (outside-diff) — model-profiles.md "Resolution Logic" section omitted the new phase-type tier. Updated the 4-step block to a 5-step block including `models[phase_type]` between override and profile, plus a paragraph noting that `model` and `reasoning_effort` derive from the same tier source. 4. Nitpick — added 2 typo-safety tests: - models.research = "haiku3" (typo) → falls through to profile - models.research = "openai/gpt-5" (full ID) → falls through to profile Plus 5 new reasoning_effort tests covering the Major fix: - exported correctly - phase-type override flips both model AND effort to same tier - inherit phase-type returns null effort - per-agent override still bypasses phase-type for effort - claude runtime ignores models.* (no effort propagation) Verification: - 24/24 pass on regression test (15 original + 2 typo-safety + 5 effort + 2 outside-diff related) - 6815/6815 full suite (7 net new from 6808) - lint-no-source-grep clean The reasoning_effort tests are written semantically (phase-type override must produce the SAME effort as a profile-only opus config) rather than hard-coding tier-specific effort strings, so changes to the runtime tier map don't break them. * fix(#3023): phase-type override beats profile=inherit (CR Major round 2) CodeRabbit caught another precedence inversion: when { model_profile: 'inherit', models: { execution: 'opus' } } both resolvers short-circuited on `profile === 'inherit'` BEFORE the phase-type override could be honored. Result: model returned 'inherit' and reasoning_effort returned null — both contradicting the documented precedence where models[phase_type] wins over model_profile. Fix in resolveModelInternal: - Compute tier from phase-type FIRST. If phase-type is a valid alias, it wins. Otherwise, fall back to profile-derived tier OR 'inherit' (when profile === 'inherit'). - Gate the runtime-resolution branch on `tier !== 'inherit'` (was `profile !== 'inherit'`) so phase-type=opus can flip runtime mapping on even when profile=inherit. - Gate the inherit-return on `tier === 'inherit'` (was `profile === 'inherit'`). Fix in resolveReasoningEffortInternal: - Remove the `if (profile === 'inherit') return null;` early-return. - Compute tier from phase-type first, fall back to profile. If phase-type is explicitly 'inherit' OR the resolved tier is 'inherit', return null (no runtime entry maps to inherit). Tests added (5 new): - model: phase-type wins over profile=inherit (with explicit opus, with haiku for one phase + planner-without-slot still inheriting) - model: profile=inherit + no models block → all agents inherit (no regression on existing inherit semantics) - model: profile=inherit + models block but agent has no slot → that agent inherits, agents with slots get phase-type tier - effort: phase-type opus + profile=inherit → produces opus-tier effort, NOT null (the original bug) Verification: - 27/27 pass on regression test (24 prior + 3 model + 1 effort) - 6820/6820 full suite (5 net new) - lint-no-source-grep clean The effort test reads the expected value by running a profile-only opus config and comparing — semantic check, not hard-coded effort string. So runtime tier map changes don't break the test.
This commit is contained in:
Tom Boucher
5
.changeset/per-phase-type-models.md
Normal file
5
.changeset/per-phase-type-models.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Added
|
||||
pr: 3030
|
||||
---
|
||||
**`models` block in `.planning/config.json` for per-phase-type model selection (#3023).** A new resolution layer between per-agent `model_overrides` and the `model_profile` tier table. Six named slots (`planning` / `discuss` / `research` / `execution` / `verification` / `completion`) accept tier aliases (`opus` / `sonnet` / `haiku` / `inherit`). Lets you express "Opus for planning, Sonnet for the rest" in two lines without learning the agent taxonomy. Fully backward compatible — configs without `models` behave exactly as today.
|
||||
@@ -16,6 +16,7 @@ GSD stores project settings in `.planning/config.json`. Created during `/gsd-new
|
||||
"granularity": "standard",
|
||||
"model_profile": "balanced",
|
||||
"model_overrides": {},
|
||||
"models": {},
|
||||
"planning": {
|
||||
"commit_docs": true,
|
||||
"search_gitignored": false,
|
||||
@@ -117,6 +118,7 @@ GSD stores project settings in `.planning/config.json`. Created during `/gsd-new
|
||||
| `model_profile` | enum | `quality`, `balanced`, `budget`, `adaptive`, `inherit` | `balanced` | Model tier for each agent (see [Model Profiles](#model-profiles)). `adaptive` was added per [#1713](https://github.com/gsd-build/get-shit-done/issues/1713) / [#1806](https://github.com/gsd-build/get-shit-done/issues/1806) and resolves the same way as the other tiers under runtime-aware profiles. |
|
||||
| `runtime` | string | `claude`, `codex`, or any string | (none) | Active runtime for [runtime-aware profile resolution](#runtime-aware-profiles-2517). When set, profile tiers (opus/sonnet/haiku) resolve to runtime-native model IDs. Today only the Codex install path emits per-agent model IDs from this resolver; other runtimes (`opencode`, `gemini`, `qwen`, `copilot`, …) consume the resolver at spawn time and gain dedicated install-path support in [#2612](https://github.com/gsd-build/get-shit-done/issues/2612). When unset (default), behavior is unchanged from prior versions. Added in v1.39 |
|
||||
| `model_profile_overrides.<runtime>.<tier>` | string \| object | per-runtime tier override | (none) | Override the runtime-aware tier mapping for a specific `(runtime, tier)`. Tier is one of `opus`, `sonnet`, `haiku`. Value is either a model ID string (e.g. `"gpt-5-pro"`) or `{ model, reasoning_effort }`. See [Runtime-Aware Profiles](#runtime-aware-profiles-2517). Added in v1.39 |
|
||||
| `models.<phase_type>` | enum | `opus`, `sonnet`, `haiku`, `inherit` | (none) | Per-phase-type model tier. Six accepted slots: `planning`, `discuss`, `research`, `execution`, `verification`, `completion`. Lets you tune at the phase level ("Opus for planning, Sonnet for the rest") without learning agent names. Resolves between `model_overrides` (higher) and `model_profile` (lower); see [Per-Phase-Type Models](#per-phase-type-models-models--added-in-v140). Added in v1.40 ([#3023](https://github.com/gsd-build/get-shit-done/pull/3030)) |
|
||||
| `project_code` | string | any short string | (none) | Prefix for phase directory names (e.g., `"ABC"` produces `ABC-01-setup/`). Added in v1.31 |
|
||||
| `response_language` | string | language code | (none) | Language for agent responses (e.g., `"pt"`, `"ko"`, `"ja"`). Propagates to all spawned agents for cross-phase language consistency. Added in v1.32 |
|
||||
| `context_window` | number | any integer | `200000` | Context window size in tokens. Set `1000000` for 1M-context models (e.g., `claude-opus-4-7[1m]`). Values `>= 500000` enable adaptive context enrichment (full-body reads of prior SUMMARY.md, deeper anti-pattern reads). Configured via `/gsd-settings-advanced`. |
|
||||
@@ -643,6 +645,89 @@ OpenCode's `task` interface do not accept an inline `model` parameter, so
|
||||
running `gsd install <runtime>` after editing `model_overrides` is required
|
||||
for the change to take effect. See issue #2256.
|
||||
|
||||
### Per-Phase-Type Models (`models`) — added in v1.40
|
||||
|
||||
> Express tuning at the **phase** level (planning, research, execution, verification) without learning the agent taxonomy. Added in [#3023](https://github.com/gsd-build/get-shit-done/pull/3030).
|
||||
|
||||
`model_overrides` is per-**agent** (precise but verbose; you have to know that `gsd-codebase-mapper` is research and `gsd-doc-writer` is execution). The `models` block lets you say "Opus for planning and execution, Sonnet for the rest" in two lines:
|
||||
|
||||
```json
|
||||
{
|
||||
"model_profile": "balanced",
|
||||
"models": {
|
||||
"planning": "opus",
|
||||
"discuss": "opus",
|
||||
"research": "sonnet",
|
||||
"execution": "opus",
|
||||
"verification": "sonnet",
|
||||
"completion": "sonnet"
|
||||
},
|
||||
"model_overrides": {
|
||||
"gsd-codebase-mapper": "haiku"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Phase-type → agent mapping
|
||||
|
||||
| Phase type | Agents |
|
||||
|---|---|
|
||||
| `planning` | `gsd-planner`, `gsd-roadmapper`, `gsd-pattern-mapper` |
|
||||
| `discuss` | (reserved — no subagent today) |
|
||||
| `research` | `gsd-phase-researcher`, `gsd-project-researcher`, `gsd-research-synthesizer`, `gsd-codebase-mapper`, `gsd-ui-researcher` |
|
||||
| `execution` | `gsd-executor`, `gsd-debugger`, `gsd-doc-writer` |
|
||||
| `verification` | `gsd-verifier`, `gsd-plan-checker`, `gsd-integration-checker`, `gsd-nyquist-auditor`, `gsd-ui-checker`, `gsd-ui-auditor`, `gsd-doc-verifier` |
|
||||
| `completion` | (reserved — no subagent today) |
|
||||
|
||||
`discuss` and `completion` are accepted by the schema for forward compatibility; setting them today is a no-op until a subagent maps to them.
|
||||
|
||||
#### Resolution precedence (highest → lowest)
|
||||
|
||||
```
|
||||
1. model_overrides[<agent>] ← per-agent; full IDs accepted; targeted exception
|
||||
2. models[<phase_type>] ← coarse phase-level tier
|
||||
3. model_profile (per-agent col) ← global tier strategy
|
||||
4. Runtime default ← when nothing else applies
|
||||
```
|
||||
|
||||
The three layers compose: `models` defaults a phase, and `model_overrides` carves an exception out of it. In the example above, all five research agents resolve to `sonnet` *except* `gsd-codebase-mapper`, which the per-agent override pins to `haiku`.
|
||||
|
||||
#### Accepted values
|
||||
|
||||
`models.<phase_type>` accepts only tier aliases:
|
||||
|
||||
| Value | Effect |
|
||||
|---|---|
|
||||
| `"opus"` / `"sonnet"` / `"haiku"` | Standard tier — runtime resolution maps to the active runtime's model for that tier |
|
||||
| `"inherit"` | Agents in this phase follow the session model (same semantics as `model_profile: "inherit"`) |
|
||||
|
||||
If you need a fully-qualified model ID (`"openai/gpt-5"`, `"google/gemini-2.5-pro"`), use `model_overrides` per agent instead. `models.*` is intentionally tier-only so the runtime-aware mapping stays correct on Codex / OpenCode / Gemini CLI installs.
|
||||
|
||||
#### When to use which
|
||||
|
||||
| You want | Use |
|
||||
|---|---|
|
||||
| One global tier strategy ("balanced everywhere") | `model_profile` |
|
||||
| Coarse phase-level tuning ("Opus for planning") | `models.<phase_type>` |
|
||||
| Per-agent precision ("force haiku on the codebase mapper") | `model_overrides[<agent>]` |
|
||||
| Full model ID for a specific agent | `model_overrides[<agent>]: "openai/gpt-5"` |
|
||||
|
||||
Mix freely — the precedence rule above resolves any overlap deterministically.
|
||||
|
||||
#### Validation
|
||||
|
||||
`config-set` rejects unknown phase-types:
|
||||
|
||||
```bash
|
||||
$ gsd config-set models.deployment opus
|
||||
Error: 'models.deployment' is not a valid config key
|
||||
|
||||
# Valid:
|
||||
$ gsd config-set models.research sonnet
|
||||
```
|
||||
|
||||
Direct edits to `.planning/config.json` are looser — the resolver simply ignores values it doesn't recognize and falls through to the profile tier — so a typo doesn't silently break tier resolution.
|
||||
|
||||
### Non-Claude Runtimes (Codex, OpenCode, Gemini CLI, Kilo)
|
||||
|
||||
When GSD is installed for a non-Claude runtime, the installer automatically sets `resolve_model_ids: "omit"` in `~/.gsd/defaults.json`. This causes GSD to return an empty model parameter for all agents, so each agent uses whatever model the runtime is configured with. No additional setup is needed for the default case.
|
||||
|
||||
@@ -923,6 +923,7 @@ continues. Drift detection cannot fail verification.
|
||||
| `mode` | enum | `interactive` | `interactive` or `yolo` (auto-approve) |
|
||||
| `granularity` | enum | `standard` | `coarse`, `standard`, or `fine` |
|
||||
| `model_profile` | enum | `balanced` | `quality`, `balanced`, `budget`, or `inherit` |
|
||||
| `models.<phase_type>` | enum | (none) | Per-phase-type tier override (`planning`, `discuss`, `research`, `execution`, `verification`, `completion`). Values: `opus`, `sonnet`, `haiku`, `inherit`. Coarse phase-level tuning that wins over `model_profile` but loses to per-agent `model_overrides`. See [CONFIGURATION.md](CONFIGURATION.md#per-phase-type-models-models--added-in-v140). Added in v1.40 |
|
||||
| `workflow.research` | boolean | `true` | Domain research before planning |
|
||||
| `workflow.plan_check` | boolean | `true` | Plan verification loop |
|
||||
| `workflow.verifier` | boolean | `true` | Post-execution verification |
|
||||
|
||||
@@ -1012,6 +1012,41 @@ Do not re-run `/gsd-execute-phase`. Use `/gsd-quick` for targeted fixes, or `/gs
|
||||
|
||||
Switch to budget profile: `/gsd-set-profile budget`. Disable research and plan-check agents via `/gsd-settings` if the domain is familiar to you (or to Claude).
|
||||
|
||||
### Tuning model cost by phase (`models`) — added in v1.40
|
||||
|
||||
If you've heard "use Opus for planning, Sonnet for verification" and want to apply that without learning the agent taxonomy, add a `models` block to `.planning/config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"model_profile": "balanced",
|
||||
"models": {
|
||||
"planning": "opus",
|
||||
"discuss": "opus",
|
||||
"research": "sonnet",
|
||||
"execution": "opus",
|
||||
"verification": "sonnet",
|
||||
"completion": "sonnet"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The six slots (`planning` / `discuss` / `research` / `execution` / `verification` / `completion`) accept tier aliases (`opus`, `sonnet`, `haiku`, `inherit`). Each slot covers a group of agents — for example, setting `models.research = "sonnet"` applies to `gsd-phase-researcher`, `gsd-codebase-mapper`, `gsd-research-synthesizer`, and the other research agents in one shot.
|
||||
|
||||
Need a per-agent exception? Add `model_overrides` alongside — it wins over `models`:
|
||||
|
||||
```json
|
||||
{
|
||||
"models": { "research": "sonnet" },
|
||||
"model_overrides": {
|
||||
"gsd-codebase-mapper": "haiku"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
That gives sonnet to all research agents *except* the codebase mapper, which runs haiku for the cheap-but-broad fan-out scan.
|
||||
|
||||
For the full mapping table and resolution-precedence rules, see [Per-Phase-Type Models](CONFIGURATION.md#per-phase-type-models-models--added-in-v140) in the configuration reference.
|
||||
|
||||
### Using Non-Claude Runtimes (Codex, OpenCode, Gemini CLI, Kilo)
|
||||
|
||||
If you installed GSD for a non-Claude runtime, the installer already configured model resolution so all agents use the runtime's default model. No manual setup is needed. Specifically, the installer sets `resolve_model_ids: "omit"` in your config, which tells GSD to skip Anthropic model ID resolution and let the runtime choose its own default model.
|
||||
|
||||
@@ -84,6 +84,12 @@ const DYNAMIC_KEY_PATTERNS = [
|
||||
// <runtime> is a free string (so users can map non-built-in runtimes); <tier> is enum-restricted.
|
||||
{ topLevel: 'model_profile_overrides', test: (k) => /^model_profile_overrides\.[a-zA-Z0-9_-]+\.(opus|sonnet|haiku)$/.test(k),
|
||||
description: 'model_profile_overrides.<runtime>.<opus|sonnet|haiku>' },
|
||||
// #3023 — per-phase-type model map: models.<phase_type> = <tier>
|
||||
// Six named slots (planning/discuss/research/execution/verification/completion);
|
||||
// unknown phase-types are rejected. Per-agent model_overrides still take
|
||||
// precedence over phase-type at resolve time.
|
||||
{ topLevel: 'models', test: (k) => /^models\.(planning|discuss|research|execution|verification|completion)$/.test(k),
|
||||
description: 'models.<planning|discuss|research|execution|verification|completion>' },
|
||||
];
|
||||
|
||||
/**
|
||||
|
||||
@@ -6,7 +6,7 @@ const fs = require('fs');
|
||||
const os = require('os');
|
||||
const path = require('path');
|
||||
const { execSync, execFileSync, spawnSync } = require('child_process');
|
||||
const { MODEL_PROFILES } = require('./model-profiles.cjs');
|
||||
const { MODEL_PROFILES, AGENT_TO_PHASE_TYPE, VALID_PHASE_TYPES } = require('./model-profiles.cjs');
|
||||
// Compatibility shim: new imports should use planning-workspace.cjs directly.
|
||||
const {
|
||||
planningDir,
|
||||
@@ -500,6 +500,12 @@ function loadConfig(cwd) {
|
||||
project_code: get('project_code') ?? defaults.project_code,
|
||||
subagent_timeout: get('subagent_timeout', { section: 'workflow', field: 'subagent_timeout' }) ?? defaults.subagent_timeout,
|
||||
model_overrides: parsed.model_overrides || null,
|
||||
// #3023 — per-phase-type model map. Six named slots
|
||||
// (planning/discuss/research/execution/verification/completion).
|
||||
// Resolves between per-agent override and profile-derived tier in
|
||||
// resolveModelInternal. Defaults to null so configs without it
|
||||
// behave exactly as today.
|
||||
models: parsed.models || null,
|
||||
// #2517 — runtime-aware profiles. `runtime` defaults to null (back-compat).
|
||||
// When null, resolveModelInternal preserves today's Claude-native behavior.
|
||||
// NOTE: `runtime` and `model_profile_overrides` are intentionally read
|
||||
@@ -560,6 +566,7 @@ function loadConfig(cwd) {
|
||||
context_window: globalDefaults.context_window ?? defaults.context_window,
|
||||
subagent_timeout: globalDefaults.subagent_timeout ?? defaults.subagent_timeout,
|
||||
model_overrides: globalDefaults.model_overrides || null,
|
||||
models: globalDefaults.models || null,
|
||||
agent_skills: globalDefaults.agent_skills || {},
|
||||
response_language: globalDefaults.response_language || null,
|
||||
};
|
||||
@@ -1518,10 +1525,37 @@ function resolveModelInternal(cwd, agentType) {
|
||||
return override;
|
||||
}
|
||||
|
||||
// 2. Compute the tier (opus/sonnet/haiku) for this agent under the active profile.
|
||||
// 2. Compute the tier (opus/sonnet/haiku/inherit) for this agent.
|
||||
//
|
||||
// #3023: phase-type slot can override the profile-derived tier.
|
||||
// Precedence: per-agent override (above) > phase-type slot > profile.
|
||||
// Phase-type values are tier aliases (opus/sonnet/haiku/inherit) — same
|
||||
// shape as model_profile output — so the runtime-resolution chain
|
||||
// (step 3), resolve_model_ids handling (step 4), and profile lookup
|
||||
// (step 5) all stay correct without further branching.
|
||||
const profile = String(config.model_profile || 'balanced').toLowerCase();
|
||||
const agentModels = MODEL_PROFILES[agentType];
|
||||
const tier = agentModels ? (agentModels[profile] || agentModels['balanced']) : null;
|
||||
const phaseType = AGENT_TO_PHASE_TYPE[agentType];
|
||||
const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object')
|
||||
? config.models[phaseType]
|
||||
: undefined;
|
||||
// Only honor phase-type tier if it's one of the recognized aliases.
|
||||
// Anything else falls through to profile lookup so a typo doesn't
|
||||
// silently break tier resolution.
|
||||
const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku', 'inherit']);
|
||||
// Resolve tier: phase-type wins when valid; else profile-derived; else
|
||||
// (when profile === 'inherit') propagate inherit so the later short-
|
||||
// circuit fires. CR Major (#3030): a config like
|
||||
// { model_profile: 'inherit', models: { execution: 'opus' } }
|
||||
// must honor the phase-type opus, not return 'inherit'. Synthesizing
|
||||
// tier='inherit' only when there's no phase-type override keeps the
|
||||
// original inherit semantics intact while letting a valid phase-type
|
||||
// tier win.
|
||||
const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier))
|
||||
? phaseTypeTier
|
||||
: (profile === 'inherit'
|
||||
? 'inherit'
|
||||
: (agentModels ? (agentModels[profile] || agentModels['balanced']) : null));
|
||||
|
||||
// 3. Runtime-aware resolution (#2517) — only when `runtime` is explicitly set
|
||||
// to a non-Claude runtime. `runtime: "claude"` is the implicit default and is
|
||||
@@ -1529,8 +1563,10 @@ function resolveModelInternal(cwd, agentType) {
|
||||
// "omit"` (review finding #4). Deliberate ordering for non-Claude runtimes:
|
||||
// explicit opt-in beats `resolve_model_ids: "omit"` so users on Codex installs
|
||||
// that auto-set "omit" can still flip on tiered behavior by setting runtime
|
||||
// alone. inherit profile is preserved verbatim.
|
||||
if (config.runtime && config.runtime !== 'claude' && profile !== 'inherit' && tier) {
|
||||
// alone. Gate on tier !== 'inherit' (not profile !== 'inherit') so a
|
||||
// valid phase-type tier flips runtime resolution on even when the
|
||||
// profile is inherit.
|
||||
if (config.runtime && config.runtime !== 'claude' && tier && tier !== 'inherit') {
|
||||
const entry = _resolveRuntimeTier(config, tier);
|
||||
if (entry?.model) return entry.model;
|
||||
// Unknown runtime with no user-supplied overrides — fall through to Claude-safe
|
||||
@@ -1546,7 +1582,9 @@ function resolveModelInternal(cwd, agentType) {
|
||||
|
||||
// 5. Profile lookup (Claude-native default).
|
||||
if (!agentModels) return 'sonnet';
|
||||
if (profile === 'inherit') return 'inherit';
|
||||
// Gate on tier (not profile) so a valid phase-type override beats
|
||||
// profile=inherit (#3030 CR Major).
|
||||
if (tier === 'inherit') return 'inherit';
|
||||
// `tier` is guaranteed truthy here: agentModels exists, and MODEL_PROFILES
|
||||
// entries always define `balanced`, so `agentModels[profile] || agentModels.balanced`
|
||||
// resolves to a string. Keep the local for readability — no defensive fallback.
|
||||
@@ -1586,11 +1624,38 @@ function resolveReasoningEffortInternal(cwd, agentType) {
|
||||
if (config.model_overrides?.[agentType]) return null;
|
||||
|
||||
const profile = String(config.model_profile || 'balanced').toLowerCase();
|
||||
if (profile === 'inherit') return null;
|
||||
const agentModels = MODEL_PROFILES[agentType];
|
||||
if (!agentModels) return null;
|
||||
const tier = agentModels[profile] || agentModels['balanced'];
|
||||
if (!tier) return null;
|
||||
|
||||
// #3023 (CR Major): mirror the phase-type tier lookup from
|
||||
// resolveModelInternal. Without this, `model` and `reasoning_effort`
|
||||
// derive from different tier sources on Codex when models.<phase_type>
|
||||
// overrides the profile.
|
||||
//
|
||||
// #3030 CR follow-up: do NOT short-circuit on profile === 'inherit'
|
||||
// before reading the phase-type tier. A config like
|
||||
// { model_profile: 'inherit', models: { execution: 'opus' } }
|
||||
// must produce the opus runtime effort, not null. Compute tier from
|
||||
// phase-type first; only fall back to profile when there's no valid
|
||||
// phase-type override; only return null when the resolved tier is
|
||||
// 'inherit' or unknown.
|
||||
const phaseType = AGENT_TO_PHASE_TYPE[agentType];
|
||||
const phaseTypeTier = (phaseType && config.models && typeof config.models === 'object')
|
||||
? config.models[phaseType]
|
||||
: undefined;
|
||||
// Explicit phase-type 'inherit' is the user opting out of tier-based
|
||||
// effort for this phase — return null instead of falling through to
|
||||
// profile (which would silently emit the profile's effort and
|
||||
// contradict the user's choice).
|
||||
if (phaseTypeTier === 'inherit') return null;
|
||||
const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku']);
|
||||
const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier))
|
||||
? phaseTypeTier
|
||||
: (profile === 'inherit'
|
||||
? 'inherit'
|
||||
: (agentModels[profile] || agentModels['balanced']));
|
||||
// 'inherit' (from profile fallback) yields no runtime effort.
|
||||
if (!tier || tier === 'inherit') return null;
|
||||
|
||||
const entry = _resolveRuntimeTier(config, tier);
|
||||
return entry?.reasoning_effort || null;
|
||||
|
||||
@@ -28,6 +28,61 @@ const MODEL_PROFILES = {
|
||||
};
|
||||
const VALID_PROFILES = [...Object.keys(MODEL_PROFILES['gsd-planner']), 'inherit'];
|
||||
|
||||
/**
|
||||
* #3023 — Phase-type → agent mapping table.
|
||||
*
|
||||
* Lets users tune model selection at the *phase-type* level (planning,
|
||||
* research, execution, verification, ...) instead of per-agent. Maps
|
||||
* each agent in MODEL_PROFILES to exactly one phase-type so resolution
|
||||
* is deterministic.
|
||||
*
|
||||
* Adding a new agent to MODEL_PROFILES requires adding an entry here too;
|
||||
* tests/feat-3023-phase-type-models.test.cjs asserts coverage.
|
||||
*
|
||||
* Phase-type semantics:
|
||||
* - planning: produces the plan (PLAN.md, ROADMAP.md, patterns)
|
||||
* - discuss: collaborative scoping (no subagent today; reserved
|
||||
* for orchestrators that may spawn one)
|
||||
* - research: gathers external/codebase context (RESEARCH.md)
|
||||
* - execution: implements the plan or writes user-facing artifacts
|
||||
* - verification: checks correctness (VERIFICATION.md, audits)
|
||||
* - completion: post-execution wrap-up (no subagent today; reserved)
|
||||
*/
|
||||
const AGENT_TO_PHASE_TYPE = {
|
||||
// Planning — produces the plan / roadmap / pattern map
|
||||
'gsd-planner': 'planning',
|
||||
'gsd-roadmapper': 'planning',
|
||||
'gsd-pattern-mapper': 'planning',
|
||||
// Research — external/codebase information gathering
|
||||
'gsd-phase-researcher': 'research',
|
||||
'gsd-project-researcher': 'research',
|
||||
'gsd-research-synthesizer': 'research',
|
||||
'gsd-codebase-mapper': 'research',
|
||||
'gsd-ui-researcher': 'research',
|
||||
// Execution — implementation, debugging, doc writing
|
||||
'gsd-executor': 'execution',
|
||||
'gsd-debugger': 'execution',
|
||||
'gsd-doc-writer': 'execution',
|
||||
// Verification — correctness checks, audits, gap analysis
|
||||
'gsd-verifier': 'verification',
|
||||
'gsd-plan-checker': 'verification',
|
||||
'gsd-integration-checker': 'verification',
|
||||
'gsd-nyquist-auditor': 'verification',
|
||||
'gsd-ui-checker': 'verification',
|
||||
'gsd-ui-auditor': 'verification',
|
||||
'gsd-doc-verifier': 'verification',
|
||||
};
|
||||
|
||||
/**
|
||||
* The six phase-type slots accepted in `.planning/config.json` `models`
|
||||
* block. `discuss` and `completion` are reserved — no current agent maps
|
||||
* to them today — so users can pre-configure those slots without
|
||||
* breaking validation when an orchestrator starts honoring them.
|
||||
*/
|
||||
const VALID_PHASE_TYPES = new Set([
|
||||
'planning', 'discuss', 'research', 'execution', 'verification', 'completion',
|
||||
]);
|
||||
|
||||
/**
|
||||
* Formats the agent-to-model mapping as a human-readable table (in string format).
|
||||
*
|
||||
@@ -68,6 +123,8 @@ function getAgentToModelMapForProfile(normalizedProfile) {
|
||||
module.exports = {
|
||||
MODEL_PROFILES,
|
||||
VALID_PROFILES,
|
||||
AGENT_TO_PHASE_TYPE,
|
||||
VALID_PHASE_TYPES,
|
||||
formatAgentToModelMapAsTable,
|
||||
getAgentToModelMapForProfile,
|
||||
};
|
||||
|
||||
@@ -19,6 +19,53 @@ Model profiles control which Claude model each GSD agent uses. This allows balan
|
||||
| gsd-integration-checker | sonnet | sonnet | haiku | haiku | inherit |
|
||||
| gsd-nyquist-auditor | sonnet | sonnet | haiku | haiku | inherit |
|
||||
|
||||
## Per-Phase-Type Model Map (#3023)
|
||||
|
||||
`.planning/config.json` accepts a coarse per-**phase-type** map under the `models` key. Use this when you want tuning at the phase level ("Opus for planning and execution, Sonnet for the rest") without learning the agent taxonomy.
|
||||
|
||||
```json
|
||||
{
|
||||
"model_profile": "balanced",
|
||||
"models": {
|
||||
"planning": "opus",
|
||||
"discuss": "opus",
|
||||
"research": "sonnet",
|
||||
"execution": "opus",
|
||||
"verification": "sonnet",
|
||||
"completion": "sonnet"
|
||||
},
|
||||
"model_overrides": {
|
||||
"gsd-codebase-mapper": "haiku"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Phase-type → agent mapping
|
||||
|
||||
| Phase type | Agents |
|
||||
|---|---|
|
||||
| `planning` | gsd-planner, gsd-roadmapper, gsd-pattern-mapper |
|
||||
| `discuss` | (reserved — no subagent today) |
|
||||
| `research` | gsd-phase-researcher, gsd-project-researcher, gsd-research-synthesizer, gsd-codebase-mapper, gsd-ui-researcher |
|
||||
| `execution` | gsd-executor, gsd-debugger, gsd-doc-writer |
|
||||
| `verification` | gsd-verifier, gsd-plan-checker, gsd-integration-checker, gsd-nyquist-auditor, gsd-ui-checker, gsd-ui-auditor, gsd-doc-verifier |
|
||||
| `completion` | (reserved — no subagent today) |
|
||||
|
||||
### Resolution precedence (highest to lowest)
|
||||
|
||||
1. **Per-agent `model_overrides[agent]`** — full IDs accepted; targeted exceptions
|
||||
2. **Phase-type `models[phase_type]`** — tier alias only (`opus` / `sonnet` / `haiku` / `inherit`)
|
||||
3. **Profile table** — the per-agent column from the active `model_profile`
|
||||
4. **Runtime default** — when nothing else applies
|
||||
|
||||
### Why two layers above the profile?
|
||||
|
||||
- **Profile** is a global tier strategy (everyone runs balanced).
|
||||
- **`models`** is coarse phase-level tuning without learning agent names.
|
||||
- **`model_overrides`** is per-agent precision (e.g. force `haiku` on `gsd-codebase-mapper` for a fan-out).
|
||||
|
||||
The three layers compose: `models` defaults a phase, `model_overrides` carves an exception out of it.
|
||||
|
||||
## Profile Philosophy
|
||||
|
||||
**quality** - Maximum reasoning power
|
||||
@@ -93,10 +140,16 @@ Orchestrators resolve model before spawning:
|
||||
```
|
||||
1. Read .planning/config.json
|
||||
2. Check model_overrides for agent-specific override
|
||||
3. If no override, look up agent in profile table
|
||||
4. Pass model parameter to Task call
|
||||
3. If no override, check models[phase_type] for a phase-type tier
|
||||
(see §Per-Phase-Type Model Map for the agent → phase-type mapping)
|
||||
4. If no phase-type slot, look up agent in profile table
|
||||
5. Pass model parameter to Task call
|
||||
```
|
||||
|
||||
The same precedence applies to `reasoning_effort` resolution on runtimes
|
||||
that support it (Codex), so `model` and `reasoning_effort` always derive
|
||||
from the same tier source — a `models[phase_type]` override flips both.
|
||||
|
||||
## Per-Agent Overrides
|
||||
|
||||
Override specific agents without changing the entire profile:
|
||||
|
||||
@@ -111,6 +111,12 @@ export const DYNAMIC_KEY_PATTERNS: readonly DynamicKeyPattern[] = [
|
||||
description: 'model_profile_overrides.<runtime>.<opus|sonnet|haiku>',
|
||||
test: (k) => /^model_profile_overrides\.[a-zA-Z0-9_-]+\.(opus|sonnet|haiku)$/.test(k),
|
||||
},
|
||||
// #3023 — per-phase-type model map: models.<phase_type> = <tier>
|
||||
{
|
||||
source: '^models\\.(planning|discuss|research|execution|verification|completion)$',
|
||||
description: 'models.<planning|discuss|research|execution|verification|completion>',
|
||||
test: (k) => /^models\.(planning|discuss|research|execution|verification|completion)$/.test(k),
|
||||
},
|
||||
];
|
||||
|
||||
/** Returns true if keyPath is a valid config key (exact or dynamic pattern). */
|
||||
|
||||
408
tests/feat-3023-phase-type-models.test.cjs
Normal file
408
tests/feat-3023-phase-type-models.test.cjs
Normal file
@@ -0,0 +1,408 @@
|
||||
/**
|
||||
* Feature test for issue #3023 — per-phase-type model map.
|
||||
*
|
||||
* Adds a `models` block to .planning/config.json that accepts phase-type
|
||||
* keys (planning / discuss / research / execution / verification /
|
||||
* completion). Resolution precedence:
|
||||
*
|
||||
* 1. Per-agent `model_overrides[agent]` (highest)
|
||||
* 2. Phase-type `models[phase_type]` (NEW)
|
||||
* 3. Profile table (`model_profile`)
|
||||
* 4. Runtime default
|
||||
*
|
||||
* Tests are typed-IR / structural — assert on the value returned by
|
||||
* resolveModelInternal, not stdout/grep. Each test seeds a temp project
|
||||
* with a fixture .planning/config.json and asserts the resolver picks
|
||||
* the right tier for each agent.
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
|
||||
process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
const { test, describe, beforeEach, afterEach } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const os = require('node:os');
|
||||
|
||||
const {
|
||||
resolveModelInternal,
|
||||
} = require('../get-shit-done/bin/lib/core.cjs');
|
||||
const {
|
||||
AGENT_TO_PHASE_TYPE,
|
||||
VALID_PHASE_TYPES,
|
||||
MODEL_PROFILES,
|
||||
} = require('../get-shit-done/bin/lib/model-profiles.cjs');
|
||||
const { isValidConfigKey } = require('../get-shit-done/bin/lib/config-schema.cjs');
|
||||
|
||||
function makeTmp(prefix) {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), `gsd-3023-${prefix}-`));
|
||||
}
|
||||
|
||||
function writeConfig(projectDir, config) {
|
||||
const planningDir = path.join(projectDir, '.planning');
|
||||
fs.mkdirSync(planningDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(planningDir, 'config.json'), JSON.stringify(config, null, 2));
|
||||
}
|
||||
|
||||
function rmr(p) {
|
||||
try { fs.rmSync(p, { recursive: true, force: true }); } catch { /* noop */ }
|
||||
}
|
||||
|
||||
// ─── Schema: AGENT_TO_PHASE_TYPE table + VALID_PHASE_TYPES ──────────────────
|
||||
|
||||
describe('#3023 phase-type schema: every agent has a phase-type assignment', () => {
|
||||
test('AGENT_TO_PHASE_TYPE is exported as a non-empty object', () => {
|
||||
assert.equal(typeof AGENT_TO_PHASE_TYPE, 'object');
|
||||
assert.ok(AGENT_TO_PHASE_TYPE !== null);
|
||||
assert.ok(Object.keys(AGENT_TO_PHASE_TYPE).length > 0);
|
||||
});
|
||||
|
||||
test('VALID_PHASE_TYPES exposes the six named slots from the issue', () => {
|
||||
// The issue specified exactly these slots. Adding new slots here is a
|
||||
// schema change that must coordinate with config-schema's dynamic
|
||||
// pattern and the docs.
|
||||
assert.deepStrictEqual(
|
||||
[...VALID_PHASE_TYPES].sort(),
|
||||
['completion', 'discuss', 'execution', 'planning', 'research', 'verification'].sort()
|
||||
);
|
||||
});
|
||||
|
||||
test('every agent in MODEL_PROFILES has a phase-type assignment', () => {
|
||||
const missing = Object.keys(MODEL_PROFILES).filter(
|
||||
(agent) => !AGENT_TO_PHASE_TYPE[agent]
|
||||
);
|
||||
assert.deepStrictEqual(missing, [],
|
||||
`every agent in MODEL_PROFILES must have a phase-type — missing: ${JSON.stringify(missing)}`);
|
||||
});
|
||||
|
||||
test('every assigned phase-type is one of the six valid slots', () => {
|
||||
const invalid = Object.entries(AGENT_TO_PHASE_TYPE).filter(
|
||||
([, phaseType]) => !VALID_PHASE_TYPES.has(phaseType)
|
||||
);
|
||||
assert.deepStrictEqual(invalid, [],
|
||||
`phase-type assignments must use VALID_PHASE_TYPES — invalid: ${JSON.stringify(invalid)}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Resolver behavior: phase-type drives tier ──────────────────────────────
|
||||
|
||||
describe('#3023 resolver: models.<phase_type> overrides profile-based tier', () => {
|
||||
let projectDir;
|
||||
beforeEach(() => { projectDir = makeTmp('resolver'); });
|
||||
afterEach(() => { rmr(projectDir); });
|
||||
|
||||
test('phase-type alone — research agents get the phase-type tier, planner gets profile default', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'balanced',
|
||||
models: { research: 'haiku' },
|
||||
});
|
||||
// gsd-phase-researcher is a research agent — should pick up 'haiku'
|
||||
// from the phase-type slot, not 'sonnet' from the balanced profile.
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'haiku');
|
||||
// gsd-codebase-mapper is also research → haiku
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku');
|
||||
// gsd-planner is planning, no models.planning set → falls through to
|
||||
// profile (balanced → opus per MODEL_PROFILES).
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus');
|
||||
});
|
||||
|
||||
test('per-agent override beats phase-type (acceptance criterion b)', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'balanced',
|
||||
models: { research: 'haiku' },
|
||||
model_overrides: { 'gsd-phase-researcher': 'opus' },
|
||||
});
|
||||
// The targeted per-agent override wins for that one agent.
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'opus');
|
||||
// Other research agents still pick up the phase-type tier.
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku');
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-research-synthesizer'), 'haiku');
|
||||
});
|
||||
|
||||
test('phase-type beats profile (acceptance criterion c)', () => {
|
||||
// model_profile=quality would normally make research agents 'opus'.
|
||||
// models.research='haiku' must win.
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'quality',
|
||||
models: { research: 'haiku' },
|
||||
});
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'haiku');
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku');
|
||||
// gsd-planner is planning, no slot set, profile=quality → opus.
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus');
|
||||
});
|
||||
|
||||
test('issue example: opus for planning/discuss/execution, sonnet for research/verification/completion', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'balanced',
|
||||
models: {
|
||||
planning: 'opus',
|
||||
discuss: 'opus',
|
||||
execution: 'opus',
|
||||
research: 'sonnet',
|
||||
verification: 'sonnet',
|
||||
completion: 'sonnet',
|
||||
},
|
||||
});
|
||||
// Planning agents → opus
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus');
|
||||
// Execution agents → opus
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-executor'), 'opus');
|
||||
// Research agents → sonnet
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet');
|
||||
// Verification agents → sonnet
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-verifier'), 'sonnet');
|
||||
});
|
||||
|
||||
test('phase-type "inherit" is honored (preserves existing inherit semantics)', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'balanced',
|
||||
models: { research: 'inherit' },
|
||||
});
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'inherit');
|
||||
});
|
||||
|
||||
test('empty models block is a no-op (acceptance criterion: backward compat)', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'balanced',
|
||||
models: {},
|
||||
});
|
||||
// Behavior must match no-models config (balanced profile).
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet');
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus');
|
||||
});
|
||||
|
||||
test('no models block at all is a no-op (acceptance criterion: backward compat)', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'balanced',
|
||||
});
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet');
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'opus');
|
||||
});
|
||||
|
||||
test('unrecognized tier value falls through to profile (typo safety) — CR follow-up', () => {
|
||||
// The VALID_TIERS guard in resolveModelInternal must reject any value
|
||||
// that isn't a known tier alias and fall back to the profile tier.
|
||||
// Without this guard a typo like "haiku3" would pollute the runtime
|
||||
// resolution chain. Locks the guard in so a future regression that
|
||||
// removes it is caught.
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'balanced',
|
||||
models: { research: 'haiku3' }, // typo; not a valid tier alias
|
||||
});
|
||||
// Falls back to balanced → sonnet for research agents.
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet');
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku',
|
||||
'gsd-codebase-mapper at balanced is haiku per profile, unaffected by typo');
|
||||
});
|
||||
|
||||
test('full model ID in models.<phase_type> is rejected; falls through to profile — CR follow-up', () => {
|
||||
// Full IDs are not valid in models.<phase_type>; they belong in
|
||||
// model_overrides per agent. The guard ensures we don't accidentally
|
||||
// hand a full ID into the runtime-tier resolution chain.
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'balanced',
|
||||
models: { research: 'openai/gpt-5' },
|
||||
});
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'sonnet');
|
||||
});
|
||||
|
||||
// ─── CR Major: phase-type beats inherit profile ─────────────────────────
|
||||
// Pre-fix bug: model_profile='inherit' + models.execution='opus' returned
|
||||
// 'inherit' because the profile short-circuit fired BEFORE the phase-type
|
||||
// override could win, violating the documented precedence where
|
||||
// models[phase_type] beats model_profile.
|
||||
|
||||
test('phase-type override wins over profile=inherit (CR Major) — model resolver', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'inherit',
|
||||
models: { execution: 'opus' },
|
||||
});
|
||||
// gsd-executor (execution) must get the phase-type opus, not inherit.
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-executor'), 'opus');
|
||||
});
|
||||
|
||||
test('phase-type "haiku" wins over profile=inherit; agents without a slot still inherit', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'inherit',
|
||||
models: { research: 'haiku' },
|
||||
});
|
||||
// research agents → haiku (phase-type wins)
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'haiku');
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-codebase-mapper'), 'haiku');
|
||||
// planning agent has no slot set → falls through to profile=inherit.
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-planner'), 'inherit');
|
||||
});
|
||||
|
||||
test('profile=inherit with no models block still returns inherit (no regression)', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'inherit',
|
||||
});
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-executor'), 'inherit');
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-phase-researcher'), 'inherit');
|
||||
});
|
||||
|
||||
test('profile=inherit with models block but agent has no slot → inherit', () => {
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'inherit',
|
||||
models: { research: 'haiku' },
|
||||
});
|
||||
// gsd-executor (execution slot) is not set → falls through to inherit.
|
||||
assert.equal(resolveModelInternal(projectDir, 'gsd-executor'), 'inherit');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── #3030 CR Major outside-diff: reasoning_effort honors phase-type ───────
|
||||
|
||||
const { resolveReasoningEffortInternal } = require('../get-shit-done/bin/lib/core.cjs');
|
||||
|
||||
describe('#3023 + #3030 CR: resolveReasoningEffortInternal honors phase-type tier (Codex)', () => {
|
||||
let projectDir;
|
||||
beforeEach(() => { projectDir = makeTmp('effort'); });
|
||||
afterEach(() => { rmr(projectDir); });
|
||||
|
||||
test('exported from core.cjs', () => {
|
||||
assert.equal(typeof resolveReasoningEffortInternal, 'function');
|
||||
});
|
||||
|
||||
test('phase-type override flips both model AND reasoning_effort to the same tier (Codex)', () => {
|
||||
// The CR Major bug: previously the model was resolved from the
|
||||
// phase-type tier (opus → gpt-5.4) but reasoning_effort still came
|
||||
// from the profile-derived sonnet tier (medium) — leading to a
|
||||
// mismatched (model, effort) pair on Codex spawn.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'balanced',
|
||||
models: { execution: 'opus' },
|
||||
});
|
||||
// gsd-executor's profile tier under balanced is sonnet, so without
|
||||
// the phase-type lookup mirror, model would resolve to opus (xhigh)
|
||||
// but effort to medium. Both must derive from the same tier source.
|
||||
const effort = resolveReasoningEffortInternal(projectDir, 'gsd-executor');
|
||||
// The exact effort value depends on the runtime tier map's opus row;
|
||||
// the test guards the relationship: it must NOT be the sonnet/medium
|
||||
// value when the phase-type forced opus.
|
||||
const sonnetEffort = (() => {
|
||||
// Read the sonnet effort by setting a config that uses the sonnet tier
|
||||
// and reading what comes back, so the assertion is semantic (effort
|
||||
// matches phase-type tier) rather than a hard-coded string.
|
||||
const sonnetDir = makeTmp('effort-sonnet');
|
||||
try {
|
||||
writeConfig(sonnetDir, {
|
||||
runtime: 'codex', model_profile: 'balanced',
|
||||
});
|
||||
return resolveReasoningEffortInternal(sonnetDir, 'gsd-executor');
|
||||
} finally {
|
||||
rmr(sonnetDir);
|
||||
}
|
||||
})();
|
||||
const opusEffort = (() => {
|
||||
const opusDir = makeTmp('effort-opus');
|
||||
try {
|
||||
writeConfig(opusDir, {
|
||||
runtime: 'codex', model_profile: 'quality', // quality → executor=opus
|
||||
});
|
||||
return resolveReasoningEffortInternal(opusDir, 'gsd-executor');
|
||||
} finally {
|
||||
rmr(opusDir);
|
||||
}
|
||||
})();
|
||||
// The phase-type override (models.execution=opus) must produce the
|
||||
// SAME effort as a profile-only opus config.
|
||||
assert.equal(effort, opusEffort,
|
||||
`phase-type override must match opus-tier effort, got ${effort}, expected ${opusEffort}`);
|
||||
// And it must NOT match sonnet effort (proving the override fired).
|
||||
if (opusEffort !== null && sonnetEffort !== null && opusEffort !== sonnetEffort) {
|
||||
assert.notEqual(effort, sonnetEffort,
|
||||
`phase-type override should not silently use sonnet effort: ${effort}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('inherit phase-type tier returns null effort (no runtime entry maps to inherit)', () => {
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'balanced',
|
||||
models: { execution: 'inherit' },
|
||||
});
|
||||
// 'inherit' has no runtime-tier entry, so the resolver returns null.
|
||||
const effort = resolveReasoningEffortInternal(projectDir, 'gsd-executor');
|
||||
assert.equal(effort, null);
|
||||
});
|
||||
|
||||
test('per-agent override still bypasses phase-type for reasoning_effort', () => {
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'balanced',
|
||||
models: { execution: 'opus' },
|
||||
model_overrides: { 'gsd-executor': 'openai/gpt-5' },
|
||||
});
|
||||
// model_overrides[agent] short-circuits resolveReasoningEffortInternal
|
||||
// (the user supplied a fully-qualified ID; effort must be set per-agent).
|
||||
assert.equal(resolveReasoningEffortInternal(projectDir, 'gsd-executor'), null);
|
||||
});
|
||||
|
||||
test('claude runtime ignores models.* for reasoning_effort (returns null)', () => {
|
||||
writeConfig(projectDir, {
|
||||
// No `runtime` set → defaults to claude, which has no reasoning_effort.
|
||||
model_profile: 'balanced',
|
||||
models: { execution: 'opus' },
|
||||
});
|
||||
assert.equal(resolveReasoningEffortInternal(projectDir, 'gsd-executor'), null);
|
||||
});
|
||||
|
||||
test('phase-type override wins over profile=inherit for effort (CR Major #3030)', () => {
|
||||
// Pre-fix bug: profile=inherit short-circuited to null even when
|
||||
// models.execution=opus would have supplied a valid tier.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'codex',
|
||||
model_profile: 'inherit',
|
||||
models: { execution: 'opus' },
|
||||
});
|
||||
// Compute the expected effort by reading what gsd-executor would
|
||||
// get under a profile-only opus config — the phase-type override
|
||||
// must produce the SAME result.
|
||||
const expected = (() => {
|
||||
const dir = makeTmp('effort-opus2');
|
||||
try {
|
||||
writeConfig(dir, { runtime: 'codex', model_profile: 'quality' });
|
||||
return resolveReasoningEffortInternal(dir, 'gsd-executor');
|
||||
} finally {
|
||||
rmr(dir);
|
||||
}
|
||||
})();
|
||||
const actual = resolveReasoningEffortInternal(projectDir, 'gsd-executor');
|
||||
assert.equal(actual, expected,
|
||||
`phase-type override over profile=inherit must produce the opus-tier effort; got ${actual}, expected ${expected}`);
|
||||
assert.notEqual(actual, null,
|
||||
'phase-type opus must NOT return null effort just because profile=inherit');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Schema validation ──────────────────────────────────────────────────────
|
||||
|
||||
describe('#3023 config-schema: models.<phase_type> validation', () => {
|
||||
test('models.planning is a valid config key', () => {
|
||||
assert.equal(isValidConfigKey('models.planning'), true);
|
||||
});
|
||||
|
||||
test('all six phase-type slots are valid config keys', () => {
|
||||
for (const slot of ['planning', 'discuss', 'research', 'execution', 'verification', 'completion']) {
|
||||
assert.equal(isValidConfigKey(`models.${slot}`), true,
|
||||
`models.${slot} must be a valid config key`);
|
||||
}
|
||||
});
|
||||
|
||||
test('unknown phase-type is rejected (acceptance criterion d)', () => {
|
||||
assert.equal(isValidConfigKey('models.deployment'), false,
|
||||
'unknown phase-type must NOT be accepted');
|
||||
assert.equal(isValidConfigKey('models.gsd-planner'), false,
|
||||
'agent name in models.* must NOT be accepted (use model_overrides for agents)');
|
||||
});
|
||||
|
||||
test('models alone (without a slot) is not a valid config-set key', () => {
|
||||
// Setting the whole block isn't a granular set; users edit JSON directly.
|
||||
assert.equal(isValidConfigKey('models'), false);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user