* feat(#3024): dynamic routing with failure-tier escalation Adds a `dynamic_routing` block to .planning/config.json that lets the resolver start agents on a cheap tier and escalate one tier up when the orchestrator detects a soft failure (verification inconclusive, plan-check FLAG, etc.). Solves the "pay Opus rates as insurance" anti-pattern by making escalation observed-quality-driven. Architecture: - AGENT_DEFAULT_TIERS map (light/standard/heavy) — every agent in MODEL_PROFILES declares a default tier; tests assert coverage so adding a new agent without updating the map fails CI. - nextTier(currentTier) helper — light → standard → heavy → heavy (heavy stays at heavy; can't go further). - resolveModelForTier(cwd, agentType, attempt) — new resolver. The orchestrator tracks the attempt counter and passes 0 for the first spawn, 1+ on escalation. The resolver caps internally at max_escalations so the orchestrator can blindly bump the counter. - Schema validation: dynamic_routing.enabled / escalate_on_failure / max_escalations / tier_models.<light|standard|heavy>. Unknown tiers and unknown sub-keys rejected at config-set time. - SDK schema mirror updated to keep CJS/SDK in lockstep (#2653). Resolution precedence (highest → lowest): 1. model_overrides[<agent>] (full IDs accepted) 2. dynamic_routing.tier_models[<tier>] (NEW; escalation-aware) 3. models[<phase_type>] (#3023 phase-type map) 4. model_profile (per-agent column) 5. Runtime default Backward compatibility: dynamic_routing is disabled by default (enabled: false or block omitted). resolveModelForTier short- circuits to resolveModelInternal in that case, so callers can adopt unconditionally without breaking existing behavior. This PR delivers the JS-layer infrastructure: schema + tier map + resolver. Orchestrator adoption (workflow markdown updates that detect soft failures and call resolveModelForTier with attempt+1) is incremental follow-up — verifier / plan-checker / integration- checker each adopt the protocol when ready. Tests (23 cases, all structural-IR — no stdout grep): - Schema invariants: AGENT_DEFAULT_TIERS coverage, VALID_AGENT_TIERS exact match, every assignment uses a valid tier - nextTier helper: light→standard→heavy→heavy, null on invalid input - Disabled mode: no block + enabled:false both no-op (back-compat) - Enabled mode: attempt=0 returns default tier model, attempt=1 escalates, beyond max_escalations caps, heavy agents stay heavy, default max_escalations=1 when omitted - Precedence: per-agent override beats dynamic_routing, dynamic_routing beats phase-type models - Validation: every settings key accepted, unknown tiers/sub-keys rejected, bare `dynamic_routing` rejected as config-set target Documentation: - get-shit-done/references/model-profiles.md — full reference section - docs/CONFIGURATION.md — full settings table + escalation flow - docs/USER-GUIDE.md — task-oriented "Cheap-by-default" section - docs/FEATURES.md — config row cross-link Verification: - 23/23 pass on regression test - 6843/6843 full suite (23 net new from 6820) - lint-no-source-grep clean (376 test files) - SDK schema mirror keeps CJS/SDK in sync per #2653 parity test Closes #3024 * fix(#3024): honor escalate_on_failure:false + 3 CR follow-ups CodeRabbit on PR #3031 (4 findings — 1 Major + 2 Minor + 1 Nitpick): 1. **Major (inline)** — get-shit-done/bin/lib/core.cjs:1668 resolveModelForTier ignored dynamic_routing.escalate_on_failure. When the user set it to false, escalation should be disabled, but the resolver only checked attempt/max_escalations. An orchestrator that always passes attempt+1 on retry would silently escalate despite the user opting out. Fix: gate effectiveAttempt on `dr.escalate_on_failure !== false` so false short-circuits every attempt back to the default tier. 2. **Minor (inline)** — docs/CONFIGURATION.md:123-126 The dynamic_routing rows in the Core Settings table had 4 cells instead of 5 (missing the Options column), breaking the table structure. Added explicit Options values for enabled / escalate_on_failure / max_escalations rows. 3. **Minor (outside-diff)** — references/model-profiles.md:179-195 "Resolution Logic" sketch was pre-#3024 and didn't include dynamic_routing in the precedence ladder. Updated to a 6-step block with dynamic_routing at step 3 (between override and phase-type). 4. **Nitpick** — tests/feat-3024-dynamic-routing.test.cjs:189+ Tests used `if (lightAgent) { ... }` guards that silent-pass when AGENT_DEFAULT_TIERS drifts. Replaced all 5 conditional skips with `assert.ok(lightAgent, '...')` preconditions so a tier-mapping change surfaces as a test failure. Plus: 2 new regression tests for the Major fix: - escalate_on_failure:false caps every attempt at default tier - escalate_on_failure:true (explicit) still escalates normally Verification: - 25/25 pass on regression test (23 prior + 2 escalate_on_failure) - 6845/6845 full suite (2 net new) - lint-no-source-grep clean * docs(#3024): align precedence + add fence language tags (CR follow-up) CodeRabbit (3 minor): 1. docs/CONFIGURATION.md:691 — "Per-Phase-Type Models → Resolution precedence" was a 4-step block written pre-#3024; readers got contradictory rules between the per-phase-type section and the later dynamic_routing section. Updated to the same 5-step ladder with dynamic_routing at step 2, and noted that dynamic_routing is disabled by default so this section's behavior is unchanged when the kill-switch is off. 2. docs/CONFIGURATION.md:770 — escalation-flow code fence missing language tag (MD040). Added `text`. 3. references/model-profiles.md:184 — resolution-ladder code fence missing language tag (MD040). Added `text`. No code changes; docs only. Verification: regression test still 25/25. * docs(#3024): clarify precedence prose — five layers, not four (CR nitpick) CodeRabbit nitpick: the "Per-Phase-Type Models → Resolution precedence" prose said "The four layers compose..." but the ladder above lists five (including Runtime default). Also "dynamic_routing escalates per-attempt above all of them" misreads as suggesting dynamic_routing wins over model_overrides — actually overrides still win at step 1. Reworded top-down so the precedence direction is unambiguous: - model_profile = base - models = phase-level override - dynamic_routing = per-attempt escalation - model_overrides = per-agent exception (top) - runtime default = fallback No code changes; docs only. * docs(#3024): note escalate_on_failure:false in escalation-flow diagram (CR) CodeRabbit nitpick: the escalation-flow diagram in docs/CONFIGURATION.md described the soft-failure → respawn → tier_models[next_tier_up] path, but didn't surface the `dynamic_routing.escalate_on_failure: false` kill-switch right next to it. Users reading the flow diagram (which is the canonical place to understand attempt behavior) wouldn't see that the kill-switch overrides the soft-failure branch. Added a one-paragraph note immediately after the flow listing, before the tier-sequence example, so the kill-switch is visible exactly where users decide whether escalation will happen. No code changes; docs only.
246 lines
10 KiB
Markdown
246 lines
10 KiB
Markdown
# Model Profiles
|
|
|
|
Model profiles control which Claude model each GSD agent uses. This allows balancing quality vs token spend, or inheriting the currently selected session model.
|
|
|
|
## Profile Definitions
|
|
|
|
| Agent | `quality` | `balanced` | `budget` | `adaptive` | `inherit` |
|
|
|-------|-----------|------------|----------|------------|-----------|
|
|
| gsd-planner | opus | opus | sonnet | opus | inherit |
|
|
| gsd-roadmapper | opus | sonnet | sonnet | sonnet | inherit |
|
|
| gsd-executor | opus | sonnet | sonnet | sonnet | inherit |
|
|
| gsd-phase-researcher | opus | sonnet | haiku | sonnet | inherit |
|
|
| gsd-project-researcher | opus | sonnet | haiku | sonnet | inherit |
|
|
| gsd-research-synthesizer | sonnet | sonnet | haiku | haiku | inherit |
|
|
| gsd-debugger | opus | sonnet | sonnet | opus | inherit |
|
|
| gsd-codebase-mapper | sonnet | haiku | haiku | haiku | inherit |
|
|
| gsd-verifier | sonnet | sonnet | haiku | sonnet | inherit |
|
|
| gsd-plan-checker | sonnet | sonnet | haiku | haiku | inherit |
|
|
| gsd-integration-checker | sonnet | sonnet | haiku | haiku | inherit |
|
|
| gsd-nyquist-auditor | sonnet | sonnet | haiku | haiku | inherit |
|
|
|
|
## Per-Phase-Type Model Map (#3023)
|
|
|
|
`.planning/config.json` accepts a coarse per-**phase-type** map under the `models` key. Use this when you want tuning at the phase level ("Opus for planning and execution, Sonnet for the rest") without learning the agent taxonomy.
|
|
|
|
```json
|
|
{
|
|
"model_profile": "balanced",
|
|
"models": {
|
|
"planning": "opus",
|
|
"discuss": "opus",
|
|
"research": "sonnet",
|
|
"execution": "opus",
|
|
"verification": "sonnet",
|
|
"completion": "sonnet"
|
|
},
|
|
"model_overrides": {
|
|
"gsd-codebase-mapper": "haiku"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Phase-type → agent mapping
|
|
|
|
| Phase type | Agents |
|
|
|---|---|
|
|
| `planning` | gsd-planner, gsd-roadmapper, gsd-pattern-mapper |
|
|
| `discuss` | (reserved — no subagent today) |
|
|
| `research` | gsd-phase-researcher, gsd-project-researcher, gsd-research-synthesizer, gsd-codebase-mapper, gsd-ui-researcher |
|
|
| `execution` | gsd-executor, gsd-debugger, gsd-doc-writer |
|
|
| `verification` | gsd-verifier, gsd-plan-checker, gsd-integration-checker, gsd-nyquist-auditor, gsd-ui-checker, gsd-ui-auditor, gsd-doc-verifier |
|
|
| `completion` | (reserved — no subagent today) |
|
|
|
|
### Resolution precedence (highest to lowest)
|
|
|
|
1. **Per-agent `model_overrides[agent]`** — full IDs accepted; targeted exceptions
|
|
2. **Phase-type `models[phase_type]`** — tier alias only (`opus` / `sonnet` / `haiku` / `inherit`)
|
|
3. **Profile table** — the per-agent column from the active `model_profile`
|
|
4. **Runtime default** — when nothing else applies
|
|
|
|
### Why two layers above the profile?
|
|
|
|
- **Profile** is a global tier strategy (everyone runs balanced).
|
|
- **`models`** is coarse phase-level tuning without learning agent names.
|
|
- **`model_overrides`** is per-agent precision (e.g. force `haiku` on `gsd-codebase-mapper` for a fan-out).
|
|
|
|
The three layers compose: `models` defaults a phase, `model_overrides` carves an exception out of it.
|
|
|
|
## Profile Philosophy
|
|
|
|
**quality** - Maximum reasoning power
|
|
- Opus for all decision-making agents
|
|
- Sonnet for read-only verification
|
|
- Use when: quota available, critical architecture work
|
|
|
|
**balanced** (default) - Smart allocation
|
|
- Opus only for planning (where architecture decisions happen)
|
|
- Sonnet for execution and research (follows explicit instructions)
|
|
- Sonnet for verification (needs reasoning, not just pattern matching)
|
|
- Use when: normal development, good balance of quality and cost
|
|
|
|
**budget** - Minimal Opus usage
|
|
- Sonnet for anything that writes code
|
|
- Haiku for research and verification
|
|
- Use when: conserving quota, high-volume work, less critical phases
|
|
|
|
**adaptive** — Role-based cost optimization
|
|
- Opus for planning and debugging (where reasoning quality has highest impact)
|
|
- Sonnet for execution, research, and verification (follows explicit instructions)
|
|
- Haiku for mapping, checking, and auditing (high volume, structured output)
|
|
- Use when: optimizing cost without sacrificing plan quality, solo development on paid API tiers
|
|
|
|
**inherit** - Follow the current session model
|
|
- All agents resolve to `inherit`
|
|
- Best when you switch models interactively (for example OpenCode or Kilo `/model`)
|
|
- **Required when using non-Anthropic providers** (OpenRouter, local models, etc.) — otherwise GSD may call Anthropic models directly, incurring unexpected costs
|
|
- Use when: you want GSD to follow your currently selected runtime model
|
|
|
|
## Using Non-Claude Runtimes (Codex, OpenCode, Gemini CLI, Kilo)
|
|
|
|
When installed for a non-Claude runtime, the GSD installer sets `resolve_model_ids: "omit"` in `~/.gsd/defaults.json`. This returns an empty model parameter for all agents, so each agent uses the runtime's default model. No manual setup is needed.
|
|
|
|
To assign different models to different agents, add `model_overrides` with model IDs your runtime recognizes:
|
|
|
|
```json
|
|
{
|
|
"resolve_model_ids": "omit",
|
|
"model_overrides": {
|
|
"gsd-planner": "o3",
|
|
"gsd-executor": "o4-mini",
|
|
"gsd-debugger": "o3",
|
|
"gsd-codebase-mapper": "o4-mini"
|
|
}
|
|
}
|
|
```
|
|
|
|
The same tiering logic applies: stronger models for planning and debugging, cheaper models for execution and mapping.
|
|
|
|
## Using Claude Code with Non-Anthropic Providers (OpenRouter, Local)
|
|
|
|
If you're using Claude Code with OpenRouter, a local model, or any non-Anthropic provider, set the `inherit` profile to prevent GSD from calling Anthropic models for subagents:
|
|
|
|
```bash
|
|
# Via settings command
|
|
/gsd-settings
|
|
# → Select "Inherit" for model profile
|
|
|
|
# Or manually in .planning/config.json
|
|
{
|
|
"model_profile": "inherit"
|
|
}
|
|
```
|
|
|
|
Without `inherit`, GSD's default `balanced` profile spawns specific Anthropic models (`opus`, `sonnet`, `haiku`) for each agent type, which can result in additional API costs through your non-Anthropic provider.
|
|
|
|
## Dynamic Routing with Failure-Tier Escalation (#3024)
|
|
|
|
When `dynamic_routing.enabled = true` in `.planning/config.json`, the resolver picks a model from a tier-mapped table based on the agent's *default tier* (light / standard / heavy) and escalates to the next tier up on orchestrator-detected soft failure.
|
|
|
|
```json
|
|
{
|
|
"dynamic_routing": {
|
|
"enabled": true,
|
|
"tier_models": {
|
|
"light": "haiku",
|
|
"standard": "sonnet",
|
|
"heavy": "opus"
|
|
},
|
|
"escalate_on_failure": true,
|
|
"max_escalations": 1
|
|
}
|
|
}
|
|
```
|
|
|
|
**Agent default tiers** (each agent in `MODEL_PROFILES` declares one):
|
|
|
|
| Tier | Agents | Use case |
|
|
|---|---|---|
|
|
| `light` | gsd-codebase-mapper, gsd-pattern-mapper, gsd-research-synthesizer, gsd-plan-checker, gsd-integration-checker, gsd-nyquist-auditor, gsd-ui-checker, gsd-ui-auditor, gsd-doc-verifier | Cheap/fast — pure mappers, scanners, low-stakes audits |
|
|
| `standard` | gsd-executor, gsd-phase-researcher, gsd-project-researcher, gsd-verifier, gsd-doc-writer, gsd-ui-researcher | Default workhorse — research, writing, primary verification |
|
|
| `heavy` | gsd-planner, gsd-roadmapper, gsd-debugger | Deep reasoning — already at top, can't escalate further |
|
|
|
|
**Escalation flow** (orchestrator-driven):
|
|
|
|
1. Orchestrator spawns agent with `attempt: 0` → resolver returns `tier_models[default_tier]`
|
|
2. If orchestrator marks the result a soft failure, it re-spawns with `attempt: 1` → resolver returns `tier_models[next_tier_up]`
|
|
3. `max_escalations` caps total retries (default 1). Beyond the cap the resolver returns the cap-tier model so the orchestrator can log without burning further budget.
|
|
4. Hard failures (exceptions) bypass escalation and surface immediately.
|
|
|
|
**Precedence with other tier sources** (highest → lowest):
|
|
|
|
1. `model_overrides[<agent>]` — full ID, always wins
|
|
2. `dynamic_routing.tier_models[escalated_tier]` — when `enabled: true`
|
|
3. `models[<phase_type>]` — coarse phase-level (#3023)
|
|
4. `model_profile` — global tier strategy
|
|
|
|
When `dynamic_routing.enabled = false` (default), behavior is identical to today.
|
|
|
|
## Resolution Logic
|
|
|
|
Orchestrators resolve model before spawning. The full precedence ladder
|
|
is (highest → lowest):
|
|
|
|
```text
|
|
1. Read .planning/config.json
|
|
2. Check model_overrides[<agent>] (full IDs accepted; targeted exceptions)
|
|
3. If dynamic_routing.enabled, return tier_models[escalated_tier]
|
|
(see §Dynamic Routing — escalation steps tier up per attempt counter)
|
|
4. If no dynamic_routing match, check models[phase_type] for a phase-type tier
|
|
(see §Per-Phase-Type Model Map for the agent → phase-type mapping)
|
|
5. If no phase-type slot, look up agent in profile table
|
|
6. Pass model parameter to Task call
|
|
```
|
|
|
|
The same precedence applies to `reasoning_effort` resolution on runtimes
|
|
that support it (Codex), so `model` and `reasoning_effort` always derive
|
|
from the same tier source — a `models[phase_type]` or
|
|
`dynamic_routing` override flips both.
|
|
|
|
## Per-Agent Overrides
|
|
|
|
Override specific agents without changing the entire profile:
|
|
|
|
```json
|
|
{
|
|
"model_profile": "balanced",
|
|
"model_overrides": {
|
|
"gsd-executor": "opus",
|
|
"gsd-planner": "haiku"
|
|
}
|
|
}
|
|
```
|
|
|
|
Overrides take precedence over the profile. Valid values: `opus`, `sonnet`, `haiku`, `inherit`, or any fully-qualified model ID (e.g., `"o3"`, `"openai/o3"`, `"google/gemini-2.5-pro"`).
|
|
|
|
## Switching Profiles
|
|
|
|
Runtime: `/gsd-set-profile <profile>`
|
|
|
|
Per-project default: Set in `.planning/config.json`:
|
|
```json
|
|
{
|
|
"model_profile": "balanced"
|
|
}
|
|
```
|
|
|
|
## Design Rationale
|
|
|
|
**Why Opus for gsd-planner?**
|
|
Planning involves architecture decisions, goal decomposition, and task design. This is where model quality has the highest impact.
|
|
|
|
**Why Sonnet for gsd-executor?**
|
|
Executors follow explicit PLAN.md instructions. The plan already contains the reasoning; execution is implementation.
|
|
|
|
**Why Sonnet (not Haiku) for verifiers in balanced?**
|
|
Verification requires goal-backward reasoning - checking if code *delivers* what the phase promised, not just pattern matching. Sonnet handles this well; Haiku may miss subtle gaps.
|
|
|
|
**Why Haiku for gsd-codebase-mapper?**
|
|
Read-only exploration and pattern extraction. No reasoning required, just structured output from file contents.
|
|
|
|
**Why `inherit` instead of passing `opus` directly?**
|
|
Claude Code's `"opus"` alias maps to a specific model version. Organizations may block older opus versions while allowing newer ones. GSD returns `"inherit"` for opus-tier agents, causing them to use whatever opus version the user has configured in their session. This avoids version conflicts and silent fallbacks to Sonnet.
|
|
|
|
**Why `inherit` profile?**
|
|
Some runtimes (including OpenCode) let users switch models at runtime (`/model`). The `inherit` profile keeps all GSD subagents aligned to that live selection.
|