Files
msd-core/get-shit-done/references/model-profiles.md
Tom Boucher e1d661ece0 feat(#3024): dynamic routing with failure-tier escalation (#3031)
* feat(#3024): dynamic routing with failure-tier escalation

Adds a `dynamic_routing` block to .planning/config.json that lets
the resolver start agents on a cheap tier and escalate one tier up
when the orchestrator detects a soft failure (verification
inconclusive, plan-check FLAG, etc.). Solves the "pay Opus rates as
insurance" anti-pattern by making escalation observed-quality-driven.

Architecture:
- AGENT_DEFAULT_TIERS map (light/standard/heavy) — every agent in
  MODEL_PROFILES declares a default tier; tests assert coverage
  so adding a new agent without updating the map fails CI.
- nextTier(currentTier) helper — light → standard → heavy → heavy
  (heavy stays at heavy; can't go further).
- resolveModelForTier(cwd, agentType, attempt) — new resolver. The
  orchestrator tracks the attempt counter and passes 0 for the
  first spawn, 1+ on escalation. The resolver caps internally at
  max_escalations so the orchestrator can blindly bump the counter.
- Schema validation: dynamic_routing.enabled / escalate_on_failure /
  max_escalations / tier_models.<light|standard|heavy>. Unknown
  tiers and unknown sub-keys rejected at config-set time.
- SDK schema mirror updated to keep CJS/SDK in lockstep (#2653).

Resolution precedence (highest → lowest):
  1. model_overrides[<agent>]              (full IDs accepted)
  2. dynamic_routing.tier_models[<tier>]   (NEW; escalation-aware)
  3. models[<phase_type>]                  (#3023 phase-type map)
  4. model_profile                         (per-agent column)
  5. Runtime default

Backward compatibility: dynamic_routing is disabled by default
(enabled: false or block omitted). resolveModelForTier short-
circuits to resolveModelInternal in that case, so callers can
adopt unconditionally without breaking existing behavior.

This PR delivers the JS-layer infrastructure: schema + tier map +
resolver. Orchestrator adoption (workflow markdown updates that
detect soft failures and call resolveModelForTier with attempt+1)
is incremental follow-up — verifier / plan-checker / integration-
checker each adopt the protocol when ready.

Tests (23 cases, all structural-IR — no stdout grep):
- Schema invariants: AGENT_DEFAULT_TIERS coverage, VALID_AGENT_TIERS
  exact match, every assignment uses a valid tier
- nextTier helper: light→standard→heavy→heavy, null on invalid input
- Disabled mode: no block + enabled:false both no-op (back-compat)
- Enabled mode: attempt=0 returns default tier model, attempt=1
  escalates, beyond max_escalations caps, heavy agents stay heavy,
  default max_escalations=1 when omitted
- Precedence: per-agent override beats dynamic_routing,
  dynamic_routing beats phase-type models
- Validation: every settings key accepted, unknown tiers/sub-keys
  rejected, bare `dynamic_routing` rejected as config-set target

Documentation:
- get-shit-done/references/model-profiles.md — full reference section
- docs/CONFIGURATION.md — full settings table + escalation flow
- docs/USER-GUIDE.md — task-oriented "Cheap-by-default" section
- docs/FEATURES.md — config row cross-link

Verification:
- 23/23 pass on regression test
- 6843/6843 full suite (23 net new from 6820)
- lint-no-source-grep clean (376 test files)
- SDK schema mirror keeps CJS/SDK in sync per #2653 parity test

Closes #3024

* fix(#3024): honor escalate_on_failure:false + 3 CR follow-ups

CodeRabbit on PR #3031 (4 findings — 1 Major + 2 Minor + 1 Nitpick):

1. **Major (inline)** — get-shit-done/bin/lib/core.cjs:1668
   resolveModelForTier ignored dynamic_routing.escalate_on_failure.
   When the user set it to false, escalation should be disabled, but
   the resolver only checked attempt/max_escalations. An orchestrator
   that always passes attempt+1 on retry would silently escalate
   despite the user opting out.
   Fix: gate effectiveAttempt on `dr.escalate_on_failure !== false`
   so false short-circuits every attempt back to the default tier.

2. **Minor (inline)** — docs/CONFIGURATION.md:123-126
   The dynamic_routing rows in the Core Settings table had 4 cells
   instead of 5 (missing the Options column), breaking the table
   structure. Added explicit Options values for enabled / escalate_on_failure
   / max_escalations rows.

3. **Minor (outside-diff)** — references/model-profiles.md:179-195
   "Resolution Logic" sketch was pre-#3024 and didn't include
   dynamic_routing in the precedence ladder. Updated to a 6-step
   block with dynamic_routing at step 3 (between override and
   phase-type).

4. **Nitpick** — tests/feat-3024-dynamic-routing.test.cjs:189+
   Tests used `if (lightAgent) { ... }` guards that silent-pass
   when AGENT_DEFAULT_TIERS drifts. Replaced all 5 conditional
   skips with `assert.ok(lightAgent, '...')` preconditions so a
   tier-mapping change surfaces as a test failure.

Plus: 2 new regression tests for the Major fix:
- escalate_on_failure:false caps every attempt at default tier
- escalate_on_failure:true (explicit) still escalates normally

Verification:
- 25/25 pass on regression test (23 prior + 2 escalate_on_failure)
- 6845/6845 full suite (2 net new)
- lint-no-source-grep clean

* docs(#3024): align precedence + add fence language tags (CR follow-up)

CodeRabbit (3 minor):

1. docs/CONFIGURATION.md:691 — "Per-Phase-Type Models → Resolution
   precedence" was a 4-step block written pre-#3024; readers got
   contradictory rules between the per-phase-type section and the
   later dynamic_routing section. Updated to the same 5-step ladder
   with dynamic_routing at step 2, and noted that dynamic_routing
   is disabled by default so this section's behavior is unchanged
   when the kill-switch is off.

2. docs/CONFIGURATION.md:770 — escalation-flow code fence missing
   language tag (MD040). Added `text`.

3. references/model-profiles.md:184 — resolution-ladder code fence
   missing language tag (MD040). Added `text`.

No code changes; docs only. Verification: regression test still 25/25.

* docs(#3024): clarify precedence prose — five layers, not four (CR nitpick)

CodeRabbit nitpick: the "Per-Phase-Type Models → Resolution
precedence" prose said "The four layers compose..." but the ladder
above lists five (including Runtime default). Also "dynamic_routing
escalates per-attempt above all of them" misreads as suggesting
dynamic_routing wins over model_overrides — actually overrides still
win at step 1.

Reworded top-down so the precedence direction is unambiguous:
  - model_profile = base
  - models = phase-level override
  - dynamic_routing = per-attempt escalation
  - model_overrides = per-agent exception (top)
  - runtime default = fallback

No code changes; docs only.

* docs(#3024): note escalate_on_failure:false in escalation-flow diagram (CR)

CodeRabbit nitpick: the escalation-flow diagram in
docs/CONFIGURATION.md described the soft-failure → respawn →
tier_models[next_tier_up] path, but didn't surface the
`dynamic_routing.escalate_on_failure: false` kill-switch right next
to it. Users reading the flow diagram (which is the canonical place
to understand attempt behavior) wouldn't see that the kill-switch
overrides the soft-failure branch.

Added a one-paragraph note immediately after the flow listing,
before the tier-sequence example, so the kill-switch is visible
exactly where users decide whether escalation will happen.

No code changes; docs only.
2026-05-02 14:26:35 -04:00

10 KiB

Model Profiles

Model profiles control which Claude model each GSD agent uses. This allows balancing quality vs token spend, or inheriting the currently selected session model.

Profile Definitions

Agent quality balanced budget adaptive inherit
gsd-planner opus opus sonnet opus inherit
gsd-roadmapper opus sonnet sonnet sonnet inherit
gsd-executor opus sonnet sonnet sonnet inherit
gsd-phase-researcher opus sonnet haiku sonnet inherit
gsd-project-researcher opus sonnet haiku sonnet inherit
gsd-research-synthesizer sonnet sonnet haiku haiku inherit
gsd-debugger opus sonnet sonnet opus inherit
gsd-codebase-mapper sonnet haiku haiku haiku inherit
gsd-verifier sonnet sonnet haiku sonnet inherit
gsd-plan-checker sonnet sonnet haiku haiku inherit
gsd-integration-checker sonnet sonnet haiku haiku inherit
gsd-nyquist-auditor sonnet sonnet haiku haiku inherit

Per-Phase-Type Model Map (#3023)

.planning/config.json accepts a coarse per-phase-type map under the models key. Use this when you want tuning at the phase level ("Opus for planning and execution, Sonnet for the rest") without learning the agent taxonomy.

{
  "model_profile": "balanced",
  "models": {
    "planning": "opus",
    "discuss": "opus",
    "research": "sonnet",
    "execution": "opus",
    "verification": "sonnet",
    "completion": "sonnet"
  },
  "model_overrides": {
    "gsd-codebase-mapper": "haiku"
  }
}

Phase-type → agent mapping

Phase type Agents
planning gsd-planner, gsd-roadmapper, gsd-pattern-mapper
discuss (reserved — no subagent today)
research gsd-phase-researcher, gsd-project-researcher, gsd-research-synthesizer, gsd-codebase-mapper, gsd-ui-researcher
execution gsd-executor, gsd-debugger, gsd-doc-writer
verification gsd-verifier, gsd-plan-checker, gsd-integration-checker, gsd-nyquist-auditor, gsd-ui-checker, gsd-ui-auditor, gsd-doc-verifier
completion (reserved — no subagent today)

Resolution precedence (highest to lowest)

  1. Per-agent model_overrides[agent] — full IDs accepted; targeted exceptions
  2. Phase-type models[phase_type] — tier alias only (opus / sonnet / haiku / inherit)
  3. Profile table — the per-agent column from the active model_profile
  4. Runtime default — when nothing else applies

Why two layers above the profile?

  • Profile is a global tier strategy (everyone runs balanced).
  • models is coarse phase-level tuning without learning agent names.
  • model_overrides is per-agent precision (e.g. force haiku on gsd-codebase-mapper for a fan-out).

The three layers compose: models defaults a phase, model_overrides carves an exception out of it.

Profile Philosophy

quality - Maximum reasoning power

  • Opus for all decision-making agents
  • Sonnet for read-only verification
  • Use when: quota available, critical architecture work

balanced (default) - Smart allocation

  • Opus only for planning (where architecture decisions happen)
  • Sonnet for execution and research (follows explicit instructions)
  • Sonnet for verification (needs reasoning, not just pattern matching)
  • Use when: normal development, good balance of quality and cost

budget - Minimal Opus usage

  • Sonnet for anything that writes code
  • Haiku for research and verification
  • Use when: conserving quota, high-volume work, less critical phases

adaptive — Role-based cost optimization

  • Opus for planning and debugging (where reasoning quality has highest impact)
  • Sonnet for execution, research, and verification (follows explicit instructions)
  • Haiku for mapping, checking, and auditing (high volume, structured output)
  • Use when: optimizing cost without sacrificing plan quality, solo development on paid API tiers

inherit - Follow the current session model

  • All agents resolve to inherit
  • Best when you switch models interactively (for example OpenCode or Kilo /model)
  • Required when using non-Anthropic providers (OpenRouter, local models, etc.) — otherwise GSD may call Anthropic models directly, incurring unexpected costs
  • Use when: you want GSD to follow your currently selected runtime model

Using Non-Claude Runtimes (Codex, OpenCode, Gemini CLI, Kilo)

When installed for a non-Claude runtime, the GSD installer sets resolve_model_ids: "omit" in ~/.gsd/defaults.json. This returns an empty model parameter for all agents, so each agent uses the runtime's default model. No manual setup is needed.

To assign different models to different agents, add model_overrides with model IDs your runtime recognizes:

{
  "resolve_model_ids": "omit",
  "model_overrides": {
    "gsd-planner": "o3",
    "gsd-executor": "o4-mini",
    "gsd-debugger": "o3",
    "gsd-codebase-mapper": "o4-mini"
  }
}

The same tiering logic applies: stronger models for planning and debugging, cheaper models for execution and mapping.

Using Claude Code with Non-Anthropic Providers (OpenRouter, Local)

If you're using Claude Code with OpenRouter, a local model, or any non-Anthropic provider, set the inherit profile to prevent GSD from calling Anthropic models for subagents:

# Via settings command
/gsd-settings
# → Select "Inherit" for model profile

# Or manually in .planning/config.json
{
  "model_profile": "inherit"
}

Without inherit, GSD's default balanced profile spawns specific Anthropic models (opus, sonnet, haiku) for each agent type, which can result in additional API costs through your non-Anthropic provider.

Dynamic Routing with Failure-Tier Escalation (#3024)

When dynamic_routing.enabled = true in .planning/config.json, the resolver picks a model from a tier-mapped table based on the agent's default tier (light / standard / heavy) and escalates to the next tier up on orchestrator-detected soft failure.

{
  "dynamic_routing": {
    "enabled": true,
    "tier_models": {
      "light":    "haiku",
      "standard": "sonnet",
      "heavy":    "opus"
    },
    "escalate_on_failure": true,
    "max_escalations": 1
  }
}

Agent default tiers (each agent in MODEL_PROFILES declares one):

Tier Agents Use case
light gsd-codebase-mapper, gsd-pattern-mapper, gsd-research-synthesizer, gsd-plan-checker, gsd-integration-checker, gsd-nyquist-auditor, gsd-ui-checker, gsd-ui-auditor, gsd-doc-verifier Cheap/fast — pure mappers, scanners, low-stakes audits
standard gsd-executor, gsd-phase-researcher, gsd-project-researcher, gsd-verifier, gsd-doc-writer, gsd-ui-researcher Default workhorse — research, writing, primary verification
heavy gsd-planner, gsd-roadmapper, gsd-debugger Deep reasoning — already at top, can't escalate further

Escalation flow (orchestrator-driven):

  1. Orchestrator spawns agent with attempt: 0 → resolver returns tier_models[default_tier]
  2. If orchestrator marks the result a soft failure, it re-spawns with attempt: 1 → resolver returns tier_models[next_tier_up]
  3. max_escalations caps total retries (default 1). Beyond the cap the resolver returns the cap-tier model so the orchestrator can log without burning further budget.
  4. Hard failures (exceptions) bypass escalation and surface immediately.

Precedence with other tier sources (highest → lowest):

  1. model_overrides[<agent>] — full ID, always wins
  2. dynamic_routing.tier_models[escalated_tier] — when enabled: true
  3. models[<phase_type>] — coarse phase-level (#3023)
  4. model_profile — global tier strategy

When dynamic_routing.enabled = false (default), behavior is identical to today.

Resolution Logic

Orchestrators resolve model before spawning. The full precedence ladder is (highest → lowest):

1. Read .planning/config.json
2. Check model_overrides[<agent>] (full IDs accepted; targeted exceptions)
3. If dynamic_routing.enabled, return tier_models[escalated_tier]
   (see §Dynamic Routing — escalation steps tier up per attempt counter)
4. If no dynamic_routing match, check models[phase_type] for a phase-type tier
   (see §Per-Phase-Type Model Map for the agent → phase-type mapping)
5. If no phase-type slot, look up agent in profile table
6. Pass model parameter to Task call

The same precedence applies to reasoning_effort resolution on runtimes that support it (Codex), so model and reasoning_effort always derive from the same tier source — a models[phase_type] or dynamic_routing override flips both.

Per-Agent Overrides

Override specific agents without changing the entire profile:

{
  "model_profile": "balanced",
  "model_overrides": {
    "gsd-executor": "opus",
    "gsd-planner": "haiku"
  }
}

Overrides take precedence over the profile. Valid values: opus, sonnet, haiku, inherit, or any fully-qualified model ID (e.g., "o3", "openai/o3", "google/gemini-2.5-pro").

Switching Profiles

Runtime: /gsd-set-profile <profile>

Per-project default: Set in .planning/config.json:

{
  "model_profile": "balanced"
}

Design Rationale

Why Opus for gsd-planner? Planning involves architecture decisions, goal decomposition, and task design. This is where model quality has the highest impact.

Why Sonnet for gsd-executor? Executors follow explicit PLAN.md instructions. The plan already contains the reasoning; execution is implementation.

Why Sonnet (not Haiku) for verifiers in balanced? Verification requires goal-backward reasoning - checking if code delivers what the phase promised, not just pattern matching. Sonnet handles this well; Haiku may miss subtle gaps.

Why Haiku for gsd-codebase-mapper? Read-only exploration and pattern extraction. No reasoning required, just structured output from file contents.

Why inherit instead of passing opus directly? Claude Code's "opus" alias maps to a specific model version. Organizations may block older opus versions while allowing newer ones. GSD returns "inherit" for opus-tier agents, causing them to use whatever opus version the user has configured in their session. This avoids version conflicts and silent fallbacks to Sonnet.

Why inherit profile? Some runtimes (including OpenCode) let users switch models at runtime (/model). The inherit profile keeps all GSD subagents aligned to that live selection.