* feat(#3024): dynamic routing with failure-tier escalation Adds a `dynamic_routing` block to .planning/config.json that lets the resolver start agents on a cheap tier and escalate one tier up when the orchestrator detects a soft failure (verification inconclusive, plan-check FLAG, etc.). Solves the "pay Opus rates as insurance" anti-pattern by making escalation observed-quality-driven. Architecture: - AGENT_DEFAULT_TIERS map (light/standard/heavy) — every agent in MODEL_PROFILES declares a default tier; tests assert coverage so adding a new agent without updating the map fails CI. - nextTier(currentTier) helper — light → standard → heavy → heavy (heavy stays at heavy; can't go further). - resolveModelForTier(cwd, agentType, attempt) — new resolver. The orchestrator tracks the attempt counter and passes 0 for the first spawn, 1+ on escalation. The resolver caps internally at max_escalations so the orchestrator can blindly bump the counter. - Schema validation: dynamic_routing.enabled / escalate_on_failure / max_escalations / tier_models.<light|standard|heavy>. Unknown tiers and unknown sub-keys rejected at config-set time. - SDK schema mirror updated to keep CJS/SDK in lockstep (#2653). Resolution precedence (highest → lowest): 1. model_overrides[<agent>] (full IDs accepted) 2. dynamic_routing.tier_models[<tier>] (NEW; escalation-aware) 3. models[<phase_type>] (#3023 phase-type map) 4. model_profile (per-agent column) 5. Runtime default Backward compatibility: dynamic_routing is disabled by default (enabled: false or block omitted). resolveModelForTier short- circuits to resolveModelInternal in that case, so callers can adopt unconditionally without breaking existing behavior. This PR delivers the JS-layer infrastructure: schema + tier map + resolver. Orchestrator adoption (workflow markdown updates that detect soft failures and call resolveModelForTier with attempt+1) is incremental follow-up — verifier / plan-checker / integration- checker each adopt the protocol when ready. Tests (23 cases, all structural-IR — no stdout grep): - Schema invariants: AGENT_DEFAULT_TIERS coverage, VALID_AGENT_TIERS exact match, every assignment uses a valid tier - nextTier helper: light→standard→heavy→heavy, null on invalid input - Disabled mode: no block + enabled:false both no-op (back-compat) - Enabled mode: attempt=0 returns default tier model, attempt=1 escalates, beyond max_escalations caps, heavy agents stay heavy, default max_escalations=1 when omitted - Precedence: per-agent override beats dynamic_routing, dynamic_routing beats phase-type models - Validation: every settings key accepted, unknown tiers/sub-keys rejected, bare `dynamic_routing` rejected as config-set target Documentation: - get-shit-done/references/model-profiles.md — full reference section - docs/CONFIGURATION.md — full settings table + escalation flow - docs/USER-GUIDE.md — task-oriented "Cheap-by-default" section - docs/FEATURES.md — config row cross-link Verification: - 23/23 pass on regression test - 6843/6843 full suite (23 net new from 6820) - lint-no-source-grep clean (376 test files) - SDK schema mirror keeps CJS/SDK in sync per #2653 parity test Closes #3024 * fix(#3024): honor escalate_on_failure:false + 3 CR follow-ups CodeRabbit on PR #3031 (4 findings — 1 Major + 2 Minor + 1 Nitpick): 1. **Major (inline)** — get-shit-done/bin/lib/core.cjs:1668 resolveModelForTier ignored dynamic_routing.escalate_on_failure. When the user set it to false, escalation should be disabled, but the resolver only checked attempt/max_escalations. An orchestrator that always passes attempt+1 on retry would silently escalate despite the user opting out. Fix: gate effectiveAttempt on `dr.escalate_on_failure !== false` so false short-circuits every attempt back to the default tier. 2. **Minor (inline)** — docs/CONFIGURATION.md:123-126 The dynamic_routing rows in the Core Settings table had 4 cells instead of 5 (missing the Options column), breaking the table structure. Added explicit Options values for enabled / escalate_on_failure / max_escalations rows. 3. **Minor (outside-diff)** — references/model-profiles.md:179-195 "Resolution Logic" sketch was pre-#3024 and didn't include dynamic_routing in the precedence ladder. Updated to a 6-step block with dynamic_routing at step 3 (between override and phase-type). 4. **Nitpick** — tests/feat-3024-dynamic-routing.test.cjs:189+ Tests used `if (lightAgent) { ... }` guards that silent-pass when AGENT_DEFAULT_TIERS drifts. Replaced all 5 conditional skips with `assert.ok(lightAgent, '...')` preconditions so a tier-mapping change surfaces as a test failure. Plus: 2 new regression tests for the Major fix: - escalate_on_failure:false caps every attempt at default tier - escalate_on_failure:true (explicit) still escalates normally Verification: - 25/25 pass on regression test (23 prior + 2 escalate_on_failure) - 6845/6845 full suite (2 net new) - lint-no-source-grep clean * docs(#3024): align precedence + add fence language tags (CR follow-up) CodeRabbit (3 minor): 1. docs/CONFIGURATION.md:691 — "Per-Phase-Type Models → Resolution precedence" was a 4-step block written pre-#3024; readers got contradictory rules between the per-phase-type section and the later dynamic_routing section. Updated to the same 5-step ladder with dynamic_routing at step 2, and noted that dynamic_routing is disabled by default so this section's behavior is unchanged when the kill-switch is off. 2. docs/CONFIGURATION.md:770 — escalation-flow code fence missing language tag (MD040). Added `text`. 3. references/model-profiles.md:184 — resolution-ladder code fence missing language tag (MD040). Added `text`. No code changes; docs only. Verification: regression test still 25/25. * docs(#3024): clarify precedence prose — five layers, not four (CR nitpick) CodeRabbit nitpick: the "Per-Phase-Type Models → Resolution precedence" prose said "The four layers compose..." but the ladder above lists five (including Runtime default). Also "dynamic_routing escalates per-attempt above all of them" misreads as suggesting dynamic_routing wins over model_overrides — actually overrides still win at step 1. Reworded top-down so the precedence direction is unambiguous: - model_profile = base - models = phase-level override - dynamic_routing = per-attempt escalation - model_overrides = per-agent exception (top) - runtime default = fallback No code changes; docs only. * docs(#3024): note escalate_on_failure:false in escalation-flow diagram (CR) CodeRabbit nitpick: the escalation-flow diagram in docs/CONFIGURATION.md described the soft-failure → respawn → tier_models[next_tier_up] path, but didn't surface the `dynamic_routing.escalate_on_failure: false` kill-switch right next to it. Users reading the flow diagram (which is the canonical place to understand attempt behavior) wouldn't see that the kill-switch overrides the soft-failure branch. Added a one-paragraph note immediately after the flow listing, before the tier-sequence example, so the kill-switch is visible exactly where users decide whether escalation will happen. No code changes; docs only.
133 lines
5.3 KiB
TypeScript
133 lines
5.3 KiB
TypeScript
/**
|
|
* SDK-side mirror of get-shit-done/bin/lib/config-schema.cjs.
|
|
*
|
|
* Single source of truth for valid config key paths accepted by
|
|
* `config-set`. MUST stay in sync with the CJS schema — enforced
|
|
* by tests/config-schema-sdk-parity.test.cjs (CI drift guard).
|
|
*
|
|
* If you add/remove a key here, make the identical change in
|
|
* get-shit-done/bin/lib/config-schema.cjs (and vice versa). The
|
|
* parity test asserts the two allowlists are set-equal and that
|
|
* DYNAMIC_KEY_PATTERN_SOURCES produce identical regex source strings.
|
|
*
|
|
* See #2653 — CJS/SDK drift caused config-set to reject documented
|
|
* keys. #2479 added CJS↔docs parity; #2653 adds CJS↔SDK parity.
|
|
*/
|
|
|
|
/** Exact-match config key paths accepted by config-set. */
|
|
export const VALID_CONFIG_KEYS: ReadonlySet<string> = new Set([
|
|
'mode', 'granularity', 'parallelization', 'commit_docs', 'model_profile',
|
|
'search_gitignored', 'brave_search', 'firecrawl', 'exa_search',
|
|
'workflow.research', 'workflow.plan_check', 'workflow.verifier',
|
|
'workflow.nyquist_validation', 'workflow.ai_integration_phase', 'workflow.ui_phase', 'workflow.ui_safety_gate',
|
|
'workflow.auto_advance', 'workflow.node_repair', 'workflow.node_repair_budget',
|
|
'workflow.tdd_mode',
|
|
'workflow.text_mode',
|
|
'workflow.research_before_questions',
|
|
'workflow.discuss_mode',
|
|
'workflow.skip_discuss',
|
|
'workflow.auto_prune_state',
|
|
'workflow.use_worktrees',
|
|
'workflow.worktree_skip_hooks',
|
|
'workflow.code_review',
|
|
'workflow.code_review_depth',
|
|
'workflow.code_review_command',
|
|
'workflow.pattern_mapper',
|
|
'workflow.plan_bounce',
|
|
'workflow.plan_bounce_script',
|
|
'workflow.plan_bounce_passes',
|
|
'workflow.plan_chunked',
|
|
'workflow.plan_review_convergence',
|
|
'workflow.post_planning_gaps',
|
|
'workflow.security_enforcement',
|
|
'workflow.security_asvs_level',
|
|
'workflow.security_block_on',
|
|
'workflow.drift_threshold',
|
|
'workflow.drift_action',
|
|
'git.branching_strategy', 'git.base_branch', 'git.phase_branch_template', 'git.milestone_branch_template', 'git.quick_branch_template',
|
|
'planning.commit_docs', 'planning.search_gitignored', 'planning.sub_repos',
|
|
'review.ollama_host', 'review.lm_studio_host', 'review.llama_cpp_host',
|
|
'workflow.cross_ai_execution', 'workflow.cross_ai_command', 'workflow.cross_ai_timeout',
|
|
'workflow.subagent_timeout',
|
|
'workflow.inline_plan_threshold',
|
|
'hooks.context_warnings',
|
|
'hooks.workflow_guard',
|
|
'workflow.context_coverage_gate',
|
|
'statusline.show_last_command',
|
|
'workflow.ui_review',
|
|
'workflow.max_discuss_passes',
|
|
'features.thinking_partner',
|
|
'context',
|
|
'features.global_learnings',
|
|
'learnings.max_inject',
|
|
'project_code', 'phase_naming',
|
|
'manager.flags.discuss', 'manager.flags.plan', 'manager.flags.execute',
|
|
'response_language',
|
|
'context_window',
|
|
'intel.enabled',
|
|
'graphify.enabled',
|
|
'graphify.build_timeout',
|
|
'claude_md_path',
|
|
'claude_md_assembly.mode',
|
|
// #2517 — runtime-aware model profiles
|
|
'runtime',
|
|
]);
|
|
|
|
/**
|
|
* Dynamic-pattern validators — keys matching these regexes are also accepted.
|
|
* Each entry's `source` MUST equal the corresponding CJS regex `.source`
|
|
* (the parity test enforces this).
|
|
*/
|
|
export interface DynamicKeyPattern {
|
|
readonly test: (k: string) => boolean;
|
|
readonly description: string;
|
|
readonly source: string;
|
|
}
|
|
|
|
export const DYNAMIC_KEY_PATTERNS: readonly DynamicKeyPattern[] = [
|
|
{
|
|
source: '^agent_skills\\.[a-zA-Z0-9_-]+$',
|
|
description: 'agent_skills.<agent-type>',
|
|
test: (k) => /^agent_skills\.[a-zA-Z0-9_-]+$/.test(k),
|
|
},
|
|
{
|
|
source: '^review\\.models\\.[a-zA-Z0-9_-]+$',
|
|
description: 'review.models.<cli-name>',
|
|
test: (k) => /^review\.models\.[a-zA-Z0-9_-]+$/.test(k),
|
|
},
|
|
{
|
|
source: '^features\\.[a-zA-Z0-9_]+$',
|
|
description: 'features.<feature_name>',
|
|
test: (k) => /^features\.[a-zA-Z0-9_]+$/.test(k),
|
|
},
|
|
{
|
|
source: '^claude_md_assembly\\.blocks\\.[a-zA-Z0-9_]+$',
|
|
description: 'claude_md_assembly.blocks.<section>',
|
|
test: (k) => /^claude_md_assembly\.blocks\.[a-zA-Z0-9_]+$/.test(k),
|
|
},
|
|
// #2517 — runtime-aware model profile overrides: model_profile_overrides.<runtime>.<tier>
|
|
{
|
|
source: '^model_profile_overrides\\.[a-zA-Z0-9_-]+\\.(opus|sonnet|haiku)$',
|
|
description: 'model_profile_overrides.<runtime>.<opus|sonnet|haiku>',
|
|
test: (k) => /^model_profile_overrides\.[a-zA-Z0-9_-]+\.(opus|sonnet|haiku)$/.test(k),
|
|
},
|
|
// #3023 — per-phase-type model map: models.<phase_type> = <tier>
|
|
{
|
|
source: '^models\\.(planning|discuss|research|execution|verification|completion)$',
|
|
description: 'models.<planning|discuss|research|execution|verification|completion>',
|
|
test: (k) => /^models\.(planning|discuss|research|execution|verification|completion)$/.test(k),
|
|
},
|
|
// #3024 — dynamic routing with failure-tier escalation
|
|
{
|
|
source: '^dynamic_routing\\.(enabled|escalate_on_failure|max_escalations|tier_models\\.(light|standard|heavy))$',
|
|
description: 'dynamic_routing.<enabled|escalate_on_failure|max_escalations|tier_models.<light|standard|heavy>>',
|
|
test: (k) => /^dynamic_routing\.(enabled|escalate_on_failure|max_escalations|tier_models\.(light|standard|heavy))$/.test(k),
|
|
},
|
|
];
|
|
|
|
/** Returns true if keyPath is a valid config key (exact or dynamic pattern). */
|
|
export function isValidConfigKeyPath(keyPath: string): boolean {
|
|
if (VALID_CONFIG_KEYS.has(keyPath)) return true;
|
|
return DYNAMIC_KEY_PATTERNS.some((p) => p.test(keyPath));
|
|
}
|