f01f6b76dd6196c8c37df2c014f6e4647f64fc0f
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tom Boucher
|
e1d661ece0 |
feat(#3024): dynamic routing with failure-tier escalation (#3031)
* feat(#3024): dynamic routing with failure-tier escalation Adds a `dynamic_routing` block to .planning/config.json that lets the resolver start agents on a cheap tier and escalate one tier up when the orchestrator detects a soft failure (verification inconclusive, plan-check FLAG, etc.). Solves the "pay Opus rates as insurance" anti-pattern by making escalation observed-quality-driven. Architecture: - AGENT_DEFAULT_TIERS map (light/standard/heavy) — every agent in MODEL_PROFILES declares a default tier; tests assert coverage so adding a new agent without updating the map fails CI. - nextTier(currentTier) helper — light → standard → heavy → heavy (heavy stays at heavy; can't go further). - resolveModelForTier(cwd, agentType, attempt) — new resolver. The orchestrator tracks the attempt counter and passes 0 for the first spawn, 1+ on escalation. The resolver caps internally at max_escalations so the orchestrator can blindly bump the counter. - Schema validation: dynamic_routing.enabled / escalate_on_failure / max_escalations / tier_models.<light|standard|heavy>. Unknown tiers and unknown sub-keys rejected at config-set time. - SDK schema mirror updated to keep CJS/SDK in lockstep (#2653). Resolution precedence (highest → lowest): 1. model_overrides[<agent>] (full IDs accepted) 2. dynamic_routing.tier_models[<tier>] (NEW; escalation-aware) 3. models[<phase_type>] (#3023 phase-type map) 4. model_profile (per-agent column) 5. Runtime default Backward compatibility: dynamic_routing is disabled by default (enabled: false or block omitted). resolveModelForTier short- circuits to resolveModelInternal in that case, so callers can adopt unconditionally without breaking existing behavior. This PR delivers the JS-layer infrastructure: schema + tier map + resolver. Orchestrator adoption (workflow markdown updates that detect soft failures and call resolveModelForTier with attempt+1) is incremental follow-up — verifier / plan-checker / integration- checker each adopt the protocol when ready. Tests (23 cases, all structural-IR — no stdout grep): - Schema invariants: AGENT_DEFAULT_TIERS coverage, VALID_AGENT_TIERS exact match, every assignment uses a valid tier - nextTier helper: light→standard→heavy→heavy, null on invalid input - Disabled mode: no block + enabled:false both no-op (back-compat) - Enabled mode: attempt=0 returns default tier model, attempt=1 escalates, beyond max_escalations caps, heavy agents stay heavy, default max_escalations=1 when omitted - Precedence: per-agent override beats dynamic_routing, dynamic_routing beats phase-type models - Validation: every settings key accepted, unknown tiers/sub-keys rejected, bare `dynamic_routing` rejected as config-set target Documentation: - get-shit-done/references/model-profiles.md — full reference section - docs/CONFIGURATION.md — full settings table + escalation flow - docs/USER-GUIDE.md — task-oriented "Cheap-by-default" section - docs/FEATURES.md — config row cross-link Verification: - 23/23 pass on regression test - 6843/6843 full suite (23 net new from 6820) - lint-no-source-grep clean (376 test files) - SDK schema mirror keeps CJS/SDK in sync per #2653 parity test Closes #3024 * fix(#3024): honor escalate_on_failure:false + 3 CR follow-ups CodeRabbit on PR #3031 (4 findings — 1 Major + 2 Minor + 1 Nitpick): 1. **Major (inline)** — get-shit-done/bin/lib/core.cjs:1668 resolveModelForTier ignored dynamic_routing.escalate_on_failure. When the user set it to false, escalation should be disabled, but the resolver only checked attempt/max_escalations. An orchestrator that always passes attempt+1 on retry would silently escalate despite the user opting out. Fix: gate effectiveAttempt on `dr.escalate_on_failure !== false` so false short-circuits every attempt back to the default tier. 2. **Minor (inline)** — docs/CONFIGURATION.md:123-126 The dynamic_routing rows in the Core Settings table had 4 cells instead of 5 (missing the Options column), breaking the table structure. Added explicit Options values for enabled / escalate_on_failure / max_escalations rows. 3. **Minor (outside-diff)** — references/model-profiles.md:179-195 "Resolution Logic" sketch was pre-#3024 and didn't include dynamic_routing in the precedence ladder. Updated to a 6-step block with dynamic_routing at step 3 (between override and phase-type). 4. **Nitpick** — tests/feat-3024-dynamic-routing.test.cjs:189+ Tests used `if (lightAgent) { ... }` guards that silent-pass when AGENT_DEFAULT_TIERS drifts. Replaced all 5 conditional skips with `assert.ok(lightAgent, '...')` preconditions so a tier-mapping change surfaces as a test failure. Plus: 2 new regression tests for the Major fix: - escalate_on_failure:false caps every attempt at default tier - escalate_on_failure:true (explicit) still escalates normally Verification: - 25/25 pass on regression test (23 prior + 2 escalate_on_failure) - 6845/6845 full suite (2 net new) - lint-no-source-grep clean * docs(#3024): align precedence + add fence language tags (CR follow-up) CodeRabbit (3 minor): 1. docs/CONFIGURATION.md:691 — "Per-Phase-Type Models → Resolution precedence" was a 4-step block written pre-#3024; readers got contradictory rules between the per-phase-type section and the later dynamic_routing section. Updated to the same 5-step ladder with dynamic_routing at step 2, and noted that dynamic_routing is disabled by default so this section's behavior is unchanged when the kill-switch is off. 2. docs/CONFIGURATION.md:770 — escalation-flow code fence missing language tag (MD040). Added `text`. 3. references/model-profiles.md:184 — resolution-ladder code fence missing language tag (MD040). Added `text`. No code changes; docs only. Verification: regression test still 25/25. * docs(#3024): clarify precedence prose — five layers, not four (CR nitpick) CodeRabbit nitpick: the "Per-Phase-Type Models → Resolution precedence" prose said "The four layers compose..." but the ladder above lists five (including Runtime default). Also "dynamic_routing escalates per-attempt above all of them" misreads as suggesting dynamic_routing wins over model_overrides — actually overrides still win at step 1. Reworded top-down so the precedence direction is unambiguous: - model_profile = base - models = phase-level override - dynamic_routing = per-attempt escalation - model_overrides = per-agent exception (top) - runtime default = fallback No code changes; docs only. * docs(#3024): note escalate_on_failure:false in escalation-flow diagram (CR) CodeRabbit nitpick: the escalation-flow diagram in docs/CONFIGURATION.md described the soft-failure → respawn → tier_models[next_tier_up] path, but didn't surface the `dynamic_routing.escalate_on_failure: false` kill-switch right next to it. Users reading the flow diagram (which is the canonical place to understand attempt behavior) wouldn't see that the kill-switch overrides the soft-failure branch. Added a one-paragraph note immediately after the flow listing, before the tier-sequence example, so the kill-switch is visible exactly where users decide whether escalation will happen. No code changes; docs only. |
||
|
Tom Boucher
|
d812c66020 |
feat(#3023): per-phase-type model map in .planning/config.json (#3030)
* feat(#3023): per-phase-type model map in .planning/config.json Adds a new `models` block to .planning/config.json with six phase-type slots (planning / discuss / research / execution / verification / completion). Lets users express coarse tuning ("Opus for planning, Sonnet for the rest") without learning the agent taxonomy. Resolution precedence (highest → lowest): 1. Per-agent `model_overrides[agent]` (full IDs; targeted exception) 2. Phase-type `models[phase_type]` (NEW; tier alias) 3. Profile table (`model_profile`) (per-agent column) 4. Runtime default The three layers compose: `models` defaults a phase, `model_overrides` carves an exception. Phase-type values are tier aliases (opus/sonnet/ haiku/inherit) so the runtime-resolution chain (#2517) stays correct end-to-end without further branching. Implementation: - model-profiles.cjs: new AGENT_TO_PHASE_TYPE map + VALID_PHASE_TYPES set. Each agent in MODEL_PROFILES gets one phase-type assignment; tests assert coverage so adding a new agent without updating the table fails CI. - core.cjs (resolveModelInternal): inserts phase-type tier lookup between per-agent override and profile-derived tier. Skips runtime resolution when the resolved tier is 'inherit' (was previously gated only on profile === 'inherit'; phase-type can now produce inherit independently). - core.cjs (loadConfig): pass `parsed.models` through both code paths so resolveModelInternal can read it. - config-schema.cjs + sdk/src/query/config-schema.ts: dynamic-pattern validator accepts only the six known phase-types; unknown slots rejected at config-set time. Backward compat: configs without `models` behave exactly as today. Tests (15 cases, all structural-IR — no stdout grep): - Schema: AGENT_TO_PHASE_TYPE coverage, VALID_PHASE_TYPES exact match - Resolver: phase-type alone; per-agent override beats phase-type; phase-type beats profile; issue's full example; "inherit"; empty block is no-op; no block is no-op - Validation: each of the 6 slots accepted; unknown slot rejected; bare `models` (no slot) rejected Verification: - 15/15 pass on new regression test - 6808/6808 full suite (5 net new), 0 fail - lint-no-source-grep clean across 375 test files Closes #3023 * docs(#3023): document `models` per-phase-type config in user-facing docs Adds `models` block coverage to the three user-facing docs that ship with each release: 1. docs/CONFIGURATION.md - New "Per-Phase-Type Models" section between "Per-Agent Overrides" and "Non-Claude Runtimes" with: * full example mixing models + model_overrides * phase-type → agent mapping table * resolution-precedence pseudocode * accepted values (tier alias only) * "When to use which" decision matrix * validation behavior + example error - Added `"models": {}` to the Full Schema snippet - Added a row for `models.<phase_type>` to the config keys table (next to model_profile_overrides for adjacency) 2. docs/FEATURES.md - Added a row for models.<phase_type> in the Configurable Settings table (right under model_profile) - Cross-link to CONFIGURATION.md for the full surface 3. docs/USER-GUIDE.md - New task-oriented "Tuning model cost by phase" section above "Using Non-Claude Runtimes" — leads with the concrete config and shows the override pattern (one-shot phase + targeted exception) - Cross-link to CONFIGURATION.md Verification: - 29/29 pass on config-schema-docs-parity + docs-update + new feature test (parity-check passes, so the config-schema entry I added in the feature commit is now matched by a docs row) - 6808/6808 full suite pass - lint-no-source-grep clean Doc style follows the same pattern used by the existing model_profile, model_overrides, and model_profile_overrides sections — example-led, table-backed, cross-referenced. Each doc surfaces the feature at the right depth (reference / settings table / task guide). * fix(#3023): mirror phase-type tier in resolveReasoningEffortInternal (CR Major) CodeRabbit caught a real Codex correctness bug + 3 minor docs/test issues: 1. **Major (outside-diff)** — resolveReasoningEffortInternal in core.cjs derived its tier exclusively from the profile table, ignoring the models.<phase_type> override added in #3023. Failure mode on Codex: Config: model_profile=balanced, models.execution=opus, agent=gsd-executor resolveModelInternal: tier=opus → gpt-5.4 resolveReasoningEffortInternal: tier=sonnet → reasoning_effort=medium ↑ WRONG — should be xhigh (opus tier on Codex) The runtime received a mismatched (model, effort) pair. Mirrored the phase-type lookup from resolveModelInternal so both functions derive from the same tier source. 'inherit' phase-type returns null effort (no runtime entry maps to 'inherit'; let runtime decide). 2. Minor — .changeset/per-phase-type-models.md `pr: TBD` → `pr: 3030`. 3. Minor (outside-diff) — model-profiles.md "Resolution Logic" section omitted the new phase-type tier. Updated the 4-step block to a 5-step block including `models[phase_type]` between override and profile, plus a paragraph noting that `model` and `reasoning_effort` derive from the same tier source. 4. Nitpick — added 2 typo-safety tests: - models.research = "haiku3" (typo) → falls through to profile - models.research = "openai/gpt-5" (full ID) → falls through to profile Plus 5 new reasoning_effort tests covering the Major fix: - exported correctly - phase-type override flips both model AND effort to same tier - inherit phase-type returns null effort - per-agent override still bypasses phase-type for effort - claude runtime ignores models.* (no effort propagation) Verification: - 24/24 pass on regression test (15 original + 2 typo-safety + 5 effort + 2 outside-diff related) - 6815/6815 full suite (7 net new from 6808) - lint-no-source-grep clean The reasoning_effort tests are written semantically (phase-type override must produce the SAME effort as a profile-only opus config) rather than hard-coding tier-specific effort strings, so changes to the runtime tier map don't break them. * fix(#3023): phase-type override beats profile=inherit (CR Major round 2) CodeRabbit caught another precedence inversion: when { model_profile: 'inherit', models: { execution: 'opus' } } both resolvers short-circuited on `profile === 'inherit'` BEFORE the phase-type override could be honored. Result: model returned 'inherit' and reasoning_effort returned null — both contradicting the documented precedence where models[phase_type] wins over model_profile. Fix in resolveModelInternal: - Compute tier from phase-type FIRST. If phase-type is a valid alias, it wins. Otherwise, fall back to profile-derived tier OR 'inherit' (when profile === 'inherit'). - Gate the runtime-resolution branch on `tier !== 'inherit'` (was `profile !== 'inherit'`) so phase-type=opus can flip runtime mapping on even when profile=inherit. - Gate the inherit-return on `tier === 'inherit'` (was `profile === 'inherit'`). Fix in resolveReasoningEffortInternal: - Remove the `if (profile === 'inherit') return null;` early-return. - Compute tier from phase-type first, fall back to profile. If phase-type is explicitly 'inherit' OR the resolved tier is 'inherit', return null (no runtime entry maps to inherit). Tests added (5 new): - model: phase-type wins over profile=inherit (with explicit opus, with haiku for one phase + planner-without-slot still inheriting) - model: profile=inherit + no models block → all agents inherit (no regression on existing inherit semantics) - model: profile=inherit + models block but agent has no slot → that agent inherits, agents with slots get phase-type tier - effort: phase-type opus + profile=inherit → produces opus-tier effort, NOT null (the original bug) Verification: - 27/27 pass on regression test (24 prior + 3 model + 1 effort) - 6820/6820 full suite (5 net new) - lint-no-source-grep clean The effort test reads the expected value by running a profile-only opus config and comparing — semantic check, not hard-coded effort string. So runtime tier map changes don't break the test. |
||
|
Tom Boucher
|
8de8acee46 |
fix(workflows): assert HEAD on per-agent branch before worktree commits (#2924) (#2941)
* fix(workflows): assert HEAD on per-agent branch before worktree commits Worktree-mode setup could leave HEAD attached to a protected branch (master), causing agent commits to land there. The previous response was a destructive self-recovery via 'git update-ref refs/heads/master <sha>', which silently rewinds the protected branch and destroys concurrent commits in multi-active scenarios (parallel agents, user committing while agent runs). - Reorder <worktree_branch_check> in execute-phase.md and quick.md to assert HEAD via 'git symbolic-ref' BEFORE any 'git reset --hard'. HALT with a blocker if HEAD is on main/master/develop/trunk/release/* or detached. - Add a per-commit HEAD assertion (step 0) to gsd-executor.md <task_commit_protocol>; HEAD attachment can drift after 'git checkout <sha>'. - Forbid 'git update-ref refs/heads/<protected>' in <destructive_git_prohibition>; surface the blocker rather than self-heal. - Remove '--no-verify' as the worktree-mode default in execute-phase.md, execute-plan.md, quick.md, and references/git-integration.md. Hooks now run on every executor commit; opt out only via workflow.worktree_skip_hooks. - Add regression test that parses the worktree_branch_check blocks structurally and asserts the symbolic-ref check precedes the reset --hard, no workflow performs update-ref on a protected ref, and --no-verify is no longer the default in any parallel-execution prompt. * fix(#2924): address CodeRabbit review findings on worktree HEAD PR - Add positive worktree-agent-* allow-list to <task_commit_protocol> step 0 in gsd-executor.md and to <worktree_branch_check> in execute-phase.md and quick.md. The deny-list (main|master|develop|trunk|release/*) silently allowed feature/* and other arbitrary branches outside the agent namespace. - Register workflow.worktree_skip_hooks in both config schemas (sdk/src/query/config-schema.ts and get-shit-done/bin/lib/config-schema.cjs) and document it in docs/CONFIGURATION.md so config-set accepts it. - Fix stash lifecycle in execute-phase.md post-wave hook validation: stash under a named ref and pop after the hook run; warn on pop failure. - Pre-dispatch PLAN.md commit in quick.md: gate on git diff --cached --quiet for idempotency and exit 1 with a clear error on commit failure (both the --no-verify and the normal branches) — no more swallowing real errors. - Test fixes (tests/bug-2924-worktree-head-attachment.test.cjs): - Parse the protected-branch alternation structurally and require main, master, develop, trunk, release/.* (release/* was previously skipped by the \\b...\\b regex). - Use fs.readdirSync(dir, { recursive: true }) so workflows in nested subdirectories are also asserted against the update-ref ban. - Add allow-list assertions for execute-phase.md, quick.md, and gsd-executor.md to lock in the new positive namespace check. * test(#2924): assert sub-section end marker exists before slicing * test(#2924): use section boundary instead of fixed window for parallel-agents slice |
||
|
Tom Boucher
|
b40110111d |
feat(#2306): plan-review-convergence v2 — CYCLE_SUMMARY contract, config gate, local model reviewers (#2718)
* feat(#2306): plan-review-convergence v2 — CYCLE_SUMMARY contract, config gate, local model reviewers Fixes the false-stall detection bug in the plan→review→replan convergence loop. REVIEWS.md accumulates history across cycles so raw grep inflated HIGH counts; HIGH count now comes from a per-cycle CYCLE_SUMMARY contract emitted in the review agent's return message. Key changes: - workflow.plan_review_convergence config gate (disabled by default, same pattern as workflow.code_review / workflow.nyquist_validation) - Review agent prompt defines CYCLE_SUMMARY: current_high=<N> contract with PARTIALLY RESOLVED / FULLY RESOLVED counting rules - Orchestrator aborts on absent/malformed CYCLE_SUMMARY (distinguishes both) - Warns when HIGH_COUNT > 0 but ## Current HIGH Concerns section is missing - Stall detection and --ws forwarding preserved and tested - Local model reviewers: --ollama, --lm-studio, --llama-cpp flags added to convergence workflow and review workflow; all three use OpenAI-compatible /v1/chat/completions endpoint with jq --rawfile for safe JSON encoding - review.ollama_host / review.lm_studio_host / review.llama_cpp_host config keys registered and documented (default to localhost:11434/1234/8080) - review.models.ollama / .lm_studio / .llama_cpp model-name config support - 58 tests (up from 29 in PR #2339), all passing Closes #2306 Closes #2339 Co-authored-by: Tom Boucher <trekkie@nomorestars.com> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(ci): sync sdk/src/query/config-schema.ts with CJS schema (#2306) Add workflow.plan_review_convergence, review.ollama_host, review.lm_studio_host, and review.llama_cpp_host to the SDK-side TypeScript mirror — required by the CJS↔SDK parity test (#2653). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#2306): resolve CodeRabbit review findings - Anchor HIGH_COUNT extraction with head -1 to prevent multi-match when agent return message contains multiple CYCLE_SUMMARY lines (e.g. quoted back from prompt context) - Replace hardcoded reviewers list in REVIEWS.md frontmatter template with runtime-derived placeholder — the static list did not reflect which reviewers were actually invoked - Broaden workflow.plan_review_convergence docs to include local reviewers (Ollama, LM Studio, llama.cpp) alongside cloud reviewers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(ci): restore reviewers frontmatter list with runtime note The cursor-reviewer.test.cjs (and equivalent per-reviewer tests) assert that each supported reviewer appears on the reviewers: line — these are wiring tests that catch when a new reviewer is added to invocation but not to the REVIEWS.md template. Replacing the list with a placeholder broke those tests. Restore the full static list and add an inline comment clarifying that the actual committed frontmatter should be filtered to only the reviewers invoked that run — satisfying both the per-reviewer tests and the CodeRabbit correctness note. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Tom Boucher
|
387c8a1f9c |
fix(#2653): eliminate SDK↔CJS config-schema drift (#2670)
The SDK's config-set kept its own hand-maintained allowlist (28-key drift vs. get-shit-done/bin/lib/config-schema.cjs), so documented keys accepted by the CJS config-set — planning.sub_repos, workflow.code_review_command, workflow.security_*, review.models.*, model_profile_overrides.*, etc. — were rejected with "Unknown config key" when routed through the SDK. Changes: - New sdk/src/query/config-schema.ts mirrors the CJS schema exactly (exact-match keys + dynamic regex sources). - config-mutation.ts imports VALID_CONFIG_KEYS / DYNAMIC_KEY_PATTERNS from the shared module instead of rolling its own set and regex branches. - Drop hand-coded agent_skills.* / features.* regex branches — now schema-driven so claude_md_assembly.blocks.*, review.models.*, and model_profile_overrides.<runtime>.<tier> are also accepted. - Add tests/config-schema-sdk-parity.test.cjs (node:test) as the CI drift guard: asserts CJS VALID_CONFIG_KEYS set-equals the literal set parsed from config-schema.ts, and that every CJS dynamic pattern source has an identical counterpart in the SDK. Parallel to the CJS↔docs parity added in #2479. - Vitest #2653 specs iterate every CJS key through the SDK validator, spot-check each dynamic pattern, and lock in planning.sub_repos. - While here: add workflow.context_coverage_gate to the CJS schema (already in docs and SDK; CJS previously rejected it) and sync the missing curated typo-suggestions (review.model, sub_repos, plan_checker, workflow.review_command) into the SDK. Fixes #2653. |