diff --git a/.changeset/silly-jaguars-swim.md b/.changeset/silly-jaguars-swim.md new file mode 100644 index 000000000..325d2f91f --- /dev/null +++ b/.changeset/silly-jaguars-swim.md @@ -0,0 +1,5 @@ +--- +type: Added +pr: 49 +--- +Add `model_policy` config surface with known-provider presets (openai/anthropic/google/qwen) and `generic` provider escape hatch. `model_policy.runtime_tiers` resolves before legacy `model_profile_overrides`. `reasoning_effort` is stripped for unsupported runtimes. diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 2745ac932..d6c2ac67e 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -139,6 +139,12 @@ GSD stores project settings in `.planning/config.json`. Created during `/gsd-new | `model_profile` | enum | `quality`, `balanced`, `budget`, `adaptive`, `inherit` | `balanced` | Model tier for each agent (see [Model Profiles](#model-profiles)). `adaptive` was added per [#1713](https://github.com/open-gsd/gsd-core/issues/1713) / [#1806](https://github.com/open-gsd/gsd-core/issues/1806) and resolves the same way as the other tiers under runtime-aware profiles. | | `runtime` | string | `claude`, `codex`, or any string | (none) | Active runtime for [runtime-aware profile resolution](#runtime-aware-profiles-2517). When set, profile tiers (opus/sonnet/haiku) resolve to runtime-native model IDs. Today only the Codex install path emits per-agent model IDs from this resolver; other runtimes (`opencode`, `gemini`, `qwen`, `copilot`, …) consume the resolver at spawn time and gain dedicated install-path support in [#2612](https://github.com/open-gsd/gsd-core/issues/2612). When unset (default), behavior is unchanged from prior versions. Added in v1.39 | | `model_profile_overrides..` | string \| object | per-runtime tier override | (none) | Override the runtime-aware tier mapping for a specific `(runtime, tier)`. Tier is one of `opus`, `sonnet`, `haiku`. Value is either a model ID string (e.g. `"gpt-5-pro"`) or `{ model, reasoning_effort }`. See [Runtime-Aware Profiles](#runtime-aware-profiles-2517). Added in v1.39 | +| `model_policy.provider` | string | `openai`, `anthropic`, `google`, `qwen`, `generic` | (none) | Declares the model provider. Known providers (`openai`, `anthropic`, `google`, `qwen`) unlock catalog-backed presets. `generic` treats all model IDs as opaque strings — no prefix inference, no reasoning-effort defaults. `model_policy.runtime_tiers` resolves before legacy `model_profile_overrides`. See [Model Policy Presets](#model-policy-presets-model_policy--added-in-v142). Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) | +| `model_policy.budget` | enum | `high`, `medium`, `low` | (none) | Selects a budget tier when using a known provider. GSD materializes the matching catalog preset into explicit tier mappings at resolve time. Ignored when `provider` is `generic` or `custom`. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) | +| `model_policy.high` | string | model ID | (none) | High-cost tier model ID for `generic`/`custom` provider. Used when `provider: "generic"` or `"custom"`. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) | +| `model_policy.medium` | string | model ID | (none) | Medium-cost tier model ID for `generic`/`custom` provider. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) | +| `model_policy.low` | string | model ID | (none) | Low-cost tier model ID for `generic`/`custom` provider. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) | +| `model_policy.runtime_tiers..` | object | `{ model, reasoning_effort? }` | (none) | Explicit per-runtime, per-tier model entry. `tier` is one of `opus`, `sonnet`, `haiku` (matching the existing profile tier names). `reasoning_effort` is forwarded only to runtimes that support it; unsupported runtimes never receive the field. Takes precedence over `model_profile_overrides`. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) | | `models.` | enum | `opus`, `sonnet`, `haiku`, `inherit` | (none) | Per-phase-type model tier. Six accepted slots: `planning`, `discuss`, `research`, `execution`, `verification`, `completion`. Lets you tune at the phase level ("Opus for planning, Sonnet for the rest") without learning agent names. Resolves between `model_overrides` (higher) and `model_profile` (lower); see [Per-Phase-Type Models](#per-phase-type-models-models--added-in-v140). Added in v1.40 ([#3023](https://github.com/open-gsd/gsd-core/pull/3030)) | | `dynamic_routing.enabled` | boolean | `true`, `false` | `false` | Master switch for [dynamic routing with failure-tier escalation](#dynamic-routing-with-failure-tier-escalation-dynamic_routing--added-in-v140). When `true`, agents resolve to `tier_models[default_tier]` and escalate one tier up on orchestrator-detected soft failure. Added in v1.40 ([#3024](https://github.com/open-gsd/gsd-core/pull/3031)) | | `dynamic_routing.tier_models.` | enum | `opus`, `sonnet`, `haiku` | (none) | Tier alias for `light`, `standard`, or `heavy`. Used when `dynamic_routing.enabled: true`. Added in v1.40 | @@ -1218,6 +1224,84 @@ This resolves `gsd-planner` → `gpt-5.5` (xhigh), `gsd-executor` → `gpt-5.3-c --- +## Model Policy Presets (`model_policy`) — Added in v1.42 + +> **[#49](https://github.com/open-gsd/gsd-core/issues/49)** — provider-neutral model policy config surface. Resolves before legacy `model_profile_overrides`. + +`model_policy` provides a simpler, provider-neutral way to configure model tiers across runtimes. It is the preferred surface for non-Anthropic runtimes where `model_profile_overrides` would require manually knowing the right model IDs. Configure it via `/gsd:settings` → Section 8 (Model Policy). + +### Known provider preset + +Choose a provider and budget level via the settings workflow; GSD writes the canonical model IDs for that provider/budget combination: + +```json +{ + "runtime": "codex", + "model_policy": { + "provider": "openai", + "budget": "medium", + "high": "gpt-5.5", + "medium": "gpt-5.3-codex", + "low": "gpt-5.4-mini" + } +} +``` + +Known providers: `openai`, `anthropic`, `google`, `qwen`. Budget levels: `high`, `medium`, `low`. + +For advanced per-runtime control, `runtime_tiers` accepts explicit entries using the internal profile tier names (`opus`, `sonnet`, `haiku`): + +```json +{ + "runtime": "codex", + "model_policy": { + "provider": "openai", + "runtime_tiers": { + "codex": { + "opus": { "model": "gpt-5.5", "reasoning_effort": "high" }, + "sonnet": { "model": "gpt-5.3-codex", "reasoning_effort": "medium" }, + "haiku": { "model": "gpt-5.4-mini", "reasoning_effort": "low" } + } + } + } +} +``` + +### Generic provider (escape hatch) + +Use `provider: "generic"` (or `"custom"`) for OpenRouter, LiteLLM, local gateways, or any runtime where you supply exact model IDs. GSD treats model IDs as opaque strings — no prefix inference, no provider-specific defaults: + +```json +{ + "runtime": "opencode", + "model_policy": { + "provider": "generic", + "high": "openrouter/anthropic/claude-opus-4-5", + "medium": "openrouter/anthropic/claude-sonnet-4-5", + "low": "openrouter/anthropic/claude-haiku-4-5" + } +} +``` + +### Reasoning effort gating + +`reasoning_effort` within a `runtime_tiers` entry is forwarded only to runtimes that declare support for it (currently: `codex`). Any runtime not on the allowlist receives the tier entry without the `reasoning_effort` field — it is silently stripped, never leaked. + +### Precedence + +`model_policy` resolution sits above `model_profile_overrides` in the resolver: + +1. `model_overrides[]` — per-agent explicit ID (highest) +2. `model_policy.runtime_tiers[][]` — explicit runtime/tier entry +3. `model_policy` flat `high`/`medium`/`low` keys — for `generic`/`custom` provider +4. `model_profile_overrides[][]` — legacy per-runtime override +5. Built-in runtime catalog default +6. `model_profile` tier alias + +**Backwards compatibility.** Configs without `model_policy` are unaffected. Existing `model_profile_overrides` blocks continue to work exactly as before. + +--- + ## Environment Variables | Variable | Purpose | diff --git a/docs/INVENTORY-MANIFEST.json b/docs/INVENTORY-MANIFEST.json index d3f3b3fcb..6f8bd3005 100644 --- a/docs/INVENTORY-MANIFEST.json +++ b/docs/INVENTORY-MANIFEST.json @@ -275,6 +275,7 @@ "command-routing-hub.cjs", "commands.cjs", "config-schema.cjs", + "config-types.cjs", "config.cjs", "configuration.cjs", "context-utilization.cjs", diff --git a/docs/INVENTORY.md b/docs/INVENTORY.md index a8ff3c9d5..052340ca2 100644 --- a/docs/INVENTORY.md +++ b/docs/INVENTORY.md @@ -362,7 +362,7 @@ The `gsd-planner` agent is decomposed into a core agent plus reference modules t --- -## CLI Modules (80 shipped) +## CLI Modules (81 shipped) Full listing: `get-shit-done/bin/lib/*.cjs`. @@ -384,6 +384,7 @@ Full listing: `get-shit-done/bin/lib/*.cjs`. | `commands.cjs` | Misc CLI commands (slug, timestamp, todos, scaffolding, stats) | | `config-schema.cjs` | Single source of truth for `VALID_CONFIG_KEYS` and dynamic key patterns; imported by both the validator and the config-schema-docs parity test | | `config.cjs` | `config.json` read/write, section initialization; imports validator from `config-schema.cjs` | +| `config-types.cjs` | TypeScript type definitions for the `model_policy` config block — `ModelPolicyConfig`, `TierEntry`, `RuntimeTiers`; compiled from `src/config-types.cts` at publish time (ADR-457) | | `configuration.cjs` | Configuration Module — canonical config loading, legacy-key normalization, defaults merge, and explicit on-disk migration; source of truth for both SDK and CJS consumers | | `context-utilization.cjs` | Pure classifier for `gsd-health --context` — turns (tokensUsed, contextWindow) into a `{ percent, state }` triage result against the 60%/70% fracture-point thresholds (#2792) | | `core.cjs` | Error handling, output formatting, shared utilities, runtime fallbacks; compatibility re-exports for planning-workspace helpers | diff --git a/get-shit-done/bin/lib/config-types.cjs b/get-shit-done/bin/lib/config-types.cjs new file mode 100644 index 000000000..23a0bebc4 --- /dev/null +++ b/get-shit-done/bin/lib/config-types.cjs @@ -0,0 +1,19 @@ +"use strict"; +/** + * TypeScript type definitions for GSD project config — model_policy block. + * + * These types reflect the model_policy config shape consumed by + * resolveModelPolicy in core.cjs and validated by config-schema.cjs. + * + * See feat #49 (model_policy presets) and config-schema.manifest.json. + * Added under ADR-457: TS sources in src/ compile to CJS artifacts in + * get-shit-done/bin/lib/ at publish time. + * + * Resolution precedence (highest → lowest): + * 1. model_overrides[agent] + * 2. model_policy.runtime_tiers[runtime][tier] (Sub-path A) + * 3. model_policy provider preset + budget (Sub-path B) + * 4. model_profile_overrides + * 5. resolve_model_ids / profile fallback + */ +Object.defineProperty(exports, "__esModule", { value: true }); diff --git a/get-shit-done/bin/lib/core.cjs b/get-shit-done/bin/lib/core.cjs index 819af8ec5..3fec86289 100644 --- a/get-shit-done/bin/lib/core.cjs +++ b/get-shit-done/bin/lib/core.cjs @@ -7,7 +7,7 @@ const os = require('os'); const path = require('path'); const { execGit, platformWriteSync, platformReadSync, platformEnsureDir } = require('./shell-command-projection.cjs'); const { MODEL_PROFILES, AGENT_TO_PHASE_TYPE, VALID_PHASE_TYPES, AGENT_DEFAULT_TIERS, VALID_AGENT_TIERS, nextTier } = require('./model-profiles.cjs'); -const { MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, KNOWN_RUNTIMES, RUNTIMES_WITH_REASONING_EFFORT, renderEffortForRuntime, RUNTIMES_WITH_FAST_MODE } = require('./model-catalog.cjs'); +const { MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, KNOWN_RUNTIMES, RUNTIMES_WITH_REASONING_EFFORT, renderEffortForRuntime, RUNTIMES_WITH_FAST_MODE, PROVIDER_PRESETS, KNOWN_PROVIDERS } = require('./model-catalog.cjs'); const { resolveWorktreeContext, parseWorktreePorcelain: parseWorktreePorcelainPolicy, @@ -514,6 +514,11 @@ function loadConfig(cwd, options = {}) { // unknown runtime/tier names at load time, not silently (review finding #10). runtime: parsed.runtime || null, model_profile_overrides: parsed.model_profile_overrides || null, + // #49 — provider-neutral model policy presets. + // model_policy is read flat (not via get()) for the same reason as + // model_profile_overrides: it is a top-level config key per CONFIGURATION.md, + // and nested resolution would introduce edge cases without benefit. + model_policy: parsed.model_policy || null, // #443 — effort/fast_mode: pass through from config.json; resolvers handle // defaults + tier lookups internally. effort: parsed.effort || null, @@ -1384,35 +1389,91 @@ function _warnUnknownProfileOverrides(parsed, configLabel) { } const overrides = parsed.model_profile_overrides; - if (!overrides || typeof overrides !== 'object') return; - for (const [overrideRuntime, tierMap] of Object.entries(overrides)) { - if (!KNOWN_RUNTIMES.has(overrideRuntime)) { - const key = `${configLabel}::override-runtime::${overrideRuntime}`; - if (!_warnedConfigKeys.has(key)) { - _warnedConfigKeys.add(key); - try { - process.stderr.write( - `gsd: warning — model_profile_overrides.${overrideRuntime}.* uses ` + - `unknown runtime "${overrideRuntime}". Known runtimes: ` + - `${[...KNOWN_RUNTIMES].sort().join(', ')}. (#2517)\n` - ); - } catch { /* ok */ } - } - } - if (!tierMap || typeof tierMap !== 'object') continue; - for (const tierName of Object.keys(tierMap)) { - if (!RUNTIME_OVERRIDE_TIERS.has(tierName)) { - const key = `${configLabel}::override-tier::${overrideRuntime}.${tierName}`; + if (overrides && typeof overrides === 'object') { + for (const [overrideRuntime, tierMap] of Object.entries(overrides)) { + if (!KNOWN_RUNTIMES.has(overrideRuntime)) { + const key = `${configLabel}::override-runtime::${overrideRuntime}`; if (!_warnedConfigKeys.has(key)) { _warnedConfigKeys.add(key); try { process.stderr.write( - `gsd: warning — model_profile_overrides.${overrideRuntime}.${tierName} ` + - `uses unknown tier "${tierName}". Allowed tiers: opus, sonnet, haiku. (#2517)\n` + `gsd: warning — model_profile_overrides.${overrideRuntime}.* uses ` + + `unknown runtime "${overrideRuntime}". Known runtimes: ` + + `${[...KNOWN_RUNTIMES].sort().join(', ')}. (#2517)\n` ); } catch { /* ok */ } } } + if (!tierMap || typeof tierMap !== 'object') continue; + for (const tierName of Object.keys(tierMap)) { + if (!RUNTIME_OVERRIDE_TIERS.has(tierName)) { + const key = `${configLabel}::override-tier::${overrideRuntime}.${tierName}`; + if (!_warnedConfigKeys.has(key)) { + _warnedConfigKeys.add(key); + try { + process.stderr.write( + `gsd: warning — model_profile_overrides.${overrideRuntime}.${tierName} ` + + `uses unknown tier "${tierName}". Allowed tiers: opus, sonnet, haiku. (#2517)\n` + ); + } catch { /* ok */ } + } + } + } + } + } + + const policy = parsed.model_policy; + if (policy && typeof policy === 'object') { + const provider = policy.provider; + // 'generic' and 'custom' are sentinel values for the manual model-ID path — not catalog entries. + const _POLICY_SENTINEL_PROVIDERS = new Set(['generic', 'custom']); + if (provider && typeof provider === 'string' && + !KNOWN_PROVIDERS.has(provider) && !_POLICY_SENTINEL_PROVIDERS.has(provider)) { + const pkey = `${configLabel}::model_policy::provider::${provider}`; + if (!_warnedConfigKeys.has(pkey)) { + _warnedConfigKeys.add(pkey); + try { + process.stderr.write( + `gsd: warning — model_policy.provider has unknown value "${provider}". ` + + `Known providers: ${[...KNOWN_PROVIDERS].sort().join(', ')}. ` + + `For manual model IDs use provider="custom". (#49)\n` + ); + } catch { /* ok */ } + } + } + + const rtOverrides = policy.runtime_tiers; + if (rtOverrides && typeof rtOverrides === 'object') { + for (const [pruntime, tierMap] of Object.entries(rtOverrides)) { + if (!KNOWN_RUNTIMES.has(pruntime)) { + const key = `${configLabel}::model_policy.runtime_tiers::${pruntime}`; + if (!_warnedConfigKeys.has(key)) { + _warnedConfigKeys.add(key); + try { + process.stderr.write( + `gsd: warning — model_policy.runtime_tiers.${pruntime}.* uses ` + + `unknown runtime "${pruntime}". Known runtimes: ` + + `${[...KNOWN_RUNTIMES].sort().join(', ')}. (#49)\n` + ); + } catch { /* ok */ } + } + } + if (!tierMap || typeof tierMap !== 'object') continue; + for (const tierName of Object.keys(tierMap)) { + if (!RUNTIME_OVERRIDE_TIERS.has(tierName)) { + const key = `${configLabel}::model_policy.runtime_tiers::${pruntime}.${tierName}`; + if (!_warnedConfigKeys.has(key)) { + _warnedConfigKeys.add(key); + try { + process.stderr.write( + `gsd: warning — model_policy.runtime_tiers.${pruntime}.${tierName} ` + + `uses unknown tier "${tierName}". Allowed: opus, sonnet, haiku. (#49)\n` + ); + } catch { /* ok */ } + } + } + } + } } } } @@ -1476,6 +1537,78 @@ function _resolveRuntimeTier(config, tier) { }); } +/** + * #49 — Provider-neutral model policy preset resolution. + * + * Signature: resolveModelPolicy(policy, tier) → string | null + * + * Resolves a model ID string from a model_policy object. Two resolution sub-paths: + * + * A. runtime_tiers sub-block (explicit per-runtime entries): + * policy.runtime_tiers[policy.runtime][tier] → model string + * The active runtime is read from policy.runtime (the caller merges + * config.runtime into the policy object before calling). + * + * B. provider + budget preset lookup (catalog-backed): + * PROVIDER_PRESETS[provider][tier][budget] → model string + * budget defaults to 'medium' if absent from policy. + * + * Returns a string model ID on a hit, null on miss. + * Never throws — unknown provider/tier/budget degrades gracefully to null + * so the caller falls through to model_profile_overrides. + * + * Precedence: runtime_tiers wins over provider presets when both are present + * for the same runtime+tier combination. + */ +function resolveModelPolicy(policy, tier) { + if (!policy || typeof policy !== 'object') return null; + if (!tier) return null; + + // Sub-path A: explicit runtime_tiers override (highest precedence within policy). + // The active runtime is read from policy.runtime — the caller is responsible for + // merging config.runtime into the policy object (see step 2.5 in resolveModelInternal). + const runtime = policy.runtime; + const rtOverrides = policy.runtime_tiers; + if (runtime && rtOverrides && typeof rtOverrides === 'object') { + const runtimeEntry = rtOverrides[runtime]; + if (runtimeEntry && typeof runtimeEntry === 'object') { + const raw = runtimeEntry[tier]; + if (raw != null) { + const entry = typeof raw === 'string' ? { model: raw } : raw; + if (entry && entry.model) return entry.model; + } + } + } + + // Sub-path B: catalog-backed provider preset. + const provider = policy.provider; + if (!provider || typeof provider !== 'string') return null; + + // Sub-path B1: generic provider — model IDs supplied directly via model_policy.high/medium/low. + // 'generic' and 'custom' are sentinel values meaning "user-supplied IDs, no catalog lookup". + // The tier-to-key mapping: 'opus' → high, 'sonnet' → medium, 'haiku' → low. + if (provider === 'generic' || provider === 'custom') { + const TIER_TO_POLICY_KEY = { opus: 'high', sonnet: 'medium', haiku: 'low' }; + const policyKey = TIER_TO_POLICY_KEY[tier]; + if (!policyKey) return null; + const v = policy[policyKey]; + return (v && typeof v === 'string') ? v : null; + } + + const presetForProvider = PROVIDER_PRESETS[provider]; + if (!presetForProvider) return null; // Unknown provider — fall through silently. + + const tierPresets = presetForProvider[tier]; + if (!tierPresets) return null; // Tier not in preset (partial preset catalog entry). + + // Budget defaults to 'medium' — mirrors the existing 'balanced' default bias. + const budget = (policy.budget && typeof policy.budget === 'string') ? policy.budget : 'medium'; + const budgetEntry = tierPresets[budget]; + if (!budgetEntry || !budgetEntry.model) return null; // Missing or null budget slot. + + return budgetEntry.model; +} + function resolveModelInternal(cwd, agentType) { const config = loadConfig(cwd); @@ -1518,6 +1651,22 @@ function resolveModelInternal(cwd, agentType) { ? 'inherit' : (agentModels ? (agentModels[profile] || agentModels['balanced']) : null)); + // 2.5. model_policy preset (#49) — higher precedence than model_profile_overrides. + // Fires when config.model_policy is set AND runtime is a non-Claude runtime with + // tier !== 'inherit'. The provider preset catalog (Sub-path B) and any + // runtime_tiers overrides (Sub-path A) are checked here before falling through to + // the existing model_profile_overrides chain in step 3. + if (config.runtime && config.runtime !== 'claude' && tier && tier !== 'inherit') { + // Merge config.runtime into the policy object so resolveModelPolicy can use + // it for Sub-path A (runtime_tiers) lookup without needing a separate argument. + const mergedPolicy = config.model_policy + ? { ...config.model_policy, runtime: config.runtime } + : null; + const policyModel = resolveModelPolicy(mergedPolicy, tier); + if (policyModel) return policyModel; + // No policy hit → fall through to step 3 (model_profile_overrides). + } + // 3. Runtime-aware resolution (#2517) — only when `runtime` is explicitly set // to a non-Claude runtime. `runtime: "claude"` is the implicit default and is // treated as a no-op here so it does not silently override `resolve_model_ids: @@ -2301,6 +2450,8 @@ module.exports = { KNOWN_RUNTIMES, RUNTIME_OVERRIDE_TIERS, resolveTierEntry, + resolveModelPolicy, + KNOWN_PROVIDERS, _resetRuntimeWarningCacheForTests, pathExistsInternal, gitWorktreeInfoInternal, diff --git a/get-shit-done/bin/lib/model-catalog.cjs b/get-shit-done/bin/lib/model-catalog.cjs index 7c0926b00..b8b86e7c0 100644 --- a/get-shit-done/bin/lib/model-catalog.cjs +++ b/get-shit-done/bin/lib/model-catalog.cjs @@ -90,6 +90,21 @@ const RUNTIMES_WITH_REASONING_EFFORT = new Set( .map(([runtime]) => runtime) ); +const PROVIDER_PRESETS = catalog.providerPresets || {}; + +// KNOWN_PROVIDERS excludes 'generic' — it is a sentinel (all null entries) that +// forces users to supply model IDs via model_profile_overrides. It is not a +// real catalog-backed provider (#49). +const KNOWN_PROVIDERS = new Set( + Object.entries(PROVIDER_PRESETS) + .filter(([, tiers]) => + Object.values(tiers).some((budgets) => + budgets && Object.values(budgets).some((entry) => entry && entry.model) + ) + ) + .map(([name]) => name) +); + function nextTier(currentTier) { const order = ['light', 'standard', 'heavy']; const idx = order.indexOf(String(currentTier)); @@ -203,6 +218,8 @@ module.exports = { RUNTIME_PROFILE_MAP, KNOWN_RUNTIMES, RUNTIMES_WITH_REASONING_EFFORT, + PROVIDER_PRESETS, + KNOWN_PROVIDERS, nextTier, formatAgentToModelMapAsTable, getAgentToModelMapForProfile, diff --git a/get-shit-done/bin/shared/config-schema.manifest.json b/get-shit-done/bin/shared/config-schema.manifest.json index 7ed43ab41..4a39a0e17 100644 --- a/get-shit-done/bin/shared/config-schema.manifest.json +++ b/get-shit-done/bin/shared/config-schema.manifest.json @@ -101,7 +101,12 @@ "effort.default", "fast_mode.enabled", "plan_review.source_grounding", - "plan_review.source_grounding_authority" + "plan_review.source_grounding_authority", + "model_policy.provider", + "model_policy.budget", + "model_policy.high", + "model_policy.medium", + "model_policy.low" ], "runtimeStateKeys": [ "workflow._auto_chain_active" @@ -171,6 +176,11 @@ "topLevel": "review", "source": "^review\\.max_prompt_tokens_per_reviewer\\.[a-zA-Z0-9_-]+$", "description": "review.max_prompt_tokens_per_reviewer." + }, + { + "topLevel": "model_policy", + "source": "^model_policy\\.runtime_tiers\\.[a-zA-Z0-9_-]+\\.(opus|sonnet|haiku)$", + "description": "model_policy.runtime_tiers.." } ] } diff --git a/get-shit-done/bin/shared/model-catalog.json b/get-shit-done/bin/shared/model-catalog.json index 73f57f36f..8e0d37d97 100644 --- a/get-shit-done/bin/shared/model-catalog.json +++ b/get-shit-done/bin/shared/model-catalog.json @@ -83,6 +83,33 @@ "haiku": null } }, + "providerPresets": { + "anthropic": { + "opus": { "low": { "model": "claude-opus-4-5" }, "medium": { "model": "claude-opus-4-8" }, "high": { "model": "claude-opus-4-8" } }, + "sonnet": { "low": { "model": "claude-haiku-4-5" }, "medium": { "model": "claude-sonnet-4-6" }, "high": { "model": "claude-opus-4-8" } }, + "haiku": { "low": { "model": "claude-haiku-4-5" }, "medium": { "model": "claude-haiku-4-5" }, "high": { "model": "claude-sonnet-4-6" } } + }, + "openai": { + "opus": { "low": { "model": "gpt-5.3-codex", "reasoning_effort": "medium" }, "medium": { "model": "gpt-5.5", "reasoning_effort": "high" }, "high": { "model": "gpt-5.5", "reasoning_effort": "xhigh" } }, + "sonnet": { "low": { "model": "gpt-5.4-mini", "reasoning_effort": "low" }, "medium": { "model": "gpt-5.3-codex", "reasoning_effort": "medium" }, "high": { "model": "gpt-5.5", "reasoning_effort": "medium" } }, + "haiku": { "low": { "model": "gpt-5.4-mini", "reasoning_effort": "minimal" }, "medium": { "model": "gpt-5.4-mini", "reasoning_effort": "medium" }, "high": { "model": "gpt-5.3-codex", "reasoning_effort": "medium" } } + }, + "google": { + "opus": { "low": { "model": "gemini-2.5-flash-lite" }, "medium": { "model": "gemini-3-flash" }, "high": { "model": "gemini-3-pro" } }, + "sonnet": { "low": { "model": "gemini-2.5-flash-lite" }, "medium": { "model": "gemini-3-flash" }, "high": { "model": "gemini-3-flash" } }, + "haiku": { "low": { "model": "gemini-2.5-flash-lite" }, "medium": { "model": "gemini-2.5-flash-lite" }, "high": { "model": "gemini-3-flash" } } + }, + "qwen": { + "opus": { "low": { "model": "qwen3-coder-plus" }, "medium": { "model": "qwen3-max-2026-01-23" }, "high": { "model": "qwen3-max-2026-01-23" } }, + "sonnet": { "low": { "model": "qwen3-coder-next" }, "medium": { "model": "qwen3-coder-plus" }, "high": { "model": "qwen3-max-2026-01-23" } }, + "haiku": { "low": { "model": "qwen3-coder-next" }, "medium": { "model": "qwen3-coder-next" }, "high": { "model": "qwen3-coder-plus" } } + }, + "generic": { + "opus": { "low": null, "medium": null, "high": null }, + "sonnet": { "low": null, "medium": null, "high": null }, + "haiku": { "low": null, "medium": null, "high": null } + } + }, "agents": { "gsd-planner": { "golden": "opus", "balanced": "opus", "budget": "sonnet", "phaseType": "planning", "routingTier": "heavy" }, "gsd-roadmapper": { "golden": "opus", "balanced": "sonnet", "budget": "sonnet", "phaseType": "planning", "routingTier": "heavy" }, diff --git a/get-shit-done/workflows/settings-advanced.md b/get-shit-done/workflows/settings-advanced.md index c760b3469..89e10ea3c 100644 --- a/get-shit-done/workflows/settings-advanced.md +++ b/get-shit-done/workflows/settings-advanced.md @@ -1,12 +1,14 @@ Interactive configuration of GSD power-user knobs — plan bounce, node repair, subagent timeouts, inline plan threshold, cross-AI execution, base branch, branch templates, response language, -context window, gitignored search, graphify build timeout, and runtime model tier overrides. +context window, gitignored search, graphify build timeout, runtime model tier overrides, and +model policy configuration (provider + budget → canonical tier mapping, or manual model ID +assignment per cost tier). This is a companion to `/gsd:settings` — the common-case prompt there covers model profile, research/plan_check/verifier toggles, branching strategy, UI/AI phase gates, and worktree isolation. This advanced command covers everything else that is user-settable, grouped into -seven sections so each prompt batch stays cognitively scoped. Every answer pre-selects the +eight sections so each prompt batch stays cognitively scoped. Every answer pre-selects the current value; numeric-input answers that are non-numeric are rejected and re-prompted. @@ -81,6 +83,13 @@ Runtime Model Tiers: - `model_profile_overrides..sonnet` (default: built-in for the runtime, or absent) - `model_profile_overrides..haiku` (default: built-in for the runtime, or absent) +Model Policy: +- `model_policy.provider` (default: `null` — known values: anthropic, openai, google, qwen) +- `model_policy.budget` (default: `null` — known values: high, medium, low) +- `model_policy.high` (default: `null` — model ID for the high-cost tier; used by generic provider path) +- `model_policy.medium` (default: `null` — model ID for the medium-cost tier; used by generic provider path) +- `model_policy.low` (default: `null` — model ID for the low-cost tier; used by generic provider path) + Each field's **current value is pre-selected** in the prompt rendering below. When the current value is absent from the config, render the documented default as the pre-selected option so the user sees what the effective value is. @@ -500,7 +509,7 @@ gsd_run query config-set model_profile_overrides.gemini.haiku null Conceptual shape after merge (unchanged top-level keys like `model_profile`, `granularity`, `mode`, `brave_search`, `agent_skills.*`, `hooks.context_warnings`, and -anything not listed in Sections 1–7 MUST survive the update): +anything not listed in Sections 1–8 MUST survive the update): ```json { @@ -542,6 +551,14 @@ anything not listed in Sections 1–7 MUST survive the update): "sonnet": , "haiku": } + }, + "model_policy": { + ...existing_model_policy, + "provider": , + "budget": , + "high": , + "medium": , + "low": } } ``` @@ -551,6 +568,170 @@ route each write through `gsd-tools.cjs query config-set` so sibling preservatio the central setter. +### Section 8 — Model Policy + +This section configures the `model_policy` key in `.planning/config.json`. Model policy +defines which AI models GSD uses at each cost tier (low / medium / high), independently +of the `runtime` and `model_profile` selections above. Two paths are offered: + +- **Known provider:** choose a provider and a budget level; GSD materializes the canonical + tier mapping for that provider. +- **Generic provider:** enter low / medium / high model IDs manually. + +**Step A — Read and display the current model policy:** + +```bash +cat "$GSD_CONFIG_PATH" | python3 -c "import sys,json; c=json.load(sys.stdin); mp=c.get('model_policy',{}); print(json.dumps(mp,indent=2))" 2>/dev/null || echo "{}" +``` + +Display the current values (or "(unset)" for any absent field) before asking: + +```text +Current model_policy: + provider : + budget : + low : + medium : + high : +``` + +**Step B — Choose configuration path:** + +```text +AskUserQuestion([ + { + question: "How do you want to configure the model policy?", + header: "Model Policy", + multiSelect: false, + options: [ + { label: "Known provider", description: "Choose a provider (Claude / OpenAI / Gemini / Qwen) and a budget level — GSD writes the canonical tier mapping automatically." }, + { label: "Generic provider", description: "Enter low / medium / high model IDs manually for any provider or custom deployment." }, + { label: "Keep current", description: "Leave model_policy unchanged." } + ] + } +]) +``` + +**If "Keep current" is selected:** skip Steps C–E and move on to the confirm step. + +**Step C — Known-provider path:** + +```text +AskUserQuestion([ + { + question: "Which provider?", + header: "Provider", + multiSelect: false, + options: [ + { label: "anthropic", description: "claude-opus-4-8 / claude-sonnet-4-6 / claude-haiku-4-5 (Anthropic / Claude)" }, + { label: "openai", description: "gpt-5.5 / gpt-5.3-codex / gpt-5.4-mini (OpenAI / Codex)" }, + { label: "google", description: "gemini-3-pro / gemini-3-flash / gemini-2.5-flash-lite (Google Gemini)" }, + { label: "qwen", description: "qwen3-max-2026-01-23 / qwen3-coder-plus / qwen3-coder-next (Qwen)" } + ] + } +]) +``` + +After the user picks a provider, ask: + +```text +AskUserQuestion([ + { + question: "Which budget level?", + header: "Budget", + multiSelect: false, + options: [ + { label: "high", description: "All tiers use the highest-quality model for the chosen provider. Highest cost." }, + { label: "medium", description: "High tier → top model; medium → mid model; low → cheapest model. Best cost/quality ratio." }, + { label: "low", description: "All tiers use the cheapest model for the chosen provider. Lowest cost." } + ] + } +]) +``` + +Canonical tier mappings by provider and budget: + +| Provider | Budget | high | medium | low | +|-----------|--------|----------------------------|----------------------------|----------------------------| +| anthropic | high | claude-opus-4-8 | claude-opus-4-8 | claude-opus-4-8 | +| anthropic | medium | claude-opus-4-8 | claude-sonnet-4-6 | claude-haiku-4-5 | +| anthropic | low | claude-haiku-4-5 | claude-haiku-4-5 | claude-haiku-4-5 | +| openai | high | gpt-5.5 | gpt-5.5 | gpt-5.5 | +| openai | medium | gpt-5.5 | gpt-5.3-codex | gpt-5.4-mini | +| openai | low | gpt-5.4-mini | gpt-5.4-mini | gpt-5.4-mini | +| google | high | gemini-3-pro | gemini-3-pro | gemini-3-pro | +| google | medium | gemini-3-pro | gemini-3-flash | gemini-2.5-flash-lite | +| google | low | gemini-2.5-flash-lite | gemini-2.5-flash-lite | gemini-2.5-flash-lite | +| qwen | high | qwen3-max-2026-01-23 | qwen3-max-2026-01-23 | qwen3-max-2026-01-23 | +| qwen | medium | qwen3-max-2026-01-23 | qwen3-coder-plus | qwen3-coder-next | +| qwen | low | qwen3-coder-next | qwen3-coder-next | qwen3-coder-next | + +Look up the selected (provider, budget) row and proceed to Step E to write those values. + +**Step D — Generic-provider path:** + +Prompt the user to enter each model ID as a free-text input. An empty input means "keep +the current value for that tier." Validate that non-empty inputs are non-blank strings +(no whitespace-only values); if validation fails, re-prompt that single field. + +```text +AskUserQuestion([ + { + question: "Model ID for the HIGH-cost tier? (most capable model — used for heavy reasoning tasks)", + header: "High-tier model", + multiSelect: false, + options: [ + { label: "Keep current", description: "Leave unchanged (current: )." }, + { label: "Enter model ID", description: "Type the exact model identifier. Non-blank string required." } + ] + }, + { + question: "Model ID for the MEDIUM-cost tier? (balanced model — used for most agents)", + header: "Medium-tier model", + multiSelect: false, + options: [ + { label: "Keep current", description: "Leave unchanged (current: )." }, + { label: "Enter model ID", description: "Type the exact model identifier." } + ] + }, + { + question: "Model ID for the LOW-cost tier? (cheapest model — used for lightweight/fast tasks)", + header: "Low-tier model", + multiSelect: false, + options: [ + { label: "Keep current", description: "Leave unchanged (current: )." }, + { label: "Enter model ID", description: "Type the exact model identifier." } + ] + } +]) +``` + +Set `provider = "custom"` and `budget = null` when writing the generic-provider result. +Proceed to Step E. + +**Step E — Write model_policy to config:** + +```bash +# Known-provider path — write all four keys atomically: +gsd_run query config-set model_policy.provider "" # e.g., anthropic / openai / google / qwen +gsd_run query config-set model_policy.budget "" # high / medium / low +gsd_run query config-set model_policy.high "" +gsd_run query config-set model_policy.medium "" +gsd_run query config-set model_policy.low "" + +# Generic-provider path — write only tiers the user changed ("Keep current" skipped): +gsd_run query config-set model_policy.provider "custom" +gsd_run query config-set model_policy.budget null +# Per-tier writes for each non-"Keep current" answer: +gsd_run query config-set model_policy.high "" # omit if user chose "Keep current" +gsd_run query config-set model_policy.medium "" # omit if user chose "Keep current" +gsd_run query config-set model_policy.low "" # omit if user chose "Keep current" +``` + +Never write a tier the user explicitly chose to keep; the existing value must survive. + + + Display: @@ -594,6 +775,11 @@ Display: | fast_mode.routing_tier_defaults.standard | {true/false} | | fast_mode.routing_tier_defaults.heavy | {true/false} | | fast_mode.agent_overrides. | {true/false} | +| model_policy.provider | {anthropic/openai/google/qwen/custom/null} | +| model_policy.budget | {high/medium/low/null} | +| model_policy.high | {model-id/null} | +| model_policy.medium | {model-id/null} | +| model_policy.low | {model-id/null} | These settings apply to future /gsd:plan-phase, /gsd:execute-phase, /gsd:discuss-phase, and /gsd:ship runs. @@ -607,7 +793,7 @@ UI/AI phase gates), use /gsd:settings. - [ ] Current config read from resolved `$GSD_CONFIG_PATH` -- [ ] Seven sections rendered (Planning, Execution, Discussion, Cross-AI, Git, Runtime/Output, Runtime Model Tiers) +- [ ] Eight sections rendered (Planning, Execution, Discussion, Cross-AI, Git, Runtime/Output, Runtime Model Tiers, Model Policy) - [ ] Every field pre-selected to its current value (or documented default if absent) - [ ] Numeric inputs validated — non-numeric rejected and re-prompted - [ ] Branch-template inputs validated — non-default must contain a placeholder @@ -616,5 +802,9 @@ UI/AI phase gates), use /gsd:settings. - [ ] Section 7 shows current runtime and built-in tier table - [ ] Group B runtimes display "(no built-in default — your runtime handles model selection)" - [ ] Override set/clear/keep paths all work correctly for each tier -- [ ] Confirmation table rendered listing all 23 fields (19 + runtime + 3 tier overrides) +- [ ] Section 8 (Model Policy) offers three top-level choices: Known provider, Generic provider, Keep current +- [ ] Known-provider path: provider + budget → canonical tier mapping written to model_policy.{provider,budget,high,medium,low} +- [ ] Generic-provider path: per-tier manual model IDs; "Keep current" tiers are never written; provider=custom budget=null +- [ ] model_policy written under the model_policy key in config.json, never as a top-level flat key +- [ ] Confirmation table rendered listing all fields including model_policy.{provider,budget,high,medium,low} diff --git a/get-shit-done/workflows/settings.md b/get-shit-done/workflows/settings.md index 672683c1e..36f237c18 100644 --- a/get-shit-done/workflows/settings.md +++ b/get-shit-done/workflows/settings.md @@ -57,6 +57,11 @@ Parse current values (default to `true` if not present): - `model_profile` — which model each agent uses (default: `balanced`) - `git.branching_strategy` — branching approach (default: `"none"`) - `workflow.use_worktrees` — whether parallel executor agents run in worktree isolation (default: `true`) +- `model_policy.provider` — provider slug for model policy (default: `null`; known values: anthropic, openai, google, qwen; set via /gsd:config --advanced) +- `model_policy.budget` — budget level for model policy (default: `null`; known values: high, medium, low; set via /gsd:config --advanced) +- `model_policy.high` — model ID for high-cost tier (default: `null`; set via /gsd:config --advanced) +- `model_policy.medium` — model ID for medium-cost tier (default: `null`; set via /gsd:config --advanced) +- `model_policy.low` — model ID for low-cost tier (default: `null`; set via /gsd:config --advanced) @@ -423,6 +428,15 @@ Merge new settings into existing config.json: "hooks": { "context_warnings": true/false, "workflow_guard": true/false + }, + "model_policy": { + // Read-only in this flow — written only by /gsd:config --advanced (Section 8). + // Listed here so safe-merge never clobbers an existing model_policy object. + "provider": , + "budget": , + "high": , + "medium": , + "low": } } ``` @@ -537,7 +551,7 @@ Quick commands: - /gsd:plan-phase --research — force research - /gsd:plan-phase --skip-research — skip research - /gsd:plan-phase --skip-verify — skip plan check -- /gsd:config --advanced — power-user tuning (plan bounce, timeouts, branch templates, cross-AI, context window) +- /gsd:config --advanced — power-user tuning (plan bounce, timeouts, branch templates, cross-AI, context window, model policy) ``` diff --git a/src/config-types.cts b/src/config-types.cts new file mode 100644 index 000000000..0c525c230 --- /dev/null +++ b/src/config-types.cts @@ -0,0 +1,62 @@ +/** + * TypeScript type definitions for GSD project config — model_policy block. + * + * These types reflect the model_policy config shape consumed by + * resolveModelPolicy in core.cjs and validated by config-schema.cjs. + * + * See feat #49 (model_policy presets) and config-schema.manifest.json. + * Added under ADR-457: TS sources in src/ compile to CJS artifacts in + * get-shit-done/bin/lib/ at publish time. + * + * Resolution precedence (highest → lowest): + * 1. model_overrides[agent] + * 2. model_policy.runtime_tiers[runtime][tier] (Sub-path A) + * 3. model_policy provider preset + budget (Sub-path B) + * 4. model_profile_overrides + * 5. resolve_model_ids / profile fallback + */ + +/** + * A single tier entry mapping a GSD tier (opus | sonnet | haiku) to a + * concrete model ID. The optional `reasoning_effort` field is forwarded to + * runtimes that accept it (e.g. opencode). + */ +export interface TierEntry { + model: string; + reasoning_effort?: string; +} + +/** + * The three standard GSD tiers for one runtime target. All fields are + * optional so callers can supply a partial override (e.g. only `opus`). + */ +export interface RuntimeTiers { + low?: TierEntry; + medium?: TierEntry; + high?: TierEntry; +} + +/** + * Top-level `model_policy` block in `.planning/config.json`. + * + * - `provider` — known provider slug (e.g. `"anthropic"`, `"openai"`). + * Drives Sub-path B catalog lookup. + * - `budget` — optional spend/quality tier that pairs with `provider` + * to select a preset from the model catalog. + * - `runtime_tiers` — explicit per-runtime, per-tier model overrides + * (Sub-path A). Keys are runtime slugs (e.g. `"opencode"`, + * `"copilot"`); values are `RuntimeTiers` maps. + */ +export interface ModelPolicyConfig { + provider: string; + budget?: string; + runtime_tiers?: Record; +} + +/** + * Minimal subset of the GSD project config that includes `model_policy`. + * Extend this interface when migrating further config keys to TypeScript. + */ +export interface ProjectConfig { + model_policy?: ModelPolicyConfig; +} diff --git a/tests/feat-49-model-policy-presets.test.cjs b/tests/feat-49-model-policy-presets.test.cjs new file mode 100644 index 000000000..3d13efe8a --- /dev/null +++ b/tests/feat-49-model-policy-presets.test.cjs @@ -0,0 +1,680 @@ +/** + * Feature test for issue #49 — model_policy presets. + * + * Adds a `model_policy` block to .planning/config.json: + * + * { + * "model_policy": { + * "provider": "anthropic", + * "budget": "high", + * "runtime_tiers": { + * "opencode": { + * "opus": { "model": "anthropic/claude-opus-4-8" } + * } + * } + * } + * } + * + * Resolution precedence in resolveModelInternal (highest → lowest): + * 1. model_overrides[agent] (per-agent full IDs; existing) + * 2. model_policy.runtime_tiers[runtime][tier] (Sub-path A: explicit runtime+tier entry) + * 3. model_policy provider preset + budget (Sub-path B: known-provider catalog lookup) + * 4. model_profile_overrides (legacy runtime-aware overrides) + * 5. resolve_model_ids / profile fallback + * + * Sub-path A (runtime_tiers) fires when config.runtime matches a key inside + * model_policy.runtime_tiers AND that key contains an entry for the resolved tier. + * + * Sub-path B (provider preset) fires when model_policy.provider is a known + * provider AND the catalog contains an entry for (tier, budget) pair. + * + * Both sub-paths return a string model ID. Failures in either sub-path fall + * through cleanly to the next step in the chain. + * + * New config keys accepted by isValidConfigKey: + * - model_policy.provider + * - model_policy.budget + * - model_policy.runtime_tiers.. + * + * Backwards compatibility: + * - model_profile_overrides continues to work when model_policy is absent. + * - When both are set, model_policy wins (fires first). + * + * KNOWN_PROVIDERS is exported from both model-catalog.cjs and core.cjs (re-export). + * + * These tests are written to FAIL before implementation. They use typed-IR / + * structural assertions on resolveModelInternal / resolveModelPolicy / isValidConfigKey + * return values — not stdout / grep. + */ + +'use strict'; + +process.env.GSD_TEST_MODE = '1'; + +const { test, describe, beforeEach, afterEach } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const os = require('node:os'); + +// ─── Imports (will fail until implementation exists) ──────────────────────── +// resolveModelPolicy is a new internal function that must be exported from core.cjs. +// KNOWN_PROVIDERS must be exported from model-catalog.cjs and re-exported by core.cjs. +const { + resolveModelInternal, + resolveModelPolicy, + KNOWN_PROVIDERS, + _resetRuntimeWarningCacheForTests, +} = require('../get-shit-done/bin/lib/core.cjs'); + +// KNOWN_PROVIDERS must also be exported directly from model-catalog.cjs +const modelCatalog = require('../get-shit-done/bin/lib/model-catalog.cjs'); + +const { isValidConfigKey } = require('../get-shit-done/bin/lib/config-schema.cjs'); +const { createTempDir } = require('./helpers.cjs'); + +const makeTmp = (prefix) => createTempDir(`gsd-49-${prefix}-`); + +function writeConfig(dir, config) { + const planningDir = path.join(dir, '.planning'); + fs.mkdirSync(planningDir, { recursive: true }); + fs.writeFileSync(path.join(planningDir, 'config.json'), JSON.stringify(config, null, 2)); +} + +function rmr(p) { + try { fs.rmSync(p, { recursive: true, force: true }); } catch { /* noop */ } +} + +// ─── resolveModelPolicy unit tests ────────────────────────────────────────── +// +// resolveModelPolicy(config, tier) is the pure resolver that takes a loaded +// config object and a resolved tier string. It returns a string model ID when +// model_policy produces a hit, or null when it falls through. + +describe('#49 resolveModelPolicy: null/absent policy returns null', () => { + test('resolveModelPolicy returns null when policy is null or absent', () => { + // policy is null + assert.strictEqual(resolveModelPolicy(null, 'opus'), null); + // policy is undefined + assert.strictEqual(resolveModelPolicy(undefined, 'opus'), null); + // policy is absent (empty object treated as absent) + assert.strictEqual(resolveModelPolicy({}, 'opus'), null); + }); + + test('resolveModelPolicy returns null when runtime or tier is missing', () => { + const policy = { provider: 'anthropic', budget: 'high' }; + // tier is null + assert.strictEqual(resolveModelPolicy(policy, null), null); + // tier is empty string + assert.strictEqual(resolveModelPolicy(policy, ''), null); + // tier is undefined + assert.strictEqual(resolveModelPolicy(policy, undefined), null); + }); +}); + +describe('#49 resolveModelPolicy Sub-path B: provider presets', () => { + test('known provider "anthropic" + tier "opus" + budget "high" returns correct model ID', () => { + // The anthropic preset catalog must contain an entry for opus+high. + // The returned model ID is the high-budget anthropic opus model. + const policy = { provider: 'anthropic', budget: 'high' }; + const result = resolveModelPolicy(policy, 'opus'); + assert.ok(typeof result === 'string' && result.length > 0, + `expected a non-empty model ID string, got: ${JSON.stringify(result)}`); + // Anthropic opus model IDs contain "claude" and "opus" + assert.match(result, /claude.*opus|opus.*claude/i, + `expected anthropic opus model ID to contain "claude" and "opus", got: ${result}`); + }); + + test('known provider "openai" + tier "sonnet" + budget "low" returns model with reasoning_effort from preset', () => { + // The openai preset catalog must contain a sonnet+low entry. + // "openai" maps to a different model family; the entry may include reasoning_effort. + const policy = { provider: 'openai', budget: 'low' }; + const result = resolveModelPolicy(policy, 'sonnet'); + assert.ok(typeof result === 'string' && result.length > 0, + `expected a non-empty model ID string for openai/sonnet/low, got: ${JSON.stringify(result)}`); + }); + + test('budget absent defaults to "medium"', () => { + // No "budget" key — defaults to "medium". The anthropic/opus/medium entry must exist. + const policyWithBudget = { provider: 'anthropic', budget: 'medium' }; + const policyNoBudget = { provider: 'anthropic' }; + const withBudget = resolveModelPolicy(policyWithBudget, 'opus'); + const withoutBudget = resolveModelPolicy(policyNoBudget, 'opus'); + // Both must return a string (not null) + assert.ok(typeof withBudget === 'string' && withBudget.length > 0, + `expected model from explicit budget:'medium'`); + assert.ok(typeof withoutBudget === 'string' && withoutBudget.length > 0, + `expected model when budget absent (should default to medium)`); + // They must resolve to the same value + assert.strictEqual(withBudget, withoutBudget, + 'absent budget must behave identically to explicit "medium"'); + }); + + test('provider "generic" (all null entries) returns null (falls through)', () => { + // provider:'generic' means opaque model IDs — there's no preset catalog for + // generic. Without a runtime_tiers hit, resolveModelPolicy returns null. + const policy = { provider: 'generic', budget: 'high' }; + const result = resolveModelPolicy(policy, 'opus'); + assert.strictEqual(result, null, + 'provider:"generic" with no runtime_tiers must return null (no preset catalog)'); + }); + + test('unknown provider string returns null without throwing', () => { + // A typo like provider:'mistral' must not crash; it degrades gracefully. + const policy = { provider: 'mistral', budget: 'high' }; + let result; + assert.doesNotThrow(() => { + result = resolveModelPolicy(policy, 'opus'); + }, 'resolveModelPolicy must not throw on unknown provider'); + assert.strictEqual(result, null, + 'unknown provider with no runtime_tiers must return null'); + }); + + test('known provider + unknown tier returns null', () => { + const policy = { provider: 'anthropic', budget: 'high' }; + const result = resolveModelPolicy(policy, 'jumbo'); + assert.strictEqual(result, null, + 'unknown tier "jumbo" must return null for anthropic provider'); + }); + + test('known provider + known tier + missing budget level returns null', () => { + // The anthropic preset for opus only defines 'high' and 'medium' but NOT 'critical'. + // A missing budget level must fall through (return null) — not crash. + const policy = { provider: 'anthropic', budget: 'critical' }; + const result = resolveModelPolicy(policy, 'opus'); + assert.strictEqual(result, null, + 'missing budget level "critical" must return null without throwing'); + }); +}); + +describe('#49 resolveModelPolicy Sub-path A: runtime_tiers', () => { + test('runtime_tiers entry wins over provider preset for same runtime+tier', () => { + // Sub-path A fires first: explicit runtime_tiers entry overrides the + // provider preset catalog. The returned model is the one in runtime_tiers, + // not what the provider preset would have returned. + const policy = { + provider: 'anthropic', + budget: 'high', + runtime: 'opencode', + runtime_tiers: { + opencode: { + opus: { model: 'anthropic/custom-opus-override' }, + }, + }, + }; + const result = resolveModelPolicy(policy, 'opus'); + assert.strictEqual(result, 'anthropic/custom-opus-override', + 'Sub-path A runtime_tiers must win over Sub-path B provider preset'); + }); + + test('runtime_tiers string shorthand normalized to { model } object', () => { + // String shorthand: `{ opencode: { opus: "some-model-id" } }` + // must be normalized to `{ model: "some-model-id" }` so the resolver + // returns the string as-is. + const policy = { + provider: 'anthropic', + budget: 'high', + runtime: 'opencode', + runtime_tiers: { + opencode: { + opus: 'anthropic/string-shorthand-model', + }, + }, + }; + const result = resolveModelPolicy(policy, 'opus'); + assert.strictEqual(result, 'anthropic/string-shorthand-model', + 'string shorthand in runtime_tiers must be normalized and returned as model ID'); + }); + + test('runtime_tiers partial entry (no matching runtime) falls through to provider preset', () => { + // runtime_tiers has entries for 'copilot' but the active runtime is 'opencode'. + // The miss on runtime_tiers falls through to Sub-path B (provider preset). + const policy = { + provider: 'anthropic', + budget: 'high', + runtime: 'opencode', + runtime_tiers: { + copilot: { + opus: { model: 'some-copilot-model' }, + }, + }, + }; + const result = resolveModelPolicy(policy, 'opus'); + // Falls through to Sub-path B (anthropic/opus/high) — must not be null. + assert.ok(typeof result === 'string' && result.length > 0, + 'runtime_tiers miss must fall through to provider preset, got: ' + JSON.stringify(result)); + // And it must NOT be the copilot model + assert.notStrictEqual(result, 'some-copilot-model'); + }); +}); + +// ─── resolveModelInternal integration tests ────────────────────────────────── +// +// These tests call resolveModelInternal through a temp project's config.json. +// They verify the full resolution chain including model_policy placement. + +describe('#49 resolveModelInternal: model_policy in the resolution chain', () => { + let projectDir; + beforeEach(() => { + projectDir = makeTmp('internal'); + _resetRuntimeWarningCacheForTests(); + }); + afterEach(() => { + rmr(projectDir); + _resetRuntimeWarningCacheForTests(); + }); + + test('model_policy fires before model_profile_overrides when both are set (model_policy wins)', () => { + // model_policy (Sub-path B: anthropic/opus/high) must win over + // model_profile_overrides when both are present. + // We use a model_profile_overrides entry that would give a DIFFERENT result. + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'quality', + model_policy: { + provider: 'anthropic', + budget: 'high', + }, + model_profile_overrides: { + opencode: { + // This legacy override would have returned this model — but model_policy must win. + opus: 'legacy-override-model-should-not-appear', + }, + }, + }); + const result = resolveModelInternal(projectDir, 'gsd-planner'); + assert.notStrictEqual(result, 'legacy-override-model-should-not-appear', + 'model_policy must fire before model_profile_overrides and win'); + assert.ok(typeof result === 'string' && result.length > 0, + 'must return a non-empty model ID'); + // model_policy anthropic/opus/high should return a claude opus model ID + assert.match(result, /claude.*opus|opus.*claude/i, + 'expected anthropic preset opus model, got: ' + result); + }); + + test('model_policy with provider:"anthropic" + budget:"high" + runtime:"opencode" resolves to preset model', () => { + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'quality', // gsd-planner quality = opus tier + model_policy: { + provider: 'anthropic', + budget: 'high', + }, + }); + const result = resolveModelInternal(projectDir, 'gsd-planner'); + assert.ok(typeof result === 'string' && result.length > 0, + 'expected a non-empty model ID'); + assert.match(result, /claude.*opus|opus.*claude/i, + 'anthropic/opus/high must resolve to an opus model ID'); + }); + + test('model_policy is skipped when runtime is absent', () => { + // No `runtime` in config — model_policy fires on any non-null policy + // only when a runtime context is available. Without runtime, the policy + // falls through entirely. + // NOTE: Sub-path B (provider preset) can fire without runtime — it only + // needs tier+budget+provider. Sub-path A requires runtime. This test + // verifies the gating behavior described in the issue: if model_policy + // is present but runtime is absent, provider preset Sub-path B still + // fires (it doesn't need runtime). So "skipped" means the runtime_tiers + // sub-path is skipped but provider preset may still fire. + // The test asserts that resolveModelInternal does not crash and returns + // a string regardless. + writeConfig(projectDir, { + model_profile: 'quality', + model_policy: { + provider: 'anthropic', + budget: 'high', + runtime_tiers: { + opencode: { + opus: { model: 'should-not-appear-no-runtime' }, + }, + }, + }, + }); + let result; + assert.doesNotThrow(() => { + result = resolveModelInternal(projectDir, 'gsd-planner'); + }); + assert.ok(typeof result === 'string', + 'resolveModelInternal must return a string even when runtime is absent'); + // The runtime_tiers entry for opencode must not appear since runtime is absent + assert.notStrictEqual(result, 'should-not-appear-no-runtime', + 'runtime_tiers must not fire when config.runtime is absent'); + }); + + test('model_policy is skipped when runtime:"claude" (no-op gate)', () => { + // runtime:"claude" is the implicit default and is treated as a no-op + // for model_policy resolution (same as the no-op gate in the existing + // runtime-aware resolution step). model_policy provider preset + // for anthropic may still fire — but runtime_tiers for claude is a no-op + // because claude-native resolution already handles that path. + writeConfig(projectDir, { + runtime: 'claude', + model_profile: 'quality', + model_policy: { + provider: 'anthropic', + budget: 'high', + runtime_tiers: { + claude: { + opus: { model: 'claude-runtime-tiers-should-not-appear' }, + }, + }, + }, + }); + let result; + assert.doesNotThrow(() => { + result = resolveModelInternal(projectDir, 'gsd-planner'); + }); + assert.ok(typeof result === 'string', 'must return a string'); + // The claude runtime_tiers entry must not appear — model_policy runtime_tiers + // is a no-op for runtime:"claude" + assert.notStrictEqual(result, 'claude-runtime-tiers-should-not-appear', + 'model_policy.runtime_tiers must be a no-op when runtime is "claude"'); + }); + + test('model_policy is skipped when tier:"inherit"', () => { + // When the resolved tier is 'inherit', model_policy must not fire. + // This mirrors the existing behavior for runtime-aware resolution. + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'inherit', + model_policy: { + provider: 'anthropic', + budget: 'high', + }, + }); + const result = resolveModelInternal(projectDir, 'gsd-planner'); + // With profile:'inherit', the result must be 'inherit' + assert.strictEqual(result, 'inherit', + 'model_policy must not fire when tier is "inherit"; resolveModelInternal must return "inherit"'); + }); + + test('model_profile_overrides still resolves when model_policy is absent (legacy fallback intact)', () => { + // No model_policy — model_profile_overrides must still work exactly as before. + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'quality', + model_profile_overrides: { + opencode: { + opus: 'legacy-overridden-model', + }, + }, + }); + const result = resolveModelInternal(projectDir, 'gsd-planner'); + assert.strictEqual(result, 'legacy-overridden-model', + 'model_profile_overrides must still win when model_policy is absent'); + }); + + test('model_policy absent + model_profile_overrides set → model_profile_overrides wins (back-compat)', () => { + // Explicit: no model_policy key at all. model_profile_overrides is the only + // custom config. The legacy chain must apply exactly as before this feature. + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'balanced', + model_profile_overrides: { + opencode: { + sonnet: 'back-compat-sonnet-model', + }, + }, + }); + // gsd-executor has balanced/opencode -> sonnet tier + const result = resolveModelInternal(projectDir, 'gsd-executor'); + assert.strictEqual(result, 'back-compat-sonnet-model', + 'legacy model_profile_overrides must be unaffected when model_policy is absent'); + }); + + test('model_policy present but runtime_tiers empty + provider:"generic" → falls through to model_profile_overrides', () => { + // model_policy is a stub: runtime_tiers is empty ({}), provider is "generic". + // The resolver must fall through all model_policy paths and land on model_profile_overrides. + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'quality', + model_policy: { + provider: 'generic', + budget: 'high', + runtime_tiers: {}, + }, + model_profile_overrides: { + opencode: { + opus: 'fallthrough-to-legacy', + }, + }, + }); + const result = resolveModelInternal(projectDir, 'gsd-planner'); + assert.strictEqual(result, 'fallthrough-to-legacy', + 'empty runtime_tiers + generic provider must fall through to model_profile_overrides'); + }); +}); + +// ─── Warning emission tests ─────────────────────────────────────────────────── + +describe('#49 resolveModelInternal: unknown provider warning behavior', () => { + let projectDir; + let origWrite; + let captured; + + beforeEach(() => { + projectDir = makeTmp('warnings'); + _resetRuntimeWarningCacheForTests(); + captured = []; + origWrite = process.stderr.write.bind(process.stderr); + process.stderr.write = (chunk) => { captured.push(String(chunk)); return true; }; + }); + + afterEach(() => { + process.stderr.write = origWrite; + rmr(projectDir); + _resetRuntimeWarningCacheForTests(); + }); + + test('unknown provider in model_policy → falls through to model_profile_overrides, emits stderr warning once', () => { + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'quality', + model_policy: { + provider: 'mistral', + budget: 'high', + }, + model_profile_overrides: { + opencode: { + opus: 'fallback-from-unknown-provider', + }, + }, + }); + const result = resolveModelInternal(projectDir, 'gsd-planner'); + // Must fall through to model_profile_overrides + assert.strictEqual(result, 'fallback-from-unknown-provider', + 'unknown provider must fall through to model_profile_overrides'); + // Must emit at least one stderr warning about the unknown provider + const joined = captured.join(''); + assert.match(joined, /model_policy.*provider.*mistral|unknown.*provider.*mistral|mistral.*unknown/i, + 'must emit a stderr warning about the unknown provider "mistral"'); + }); + + test('unknown provider warning is deduplicated (emitted only once per config label)', () => { + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'quality', + model_policy: { + provider: 'mistral', + budget: 'high', + }, + }); + // Call resolveModelInternal multiple times for different agents — the + // warning about the unknown provider must be emitted only once. + resolveModelInternal(projectDir, 'gsd-planner'); + resolveModelInternal(projectDir, 'gsd-executor'); + resolveModelInternal(projectDir, 'gsd-verifier'); + const joined = captured.join(''); + // Count occurrences of "mistral" in the warning output + const matches = (joined.match(/mistral/gi) || []).length; + assert.ok(matches >= 1, 'expected at least one warning about "mistral"'); + assert.ok(matches <= 2, `warning for unknown provider must be deduplicated — saw ${matches} occurrences`); + }); + + test('model_policy.runtime_tiers with unknown runtime emits one-shot stderr warning', () => { + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'quality', + model_policy: { + provider: 'anthropic', + budget: 'high', + runtime_tiers: { + unknownrt: { + opus: { model: 'some-model' }, + }, + }, + }, + }); + resolveModelInternal(projectDir, 'gsd-planner'); + const joined = captured.join(''); + // Must emit a warning about the unknown runtime key in runtime_tiers + assert.match(joined, /unknownrt|unknown.*runtime|runtime_tiers.*unknown/i, + 'must emit a stderr warning about unknown runtime "unknownrt" in model_policy.runtime_tiers'); + }); + + test('model_policy.runtime_tiers with invalid tier name emits one-shot stderr warning', () => { + writeConfig(projectDir, { + runtime: 'opencode', + model_profile: 'quality', + model_policy: { + provider: 'anthropic', + budget: 'high', + runtime_tiers: { + opencode: { + jumbo: { model: 'invalid-tier-model' }, + }, + }, + }, + }); + resolveModelInternal(projectDir, 'gsd-planner'); + const joined = captured.join(''); + // Must emit a warning about the invalid tier name "jumbo" + assert.match(joined, /jumbo|invalid.*tier|tier.*invalid|unknown.*tier/i, + 'must emit a stderr warning about invalid tier "jumbo" in model_policy.runtime_tiers.opencode'); + }); +}); + +// ─── reasoning_effort passthrough tests ────────────────────────────────────── + +describe('#49 reasoning_effort in model_policy entries', () => { + let projectDir; + beforeEach(() => { projectDir = makeTmp('effort'); }); + afterEach(() => { rmr(projectDir); }); + + test('reasoning_effort in preset entry is returned as part of the entry object (caller decides whether to emit)', () => { + // When a provider preset includes reasoning_effort (e.g. openai opus/high), + // resolveModelPolicy must return the full entry object (or at minimum the model + // string) without stripping reasoning_effort internally. + // This is checked via the internal resolveModelPolicy function directly. + // The policy object includes a runtime_tiers entry that has reasoning_effort. + const policy = { + provider: 'anthropic', + budget: 'high', + runtime: 'opencode', + runtime_tiers: { + opencode: { + opus: { model: 'anthropic/claude-opus-4-8', reasoning_effort: 'high' }, + }, + }, + }; + // resolveModelPolicy must return the model string (at minimum). + // The caller (resolveModelInternal) is responsible for deciding what to + // emit — the resolver just returns the model ID string. + const result = resolveModelPolicy(policy, 'opus'); + assert.strictEqual(result, 'anthropic/claude-opus-4-8', + 'resolveModelPolicy must return the model string from the runtime_tiers entry'); + }); + + test('reasoning_effort in model_policy.runtime_tiers entry is returned verbatim; renderEffortForRuntime strips it when runtime not in RUNTIMES_WITH_REASONING_EFFORT', () => { + // The renderEffortForRuntime function (already existing) handles the stripping. + // This test verifies the contract: resolveModelPolicy returns the model string, + // and for runtimes not in RUNTIMES_WITH_REASONING_EFFORT, the caller must not + // emit reasoning_effort. + const { renderEffortForRuntime, RUNTIMES_WITH_REASONING_EFFORT } = require('../get-shit-done/bin/lib/model-catalog.cjs'); + + // 'opencode' is NOT in RUNTIMES_WITH_REASONING_EFFORT (only codex has reasoning_effort in catalog) + assert.ok(!RUNTIMES_WITH_REASONING_EFFORT.has('opencode'), + 'opencode must not be in RUNTIMES_WITH_REASONING_EFFORT for this test to be meaningful'); + + // renderEffortForRuntime for a non-effort runtime returns channel:null + const rendered = renderEffortForRuntime('opencode', 'high'); + assert.strictEqual(rendered.channel, null, + 'renderEffortForRuntime must return channel:null for runtimes not supporting reasoning_effort'); + + // The resolveModelPolicy function returns just the model string — reasoning_effort + // is stripped at the emit layer, not inside resolveModelPolicy. + const policy = { + runtime: 'opencode', + provider: 'anthropic', + budget: 'high', + runtime_tiers: { + opencode: { + opus: { model: 'anthropic/claude-opus-4-8', reasoning_effort: 'high' }, + }, + }, + }; + const result = resolveModelPolicy(policy, 'opus'); + assert.strictEqual(result, 'anthropic/claude-opus-4-8', + 'resolveModelPolicy must return model string; reasoning_effort is stripped downstream'); + }); +}); + +// ─── isValidConfigKey: model_policy.* schema validation ────────────────────── + +describe('#49 isValidConfigKey: model_policy.* keys accepted/rejected', () => { + test('isValidConfigKey accepts "model_policy.provider"', () => { + assert.strictEqual(isValidConfigKey('model_policy.provider'), true, + '"model_policy.provider" must be a valid config key'); + }); + + test('isValidConfigKey accepts "model_policy.budget"', () => { + assert.strictEqual(isValidConfigKey('model_policy.budget'), true, + '"model_policy.budget" must be a valid config key'); + }); + + test('isValidConfigKey accepts "model_policy.runtime_tiers.opencode.opus"', () => { + assert.strictEqual(isValidConfigKey('model_policy.runtime_tiers.opencode.opus'), true, + '"model_policy.runtime_tiers.opencode.opus" must be a valid config key'); + }); + + test('isValidConfigKey rejects "model_policy.runtime_tiers.opencode.banana" (invalid tier)', () => { + assert.strictEqual(isValidConfigKey('model_policy.runtime_tiers.opencode.banana'), false, + '"model_policy.runtime_tiers.opencode.banana" must be rejected (banana is not a valid tier)'); + }); +}); + +// ─── KNOWN_PROVIDERS export tests ───────────────────────────────────────────── + +describe('#49 KNOWN_PROVIDERS exports from model-catalog.cjs and core.cjs', () => { + test('KNOWN_PROVIDERS exported from core.cjs includes all keys from providerPresets in catalog', () => { + // KNOWN_PROVIDERS must be a Set (or array) exported from core.cjs. + assert.ok(KNOWN_PROVIDERS != null, + 'KNOWN_PROVIDERS must be exported from core.cjs'); + const isIterable = typeof KNOWN_PROVIDERS[Symbol.iterator] === 'function'; + assert.ok(isIterable, + 'KNOWN_PROVIDERS must be iterable (Set or array)'); + const providers = [...KNOWN_PROVIDERS]; + assert.ok(providers.length > 0, + 'KNOWN_PROVIDERS must not be empty'); + // 'anthropic' must be in the set since it is a required provider preset + assert.ok(providers.includes('anthropic'), + 'KNOWN_PROVIDERS must include "anthropic"'); + // 'generic' is a special fallback, not a real provider — it must NOT be in KNOWN_PROVIDERS + // (KNOWN_PROVIDERS lists only providers with catalog entries) + assert.ok(!providers.includes('generic'), + 'KNOWN_PROVIDERS must not include "generic" (it is not a catalog-backed provider)'); + }); + + test('KNOWN_PROVIDERS exported from model-catalog.cjs matches core.cjs re-export', () => { + // model-catalog.cjs must also export KNOWN_PROVIDERS (the canonical source). + // core.cjs re-exports it. Both must be identical. + assert.ok(modelCatalog.KNOWN_PROVIDERS != null, + 'KNOWN_PROVIDERS must be exported from model-catalog.cjs'); + const fromCatalog = [...modelCatalog.KNOWN_PROVIDERS].sort(); + const fromCore = [...KNOWN_PROVIDERS].sort(); + assert.deepStrictEqual(fromCore, fromCatalog, + 'KNOWN_PROVIDERS from core.cjs (re-export) must match model-catalog.cjs canonical export'); + }); +});