feat(#49): provider-neutral model policy presets
* feat(#49): provider-neutral model policy presets Adds model_policy config surface with known-provider presets (openai/anthropic/google/qwen) and generic provider escape hatch. model_policy.runtime_tiers resolves before legacy model_profile_overrides. reasoning_effort is stripped for unsupported runtimes. Closes #49 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#49): replace unregistered /gsd-settings-advanced token in docs docs-parity-live-registry enforces every /token in docs/*.md maps to a live command. /gsd-settings-advanced is a workflow filename, not a registered command — use /gsd:settings instead. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(#49): update INVENTORY.md count and manifest for config-types.cjs inventory-counts and inventory-manifest-sync tests require the headline count and INVENTORY-MANIFEST.json to reflect every file in bin/lib/. config-types.cjs (new module added by feat(#49)) was missing from both. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
5
.changeset/silly-jaguars-swim.md
Normal file
5
.changeset/silly-jaguars-swim.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Added
|
||||
pr: 49
|
||||
---
|
||||
Add `model_policy` config surface with known-provider presets (openai/anthropic/google/qwen) and `generic` provider escape hatch. `model_policy.runtime_tiers` resolves before legacy `model_profile_overrides`. `reasoning_effort` is stripped for unsupported runtimes.
|
||||
@@ -139,6 +139,12 @@ GSD stores project settings in `.planning/config.json`. Created during `/gsd-new
|
||||
| `model_profile` | enum | `quality`, `balanced`, `budget`, `adaptive`, `inherit` | `balanced` | Model tier for each agent (see [Model Profiles](#model-profiles)). `adaptive` was added per [#1713](https://github.com/open-gsd/gsd-core/issues/1713) / [#1806](https://github.com/open-gsd/gsd-core/issues/1806) and resolves the same way as the other tiers under runtime-aware profiles. |
|
||||
| `runtime` | string | `claude`, `codex`, or any string | (none) | Active runtime for [runtime-aware profile resolution](#runtime-aware-profiles-2517). When set, profile tiers (opus/sonnet/haiku) resolve to runtime-native model IDs. Today only the Codex install path emits per-agent model IDs from this resolver; other runtimes (`opencode`, `gemini`, `qwen`, `copilot`, …) consume the resolver at spawn time and gain dedicated install-path support in [#2612](https://github.com/open-gsd/gsd-core/issues/2612). When unset (default), behavior is unchanged from prior versions. Added in v1.39 |
|
||||
| `model_profile_overrides.<runtime>.<tier>` | string \| object | per-runtime tier override | (none) | Override the runtime-aware tier mapping for a specific `(runtime, tier)`. Tier is one of `opus`, `sonnet`, `haiku`. Value is either a model ID string (e.g. `"gpt-5-pro"`) or `{ model, reasoning_effort }`. See [Runtime-Aware Profiles](#runtime-aware-profiles-2517). Added in v1.39 |
|
||||
| `model_policy.provider` | string | `openai`, `anthropic`, `google`, `qwen`, `generic` | (none) | Declares the model provider. Known providers (`openai`, `anthropic`, `google`, `qwen`) unlock catalog-backed presets. `generic` treats all model IDs as opaque strings — no prefix inference, no reasoning-effort defaults. `model_policy.runtime_tiers` resolves before legacy `model_profile_overrides`. See [Model Policy Presets](#model-policy-presets-model_policy--added-in-v142). Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) |
|
||||
| `model_policy.budget` | enum | `high`, `medium`, `low` | (none) | Selects a budget tier when using a known provider. GSD materializes the matching catalog preset into explicit tier mappings at resolve time. Ignored when `provider` is `generic` or `custom`. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) |
|
||||
| `model_policy.high` | string | model ID | (none) | High-cost tier model ID for `generic`/`custom` provider. Used when `provider: "generic"` or `"custom"`. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) |
|
||||
| `model_policy.medium` | string | model ID | (none) | Medium-cost tier model ID for `generic`/`custom` provider. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) |
|
||||
| `model_policy.low` | string | model ID | (none) | Low-cost tier model ID for `generic`/`custom` provider. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) |
|
||||
| `model_policy.runtime_tiers.<runtime>.<tier>` | object | `{ model, reasoning_effort? }` | (none) | Explicit per-runtime, per-tier model entry. `tier` is one of `opus`, `sonnet`, `haiku` (matching the existing profile tier names). `reasoning_effort` is forwarded only to runtimes that support it; unsupported runtimes never receive the field. Takes precedence over `model_profile_overrides`. Added in v1.42 ([#49](https://github.com/open-gsd/gsd-core/issues/49)) |
|
||||
| `models.<phase_type>` | enum | `opus`, `sonnet`, `haiku`, `inherit` | (none) | Per-phase-type model tier. Six accepted slots: `planning`, `discuss`, `research`, `execution`, `verification`, `completion`. Lets you tune at the phase level ("Opus for planning, Sonnet for the rest") without learning agent names. Resolves between `model_overrides` (higher) and `model_profile` (lower); see [Per-Phase-Type Models](#per-phase-type-models-models--added-in-v140). Added in v1.40 ([#3023](https://github.com/open-gsd/gsd-core/pull/3030)) |
|
||||
| `dynamic_routing.enabled` | boolean | `true`, `false` | `false` | Master switch for [dynamic routing with failure-tier escalation](#dynamic-routing-with-failure-tier-escalation-dynamic_routing--added-in-v140). When `true`, agents resolve to `tier_models[default_tier]` and escalate one tier up on orchestrator-detected soft failure. Added in v1.40 ([#3024](https://github.com/open-gsd/gsd-core/pull/3031)) |
|
||||
| `dynamic_routing.tier_models.<tier>` | enum | `opus`, `sonnet`, `haiku` | (none) | Tier alias for `light`, `standard`, or `heavy`. Used when `dynamic_routing.enabled: true`. Added in v1.40 |
|
||||
@@ -1218,6 +1224,84 @@ This resolves `gsd-planner` → `gpt-5.5` (xhigh), `gsd-executor` → `gpt-5.3-c
|
||||
|
||||
---
|
||||
|
||||
## Model Policy Presets (`model_policy`) — Added in v1.42
|
||||
|
||||
> **[#49](https://github.com/open-gsd/gsd-core/issues/49)** — provider-neutral model policy config surface. Resolves before legacy `model_profile_overrides`.
|
||||
|
||||
`model_policy` provides a simpler, provider-neutral way to configure model tiers across runtimes. It is the preferred surface for non-Anthropic runtimes where `model_profile_overrides` would require manually knowing the right model IDs. Configure it via `/gsd:settings` → Section 8 (Model Policy).
|
||||
|
||||
### Known provider preset
|
||||
|
||||
Choose a provider and budget level via the settings workflow; GSD writes the canonical model IDs for that provider/budget combination:
|
||||
|
||||
```json
|
||||
{
|
||||
"runtime": "codex",
|
||||
"model_policy": {
|
||||
"provider": "openai",
|
||||
"budget": "medium",
|
||||
"high": "gpt-5.5",
|
||||
"medium": "gpt-5.3-codex",
|
||||
"low": "gpt-5.4-mini"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Known providers: `openai`, `anthropic`, `google`, `qwen`. Budget levels: `high`, `medium`, `low`.
|
||||
|
||||
For advanced per-runtime control, `runtime_tiers` accepts explicit entries using the internal profile tier names (`opus`, `sonnet`, `haiku`):
|
||||
|
||||
```json
|
||||
{
|
||||
"runtime": "codex",
|
||||
"model_policy": {
|
||||
"provider": "openai",
|
||||
"runtime_tiers": {
|
||||
"codex": {
|
||||
"opus": { "model": "gpt-5.5", "reasoning_effort": "high" },
|
||||
"sonnet": { "model": "gpt-5.3-codex", "reasoning_effort": "medium" },
|
||||
"haiku": { "model": "gpt-5.4-mini", "reasoning_effort": "low" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Generic provider (escape hatch)
|
||||
|
||||
Use `provider: "generic"` (or `"custom"`) for OpenRouter, LiteLLM, local gateways, or any runtime where you supply exact model IDs. GSD treats model IDs as opaque strings — no prefix inference, no provider-specific defaults:
|
||||
|
||||
```json
|
||||
{
|
||||
"runtime": "opencode",
|
||||
"model_policy": {
|
||||
"provider": "generic",
|
||||
"high": "openrouter/anthropic/claude-opus-4-5",
|
||||
"medium": "openrouter/anthropic/claude-sonnet-4-5",
|
||||
"low": "openrouter/anthropic/claude-haiku-4-5"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Reasoning effort gating
|
||||
|
||||
`reasoning_effort` within a `runtime_tiers` entry is forwarded only to runtimes that declare support for it (currently: `codex`). Any runtime not on the allowlist receives the tier entry without the `reasoning_effort` field — it is silently stripped, never leaked.
|
||||
|
||||
### Precedence
|
||||
|
||||
`model_policy` resolution sits above `model_profile_overrides` in the resolver:
|
||||
|
||||
1. `model_overrides[<agent>]` — per-agent explicit ID (highest)
|
||||
2. `model_policy.runtime_tiers[<runtime>][<tier>]` — explicit runtime/tier entry
|
||||
3. `model_policy` flat `high`/`medium`/`low` keys — for `generic`/`custom` provider
|
||||
4. `model_profile_overrides[<runtime>][<tier>]` — legacy per-runtime override
|
||||
5. Built-in runtime catalog default
|
||||
6. `model_profile` tier alias
|
||||
|
||||
**Backwards compatibility.** Configs without `model_policy` are unaffected. Existing `model_profile_overrides` blocks continue to work exactly as before.
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Purpose |
|
||||
|
||||
@@ -275,6 +275,7 @@
|
||||
"command-routing-hub.cjs",
|
||||
"commands.cjs",
|
||||
"config-schema.cjs",
|
||||
"config-types.cjs",
|
||||
"config.cjs",
|
||||
"configuration.cjs",
|
||||
"context-utilization.cjs",
|
||||
|
||||
@@ -362,7 +362,7 @@ The `gsd-planner` agent is decomposed into a core agent plus reference modules t
|
||||
|
||||
---
|
||||
|
||||
## CLI Modules (80 shipped)
|
||||
## CLI Modules (81 shipped)
|
||||
|
||||
Full listing: `get-shit-done/bin/lib/*.cjs`.
|
||||
|
||||
@@ -384,6 +384,7 @@ Full listing: `get-shit-done/bin/lib/*.cjs`.
|
||||
| `commands.cjs` | Misc CLI commands (slug, timestamp, todos, scaffolding, stats) |
|
||||
| `config-schema.cjs` | Single source of truth for `VALID_CONFIG_KEYS` and dynamic key patterns; imported by both the validator and the config-schema-docs parity test |
|
||||
| `config.cjs` | `config.json` read/write, section initialization; imports validator from `config-schema.cjs` |
|
||||
| `config-types.cjs` | TypeScript type definitions for the `model_policy` config block — `ModelPolicyConfig`, `TierEntry`, `RuntimeTiers`; compiled from `src/config-types.cts` at publish time (ADR-457) |
|
||||
| `configuration.cjs` | Configuration Module — canonical config loading, legacy-key normalization, defaults merge, and explicit on-disk migration; source of truth for both SDK and CJS consumers |
|
||||
| `context-utilization.cjs` | Pure classifier for `gsd-health --context` — turns (tokensUsed, contextWindow) into a `{ percent, state }` triage result against the 60%/70% fracture-point thresholds (#2792) |
|
||||
| `core.cjs` | Error handling, output formatting, shared utilities, runtime fallbacks; compatibility re-exports for planning-workspace helpers |
|
||||
|
||||
19
get-shit-done/bin/lib/config-types.cjs
Normal file
19
get-shit-done/bin/lib/config-types.cjs
Normal file
@@ -0,0 +1,19 @@
|
||||
"use strict";
|
||||
/**
|
||||
* TypeScript type definitions for GSD project config — model_policy block.
|
||||
*
|
||||
* These types reflect the model_policy config shape consumed by
|
||||
* resolveModelPolicy in core.cjs and validated by config-schema.cjs.
|
||||
*
|
||||
* See feat #49 (model_policy presets) and config-schema.manifest.json.
|
||||
* Added under ADR-457: TS sources in src/ compile to CJS artifacts in
|
||||
* get-shit-done/bin/lib/ at publish time.
|
||||
*
|
||||
* Resolution precedence (highest → lowest):
|
||||
* 1. model_overrides[agent]
|
||||
* 2. model_policy.runtime_tiers[runtime][tier] (Sub-path A)
|
||||
* 3. model_policy provider preset + budget (Sub-path B)
|
||||
* 4. model_profile_overrides
|
||||
* 5. resolve_model_ids / profile fallback
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
@@ -7,7 +7,7 @@ const os = require('os');
|
||||
const path = require('path');
|
||||
const { execGit, platformWriteSync, platformReadSync, platformEnsureDir } = require('./shell-command-projection.cjs');
|
||||
const { MODEL_PROFILES, AGENT_TO_PHASE_TYPE, VALID_PHASE_TYPES, AGENT_DEFAULT_TIERS, VALID_AGENT_TIERS, nextTier } = require('./model-profiles.cjs');
|
||||
const { MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, KNOWN_RUNTIMES, RUNTIMES_WITH_REASONING_EFFORT, renderEffortForRuntime, RUNTIMES_WITH_FAST_MODE } = require('./model-catalog.cjs');
|
||||
const { MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, KNOWN_RUNTIMES, RUNTIMES_WITH_REASONING_EFFORT, renderEffortForRuntime, RUNTIMES_WITH_FAST_MODE, PROVIDER_PRESETS, KNOWN_PROVIDERS } = require('./model-catalog.cjs');
|
||||
const {
|
||||
resolveWorktreeContext,
|
||||
parseWorktreePorcelain: parseWorktreePorcelainPolicy,
|
||||
@@ -514,6 +514,11 @@ function loadConfig(cwd, options = {}) {
|
||||
// unknown runtime/tier names at load time, not silently (review finding #10).
|
||||
runtime: parsed.runtime || null,
|
||||
model_profile_overrides: parsed.model_profile_overrides || null,
|
||||
// #49 — provider-neutral model policy presets.
|
||||
// model_policy is read flat (not via get()) for the same reason as
|
||||
// model_profile_overrides: it is a top-level config key per CONFIGURATION.md,
|
||||
// and nested resolution would introduce edge cases without benefit.
|
||||
model_policy: parsed.model_policy || null,
|
||||
// #443 — effort/fast_mode: pass through from config.json; resolvers handle
|
||||
// defaults + tier lookups internally.
|
||||
effort: parsed.effort || null,
|
||||
@@ -1384,35 +1389,91 @@ function _warnUnknownProfileOverrides(parsed, configLabel) {
|
||||
}
|
||||
|
||||
const overrides = parsed.model_profile_overrides;
|
||||
if (!overrides || typeof overrides !== 'object') return;
|
||||
for (const [overrideRuntime, tierMap] of Object.entries(overrides)) {
|
||||
if (!KNOWN_RUNTIMES.has(overrideRuntime)) {
|
||||
const key = `${configLabel}::override-runtime::${overrideRuntime}`;
|
||||
if (!_warnedConfigKeys.has(key)) {
|
||||
_warnedConfigKeys.add(key);
|
||||
try {
|
||||
process.stderr.write(
|
||||
`gsd: warning — model_profile_overrides.${overrideRuntime}.* uses ` +
|
||||
`unknown runtime "${overrideRuntime}". Known runtimes: ` +
|
||||
`${[...KNOWN_RUNTIMES].sort().join(', ')}. (#2517)\n`
|
||||
);
|
||||
} catch { /* ok */ }
|
||||
}
|
||||
}
|
||||
if (!tierMap || typeof tierMap !== 'object') continue;
|
||||
for (const tierName of Object.keys(tierMap)) {
|
||||
if (!RUNTIME_OVERRIDE_TIERS.has(tierName)) {
|
||||
const key = `${configLabel}::override-tier::${overrideRuntime}.${tierName}`;
|
||||
if (overrides && typeof overrides === 'object') {
|
||||
for (const [overrideRuntime, tierMap] of Object.entries(overrides)) {
|
||||
if (!KNOWN_RUNTIMES.has(overrideRuntime)) {
|
||||
const key = `${configLabel}::override-runtime::${overrideRuntime}`;
|
||||
if (!_warnedConfigKeys.has(key)) {
|
||||
_warnedConfigKeys.add(key);
|
||||
try {
|
||||
process.stderr.write(
|
||||
`gsd: warning — model_profile_overrides.${overrideRuntime}.${tierName} ` +
|
||||
`uses unknown tier "${tierName}". Allowed tiers: opus, sonnet, haiku. (#2517)\n`
|
||||
`gsd: warning — model_profile_overrides.${overrideRuntime}.* uses ` +
|
||||
`unknown runtime "${overrideRuntime}". Known runtimes: ` +
|
||||
`${[...KNOWN_RUNTIMES].sort().join(', ')}. (#2517)\n`
|
||||
);
|
||||
} catch { /* ok */ }
|
||||
}
|
||||
}
|
||||
if (!tierMap || typeof tierMap !== 'object') continue;
|
||||
for (const tierName of Object.keys(tierMap)) {
|
||||
if (!RUNTIME_OVERRIDE_TIERS.has(tierName)) {
|
||||
const key = `${configLabel}::override-tier::${overrideRuntime}.${tierName}`;
|
||||
if (!_warnedConfigKeys.has(key)) {
|
||||
_warnedConfigKeys.add(key);
|
||||
try {
|
||||
process.stderr.write(
|
||||
`gsd: warning — model_profile_overrides.${overrideRuntime}.${tierName} ` +
|
||||
`uses unknown tier "${tierName}". Allowed tiers: opus, sonnet, haiku. (#2517)\n`
|
||||
);
|
||||
} catch { /* ok */ }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const policy = parsed.model_policy;
|
||||
if (policy && typeof policy === 'object') {
|
||||
const provider = policy.provider;
|
||||
// 'generic' and 'custom' are sentinel values for the manual model-ID path — not catalog entries.
|
||||
const _POLICY_SENTINEL_PROVIDERS = new Set(['generic', 'custom']);
|
||||
if (provider && typeof provider === 'string' &&
|
||||
!KNOWN_PROVIDERS.has(provider) && !_POLICY_SENTINEL_PROVIDERS.has(provider)) {
|
||||
const pkey = `${configLabel}::model_policy::provider::${provider}`;
|
||||
if (!_warnedConfigKeys.has(pkey)) {
|
||||
_warnedConfigKeys.add(pkey);
|
||||
try {
|
||||
process.stderr.write(
|
||||
`gsd: warning — model_policy.provider has unknown value "${provider}". ` +
|
||||
`Known providers: ${[...KNOWN_PROVIDERS].sort().join(', ')}. ` +
|
||||
`For manual model IDs use provider="custom". (#49)\n`
|
||||
);
|
||||
} catch { /* ok */ }
|
||||
}
|
||||
}
|
||||
|
||||
const rtOverrides = policy.runtime_tiers;
|
||||
if (rtOverrides && typeof rtOverrides === 'object') {
|
||||
for (const [pruntime, tierMap] of Object.entries(rtOverrides)) {
|
||||
if (!KNOWN_RUNTIMES.has(pruntime)) {
|
||||
const key = `${configLabel}::model_policy.runtime_tiers::${pruntime}`;
|
||||
if (!_warnedConfigKeys.has(key)) {
|
||||
_warnedConfigKeys.add(key);
|
||||
try {
|
||||
process.stderr.write(
|
||||
`gsd: warning — model_policy.runtime_tiers.${pruntime}.* uses ` +
|
||||
`unknown runtime "${pruntime}". Known runtimes: ` +
|
||||
`${[...KNOWN_RUNTIMES].sort().join(', ')}. (#49)\n`
|
||||
);
|
||||
} catch { /* ok */ }
|
||||
}
|
||||
}
|
||||
if (!tierMap || typeof tierMap !== 'object') continue;
|
||||
for (const tierName of Object.keys(tierMap)) {
|
||||
if (!RUNTIME_OVERRIDE_TIERS.has(tierName)) {
|
||||
const key = `${configLabel}::model_policy.runtime_tiers::${pruntime}.${tierName}`;
|
||||
if (!_warnedConfigKeys.has(key)) {
|
||||
_warnedConfigKeys.add(key);
|
||||
try {
|
||||
process.stderr.write(
|
||||
`gsd: warning — model_policy.runtime_tiers.${pruntime}.${tierName} ` +
|
||||
`uses unknown tier "${tierName}". Allowed: opus, sonnet, haiku. (#49)\n`
|
||||
);
|
||||
} catch { /* ok */ }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1476,6 +1537,78 @@ function _resolveRuntimeTier(config, tier) {
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* #49 — Provider-neutral model policy preset resolution.
|
||||
*
|
||||
* Signature: resolveModelPolicy(policy, tier) → string | null
|
||||
*
|
||||
* Resolves a model ID string from a model_policy object. Two resolution sub-paths:
|
||||
*
|
||||
* A. runtime_tiers sub-block (explicit per-runtime entries):
|
||||
* policy.runtime_tiers[policy.runtime][tier] → model string
|
||||
* The active runtime is read from policy.runtime (the caller merges
|
||||
* config.runtime into the policy object before calling).
|
||||
*
|
||||
* B. provider + budget preset lookup (catalog-backed):
|
||||
* PROVIDER_PRESETS[provider][tier][budget] → model string
|
||||
* budget defaults to 'medium' if absent from policy.
|
||||
*
|
||||
* Returns a string model ID on a hit, null on miss.
|
||||
* Never throws — unknown provider/tier/budget degrades gracefully to null
|
||||
* so the caller falls through to model_profile_overrides.
|
||||
*
|
||||
* Precedence: runtime_tiers wins over provider presets when both are present
|
||||
* for the same runtime+tier combination.
|
||||
*/
|
||||
function resolveModelPolicy(policy, tier) {
|
||||
if (!policy || typeof policy !== 'object') return null;
|
||||
if (!tier) return null;
|
||||
|
||||
// Sub-path A: explicit runtime_tiers override (highest precedence within policy).
|
||||
// The active runtime is read from policy.runtime — the caller is responsible for
|
||||
// merging config.runtime into the policy object (see step 2.5 in resolveModelInternal).
|
||||
const runtime = policy.runtime;
|
||||
const rtOverrides = policy.runtime_tiers;
|
||||
if (runtime && rtOverrides && typeof rtOverrides === 'object') {
|
||||
const runtimeEntry = rtOverrides[runtime];
|
||||
if (runtimeEntry && typeof runtimeEntry === 'object') {
|
||||
const raw = runtimeEntry[tier];
|
||||
if (raw != null) {
|
||||
const entry = typeof raw === 'string' ? { model: raw } : raw;
|
||||
if (entry && entry.model) return entry.model;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sub-path B: catalog-backed provider preset.
|
||||
const provider = policy.provider;
|
||||
if (!provider || typeof provider !== 'string') return null;
|
||||
|
||||
// Sub-path B1: generic provider — model IDs supplied directly via model_policy.high/medium/low.
|
||||
// 'generic' and 'custom' are sentinel values meaning "user-supplied IDs, no catalog lookup".
|
||||
// The tier-to-key mapping: 'opus' → high, 'sonnet' → medium, 'haiku' → low.
|
||||
if (provider === 'generic' || provider === 'custom') {
|
||||
const TIER_TO_POLICY_KEY = { opus: 'high', sonnet: 'medium', haiku: 'low' };
|
||||
const policyKey = TIER_TO_POLICY_KEY[tier];
|
||||
if (!policyKey) return null;
|
||||
const v = policy[policyKey];
|
||||
return (v && typeof v === 'string') ? v : null;
|
||||
}
|
||||
|
||||
const presetForProvider = PROVIDER_PRESETS[provider];
|
||||
if (!presetForProvider) return null; // Unknown provider — fall through silently.
|
||||
|
||||
const tierPresets = presetForProvider[tier];
|
||||
if (!tierPresets) return null; // Tier not in preset (partial preset catalog entry).
|
||||
|
||||
// Budget defaults to 'medium' — mirrors the existing 'balanced' default bias.
|
||||
const budget = (policy.budget && typeof policy.budget === 'string') ? policy.budget : 'medium';
|
||||
const budgetEntry = tierPresets[budget];
|
||||
if (!budgetEntry || !budgetEntry.model) return null; // Missing or null budget slot.
|
||||
|
||||
return budgetEntry.model;
|
||||
}
|
||||
|
||||
function resolveModelInternal(cwd, agentType) {
|
||||
const config = loadConfig(cwd);
|
||||
|
||||
@@ -1518,6 +1651,22 @@ function resolveModelInternal(cwd, agentType) {
|
||||
? 'inherit'
|
||||
: (agentModels ? (agentModels[profile] || agentModels['balanced']) : null));
|
||||
|
||||
// 2.5. model_policy preset (#49) — higher precedence than model_profile_overrides.
|
||||
// Fires when config.model_policy is set AND runtime is a non-Claude runtime with
|
||||
// tier !== 'inherit'. The provider preset catalog (Sub-path B) and any
|
||||
// runtime_tiers overrides (Sub-path A) are checked here before falling through to
|
||||
// the existing model_profile_overrides chain in step 3.
|
||||
if (config.runtime && config.runtime !== 'claude' && tier && tier !== 'inherit') {
|
||||
// Merge config.runtime into the policy object so resolveModelPolicy can use
|
||||
// it for Sub-path A (runtime_tiers) lookup without needing a separate argument.
|
||||
const mergedPolicy = config.model_policy
|
||||
? { ...config.model_policy, runtime: config.runtime }
|
||||
: null;
|
||||
const policyModel = resolveModelPolicy(mergedPolicy, tier);
|
||||
if (policyModel) return policyModel;
|
||||
// No policy hit → fall through to step 3 (model_profile_overrides).
|
||||
}
|
||||
|
||||
// 3. Runtime-aware resolution (#2517) — only when `runtime` is explicitly set
|
||||
// to a non-Claude runtime. `runtime: "claude"` is the implicit default and is
|
||||
// treated as a no-op here so it does not silently override `resolve_model_ids:
|
||||
@@ -2301,6 +2450,8 @@ module.exports = {
|
||||
KNOWN_RUNTIMES,
|
||||
RUNTIME_OVERRIDE_TIERS,
|
||||
resolveTierEntry,
|
||||
resolveModelPolicy,
|
||||
KNOWN_PROVIDERS,
|
||||
_resetRuntimeWarningCacheForTests,
|
||||
pathExistsInternal,
|
||||
gitWorktreeInfoInternal,
|
||||
|
||||
@@ -90,6 +90,21 @@ const RUNTIMES_WITH_REASONING_EFFORT = new Set(
|
||||
.map(([runtime]) => runtime)
|
||||
);
|
||||
|
||||
const PROVIDER_PRESETS = catalog.providerPresets || {};
|
||||
|
||||
// KNOWN_PROVIDERS excludes 'generic' — it is a sentinel (all null entries) that
|
||||
// forces users to supply model IDs via model_profile_overrides. It is not a
|
||||
// real catalog-backed provider (#49).
|
||||
const KNOWN_PROVIDERS = new Set(
|
||||
Object.entries(PROVIDER_PRESETS)
|
||||
.filter(([, tiers]) =>
|
||||
Object.values(tiers).some((budgets) =>
|
||||
budgets && Object.values(budgets).some((entry) => entry && entry.model)
|
||||
)
|
||||
)
|
||||
.map(([name]) => name)
|
||||
);
|
||||
|
||||
function nextTier(currentTier) {
|
||||
const order = ['light', 'standard', 'heavy'];
|
||||
const idx = order.indexOf(String(currentTier));
|
||||
@@ -203,6 +218,8 @@ module.exports = {
|
||||
RUNTIME_PROFILE_MAP,
|
||||
KNOWN_RUNTIMES,
|
||||
RUNTIMES_WITH_REASONING_EFFORT,
|
||||
PROVIDER_PRESETS,
|
||||
KNOWN_PROVIDERS,
|
||||
nextTier,
|
||||
formatAgentToModelMapAsTable,
|
||||
getAgentToModelMapForProfile,
|
||||
|
||||
@@ -101,7 +101,12 @@
|
||||
"effort.default",
|
||||
"fast_mode.enabled",
|
||||
"plan_review.source_grounding",
|
||||
"plan_review.source_grounding_authority"
|
||||
"plan_review.source_grounding_authority",
|
||||
"model_policy.provider",
|
||||
"model_policy.budget",
|
||||
"model_policy.high",
|
||||
"model_policy.medium",
|
||||
"model_policy.low"
|
||||
],
|
||||
"runtimeStateKeys": [
|
||||
"workflow._auto_chain_active"
|
||||
@@ -171,6 +176,11 @@
|
||||
"topLevel": "review",
|
||||
"source": "^review\\.max_prompt_tokens_per_reviewer\\.[a-zA-Z0-9_-]+$",
|
||||
"description": "review.max_prompt_tokens_per_reviewer.<reviewer-slug>"
|
||||
},
|
||||
{
|
||||
"topLevel": "model_policy",
|
||||
"source": "^model_policy\\.runtime_tiers\\.[a-zA-Z0-9_-]+\\.(opus|sonnet|haiku)$",
|
||||
"description": "model_policy.runtime_tiers.<runtime>.<opus|sonnet|haiku>"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -83,6 +83,33 @@
|
||||
"haiku": null
|
||||
}
|
||||
},
|
||||
"providerPresets": {
|
||||
"anthropic": {
|
||||
"opus": { "low": { "model": "claude-opus-4-5" }, "medium": { "model": "claude-opus-4-8" }, "high": { "model": "claude-opus-4-8" } },
|
||||
"sonnet": { "low": { "model": "claude-haiku-4-5" }, "medium": { "model": "claude-sonnet-4-6" }, "high": { "model": "claude-opus-4-8" } },
|
||||
"haiku": { "low": { "model": "claude-haiku-4-5" }, "medium": { "model": "claude-haiku-4-5" }, "high": { "model": "claude-sonnet-4-6" } }
|
||||
},
|
||||
"openai": {
|
||||
"opus": { "low": { "model": "gpt-5.3-codex", "reasoning_effort": "medium" }, "medium": { "model": "gpt-5.5", "reasoning_effort": "high" }, "high": { "model": "gpt-5.5", "reasoning_effort": "xhigh" } },
|
||||
"sonnet": { "low": { "model": "gpt-5.4-mini", "reasoning_effort": "low" }, "medium": { "model": "gpt-5.3-codex", "reasoning_effort": "medium" }, "high": { "model": "gpt-5.5", "reasoning_effort": "medium" } },
|
||||
"haiku": { "low": { "model": "gpt-5.4-mini", "reasoning_effort": "minimal" }, "medium": { "model": "gpt-5.4-mini", "reasoning_effort": "medium" }, "high": { "model": "gpt-5.3-codex", "reasoning_effort": "medium" } }
|
||||
},
|
||||
"google": {
|
||||
"opus": { "low": { "model": "gemini-2.5-flash-lite" }, "medium": { "model": "gemini-3-flash" }, "high": { "model": "gemini-3-pro" } },
|
||||
"sonnet": { "low": { "model": "gemini-2.5-flash-lite" }, "medium": { "model": "gemini-3-flash" }, "high": { "model": "gemini-3-flash" } },
|
||||
"haiku": { "low": { "model": "gemini-2.5-flash-lite" }, "medium": { "model": "gemini-2.5-flash-lite" }, "high": { "model": "gemini-3-flash" } }
|
||||
},
|
||||
"qwen": {
|
||||
"opus": { "low": { "model": "qwen3-coder-plus" }, "medium": { "model": "qwen3-max-2026-01-23" }, "high": { "model": "qwen3-max-2026-01-23" } },
|
||||
"sonnet": { "low": { "model": "qwen3-coder-next" }, "medium": { "model": "qwen3-coder-plus" }, "high": { "model": "qwen3-max-2026-01-23" } },
|
||||
"haiku": { "low": { "model": "qwen3-coder-next" }, "medium": { "model": "qwen3-coder-next" }, "high": { "model": "qwen3-coder-plus" } }
|
||||
},
|
||||
"generic": {
|
||||
"opus": { "low": null, "medium": null, "high": null },
|
||||
"sonnet": { "low": null, "medium": null, "high": null },
|
||||
"haiku": { "low": null, "medium": null, "high": null }
|
||||
}
|
||||
},
|
||||
"agents": {
|
||||
"gsd-planner": { "golden": "opus", "balanced": "opus", "budget": "sonnet", "phaseType": "planning", "routingTier": "heavy" },
|
||||
"gsd-roadmapper": { "golden": "opus", "balanced": "sonnet", "budget": "sonnet", "phaseType": "planning", "routingTier": "heavy" },
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
<purpose>
|
||||
Interactive configuration of GSD power-user knobs — plan bounce, node repair, subagent timeouts,
|
||||
inline plan threshold, cross-AI execution, base branch, branch templates, response language,
|
||||
context window, gitignored search, graphify build timeout, and runtime model tier overrides.
|
||||
context window, gitignored search, graphify build timeout, runtime model tier overrides, and
|
||||
model policy configuration (provider + budget → canonical tier mapping, or manual model ID
|
||||
assignment per cost tier).
|
||||
|
||||
This is a companion to `/gsd:settings` — the common-case prompt there covers model profile,
|
||||
research/plan_check/verifier toggles, branching strategy, UI/AI phase gates, and worktree
|
||||
isolation. This advanced command covers everything else that is user-settable, grouped into
|
||||
seven sections so each prompt batch stays cognitively scoped. Every answer pre-selects the
|
||||
eight sections so each prompt batch stays cognitively scoped. Every answer pre-selects the
|
||||
current value; numeric-input answers that are non-numeric are rejected and re-prompted.
|
||||
</purpose>
|
||||
|
||||
@@ -81,6 +83,13 @@ Runtime Model Tiers:
|
||||
- `model_profile_overrides.<runtime>.sonnet` (default: built-in for the runtime, or absent)
|
||||
- `model_profile_overrides.<runtime>.haiku` (default: built-in for the runtime, or absent)
|
||||
|
||||
Model Policy:
|
||||
- `model_policy.provider` (default: `null` — known values: anthropic, openai, google, qwen)
|
||||
- `model_policy.budget` (default: `null` — known values: high, medium, low)
|
||||
- `model_policy.high` (default: `null` — model ID for the high-cost tier; used by generic provider path)
|
||||
- `model_policy.medium` (default: `null` — model ID for the medium-cost tier; used by generic provider path)
|
||||
- `model_policy.low` (default: `null` — model ID for the low-cost tier; used by generic provider path)
|
||||
|
||||
Each field's **current value is pre-selected** in the prompt rendering below. When the
|
||||
current value is absent from the config, render the documented default as the pre-selected
|
||||
option so the user sees what the effective value is.
|
||||
@@ -500,7 +509,7 @@ gsd_run query config-set model_profile_overrides.gemini.haiku null
|
||||
|
||||
Conceptual shape after merge (unchanged top-level keys like `model_profile`,
|
||||
`granularity`, `mode`, `brave_search`, `agent_skills.*`, `hooks.context_warnings`, and
|
||||
anything not listed in Sections 1–7 MUST survive the update):
|
||||
anything not listed in Sections 1–8 MUST survive the update):
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -542,6 +551,14 @@ anything not listed in Sections 1–7 MUST survive the update):
|
||||
"sonnet": <new|existing|null>,
|
||||
"haiku": <new|existing|null>
|
||||
}
|
||||
},
|
||||
"model_policy": {
|
||||
...existing_model_policy,
|
||||
"provider": <new|existing|null>,
|
||||
"budget": <new|existing|null>,
|
||||
"high": <new|existing|null>,
|
||||
"medium": <new|existing|null>,
|
||||
"low": <new|existing|null>
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -551,6 +568,170 @@ route each write through `gsd-tools.cjs query config-set` so sibling preservatio
|
||||
the central setter.
|
||||
</step>
|
||||
|
||||
### Section 8 — Model Policy
|
||||
|
||||
This section configures the `model_policy` key in `.planning/config.json`. Model policy
|
||||
defines which AI models GSD uses at each cost tier (low / medium / high), independently
|
||||
of the `runtime` and `model_profile` selections above. Two paths are offered:
|
||||
|
||||
- **Known provider:** choose a provider and a budget level; GSD materializes the canonical
|
||||
tier mapping for that provider.
|
||||
- **Generic provider:** enter low / medium / high model IDs manually.
|
||||
|
||||
**Step A — Read and display the current model policy:**
|
||||
|
||||
```bash
|
||||
cat "$GSD_CONFIG_PATH" | python3 -c "import sys,json; c=json.load(sys.stdin); mp=c.get('model_policy',{}); print(json.dumps(mp,indent=2))" 2>/dev/null || echo "{}"
|
||||
```
|
||||
|
||||
Display the current values (or "(unset)" for any absent field) before asking:
|
||||
|
||||
```text
|
||||
Current model_policy:
|
||||
provider : <value or "(unset)">
|
||||
budget : <value or "(unset)">
|
||||
low : <value or "(unset)">
|
||||
medium : <value or "(unset)">
|
||||
high : <value or "(unset)">
|
||||
```
|
||||
|
||||
**Step B — Choose configuration path:**
|
||||
|
||||
```text
|
||||
AskUserQuestion([
|
||||
{
|
||||
question: "How do you want to configure the model policy?",
|
||||
header: "Model Policy",
|
||||
multiSelect: false,
|
||||
options: [
|
||||
{ label: "Known provider", description: "Choose a provider (Claude / OpenAI / Gemini / Qwen) and a budget level — GSD writes the canonical tier mapping automatically." },
|
||||
{ label: "Generic provider", description: "Enter low / medium / high model IDs manually for any provider or custom deployment." },
|
||||
{ label: "Keep current", description: "Leave model_policy unchanged." }
|
||||
]
|
||||
}
|
||||
])
|
||||
```
|
||||
|
||||
**If "Keep current" is selected:** skip Steps C–E and move on to the confirm step.
|
||||
|
||||
**Step C — Known-provider path:**
|
||||
|
||||
```text
|
||||
AskUserQuestion([
|
||||
{
|
||||
question: "Which provider?",
|
||||
header: "Provider",
|
||||
multiSelect: false,
|
||||
options: [
|
||||
{ label: "anthropic", description: "claude-opus-4-8 / claude-sonnet-4-6 / claude-haiku-4-5 (Anthropic / Claude)" },
|
||||
{ label: "openai", description: "gpt-5.5 / gpt-5.3-codex / gpt-5.4-mini (OpenAI / Codex)" },
|
||||
{ label: "google", description: "gemini-3-pro / gemini-3-flash / gemini-2.5-flash-lite (Google Gemini)" },
|
||||
{ label: "qwen", description: "qwen3-max-2026-01-23 / qwen3-coder-plus / qwen3-coder-next (Qwen)" }
|
||||
]
|
||||
}
|
||||
])
|
||||
```
|
||||
|
||||
After the user picks a provider, ask:
|
||||
|
||||
```text
|
||||
AskUserQuestion([
|
||||
{
|
||||
question: "Which budget level?",
|
||||
header: "Budget",
|
||||
multiSelect: false,
|
||||
options: [
|
||||
{ label: "high", description: "All tiers use the highest-quality model for the chosen provider. Highest cost." },
|
||||
{ label: "medium", description: "High tier → top model; medium → mid model; low → cheapest model. Best cost/quality ratio." },
|
||||
{ label: "low", description: "All tiers use the cheapest model for the chosen provider. Lowest cost." }
|
||||
]
|
||||
}
|
||||
])
|
||||
```
|
||||
|
||||
Canonical tier mappings by provider and budget:
|
||||
|
||||
| Provider | Budget | high | medium | low |
|
||||
|-----------|--------|----------------------------|----------------------------|----------------------------|
|
||||
| anthropic | high | claude-opus-4-8 | claude-opus-4-8 | claude-opus-4-8 |
|
||||
| anthropic | medium | claude-opus-4-8 | claude-sonnet-4-6 | claude-haiku-4-5 |
|
||||
| anthropic | low | claude-haiku-4-5 | claude-haiku-4-5 | claude-haiku-4-5 |
|
||||
| openai | high | gpt-5.5 | gpt-5.5 | gpt-5.5 |
|
||||
| openai | medium | gpt-5.5 | gpt-5.3-codex | gpt-5.4-mini |
|
||||
| openai | low | gpt-5.4-mini | gpt-5.4-mini | gpt-5.4-mini |
|
||||
| google | high | gemini-3-pro | gemini-3-pro | gemini-3-pro |
|
||||
| google | medium | gemini-3-pro | gemini-3-flash | gemini-2.5-flash-lite |
|
||||
| google | low | gemini-2.5-flash-lite | gemini-2.5-flash-lite | gemini-2.5-flash-lite |
|
||||
| qwen | high | qwen3-max-2026-01-23 | qwen3-max-2026-01-23 | qwen3-max-2026-01-23 |
|
||||
| qwen | medium | qwen3-max-2026-01-23 | qwen3-coder-plus | qwen3-coder-next |
|
||||
| qwen | low | qwen3-coder-next | qwen3-coder-next | qwen3-coder-next |
|
||||
|
||||
Look up the selected (provider, budget) row and proceed to Step E to write those values.
|
||||
|
||||
**Step D — Generic-provider path:**
|
||||
|
||||
Prompt the user to enter each model ID as a free-text input. An empty input means "keep
|
||||
the current value for that tier." Validate that non-empty inputs are non-blank strings
|
||||
(no whitespace-only values); if validation fails, re-prompt that single field.
|
||||
|
||||
```text
|
||||
AskUserQuestion([
|
||||
{
|
||||
question: "Model ID for the HIGH-cost tier? (most capable model — used for heavy reasoning tasks)",
|
||||
header: "High-tier model",
|
||||
multiSelect: false,
|
||||
options: [
|
||||
{ label: "Keep current", description: "Leave unchanged (current: <model_policy.high or '(unset)'>)." },
|
||||
{ label: "Enter model ID", description: "Type the exact model identifier. Non-blank string required." }
|
||||
]
|
||||
},
|
||||
{
|
||||
question: "Model ID for the MEDIUM-cost tier? (balanced model — used for most agents)",
|
||||
header: "Medium-tier model",
|
||||
multiSelect: false,
|
||||
options: [
|
||||
{ label: "Keep current", description: "Leave unchanged (current: <model_policy.medium or '(unset)'>)." },
|
||||
{ label: "Enter model ID", description: "Type the exact model identifier." }
|
||||
]
|
||||
},
|
||||
{
|
||||
question: "Model ID for the LOW-cost tier? (cheapest model — used for lightweight/fast tasks)",
|
||||
header: "Low-tier model",
|
||||
multiSelect: false,
|
||||
options: [
|
||||
{ label: "Keep current", description: "Leave unchanged (current: <model_policy.low or '(unset)'>)." },
|
||||
{ label: "Enter model ID", description: "Type the exact model identifier." }
|
||||
]
|
||||
}
|
||||
])
|
||||
```
|
||||
|
||||
Set `provider = "custom"` and `budget = null` when writing the generic-provider result.
|
||||
Proceed to Step E.
|
||||
|
||||
**Step E — Write model_policy to config:**
|
||||
|
||||
```bash
|
||||
# Known-provider path — write all four keys atomically:
|
||||
gsd_run query config-set model_policy.provider "<provider>" # e.g., anthropic / openai / google / qwen
|
||||
gsd_run query config-set model_policy.budget "<budget>" # high / medium / low
|
||||
gsd_run query config-set model_policy.high "<high-id>"
|
||||
gsd_run query config-set model_policy.medium "<medium-id>"
|
||||
gsd_run query config-set model_policy.low "<low-id>"
|
||||
|
||||
# Generic-provider path — write only tiers the user changed ("Keep current" skipped):
|
||||
gsd_run query config-set model_policy.provider "custom"
|
||||
gsd_run query config-set model_policy.budget null
|
||||
# Per-tier writes for each non-"Keep current" answer:
|
||||
gsd_run query config-set model_policy.high "<high-id>" # omit if user chose "Keep current"
|
||||
gsd_run query config-set model_policy.medium "<medium-id>" # omit if user chose "Keep current"
|
||||
gsd_run query config-set model_policy.low "<low-id>" # omit if user chose "Keep current"
|
||||
```
|
||||
|
||||
Never write a tier the user explicitly chose to keep; the existing value must survive.
|
||||
|
||||
</step>
|
||||
|
||||
<step name="confirm">
|
||||
Display:
|
||||
|
||||
@@ -594,6 +775,11 @@ Display:
|
||||
| fast_mode.routing_tier_defaults.standard | {true/false} |
|
||||
| fast_mode.routing_tier_defaults.heavy | {true/false} |
|
||||
| fast_mode.agent_overrides.<agent-id> | {true/false} |
|
||||
| model_policy.provider | {anthropic/openai/google/qwen/custom/null} |
|
||||
| model_policy.budget | {high/medium/low/null} |
|
||||
| model_policy.high | {model-id/null} |
|
||||
| model_policy.medium | {model-id/null} |
|
||||
| model_policy.low | {model-id/null} |
|
||||
|
||||
These settings apply to future /gsd:plan-phase, /gsd:execute-phase, /gsd:discuss-phase,
|
||||
and /gsd:ship runs.
|
||||
@@ -607,7 +793,7 @@ UI/AI phase gates), use /gsd:settings.
|
||||
|
||||
<success_criteria>
|
||||
- [ ] Current config read from resolved `$GSD_CONFIG_PATH`
|
||||
- [ ] Seven sections rendered (Planning, Execution, Discussion, Cross-AI, Git, Runtime/Output, Runtime Model Tiers)
|
||||
- [ ] Eight sections rendered (Planning, Execution, Discussion, Cross-AI, Git, Runtime/Output, Runtime Model Tiers, Model Policy)
|
||||
- [ ] Every field pre-selected to its current value (or documented default if absent)
|
||||
- [ ] Numeric inputs validated — non-numeric rejected and re-prompted
|
||||
- [ ] Branch-template inputs validated — non-default must contain a placeholder
|
||||
@@ -616,5 +802,9 @@ UI/AI phase gates), use /gsd:settings.
|
||||
- [ ] Section 7 shows current runtime and built-in tier table
|
||||
- [ ] Group B runtimes display "(no built-in default — your runtime handles model selection)"
|
||||
- [ ] Override set/clear/keep paths all work correctly for each tier
|
||||
- [ ] Confirmation table rendered listing all 23 fields (19 + runtime + 3 tier overrides)
|
||||
- [ ] Section 8 (Model Policy) offers three top-level choices: Known provider, Generic provider, Keep current
|
||||
- [ ] Known-provider path: provider + budget → canonical tier mapping written to model_policy.{provider,budget,high,medium,low}
|
||||
- [ ] Generic-provider path: per-tier manual model IDs; "Keep current" tiers are never written; provider=custom budget=null
|
||||
- [ ] model_policy written under the model_policy key in config.json, never as a top-level flat key
|
||||
- [ ] Confirmation table rendered listing all fields including model_policy.{provider,budget,high,medium,low}
|
||||
</success_criteria>
|
||||
|
||||
@@ -57,6 +57,11 @@ Parse current values (default to `true` if not present):
|
||||
- `model_profile` — which model each agent uses (default: `balanced`)
|
||||
- `git.branching_strategy` — branching approach (default: `"none"`)
|
||||
- `workflow.use_worktrees` — whether parallel executor agents run in worktree isolation (default: `true`)
|
||||
- `model_policy.provider` — provider slug for model policy (default: `null`; known values: anthropic, openai, google, qwen; set via /gsd:config --advanced)
|
||||
- `model_policy.budget` — budget level for model policy (default: `null`; known values: high, medium, low; set via /gsd:config --advanced)
|
||||
- `model_policy.high` — model ID for high-cost tier (default: `null`; set via /gsd:config --advanced)
|
||||
- `model_policy.medium` — model ID for medium-cost tier (default: `null`; set via /gsd:config --advanced)
|
||||
- `model_policy.low` — model ID for low-cost tier (default: `null`; set via /gsd:config --advanced)
|
||||
</step>
|
||||
|
||||
<step name="present_settings">
|
||||
@@ -423,6 +428,15 @@ Merge new settings into existing config.json:
|
||||
"hooks": {
|
||||
"context_warnings": true/false,
|
||||
"workflow_guard": true/false
|
||||
},
|
||||
"model_policy": {
|
||||
// Read-only in this flow — written only by /gsd:config --advanced (Section 8).
|
||||
// Listed here so safe-merge never clobbers an existing model_policy object.
|
||||
"provider": <existing|null>,
|
||||
"budget": <existing|null>,
|
||||
"high": <existing|null>,
|
||||
"medium": <existing|null>,
|
||||
"low": <existing|null>
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -537,7 +551,7 @@ Quick commands:
|
||||
- /gsd:plan-phase --research — force research
|
||||
- /gsd:plan-phase --skip-research — skip research
|
||||
- /gsd:plan-phase --skip-verify — skip plan check
|
||||
- /gsd:config --advanced — power-user tuning (plan bounce, timeouts, branch templates, cross-AI, context window)
|
||||
- /gsd:config --advanced — power-user tuning (plan bounce, timeouts, branch templates, cross-AI, context window, model policy)
|
||||
```
|
||||
</step>
|
||||
|
||||
|
||||
62
src/config-types.cts
Normal file
62
src/config-types.cts
Normal file
@@ -0,0 +1,62 @@
|
||||
/**
|
||||
* TypeScript type definitions for GSD project config — model_policy block.
|
||||
*
|
||||
* These types reflect the model_policy config shape consumed by
|
||||
* resolveModelPolicy in core.cjs and validated by config-schema.cjs.
|
||||
*
|
||||
* See feat #49 (model_policy presets) and config-schema.manifest.json.
|
||||
* Added under ADR-457: TS sources in src/ compile to CJS artifacts in
|
||||
* get-shit-done/bin/lib/ at publish time.
|
||||
*
|
||||
* Resolution precedence (highest → lowest):
|
||||
* 1. model_overrides[agent]
|
||||
* 2. model_policy.runtime_tiers[runtime][tier] (Sub-path A)
|
||||
* 3. model_policy provider preset + budget (Sub-path B)
|
||||
* 4. model_profile_overrides
|
||||
* 5. resolve_model_ids / profile fallback
|
||||
*/
|
||||
|
||||
/**
|
||||
* A single tier entry mapping a GSD tier (opus | sonnet | haiku) to a
|
||||
* concrete model ID. The optional `reasoning_effort` field is forwarded to
|
||||
* runtimes that accept it (e.g. opencode).
|
||||
*/
|
||||
export interface TierEntry {
|
||||
model: string;
|
||||
reasoning_effort?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The three standard GSD tiers for one runtime target. All fields are
|
||||
* optional so callers can supply a partial override (e.g. only `opus`).
|
||||
*/
|
||||
export interface RuntimeTiers {
|
||||
low?: TierEntry;
|
||||
medium?: TierEntry;
|
||||
high?: TierEntry;
|
||||
}
|
||||
|
||||
/**
|
||||
* Top-level `model_policy` block in `.planning/config.json`.
|
||||
*
|
||||
* - `provider` — known provider slug (e.g. `"anthropic"`, `"openai"`).
|
||||
* Drives Sub-path B catalog lookup.
|
||||
* - `budget` — optional spend/quality tier that pairs with `provider`
|
||||
* to select a preset from the model catalog.
|
||||
* - `runtime_tiers` — explicit per-runtime, per-tier model overrides
|
||||
* (Sub-path A). Keys are runtime slugs (e.g. `"opencode"`,
|
||||
* `"copilot"`); values are `RuntimeTiers` maps.
|
||||
*/
|
||||
export interface ModelPolicyConfig {
|
||||
provider: string;
|
||||
budget?: string;
|
||||
runtime_tiers?: Record<string, RuntimeTiers>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Minimal subset of the GSD project config that includes `model_policy`.
|
||||
* Extend this interface when migrating further config keys to TypeScript.
|
||||
*/
|
||||
export interface ProjectConfig {
|
||||
model_policy?: ModelPolicyConfig;
|
||||
}
|
||||
680
tests/feat-49-model-policy-presets.test.cjs
Normal file
680
tests/feat-49-model-policy-presets.test.cjs
Normal file
@@ -0,0 +1,680 @@
|
||||
/**
|
||||
* Feature test for issue #49 — model_policy presets.
|
||||
*
|
||||
* Adds a `model_policy` block to .planning/config.json:
|
||||
*
|
||||
* {
|
||||
* "model_policy": {
|
||||
* "provider": "anthropic",
|
||||
* "budget": "high",
|
||||
* "runtime_tiers": {
|
||||
* "opencode": {
|
||||
* "opus": { "model": "anthropic/claude-opus-4-8" }
|
||||
* }
|
||||
* }
|
||||
* }
|
||||
* }
|
||||
*
|
||||
* Resolution precedence in resolveModelInternal (highest → lowest):
|
||||
* 1. model_overrides[agent] (per-agent full IDs; existing)
|
||||
* 2. model_policy.runtime_tiers[runtime][tier] (Sub-path A: explicit runtime+tier entry)
|
||||
* 3. model_policy provider preset + budget (Sub-path B: known-provider catalog lookup)
|
||||
* 4. model_profile_overrides (legacy runtime-aware overrides)
|
||||
* 5. resolve_model_ids / profile fallback
|
||||
*
|
||||
* Sub-path A (runtime_tiers) fires when config.runtime matches a key inside
|
||||
* model_policy.runtime_tiers AND that key contains an entry for the resolved tier.
|
||||
*
|
||||
* Sub-path B (provider preset) fires when model_policy.provider is a known
|
||||
* provider AND the catalog contains an entry for (tier, budget) pair.
|
||||
*
|
||||
* Both sub-paths return a string model ID. Failures in either sub-path fall
|
||||
* through cleanly to the next step in the chain.
|
||||
*
|
||||
* New config keys accepted by isValidConfigKey:
|
||||
* - model_policy.provider
|
||||
* - model_policy.budget
|
||||
* - model_policy.runtime_tiers.<runtime>.<tier>
|
||||
*
|
||||
* Backwards compatibility:
|
||||
* - model_profile_overrides continues to work when model_policy is absent.
|
||||
* - When both are set, model_policy wins (fires first).
|
||||
*
|
||||
* KNOWN_PROVIDERS is exported from both model-catalog.cjs and core.cjs (re-export).
|
||||
*
|
||||
* These tests are written to FAIL before implementation. They use typed-IR /
|
||||
* structural assertions on resolveModelInternal / resolveModelPolicy / isValidConfigKey
|
||||
* return values — not stdout / grep.
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
|
||||
process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
const { test, describe, beforeEach, afterEach } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const os = require('node:os');
|
||||
|
||||
// ─── Imports (will fail until implementation exists) ────────────────────────
|
||||
// resolveModelPolicy is a new internal function that must be exported from core.cjs.
|
||||
// KNOWN_PROVIDERS must be exported from model-catalog.cjs and re-exported by core.cjs.
|
||||
const {
|
||||
resolveModelInternal,
|
||||
resolveModelPolicy,
|
||||
KNOWN_PROVIDERS,
|
||||
_resetRuntimeWarningCacheForTests,
|
||||
} = require('../get-shit-done/bin/lib/core.cjs');
|
||||
|
||||
// KNOWN_PROVIDERS must also be exported directly from model-catalog.cjs
|
||||
const modelCatalog = require('../get-shit-done/bin/lib/model-catalog.cjs');
|
||||
|
||||
const { isValidConfigKey } = require('../get-shit-done/bin/lib/config-schema.cjs');
|
||||
const { createTempDir } = require('./helpers.cjs');
|
||||
|
||||
const makeTmp = (prefix) => createTempDir(`gsd-49-${prefix}-`);
|
||||
|
||||
function writeConfig(dir, config) {
|
||||
const planningDir = path.join(dir, '.planning');
|
||||
fs.mkdirSync(planningDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(planningDir, 'config.json'), JSON.stringify(config, null, 2));
|
||||
}
|
||||
|
||||
function rmr(p) {
|
||||
try { fs.rmSync(p, { recursive: true, force: true }); } catch { /* noop */ }
|
||||
}
|
||||
|
||||
// ─── resolveModelPolicy unit tests ──────────────────────────────────────────
|
||||
//
|
||||
// resolveModelPolicy(config, tier) is the pure resolver that takes a loaded
|
||||
// config object and a resolved tier string. It returns a string model ID when
|
||||
// model_policy produces a hit, or null when it falls through.
|
||||
|
||||
describe('#49 resolveModelPolicy: null/absent policy returns null', () => {
|
||||
test('resolveModelPolicy returns null when policy is null or absent', () => {
|
||||
// policy is null
|
||||
assert.strictEqual(resolveModelPolicy(null, 'opus'), null);
|
||||
// policy is undefined
|
||||
assert.strictEqual(resolveModelPolicy(undefined, 'opus'), null);
|
||||
// policy is absent (empty object treated as absent)
|
||||
assert.strictEqual(resolveModelPolicy({}, 'opus'), null);
|
||||
});
|
||||
|
||||
test('resolveModelPolicy returns null when runtime or tier is missing', () => {
|
||||
const policy = { provider: 'anthropic', budget: 'high' };
|
||||
// tier is null
|
||||
assert.strictEqual(resolveModelPolicy(policy, null), null);
|
||||
// tier is empty string
|
||||
assert.strictEqual(resolveModelPolicy(policy, ''), null);
|
||||
// tier is undefined
|
||||
assert.strictEqual(resolveModelPolicy(policy, undefined), null);
|
||||
});
|
||||
});
|
||||
|
||||
describe('#49 resolveModelPolicy Sub-path B: provider presets', () => {
|
||||
test('known provider "anthropic" + tier "opus" + budget "high" returns correct model ID', () => {
|
||||
// The anthropic preset catalog must contain an entry for opus+high.
|
||||
// The returned model ID is the high-budget anthropic opus model.
|
||||
const policy = { provider: 'anthropic', budget: 'high' };
|
||||
const result = resolveModelPolicy(policy, 'opus');
|
||||
assert.ok(typeof result === 'string' && result.length > 0,
|
||||
`expected a non-empty model ID string, got: ${JSON.stringify(result)}`);
|
||||
// Anthropic opus model IDs contain "claude" and "opus"
|
||||
assert.match(result, /claude.*opus|opus.*claude/i,
|
||||
`expected anthropic opus model ID to contain "claude" and "opus", got: ${result}`);
|
||||
});
|
||||
|
||||
test('known provider "openai" + tier "sonnet" + budget "low" returns model with reasoning_effort from preset', () => {
|
||||
// The openai preset catalog must contain a sonnet+low entry.
|
||||
// "openai" maps to a different model family; the entry may include reasoning_effort.
|
||||
const policy = { provider: 'openai', budget: 'low' };
|
||||
const result = resolveModelPolicy(policy, 'sonnet');
|
||||
assert.ok(typeof result === 'string' && result.length > 0,
|
||||
`expected a non-empty model ID string for openai/sonnet/low, got: ${JSON.stringify(result)}`);
|
||||
});
|
||||
|
||||
test('budget absent defaults to "medium"', () => {
|
||||
// No "budget" key — defaults to "medium". The anthropic/opus/medium entry must exist.
|
||||
const policyWithBudget = { provider: 'anthropic', budget: 'medium' };
|
||||
const policyNoBudget = { provider: 'anthropic' };
|
||||
const withBudget = resolveModelPolicy(policyWithBudget, 'opus');
|
||||
const withoutBudget = resolveModelPolicy(policyNoBudget, 'opus');
|
||||
// Both must return a string (not null)
|
||||
assert.ok(typeof withBudget === 'string' && withBudget.length > 0,
|
||||
`expected model from explicit budget:'medium'`);
|
||||
assert.ok(typeof withoutBudget === 'string' && withoutBudget.length > 0,
|
||||
`expected model when budget absent (should default to medium)`);
|
||||
// They must resolve to the same value
|
||||
assert.strictEqual(withBudget, withoutBudget,
|
||||
'absent budget must behave identically to explicit "medium"');
|
||||
});
|
||||
|
||||
test('provider "generic" (all null entries) returns null (falls through)', () => {
|
||||
// provider:'generic' means opaque model IDs — there's no preset catalog for
|
||||
// generic. Without a runtime_tiers hit, resolveModelPolicy returns null.
|
||||
const policy = { provider: 'generic', budget: 'high' };
|
||||
const result = resolveModelPolicy(policy, 'opus');
|
||||
assert.strictEqual(result, null,
|
||||
'provider:"generic" with no runtime_tiers must return null (no preset catalog)');
|
||||
});
|
||||
|
||||
test('unknown provider string returns null without throwing', () => {
|
||||
// A typo like provider:'mistral' must not crash; it degrades gracefully.
|
||||
const policy = { provider: 'mistral', budget: 'high' };
|
||||
let result;
|
||||
assert.doesNotThrow(() => {
|
||||
result = resolveModelPolicy(policy, 'opus');
|
||||
}, 'resolveModelPolicy must not throw on unknown provider');
|
||||
assert.strictEqual(result, null,
|
||||
'unknown provider with no runtime_tiers must return null');
|
||||
});
|
||||
|
||||
test('known provider + unknown tier returns null', () => {
|
||||
const policy = { provider: 'anthropic', budget: 'high' };
|
||||
const result = resolveModelPolicy(policy, 'jumbo');
|
||||
assert.strictEqual(result, null,
|
||||
'unknown tier "jumbo" must return null for anthropic provider');
|
||||
});
|
||||
|
||||
test('known provider + known tier + missing budget level returns null', () => {
|
||||
// The anthropic preset for opus only defines 'high' and 'medium' but NOT 'critical'.
|
||||
// A missing budget level must fall through (return null) — not crash.
|
||||
const policy = { provider: 'anthropic', budget: 'critical' };
|
||||
const result = resolveModelPolicy(policy, 'opus');
|
||||
assert.strictEqual(result, null,
|
||||
'missing budget level "critical" must return null without throwing');
|
||||
});
|
||||
});
|
||||
|
||||
describe('#49 resolveModelPolicy Sub-path A: runtime_tiers', () => {
|
||||
test('runtime_tiers entry wins over provider preset for same runtime+tier', () => {
|
||||
// Sub-path A fires first: explicit runtime_tiers entry overrides the
|
||||
// provider preset catalog. The returned model is the one in runtime_tiers,
|
||||
// not what the provider preset would have returned.
|
||||
const policy = {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime: 'opencode',
|
||||
runtime_tiers: {
|
||||
opencode: {
|
||||
opus: { model: 'anthropic/custom-opus-override' },
|
||||
},
|
||||
},
|
||||
};
|
||||
const result = resolveModelPolicy(policy, 'opus');
|
||||
assert.strictEqual(result, 'anthropic/custom-opus-override',
|
||||
'Sub-path A runtime_tiers must win over Sub-path B provider preset');
|
||||
});
|
||||
|
||||
test('runtime_tiers string shorthand normalized to { model } object', () => {
|
||||
// String shorthand: `{ opencode: { opus: "some-model-id" } }`
|
||||
// must be normalized to `{ model: "some-model-id" }` so the resolver
|
||||
// returns the string as-is.
|
||||
const policy = {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime: 'opencode',
|
||||
runtime_tiers: {
|
||||
opencode: {
|
||||
opus: 'anthropic/string-shorthand-model',
|
||||
},
|
||||
},
|
||||
};
|
||||
const result = resolveModelPolicy(policy, 'opus');
|
||||
assert.strictEqual(result, 'anthropic/string-shorthand-model',
|
||||
'string shorthand in runtime_tiers must be normalized and returned as model ID');
|
||||
});
|
||||
|
||||
test('runtime_tiers partial entry (no matching runtime) falls through to provider preset', () => {
|
||||
// runtime_tiers has entries for 'copilot' but the active runtime is 'opencode'.
|
||||
// The miss on runtime_tiers falls through to Sub-path B (provider preset).
|
||||
const policy = {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime: 'opencode',
|
||||
runtime_tiers: {
|
||||
copilot: {
|
||||
opus: { model: 'some-copilot-model' },
|
||||
},
|
||||
},
|
||||
};
|
||||
const result = resolveModelPolicy(policy, 'opus');
|
||||
// Falls through to Sub-path B (anthropic/opus/high) — must not be null.
|
||||
assert.ok(typeof result === 'string' && result.length > 0,
|
||||
'runtime_tiers miss must fall through to provider preset, got: ' + JSON.stringify(result));
|
||||
// And it must NOT be the copilot model
|
||||
assert.notStrictEqual(result, 'some-copilot-model');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── resolveModelInternal integration tests ──────────────────────────────────
|
||||
//
|
||||
// These tests call resolveModelInternal through a temp project's config.json.
|
||||
// They verify the full resolution chain including model_policy placement.
|
||||
|
||||
describe('#49 resolveModelInternal: model_policy in the resolution chain', () => {
|
||||
let projectDir;
|
||||
beforeEach(() => {
|
||||
projectDir = makeTmp('internal');
|
||||
_resetRuntimeWarningCacheForTests();
|
||||
});
|
||||
afterEach(() => {
|
||||
rmr(projectDir);
|
||||
_resetRuntimeWarningCacheForTests();
|
||||
});
|
||||
|
||||
test('model_policy fires before model_profile_overrides when both are set (model_policy wins)', () => {
|
||||
// model_policy (Sub-path B: anthropic/opus/high) must win over
|
||||
// model_profile_overrides when both are present.
|
||||
// We use a model_profile_overrides entry that would give a DIFFERENT result.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'quality',
|
||||
model_policy: {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
},
|
||||
model_profile_overrides: {
|
||||
opencode: {
|
||||
// This legacy override would have returned this model — but model_policy must win.
|
||||
opus: 'legacy-override-model-should-not-appear',
|
||||
},
|
||||
},
|
||||
});
|
||||
const result = resolveModelInternal(projectDir, 'gsd-planner');
|
||||
assert.notStrictEqual(result, 'legacy-override-model-should-not-appear',
|
||||
'model_policy must fire before model_profile_overrides and win');
|
||||
assert.ok(typeof result === 'string' && result.length > 0,
|
||||
'must return a non-empty model ID');
|
||||
// model_policy anthropic/opus/high should return a claude opus model ID
|
||||
assert.match(result, /claude.*opus|opus.*claude/i,
|
||||
'expected anthropic preset opus model, got: ' + result);
|
||||
});
|
||||
|
||||
test('model_policy with provider:"anthropic" + budget:"high" + runtime:"opencode" resolves to preset model', () => {
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'quality', // gsd-planner quality = opus tier
|
||||
model_policy: {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
},
|
||||
});
|
||||
const result = resolveModelInternal(projectDir, 'gsd-planner');
|
||||
assert.ok(typeof result === 'string' && result.length > 0,
|
||||
'expected a non-empty model ID');
|
||||
assert.match(result, /claude.*opus|opus.*claude/i,
|
||||
'anthropic/opus/high must resolve to an opus model ID');
|
||||
});
|
||||
|
||||
test('model_policy is skipped when runtime is absent', () => {
|
||||
// No `runtime` in config — model_policy fires on any non-null policy
|
||||
// only when a runtime context is available. Without runtime, the policy
|
||||
// falls through entirely.
|
||||
// NOTE: Sub-path B (provider preset) can fire without runtime — it only
|
||||
// needs tier+budget+provider. Sub-path A requires runtime. This test
|
||||
// verifies the gating behavior described in the issue: if model_policy
|
||||
// is present but runtime is absent, provider preset Sub-path B still
|
||||
// fires (it doesn't need runtime). So "skipped" means the runtime_tiers
|
||||
// sub-path is skipped but provider preset may still fire.
|
||||
// The test asserts that resolveModelInternal does not crash and returns
|
||||
// a string regardless.
|
||||
writeConfig(projectDir, {
|
||||
model_profile: 'quality',
|
||||
model_policy: {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime_tiers: {
|
||||
opencode: {
|
||||
opus: { model: 'should-not-appear-no-runtime' },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
let result;
|
||||
assert.doesNotThrow(() => {
|
||||
result = resolveModelInternal(projectDir, 'gsd-planner');
|
||||
});
|
||||
assert.ok(typeof result === 'string',
|
||||
'resolveModelInternal must return a string even when runtime is absent');
|
||||
// The runtime_tiers entry for opencode must not appear since runtime is absent
|
||||
assert.notStrictEqual(result, 'should-not-appear-no-runtime',
|
||||
'runtime_tiers must not fire when config.runtime is absent');
|
||||
});
|
||||
|
||||
test('model_policy is skipped when runtime:"claude" (no-op gate)', () => {
|
||||
// runtime:"claude" is the implicit default and is treated as a no-op
|
||||
// for model_policy resolution (same as the no-op gate in the existing
|
||||
// runtime-aware resolution step). model_policy provider preset
|
||||
// for anthropic may still fire — but runtime_tiers for claude is a no-op
|
||||
// because claude-native resolution already handles that path.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'claude',
|
||||
model_profile: 'quality',
|
||||
model_policy: {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime_tiers: {
|
||||
claude: {
|
||||
opus: { model: 'claude-runtime-tiers-should-not-appear' },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
let result;
|
||||
assert.doesNotThrow(() => {
|
||||
result = resolveModelInternal(projectDir, 'gsd-planner');
|
||||
});
|
||||
assert.ok(typeof result === 'string', 'must return a string');
|
||||
// The claude runtime_tiers entry must not appear — model_policy runtime_tiers
|
||||
// is a no-op for runtime:"claude"
|
||||
assert.notStrictEqual(result, 'claude-runtime-tiers-should-not-appear',
|
||||
'model_policy.runtime_tiers must be a no-op when runtime is "claude"');
|
||||
});
|
||||
|
||||
test('model_policy is skipped when tier:"inherit"', () => {
|
||||
// When the resolved tier is 'inherit', model_policy must not fire.
|
||||
// This mirrors the existing behavior for runtime-aware resolution.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'inherit',
|
||||
model_policy: {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
},
|
||||
});
|
||||
const result = resolveModelInternal(projectDir, 'gsd-planner');
|
||||
// With profile:'inherit', the result must be 'inherit'
|
||||
assert.strictEqual(result, 'inherit',
|
||||
'model_policy must not fire when tier is "inherit"; resolveModelInternal must return "inherit"');
|
||||
});
|
||||
|
||||
test('model_profile_overrides still resolves when model_policy is absent (legacy fallback intact)', () => {
|
||||
// No model_policy — model_profile_overrides must still work exactly as before.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'quality',
|
||||
model_profile_overrides: {
|
||||
opencode: {
|
||||
opus: 'legacy-overridden-model',
|
||||
},
|
||||
},
|
||||
});
|
||||
const result = resolveModelInternal(projectDir, 'gsd-planner');
|
||||
assert.strictEqual(result, 'legacy-overridden-model',
|
||||
'model_profile_overrides must still win when model_policy is absent');
|
||||
});
|
||||
|
||||
test('model_policy absent + model_profile_overrides set → model_profile_overrides wins (back-compat)', () => {
|
||||
// Explicit: no model_policy key at all. model_profile_overrides is the only
|
||||
// custom config. The legacy chain must apply exactly as before this feature.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'balanced',
|
||||
model_profile_overrides: {
|
||||
opencode: {
|
||||
sonnet: 'back-compat-sonnet-model',
|
||||
},
|
||||
},
|
||||
});
|
||||
// gsd-executor has balanced/opencode -> sonnet tier
|
||||
const result = resolveModelInternal(projectDir, 'gsd-executor');
|
||||
assert.strictEqual(result, 'back-compat-sonnet-model',
|
||||
'legacy model_profile_overrides must be unaffected when model_policy is absent');
|
||||
});
|
||||
|
||||
test('model_policy present but runtime_tiers empty + provider:"generic" → falls through to model_profile_overrides', () => {
|
||||
// model_policy is a stub: runtime_tiers is empty ({}), provider is "generic".
|
||||
// The resolver must fall through all model_policy paths and land on model_profile_overrides.
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'quality',
|
||||
model_policy: {
|
||||
provider: 'generic',
|
||||
budget: 'high',
|
||||
runtime_tiers: {},
|
||||
},
|
||||
model_profile_overrides: {
|
||||
opencode: {
|
||||
opus: 'fallthrough-to-legacy',
|
||||
},
|
||||
},
|
||||
});
|
||||
const result = resolveModelInternal(projectDir, 'gsd-planner');
|
||||
assert.strictEqual(result, 'fallthrough-to-legacy',
|
||||
'empty runtime_tiers + generic provider must fall through to model_profile_overrides');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Warning emission tests ───────────────────────────────────────────────────
|
||||
|
||||
describe('#49 resolveModelInternal: unknown provider warning behavior', () => {
|
||||
let projectDir;
|
||||
let origWrite;
|
||||
let captured;
|
||||
|
||||
beforeEach(() => {
|
||||
projectDir = makeTmp('warnings');
|
||||
_resetRuntimeWarningCacheForTests();
|
||||
captured = [];
|
||||
origWrite = process.stderr.write.bind(process.stderr);
|
||||
process.stderr.write = (chunk) => { captured.push(String(chunk)); return true; };
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.stderr.write = origWrite;
|
||||
rmr(projectDir);
|
||||
_resetRuntimeWarningCacheForTests();
|
||||
});
|
||||
|
||||
test('unknown provider in model_policy → falls through to model_profile_overrides, emits stderr warning once', () => {
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'quality',
|
||||
model_policy: {
|
||||
provider: 'mistral',
|
||||
budget: 'high',
|
||||
},
|
||||
model_profile_overrides: {
|
||||
opencode: {
|
||||
opus: 'fallback-from-unknown-provider',
|
||||
},
|
||||
},
|
||||
});
|
||||
const result = resolveModelInternal(projectDir, 'gsd-planner');
|
||||
// Must fall through to model_profile_overrides
|
||||
assert.strictEqual(result, 'fallback-from-unknown-provider',
|
||||
'unknown provider must fall through to model_profile_overrides');
|
||||
// Must emit at least one stderr warning about the unknown provider
|
||||
const joined = captured.join('');
|
||||
assert.match(joined, /model_policy.*provider.*mistral|unknown.*provider.*mistral|mistral.*unknown/i,
|
||||
'must emit a stderr warning about the unknown provider "mistral"');
|
||||
});
|
||||
|
||||
test('unknown provider warning is deduplicated (emitted only once per config label)', () => {
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'quality',
|
||||
model_policy: {
|
||||
provider: 'mistral',
|
||||
budget: 'high',
|
||||
},
|
||||
});
|
||||
// Call resolveModelInternal multiple times for different agents — the
|
||||
// warning about the unknown provider must be emitted only once.
|
||||
resolveModelInternal(projectDir, 'gsd-planner');
|
||||
resolveModelInternal(projectDir, 'gsd-executor');
|
||||
resolveModelInternal(projectDir, 'gsd-verifier');
|
||||
const joined = captured.join('');
|
||||
// Count occurrences of "mistral" in the warning output
|
||||
const matches = (joined.match(/mistral/gi) || []).length;
|
||||
assert.ok(matches >= 1, 'expected at least one warning about "mistral"');
|
||||
assert.ok(matches <= 2, `warning for unknown provider must be deduplicated — saw ${matches} occurrences`);
|
||||
});
|
||||
|
||||
test('model_policy.runtime_tiers with unknown runtime emits one-shot stderr warning', () => {
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'quality',
|
||||
model_policy: {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime_tiers: {
|
||||
unknownrt: {
|
||||
opus: { model: 'some-model' },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
resolveModelInternal(projectDir, 'gsd-planner');
|
||||
const joined = captured.join('');
|
||||
// Must emit a warning about the unknown runtime key in runtime_tiers
|
||||
assert.match(joined, /unknownrt|unknown.*runtime|runtime_tiers.*unknown/i,
|
||||
'must emit a stderr warning about unknown runtime "unknownrt" in model_policy.runtime_tiers');
|
||||
});
|
||||
|
||||
test('model_policy.runtime_tiers with invalid tier name emits one-shot stderr warning', () => {
|
||||
writeConfig(projectDir, {
|
||||
runtime: 'opencode',
|
||||
model_profile: 'quality',
|
||||
model_policy: {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime_tiers: {
|
||||
opencode: {
|
||||
jumbo: { model: 'invalid-tier-model' },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
resolveModelInternal(projectDir, 'gsd-planner');
|
||||
const joined = captured.join('');
|
||||
// Must emit a warning about the invalid tier name "jumbo"
|
||||
assert.match(joined, /jumbo|invalid.*tier|tier.*invalid|unknown.*tier/i,
|
||||
'must emit a stderr warning about invalid tier "jumbo" in model_policy.runtime_tiers.opencode');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── reasoning_effort passthrough tests ──────────────────────────────────────
|
||||
|
||||
describe('#49 reasoning_effort in model_policy entries', () => {
|
||||
let projectDir;
|
||||
beforeEach(() => { projectDir = makeTmp('effort'); });
|
||||
afterEach(() => { rmr(projectDir); });
|
||||
|
||||
test('reasoning_effort in preset entry is returned as part of the entry object (caller decides whether to emit)', () => {
|
||||
// When a provider preset includes reasoning_effort (e.g. openai opus/high),
|
||||
// resolveModelPolicy must return the full entry object (or at minimum the model
|
||||
// string) without stripping reasoning_effort internally.
|
||||
// This is checked via the internal resolveModelPolicy function directly.
|
||||
// The policy object includes a runtime_tiers entry that has reasoning_effort.
|
||||
const policy = {
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime: 'opencode',
|
||||
runtime_tiers: {
|
||||
opencode: {
|
||||
opus: { model: 'anthropic/claude-opus-4-8', reasoning_effort: 'high' },
|
||||
},
|
||||
},
|
||||
};
|
||||
// resolveModelPolicy must return the model string (at minimum).
|
||||
// The caller (resolveModelInternal) is responsible for deciding what to
|
||||
// emit — the resolver just returns the model ID string.
|
||||
const result = resolveModelPolicy(policy, 'opus');
|
||||
assert.strictEqual(result, 'anthropic/claude-opus-4-8',
|
||||
'resolveModelPolicy must return the model string from the runtime_tiers entry');
|
||||
});
|
||||
|
||||
test('reasoning_effort in model_policy.runtime_tiers entry is returned verbatim; renderEffortForRuntime strips it when runtime not in RUNTIMES_WITH_REASONING_EFFORT', () => {
|
||||
// The renderEffortForRuntime function (already existing) handles the stripping.
|
||||
// This test verifies the contract: resolveModelPolicy returns the model string,
|
||||
// and for runtimes not in RUNTIMES_WITH_REASONING_EFFORT, the caller must not
|
||||
// emit reasoning_effort.
|
||||
const { renderEffortForRuntime, RUNTIMES_WITH_REASONING_EFFORT } = require('../get-shit-done/bin/lib/model-catalog.cjs');
|
||||
|
||||
// 'opencode' is NOT in RUNTIMES_WITH_REASONING_EFFORT (only codex has reasoning_effort in catalog)
|
||||
assert.ok(!RUNTIMES_WITH_REASONING_EFFORT.has('opencode'),
|
||||
'opencode must not be in RUNTIMES_WITH_REASONING_EFFORT for this test to be meaningful');
|
||||
|
||||
// renderEffortForRuntime for a non-effort runtime returns channel:null
|
||||
const rendered = renderEffortForRuntime('opencode', 'high');
|
||||
assert.strictEqual(rendered.channel, null,
|
||||
'renderEffortForRuntime must return channel:null for runtimes not supporting reasoning_effort');
|
||||
|
||||
// The resolveModelPolicy function returns just the model string — reasoning_effort
|
||||
// is stripped at the emit layer, not inside resolveModelPolicy.
|
||||
const policy = {
|
||||
runtime: 'opencode',
|
||||
provider: 'anthropic',
|
||||
budget: 'high',
|
||||
runtime_tiers: {
|
||||
opencode: {
|
||||
opus: { model: 'anthropic/claude-opus-4-8', reasoning_effort: 'high' },
|
||||
},
|
||||
},
|
||||
};
|
||||
const result = resolveModelPolicy(policy, 'opus');
|
||||
assert.strictEqual(result, 'anthropic/claude-opus-4-8',
|
||||
'resolveModelPolicy must return model string; reasoning_effort is stripped downstream');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── isValidConfigKey: model_policy.* schema validation ──────────────────────
|
||||
|
||||
describe('#49 isValidConfigKey: model_policy.* keys accepted/rejected', () => {
|
||||
test('isValidConfigKey accepts "model_policy.provider"', () => {
|
||||
assert.strictEqual(isValidConfigKey('model_policy.provider'), true,
|
||||
'"model_policy.provider" must be a valid config key');
|
||||
});
|
||||
|
||||
test('isValidConfigKey accepts "model_policy.budget"', () => {
|
||||
assert.strictEqual(isValidConfigKey('model_policy.budget'), true,
|
||||
'"model_policy.budget" must be a valid config key');
|
||||
});
|
||||
|
||||
test('isValidConfigKey accepts "model_policy.runtime_tiers.opencode.opus"', () => {
|
||||
assert.strictEqual(isValidConfigKey('model_policy.runtime_tiers.opencode.opus'), true,
|
||||
'"model_policy.runtime_tiers.opencode.opus" must be a valid config key');
|
||||
});
|
||||
|
||||
test('isValidConfigKey rejects "model_policy.runtime_tiers.opencode.banana" (invalid tier)', () => {
|
||||
assert.strictEqual(isValidConfigKey('model_policy.runtime_tiers.opencode.banana'), false,
|
||||
'"model_policy.runtime_tiers.opencode.banana" must be rejected (banana is not a valid tier)');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── KNOWN_PROVIDERS export tests ─────────────────────────────────────────────
|
||||
|
||||
describe('#49 KNOWN_PROVIDERS exports from model-catalog.cjs and core.cjs', () => {
|
||||
test('KNOWN_PROVIDERS exported from core.cjs includes all keys from providerPresets in catalog', () => {
|
||||
// KNOWN_PROVIDERS must be a Set (or array) exported from core.cjs.
|
||||
assert.ok(KNOWN_PROVIDERS != null,
|
||||
'KNOWN_PROVIDERS must be exported from core.cjs');
|
||||
const isIterable = typeof KNOWN_PROVIDERS[Symbol.iterator] === 'function';
|
||||
assert.ok(isIterable,
|
||||
'KNOWN_PROVIDERS must be iterable (Set or array)');
|
||||
const providers = [...KNOWN_PROVIDERS];
|
||||
assert.ok(providers.length > 0,
|
||||
'KNOWN_PROVIDERS must not be empty');
|
||||
// 'anthropic' must be in the set since it is a required provider preset
|
||||
assert.ok(providers.includes('anthropic'),
|
||||
'KNOWN_PROVIDERS must include "anthropic"');
|
||||
// 'generic' is a special fallback, not a real provider — it must NOT be in KNOWN_PROVIDERS
|
||||
// (KNOWN_PROVIDERS lists only providers with catalog entries)
|
||||
assert.ok(!providers.includes('generic'),
|
||||
'KNOWN_PROVIDERS must not include "generic" (it is not a catalog-backed provider)');
|
||||
});
|
||||
|
||||
test('KNOWN_PROVIDERS exported from model-catalog.cjs matches core.cjs re-export', () => {
|
||||
// model-catalog.cjs must also export KNOWN_PROVIDERS (the canonical source).
|
||||
// core.cjs re-exports it. Both must be identical.
|
||||
assert.ok(modelCatalog.KNOWN_PROVIDERS != null,
|
||||
'KNOWN_PROVIDERS must be exported from model-catalog.cjs');
|
||||
const fromCatalog = [...modelCatalog.KNOWN_PROVIDERS].sort();
|
||||
const fromCore = [...KNOWN_PROVIDERS].sort();
|
||||
assert.deepStrictEqual(fromCore, fromCatalog,
|
||||
'KNOWN_PROVIDERS from core.cjs (re-export) must match model-catalog.cjs canonical export');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user