Files
msd-core/src/model-catalog.cts
Tom Boucher 58d73dd220 enhance(#3241): omit the codex per-agent model by default (#3276)
* test(#3241): failing-first suite for the codex passive model posture

Locks ADR-2313's D1-D5 before any production code exists, so the tests
bind to the behavior rather than to whatever the implementation happens
to do.

Red-first (fail against the current tree):
  - the resolver path emits no `model` and no `model_reasoning_effort`
  - a whitespace-only model_overrides value yields no pin
  - isAnthropicFlavoredModel / CLAUDE_AGENT_ALIASES on model-catalog
  - the one-time install notice, and its once-per-install dedupe

Regression guards (pass today, must keep passing): resolver-null via
`inherit` and via absent runtime; a resolver that resolves to nothing;
empty-string and non-string overrides; and the light-tier
service_tier/model_verbosity fields, which are NOT coupled to the model
pin and would silently regress if the implementation coupled them.

Classifying each test as red-first or regression guard is deliberate.
A test that passes on both sides of the change proves nothing, and this
epic has already shipped two such rows before catching them.

The whitespace case is a live defect, not a quirk: `'   '` is truthy,
survives the type guard, is not Anthropic-flavored, and is embedded
verbatim as `model = "   "` — the same class the #2310 guard exists to
stop. Same function, same path, fixed in this phase per CLAUDE.md §3.

Two matrix rows were dropped as vacuous rather than shipped green: a
64-char truncation case (the pinned notice interpolates no
user-controlled value, so it cannot exhibit truncation) and a newline
hazard that the input surface cannot reach.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(#3241): omit the codex per-agent model by default

Implements ADR-2313 D1-D5. generateCodexAgentToml no longer embeds the
runtime resolver's per-tier Codex model, so an agent inherits the
always-available session model instead of a pin a ChatGPT-account Codex
may not expose. model_reasoning_effort disappears with it via the
existing hasPinnedModel coupling (#838) — no logic change needed there.

Supersedes #2517's embedding on the default path only. An explicit
real-Codex model_overrides pin is still embedded verbatim, and the #2310
Anthropic-flavored guard is retained: the model_overrides route to it is
still live even though the resolver route is now unreachable.

The shared rule moves down a layer. CLAUDE_AGENT_ALIASES leaves
model-resolver for model-catalog — a genuine leaf importing only
node:path and its own JSON — with isAnthropicFlavoredModel defined beside
it, and is re-exported from model-resolver so every existing importer is
untouched. This is what lets Phase 2's install-check and Phase 3's sync
consume the rule without taking the config-loader dependency
model-resolver would have dragged into a module documented as pure
read/verify with 33 dependents. A parity test fails if the two ever fork.

Also fixes a live defect surfaced while writing the tests: a
whitespace-only model_overrides value was truthy, survived the type
guard, was not Anthropic-flavored, and so was embedded verbatim as
`model = "   "` — the same class the #2310 guard exists to stop, reached
by a different route. Trimmed before the truthiness test. It is
deliberately not routed through _warnCodexModelOverrideDropped, whose
text would misdescribe a blank field as a mis-typed model.

Adds the one-time install notice (maintainer direction, recorded as an
ADR-2313 amendment): one stderr line naming model_overrides and the
session model, deduped per install rather than per agent, and emitted
only for the population that actually loses a pin.

service_tier and model_verbosity stay decoupled from the model (#774);
a regression guard asserts they still emit with nothing pinned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(#3241): amend ADR-2313, add the model-catalog glossary entry

ADR-2313 gains two dated amendments rather than edits to its merged
text, since ADRs here are append-only.

The first records that a deprecation notice IS offered, reversing the
Migration section's "no deprecation window" position, and states why
that position was wrong rather than just superseding it: the ADR
identified the API-key population as losing something real and then
declined to warn it, in the same document. Hyrum's guidance was applied
to the recourse and not to the notice.

The second records the whitespace-only model_overrides defect and notes
that D2 always implied the fix — the implementation simply never
enforced it and no test covered the case.

CONTEXT.md gains a Model Catalog Module entry. The module had none,
which is why the glossary gate passed without one: check-glossary-refs
verifies that references resolve, not that modules are documented. The
entry records why the Anthropic-flavored rule lives there rather than in
model-resolver, so a later reader does not "helpfully" move it back. The
Model Resolver entry is updated to point at its new home and note the
back-compat re-export.

docs/CONFIGURATION.md carried a claim that is now false: that the
resolved tier ID is embedded into agent frontmatter at install time on
codex and opencode. Corrected to name codex as the exception, with the
400 symptom and the model_overrides recourse.

Changeset leads with the user-visible change and the migration line
rather than the implementation, per the ADR's Hyrum's-Law analysis.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3241): only notice a lost pin when one was actually embeddable

Review finding from an isolated reviewer. The deprecation notice gated
on whether the runtime resolver would have returned *any* model, but the
question that matters is whether that model would have been *embedded*.

Those differ. The #2310 safety gate already rejected an Anthropic-
flavored model arriving from the resolver path before Phase 1 — so for a
mixed-runtime config resolving to a claude-* id against a Codex install
target, the user never had that pin. The notice told them they lost
something they never got, and pointed them at model_overrides for no
reason.

The existing #2310 test drives exactly that path but asserts only the
emitted `model` line, never stderr, which is why it slipped through. Now
covered.

Deliberately unchanged: an Anthropic-flavored model_overrides value plus
a legal resolver model fires BOTH the override warning and the notice.
That is correct — pre-Phase-1 the guard dropped the override, execution
fell through to the resolver, and the resolver's model was embedded, so
that user did lose a pin. Two messages, two distinct true facts, and the
prefixes differ (`gsd: warning — ` vs `gsd: notice — `) so the
one-notice-per-install contract holds. A regression test now pins that
behavior so it does not get "simplified" away.

Of the three tests added, only the first is red-first; the other two
pass on both sides by design and are labelled as guards — one against
over-correcting the fix into silence, one against removing the
intentional double message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3241): reset the notice dedupe via a seam, not a require.cache bust

The remote runner caught a regression I introduced: the #2760
post-write-validation test began failing with the validator override no
longer intercepting.

Cause, confirmed by trace rather than guessed: the new #3241 review
tests deleted require.cache for bin/install.js and re-required it mid
suite, to clear the notice's module-level dedupe flag. But
runCodexInstall destructures `install` at file load, closing over the
ORIGINAL module's exports. After the cache bust a second instance
existed, so the test's `installModule.__codexSchemaValidator = ...`
mutated the new object while the code under test still called the old
one. The override silently stopped intercepting, the real validator ran
and passed on GSD-emitted output, and the abort-and-restore path was
never exercised.

Cache-busting a module mid-suite breaks every later test that assumes a
single instance, which every other test in the file is entitled to. So
the fix is a seam, not a workaround: bin/install.js exports
_resetCodexNoticeDedupeForTests(), and the three tests call it directly
instead of reloading the module.

The flag is module-level by design — the dedupe is per-install and
install() already resets it — so a unit test driving
generateCodexAgentToml directly needs an explicit way to reset it. That
is now what it has.

Swept the rest of the #3241 diff for the same hazard; this flag was the
only shared module-level state introduced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3241): reset both codex dedupe stores, not just the notice flag

Second incomplete fix, same class one layer down. bin/install.js keeps
TWO module-level dedupe stores and the require.cache bust I removed had
been papering over both; my replacement seam cleared only one.

_codexModelOverrideDroppedWarned is a Set keyed `${agent}::${value}`.
tests/codex-config.test.cjs:558 already emits for `gsd-executor::sonnet`,
so by the time the review test using the same agent and value ran,
_warnCodexModelOverrideDropped was a silent no-op and the expected
warning never appeared.

The seam now clears both stores and is renamed to say so. Its comment
records that per-install dedupe lives in module scope deliberately and
that this is the single sanctioned way for a unit test to clear it.

Swept bin/install.js for every other module-scope mutable a test could
latch. Two are inert (capability registries assigned once at require
time; selectedRuntimes computed once from argv). One is a genuine latent
hazard and is deliberately NOT folded in: attributionCache (:1654)
memoizes getCommitAttribution by runtime name for process lifetime, so
two in-process installs of one runtime with differing attribution config
would collide. It is unreachable from any current test and is a
different concern from Codex warning dedupe, so it stays out of this PR
rather than widening it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(#3241): correct the codex tier-routing how-to

The docs gate forced the task-oriented quadrant and found the worst
defect in this change's documentation surface.

docs/how-to/configure-model-profiles.md carried a section titled "If you
want tiered models on Codex" telling users to set runtime:codex +
model_profile:balanced, promising "GSD resolves each tier alias to the
Codex-native model and reasoning effort defined in the runtime tier
map." That is exactly the behavior this PR removes — a how-to page
confidently instructing users to do something that no longer works,
which is worse than a missing page because it fails at the moment of
use.

Rewritten to state that Codex does no tier routing, give the
model_overrides pin as the supported alternative, and name the two
constraints on what may be pinned: it must be a real Codex model id, and
the account must actually expose it — GSD cannot verify the second, so
the honest advice when unsure is to omit the pin. Carries an upgrade
note for both account types, since the change is a no-op for ChatGPT
accounts and a real loss for API-key ones.

Also tightened the same page's claim that Codex "embeds the resolved
model" at install time — now true only of an explicit override. The
re-install instruction it supports is still correct and still needed, so
only the premise moved.

Both the required-docs set (COMMANDS.md + FEATURES.md) and
lint-docs-required.cjs would have passed before this commit, since
CONFIGURATION.md and the ADR had already moved. Neither checks the
quadrant a user in trouble actually opens.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#3241): backfill changeset pr number (#3276)

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 19:16:04 -04:00

341 lines
14 KiB
TypeScript

/**
* Model catalog — typed access to model-catalog.json.
*
* ADR-457 build-at-publish: the hand-written bin/lib/model-catalog.cjs
* collapsed to a TypeScript source of truth. Behaviour is preserved
* byte-for-behaviour from the prior hand-written .cjs; only types are added.
*/
import path from 'node:path';
// In .cts (CommonJS output) files, `require` is available as a global;
// we use it directly to load JSON candidates.
const _require: NodeRequire = require;
// Resolve model-catalog.json via a prioritised candidate list so the module
// works in every layout:
//
// 1. Co-located install path — gsd-core/bin/shared/model-catalog.json
// 2. GSD_MODEL_CATALOG env override
//
// A third candidate — `sdk/shared/model-catalog.json`, three levels up — used to
// sit between them. It was the legacy source-repo path kept as a fallback by the
// #3288 fix, whose contract was "check the co-located path FIRST, before the
// legacy source-repo path". ADR-0174 then retired the `@opengsd/gsd-sdk` package
// boundary and deleted the `sdk/` tree outright, so that candidate can no longer
// resolve in any layout: in a source repo there is no `sdk/`, and in an install
// layout it points at `~/.claude/sdk/shared/`, which the installer never writes
// (the original #3288 bug). It is removed rather than left as dead weight that
// implies a package boundary this repo no longer has.
const _catalogCandidates: string[] = [
path.resolve(__dirname, '..', 'shared', 'model-catalog.json'),
...(process.env['GSD_MODEL_CATALOG'] ? [path.resolve(process.env['GSD_MODEL_CATALOG'])] : []),
];
/** Typed tier entry from model-catalog.json (model + optional reasoning_effort). */
export interface TierEntry {
model: string;
reasoning_effort?: string;
}
/** Per-agent model mapping in the catalog. */
export interface AgentMeta {
golden: string;
balanced: string;
budget: string;
phaseType: string;
routingTier: string;
}
/** The shape of model-catalog.json. */
export interface ModelCatalog {
profiles: string[];
phaseTypes: string[];
adaptiveTierMap: Record<string, string>;
runtimeTierDefaults: Record<string, Record<string, TierEntry | null>>;
providerPresets: Record<string, Record<string, Record<string, TierEntry | null>>>;
agents: Record<string, AgentMeta>;
}
let catalog: ModelCatalog | null = null;
let _catalogLastErr: Error | null = null;
for (const _p of _catalogCandidates) {
try {
catalog = _require(_p) as ModelCatalog;
break;
} catch (e) {
const isMissingCandidate =
(e && (e as NodeJS.ErrnoException).code === 'MODULE_NOT_FOUND' && String((e as Error).message || '').includes(_p)) ||
(e && (e as NodeJS.ErrnoException).code === 'ENOENT');
if (!isMissingCandidate) throw e;
_catalogLastErr = e as Error;
}
}
if (!catalog) {
throw new Error(
`model-catalog.json not found. Tried:\n${_catalogCandidates.map((p) => ` ${p}`).join('\n')}\nLast error: ${_catalogLastErr?.message}`
);
}
// After the throw guard above, catalog is guaranteed non-null.
const _catalog = catalog;
export { _catalog as catalog };
export const VALID_PROFILES: string[] = [..._catalog.profiles];
export const VALID_PHASE_TYPES: Set<string> = new Set(_catalog.phaseTypes);
export const VALID_AGENT_TIERS: Set<string> = new Set(Object.keys(_catalog.adaptiveTierMap));
// Catalog-derived so this can never drift from the resolver's tier gate:
// Object.values(adaptiveTierMap) === ['opus', 'sonnet', 'haiku'] today, plus 'inherit'.
export const VALID_TIERS: Set<string> = new Set([...Object.values(_catalog.adaptiveTierMap), 'inherit']);
// Same catalog-derived tier values as VALID_TIERS but WITHOUT 'inherit' — used
// by config-loader's runtime-override validation (model_profile_overrides /
// model_policy.runtime_tiers), which does not accept 'inherit' as a tier.
export const ADAPTIVE_TIER_VALUES: Set<string> = new Set(Object.values(_catalog.adaptiveTierMap));
/** Per-profile model slots for each agent. */
export interface AgentModelProfiles {
quality: string;
balanced: string;
budget: string;
adaptive: string;
}
export const MODEL_PROFILES: Record<string, AgentModelProfiles> = Object.fromEntries(
Object.entries(_catalog.agents).map(([agent, meta]) => [agent, {
quality: meta.golden,
balanced: meta.balanced,
budget: meta.budget,
adaptive: _catalog.adaptiveTierMap[meta.routingTier],
}])
);
export const AGENT_TO_PHASE_TYPE: Record<string, string> = Object.fromEntries(
Object.entries(_catalog.agents).map(([agent, meta]) => [agent, meta.phaseType])
);
export const AGENT_DEFAULT_TIERS: Record<string, string> = Object.fromEntries(
Object.entries(_catalog.agents).map(([agent, meta]) => [agent, meta.routingTier])
);
export const MODEL_ALIAS_MAP: Record<string, string | undefined> = Object.fromEntries(
Object.entries(_catalog.runtimeTierDefaults['claude'] ?? {}).map(([tier, entry]) => [tier, entry?.model])
);
export const RUNTIME_PROFILE_MAP: Record<string, Record<string, TierEntry>> = (() => {
const result: Record<string, Record<string, TierEntry>> = {};
for (const [runtime, tiers] of Object.entries(_catalog.runtimeTierDefaults)) {
const filtered: Record<string, TierEntry> = {};
for (const [tier, entry] of Object.entries(tiers)) {
if (entry) filtered[tier] = entry;
}
if (Object.keys(filtered).length > 0) result[runtime] = filtered;
}
return result;
})();
export const KNOWN_RUNTIMES: Set<string> = new Set(Object.keys(_catalog.runtimeTierDefaults));
export const RUNTIMES_WITH_REASONING_EFFORT: Set<string> = new Set(
Object.entries(_catalog.runtimeTierDefaults)
.filter(([, tiers]) => Object.values(tiers).some((entry) => entry && entry.reasoning_effort))
.map(([runtime]) => runtime)
);
export const PROVIDER_PRESETS: Record<string, Record<string, Record<string, TierEntry | null>>> =
_catalog.providerPresets ?? {};
// KNOWN_PROVIDERS excludes 'generic' — it is a sentinel (all null entries) that
// forces users to supply model IDs via model_profile_overrides. It is not a
// real catalog-backed provider (#49).
export const KNOWN_PROVIDERS: Set<string> = new Set(
Object.entries(PROVIDER_PRESETS)
.filter(([, tiers]) =>
Object.values(tiers).some((budgets) =>
budgets && Object.values(budgets).some((entry) => entry && entry.model)
)
)
.map(([name]) => name)
);
// ─── #3241 — Anthropic-flavored model detection ──────────────────────────────
//
// Moved here from src/model-resolver.cts (the "seam decision" in
// .gsd/phase/feat-3241-codex-omit-model-by-default/40-design.md): this leaf
// module is the one common dependency both model-resolver and the (layering-
// restricted) install-time Codex-posture checks can share without pulling
// model-resolver's config-loader dependency chain into a "pure read/verify"
// caller. model-resolver re-exports both names for back-compat.
//
// #2310 — True if `model` is an Anthropic-flavored value that must never appear as a
// Codex agent `.toml` `model`. Two forms: (a) a bare Claude Agent-tool tier alias
// (opus/sonnet/haiku/fable — CLAUDE_AGENT_ALIASES below); (b) any Claude model id in
// any provider namespacing — `claude-*`, `anthropic/claude-*`, `us.anthropic.claude-*`
// (the forms the catalog assigns to opencode/hermes/kilo, reachable on a Codex .toml
// via the runtime-resolver path). No OpenAI/Codex model id contains "claude", so a
// case-insensitive substring test is a safe, exhaustive guard for (b). Codex/ChatGPT
// rejects all of these.
export const CLAUDE_AGENT_ALIASES: Set<string> = new Set(['opus', 'sonnet', 'haiku', 'fable']);
export function isAnthropicFlavoredModel(model: unknown): boolean {
return typeof model === 'string' && (CLAUDE_AGENT_ALIASES.has(model) || model.toLowerCase().includes('claude'));
}
export function nextTier(currentTier: string): string | null {
const order = ['light', 'standard', 'heavy'];
const idx = order.indexOf(String(currentTier));
if (idx === -1) return null;
return order[Math.min(idx + 1, order.length - 1)];
}
export function formatAgentToModelMapAsTable(agentToModelMap: Record<string, string>): string {
const agentWidth = Math.max('Agent'.length, ...Object.keys(agentToModelMap).map((a) => a.length));
const modelWidth = Math.max('Model'.length, ...Object.values(agentToModelMap).map((m) => m.length));
const sep = '─'.repeat(agentWidth + 2) + '┼' + '─'.repeat(modelWidth + 2);
const header = ` ${'Agent'.padEnd(agentWidth)} │ ${'Model'.padEnd(modelWidth)}`;
let out = `${header}\n${sep}\n`;
for (const [agent, model] of Object.entries(agentToModelMap)) {
out += ` ${agent.padEnd(agentWidth)} │ ${model.padEnd(modelWidth)}\n`;
}
return out;
}
export function getAgentToModelMapForProfile(normalizedProfile: string): Record<string, string> {
const profile = VALID_PROFILES.includes(normalizedProfile) ? normalizedProfile : 'balanced';
const out: Record<string, string> = {};
for (const [agent, profiles] of Object.entries(MODEL_PROFILES)) {
const profilesRec = profiles as unknown as Record<string, string>;
out[agent] = profile === 'inherit' ? 'inherit' : (profilesRec[profile] ?? profiles.balanced);
}
return out;
}
// ─── Effort rendering ────────────────────────────────────────────────────────
export interface EffortSpec {
param: string;
channel: string;
supported: Set<string>;
clamp(level: string): string;
}
export const EFFORT_RENDERING: Record<string, EffortSpec> = {
claude: {
param: 'output_config.effort',
channel: 'frontmatter',
supported: new Set(['low', 'medium', 'high', 'xhigh', 'max']),
clamp(level: string): string {
if (level === 'minimal') return 'low';
return level;
},
},
codex: {
param: 'model_reasoning_effort',
channel: 'api',
supported: new Set(['minimal', 'low', 'medium', 'high', 'xhigh']),
clamp(level: string): string {
if (level === 'max') return 'xhigh';
return level;
},
},
};
export interface RenderedEffort {
value: string;
param: string | null;
channel: string | null;
}
// ─── Invocation-time (argv) effort rendering ─────────────────────────────────
//
// ADR-1239 amendment (#2481) / ADR-443 path (a). EFFORT_RENDERING above covers the
// two INSTALL-TIME channels (`frontmatter`, `api`) — effort baked into a generated
// artifact. This table covers the INVOCATION-TIME channel: the argument appended to
// a host CLI spawned as a subprocess.
//
// WHETHER to emit is not decided here — it is the negotiated `effortSurface` axis on
// the host's descriptor (`argv` | `none`). This table only knows the
// syntax for hosts whose surface is `argv`. A host absent from this table renders
// null, so an undeclared or undocumented host silently gets nothing rather than a
// guessed flag.
export interface EffortArgvSpec {
/** Render the argv fragment for an already-clamped effort level. */
render(level: string): string[];
/** Levels this CLI accepts; anything else clamps via `clamp` first. */
supported: Set<string>;
clamp(level: string): string;
}
export const EFFORT_ARGV: Record<string, EffortArgvSpec> = {
// Verified against `claude --help`: `--effort <level>`.
claude: {
render: (level: string): string[] => ['--effort', level],
supported: new Set(['low', 'medium', 'high', 'xhigh', 'max']),
clamp: (level: string): string => (level === 'minimal' ? 'low' : level),
},
// Verified against `opencode run --help`: `--variant` — "model variant
// (provider-specific reasoning effort, e.g., high, max, minimal)".
opencode: {
render: (level: string): string[] => ['--variant', level],
supported: new Set(['minimal', 'low', 'medium', 'high', 'xhigh', 'max']),
clamp: (level: string): string => level,
},
// First-party Codex docs: `model_reasoning_effort` is a config-only key with no
// dedicated flag, so the generic `-c key=value` override is the only argv route.
codex: {
render: (level: string): string[] => ['-c', `model_reasoning_effort=${level}`],
supported: new Set(['minimal', 'low', 'medium', 'high', 'xhigh']),
clamp: (level: string): string => (level === 'max' ? 'xhigh' : level),
},
};
export interface RenderedEffortArgv {
argv: string[];
value: string | null;
host: string;
}
/**
* Render the invocation-time effort argument for a host.
*
* `effortSurface` is the host's negotiated axis value. Only `argv` produces an
* argument; `none`, `undocumented`, and anything unrecognised produce nothing.
* Never throws.
*/
export function renderEffortArgv(
host: string,
universalEffort: string,
effortSurface: string | null | undefined,
): RenderedEffortArgv {
const empty: RenderedEffortArgv = { argv: [], value: null, host };
if (effortSurface !== 'argv') return empty;
// Own-property lookup only. A plain `EFFORT_ARGV[host]` resolves `__proto__`
// (and `constructor`/`toString`) to inherited members, which are truthy but
// carry no `clamp`/`render` — a hostile host id would throw instead of
// degrading. The host id reaches here from a descriptor, i.e. untrusted JSON.
if (typeof host !== 'string' || !Object.prototype.hasOwnProperty.call(EFFORT_ARGV, host)) return empty;
const spec = EFFORT_ARGV[host];
if (!spec || typeof spec.clamp !== 'function' || typeof spec.render !== 'function') return empty;
if (typeof universalEffort !== 'string' || universalEffort.length === 0) return empty;
const clamped = spec.clamp(universalEffort);
if (!spec.supported.has(clamped)) return empty;
return { argv: spec.render(clamped), value: clamped, host };
}
/**
* Render a universal effort string for a specific runtime.
*/
export function renderEffortForRuntime(runtime: string, universalEffort: string): RenderedEffort {
const spec = EFFORT_RENDERING[runtime];
if (!spec) {
return { value: universalEffort, param: null, channel: null };
}
return {
value: spec.clamp(universalEffort),
param: spec.param,
channel: spec.channel,
};
}
// ─── Fast mode propagation ───────────────────────────────────────────────────
export const RUNTIMES_WITH_FAST_MODE: Set<string> = new Set(['api']);