enhance(#3241): omit the codex per-agent model by default (#3276)

* test(#3241): failing-first suite for the codex passive model posture

Locks ADR-2313's D1-D5 before any production code exists, so the tests
bind to the behavior rather than to whatever the implementation happens
to do.

Red-first (fail against the current tree):
  - the resolver path emits no `model` and no `model_reasoning_effort`
  - a whitespace-only model_overrides value yields no pin
  - isAnthropicFlavoredModel / CLAUDE_AGENT_ALIASES on model-catalog
  - the one-time install notice, and its once-per-install dedupe

Regression guards (pass today, must keep passing): resolver-null via
`inherit` and via absent runtime; a resolver that resolves to nothing;
empty-string and non-string overrides; and the light-tier
service_tier/model_verbosity fields, which are NOT coupled to the model
pin and would silently regress if the implementation coupled them.

Classifying each test as red-first or regression guard is deliberate.
A test that passes on both sides of the change proves nothing, and this
epic has already shipped two such rows before catching them.

The whitespace case is a live defect, not a quirk: `'   '` is truthy,
survives the type guard, is not Anthropic-flavored, and is embedded
verbatim as `model = "   "` — the same class the #2310 guard exists to
stop. Same function, same path, fixed in this phase per CLAUDE.md §3.

Two matrix rows were dropped as vacuous rather than shipped green: a
64-char truncation case (the pinned notice interpolates no
user-controlled value, so it cannot exhibit truncation) and a newline
hazard that the input surface cannot reach.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(#3241): omit the codex per-agent model by default

Implements ADR-2313 D1-D5. generateCodexAgentToml no longer embeds the
runtime resolver's per-tier Codex model, so an agent inherits the
always-available session model instead of a pin a ChatGPT-account Codex
may not expose. model_reasoning_effort disappears with it via the
existing hasPinnedModel coupling (#838) — no logic change needed there.

Supersedes #2517's embedding on the default path only. An explicit
real-Codex model_overrides pin is still embedded verbatim, and the #2310
Anthropic-flavored guard is retained: the model_overrides route to it is
still live even though the resolver route is now unreachable.

The shared rule moves down a layer. CLAUDE_AGENT_ALIASES leaves
model-resolver for model-catalog — a genuine leaf importing only
node:path and its own JSON — with isAnthropicFlavoredModel defined beside
it, and is re-exported from model-resolver so every existing importer is
untouched. This is what lets Phase 2's install-check and Phase 3's sync
consume the rule without taking the config-loader dependency
model-resolver would have dragged into a module documented as pure
read/verify with 33 dependents. A parity test fails if the two ever fork.

Also fixes a live defect surfaced while writing the tests: a
whitespace-only model_overrides value was truthy, survived the type
guard, was not Anthropic-flavored, and so was embedded verbatim as
`model = "   "` — the same class the #2310 guard exists to stop, reached
by a different route. Trimmed before the truthiness test. It is
deliberately not routed through _warnCodexModelOverrideDropped, whose
text would misdescribe a blank field as a mis-typed model.

Adds the one-time install notice (maintainer direction, recorded as an
ADR-2313 amendment): one stderr line naming model_overrides and the
session model, deduped per install rather than per agent, and emitted
only for the population that actually loses a pin.

service_tier and model_verbosity stay decoupled from the model (#774);
a regression guard asserts they still emit with nothing pinned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(#3241): amend ADR-2313, add the model-catalog glossary entry

ADR-2313 gains two dated amendments rather than edits to its merged
text, since ADRs here are append-only.

The first records that a deprecation notice IS offered, reversing the
Migration section's "no deprecation window" position, and states why
that position was wrong rather than just superseding it: the ADR
identified the API-key population as losing something real and then
declined to warn it, in the same document. Hyrum's guidance was applied
to the recourse and not to the notice.

The second records the whitespace-only model_overrides defect and notes
that D2 always implied the fix — the implementation simply never
enforced it and no test covered the case.

CONTEXT.md gains a Model Catalog Module entry. The module had none,
which is why the glossary gate passed without one: check-glossary-refs
verifies that references resolve, not that modules are documented. The
entry records why the Anthropic-flavored rule lives there rather than in
model-resolver, so a later reader does not "helpfully" move it back. The
Model Resolver entry is updated to point at its new home and note the
back-compat re-export.

docs/CONFIGURATION.md carried a claim that is now false: that the
resolved tier ID is embedded into agent frontmatter at install time on
codex and opencode. Corrected to name codex as the exception, with the
400 symptom and the model_overrides recourse.

Changeset leads with the user-visible change and the migration line
rather than the implementation, per the ADR's Hyrum's-Law analysis.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3241): only notice a lost pin when one was actually embeddable

Review finding from an isolated reviewer. The deprecation notice gated
on whether the runtime resolver would have returned *any* model, but the
question that matters is whether that model would have been *embedded*.

Those differ. The #2310 safety gate already rejected an Anthropic-
flavored model arriving from the resolver path before Phase 1 — so for a
mixed-runtime config resolving to a claude-* id against a Codex install
target, the user never had that pin. The notice told them they lost
something they never got, and pointed them at model_overrides for no
reason.

The existing #2310 test drives exactly that path but asserts only the
emitted `model` line, never stderr, which is why it slipped through. Now
covered.

Deliberately unchanged: an Anthropic-flavored model_overrides value plus
a legal resolver model fires BOTH the override warning and the notice.
That is correct — pre-Phase-1 the guard dropped the override, execution
fell through to the resolver, and the resolver's model was embedded, so
that user did lose a pin. Two messages, two distinct true facts, and the
prefixes differ (`gsd: warning — ` vs `gsd: notice — `) so the
one-notice-per-install contract holds. A regression test now pins that
behavior so it does not get "simplified" away.

Of the three tests added, only the first is red-first; the other two
pass on both sides by design and are labelled as guards — one against
over-correcting the fix into silence, one against removing the
intentional double message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3241): reset the notice dedupe via a seam, not a require.cache bust

The remote runner caught a regression I introduced: the #2760
post-write-validation test began failing with the validator override no
longer intercepting.

Cause, confirmed by trace rather than guessed: the new #3241 review
tests deleted require.cache for bin/install.js and re-required it mid
suite, to clear the notice's module-level dedupe flag. But
runCodexInstall destructures `install` at file load, closing over the
ORIGINAL module's exports. After the cache bust a second instance
existed, so the test's `installModule.__codexSchemaValidator = ...`
mutated the new object while the code under test still called the old
one. The override silently stopped intercepting, the real validator ran
and passed on GSD-emitted output, and the abort-and-restore path was
never exercised.

Cache-busting a module mid-suite breaks every later test that assumes a
single instance, which every other test in the file is entitled to. So
the fix is a seam, not a workaround: bin/install.js exports
_resetCodexNoticeDedupeForTests(), and the three tests call it directly
instead of reloading the module.

The flag is module-level by design — the dedupe is per-install and
install() already resets it — so a unit test driving
generateCodexAgentToml directly needs an explicit way to reset it. That
is now what it has.

Swept the rest of the #3241 diff for the same hazard; this flag was the
only shared module-level state introduced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3241): reset both codex dedupe stores, not just the notice flag

Second incomplete fix, same class one layer down. bin/install.js keeps
TWO module-level dedupe stores and the require.cache bust I removed had
been papering over both; my replacement seam cleared only one.

_codexModelOverrideDroppedWarned is a Set keyed `${agent}::${value}`.
tests/codex-config.test.cjs:558 already emits for `gsd-executor::sonnet`,
so by the time the review test using the same agent and value ran,
_warnCodexModelOverrideDropped was a silent no-op and the expected
warning never appeared.

The seam now clears both stores and is renamed to say so. Its comment
records that per-install dedupe lives in module scope deliberately and
that this is the single sanctioned way for a unit test to clear it.

Swept bin/install.js for every other module-scope mutable a test could
latch. Two are inert (capability registries assigned once at require
time; selectedRuntimes computed once from argv). One is a genuine latent
hazard and is deliberately NOT folded in: attributionCache (:1654)
memoizes getCommitAttribution by runtime name for process lifetime, so
two in-process installs of one runtime with differing attribution config
would collide. It is unreachable from any current test and is a
different concern from Codex warning dedupe, so it stays out of this PR
rather than widening it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(#3241): correct the codex tier-routing how-to

The docs gate forced the task-oriented quadrant and found the worst
defect in this change's documentation surface.

docs/how-to/configure-model-profiles.md carried a section titled "If you
want tiered models on Codex" telling users to set runtime:codex +
model_profile:balanced, promising "GSD resolves each tier alias to the
Codex-native model and reasoning effort defined in the runtime tier
map." That is exactly the behavior this PR removes — a how-to page
confidently instructing users to do something that no longer works,
which is worse than a missing page because it fails at the moment of
use.

Rewritten to state that Codex does no tier routing, give the
model_overrides pin as the supported alternative, and name the two
constraints on what may be pinned: it must be a real Codex model id, and
the account must actually expose it — GSD cannot verify the second, so
the honest advice when unsure is to omit the pin. Carries an upgrade
note for both account types, since the change is a no-op for ChatGPT
accounts and a real loss for API-key ones.

Also tightened the same page's claim that Codex "embeds the resolved
model" at install time — now true only of an explicit override. The
re-install instruction it supports is still correct and still needed, so
only the premise moved.

Both the required-docs set (COMMANDS.md + FEATURES.md) and
lint-docs-required.cjs would have passed before this commit, since
CONFIGURATION.md and the ADR had already moved. Neither checks the
quadrant a user in trouble actually opens.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#3241): backfill changeset pr number (#3276)

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-08-09 19:16:04 -04:00
committed by GitHub
parent 4b69dc346b
commit 58d73dd220
11 changed files with 679 additions and 50 deletions

View File

@@ -468,10 +468,10 @@ const _gsdLibDir = path.join(__dirname, '..', 'gsd-core', 'bin', 'lib');
const { MODEL_PROFILES: GSD_MODEL_PROFILES } = require(path.join(_gsdLibDir, 'model-profiles.cjs'));
const {
RUNTIME_PROFILE_MAP: GSD_RUNTIME_PROFILE_MAP,
isAnthropicFlavoredModel: gsdIsAnthropicFlavoredModel,
} = require(path.join(_gsdLibDir, 'model-catalog.cjs'));
const {
resolveTierEntry: gsdResolveTierEntry,
CLAUDE_AGENT_ALIASES,
} = require(path.join(_gsdLibDir, 'model-resolver.cjs'));
// #2071 — install-time effort resolution (readGsdEffectiveEffortConfig /
@@ -4074,15 +4074,19 @@ purpose: ${toSingleLine(description)}
/**
* #2310 — True if `model` is an Anthropic-flavored value that must never appear as a
* Codex agent `.toml` `model`. Two forms: (a) a bare Claude Agent-tool tier alias
* (opus/sonnet/haiku/fable — the canonical CLAUDE_AGENT_ALIASES, imported from
* src/model-resolver.cts so it can't diverge); (b) any Claude model id in any provider
* namespacing — `claude-*`, `anthropic/claude-*`, `us.anthropic.claude-*` (the forms the
* catalog assigns to opencode/hermes/kilo, reachable on a Codex .toml via the runtime-
* resolver path). No OpenAI/Codex model id contains "claude", so a case-insensitive
* substring test is a safe, exhaustive guard for (b). Codex/ChatGPT rejects all of these.
* (opus/sonnet/haiku/fable — the canonical CLAUDE_AGENT_ALIASES); (b) any Claude model
* id in any provider namespacing — `claude-*`, `anthropic/claude-*`, `us.anthropic.claude-*`
* (the forms the catalog assigns to opencode/hermes/kilo, reachable on a Codex .toml via
* the runtime-resolver path). No OpenAI/Codex model id contains "claude", so a
* case-insensitive substring test is a safe, exhaustive guard for (b). Codex/ChatGPT
* rejects all of these.
*
* #3241 — thin delegation to the shared predicate on src/model-catalog.cts (moved there
* so it can't diverge across Codex-posture surfaces); kept as a local name because it
* reads better at the call sites below.
*/
function _isAnthropicFlavoredModel(model) {
return typeof model === 'string' && (CLAUDE_AGENT_ALIASES.has(model) || model.toLowerCase().includes('claude'));
return gsdIsAnthropicFlavoredModel(model);
}
// #2310 — dedupe stderr warnings so repeated agent emits don't spam (mirrors the
@@ -4101,6 +4105,42 @@ function _warnCodexModelOverrideDropped(agentName, value) {
);
}
// #3241 — one-time per-install deprecation notice: the automatic runtime-resolver
// per-tier Codex model embed was removed (D1/D5, ADR-2313 passive-posture epic). When
// the resolver *would have* supplied a model and nothing else ends up pinned, this
// notice points the user at model_overrides as the explicit-pin replacement. Dedupes
// with a module-level boolean (mirrors _codexModelOverrideDroppedWarned's Set above)
// so a multi-agent install — every Codex agent hits this condition simultaneously —
// emits exactly one line, not one per agent. Reset once per install() call (see
// install()) so the "at most once" window is per-install, not per-process. That
// reset lives ONLY inside install() (~:10116) — generateCodexAgentToml is also
// exported standalone (~:13460), and a caller invoking it directly/repeatedly
// outside install() gets process-lifetime dedupe instead of per-install. No
// current test depends on the standalone caller's dedupe window.
let _codexResolverModelOmittedWarned = false;
function _warnCodexResolverModelOmitted() {
if (_codexResolverModelOmittedWarned) return;
_codexResolverModelOmittedWarned = true;
process.stderr.write(
'gsd: notice — Codex agents no longer auto-pin a per-tier model from the runtime ' +
'resolver; set model_overrides for an agent if you want a specific Codex model ' +
'instead of the session model.\n',
);
}
// Test seam only — bin/install.js deliberately keeps per-install warning/notice
// dedupe in module scope (both the _codexModelOverrideDroppedWarned Set above and
// the _codexResolverModelOmittedWarned boolean; install() resets the latter at
// ~:10116). A unit test that drives generateCodexAgentToml() directly, without
// going through install(), has no other way to reset either store between
// assertions without busting the require.cache (which breaks module-instance
// sharing with the rest of the suite). This is the single sanctioned way for a
// unit test to clear both dedupe stores — exported so tests can call it instead.
function _resetCodexWarningDedupeForTests() {
_codexModelOverrideDroppedWarned.clear();
_codexResolverModelOmittedWarned = false;
}
/**
* Generate a per-agent .toml config file for Codex.
* Sets required agent metadata, sandbox_mode, and developer_instructions
@@ -4133,38 +4173,58 @@ function generateCodexAgentToml(agentName, agentContent, modelOverrides = null,
// Embed model override when configured in ~/.gsd/defaults.json so that
// model_overrides is respected on Codex (which uses static TOML, not inline
// Task() model parameters). See #2256.
// Precedence: per-agent model_overrides > runtime-aware tier resolution (#2517).
// #2310 — a Codex .toml `model` MUST be a real Codex/OpenAI model id. Codex is a
// passive/session-only model host (ADR-1239): GSD cannot reliably route per-agent
// tiers, and a bare GSD/Claude tier alias (opus/sonnet/haiku/fable) or a claude-*
// id 400s on a ChatGPT-account Codex ("The 'sonnet' model is not supported when
// using Codex with a ChatGPT account"). So: embed ONLY an explicit real-Codex
// model pin from model_overrides; omit anything Anthropic-flavored so the agent
// inherits the always-available session model. (Removing the runtime-resolver
// per-tier embedding below is the ADR-2310 passive-posture epic.)
// inherits the always-available session model.
// #3241 (D1/D5) — the runtime-aware tier-resolver auto-embed that used to fall
// through here when model_overrides had nothing was removed: Codex is passive by
// default now, and only an explicit model_overrides pin survives. See the
// deprecation-notice block below for the population that used to get a pin from
// the resolver and no longer does.
const rawModelOverride = modelOverrides?.[resolvedName] || modelOverrides?.[agentName];
let pinnedModel = null;
if (rawModelOverride) {
if (typeof rawModelOverride === 'string' && rawModelOverride && !_isAnthropicFlavoredModel(rawModelOverride)) {
pinnedModel = rawModelOverride; // explicit real-Codex model pin → embed verbatim (#2256)
// Trim before the truthiness test (#3241 defect fix): a whitespace-only value
// (e.g. ' ') is a truthy JS string but not a model id — it must not be
// embedded verbatim (`model = " "`, a live pre-fix defect) or routed to
// _warnCodexModelOverrideDropped, whose "is not a valid Codex model
// (Anthropic alias/id)" text would misdescribe a blank config field. It is
// silently dropped, matching how '' already behaves (no pin, no warning).
const trimmedOverride = typeof rawModelOverride === 'string' ? rawModelOverride.trim() : rawModelOverride;
if (typeof trimmedOverride === 'string' && trimmedOverride && !_isAnthropicFlavoredModel(trimmedOverride)) {
pinnedModel = trimmedOverride; // explicit real-Codex model pin → embed verbatim (#2256)
} else if (typeof rawModelOverride === 'string' && trimmedOverride === '') {
// whitespace-only override — no pin, no warning (#3241).
} else {
_warnCodexModelOverrideDropped(resolvedName, rawModelOverride); // alias/claude-* → omit
}
}
if (!pinnedModel && runtimeResolver) {
// #2517 — runtime-aware tier resolution. Embeds Codex-native model + reasoning_effort
// from RUNTIME_PROFILE_MAP / model_profile_overrides for the configured tier.
// (Superseded on the default path by the ADR-2310 passive-posture epic.)
const entry = runtimeResolver.resolve(resolvedName) || runtimeResolver.resolve(agentName);
if (entry?.model) pinnedModel = entry.model;
}
// #2310 — final safety gate: never emit an Anthropic-flavored model into a Codex
// .toml, even from the runtime-resolver path (e.g. a defaults.json runtime that
// does not match the codex install target).
// .toml, even one that reached here through some other path than the override
// check above.
if (pinnedModel && _isAnthropicFlavoredModel(pinnedModel)) {
_warnCodexModelOverrideDropped(resolvedName, pinnedModel);
pinnedModel = null;
}
// #3241 — one-time deprecation notice: if nothing ends up pinned but the
// runtime resolver would have supplied a per-tier model that would actually
// have been EMBEDDED (the population that loses a pin now that the
// auto-embed above is gone), point the user at model_overrides. The would-be
// model must also clear the #2310 Anthropic-flavored gate above — if it
// wouldn't have survived that gate, the user never had that pin pre-Phase-1
// either, and the notice would be false. Never fires when the resolver is
// null, resolves to nothing, resolves to an Anthropic-flavored model, or an
// explicit real-Codex pin survived — in all of those cases nothing was lost.
if (!pinnedModel && runtimeResolver) {
const wouldHavePinned = runtimeResolver.resolve(resolvedName) || runtimeResolver.resolve(agentName);
if (wouldHavePinned?.model && !_isAnthropicFlavoredModel(wouldHavePinned.model)) {
_warnCodexResolverModelOmitted();
}
}
let hasPinnedModel = false;
if (pinnedModel) {
lines.push(`model = ${JSON.stringify(pinnedModel)}`);
@@ -10069,6 +10129,12 @@ function install(isGlobal, runtime = DEFAULT_RUNTIME, options = {}) {
const dirName = getDirName(runtime);
const src = path.join(__dirname, '..');
// #3241 — the Codex resolver-model-omitted notice dedupes "at most once", but
// scoped per install() call rather than per process — each install() run gets
// its own fresh window so a second install (e.g. a second runtime, or a test
// re-running install()) can warn again if the same condition recurs.
_codexResolverModelOmittedWarned = false;
if (_hostBehaviors(runtime).localInstallDeferred && !isGlobal) {
console.log(` ${yellow}⚠${reset} Kimi local install is deferred for Phase 2.`);
console.log(` No .kimi-code/skills or .agents/skills project artifacts were written.`);
@@ -13458,6 +13524,7 @@ module.exports = {
convertClaudeAgentToCursorAgent,
convertClaudeAgentToCodexAgent,
generateCodexAgentToml,
_resetCodexWarningDedupeForTests,
cleanupCodexSkillMetadataSidecars,
cleanupWindsurfLegacyDevinSkills,
cleanupMovedSkillsOldLocation,