golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

fix(#2070): accept adaptive model_profile in validate health; warn on invalid models tiers (W022) (#2336)

Browse Source 
* test(#2070): fail-first tests for adaptive model_profile and models tier validation

Encodes the three acceptance criteria from #2070 plus the boundary cases the
resolver silently ignores today (non-string values, empty string, mistyped
phase-type key), and pins VALID_TIERS to a catalog-derived set.

Red phase: these fail against current src/ by design.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* fix(#2070): accept adaptive model_profile in validate health; warn on invalid models tiers (W022)

W004 sourced its profile list from a hand-maintained literal that predated the
adaptive profile, so `"model_profile": "adaptive"` was false-flagged. It now
reads VALID_PROFILES, which model-catalog.cts derives from model-catalog.json.

models.<phase_type> was validated nowhere: the resolver's tier gate silently
drops unknown values, so a typo like `"planning": "opuss"` was an undiagnosable
no-op. A new W022 flags unknown phase-type keys and invalid tier values
(including non-string values, which the same gate also drops).

VALID_TIERS moves from a function-local literal in model-resolver.cts to a
catalog-derived export, so health and the resolver cannot disagree by
construction rather than by parity test. Object.values(adaptiveTierMap) is
['opus','sonnet','haiku'] plus 'inherit' — identical to the previous literal,
so resolution behavior is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* docs(#2070): changeset for validate health adaptive profile + W022

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* fix(#2070): close review findings — malformed models, tier-list duplication, changeset gate

Review of the initial fix surfaced three real defects, folded in per the
no-defer rule:

1. verify.cts: the W022 guard skipped a top-level `models` that is present but
   not a plain object (`[]`, `"opus"`, `5`, `true`). The resolver ignores those
   identically, so they were the same undiagnosable no-op #2070 targets — just
   one level up. They now warn; absent/null/{} stay silent.

2. config-loader.cts: RUNTIME_OVERRIDE_TIERS was a second hardcoded copy of the
   tier vocabulary this change had just de-hardcoded elsewhere. It now derives
   from the catalog via ADAPTIVE_TIER_VALUES (no 'inherit' — runtime overrides
   resolve to a concrete tier). Byte-equivalent to the old literal.

3. scripts/changeset/lint.cjs: USER_FACING_PREFIXES omitted `src/`. Post-ADR-457
   the product source is src/*.cts compiled to a gitignored gsd-core/bin/lib,
   so the `gsd-core/` prefix is dead coverage for library code and a src/-only
   PR could merge with no release note — including this one. Adding `src/`
   closes the gate; tests/ stays non-user-facing.

Also corrects a false docstring in the VALID_TIERS test: value-equality cannot
detect a re-hardcoded literal, so the test no longer claims it does.

Two review findings were rejected with evidence rather than actioned:
- W021 double-allocation is governed by ADR-612 ("W021 renumber -> void ...
  kept, message-disambiguated"), not a defect.
- Global-defaults validation would be a false-positive generator: config-loader
  reads ~/.gsd/defaults.json only on the "no .planning/" branch, and health
  early-returns E001 without .planning/, so those values provably never affect
  resolution in any context health can run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* test(#2070): regenerate install goldens for the changeset-lint change

scripts/ ships as an installed artifact, so scripts/changeset/lint.cjs's content
hash is pinned in all 18 runtime golden fixtures. Adding 'src/' to
USER_FACING_PREFIXES changed that hash and tripped every golden parity check.
Regenerated via `npm run gen:golden`; the only delta is the lint.cjs hash.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* docs(#2070): backfill PR number 2336 into changeset

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-07-17 06:48:04 -04:00
committed by GitHub
parent a30fb75b51
commit b2961c3f69
27 changed files with 511 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.changeset/gentle-koalas-hum.md Normal file
View File

@@ -0,0 +1,5 @@
---
type: Fixed
pr: 2336
---
**`validate health` no longer false-flags the `adaptive` model profile, and now warns when a `models.<phase_type>` tier is invalid** — health reported `W004 invalid model_profile "adaptive"` for a profile that has been valid since v1.40, and a typo like `"planning": "opuss"` was accepted in silence while the resolver quietly ignored it. Health now sources its profile list from the model catalog and emits `W022` for unknown phase types and invalid tier values.

1
scripts/changeset/lint.cjs
View File

@@ -27,6 +27,7 @@ const OPT_OUT_LABEL = 'no-changelog';
const USER_FACING_PREFIXES = [
'bin/',
'gsd-core/',
'src/',
'agents/',
'commands/',
'hooks/',

10
src/config-loader.cts
View File

@@ -17,7 +17,7 @@
* - ./planning-workspace.cjs (planningDir, planningRoot)
* - ./shell-command-projection.cjs (execGit, platformWriteSync, platformReadSync)
* - ./core-utils.cjs (detectSubRepos)
* - ./model-catalog.cjs (KNOWN_RUNTIMES, KNOWN_PROVIDERS)
* - ./model-catalog.cjs (KNOWN_RUNTIMES, KNOWN_PROVIDERS, ADAPTIVE_TIER_VALUES)
*/
import fs from 'node:fs';
@@ -35,7 +35,7 @@ import { CONFIG_DEFAULTS as CANONICAL_CONFIG_DEFAULTS, normalizeLegacyKeys } fro
// eslint-disable-next-line @typescript-eslint/no-require-imports
import configSchema = require('./config-schema.cjs');
const { VALID_CONFIG_KEYS, DYNAMIC_KEY_PATTERNS, isCentralConfigKey: _isCentralConfigKeyFn } = configSchema;
import { KNOWN_RUNTIMES, KNOWN_PROVIDERS } from './model-catalog.cjs';
import { KNOWN_RUNTIMES, KNOWN_PROVIDERS, ADAPTIVE_TIER_VALUES } from './model-catalog.cjs';
// ─── Federated Config (ADR-857 phase 3b) ─────────────────────────────────────
// eslint-disable-next-line @typescript-eslint/no-require-imports
import federatedConfigModule = require('./federated-config.cjs');
@@ -190,7 +190,11 @@ function isGitIgnored(cwd: string, targetPath: string): boolean {
// ─── Model alias resolution ───────────────────────────────────────────────────
const RUNTIME_OVERRIDE_TIERS = new Set(['opus', 'sonnet', 'haiku']);
// Catalog-derived (model-catalog.cts) so this vocabulary can never drift from
// VALID_TIERS in verify.cts — see #2070 "Generative Fix Divergence". Excludes
// 'inherit' (unlike VALID_TIERS): runtime overrides always resolve to a
// concrete tier, never the adaptive sentinel.
const RUNTIME_OVERRIDE_TIERS = ADAPTIVE_TIER_VALUES;
const _warnedConfigKeys = new Set<string>();
function _warnUnknownProfileOverrides(parsed: Record<string, unknown>, configLabel: string): void {

7
src/model-catalog.cts
View File

@@ -77,6 +77,13 @@ export { _catalog as catalog };
export const VALID_PROFILES: string[] = [..._catalog.profiles];
export const VALID_PHASE_TYPES: Set<string> = new Set(_catalog.phaseTypes);
export const VALID_AGENT_TIERS: Set<string> = new Set(Object.keys(_catalog.adaptiveTierMap));
// Catalog-derived so this can never drift from the resolver's tier gate:
// Object.values(adaptiveTierMap) === ['opus', 'sonnet', 'haiku'] today, plus 'inherit'.
export const VALID_TIERS: Set<string> = new Set([...Object.values(_catalog.adaptiveTierMap), 'inherit']);
// Same catalog-derived tier values as VALID_TIERS but WITHOUT 'inherit' — used
// by config-loader's runtime-override validation (model_profile_overrides /
// model_policy.runtime_tiers), which does not accept 'inherit' as a tier.
export const ADAPTIVE_TIER_VALUES: Set<string> = new Set(Object.values(_catalog.adaptiveTierMap));
/** Per-profile model slots for each agent. */
export interface AgentModelProfiles {

5
src/model-resolver.cts
View File

@@ -16,7 +16,7 @@
* - ./config-loader.cjs (loadConfig)
* - ./configuration.cjs (CONFIG_DEFAULTS as CANONICAL_CONFIG_DEFAULTS)
* - ./model-profiles.cjs (MODEL_PROFILES, AGENT_TO_PHASE_TYPE, AGENT_DEFAULT_TIERS, VALID_AGENT_TIERS, nextTier)
* - ./model-catalog.cjs (MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, PROVIDER_PRESETS)
* - ./model-catalog.cjs (MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, PROVIDER_PRESETS, VALID_TIERS)
*/
// eslint-disable-next-line @typescript-eslint/no-require-imports
@@ -30,7 +30,7 @@ import { CONFIG_DEFAULTS as CANONICAL_CONFIG_DEFAULTS } from './configuration.cj
import modelProfiles = require('./model-profiles.cjs');
const { MODEL_PROFILES, AGENT_TO_PHASE_TYPE, AGENT_DEFAULT_TIERS, VALID_AGENT_TIERS, nextTier } = modelProfiles;
import { MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, PROVIDER_PRESETS } from './model-catalog.cjs';
import { MODEL_ALIAS_MAP, RUNTIME_PROFILE_MAP, PROVIDER_PRESETS, VALID_TIERS } from './model-catalog.cjs';
import fs from 'node:fs';
import path from 'node:path';
@@ -333,7 +333,6 @@ function resolveModelInternal(cwd: string, agentType: string): string {
const phaseTypeTier = (phaseType && configModels && typeof configModels === 'object')
? configModels[phaseType]
: undefined;
const VALID_TIERS = new Set(['opus', 'sonnet', 'haiku', 'inherit']);
const tier = (phaseTypeTier && VALID_TIERS.has(phaseTypeTier))
? phaseTypeTier
: (profile === 'inherit'

33
src/verify.cts
View File

@@ -28,6 +28,7 @@ import { formatGsdSlash, resolveRuntime } from './runtime-slash.cjs';
import { detectSchemaFiles, checkSchemaDrift } from './schema-detect.cjs';
import { isCanonicalPlanningFile } from './artifacts.cjs';
import { extractTaggedBlocks } from './markdown-sectionizer.cjs';
import { VALID_PROFILES, VALID_TIERS, VALID_PHASE_TYPES } from './model-catalog.cjs';
// eslint-disable-next-line @typescript-eslint/no-require-imports -- agent-install-check.cjs is an export= CommonJS module
import agentInstallCheck = require('./agent-install-check.cjs');
const { checkAgentsInstalled } = agentInstallCheck;
@@ -1354,13 +1355,39 @@ function cmdValidateHealth(
try {
const rawCfg = fs.readFileSync(configPath, 'utf-8');
const parsed = JSON.parse(rawCfg) as Record<string, unknown>;
const validProfiles = ['quality', 'balanced', 'budget', 'inherit'];
if (parsed['model_profile'] && !validProfiles.includes(parsed['model_profile'] as string)) {
if (parsed['model_profile'] && !VALID_PROFILES.includes(parsed['model_profile'] as string)) {
addIssue(
'warning',
'W004',
`config.json: invalid model_profile "${parsed['model_profile'] as string}"`,
`Valid values: ${validProfiles.join(', ')}`,
`Valid values: ${VALID_PROFILES.join(', ')}`,
);
}
const configModels = parsed['models'];
if (configModels && typeof configModels === 'object' && !Array.isArray(configModels)) {
for (const [phaseType, tierValue] of Object.entries(configModels as Record<string, unknown>)) {
if (!VALID_PHASE_TYPES.has(phaseType)) {
addIssue(
'warning',
'W022',
`config.json: models has an unknown phase type "${phaseType}" which will be ignored`,
`Valid phase types: ${[...VALID_PHASE_TYPES].join(', ')}`,
);
} else if (typeof tierValue !== 'string' || !VALID_TIERS.has(tierValue)) {
addIssue(
'warning',
'W022',
`config.json: models.${phaseType} has an invalid tier value ${JSON.stringify(tierValue)} which will be ignored`,
`Valid tiers: ${[...VALID_TIERS].join(', ')}`,
);
}
}
} else if (configModels !== undefined && configModels !== null) {
addIssue(
'warning',
'W022',
`config.json: models is set to ${JSON.stringify(configModels)}, but must be an object mapping phase types to tiers — this value will be ignored`,
`Set models to an object like {"planning": "sonnet"}, or remove the key to use profile defaults`,
);
}
} catch (err) {

32
tests/changeset-lint.test.cjs
View File

@@ -175,6 +175,38 @@ describe('changeset lint: pure verdict (#2975)', () => {
});
});
// ---------------------------------------------------------------------------
// src/ is user-facing (#2070 fix 2): post-ADR-457 the .cts sources under
// src/ are the product source of truth — gsd-core/bin/lib/*.cjs is a
// gitignored build artifact, so a src/-only PR previously had zero
// USER_FACING_PREFIXES coverage and could merge with no release note.
// ---------------------------------------------------------------------------
describe('changeset lint: src/ is user-facing (#2070)', () => {
test('FAIL_MISSING_FRAGMENT when only a src/*.cts file changes without a fragment', () => {
const verdict = evaluateLint({
changedFiles: ['src/verify.cts'],
labels: [],
});
assert.deepEqual(verdict, { ok: false, reason: LINT_REASON.FAIL_MISSING_FRAGMENT });
});
test('OK_FRAGMENT_PRESENT when a src/*.cts change is accompanied by a fragment', () => {
const verdict = evaluateLint({
changedFiles: ['src/verify.cts', '.changeset/silly-bears-dance.md'],
labels: [],
});
assert.deepEqual(verdict, { ok: true, reason: LINT_REASON.OK_FRAGMENT_PRESENT });
});
test('OK_NO_USER_FACING_CHANGES when only a tests/*.test.cjs file changes (tests are not user-facing)', () => {
const verdict = evaluateLint({
changedFiles: ['tests/foo.test.cjs'],
labels: [],
});
assert.deepEqual(verdict, { ok: true, reason: LINT_REASON.OK_NO_USER_FACING_CHANGES });
});
});
// ---------------------------------------------------------------------------
// End-to-end integration: real main() wiring via temp git repo (#1006)
// ---------------------------------------------------------------------------

2
tests/fixtures/golden-install-parity/antigravity.json vendored
View File

@@ -344,7 +344,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/augment.json vendored
View File

@@ -414,7 +414,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/claude-local.json vendored
View File

@@ -413,7 +413,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/claude.json vendored
View File

@@ -342,7 +342,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/cline.json vendored
View File

@@ -318,7 +318,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/codebuddy.json vendored
View File

@@ -414,7 +414,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/codex.json vendored
View File

@@ -423,7 +423,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/copilot.json vendored
View File

@@ -317,7 +317,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/cursor.json vendored
View File

@@ -392,7 +392,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/hermes.json vendored
View File

@@ -343,7 +343,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/kilo.json vendored
View File

@@ -388,7 +388,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/kimi.json vendored
View File

@@ -379,7 +379,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/opencode.json vendored
View File

@@ -416,7 +416,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/pi.json vendored
View File

@@ -310,7 +310,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/qwen.json vendored
View File

@@ -343,7 +343,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/trae.json vendored
View File

@@ -315,7 +315,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/windsurf.json vendored
View File

@@ -317,7 +317,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/zcode.json vendored
View File

@@ -386,7 +386,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

80
tests/model-catalog-valid-tiers.test.cjs Normal file
View File

@@ -0,0 +1,80 @@
'use strict';
/**
* Regression/parity tests for VALID_TIERS and RUNTIME_OVERRIDE_TIERS (#2070).
*
* src/model-resolver.cts:336 currently declares a function-local,
* non-exported `VALID_TIERS` set. src/model-catalog.cts now exports a
* `VALID_TIERS` computed as `[...Object.values(catalog.adaptiveTierMap),
* 'inherit']`. NOTE: a value-equality assertion against that same derivation
* cannot, by construction, distinguish "genuinely catalog-derived" from "a
* hardcoded literal that happens to equal today's catalog output" — both
* would pass this test. What this test actually guarantees is narrower but
* still useful: VALID_TIERS equals the catalog-derived set right now, so it
* fails loudly the moment the two go out of sync (e.g. someone drops
* 'inherit', or model-catalog.json gains/loses a tier and VALID_TIERS isn't
* updated to match) — an impl/catalog divergence guard, not a
* hardcoding detector.
*
* The RUNTIME_OVERRIDE_TIERS tests below cover review finding #7: a second,
* independent hardcoded tier list at src/config-loader.cts:193
* (`new Set(['opus', 'sonnet', 'haiku'])`, exported at line 769) is a
* parallel surface to VALID_TIERS. Fixed to derive from
* `Object.values(catalog.adaptiveTierMap)` (which — unlike VALID_TIERS —
* does NOT include 'inherit'), the parity assertion below fails loudly if
* RUNTIME_OVERRIDE_TIERS and VALID_TIERS ever drift apart.
*/
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const { catalog, VALID_TIERS } = require('../gsd-core/bin/lib/model-catalog.cjs');
const { RUNTIME_OVERRIDE_TIERS } = require('../gsd-core/bin/lib/config-loader.cjs');
describe('model-catalog VALID_TIERS derivation (#2070)', () => {
test('VALID_TIERS equals the catalog-derived set (adaptiveTierMap values plus "inherit") — catches impl/catalog divergence', () => {
const expected = new Set([...Object.values(catalog.adaptiveTierMap), 'inherit']);
assert.deepStrictEqual(
VALID_TIERS,
expected,
`VALID_TIERS must match catalog.adaptiveTierMap-derived tiers plus 'inherit': ${JSON.stringify([...(VALID_TIERS || [])])}`
);
});
test('VALID_TIERS pins current tier set to opus/sonnet/haiku/inherit', () => {
assert.deepStrictEqual(
VALID_TIERS,
new Set(['opus', 'sonnet', 'haiku', 'inherit']),
`Expected VALID_TIERS to equal {opus, sonnet, haiku, inherit}, got: ${JSON.stringify([...(VALID_TIERS || [])])}`
);
});
});
describe('config-loader RUNTIME_OVERRIDE_TIERS derivation (#2070 review finding 7)', () => {
test('RUNTIME_OVERRIDE_TIERS equals catalog.adaptiveTierMap values (no "inherit")', () => {
const expected = new Set(Object.values(catalog.adaptiveTierMap));
assert.deepStrictEqual(
RUNTIME_OVERRIDE_TIERS,
expected,
`RUNTIME_OVERRIDE_TIERS must be derived from catalog.adaptiveTierMap, not hardcoded: ${JSON.stringify([...(RUNTIME_OVERRIDE_TIERS || [])])}`
);
});
test('RUNTIME_OVERRIDE_TIERS pins current tier set to opus/sonnet/haiku', () => {
assert.deepStrictEqual(
RUNTIME_OVERRIDE_TIERS,
new Set(['opus', 'sonnet', 'haiku']),
`Expected RUNTIME_OVERRIDE_TIERS to equal {opus, sonnet, haiku}, got: ${JSON.stringify([...(RUNTIME_OVERRIDE_TIERS || [])])}`
);
});
test('RUNTIME_OVERRIDE_TIERS ∪ {"inherit"} equals VALID_TIERS (parity — fails loudly if the two surfaces drift)', () => {
const union = new Set([...RUNTIME_OVERRIDE_TIERS, 'inherit']);
assert.deepStrictEqual(
union,
VALID_TIERS,
`RUNTIME_OVERRIDE_TIERS plus 'inherit' must equal VALID_TIERS — these are two parallel tier-list surfaces that must never diverge: ` +
`RUNTIME_OVERRIDE_TIERS=${JSON.stringify([...(RUNTIME_OVERRIDE_TIERS || [])])}, VALID_TIERS=${JSON.stringify([...(VALID_TIERS || [])])}`
);
});
});

329
tests/verify-health.test.cjs
View File

@@ -21,6 +21,7 @@ const assert = require('node:assert/strict');
const fs = require('fs');
const path = require('path');
const { runGsdTools, createTempProject, cleanup } = require('./helpers.cjs');
const { VALID_PROFILES } = require('../gsd-core/bin/lib/model-catalog.cjs');
// ─── Helpers for setting up minimal valid projects ────────────────────────────
@@ -351,6 +352,334 @@ describe('validate health command', () => {
);
});
// ─── Check 5c: model_profile "adaptive" + per-phase-type tier validation
// (W022, #2070) ──────────────────────────────────────────────────────────
// W004 must be sourced from the catalog's VALID_PROFILES (so "adaptive"
// is accepted), and a new W022 must fire for any `models.<phase_type>`
// entry whose value is not a valid tier (opus/sonnet/haiku/inherit),
// including non-string values, and for keys that are not valid phase types.
test('accepts adaptive model_profile as valid (no W004)', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'adaptive' })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W004'),
`Should not warn W004 for adaptive model_profile: ${JSON.stringify(output.warnings)}`
);
});
test('every catalog profile is accepted without W004', () => {
// Loop over VALID_PROFILES so this can never drift from the catalog.
for (const profile of VALID_PROFILES) {
const dir = createTempProject();
try {
writeMinimalProjectMd(dir);
writeMinimalRoadmap(dir, ['1']);
writeMinimalStateMd(dir);
fs.writeFileSync(
path.join(dir, '.planning', 'config.json'),
JSON.stringify({ model_profile: profile })
);
fs.mkdirSync(path.join(dir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', dir);
assert.ok(result.success, `Command failed for profile "${profile}": ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W004'),
`Should not warn W004 for catalog profile "${profile}": ${JSON.stringify(output.warnings)}`
);
} finally {
cleanup(dir);
}
}
});
test('still warns W004 for a genuinely invalid model_profile (regression guard)', () => {
// Don't let the adaptive-profile fix over-broaden into accepting everything.
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'bogus' })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
output.warnings.some(w => w.code === 'W004'),
`Expected W004 to still fire for a bogus model_profile: ${JSON.stringify(output.warnings)}`
);
});
test('warns W022 for an invalid tier value under models.<phase_type>', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: 'opuss' } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
const w022 = output.warnings.find(w => w.code === 'W022');
assert.ok(w022, `Expected W022 in warnings: ${JSON.stringify(output.warnings)}`);
const fullText = `${w022.message} ${w022.fix || ''}`;
assert.ok(fullText.includes('planning'), `Expected W022 message to name "planning": ${JSON.stringify(w022)}`);
assert.ok(fullText.includes('opuss'), `Expected W022 message to name "opuss": ${JSON.stringify(w022)}`);
assert.ok(
/ignor/i.test(fullText),
`Expected W022 message to indicate the value will be ignored: ${JSON.stringify(w022)}`
);
});
test('does not warn W022 or W004 for a valid tier value under models.<phase_type>', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: 'opus' } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not have W022 for valid tier "opus": ${JSON.stringify(output.warnings)}`
);
assert.ok(
!output.warnings.some(w => w.code === 'W004'),
`Should not have W004: ${JSON.stringify(output.warnings)}`
);
});
for (const tier of ['sonnet', 'haiku', 'inherit']) {
test(`does not warn W022 for the valid tier value "${tier}" under models.<phase_type>`, () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: tier } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not have W022 for valid tier "${tier}": ${JSON.stringify(output.warnings)}`
);
});
}
test('warns W022 for a numeric (non-string) models.<phase_type> value', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: 5 } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
output.warnings.some(w => w.code === 'W022'),
`Expected W022 for numeric models.planning value: ${JSON.stringify(output.warnings)}`
);
});
test('warns W022 for a null models.<phase_type> value', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: null } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
output.warnings.some(w => w.code === 'W022'),
`Expected W022 for null models.planning value: ${JSON.stringify(output.warnings)}`
);
});
test('warns W022 for an empty-string models.<phase_type> value', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: '' } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
output.warnings.some(w => w.code === 'W022'),
`Expected W022 for empty-string models.planning value: ${JSON.stringify(output.warnings)}`
);
});
test('warns W022 for a mistyped models phase-type key, naming the unknown key', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { plannning: 'opus' } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
const w022 = output.warnings.find(w => w.code === 'W022');
assert.ok(w022, `Expected W022 in warnings: ${JSON.stringify(output.warnings)}`);
const fullText = `${w022.message} ${w022.fix || ''}`;
assert.ok(
fullText.includes('plannning'),
`Expected W022 message to name the unknown key "plannning": ${JSON.stringify(w022)}`
);
});
// ─── #2070 review finding 3: malformed top-level `models` is silently
// skipped today (src/verify.cts guards `typeof === 'object' && !Array`
// with no else branch, so an array/string/number/boolean `models` value
// produces ZERO diagnostics — the same undiagnosable-no-op class #2070
// targets). `models` absent or explicitly `null` must stay silent. ────────
for (const [label, malformedModels] of [
['an array', []],
['a string', 'opus'],
['a number', 5],
['a boolean', true],
]) {
test(`warns W022 for a top-level models value that is ${label} (not a plain object)`, () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: malformedModels })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
const w022 = output.warnings.find(w => w.code === 'W022');
assert.ok(
w022,
`Expected W022 when top-level models is ${label} (${JSON.stringify(malformedModels)}): ${JSON.stringify(output.warnings)}`
);
const fullText = `${w022.message} ${w022.fix || ''}`;
assert.ok(fullText.includes('models'), `Expected W022 message to mention "models": ${JSON.stringify(w022)}`);
assert.ok(/object/i.test(fullText), `Expected W022 message to indicate models must be an object: ${JSON.stringify(w022)}`);
assert.ok(/ignor/i.test(fullText), `Expected W022 message to indicate the value will be ignored: ${JSON.stringify(w022)}`);
});
}
test('does not warn W022 for an empty-object top-level models value (valid, just empty)', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: {} })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not warn W022 for an empty models object: ${JSON.stringify(output.warnings)}`
);
});
test('does not warn W022 for a top-level models value of null (explicit unset)', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: null })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not warn W022 for models: null: ${JSON.stringify(output.warnings)}`
);
});
test('does not warn W022 when the models key is absent entirely', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced' })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not warn W022 when models is absent: ${JSON.stringify(output.warnings)}`
);
});
// ─── Check 6: Phase directory naming (NN-name format) ─────────────────────
test('warns about incorrectly named phase directories', () => {