golem15/msd-core
4
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(#2070): accept adaptive model_profile in validate health; warn on invalid models tiers (W022) (#2336)

Browse Source 
* test(#2070): fail-first tests for adaptive model_profile and models tier validation

Encodes the three acceptance criteria from #2070 plus the boundary cases the
resolver silently ignores today (non-string values, empty string, mistyped
phase-type key), and pins VALID_TIERS to a catalog-derived set.

Red phase: these fail against current src/ by design.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* fix(#2070): accept adaptive model_profile in validate health; warn on invalid models tiers (W022)

W004 sourced its profile list from a hand-maintained literal that predated the
adaptive profile, so `"model_profile": "adaptive"` was false-flagged. It now
reads VALID_PROFILES, which model-catalog.cts derives from model-catalog.json.

models.<phase_type> was validated nowhere: the resolver's tier gate silently
drops unknown values, so a typo like `"planning": "opuss"` was an undiagnosable
no-op. A new W022 flags unknown phase-type keys and invalid tier values
(including non-string values, which the same gate also drops).

VALID_TIERS moves from a function-local literal in model-resolver.cts to a
catalog-derived export, so health and the resolver cannot disagree by
construction rather than by parity test. Object.values(adaptiveTierMap) is
['opus','sonnet','haiku'] plus 'inherit' — identical to the previous literal,
so resolution behavior is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* docs(#2070): changeset for validate health adaptive profile + W022

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* fix(#2070): close review findings — malformed models, tier-list duplication, changeset gate

Review of the initial fix surfaced three real defects, folded in per the
no-defer rule:

1. verify.cts: the W022 guard skipped a top-level `models` that is present but
   not a plain object (`[]`, `"opus"`, `5`, `true`). The resolver ignores those
   identically, so they were the same undiagnosable no-op #2070 targets — just
   one level up. They now warn; absent/null/{} stay silent.

2. config-loader.cts: RUNTIME_OVERRIDE_TIERS was a second hardcoded copy of the
   tier vocabulary this change had just de-hardcoded elsewhere. It now derives
   from the catalog via ADAPTIVE_TIER_VALUES (no 'inherit' — runtime overrides
   resolve to a concrete tier). Byte-equivalent to the old literal.

3. scripts/changeset/lint.cjs: USER_FACING_PREFIXES omitted `src/`. Post-ADR-457
   the product source is src/*.cts compiled to a gitignored gsd-core/bin/lib,
   so the `gsd-core/` prefix is dead coverage for library code and a src/-only
   PR could merge with no release note — including this one. Adding `src/`
   closes the gate; tests/ stays non-user-facing.

Also corrects a false docstring in the VALID_TIERS test: value-equality cannot
detect a re-hardcoded literal, so the test no longer claims it does.

Two review findings were rejected with evidence rather than actioned:
- W021 double-allocation is governed by ADR-612 ("W021 renumber -> void ...
  kept, message-disambiguated"), not a defect.
- Global-defaults validation would be a false-positive generator: config-loader
  reads ~/.gsd/defaults.json only on the "no .planning/" branch, and health
  early-returns E001 without .planning/, so those values provably never affect
  resolution in any context health can run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* test(#2070): regenerate install goldens for the changeset-lint change

scripts/ ships as an installed artifact, so scripts/changeset/lint.cjs's content
hash is pinned in all 18 runtime golden fixtures. Adding 'src/' to
USER_FACING_PREFIXES changed that hash and tripped every golden parity check.
Regenerated via `npm run gen:golden`; the only delta is the lint.cjs hash.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

* docs(#2070): backfill PR number 2336 into changeset

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SLufH5sDuqA1AiEGu45cuA

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-07-17 06:48:04 -04:00
committed by GitHub
parent a30fb75b51
commit b2961c3f69
27 changed files with 511 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
tests/changeset-lint.test.cjs
View File

@@ -175,6 +175,38 @@ describe('changeset lint: pure verdict (#2975)', () => {
});
});
// ---------------------------------------------------------------------------
// src/ is user-facing (#2070 fix 2): post-ADR-457 the .cts sources under
// src/ are the product source of truth — gsd-core/bin/lib/*.cjs is a
// gitignored build artifact, so a src/-only PR previously had zero
// USER_FACING_PREFIXES coverage and could merge with no release note.
// ---------------------------------------------------------------------------
describe('changeset lint: src/ is user-facing (#2070)', () => {
test('FAIL_MISSING_FRAGMENT when only a src/*.cts file changes without a fragment', () => {
const verdict = evaluateLint({
changedFiles: ['src/verify.cts'],
labels: [],
});
assert.deepEqual(verdict, { ok: false, reason: LINT_REASON.FAIL_MISSING_FRAGMENT });
});
test('OK_FRAGMENT_PRESENT when a src/*.cts change is accompanied by a fragment', () => {
const verdict = evaluateLint({
changedFiles: ['src/verify.cts', '.changeset/silly-bears-dance.md'],
labels: [],
});
assert.deepEqual(verdict, { ok: true, reason: LINT_REASON.OK_FRAGMENT_PRESENT });
});
test('OK_NO_USER_FACING_CHANGES when only a tests/*.test.cjs file changes (tests are not user-facing)', () => {
const verdict = evaluateLint({
changedFiles: ['tests/foo.test.cjs'],
labels: [],
});
assert.deepEqual(verdict, { ok: true, reason: LINT_REASON.OK_NO_USER_FACING_CHANGES });
});
});
// ---------------------------------------------------------------------------
// End-to-end integration: real main() wiring via temp git repo (#1006)
// ---------------------------------------------------------------------------

2
tests/fixtures/golden-install-parity/antigravity.json vendored
View File

@@ -344,7 +344,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/augment.json vendored
View File

@@ -414,7 +414,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/claude-local.json vendored
View File

@@ -413,7 +413,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/claude.json vendored
View File

@@ -342,7 +342,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/cline.json vendored
View File

@@ -318,7 +318,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/codebuddy.json vendored
View File

@@ -414,7 +414,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/codex.json vendored
View File

@@ -423,7 +423,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/copilot.json vendored
View File

@@ -317,7 +317,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/cursor.json vendored
View File

@@ -392,7 +392,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/hermes.json vendored
View File

@@ -343,7 +343,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/kilo.json vendored
View File

@@ -388,7 +388,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/kimi.json vendored
View File

@@ -379,7 +379,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/opencode.json vendored
View File

@@ -416,7 +416,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/pi.json vendored
View File

@@ -310,7 +310,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/qwen.json vendored
View File

@@ -343,7 +343,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/trae.json vendored
View File

@@ -315,7 +315,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/windsurf.json vendored
View File

@@ -317,7 +317,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

2
tests/fixtures/golden-install-parity/zcode.json vendored
View File

@@ -386,7 +386,7 @@
"scripts/changeset/README.md": "86ff89331dfd94b2",
"scripts/changeset/cli.cjs": "68f92a344b199271",
"scripts/changeset/github-release-notes.cjs": "795677f0c009b132",
"scripts/changeset/lint.cjs": "0066faed159154f0",
"scripts/changeset/lint.cjs": "23cb53f77a6ea180",
"scripts/changeset/new.cjs": "4991e21fd17f5541",
"scripts/changeset/parse.cjs": "f9a949cbcab56445",
"scripts/changeset/render.cjs": "e47bc3e1587c3cae",

80
tests/model-catalog-valid-tiers.test.cjs Normal file
View File

@@ -0,0 +1,80 @@
'use strict';
/**
* Regression/parity tests for VALID_TIERS and RUNTIME_OVERRIDE_TIERS (#2070).
*
* src/model-resolver.cts:336 currently declares a function-local,
* non-exported `VALID_TIERS` set. src/model-catalog.cts now exports a
* `VALID_TIERS` computed as `[...Object.values(catalog.adaptiveTierMap),
* 'inherit']`. NOTE: a value-equality assertion against that same derivation
* cannot, by construction, distinguish "genuinely catalog-derived" from "a
* hardcoded literal that happens to equal today's catalog output" — both
* would pass this test. What this test actually guarantees is narrower but
* still useful: VALID_TIERS equals the catalog-derived set right now, so it
* fails loudly the moment the two go out of sync (e.g. someone drops
* 'inherit', or model-catalog.json gains/loses a tier and VALID_TIERS isn't
* updated to match) — an impl/catalog divergence guard, not a
* hardcoding detector.
*
* The RUNTIME_OVERRIDE_TIERS tests below cover review finding #7: a second,
* independent hardcoded tier list at src/config-loader.cts:193
* (`new Set(['opus', 'sonnet', 'haiku'])`, exported at line 769) is a
* parallel surface to VALID_TIERS. Fixed to derive from
* `Object.values(catalog.adaptiveTierMap)` (which — unlike VALID_TIERS —
* does NOT include 'inherit'), the parity assertion below fails loudly if
* RUNTIME_OVERRIDE_TIERS and VALID_TIERS ever drift apart.
*/
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const { catalog, VALID_TIERS } = require('../gsd-core/bin/lib/model-catalog.cjs');
const { RUNTIME_OVERRIDE_TIERS } = require('../gsd-core/bin/lib/config-loader.cjs');
describe('model-catalog VALID_TIERS derivation (#2070)', () => {
test('VALID_TIERS equals the catalog-derived set (adaptiveTierMap values plus "inherit") — catches impl/catalog divergence', () => {
const expected = new Set([...Object.values(catalog.adaptiveTierMap), 'inherit']);
assert.deepStrictEqual(
VALID_TIERS,
expected,
`VALID_TIERS must match catalog.adaptiveTierMap-derived tiers plus 'inherit': ${JSON.stringify([...(VALID_TIERS || [])])}`
);
});
test('VALID_TIERS pins current tier set to opus/sonnet/haiku/inherit', () => {
assert.deepStrictEqual(
VALID_TIERS,
new Set(['opus', 'sonnet', 'haiku', 'inherit']),
`Expected VALID_TIERS to equal {opus, sonnet, haiku, inherit}, got: ${JSON.stringify([...(VALID_TIERS || [])])}`
);
});
});
describe('config-loader RUNTIME_OVERRIDE_TIERS derivation (#2070 review finding 7)', () => {
test('RUNTIME_OVERRIDE_TIERS equals catalog.adaptiveTierMap values (no "inherit")', () => {
const expected = new Set(Object.values(catalog.adaptiveTierMap));
assert.deepStrictEqual(
RUNTIME_OVERRIDE_TIERS,
expected,
`RUNTIME_OVERRIDE_TIERS must be derived from catalog.adaptiveTierMap, not hardcoded: ${JSON.stringify([...(RUNTIME_OVERRIDE_TIERS || [])])}`
);
});
test('RUNTIME_OVERRIDE_TIERS pins current tier set to opus/sonnet/haiku', () => {
assert.deepStrictEqual(
RUNTIME_OVERRIDE_TIERS,
new Set(['opus', 'sonnet', 'haiku']),
`Expected RUNTIME_OVERRIDE_TIERS to equal {opus, sonnet, haiku}, got: ${JSON.stringify([...(RUNTIME_OVERRIDE_TIERS || [])])}`
);
});
test('RUNTIME_OVERRIDE_TIERS ∪ {"inherit"} equals VALID_TIERS (parity — fails loudly if the two surfaces drift)', () => {
const union = new Set([...RUNTIME_OVERRIDE_TIERS, 'inherit']);
assert.deepStrictEqual(
union,
VALID_TIERS,
`RUNTIME_OVERRIDE_TIERS plus 'inherit' must equal VALID_TIERS — these are two parallel tier-list surfaces that must never diverge: ` +
`RUNTIME_OVERRIDE_TIERS=${JSON.stringify([...(RUNTIME_OVERRIDE_TIERS || [])])}, VALID_TIERS=${JSON.stringify([...(VALID_TIERS || [])])}`
);
});
});

329
tests/verify-health.test.cjs
View File

@@ -21,6 +21,7 @@ const assert = require('node:assert/strict');
const fs = require('fs');
const path = require('path');
const { runGsdTools, createTempProject, cleanup } = require('./helpers.cjs');
const { VALID_PROFILES } = require('../gsd-core/bin/lib/model-catalog.cjs');
// ─── Helpers for setting up minimal valid projects ────────────────────────────
@@ -351,6 +352,334 @@ describe('validate health command', () => {
);
});
// ─── Check 5c: model_profile "adaptive" + per-phase-type tier validation
// (W022, #2070) ──────────────────────────────────────────────────────────
// W004 must be sourced from the catalog's VALID_PROFILES (so "adaptive"
// is accepted), and a new W022 must fire for any `models.<phase_type>`
// entry whose value is not a valid tier (opus/sonnet/haiku/inherit),
// including non-string values, and for keys that are not valid phase types.
test('accepts adaptive model_profile as valid (no W004)', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'adaptive' })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W004'),
`Should not warn W004 for adaptive model_profile: ${JSON.stringify(output.warnings)}`
);
});
test('every catalog profile is accepted without W004', () => {
// Loop over VALID_PROFILES so this can never drift from the catalog.
for (const profile of VALID_PROFILES) {
const dir = createTempProject();
try {
writeMinimalProjectMd(dir);
writeMinimalRoadmap(dir, ['1']);
writeMinimalStateMd(dir);
fs.writeFileSync(
path.join(dir, '.planning', 'config.json'),
JSON.stringify({ model_profile: profile })
);
fs.mkdirSync(path.join(dir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', dir);
assert.ok(result.success, `Command failed for profile "${profile}": ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W004'),
`Should not warn W004 for catalog profile "${profile}": ${JSON.stringify(output.warnings)}`
);
} finally {
cleanup(dir);
}
}
});
test('still warns W004 for a genuinely invalid model_profile (regression guard)', () => {
// Don't let the adaptive-profile fix over-broaden into accepting everything.
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'bogus' })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
output.warnings.some(w => w.code === 'W004'),
`Expected W004 to still fire for a bogus model_profile: ${JSON.stringify(output.warnings)}`
);
});
test('warns W022 for an invalid tier value under models.<phase_type>', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: 'opuss' } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
const w022 = output.warnings.find(w => w.code === 'W022');
assert.ok(w022, `Expected W022 in warnings: ${JSON.stringify(output.warnings)}`);
const fullText = `${w022.message} ${w022.fix || ''}`;
assert.ok(fullText.includes('planning'), `Expected W022 message to name "planning": ${JSON.stringify(w022)}`);
assert.ok(fullText.includes('opuss'), `Expected W022 message to name "opuss": ${JSON.stringify(w022)}`);
assert.ok(
/ignor/i.test(fullText),
`Expected W022 message to indicate the value will be ignored: ${JSON.stringify(w022)}`
);
});
test('does not warn W022 or W004 for a valid tier value under models.<phase_type>', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: 'opus' } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not have W022 for valid tier "opus": ${JSON.stringify(output.warnings)}`
);
assert.ok(
!output.warnings.some(w => w.code === 'W004'),
`Should not have W004: ${JSON.stringify(output.warnings)}`
);
});
for (const tier of ['sonnet', 'haiku', 'inherit']) {
test(`does not warn W022 for the valid tier value "${tier}" under models.<phase_type>`, () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: tier } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not have W022 for valid tier "${tier}": ${JSON.stringify(output.warnings)}`
);
});
}
test('warns W022 for a numeric (non-string) models.<phase_type> value', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: 5 } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
output.warnings.some(w => w.code === 'W022'),
`Expected W022 for numeric models.planning value: ${JSON.stringify(output.warnings)}`
);
});
test('warns W022 for a null models.<phase_type> value', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: null } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
output.warnings.some(w => w.code === 'W022'),
`Expected W022 for null models.planning value: ${JSON.stringify(output.warnings)}`
);
});
test('warns W022 for an empty-string models.<phase_type> value', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { planning: '' } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
output.warnings.some(w => w.code === 'W022'),
`Expected W022 for empty-string models.planning value: ${JSON.stringify(output.warnings)}`
);
});
test('warns W022 for a mistyped models phase-type key, naming the unknown key', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: { plannning: 'opus' } })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
const w022 = output.warnings.find(w => w.code === 'W022');
assert.ok(w022, `Expected W022 in warnings: ${JSON.stringify(output.warnings)}`);
const fullText = `${w022.message} ${w022.fix || ''}`;
assert.ok(
fullText.includes('plannning'),
`Expected W022 message to name the unknown key "plannning": ${JSON.stringify(w022)}`
);
});
// ─── #2070 review finding 3: malformed top-level `models` is silently
// skipped today (src/verify.cts guards `typeof === 'object' && !Array`
// with no else branch, so an array/string/number/boolean `models` value
// produces ZERO diagnostics — the same undiagnosable-no-op class #2070
// targets). `models` absent or explicitly `null` must stay silent. ────────
for (const [label, malformedModels] of [
['an array', []],
['a string', 'opus'],
['a number', 5],
['a boolean', true],
]) {
test(`warns W022 for a top-level models value that is ${label} (not a plain object)`, () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: malformedModels })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
const w022 = output.warnings.find(w => w.code === 'W022');
assert.ok(
w022,
`Expected W022 when top-level models is ${label} (${JSON.stringify(malformedModels)}): ${JSON.stringify(output.warnings)}`
);
const fullText = `${w022.message} ${w022.fix || ''}`;
assert.ok(fullText.includes('models'), `Expected W022 message to mention "models": ${JSON.stringify(w022)}`);
assert.ok(/object/i.test(fullText), `Expected W022 message to indicate models must be an object: ${JSON.stringify(w022)}`);
assert.ok(/ignor/i.test(fullText), `Expected W022 message to indicate the value will be ignored: ${JSON.stringify(w022)}`);
});
}
test('does not warn W022 for an empty-object top-level models value (valid, just empty)', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: {} })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not warn W022 for an empty models object: ${JSON.stringify(output.warnings)}`
);
});
test('does not warn W022 for a top-level models value of null (explicit unset)', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced', models: null })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not warn W022 for models: null: ${JSON.stringify(output.warnings)}`
);
});
test('does not warn W022 when the models key is absent entirely', () => {
writeMinimalProjectMd(tmpDir);
writeMinimalRoadmap(tmpDir, ['1']);
writeMinimalStateMd(tmpDir);
fs.writeFileSync(
path.join(tmpDir, '.planning', 'config.json'),
JSON.stringify({ model_profile: 'balanced' })
);
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
const result = runGsdTools('validate health', tmpDir);
assert.ok(result.success, `Command failed: ${result.error}`);
const output = JSON.parse(result.output);
assert.ok(
!output.warnings.some(w => w.code === 'W022'),
`Should not warn W022 when models is absent: ${JSON.stringify(output.warnings)}`
);
});
// ─── Check 6: Phase directory naming (NN-name format) ─────────────────────
test('warns about incorrectly named phase directories', () => {