Commit Graph

5 Commits

Author SHA1 Message Date
Tom Boucher
96806003c5 fix(#3229): shared model catalog source of truth for agent profiles + runtime tier defaults (#3230)
* docs(adr): add ADR-0003 model catalog module

* fix(#3229): add shared model catalog as source of truth for agent profiles and runtime tier defaults

Research / design (ADR-0003):
- Existing drift came from 4 independent model truths:
  1. CJS model-profiles.cjs
  2. SDK config-query.ts stale copy (18 agents)
  3. settings-advanced.md runtime tier table
  4. session-runner Claude-only profile map
- New design: one machine-readable Model Catalog Module in sdk/shared/
  that both packages ship and consume.

Implementation:
- sdk/shared/model-catalog.json — canonical source of truth for:
  - full 33-agent registry
  - per-agent golden (quality) alias + balanced/budget aliases
  - adaptive derivation from routingTier
  - agent→phaseType map
  - agent→dynamic-routing default tier map
  - runtime tier defaults for all supported runtimes
- get-shit-done/bin/lib/model-catalog.cjs — CJS adapter over the catalog
- sdk/src/model-catalog.ts — SDK adapter over the same catalog
- CJS model-profiles.cjs now re-exports derived data from model-catalog.cjs
- SDK config-query.ts now re-exports MODEL_PROFILES/VALID_PROFILES from
  model-catalog.ts instead of maintaining its own list
- sdk/src/query/helpers.ts runtime list now comes from the catalog (fixes hermes drift)
- sdk/src/session-runner.ts Claude profile→model-id mapping now resolves via catalog
- docs/CONFIGURATION.md + settings-advanced.md runtime tables updated to match catalog

Behavior changes:
- resolve-model now covers every shipped agent file on disk (33 agents)
- unknown-agent fallback is profile-semantic, not hardcoded sonnet:
  quality→opus, budget→haiku, balanced/adaptive→sonnet, inherit→inherit
- Group B runtimes remain known runtimes but do not get built-in tier defaults

Tests (RED→GREEN):
- root tests: shipped agent files must equal MODEL_PROFILES keys
- sdk tests: shipped agent files must equal MODEL_PROFILES keys
- direct fix assertion: gsd-code-reviewer resolves to opus under quality with no unknown_agent
- runtime defaults parity test: settings-advanced.md + CONFIGURATION.md tables must match catalog
- helper tests: hermes included in SUPPORTED_RUNTIMES and getRuntimeConfigDir()

Closes #3229

* chore(changeset): update #3229 changeset pr field to 3230

* fix(ci): update inherit fallback expectations and inventory parity for model catalog
2026-05-08 21:25:37 -04:00
Tom Boucher
8393f4b355 fix(#2601): config-set-model-profile now accepts 'inherit' value (#2707)
VALID_PROFILES was derived solely from Object.keys(MODEL_PROFILES['gsd-planner']),
which only contained the named tiers (quality/balanced/budget/adaptive). The
cmdConfigSetModelProfile validator rejected 'inherit' even though the runtime has
supported it since #1829. Fix: append 'inherit' to VALID_PROFILES and handle it
in getAgentToModelMapForProfile so the agent→model table shows 'inherit' instead
of undefined.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 12:09:27 -04:00
Tibsfox
a2a49ecd14 feat(model-profiles): add adaptive preset with role-based model assignment (#1806)
Add a fourth model profile preset that assigns models by agent role:
opus for planning and debugging (reasoning-critical), sonnet for
execution and research (follows instructions), haiku for mapping and
checking (high volume, structured output).

This gives solo developers on paid API tiers a cost-effective middle
ground — quality where it matters most (planning) without overspending
on mechanical tasks (mapping, checking).

Per-agent overrides via model_overrides continue to take precedence
over any profile preset, including adaptive.

Closes #1713

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 17:03:17 -04:00
Tom Boucher
2703422be8 refactor(tests): standardize to node:assert/strict and t.after() per CONTRIBUTING.md (#1675)
* refactor(tests): standardize to node:assert/strict and t.after() per CONTRIBUTING.md

- Replace require('node:assert') with require('node:assert/strict') across
  all 73 test files to enforce strict equality (no type coercion)
- Replace try/finally cleanup blocks with t.after() hooks in core.test.cjs
  and hooks-opt-in.test.cjs per the test lifecycle standards
- Utility functions in codex-config and security-scan retain try/finally
  as that is appropriate for per-function resource guards, not lifecycle hooks

Closes #1674

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(tests): add --test-concurrency=4 to test runner for parallel file execution

Node.js --test-concurrency controls how many test files run as parallel child
processes. Set to 4 by default, configurable via TEST_CONCURRENCY env var.
Fixes tests at a known level rather than inheriting os.availableParallelism()
which varies across CI environments.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): allowlist verify.test.cjs in prompt-injection scanner

tests/verify.test.cjs uses <human>...</human> as GSD phase task-type
XML (meaning "a human should verify this step"), which matches the
scanner's fake-message-boundary pattern for LLM APIs. This is a
false positive — add it to the allowlist alongside the other test files
that legitimately contain injection-adjacent patterns.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 14:29:03 -04:00
Tom Boucher
93dc3d134f test: add coverage for model-profiles, template, profile-pipeline, profile-output (#1170)
New test files for 4 previously untested modules:

model-profiles.test.cjs (15 tests):
- MODEL_PROFILES data integrity (all agents, all profiles, valid aliases)
- VALID_PROFILES list validation
- getAgentToModelMapForProfile (balanced, budget, quality, agent count)
- formatAgentToModelMapAsTable (header, separator, column alignment)

template.test.cjs (11 tests):
- template select: minimal/standard/complex heuristics, fallback
- template fill: summary/plan/verification generation, --plan option
- Error paths: existing file, unknown type, missing phase

profile-pipeline.test.cjs (7 tests):
- scan-sessions: empty dir, synthetic project, multi-session
- extract-messages: user message extraction, meta/internal filtering
- profile-questionnaire: structure validation

profile-output.test.cjs (13 tests):
- PROFILING_QUESTIONS data (fields, options, uniqueness)
- CLAUDE_INSTRUCTIONS coverage (dimensions, instruction mapping)
- write-profile: analysis JSON → USER-PROFILE.md
- generate-claude-md: --auto generation, --force protection
- generate-dev-preferences: analysis → preferences command

Test count: 744 → 798 (+54 new tests, 0 failures)
2026-03-18 09:55:55 -06:00