Files
msd-core/tests/config-schema-sdk-parity.test.cjs
Tom Boucher 6a5fa59129 feat(3081): auto-trim review prompts for small-context model reviewers (#3708)
* feat(3081): auto-trim review prompts for small-context model reviewers

Adds review.max_prompt_tokens and review.max_prompt_tokens_per_reviewer
config keys. When configured, the /gsd-review workflow deterministically
trims the assembled prompt before sending to each reviewer (drop CONTEXT
→ RESEARCH → REQUIREMENTS; head-shrink PROJECT.md; tail-truncate PLANs
proportionally; reserve disclosure-note tokens upfront). Trim metadata
is recorded in REVIEWS.md frontmatter. Reviewer is skipped with a
warning if even the minimum review set exceeds the budget.

Closes #3081

* fix(3081): register prompt-budget in SDK query registry and update inventory manifest

review.md references `gsd-sdk query prompt-budget` at three call sites, but the
command had no handler in the SDK registry — failing the registry-integration
drift-guard test on all 6 CI matrix legs. Added a native TypeScript SDK handler
(sdk/src/query/prompt-budget.ts) that ports the applyBudget logic from the CJS
module, registered it in DOMAIN_STATIC_CATALOG, and regenerated
docs/INVENTORY-MANIFEST.json to include the new cli_modules/prompt-budget.cjs entry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(3081): bump ws to 8.20.1 and allowlist prompt-budget sibling pair

Two additional CI failures after the registry fix:

1. ws moderate CVE (GHSA-58qx-3vcg-4xpx, uninitialized memory disclosure):
   The advisory covers ws >=8.0.0 <8.20.1. Both root and sdk/package.json
   pinned ^8.20.0 which resolved to 8.20.0. Bumped both to 8.20.1 to clear
   the npm audit drift-guard test (bug-3588-npm-audit-clean.test.cjs).

2. lint-shared-module-handsync detected the new prompt-budget.ts / prompt-budget.cjs
   sibling pair without an allowlist entry. Added a cooperatingSiblings entry
   to scripts/shared-module-handsync-allowlist.json with classification and
   justification matching the established pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(3081): align prompt-budget skip semantics across CJS and SDK dispatch paths

Replace brittle `[ $EXIT -eq 2 ]` guards with `[ $EXIT -ne 0 ]` in all three
local-reviewer blocks (Ollama, LM Studio, llama.cpp) in workflows/review.md.
Any non-zero exit from prompt-budget now triggers a skip with a descriptive
warning — exit 2/11 prints "budget too small", any other non-zero prints
"unexpected exit code". This ensures the SDK bridge dispatch path (exit 11
via GSDError(Blocked)) triggers the same skip as the CJS path (exit 2).

The SDK handler (sdk/src/query/prompt-budget.ts) already writes both metadata
and prompt files before throwing, so no change needed there.

The Ollama block also gains the missing OLLAMA_SKIP guard so the reviewer
invocation is actually skipped (previously the block only suppressed the
OLLAMA_PROMPT_FILE update but still ran the curl invocation).

SDK integration path (hardFailed via GSDError(Blocked) → exit 11) is covered
by handler unit tests in tests/prompt-budget.test.cjs; no gsd-sdk-*.test.cjs
exercising the full bridge dispatch for this command exists yet — that gap
remains and is documented here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix prompt-budget trim ordering and review guard follow-ups

* perf: optimize prompt-budget and dedup reviewer trim workflow

* fix(3708): drop source-grep theater tests to satisfy lint-no-source-grep

All four test files added in commit 2df566ed were pure source-grep theater:
they read .cjs / .ts / .md source files and asserted that specific string
literals were present or absent. None exercised runtime behaviour.

Deleted:
- tests/gsd-tools-memory-optimizer.test.cjs   — 7 includes() on gsd-tools.cjs
- tests/prompt-budget-hotpath-optimizer.test.cjs — includes() on prompt-budget.cjs + .ts
- tests/prompt-budget-io-optimizer.test.cjs   — includes() on prompt-budget.ts + gsd-tools.cjs
- tests/review-workflow-budget-dedup.test.cjs — includes() on review.md

Behavioural coverage for the prompt-budget feature already exists in
tests/prompt-budget.test.cjs and tests/prompt-budget-cli.test.cjs (also
added by this PR). No replacement tests needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(3708): correct budget-pressure threshold and minSet accounting

Two bugs in applyBudget caused premature trimming and false hard-fails:

1. UNNEEDED_TRIM: budgetUnderPressure compared baseTokens against
   effectiveBudget - NOTE_RESERVE_TOKENS, triggering trim pressure 80
   tokens before the budget was actually exceeded. Fix: compare against
   effectiveBudget directly; NOTE_RESERVE_TOKENS are still reserved in
   contentBudget once real pressure is confirmed.

2. FALSE_HARDFAIL: minSet included NOTE_RESERVE_TOKENS unconditionally,
   treating the note as mandatory even when no trim would occur and no
   note would be injected. Fix: exclude NOTE_RESERVE_TOKENS from minSet;
   a prompt that fits untrimmed needs no note and must not hard-fail.

Both fixes applied in CJS and TypeScript implementations. Two regression
tests added (cycles 11 and 12) that reproduce each case behaviorally.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 23:13:09 -04:00

148 lines
6.4 KiB
JavaScript

'use strict';
// allow-test-rule: structural source assertion — verifies that sdk/src/query/config-schema.ts
// is a re-export shell (no inline literals) rather than a redeclared Set. Runtime/IR comparison
// cannot distinguish a re-export from a redeclared `new Set([...])` with identical contents
// because both would deep-equal the manifest set; only source-shape inspection catches drift
// back to inline literals. See test "SDK config-schema.ts re-exports from configuration module".
/**
* Manifest-as-source-of-truth guard (Phase 2, Cycle 5, #3536).
*
* Prior to Cycle 5, the CJS and SDK schema files each had independent inline
* literals. This test existed to prevent drift between them. After Cycle 5,
* BOTH sides derive their data from sdk/shared/config-schema.manifest.json,
* so there is nothing to drift — but the guard still serves a purpose:
*
* 1. Confirm that VALID_CONFIG_KEYS loaded at runtime from config-schema.cjs
* exactly matches the manifest's validKeys array.
* 2. Confirm that VALID_CONFIG_KEYS exported from sdk/src/query/config-schema.ts
* (via sdk/dist/) equals the same manifest.
* 3. Confirm that DYNAMIC_KEY_PATTERNS from config-schema.cjs have .source
* fields matching the manifest's dynamicKeyPatterns.
* 4. Confirm RUNTIME_STATE_KEYS from config-schema.cjs matches the manifest.
*
* This ensures neither side has accidentally disconnected from the manifest
* (e.g. reverted to inline literals or switched to a different data source).
*/
const { test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const ROOT = path.resolve(__dirname, '..');
const MANIFEST_PATH = path.join(ROOT, 'sdk', 'shared', 'config-schema.manifest.json');
const manifest = JSON.parse(fs.readFileSync(MANIFEST_PATH, 'utf8'));
const manifestValidKeys = new Set(manifest.validKeys);
const manifestRuntimeKeys = new Set(manifest.runtimeStateKeys);
const manifestPatternSources = manifest.dynamicKeyPatterns.map((p) => p.source);
const {
VALID_CONFIG_KEYS: CJS_KEYS,
RUNTIME_STATE_KEYS: CJS_RUNTIME_KEYS,
DYNAMIC_KEY_PATTERNS: CJS_PATTERNS,
} = require('../get-shit-done/bin/lib/config-schema.cjs');
// ─── CJS side: verify manifest-sourced values ─────────────────────────────
test('CJS VALID_CONFIG_KEYS matches manifest validKeys exactly', () => {
const missingInCjs = [...manifestValidKeys].filter((k) => !CJS_KEYS.has(k));
const extraInCjs = [...CJS_KEYS].filter((k) => !manifestValidKeys.has(k));
assert.deepStrictEqual(
missingInCjs,
[],
'Manifest keys missing from CJS VALID_CONFIG_KEYS:\n' +
missingInCjs.map((k) => ' ' + k).join('\n'),
);
assert.deepStrictEqual(
extraInCjs,
[],
'CJS VALID_CONFIG_KEYS has keys not in manifest:\n' +
extraInCjs.map((k) => ' ' + k).join('\n'),
);
});
test('CJS RUNTIME_STATE_KEYS matches manifest runtimeStateKeys exactly', () => {
const missingInCjs = [...manifestRuntimeKeys].filter((k) => !CJS_RUNTIME_KEYS.has(k));
const extraInCjs = [...CJS_RUNTIME_KEYS].filter((k) => !manifestRuntimeKeys.has(k));
assert.deepStrictEqual(missingInCjs, [], 'Manifest runtime keys missing from CJS RUNTIME_STATE_KEYS');
assert.deepStrictEqual(extraInCjs, [], 'CJS RUNTIME_STATE_KEYS has keys not in manifest');
});
test('CJS DYNAMIC_KEY_PATTERNS .source fields match manifest dynamicKeyPatterns', () => {
assert.strictEqual(
CJS_PATTERNS.length,
manifestPatternSources.length,
`CJS has ${CJS_PATTERNS.length} patterns but manifest has ${manifestPatternSources.length}`,
);
for (let i = 0; i < manifestPatternSources.length; i++) {
const expected = manifestPatternSources[i];
const actual = CJS_PATTERNS[i].source;
assert.strictEqual(
actual,
expected,
`CJS pattern[${i}].source mismatch: expected "${expected}", got "${actual}"`,
);
}
});
// ─── SDK side: verify config-schema.ts re-exports from configuration module ─
test('SDK config-schema.ts re-exports from configuration module (not inline literals)', () => {
const SDK_SCHEMA_PATH = path.join(ROOT, 'sdk', 'src', 'query', 'config-schema.ts');
const src = fs.readFileSync(SDK_SCHEMA_PATH, 'utf8');
// After Cycle 5, the file must NOT contain inline key literals.
// It should import/re-export from '../configuration/index.js'.
assert.ok(
src.includes("from '../configuration/index.js'"),
'sdk/src/query/config-schema.ts must re-export from ../configuration/index.js (not inline literals)',
);
// Must NOT contain a standalone new Set([...]) block with key literals.
// A minimal check: the file should not define VALID_CONFIG_KEYS as a Set literal.
assert.ok(
!src.includes("new Set([\n 'mode'") && !src.includes("new Set(['mode'"),
'sdk/src/query/config-schema.ts must not contain an inline VALID_CONFIG_KEYS Set literal',
);
});
// ─── Cross-check: CJS equals SDK via manifest ─────────────────────────────
test('#2653 — CJS and SDK both source from the same manifest (set equality via manifest)', () => {
// Since both sides derive from sdk/shared/config-schema.manifest.json,
// the CJS runtime set must equal the manifest set (verified above).
// This test is the explicit statement of the invariant for audit purposes.
const cjsKeysSorted = [...CJS_KEYS].sort();
const manifestKeysSorted = [...manifestValidKeys].sort();
assert.deepStrictEqual(
cjsKeysSorted,
manifestKeysSorted,
'CJS VALID_CONFIG_KEYS must equal manifest validKeys — both sides source from the manifest',
);
});
test('#2653 — CJS DYNAMIC_KEY_PATTERNS test functions work correctly', () => {
// Verify that each pattern's test() function (reconstructed from manifest source)
// correctly accepts sample keys and rejects non-matching ones.
const samples = [
['agent_skills.gsd-planner', 0],
['review.models.claude', 1],
['features.some_feature', 2],
['claude_md_assembly.blocks.intro', 3],
['model_profile_overrides.codex.opus', 4],
['models.planning', 5],
['dynamic_routing.enabled', 6],
['model_overrides.my-agent', 7],
['review.max_prompt_tokens_per_reviewer.ollama', 8],
];
for (const [key, idx] of samples) {
assert.ok(
CJS_PATTERNS[idx].test(key),
`CJS pattern[${idx}] must accept "${key}"`,
);
}
});