Files
msd-core/tests/helpers.cjs
Tom Boucher 6a5fa59129 feat(3081): auto-trim review prompts for small-context model reviewers (#3708)
* feat(3081): auto-trim review prompts for small-context model reviewers

Adds review.max_prompt_tokens and review.max_prompt_tokens_per_reviewer
config keys. When configured, the /gsd-review workflow deterministically
trims the assembled prompt before sending to each reviewer (drop CONTEXT
→ RESEARCH → REQUIREMENTS; head-shrink PROJECT.md; tail-truncate PLANs
proportionally; reserve disclosure-note tokens upfront). Trim metadata
is recorded in REVIEWS.md frontmatter. Reviewer is skipped with a
warning if even the minimum review set exceeds the budget.

Closes #3081

* fix(3081): register prompt-budget in SDK query registry and update inventory manifest

review.md references `gsd-sdk query prompt-budget` at three call sites, but the
command had no handler in the SDK registry — failing the registry-integration
drift-guard test on all 6 CI matrix legs. Added a native TypeScript SDK handler
(sdk/src/query/prompt-budget.ts) that ports the applyBudget logic from the CJS
module, registered it in DOMAIN_STATIC_CATALOG, and regenerated
docs/INVENTORY-MANIFEST.json to include the new cli_modules/prompt-budget.cjs entry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(3081): bump ws to 8.20.1 and allowlist prompt-budget sibling pair

Two additional CI failures after the registry fix:

1. ws moderate CVE (GHSA-58qx-3vcg-4xpx, uninitialized memory disclosure):
   The advisory covers ws >=8.0.0 <8.20.1. Both root and sdk/package.json
   pinned ^8.20.0 which resolved to 8.20.0. Bumped both to 8.20.1 to clear
   the npm audit drift-guard test (bug-3588-npm-audit-clean.test.cjs).

2. lint-shared-module-handsync detected the new prompt-budget.ts / prompt-budget.cjs
   sibling pair without an allowlist entry. Added a cooperatingSiblings entry
   to scripts/shared-module-handsync-allowlist.json with classification and
   justification matching the established pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(3081): align prompt-budget skip semantics across CJS and SDK dispatch paths

Replace brittle `[ $EXIT -eq 2 ]` guards with `[ $EXIT -ne 0 ]` in all three
local-reviewer blocks (Ollama, LM Studio, llama.cpp) in workflows/review.md.
Any non-zero exit from prompt-budget now triggers a skip with a descriptive
warning — exit 2/11 prints "budget too small", any other non-zero prints
"unexpected exit code". This ensures the SDK bridge dispatch path (exit 11
via GSDError(Blocked)) triggers the same skip as the CJS path (exit 2).

The SDK handler (sdk/src/query/prompt-budget.ts) already writes both metadata
and prompt files before throwing, so no change needed there.

The Ollama block also gains the missing OLLAMA_SKIP guard so the reviewer
invocation is actually skipped (previously the block only suppressed the
OLLAMA_PROMPT_FILE update but still ran the curl invocation).

SDK integration path (hardFailed via GSDError(Blocked) → exit 11) is covered
by handler unit tests in tests/prompt-budget.test.cjs; no gsd-sdk-*.test.cjs
exercising the full bridge dispatch for this command exists yet — that gap
remains and is documented here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix prompt-budget trim ordering and review guard follow-ups

* perf: optimize prompt-budget and dedup reviewer trim workflow

* fix(3708): drop source-grep theater tests to satisfy lint-no-source-grep

All four test files added in commit 2df566ed were pure source-grep theater:
they read .cjs / .ts / .md source files and asserted that specific string
literals were present or absent. None exercised runtime behaviour.

Deleted:
- tests/gsd-tools-memory-optimizer.test.cjs   — 7 includes() on gsd-tools.cjs
- tests/prompt-budget-hotpath-optimizer.test.cjs — includes() on prompt-budget.cjs + .ts
- tests/prompt-budget-io-optimizer.test.cjs   — includes() on prompt-budget.ts + gsd-tools.cjs
- tests/review-workflow-budget-dedup.test.cjs — includes() on review.md

Behavioural coverage for the prompt-budget feature already exists in
tests/prompt-budget.test.cjs and tests/prompt-budget-cli.test.cjs (also
added by this PR). No replacement tests needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(3708): correct budget-pressure threshold and minSet accounting

Two bugs in applyBudget caused premature trimming and false hard-fails:

1. UNNEEDED_TRIM: budgetUnderPressure compared baseTokens against
   effectiveBudget - NOTE_RESERVE_TOKENS, triggering trim pressure 80
   tokens before the budget was actually exceeded. Fix: compare against
   effectiveBudget directly; NOTE_RESERVE_TOKENS are still reserved in
   contentBudget once real pressure is confirmed.

2. FALSE_HARDFAIL: minSet included NOTE_RESERVE_TOKENS unconditionally,
   treating the note as mandatory even when no trim would occur and no
   note would be injected. Fix: exclude NOTE_RESERVE_TOKENS from minSet;
   a prompt that fits untrimmed needs no note and must not hard-fail.

Both fixes applied in CJS and TypeScript implementations. Two regression
tests added (cycles 11 and 12) that reproduce each case behaviorally.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 23:13:09 -04:00

260 lines
9.8 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* GSD Tools Test Helpers
*/
const { execSync, execFileSync } = require('child_process');
const fs = require('fs');
const path = require('path');
const TOOLS_PATH = path.join(__dirname, '..', 'get-shit-done', 'bin', 'gsd-tools.cjs');
const TEST_ENV_BASE = {
GSD_SESSION_KEY: '',
CODEX_THREAD_ID: '',
CLAUDE_SESSION_ID: '',
CLAUDE_CODE_SSE_PORT: '',
OPENCODE_SESSION_ID: '',
GEMINI_SESSION_ID: '',
CURSOR_SESSION_ID: '',
WINDSURF_SESSION_ID: '',
TERM_SESSION_ID: '',
WT_SESSION: '',
TMUX_PANE: '',
ZELLIJ_SESSION_NAME: '',
TTY: '',
SSH_TTY: '',
};
/**
* Run gsd-tools command.
*
* @param {string|string[]} args - Command string (shell-interpreted) or array
* of arguments (shell-bypassed via execFileSync, safe for JSON and dollar signs).
* @param {string} cwd - Working directory.
* @param {object} [env] - Optional env overrides merged on top of process.env.
* Pass { HOME: cwd } to sandbox ~/.gsd/ lookups in tests that assert concrete
* config values that could be overridden by a developer's defaults.json.
*/
function runGsdTools(args, cwd = process.cwd(), env = {}) {
try {
let result;
const childEnv = { ...process.env, ...TEST_ENV_BASE, ...env };
if (Array.isArray(args)) {
result = execFileSync(process.execPath, [TOOLS_PATH, ...args], {
cwd,
encoding: 'utf-8',
stdio: ['pipe', 'pipe', 'pipe'],
env: childEnv,
});
} else {
// Split shell-style string into argv, stripping surrounding quotes, so we
// can invoke execFileSync with process.execPath instead of relying on
// `node` being on PATH (it isn't in Claude Code shell sessions).
// Apply shell-style quote removal: strip surrounding quotes from quoted
// sequences anywhere in a token (handles both "foo bar" and --"foo bar").
const argv = (args.match(/(?:[^\s"']+|"[^"]*"|'[^']*')+/g) || [])
.map(t => t.replace(/"([^"]*)"/g, '$1').replace(/'([^']*)'/g, '$1'));
result = execFileSync(process.execPath, [TOOLS_PATH, ...argv], {
cwd,
encoding: 'utf-8',
stdio: ['pipe', 'pipe', 'pipe'],
env: childEnv,
});
}
return { success: true, output: result.trim(), exitCode: 0 };
} catch (err) {
return {
success: false,
output: err.stdout?.toString().trim() || '',
error: err.stderr?.toString().trim() || err.message,
exitCode: err.status ?? 1,
};
}
}
// Create a bare temp directory (no .planning/ structure)
function createTempDir(prefix = 'gsd-test-') {
return fs.mkdtempSync(path.join(require('os').tmpdir(), prefix));
}
// Create temp directory structure
function createTempProject(prefix = 'gsd-test-') {
const tmpDir = fs.mkdtempSync(path.join(require('os').tmpdir(), prefix));
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases'), { recursive: true });
return tmpDir;
}
// Create temp directory with initialized git repo and at least one commit
function createTempGitProject(prefix = 'gsd-test-') {
const tmpDir = fs.mkdtempSync(path.join(require('os').tmpdir(), prefix));
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases'), { recursive: true });
execSync('git init', { cwd: tmpDir, stdio: 'pipe' });
execSync('git config user.email "test@test.com"', { cwd: tmpDir, stdio: 'pipe' });
execSync('git config user.name "Test"', { cwd: tmpDir, stdio: 'pipe' });
execSync('git config commit.gpgsign false', { cwd: tmpDir, stdio: 'pipe' });
fs.writeFileSync(
path.join(tmpDir, '.planning', 'PROJECT.md'),
'# Project\n\nTest project.\n'
);
execSync('git add -A', { cwd: tmpDir, stdio: 'pipe' });
execSync('git commit -m "initial commit"', { cwd: tmpDir, stdio: 'pipe' });
return tmpDir;
}
function cleanup(tmpDir) {
if (typeof tmpDir !== 'string' || tmpDir.length === 0) return;
const target = path.resolve(tmpDir);
const cwd = path.resolve(process.cwd());
if (cwd === target || cwd.startsWith(`${target}${path.sep}`)) {
// Windows cannot remove a directory that is the current working directory.
process.chdir(path.dirname(target));
}
// maxRetries/retryDelay absorbs transient Windows EBUSY where AV scanners,
// file-indexers, or just-exited child processes still hold handles when
// teardown runs. On POSIX the retry loop is a no-op (rmSync succeeds first try).
// Budget: 20 × 250ms = 5s total — Windows Defender's deferred scan can hold
// newly-written files for several seconds on cold runners.
fs.rmSync(target, { recursive: true, force: true, maxRetries: 20, retryDelay: 250 });
}
/**
* Parse a Markdown frontmatter block into a flat key→value map.
*
* Handles the YAML scalar forms emitted by the install converters:
* key: "json-encoded value" → JSON.parse
* key: 'value with ''escape'' → strip quotes, unescape ''
* key: bare value → trimmed string
*
* Multi-line and block scalars are out of scope — every converter in
* `bin/install.js` emits single-line scalars only. Throws if the content
* has no closed `---` block so a regression in the emitter shape fails
* loudly rather than silently returning {}.
*
* Tests use this helper instead of `result.includes('key: value')` to
* follow the project's "tests parse, never grep" convention.
*
* @param {string} content - Full file content beginning with `---`.
* @returns {Record<string, string>} Map of frontmatter keys to decoded values.
*/
function parseFrontmatter(content) {
if (!content.startsWith('---')) {
throw new Error(`parseFrontmatter: content must start with '---', got: ${content.slice(0, 40)}`);
}
// CRLF tolerance: a Windows-authored file split on `\n` would leave a
// trailing `\r` on every line, making `lines[i] === '---'` fail to
// recognize delimiters. Same goes for whitespace-padded delimiter lines.
// Normalize via a CRLF-aware split + trimmed comparison.
const lines = content.split(/\r?\n/);
let openIdx = -1;
let closeIdx = -1;
for (let i = 0; i < lines.length; i += 1) {
if (lines[i].trim() === '---') {
if (openIdx === -1) openIdx = i;
else { closeIdx = i; break; }
}
}
if (openIdx === -1 || closeIdx === -1) {
throw new Error('parseFrontmatter: no closed --- block');
}
const fields = {};
for (const line of lines.slice(openIdx + 1, closeIdx)) {
const match = line.match(/^([A-Za-z][A-Za-z0-9_-]*):\s*(.*)$/);
if (!match) continue; // skip block-list items, blank lines, comments
const [, key, rawValue] = match;
const value = rawValue.trim();
if (value.startsWith('"') && value.endsWith('"') && value.length >= 2) {
fields[key] = JSON.parse(value);
} else if (value.startsWith("'") && value.endsWith("'") && value.length >= 2) {
fields[key] = value.slice(1, -1).replace(/''/g, "'");
} else {
fields[key] = value;
}
}
return fields;
}
// #3026 CR: shared `--help` output check used by bug-1818 + bug-3019 tests.
// Render-on-help shape is `Usage: gsd-tools …\nCommands: …` — both lines
// must be present; structural test, not prose substring matching.
function isUsageOutput(text) {
return /Usage:\s*gsd-tools/.test(text) && /Commands:/.test(text);
}
/**
* Run `fn` with console.log/warn/error captured, returning {stdout, stderr}
* with ANSI colors stripped. Re-throws any exception fn threw AFTER restoring
* the real console so the caller's assertion path sees the failure (without
* this, a fn that crashes before printing would falsely pass !hasReady-style
* assertions). #2775 CR follow-up established this exact contract.
*
* Previously duplicated in bug-2775, bug-2829, bug-3033, bug-3211, bug-3231,
* bug-3359, and installer-migration-install-integration.
*/
function captureConsole(fn) {
const stdout = [];
const stderr = [];
const origLog = console.log;
const origWarn = console.warn;
const origError = console.error;
console.log = (...a) => stdout.push(a.join(' '));
console.warn = (...a) => stderr.push(a.join(' '));
console.error = (...a) => stderr.push(a.join(' '));
let threw = null;
try {
fn();
} catch (e) {
threw = e;
} finally {
console.log = origLog;
console.warn = origWarn;
console.error = origError;
}
if (threw) throw threw;
const strip = (s) => s.replace(/\x1b\[[0-9;]*m/g, '');
return {
stdout: stdout.map(strip).join('\n'),
stderr: stderr.map(strip).join('\n'),
};
}
/**
* Normalize platform path separators to POSIX forward slashes. Use for
* cross-platform path comparisons in test assertions where the runtime
* emits the platform-native separator (\ on Windows) but the test
* fixture or expected literal is POSIX. Returns the input unchanged if
* null/undefined so it composes safely with optional chaining.
*/
function toPosixPath(p) {
return p == null ? p : p.split(path.sep).join('/');
}
/**
* Run an npm command via execFileSync with cross-platform portability.
*
* Handles the Windows `npm.cmd` vs POSIX `npm` distinction and the
* `shell: true` requirement on Windows so tests do not need to
* re-implement platform detection inline.
*
* @param {string[]} args - npm subcommand and flags (e.g. ['pack', '--pack-destination', dir]).
* @param {object} [options] - execFileSync options merged with platform defaults.
* `cwd`, `encoding`, `timeout`, and `env` are the commonly overridden keys.
* @returns {string} trimmed stdout string (encoding: 'utf-8').
* @throws {Error} re-throws the execFileSync error on non-zero exit so callers
* get the full stderr in the error message.
*/
function runNpm(args, options = {}) {
const isWindows = process.platform === 'win32';
const npmCmd = isWindows ? 'npm.cmd' : 'npm';
const defaults = {
encoding: 'utf-8',
shell: isWindows,
timeout: 180000,
};
return execFileSync(npmCmd, args, { ...defaults, ...options }).trim();
}
module.exports = { runGsdTools, createTempDir, createTempProject, createTempGitProject, cleanup, parseFrontmatter, isUsageOutput, captureConsole, toPosixPath, runNpm, TOOLS_PATH };