fix(#3691): let every reviewer lane take a prompt cap, and make the documented global resolve (#3832)
* test(#3691): failing-first coverage for the reviewer prompt budget No prompt cap can reach any CLI reviewer lane, by any configuration. Two independent defects compound: all nine `transport: spawn` lanes declare `promptBudgetKey: null`, so `budgetFor` returns on its first line; and the documented global `review.max_prompt_tokens` is advertised in the schema manifest but declared nowhere, so the resolver never materializes it and `budgetFor`'s fallback is dead code. Adds to tests/reviewer-config-federation.test.cjs, which already owns the per-reviewer budget config-set/config-get idiom: - a CLI lane inherits the global cap (RED: reports null) - an http lane with the -1 sentinel inherits the global cap (RED: reports null) - the resolved review surface carries max_prompt_tokens at all (RED: absent) - per-lane overrides the global on a CLI lane - the sentinel boundary: -1 inherits, 0 means do-not-trim and must NOT read as unset, 1 is the smallest real budget — the regression budgetFor's own comment warns about - anti-tightening pins that must stay green: an empty config leaves every lane null, the three existing budgeted lanes are unchanged, and config-set still rejects a per-reviewer key naming something that is not a declared lane - a fast-check property over the resolution contract itself, with -1, 0 and non-finite inputs generated explicitly rather than left to chance Every row was reproduced by hand against the real CLI before being written, so the RED/GREEN split is observed rather than predicted. Refs #3691 * fix(#3691): let every reviewer lane take a prompt cap, and make the global resolve No prompt cap could reach any CLI reviewer lane, by any configuration. Two independent defects compounded. The nine spawn-transport lanes — claude, coderabbit, antigravity, cursor, gemini, codex, kimi-code, opencode, qwen — declared `promptBudgetKey: null`, so `budgetFor` returned on its first line and `review-lane plan` reported `promptBudget: null` no matter what was configured. Each now declares `review.max_prompt_tokens_per_reviewer.<slug>` with the same `-1`-is-unset sentinel the three local-server lanes already use. Separately, the central `review.max_prompt_tokens` was listed in the schema manifest's validKeys and documented as a supported setting, but declared nowhere — the resolved surface is built from capability declarations plus the defaults manifest, and neither carried it. `configGet` returned undefined and `budgetFor`'s documented fallback was dead code. It is now declared with a `null` default, exactly as docs/CONFIGURATION.md already specified, so the default behavior is unchanged: nothing configured means nothing trims. Two things the diagnosis had not predicted, found and fixed while implementing: - `REVIEWER_LANES` in src/review-lane-descriptor.cts is a second, hardcoded registration site that `mergeReviewerLanes` prefers over the capability registry on a slug collision. Editing only the capability files left every CLI lane still null. Both sites now agree. - The generated `gsd-core/bin/lib/capability-registry.cjs` was stale and masked the capability edits; regenerated with `npm run gen:capability-registry` rather than hand-edited. docs/CONFIGURATION.md said "Only lanes that declare a budget key accept one — today ollama, lm_studio and llama_cpp". That is false as of this change and is corrected rather than left to rot. The trim-versus-refuse question the issue raises is deliberately not taken up here: the refusal path already exists for the case that matters — a reviewer whose minimum set exceeds its budget is skipped rather than sent a misleading prompt — and trimming above that floor is the documented, shipped design of the feature. Changing it would alter behavior for the three lanes that already work, which is not what the issue asks for. Fixes #3691 * fix(#3691): document the new global and narrow an invariant this change obsoleted The full suite surfaced two consequences of giving every CLI lane a budget key. `review.max_prompt_tokens` entered CONFIG_DEFAULTS without a matching entry in the planning-config reference, which config-field-docs guards. Documented, including the sentinel semantics a reader needs: a per-lane value overrides the global, `-1` means unset and inherits it, and `0` means "do not trim that lane" and is not unset. The #2797 federation guard asserted that "a lane with no model flag and no host owns no config keys". That held only because budget keys existed solely on the three local-server lanes, all of which have hosts. A lane can now legitimately own a config key for a third reason, so qwen tripped it. The assertion is narrowed rather than weakened: such a lane must still own no model key and no host key, and may own at most its own `review.max_prompt_tokens_per_reviewer.<slug>` — never another lane's. That is strictly more specific in the dimensions that still matter. Proven to still bite: hypothetically giving qwen a `review.models.qwen` key fails it with `model/host: review.models.qwen`. The name and comment cite #3691 for why the premise changed, so a reader sees a deliberate narrowing, not erosion. Checked the sibling assertions in that describe block; the other three do not rest on the obsolete premise and are untouched. Refs #3691 * fix(#3685): port the write-flag content-change contract to its three sibling sites #3685 fixed `phase complete`'s `roadmap_updated` / `state_updated`, which reported `fs.existsSync(path)` rather than whether the transaction wrote anything. Three sibling sites carried the identical defect and are ported here. - `cmdPhaseRemove` reported `roadmap_updated: true`, hardcoded. `updateRoadmapAfterPhaseRemoval` now returns whether the content changed and the flag reports it. #2640/#2974 already fixed `state_updated` at this same call site and left this one behind, so the correct shape was adjacent. - `cmdMilestoneComplete` reported `state_updated: fs.existsSync(statePath)` — byte-identical to #3685's bug in a different command. - `cmdMilestoneComplete` reported `milestones_updated: true`, hardcoded, never consulting the MILESTONES.md write. `gsd-core/workflows/remove-phase.md:100` extracts `roadmap_updated` for display and never branches on it, so the flip from always-true to content-based changes no workflow behavior. Verified by reading the step, not assumed. One trap found while implementing: the obvious in-memory `finalContent !== originalStateContent` comparison — copying `cmdPhaseComplete`'s shipped shape verbatim — gives a FALSE POSITIVE for milestone completion. `platformWriteSync` normalizes Markdown at write time, and the milestone-closure transform regenerates `## Current Position` fresh on every call, so its pre-normalize output always differs from the already-normalized file on disk even when the persisted bytes are identical. The comparison is therefore made against the post-write on-disk content. `cmdPhaseComplete`'s own comparisons are left untouched — their repeat-no-op tests pass, so they are not exposed to this artifact. `milestones_updated` has no reachable no-op: the MILESTONES.md write unconditionally appends an entry every call. Only the true direction is pinned, documented inline rather than faked with a passing test. Refs #3685 * fix(#3685): compare write-flag content through the writer's own normalizer An independent reviewer disproved a claim made while porting #3685's contract to its sibling sites: that `cmdPhaseComplete`'s comparisons were not exposed to the Markdown-normalization artifact already diagnosed in `cmdMilestoneComplete`. `platformWriteSync` normalizes on write — CRLF stripped, blank-line runs collapsed, a blank line inserted after a heading, a single trailing newline enforced. Every flag that compares the PRE-normalization in-memory string against the on-disk pre-image can therefore report a change when the persisted bytes are identical. `cmdMilestoneComplete` had been worked around by re-reading the file after the write; the other sites compared raw strings. All of them now go through one exported seam, `contentChangedAfterNormalize(filePath, before, after)`, which normalizes both sides exactly as the writer does. That removes the extra disk read the milestone workaround needed, and makes the sites agree by construction rather than by four independent implementations of one rule — the divergence the repo names as an anti-pattern. Reachability, stated precisely rather than uniformly: the seam is load-bearing at `cmdPhaseComplete`'s `roadmapUpdated`, `requirementsUpdated` and `stateUpdated`, where section-rewrite logic genuinely regenerates content into a different-but-normalization-equivalent shape. At `updateRoadmapAfterPhaseRemoval` it is defense-in-depth: the no-match branch never reassigns `content`, so the raw comparison was already correct there. The first analysis claimed the reverse; this is the corrected finding. Also fixes an unsound test premise the remote suite caught. The byte-identity precondition in `roadmap_updated is false when ROADMAP.md comes out byte-identical` asserted against a hand-authored, un-normalized fixture — so the very first write reformatted it and the file could not come back identical. The fixture is now written already-normalized, so the assertion compares a normalized pre-image against a normalized post-image and still fails if the flag regresses to a hardcoded `true`. Not platform-specific; it reproduces on macOS too, and the earlier local check simply never exercised it. The sibling true-direction and milestone tests were checked for the same premise and do not share it — they assert `notEqual`, or compare two post-write states produced through the same normalizing seam. Refs #3685 * chore(changeset): backfill PR number for #3691 fragment --------- Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
@@ -515,6 +515,66 @@ describe('milestone complete command', () => {
|
||||
assert.strictEqual(output.plans, 0);
|
||||
assert.strictEqual(output.tasks, 0);
|
||||
});
|
||||
|
||||
// #3685: state_updated/milestones_updated were reported via
|
||||
// fs.existsSync(statePath) and a hardcoded `true` respectively — neither
|
||||
// consulted whether the file's content actually changed in the
|
||||
// transaction. Both now mirror requirements_updated's diff-tracking
|
||||
// contract. Clock is pinned (GSD_TEST_MODE + GSD_NOW_MS) because
|
||||
// syncStateFrontmatter stamps a millisecond-resolution `last_updated:`
|
||||
// field on every STATE.md write — an unpinned second run would genuinely
|
||||
// differ by that timestamp alone, masking the no-op this test needs to
|
||||
// observe.
|
||||
describe('write-flag content-change contract (#3685)', () => {
|
||||
const PINNED_CLOCK_ENV = { GSD_TEST_MODE: '1', GSD_NOW_MS: '1750000000000' };
|
||||
|
||||
test('state_updated is true and STATE.md content actually changes on a genuine completion', () => {
|
||||
writeRoadmap(tmpDir, `# Roadmap v1.0\n`);
|
||||
writeState(tmpDir);
|
||||
const statePath = path.join(tmpDir, '.planning', 'STATE.md');
|
||||
const stateBefore = fs.readFileSync(statePath, 'utf-8');
|
||||
|
||||
const result = runGsdTools(['milestone', 'complete', 'v1.0', '--name', 'Test'], tmpDir, PINNED_CLOCK_ENV);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
const stateAfter = fs.readFileSync(statePath, 'utf-8');
|
||||
assert.notEqual(stateAfter, stateBefore, 'precondition: STATE.md content must actually change');
|
||||
assert.strictEqual(output.state_updated, true, 'state_updated must be true for a genuine rewrite');
|
||||
assert.strictEqual(output.milestones_updated, true, 'milestones_updated must be true for a genuine rewrite');
|
||||
});
|
||||
|
||||
test('state_updated is false when a second identical completion rewrites nothing (#3685)', () => {
|
||||
writeRoadmap(tmpDir, `# Roadmap v1.0\n`);
|
||||
writeState(tmpDir);
|
||||
const statePath = path.join(tmpDir, '.planning', 'STATE.md');
|
||||
|
||||
const run1 = runGsdTools(['milestone', 'complete', 'v1.0', '--name', 'Test', '--force'], tmpDir, PINNED_CLOCK_ENV);
|
||||
assert.ok(run1.success, `first milestone complete failed: ${run1.error}`);
|
||||
const stateAfter1 = fs.readFileSync(statePath, 'utf-8');
|
||||
|
||||
// Second call: STATE.md is already in its "milestone complete" closure
|
||||
// shape, so re-running the same closure transform against it is a
|
||||
// genuine no-op for STATE.md content, even though MILESTONES.md still
|
||||
// gains a new (duplicate-looking) entry each call — the two flags are
|
||||
// independent and must not be conflated.
|
||||
const run2 = runGsdTools(['milestone', 'complete', 'v1.0', '--name', 'Test', '--force'], tmpDir, PINNED_CLOCK_ENV);
|
||||
assert.ok(run2.success, `second milestone complete failed: ${run2.error}`);
|
||||
const stateAfter2 = fs.readFileSync(statePath, 'utf-8');
|
||||
|
||||
assert.equal(stateAfter2, stateAfter1, 'STATE.md must be byte-identical across the no-op second run');
|
||||
const output2 = JSON.parse(run2.output);
|
||||
assert.strictEqual(
|
||||
output2.state_updated, false,
|
||||
'fs.existsSync() reported true here, masking the no-op (#3685)',
|
||||
);
|
||||
// milestones_updated has no reachable no-op path: the MILESTONES.md
|
||||
// write unconditionally inserts a new entry block every call, so its
|
||||
// content always differs from the pre-call file — pinning the
|
||||
// true-direction here instead of fabricating a no-op case.
|
||||
assert.strictEqual(output2.milestones_updated, true, 'milestones_updated stays true — MILESTONES.md always gains a new entry');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -3244,6 +3244,65 @@ Plans:
|
||||
const out = JSON.parse(result.output);
|
||||
assert.strictEqual(out.state_updated, false, 'state_updated must be false when no STATE.md exists');
|
||||
});
|
||||
|
||||
// #3685: roadmap_updated used to be reported as a hardcoded `true` —
|
||||
// #2640/#2974 already fixed this call site's sibling `state_updated` flag
|
||||
// to reflect a real content diff (via readModifyWriteStateMd's returned
|
||||
// boolean); roadmap_updated is now fixed the same way, via
|
||||
// updateRoadmapAfterPhaseRemoval's own before/after content comparison.
|
||||
test('roadmap_updated is false when ROADMAP.md comes out byte-identical (#3685)', () => {
|
||||
// Target phase number appears nowhere in ROADMAP.md (no heading, no
|
||||
// dependency reference, no progress-table row, and higher than every
|
||||
// existing phase number so no renumbering fires) and has no directory —
|
||||
// updateRoadmapAfterPhaseRemoval's section-delete/renumber/row-delete
|
||||
// passes are all no-matches, so `content` never diverges from
|
||||
// `originalContent`.
|
||||
//
|
||||
// The fixture is written in ALREADY-NORMALIZED form (a blank line after
|
||||
// the `###` heading) so the byte-identity assertion below compares a
|
||||
// normalized pre-image against a normalized post-image. A hand-authored
|
||||
// fixture that skips that blank line is NOT in the shape
|
||||
// platformWriteSync's own normalizer produces, so writing it back
|
||||
// through the same normalizing write path gains the blank line even
|
||||
// though no phase data changed — that's the writer's own formatting
|
||||
// pass reformatting an un-normalized input, not a real content change,
|
||||
// and asserting byte-identity against such a fixture is unsound.
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, '.planning', 'ROADMAP.md'),
|
||||
`# Roadmap\n\n### Phase 1: Foundation\n\n**Goal:** Setup\n\n## Progress\n\n| Phase | Status |\n|-------|--------|\n| 1 | Done |\n`,
|
||||
);
|
||||
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-foundation'), { recursive: true });
|
||||
const roadmapBefore = fs.readFileSync(path.join(tmpDir, '.planning', 'ROADMAP.md'), 'utf-8');
|
||||
|
||||
const result = runGsdTools('phase remove 5', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const roadmapAfter = fs.readFileSync(path.join(tmpDir, '.planning', 'ROADMAP.md'), 'utf-8');
|
||||
assert.equal(roadmapAfter, roadmapBefore, 'ROADMAP.md must be byte-identical when the removed phase is absent from it');
|
||||
const out = JSON.parse(result.output);
|
||||
assert.strictEqual(
|
||||
out.roadmap_updated, false,
|
||||
'roadmap_updated was hardcoded true here, masking the no-op (#3685)',
|
||||
);
|
||||
});
|
||||
|
||||
test('roadmap_updated is true and ROADMAP.md content actually changes on a genuine removal (#3685)', () => {
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, '.planning', 'ROADMAP.md'),
|
||||
`# Roadmap\n\n### Phase 1: Foundation\n**Goal:** Setup\n\n### Phase 2: Auth\n**Goal:** Authentication\n\n## Progress\n\n| Phase | Status |\n|-------|--------|\n| 1 | Done |\n| 2 | Planned |\n`,
|
||||
);
|
||||
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-foundation'), { recursive: true });
|
||||
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '02-auth'), { recursive: true });
|
||||
const roadmapBefore = fs.readFileSync(path.join(tmpDir, '.planning', 'ROADMAP.md'), 'utf-8');
|
||||
|
||||
const result = runGsdTools('phase remove 2', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const roadmapAfter = fs.readFileSync(path.join(tmpDir, '.planning', 'ROADMAP.md'), 'utf-8');
|
||||
assert.notEqual(roadmapAfter, roadmapBefore, 'precondition: ROADMAP.md content must actually change');
|
||||
const out = JSON.parse(result.output);
|
||||
assert.strictEqual(out.roadmap_updated, true, 'roadmap_updated must be true for a genuine removal');
|
||||
});
|
||||
});
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -21,12 +21,15 @@ const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const fc = require('fast-check');
|
||||
|
||||
const { createTempProject, createTempDir, cleanup, runGsdTools } = require('./helpers.cjs');
|
||||
const configSchema = require('../gsd-core/bin/lib/config-schema.cjs');
|
||||
const capValidator = require('../gsd-core/bin/lib/capability-validator.cjs');
|
||||
const registry = require('../gsd-core/bin/lib/capability-registry.cjs');
|
||||
const gen = require('../scripts/gen-capability-registry.cjs');
|
||||
const configLoader = require('../gsd-core/bin/lib/config-loader.cjs');
|
||||
const { REVIEWER_LANES } = require('../gsd-core/bin/lib/review-lane-descriptor.cjs');
|
||||
|
||||
/** Keys that moved to a lane capability, with the lane that must own each. */
|
||||
const FEDERATED = {
|
||||
@@ -98,12 +101,26 @@ describe('reviewer config federation — provenance actually moved (#2797)', ()
|
||||
assert.equal(configSchema.isCentralConfigKey('review.max_prompt_tokens_per_reviewer'), true);
|
||||
});
|
||||
|
||||
test('a lane with no model flag and no host owns no config keys', () => {
|
||||
test('a lane with no model flag and no host owns no MODEL or HOST key (#3691 narrows #2797)', () => {
|
||||
// Absent-safe (ADR-2782 D4): qwen, cursor and coderabbit take neither a model
|
||||
// argument nor a host, so they declare nothing. That is not a coverage hole.
|
||||
const owners = Object.values(registry.configSchema || {}).map((e) => e && e.owner);
|
||||
for (const laneId of ['qwen', 'cursor', 'coderabbit']) {
|
||||
assert.equal(owners.includes(laneId), false, `${laneId} must declare no config keys`);
|
||||
// argument nor a host. Under #2797 that meant "declares nothing" — the only
|
||||
// way a lane owned a config key was via a model flag or a host. #3691 gave
|
||||
// every CLI lane a `review.max_prompt_tokens_per_reviewer.<slug>` key, a
|
||||
// third legitimate reason to own a key, so qwen/cursor/coderabbit now
|
||||
// legitimately own their own budget key. The part of the #2797 invariant
|
||||
// that still holds — a lane must never own a MODEL or HOST key, or another
|
||||
// lane's budget key, it has no use for — is what this asserts directly.
|
||||
for (const [key, entry] of Object.entries(registry.configSchema || {})) {
|
||||
const owner = entry && entry.owner;
|
||||
if (!['qwen', 'cursor', 'coderabbit'].includes(owner)) continue;
|
||||
assert.ok(
|
||||
!key.startsWith('review.models.') && !key.endsWith('_host'),
|
||||
`${owner} must not own a model or host key, but owns "${key}"`,
|
||||
);
|
||||
assert.equal(
|
||||
key, `review.max_prompt_tokens_per_reviewer.${owner}`,
|
||||
`${owner} must own no key other than its own budget key, but owns "${key}"`,
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -362,3 +379,226 @@ describe('exclusivity gate sees dynamic patterns (#2797)', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// #3691 — no prompt cap reaches any CLI reviewer lane.
|
||||
//
|
||||
// Two independent defects, per .gsd/bug/fix-3691-reviewer-prompt-budget/10-diagnosis.md:
|
||||
// (1) every `transport: spawn` lane (claude, coderabbit, antigravity, cursor, gemini,
|
||||
// codex, kimi-code, opencode, qwen) declares `promptBudgetKey: null`, so
|
||||
// `budgetFor` (gsd-core/bin/gsd-tools.cjs) returns null for them unconditionally;
|
||||
// (2) `review.max_prompt_tokens` is documented and in validKeys but
|
||||
// `config-defaults.manifest.json` has no `review` section, so the resolved
|
||||
// config surface never materializes the key at all — `budgetFor`'s global
|
||||
// fallback is dead code.
|
||||
//
|
||||
// `budgetFor` is an unexported closure inside `routeReviewLane`
|
||||
// (gsd-core/bin/gsd-tools.cjs:1373-1380, confirmed via `module.exports` at
|
||||
// gsd-tools.cjs:4410 — it is not there), so there is no in-process seam to call
|
||||
// directly; every row below drives the real `review-lane plan` CLI end-to-end,
|
||||
// same idiom as the federation suite above.
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
describe('reviewer prompt budget — #3691 (CLI lanes cannot receive a cap)', () => {
|
||||
/** Write `.planning/config.json` for a temp project (overwrites any existing one). */
|
||||
function writeReviewConfig(tmpDir, cfg) {
|
||||
fs.writeFileSync(path.join(tmpDir, '.planning', 'config.json'), JSON.stringify(cfg, null, 2));
|
||||
}
|
||||
|
||||
/** Run `review-lane plan --selected <slugs>` and return the parsed plan array. */
|
||||
function planLanes(tmpDir, slugs) {
|
||||
const runDir = path.join(tmpDir, 'run');
|
||||
const r = runGsdTools(
|
||||
['review-lane', 'plan', '--selected', slugs.join(','), '--run-dir', runDir, '--repo-root', tmpDir],
|
||||
tmpDir,
|
||||
);
|
||||
assert.equal(r.success, true, `review-lane plan failed: ${r.error || r.output}`);
|
||||
return JSON.parse(r.output);
|
||||
}
|
||||
|
||||
/** Run `review-lane plan` for exactly one slug and return its entry. */
|
||||
function planLane(tmpDir, slug) {
|
||||
const entry = planLanes(tmpDir, [slug]).find((e) => e.slug === slug);
|
||||
assert.ok(entry, `no plan entry for slug "${slug}"`);
|
||||
return entry;
|
||||
}
|
||||
|
||||
test('row1 (regression): a CLI spawn lane with only the central global set still reports null today', (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, { review: { max_prompt_tokens: 50000 } });
|
||||
assert.equal(
|
||||
planLane(tmpDir, 'codex').promptBudget, 50000,
|
||||
'codex must inherit the central global once it declares a promptBudgetKey',
|
||||
);
|
||||
});
|
||||
|
||||
test('row2 (regression): the -1 per-lane sentinel on an already-budgeted lane must inherit the global', (t) => {
|
||||
// The exact repro from 10-diagnosis.md.
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, {
|
||||
review: { max_prompt_tokens: 50000, max_prompt_tokens_per_reviewer: { ollama: -1 } },
|
||||
});
|
||||
assert.equal(
|
||||
planLane(tmpDir, 'ollama').promptBudget, 50000,
|
||||
'ollama already declares a promptBudgetKey, so this fails purely on defect 2 (the dead global)',
|
||||
);
|
||||
});
|
||||
|
||||
test('row3 (regression): the resolved config surface must carry review.max_prompt_tokens', (t) => {
|
||||
// Asserts on the resolver's own surface, not only on promptBudget — the two
|
||||
// defects are independent, and fixing only the lane keys would leave THIS
|
||||
// row red even after row1/row2 go green.
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, {
|
||||
review: { max_prompt_tokens: 50000, max_prompt_tokens_per_reviewer: { ollama: -1 } },
|
||||
});
|
||||
const resolved = configLoader.loadConfigResolved(tmpDir);
|
||||
const reviewKeys = Object.keys(resolved.config.review || {});
|
||||
assert.ok(
|
||||
Object.prototype.hasOwnProperty.call(resolved.config.review || {}, 'max_prompt_tokens'),
|
||||
`resolved review surface is missing max_prompt_tokens, got keys: ${JSON.stringify(reviewKeys)}`,
|
||||
);
|
||||
assert.equal(resolved.config.review.max_prompt_tokens, 50000);
|
||||
});
|
||||
|
||||
test('row4 (happy path): a per-lane value overrides the global', (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, {
|
||||
review: { max_prompt_tokens: 50000, max_prompt_tokens_per_reviewer: { codex: 12345 } },
|
||||
});
|
||||
assert.equal(planLane(tmpDir, 'codex').promptBudget, 12345);
|
||||
});
|
||||
|
||||
test('row5 (boundary, limit-1): an explicit per-lane 0 means "do not trim", never the global', (t) => {
|
||||
// The specific regression budgetFor's own comment warns about — 0 is a real
|
||||
// value, not the unset sentinel, and must not be silently promoted to the
|
||||
// global budget.
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, {
|
||||
review: { max_prompt_tokens: 50000, max_prompt_tokens_per_reviewer: { codex: 0 } },
|
||||
});
|
||||
assert.equal(planLane(tmpDir, 'codex').promptBudget, 0);
|
||||
});
|
||||
|
||||
test('row6 (boundary, limit): the -1 sentinel inherits the global', (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, {
|
||||
review: { max_prompt_tokens: 50000, max_prompt_tokens_per_reviewer: { codex: -1 } },
|
||||
});
|
||||
assert.equal(planLane(tmpDir, 'codex').promptBudget, 50000);
|
||||
});
|
||||
|
||||
test('row7 (boundary, limit+1): a real one-token budget is not read as a sentinel', (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, {
|
||||
review: { max_prompt_tokens: 50000, max_prompt_tokens_per_reviewer: { codex: 1 } },
|
||||
});
|
||||
assert.equal(planLane(tmpDir, 'codex').promptBudget, 1);
|
||||
});
|
||||
|
||||
test('row8 (anti-tightening pin, green today and after): no config at all trims nothing, on every declared lane', (t) => {
|
||||
// Guards against a "fix" that hard-codes a budget onto every lane: that
|
||||
// would satisfy rows 1-7 while breaking every user who configured nothing.
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, {});
|
||||
const allSlugs = REVIEWER_LANES.map((l) => l.slug);
|
||||
const plans = planLanes(tmpDir, allSlugs);
|
||||
for (const slug of allSlugs) {
|
||||
const entry = plans.find((e) => e.slug === slug);
|
||||
assert.ok(entry, `no plan entry for slug "${slug}"`);
|
||||
assert.equal(entry.promptBudget, null, `${slug}: the default resolved surface must not gain trimming`);
|
||||
}
|
||||
});
|
||||
|
||||
test('row9 (independence pin, green today and after): the three pre-existing http lanes are unaffected', (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
writeReviewConfig(tmpDir, { review: { max_prompt_tokens_per_reviewer: { ollama: 777 } } });
|
||||
const plans = planLanes(tmpDir, ['ollama', 'lm_studio', 'llama_cpp']);
|
||||
assert.equal(plans.find((e) => e.slug === 'ollama').promptBudget, 777, 'ollama must keep resolving its own configured value');
|
||||
assert.equal(plans.find((e) => e.slug === 'lm_studio').promptBudget, null, 'lm_studio must not shift with no config of its own');
|
||||
assert.equal(plans.find((e) => e.slug === 'llama_cpp').promptBudget, null, 'llama_cpp must not shift with no config of its own');
|
||||
});
|
||||
|
||||
test('row10 (negative space, green today and after): config-set still rejects an unknown per-lane slug', (t) => {
|
||||
// #2841 made an unknown `review.max_prompt_tokens_per_reviewer.<x>` slug an
|
||||
// error; extending the family to nine more lanes must not loosen that.
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
const set = runGsdTools('config-set review.max_prompt_tokens_per_reviewer.not_a_lane 5', tmpDir);
|
||||
assert.equal(set.success, false, 'an unknown lane slug must still be rejected, not silently accepted');
|
||||
});
|
||||
|
||||
// ─── property: the budget-resolution contract ──────────────────────────
|
||||
//
|
||||
// `budgetFor`'s contract (gsd-core/bin/gsd-tools.cjs:1361-1380): the resolved
|
||||
// budget is the per-lane value when it is a finite number other than -1;
|
||||
// otherwise the global when finite; otherwise null.
|
||||
//
|
||||
// Driven through the REAL CLI (review-lane plan against a temp project), not
|
||||
// a reimplementation of the formula — `budgetFor` is not exported (see the
|
||||
// describe-block header), so this is the lowest reachable seam that still
|
||||
// exercises production code rather than a copy of it.
|
||||
//
|
||||
// Generator note on non-finite numbers: JSON cannot encode a literal NaN or
|
||||
// Infinity (`JSON.stringify(NaN) === 'null'`), so a real `.planning/config.json`
|
||||
// can never carry a numeric NaN/Infinity in the first place — driving those
|
||||
// exact values through this seam would not be testing anything reachable.
|
||||
// The string forms below ('NaN', 'Infinity', 'not-a-number') ARE reachable
|
||||
// (JSON strings survive the round trip) and exercise the identical
|
||||
// `typeof v === 'number' && Number.isFinite(v)` guard: a string is rejected
|
||||
// by `typeof` exactly as a real NaN would be rejected by `Number.isFinite`.
|
||||
test('property: per-lane wins when finite and not -1, else the global when finite, else null', () => {
|
||||
const perLaneArb = fc.oneof(
|
||||
fc.integer({ min: -1000, max: 1000000 }),
|
||||
fc.constantFrom(-1, 0),
|
||||
fc.constantFrom('NaN', 'Infinity', 'not-a-number'),
|
||||
fc.constant(undefined),
|
||||
);
|
||||
const globalArb = fc.oneof(
|
||||
fc.integer({ min: 0, max: 1000000 }),
|
||||
fc.constant(null),
|
||||
fc.constantFrom('NaN', 'not-a-number'),
|
||||
fc.constant(undefined),
|
||||
);
|
||||
|
||||
fc.assert(
|
||||
fc.property(perLaneArb, globalArb, (p, g) => {
|
||||
const review = {};
|
||||
if (g !== undefined) review.max_prompt_tokens = g;
|
||||
if (p !== undefined) review.max_prompt_tokens_per_reviewer = { codex: p };
|
||||
|
||||
const tmpDir = createTempProject();
|
||||
try {
|
||||
fs.writeFileSync(path.join(tmpDir, '.planning', 'config.json'), JSON.stringify({ review }, null, 2));
|
||||
const runDir = path.join(tmpDir, 'run');
|
||||
const r = runGsdTools(
|
||||
['review-lane', 'plan', '--selected', 'codex', '--run-dir', runDir, '--repo-root', tmpDir],
|
||||
tmpDir,
|
||||
);
|
||||
if (!r.success) return false;
|
||||
const entry = JSON.parse(r.output).find((e) => e.slug === 'codex');
|
||||
if (!entry) return false;
|
||||
|
||||
const isNum = (v) => typeof v === 'number' && Number.isFinite(v);
|
||||
const expected = isNum(p) && p !== -1 ? p : (isNum(g) ? g : null);
|
||||
return entry.promptBudget === expected;
|
||||
} finally {
|
||||
cleanup(tmpDir);
|
||||
}
|
||||
}),
|
||||
// Bounded low: each run spawns a real gsd-tools child process (plus its own
|
||||
// nested `query resolve-execution` spawn), so this is deliberately far
|
||||
// below the suite's usual 200-run property budget — see the file header
|
||||
// note on why the CLI is nonetheless the right seam.
|
||||
{ seed: 36910824, numRuns: 20 },
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -30,6 +30,7 @@ const {
|
||||
escapePosixDoubleQuoted,
|
||||
escapeSingleQuotedShellLiteral,
|
||||
retryRenameSync,
|
||||
contentChangedAfterNormalize,
|
||||
} = require(path.join(__dirname, '..', 'gsd-core', 'bin', 'lib', 'shell-command-projection.cjs'));
|
||||
|
||||
const { createTempGitProject, createTempDir, cleanup } = require('./helpers.cjs');
|
||||
@@ -1291,6 +1292,71 @@ describe('normalizeContent', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ─── contentChangedAfterNormalize ─────────────────────────────────────────────
|
||||
|
||||
// #3685 / #3691: `platformWriteSync` runs Markdown normalization before
|
||||
// persisting. `roadmap_updated`/`state_updated`/`requirements_updated`-style
|
||||
// flags computed via a raw `after !== before` on the PRE-normalize strings
|
||||
// false-positive whenever the two sides differ only in a way normalization
|
||||
// erases (CRLF, blank-line-run collapse, trailing-newline count) — the exact
|
||||
// artifact class the milestone.cts #3685 fix diagnosed. This seam is the
|
||||
// single point every such flag must go through instead.
|
||||
describe('contentChangedAfterNormalize (#3685 / #3691)', () => {
|
||||
test('reports false when the only difference is a CRLF vs LF artifact', () => {
|
||||
const before = 'line1\r\nline2\r\n';
|
||||
const after = 'line1\nline2\n';
|
||||
assert.strictEqual(
|
||||
contentChangedAfterNormalize('STATE.md', before, after), false,
|
||||
'CRLF-only difference must not report a content change',
|
||||
);
|
||||
});
|
||||
|
||||
test('reports false when the only difference is a collapsible blank-line run', () => {
|
||||
const before = '# Title\n\nparagraph\n';
|
||||
const after = '# Title\n\n\nparagraph\n'; // extra blank line — collapses under normalize
|
||||
assert.strictEqual(
|
||||
contentChangedAfterNormalize('ROADMAP.md', before, after), false,
|
||||
'a blank-line-run artifact that normalizes away must not report a content change',
|
||||
);
|
||||
});
|
||||
|
||||
test('reports false when the only difference is trailing-newline count', () => {
|
||||
const before = '# Title\n\nbody\n';
|
||||
const after = '# Title\n\nbody\n\n\n';
|
||||
assert.strictEqual(
|
||||
contentChangedAfterNormalize('STATE.md', before, after), false,
|
||||
'trailing-newline-count-only difference must not report a content change',
|
||||
);
|
||||
});
|
||||
|
||||
test('reports true for a genuine content change', () => {
|
||||
const before = '# Title\n\nold body\n';
|
||||
const after = '# Title\n\nnew body\n';
|
||||
assert.strictEqual(
|
||||
contentChangedAfterNormalize('ROADMAP.md', before, after), true,
|
||||
'a real content change must still report true',
|
||||
);
|
||||
});
|
||||
|
||||
test('reports true for a genuine change even when disguised by normalize-equivalent formatting on both sides', () => {
|
||||
const before = '# Title\r\n\r\n\r\nold body\r\n';
|
||||
const after = '# Title\n\nnew body\n\n\n';
|
||||
assert.strictEqual(
|
||||
contentChangedAfterNormalize('ROADMAP.md', before, after), true,
|
||||
'formatting noise on both sides must not mask a real semantic change',
|
||||
);
|
||||
});
|
||||
|
||||
test('non-.md files still normalize (CRLF strip + trailing newline) before comparing', () => {
|
||||
const before = 'a: 1\r\nb: 2';
|
||||
const after = 'a: 1\nb: 2\n';
|
||||
assert.strictEqual(
|
||||
contentChangedAfterNormalize('config.json', before, after), false,
|
||||
'non-.md normalization (CRLF + trailing newline) must also suppress a false positive',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── platformWriteSync ───────────────────────────────────────────────────────
|
||||
|
||||
describe('platformWriteSync', () => {
|
||||
|
||||
Reference in New Issue
Block a user