Files
msd-core/tests/adr-15-progress-converge.test.cjs
Tom Boucher 7372d99a26 enhance(#2800): derive reviewer flag lists and gate reviewer lane docs across locales (#2882)
* chore(#2800): derive reviewer flag lists and gate reviewer lane docs across locales

The reviewer lane roster was hand-enumerated across five documentation
surfaces and three workflow files that had drifted apart: --kimi-code was
missing from all four translated COMMANDS.md mirrors, --coderabbit from
every workflow forwarding list, and --antigravity from FEATURES.md.

Adds checkReviewerDocsParity, a second pure gate deliberately separate from
checkReviewerLaneParity so a stale doc cannot make the runtime checker look
red. Workflows now derive their flag lists from a new review-lane flags
query instead of hand-enumerating them, which also retires the unanchored
grep that matched --agy inside --antigravity.

Documents the previously absent reviewer body and hostBehaviors field in
the capability manifest reference.

Closes #2800
Closes #2781
Closes #2272

* fix(#2800): key the docs parity table arm on first-cell position

Review found the flag arm was file-scoped, so the forwarding row that lists
every flag in its third cell satisfied it on its own. Deleting a lane's own
reviewer-table row -- the #2781 regression this gate exists to prevent --
therefore passed undetected.

Arm 4 keys on the FIRST table cell, which separates a lane row from the
forwarding row structurally and in every locale. Regression test included.

* fix(#2800): shape-filter the flags subcommand output

All three consumers read review-lane flags through an unquoted command
substitution so the output word-splits into loop items. Phase 2 admits
third-party overlay lanes, so an overlay flag containing whitespace would
inject a second loop item and one containing a glob would expand against
the cwd. Emit only well-formed flags so neither reaches the shell.

* fix(#2800): remove the regex length ceiling and count only prose mentions

Review found two real defects in the docs parity gate.

The never-throws contract was false: building a RegExp from a declared flag
or section title throws SyntaxError past ~100k chars, and Phase 2 admits
overlay lanes whose declared strings are untrusted in length. Every one of
these matches is literal, so String.includes replaces the regex outright,
which also deletes escapeLiteral and the llama.cpp escaping it existed for.

Arm 1 was context-blind: a flag mentioned only inside a fenced example or a
commented-out row counted as documented. Both are stripped before matching.

Also advertises all 13 lane flags in the argument-hint and corrects a stale
eleven-lane count in the slug grammar note.

* test(#2800): repoint the convergence suite off deleted workflow text

The derived flag loop deleted the literal per-flag grep lines four tests
matched on. Two of those failed loudly. The behavioral and property tests
failed SILENTLY instead: their end marker no longer resolved, so the parse
block extracted empty and both passed vacuously, and the property test's
gsd_run stub had a no-op default that hid it.

All now share one extractor and execute the real deployed block through a
gsd_run shim backed by the actual binary. The whitelist assertions become an
anti-parity check: re-adding a hand-written flag list must fail.

Also repairs two vacuous cases in the docs parity suite. The unreadable-doc
test called its own mock rather than the reader, and the integration test
bounded nothing, so a doc losing its marker would have been silently skipped
and still passed green.

* fix(#2800): run the derived flag loop after the launcher preamble

The remote matrix caught a real runtime bug, not a test artifact. In
autonomous.md and plan-review-convergence.md the launcher preamble that
defines gsd_run lives in a separate, LATER bash fence than the derived loop.
Each fence is its own shell, so gsd_run was undefined where the loop ran:
the command substitution yielded nothing and zero reviewer flags would have
been forwarded. Worse than the drift this epic fixes, and silent.

The whole CONVERGENCE_ARGS construction moves as one unit, because the
--max-cycles append sits between the loop and the preamble and would
otherwise have run against an uninitialized variable and then been dropped
by the relocated initializer.

Also documents all 13 lane flags in help/modes/full.md, which the repo gates
bidirectionally against each command's argument-hint.

* test(#2800): repoint the two converge suites off deleted flag literals

Both asserted workflow.includes('--codex') against the hand-enumerated list
the derived loop removed. They now assert the derivation itself, keep --all
and --text (convergence controls, still literal), and add an anti-parity
guard so re-adding a hardcoded list fails.

The lost pass-through proof is replaced with a real one: every flag the
tests used to hardcode is asserted present in the actual roster emitted by
the binary, which is the property the old assertion was protecting.

* test(#2800): acknowledge the workflow byte growth from the derived flag loop

* chore(#2800): backfill changeset pr number to 2882

* fix(#2800): strip HTML comments to a fixed point in the parity gate

CodeQL js/incomplete-multi-character-sanitization (high) on PR #2882: the
single-pass <!--...--> strip can leave a live <!-- behind, so a join-trick
construction smuggles a commented-out row past the gate and it counts as
documented. Not an injection risk here since nothing is rendered, but it is
the exact false pass this helper exists to prevent.

Strips to a fixed point, then treats any surviving opener as unterminated so
the multi-line branch closes it on a later line. Terminates because every
pass strictly shortens the string.

* test(#2800): pin the comment-smuggling regression with a real reproducer

The obvious fixture for this class does not reproduce it: <!--<!---->-->
leaves a dangling --> rather than a live <!--, and is caught either way, so
it would have passed with and without the fix. The join-trick construction
(<!- + <!--DUMMY--> + -...-->), the <scr<script>ipt> shape, genuinely
regresses on the single-pass strip and is what the test now uses.

---------

Co-authored-by: Test <test@example.com>
2026-07-30 19:14:13 -04:00

217 lines
8.4 KiB
JavaScript

// allow-test-rule: source-text-is-the-product #1190
// The progress command and next workflow markdown are runtime-loaded contracts.
// Checking their text verifies the shipped slash-command behavior.
'use strict';
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const cp = require('node:child_process');
const REPO_ROOT = path.join(__dirname, '..');
const COMMAND_PATH = path.join(REPO_ROOT, 'commands', 'gsd', 'progress.md');
const WORKFLOW_PATH = path.join(REPO_ROOT, 'gsd-core', 'workflows', 'next.md');
const FULL_MD_PATH = path.join(REPO_ROOT, 'gsd-core', 'workflows', 'help', 'modes', 'full.md');
const COMMANDS_DOC_PATH = path.join(REPO_ROOT, 'docs', 'COMMANDS.md');
const HOW_TO_PATH = path.join(REPO_ROOT, 'docs', 'how-to', 'run-phases-autonomously.md');
const TOOLS = path.join(REPO_ROOT, 'gsd-core', 'bin', 'gsd-tools.cjs');
function read(filePath) {
return fs.readFileSync(filePath, 'utf8');
}
describe('ADR-15: /gsd:progress --next --auto --converge (#1190)', () => {
test('progress command advertises --converge, --auto, and notes --cross-ai alias', () => {
const command = read(COMMAND_PATH);
assert.match(
command,
/^argument-hint:.*--converge/m,
'progress command should advertise --converge in argument-hint',
);
assert.match(
command,
/^argument-hint:.*--auto/m,
'progress command should advertise --auto in argument-hint',
);
assert.match(command, /--cross-ai/, 'progress command should document --cross-ai alias');
assert.match(
command,
/workflow\.plan_review_convergence=true/,
'progress command should mention the convergence feature gate',
);
});
test('next workflow parses converge aliases into a plan strategy', () => {
const workflow = read(WORKFLOW_PATH);
assert.match(workflow, /PLAN_STRATEGY="local"/, 'workflow should default to local planning');
assert.match(workflow, /PLAN_STRATEGY="converge"/, 'workflow should opt into converge planning');
assert.match(workflow, /converge\|cross-ai/, 'workflow should accept --converge and --cross-ai');
});
test('next workflow fails fast when convergence is requested but disabled', () => {
const workflow = read(WORKFLOW_PATH);
assert.match(
workflow,
/config-get workflow\.plan_review_convergence/,
'workflow should check workflow.plan_review_convergence before planning',
);
assert.match(
workflow,
/gsd config-set workflow\.plan_review_convergence true/,
'workflow should print the enable command instead of silently downgrading',
);
});
test('next workflow routes Route 3 through plan-review-convergence when PLAN_STRATEGY=converge', () => {
const workflow = read(WORKFLOW_PATH);
assert.match(
workflow,
/\/gsd:plan-review-convergence/,
'next workflow should reference /gsd:plan-review-convergence for the converge route',
);
assert.match(
workflow,
/PLAN_STRATEGY=converge/,
'next workflow should check PLAN_STRATEGY for the converge override',
);
assert.match(
workflow,
/gsd:plan-phase/,
'local planning path should remain available for default next runs',
);
// Args-forwarding contract: Route 3 invocation must pass ${CONVERGENCE_ARGS} to the convergence
// command — not just route to the command name but actually forward the built args variable.
assert.match(
workflow,
/\/gsd:plan-review-convergence[^\n]*\$\{CONVERGENCE_ARGS\}/,
'Route 3 convergence invocation must include ${CONVERGENCE_ARGS} on the same line as the command',
);
});
test('next workflow forwards reviewer flags and max cycles to convergence', () => {
const workflow = read(WORKFLOW_PATH);
// Non-lane convergence controls remain hand-written literals in the workflow.
const convergenceControls = ['--all', '--text'];
// Reviewer lane flags that were formerly hand-enumerated in the workflow text.
// They must now be DERIVED at runtime via `gsd_run review-lane flags`, not listed.
const formerlyHardcodedLaneFlags = [
'--codex',
'--gemini',
'--claude',
'--opencode',
'--ollama',
'--lm-studio',
'--llama-cpp',
];
// The literal-absence guard below excludes '--claude': the runtime-launcher
// preamble legitimately contains an unrelated "npx ... --claude --local"
// install-runtime flag, so a substring match on '--claude' would false-positive
// against that literal, not against a re-added reviewer-flag list.
const antiParityLaneFlags = formerlyHardcodedLaneFlags.filter((flag) => flag !== '--claude');
assert.match(workflow, /CONVERGENCE_ARGS/, 'workflow should build convergence pass-through args');
assert.match(
workflow,
/gsd_run review-lane flags/,
'workflow should derive reviewer flags from the review-lane roster instead of hand-listing them',
);
for (const flag of convergenceControls) {
assert.ok(workflow.includes(flag), `workflow should pass through ${flag}`);
}
assert.match(workflow, /--max-cycles/, 'workflow should pass through --max-cycles N');
// Anti-parity guard (deliberately inverted polarity): the whole point of the
// review-lane-flags derivation is that reviewer lane flags are declared ONCE
// (in the review-lane roster) and never hand-listed again in workflow prose.
// If a future edit re-adds a hardcoded reviewer-flag list here, that is the
// regression this test exists to catch — so this assertion must FAIL when
// any of these flags reappear as literals in the workflow text.
for (const flag of antiParityLaneFlags) {
assert.ok(
!workflow.includes(flag),
`workflow should NOT hand-enumerate reviewer lane flag ${flag}; it must be derived via review-lane flags`,
);
}
// Behavioral coverage: prove the roster the workflow derives from actually
// yields the flags this test used to hardcode, so the derivation is not vacuous.
const laneFlags = cp
.execFileSync(process.execPath, [TOOLS, 'review-lane', 'flags'], { encoding: 'utf8' })
.split('\n')
.filter(Boolean);
for (const flag of formerlyHardcodedLaneFlags) {
assert.ok(laneFlags.includes(flag), `review-lane flags should include ${flag}`);
}
});
test('next workflow preserves --auto re-invocation chaining', () => {
const workflow = read(WORKFLOW_PATH);
assert.match(
workflow,
/--auto/,
'workflow should document the --auto chaining behavior',
);
assert.match(
workflow,
/\/gsd:progress --next --auto/,
'workflow should re-invoke /gsd:progress --next --auto for chaining',
);
// Forwarding contract: the --auto re-invocation must explicitly state that --converge/--cross-ai
// and reviewer flags are forwarded — not just re-invoke --auto alone.
assert.match(
workflow,
/\/gsd:progress --next --auto[^\n]*(forwarding|--converge)/,
'workflow --auto re-invocation should document forwarding --converge/--cross-ai and reviewer flags',
);
});
test('full.md documents --converge, --auto, and --cross-ai for /gsd:progress', () => {
const fullMd = read(FULL_MD_PATH);
assert.match(
fullMd,
/\/gsd:progress --next --auto --converge/,
'full.md should show /gsd:progress --next --auto --converge usage',
);
assert.match(
fullMd,
/--auto/,
'full.md should document --auto for progress',
);
assert.match(
fullMd,
/--cross-ai/,
'full.md should mention --cross-ai alias for convergence',
);
});
test('COMMANDS.md documents progress convergence flags and usage', () => {
const commandsDoc = read(COMMANDS_DOC_PATH);
assert.match(
commandsDoc,
/\/gsd-progress --next --auto --converge/,
'COMMANDS.md should show /gsd-progress --next --auto --converge usage example',
);
assert.match(commandsDoc, /--converge/, 'COMMANDS.md should document --converge for progress');
assert.match(commandsDoc, /--cross-ai/, 'COMMANDS.md should document --cross-ai alias for progress');
});
test('how-to shows /gsd-progress --next --auto --converge usage', () => {
const howTo = read(HOW_TO_PATH);
assert.match(
howTo,
/\/gsd-progress --next --auto --converge/,
'how-to should show /gsd-progress --next --auto --converge usage',
);
});
});