* test(#4109): add zsh regression coverage for gate-check and invoke_reviewers dispatch Extends tests/review-plan-coverage-manifest.test.cjs to cover the two DISPATCH_SLUGS consumption sites the existing #3301 coverage-check tests don't reach: the write_reviews gate-check block (ALL_LANES_SKIPPED / TOTAL_LANE_FAILURE counting) and invoke_reviewers' dispatch + join loops. Both extract the real shipped bash from review.md and run it under bash and zsh, same convention as the existing coverage-check rows. Expected RED under zsh pre-fix, GREEN post-fix. * fix(#4109): rewrap unquoted DISPATCH_SLUGS-shaped consumption to survive zsh Bash word-splits an unquoted scalar on IFS by default; zsh does not (no setopt SH_WORD_SPLIT anywhere in these files), so a scalar accumulated from multiple space-separated tokens and then consumed via bare `for x in $VAR` collapses onto one bogus iteration under zsh whenever it holds 2+ tokens. Fixes all four review.md sites reported by #4108/#4109 (invoke_reviewers dispatch + join loops, the gate-check block, and the #3301 coverage-check block), plus the identical pattern found by a repo-wide sweep in complete-milestone.md, code-review.md, pr-branch.md, sync-skills.md, and execute-phase/steps/per-plan-worktree-gate.md. Each site is rewrapped in unquoted command substitution (`$(printf '%s' "$VAR")`), which re-splits identically under both shells regardless of SH_WORD_SPLIT — the same mechanism that already made the accumulator-building loops in these files shell-safe. * test(#4109): warn loudly when the zsh probe fails instead of skipping silently detectShells() drops the zsh test lane whenever a live zsh probe fails (e.g. on a CI runner without zsh installed), and a dropped lane reads identically to a passing one in the suite's own output — exactly the blind spot that let #4109's bug class ship undetected. Both copies of this helper (this file and its byte-identical duplicate in review-build-prompt-optional-sections.test.cjs) now emit a greppable console.warn when the probe fails, so a run without zsh reads as "zsh coverage unknown" rather than "all lanes green". * ci(#4109): gate workflows/*.md's embedded bash blocks in lint:ci Adds scripts/lint-workflow-shellcheck.cjs, wired into npm run lint:ci, so this bug class can't land undetected a third time. Two independent checks run over every ```bash block in gsd-core/workflows/**/*.md: - ShellCheck (new `shellcheck` devDependency, downloads and caches the real koalaman/shellcheck binary) catches the general unquoted-expansion/ word-splitting family in argument position. 212 pre-existing findings across the tree are absorbed into scripts/lint-workflow-shellcheck-baseline.json (matched on {file, code, message}, not line number, so unrelated edits elsewhere in a file can't spuriously un-baseline anything) — fixing all of them is out of scope for this issue; only NEW findings fail the build. - A custom structural check specifically for #4109's own shape: ShellCheck does not flag a bare `for x in $VAR` word-list — it treats that as an intentional idiom under any ruleset (confirmed empirically). This check does, and gates the build on any occurrence, with zero tolerance (no baseline) since every known site was already swept and fixed on this branch. * test(#4109): update pr-branch cherry-pick loop test anchor for the zsh fix extractPickLoop()'s PICK_LOOP_MARKER located the create_pr_branch cherry-pick loop by the literal substring "for HASH in $INCLUDED_COMMITS", which no longer appears verbatim after this issue's fix rewrapped that loop in unquoted command substitution. Updates the anchor (and its error message, now derived from the same constant instead of duplicating stale text) to the new literal form. No behavior change to the extraction logic itself. Emitted-Drift-Ack-Growth: review.md — #4109's fix adds explanatory comments at 4 sites; net code is functionally equivalent, comment expansion grows the file Emitted-Drift-Ack-Growth: complete-milestone.md — #4109's fix adds explanatory comments at 3 sites documenting the zsh word-splitting divergence Emitted-Drift-Ack-Growth: code-review.md — #4109's fix adds an explanatory comment documenting the zsh word-splitting divergence Emitted-Drift-Ack-Growth: pr-branch.md — #4109's fix adds explanatory comments at 3 sites (TRANSIENT_DIRS, INCLUDED_COMMITS, FILTER_PATHS) Emitted-Drift-Ack-Growth: sync-skills.md — #4109's fix adds explanatory comments at 2 sites (CREATE_LIST/UPDATE_LIST, REMOVE_LIST) * test(#4109): cover lint-workflow-shellcheck parsers + add timeout Adds tests/lint-workflow-shellcheck.test.cjs covering the hand-rolled parser/logic functions exported by scripts/lint-workflow-shellcheck.cjs (stripCommandSubstitutions, stripShellComments, substitutePlaceholders, extractForLoops, findBareForLoopSplits, findingKey, partitionAgainstBaseline) that shipped with zero coverage — CLAUDE.md requires at least one fast-check property test for parsers, included here (bare/braced forms always flagged naming the variable; quoted/substituted/literal forms never are). Also bounds runShellcheck's subprocess with a 60s timeout: the `shellcheck` npm package's own API has no timeout option and internally blocks on a synchronous spawnSync, so this reimplements the binary resolve/download step via the package's own exported config/download and calls spawnSync directly with a native timeout. And guards the CLI entry point with `if (require.main === module)`, matching this repo's sibling dual-purpose lint scripts — without it, requiring the module for its exported functions (as the new test file does) also triggered a live ShellCheck run as a side effect. * docs(#4109): add changeset for the zsh word-splitting fix * chore(#4109): backfill changeset PR number (#4116) --------- Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
@@ -85,7 +85,7 @@ const readWorkflow = () => parseWorkflow(fs.readFileSync(WORKFLOW_PATH, 'utf-8')
|
||||
|
||||
const BASH_FENCE_OPEN_RE = /^```bash\s*$/;
|
||||
const BASH_FENCE_CLOSE_RE = /^```\s*$/;
|
||||
const PICK_LOOP_MARKER = 'for HASH in $INCLUDED_COMMITS';
|
||||
const PICK_LOOP_MARKER = 'for HASH in $(printf \'%s\' "$INCLUDED_COMMITS")';
|
||||
|
||||
// Scans `text` for fenced ```bash blocks and returns the verbatim body
|
||||
// (fence markers stripped, lines rejoined with '\n') of the single block
|
||||
@@ -119,7 +119,7 @@ const extractPickLoop = (text) => {
|
||||
}
|
||||
|
||||
if (matches.length === 0) {
|
||||
throw new Error('pr-branch.md: no create_pr_branch cherry-pick loop found (expected a bash block containing "for HASH in $INCLUDED_COMMITS")');
|
||||
throw new Error(`pr-branch.md: no create_pr_branch cherry-pick loop found (expected a bash block containing "${PICK_LOOP_MARKER}")`);
|
||||
}
|
||||
if (matches.length > 1) {
|
||||
throw new Error(`pr-branch.md: cherry-pick loop found in ${matches.length} bash blocks — the recipe must have exactly one canonical form`);
|
||||
|
||||
277
tests/lint-workflow-shellcheck.test.cjs
Normal file
277
tests/lint-workflow-shellcheck.test.cjs
Normal file
@@ -0,0 +1,277 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* lint-workflow-shellcheck.test.cjs — unit + property coverage for the
|
||||
* hand-rolled shell-text parser/logic functions exported by
|
||||
* scripts/lint-workflow-shellcheck.cjs (#4109 follow-up).
|
||||
*
|
||||
* Per this repo's CLAUDE.md: "Parsers, budget limits, and bijective
|
||||
* contracts must include at least one fast-check (`fc`) property test."
|
||||
* These functions ARE parsers (shell-text extraction/transformation over
|
||||
* workflow-authored markdown), so the property-test requirement below is a
|
||||
* binding gate, not optional polish.
|
||||
*
|
||||
* Note: scripts/lint-workflow-shellcheck.cjs guards its CLI entry point
|
||||
* with `if (require.main === module) runMain(main);`, so requiring it here
|
||||
* for its exported pure functions does not also trigger a live ShellCheck
|
||||
* run against the real gsd-core/workflows/ tree.
|
||||
*/
|
||||
|
||||
const { test, describe } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const path = require('node:path');
|
||||
const fc = require('./helpers/fast-check-setup.cjs');
|
||||
|
||||
const {
|
||||
substitutePlaceholders,
|
||||
stripCommandSubstitutions,
|
||||
stripShellComments,
|
||||
extractForLoops,
|
||||
findBareForLoopSplits,
|
||||
findingKey,
|
||||
partitionAgainstBaseline,
|
||||
} = require(path.join(__dirname, '..', 'scripts', 'lint-workflow-shellcheck.cjs'));
|
||||
|
||||
/** Minimal block shape findBareForLoopSplits expects (see the source's own
|
||||
* extractBashBlocks for the real shape this stands in for). */
|
||||
function mkBlock(body, overrides = {}) {
|
||||
return { file: 'w.md', blockIdx: 0, firstBodyLine: 1, body, ...overrides };
|
||||
}
|
||||
|
||||
describe('stripCommandSubstitutions', () => {
|
||||
test('empty string', () => {
|
||||
assert.equal(stripCommandSubstitutions(''), '');
|
||||
});
|
||||
|
||||
test('no substitution present leaves the string intact', () => {
|
||||
assert.equal(stripCommandSubstitutions('echo hello'), 'echo hello');
|
||||
});
|
||||
|
||||
test('substitution at the start', () => {
|
||||
assert.equal(stripCommandSubstitutions('$(echo hi) world'), ' world');
|
||||
});
|
||||
|
||||
test('substitution in the middle', () => {
|
||||
assert.equal(stripCommandSubstitutions('a $(echo hi) b'), 'a b');
|
||||
});
|
||||
|
||||
test('substitution at the end', () => {
|
||||
assert.equal(stripCommandSubstitutions('a $(echo hi)'), 'a ');
|
||||
});
|
||||
|
||||
test('nested command substitution — $(echo $(nested))', () => {
|
||||
assert.equal(stripCommandSubstitutions('pre $(echo $(nested)) post'), 'pre post');
|
||||
});
|
||||
|
||||
test('unterminated substitution does not crash or hang, consumes to end of string', () => {
|
||||
assert.equal(stripCommandSubstitutions('a $(echo hi'), 'a ');
|
||||
});
|
||||
|
||||
test('malformed nested-unterminated input does not crash or hang', () => {
|
||||
assert.equal(stripCommandSubstitutions('a $(echo $(nested'), 'a ');
|
||||
});
|
||||
});
|
||||
|
||||
describe('stripShellComments', () => {
|
||||
test('strips a full-line comment to spaces, preserving the newline', () => {
|
||||
assert.equal(stripShellComments('# a comment\necho hi'), ' \necho hi');
|
||||
});
|
||||
|
||||
test('strips a trailing comment after code, preserving preceding code', () => {
|
||||
assert.equal(stripShellComments('echo hi # trailing'), 'echo hi ');
|
||||
});
|
||||
|
||||
test('does NOT strip ${VAR#pattern} parameter-expansion # (the false-positive this pass exists to avoid)', () => {
|
||||
assert.equal(stripShellComments('sm_raw=${sm_raw#./}'), 'sm_raw=${sm_raw#./}');
|
||||
});
|
||||
|
||||
test('does not strip a # immediately following a non-whitespace identifier character', () => {
|
||||
assert.equal(stripShellComments('x=${foo#bar} y=1'), 'x=${foo#bar} y=1');
|
||||
});
|
||||
|
||||
test('a # inside single quotes is preserved literally, not treated as a comment start', () => {
|
||||
assert.equal(stripShellComments("echo 'a # b'"), "echo 'a # b'");
|
||||
});
|
||||
});
|
||||
|
||||
describe('substitutePlaceholders', () => {
|
||||
test('substitutes a single-token placeholder ({run_dir}) to a shell-safe bareword', () => {
|
||||
assert.equal(substitutePlaceholders('cd {run_dir}'), 'cd PLACEHOLDER_run_dir');
|
||||
});
|
||||
|
||||
test('substitutes a multi-word prose placeholder ({discovered test command})', () => {
|
||||
assert.equal(
|
||||
substitutePlaceholders('run {discovered test command}'),
|
||||
'run PLACEHOLDER_discovered_test_command',
|
||||
);
|
||||
});
|
||||
|
||||
test('does NOT substitute real ${VAR} parameter expansion', () => {
|
||||
assert.equal(substitutePlaceholders('echo ${VAR}'), 'echo ${VAR}');
|
||||
});
|
||||
|
||||
test('does NOT clobber {1..5} POSIX numeric brace expansion (pinning the corrected behavior)', () => {
|
||||
assert.equal(substitutePlaceholders('echo {1..5}'), 'echo {1..5}');
|
||||
});
|
||||
|
||||
test('does NOT clobber {a,b,c} POSIX brace-expansion list (pinning the corrected behavior)', () => {
|
||||
assert.equal(substitutePlaceholders('echo {a,b,c}'), 'echo {a,b,c}');
|
||||
});
|
||||
|
||||
test('does NOT clobber { cmd1; cmd2; } compound-command grouping (pinning the corrected behavior)', () => {
|
||||
const input = '{ echo hi; echo bye; }';
|
||||
assert.equal(substitutePlaceholders(input), input);
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractForLoops', () => {
|
||||
test('captures loopVar and listExpr for a simple for-header', () => {
|
||||
const [loop] = extractForLoops('for x in $VAR; do\n echo "$x"\ndone\n');
|
||||
assert.equal(loop.loopVar, 'x');
|
||||
assert.equal(loop.listExpr, '$VAR');
|
||||
});
|
||||
|
||||
test('does not truncate the list-expr at a `;` nested inside $( ... )', () => {
|
||||
const [loop] = extractForLoops('for x in $(echo a; echo b); do\n echo "$x"\ndone\n');
|
||||
assert.equal(loop.listExpr, '$(echo a; echo b)');
|
||||
});
|
||||
|
||||
test('finds multiple for-loops in one body', () => {
|
||||
const loops = extractForLoops('for a in 1 2; do :; done\nfor b in $Y; do :; done\n');
|
||||
assert.equal(loops.length, 2);
|
||||
assert.equal(loops[0].loopVar, 'a');
|
||||
assert.equal(loops[1].loopVar, 'b');
|
||||
});
|
||||
});
|
||||
|
||||
describe('findBareForLoopSplits — the core #4109 detector', () => {
|
||||
test('POSITIVE: bare unquoted `for x in $VAR; do ... done` is flagged', () => {
|
||||
const findings = findBareForLoopSplits([mkBlock('for x in $VAR; do\n echo "$x"\ndone\n')]);
|
||||
assert.equal(findings.length, 1);
|
||||
assert.equal(findings[0].loopVar, 'x');
|
||||
assert.equal(findings[0].varName, 'VAR');
|
||||
});
|
||||
|
||||
test('POSITIVE: bare unquoted `${VAR}` braced form is also flagged', () => {
|
||||
const findings = findBareForLoopSplits([mkBlock('for x in ${VAR}; do\n echo "$x"\ndone\n')]);
|
||||
assert.equal(findings.length, 1);
|
||||
assert.equal(findings[0].varName, 'VAR');
|
||||
});
|
||||
|
||||
test('NEGATIVE: command-substitution-wrapped form (the actual #4109 fix pattern) is NOT flagged', () => {
|
||||
const findings = findBareForLoopSplits([
|
||||
mkBlock('for x in $(printf \'%s\' "$VAR"); do\n echo "$x"\ndone\n'),
|
||||
]);
|
||||
assert.equal(findings.length, 0);
|
||||
});
|
||||
|
||||
test('NEGATIVE: quoted `"$VAR"` form is NOT flagged', () => {
|
||||
const findings = findBareForLoopSplits([mkBlock('for x in "$VAR"; do\n echo "$x"\ndone\n')]);
|
||||
assert.equal(findings.length, 0);
|
||||
});
|
||||
|
||||
test('NEGATIVE: literal words (no variable at all) are NOT flagged', () => {
|
||||
const findings = findBareForLoopSplits([mkBlock('for x in a b c; do\n echo "$x"\ndone\n')]);
|
||||
assert.equal(findings.length, 0);
|
||||
});
|
||||
|
||||
test('a for-loop shape quoted inside a `#` comment (prose referencing a past bug) is NOT flagged', () => {
|
||||
const findings = findBareForLoopSplits([
|
||||
mkBlock('# old bug: for x in $VAR; do ... done\necho ok\n'),
|
||||
]);
|
||||
assert.equal(findings.length, 0);
|
||||
});
|
||||
|
||||
// --- fast-check property test (CLAUDE.md-mandated for parsers) ---
|
||||
//
|
||||
// Property: for any generated loop-var/var-name pair and any of the SAFE
|
||||
// forms (quoted, command-substitution-wrapped, literal-words), the
|
||||
// detector reports zero findings; for either UNSAFE bare form (bare $VAR
|
||||
// or braced ${VAR}), it reports exactly one finding naming that variable.
|
||||
const identArb = fc.stringMatching(/^[A-Za-z_][A-Za-z0-9_]{0,6}$/);
|
||||
const formArb = fc.constantFrom('bare', 'braced', 'quoted', 'substituted', 'literal');
|
||||
|
||||
function buildLoopBody(loopVar, varName, form) {
|
||||
let listExpr;
|
||||
switch (form) {
|
||||
case 'bare':
|
||||
listExpr = `$${varName}`;
|
||||
break;
|
||||
case 'braced':
|
||||
listExpr = `\${${varName}}`;
|
||||
break;
|
||||
case 'quoted':
|
||||
listExpr = `"$${varName}"`;
|
||||
break;
|
||||
case 'substituted':
|
||||
listExpr = `$(printf '%s' "$${varName}")`;
|
||||
break;
|
||||
case 'literal':
|
||||
listExpr = 'a b c';
|
||||
break;
|
||||
default:
|
||||
throw new Error(`unreachable form: ${form}`);
|
||||
}
|
||||
return `for ${loopVar} in ${listExpr}; do\n echo "$${loopVar}"\ndone\n`;
|
||||
}
|
||||
|
||||
test('property: bare/braced forms are flagged naming the variable; quoted/substituted/literal forms never are', () => {
|
||||
fc.assert(
|
||||
fc.property(identArb, identArb, formArb, (loopVar, varName, form) => {
|
||||
const body = buildLoopBody(loopVar, varName, form);
|
||||
const findings = findBareForLoopSplits([mkBlock(body)]);
|
||||
if (form === 'bare' || form === 'braced') {
|
||||
assert.equal(findings.length, 1);
|
||||
assert.equal(findings[0].varName, varName);
|
||||
} else {
|
||||
assert.equal(findings.length, 0);
|
||||
}
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('findingKey / partitionAgainstBaseline', () => {
|
||||
test('findingKey matches on {file, code, message} only — ignores line number', () => {
|
||||
const a = { file: 'w.md', code: '2086', message: 'msg', line: 10 };
|
||||
const b = { file: 'w.md', code: '2086', message: 'msg', line: 999 };
|
||||
assert.equal(findingKey(a), findingKey(b));
|
||||
});
|
||||
|
||||
test('a finding shifted to a different line, same file/code/message, still matches the baseline (drift tolerance)', () => {
|
||||
const baseline = [{ file: 'w.md', code: '2086', message: 'msg' }];
|
||||
const current = [{ file: 'w.md', code: '2086', message: 'msg', line: 999 }];
|
||||
const { newFindings, baselinedFindings } = partitionAgainstBaseline(current, baseline);
|
||||
assert.equal(newFindings.length, 0);
|
||||
assert.equal(baselinedFindings.length, 1);
|
||||
});
|
||||
|
||||
test('a THIRD occurrence of a message with only two accepted baseline instances is reported new (multiset semantics)', () => {
|
||||
const baseline = [
|
||||
{ file: 'w.md', code: '2086', message: 'msg' },
|
||||
{ file: 'w.md', code: '2086', message: 'msg' },
|
||||
];
|
||||
const current = [
|
||||
{ file: 'w.md', code: '2086', message: 'msg', line: 1 },
|
||||
{ file: 'w.md', code: '2086', message: 'msg', line: 2 },
|
||||
{ file: 'w.md', code: '2086', message: 'msg', line: 3 },
|
||||
];
|
||||
const { newFindings, baselinedFindings } = partitionAgainstBaseline(current, baseline);
|
||||
assert.equal(newFindings.length, 1);
|
||||
assert.equal(baselinedFindings.length, 2);
|
||||
});
|
||||
|
||||
test('a finding with a different code (same file/message) is NOT matched against the baseline', () => {
|
||||
const baseline = [{ file: 'w.md', code: '2086', message: 'msg' }];
|
||||
const current = [{ file: 'w.md', code: '2046', message: 'msg', line: 1 }];
|
||||
const { newFindings } = partitionAgainstBaseline(current, baseline);
|
||||
assert.equal(newFindings.length, 1);
|
||||
});
|
||||
|
||||
test('an empty baseline reports every current finding as new', () => {
|
||||
const current = [{ file: 'w.md', code: '2086', message: 'msg', line: 1 }];
|
||||
const { newFindings, baselinedFindings } = partitionAgainstBaseline(current, []);
|
||||
assert.equal(newFindings.length, 1);
|
||||
assert.equal(baselinedFindings.length, 0);
|
||||
});
|
||||
});
|
||||
@@ -57,7 +57,18 @@ const STDIN_BOUND_MS = 5000;
|
||||
function detectShells() {
|
||||
const shells = [{ name: 'bash', cmd: 'bash' }];
|
||||
const probe = spawnSync('zsh', ['-c', 'exit 0'], { timeout: PROBE_TIMEOUT_MS, windowsHide: true });
|
||||
if (!probe.error && probe.status === 0) shells.push({ name: 'zsh', cmd: 'zsh' });
|
||||
if (!probe.error && probe.status === 0) {
|
||||
shells.push({ name: 'zsh', cmd: 'zsh' });
|
||||
} else {
|
||||
// gsd-core#4109: a skipped zsh lane reads identically to a passing one in
|
||||
// this suite's own output, which is exactly why the bash/zsh
|
||||
// word-splitting bug class went undetected in CI as long as it did. Make
|
||||
// the skip loud so a zsh-less run (e.g. some ubuntu CI images) reads as
|
||||
// "zsh coverage unknown", not "all lanes green".
|
||||
console.warn(
|
||||
'[review-build-prompt-optional-sections.test.cjs] zsh not available — zsh-lane tests SKIPPED, coverage for this shell is UNKNOWN, not verified',
|
||||
);
|
||||
}
|
||||
return shells;
|
||||
}
|
||||
const SHELLS = detectShells();
|
||||
|
||||
@@ -44,7 +44,18 @@ const REPO_ROOT = path.join(__dirname, '..');
|
||||
function detectShells() {
|
||||
const shells = [{ name: 'bash', cmd: 'bash' }];
|
||||
const probe = spawnSync('zsh', ['-c', 'exit 0'], { timeout: PROBE_TIMEOUT_MS, windowsHide: true });
|
||||
if (!probe.error && probe.status === 0) shells.push({ name: 'zsh', cmd: 'zsh' });
|
||||
if (!probe.error && probe.status === 0) {
|
||||
shells.push({ name: 'zsh', cmd: 'zsh' });
|
||||
} else {
|
||||
// gsd-core#4109: a skipped zsh lane reads identically to a passing one in
|
||||
// this suite's own output, which is exactly why the bash/zsh
|
||||
// word-splitting bug class went undetected in CI as long as it did. Make
|
||||
// the skip loud so a zsh-less run (e.g. some ubuntu CI images) reads as
|
||||
// "zsh coverage unknown", not "all lanes green".
|
||||
console.warn(
|
||||
'[review-plan-coverage-manifest.test.cjs] zsh not available — zsh-lane tests SKIPPED, coverage for this shell is UNKNOWN, not verified',
|
||||
);
|
||||
}
|
||||
return shells;
|
||||
}
|
||||
const SHELLS = detectShells();
|
||||
@@ -186,6 +197,101 @@ function readCoverageJson(runDir, slug) {
|
||||
return fs.existsSync(p) ? JSON.parse(fs.readFileSync(p, 'utf8')) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* #4109 — extract the write_reviews gate-check block that counts dispatched
|
||||
* vs skipped lanes and sets ALL_LANES_SKIPPED / TOTAL_LANE_FAILURE. Anchored
|
||||
* on the JSONL variable at the top of that block (distinct from, and earlier
|
||||
* than, the .plans-manifest.md anchor extractCoverageCheckBlock() uses).
|
||||
*/
|
||||
function extractGateCheckBlock() {
|
||||
const content = readWorkflowCombined(REVIEW_WORKFLOW);
|
||||
const anchorIdx = content.indexOf('JSONL="$RUN_DIR/gsd-review-lane-results.jsonl"');
|
||||
assert.notEqual(anchorIdx, -1, 'write_reviews must reference gsd-review-lane-results.jsonl — the gate-check block is missing');
|
||||
const before = content.slice(0, anchorIdx);
|
||||
const fenceOpenRe = /```bash\r?\n/g;
|
||||
let lastOpen = -1;
|
||||
let m;
|
||||
while ((m = fenceOpenRe.exec(before)) !== null) lastOpen = m.index + m[0].length;
|
||||
assert.notEqual(lastOpen, -1, 'gsd-review-lane-results.jsonl reference is not inside a ```bash fence of write_reviews');
|
||||
const after = content.slice(lastOpen);
|
||||
const closeIdx = after.indexOf('\n```');
|
||||
assert.notEqual(closeIdx, -1, 'unterminated ```bash fence around the gate-check block');
|
||||
const body = after.slice(0, closeIdx);
|
||||
assert.ok(
|
||||
body.includes('ALL_LANES_SKIPPED'),
|
||||
'extracted block references JSONL but not ALL_LANES_SKIPPED — wrong block',
|
||||
);
|
||||
return body;
|
||||
}
|
||||
|
||||
/**
|
||||
* #4109 — extract the invoke_reviewers dispatch + join loops, from the
|
||||
* "Split ONCE, de-duplicated" comment (immediately before the DISPATCH_SLUGS
|
||||
* accumulator at this site) through the block's own closing fence. This span
|
||||
* references run_review_lane() and PARALLEL_LANES, both defined earlier in
|
||||
* the SAME fence but out of scope for this extractor — callers must prepend
|
||||
* a stub (see buildDispatchFixture / DISPATCH_JOIN_STUB below).
|
||||
*/
|
||||
function extractDispatchJoinBlock() {
|
||||
const content = readWorkflowCombined(REVIEW_WORKFLOW);
|
||||
const anchorIdx = content.indexOf('# Split ONCE, de-duplicated');
|
||||
assert.notEqual(anchorIdx, -1, 'invoke_reviewers must contain the "Split ONCE, de-duplicated" comment anchor');
|
||||
const after = content.slice(anchorIdx);
|
||||
const closeIdx = after.indexOf('\n```');
|
||||
assert.notEqual(closeIdx, -1, 'unterminated ```bash fence after the dispatch/join anchor');
|
||||
const body = after.slice(0, closeIdx);
|
||||
assert.ok(body.includes('DISPATCH_SLUGS='), 'extracted span does not include the DISPATCH_SLUGS accumulator — wrong anchor');
|
||||
assert.ok(body.includes('wait'), 'extracted span does not include the join `wait` — anchor did not reach the join loop');
|
||||
return body;
|
||||
}
|
||||
|
||||
const DISPATCH_JOIN_STUB = 'run_review_lane() { echo "$1" >> "$RUN_DIR/dispatch-log.txt"; }\nPARALLEL_LANES="false"\n';
|
||||
|
||||
/** Fixture for the gate-check block: a RUN_DIR with per-slug stub files, no aggregate JSONL. */
|
||||
function buildGateCheckFixture(slugs, skippedSlugs) {
|
||||
const root = createTempDir('gsd-4109-gate-');
|
||||
const runDir = path.join(root, 'run');
|
||||
fs.mkdirSync(runDir);
|
||||
for (const slug of slugs) {
|
||||
if (skippedSlugs.includes(slug)) {
|
||||
fs.writeFileSync(
|
||||
path.join(runDir, `gsd-review-${slug}.md`),
|
||||
`${slug} review skipped: prompt budget (500 tokens) too small for the minimum review set.\n`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return {
|
||||
root,
|
||||
runDir,
|
||||
env: { SELECTED_REVIEWERS: slugs.join(',') },
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fixture for the dispatch/join loops: a RUN_DIR and SELECTED_REVIEWERS.
|
||||
* RUN_DIR is passed as an env var (not the `{run_dir}` placeholder) because
|
||||
* extractDispatchJoinBlock() starts at the "Split ONCE" comment, AFTER the
|
||||
* `RUN_DIR="{run_dir}"` assignment earlier in the same real fence — the
|
||||
* aggregate/join loop still references `$RUN_DIR` directly, so it must come
|
||||
* from the environment here.
|
||||
*/
|
||||
function buildDispatchFixture(selectedReviewersCsv) {
|
||||
const root = createTempDir('gsd-4109-dispatch-');
|
||||
const runDir = path.join(root, 'run');
|
||||
fs.mkdirSync(runDir);
|
||||
return {
|
||||
root,
|
||||
runDir,
|
||||
env: { SELECTED_REVIEWERS: selectedReviewersCsv, RUN_DIR: runDir },
|
||||
};
|
||||
}
|
||||
|
||||
function readDispatchLog(runDir) {
|
||||
const content = readIfPresent(path.join(runDir, 'dispatch-log.txt'));
|
||||
if (content === null) return [];
|
||||
return content.split(/\r?\n/).map((l) => l.trim()).filter((l) => l.length > 0);
|
||||
}
|
||||
|
||||
describe('#3301 build_prompt derives and appends a plan coverage manifest', () => {
|
||||
for (const shell of SHELLS) {
|
||||
test(`[${shell.name}] zero plans: manifest reports Total plans: 0 and no ids`, (t) => {
|
||||
@@ -357,6 +463,11 @@ describe('#3301 write_reviews grades each lane against the plan coverage manifes
|
||||
const offenders = extractAllBashBlocks().filter((b) => /\[\s*"\$SLUG"\s*=\s*"(?!coderabbit)/.test(b));
|
||||
assert.deepEqual(offenders, [], 'only the coderabbit exemption may hardcode a slug comparison');
|
||||
});
|
||||
|
||||
test('structural: no bare-unquoted DISPATCH_SLUGS consumption remains (#4109)', () => {
|
||||
const offenders = extractAllBashBlocks().filter((b) => /for SLUG in \$DISPATCH_SLUGS;/.test(b));
|
||||
assert.deepEqual(offenders, [], 'DISPATCH_SLUGS must be quoted or word-split explicitly, not consumed bare — zsh does not IFS-split an unquoted scalar');
|
||||
});
|
||||
});
|
||||
|
||||
describe('#3301 REVIEWS.md documents the plan_coverage frontmatter key', () => {
|
||||
@@ -370,3 +481,109 @@ describe('#3301 REVIEWS.md documents the plan_coverage frontmatter key', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* #4109 — a zsh word-splitting bug: DISPATCH_SLUGS is built as a
|
||||
* space-separated scalar accumulator and consumed via unquoted
|
||||
* `for SLUG in $DISPATCH_SLUGS; do`. Bash IFS-splits an unquoted scalar by
|
||||
* default; zsh does not, so the entire accumulator (leading space and all)
|
||||
* collapses onto ONE bogus iteration under zsh whenever 2+ reviewers are
|
||||
* selected. These rows extract the REAL shipped bash from the two remaining
|
||||
* untested sites (write_reviews' gate-check block, and invoke_reviewers'
|
||||
* dispatch + join loops) and execute it under both shells.
|
||||
*/
|
||||
describe('#4109 gate-check counts every dispatched reviewer under both shells', () => {
|
||||
for (const shell of SHELLS) {
|
||||
test(`[${shell.name}] 2 reviewers both skipped: dispatched=2 skipped=2 all_lanes_skipped=true`, (t) => {
|
||||
const fx = buildGateCheckFixture(['claude', 'codex'], ['claude', 'codex']);
|
||||
t.after(() => cleanup(fx.root));
|
||||
const body = extractGateCheckBlock()
|
||||
+ '\necho "{\\"dispatched_count\\":${DISPATCHED_COUNT:-0},\\"skipped_count\\":${SKIPPED_COUNT:-0},'
|
||||
+ '\\"all_lanes_skipped\\":\\"${ALL_LANES_SKIPPED:-false}\\",\\"total_lane_failure\\":\\"${TOTAL_LANE_FAILURE:-false}\\"}"\n';
|
||||
const res = runScript(shell, body, fx.root, fx.runDir, fx.env, REPO_ROOT);
|
||||
assert.strictEqual(res.status, 0, `block exited ${res.status}: ${res.stderr}`);
|
||||
const out = JSON.parse(res.stdout.trim());
|
||||
assert.strictEqual(out.dispatched_count, 2, `expected 2 dispatched slugs, got ${res.stdout}`);
|
||||
assert.strictEqual(out.skipped_count, 2, `expected 2 skipped slugs, got ${res.stdout}`);
|
||||
assert.strictEqual(out.all_lanes_skipped, 'true');
|
||||
assert.strictEqual(out.total_lane_failure, 'false');
|
||||
});
|
||||
|
||||
test(`[${shell.name}] mixed skip and total failure: dispatched=2 skipped=1 total_lane_failure=true`, (t) => {
|
||||
const fx = buildGateCheckFixture(['claude', 'codex'], ['claude']);
|
||||
t.after(() => cleanup(fx.root));
|
||||
const body = extractGateCheckBlock()
|
||||
+ '\necho "{\\"dispatched_count\\":${DISPATCHED_COUNT:-0},\\"skipped_count\\":${SKIPPED_COUNT:-0},'
|
||||
+ '\\"all_lanes_skipped\\":\\"${ALL_LANES_SKIPPED:-false}\\",\\"total_lane_failure\\":\\"${TOTAL_LANE_FAILURE:-false}\\"}"\n';
|
||||
const res = runScript(shell, body, fx.root, fx.runDir, fx.env, REPO_ROOT);
|
||||
assert.strictEqual(res.status, 0, `block exited ${res.status}: ${res.stderr}`);
|
||||
const out = JSON.parse(res.stdout.trim());
|
||||
assert.strictEqual(out.dispatched_count, 2, `expected 2 dispatched slugs, got ${res.stdout}`);
|
||||
assert.strictEqual(out.skipped_count, 1, `expected 1 skipped slug, got ${res.stdout}`);
|
||||
assert.strictEqual(out.all_lanes_skipped, 'false');
|
||||
assert.strictEqual(out.total_lane_failure, 'true');
|
||||
});
|
||||
|
||||
test(`[${shell.name}] single reviewer still counts correctly (boundary)`, (t) => {
|
||||
const fx = buildGateCheckFixture(['claude'], ['claude']);
|
||||
t.after(() => cleanup(fx.root));
|
||||
const body = extractGateCheckBlock()
|
||||
+ '\necho "{\\"dispatched_count\\":${DISPATCHED_COUNT:-0},\\"skipped_count\\":${SKIPPED_COUNT:-0},'
|
||||
+ '\\"all_lanes_skipped\\":\\"${ALL_LANES_SKIPPED:-false}\\",\\"total_lane_failure\\":\\"${TOTAL_LANE_FAILURE:-false}\\"}"\n';
|
||||
const res = runScript(shell, body, fx.root, fx.runDir, fx.env, REPO_ROOT);
|
||||
assert.strictEqual(res.status, 0, `block exited ${res.status}: ${res.stderr}`);
|
||||
const out = JSON.parse(res.stdout.trim());
|
||||
assert.strictEqual(out.dispatched_count, 1, `expected 1 dispatched slug, got ${res.stdout}`);
|
||||
assert.strictEqual(out.skipped_count, 1, `expected 1 skipped slug, got ${res.stdout}`);
|
||||
assert.strictEqual(out.all_lanes_skipped, 'true');
|
||||
});
|
||||
|
||||
test(`[${shell.name}] 3 reviewers all skipped (boundary)`, (t) => {
|
||||
const fx = buildGateCheckFixture(['claude', 'codex', 'gemini'], ['claude', 'codex', 'gemini']);
|
||||
t.after(() => cleanup(fx.root));
|
||||
const body = extractGateCheckBlock()
|
||||
+ '\necho "{\\"dispatched_count\\":${DISPATCHED_COUNT:-0},\\"skipped_count\\":${SKIPPED_COUNT:-0},'
|
||||
+ '\\"all_lanes_skipped\\":\\"${ALL_LANES_SKIPPED:-false}\\",\\"total_lane_failure\\":\\"${TOTAL_LANE_FAILURE:-false}\\"}"\n';
|
||||
const res = runScript(shell, body, fx.root, fx.runDir, fx.env, REPO_ROOT);
|
||||
assert.strictEqual(res.status, 0, `block exited ${res.status}: ${res.stderr}`);
|
||||
const out = JSON.parse(res.stdout.trim());
|
||||
assert.strictEqual(out.dispatched_count, 3, `expected 3 dispatched slugs, got ${res.stdout}`);
|
||||
assert.strictEqual(out.skipped_count, 3, `expected 3 skipped slugs, got ${res.stdout}`);
|
||||
assert.strictEqual(out.all_lanes_skipped, 'true');
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('#4109 invoke_reviewers dispatches every deduped reviewer exactly once under both shells', () => {
|
||||
for (const shell of SHELLS) {
|
||||
test(`[${shell.name}] 2 reviewers each dispatched exactly once`, (t) => {
|
||||
const fx = buildDispatchFixture('claude,codex');
|
||||
t.after(() => cleanup(fx.root));
|
||||
const body = DISPATCH_JOIN_STUB + '\n' + extractDispatchJoinBlock();
|
||||
const res = runScript(shell, body, fx.root, fx.runDir, fx.env, REPO_ROOT);
|
||||
assert.strictEqual(res.status, 0, `block exited ${res.status}: ${res.stderr}`);
|
||||
const lines = readDispatchLog(fx.runDir);
|
||||
assert.deepEqual(lines, ['claude', 'codex']);
|
||||
});
|
||||
|
||||
test(`[${shell.name}] duplicate slug in SELECTED_REVIEWERS deduped, dispatched once`, (t) => {
|
||||
const fx = buildDispatchFixture('claude,codex,claude');
|
||||
t.after(() => cleanup(fx.root));
|
||||
const body = DISPATCH_JOIN_STUB + '\n' + extractDispatchJoinBlock();
|
||||
const res = runScript(shell, body, fx.root, fx.runDir, fx.env, REPO_ROOT);
|
||||
assert.strictEqual(res.status, 0, `block exited ${res.status}: ${res.stderr}`);
|
||||
const lines = readDispatchLog(fx.runDir);
|
||||
assert.deepEqual(lines, ['claude', 'codex']);
|
||||
});
|
||||
|
||||
test(`[${shell.name}] 3 reviewers each dispatched exactly once (boundary)`, (t) => {
|
||||
const fx = buildDispatchFixture('claude,codex,gemini');
|
||||
t.after(() => cleanup(fx.root));
|
||||
const body = DISPATCH_JOIN_STUB + '\n' + extractDispatchJoinBlock();
|
||||
const res = runScript(shell, body, fx.root, fx.runDir, fx.env, REPO_ROOT);
|
||||
assert.strictEqual(res.status, 0, `block exited ${res.status}: ${res.stderr}`);
|
||||
const lines = readDispatchLog(fx.runDir);
|
||||
assert.deepEqual(lines, ['claude', 'codex', 'gemini']);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user