fix(#4097): evidence preservation no longer sweeps the run's own input copies into .review-diagnostics/ (#4329)

* test(#4097): failing-first regression — run input copies swept into .review-diagnostics

The present_results preserve+cleanup glob `_DIAG_MD=( "$RUN_DIR"/gsd-review-*.md )`
matches both lane outputs and the run's own assembled input copies (prompt,
instructions, roadmap, per-plan copies, project/context/research/requirements,
per-lane trimmed prompts) because both share the gsd-review- prefix.

Seed the full input-copy set into the existing runWriteReviewsFlow fixture and
assert (a) the diagnostics dir holds exactly the lane report + .err sidecar and
no input basenames, and (b) an inputs-only run creates no diagnostics dir at all
and still cleans up. Both RED against the shipped block.

* fix(#4097): preserve lane output only — exclude the run's input copies from evidence

The preserve+cleanup glob treated every gsd-review-*.md in RUN_DIR as lane
evidence, but the workflow itself writes the run's assembled INPUTS there under
the same prefix (prompt, instructions, roadmap, per-plan copies, project/
context/research/requirements, per-lane trimmed prompts). Filter _DIAG_MD by
basename against that closed input set instead.

Direct glob iteration + case filter — identical under bash and zsh (#4099/
#4109), nullglob-safe (#2962), and the exclusion list is closed and owned in
this step: a future input basename cannot silently rejoin the evidence set.
Lane reports, diagnostic stubs and non-empty .err sidecars are unchanged.

Emitted-Drift-Ack-Growth: review.md — #4097 narrows the present_results evidence-preservation glob: the closed input-basename exclusion list (case filter) plus its rationale note are deliberate additions so a future input basename cannot silently rejoin the evidence set.

* changeset(#4097): fixed — review diagnostics no longer sweep run input copies

* changeset(#4097): backfill PR number 4329

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-09-05 16:45:43 -04:00
committed by GitHub
parent 4c60879b5d
commit 2f920c5af3
3 changed files with 119 additions and 1 deletions

View File

@@ -0,0 +1,5 @@
---
type: Fixed
pr: 4329
---
**`/gsd-review` no longer sweeps the run's own prompt/plan copies into `.review-diagnostics/`** — after a review, the preserved diagnostics folder is now dominated by actual evidence (reviewer reports and stderr sidecars) instead of byte-identical duplicates of the prompt, instructions, roadmap, and every plan under review. (#4097)

View File

@@ -845,6 +845,19 @@ NOT a preservation failure. This copy is deliberately NOT part of the commit abo
step names only `{padded_phase}-REVIEWS.md` explicitly, never a directory glob, so step names only `{padded_phase}-REVIEWS.md` explicitly, never a directory glob, so
`.review-diagnostics/` is never swept into it. `.review-diagnostics/` is never swept into it.
**#4097: preserve lane OUTPUT, never the run's own input copies.** `RUN_DIR` holds not only
lane outputs — prompt assembly (the `gather_context`/section-copy step above) also writes the
run's assembled INPUTS there under the same `gsd-review-` prefix: the combined prompt, the
instructions/roadmap sections, a copy of every plan under review, the project/context/research/
requirements sections, and the per-lane trimmed prompts. Those are byte-identical duplicates of
files already committed under `.planning/`; sweeping them into `.review-diagnostics/` buries
the actual evidence under plan duplicates and grows the phase directory on every run. The
exclusion list below is CLOSED and owned here: this workflow itself writes every input
basename at prompt-assembly time, so a future input file CANNOT silently join the evidence
set — adding one means adding its stem to this list consciously. Lane slugs never begin with
any excluded stem (`prompt`, `instructions`, `plan-`, `project`, `roadmap`, `context`,
`research`, `requirements`), so a lane report can never be excluded by accident.
Preservation and cleanup MUST run in the same fenced block below (a shell variable cannot Preservation and cleanup MUST run in the same fenced block below (a shell variable cannot
survive across separate fences — each is its own process). `mkdir -p` and every `cp` are survive across separate fences — each is its own process). `mkdir -p` and every `cp` are
exit-status checked; `rm -rf "$RUN_DIR"` runs ONLY if nothing was preserved (nothing to exit-status checked; `rm -rf "$RUN_DIR"` runs ONLY if nothing was preserved (nothing to
@@ -858,7 +871,20 @@ shopt -s nullglob 2>/dev/null; setopt NULL_GLOB 2>/dev/null
RUN_DIR="{run_dir}" RUN_DIR="{run_dir}"
DIAG_DIR="{phase_dir}/.review-diagnostics" DIAG_DIR="{phase_dir}/.review-diagnostics"
_DIAG_MD=( "$RUN_DIR"/gsd-review-*.md ) # #4097: `gsd-review-*.md` matches BOTH lane outputs (reports, diagnostic stubs) and the
# run's own assembled input copies (see the #4097 note above). Filter by basename against
# the closed input set this workflow itself writes — direct glob iteration with a `case`
# filter, no string accumulator, identical under bash and zsh (#4099/#4109), and the
# `nullglob` set at the top of this fence keeps an empty RUN_DIR an empty array (#2962).
# `gsd-review-prompt*` deliberately covers BOTH the combined prompt (`gsd-review-prompt.md`)
# and the per-lane trimmed prompts (`gsd-review-prompt-<slug>.md`).
_DIAG_MD=()
for f in "$RUN_DIR"/gsd-review-*.md; do
case "$(basename "$f")" in
gsd-review-prompt*|gsd-review-instructions*|gsd-review-plan-*|gsd-review-project*|gsd-review-roadmap*|gsd-review-context*|gsd-review-research*|gsd-review-requirements*) ;;
*) _DIAG_MD+=("$f") ;;
esac
done
_DIAG_ERR=() _DIAG_ERR=()
for f in "$RUN_DIR"/gsd-review-*.err; do for f in "$RUN_DIR"/gsd-review-*.err; do
[ -s "$f" ] && _DIAG_ERR+=("$f") [ -s "$f" ] && _DIAG_ERR+=("$f")

View File

@@ -708,6 +708,14 @@ function runWriteReviewsFlow(t, opts) {
fs.writeFileSync(path.join(phaseDir, '.review-diagnostics'), 'blocking file, not a directory\n'); fs.writeFileSync(path.join(phaseDir, '.review-diagnostics'), 'blocking file, not a directory\n');
} }
// #4097: stage the run's OWN input copies exactly as prompt assembly
// (review.md's section-copy fence) writes them, so the preserve+cleanup
// block can be observed deciding input vs. evidence. `name` is a bare
// basename written into the fixture run dir.
for (const [name, content] of Object.entries(opts.seedFiles || {})) {
fs.writeFileSync(path.join(runDir, name), content);
}
if (opts.jsonlLines && opts.jsonlLines.length > 0) { if (opts.jsonlLines && opts.jsonlLines.length > 0) {
fs.writeFileSync( fs.writeFileSync(
path.join(runDir, 'gsd-review-lane-results.jsonl'), path.join(runDir, 'gsd-review-lane-results.jsonl'),
@@ -997,6 +1005,85 @@ describe('#3885 failed preservation leaves run_dir intact (no silent swallow)',
}); });
}); });
describe('#4097 the run\'s own input copies are not preserved as diagnostics', () => {
// The preserve+cleanup glob assumes every `gsd-review-*.md` in RUN_DIR is
// lane OUTPUT. But the workflow itself writes the run's assembled INPUTS
// into RUN_DIR under the same prefix at prompt-assembly time (review.md's
// section-copy fence): the combined prompt, instructions, roadmap, one copy
// of every plan under review, project/context/research/requirements
// sections, and the per-lane trimmed prompts. On a multi-plan phase those
// byte-identical `.planning/` duplicates bury the actual evidence (a
// handful of reviewer reports and `.err` sidecars) and grow the phase
// directory on every review run (#4097).
const INPUT_COPIES_4097 = {
'gsd-review-prompt.md': 'combined reviewer prompt\n',
'gsd-review-instructions.md': 'instructions section\n',
'gsd-review-roadmap.md': 'roadmap section\n',
'gsd-review-project.md': 'PROJECT.md copy\n',
'gsd-review-context.md': 'CONTEXT.md concatenation\n',
'gsd-review-research.md': 'RESEARCH.md concatenation\n',
'gsd-review-requirements.md': 'REQUIREMENTS.md copy\n',
'gsd-review-plan-12.6-01.md': 'plan 12.6-01 duplicate\n',
'gsd-review-plan-12.6-02.md': 'plan 12.6-02 duplicate\n',
// Per-lane trimmed prompt written by prepare_trimmed_prompt_for_reviewer
// — also a run input, also matched by the issue's `gsd-review-prompt*`
// exclusion prefix.
'gsd-review-prompt-claude.md': 'budget-trimmed prompt for lane claude\n',
};
test('inputCopiesAreNotSweptIntoDiagnostics_4097', (t) => {
const result = runWriteReviewsFlow(t, {
selected: 'claude',
jsonlLines: [{ slug: 'claude' }],
seedFiles: INPUT_COPIES_4097,
lanes: {
claude: { md: '# Claude review\nok\n', err: 'mild stderr warning\n' },
},
});
assert.equal(result.outcome, 'exited');
assert.equal(result.runDirExists, false, 'successful preservation must still clean up the run dir');
assert.equal(result.diagDirExists, true, 'real lane evidence must still be preserved');
const preserved = fs.existsSync(result.diagDir)
? fs.readdirSync(result.diagDir).sort()
: [];
// Only the lane's own output belongs in the diagnostics folder.
assert.deepEqual(
preserved,
['gsd-review-claude.err', 'gsd-review-claude.md'],
`diagnostics must hold exactly the lane report and stderr sidecar, not the run's input copies; got: ${preserved.join(', ')}`,
);
assert.equal(
fs.readFileSync(path.join(result.diagDir, 'gsd-review-claude.md'), 'utf-8'),
'# Claude review\nok\n',
'the lane report must be preserved byte-identically',
);
for (const inputName of Object.keys(INPUT_COPIES_4097)) {
assert.equal(
fs.existsSync(path.join(result.diagDir, inputName)),
false,
`input copy ${inputName} must NOT be swept into .review-diagnostics/ (#4097)`,
);
}
});
test('inputsOnlyRunLeavesNoDiagnosticsDir_4097', (t) => {
// Inputs only, no lane artifacts at all: inputs are not evidence, so
// this is the "nothing to preserve" branch — no diagnostics directory is
// created and cleanup proceeds (#4097 narrowing of #3352's glob).
const result = runWriteReviewsFlow(t, {
selected: 'claude',
jsonlLines: [],
seedFiles: INPUT_COPIES_4097,
lanes: {},
});
assert.equal(result.runDirExists, false, 'nothing to preserve is not a failure — run dir must still be removed');
assert.equal(result.diagDirExists, false, 'a run that produced only input copies has no evidence to preserve');
});
});
describe('#3352 preserved evidence is never swept into the commit (N6)', () => { describe('#3352 preserved evidence is never swept into the commit (N6)', () => {
test('preservedEvidenceIsNotSweptIntoTheCommit', (t) => { test('preservedEvidenceIsNotSweptIntoTheCommit', (t) => {
const result = runWriteReviewsFlow(t, { const result = runWriteReviewsFlow(t, {