* chore(#3677): checkpoint design artifacts (gitignored, dev-only) * test(#3677): add failing regression test for the crash-window duplicate-dispatch gap (RED) Independently re-traces resume-mode.md/planner-wave.md/worktree-dispatch.md/ merge-wave.md and src/quick-batch.cts's resumeBatch (lines 894-899) and confirms the prior research pass's Open Question 1: a coordinator crash between Step 6 (executor commits, SUMMARY.md written) and Step 7 (merge) leaves BATCH.json at "pending" with no STATE.md row yet (only written in Step 9), so --resume's eligibility re-derivation would dispatch a second executor into a new worktree for the same item, orphaning the first. This test asserts worktree-dispatch.md's Step 6 excludes an item whose SUMMARY.md already exists from the spawn set, mirroring planner-wave.md's existing PLAN.md-existence check one layer earlier. Fails against the current worktree-dispatch.md, which has no such guard. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §1 for the full trace and fix-location rationale. * fix(#3677): guard worktree-dispatch.md against re-dispatching an already-executed item (GREEN) worktree-dispatch.md's Step 6 re-derives eligibility every dispatch round via the same quick-batch resume call resume-mode.md uses, but had no check for "did this item already finish executing" the way planner-wave.md already checks "did this item already get planned" (PLAN.md existence) before re-planning. A coordinator crash between Step 6 (executor commits, SUMMARY.md written) and Step 7 (merge) left the item eligible for a second dispatch on --resume, orphaning the first worktree's real, already- committed work and silently losing it once the second executor's SUMMARY.md write clobbered the first at the same item_dir path. Adds a SUMMARY.md-existence exclusion before spawn-plan is computed, symmetric to planner-wave.md's PLAN.md check. The excluded item is not lost: merge-wave.md's own mergeable-wave criterion (status=pending, SUMMARY.md on disk, not yet merged) already picks it up independently of this eligible/spawn list. Workflow-prose-only fix — touches no already-merged/reviewed .cts module. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §1 for the fix-location rationale (why not resumeBatch itself). * test(#3677): add real-git coverage for worktree-ownership tampering, scope drift, and submodules Closes the three coverage gaps identified in 40-design.md §2/§3 (#3677, epic #3344 Phase 5's own AC bullets: "arbitrary-worktree ownership attempts", "scope drift", "submodules"): - Arbitrary-worktree ownership tampering: a manifest entry naming a non-agent branch is silently dropped at normalization before any git subprocess runs; a manifest entry naming a plausible agent-branch that was never actually created by this repo's own worktree.create (a genuinely foreign repo/branch) is blocked via base_mismatch. Both leave the foreign location and repoRoot's HEAD provably untouched. - Advisory scope drift: a committed path outside declared files_modified still merges successfully (advisory, never blocking) while surfacing a scope_out_of_declared warning naming the drifted path; an exact declared-scope match produces zero warnings (boundary case). - Real .gitmodules submodule integration: a repo containing a real local git submodule merges cleanly through executeWorktreeWaveCleanupPlan for an unrelated plan; a real gitlink pointer bump (declared) merges cleanly with the superproject tree reflecting the new pinned commit; an undeclared bump is advisory-only and surfaces a scope warning naming vendor/sub, same as any other undeclared modification. No src/*.cts changes — all three gaps were coverage-only; the underlying primitives already behaved correctly (independently verified against real git subprocess output before writing each assertion). * docs(#3677): document how to diagnose a preserved quick-batch worktree Extends the one-sentence "worktree is preserved (never deleted)" mention into a concrete diagnosis procedure: where the preserved directory is, how to read the executor's real commits/diff against the plan's declared files_modified, how to read the item's own SUMMARY.md independent of merge outcome, how to manually merge-and-clean-up or discard, and how to re-run --resume afterward. Also documents that a SUMMARY.md-written-but-still- pending item (the crash-window case fixed in this same PR) needs no manual intervention — --resume routes it straight to the merge step. * chore(#3677): checkpoint final acceptance-evidence mapping (gitignored, dev-only) * fix(#3677): make crash-window duplicate-dispatch guard behaviorally provable and durably recoverable Orthogonal review (Spec finding): the crash-window regression test added earlier this phase only asserted readStep('worktree-dispatch.md') + regex matches against the markdown prose — proving the DOCUMENTATION says the right thing, never that the runtime condition (pending status + on-disk SUMMARY.md + absent STATE row) is actually handled correctly. #3677's own "Alternatives considered" explicitly rejects "document recovery without fault injection" for exactly this reason. Extracts the filtering decision into a pure, independently testable function, filterAlreadyExecuted(eligibleIds, executedIds) in src/quick-batch-dispatch.cts, wired to a new `quick-batch filter-executed` CLI verb (src/quick-batch-command-router.cts) — the same pure-decision- then-CLI-wired pattern computeSpawnPlan/computeMergeOrder already establish. worktree-dispatch.md now calls this verb explicitly instead of only describing the decision in prose. A genuine fixture-based test in tests/quick-batch.test.cjs constructs a REAL BATCH.json (createBatch), writes a REAL SUMMARY.md on disk at the item's real item_dir, calls the REAL resumeBatch, and proves both that resumeBatch alone still reports the item eligible AND that filterAlreadyExecuted (fed a real filesystem check) correctly excludes it. The prior prose-assertion tests are kept — they now prove the workflow markdown is correctly WIRED to the verb — but are no longer the only proof. Self-discovered defect while building that fixture (fixed inline, not deferred): tracing merge-wave.md against /gsd:quick's own prior art (QUICK_WORKTREE_MANIFEST=$(mktemp ...), quick.md:415) showed $QUICK_BATCH_WORKTREE_MANIFEST is a fresh PER-PROCESS temp file. A resumed coordinator correctly does not re-dispatch an already-executed item (this fix), but nothing durably recorded that item's worktree_path/branch/base either — Step 7 in the resumed process would have had no data to build its cleanup-wave entry from. Adds dispatched_worktree/dispatched_branch/ dispatched_base to QuickBatchItem (src/quick-batch.cts) — deliberately NOT a reuse of the pre-existing `worktree` field, whose loadBatch validation requires the path to exist on disk (verified empirically: reusing it made the batch permanently unloadable the moment a legitimately-merged worktree was removed). worktree-dispatch.md persists the triple once a worktree is created; merge-wave.md falls back to it when the ephemeral manifest lacks an entry, clears it after a successful merge, and fails closed rather than guessing if no record exists anywhere. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §9.1 and §9.3 for the full trace, empirical verification notes, and rejected alternatives (reusing `worktree` directly). * test(#3677): prove the arbitrary-worktree-ownership boundary against two real sibling worktrees Orthogonal review (Security finding): the two existing ownership-tampering tests didn't test ownership — one was trivially rejected by WORKTREE_AGENT_BRANCH_RE's shape check before any git call (proves branch- NAME filtering, not ownership), the other pointed at a wholly separate, never-linked foreign repo, so merge-base failed immediately because the branch didn't exist as a ref at all. Neither exercised the real scenario: a manifest entry whose worktree_path/branch are swapped to point at a DIFFERENT, GENUINELY-REGISTERED sibling worktree of the SAME repoRoot, with a branch name passing the shape check and a base in allowed_bases. Investigated executeWorktreeWaveCleanupPlan (src/worktree-safety.cts) directly: this is NOT a reachable gap. Git enforces branch-per-worktree uniqueness, so a swapped-in entry.branch can only match worktree_path's ACTUAL checked-out branch if it names that sibling's own real, uniquely- generated branch name — which manifest tampering confined to one batch's own record has no way to know (branch names are agent-<quick_id>[-<timestamp>]-shaped, and quick_id allocation is collision-checked GLOBALLY across every existing quick task and batch, not merely within one batch). Adds a stronger test that empirically proves this: two REAL, concurrently- alive sibling worktrees of the same repo (both via real `git worktree add`, both WORKTREE_AGENT_BRANCH_RE-passing, both sharing one merge-base), with worktree_path/branch swapped between them in both directions. Both attempts are blocked via branch_mismatch; both real worktrees, their branches, and one sibling's real uncommitted-to-main commit survive completely untouched. Supplements (does not replace) the original two tests, which still prove distinct, real boundaries. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §9.2 for the full trace, including the one explicitly-documented (not fixed) trust boundary this investigation surfaced: the primitive defends against fabricated data, not a caller bug that misattributes a real-but-wrong item's own triple to a different item. * chore(#3677): checkpoint design-doc addendum for review pass 2 findings (gitignored, dev-only) * docs(#3677): add changeset for PR 4240 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
@@ -43,6 +43,14 @@ const { appendQuickTaskRow } = require('../gsd-core/bin/lib/markdown-table.cjs')
|
||||
const { makeFakeClock } = require('./helpers/clock.cjs');
|
||||
const { runGsdTools, cleanup } = require('./helpers.cjs');
|
||||
|
||||
// #3677: crash-window duplicate-dispatch guard — combines resumeBatch (this
|
||||
// module) with filterAlreadyExecuted (the pure decision extracted into
|
||||
// quick-batch-dispatch.cts) and a REAL on-disk SUMMARY.md, the same three
|
||||
// pieces worktree-dispatch.md's Step 6 wires together at runtime.
|
||||
const { filterAlreadyExecuted } = require('../gsd-core/bin/lib/quick-batch-dispatch.cjs');
|
||||
const { generateSlugInternal } = require('../gsd-core/bin/lib/core-utils.cjs');
|
||||
const { planningPaths } = require('../gsd-core/bin/lib/planning-workspace.cjs');
|
||||
|
||||
// ─── Shared fixtures ────────────────────────────────────────────────────────────
|
||||
|
||||
function mkTmpProject() {
|
||||
@@ -738,6 +746,118 @@ describe('quick-batch: exactly-once STATE completion', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ─── crash-window duplicate-dispatch guard (#3677, epic #3344 Phase 5) ─────
|
||||
//
|
||||
// A DIFFERENT crash window than row 31 above: row 31 crashes AFTER the
|
||||
// STATE.md row is written (Step 9) but before BATCH.json records `complete`
|
||||
// — resumeBatch's own hasQuickTaskRow detection covers that one directly.
|
||||
// This one crashes EARLIER — after Step 6 (executor committed, SUMMARY.md
|
||||
// written) but before Step 7 (merge), so NO STATE.md row exists yet.
|
||||
// resumeBatch alone cannot detect this; worktree-dispatch.md's Step 6 must
|
||||
// additionally check for an on-disk SUMMARY.md via filterAlreadyExecuted.
|
||||
// Real fixture (real BATCH.json via createBatch, real SUMMARY.md file on
|
||||
// disk, real resumeBatch call), not a prose/structural proxy — see
|
||||
// `.gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md` §1.
|
||||
|
||||
describe('quick-batch: crash-window duplicate-dispatch guard (#3677) — SUMMARY.md written but BATCH.json still pending', () => {
|
||||
test('resumeBatch alone still reports the crashed item eligible; filterAlreadyExecuted (fed a REAL on-disk SUMMARY.md check) excludes it from spawnEligible', () => {
|
||||
const dir = mkTmpProject();
|
||||
writeState(dir, stateWithQuickTasksSection());
|
||||
try {
|
||||
const created = createBatch(dir, [{ description: 'crashed item' }, { description: 'clean item' }]);
|
||||
assert.equal(created.ok, true);
|
||||
const [crashed, clean] = created.value.manifest.items;
|
||||
|
||||
// Simulate the real Step-6-to-Step-7 crash window: the executor
|
||||
// finished (a real commit, in the real design; here a real SUMMARY.md
|
||||
// file is what matters) but the coordinator crashed before Step 7's
|
||||
// merge. BATCH.json is untouched — still `pending` — and no STATE.md
|
||||
// row exists (that is written only in Step 9, well after this point).
|
||||
const slug = generateSlugInternal(crashed.description);
|
||||
const itemDir = path.join(planningPaths(dir).quick, `${crashed.quick_id}-${slug}`);
|
||||
fs.mkdirSync(itemDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(itemDir, `${crashed.quick_id}-SUMMARY.md`),
|
||||
'---\nstatus: complete\n---\n\n# Summary\n\nDid the thing.\n',
|
||||
);
|
||||
|
||||
const preResume = loadBatch(dir, created.value.batchId);
|
||||
assert.equal(
|
||||
preResume.value.items.find((it) => it.quick_id === crashed.quick_id).status,
|
||||
'pending',
|
||||
'BATCH.json is untouched by the crash — still pending',
|
||||
);
|
||||
|
||||
// --resume re-derives eligibility exactly as worktree-dispatch.md's
|
||||
// Step 6 substep 1 does. BOTH items come back eligible: resumeBatch's
|
||||
// own crash-window detection (hasQuickTaskRow) only fires on a
|
||||
// STATE.md row, which does not exist yet for the crashed item.
|
||||
const resumed = resumeBatch(dir, created.value.batchId);
|
||||
assert.equal(resumed.ok, true);
|
||||
assert.deepEqual(
|
||||
resumed.value.eligible.slice().sort(),
|
||||
[clean.quick_id, crashed.quick_id].sort(),
|
||||
'resumeBatch alone does not know the crashed item already finished executing',
|
||||
);
|
||||
|
||||
// worktree-dispatch.md's own crash-window guard: a REAL filesystem
|
||||
// check for each eligible id's SUMMARY.md (same item_dir derivation
|
||||
// via generateSlugInternal every other step uses), fed into the pure
|
||||
// filterAlreadyExecuted decision.
|
||||
const executedIds = resumed.value.eligible.filter((quickId) => {
|
||||
const item = resumed.value.manifest.items.find((it) => it.quick_id === quickId);
|
||||
const itemSlug = generateSlugInternal(item.description);
|
||||
const summaryPath = path.join(planningPaths(dir).quick, `${quickId}-${itemSlug}`, `${quickId}-SUMMARY.md`);
|
||||
return fs.existsSync(summaryPath);
|
||||
});
|
||||
assert.deepEqual(executedIds, [crashed.quick_id], 'only the crashed item has a real on-disk SUMMARY.md');
|
||||
|
||||
const filtered = filterAlreadyExecuted(resumed.value.eligible, executedIds);
|
||||
assert.deepEqual(filtered.spawnEligible, [clean.quick_id], 'the crashed item must NOT be re-dispatched into a second worktree');
|
||||
assert.deepEqual(filtered.alreadyExecuted, [crashed.quick_id]);
|
||||
|
||||
// The crashed item is not lost — it is exactly the shape
|
||||
// merge-wave.md's OWN independent criterion (status=pending,
|
||||
// SUMMARY.md on disk, not yet merged) already expects to find.
|
||||
assert.equal(fs.existsSync(path.join(itemDir, `${crashed.quick_id}-SUMMARY.md`)), true);
|
||||
assert.equal(
|
||||
loadBatch(dir, created.value.batchId).value.items.find((it) => it.quick_id === crashed.quick_id).status,
|
||||
'pending',
|
||||
'filterAlreadyExecuted performs no BATCH.json write — merge-wave.md still finds it pending with an on-disk SUMMARY.md',
|
||||
);
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
test('an item with NO on-disk SUMMARY.md is unaffected — the guard only excludes genuinely already-executed items', () => {
|
||||
const dir = mkTmpProject();
|
||||
writeState(dir, stateWithQuickTasksSection());
|
||||
try {
|
||||
const created = createBatch(dir, [{ description: 'never started' }]);
|
||||
assert.equal(created.ok, true);
|
||||
const [item] = created.value.manifest.items;
|
||||
|
||||
const resumed = resumeBatch(dir, created.value.batchId);
|
||||
assert.deepEqual(resumed.value.eligible, [item.quick_id]);
|
||||
|
||||
// No SUMMARY.md written anywhere — the filesystem check must find nothing.
|
||||
const executedIds = resumed.value.eligible.filter((quickId) => {
|
||||
const it = resumed.value.manifest.items.find((x) => x.quick_id === quickId);
|
||||
const s = generateSlugInternal(it.description);
|
||||
return fs.existsSync(path.join(planningPaths(dir).quick, `${quickId}-${s}`, `${quickId}-SUMMARY.md`));
|
||||
});
|
||||
assert.deepEqual(executedIds, []);
|
||||
|
||||
const filtered = filterAlreadyExecuted(resumed.value.eligible, executedIds);
|
||||
assert.deepEqual(filtered.spawnEligible, [item.quick_id], 'an item that never started must still be dispatched normally');
|
||||
assert.deepEqual(filtered.alreadyExecuted, []);
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── 34-35: Independence + regression ────────────────────────────────────────────
|
||||
|
||||
describe('quick-batch: independence from scanQuickTasks, regression on existing quick paths', () => {
|
||||
@@ -1071,6 +1191,114 @@ describe('quick-batch: updateBatchItems — basic mutation + persistence', () =>
|
||||
});
|
||||
});
|
||||
|
||||
// #3677: durable worktree-recovery fields (dispatched_worktree/branch/base)
|
||||
// — the crash-window duplicate-dispatch guard's fallback for a RESUMED
|
||||
// coordinator process whose own ephemeral $QUICK_BATCH_WORKTREE_MANIFEST
|
||||
// (a fresh mktemp file every process) has no entry for an item dispatched
|
||||
// by a PRIOR, now-dead process. Deliberately separate fields from the
|
||||
// pre-existing `worktree` (which requires the path to exist on disk via
|
||||
// loadBatch's validation) — these three carry NO existence check, because
|
||||
// their entire purpose is to stay readable (and clearable) after a
|
||||
// legitimate post-merge worktree removal.
|
||||
describe('quick-batch: updateBatchItems — durable worktree-recovery fields (#3677)', () => {
|
||||
test('persists dispatchedWorktree/dispatchedBranch/dispatchedBase and a fresh loadBatch (simulating a new coordinator process) reads them back', () => {
|
||||
const dir = mkTmpProject();
|
||||
try {
|
||||
const created = createBatch(dir, [{ description: 'a' }, { description: 'b' }]);
|
||||
assert.equal(created.ok, true);
|
||||
const [itemA, itemB] = created.value.manifest.items;
|
||||
assert.equal(itemA.dispatched_worktree, null, 'null until Step 6 actually dispatches');
|
||||
assert.equal(itemA.dispatched_branch, null);
|
||||
assert.equal(itemA.dispatched_base, null);
|
||||
|
||||
const wtDir = path.join(os.tmpdir(), `qb-dispatched-wt-${itemA.quick_id}`);
|
||||
fs.mkdirSync(wtDir, { recursive: true });
|
||||
try {
|
||||
const result = updateBatchItems(dir, created.value.batchId, [
|
||||
{ quickId: itemA.quick_id, dispatchedWorktree: wtDir, dispatchedBranch: `agent-${itemA.quick_id}`, dispatchedBase: 'deadbeef' },
|
||||
]);
|
||||
assert.equal(result.ok, true, result.ok ? '' : result.reason);
|
||||
|
||||
// A FRESH loadBatch call — standing in for a brand-new coordinator
|
||||
// process (its own ephemeral worktree manifest would be empty) —
|
||||
// must still recover the durable triple.
|
||||
const reloaded = loadBatch(dir, created.value.batchId);
|
||||
assert.equal(reloaded.ok, true);
|
||||
const reloadedA = reloaded.value.items.find((it) => it.quick_id === itemA.quick_id);
|
||||
assert.equal(reloadedA.dispatched_worktree, wtDir);
|
||||
assert.equal(reloadedA.dispatched_branch, `agent-${itemA.quick_id}`);
|
||||
assert.equal(reloadedA.dispatched_base, 'deadbeef');
|
||||
|
||||
// The untouched sibling item is unaffected.
|
||||
const reloadedB = reloaded.value.items.find((it) => it.quick_id === itemB.quick_id);
|
||||
assert.equal(reloadedB.dispatched_worktree, null);
|
||||
} finally {
|
||||
cleanup(wtDir);
|
||||
}
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
test('clearing the triple (post-merge) succeeds and remains loadable even though the worktree path no longer exists on disk', () => {
|
||||
const dir = mkTmpProject();
|
||||
try {
|
||||
const created = createBatch(dir, [{ description: 'a' }]);
|
||||
assert.equal(created.ok, true);
|
||||
const [itemA] = created.value.manifest.items;
|
||||
|
||||
const wtDir = path.join(os.tmpdir(), `qb-dispatched-wt-clear-${itemA.quick_id}`);
|
||||
fs.mkdirSync(wtDir, { recursive: true });
|
||||
const set = updateBatchItems(dir, created.value.batchId, [
|
||||
{ quickId: itemA.quick_id, dispatchedWorktree: wtDir, dispatchedBranch: `agent-${itemA.quick_id}`, dispatchedBase: 'deadbeef' },
|
||||
]);
|
||||
assert.equal(set.ok, true);
|
||||
|
||||
// Merge-wave.md's own post-merge step: the real worktree is removed
|
||||
// (git worktree remove, out of scope here), THEN the durable triple
|
||||
// is cleared.
|
||||
cleanup(wtDir);
|
||||
const cleared = updateBatchItems(dir, created.value.batchId, [
|
||||
{ quickId: itemA.quick_id, dispatchedWorktree: null, dispatchedBranch: null, dispatchedBase: null },
|
||||
]);
|
||||
assert.equal(cleared.ok, true, 'clearing must succeed even though the path no longer exists on disk — this is exactly why these fields are NOT the pre-existing `worktree` field');
|
||||
|
||||
const reloaded = loadBatch(dir, created.value.batchId);
|
||||
assert.equal(reloaded.ok, true, 'a SUBSEQUENT loadBatch must not fail on the now-null dispatched_worktree field');
|
||||
const reloadedA = reloaded.value.items.find((it) => it.quick_id === itemA.quick_id);
|
||||
assert.equal(reloadedA.dispatched_worktree, null);
|
||||
assert.equal(reloadedA.dispatched_branch, null);
|
||||
assert.equal(reloadedA.dispatched_base, null);
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects a non-string, non-null dispatchedBranch without persisting anything (schema validation)', () => {
|
||||
const dir = mkTmpProject();
|
||||
try {
|
||||
const created = createBatch(dir, [{ description: 'a' }]);
|
||||
assert.equal(created.ok, true);
|
||||
|
||||
// updateBatchItems itself performs no shape check on these fields
|
||||
// (same "opaque identifier" convention as `commit`) — the schema
|
||||
// guard fires on the NEXT loadBatch, exactly like every other
|
||||
// malformed field in this manifest. Simulate a corrupt write directly
|
||||
// to prove loadBatch's validateBatchSchema catches it.
|
||||
const manifestPath = path.join(dir, '.planning', 'quick-batches', created.value.batchId, 'BATCH.json');
|
||||
const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf-8'));
|
||||
manifest.items[0].dispatched_branch = 12345;
|
||||
fs.writeFileSync(manifestPath, JSON.stringify(manifest, null, 2));
|
||||
|
||||
const reloaded = loadBatch(dir, created.value.batchId);
|
||||
assert.equal(reloaded.ok, false);
|
||||
assert.match(reloaded.reason, /dispatched_branch/);
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('quick-batch: updateBatchItems — fails closed, never persists on a bad update', () => {
|
||||
test('rejects an unknown quickId without persisting anything', () => {
|
||||
const dir = mkTmpProject();
|
||||
|
||||
Reference in New Issue
Block a user