* chore(#3677): checkpoint design artifacts (gitignored, dev-only) * test(#3677): add failing regression test for the crash-window duplicate-dispatch gap (RED) Independently re-traces resume-mode.md/planner-wave.md/worktree-dispatch.md/ merge-wave.md and src/quick-batch.cts's resumeBatch (lines 894-899) and confirms the prior research pass's Open Question 1: a coordinator crash between Step 6 (executor commits, SUMMARY.md written) and Step 7 (merge) leaves BATCH.json at "pending" with no STATE.md row yet (only written in Step 9), so --resume's eligibility re-derivation would dispatch a second executor into a new worktree for the same item, orphaning the first. This test asserts worktree-dispatch.md's Step 6 excludes an item whose SUMMARY.md already exists from the spawn set, mirroring planner-wave.md's existing PLAN.md-existence check one layer earlier. Fails against the current worktree-dispatch.md, which has no such guard. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §1 for the full trace and fix-location rationale. * fix(#3677): guard worktree-dispatch.md against re-dispatching an already-executed item (GREEN) worktree-dispatch.md's Step 6 re-derives eligibility every dispatch round via the same quick-batch resume call resume-mode.md uses, but had no check for "did this item already finish executing" the way planner-wave.md already checks "did this item already get planned" (PLAN.md existence) before re-planning. A coordinator crash between Step 6 (executor commits, SUMMARY.md written) and Step 7 (merge) left the item eligible for a second dispatch on --resume, orphaning the first worktree's real, already- committed work and silently losing it once the second executor's SUMMARY.md write clobbered the first at the same item_dir path. Adds a SUMMARY.md-existence exclusion before spawn-plan is computed, symmetric to planner-wave.md's PLAN.md check. The excluded item is not lost: merge-wave.md's own mergeable-wave criterion (status=pending, SUMMARY.md on disk, not yet merged) already picks it up independently of this eligible/spawn list. Workflow-prose-only fix — touches no already-merged/reviewed .cts module. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §1 for the fix-location rationale (why not resumeBatch itself). * test(#3677): add real-git coverage for worktree-ownership tampering, scope drift, and submodules Closes the three coverage gaps identified in 40-design.md §2/§3 (#3677, epic #3344 Phase 5's own AC bullets: "arbitrary-worktree ownership attempts", "scope drift", "submodules"): - Arbitrary-worktree ownership tampering: a manifest entry naming a non-agent branch is silently dropped at normalization before any git subprocess runs; a manifest entry naming a plausible agent-branch that was never actually created by this repo's own worktree.create (a genuinely foreign repo/branch) is blocked via base_mismatch. Both leave the foreign location and repoRoot's HEAD provably untouched. - Advisory scope drift: a committed path outside declared files_modified still merges successfully (advisory, never blocking) while surfacing a scope_out_of_declared warning naming the drifted path; an exact declared-scope match produces zero warnings (boundary case). - Real .gitmodules submodule integration: a repo containing a real local git submodule merges cleanly through executeWorktreeWaveCleanupPlan for an unrelated plan; a real gitlink pointer bump (declared) merges cleanly with the superproject tree reflecting the new pinned commit; an undeclared bump is advisory-only and surfaces a scope warning naming vendor/sub, same as any other undeclared modification. No src/*.cts changes — all three gaps were coverage-only; the underlying primitives already behaved correctly (independently verified against real git subprocess output before writing each assertion). * docs(#3677): document how to diagnose a preserved quick-batch worktree Extends the one-sentence "worktree is preserved (never deleted)" mention into a concrete diagnosis procedure: where the preserved directory is, how to read the executor's real commits/diff against the plan's declared files_modified, how to read the item's own SUMMARY.md independent of merge outcome, how to manually merge-and-clean-up or discard, and how to re-run --resume afterward. Also documents that a SUMMARY.md-written-but-still- pending item (the crash-window case fixed in this same PR) needs no manual intervention — --resume routes it straight to the merge step. * chore(#3677): checkpoint final acceptance-evidence mapping (gitignored, dev-only) * fix(#3677): make crash-window duplicate-dispatch guard behaviorally provable and durably recoverable Orthogonal review (Spec finding): the crash-window regression test added earlier this phase only asserted readStep('worktree-dispatch.md') + regex matches against the markdown prose — proving the DOCUMENTATION says the right thing, never that the runtime condition (pending status + on-disk SUMMARY.md + absent STATE row) is actually handled correctly. #3677's own "Alternatives considered" explicitly rejects "document recovery without fault injection" for exactly this reason. Extracts the filtering decision into a pure, independently testable function, filterAlreadyExecuted(eligibleIds, executedIds) in src/quick-batch-dispatch.cts, wired to a new `quick-batch filter-executed` CLI verb (src/quick-batch-command-router.cts) — the same pure-decision- then-CLI-wired pattern computeSpawnPlan/computeMergeOrder already establish. worktree-dispatch.md now calls this verb explicitly instead of only describing the decision in prose. A genuine fixture-based test in tests/quick-batch.test.cjs constructs a REAL BATCH.json (createBatch), writes a REAL SUMMARY.md on disk at the item's real item_dir, calls the REAL resumeBatch, and proves both that resumeBatch alone still reports the item eligible AND that filterAlreadyExecuted (fed a real filesystem check) correctly excludes it. The prior prose-assertion tests are kept — they now prove the workflow markdown is correctly WIRED to the verb — but are no longer the only proof. Self-discovered defect while building that fixture (fixed inline, not deferred): tracing merge-wave.md against /gsd:quick's own prior art (QUICK_WORKTREE_MANIFEST=$(mktemp ...), quick.md:415) showed $QUICK_BATCH_WORKTREE_MANIFEST is a fresh PER-PROCESS temp file. A resumed coordinator correctly does not re-dispatch an already-executed item (this fix), but nothing durably recorded that item's worktree_path/branch/base either — Step 7 in the resumed process would have had no data to build its cleanup-wave entry from. Adds dispatched_worktree/dispatched_branch/ dispatched_base to QuickBatchItem (src/quick-batch.cts) — deliberately NOT a reuse of the pre-existing `worktree` field, whose loadBatch validation requires the path to exist on disk (verified empirically: reusing it made the batch permanently unloadable the moment a legitimately-merged worktree was removed). worktree-dispatch.md persists the triple once a worktree is created; merge-wave.md falls back to it when the ephemeral manifest lacks an entry, clears it after a successful merge, and fails closed rather than guessing if no record exists anywhere. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §9.1 and §9.3 for the full trace, empirical verification notes, and rejected alternatives (reusing `worktree` directly). * test(#3677): prove the arbitrary-worktree-ownership boundary against two real sibling worktrees Orthogonal review (Security finding): the two existing ownership-tampering tests didn't test ownership — one was trivially rejected by WORKTREE_AGENT_BRANCH_RE's shape check before any git call (proves branch- NAME filtering, not ownership), the other pointed at a wholly separate, never-linked foreign repo, so merge-base failed immediately because the branch didn't exist as a ref at all. Neither exercised the real scenario: a manifest entry whose worktree_path/branch are swapped to point at a DIFFERENT, GENUINELY-REGISTERED sibling worktree of the SAME repoRoot, with a branch name passing the shape check and a base in allowed_bases. Investigated executeWorktreeWaveCleanupPlan (src/worktree-safety.cts) directly: this is NOT a reachable gap. Git enforces branch-per-worktree uniqueness, so a swapped-in entry.branch can only match worktree_path's ACTUAL checked-out branch if it names that sibling's own real, uniquely- generated branch name — which manifest tampering confined to one batch's own record has no way to know (branch names are agent-<quick_id>[-<timestamp>]-shaped, and quick_id allocation is collision-checked GLOBALLY across every existing quick task and batch, not merely within one batch). Adds a stronger test that empirically proves this: two REAL, concurrently- alive sibling worktrees of the same repo (both via real `git worktree add`, both WORKTREE_AGENT_BRANCH_RE-passing, both sharing one merge-base), with worktree_path/branch swapped between them in both directions. Both attempts are blocked via branch_mismatch; both real worktrees, their branches, and one sibling's real uncommitted-to-main commit survive completely untouched. Supplements (does not replace) the original two tests, which still prove distinct, real boundaries. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §9.2 for the full trace, including the one explicitly-documented (not fixed) trust boundary this investigation surfaced: the primitive defends against fabricated data, not a caller bug that misattributes a real-but-wrong item's own triple to a different item. * chore(#3677): checkpoint design-doc addendum for review pass 2 findings (gitignored, dev-only) * docs(#3677): add changeset for PR 4240 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
@@ -45,7 +45,24 @@ _GSD_SHIM_NAME="gsd-tools.cjs"; _GSD_RUNTIME_ROOT="${RUNTIME_DIR:-$(git rev-pars
|
||||
done
|
||||
```
|
||||
(`$WT_PATH`/`$WT_BRANCH`/`$EXPECTED_BASE` per item come from the recorded
|
||||
`$QUICK_BATCH_WORKTREE_MANIFEST` entry Step 6 wrote for that `agent_id`.)
|
||||
`$QUICK_BATCH_WORKTREE_MANIFEST` entry Step 6 wrote for that `agent_id`
|
||||
THIS process, when present.
|
||||
|
||||
**Durable fallback (#3677):** for an item Step 6 did NOT dispatch this
|
||||
process — the crash-window guard correctly skipped it because
|
||||
`SUMMARY.md` already existed from a PRIOR, now-dead coordinator process —
|
||||
`$QUICK_BATCH_WORKTREE_MANIFEST` has no entry for it at all (it is a
|
||||
fresh per-process `mktemp` file). Read `$WT_PATH`/`$WT_BRANCH`/
|
||||
`$EXPECTED_BASE` from that item's OWN durable
|
||||
`dispatched_worktree`/`dispatched_branch`/`dispatched_base` fields in
|
||||
`$BATCH_MANIFEST_JSON` instead — persisted by Step 6's own durable-
|
||||
persistence step at the time it actually created the worktree, in
|
||||
whichever process that was. If ALL THREE are still `null` (the item was
|
||||
never durably recorded — should not happen once Step 6 always persists
|
||||
on dispatch, but fail closed rather than guess): route this entry via
|
||||
`merge-routing --kind merge_failed --detail "missing durable worktree
|
||||
record"` the same as any other blocked entry below, and do NOT attempt
|
||||
the cleanup-wave call for it.)
|
||||
|
||||
4. **Merge, one at a time, via the SAME bounded primitive every other worktree
|
||||
consumer uses** (never hand-roll `git merge`):
|
||||
@@ -61,7 +78,13 @@ _GSD_SHIM_NAME="gsd-tools.cjs"; _GSD_RUNTIME_ROOT="${RUNTIME_DIR:-$(git rev-pars
|
||||
5. **Route each entry's result:**
|
||||
- `status == "merged_removed"`: success. Mark the item's completion pending
|
||||
(Step 9 calls `quick-batch complete` for it — do NOT call it here; a
|
||||
`--validate` item still has verification ahead of it).
|
||||
`--validate` item still has verification ahead of it). **Clear the
|
||||
durable worktree-recovery fields now (#3677)** — the worktree no
|
||||
longer exists on disk, so its `dispatched_worktree`/`dispatched_branch`/
|
||||
`dispatched_base` must not keep pointing at a removed path:
|
||||
```bash
|
||||
gsd_run quick-batch update --batch "$BATCH_ID" --updates '[{"quickId":"'"$quick_id"'","dispatchedWorktree":null,"dispatchedBranch":null,"dispatchedBase":null}]'
|
||||
```
|
||||
- Any other status: route via
|
||||
```bash
|
||||
gsd_run quick-batch merge-routing --kind merge_failed --detail "$reason" --raw
|
||||
|
||||
@@ -37,6 +37,36 @@ EXEC_CONCURRENCY=$(printf '%s' "$QB_EXEC_CONC_JSON" | node -e 'let s="";process.
|
||||
Parse `eligible` (quick ids ready to execute — every dependency already
|
||||
`complete`) and refresh `$BATCH_MANIFEST_JSON` from its `manifest`.
|
||||
|
||||
**Crash-window guard (mirrors `planner-wave.md`'s PLAN.md-existence
|
||||
check one layer earlier — #3677):** `quick-batch resume`'s `eligible` is
|
||||
purely status/dependency-derived; it does NOT know an item already
|
||||
finished executing. A coordinator crash between this step returning
|
||||
(executor committed, `SUMMARY.md` written) and Step 7's merge leaves
|
||||
`BATCH.json` at `pending` with no STATE.md row yet (that row is written
|
||||
only in Step 9) — so on `--resume`, such an item still comes back
|
||||
`eligible` here. Before computing `spawn-plan`, determine which eligible
|
||||
items already have `${item_dir}/${quick_id}-SUMMARY.md` on disk (same
|
||||
`item_dir` derivation via `generate-slug` every other step uses), then
|
||||
let the PURE `quick-batch filter-executed` verb
|
||||
(`filterAlreadyExecuted`, `src/quick-batch-dispatch.cts`) decide which
|
||||
ids are actually safe to spawn — never re-derive that split inline:
|
||||
```bash
|
||||
EXECUTED_IDS_JSON="[]" # JSON array of quick_ids whose SUMMARY.md already exists on disk this round
|
||||
QB_FILTER_JSON=$(gsd_run quick-batch filter-executed --eligible "$ELIGIBLE_IDS_JSON" --executed "$EXECUTED_IDS_JSON" --raw)
|
||||
```
|
||||
Parse `spawnEligible` (safe to spawn this round) and `alreadyExecuted`
|
||||
(diagnostic only — report these as "already executed, routing to merge"
|
||||
rather than dispatching them). Replace `$ELIGIBLE_IDS_JSON` with
|
||||
`spawnEligible` before continuing to backpressure below.
|
||||
|
||||
NEVER re-dispatch it into a second worktree for any id `filter-executed`
|
||||
returns in `alreadyExecuted`. It is not lost: Step 7's own mergeable-wave
|
||||
criterion
|
||||
(`gsd-core/workflows/quick-batch/steps/merge-wave.md` Step 1) already
|
||||
picks up any `pending` item with an on-disk `SUMMARY.md` that isn't yet
|
||||
merged, independent of this eligible/spawn list — dropping it here only
|
||||
prevents the duplicate dispatch, it does not remove it from the batch.
|
||||
|
||||
2. **Backpressure.** Not every eligible item necessarily spawns this round —
|
||||
cap fan-out at `$EXEC_CONCURRENCY` minus current in-flight count (row
|
||||
27/39):
|
||||
@@ -113,6 +143,21 @@ EXEC_CONCURRENCY=$(printf '%s' "$QB_EXEC_CONC_JSON" | node -e 'let s="";process.
|
||||
primary checkout (same prompt, minus the worktree-only framing), one item
|
||||
at a time — `EXEC_CONCURRENCY` is already forced to 1 in this mode.
|
||||
|
||||
**Durable worktree-recovery persistence (#3677 — `harness-worktree`/
|
||||
`orchestrator-worktree` only, skip for `none`):** immediately after
|
||||
recording `{agent_id, worktree_path, branch, expected_base}` into the
|
||||
EPHEMERAL `$QUICK_BATCH_WORKTREE_MANIFEST` above, ALSO persist the same
|
||||
triple durably onto this item in `BATCH.json`:
|
||||
```bash
|
||||
gsd_run quick-batch update --batch "$BATCH_ID" --updates '[{"quickId":"'"$quick_id"'","dispatchedWorktree":"'"$worktree_path"'","dispatchedBranch":"'"$branch"'","dispatchedBase":"'"$expected_base"'"}]'
|
||||
```
|
||||
The ephemeral manifest is a per-process `mktemp` file (same shape
|
||||
`/gsd:quick`'s own `QUICK_WORKTREE_MANIFEST` uses) — it does NOT survive
|
||||
a coordinator crash/restart. A RESUMED coordinator's Step 7
|
||||
(`merge-wave.md`) reads this durable BATCH.json triple as its fallback
|
||||
for any item the crash-window guard above correctly did NOT re-dispatch
|
||||
in the current process.
|
||||
|
||||
> **ORCHESTRATOR RULE — CODEX RUNTIME**: after each `Agent()` call above, wait for it to return before starting the next worktree create.
|
||||
|
||||
4. **After every item dispatched this round returns:** verify
|
||||
|
||||
Reference in New Issue
Block a user