515191f07d
* chore(#3677): checkpoint design artifacts (gitignored, dev-only) * test(#3677): add failing regression test for the crash-window duplicate-dispatch gap (RED) Independently re-traces resume-mode.md/planner-wave.md/worktree-dispatch.md/ merge-wave.md and src/quick-batch.cts's resumeBatch (lines 894-899) and confirms the prior research pass's Open Question 1: a coordinator crash between Step 6 (executor commits, SUMMARY.md written) and Step 7 (merge) leaves BATCH.json at "pending" with no STATE.md row yet (only written in Step 9), so --resume's eligibility re-derivation would dispatch a second executor into a new worktree for the same item, orphaning the first. This test asserts worktree-dispatch.md's Step 6 excludes an item whose SUMMARY.md already exists from the spawn set, mirroring planner-wave.md's existing PLAN.md-existence check one layer earlier. Fails against the current worktree-dispatch.md, which has no such guard. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §1 for the full trace and fix-location rationale. * fix(#3677): guard worktree-dispatch.md against re-dispatching an already-executed item (GREEN) worktree-dispatch.md's Step 6 re-derives eligibility every dispatch round via the same quick-batch resume call resume-mode.md uses, but had no check for "did this item already finish executing" the way planner-wave.md already checks "did this item already get planned" (PLAN.md existence) before re-planning. A coordinator crash between Step 6 (executor commits, SUMMARY.md written) and Step 7 (merge) left the item eligible for a second dispatch on --resume, orphaning the first worktree's real, already- committed work and silently losing it once the second executor's SUMMARY.md write clobbered the first at the same item_dir path. Adds a SUMMARY.md-existence exclusion before spawn-plan is computed, symmetric to planner-wave.md's PLAN.md check. The excluded item is not lost: merge-wave.md's own mergeable-wave criterion (status=pending, SUMMARY.md on disk, not yet merged) already picks it up independently of this eligible/spawn list. Workflow-prose-only fix — touches no already-merged/reviewed .cts module. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §1 for the fix-location rationale (why not resumeBatch itself). * test(#3677): add real-git coverage for worktree-ownership tampering, scope drift, and submodules Closes the three coverage gaps identified in 40-design.md §2/§3 (#3677, epic #3344 Phase 5's own AC bullets: "arbitrary-worktree ownership attempts", "scope drift", "submodules"): - Arbitrary-worktree ownership tampering: a manifest entry naming a non-agent branch is silently dropped at normalization before any git subprocess runs; a manifest entry naming a plausible agent-branch that was never actually created by this repo's own worktree.create (a genuinely foreign repo/branch) is blocked via base_mismatch. Both leave the foreign location and repoRoot's HEAD provably untouched. - Advisory scope drift: a committed path outside declared files_modified still merges successfully (advisory, never blocking) while surfacing a scope_out_of_declared warning naming the drifted path; an exact declared-scope match produces zero warnings (boundary case). - Real .gitmodules submodule integration: a repo containing a real local git submodule merges cleanly through executeWorktreeWaveCleanupPlan for an unrelated plan; a real gitlink pointer bump (declared) merges cleanly with the superproject tree reflecting the new pinned commit; an undeclared bump is advisory-only and surfaces a scope warning naming vendor/sub, same as any other undeclared modification. No src/*.cts changes — all three gaps were coverage-only; the underlying primitives already behaved correctly (independently verified against real git subprocess output before writing each assertion). * docs(#3677): document how to diagnose a preserved quick-batch worktree Extends the one-sentence "worktree is preserved (never deleted)" mention into a concrete diagnosis procedure: where the preserved directory is, how to read the executor's real commits/diff against the plan's declared files_modified, how to read the item's own SUMMARY.md independent of merge outcome, how to manually merge-and-clean-up or discard, and how to re-run --resume afterward. Also documents that a SUMMARY.md-written-but-still- pending item (the crash-window case fixed in this same PR) needs no manual intervention — --resume routes it straight to the merge step. * chore(#3677): checkpoint final acceptance-evidence mapping (gitignored, dev-only) * fix(#3677): make crash-window duplicate-dispatch guard behaviorally provable and durably recoverable Orthogonal review (Spec finding): the crash-window regression test added earlier this phase only asserted readStep('worktree-dispatch.md') + regex matches against the markdown prose — proving the DOCUMENTATION says the right thing, never that the runtime condition (pending status + on-disk SUMMARY.md + absent STATE row) is actually handled correctly. #3677's own "Alternatives considered" explicitly rejects "document recovery without fault injection" for exactly this reason. Extracts the filtering decision into a pure, independently testable function, filterAlreadyExecuted(eligibleIds, executedIds) in src/quick-batch-dispatch.cts, wired to a new `quick-batch filter-executed` CLI verb (src/quick-batch-command-router.cts) — the same pure-decision- then-CLI-wired pattern computeSpawnPlan/computeMergeOrder already establish. worktree-dispatch.md now calls this verb explicitly instead of only describing the decision in prose. A genuine fixture-based test in tests/quick-batch.test.cjs constructs a REAL BATCH.json (createBatch), writes a REAL SUMMARY.md on disk at the item's real item_dir, calls the REAL resumeBatch, and proves both that resumeBatch alone still reports the item eligible AND that filterAlreadyExecuted (fed a real filesystem check) correctly excludes it. The prior prose-assertion tests are kept — they now prove the workflow markdown is correctly WIRED to the verb — but are no longer the only proof. Self-discovered defect while building that fixture (fixed inline, not deferred): tracing merge-wave.md against /gsd:quick's own prior art (QUICK_WORKTREE_MANIFEST=$(mktemp ...), quick.md:415) showed $QUICK_BATCH_WORKTREE_MANIFEST is a fresh PER-PROCESS temp file. A resumed coordinator correctly does not re-dispatch an already-executed item (this fix), but nothing durably recorded that item's worktree_path/branch/base either — Step 7 in the resumed process would have had no data to build its cleanup-wave entry from. Adds dispatched_worktree/dispatched_branch/ dispatched_base to QuickBatchItem (src/quick-batch.cts) — deliberately NOT a reuse of the pre-existing `worktree` field, whose loadBatch validation requires the path to exist on disk (verified empirically: reusing it made the batch permanently unloadable the moment a legitimately-merged worktree was removed). worktree-dispatch.md persists the triple once a worktree is created; merge-wave.md falls back to it when the ephemeral manifest lacks an entry, clears it after a successful merge, and fails closed rather than guessing if no record exists anywhere. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §9.1 and §9.3 for the full trace, empirical verification notes, and rejected alternatives (reusing `worktree` directly). * test(#3677): prove the arbitrary-worktree-ownership boundary against two real sibling worktrees Orthogonal review (Security finding): the two existing ownership-tampering tests didn't test ownership — one was trivially rejected by WORKTREE_AGENT_BRANCH_RE's shape check before any git call (proves branch- NAME filtering, not ownership), the other pointed at a wholly separate, never-linked foreign repo, so merge-base failed immediately because the branch didn't exist as a ref at all. Neither exercised the real scenario: a manifest entry whose worktree_path/branch are swapped to point at a DIFFERENT, GENUINELY-REGISTERED sibling worktree of the SAME repoRoot, with a branch name passing the shape check and a base in allowed_bases. Investigated executeWorktreeWaveCleanupPlan (src/worktree-safety.cts) directly: this is NOT a reachable gap. Git enforces branch-per-worktree uniqueness, so a swapped-in entry.branch can only match worktree_path's ACTUAL checked-out branch if it names that sibling's own real, uniquely- generated branch name — which manifest tampering confined to one batch's own record has no way to know (branch names are agent-<quick_id>[-<timestamp>]-shaped, and quick_id allocation is collision-checked GLOBALLY across every existing quick task and batch, not merely within one batch). Adds a stronger test that empirically proves this: two REAL, concurrently- alive sibling worktrees of the same repo (both via real `git worktree add`, both WORKTREE_AGENT_BRANCH_RE-passing, both sharing one merge-base), with worktree_path/branch swapped between them in both directions. Both attempts are blocked via branch_mismatch; both real worktrees, their branches, and one sibling's real uncommitted-to-main commit survive completely untouched. Supplements (does not replace) the original two tests, which still prove distinct, real boundaries. See .gsd/phase/feat-3677-quick-batch-hardening-acceptance/40-design.md §9.2 for the full trace, including the one explicitly-documented (not fixed) trust boundary this investigation surfaced: the primitive defends against fabricated data, not a caller bug that misattributes a real-but-wrong item's own triple to a different item. * chore(#3677): checkpoint design-doc addendum for review pass 2 findings (gitignored, dev-only) * docs(#3677): add changeset for PR 4240 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
170 lines
8.2 KiB
Markdown
170 lines
8.2 KiB
Markdown
# How to batch quick tasks
|
|
|
|
`/gsd-quick-batch` runs several `/gsd-quick`-shaped tasks together as ONE
|
|
coordinated run: one coordinator parses the task list, plans and dispatches
|
|
each item (planner, and optionally researcher/plan-checker/verifier leaves),
|
|
merges them in a deterministic order, and owns every shared write
|
|
(`BATCH.json`, `STATE.md`, worktree create/merge/cleanup) so the leaves never
|
|
race each other (ADR-1239 "Quick-batch binding").
|
|
|
|
Use it instead of running `/gsd-quick` N separate times when you have several
|
|
independent (or lightly interdependent) small tasks you want planned and
|
|
executed together, with parallelism where the tasks allow it.
|
|
|
|
For the single-task case, see [Handle quick and fast tasks](handle-quick-and-fast-tasks.md).
|
|
|
|
---
|
|
|
|
## Basic use
|
|
|
|
Pass an inline task list — a bulleted or numbered list, at least 2 items,
|
|
one per line:
|
|
|
|
```bash
|
|
/gsd-quick-batch
|
|
- Fix the login timeout on mobile Safari
|
|
- Add a retry banner when the API call fails
|
|
- Update the README's setup instructions
|
|
```
|
|
|
|
Or point at a file containing the list:
|
|
|
|
```bash
|
|
/gsd-quick-batch --file .planning/my-tasks.md
|
|
```
|
|
|
|
Each item gets its own quick id, its own directory under
|
|
`.planning/quick/`, and (when isolation is available) its own worktree — the
|
|
same artifact shape a standalone `/gsd-quick` task produces, just planned and
|
|
dispatched together.
|
|
|
|
---
|
|
|
|
## Flags
|
|
|
|
| Flag | What it does |
|
|
|------|-------------|
|
|
| `--jobs auto\|N` | `auto` (default) uses the negotiated dispatch capacity as-is. `N` caps effective concurrency at `min(task count, N, capacity)` — never more than the number of tasks, never more than what the runtime negotiated. A non-numeric or non-positive `N` is rejected before any dispatch. |
|
|
| `--validate` | Enables the per-item plan-checker loop (max 2 iterations, same cap as `/gsd-quick --validate`) and post-merge verification. |
|
|
| `--research` | Dispatches a focused researcher per item before planning. |
|
|
| `--resume <batch-id>` | Skips task-list parsing and batch creation entirely — loads the existing batch and dispatches only its still-eligible items. |
|
|
|
|
**Not supported in v1:** `--discuss` and `--full` are rejected with a usage
|
|
error before any dispatch. If a task genuinely needs a discussion phase, run
|
|
it through `/gsd-quick --discuss` on its own instead of including it in a
|
|
batch.
|
|
|
|
```bash
|
|
/gsd-quick-batch --jobs 2 --validate --research # research, up to 2 concurrent, plan-checked + verified
|
|
/gsd-quick-batch --resume 260101-abc # resume an interrupted batch
|
|
```
|
|
|
|
---
|
|
|
|
## How capacity and isolation interact
|
|
|
|
Effective concurrency is computed from three things: `--jobs`, the
|
|
negotiated dispatch capacity (how many subagents your runtime can run at
|
|
once), and — for the mutating stage (worktree create → execute → merge) —
|
|
the isolation mode:
|
|
|
|
| Isolation | Effect on the mutating (executor/worktree) stage |
|
|
|---|---|
|
|
| `harness-worktree` / `orchestrator-worktree` | Runs up to the effective concurrency computed above. |
|
|
| `none` (no worktree isolation available, or `workflow.use_worktrees=false`) | Forced to concurrency **1**, regardless of `--jobs` or capacity — everything executes sequentially on the primary checkout. |
|
|
|
|
This cap applies **only** to the mutating stage. Planning and research are
|
|
never worktree-isolated, so they run at full effective concurrency even when
|
|
isolation is `none`.
|
|
|
|
Worktree creation, merging, and cleanup are always serialized one at a time
|
|
(`git worktree add`/`git merge`/`git worktree remove` never overlap) —
|
|
concurrency is about how many already-created worktrees' agents run at once,
|
|
not about the git operations themselves. Merges apply in the same
|
|
deterministic order the batch's dependency/file-overlap waves were computed
|
|
in, never in whichever order an executor happens to finish first.
|
|
|
|
---
|
|
|
|
## Dependencies and file overlap
|
|
|
|
Before planning, `/gsd-quick-batch` has no signal about which items depend
|
|
on each other or touch the same files — every item starts in the same wave.
|
|
Each item's planner is shown the full batch's task catalog (every item's id
|
|
and description) and is required to declare, in its plan's frontmatter,
|
|
which sibling items (if any) it depends on and which files it will touch.
|
|
After each planning round, the coordinator recomputes execution waves from
|
|
those declarations — independent items with disjoint files run in parallel;
|
|
a dependent item's wave always comes strictly after its dependency's.
|
|
|
|
---
|
|
|
|
## Resuming and failure recovery
|
|
|
|
A batch's `BATCH.json` (`.planning/quick-batches/<batch-id>/`) tracks every
|
|
item's status. Re-run with `--resume <batch-id>` at any point — including
|
|
after a crash — and the coordinator re-derives which items are still
|
|
runnable:
|
|
|
|
| Outcome | What happens | Recoverable via `--resume`? |
|
|
|---|---|---|
|
|
| Item completes normally | Marked `complete`; a `Quick Tasks Completed` STATE.md row is appended. | N/A |
|
|
| Verifier reports `human_needed` (`--validate` only) | Terminal for that item — no STATE row is appended. Review it yourself, then fix and re-run if needed. | Yes, once resolved |
|
|
| Verifier reports `gaps_found` (`--validate` only) | The item is marked `failed`. Its already-merged commit is **not** rolled back, and there is no automatic gap-fix retry. | Yes — resume re-evaluates it |
|
|
| A merge conflicts, or a committed diff includes an undeclared file deletion | The item is marked `failed` with a reason; its worktree is **preserved** (never deleted) so you can inspect what happened. | Yes, after you resolve the worktree by hand |
|
|
| An item this item depends on failed | The dependent item is automatically marked `blocked` on the next `--resume`. | Yes, once the blocking item is resolved |
|
|
|
|
Items unrelated to a failure continue normally in the same or a later batch
|
|
run — one item's problem never blocks the rest of the batch.
|
|
|
|
### Diagnosing a preserved worktree
|
|
|
|
When a merge conflicts or a committed diff includes an undeclared file
|
|
deletion, the coordinator preserves that item's worktree instead of deleting
|
|
it, so you can inspect exactly what the executor did:
|
|
|
|
1. **Find it.** The preserved directory is
|
|
`<repo-root>/../<quick_id>-<slug>-wt/` (or wherever your runtime's
|
|
worktree layout places it) — the item's own quick directory,
|
|
`.planning/quick/<quick_id>-<slug>/`, still has the `PLAN.md` the
|
|
executor was given, which tells you what it was trying to do.
|
|
2. **See what actually changed.** From the preserved worktree:
|
|
`git log <base>..HEAD` shows the executor's real commit(s);
|
|
`git diff <base>...HEAD --stat` shows exactly which files it touched
|
|
(compare that against `PLAN.md`'s declared `files_modified` if you want
|
|
to confirm whether the failure was a genuine conflict or an
|
|
out-of-scope change).
|
|
3. **Read the item's own `SUMMARY.md`** in its quick directory — the
|
|
executor wrote it before the merge was attempted, so it still describes
|
|
what the executor believed it accomplished, independent of whether the
|
|
merge itself succeeded.
|
|
4. **Decide how to resolve it:**
|
|
- If the work is good and only the automated merge failed (a real
|
|
conflict, or a deletion that should have been declared): merge the
|
|
worktree's branch by hand (`git merge <branch> --no-ff`), resolve any
|
|
conflicts, then remove the worktree yourself
|
|
(`git worktree remove <path> --force`) and its branch
|
|
(`git branch -D <branch>`).
|
|
- If the work should be discarded: remove the worktree and branch the
|
|
same way, without merging.
|
|
- If a coordinator crash left the item's `SUMMARY.md` written but its
|
|
`BATCH.json` status still `pending` (the executor finished before the
|
|
coordinator process crashed, before the merge step ran) — this is NOT
|
|
a preserved-worktree failure and needs no manual merge. `--resume`
|
|
recognizes the on-disk `SUMMARY.md` and routes the item straight to
|
|
the merge step on its own; it will not re-dispatch a second executor
|
|
for it.
|
|
5. **Re-run** `/gsd-quick-batch --resume <batch-id>` once you're done. A
|
|
`failed`/`merge_failed`/`scope_violation` item you resolved manually
|
|
(merged and cleaned up yourself) is picked up as already-merged on the
|
|
next resume; an item you decided to abandon stays `failed` and is
|
|
skipped.
|
|
|
|
---
|
|
|
|
## Related
|
|
|
|
- [Handle quick and fast tasks](handle-quick-and-fast-tasks.md)
|
|
- [Commands](../COMMANDS.md)
|
|
- [Docs index](../README.md)
|