Merge pull request #420 from davesienkowski/fix/executor-completion-verification

fix(executor): add completion verification to prevent hallucinated success (#315)
This commit is contained in:
TÂCHES
2026-02-05 08:04:45 -06:00
committed by GitHub
4 changed files with 61 additions and 1 deletions

View File

@@ -678,6 +678,45 @@ During execution, these authentication requirements were handled:
</summary_creation>
<self_check>
After writing SUMMARY.md, verify your own claims before proceeding.
**1. Check created files exist:**
Parse `key-files.created` from the SUMMARY frontmatter. For each file listed:
```bash
[ -f "path/to/file" ] && echo "FOUND: path/to/file" || echo "MISSING: path/to/file"
```
If `key-files.created` is empty (docs-only plans), skip this check.
**2. Check commits exist:**
Parse commit hashes from the "Task Commits" section. Verify each hash exists:
```bash
git log --oneline --all | grep -q "{hash}" && echo "FOUND: {hash}" || echo "MISSING: {hash}"
```
**3. Append self-check result to SUMMARY.md:**
If ANY file or commit is missing, append to SUMMARY.md:
```markdown
## Self-Check: FAILED
Missing files:
- path/to/missing-file.ts
Missing commits:
- abc123f
```
If all checks pass, append:
```markdown
## Self-Check: PASSED
```
Do NOT skip this step. Do NOT proceed to state updates if self-check fails — the SUMMARY must reflect reality.
</self_check>
<state_updates>
After creating SUMMARY.md, update STATE.md.

View File

@@ -76,7 +76,7 @@ Phase: $ARGUMENTS
For each wave in order:
- Spawn `gsd-executor` for each plan in wave (parallel Task calls)
- Wait for completion (Task blocks)
- Verify SUMMARYs created
- Verify SUMMARYs created and spot-check claims
- Proceed to next wave
5. **Aggregate results**

View File

@@ -337,6 +337,20 @@ Execute each wave in sequence. Autonomous plans within a wave run in parallel **
- Read SUMMARY.md to extract what was built
- Note any issues or deviations
**Spot-check claims before trusting SUMMARY:**
For each completed plan's SUMMARY.md:
- Pick the first 2 files from `key-files.created` frontmatter — verify they exist on disk with `[ -f ]`
- Check `git log --oneline --all --grep="{phase}-{plan}"` returns at least 1 commit
- Check SUMMARY.md for `## Self-Check: FAILED` marker
If ANY spot-check fails:
- Do NOT proceed silently
- Report which plan failed verification and what was missing
- Route to failure handler (step 4): ask user "Retry plan?" or "Continue with remaining waves?"
If spot-checks pass: proceed normally.
**Output:**
```
---

View File

@@ -460,6 +460,13 @@ For Pattern A (fully autonomous) and Pattern C (decision-dependent), skip this s
D. Report completion
E. Self-check the aggregated SUMMARY:
- Parse `key-files.created` from the SUMMARY frontmatter
- Verify first 2 files exist on disk with `[ -f ]`
- Check `git log --oneline --all --grep="{phase}-{plan}"` returns at least 1 commit
- If any check fails: append `## Self-Check: FAILED` section to SUMMARY listing missing items
- If all pass: append `## Self-Check: PASSED`
**Example execution trace:**
````