fix(#222): orchestrator self-heal when synthesizer returns SUMMARY.md inline (#1042)

* fix(#222): orchestrator self-heal when synthesizer returns SUMMARY.md inline

The gsd-research-synthesizer agent intermittently hits an LLM false-refusal:
instead of writing .planning/research/SUMMARY.md with the Write tool, it returns
the SUMMARY.md content inline and fabricates a non-existent write restriction
(e.g. "the runtime is blocking file writes"). The shipped prompt hardening
(#240) is necessary but insufficient — the false-refusal recurs under some
context loads, and a drifting subagent then leaves gsd-roadmapper to fail with
"SUMMARY.md not found".

Adds an orchestrator-level self-heal to new-project.md and new-milestone.md:
after the synthesizer returns, verify .planning/research/SUMMARY.md exists; if it
is missing but the agent returned content inline, the orchestrator persists that
content with the Write tool (logging a warning) before spawning gsd-roadmapper;
if missing with no content, surface the error and stop rather than proceed
against a missing SUMMARY.md. This absorbs the failure mode deterministically
instead of depending on the subagent never drifting.

Closes #222

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(#222): backfill changeset PR number to 1042

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

.changeset/222-synthesizer-summary-self-heal.md

@@ -0,0 +1,5 @@
+---
+type: Fixed
+pr: 1042
+---
+**`/gsd-new-project` and `/gsd-new-milestone` now self-heal when the research synthesizer returns `SUMMARY.md` inline instead of writing it** — under some context loads the `gsd-research-synthesizer` agent hits an LLM false-refusal (fabricating a non-existent write restriction) and returns the SUMMARY.md content in its reply rather than writing `.planning/research/SUMMARY.md`. Prompt hardening (#240) reduced but did not eliminate this. Both workflows now verify the file exists after the synthesizer returns and, if it is missing but content came back inline, the orchestrator persists it before spawning `gsd-roadmapper` — so the roadmapper never fails with "SUMMARY.md not found". (#222)

gsd-core/workflows/new-milestone.md

@@ -362,6 +362,14 @@ Commit after writing.
 
 > **ORCHESTRATOR RULE — CODEX RUNTIME**: After calling Agent() above, stop working on this task immediately. Do not read more files, edit code, or run tests related to this task while the subagent is active. Wait for the subagent to return its result. This prevents duplicate work, conflicting edits, and wasted context. Only resume when the subagent result is available.
 
+**Synthesizer output self-heal (#222) — verify SUMMARY.md materialized:** The synthesizer's canonical output is `.planning/research/SUMMARY.md` on disk; its brief structured return (`## SYNTHESIS COMPLETE` plus a few `###` confirmation lines) is NOT the file content. A known LLM false-refusal (issue #222) sometimes makes the agent return the full SUMMARY.md document inline — fabricating a write restriction (e.g. "the runtime is blocking file writes") — instead of writing the file. Prompt hardening alone does not fully eliminate it, so the orchestrator MUST absorb the failure deterministically before spawning `gsd-roadmapper`:
+
+1. Verify `.planning/research/SUMMARY.md` exists AND is substantive — non-empty, and free of any leftover `<!-- gsd:write-continue -->` continuation sentinel (which marks a truncated/incomplete write). You may validate with `gsd-tools verify-summary .planning/research/SUMMARY.md` — it exits 0 regardless, so check its JSON `passed` field (`"passed": false` means missing or invalid), not the process exit code. If it passes, continue normally.
+2. If it is MISSING or invalid AND the synthesizer's return message contains the FULL SUMMARY.md document — recognizable by the template's top-level markers `# Project Research Summary`, `## Key Findings`, `## Implications for Roadmap`, and `## Sources`, not merely the brief `## SYNTHESIS COMPLETE` confirmation — the false-refusal fired: write that returned document to `.planning/research/SUMMARY.md` with the Write tool, then commit ALL research artifacts the synthesizer owns (it commits on behalf of the four researchers) with `gsd-tools query commit "docs: complete project research" --files .planning/research/` unless they are already committed. Log `⚠ #222 self-heal: synthesizer returned SUMMARY.md inline without writing it; orchestrator persisted the file.`
+3. If it is MISSING or invalid AND the return is only a brief confirmation (no full SUMMARY document to recover), the synthesizer genuinely failed — surface the error and stop; do NOT spawn `gsd-roadmapper` against a missing or incomplete SUMMARY.md.
+
+This guarantees `gsd-roadmapper` (which lists SUMMARY.md as required reading) never runs against a missing or truncated SUMMARY.md.
+
 Display key findings from SUMMARY.md:
 ```
 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

gsd-core/workflows/new-project.md

@@ -1069,6 +1069,14 @@ Commit after writing.
 
 > **ORCHESTRATOR RULE — CODEX RUNTIME**: After calling Agent() above, stop working on this task immediately. Do not read more files, edit code, or run tests related to this task while the subagent is active. Wait for the subagent to return its result. This prevents duplicate work, conflicting edits, and wasted context. Only resume when the subagent result is available.
 
+**Synthesizer output self-heal (#222) — verify SUMMARY.md materialized:** The synthesizer's canonical output is `.planning/research/SUMMARY.md` on disk; its brief structured return (`## SYNTHESIS COMPLETE` plus a few `###` confirmation lines) is NOT the file content. A known LLM false-refusal (issue #222) sometimes makes the agent return the full SUMMARY.md document inline — fabricating a write restriction (e.g. "the runtime is blocking file writes") — instead of writing the file. Prompt hardening alone does not fully eliminate it, so the orchestrator MUST absorb the failure deterministically before spawning `gsd-roadmapper`:
+
+1. Verify `.planning/research/SUMMARY.md` exists AND is substantive — non-empty, and free of any leftover `<!-- gsd:write-continue -->` continuation sentinel (which marks a truncated/incomplete write). You may validate with `gsd-tools verify-summary .planning/research/SUMMARY.md` — it exits 0 regardless, so check its JSON `passed` field (`"passed": false` means missing or invalid), not the process exit code. If it passes, continue normally.
+2. If it is MISSING or invalid AND the synthesizer's return message contains the FULL SUMMARY.md document — recognizable by the template's top-level markers `# Project Research Summary`, `## Key Findings`, `## Implications for Roadmap`, and `## Sources`, not merely the brief `## SYNTHESIS COMPLETE` confirmation — the false-refusal fired: write that returned document to `.planning/research/SUMMARY.md` with the Write tool, then commit ALL research artifacts the synthesizer owns (it commits on behalf of the four researchers) with `gsd-tools query commit "docs: complete project research" --files .planning/research/` unless they are already committed. Log `⚠ #222 self-heal: synthesizer returned SUMMARY.md inline without writing it; orchestrator persisted the file.`
+3. If it is MISSING or invalid AND the return is only a brief confirmation (no full SUMMARY document to recover), the synthesizer genuinely failed — surface the error and stop; do NOT spawn `gsd-roadmapper` against a missing or incomplete SUMMARY.md.
+
+This guarantees `gsd-roadmapper` (which lists SUMMARY.md as required reading) never runs against a missing or truncated SUMMARY.md.
+
 Display research complete banner and key findings:
 
 ```

tests/bug-222-research-synthesizer-write-contract.test.cjs

@@ -54,3 +54,62 @@ describe('bug #222: research synthesizer must write SUMMARY.md via Write tool',
     );
   });
 });
+
+describe('bug #222 recurrence: orchestrator self-heals when synthesizer returns SUMMARY.md inline', () => {
+  const WORKFLOWS = [
+    path.join(REPO_ROOT, 'gsd-core', 'workflows', 'new-project.md'),
+    path.join(REPO_ROOT, 'gsd-core', 'workflows', 'new-milestone.md'),
+  ];
+
+  for (const wf of WORKFLOWS) {
+    const name = path.basename(wf);
+
+    test(`${name} has the #222 synthesizer SUMMARY.md self-heal guard`, () => {
+      const text = fs.readFileSync(wf, 'utf8');
+
+      // Marker tying the guard to the issue
+      assert.match(text, /#222[^\n]*self-heal|self-heal[^\n]*#222/i,
+        `${name} must contain a #222-tagged self-heal guard after the synthesizer returns.`);
+
+      // Verifies the file exists AND is substantive/non-empty
+      assert.match(text, /SUMMARY\.md[\s\S]{0,120}?(non-empty|substantive|exists)/i,
+        `${name} must verify .planning/research/SUMMARY.md exists AND is substantive — non-empty.`);
+
+      // Truncation/validator guard: references the continuation sentinel OR the verify-summary CLI
+      assert.match(text, /gsd:write-continue|verify-summary/i,
+        `${name} must guard against truncated/invalid SUMMARY.md (sentinel or verify-summary).`);
+
+      // Self-heal must commit ALL research artifacts, not just SUMMARY.md
+      assert.match(text, /--files \.planning\/research\//,
+        `${name} self-heal must commit ALL research artifacts, not just SUMMARY.md.`);
+
+      // Persists inline-returned document via Write rather than trusting the agent
+      assert.match(text, /returned[\s\S]{0,200}?document[\s\S]{0,200}?Write tool/i,
+        `${name} must instruct the orchestrator to persist inline-returned document with the Write tool.`);
+
+      // Must not proceed to roadmapper against a missing or incomplete SUMMARY.md
+      assert.match(text, /gsd-roadmapper[\s\S]{0,200}?(missing|incomplete|do NOT)/i,
+        `${name} must block spawning gsd-roadmapper when SUMMARY.md is missing or incomplete.`);
+
+      // Must name the FULL SUMMARY template markers so the orchestrator persists the real
+      // document, not the brief structured return (resolves the HIGH finding).
+      assert.match(text, /# Project Research Summary[\s\S]{0,260}?## Sources/,
+        `${name}: self-heal must name the full SUMMARY.md template markers (# Project Research Summary … ## Sources).`);
+      // Must reference the brief confirmation marker it must NOT mistake for file content.
+      assert.match(text, /## SYNTHESIS COMPLETE/,
+        `${name}: self-heal must distinguish the brief ## SYNTHESIS COMPLETE confirmation from the real document.`);
+    });
+
+    test(`${name} runs the #222 self-heal AFTER the synthesizer and BEFORE gsd-roadmapper`, () => {
+      const text = fs.readFileSync(wf, 'utf8');
+      const synthIdx = text.indexOf('subagent_type="gsd-research-synthesizer"');
+      const healIdx = text.indexOf('Synthesizer output self-heal (#222)');
+      const roadIdx = text.indexOf('subagent_type="gsd-roadmapper"');
+      assert.ok(synthIdx >= 0, `${name}: synthesizer dispatch not found`);
+      assert.ok(healIdx >= 0, `${name}: #222 self-heal block not found`);
+      assert.ok(roadIdx >= 0, `${name}: gsd-roadmapper dispatch not found`);
+      assert.ok(healIdx > synthIdx, `${name}: self-heal must come AFTER the synthesizer dispatch`);
+      assert.ok(roadIdx > healIdx, `${name}: self-heal must come BEFORE the gsd-roadmapper dispatch`);
+    });
+  }
+});