feat(3309): workflow.human_verify_mode = end-of-phase (new default; mid-flight opt-back-in) (#3325)

* test(3309): red — workflow.human_verify_mode contract

New behavioral test file covers:
- workflow.human_verify_mode is a recognized config key (VALID_CONFIG_KEYS)
- defaults to 'mid-flight' (preserves current behavior)
- config-set / config-get round-trips for both values
- persists in config.json as string
- planner agent file references the flag with canonical wording, couples
  end-of-phase mode with the rule that checkpoint:human-verify is not
  emitted, and documents the <verify><human-check> deferred-item shape
- verifier agent file references harvesting <verify><human-check> blocks
- references/checkpoints.md documents the cost-control alternative

Source-text assertions on agent .md files are exempted via
allow-test-rule: source-text-is-the-product — those files ARE the
runtime contract loaded by AI runtimes, so asserting their wording is
the only way to verify the agents will respect the flag.

Fails 10/11 against current source. Will pass after the fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(3309): add workflow.human_verify_mode = end-of-phase opt-out

Each mid-flight checkpoint:human-verify halt costs a full executor
cold-start (CLAUDE.md, MEMORY.md, STATE.md, plan re-read on every
respawn) because subagent context is discarded across the pause. A plan
with N human-verify checkpoints pays the cold-start cost N+1 times. The
reporter (rentanything-nb) measured this at "tens of thousands of tokens"
per round-trip and "hundreds of thousands per week."

This adds workflow.human_verify_mode (default 'mid-flight') with an
'end-of-phase' value that:
- instructs gsd-planner to NOT emit <task type="checkpoint:human-verify">
  tasks; verification details go into a <verify><human-check> sub-block
  on the relevant auto task instead
- instructs gsd-verifier (Step 8) to harvest those <verify><human-check>
  blocks at end-of-phase and merge them into its own human-verification
  list
- the existing human_needed → HUMAN-UAT.md flow in execute-phase.md is
  the single sink — no new file/writer is created

checkpoint:decision and checkpoint:human-action are unaffected — those
gate the work itself, not post-hoc verification.

Surfaces touched:
- bin/lib/config-schema.cjs, bin/lib/config.cjs — register key + default
- sdk/src/config.ts, sdk/src/query/config-schema.ts — SDK parity
- agents/gsd-planner.md — slim Detection section + reference link
- agents/gsd-verifier.md — Step 8 harvest instruction
- get-shit-done/references/planner-human-verify-mode.md — full rules,
  loaded conditionally to keep planner.md under its size budget
- get-shit-done/references/checkpoints.md — surface the alternative
- docs/CONFIGURATION.md — config table row
- docs/INVENTORY.md, docs/INVENTORY-MANIFEST.json — track new reference

Tag name <human-check> chosen instead of <human> to avoid the
prompt-injection scan pattern that flags <system|assistant|human> tags.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(3309): align changeset pr: to actual PR number

The pr: field was authored as 3319 (a guess at the next number) before
the PR was opened. Actual PR is #3325.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(3309): flip workflow.human_verify_mode default to end-of-phase

Per maintainer direction on PR #3325, end-of-phase is the new project
default. Mid-flight checkpoint:human-verify halts cost a full executor
cold-start (CLAUDE.md, MEMORY.md, STATE.md, plan re-read on respawn) per
round-trip — reported at "tens of thousands of tokens" per round-trip,
"hundreds of thousands per week" on real projects. The cost-control
mode is what new projects should get out of the box.

mid-flight remains a one-line opt-back-in via:

    gsd config-set workflow.human_verify_mode mid-flight

Behavior change for existing projects: the new default takes effect
when .planning/config.json is rewritten (config-set, fresh project).
Existing in-flight PLAN.md files with checkpoint:human-verify tasks
continue to work in either mode — the flag only changes what the
planner emits next time it runs.

Surfaces updated:
- bin/lib/config.cjs, sdk/src/config.ts — default flipped
- sdk/src/config.ts docstring — describes new default + opt-back-in
- agents/gsd-planner.md — Detection section explains new default
- references/planner-human-verify-mode.md — reordered modes; added
  guidance on when to opt back into mid-flight
- references/checkpoints.md — surface the default flip and the why
- docs/CONFIGURATION.md — table row reflects new default + reason
- tests/feat-3309-human-verify-mode.test.cjs — default test asserts
  end-of-phase
- .changeset/fierce-geese-march.md — describes the default flip and
  the migration semantics

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: address human verify mode review

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-05-09 23:29:11 -04:00
committed by GitHub
parent e7942c21b3
commit 25fb81d01e
13 changed files with 294 additions and 2 deletions

View File

@@ -20,6 +20,7 @@ const VALID_CONFIG_KEYS = new Set([
'workflow.nyquist_validation', 'workflow.ai_integration_phase', 'workflow.ui_phase', 'workflow.ui_safety_gate',
'workflow.auto_advance', 'workflow.node_repair', 'workflow.node_repair_budget',
'workflow.tdd_mode',
'workflow.human_verify_mode',
'workflow.text_mode',
'workflow.research_before_questions',
'workflow.discuss_mode',

View File

@@ -109,6 +109,7 @@ function buildNewProjectConfig(userChoices) {
ui_safety_gate: true,
ai_integration_phase: true,
tdd_mode: false,
human_verify_mode: 'end-of-phase',
text_mode: false,
research_before_questions: false,
discuss_mode: 'discuss',
@@ -354,6 +355,12 @@ function cmdConfigSet(cwd, keyPath, value, raw) {
}
}
// Human verification checkpoint mode (#3309)
const VALID_HUMAN_VERIFY_MODES = ['mid-flight', 'end-of-phase'];
if (keyPath === 'workflow.human_verify_mode' && !VALID_HUMAN_VERIFY_MODES.includes(String(parsedValue))) {
error(`Invalid workflow.human_verify_mode '${value}'. Valid values: ${VALID_HUMAN_VERIFY_MODES.join(', ')}`);
}
const setConfigValueResult = setConfigValue(cwd, keyPath, parsedValue);
// Mask secrets in both JSON and text output. The plaintext is written

View File

@@ -18,6 +18,12 @@ Plans execute autonomously. Checkpoints formalize interaction points where human
**When:** Claude completed automated work, human confirms it works correctly.
> **Default mode (#3309): `workflow.human_verify_mode = end-of-phase`.** New projects do NOT halt mid-flight at `checkpoint:human-verify`. The planner suppresses those task emissions and embeds the verification details into the relevant `auto` task's `<verify><human-check>` block; the verifier harvests every `<verify><human-check>` at end-of-phase (Step 8) and consolidates them into the existing `human_needed` → HUMAN-UAT.md flow in `workflows/execute-phase.md`. The user reviews everything in one batch.
>
> **Why this is the default:** every mid-flight halt costs a full executor cold-start (CLAUDE.md, MEMORY.md, STATE.md, plan re-read on respawn) because subagent context is discarded across the pause. A plan with N human-verify checkpoints pays the cold-start cost N+1 times — measured at "tens of thousands of tokens" per round-trip on real projects.
>
> Set `workflow.human_verify_mode = mid-flight` in `.planning/config.json` to opt back into the pre-#3309 behavior of halting at every checkpoint. `checkpoint:decision` and `checkpoint:human-action` are unaffected by either value — those gate the work itself, not post-hoc verification.
**Use for:**
- Visual UI checks (layout, styling, responsiveness)
- Interactive flows (click through wizard, test user flows)

View File

@@ -0,0 +1,57 @@
# Planner — Human Verification Mode
> Loaded by `gsd-planner` when deciding whether to emit `<task type="checkpoint:human-verify">` tasks. Read `workflow.human_verify_mode` from `.planning/config.json` (default `end-of-phase` since #3309).
## The two modes
### `end-of-phase` (default — issue #3309)
Do **not** emit any `<task type="checkpoint:human-verify">` tasks. Every mid-flight halt costs a full executor cold-start (CLAUDE.md, MEMORY.md, STATE.md, plan re-read on respawn) because subagent context is discarded across the pause; a plan with N human-verify checkpoints pays the cold-start cost N+1 times — measured at "tens of thousands of tokens" per round-trip on real projects. This is the default for that reason.
Instead, fold each would-be verification step into the relevant `auto` task using a `<verify><human-check>` sub-block:
```xml
<task type="auto">
<name>Wire dashboard route</name>
<files>app/dashboard/page.tsx, app/api/dashboard/route.ts</files>
<action>...</action>
<verify>
<automated>npm test -- --filter=dashboard</automated>
<human-check>
<test>Visit http://localhost:3000/dashboard</test>
<expected>Sidebar left, content right on desktop &gt;1024px; collapses to hamburger at 768px</expected>
<why_human>Visual layout — grep cannot verify breakpoint behavior</why_human>
</human-check>
</verify>
<done>Layout renders correctly across breakpoints</done>
</task>
```
The verifier (Step 8) harvests every `<verify><human-check>` block at end-of-phase and consolidates them into the existing `human_needed` → HUMAN-UAT.md path in `workflows/execute-phase.md`. The user reviews everything in one batch instead of paying a cold-start cost per item.
### `mid-flight` (opt-back-in — pre-#3309 behavior)
Set `gsd config-set workflow.human_verify_mode mid-flight` to restore the canonical mid-flight pattern: emit `<task type="checkpoint:human-verify">` tasks at the points where human confirmation is required, and the executor halts at each one to ask the user.
```xml
<task type="checkpoint:human-verify" gate="blocking">
<what-built>Dev server running at http://localhost:3000</what-built>
<how-to-verify>
1. Visit /dashboard
2. Sidebar collapses at 768px
</how-to-verify>
<resume-signal>"approved" or describe issues</resume-signal>
</task>
```
Choose `mid-flight` when you genuinely need the work to stop before any subsequent task runs (e.g., the next task depends on visual confirmation of the previous one), and you accept the cold-start cost as the price of that hard barrier.
## What is *not* affected
`checkpoint:decision` and `checkpoint:human-action` tasks are still emitted in `end-of-phase` mode. Those gate the work itself (a choice the executor needs from the user, or an auth step only the user can perform), not post-hoc verification of completed work. Only `checkpoint:human-verify` is suppressed.
## Compatibility with other modes
- **`workflow.tdd_mode`**: orthogonal. TDD tasks still emit `tdd="true"` and `<behavior>`; the `<verify>` block carries the human-check sub-element when `human_verify_mode = end-of-phase`.
- **`MVP_MODE`**: orthogonal. Vertical-slice ordering is unchanged. The first task remains a failing end-to-end test; later auto tasks may carry `<verify><human-check>` instead of standalone checkpoint tasks.
- **`workflow.auto_advance` / `_auto_chain_active`**: in mid-flight mode these auto-approve checkpoint:human-verify halts. In end-of-phase mode there are no halts to auto-approve, so the flags have no effect on this code path.