fix(verification): keep passed uat results across verifier re-runs

A phase whose human UAT passed could loop forever between execute-phase
and verify-work. A gap-closure plan, or a later phase editing a covered
file, made the report stale; verify-work only recorded a passed UAT
against human_needed; the re-run verifier could not see the UAT file and
re-emitted human_needed; and execute-phase's human_needed branch then
rewrote *-UAT.md with every row back to [pending].

- verification.uat-evidence: recorded UAT rows plus the covered
  implementation files changed since the UAT. The verifier (Step 8b)
  treats a row that passed on unchanged code as verified and re-lists
  only affected rows, each with a retest_reason.
- verification.seed-uat: merges the report's human items into *-UAT.md.
  Existing rows are kept byte-for-byte; a pass is reset only with a
  recorded reason.
- verification.canonicalize-uat: the single human_needed -> passed flip,
  gated on the uat-only row predicate; refuses a stale report.
- verification.status reports stale_reason; init.progress lists every
  executed-but-stale phase in reverify_phases.
- verify-work and progress re-run the verifier themselves for a stale
  report instead of routing to execute-phase; execute-phase records a
  gap-closure checkpoint's UAT answers before dispatching the verifier.

Fail-closed properties are unchanged: a malformed fingerprint and a real
change to covered code still read stale.

Emitted-Drift-Ack-Growth: execute-phase.md — verify_phase_goal gains the gap-closure uat-record pointer and the merge-not-overwrite human_needed branch; the uat template it replaced moved into verification.seed-uat
Emitted-Drift-Ack-Growth: msd-verifier.md — new step 8b pointer to the lazily loaded verifier-uat-evidence reference plus the human_verified frontmatter key
Emitted-Drift-Ack-Growth: progress.md — route v.stale now re-verifies every stale executed phase in place instead of naming one command
This commit is contained in:
Jakub Zych
2026-10-11 01:44:30 +02:00
parent d1a3a31731
commit aa03fc7fa9
26 changed files with 1617 additions and 85 deletions

View File

@@ -1189,6 +1189,8 @@ If `section_manifest` is `null` or `"regression-gate"` is in its `included` list
<step name="verify_phase_goal">
Verify phase achieved its GOAL, not just completed tasks.
**Gap-closure runs:** if this run executed a `gap_closure: true` plan, first follow `execute-phase/steps/gap-closure-uat-record.md` — the human's checkpoint results go into `*-UAT.md` BEFORE the verifier is dispatched, so the same run can reach `passed`.
```bash
VERIFIER_SKILLS=$(msd_run query agent-skills msd-verifier)
```
@@ -1208,6 +1210,7 @@ Create VERIFICATION.md.
Read these files before verification:
- {phase_dir}/*-PLAN.md (All plans — understand intent, check must_haves)
- {phase_dir}/*-SUMMARY.md (All summaries — cross-reference claimed vs actual)
- {phase_dir}/*-UAT.md (If present: recorded human UAT — evidence, verifier Step 8b)
- {requirements_path} (Requirement traceability)
${CONTEXT_WINDOW >= 500000 ? `- {phase_dir}/*-CONTEXT.md (User decisions — verify they were honored)
- {phase_dir}/*-RESEARCH.md (Known pitfalls — check for traps)
@@ -1235,60 +1238,30 @@ Route on `$STATUS`: if `passed`, proceed to update_roadmap. Otherwise keep the p
**If human_needed:**
**Step A: Persist human verification items as UAT file.**
**Step A: Merge human verification items into the UAT file — never overwrite it.**
Create `{phase_dir}/{phase_num}-UAT.md` using UAT template format:
```markdown
---
status: testing
phase: {phase_num}-{phase_name}
source: [{phase_num}-VERIFICATION.md]
started: [now ISO]
updated: [now ISO]
---
## Current Test
number: 1
name: {first human_verification item description}
expected: |
{expected behavior from VERIFICATION.md}
awaiting: user response
## Tests
{For each human_verification item from VERIFICATION.md:}
### {N}. {item description}
expected: {expected behavior from VERIFICATION.md}
result: [pending]
## Summary
total: {count}
passed: 0
issues: 0
pending: {count}
skipped: 0
blocked: 0
## Gaps
```
Commit the file:
```bash
msd_run query commit "test({phase_num}): persist human verification items as UAT" --files "{phase_dir}/{phase_num}-UAT.md"
SEED=$(msd_run query verification.seed-uat "$PHASE_DIR")
SEED_UAT=$(printf '%s' "$SEED" | jq -r '.uat_file // empty')
SEED_PENDING=$(printf '%s' "$SEED" | jq -r '.pending // 0')
```
`verification.seed-uat` creates `{phase_num}-UAT.md` when none exists, and otherwise MERGES: every existing row and its recorded result is kept, only unseen items are appended as `[pending]`, and a passing row is reset only when the verifier recorded a `retest_reason` for it. Do NOT hand-write or regenerate the UAT file. When `.changed` is `true`, commit it:
```bash
msd_run query commit "test({phase_num}): persist human verification items as UAT" --files "$SEED_UAT"
```
**If `SEED_PENDING` is `0`** — every UAT row already has a result, so no human test is owed. Do NOT present the items as tests to run. Say so, and hand off: `/msd:verify-work {X} ${MSD_WS}` records the pass against the report at once (`verification.canonicalize-uat`) and completes the phase without re-asking a single row. Skip Step B.
**Step B: Present to user**:
```
## ◷ Phase {X}: {Name} — Human Verification Needed
All automated checks passed. {N} item(s) require human testing before this phase can be marked complete:
All automated checks passed. {SEED_PENDING} item(s) require human testing before this phase can be marked complete:
{From VERIFICATION.md human_verification section}
{The pending rows of {phase_num}-UAT.md — rows that already passed are NOT re-tested}
Tests saved to `{phase_num}-UAT.md`.