Files
msd-core/msd-core/references/verifier-uat-evidence.md
Jakub Zych aa03fc7fa9 fix(verification): keep passed uat results across verifier re-runs
A phase whose human UAT passed could loop forever between execute-phase
and verify-work. A gap-closure plan, or a later phase editing a covered
file, made the report stale; verify-work only recorded a passed UAT
against human_needed; the re-run verifier could not see the UAT file and
re-emitted human_needed; and execute-phase's human_needed branch then
rewrote *-UAT.md with every row back to [pending].

- verification.uat-evidence: recorded UAT rows plus the covered
  implementation files changed since the UAT. The verifier (Step 8b)
  treats a row that passed on unchanged code as verified and re-lists
  only affected rows, each with a retest_reason.
- verification.seed-uat: merges the report's human items into *-UAT.md.
  Existing rows are kept byte-for-byte; a pass is reset only with a
  recorded reason.
- verification.canonicalize-uat: the single human_needed -> passed flip,
  gated on the uat-only row predicate; refuses a stale report.
- verification.status reports stale_reason; init.progress lists every
  executed-but-stale phase in reverify_phases.
- verify-work and progress re-run the verifier themselves for a stale
  report instead of routing to execute-phase; execute-phase records a
  gap-closure checkpoint's UAT answers before dispatching the verifier.

Fail-closed properties are unchanged: a malformed fingerprint and a real
change to covered code still read stale.

Emitted-Drift-Ack-Growth: execute-phase.md — verify_phase_goal gains the gap-closure uat-record pointer and the merge-not-overwrite human_needed branch; the uat template it replaced moved into verification.seed-uat
Emitted-Drift-Ack-Growth: msd-verifier.md — new step 8b pointer to the lazily loaded verifier-uat-evidence reference plus the human_verified frontmatter key
Emitted-Drift-Ack-Growth: progress.md — route v.stale now re-verifies every stale executed phase in place instead of naming one command
2026-10-11 01:44:30 +02:00

3.1 KiB

Verifier UAT Evidence

Loaded by agents/msd-verifier.md at Step 8b, only when the phase directory holds a *-UAT.md. A human who already ran a check is evidence. Re-asking for it on every verifier re-run — after a gap-closure plan, or after a covered file drifted — is the loop this gate exists to end. msd_run is the launcher shim from the agent's Step 1.

1. Read the evidence

Pass the ROOT-relative implementation files you are about to declare in covered_files:

msd_run query verification.uat-evidence "$PHASE_DIR" {impl file}...

The result carries rows (each UAT row: test, name, result, passing, expected), recorded_at (when the UAT file last changed), and changed_since_uat (covered implementation files modified, or gone, AFTER the UAT was recorded). An empty uat_file means there is no UAT evidence — skip this gate.

2. Settle each Step 8 human item against its row

Match by meaning, not by wording: a row covers an item when a human performing the row's check would have observed what the item asks for.

Row for the item changed_since_uat Outcome
passing: true empty, or no changed file bears on what the row exercised Verified by human UAT. Remove the item from the human verification list. Record it under human_verified (below) and mark the truth ✓ VERIFIED (human UAT, row N).
passing: true a changed file bears on what the row exercised Re-test. Keep the item in human_verification with retest_reason naming the file and recorded_at. Use the row's exact name as the item's test.
not passing (pending, issue, blocked, skipped, missing) any Ordinary human item. Use the row's exact name as test; no retest_reason.
no row any Ordinary new human item.

Rules:

  • Unsure whether a changed file affects a row → re-test. A changed file you cannot rule out is a changed file that bears on it. Real change still costs a re-test.
  • A pass is only ever withdrawn with a reason. retest_reason is what lets the UAT writer reset that row; an item re-listed without one leaves the recorded pass in place.
  • A ⚠️ PRESENT_BEHAVIOR_UNVERIFIED truth whose row passed on unchanged code has directly observed behavior: it is VERIFIED by that row and leaves behavior_unverified_items.
  • Step 9 then runs on what is LEFT. If every human item was verified by UAT, the human verification section is empty and the ordinary tree yields passed (or gaps_found).
  • Never write or edit *-UAT.md. The orchestrator merges your human_verification items into it with verification.seed-uat, which keeps every recorded result.

3. Frontmatter

human_verified: # Only if UAT evidence settled at least one item
  - test: "Exact UAT row name"
    uat_row: 3
    recorded_at: "ISO time from uat-evidence"
human_verification: # Only if status: human_needed
  - test: "Exact UAT row name"
    expected: "What should happen"
    why_human: "Why can't verify programmatically"
    retest_reason: "src/auth/session.ts changed after the UAT pass (2026-10-05T…)" # ONLY for a row that had passed