* enh(#966): gate behavior-dependent truths on behavioral evidence in gsd-verifier Introduce a per-truth PRESENT_BEHAVIOR_UNVERIFIED state for must-haves that assert a state transition or a cancellation/cleanup/ordering invariant whose only evidence is symbol presence + wiring. Such truths are excluded from the verified_truths score, reported as a behavior_unverified count, recorded in an always-on behavior_unverified_items frontmatter list, and routed to the existing human_needed sink — so a clean N/N can no longer be reached on symbol presence alone. The overall-status vocabulary and the src/verification.cts seam are unchanged (the new state is per-truth only); gaps_found keeps decision-tree precedence; override-passed truths still count toward verified_truths. Mirrors the calibration into the shipped verify-phase.md workflow (with an infra/foundation carve-out), the VERIFICATION.md templates, and docs (planning-artifacts.md, AGENTS.md). gsd-verifier.md kept under its 48KB LARGE cap; size baselines regenerated. Closes #966 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(#966): add changeset fragment for gsd-verifier behavior-unverified calibration Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -12,6 +12,12 @@ phase: XX-name
|
||||
verified: YYYY-MM-DDTHH:MM:SSZ
|
||||
status: passed | gaps_found | human_needed
|
||||
score: N/M must-haves verified
|
||||
behavior_unverified: 0 # Count of ⚠️ PRESENT_BEHAVIOR_UNVERIFIED truths (present + wired, behavior not exercised)
|
||||
behavior_unverified_items: # Only if behavior_unverified > 0 — the truths above as structured items; emitted regardless of overall status
|
||||
- truth: "Observable truth whose state transition or cancellation/cleanup/ordering invariant no test exercises"
|
||||
test: "What to trigger"
|
||||
expected: "What state must hold afterward"
|
||||
why_human: "Why presence checks can't see it"
|
||||
---
|
||||
|
||||
# Phase {X}: {Name} Verification Report
|
||||
@@ -28,9 +34,10 @@ score: N/M must-haves verified
|
||||
|---|-------|--------|----------|
|
||||
| 1 | {truth from must_haves} | ✓ VERIFIED | {what confirmed it} |
|
||||
| 2 | {truth from must_haves} | ✗ FAILED | {what's wrong} |
|
||||
| 3 | {truth from must_haves} | ? UNCERTAIN | {why can't verify} |
|
||||
| 3 | {truth from must_haves} | ⚠️ PRESENT_BEHAVIOR_UNVERIFIED | {present + wired; transition/invariant not exercised by a test — see Human Verification} |
|
||||
| 4 | {truth from must_haves} | ? UNCERTAIN | {why can't verify} |
|
||||
|
||||
**Score:** {N}/{M} truths verified
|
||||
**Score:** {N}/{M} truths verified ({P} present, behavior-unverified)
|
||||
|
||||
### Required Artifacts
|
||||
|
||||
@@ -161,11 +168,17 @@ None — all verifiable items checked programmatically.
|
||||
|
||||
## Guidelines
|
||||
|
||||
**Status values:**
|
||||
**Status values (overall, frontmatter `status:`):**
|
||||
- `passed` — All must-haves verified, no blockers
|
||||
- `gaps_found` — One or more critical gaps found
|
||||
- `human_needed` — Automated checks pass but human verification required
|
||||
|
||||
**Per-truth states (Observable Truths `Status` column):**
|
||||
- `✓ VERIFIED` — supporting artifacts pass all checks; for a behavior-dependent truth, a behavioral test exercised the asserted behavior
|
||||
- `⚠️ PRESENT_BEHAVIOR_UNVERIFIED` — present + wired, but a state transition or cancellation/cleanup/ordering invariant was not exercised by any test. Counts toward `behavior_unverified`, routes to human verification, and is *excluded* from the verified score. Per-truth only — on its own the overall `status:` becomes `human_needed` (unless a higher-precedence `gaps_found` also applies); the item is preserved in `behavior_unverified_items` regardless.
|
||||
- `✗ FAILED` — artifact missing, stub, or unwired
|
||||
- `? UNCERTAIN` — can't verify programmatically
|
||||
|
||||
**Evidence types:**
|
||||
- For EXISTS: "File at path, exports X"
|
||||
- For SUBSTANTIVE: "N lines, has patterns X, Y, Z"
|
||||
|
||||
Reference in New Issue
Block a user