enh(#966): gate behavior-dependent truths on behavioral evidence in gsd-verifier (#1271)

* enh(#966): gate behavior-dependent truths on behavioral evidence in gsd-verifier

Introduce a per-truth PRESENT_BEHAVIOR_UNVERIFIED state for must-haves that assert a state transition or a cancellation/cleanup/ordering invariant whose only evidence is symbol presence + wiring. Such truths are excluded from the verified_truths score, reported as a behavior_unverified count, recorded in an always-on behavior_unverified_items frontmatter list, and routed to the existing human_needed sink — so a clean N/N can no longer be reached on symbol presence alone.

The overall-status vocabulary and the src/verification.cts seam are unchanged (the new state is per-truth only); gaps_found keeps decision-tree precedence; override-passed truths still count toward verified_truths. Mirrors the calibration into the shipped verify-phase.md workflow (with an infra/foundation carve-out), the VERIFICATION.md templates, and docs (planning-artifacts.md, AGENTS.md). gsd-verifier.md kept under its 48KB LARGE cap; size baselines regenerated.

Closes #966

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(#966): add changeset fragment for gsd-verifier behavior-unverified calibration

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-06-15 15:24:56 -04:00
committed by GitHub
parent 48d9cec6fe
commit 0a856f06cd
9 changed files with 209 additions and 24 deletions

View File

@@ -12,6 +12,12 @@ phase: XX-name
verified: YYYY-MM-DDTHH:MM:SSZ
status: passed | gaps_found | human_needed
score: N/M must-haves verified
behavior_unverified: 0 # Count of ⚠️ PRESENT_BEHAVIOR_UNVERIFIED truths (present + wired, behavior not exercised)
behavior_unverified_items: # Only if behavior_unverified > 0 — the truths above as structured items; emitted regardless of overall status
- truth: "Observable truth whose state transition or cancellation/cleanup/ordering invariant no test exercises"
test: "What to trigger"
expected: "What state must hold afterward"
why_human: "Why presence checks can't see it"
---
# Phase {X}: {Name} Verification Report
@@ -28,9 +34,10 @@ score: N/M must-haves verified
|---|-------|--------|----------|
| 1 | {truth from must_haves} | ✓ VERIFIED | {what confirmed it} |
| 2 | {truth from must_haves} | ✗ FAILED | {what's wrong} |
| 3 | {truth from must_haves} | ? UNCERTAIN | {why can't verify} |
| 3 | {truth from must_haves} | ⚠️ PRESENT_BEHAVIOR_UNVERIFIED | {present + wired; transition/invariant not exercised by a test — see Human Verification} |
| 4 | {truth from must_haves} | ? UNCERTAIN | {why can't verify} |
**Score:** {N}/{M} truths verified
**Score:** {N}/{M} truths verified ({P} present, behavior-unverified)
### Required Artifacts
@@ -161,11 +168,17 @@ None — all verifiable items checked programmatically.
## Guidelines
**Status values:**
**Status values (overall, frontmatter `status:`):**
- `passed` — All must-haves verified, no blockers
- `gaps_found` — One or more critical gaps found
- `human_needed` — Automated checks pass but human verification required
**Per-truth states (Observable Truths `Status` column):**
- `✓ VERIFIED` — supporting artifacts pass all checks; for a behavior-dependent truth, a behavioral test exercised the asserted behavior
- `⚠️ PRESENT_BEHAVIOR_UNVERIFIED` — present + wired, but a state transition or cancellation/cleanup/ordering invariant was not exercised by any test. Counts toward `behavior_unverified`, routes to human verification, and is *excluded* from the verified score. Per-truth only — on its own the overall `status:` becomes `human_needed` (unless a higher-precedence `gaps_found` also applies); the item is preserved in `behavior_unverified_items` regardless.
- `✗ FAILED` — artifact missing, stub, or unwired
- `? UNCERTAIN` — can't verify programmatically
**Evidence types:**
- For EXISTS: "File at path, exports X"
- For SUBSTANTIVE: "N lines, has patterns X, Y, Z"