golem15/msd-core
4
0
Fork 0
Code Issues 1 Pull Requests Actions 2 Packages Projects Releases Wiki Activity

feat(#1955): flag coincidental reliance in goal-backward verification (#3250)

Browse Source 
* test(#1955): failing-first contract for verifier coincidental-reliance advisory

* test(#1955): anchor coincidental-reliance assertions on the frontmatter block

* feat(#1955): flag coincidental reliance in goal-backward verification

* chore(#1955): correct stale workflow tier high-water comment

* fix(#1955): close the verify-phase divergence and state the endogeneity limit

* docs(#1955): backfill changeset pr number

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-09 12:07:41 -04:00
committed by GitHub
parent bc5619dd27
commit 6e59f97dd5
8 changed files with 328 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
agents/gsd-verifier.md
View File

@@ -200,6 +200,7 @@ For each truth:
- No such test exists, or it can't run without a server/state mutation → ⚠️ PRESENT_BEHAVIOR_UNVERIFIED. Emit a human-verification item (Step 8) and do not count it toward the verified score (Step 9).
- An accepted override (Step 3b) carries the truth as PASSED (override), exactly as it does for a FAILED truth.
5b. **Non-inferable (`backstop`) truths:** a `verification: backstop` truth (via `truthVerification()`) abstains unless confirmed by explicit evidence — mark `insufficient_spec` -> a human-verification item -> `human_needed`. See `references/honest-verifier.md`.
5c. **Reliance check (advisory, #1955).** Before finalizing a ✓ VERIFIED truth, ask *why* it holds. Classify the evidence already recorded, not your confidence in it. Endogenous, and so weaker than the exogenous `backstop` tag (`references/honest-verifier.md`) — advisory for exactly that reason. Flag `coincidental-reliance` when the evidence names one of: **undeclared-precondition** (state nothing in the phase's artifacts or a declared prerequisite guarantees), **incidental-ordering** (an order or side effect nothing in the code enforces), **fixture-only** (the test's own setup establishes the precondition; the production path has no equivalent). **Do NOT flag:** a precondition the code establishes or explicitly defaults; ordering the code enforces (await, explicit sequencing); a fixture merely supplying input the real caller also supplies; unease naming no specific state, ordering, or fixture. Out of scope: ⚠️ PRESENT_BEHAVIOR_UNVERIFIED and ⚠️ `insufficient_spec` (already routed to human), and PASSED (override) truths. Record `✓ VERIFIED (coincidental-reliance)` and add a `coincidental_reliance_items` entry. **Advisory only — not the score, not the status, and never a human-verification item** (Step 9 rule 2 would flip a passing phase to `human_needed`). The usual fix: promote the hidden assumption into a declared precondition.
6. Determine truth status
## Step 3b: Check Verification Overrides
@@ -554,6 +555,7 @@ Classify status using this decision tree IN ORDER (most restrictive first):
- `verified_truths` counts ✓ VERIFIED truths plus PASSED (override) truths (Step 3b). For a behavior-dependent truth, VERIFIED means a behavioral test passed, not just that symbols are present.
- ⚠️ PRESENT_BEHAVIOR_UNVERIFIED truths are the *only* ones excluded from `verified_truths`; they are reported separately as `behavior_unverified`.
- `✓ VERIFIED (coincidental-reliance)` counts as VERIFIED — the advisory changes no score and no status.
```text
score: verified_truths / total_truths # e.g. 6/7
@@ -701,6 +703,10 @@ behavior_unverified_items: # Only if behavior_unverified > 0 — emitted regardl
test: "What to trigger"
expected: "What state must hold afterward"
why_human: "Why presence checks can't see it"
coincidental_reliance_items: # Only if a ✓ VERIFIED truth holds incidentally — emitted regardless of overall status (survives gaps_found)
- truth: "Observable truth that holds incidentally"
reason: undeclared-precondition | incidental-ordering | fixture-only
harden: "Precondition/ordering to declare or enforce"
human_verification: # Only if status: human_needed
- test: "What to do"
expected: "What should happen"
@@ -723,6 +729,7 @@ human_verification: # Only if status: human_needed
| 1 | {truth} | ✓ VERIFIED | {evidence} |
| 2 | {truth} | ✗ FAILED | {what's wrong} |
| 3 | {truth} | ⚠️ PRESENT_BEHAVIOR_UNVERIFIED | {present + wired; no test exercises the transition/invariant — see Human Verification} |
| 4 | {truth} | ✓ VERIFIED (coincidental-reliance) | {holds, but incidentally — see coincidental_reliance_items} |
**Score:** {N}/{M} truths verified ({P} present, behavior-unverified)