* test(#1958): add failing-first guardrail contract tests Epic #1957 Phase 1A. Adds source-text-is-the-product tests asserting the 5-signal fix-acceptance guardrail contract (target test, mutation check, no-op/deletion detector, adjacent tests, revert-and-reconfirm), graceful degradation, FIX REJECTED BY GUARDRAIL return path, per-signal debug-file recording, and subprocess bounding. Failing-first: reference file and agent sections do not yet exist. * feat(#1958): add multi-signal fix-acceptance guardrail to gsd-debugger Epic #1957 Phase 1A. Prevents accepting a fix that merely greens the test (Goodhart defense / APR overfitting). Adds a 5-signal gate run before fix acceptance: target test, mutation check (Stryker), no-op/behavior-deleting detector, adjacent/held-out tests, revert-and-reconfirm. Degrades gracefully when Stryker or a test suite is absent (each skip logged, never a silent pass), records per-signal results under Resolution.verification, and returns a FIX REJECTED BY GUARDRAIL outcome the session-manager surfaces for revise / accept-as-debt / abandon. Full rules extracted to gsd-core/references/debugger-fix-acceptance.md (slim routing kept in the agent to respect the agent-size cap). Debug template + INVENTORY + manifest + agent-size baseline + AGENTS.md updated. * test(#1958): correct newline-tolerant assertion + regen install-parity goldens The revert-and-reconfirm assertion collapsed whitespace before matching so markdown line-wrapping does not break it. Regenerated the golden-install-parity and install-tree fixtures (npm run gen:golden) to absorb the intentional gsd-debugger.md / gsd-debug-session-manager.md / DEBUG.md / new reference-file changes to the installed artifact tree. * fix(#1958): tighten guardrail per orthogonal review Addresses the isolated reviewer's findings: - signal 5 now states its recorded-repro dependency and routes the no-repro case to the degradation row; revert mechanism specified (git stash / git revert -n); minimality flag tied to diff structure, not revert-ability. - bounded-subprocesses section now bounds the git subprocess (5-30s) too, requires argv-array argument passing, and scopes Stryker to the driving regression test (a mutant killed only by a non-driving test is a finding). - new test-provenance (security) clause: the driving test must be agent-authored; bug-report repro scripts are DATA, never executed verbatim. - tightened 3 contract assertions to bind to specific clauses (guardrail_verdict field, deletion-reject-unless-RCA, 60s+git bounding). - Goodhart framing softened to 'partially-independent'; DEBUG.md template verification field notes the nested map shape. * chore(#1958): backfill changeset pr number (PR #2396) * fix(#1958): add issue ref to allow-test-rule annotation (ADR-456) CI lint-allow-test-rule-refs requires every allow-test-rule exemption to carry a 'see #NNN' issue ref per ADR-456. The new test file's annotation lacked it; this adds (see #1958).
This commit is contained in:
Tom Boucher
@@ -270,6 +270,29 @@ If user selects 1 or 2: spawn continuation agent (with any additional context pr
|
||||
|
||||
If user selects 3: proceed to Step 4 with fix = "not applied".
|
||||
|
||||
### 3f. FIX REJECTED BY GUARDRAIL
|
||||
|
||||
When agent returns `## FIX REJECTED BY GUARDRAIL`:
|
||||
|
||||
Present the failing signal and evidence to the user via AskUserQuestion:
|
||||
```
|
||||
Fix rejected by the acceptance guardrail.
|
||||
|
||||
Failing signal: {failing signal}
|
||||
Evidence: {why it failed}
|
||||
|
||||
Options:
|
||||
1. Revise fix — spawn continuation agent to revise the fix so the signal passes
|
||||
2. Accept as technical debt — record the unmet signal + justification (the fix lands without the gate passing; this is never silent)
|
||||
3. Abandon — stop; session stays unresolved
|
||||
```
|
||||
|
||||
If user selects 1: spawn continuation agent with `goal: find_and_fix` naming the failing signal to revise. Loop back to Step 3.
|
||||
|
||||
If user selects 2: spawn continuation agent instructed to record `guardrail_verdict: accepted_debt` + the justification in the debug file, then proceed to request_human_verification. Loop back to Step 3.
|
||||
|
||||
If user selects 3: proceed to Step 4 with fix = "not applied (guardrail rejected)".
|
||||
|
||||
## Step 4: Return Compact Summary
|
||||
|
||||
**Non-terminal early stop — check this FIRST.** Before returning any summary below, ask: is your own turn/context budget exhausted while the debugger (`gsd-debugger`) is still investigating — i.e. you have NOT reached `DEBUG COMPLETE`, a user-chosen `ABANDONED`, or exhausted the `INVESTIGATION INCONCLUSIVE` options? If so, do NOT fabricate a `DEBUG SESSION COMPLETE` or `ABANDONED` summary to fit this shape. Return the non-terminal marker instead:
|
||||
|
||||
@@ -1091,11 +1091,15 @@ Update status to "fixing".
|
||||
- Make SMALLEST change that addresses root cause
|
||||
- Update Resolution.fix and Resolution.files_changed
|
||||
|
||||
**2. Verify**
|
||||
**2. Verify (Fix-Acceptance Guardrail)**
|
||||
- Update status to "verifying"
|
||||
- Test against original Symptoms
|
||||
- If verification FAILS: status -> "investigating", return to investigation_loop
|
||||
- If verification PASSES: Update Resolution.verification, proceed to request_human_verification
|
||||
- Run the multi-signal guardrail before accepting the fix:
|
||||
|
||||
@~/.claude/gsd-core/references/debugger-fix-acceptance.md
|
||||
|
||||
- Record every signal's result under `Resolution.verification` (per-signal schema in the reference)
|
||||
- If ANY applicable signal fails (and no documented technical-debt escape applies): return `## FIX REJECTED BY GUARDRAIL` (see structured_returns) — do NOT request human verification
|
||||
- If all applicable signals pass: set `guardrail_verdict: accepted`, proceed to request_human_verification
|
||||
</step>
|
||||
|
||||
<step name="request_human_verification">
|
||||
@@ -1334,6 +1338,16 @@ Orchestrator presents checkpoint to user, gets response, spawns fresh continuati
|
||||
|
||||
Only return this after human verification confirms the fix.
|
||||
|
||||
## FIX REJECTED BY GUARDRAIL
|
||||
|
||||
Returned when a fix-acceptance guardrail signal fails (see `@~/.claude/gsd-core/references/debugger-fix-acceptance.md`). Do **not** mark the session resolved.
|
||||
|
||||
**Debug Session:** .planning/debug/{slug}.md
|
||||
**Failing signal:** {signal 1–5 name}
|
||||
**Evidence:** {why the signal failed — e.g. "mutant at fix site survived", "deletion-only diff with no RCA justification", "bug did not return on revert"}
|
||||
|
||||
The session-manager continuation surfaces this and offers revise / accept-as-debt / abandon.
|
||||
|
||||
## INVESTIGATION INCONCLUSIVE
|
||||
|
||||
```markdown
|
||||
|
||||
Reference in New Issue
Block a user