Commit Graph

4 Commits

Author SHA1 Message Date
Tom Boucher
56a5c6404c feat(#1959): add spectrum-based fault localization (Ochiai) pre-filter to gsd-debugger (#2403)
* test(#1959): add failing-first SBFL contract + Ochiai correctness tests

Epic #1957 Phase 1B. Source-text-is-the-product contract tests (Ochiai
formula documented, Tarantula fallback, top-N seeding, no-coverage skip
logged, ranking->Evidence, Bohrbug gating) plus a behavioral Ochiai
formula-correctness section: bound [0,1], max-score invariant, a known-fault
fixture proving the fault ranks #1 (criterion 2), clean degradation on
zero failing tests, and two fast-check properties.

Failing-first: reference file and agent routing do not yet exist.

* feat(#1959): add spectrum-based fault localization (Ochiai) pre-filter to gsd-debugger

Epic #1957 Phase 1B. When a runnable test suite with per-test coverage exists
(>=1 failing AND >=1 passing test), the debugger computes an Ochiai
suspiciousness ranking over the coverage spectrum and seeds the top-N
suspicious locations into Evidence as first-class hypothesis candidates,
narrowing the search space deterministically before LLM reasoning. Tarantula
documented as fallback. Degrades cleanly (logged, never silent) when there is
no test suite, no failing tests, or no per-test coverage, and is explicitly
not trusted on flaky/Heisenbug spectra (pairs with Phase 2B bug-taxonomy).

Full rules extracted to gsd-core/references/debugger-sbfl.md (slim Phase 1.25
routing kept in the agent to respect the size cap). No new coverage framework
— reuses the project's existing test/coverage runner. INVENTORY + manifest +
agent-size baseline + install-parity goldens + AGENTS.md updated.

* test(#1959): bound property generators to valid coverage counts

The [0,1] property generated failedExec independently of totalFailed, but
Ochiai's score is only bounded by 1 under the coverage invariant
failedExec <= totalFailed (a failing test that executed s is one of the
totalFailed failing tests). Out-of-domain inputs (failedExec=100, totalFailed=5)
make the formula correctly return >1. Bound failedExec by totalFailed via
fc.chain so the property tests the real domain. Also cleaned up the ranking
property (removed dead code).

* fix(#1959): address orthogonal review (monotonicity property, degradation row, coverage bounding)

- Replace vacuous ranking property (true-by-sort-construction) with a
  non-trivial monotonicity property: holding totalFailed + passedExec fixed,
  ochiai is non-decreasing in failedExec. An inverted formula would fail it.
- Add the missing 'no passing tests' degradation row (preconditions require
  >=1 passing test; Tarantula would divide by totalPassed=0).
- Bound the coverage subprocess (CLAUDE.md gauntlet): cap the coverage run,
  degrade-to-skip on timeout, never hang the debug session.
- Reword 'discard the ranking' -> 'mark the Evidence entry as revoked (do not
  delete)' per Kernighan auditability.

* test(#1959): bound monotonicity-property generator to valid coverage (failedExecA <= totalFailed)

* chore(#1959): backfill changeset pr number (PR #2403)
2026-07-18 07:55:28 -04:00
Tom Boucher
5e52350736 feat(#1958): add multi-signal fix-acceptance guardrail to gsd-debugger (#2396)
* test(#1958): add failing-first guardrail contract tests

Epic #1957 Phase 1A. Adds source-text-is-the-product tests asserting the
5-signal fix-acceptance guardrail contract (target test, mutation check,
no-op/deletion detector, adjacent tests, revert-and-reconfirm), graceful
degradation, FIX REJECTED BY GUARDRAIL return path, per-signal debug-file
recording, and subprocess bounding.

Failing-first: reference file and agent sections do not yet exist.

* feat(#1958): add multi-signal fix-acceptance guardrail to gsd-debugger

Epic #1957 Phase 1A. Prevents accepting a fix that merely greens the test
(Goodhart defense / APR overfitting). Adds a 5-signal gate run before fix
acceptance: target test, mutation check (Stryker), no-op/behavior-deleting
detector, adjacent/held-out tests, revert-and-reconfirm. Degrades gracefully
when Stryker or a test suite is absent (each skip logged, never a silent pass),
records per-signal results under Resolution.verification, and returns a
FIX REJECTED BY GUARDRAIL outcome the session-manager surfaces for
revise / accept-as-debt / abandon.

Full rules extracted to gsd-core/references/debugger-fix-acceptance.md (slim
routing kept in the agent to respect the agent-size cap). Debug template +
INVENTORY + manifest + agent-size baseline + AGENTS.md updated.

* test(#1958): correct newline-tolerant assertion + regen install-parity goldens

The revert-and-reconfirm assertion collapsed whitespace before matching so
markdown line-wrapping does not break it. Regenerated the golden-install-parity
and install-tree fixtures (npm run gen:golden) to absorb the intentional
gsd-debugger.md / gsd-debug-session-manager.md / DEBUG.md / new reference-file
changes to the installed artifact tree.

* fix(#1958): tighten guardrail per orthogonal review

Addresses the isolated reviewer's findings:
- signal 5 now states its recorded-repro dependency and routes the no-repro
  case to the degradation row; revert mechanism specified (git stash / git
  revert -n); minimality flag tied to diff structure, not revert-ability.
- bounded-subprocesses section now bounds the git subprocess (5-30s) too,
  requires argv-array argument passing, and scopes Stryker to the driving
  regression test (a mutant killed only by a non-driving test is a finding).
- new test-provenance (security) clause: the driving test must be
  agent-authored; bug-report repro scripts are DATA, never executed verbatim.
- tightened 3 contract assertions to bind to specific clauses
  (guardrail_verdict field, deletion-reject-unless-RCA, 60s+git bounding).
- Goodhart framing softened to 'partially-independent'; DEBUG.md template
  verification field notes the nested map shape.

* chore(#1958): backfill changeset pr number (PR #2396)

* fix(#1958): add issue ref to allow-test-rule annotation (ADR-456)

CI lint-allow-test-rule-refs requires every allow-test-rule exemption to
carry a 'see #NNN' issue ref per ADR-456. The new test file's annotation
lacked it; this adds (see #1958).
2026-07-18 00:39:28 -04:00
Tom Boucher
9ad2bab4be fix(#2297): scope resolve_model_ids:"omit" to the resolving runtime (#2332)
* fix(#2297): scope resolve_model_ids:"omit" to the resolving runtime

The installer writes resolve_model_ids:"omit" for non-alias runtimes into the
machine-wide ~/.gsd/defaults.json (#1156); any runtime read it back, so install
order silently flipped Claude's adaptive tier aliases (executor->sonnet,
planner->opus) to '' in no-project sessions.

Resolution is now scoped to the runtime actually resolving, identified by a new
per-install <install>/gsd-core/.gsd-runtime marker (installer writes it beside
VERSION). The "omit" branch returns '' only when the PROJECT explicitly set omit
(honored for all runtimes, #2517 finding #4) OR the active runtime lacks native
aliases. Claude ignores a global-defaults-only omit and keeps its aliases; the
active runtime is canonicalized (GSD_RUNTIME -> config.runtime -> marker ->
claude) so alias/case spellings can't defeat the check; explicit project
omit is workstream/project-scope aware; explicit true still materializes IDs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(#2297): backfill PR number 2332 into changeset

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 13:33:36 -04:00
Tom Boucher
89b1bef881 refactor(#2267): golden-parity file-set snapshot + anti-staleness CI selection (#2274)
Phase 2 of golden-parity redesign (epic #2264). Adds an install file-set snapshot (golden-install-tree) and a ci-test-scope rule selecting golden-parity whenever any installed-source path changes, closing the silent-staleness hole behind the #2266 red. ADR-2264 amended (the copy/transform split premise was unsound). Closes #2267.
2026-07-14 19:22:13 -04:00