msd-core/gsd-core/references/debugger-sbfl.md

# Spectrum-Based Fault Localization (SBFL) Pre-filter

Loaded by `gsd-debugger` via `@-include`. A deterministic ranked "where to look
first" list derived from existing test pass/fail coverage, computed before LLM
reasoning over an unranked search space.

## Why this exists

`investigation_loop` Phase 1 searches the codebase and reads files to seed
hypotheses via (expensive, non-deterministic) LLM reasoning over an unranked
space. When a runnable test suite with per-test coverage exists, there is a
cheap, deterministic signal being left on the table: which code is
disproportionately executed by **failing** vs **passing** tests. SBFL turns
that coverage spectrum into a suspiciousness ranking. The agent currently has no
fault-localization step at all.

## When to run it (Phase 1.25 — after initial evidence, before hypothesis formation)

Run only when ALL hold:
- A runnable test suite exists for the failing area.
- At least one failing test AND at least one passing test exist (a spectrum
  requires both).
- Per-test coverage is available (which tests executed which code
  element — function, line, or branch).

If any precondition fails, **skip** this step with a logged note (see
Degradation) and proceed with Phase 1's normal evidence gathering unchanged.

## The Ochiai formula

For each code element `s` executed by the test suite:

```
ochiai(s) = failed(s) / sqrt(totalFailed × (failed(s) + passed(s)))
```

where:
- `failed(s)` = number of **failing** tests that executed `s`
- `passed(s)` = number of **passing** tests that executed `s`
- `totalFailed` = total number of failing tests in the suite

The score is in `[0, 1]`. An element executed by every failing test and no
passing test scores `1.0` (maximum suspiciousness). An element touched only by
passing tests scores `0`. Ochiai is empirically stronger than Tarantula across
the SBFL literature; **Tarantula** is the documented fallback formula
(`tarantula(s) = (failed(s)/totalFailed) / ((failed(s)/totalFailed) + (passed(s)/totalPassed))`)
if a comparison or secondary signal is wanted.

## Output — top-N shortlist seeded into the hypothesis space

Rank all executed elements by descending Ochiai score and take the **top-N**
(N is judgment — 5–10 is typical; bounded by what narrows the search without
flooding it). Append each top-N element to the debug file's **Evidence**
section as a first-class hypothesis candidate:

```
- timestamp: <now>
  checked: SBFL Ochiai ranking (Phase 1.25)
  found: top-N suspicious locations —
    1. path/to/file.cts:LINE (score 0.89) — <symbol>
    2. path/to/other.cts:LINE (score 0.77) — <symbol>
    ...
  implication: investigate these before forming broader hypotheses
```

This narrows the search space by orders of magnitude before any LLM tokens are
spent forming hypotheses. Each top-N entry becomes a candidate for Phase 2
hypothesis formation, ranked ahead of un-evidenced guesses.

## Degradation (Gall's Law — optional step, degrades onto the working agent)

This step is **purely additive**. Every miss degrades to today's behavior; a
skipped step is logged, never a silent pass (Kernighan — the debugger stays
auditable).

| Condition | Behavior |
|---|---|
| No test suite for the failing area | **skip** with a logged note in Evidence ("SBFL skipped: no test suite"); Phase 1 proceeds unchanged |
| Test suite but no failing tests | **skip** with a logged note ("SBFL skipped: no failing tests — no spectrum"); Phase 1 proceeds unchanged |
| Test suite but no passing tests | **skip** with a logged note ("SBFL skipped: no spectrum — no passing tests"); Phase 1 proceeds unchanged (Tarantula would divide by `totalPassed=0`; do not run it) |
| Test suite but no per-test coverage | **skip** with a logged note ("SBFL skipped: no per-test coverage available"); Phase 1 proceeds unchanged |
| Coverage exists but is coarse (file-level, not line/function) | run anyway, rank at the available granularity, and note the granularity in Evidence |

## Bug-class gating (pairs with Phase 2B bug-taxonomy routing)

SBFL is the go-to pre-filter for **deterministic failures (Bohrbugs)** — bugs
that reproduce reliably. It is explicitly **not trusted** on
**Heisenbug/Mandelbug** spectra (timing, races, environment-dependent failures):
a flaky suite pollutes the spectrum (a "failing" test that sometimes passes
poisons `failed(s)`), so the ranking becomes noise. When the failure is
non-deterministic (Phase 2B classifies it), **skip SBFL** and route to
record-replay or stability-stress instead. If SBFL has already run before
classification and the class later resolves to Heisenbug/Mandelbug, mark the
prior SBFL Evidence entry as revoked (do not delete it — Kernighan
auditability) and note why in Evidence.

## Scope boundary (Zawinski's Law)

This is a deterministic pre-filter that reuses the project's existing
test/coverage runner — it adds **no new coverage framework** and no new
subsystem. It narrows the LLM's search space; it does not replace hypothesis
formation, fix-and-verify, or the knowledge base. Coverage acquisition is the
agent's adaptive job (use whatever coverage the project produces); the formula
above is the canonical ranking.

**Bound the coverage run** (CLAUDE.md gauntlet — unbounded subprocess): a
coverage run is often 2–3× slower than a plain test run due to instrumentation,
so cap it (60s for npm-tier suites; scale with suite size) and **degrade to
skip with a logged note on timeout** — never let coverage acquisition hang the
debug session.