6addeccd19
Full API Coverage by Default — Opt Out, Never Opt In. A phase that integrates an external API/SDK/service can no longer seal without a decided coverage matrix. - src/api-coverage.cts: deterministic detector (compound verb+noun signal + <Service> API/SDK surface; stopword-guarded; strips fenced code) + matrix parse/validate/render with field-length caps. - check api-coverage.verify-pre: blocking seal-time gate; phase arg resolved as a token under .planning/phases/ only (traversal-neutralized); validates COVERAGE.md or blocks iff a strong integration signal is detected and no matrix exists; fail-closed when phases tree exists but phase unresolvable. - capabilities/ai-integration: workflow.api_coverage_gate config key (default true), plan:pre contribution, blocking verify:pre gate. Data-driven. - gsd-core/workflows/verify-work.md: generic verify:pre gate dispatch. - Tests: detector FP/FN + matrix validation + fast-check bijection; gate e2e. Code+security review findings fixed (stopword FP, scope containment, pipe/cap rejection, prompt-injection message hygiene). - Regenerated registry/matrix/loop-host-contract/goldens/baseline + docs. Closes #1562
105 lines
4.5 KiB
Markdown
105 lines
4.5 KiB
Markdown
# API Coverage Gate (Full Coverage by Default — Opt Out, Never Opt In)
|
|
|
|
> Reference for the `api-coverage` gate on the `ai-integration` capability (#1562).
|
|
> Config key: `workflow.api_coverage_gate` (default `true`). Gate point: `verify:pre`.
|
|
|
|
## The problem this closes
|
|
|
|
"We integrated the API" too often silently means "we integrated whatever the
|
|
first use case exercised." Every un-built capability is then an invisible hole,
|
|
discovered later by a user who reasonably expected it to work. The phase sealed
|
|
green because its tasks completed — nobody *decided* the gaps were acceptable,
|
|
because nobody *enumerated* them.
|
|
|
|
This gate makes the API surface **visible and decided** before the phase can
|
|
seal. Full coverage is the default starting position; the coverage matrix is the
|
|
*subtraction record*. Every gap is an explicit, reasoned opt-out rather than a
|
|
surprise.
|
|
|
|
## When it fires
|
|
|
|
The gate runs at `verify:pre` (before `/gsd:verify-work` begins UAT). A phase is
|
|
treated as an external-API integration when **either**:
|
|
|
|
1. a `COVERAGE.md` matrix is present in the phase directory (the planner produced
|
|
one at `plan:pre`), **or**
|
|
2. the phase scope shows a strong external-API-integration signal (an integration
|
|
verb co-occurring with an external-API noun, or an explicit `<Service>
|
|
API|SDK|REST|GraphQL` surface) and no matrix yet exists.
|
|
|
|
Non-API phases (refactors, bug fixes, internal-only work, features that merely
|
|
*mention* an existing internal API) do **not** fire the gate — the trigger
|
|
requires a compound signal, so a bare word like "api" in "the public API of
|
|
UserController" is intentionally ignored.
|
|
|
|
## The two touch points
|
|
|
|
1. **Plan time (`plan:pre`).** A contribution to the planner prompts it to run
|
|
the deterministic detector over the phase scope and, when an integration is
|
|
detected, produce `COVERAGE.md`. See
|
|
`capabilities/ai-integration/fragments/api-coverage-plan-pre.md`.
|
|
2. **Seal time (`verify:pre`).** The blocking `api-coverage.verify-pre` gate
|
|
runs `check api-coverage.verify-pre <phase-dir>` and blocks unless a valid
|
|
matrix exists (or no integration is detected).
|
|
|
|
## The coverage matrix format
|
|
|
|
Canonical form — a markdown table (human-editable, diff-friendly):
|
|
|
|
```markdown
|
|
# API Coverage — <service>
|
|
|
|
> Full coverage by default. Opt-outs are explicit, reasoned decisions.
|
|
|
|
| capability | decision | reason |
|
|
|---|---|---|
|
|
| search | INTEGRATE | |
|
|
| playlists | INTEGRATE | |
|
|
| skip | OPT-OUT | not needed yet — tracked for follow-up phase |
|
|
```
|
|
|
|
- **`INTEGRATE` is the default.** Every capability starts as INTEGRATE.
|
|
- **Every `OPT-OUT` MUST carry a one-line reason** (`not needed`, `not needed
|
|
yet`, `explicitly out of scope`, …). An opt-out without a reason is an
|
|
un-decided hole — the exact failure mode this gate exists to close.
|
|
- A fenced ` ```coverage ` JSON block (`[{"capability":…,"decision":…,"reason":…}]`)
|
|
is also accepted for machine-generated matrices.
|
|
|
|
Rules enforced at seal time: the matrix must be non-empty; every capability name
|
|
must be non-empty and unique; every decision must be `INTEGRATE` or `OPT-OUT`;
|
|
every `OPT-OUT` must have a reason. Violations block the seal with a precise
|
|
error.
|
|
|
|
## A second integration against the same need
|
|
|
|
A second platform for an existing capability (e.g. adding YouTube alongside
|
|
Spotify for media playback) starts from the **same full-coverage baseline** as
|
|
the first. Do not carry over the first integration's opt-outs silently —
|
|
re-decide each capability for the new surface, so a first-class/fallback
|
|
asymmetry cannot accumulate into a later user-facing bug.
|
|
|
|
## The matrix persists
|
|
|
|
`COVERAGE.md` is a phase artifact. A future phase that extends the same
|
|
integration starts from the recorded surface and decisions rather than from
|
|
zero — the matrix is the durable subtraction record.
|
|
|
|
## Tuning
|
|
|
|
- **Disable entirely:** set `workflow.api_coverage_gate: false` in
|
|
`.planning/config.json` (the gate unregisters from `verify:pre`).
|
|
- **Widen the trigger vocabulary:** the detector accepts `--verbs` / `--nouns`
|
|
overrides (see `capabilities/ai-integration/fragments/api-coverage-plan-pre.md`).
|
|
The default vocabulary is additive-only.
|
|
|
|
## Detector CLI
|
|
|
|
```bash
|
|
echo "$PHASE_SCOPE" | node gsd-core/bin/lib/api-coverage.cjs --json
|
|
# exit 0 = integration detected, 1 = none, 2 = startup error
|
|
```
|
|
|
|
The detector is a pure function (`detectApiIntegration` → `{ detected, signals,
|
|
terms }`) shared by the plan-time prompt and the seal-time gate, so the
|
|
low-false-positive guarantee is testable rather than a judgment call.
|