6addeccd19
Full API Coverage by Default — Opt Out, Never Opt In. A phase that integrates an external API/SDK/service can no longer seal without a decided coverage matrix. - src/api-coverage.cts: deterministic detector (compound verb+noun signal + <Service> API/SDK surface; stopword-guarded; strips fenced code) + matrix parse/validate/render with field-length caps. - check api-coverage.verify-pre: blocking seal-time gate; phase arg resolved as a token under .planning/phases/ only (traversal-neutralized); validates COVERAGE.md or blocks iff a strong integration signal is detected and no matrix exists; fail-closed when phases tree exists but phase unresolvable. - capabilities/ai-integration: workflow.api_coverage_gate config key (default true), plan:pre contribution, blocking verify:pre gate. Data-driven. - gsd-core/workflows/verify-work.md: generic verify:pre gate dispatch. - Tests: detector FP/FN + matrix validation + fast-check bijection; gate e2e. Code+security review findings fixed (stopword FP, scope containment, pipe/cap rejection, prompt-injection message hygiene). - Regenerated registry/matrix/loop-host-contract/goldens/baseline + docs. Closes #1562
106 lines
4.5 KiB
Markdown
106 lines
4.5 KiB
Markdown
# API Coverage Decision Checkpoint
|
|
|
|
> Full API Coverage by Default — Opt Out, Never Opt In. Fires when a phase
|
|
> integrates an external API / SDK / service. Most non-API phases will not fire
|
|
> it — that is the point.
|
|
|
|
## Why this exists
|
|
|
|
"We integrated the API" too often silently means "we integrated whatever the
|
|
first use case exercised." Every un-built capability is then an invisible hole,
|
|
discovered later by a user who reasonably expected it to work. The phase sealed
|
|
green because its tasks completed; nobody decided the gaps were acceptable,
|
|
because nobody enumerated them. This checkpoint makes the surface **visible and
|
|
decided** before the phase can seal.
|
|
|
|
## Detect whether this phase integrates an external API
|
|
|
|
The detector is a deterministic scan over the phase scope. It strips fenced
|
|
code blocks first, so a trigger term inside a code snippet does not fire. It
|
|
returns a typed result: `{ detected, signals[], terms }`. Run it on the phase
|
|
scope (the concatenation of this phase's ROADMAP section + the PLAN body):
|
|
|
|
```bash
|
|
SCOPE="$(cat "${PHASE_DIR}"/*-PLAN.md 2>/dev/null) $(gsd_run query roadmap.get-phase "${PHASE}" 2>/dev/null || true)"
|
|
API_COVERAGE_JSON=$(printf '%s' "$SCOPE" | node gsd-core/bin/lib/api-coverage.cjs --json 2>/dev/null || echo '{"detected":false,"signals":[]}')
|
|
```
|
|
|
|
Read `API_COVERAGE_JSON.detected`. Act on it only — do **not** pattern-match the
|
|
prose yourself.
|
|
|
|
**If `detected` is `false`:** this phase does not integrate an external API. Skip
|
|
the checkpoint entirely and continue planning. Do not raise it with the user.
|
|
|
|
**If `detected` is `true`:** an external-API integration is in scope. You MUST
|
|
produce a **coverage matrix** before the plan is finalized.
|
|
|
|
## Produce the coverage matrix
|
|
|
|
Enumerate the external API's full **capability surface** — the verb/endpoint/method
|
|
list (e.g. for a music service: `search`, `play`, `pause`, `skip`, `set_volume`,
|
|
`get_playlist`, `create_playlist`, `add_to_playlist`, …). For each capability
|
|
record a decision, starting from **full coverage** as the default:
|
|
|
|
| capability | decision | reason |
|
|
|---|---|---|
|
|
| `<capability-id>` | `INTEGRATE` \| `OPT-OUT` | `<one-line reason if OPT-OUT>` |
|
|
|
|
Rules:
|
|
|
|
- **`INTEGRATE` is the default.** Every capability starts as INTEGRATE; the
|
|
matrix is the *subtraction record*.
|
|
- **Every `OPT-OUT` MUST carry a one-line reason** (`not needed`, `not needed
|
|
yet`, `explicitly out of scope`, …). An opt-out without a reason is an
|
|
un-decided hole — the exact failure mode this gate exists to close.
|
|
- **A second integration against the same need** (e.g. a second platform for the
|
|
same capability) starts from the **same full-coverage baseline** as the first.
|
|
Do not carry over the first integration's opt-outs silently — re-decide each
|
|
capability for the new surface, so a first-class/fallback asymmetry cannot
|
|
accumulate.
|
|
|
|
Write the matrix to `${PHASE_DIR}/COVERAGE.md` (canonical markdown-table form):
|
|
|
|
```markdown
|
|
# API Coverage — <service>
|
|
|
|
> Full coverage by default. Opt-outs are explicit, reasoned decisions.
|
|
|
|
| capability | decision | reason |
|
|
|---|---|---|
|
|
| search | INTEGRATE | |
|
|
| playlists | INTEGRATE | |
|
|
| skip | OPT-OUT | not needed yet — tracked for follow-up phase |
|
|
```
|
|
|
|
A fenced ` ```coverage ` JSON block is also accepted for machine-generated
|
|
matrices; the markdown table is preferred (human-editable, diff-friendly).
|
|
|
|
## The seal-time gate
|
|
|
|
This checkpoint is enforced. At `verify:pre` the `api-coverage.verify-pre` gate
|
|
runs `check api-coverage.verify-pre <phase-dir>`:
|
|
|
|
- If `COVERAGE.md` exists, it is validated — every row needs a valid decision and
|
|
every `OPT-OUT` a reason. A malformed/partial matrix **blocks the seal**.
|
|
- If `COVERAGE.md` is absent, the detector runs again over the phase scope. If a
|
|
strong external-API-integration signal is found, the seal is **blocked** until a
|
|
matrix is produced. If no signal is found, the phase is treated as a non-API
|
|
phase and the seal proceeds.
|
|
|
|
So: an API-integrating phase cannot seal without a decided matrix. Produce it at
|
|
plan time; do not leave it for seal time.
|
|
|
|
## Tuning the vocabulary (optional)
|
|
|
|
The trigger vocabulary is a curated, additive-only set in
|
|
`gsd-core/bin/lib/api-coverage.cjs` (`DEFAULT_API_COVERAGE_TERMS`). To widen it
|
|
for a project, override at the call site:
|
|
|
|
```bash
|
|
printf '%s' "$SCOPE" | node gsd-core/bin/lib/api-coverage.cjs --json \
|
|
--verbs integrate,wrap,connect,embed --nouns api,sdk,rest,grpc,webhook,plugin
|
|
```
|
|
|
|
The whole checkpoint is toggleable via `workflow.api_coverage_gate` in
|
|
`.planning/config.json`.
|