Files
msd-core/capabilities/ai-integration/fragments/api-coverage-plan-pre.md
Tom Boucher 6addeccd19 feat(ai-integration): API-coverage verify:pre gate (#1562)
Full API Coverage by Default — Opt Out, Never Opt In. A phase that integrates
an external API/SDK/service can no longer seal without a decided coverage matrix.

- src/api-coverage.cts: deterministic detector (compound verb+noun signal +
  <Service> API/SDK surface; stopword-guarded; strips fenced code) + matrix
  parse/validate/render with field-length caps.
- check api-coverage.verify-pre: blocking seal-time gate; phase arg resolved as
  a token under .planning/phases/ only (traversal-neutralized); validates
  COVERAGE.md or blocks iff a strong integration signal is detected and no
  matrix exists; fail-closed when phases tree exists but phase unresolvable.
- capabilities/ai-integration: workflow.api_coverage_gate config key (default
  true), plan:pre contribution, blocking verify:pre gate. Data-driven.
- gsd-core/workflows/verify-work.md: generic verify:pre gate dispatch.
- Tests: detector FP/FN + matrix validation + fast-check bijection; gate e2e.
  Code+security review findings fixed (stopword FP, scope containment, pipe/cap
  rejection, prompt-injection message hygiene).
- Regenerated registry/matrix/loop-host-contract/goldens/baseline + docs.

Closes #1562
2026-07-07 15:11:12 -04:00

4.5 KiB

API Coverage Decision Checkpoint

Full API Coverage by Default — Opt Out, Never Opt In. Fires when a phase integrates an external API / SDK / service. Most non-API phases will not fire it — that is the point.

Why this exists

"We integrated the API" too often silently means "we integrated whatever the first use case exercised." Every un-built capability is then an invisible hole, discovered later by a user who reasonably expected it to work. The phase sealed green because its tasks completed; nobody decided the gaps were acceptable, because nobody enumerated them. This checkpoint makes the surface visible and decided before the phase can seal.

Detect whether this phase integrates an external API

The detector is a deterministic scan over the phase scope. It strips fenced code blocks first, so a trigger term inside a code snippet does not fire. It returns a typed result: { detected, signals[], terms }. Run it on the phase scope (the concatenation of this phase's ROADMAP section + the PLAN body):

SCOPE="$(cat "${PHASE_DIR}"/*-PLAN.md 2>/dev/null) $(gsd_run query roadmap.get-phase "${PHASE}" 2>/dev/null || true)"
API_COVERAGE_JSON=$(printf '%s' "$SCOPE" | node gsd-core/bin/lib/api-coverage.cjs --json 2>/dev/null || echo '{"detected":false,"signals":[]}')

Read API_COVERAGE_JSON.detected. Act on it only — do not pattern-match the prose yourself.

If detected is false: this phase does not integrate an external API. Skip the checkpoint entirely and continue planning. Do not raise it with the user.

If detected is true: an external-API integration is in scope. You MUST produce a coverage matrix before the plan is finalized.

Produce the coverage matrix

Enumerate the external API's full capability surface — the verb/endpoint/method list (e.g. for a music service: search, play, pause, skip, set_volume, get_playlist, create_playlist, add_to_playlist, …). For each capability record a decision, starting from full coverage as the default:

capability decision reason
<capability-id> INTEGRATE | OPT-OUT <one-line reason if OPT-OUT>

Rules:

  • INTEGRATE is the default. Every capability starts as INTEGRATE; the matrix is the subtraction record.
  • Every OPT-OUT MUST carry a one-line reason (not needed, not needed yet, explicitly out of scope, …). An opt-out without a reason is an un-decided hole — the exact failure mode this gate exists to close.
  • A second integration against the same need (e.g. a second platform for the same capability) starts from the same full-coverage baseline as the first. Do not carry over the first integration's opt-outs silently — re-decide each capability for the new surface, so a first-class/fallback asymmetry cannot accumulate.

Write the matrix to ${PHASE_DIR}/COVERAGE.md (canonical markdown-table form):

# API Coverage — <service>

> Full coverage by default. Opt-outs are explicit, reasoned decisions.

| capability | decision | reason |
|---|---|---|
| search | INTEGRATE | |
| playlists | INTEGRATE | |
| skip | OPT-OUT | not needed yet — tracked for follow-up phase |

A fenced ```coverage JSON block is also accepted for machine-generated matrices; the markdown table is preferred (human-editable, diff-friendly).

The seal-time gate

This checkpoint is enforced. At verify:pre the api-coverage.verify-pre gate runs check api-coverage.verify-pre <phase-dir>:

  • If COVERAGE.md exists, it is validated — every row needs a valid decision and every OPT-OUT a reason. A malformed/partial matrix blocks the seal.
  • If COVERAGE.md is absent, the detector runs again over the phase scope. If a strong external-API-integration signal is found, the seal is blocked until a matrix is produced. If no signal is found, the phase is treated as a non-API phase and the seal proceeds.

So: an API-integrating phase cannot seal without a decided matrix. Produce it at plan time; do not leave it for seal time.

Tuning the vocabulary (optional)

The trigger vocabulary is a curated, additive-only set in gsd-core/bin/lib/api-coverage.cjs (DEFAULT_API_COVERAGE_TERMS). To widen it for a project, override at the call site:

printf '%s' "$SCOPE" | node gsd-core/bin/lib/api-coverage.cjs --json \
  --verbs integrate,wrap,connect,embed --nouns api,sdk,rest,grpc,webhook,plugin

The whole checkpoint is toggleable via workflow.api_coverage_gate in .planning/config.json.