Files

Tom Boucher c5f2b94b27 enhance(#3907 ): gates report no-input instead of a verdict they never reached (#3932 )

* feat(#3907): gates report no-input instead of asserting a verdict they never reached

The three stdin-reading gates bound 2 to a stdin read error only, with no arm for stdin closed at zero bytes - so empty input flowed into the detector, found nothing, and exited 1, which each module's own comment defines as a negative verdict. An unset PHASE_SECTION made the UI gate assert the phase has no UI. Empty and whitespace-only input now exit NO_INPUT, and a read error exits UNAVAILABLE rather than a locally-invented 2, both resolved through the registry and delivered by terminateNow.

The exit code was only half of it: under --json the same input emitted {detected:false}, byte-identical to the fabricated payload #3909 exists to fix, and the blocking coverage gate reads that payload. Empty input now emits the in-tree {skipped:true,reason} form with no detected key at all.

teams-status is excluded: it never reads stdin and has no invented 2, so the four-module framing in the issue and ADR is wrong. The dead root bin/lib/ui-safety-gate.cjs is deleted - no installer reference, no workflow invocation, and the live fallback chains are for other modules. Its removal restores the unit tests to the module that actually ships; they had been asserting the stale copy's two-field shape, which is why it drifted unnoticed.

* fix(#3907): drive gate tests through the process seam, and make removed-but-needed basename-precise

CONTRIBUTING requires every subprocess go through tests/helpers/process-seam.cjs; two of the three gate suites hand-rolled spawnSync while the third, added in the same change, used runNode correctly for the identical injection case. Converted the blocks this change added, leaving pre-existing ones alone.

Deleting one of two files sharing a basename made lint-removed-but-needed report 14 references that were all to the surviving canonical module - the false-positive class its own docstring names. It now matches on the deleted file's full path when a surviving file shares its basename, which is more precise rather than weaker: a genuine full-path reference still fails, and behaviour is unchanged when no basename collides. It immediately caught a docstring on this branch that spelled the deleted path.

* test(#3907): update the one existing assertion that pinned the old empty-stdin verdict

A pre-existing test asserted exit 1 on empty stdin - the defect this phase removes - and was missed because the change added new blocks without auditing existing ones pinning the old contract. Audited the rest: the other three status-1 assertions in that file all feed real input and are the genuine-negative controls that must keep returning 1, so exactly one was stale. The retired 2 is gone from the describe's contract comment too.

* chore(#3907): backfill changeset pr number

---------

Co-authored-by: sim <sim@local>

2026-08-27 11:37:31 -04:00

6.5 KiB

Raw Blame History

API Coverage Gate (Full Coverage by Default — Opt Out, Never Opt In)

Reference for the api-coverage gate on the ai-integration capability (#1562). Config key: workflow.api_coverage_gate (default true). Gate point: verify:pre.

The problem this closes

"We integrated the API" too often silently means "we integrated whatever the first use case exercised." Every un-built capability is then an invisible hole, discovered later by a user who reasonably expected it to work. The phase sealed green because its tasks completed — nobody decided the gaps were acceptable, because nobody enumerated them.

This gate makes the API surface visible and decided before the phase can seal. Full coverage is the default starting position; the coverage matrix is the subtraction record. Every gap is an explicit, reasoned opt-out rather than a surprise.

When it fires

The gate runs at verify:pre (before /gsd:verify-work begins UAT). A phase is treated as an external-API integration when either:

a COVERAGE.md matrix is present in the phase directory (the planner produced one at plan:pre), or
the phase scope shows a strong external-API-integration signal (an integration verb and an external-API noun in the same clause, or an explicit <Service> API|SDK|REST|GraphQL surface naming a real service) and no matrix yet exists.

The detector is deliberately fail-closed: it leans toward firing, because a false positive is dismissed by a one-line COVERAGE.md "no external API integration" declaration, whereas a false negative silently lets a real external-API phase past this blocking gate — strictly worse. So it suppresses only prose that is unambiguously not external integration. A bare word like "api" in "the public API of UserController" is ignored (no integration verb + named service); the clause boundary is the whole relationship test, so an integration verb and an API noun in different clauses do not pair. Since #2365 the detector also excludes non-prose spans before matching: fenced code blocks, inline `code` spans, and path-shaped tokens (a first-party src/app/api/profile/route.ts route is a file path, not an external API, while an external host like api.stripe.com/v1 still counts). In the <Service> API surface position it rejects capitalized sentence starters ("The API"), locality/protocol descriptors ("Internal API", "REST API"), compound modifiers ("Resolver-only API"), and first-party-qualified services ("internal Payments API") — a real vendor name is none of these.

The two touch points

Plan time (plan:pre). A contribution to the planner prompts it to run the deterministic detector over the phase scope and, when an integration is detected, produce COVERAGE.md. See capabilities/ai-integration/fragments/api-coverage-plan-pre.md.
Seal time (verify:pre). The blocking api-coverage.verify-pre gate runs check api-coverage.verify-pre <phase-dir> and blocks unless a valid matrix exists (or no integration is detected).

The coverage matrix format

Canonical form — a markdown table (human-editable, diff-friendly):

# API Coverage — <service>

> Full coverage by default. Opt-outs are explicit, reasoned decisions.

| capability | decision | reason |
|---|---|---|
| search | INTEGRATE | |
| playlists | INTEGRATE | |
| skip | OPT-OUT | not needed yet — tracked for follow-up phase |

INTEGRATE is the default. Every capability starts as INTEGRATE.
Every OPT-OUT MUST carry a one-line reason (not needed, not needed yet, explicitly out of scope, …). An opt-out without a reason is an un-decided hole — the exact failure mode this gate exists to close.
A fenced ```coverage JSON block ([{"capability":…,"decision":…,"reason":…}]) is also accepted for machine-generated matrices.

Rules enforced at seal time: the matrix must be non-empty; every capability name must be non-empty and unique; every decision must be INTEGRATE or OPT-OUT; every OPT-OUT must have a reason. Violations block the seal with a precise error.

Declaring "no external API integration" (#2365)

A phase that integrates no external API/SDK/service — but was still asked for a matrix (e.g. the detector over-fired, or a team wants the decision on record) — declares it instead of fabricating a row:

No external API integration: UI-only phase, no third-party surface.

The reason is required, exactly like an OPT-OUT reason — the declaration is a reasoned decision, not a bypass. A COVERAGE.md containing both the declaration and coverage rows is contradictory and blocks the seal. When the detector still finds integration signals in the phase scope, the declaration wins (it is the human overrule for a fallible detector) but the gate output surfaces the overridden signals so the contradiction is visible, not silent.

A second integration against the same need

A second platform for an existing capability (e.g. adding YouTube alongside Spotify for media playback) starts from the same full-coverage baseline as the first. Do not carry over the first integration's opt-outs silently — re-decide each capability for the new surface, so a first-class/fallback asymmetry cannot accumulate into a later user-facing bug.

The matrix persists

COVERAGE.md is a phase artifact. A future phase that extends the same integration starts from the recorded surface and decisions rather than from zero — the matrix is the durable subtraction record.

Tuning

Disable entirely: set workflow.api_coverage_gate: false in .planning/config.json (the gate unregisters from verify:pre).
Widen the trigger vocabulary: the detector accepts --verbs / --nouns overrides (see capabilities/ai-integration/fragments/api-coverage-plan-pre.md). The default vocabulary is additive-only.

Detector CLI

echo "$PHASE_SCOPE" | node gsd-core/bin/lib/api-coverage.cjs --json
# exit 0 = integration detected, 1 = none (real input, examined, no signal)
# NO_INPUT = stdin was empty or whitespace-only (registry code — ADR-3889 Phase 3, #3907)
# UNAVAILABLE = stdin read failed (registry code)
# NO_INPUT/UNAVAILABLE emit {"skipped":true,"reason":"no_input"|"stdin_error"} — no `detected` key

The detector is a pure function (detectApiIntegration → { detected, signals, terms }) shared by the plan-time prompt and the seal-time gate, so the low-false-positive guarantee is testable rather than a judgment call.

6.5 KiB Raw Blame History