diff --git a/.changeset/plucky-hawks-sing.md b/.changeset/plucky-hawks-sing.md new file mode 100644 index 000000000..7bad0cb91 --- /dev/null +++ b/.changeset/plucky-hawks-sing.md @@ -0,0 +1,5 @@ +--- +type: Changed +pr: 3944 +--- +**The API-coverage seal gate no longer clears a phase it never examined** — a phase with no plan body and no roadmap section previously ran the detector over zero bytes and sealed as "no external-API integration"; it is now held with `scope_unavailable`, and the assumption-delta checkpoint reports `skipped` instead of a fabricated `detected:false` when it cannot resolve a phase section. (#3909) diff --git a/capabilities/ai-integration/fragments/api-coverage-plan-pre.md b/capabilities/ai-integration/fragments/api-coverage-plan-pre.md index 2cfd6fb8c..5d3d10b60 100644 --- a/capabilities/ai-integration/fragments/api-coverage-plan-pre.md +++ b/capabilities/ai-integration/fragments/api-coverage-plan-pre.md @@ -22,9 +22,17 @@ scope (the concatenation of this phase's ROADMAP section + the PLAN body): ```bash SCOPE="$(cat "${PHASE_DIR}"/*-PLAN.md 2>/dev/null) $(gsd_run query roadmap.get-phase "${PHASE}" 2>/dev/null || true)" -API_COVERAGE_JSON=$(printf '%s' "$SCOPE" | node gsd-core/bin/lib/api-coverage.cjs --json 2>/dev/null || echo '{"detected":false,"signals":[]}') +API_COVERAGE_JSON=$(printf '%s' "$SCOPE" | node gsd-core/bin/lib/api-coverage.cjs --json 2>/dev/null) || true +[ -n "$API_COVERAGE_JSON" ] || API_COVERAGE_JSON='{"skipped":true,"reason":"probe_unavailable"}' ``` +The `|| true` neutralizes the assignment's status without discarding the +detector's own payload: the detector exits **1** for a real "no integration" +verdict, so treating any non-zero exit as failure would throw away a correct +answer. Emptiness — not exit status — is what proves the probe never ran, and +the second line is the only place the fragment manufactures a payload of its +own — one that records the *absence* of a verdict rather than asserting one. + The detector's exit code and `--json` payload now distinguish a real negative from an unexamined input (ADR-3889 Phase 3, #3907): empty/whitespace-only `$SCOPE` or a stdin read failure emit `{"skipped":true,"reason":"no_input"| diff --git a/capabilities/assumption-delta/fragments/plan-pre.md b/capabilities/assumption-delta/fragments/plan-pre.md index 2864d5c2a..f2cc2c0e2 100644 --- a/capabilities/assumption-delta/fragments/plan-pre.md +++ b/capabilities/assumption-delta/fragments/plan-pre.md @@ -11,10 +11,11 @@ Most quietly-imported architectural debt does not come from a missing upfront de The detector is a deterministic scan over the phase scope text. It strips fenced code blocks first, so a trigger word that appears only inside a code snippet does not fire. It returns a typed result: `{ detected, signals[], terms }`. Resolve it through the `assumption-delta scan` query (same phase-section resolver as `roadmap.get-phase`): ```bash -ASSUMPTION_DELTA_JSON=$(gsd_run query assumption-delta scan "${PHASE}" --json 2>/dev/null || echo '{"detected":false,"signals":[],"terms":{}}') +ASSUMPTION_DELTA_JSON=$(gsd_run query assumption-delta scan "${PHASE}" --json 2>/dev/null) || true +[ -n "$ASSUMPTION_DELTA_JSON" ] || ASSUMPTION_DELTA_JSON='{"skipped":true,"reason":"probe_unavailable"}' ``` -> If the phase section cannot be resolved (no `ROADMAP.md` / unknown phase), the query emits `{ "detected": false, ... }` — the checkpoint does not fire. Do not block on it. +> If the phase section cannot be resolved (no `ROADMAP.md` / unknown phase, or a section with no body), the query emits `{ "skipped": true, "reason": "phase_unresolved" }` — **not** `detected:false`. A probe that never had input does not get to assert that this phase changes no core assumption. The checkpoint does not fire either way; the difference is that a skip is now distinguishable from a real negative. Do not block on it. > > Optional tuning — pass `--terms ` to replace the curated pluralization cues for this project (the `optional`/`chosen` cues keep their defaults): `gsd_run query assumption-delta scan "${PHASE}" --json --terms second,alternative,fallback`. @@ -22,6 +23,8 @@ ASSUMPTION_DELTA_JSON=$(gsd_run query assumption-delta scan "${PHASE}" --json 2> Read `ASSUMPTION_DELTA_JSON`. Act on `detected` only — do **not** pattern-match the human prose. +**If `skipped` is `true`:** the detector never examined a phase section — it could not resolve one (`phase_unresolved`) or could not run at all (`probe_unavailable`). Skip the checkpoint for this run rather than asserting a verdict about input that was never examined; do not raise it with the user. **Check for `skipped` before reading `detected`** — a skipped payload carries no `detected` key, and treating its absence as `false` re-creates the fabrication this branch exists to prevent. + **If `detected` is `false`:** this phase does not change a core assumption. Skip the checkpoint entirely and continue planning. Do not raise it with the user. **If `detected` is `true`:** a core assumption may have lost its monopoly. The `signals[]` array tells you which family fired: diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 51c452220..d427a9cc4 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -395,7 +395,7 @@ All workflow toggles follow the **absent = enabled** pattern. If a key is missin | `workflow.nyquist_validation` | boolean | `true` | Test coverage mapping during plan-phase research | | `workflow.ui_phase` | boolean | `true` | Generate UI design contracts for frontend phases | | `workflow.ui_safety_gate` | boolean | `true` | Prompt to run /gsd-ui-phase for frontend phases during plan-phase | -| `workflow.assumption_delta` | boolean | `true` | Advisory architecture checkpoint during planning. When a phase makes something **plural, optional, or chosen** that used to be **singular, required, or derived** (e.g. a second auth method, a required field becoming optional, a constant becoming a parameter), the planner is prompted to re-ask whether the primary key / identity model still names the right thing (promote the new general representation vs. add it alongside). Non-blocking; fires only on a detected signal. Bare "or" is intentionally excluded (prose false-positives). Inspect a phase with `gsd_run query assumption-delta scan `. Added in #1561 | +| `workflow.assumption_delta` | boolean | `true` | Advisory architecture checkpoint during planning. When a phase makes something **plural, optional, or chosen** that used to be **singular, required, or derived** (e.g. a second auth method, a required field becoming optional, a constant becoming a parameter), the planner is prompted to re-ask whether the primary key / identity model still names the right thing (promote the new general representation vs. add it alongside). Non-blocking; fires only on a detected signal. Bare "or" is intentionally excluded (prose false-positives). Inspect a phase with `gsd_run query assumption-delta scan `. Added in #1561. A phase section that cannot be resolved returns `{"skipped":true,"reason":"phase_unresolved"}` rather than a fabricated `detected:false` (#3909) | | `workflow.ui_review` | boolean | `true` | Run visual quality audit (`/gsd-ui-review`) after phase execution in autonomous mode. When `false`, the UI audit step is skipped. | | `workflow.live_dom_uat` | boolean | `false` | **Default-off.** Enable live-DOM verification (#2856). When `true`, a `gsd-dom-verifier` step runs after each execution wave and writes `{phase}-DOM-VERIFY.md`, and the orchestrator's automated UI verification will additionally consider `mcp__chrome-devtools__*` / `mcp__claude-in-chrome__*` when present. Browser reach is confined to `gsd-dom-verifier` — `gsd-executor`'s tool surface is unchanged in every configuration. Presence of a browser MCP server is **not** sufficient on its own: a server configured for unrelated work is never driven unless this key is on. The pre-existing `mcp__playwright__*` path is unaffected by this key. Note `chrome-devtools-mcp` holds an exclusive browser-profile lock, so concurrent waves need `--isolated` on **your** MCP server registration — GSD cannot pass it. See [Enable live-DOM verification](how-to/enable-live-dom-verification.md). | | `workflow.node_repair` | boolean | `true` | Autonomous task repair on verification failure | @@ -428,7 +428,7 @@ All workflow toggles follow the **absent = enabled** pattern. If a key is missin | `workflow.cross_ai_timeout` | number | `300` | Timeout in seconds for cross-AI execution commands. Prevents runaway external processes. Added in v1.36 | | `workflow.test_gate_timeout` | number | `600` | Wall-clock timeout (seconds) for a verification test gate; a watch-mode runner (vitest/jest) that never exits is aborted after this budget instead of hanging the orchestrator (#1857) | | `workflow.ai_integration_phase` | boolean | `true` | Enable the `/gsd-ai-integration-phase` command. When `false`, the command exits with a configuration gate message | -| `workflow.api_coverage_gate` | boolean | `true` | Require an explicit API-coverage decision before a phase that integrates an external API/SDK/service can seal. At `plan:pre` the planner is prompted to produce a `COVERAGE.md` matrix (full coverage by default, every opt-out reasoned); at `verify:pre` a blocking gate fails the seal unless the matrix is complete. Independent of `ai_integration_phase` (#1562) | +| `workflow.api_coverage_gate` | boolean | `true` | Require an explicit API-coverage decision before a phase that integrates an external API/SDK/service can seal. At `plan:pre` the planner is prompted to produce a `COVERAGE.md` matrix (full coverage by default, every opt-out reasoned); at `verify:pre` a blocking gate fails the seal unless the matrix is complete. Independent of `ai_integration_phase` (#1562). A phase whose scope cannot be established at all (no plan body and no roadmap section) is held rather than passed, reporting `scope_unavailable` — see [Resolve a skipped capability probe](how-to/resolve-a-skipped-capability-probe.md) (#3909) | | `workflow.auto_prune_state` | boolean | `false` | When `true`, automatically prune stale entries from STATE.md at phase boundaries instead of prompting | | `workflow.pattern_mapper` | boolean | `true` | Run the `gsd-pattern-mapper` agent between research and planning to map new files to existing codebase analogs | | `workflow.subagent_timeout` | number | `300000` | Timeout in milliseconds for parallel subagent tasks (e.g. codebase mapping). Increase for large codebases or slower models. Default: 300000 (5 minutes) | diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 5f31a4afa..42258c28e 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -3412,6 +3412,12 @@ The load-bearing wire is the `plan-phase` lift into `must_haves.prohibitions`, s **Purpose:** A phase that integrates an external API, SDK, or service can no longer seal verification without a decided coverage matrix (#1562). +**Behavior:** At seal time the gate reads the phase scope — the plan bodies, falling back to this phase's ROADMAP section — and runs the deterministic detector over it. An integration signal without a `COVERAGE.md` matrix blocks the seal; no signal passes. + +**Unestablished scope is not a negative verdict (#3909).** A phase with no plan body *and* no roadmap section gives the detector nothing to examine. The gate used to run detection over zero bytes and pass, certifying "no external-API integration" from a probe that never looked. It now holds the seal instead, reporting `scope_unavailable: true`. A phase whose plans are real and simply contain no API vocabulary is unaffected — the discriminator is *bytes examined*, never *signals found*. + +**Breaking change:** a phase that previously sealed because its detector could not establish a scope is now correctly held. Add the phase plan, or record a reasoned `No external API integration: ` declaration in `COVERAGE.md`. See [Resolve a skipped capability probe](how-to/resolve-a-skipped-capability-probe.md). + --- ### 157. State Rebuild & Configurable Graph Path diff --git a/docs/INVENTORY.md b/docs/INVENTORY.md index 2d1c4768e..89f42a921 100644 --- a/docs/INVENTORY.md +++ b/docs/INVENTORY.md @@ -375,7 +375,7 @@ Full roster at `gsd-core/references/*.md`. References are shared knowledge docum | `autonomous-ui-design-contract.md` | Autonomous-mode step 3a.5 — resolve whether a frontend phase needs a UI-SPEC.md and generate one through active `plan:pre` hooks; always non-blocking. | | `ios-scaffold.md` | iOS application scaffolding patterns. | | `ai-evals.md` | AI evaluation design reference for `/gsd-ai-integration-phase`. | -| `api-coverage.md` | API-coverage gate reference (full-coverage-by-default) for the `ai-integration` capability's `verify:pre` blocking gate (#1562) — matrix format, trigger, tuning, detector CLI. | +| `api-coverage.md` | API-coverage gate reference (full-coverage-by-default) for the `ai-integration` capability's `verify:pre` blocking gate (#1562) — matrix format, trigger, tuning, detector CLI, and the seal-time outcome table naming every pass/block arm including `scope_unavailable` (#3909). | | `ai-frameworks.md` | AI framework decision-matrix reference for `gsd-framework-selector`. | | `executor-examples.md` | Worked examples for the gsd-executor agent. | | `doc-conflict-engine.md` | Shared conflict-detection contract for ingest/import workflows. | diff --git a/docs/README.md b/docs/README.md index b00cd448c..b96624c24 100644 --- a/docs/README.md +++ b/docs/README.md @@ -30,6 +30,7 @@ Language versions: [English](README.md) · [Português (pt-BR)](pt-BR/README.md) - [State a failing direction](how-to/state-a-failing-direction.md) — say what output constitutes failure for an `` verify command, and migrate a phase planned before the rule - [Resolve a contract-drift finding](how-to/resolve-contract-drift-findings.md) — bring an agent's completion contract, read-tag gate, or deleted-file test reference back into agreement with the registry - [Resolve unreachable-guard findings](how-to/resolve-unreachable-guard-findings.md) — fix shell guards whose fallback arm cannot run, and tell "nothing to report" apart from "could not look" +- [Resolve a skipped capability probe](how-to/resolve-a-skipped-capability-probe.md) — act on a coverage gate that held your phase for an unestablished scope, or a planning checkpoint that reported `skipped` instead of a verdict - [Diagnose which gsd-tools is running](how-to/diagnose-a-foreign-gsd-tools.md) — tell this package's tool apart from the predecessor's colliding binary and from a gsd-core too old to identify itself - [Resolve an ESLint glob-coverage finding](how-to/resolve-eslint-coverage-findings.md) — bring a source file that matches no lint rule under coverage, or record a reasoned exemption - [Read the statusline freshness marker](how-to/read-the-statusline-freshness-marker.md) — turn on `state ~N commits back`, and tell "STATE.md is fresh" apart from "freshness could not be established" diff --git a/docs/features/api-coverage-gate.md b/docs/features/api-coverage-gate.md index 19631e68d..99bee272b 100644 --- a/docs/features/api-coverage-gate.md +++ b/docs/features/api-coverage-gate.md @@ -7,3 +7,9 @@ group: v1.7.0 Features **Command:** `/gsd-verify-work` **Purpose:** A phase that integrates an external API, SDK, or service can no longer seal verification without a decided coverage matrix (#1562). + +**Behavior:** At seal time the gate reads the phase scope — the plan bodies, falling back to this phase's ROADMAP section — and runs the deterministic detector over it. An integration signal without a `COVERAGE.md` matrix blocks the seal; no signal passes. + +**Unestablished scope is not a negative verdict (#3909).** A phase with no plan body *and* no roadmap section gives the detector nothing to examine. The gate used to run detection over zero bytes and pass, certifying "no external-API integration" from a probe that never looked. It now holds the seal instead, reporting `scope_unavailable: true`. A phase whose plans are real and simply contain no API vocabulary is unaffected — the discriminator is *bytes examined*, never *signals found*. + +**Breaking change:** a phase that previously sealed because its detector could not establish a scope is now correctly held. Add the phase plan, or record a reasoned `No external API integration: ` declaration in `COVERAGE.md`. See [Resolve a skipped capability probe](how-to/resolve-a-skipped-capability-probe.md). diff --git a/docs/how-to/resolve-a-skipped-capability-probe.md b/docs/how-to/resolve-a-skipped-capability-probe.md new file mode 100644 index 000000000..dbc195a36 --- /dev/null +++ b/docs/how-to/resolve-a-skipped-capability-probe.md @@ -0,0 +1,116 @@ +# Resolve a skipped capability probe + +A phase-scope probe — the API-coverage detector or the assumption-delta +detector — needs real text to examine before it can assert a verdict. When it +gets none, it says so instead of guessing. This page covers reading that +signal and clearing it. + +## What you saw + +One of two things, depending on which surface you hit: + +- **The seal gate held your phase.** `verify:pre` reported a block with + `scope_unavailable: true` instead of the usual "no external-API integration + detected" pass. +- **A planning checkpoint reported `skipped` instead of a verdict.** The + `assumption-delta` or `api-coverage` plan-time checkpoint printed + `{"skipped":true,"reason":"..."}` and produced no decision either way. + +Both are the same underlying fix (#3909): a probe that never examined real +input used to fabricate `detected:false`, which reads as "nothing here" when +the true answer is "nobody looked." Now it says which one happened. + +## The reason-code table + +| `reason` | What it means | What actually happened | Remedy | +|---|---|---|---| +| `scope_unavailable` | The seal-time gate found no phase scope at all | No plan body (`.planning/phases//*-PLAN.md`) *and* no ROADMAP section for the phase | Add the plan or roadmap section, or write a reasoned `No external API integration: ` declaration to `COVERAGE.md` | +| `phase_unresolved` | The `assumption-delta scan` query could not resolve a phase section to scan | No `ROADMAP.md`, an unknown phase number, or a phase section with no body | If you expected a real scan, fix the phase reference or roadmap section; otherwise no action — the checkpoint correctly stays silent | +| `probe_unavailable` | The detector process itself produced no output | The probe crashed, was not found, or its stdout was empty for a reason unrelated to input content | Check that `gsd-core/bin/lib/api-coverage.cjs` or `gsd-core/bin/lib/assumption-delta.cjs` runs standalone; re-run the checkpoint once the probe itself is healthy | +| `no_input` | The detector ran but stdin was empty or whitespace-only | The phase scope resolved to nothing (empty plan body and empty roadmap section) | Same as `scope_unavailable` — give the detector something to read | +| `stdin_error` | The detector could not read stdin at all | A pipe/read failure upstream of the detector, not an empty-input case | Re-run; if it recurs, the caller constructing `$SCOPE` is the thing to fix, not the detector | + +## The seal gate held my phase + +1. Confirm the reason directly: + + ```bash + gsd_run check api-coverage.verify-pre --raw + ``` + + Look for `"scope_unavailable": true` in the output. That confirms this is + the fail-closed arm, not a real "integration detected" block. + +2. Check whether the phase actually has scope to read: + + ```bash + ls .planning/phases//*-PLAN.md 2>/dev/null + gsd_run query roadmap.get-phase + ``` + + If both come back empty, the gate is correct — there is genuinely nothing + for the detector to examine. + +3. Resolve it one of two ways: + - **Add the missing scope.** Write the phase plan, or add the phase's + section to `ROADMAP.md`, then re-run detection. + - **Record the reasoned declaration anyway.** If the phase truly has no + plan body worth writing (rare), put the human decision directly in + `COVERAGE.md`: + + ```markdown + No external API integration: . + ``` + + This is the same reasoned overrule the gate already accepts when a + detector *does* find (or falsely flags) a signal — see + [`gsd-core/references/api-coverage.md`](../../gsd-core/references/api-coverage.md#declaring-no-external-api-integration-2365). + +4. Re-run the gate to confirm it clears: + + ```bash + gsd_run check api-coverage.verify-pre --raw + ``` + +**Turning the gate off does not answer the question.** Setting +`workflow.api_coverage_gate: false` in `.planning/config.json` silences the +hold, but the phase's API surface is still undecided — you have just stopped +being told. Prefer resolving the scope or writing the declaration. + +## A checkpoint reported skipped + +The `assumption-delta` and `api-coverage` plan-time checkpoints are advisory: +when they cannot resolve a phase section, they skip rather than fire, and that +is correct, non-blocking behavior. You do not need to do anything except +notice that the phase was not actually cleared by a real scan — a `skipped` +payload carries no `detected` key and is not the same as "no signal found." +If you expected a real scan and got a skip instead, treat it like the +`phase_unresolved` row above: check that the phase reference and roadmap +section actually exist. + +## Why this is not just a stricter gate + +A probe reporting `detected:false` from an input it never read is a false +negative on a *blocking* gate — the one direction a gate must never fail +silently. A false positive here costs one line in `COVERAGE.md`; a false +negative lets a real external-API integration seal with an undecided surface, +discovered later by a user who reasonably expected it to work. This change can +turn a false green red. It can never turn a red green — every phase that +passed because a signal was genuinely absent from scope the detector actually +read still passes, unchanged. + +## Telling "nothing to report" apart from "could not look" + +This is the recurring distinction across this doc set (see also +[Resolve unreachable-guard findings](resolve-unreachable-guard-findings.md) and +[Consume the planning snapshot](consume-the-planning-snapshot.md)). A verdict +of `detected: false` or `passed: true` means the detector examined real text +and found nothing. A `skipped` payload or a `scope_unavailable` block means +the detector examined nothing at all. Only the first is good news; the second +is a request for more scope, not a clean bill of health. + +## Related + +- [`gsd-core/references/api-coverage.md`](../../gsd-core/references/api-coverage.md) — the full API-coverage gate reference, including the seal-time outcome table +- [Resolve unreachable-guard findings](resolve-unreachable-guard-findings.md) — the same "nothing to report vs. could not look" distinction, one layer down +- [Consume the planning snapshot](consume-the-planning-snapshot.md) — the `scope` field's version of this same rule diff --git a/gsd-core/bin/gsd-tools.cjs b/gsd-core/bin/gsd-tools.cjs index 058c101f4..8e73ee6a0 100755 --- a/gsd-core/bin/gsd-tools.cjs +++ b/gsd-core/bin/gsd-tools.cjs @@ -2419,8 +2419,19 @@ function dispatchOverlayCapabilityCommand({ command, args, cwd, raw, error, load .filter((t) => t.length > 0); termsOverride = list.length > 0 ? { pluralization: list } : undefined; } + // An unresolvable phase section is not a negative verdict. Feeding + // `''` to the detector reported "examined, found nothing" for a + // probe that never had input — no ROADMAP.md, or a phase number + // absent from it, both read as a confident `detected:false` + // (ADR-3889 failure class (c), #3909). Exit stays 0: this is an + // ADR-2980 degraded result carried in the payload, and ADR-3889 P8 + // pins the gsd-tools exit projection at v1. const section = roadmap.getRoadmapPhaseWithFallback(cwd, phaseNum); - const result = detectAssumptionDelta(section ?? '', termsOverride); + if (typeof section !== 'string' || section.trim() === '') { + output({ skipped: true, reason: 'phase_unresolved' }, raw); + return; + } + const result = detectAssumptionDelta(section, termsOverride); output(result, raw); return; } diff --git a/gsd-core/bin/lib/capability-registry.cjs b/gsd-core/bin/lib/capability-registry.cjs index 69b9debbe..9eb399dda 100644 --- a/gsd-core/bin/lib/capability-registry.cjs +++ b/gsd-core/bin/lib/capability-registry.cjs @@ -68,7 +68,7 @@ const capabilities = { "into": "planner", "fragment": { "path": "fragments/api-coverage-plan-pre.md", - "inline": "# API Coverage Decision Checkpoint\n\n> Full API Coverage by Default — Opt Out, Never Opt In. Fires when a phase\n> integrates an external API / SDK / service. Most non-API phases will not fire\n> it — that is the point.\n\n## Why this exists\n\n\"We integrated the API\" too often silently means \"we integrated whatever the\nfirst use case exercised.\" Every un-built capability is then an invisible hole,\ndiscovered later by a user who reasonably expected it to work. The phase sealed\ngreen because its tasks completed; nobody decided the gaps were acceptable,\nbecause nobody enumerated them. This checkpoint makes the surface **visible and\ndecided** before the phase can seal.\n\n## Detect whether this phase integrates an external API\n\nThe detector is a deterministic scan over the phase scope. It strips fenced\ncode blocks first, so a trigger term inside a code snippet does not fire. It\nreturns a typed result: `{ detected, signals[], terms }`. Run it on the phase\nscope (the concatenation of this phase's ROADMAP section + the PLAN body):\n\n```bash\nSCOPE=\"$(cat \"${PHASE_DIR}\"/*-PLAN.md 2>/dev/null) $(gsd_run query roadmap.get-phase \"${PHASE}\" 2>/dev/null || true)\"\nAPI_COVERAGE_JSON=$(printf '%s' \"$SCOPE\" | node gsd-core/bin/lib/api-coverage.cjs --json 2>/dev/null || echo '{\"detected\":false,\"signals\":[]}')\n```\n\nThe detector's exit code and `--json` payload now distinguish a real negative\nfrom an unexamined input (ADR-3889 Phase 3, #3907): empty/whitespace-only\n`$SCOPE` or a stdin read failure emit `{\"skipped\":true,\"reason\":\"no_input\"|\n\"stdin_error\"}` — no `detected` key at all. **Check for `skipped` before\nreading `detected`**: a `skipped` payload is not a confirmed \"no API\nintegration\" verdict, it means the detector never examined real input. Do not\ntreat it as `detected:false`. Read `API_COVERAGE_JSON.detected` only when\n`skipped` is absent — act on it only, do **not** pattern-match the prose\nyourself.\n\n**If `skipped` is `true`:** the detector could not establish a scope (empty\n`$SCOPE`) or failed to run (stdin read error). Skip the checkpoint for this\nrun rather than asserting a verdict about input that was never examined; do\nnot raise it with the user.\n\n**If `detected` is `false`:** this phase does not integrate an external API. Skip\nthe checkpoint entirely and continue planning. Do not raise it with the user.\n\n**If `detected` is `true`:** an external-API integration is in scope. You MUST\nproduce a **coverage matrix** before the plan is finalized.\n\n**If `detected` is `true` but the phase genuinely integrates no external API**\n(the detector is deterministic, not infallible — confirm by re-reading the phase\nscope, not by preference): do NOT fabricate a matrix row for a capability that\ndoes not exist. Write a reasoned declaration to `${PHASE_DIR}/COVERAGE.md`\ninstead:\n\n```markdown\nNo external API integration: .\n```\n\nThe reason is required, exactly like an `OPT-OUT` reason. The seal-time gate\naccepts this declaration in place of a matrix.\n\n## Produce the coverage matrix\n\nEnumerate the external API's full **capability surface** — the verb/endpoint/method\nlist (e.g. for a music service: `search`, `play`, `pause`, `skip`, `set_volume`,\n`get_playlist`, `create_playlist`, `add_to_playlist`, …). For each capability\nrecord a decision, starting from **full coverage** as the default:\n\n| capability | decision | reason |\n|---|---|---|\n| `` | `INTEGRATE` \\| `OPT-OUT` | `` |\n\nRules:\n\n- **`INTEGRATE` is the default.** Every capability starts as INTEGRATE; the\n matrix is the *subtraction record*.\n- **Every `OPT-OUT` MUST carry a one-line reason** (`not needed`, `not needed\n yet`, `explicitly out of scope`, …). An opt-out without a reason is an\n un-decided hole — the exact failure mode this gate exists to close.\n- **A second integration against the same need** (e.g. a second platform for the\n same capability) starts from the **same full-coverage baseline** as the first.\n Do not carry over the first integration's opt-outs silently — re-decide each\n capability for the new surface, so a first-class/fallback asymmetry cannot\n accumulate.\n\nWrite the matrix to `${PHASE_DIR}/COVERAGE.md` (canonical markdown-table form):\n\n```markdown\n# API Coverage — \n\n> Full coverage by default. Opt-outs are explicit, reasoned decisions.\n\n| capability | decision | reason |\n|---|---|---|\n| search | INTEGRATE | |\n| playlists | INTEGRATE | |\n| skip | OPT-OUT | not needed yet — tracked for follow-up phase |\n```\n\nA fenced ` ```coverage ` JSON block is also accepted for machine-generated\nmatrices; the markdown table is preferred (human-editable, diff-friendly).\n\n## The seal-time gate\n\nThis checkpoint is enforced. At `verify:pre` the `api-coverage.verify-pre` gate\nruns `check api-coverage.verify-pre `:\n\n- If `COVERAGE.md` exists, it is validated — every row needs a valid decision and\n every `OPT-OUT` a reason. A malformed/partial matrix **blocks the seal**. A\n reasoned `No external API integration: …` declaration (and no rows) passes.\n- If `COVERAGE.md` is absent, the detector runs again over the phase scope. If a\n strong external-API-integration signal is found, the seal is **blocked** until a\n matrix is produced. If no signal is found, the phase is treated as a non-API\n phase and the seal proceeds.\n\nSo: an API-integrating phase cannot seal without a decided matrix. Produce it at\nplan time; do not leave it for seal time.\n\n## Tuning the vocabulary (optional)\n\nThe trigger vocabulary is a curated, additive-only set in\n`gsd-core/bin/lib/api-coverage.cjs` (`DEFAULT_API_COVERAGE_TERMS`). To widen it\nfor a project, override at the call site:\n\n```bash\nprintf '%s' \"$SCOPE\" | node gsd-core/bin/lib/api-coverage.cjs --json \\\n --verbs integrate,wrap,connect,embed --nouns api,sdk,rest,grpc,webhook,plugin\n```\n\nThe whole checkpoint is toggleable via `workflow.api_coverage_gate` in\n`.planning/config.json`.\n" + "inline": "# API Coverage Decision Checkpoint\n\n> Full API Coverage by Default — Opt Out, Never Opt In. Fires when a phase\n> integrates an external API / SDK / service. Most non-API phases will not fire\n> it — that is the point.\n\n## Why this exists\n\n\"We integrated the API\" too often silently means \"we integrated whatever the\nfirst use case exercised.\" Every un-built capability is then an invisible hole,\ndiscovered later by a user who reasonably expected it to work. The phase sealed\ngreen because its tasks completed; nobody decided the gaps were acceptable,\nbecause nobody enumerated them. This checkpoint makes the surface **visible and\ndecided** before the phase can seal.\n\n## Detect whether this phase integrates an external API\n\nThe detector is a deterministic scan over the phase scope. It strips fenced\ncode blocks first, so a trigger term inside a code snippet does not fire. It\nreturns a typed result: `{ detected, signals[], terms }`. Run it on the phase\nscope (the concatenation of this phase's ROADMAP section + the PLAN body):\n\n```bash\nSCOPE=\"$(cat \"${PHASE_DIR}\"/*-PLAN.md 2>/dev/null) $(gsd_run query roadmap.get-phase \"${PHASE}\" 2>/dev/null || true)\"\nAPI_COVERAGE_JSON=$(printf '%s' \"$SCOPE\" | node gsd-core/bin/lib/api-coverage.cjs --json 2>/dev/null) || true\n[ -n \"$API_COVERAGE_JSON\" ] || API_COVERAGE_JSON='{\"skipped\":true,\"reason\":\"probe_unavailable\"}'\n```\n\nThe `|| true` neutralizes the assignment's status without discarding the\ndetector's own payload: the detector exits **1** for a real \"no integration\"\nverdict, so treating any non-zero exit as failure would throw away a correct\nanswer. Emptiness — not exit status — is what proves the probe never ran, and\nthe second line is the only place the fragment manufactures a payload of its\nown — one that records the *absence* of a verdict rather than asserting one.\n\nThe detector's exit code and `--json` payload now distinguish a real negative\nfrom an unexamined input (ADR-3889 Phase 3, #3907): empty/whitespace-only\n`$SCOPE` or a stdin read failure emit `{\"skipped\":true,\"reason\":\"no_input\"|\n\"stdin_error\"}` — no `detected` key at all. **Check for `skipped` before\nreading `detected`**: a `skipped` payload is not a confirmed \"no API\nintegration\" verdict, it means the detector never examined real input. Do not\ntreat it as `detected:false`. Read `API_COVERAGE_JSON.detected` only when\n`skipped` is absent — act on it only, do **not** pattern-match the prose\nyourself.\n\n**If `skipped` is `true`:** the detector could not establish a scope (empty\n`$SCOPE`) or failed to run (stdin read error). Skip the checkpoint for this\nrun rather than asserting a verdict about input that was never examined; do\nnot raise it with the user.\n\n**If `detected` is `false`:** this phase does not integrate an external API. Skip\nthe checkpoint entirely and continue planning. Do not raise it with the user.\n\n**If `detected` is `true`:** an external-API integration is in scope. You MUST\nproduce a **coverage matrix** before the plan is finalized.\n\n**If `detected` is `true` but the phase genuinely integrates no external API**\n(the detector is deterministic, not infallible — confirm by re-reading the phase\nscope, not by preference): do NOT fabricate a matrix row for a capability that\ndoes not exist. Write a reasoned declaration to `${PHASE_DIR}/COVERAGE.md`\ninstead:\n\n```markdown\nNo external API integration: .\n```\n\nThe reason is required, exactly like an `OPT-OUT` reason. The seal-time gate\naccepts this declaration in place of a matrix.\n\n## Produce the coverage matrix\n\nEnumerate the external API's full **capability surface** — the verb/endpoint/method\nlist (e.g. for a music service: `search`, `play`, `pause`, `skip`, `set_volume`,\n`get_playlist`, `create_playlist`, `add_to_playlist`, …). For each capability\nrecord a decision, starting from **full coverage** as the default:\n\n| capability | decision | reason |\n|---|---|---|\n| `` | `INTEGRATE` \\| `OPT-OUT` | `` |\n\nRules:\n\n- **`INTEGRATE` is the default.** Every capability starts as INTEGRATE; the\n matrix is the *subtraction record*.\n- **Every `OPT-OUT` MUST carry a one-line reason** (`not needed`, `not needed\n yet`, `explicitly out of scope`, …). An opt-out without a reason is an\n un-decided hole — the exact failure mode this gate exists to close.\n- **A second integration against the same need** (e.g. a second platform for the\n same capability) starts from the **same full-coverage baseline** as the first.\n Do not carry over the first integration's opt-outs silently — re-decide each\n capability for the new surface, so a first-class/fallback asymmetry cannot\n accumulate.\n\nWrite the matrix to `${PHASE_DIR}/COVERAGE.md` (canonical markdown-table form):\n\n```markdown\n# API Coverage — \n\n> Full coverage by default. Opt-outs are explicit, reasoned decisions.\n\n| capability | decision | reason |\n|---|---|---|\n| search | INTEGRATE | |\n| playlists | INTEGRATE | |\n| skip | OPT-OUT | not needed yet — tracked for follow-up phase |\n```\n\nA fenced ` ```coverage ` JSON block is also accepted for machine-generated\nmatrices; the markdown table is preferred (human-editable, diff-friendly).\n\n## The seal-time gate\n\nThis checkpoint is enforced. At `verify:pre` the `api-coverage.verify-pre` gate\nruns `check api-coverage.verify-pre `:\n\n- If `COVERAGE.md` exists, it is validated — every row needs a valid decision and\n every `OPT-OUT` a reason. A malformed/partial matrix **blocks the seal**. A\n reasoned `No external API integration: …` declaration (and no rows) passes.\n- If `COVERAGE.md` is absent, the detector runs again over the phase scope. If a\n strong external-API-integration signal is found, the seal is **blocked** until a\n matrix is produced. If no signal is found, the phase is treated as a non-API\n phase and the seal proceeds.\n\nSo: an API-integrating phase cannot seal without a decided matrix. Produce it at\nplan time; do not leave it for seal time.\n\n## Tuning the vocabulary (optional)\n\nThe trigger vocabulary is a curated, additive-only set in\n`gsd-core/bin/lib/api-coverage.cjs` (`DEFAULT_API_COVERAGE_TERMS`). To widen it\nfor a project, override at the call site:\n\n```bash\nprintf '%s' \"$SCOPE\" | node gsd-core/bin/lib/api-coverage.cjs --json \\\n --verbs integrate,wrap,connect,embed --nouns api,sdk,rest,grpc,webhook,plugin\n```\n\nThe whole checkpoint is toggleable via `workflow.api_coverage_gate` in\n`.planning/config.json`.\n" }, "produces": [ "COVERAGE.md" @@ -280,7 +280,7 @@ const capabilities = { "into": "planner", "fragment": { "path": "fragments/plan-pre.md", - "inline": "# Assumption-Delta Architecture Checkpoint\n\n> Advisory, non-blocking. Fires **only** when the phase scope shows a singular→plural / required→optional / derived→chosen transition. When it fires, it surfaces ONE identity-model question before the plan is finalized. Most phases will not fire it — that is the point.\n\n## Why this exists\n\nMost quietly-imported architectural debt does not come from a missing upfront design phase. It comes at the *seam*: a later phase introduces a second case (a second platform, auth method, tenant, region, source of truth) and nobody re-asks whether the original abstraction still names the right thing. The phase that adds the second case is exactly the 20-minute conversation that prevents an afternoon of later cleanup.\n\n## Run the detector\n\nThe detector is a deterministic scan over the phase scope text. It strips fenced code blocks first, so a trigger word that appears only inside a code snippet does not fire. It returns a typed result: `{ detected, signals[], terms }`. Resolve it through the `assumption-delta scan` query (same phase-section resolver as `roadmap.get-phase`):\n\n```bash\nASSUMPTION_DELTA_JSON=$(gsd_run query assumption-delta scan \"${PHASE}\" --json 2>/dev/null || echo '{\"detected\":false,\"signals\":[],\"terms\":{}}')\n```\n\n> If the phase section cannot be resolved (no `ROADMAP.md` / unknown phase), the query emits `{ \"detected\": false, ... }` — the checkpoint does not fire. Do not block on it.\n>\n> Optional tuning — pass `--terms ` to replace the curated pluralization cues for this project (the `optional`/`chosen` cues keep their defaults): `gsd_run query assumption-delta scan \"${PHASE}\" --json --terms second,alternative,fallback`.\n\n## Decision branch\n\nRead `ASSUMPTION_DELTA_JSON`. Act on `detected` only — do **not** pattern-match the human prose.\n\n**If `detected` is `false`:** this phase does not change a core assumption. Skip the checkpoint entirely and continue planning. Do not raise it with the user.\n\n**If `detected` is `true`:** a core assumption may have lost its monopoly. The `signals[]` array tells you which family fired:\n\n| `kind` | What changed | The question to answer |\n|---|---|---|\n| `pluralization` | A second X was introduced where there was one (second platform / auth method / tenant / region / source of truth) | Does the current primary key / identity model still name the right noun? |\n| `optional` | A required / `only` field became optional | Is the field still the right anchor, or has the anchor moved? |\n| `chosen` | A derived value became chosen, or a constant became a parameter | Has a configuration decision become a modeling decision? |\n\nBefore finalizing the plan, answer this for the user and record the decision explicitly:\n\n> **Promote vs. add-alongside.** The usual correct move when a generalization occurs is to **promote** the new general representation to the primary and **demote** the old specific one to a detail of one variant — *not* to add the new one alongside the still-required old one. Adding alongside silently contradicts the generalized intent (a later variant that does not fit the old primary can be stored but never confirmed as a default).\n\nRecord the outcome in the PLAN.md front matter / a `` block:\n\n- The **noun** that is now primary (the generalized identity).\n- The **decision**: `promote` | `add-alongside` | `no-change`, with a one-line rationale.\n- If `add-alongside`: call it out as accepted debt and note what would force a later promote.\n\n## Optional companion: an invariant test\n\nWhen `detected` is `true`, suggest (do not require) a contract/invariant test that encodes the now-generalized intent — e.g. *\"every confirmed default round-trips through the primary use-path, for every supported variant.\"* That test goes red the instant a future phase reintroduces the singular assumption, so the regression cannot land silently. If the user accepts, add the test as a task in the plan.\n\n## Tuning the vocabulary (optional)\n\nThe trigger vocabulary is a curated, additive-only set in `gsd-core/bin/lib/assumption-delta.cjs` (`DEFAULT_ASSUMPTION_DELTA_TERMS`). Bare \"or\" is intentionally excluded — it is too common in prose and would make the gate fire constantly. To widen or narrow the cues for a project, override at the call site with `--terms ` (replaces the pluralization cues; `optional`/`chosen` keep defaults). The whole checkpoint is toggleable via `workflow.assumption_delta` in `.planning/config.json`.\n\nThis checkpoint is advisory: it informs and records; it never blocks the phase.\n" + "inline": "# Assumption-Delta Architecture Checkpoint\n\n> Advisory, non-blocking. Fires **only** when the phase scope shows a singular→plural / required→optional / derived→chosen transition. When it fires, it surfaces ONE identity-model question before the plan is finalized. Most phases will not fire it — that is the point.\n\n## Why this exists\n\nMost quietly-imported architectural debt does not come from a missing upfront design phase. It comes at the *seam*: a later phase introduces a second case (a second platform, auth method, tenant, region, source of truth) and nobody re-asks whether the original abstraction still names the right thing. The phase that adds the second case is exactly the 20-minute conversation that prevents an afternoon of later cleanup.\n\n## Run the detector\n\nThe detector is a deterministic scan over the phase scope text. It strips fenced code blocks first, so a trigger word that appears only inside a code snippet does not fire. It returns a typed result: `{ detected, signals[], terms }`. Resolve it through the `assumption-delta scan` query (same phase-section resolver as `roadmap.get-phase`):\n\n```bash\nASSUMPTION_DELTA_JSON=$(gsd_run query assumption-delta scan \"${PHASE}\" --json 2>/dev/null) || true\n[ -n \"$ASSUMPTION_DELTA_JSON\" ] || ASSUMPTION_DELTA_JSON='{\"skipped\":true,\"reason\":\"probe_unavailable\"}'\n```\n\n> If the phase section cannot be resolved (no `ROADMAP.md` / unknown phase, or a section with no body), the query emits `{ \"skipped\": true, \"reason\": \"phase_unresolved\" }` — **not** `detected:false`. A probe that never had input does not get to assert that this phase changes no core assumption. The checkpoint does not fire either way; the difference is that a skip is now distinguishable from a real negative. Do not block on it.\n>\n> Optional tuning — pass `--terms ` to replace the curated pluralization cues for this project (the `optional`/`chosen` cues keep their defaults): `gsd_run query assumption-delta scan \"${PHASE}\" --json --terms second,alternative,fallback`.\n\n## Decision branch\n\nRead `ASSUMPTION_DELTA_JSON`. Act on `detected` only — do **not** pattern-match the human prose.\n\n**If `skipped` is `true`:** the detector never examined a phase section — it could not resolve one (`phase_unresolved`) or could not run at all (`probe_unavailable`). Skip the checkpoint for this run rather than asserting a verdict about input that was never examined; do not raise it with the user. **Check for `skipped` before reading `detected`** — a skipped payload carries no `detected` key, and treating its absence as `false` re-creates the fabrication this branch exists to prevent.\n\n**If `detected` is `false`:** this phase does not change a core assumption. Skip the checkpoint entirely and continue planning. Do not raise it with the user.\n\n**If `detected` is `true`:** a core assumption may have lost its monopoly. The `signals[]` array tells you which family fired:\n\n| `kind` | What changed | The question to answer |\n|---|---|---|\n| `pluralization` | A second X was introduced where there was one (second platform / auth method / tenant / region / source of truth) | Does the current primary key / identity model still name the right noun? |\n| `optional` | A required / `only` field became optional | Is the field still the right anchor, or has the anchor moved? |\n| `chosen` | A derived value became chosen, or a constant became a parameter | Has a configuration decision become a modeling decision? |\n\nBefore finalizing the plan, answer this for the user and record the decision explicitly:\n\n> **Promote vs. add-alongside.** The usual correct move when a generalization occurs is to **promote** the new general representation to the primary and **demote** the old specific one to a detail of one variant — *not* to add the new one alongside the still-required old one. Adding alongside silently contradicts the generalized intent (a later variant that does not fit the old primary can be stored but never confirmed as a default).\n\nRecord the outcome in the PLAN.md front matter / a `` block:\n\n- The **noun** that is now primary (the generalized identity).\n- The **decision**: `promote` | `add-alongside` | `no-change`, with a one-line rationale.\n- If `add-alongside`: call it out as accepted debt and note what would force a later promote.\n\n## Optional companion: an invariant test\n\nWhen `detected` is `true`, suggest (do not require) a contract/invariant test that encodes the now-generalized intent — e.g. *\"every confirmed default round-trips through the primary use-path, for every supported variant.\"* That test goes red the instant a future phase reintroduces the singular assumption, so the regression cannot land silently. If the user accepts, add the test as a task in the plan.\n\n## Tuning the vocabulary (optional)\n\nThe trigger vocabulary is a curated, additive-only set in `gsd-core/bin/lib/assumption-delta.cjs` (`DEFAULT_ASSUMPTION_DELTA_TERMS`). Bare \"or\" is intentionally excluded — it is too common in prose and would make the gate fire constantly. To widen or narrow the cues for a project, override at the call site with `--terms ` (replaces the pluralization cues; `optional`/`chosen` keep defaults). The whole checkpoint is toggleable via `workflow.assumption_delta` in `.planning/config.json`.\n\nThis checkpoint is advisory: it informs and records; it never blocks the phase.\n" }, "produces": [], "consumes": [ @@ -4256,7 +4256,7 @@ const byLoopPoint = { "into": "planner", "fragment": { "path": "fragments/api-coverage-plan-pre.md", - "inline": "# API Coverage Decision Checkpoint\n\n> Full API Coverage by Default — Opt Out, Never Opt In. Fires when a phase\n> integrates an external API / SDK / service. Most non-API phases will not fire\n> it — that is the point.\n\n## Why this exists\n\n\"We integrated the API\" too often silently means \"we integrated whatever the\nfirst use case exercised.\" Every un-built capability is then an invisible hole,\ndiscovered later by a user who reasonably expected it to work. The phase sealed\ngreen because its tasks completed; nobody decided the gaps were acceptable,\nbecause nobody enumerated them. This checkpoint makes the surface **visible and\ndecided** before the phase can seal.\n\n## Detect whether this phase integrates an external API\n\nThe detector is a deterministic scan over the phase scope. It strips fenced\ncode blocks first, so a trigger term inside a code snippet does not fire. It\nreturns a typed result: `{ detected, signals[], terms }`. Run it on the phase\nscope (the concatenation of this phase's ROADMAP section + the PLAN body):\n\n```bash\nSCOPE=\"$(cat \"${PHASE_DIR}\"/*-PLAN.md 2>/dev/null) $(gsd_run query roadmap.get-phase \"${PHASE}\" 2>/dev/null || true)\"\nAPI_COVERAGE_JSON=$(printf '%s' \"$SCOPE\" | node gsd-core/bin/lib/api-coverage.cjs --json 2>/dev/null || echo '{\"detected\":false,\"signals\":[]}')\n```\n\nThe detector's exit code and `--json` payload now distinguish a real negative\nfrom an unexamined input (ADR-3889 Phase 3, #3907): empty/whitespace-only\n`$SCOPE` or a stdin read failure emit `{\"skipped\":true,\"reason\":\"no_input\"|\n\"stdin_error\"}` — no `detected` key at all. **Check for `skipped` before\nreading `detected`**: a `skipped` payload is not a confirmed \"no API\nintegration\" verdict, it means the detector never examined real input. Do not\ntreat it as `detected:false`. Read `API_COVERAGE_JSON.detected` only when\n`skipped` is absent — act on it only, do **not** pattern-match the prose\nyourself.\n\n**If `skipped` is `true`:** the detector could not establish a scope (empty\n`$SCOPE`) or failed to run (stdin read error). Skip the checkpoint for this\nrun rather than asserting a verdict about input that was never examined; do\nnot raise it with the user.\n\n**If `detected` is `false`:** this phase does not integrate an external API. Skip\nthe checkpoint entirely and continue planning. Do not raise it with the user.\n\n**If `detected` is `true`:** an external-API integration is in scope. You MUST\nproduce a **coverage matrix** before the plan is finalized.\n\n**If `detected` is `true` but the phase genuinely integrates no external API**\n(the detector is deterministic, not infallible — confirm by re-reading the phase\nscope, not by preference): do NOT fabricate a matrix row for a capability that\ndoes not exist. Write a reasoned declaration to `${PHASE_DIR}/COVERAGE.md`\ninstead:\n\n```markdown\nNo external API integration: .\n```\n\nThe reason is required, exactly like an `OPT-OUT` reason. The seal-time gate\naccepts this declaration in place of a matrix.\n\n## Produce the coverage matrix\n\nEnumerate the external API's full **capability surface** — the verb/endpoint/method\nlist (e.g. for a music service: `search`, `play`, `pause`, `skip`, `set_volume`,\n`get_playlist`, `create_playlist`, `add_to_playlist`, …). For each capability\nrecord a decision, starting from **full coverage** as the default:\n\n| capability | decision | reason |\n|---|---|---|\n| `` | `INTEGRATE` \\| `OPT-OUT` | `` |\n\nRules:\n\n- **`INTEGRATE` is the default.** Every capability starts as INTEGRATE; the\n matrix is the *subtraction record*.\n- **Every `OPT-OUT` MUST carry a one-line reason** (`not needed`, `not needed\n yet`, `explicitly out of scope`, …). An opt-out without a reason is an\n un-decided hole — the exact failure mode this gate exists to close.\n- **A second integration against the same need** (e.g. a second platform for the\n same capability) starts from the **same full-coverage baseline** as the first.\n Do not carry over the first integration's opt-outs silently — re-decide each\n capability for the new surface, so a first-class/fallback asymmetry cannot\n accumulate.\n\nWrite the matrix to `${PHASE_DIR}/COVERAGE.md` (canonical markdown-table form):\n\n```markdown\n# API Coverage — \n\n> Full coverage by default. Opt-outs are explicit, reasoned decisions.\n\n| capability | decision | reason |\n|---|---|---|\n| search | INTEGRATE | |\n| playlists | INTEGRATE | |\n| skip | OPT-OUT | not needed yet — tracked for follow-up phase |\n```\n\nA fenced ` ```coverage ` JSON block is also accepted for machine-generated\nmatrices; the markdown table is preferred (human-editable, diff-friendly).\n\n## The seal-time gate\n\nThis checkpoint is enforced. At `verify:pre` the `api-coverage.verify-pre` gate\nruns `check api-coverage.verify-pre `:\n\n- If `COVERAGE.md` exists, it is validated — every row needs a valid decision and\n every `OPT-OUT` a reason. A malformed/partial matrix **blocks the seal**. A\n reasoned `No external API integration: …` declaration (and no rows) passes.\n- If `COVERAGE.md` is absent, the detector runs again over the phase scope. If a\n strong external-API-integration signal is found, the seal is **blocked** until a\n matrix is produced. If no signal is found, the phase is treated as a non-API\n phase and the seal proceeds.\n\nSo: an API-integrating phase cannot seal without a decided matrix. Produce it at\nplan time; do not leave it for seal time.\n\n## Tuning the vocabulary (optional)\n\nThe trigger vocabulary is a curated, additive-only set in\n`gsd-core/bin/lib/api-coverage.cjs` (`DEFAULT_API_COVERAGE_TERMS`). To widen it\nfor a project, override at the call site:\n\n```bash\nprintf '%s' \"$SCOPE\" | node gsd-core/bin/lib/api-coverage.cjs --json \\\n --verbs integrate,wrap,connect,embed --nouns api,sdk,rest,grpc,webhook,plugin\n```\n\nThe whole checkpoint is toggleable via `workflow.api_coverage_gate` in\n`.planning/config.json`.\n" + "inline": "# API Coverage Decision Checkpoint\n\n> Full API Coverage by Default — Opt Out, Never Opt In. Fires when a phase\n> integrates an external API / SDK / service. Most non-API phases will not fire\n> it — that is the point.\n\n## Why this exists\n\n\"We integrated the API\" too often silently means \"we integrated whatever the\nfirst use case exercised.\" Every un-built capability is then an invisible hole,\ndiscovered later by a user who reasonably expected it to work. The phase sealed\ngreen because its tasks completed; nobody decided the gaps were acceptable,\nbecause nobody enumerated them. This checkpoint makes the surface **visible and\ndecided** before the phase can seal.\n\n## Detect whether this phase integrates an external API\n\nThe detector is a deterministic scan over the phase scope. It strips fenced\ncode blocks first, so a trigger term inside a code snippet does not fire. It\nreturns a typed result: `{ detected, signals[], terms }`. Run it on the phase\nscope (the concatenation of this phase's ROADMAP section + the PLAN body):\n\n```bash\nSCOPE=\"$(cat \"${PHASE_DIR}\"/*-PLAN.md 2>/dev/null) $(gsd_run query roadmap.get-phase \"${PHASE}\" 2>/dev/null || true)\"\nAPI_COVERAGE_JSON=$(printf '%s' \"$SCOPE\" | node gsd-core/bin/lib/api-coverage.cjs --json 2>/dev/null) || true\n[ -n \"$API_COVERAGE_JSON\" ] || API_COVERAGE_JSON='{\"skipped\":true,\"reason\":\"probe_unavailable\"}'\n```\n\nThe `|| true` neutralizes the assignment's status without discarding the\ndetector's own payload: the detector exits **1** for a real \"no integration\"\nverdict, so treating any non-zero exit as failure would throw away a correct\nanswer. Emptiness — not exit status — is what proves the probe never ran, and\nthe second line is the only place the fragment manufactures a payload of its\nown — one that records the *absence* of a verdict rather than asserting one.\n\nThe detector's exit code and `--json` payload now distinguish a real negative\nfrom an unexamined input (ADR-3889 Phase 3, #3907): empty/whitespace-only\n`$SCOPE` or a stdin read failure emit `{\"skipped\":true,\"reason\":\"no_input\"|\n\"stdin_error\"}` — no `detected` key at all. **Check for `skipped` before\nreading `detected`**: a `skipped` payload is not a confirmed \"no API\nintegration\" verdict, it means the detector never examined real input. Do not\ntreat it as `detected:false`. Read `API_COVERAGE_JSON.detected` only when\n`skipped` is absent — act on it only, do **not** pattern-match the prose\nyourself.\n\n**If `skipped` is `true`:** the detector could not establish a scope (empty\n`$SCOPE`) or failed to run (stdin read error). Skip the checkpoint for this\nrun rather than asserting a verdict about input that was never examined; do\nnot raise it with the user.\n\n**If `detected` is `false`:** this phase does not integrate an external API. Skip\nthe checkpoint entirely and continue planning. Do not raise it with the user.\n\n**If `detected` is `true`:** an external-API integration is in scope. You MUST\nproduce a **coverage matrix** before the plan is finalized.\n\n**If `detected` is `true` but the phase genuinely integrates no external API**\n(the detector is deterministic, not infallible — confirm by re-reading the phase\nscope, not by preference): do NOT fabricate a matrix row for a capability that\ndoes not exist. Write a reasoned declaration to `${PHASE_DIR}/COVERAGE.md`\ninstead:\n\n```markdown\nNo external API integration: .\n```\n\nThe reason is required, exactly like an `OPT-OUT` reason. The seal-time gate\naccepts this declaration in place of a matrix.\n\n## Produce the coverage matrix\n\nEnumerate the external API's full **capability surface** — the verb/endpoint/method\nlist (e.g. for a music service: `search`, `play`, `pause`, `skip`, `set_volume`,\n`get_playlist`, `create_playlist`, `add_to_playlist`, …). For each capability\nrecord a decision, starting from **full coverage** as the default:\n\n| capability | decision | reason |\n|---|---|---|\n| `` | `INTEGRATE` \\| `OPT-OUT` | `` |\n\nRules:\n\n- **`INTEGRATE` is the default.** Every capability starts as INTEGRATE; the\n matrix is the *subtraction record*.\n- **Every `OPT-OUT` MUST carry a one-line reason** (`not needed`, `not needed\n yet`, `explicitly out of scope`, …). An opt-out without a reason is an\n un-decided hole — the exact failure mode this gate exists to close.\n- **A second integration against the same need** (e.g. a second platform for the\n same capability) starts from the **same full-coverage baseline** as the first.\n Do not carry over the first integration's opt-outs silently — re-decide each\n capability for the new surface, so a first-class/fallback asymmetry cannot\n accumulate.\n\nWrite the matrix to `${PHASE_DIR}/COVERAGE.md` (canonical markdown-table form):\n\n```markdown\n# API Coverage — \n\n> Full coverage by default. Opt-outs are explicit, reasoned decisions.\n\n| capability | decision | reason |\n|---|---|---|\n| search | INTEGRATE | |\n| playlists | INTEGRATE | |\n| skip | OPT-OUT | not needed yet — tracked for follow-up phase |\n```\n\nA fenced ` ```coverage ` JSON block is also accepted for machine-generated\nmatrices; the markdown table is preferred (human-editable, diff-friendly).\n\n## The seal-time gate\n\nThis checkpoint is enforced. At `verify:pre` the `api-coverage.verify-pre` gate\nruns `check api-coverage.verify-pre `:\n\n- If `COVERAGE.md` exists, it is validated — every row needs a valid decision and\n every `OPT-OUT` a reason. A malformed/partial matrix **blocks the seal**. A\n reasoned `No external API integration: …` declaration (and no rows) passes.\n- If `COVERAGE.md` is absent, the detector runs again over the phase scope. If a\n strong external-API-integration signal is found, the seal is **blocked** until a\n matrix is produced. If no signal is found, the phase is treated as a non-API\n phase and the seal proceeds.\n\nSo: an API-integrating phase cannot seal without a decided matrix. Produce it at\nplan time; do not leave it for seal time.\n\n## Tuning the vocabulary (optional)\n\nThe trigger vocabulary is a curated, additive-only set in\n`gsd-core/bin/lib/api-coverage.cjs` (`DEFAULT_API_COVERAGE_TERMS`). To widen it\nfor a project, override at the call site:\n\n```bash\nprintf '%s' \"$SCOPE\" | node gsd-core/bin/lib/api-coverage.cjs --json \\\n --verbs integrate,wrap,connect,embed --nouns api,sdk,rest,grpc,webhook,plugin\n```\n\nThe whole checkpoint is toggleable via `workflow.api_coverage_gate` in\n`.planning/config.json`.\n" }, "produces": [ "COVERAGE.md" @@ -4273,7 +4273,7 @@ const byLoopPoint = { "into": "planner", "fragment": { "path": "fragments/plan-pre.md", - "inline": "# Assumption-Delta Architecture Checkpoint\n\n> Advisory, non-blocking. Fires **only** when the phase scope shows a singular→plural / required→optional / derived→chosen transition. When it fires, it surfaces ONE identity-model question before the plan is finalized. Most phases will not fire it — that is the point.\n\n## Why this exists\n\nMost quietly-imported architectural debt does not come from a missing upfront design phase. It comes at the *seam*: a later phase introduces a second case (a second platform, auth method, tenant, region, source of truth) and nobody re-asks whether the original abstraction still names the right thing. The phase that adds the second case is exactly the 20-minute conversation that prevents an afternoon of later cleanup.\n\n## Run the detector\n\nThe detector is a deterministic scan over the phase scope text. It strips fenced code blocks first, so a trigger word that appears only inside a code snippet does not fire. It returns a typed result: `{ detected, signals[], terms }`. Resolve it through the `assumption-delta scan` query (same phase-section resolver as `roadmap.get-phase`):\n\n```bash\nASSUMPTION_DELTA_JSON=$(gsd_run query assumption-delta scan \"${PHASE}\" --json 2>/dev/null || echo '{\"detected\":false,\"signals\":[],\"terms\":{}}')\n```\n\n> If the phase section cannot be resolved (no `ROADMAP.md` / unknown phase), the query emits `{ \"detected\": false, ... }` — the checkpoint does not fire. Do not block on it.\n>\n> Optional tuning — pass `--terms ` to replace the curated pluralization cues for this project (the `optional`/`chosen` cues keep their defaults): `gsd_run query assumption-delta scan \"${PHASE}\" --json --terms second,alternative,fallback`.\n\n## Decision branch\n\nRead `ASSUMPTION_DELTA_JSON`. Act on `detected` only — do **not** pattern-match the human prose.\n\n**If `detected` is `false`:** this phase does not change a core assumption. Skip the checkpoint entirely and continue planning. Do not raise it with the user.\n\n**If `detected` is `true`:** a core assumption may have lost its monopoly. The `signals[]` array tells you which family fired:\n\n| `kind` | What changed | The question to answer |\n|---|---|---|\n| `pluralization` | A second X was introduced where there was one (second platform / auth method / tenant / region / source of truth) | Does the current primary key / identity model still name the right noun? |\n| `optional` | A required / `only` field became optional | Is the field still the right anchor, or has the anchor moved? |\n| `chosen` | A derived value became chosen, or a constant became a parameter | Has a configuration decision become a modeling decision? |\n\nBefore finalizing the plan, answer this for the user and record the decision explicitly:\n\n> **Promote vs. add-alongside.** The usual correct move when a generalization occurs is to **promote** the new general representation to the primary and **demote** the old specific one to a detail of one variant — *not* to add the new one alongside the still-required old one. Adding alongside silently contradicts the generalized intent (a later variant that does not fit the old primary can be stored but never confirmed as a default).\n\nRecord the outcome in the PLAN.md front matter / a `` block:\n\n- The **noun** that is now primary (the generalized identity).\n- The **decision**: `promote` | `add-alongside` | `no-change`, with a one-line rationale.\n- If `add-alongside`: call it out as accepted debt and note what would force a later promote.\n\n## Optional companion: an invariant test\n\nWhen `detected` is `true`, suggest (do not require) a contract/invariant test that encodes the now-generalized intent — e.g. *\"every confirmed default round-trips through the primary use-path, for every supported variant.\"* That test goes red the instant a future phase reintroduces the singular assumption, so the regression cannot land silently. If the user accepts, add the test as a task in the plan.\n\n## Tuning the vocabulary (optional)\n\nThe trigger vocabulary is a curated, additive-only set in `gsd-core/bin/lib/assumption-delta.cjs` (`DEFAULT_ASSUMPTION_DELTA_TERMS`). Bare \"or\" is intentionally excluded — it is too common in prose and would make the gate fire constantly. To widen or narrow the cues for a project, override at the call site with `--terms ` (replaces the pluralization cues; `optional`/`chosen` keep defaults). The whole checkpoint is toggleable via `workflow.assumption_delta` in `.planning/config.json`.\n\nThis checkpoint is advisory: it informs and records; it never blocks the phase.\n" + "inline": "# Assumption-Delta Architecture Checkpoint\n\n> Advisory, non-blocking. Fires **only** when the phase scope shows a singular→plural / required→optional / derived→chosen transition. When it fires, it surfaces ONE identity-model question before the plan is finalized. Most phases will not fire it — that is the point.\n\n## Why this exists\n\nMost quietly-imported architectural debt does not come from a missing upfront design phase. It comes at the *seam*: a later phase introduces a second case (a second platform, auth method, tenant, region, source of truth) and nobody re-asks whether the original abstraction still names the right thing. The phase that adds the second case is exactly the 20-minute conversation that prevents an afternoon of later cleanup.\n\n## Run the detector\n\nThe detector is a deterministic scan over the phase scope text. It strips fenced code blocks first, so a trigger word that appears only inside a code snippet does not fire. It returns a typed result: `{ detected, signals[], terms }`. Resolve it through the `assumption-delta scan` query (same phase-section resolver as `roadmap.get-phase`):\n\n```bash\nASSUMPTION_DELTA_JSON=$(gsd_run query assumption-delta scan \"${PHASE}\" --json 2>/dev/null) || true\n[ -n \"$ASSUMPTION_DELTA_JSON\" ] || ASSUMPTION_DELTA_JSON='{\"skipped\":true,\"reason\":\"probe_unavailable\"}'\n```\n\n> If the phase section cannot be resolved (no `ROADMAP.md` / unknown phase, or a section with no body), the query emits `{ \"skipped\": true, \"reason\": \"phase_unresolved\" }` — **not** `detected:false`. A probe that never had input does not get to assert that this phase changes no core assumption. The checkpoint does not fire either way; the difference is that a skip is now distinguishable from a real negative. Do not block on it.\n>\n> Optional tuning — pass `--terms ` to replace the curated pluralization cues for this project (the `optional`/`chosen` cues keep their defaults): `gsd_run query assumption-delta scan \"${PHASE}\" --json --terms second,alternative,fallback`.\n\n## Decision branch\n\nRead `ASSUMPTION_DELTA_JSON`. Act on `detected` only — do **not** pattern-match the human prose.\n\n**If `skipped` is `true`:** the detector never examined a phase section — it could not resolve one (`phase_unresolved`) or could not run at all (`probe_unavailable`). Skip the checkpoint for this run rather than asserting a verdict about input that was never examined; do not raise it with the user. **Check for `skipped` before reading `detected`** — a skipped payload carries no `detected` key, and treating its absence as `false` re-creates the fabrication this branch exists to prevent.\n\n**If `detected` is `false`:** this phase does not change a core assumption. Skip the checkpoint entirely and continue planning. Do not raise it with the user.\n\n**If `detected` is `true`:** a core assumption may have lost its monopoly. The `signals[]` array tells you which family fired:\n\n| `kind` | What changed | The question to answer |\n|---|---|---|\n| `pluralization` | A second X was introduced where there was one (second platform / auth method / tenant / region / source of truth) | Does the current primary key / identity model still name the right noun? |\n| `optional` | A required / `only` field became optional | Is the field still the right anchor, or has the anchor moved? |\n| `chosen` | A derived value became chosen, or a constant became a parameter | Has a configuration decision become a modeling decision? |\n\nBefore finalizing the plan, answer this for the user and record the decision explicitly:\n\n> **Promote vs. add-alongside.** The usual correct move when a generalization occurs is to **promote** the new general representation to the primary and **demote** the old specific one to a detail of one variant — *not* to add the new one alongside the still-required old one. Adding alongside silently contradicts the generalized intent (a later variant that does not fit the old primary can be stored but never confirmed as a default).\n\nRecord the outcome in the PLAN.md front matter / a `` block:\n\n- The **noun** that is now primary (the generalized identity).\n- The **decision**: `promote` | `add-alongside` | `no-change`, with a one-line rationale.\n- If `add-alongside`: call it out as accepted debt and note what would force a later promote.\n\n## Optional companion: an invariant test\n\nWhen `detected` is `true`, suggest (do not require) a contract/invariant test that encodes the now-generalized intent — e.g. *\"every confirmed default round-trips through the primary use-path, for every supported variant.\"* That test goes red the instant a future phase reintroduces the singular assumption, so the regression cannot land silently. If the user accepts, add the test as a task in the plan.\n\n## Tuning the vocabulary (optional)\n\nThe trigger vocabulary is a curated, additive-only set in `gsd-core/bin/lib/assumption-delta.cjs` (`DEFAULT_ASSUMPTION_DELTA_TERMS`). Bare \"or\" is intentionally excluded — it is too common in prose and would make the gate fire constantly. To widen or narrow the cues for a project, override at the call site with `--terms ` (replaces the pluralization cues; `optional`/`chosen` keep defaults). The whole checkpoint is toggleable via `workflow.assumption_delta` in `.planning/config.json`.\n\nThis checkpoint is advisory: it informs and records; it never blocks the phase.\n" }, "produces": [], "consumes": [ diff --git a/gsd-core/references/api-coverage.md b/gsd-core/references/api-coverage.md index 7d30ba287..ecb05950d 100644 --- a/gsd-core/references/api-coverage.md +++ b/gsd-core/references/api-coverage.md @@ -53,7 +53,26 @@ compound modifiers ("Resolver-only API"), and first-party-qualified services `capabilities/ai-integration/fragments/api-coverage-plan-pre.md`. 2. **Seal time (`verify:pre`).** The blocking `api-coverage.verify-pre` gate runs `check api-coverage.verify-pre ` and blocks unless a valid - matrix exists (or no integration is detected). + matrix exists (or no integration is detected in a scope it could actually + read). + +### Seal-time outcomes + +| Condition | Verdict | +|---|---| +| Valid `COVERAGE.md` matrix | pass | +| `No external API integration: ` declaration | pass (the reasoned human overrule) | +| Malformed / partial matrix | **block** | +| No matrix, integration signal found | **block** | +| No matrix, no signal, scope was read | pass | +| No matrix, **scope could not be established** | **block** — `scope_unavailable: true` (#3909) | +| Phase token unresolvable | **block** | +| Plan file exists but unreadable | **block** | +| No `.planning/phases` tree at all | pass — not a GSD project layout | + +The last four are the fail-closed arms: the gate refuses to certify "no +external-API integration" from scope it did not read. A phase with real plans +and no API vocabulary is the fifth row, and is unaffected. ## The coverage matrix format diff --git a/src/check-command-router.cts b/src/check-command-router.cts index 77f124bc3..46ffcb9fb 100644 --- a/src/check-command-router.cts +++ b/src/check-command-router.cts @@ -1505,6 +1505,33 @@ function cmdApiCoverageVerifyPre(projectDir: string, args: string[], raw: boolea ); return; } + // An EMPTY scope is not a negative verdict. This gate's neighbouring arms + // already fail closed (unresolvable phase → block; unreadable plan → block), + // but a phase with no plan body AND no roadmap section fell through to + // detection over zero bytes and CERTIFIED "no external-API integration" — + // clearing a blocking seal gate on a probe that examined nothing + // (ADR-3889 failure class (c), #3909). The discriminator is BYTES EXAMINED, + // never SIGNALS FOUND: a phase with real plans and no API vocabulary still + // reaches the pass below unchanged. + if (scope.text.trim() === '') { + output( + { + block: true, + passed: false, + coverage_present: false, + detected: false, + scope_unavailable: true, + message: + 'api-coverage: the phase scope is empty — no plan body and no roadmap section were ' + + 'found, so nothing was examined. Refusing to certify no external-API integration ' + + 'from an unestablished scope. Add the phase plan, or add a COVERAGE.md declaration.', + }, + raw, + undefined, + ); + return; + } + const detection = detectApiIntegration(scope.text); if (detection.detected) { // Surface only verb/noun (typed, bounded) — NOT raw prose snippets — so the diff --git a/tests/api-coverage-gate-e2e.test.cjs b/tests/api-coverage-gate-e2e.test.cjs index 4cb629afb..809912728 100644 --- a/tests/api-coverage-gate-e2e.test.cjs +++ b/tests/api-coverage-gate-e2e.test.cjs @@ -16,6 +16,7 @@ const { describe, test, afterEach } = require('node:test'); const assert = require('node:assert/strict'); +const fc = require('fast-check'); const fs = require('node:fs'); const os = require('node:os'); const path = require('node:path'); @@ -369,3 +370,162 @@ describe('readPhaseScope — fail-closed on a real read failure (#2365 review)', assert.strictEqual(res.text, ''); }); }); + +// ─── Unestablished scope is not a negative verdict (#3909, ADR-3889 P5) ─────── +// The gate's two neighbouring arms already fail closed (unresolvable phase → +// block; unreadable plan → block). Certifying "no external-API integration" +// from ZERO examined bytes was the remaining fabrication: nothing was read, yet +// the blocking seal gate cleared. The discriminator is BYTES EXAMINED, never +// SIGNALS FOUND — a phase with real plans and no API vocabulary must keep +// passing exactly as it did before. +describe('api-coverage.verify-pre — unestablished scope blocks (#3909)', () => { + let tmpDir; + afterEach(() => { if (tmpDir) { cleanup(tmpDir); tmpDir = null; } }); + + function gateJson(phaseDir) { + return JSON.parse(runGate(tmpDir, phaseDir).output); + } + + test('CONTROL: a real plan with no API vocabulary still passes', () => { + tmpDir = makeProject({ api_coverage_gate: true }); + const phaseDir = makePhaseDir(tmpDir, '01-refactor'); + writePlan(phaseDir, '01-PLAN.md', '# Plan\nRefactor the internal state machine and rename fields.'); + const j = gateJson(phaseDir); + assert.strictEqual(j.block, false, 'an examined phase with no signal must still pass'); + assert.strictEqual(j.passed, true); + assert.strictEqual(j.detected, false); + assert.strictEqual( + j.scope_unavailable, + undefined, + 'an examined scope must NOT be reported as unavailable', + ); + }); + + test('a phase dir with no plans and no roadmap section BLOCKS instead of certifying', () => { + tmpDir = makeProject({ api_coverage_gate: true }); + const phaseDir = makePhaseDir(tmpDir, '02-empty'); + const j = gateJson(phaseDir); + assert.strictEqual(j.block, true, 'zero examined bytes must not clear the blocking seal gate'); + assert.strictEqual(j.passed, false); + assert.strictEqual(j.scope_unavailable, true, 'the reason must be reported, not implied'); + assert.strictEqual( + j.detected, + false, + 'detected stays false — nothing was found because nothing was examined', + ); + assert.match(j.message, /scope/i); + }); + + test('BOUNDARY: whitespace-only scope is unestablished', () => { + tmpDir = makeProject({ api_coverage_gate: true }); + const phaseDir = makePhaseDir(tmpDir, '03-ws'); + writePlan(phaseDir, '01-PLAN.md', ' \n\t\n '); + const j = gateJson(phaseDir); + assert.strictEqual(j.block, true, 'whitespace is not examined content'); + assert.strictEqual(j.scope_unavailable, true); + }); + + test('BOUNDARY: CRLF-only scope is unestablished', () => { + tmpDir = makeProject({ api_coverage_gate: true }); + const phaseDir = makePhaseDir(tmpDir, '04-crlf'); + writePlan(phaseDir, '01-PLAN.md', '\r\n\r\n'); + const j = gateJson(phaseDir); + assert.strictEqual(j.block, true, 'CRLF-only content carries no scope'); + assert.strictEqual(j.scope_unavailable, true); + }); + + test('BOUNDARY: a single non-whitespace character IS examined content', () => { + tmpDir = makeProject({ api_coverage_gate: true }); + const phaseDir = makePhaseDir(tmpDir, '05-one'); + writePlan(phaseDir, '01-PLAN.md', 'x'); + const j = gateJson(phaseDir); + assert.strictEqual(j.block, false, 'one byte of real scope is examined — normal detection applies'); + assert.strictEqual(j.scope_unavailable, undefined); + }); + + test('a COVERAGE.md matrix short-circuits before scope is ever read', () => { + tmpDir = makeProject({ api_coverage_gate: true }); + const phaseDir = makePhaseDir(tmpDir, '06-matrix'); + // No plans at all — scope would be unestablished, but the matrix decides. + writeCoverage(phaseDir, [ + '# API Coverage — Stripe', + '', + '| capability | decision | reason |', + '|---|---|---|', + '| charge | INTEGRATE | |', + ].join('\n')); + const j = gateJson(phaseDir); + assert.strictEqual(j.block, false, 'a valid matrix passes regardless of scope readability'); + assert.strictEqual(j.coverage_present, true); + assert.strictEqual(j.scope_unavailable, undefined); + }); + + test('a no-integration DECLARATION passes even with an unestablished scope', () => { + tmpDir = makeProject({ api_coverage_gate: true }); + const phaseDir = makePhaseDir(tmpDir, '07-decl'); + writeCoverage(phaseDir, 'No external API integration: pure internal refactor.\n'); + const j = gateJson(phaseDir); + assert.strictEqual(j.block, false, 'the human declaration is the reasoned overrule'); + assert.strictEqual(j.none_declared, true); + assert.strictEqual(j.scope_unavailable, undefined); + }); + + test('a project with no .planning/phases tree still fails OPEN (not a scope block)', () => { + const noPhases = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-apicov-nophases-')); + try { + fs.mkdirSync(path.join(noPhases, '.planning'), { recursive: true }); + fs.writeFileSync( + path.join(noPhases, '.planning', 'config.json'), + JSON.stringify({ workflow: { api_coverage_gate: true } }), + 'utf8', + ); + const r = runTools(['check', 'api-coverage.verify-pre', '01-x', '--raw'], noPhases); + const j = JSON.parse(r.output); + assert.strictEqual(j.block, false, 'a non-GSD layout must stay fail-open'); + assert.strictEqual(j.passed, true); + assert.strictEqual( + j.scope_unavailable, + undefined, + 'the new block must not widen to swallow the non-GSD-project pass', + ); + } finally { + cleanup(noPhases); + } + }); +}); + +// ─── Property: the scope discriminator is emptiness, nothing else (#3909) ───── +// The gate reports `scope_unavailable` exactly when the scope it read is +// whitespace-only. This pins that predicate against arbitrary input — unicode +// whitespace, CRLF, strings that merely look blank — so the discriminator can +// never quietly become "no signals found" instead of "nothing examined". +describe('api-coverage scope emptiness — properties (#3909)', () => { + let tmpDir; + afterEach(() => { if (tmpDir) { cleanup(tmpDir); tmpDir = null; } }); + + test('P1: the scope read back is whitespace-only iff the plan body was', () => { + fc.assert( + fc.property(fc.string(), (body) => { + tmpDir = makeProject({ api_coverage_gate: true }); + try { + const phaseDir = makePhaseDir(tmpDir, '01-prop'); + writePlan(phaseDir, '01-PLAN.md', body); + // No ROADMAP.md is written, so the roadmap fallback contributes + // nothing and the plan body is the entire scope. + const res = readPhaseScope(tmpDir, phaseDir, '01'); + assert.strictEqual(res.readError, null, 'a written plan file must always be readable'); + assert.strictEqual( + res.text.trim() === '', + body.trim() === '', + 'the gate blocks on an unestablished scope, so its emptiness test must agree ' + + 'with the emptiness of what the author actually wrote', + ); + } finally { + cleanup(tmpDir); + tmpDir = null; + } + }), + { numRuns: 200, seed: 3909 }, + ); + }); +}); diff --git a/tests/assumption-delta-checkpoint-e2e.test.cjs b/tests/assumption-delta-checkpoint-e2e.test.cjs index b19895d7d..18b55c0c7 100644 --- a/tests/assumption-delta-checkpoint-e2e.test.cjs +++ b/tests/assumption-delta-checkpoint-e2e.test.cjs @@ -127,11 +127,19 @@ describe('assumption-delta scan query — phase-section detection (#1561)', () = assert.deepStrictEqual(parsed.signals, []); }); - test('unknown phase → detected:false, no throw (graceful)', () => { + test('unknown phase → skipped, no throw (graceful) (#3909)', () => { tmpDir = makeRoadmapProject(); const r = scanQuery(tmpDir, 999, ['--json']); assert.ok(r.success, `scan should succeed on unknown phase. stderr: ${r.error}`); - assert.strictEqual(JSON.parse(r.output).detected, false); + const parsed = JSON.parse(r.output); + // This asserted `detected:false` until #3909. That was the fabrication + // itself pinned as intended behavior: phase 999 does not exist, so the + // detector was handed the empty string and its "no core assumption + // changed" answer described nothing. The graceful-degradation contract + // the test was protecting (succeeds, does not throw) is unchanged. + assert.strictEqual(parsed.skipped, true); + assert.strictEqual(parsed.reason, 'phase_unresolved'); + assert.strictEqual(parsed.detected, undefined); }); test('--terms override narrows the vocabulary (custom cue fires, default cue does not)', () => { @@ -226,3 +234,84 @@ describe('assumption-delta capability — plan:pre render-hooks wiring (#1561)', ); }); }); + +// ─── An unresolved phase section is not a negative verdict (#3909, P5) ──────── +// `routeAssumptionDelta` fed `detectAssumptionDelta(section ?? '')`, so a phase +// section that could not be resolved at all (no ROADMAP.md, unknown phase +// number) was scanned as the empty string and reported as an examined-and- +// negative result. That is a fabricated verdict: the probe never had input. +// The route now reports the skipped-with-reason shape instead. Exit code stays +// 0 — this is an ADR-2980 degraded result reported through the payload, and +// ADR-3889 P8 pins the gsd-tools projection at v1. +describe('query assumption-delta scan — unresolved phase reports skipped (#3909)', () => { + let tmpDir; + afterEach(() => { if (tmpDir) { cleanup(tmpDir); tmpDir = null; } }); + + function writeRoadmap(dir, body) { + fs.writeFileSync(path.join(dir, '.planning', 'ROADMAP.md'), body, 'utf8'); + } + + function scan(cwd, phase, extra = []) { + const r = runTools(['query', 'assumption-delta', 'scan', phase, '--json', ...extra], cwd); + return { raw: r, json: JSON.parse(r.output) }; + } + + test('CONTROL: a resolved section with no cues still reports detected:false', () => { + tmpDir = makeProject({ workflow: { assumption_delta: true } }); + writeRoadmap(tmpDir, '# Roadmap\n\n### Phase 01: Cleanup\n\nRefactor the internal state machine.\n'); + const { raw, json } = scan(tmpDir, '01'); + assert.strictEqual(raw.exitCode, 0); + assert.strictEqual(json.detected, false, 'an examined section with no cues is a real negative'); + assert.strictEqual(json.skipped, undefined, 'a real scan must NOT be reported as skipped'); + }); + + test('CONTROL: a resolved section with a pluralization cue still detects', () => { + tmpDir = makeProject({ workflow: { assumption_delta: true } }); + writeRoadmap(tmpDir, '# Roadmap\n\n### Phase 01: Auth\n\nAdd a second authentication method.\n'); + const { json } = scan(tmpDir, '01'); + assert.strictEqual(json.detected, true); + assert.strictEqual(json.skipped, undefined); + }); + + test('no ROADMAP.md at all reports skipped, not detected:false', () => { + tmpDir = makeProject({ workflow: { assumption_delta: true } }); + const { raw, json } = scan(tmpDir, '01'); + assert.strictEqual(json.skipped, true, 'a probe with no input must not assert a verdict'); + assert.strictEqual(json.reason, 'phase_unresolved'); + assert.strictEqual( + json.detected, + undefined, + 'a skipped payload must carry no detected key — that is what makes it distinguishable', + ); + assert.strictEqual(raw.exitCode, 0, 'degraded result stays exit 0 (ADR-2980; P8 pins v1)'); + }); + + test('a ROADMAP.md that does not contain the phase reports skipped', () => { + tmpDir = makeProject({ workflow: { assumption_delta: true } }); + writeRoadmap(tmpDir, '# Roadmap\n\n### Phase 01: Auth\n\nAdd a second authentication method.\n'); + const { json } = scan(tmpDir, '99'); + assert.strictEqual(json.skipped, true, 'an unknown phase number resolves to nothing'); + assert.strictEqual(json.reason, 'phase_unresolved'); + }); + + test('BOUNDARY: a resolved but body-less section is EXAMINED, not skipped', () => { + tmpDir = makeProject({ workflow: { assumption_delta: true } }); + writeRoadmap(tmpDir, '# Roadmap\n\n### Phase 01: Empty\n\n \n\t\n\n### Phase 02: Next\n\nBody.\n'); + const { json } = scan(tmpDir, '01'); + // The resolver returns the heading line alone for a body-less section, so + // the phase WAS found and the detector did examine what the roadmap holds + // for it. `skipped` is reserved for the cases where the resolver returns + // null — an unknown phase number, or no ROADMAP.md at all. Pinning that + // line here keeps the distinction the issue actually asks for (found vs + // not found) from drifting into a vaguer "looks empty to me". + assert.strictEqual(json.skipped, undefined, 'a resolved section is not a skip'); + assert.strictEqual(json.detected, false, 'the heading alone carries no cues — a real negative'); + }); + + test('--terms does not rescue an unresolved phase', () => { + tmpDir = makeProject({ workflow: { assumption_delta: true } }); + const { json } = scan(tmpDir, '01', ['--terms', 'second,alternative']); + assert.strictEqual(json.skipped, true, 'a vocabulary override cannot manufacture a scan'); + assert.strictEqual(json.reason, 'phase_unresolved'); + }); +}); diff --git a/tests/capability-probe-fallback.test.cjs b/tests/capability-probe-fallback.test.cjs new file mode 100644 index 000000000..b41183fd6 --- /dev/null +++ b/tests/capability-probe-fallback.test.cjs @@ -0,0 +1,290 @@ +'use strict'; + +/** + * The capability fragments' probe fallbacks must be honest (#3909, ADR-3889 P5). + * + * Two capability fragments carry a shell snippet that runs a detector and + * captures its JSON. Those snippets used `… 2>/dev/null || echo '{"detected":false}'`, + * which FABRICATES a negative verdict whenever the probe exits non-zero. That is + * wrong three separate ways, and all three are covered here: + * + * 1. `||` fires on exit 1 — which ADR-3889 P3 made the LEGITIMATE negative — + * so a correct "no integration" answer got a second object appended to it. + * 2. `$( )` captures the whole compound's stdout, so the fallback APPENDS + * rather than replaces: an honest `{"skipped":true}` was immediately + * contradicted by a fabricated `{"detected":false}` in the same string. + * 3. A probe that genuinely could not run produced a clean, confident, + * wrong `detected:false`. + * + * BEHAVIORAL, not source-grep: each test extracts the fragment's own fenced + * bash block, executes it under `bash` with the surrounding contract stubbed + * (`gsd_run`, `PHASE_DIR`, `PHASE`), and asserts on the captured variable. + */ + +const { describe, test, afterEach } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); + +const { cleanup } = require('./helpers.cjs'); +const { runNode, OUTCOME } = require('./helpers/process-seam.cjs'); +const { splitLines } = require('../gsd-core/bin/lib/text-lines.cjs'); + +const REPO_ROOT = path.join(__dirname, '..'); +const API_FRAGMENT = path.join( + REPO_ROOT, 'capabilities', 'ai-integration', 'fragments', 'api-coverage-plan-pre.md'); +const DELTA_FRAGMENT = path.join( + REPO_ROOT, 'capabilities', 'assumption-delta', 'fragments', 'plan-pre.md'); +const TOOLS_PATH = path.join(REPO_ROOT, 'gsd-core', 'bin', 'gsd-tools.cjs'); + +/** + * Pull the fragment's probe snippet out of its markdown: the first fenced + * ```bash block that assigns `varName`. The block is returned verbatim so the + * test executes exactly the bytes the planner is handed. + */ +function extractProbeBlock(fragmentPath, varName) { + const md = splitLines(fs.readFileSync(fragmentPath, 'utf8')); + const fences = []; + let current = null; + for (const line of md) { + if (current === null) { + if (line.trim() === '```bash') { + current = []; + } + continue; + } + if (line.trim() === '```') { + fences.push(current.join('\n')); + current = null; + continue; + } + current.push(line); + } + const block = fences.find((body) => body.includes(`${varName}=`)); + assert.ok( + block, + `${path.basename(fragmentPath)} must contain a fenced bash block assigning ${varName}`, + ); + return block; +} + +/** + * Run a fragment snippet under bash and return the captured variable's value. + * `prelude` stubs the surrounding workflow contract; `cwd` decides whether the + * detector module is reachable (an unreachable one is how "the probe could not + * launch" is simulated — no chmod, no monkeypatch, just a different cwd). + */ +function runSnippet({ block, varName, prelude, cwd }) { + const script = `set -u\n${prelude}\n${block}\nprintf '%s' "\${${varName}}"\n`; + const scriptFile = path.join(fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-frag-')), 'run.sh'); + fs.writeFileSync(scriptFile, script, 'utf8'); + try { + const r = runNode(['-e', ` + const { spawnSync } = require('node:child_process'); + const r = spawnSync('bash', [process.argv[1]], { cwd: process.argv[2], encoding: 'utf8', timeout: 60000 }); + process.stdout.write(r.stdout || ''); + `, scriptFile, cwd], { cwd, timeoutMs: 60000 }); + assert.strictEqual(r.outcome, OUTCOME.EXITED, `snippet runner outcome: ${r.outcome}`); + return r.stdout; + } finally { + cleanup(path.dirname(scriptFile)); + } +} + +/** Assert the captured value is exactly ONE JSON object, and return it. */ +function parseSingleObject(captured, what) { + assert.notStrictEqual(captured.trim(), '', `${what}: the fragment captured nothing at all`); + let parsed; + try { + parsed = JSON.parse(captured); + } catch (err) { + assert.fail( + `${what}: the fragment produced text that is not a single JSON object — ` + + `a concatenated fallback is exactly this failure. Captured: ${JSON.stringify(captured)} ` + + `(${err.message})`, + ); + } + assert.strictEqual(typeof parsed, 'object', `${what}: payload must be an object`); + assert.notStrictEqual(parsed, null, `${what}: payload must not be null`); + return parsed; +} + +// ─── api-coverage fragment ──────────────────────────────────────────────────── + +describe('api-coverage fragment probe fallback is honest (#3909)', () => { + let tmpDir; + afterEach(() => { if (tmpDir) { cleanup(tmpDir); tmpDir = null; } }); + + // The fragment's snippet builds SCOPE from `${PHASE_DIR}/*-PLAN.md` plus a + // `gsd_run query roadmap.get-phase` call. Both are stubbed so the test + // controls exactly what reaches the detector. + function preludeFor(planBody) { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-apifrag-')); + if (planBody !== null) { + fs.writeFileSync(path.join(tmpDir, '01-PLAN.md'), planBody, 'utf8'); + } + return [ + `PHASE_DIR=${JSON.stringify(tmpDir)}`, + 'PHASE=01', + 'gsd_run() { return 0; }', + ].join('\n'); + } + + function capture(planBody, { cwd = REPO_ROOT } = {}) { + return runSnippet({ + block: extractProbeBlock(API_FRAGMENT, 'API_COVERAGE_JSON'), + varName: 'API_COVERAGE_JSON', + prelude: preludeFor(planBody), + cwd, + }); + } + + test('CONTROL: a scope with API vocabulary captures a detected verdict', () => { + const j = parseSingleObject( + capture('# Plan\nIntegrate the Stripe API and wrap its SDK.'), 'detected case'); + assert.strictEqual(j.detected, true); + assert.strictEqual(j.skipped, undefined); + }); + + test('a LEGITIMATE negative (probe exit 1) is captured as ONE valid object', () => { + // Regression: the detector exits 1 for a real negative, so `|| echo …` + // fired on the success path and appended a second object. + const j = parseSingleObject( + capture('# Plan\nRefactor the internal state machine.'), 'legit negative'); + assert.strictEqual(j.detected, false, 'a real negative must survive intact'); + assert.strictEqual(j.skipped, undefined, 'a real negative is not a skip'); + }); + + test('an HONEST skip (empty scope) is not contradicted by a fabricated verdict', () => { + const j = parseSingleObject(capture(null), 'empty scope'); + assert.strictEqual(j.skipped, true, 'an unexamined scope must report skipped'); + assert.strictEqual( + j.detected, + undefined, + 'the skip must not carry a detected key — that contradiction is the defect', + ); + }); + + test('a probe that CANNOT LAUNCH reports skipped, never detected:false', () => { + // cwd without the module → node fails, stdout empty. The fragment must not + // manufacture a verdict from that. + const away = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-nomodule-')); + try { + const j = parseSingleObject( + capture('# Plan\nIntegrate the Stripe API.', { cwd: away }), 'probe unavailable'); + assert.strictEqual(j.skipped, true, 'a probe that could not run must report skipped'); + assert.strictEqual(j.reason, 'probe_unavailable'); + assert.strictEqual( + j.detected, + undefined, + 'asserting detected:false from a probe that never ran is the bug this closes', + ); + } finally { + cleanup(away); + } + }); +}); + +// ─── assumption-delta fragment ──────────────────────────────────────────────── + +describe('assumption-delta fragment probe fallback is honest (#3909)', () => { + let tmpDir; + afterEach(() => { if (tmpDir) { cleanup(tmpDir); tmpDir = null; } }); + + function projectWithRoadmap(body) { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-deltafrag-')); + fs.mkdirSync(path.join(tmpDir, '.planning'), { recursive: true }); + if (body !== null) { + fs.writeFileSync(path.join(tmpDir, '.planning', 'ROADMAP.md'), body, 'utf8'); + } + return tmpDir; + } + + // `gsd_run` is the workflow launcher's shell function; stub it to the real + // CLI so the snippet exercises the genuine query route. + function realGsdRunPrelude(projectDir) { + return [ + 'PHASE=01', + `gsd_run() { ( cd ${JSON.stringify(projectDir)} && ` + + `node ${JSON.stringify(TOOLS_PATH)} "$@" ); }`, + ].join('\n'); + } + + function capture(prelude) { + return runSnippet({ + block: extractProbeBlock(DELTA_FRAGMENT, 'ASSUMPTION_DELTA_JSON'), + varName: 'ASSUMPTION_DELTA_JSON', + prelude, + cwd: REPO_ROOT, + }); + } + + test('CONTROL: a resolved section with a cue captures a detected verdict', () => { + const dir = projectWithRoadmap( + '# Roadmap\n\n### Phase 01: Auth\n\nAdd a second authentication method.\n'); + const j = parseSingleObject(capture(realGsdRunPrelude(dir)), 'detected case'); + assert.strictEqual(j.detected, true); + assert.strictEqual(j.skipped, undefined); + }); + + test('CONTROL: a resolved section with no cue captures ONE valid negative', () => { + const dir = projectWithRoadmap( + '# Roadmap\n\n### Phase 01: Cleanup\n\nRefactor the internal state machine.\n'); + const j = parseSingleObject(capture(realGsdRunPrelude(dir)), 'legit negative'); + assert.strictEqual(j.detected, false); + assert.strictEqual(j.skipped, undefined); + }); + + test('an unresolvable phase captures skipped, not a fabricated negative', () => { + const dir = projectWithRoadmap(null); + const j = parseSingleObject(capture(realGsdRunPrelude(dir)), 'unresolved phase'); + assert.strictEqual(j.skipped, true); + assert.strictEqual(j.detected, undefined); + }); + + test('a probe that CANNOT LAUNCH reports skipped, never detected:false', () => { + const j = parseSingleObject( + capture(['PHASE=01', 'gsd_run() { return 127; }'].join('\n')), 'probe unavailable'); + assert.strictEqual(j.skipped, true, 'a launcher that failed must not yield a verdict'); + assert.strictEqual(j.reason, 'probe_unavailable'); + assert.strictEqual(j.detected, undefined); + }); +}); + +// ─── Parity: one vocabulary across both fragments ───────────────────────────── + +describe('both fragments share one skipped-with-reason vocabulary (#3909)', () => { + // Generative-fix divergence guard: two surfaces adopting one convention must + // fail this test the moment they drift apart. + test('both fragments emit the same probe-unavailable reason token', () => { + const results = [ + { name: 'api-coverage', block: extractProbeBlock(API_FRAGMENT, 'API_COVERAGE_JSON') }, + { name: 'assumption-delta', block: extractProbeBlock(DELTA_FRAGMENT, 'ASSUMPTION_DELTA_JSON') }, + ].map(({ name, block }) => { + const varName = name === 'api-coverage' ? 'API_COVERAGE_JSON' : 'ASSUMPTION_DELTA_JSON'; + const captured = runSnippet({ + block, + varName, + // Force the unavailable path for both: no PHASE_DIR contents, and a + // launcher/cwd that cannot produce output. + prelude: [ + 'PHASE_DIR=/nonexistent-phase-dir-3909', + 'PHASE=01', + 'gsd_run() { return 127; }', + ].join('\n'), + cwd: os.tmpdir(), + }); + return { name, payload: parseSingleObject(captured, name) }; + }); + + for (const { name, payload } of results) { + assert.strictEqual(payload.skipped, true, `${name} must report skipped when the probe cannot run`); + } + assert.strictEqual( + results[0].payload.reason, + results[1].payload.reason, + 'the two fragments must not invent different reason tokens for the same condition', + ); + }); +});