From 34bb8ed5f640cfe306c7cd11b5480ea42e90bb5c Mon Sep 17 00:00:00 2001 From: Tom Boucher Date: Fri, 12 Jun 2026 15:37:25 -0400 Subject: [PATCH] =?UTF-8?q?docs(857):=20settle=20phase-6=20predicate=20bou?= =?UTF-8?q?ndary=20=E2=80=94=20verification=20substrate=20vs.=20plug-in=20?= =?UTF-8?q?tier?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Classify edge-probe/prohibition-probe predicate-generation as core verification substrate (not an off-by-default Feature Capability), settled before ADR-857 phase 6 (Migrate) freezes the core/plug-in line. Prompted by @davesienkowski's boundary analysis on #857. - ADR-857: amendment note + top-level decision carve-out + Loop Extension Points exemption + new "Verification substrate vs. plug-in tier" subsection + 2 Alternatives rows + phase-6 rollout exception + Open Questions resolution. - ADR-550: cross-reference pinning the core-substrate classification; ties "exogenous grading" to Decision 4 (judgment-tier) and Decision 5 (test the contract, not the classifier). - CONTEXT.md: glossary — Probe Core / Edge Probe reclassified (probe-core on the contract side, adapters as the generator) + new "Verification substrate (predicate boundary)" term. Decomposition: the verifier<->predicate contract is core/non-toggleable; the generator (probe adapters) is core-default but independently versionable. Docs-only; no user-facing change. Closes #1120 Refs #857 Co-Authored-By: Claude Opus 4.8 --- CONTEXT.md | 7 ++++-- docs/adr/550-spec-phase-probe-contract.md | 9 ++++++++ docs/adr/857-capability-system.md | 26 ++++++++++++++++++++--- 3 files changed, 37 insertions(+), 5 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 63a45ea66..b2ced6b96 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -205,10 +205,13 @@ The GSD-RESEARCH capability behind an L2-hybrid seam: code owns cache + provider Runtime-neutral predicate evaluating `*-UAT.md` / `*-VERIFICATION.md` result fields with markdown-aware parsing that ignores false-positive contexts (frontmatter body, fenced code, HTML comments, blockquotes). Returns `passed: true` only when all required checks pass; supports `--require-verification` to demand at least one VERIFICATION.md file alongside UAT results. Output envelope: `{ passed, uat_files[], verification_files[], checks[], blockers[], policy }`. Source: `gsd-core/bin/lib/uat-predicate.cjs` (generated from `src/uat-predicate.cts`). Wired via `phase uat-passed` alias → `phase-command-router` → `cmdPhaseUatPassed`. ### Probe Core Module -Generic spec-phase probe resolution model — the shared seam underlying spec-completeness probes (ADR-550 Decision 7). Owns the `status × verification` model (`status: resolved | dismissed | unresolved` × a per-probe `verification` tier), structural validation (`validateResolution`, `validateRequirement` — fail-closed: `verification` must be null unless status is `resolved`, and an out-of-enum status, a `dismissed`-without-`reason`, or an `unresolved` carrying a `resolution`/`reason`/tier payload all throw rather than silently miscount), the `analyzeCoverage(items, resolutions?, validators)` merge/rollup/orphan-reject pipeline, the `byVerification` per-tier rollup, and the `runProbeCli` I/O scaffold (parse → validate → analyze → emit, structurally guarding the report shape before write — a malformed report fails closed with stderr + exit 2 instead of stringifying as green). Adapter-agnostic: consumed by the Edge Probe Module today and the prohibition probe (#644) next. Exports: `VALID_STATUS`, `validateResolution`, `validateRequirement`, `analyzeCoverage`, `runProbeCli`. Source of truth: `gsd-core/bin/lib/probe-core.cjs` (generated from `src/probe-core.cts`, gitignored per ADR-457). Tests: `tests/probe-core.test.cjs`. See ADR-550 and Edge Probe Module. +Generic spec-phase probe resolution model — the shared seam underlying spec-completeness probes (ADR-550 Decision 7). Owns the `status × verification` model (`status: resolved | dismissed | unresolved` × a per-probe `verification` tier), structural validation (`validateResolution`, `validateRequirement` — fail-closed: `verification` must be null unless status is `resolved`, and an out-of-enum status, a `dismissed`-without-`reason`, or an `unresolved` carrying a `resolution`/`reason`/tier payload all throw rather than silently miscount), the `analyzeCoverage(items, resolutions?, validators)` merge/rollup/orphan-reject pipeline, the `byVerification` per-tier rollup, and the `runProbeCli` I/O scaffold (parse → validate → analyze → emit, structurally guarding the report shape before write — a malformed report fails closed with stderr + exit 2 instead of stringifying as green). Adapter-agnostic: consumed by the Edge Probe Module today and the prohibition probe (#644) next. Exports: `VALID_STATUS`, `validateResolution`, `validateRequirement`, `analyzeCoverage`, `runProbeCli`. Source of truth: `gsd-core/bin/lib/probe-core.cjs` (generated from `src/probe-core.cts`, gitignored per ADR-457). Tests: `tests/probe-core.test.cjs`. See ADR-550 and Edge Probe Module. Under ADR-857 (phase-6 boundary, settled 2026-06-12) this seam is classified **core verification substrate** on the *contract* side: its deterministic validators are the verifier↔predicate contract's CI-testable surface (ADR-550 Decision 5) — core and non-toggleable, never an off-by-default Feature Capability. (The recall-gapped *generator* is the probe adapters that propose predicates, not this resolution engine — see Edge Probe Module and Verification substrate (predicate boundary).) ### Edge Probe Module -First adapter of the Probe Core Module (ADR-550 Decision 7): the spec-phase edge-completeness probe wired into spec-phase Step 5.5. Owns shape classification (`classifyShape`), the applicable-category relevance filter (`applicableCategories` over the 8-category edge `TAXONOMY`), edge proposal (`proposeEdges`), and the `{explicit, backstop}` verification validators; delegates merge/rollup/CLI to `probe-core`. Fail-closed input contract: an edge requirement with missing/empty `text` and no `shapes` override is rejected (a `{ id }`-only requirement no longer classifies to zero edges and silently drops), while the legitimate `shapes: []` opt-out is preserved. Downstream, the `plan-phase` planner lifts every `covered`/`backstop` edge from the SPEC `## Edge Coverage` section into `must_haves.truths`. Exports (locked surface): `classifyShape`, `applicableCategories`, `proposeEdges`, `analyzeCoverage`, `validateResolution`, `validateRequirement`, plus the constants `TAXONOMY` (the 8 edge categories), `VALID_SHAPES`, `SHAPE_CUES`, and `EDGE_VALIDATORS` (the `{explicit, backstop}` validators bundle injected into `probe-core`'s generic engine). Source of truth: `gsd-core/bin/lib/edge-probe.cjs` (generated from `src/edge-probe.cts`, gitignored per ADR-457). Tests: `tests/edge-probe.test.cjs`, `tests/edge-probe-spec-phase-contract.test.cjs`, `tests/edge-probe-planner-contract.test.cjs`. See ADR-550 and Probe Core Module. +First adapter of the Probe Core Module (ADR-550 Decision 7): the spec-phase edge-completeness probe wired into spec-phase Step 5.5. Owns shape classification (`classifyShape`), the applicable-category relevance filter (`applicableCategories` over the 8-category edge `TAXONOMY`), edge proposal (`proposeEdges`), and the `{explicit, backstop}` verification validators; delegates merge/rollup/CLI to `probe-core`. Fail-closed input contract: an edge requirement with missing/empty `text` and no `shapes` override is rejected (a `{ id }`-only requirement no longer classifies to zero edges and silently drops), while the legitimate `shapes: []` opt-out is preserved. Downstream, the `plan-phase` planner lifts every `covered`/`backstop` edge from the SPEC `## Edge Coverage` section into `must_haves.truths`. Exports (locked surface): `classifyShape`, `applicableCategories`, `proposeEdges`, `analyzeCoverage`, `validateResolution`, `validateRequirement`, plus the constants `TAXONOMY` (the 8 edge categories), `VALID_SHAPES`, `SHAPE_CUES`, and `EDGE_VALIDATORS` (the `{explicit, backstop}` validators bundle injected into `probe-core`'s generic engine). Source of truth: `gsd-core/bin/lib/edge-probe.cjs` (generated from `src/edge-probe.cts`, gitignored per ADR-457). Tests: `tests/edge-probe.test.cjs`, `tests/edge-probe-spec-phase-contract.test.cjs`, `tests/edge-probe-planner-contract.test.cjs`. See ADR-550 and Probe Core Module. Per ADR-857's phase-6 boundary (2026-06-12) the predicates this module generates are **core verification substrate** (they set the verifier's reach), so it is wired onto the core predicate rail as a core-default module rather than migrated to an off-by-default `capabilities/edge-probe/` Feature Capability. + +### Verification substrate (predicate boundary) +The ADR-857 classification (settled 2026-06-12, prompted by @davesienkowski's boundary analysis on #857) that predicate-generation — the must-NOT-have / edge predicates that set the verifier's reach — is **core**, not an off-by-default Feature Capability. Load-bearing premise: *verifier reach = spec reach* (the verifier can only catch what the spec concretely names). Decomposes into: the **verifier↔predicate contract** (the verifier always expects predicates and grades **exogenously** against them — core, non-toggleable, a stability contract alongside the Loop Extension Point names), and the **generator** (the probe *adapters* that propose predicates — edge-probe's `classifyShape`/`proposeEdges`, the prohibition probe's adversarial LLM-propose — core-default but independently versionable, kept their own module because of a measured recall gap; `probe-core`'s deterministic validators sit on the contract side, not the generator side). Altitude rule distinguishing it from `gate` hooks: a gate runs *against* the spec (hook); predicate-generation defines the spec's *reach* (core). The decision-#6 `produces`/`consumes` artifact flow is the internal rail from generation to the core verifier. See ADR-857 *Verification substrate vs. plug-in tier (the predicate boundary)* and the ADR-550 cross-reference. ### MVP Mode Phase-level planning mode that frames work as a vertical slice (UI → API → DB) of one user-visible capability instead of horizontal layers. Resolved at workflow init via the precedence chain: `--mvp` CLI flag → ROADMAP.md `**Mode:** mvp` field → `workflow.mvp_mode` config → false. All-or-nothing per phase (PRD #2826 Q1). Surfaced as `MVP_MODE=true|false` to the planner, executor, verifier, and discovery surfaces (progress, stats, graphify). Canonical parser: `roadmap.cjs` `**Mode:**` field; canonical resolution chain documented in `workflows/plan-phase.md`. Concept index: `references/mvp-concepts.md`. diff --git a/docs/adr/550-spec-phase-probe-contract.md b/docs/adr/550-spec-phase-probe-contract.md index 422cda73c..16f9ba558 100644 --- a/docs/adr/550-spec-phase-probe-contract.md +++ b/docs/adr/550-spec-phase-probe-contract.md @@ -75,3 +75,12 @@ The values/safety class #644 targets (*"must not become a guilt mechanic"*) is f - **Docs debt:** `spec-phase` is undocumented in `FEATURES.md`/`COMMANDS.md` despite shipping in v1.38; the docs-required gate bites both probes. PR #584 establishes the spec-phase docs section. - **Glossary:** `probe family`, `probe-core`, `verification tier`, and `bespoke vs canon prohibition` are added to `CONTEXT.md` when #584 merges (not while the contract is pre-merge); this ADR is their interim home. - **Sequencing:** PR #584 lands ADR 550 + `probe-core` + the `edge-probe.cts` refactor + enum re-cut + fixture re-gen + the spec-phase docs section. #644 then adds the prohibition adapter (reference + `prohibitions:` block/parser/parity + tiered verification). #644 stays blocked until #584 merges. + +## Cross-reference: ADR-857 capability boundary (2026-06-12) + +The capability system (ADR-857) classifies **predicate-generation as core verification substrate, not an off-by-default Feature Capability** — settled before ADR-857's phase-6 (Migrate) freezes the core/plug-in line. This pins where the probe family sits relative to the loop: + +- The **verifier↔predicate contract** — the verifier always expects must-NOT-have / edge predicates and grades **exogenously** against them — is **core and non-toggleable**. *Verifier reach = spec reach:* the predicates **are** the reach of the spec the verifier verifies, so the contract cannot live in an off-by-default plug-in without making core reliability a function of an optional capability (ADR-857 decision #6 blast-radius rule). ADR-857's `produces`/`consumes` artifact flow (its decision #6) is the **internal rail** carrying predicates into the core verifier; in this ADR's terms that rail is the `SPEC.md` ↔ `must_haves` representation/projection (Decision 3, round-tripped per Decision 5). +- **"Exogenous grading"** is the verify-time half of this ADR's **judgment-tier** (Decision 4): the autonomous verifier records an LLM-judge verdict **marked non-authoritative** and flags *"unverified prohibition — human review recommended"* — grading against the spec's explicit predicates, never a self-graded restatement (the prose-drift / self-graded-review failure modes reproduced on #664). The judgment-tier soft-gate **is** the contract's "never a silent pass" guarantee at the core boundary. +- **The probe *adapters* are the core-default *generator*** — `edge-probe`'s `classifyShape`/`proposeEdges` and the prohibition probe's adversarial LLM-propose, the surfaces that *propose* predicates — default-on and non-removable, but **independently versionable**. The classifier's measured recall gap (the prose→shape under-fire on terse prose) is the reason they stay their own modules under this ADR rather than being folded into the slow core rail. **`probe-core` is not the generator:** per Decision 7b it ingests already-proposed items, and its deterministic validators are the **contract**'s CI-testable surface (Decision 5) — so `probe-core` sits on the contract side, the adapters on the generator side. ADR-857 phase 6 wires these modules onto the core predicate rail; it does **not** relabel them as a `capabilities/edge-probe/` plug-in. +- Decision 5's rule — *the CI-testable surface is the contract, not the classifier* — extends to ADR-857's core rail: the deterministic conformance test is the contract shape the verifier consumes, never the LLM's judgment. diff --git a/docs/adr/857-capability-system.md b/docs/adr/857-capability-system.md index 4d58a933d..ed91a2763 100644 --- a/docs/adr/857-capability-system.md +++ b/docs/adr/857-capability-system.md @@ -5,6 +5,7 @@ - **Issue:** #857 - **Supersedes (generalizes):** Skill Surface Budget Module (ADR-0011), Runtime Install Policy Module (ADR-0058) - **Builds on:** CommandRoutingHub (ADR-0012), Runtime Artifact Layout Module (ADR-3660), generated-cjs single source (ADR-457) +- **Amended:** 2026-06-12 — phase-6 boundary settled before the Migrate phase freezes it: the **verifier↔predicate contract** is classified as core verification substrate (not an off-by-default Feature Capability). See *Verification substrate vs. plug-in tier (the predicate boundary)* below. Prompted by @davesienkowski's boundary analysis on #857; coordinates with ADR-550 (spec-phase probe contract). ## Context @@ -21,7 +22,7 @@ The healthier news from the architecture review: the lower seams are already in ## Decision -Introduce a **Capability** system. The five-step loop plus shared-infrastructure skills (`phase`, `config`, `help`, `update`, `surface`, `progress`) are the **privileged host/core**. Every other feature is a **Capability** — a plug-in selectable at install and toggleable after restart. +Introduce a **Capability** system. The five-step loop plus shared-infrastructure skills (`phase`, `config`, `help`, `update`, `surface`, `progress`) are the **privileged host/core**. Every other feature is a **Capability** — a plug-in selectable at install and toggleable after restart. **One settled exception** (2026-06-12): the verifier↔predicate contract is **core verification substrate**, not a Capability — the predicates that set the verifier's reach cannot live in an off-by-default plug-in. See *Verification substrate vs. plug-in tier (the predicate boundary)*. The design was resolved across seven decisions: @@ -49,7 +50,22 @@ These were grilled to resolution after the initial eight decisions. ### Loop Extension Points (the 12) -`discuss:pre`, `discuss:post`, `plan:pre`, `plan:post`, `execute:pre`, `execute:wave:pre`, `execute:wave:post`, `execute:post`, `verify:pre`, `verify:post`, `ship:pre`, `ship:post`. The planner/checker loop, the verifier, and the verify-work gap-closure loop remain **core** (not hooks). Today's `§`-point features map on as: research / ui-spec / ai-spec / pattern-mapper (`step`) and security / schema-gate / tdd (`contribution`) at `plan:pre`; nyquist / gap-analysis (`gate`) at `plan:post`; build+test / code-review / drift (`gate`/`step`) at `execute:wave:post`; `verification.status` preflight (`gate`) at `ship:pre`; PR-body sections (`contribution`) at `ship:post`. The names are a stability contract — additive-only across versions. +`discuss:pre`, `discuss:post`, `plan:pre`, `plan:post`, `execute:pre`, `execute:wave:pre`, `execute:wave:post`, `execute:post`, `verify:pre`, `verify:post`, `ship:pre`, `ship:post`. The planner/checker loop, the verifier, the verify-work gap-closure loop, **and the verifier↔predicate contract** (the spec-reach substrate — see *Verification substrate vs. plug-in tier* below) remain **core** (not hooks). Today's `§`-point features map on as: research / ui-spec / ai-spec / pattern-mapper (`step`) and security / schema-gate / tdd (`contribution`) at `plan:pre`; nyquist / gap-analysis (`gate`) at `plan:post`; build+test / code-review / drift (`gate`/`step`) at `execute:wave:post`; `verification.status` preflight (`gate`) at `ship:pre`; PR-body sections (`contribution`) at `ship:post`. The names are a stability contract — additive-only across versions. + +### Verification substrate vs. plug-in tier (the predicate boundary) + +Settled before phase 6 (Migrate) freezes the core/plug-in line. **The probe family that generates must-NOT-have and edge predicates is core verification substrate, not an off-by-default Feature Capability** — split into a non-toggleable contract and a core-default generator. + +**The load-bearing finding:** *verifier reach = spec reach.* The verifier can only catch what the spec concretely names; the must-NOT-have / edge predicates **are** the reach of the spec the verifier verifies. Placing the verifier in core (already exempted above) while leaving the input that sets its reach in an off-by-default Capability would make the core's reliability a function of an optional plug-in — the exact blast-radius leak decision #6 exists to prevent. (The live case: prose-drift and a self-graded review rationalizing a defect away — both reproduced on #664 — are why grading must be **exogenous**, i.e. against externally-supplied predicates rather than the verifier's own restated understanding.) + +**Altitude rule (where the line falls).** A `gate` runs *against* the spec at a point and may block (hook). Predicate-generation defines *what the verifier is allowed to see* — it is upstream of and constitutive of verification, not a check within it. So: **gates run against the spec (hook); predicate-generation defines the spec's reach (core).** This is why nyquist / gap-analysis remain `gate` hooks while the predicate contract does not. + +**Decomposition** (keeps the fallible part out of the core blast radius without making "off" silently shrink the verifier's reach): + +- **Core, non-negotiable — the contract.** The verifier always expects must-NOT-have predicates and grades **exogenously** against them. This substrate is **not toggleable**; no `capabilities/edge-probe/` Feature Capability may remove it. The decision-#6 `produces`/`consumes` wire is the **internal rail** from predicate-generation to the core verifier (file-artifact data flow surviving `/clear`), not a Loop Extension Point a plug-in can detach. The contract is a **stability contract** alongside the Loop Extension Point names (Hyrum's Law: once relied on, it is a depended-upon interface — name it and keep it compatible). +- **Core-default but independently versionable — the generator.** The **probe adapters** — the taxonomy + classifier that *propose* predicates (edge-probe's `classifyShape`/`proposeEdges`, the prohibition probe's adversarial LLM-propose), governed by ADR-550 — are the generator. They keep their own module precisely because the classifier has a measured recall gap (the prose→shape classifier under-fires on terse prose without erroring) and must keep improving without churning the contract. The generator is **default-on and non-removable**, but versioned separately (Gall's Law: the minimal core rail evolves slowly; the complex fallible generator evolves on its own cadence). Its fallibility is contained the same way the federated-config merge is — a generator miss is a recall gap to improve, never a core-load break. (Note: ADR-550's `probe-core` deterministic resolution/validation engine is *not* the generator — per Decision 7b it ingests already-proposed items; its validators are the **contract**'s CI-testable surface, Decision 5.) + +The ownership seam is clean (Conway's Law): the **contract** is owned by the core verifier plus `probe-core`'s deterministic validation/rollup engine; the **generator** is owned by the probe adapters; the `produces`/`consumes` artifact rail (decision #6) joins them. Consequently, **phase 6 does not migrate predicate-generation to an off-by-default Capability** — it wires the existing edge-probe/prohibition-probe modules onto the core predicate rail as core-default substrate. (Attribution: boundary analysis by @davesienkowski on #857, accepted by the maintainer; cross-referenced from ADR-550.) ### Contribution merge @@ -87,6 +103,8 @@ Made light by the closed vocabulary: (1) validate the descriptor against its JSO | Runtime interface | Code adapter, third-party loadable | Ships the trust/load/security surface prematurely; not needed at launch | | Runtime interface | Code adapter, first-party only | Forces a later retrofit to a descriptor format — the exact rework ADR-857 is unwinding for features | | Runtime scope | Drop the 12 non-tier-1 runtimes | Regresses working runtime support for current users | +| Predicate boundary | Predicate-generation as an off-by-default `capabilities/edge-probe/` Feature Capability | Makes the core verifier's reach (and thus its reliability) a function of an optional plug-in — the blast-radius leak decision #6 forbids; *verifier reach = spec reach* | +| Predicate boundary | Promote the whole probe (taxonomy + classifier) into core wholesale | The classifier has a measured recall gap and must keep improving; folding it into the slow core rail grows core complexity and couples contract churn to generator iteration (Gall's Law) — decompose into core contract + core-default generator instead | ## Consequences @@ -117,7 +135,7 @@ Phased; `next` stays green at each step. (Maps to the candidate sequence from th 3. **Define** — land the Capability Registry generation, the federated config loader, and the Loop Extension Point resolver (`loop.render-hooks`, extending `init.*`). Define the ~12 stable points. 4. **Wire** — collapse `.gsd-profile` + `.gsd-surface.json` + `config.json workflow.*` into one resolved capability state; open the `gsd-tools.cjs:runCommand` entrypoint (registry) so first-party code modules register as Capabilities. 5. **Runtime seam** — finish the InstallPlan adapter registry (ADR-0058) as a declarative descriptor over a primitive vocabulary; re-author the 15 runtimes as descriptors (tier-1: Claude/Codex/Antigravity); registry loads in-tree descriptors only (third-party loader deferred). -6. **Migrate** — convert existing optional features (UI, AI/eval, research, security, nyquist, code-review, graphify, …) to Capabilities; shrink the loop workflow bodies. +6. **Migrate** — convert existing optional features (UI, AI/eval, research, security, nyquist, code-review, graphify, …) to Capabilities; shrink the loop workflow bodies. **Exception (settled 2026-06-12):** the edge-probe / prohibition-probe predicate-generation (ADR-550) is **not** migrated to an off-by-default Feature Capability — the verifier↔predicate contract is core substrate and the generator is a core-default module (see *Verification substrate vs. plug-in tier*). Phase 6 wires these modules onto the core predicate rail rather than relabeling them as plug-ins. (#999's Impeccable migration is unaffected — it is a genuine Feature Capability.) Each phase is its own `approved-*` issue under #857 (an approved epic does not approve its children). @@ -125,3 +143,5 @@ Each phase is its own `approved-*` issue under #857 (an approved epic does not a - Migration ordering among features with cross-dependencies (e.g. UI-spec → plan, code-review → execute) under the default-resilient failure model. - Whether tier-1 (Claude Code / Codex / Antigravity) implies an automated cross-runtime test matrix as a merge gate. +- ~~Whether predicate-generation (edge/prohibition probes) is core substrate or a Feature Capability~~ — **resolved 2026-06-12**: core substrate (contract) + core-default generator; not an off-by-default Capability. See *Verification substrate vs. plug-in tier*. +- Whether the verifier↔predicate **contract** warrants a deterministic CI conformance test (a core-default generator producing a contract-shaped predicate set the verifier consumes), extending ADR-550 Decision 5's "test the contract, not the classifier" rule to the core rail — likely yes; deferred to the phase-3/phase-6 implementation issue.