From 3556450b0de850f181b3f43b4f8d40e5e535a37c Mon Sep 17 00:00:00 2001 From: Rezolv Date: Sun, 14 Jun 2026 21:44:21 -0400 Subject: [PATCH] feat(spec-phase): surface zero-classification edge-probe requirements as unclassified candidates (#1110) (#1117) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Edge-probe now surfaces a zero-classification requirement (non-empty prose, no shape cue matched, no `shapes` override) as a single soft `unclassified — review manually` candidate instead of silently dropping it — the exact blind spot the probe exists to catch. Dismissible like any edge; the `shapes: []` opt-out stays silent; `TAXONOMY` (the closed 8 categories) is unchanged. Under `--auto` the candidate is left `unresolved`, never auto-`backstop` (a missing shape is not evidence an edge exists). Closes #1110 --- .changeset/serene-mice-swim.md | 5 ++ CONTEXT.md | 2 +- docs/FEATURES.md | 5 +- docs/how-to/resolve-edge-coverage-findings.md | 2 + gsd-core/references/edge-probe.md | 11 ++++ gsd-core/workflows/spec-phase.md | 11 ++++ src/edge-probe.cts | 26 +++++++++- tests/edge-probe.test.cjs | 51 +++++++++++++++++++ tests/workflow-size-baseline.json | 2 +- 9 files changed, 111 insertions(+), 4 deletions(-) create mode 100644 .changeset/serene-mice-swim.md diff --git a/.changeset/serene-mice-swim.md b/.changeset/serene-mice-swim.md new file mode 100644 index 000000000..4e70dc202 --- /dev/null +++ b/.changeset/serene-mice-swim.md @@ -0,0 +1,5 @@ +--- +type: Changed +pr: 1117 +--- +Edge-probe now surfaces a zero-classification requirement (non-empty prose, no shape cue matched, no `shapes` override) as a single soft `unclassified — review manually` candidate instead of silently dropping it. Dismissible like any edge; `shapes: []` opt-out stays silent; TAXONOMY unchanged. diff --git a/CONTEXT.md b/CONTEXT.md index 95d692bd4..27eb503d2 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -238,7 +238,7 @@ Runtime-neutral predicate evaluating `*-UAT.md` / `*-VERIFICATION.md` result fie Generic spec-phase probe resolution model — the shared seam underlying spec-completeness probes (ADR-550 Decision 7). Owns the `status × verification` model (`status: resolved | dismissed | unresolved` × a per-probe `verification` tier), structural validation (`validateResolution`, `validateRequirement` — fail-closed: `verification` must be null unless status is `resolved`, and an out-of-enum status, a `dismissed`-without-`reason`, or an `unresolved` carrying a `resolution`/`reason`/tier payload all throw rather than silently miscount), the `analyzeCoverage(items, resolutions?, validators)` merge/rollup/orphan-reject pipeline, the `byVerification` per-tier rollup, and the `runProbeCli` I/O scaffold (parse → validate → analyze → emit, structurally guarding the report shape before write — a malformed report fails closed with stderr + exit 2 instead of stringifying as green). Adapter-agnostic: consumed by the Edge Probe Module today and the Prohibition Probe Module (#644) next. Exports (generic surface): `VALID_STATUS`, `validateResolution`, `validateRequirement`, `analyzeCoverage`, `runProbeCli` — the prohibition adapter exports that also ship from this module (`projectProhibitions`, `PROHIBITION_VALIDATORS`, `validateProhibitionResolution`, `dispositionForProhibition`) are documented under the Prohibition Probe Module's own locked-surface line. Source of truth: `gsd-core/bin/lib/probe-core.cjs` (generated from `src/probe-core.cts`, gitignored per ADR-457). Tests: `tests/probe-core.test.cjs`. See ADR-550 and Edge Probe Module. Under ADR-857 (phase-6 boundary, settled 2026-06-12) this seam is classified **core verification substrate** on the *contract* side: its deterministic validators are the verifier↔predicate contract's CI-testable surface (ADR-550 Decision 5) — core and non-toggleable, never an off-by-default Feature Capability. (The recall-gapped *generator* is the probe adapters that propose predicates, not this resolution engine — see Edge Probe Module and Verification substrate (predicate boundary).) ### Edge Probe Module -First adapter of the Probe Core Module (ADR-550 Decision 7): the spec-phase edge-completeness probe wired into spec-phase Step 5.5. Owns shape classification (`classifyShape`), the applicable-category relevance filter (`applicableCategories` over the 8-category edge `TAXONOMY`), edge proposal (`proposeEdges`), and the `{explicit, backstop}` verification validators; delegates merge/rollup/CLI to `probe-core`. Fail-closed input contract: an edge requirement with missing/empty `text` and no `shapes` override is rejected (a `{ id }`-only requirement no longer classifies to zero edges and silently drops), while the legitimate `shapes: []` opt-out is preserved. Downstream, the `plan-phase` planner lifts every `covered`/`backstop` edge from the SPEC `## Edge Coverage` section into `must_haves.truths`. Exports (locked surface): `classifyShape`, `applicableCategories`, `proposeEdges`, `analyzeCoverage`, `validateResolution`, `validateRequirement`, plus the constants `TAXONOMY` (the 8 edge categories), `VALID_SHAPES`, `SHAPE_CUES`, and `EDGE_VALIDATORS` (the `{explicit, backstop}` validators bundle injected into `probe-core`'s generic engine). Source of truth: `gsd-core/bin/lib/edge-probe.cjs` (generated from `src/edge-probe.cts`, gitignored per ADR-457). Tests: `tests/edge-probe.test.cjs`, `tests/edge-probe-spec-phase-contract.test.cjs`, `tests/edge-probe-planner-contract.test.cjs`. See ADR-550 and Probe Core Module. Per ADR-857's phase-6 boundary (2026-06-12) the predicates this module generates are **core verification substrate** (they set the verifier's reach), so it is wired onto the core predicate rail as a core-default module rather than migrated to an off-by-default `capabilities/edge-probe/` Feature Capability. +First adapter of the Probe Core Module (ADR-550 Decision 7): the spec-phase edge-completeness probe wired into spec-phase Step 5.5. Owns shape classification (`classifyShape`), the applicable-category relevance filter (`applicableCategories` over the 8-category edge `TAXONOMY`), edge proposal (`proposeEdges`), and the `{explicit, backstop}` verification validators; delegates merge/rollup/CLI to `probe-core`. Fail-closed input contract: an edge requirement with missing/empty `text` and no `shapes` override is rejected (a `{ id }`-only requirement no longer classifies to zero edges and silently drops), while the legitimate `shapes: []` opt-out is preserved. Downstream, the `plan-phase` planner lifts every `covered`/`backstop` edge from the SPEC `## Edge Coverage` section into `must_haves.truths`. Exports (locked surface): `classifyShape`, `applicableCategories`, `proposeEdges`, `analyzeCoverage`, `validateResolution`, `validateRequirement`, plus the constants `TAXONOMY` (the **closed** 8 edge categories), `UNCLASSIFIED_CATEGORY` (the `unclassified` review-manually sentinel for zero-cue prose, #1110 — deliberately **not** a 9th taxonomy entry: it stays out of `TAXONOMY` but is present in `EDGE_VALIDATORS.categories`), `VALID_SHAPES`, `SHAPE_CUES`, and `EDGE_VALIDATORS` (the `{explicit, backstop}` validators bundle injected into `probe-core`'s generic engine; its `categories` = the `TAXONOMY` ids plus `UNCLASSIFIED_CATEGORY`). Source of truth: `gsd-core/bin/lib/edge-probe.cjs` (generated from `src/edge-probe.cts`, gitignored per ADR-457). Tests: `tests/edge-probe.test.cjs`, `tests/edge-probe-spec-phase-contract.test.cjs`, `tests/edge-probe-planner-contract.test.cjs`. See ADR-550 and Probe Core Module. Per ADR-857's phase-6 boundary (2026-06-12) the predicates this module generates are **core verification substrate** (they set the verifier's reach), so it is wired onto the core predicate rail as a core-default module rather than migrated to an off-by-default `capabilities/edge-probe/` Feature Capability. ### Verification substrate (predicate boundary) The ADR-857 classification (settled 2026-06-12, prompted by @davesienkowski's boundary analysis on #857) that predicate-generation — the must-NOT-have / edge predicates that set the verifier's reach — is **core**, not an off-by-default Feature Capability. Load-bearing premise: *verifier reach = spec reach* (the verifier can only catch what the spec concretely names). Decomposes into: the **verifier↔predicate contract** (the verifier always expects predicates and grades **exogenously** against them — core, non-toggleable, a stability contract alongside the Loop Extension Point names), and the **generator** (the probe *adapters* that propose predicates — edge-probe's `classifyShape`/`proposeEdges`, the prohibition probe's adversarial LLM-propose — core-default but independently versionable, kept their own module because of a measured recall gap; `probe-core`'s deterministic validators sit on the contract side, not the generator side). Altitude rule distinguishing it from `gate` hooks: a gate runs *against* the spec (hook); predicate-generation defines the spec's *reach* (core). The decision-#6 `produces`/`consumes` artifact flow is the internal rail from generation to the core verifier. See ADR-857 *Verification substrate vs. plug-in tier (the predicate boundary)* and the ADR-550 cross-reference. diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 0797f176f..8f6e038c0 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -3109,7 +3109,9 @@ explicit reviewer flags -> --all -> review.default_reviewers -> all detected rev | `backstop` | Intent recorded, needs a held-out/property-based test | Lifted into `must_haves.truths` as a non-inferable check | | `unresolved` | Deferred | Soft-gates the spec; row stamped `⚠ Edge unresolved — planner must treat as assumption` | -The resolved edges populate a `## Edge Coverage` section in `SPEC.md`. Unresolved *applicable* edges trigger a soft gate (Resolve / Write-anyway-flagged / Keep-probing) rather than a hard block. Under `--auto`, the probe **never auto-dismisses** — it auto-covers where a defensible criterion exists, otherwise auto-backstops, and logs `[auto] edge coverage: C covered, B backstop, U unresolved`. +When a requirement's prose matches **no** shape cue, the probe does not silently drop it (#1110): it emits a single `unclassified — review manually` candidate so the zero-cue requirement is surfaced for the author to resolve like any other (specify / dismiss-with-reason / defer) — a manual-review nudge, not a hard block. + +The resolved edges populate a `## Edge Coverage` section in `SPEC.md`. Unresolved *applicable* edges trigger a soft gate (Resolve / Write-anyway-flagged / Keep-probing) rather than a hard block. Under `--auto`, the probe **never auto-dismisses** — it auto-covers where a defensible criterion exists, otherwise auto-backstops, and logs `[auto] edge coverage: C covered, B backstop, U unresolved`. The one exception is an `unclassified` candidate: `--auto` leaves it **`unresolved`** (surfaced as a flagged assumption), never auto-`backstop` — a missing shape is not evidence an edge exists, so minting a held-out edge obligation would be a false claim. The load-bearing wire is the `plan-phase` lift: `covered` and `backstop` edges become `must_haves.truths` the verifier can check, so the section is not merely documentation. @@ -3120,6 +3122,7 @@ The load-bearing wire is the `plan-phase` lift: `covered` and `backstop` edges b - REQ-EDGE-04: An unresolved applicable edge MUST trigger the soft gate; write-anyway stamps the row as a planner assumption. - REQ-EDGE-05: `--auto` MUST never auto-dismiss — auto-cover or auto-backstop only. - REQ-EDGE-06: `plan-phase` MUST lift `covered` criteria and `backstop` notes into `must_haves.truths`. +- REQ-EDGE-07: A requirement whose prose matches no shape cue MUST surface an `unclassified — review manually` candidate (never silently dropped); `--auto` MUST leave it `unresolved`, never auto-`backstop`. **Reference:** [Edge Probe](../gsd-core/references/edge-probe.md) diff --git a/docs/how-to/resolve-edge-coverage-findings.md b/docs/how-to/resolve-edge-coverage-findings.md index 8dac0bdc3..9955ea90f 100644 --- a/docs/how-to/resolve-edge-coverage-findings.md +++ b/docs/how-to/resolve-edge-coverage-findings.md @@ -16,6 +16,8 @@ Each finding is one **applicable** edge for one requirement — a boundary the p You must resolve each finding into exactly one of four states. Claude presents them as a numbered choice (or an `AskUserQuestion` menu). +You may also see an **`unclassified — review manually`** finding. The relevance filter is a heuristic over prose cues, so a requirement whose wording is edge-relevant but matched no cue surfaces this single soft candidate instead of being silently dropped. Resolve it like any other finding — specify a criterion, or **dismiss it with a reason** if the requirement is genuinely edge-free (e.g. "static asset, no input"). It is a manual-review nudge, never a hard block. + --- ## Specify it — write an acceptance criterion diff --git a/gsd-core/references/edge-probe.md b/gsd-core/references/edge-probe.md index 2cfb2de70..ae6171591 100644 --- a/gsd-core/references/edge-probe.md +++ b/gsd-core/references/edge-probe.md @@ -77,6 +77,17 @@ Two rules keep the probe honest and prevent an "everything is N/A" failure mode: 2. **Dismissal requires a reason string.** "N/A — input is a bounded enum, no boundary exists" is valid; silence is not. The reason string is the audit trail. +**Zero-classification surfaces an `unclassified` candidate (#1110).** The relevance filter is +a heuristic over prose cues, so a requirement whose wording *is* edge-relevant but matches no +shape cue would otherwise classify to zero shapes → zero edges and vanish from coverage with +no signal — the same silent blind spot the probe exists to catch. Instead, a requirement with +non-empty prose, no authored `shapes`, and zero matched shapes surfaces exactly one soft +`unclassified — review manually` candidate (`category: "unclassified"`, `status: "unresolved"`). +It is a dismissible nudge — resolve it, or dismiss it with a reason (e.g. a genuinely edge-free +static-asset requirement) — never a hard block. `unclassified` is a review signal, **not** a +ninth taxonomy category: the closed eight above are unchanged, and an explicit `shapes: []` +opt-out stays silent (the author's deliberate "no edge surface"). + Each raised edge carries two orthogonal axes — a resolution **lifecycle** and, when resolved, a **verification** tier (ADR-550 Decision 7, the shared probe-core model): diff --git a/gsd-core/workflows/spec-phase.md b/gsd-core/workflows/spec-phase.md index 16266618a..9a93c6758 100644 --- a/gsd-core/workflows/spec-phase.md +++ b/gsd-core/workflows/spec-phase.md @@ -295,6 +295,9 @@ For each Requirement gathered so far: - **Dismiss (reason)** → mark `dismissed` with a required non-empty reason. - **Backstop with a test** → mark `backstop`; note "held-out edge test" for plan-phase. - **Defer** → leave `unresolved`. + - An `unclassified` row (probe `unclassified — review manually`) means the requirement's + prose matched no shape cue (#1110) — treat it like any other candidate (**Specify**, + **Dismiss (reason)**, or **Defer**). A manual-review nudge, not a hard block. **Soft gate (after resolving):** - All applicable edges resolved → proceed to Step 6. @@ -310,6 +313,14 @@ For each Requirement gathered so far: otherwise auto-`backstop` (never auto-dismiss — a wrong dismissal is the exact silent failure being eliminated). Log: `[auto] edge coverage: C covered, B backstop, U unresolved`. +**`unclassified` exception (#1110):** `--auto` leaves an `unclassified` candidate +**`unresolved`** (the soft gate surfaces it as a flagged planner assumption) — it never +auto-`backstop`s it. A missing shape is not evidence an edge exists, so minting a held-out +edge obligation on a requirement that may be genuinely edge-free would be a false claim and +risks a vacuous edge test. Leaving it `unresolved` keeps the zero-cue requirement visible +(never a silent drop) without fabricating an edge — which is exactly #1110's purpose: surface +it for review, do not auto-handle it. + Populate the `## Edge Coverage` section of SPEC.md from the resolved edges. ## Step 5.6: Prohibition-Completeness Probe (must-NOT) diff --git a/src/edge-probe.cts b/src/edge-probe.cts index 2a4ec793a..8f6df73c1 100644 --- a/src/edge-probe.cts +++ b/src/edge-probe.cts @@ -104,8 +104,16 @@ export function applicableCategories(shapes: Shape[]): string[] { * taxonomy; both verification tiers require a non-empty `resolution` (an explicit AC's text * or a backstop note) so plan-phase has a criterion to lift. */ +/** + * Pseudo-category for a requirement whose prose matched NO shape cue (#1110). It is a soft + * "review manually" signal, NOT a 9th taxonomy category: it stays out of `TAXONOMY` (the closed + * eight) and only joins `EDGE_VALIDATORS.categories` so `analyzeCoverage` accepts the item. + */ +export const UNCLASSIFIED_CATEGORY = 'unclassified'; +const UNCLASSIFIED_PROBE = 'unclassified — review manually'; + export const EDGE_VALIDATORS: Validators = { - categories: TAXONOMY.map((c) => c.id), + categories: [...TAXONOMY.map((c) => c.id), UNCLASSIFIED_CATEGORY], verification: ['explicit', 'backstop'], requiredFieldsByVerification: { explicit: ['resolution'], backstop: ['resolution'] }, }; @@ -162,6 +170,22 @@ export function proposeEdges(requirement: Requirement): Edge[] { shapes = requirement.shapes; } else { shapes = classifyShape(requirement.text); + if (shapes.length === 0) { + // Prose present but no shape cue matched. Do NOT silently drop it (#1110): an + // edge-relevant requirement whose phrasing missed every cue would otherwise vanish from + // coverage with no signal — the exact blind spot this probe exists to catch. Surface ONE + // soft, dismissible "unclassified — review manually" candidate. The explicit `shapes: []` + // opt-out (handled above) stays silent — that is the author's deliberate "no edge surface". + return [{ + requirement_id: requirement.id, + category: UNCLASSIFIED_CATEGORY, + status: 'unresolved', + verification: null, + resolution: null, + reason: null, + probe: UNCLASSIFIED_PROBE, + }]; + } } return applicableCategories(shapes).map((catId): Edge => { const cat = TAXONOMY.find((c) => c.id === catId); diff --git a/tests/edge-probe.test.cjs b/tests/edge-probe.test.cjs index 2436bc1c0..660ef4ec9 100644 --- a/tests/edge-probe.test.cjs +++ b/tests/edge-probe.test.cjs @@ -219,6 +219,57 @@ describe('edge-probe: proposeEdges — empty-shapes override (RR-06)', () => { }); }); +describe('edge-probe: proposeEdges — unclassified candidate for prose-zero-cue (#1110)', () => { + // A requirement with non-empty prose that matches NO shape cue must not be silently + // dropped (zero edges, no signal). It now surfaces ONE soft "unclassified — review + // manually" candidate instead. The explicit `shapes: []` opt-out stays silent. + test('prose with no shape cue surfaces a single unclassified candidate', () => { + const edges = ep.proposeEdges({ id: 'R1', text: 'Display the company logo' }); + assert.equal(edges.length, 1, 'prose-zero-cue must surface exactly one unclassified candidate'); + assert.deepEqual(edges[0], { + requirement_id: 'R1', + category: 'unclassified', + status: 'unresolved', + verification: null, + resolution: null, + reason: null, + probe: 'unclassified — review manually', + }); + }); + + test('UNCLASSIFIED_CATEGORY is a valid item category but NOT a taxonomy category', () => { + assert.equal(ep.UNCLASSIFIED_CATEGORY, 'unclassified'); + assert.ok(ep.EDGE_VALIDATORS.categories.includes('unclassified'), 'analyzeCoverage must accept the unclassified item category'); + assert.ok(!ep.TAXONOMY.some((c) => c.id === 'unclassified'), 'unclassified must NOT pollute the closed 8-category taxonomy'); + }); + + test('explicit shapes: [] stays silent (deliberate opt-out — NOT unclassified)', () => { + const edges = ep.proposeEdges({ id: 'R1', text: 'Display the company logo', shapes: [] }); + assert.deepEqual(edges, []); + }); + + test('prose that DOES classify proposes real edges, never an unclassified candidate', () => { + const edges = ep.proposeEdges({ id: 'R1', text: 'Round a number to N decimal places' }); + assert.ok(edges.length > 0); + assert.ok(!edges.some((e) => e.category === 'unclassified'), 'a classifiable requirement must not emit unclassified'); + }); + + test('analyzeCoverage surfaces the unclassified candidate as unresolved (no throw)', () => { + const report = ep.analyzeCoverage([{ id: 'R1', text: 'Display the company logo' }]); + assert.equal(report.coverage.applicable, 1); + assert.equal(report.coverage.unresolved, 1); + assert.equal(report.items[0].category, 'unclassified'); + }); + + test('an unclassified candidate can be dismissed with a reason (edge-probe parity)', () => { + const report = ep.analyzeCoverage( + [{ id: 'R1', text: 'Display the company logo' }], + [{ requirement_id: 'R1', category: 'unclassified', status: 'dismissed', reason: 'genuinely edge-free — static asset' }], + ); + assert.equal(report.items[0].status, 'dismissed'); + }); +}); + describe('edge-probe: proposeEdges — invalid authored shapes fail closed (re-review #3 High)', () => { // A non-empty but INVALID shapes array must NOT silently suppress every probe. // shapes:['numeric'] (typo for the locked 'numeric-range') previously passed diff --git a/tests/workflow-size-baseline.json b/tests/workflow-size-baseline.json index e445abe80..21ef79696 100644 --- a/tests/workflow-size-baseline.json +++ b/tests/workflow-size-baseline.json @@ -72,7 +72,7 @@ "ship.md": 24388, "sketch-wrap-up.md": 14223, "sketch.md": 19960, - "spec-phase.md": 27589, + "spec-phase.md": 28438, "spike-wrap-up.md": 15092, "spike.md": 24517, "stats.md": 6718,