feat(spec-phase): surface zero-classification edge-probe requirements as unclassified candidates (#1110) (#1117)

Edge-probe now surfaces a zero-classification requirement (non-empty prose, no
shape cue matched, no `shapes` override) as a single soft `unclassified — review
manually` candidate instead of silently dropping it — the exact blind spot the
probe exists to catch. Dismissible like any edge; the `shapes: []` opt-out stays
silent; `TAXONOMY` (the closed 8 categories) is unchanged. Under `--auto` the
candidate is left `unresolved`, never auto-`backstop` (a missing shape is not
evidence an edge exists).

Closes #1110
This commit is contained in:
Rezolv
2026-06-14 21:44:21 -04:00
committed by GitHub
parent 395fb519e7
commit 3556450b0d
9 changed files with 111 additions and 4 deletions

View File

@@ -0,0 +1,5 @@
---
type: Changed
pr: 1117
---
Edge-probe now surfaces a zero-classification requirement (non-empty prose, no shape cue matched, no `shapes` override) as a single soft `unclassified — review manually` candidate instead of silently dropping it. Dismissible like any edge; `shapes: []` opt-out stays silent; TAXONOMY unchanged.

View File

@@ -238,7 +238,7 @@ Runtime-neutral predicate evaluating `*-UAT.md` / `*-VERIFICATION.md` result fie
Generic spec-phase probe resolution model — the shared seam underlying spec-completeness probes (ADR-550 Decision 7). Owns the `status × verification` model (`status: resolved | dismissed | unresolved` × a per-probe `verification` tier), structural validation (`validateResolution`, `validateRequirement` — fail-closed: `verification` must be null unless status is `resolved`, and an out-of-enum status, a `dismissed`-without-`reason`, or an `unresolved` carrying a `resolution`/`reason`/tier payload all throw rather than silently miscount), the `analyzeCoverage(items, resolutions?, validators)` merge/rollup/orphan-reject pipeline, the `byVerification` per-tier rollup, and the `runProbeCli` I/O scaffold (parse → validate → analyze → emit, structurally guarding the report shape before write — a malformed report fails closed with stderr + exit 2 instead of stringifying as green). Adapter-agnostic: consumed by the Edge Probe Module today and the Prohibition Probe Module (#644) next. Exports (generic surface): `VALID_STATUS`, `validateResolution`, `validateRequirement`, `analyzeCoverage`, `runProbeCli` — the prohibition adapter exports that also ship from this module (`projectProhibitions`, `PROHIBITION_VALIDATORS`, `validateProhibitionResolution`, `dispositionForProhibition`) are documented under the Prohibition Probe Module's own locked-surface line. Source of truth: `gsd-core/bin/lib/probe-core.cjs` (generated from `src/probe-core.cts`, gitignored per ADR-457). Tests: `tests/probe-core.test.cjs`. See ADR-550 and Edge Probe Module. Under ADR-857 (phase-6 boundary, settled 2026-06-12) this seam is classified **core verification substrate** on the *contract* side: its deterministic validators are the verifier↔predicate contract's CI-testable surface (ADR-550 Decision 5) — core and non-toggleable, never an off-by-default Feature Capability. (The recall-gapped *generator* is the probe adapters that propose predicates, not this resolution engine — see Edge Probe Module and Verification substrate (predicate boundary).)
### Edge Probe Module
First adapter of the Probe Core Module (ADR-550 Decision 7): the spec-phase edge-completeness probe wired into spec-phase Step 5.5. Owns shape classification (`classifyShape`), the applicable-category relevance filter (`applicableCategories` over the 8-category edge `TAXONOMY`), edge proposal (`proposeEdges`), and the `{explicit, backstop}` verification validators; delegates merge/rollup/CLI to `probe-core`. Fail-closed input contract: an edge requirement with missing/empty `text` and no `shapes` override is rejected (a `{ id }`-only requirement no longer classifies to zero edges and silently drops), while the legitimate `shapes: []` opt-out is preserved. Downstream, the `plan-phase` planner lifts every `covered`/`backstop` edge from the SPEC `## Edge Coverage` section into `must_haves.truths`. Exports (locked surface): `classifyShape`, `applicableCategories`, `proposeEdges`, `analyzeCoverage`, `validateResolution`, `validateRequirement`, plus the constants `TAXONOMY` (the 8 edge categories), `VALID_SHAPES`, `SHAPE_CUES`, and `EDGE_VALIDATORS` (the `{explicit, backstop}` validators bundle injected into `probe-core`'s generic engine). Source of truth: `gsd-core/bin/lib/edge-probe.cjs` (generated from `src/edge-probe.cts`, gitignored per ADR-457). Tests: `tests/edge-probe.test.cjs`, `tests/edge-probe-spec-phase-contract.test.cjs`, `tests/edge-probe-planner-contract.test.cjs`. See ADR-550 and Probe Core Module. Per ADR-857's phase-6 boundary (2026-06-12) the predicates this module generates are **core verification substrate** (they set the verifier's reach), so it is wired onto the core predicate rail as a core-default module rather than migrated to an off-by-default `capabilities/edge-probe/` Feature Capability.
First adapter of the Probe Core Module (ADR-550 Decision 7): the spec-phase edge-completeness probe wired into spec-phase Step 5.5. Owns shape classification (`classifyShape`), the applicable-category relevance filter (`applicableCategories` over the 8-category edge `TAXONOMY`), edge proposal (`proposeEdges`), and the `{explicit, backstop}` verification validators; delegates merge/rollup/CLI to `probe-core`. Fail-closed input contract: an edge requirement with missing/empty `text` and no `shapes` override is rejected (a `{ id }`-only requirement no longer classifies to zero edges and silently drops), while the legitimate `shapes: []` opt-out is preserved. Downstream, the `plan-phase` planner lifts every `covered`/`backstop` edge from the SPEC `## Edge Coverage` section into `must_haves.truths`. Exports (locked surface): `classifyShape`, `applicableCategories`, `proposeEdges`, `analyzeCoverage`, `validateResolution`, `validateRequirement`, plus the constants `TAXONOMY` (the **closed** 8 edge categories), `UNCLASSIFIED_CATEGORY` (the `unclassified` review-manually sentinel for zero-cue prose, #1110 — deliberately **not** a 9th taxonomy entry: it stays out of `TAXONOMY` but is present in `EDGE_VALIDATORS.categories`), `VALID_SHAPES`, `SHAPE_CUES`, and `EDGE_VALIDATORS` (the `{explicit, backstop}` validators bundle injected into `probe-core`'s generic engine; its `categories` = the `TAXONOMY` ids plus `UNCLASSIFIED_CATEGORY`). Source of truth: `gsd-core/bin/lib/edge-probe.cjs` (generated from `src/edge-probe.cts`, gitignored per ADR-457). Tests: `tests/edge-probe.test.cjs`, `tests/edge-probe-spec-phase-contract.test.cjs`, `tests/edge-probe-planner-contract.test.cjs`. See ADR-550 and Probe Core Module. Per ADR-857's phase-6 boundary (2026-06-12) the predicates this module generates are **core verification substrate** (they set the verifier's reach), so it is wired onto the core predicate rail as a core-default module rather than migrated to an off-by-default `capabilities/edge-probe/` Feature Capability.
### Verification substrate (predicate boundary)
The ADR-857 classification (settled 2026-06-12, prompted by @davesienkowski's boundary analysis on #857) that predicate-generation — the must-NOT-have / edge predicates that set the verifier's reach — is **core**, not an off-by-default Feature Capability. Load-bearing premise: *verifier reach = spec reach* (the verifier can only catch what the spec concretely names). Decomposes into: the **verifier↔predicate contract** (the verifier always expects predicates and grades **exogenously** against them — core, non-toggleable, a stability contract alongside the Loop Extension Point names), and the **generator** (the probe *adapters* that propose predicates — edge-probe's `classifyShape`/`proposeEdges`, the prohibition probe's adversarial LLM-propose — core-default but independently versionable, kept their own module because of a measured recall gap; `probe-core`'s deterministic validators sit on the contract side, not the generator side). Altitude rule distinguishing it from `gate` hooks: a gate runs *against* the spec (hook); predicate-generation defines the spec's *reach* (core). The decision-#6 `produces`/`consumes` artifact flow is the internal rail from generation to the core verifier. See ADR-857 *Verification substrate vs. plug-in tier (the predicate boundary)* and the ADR-550 cross-reference.

View File

@@ -3109,7 +3109,9 @@ explicit reviewer flags -> --all -> review.default_reviewers -> all detected rev
| `backstop` | Intent recorded, needs a held-out/property-based test | Lifted into `must_haves.truths` as a non-inferable check |
| `unresolved` | Deferred | Soft-gates the spec; row stamped `⚠ Edge unresolved — planner must treat as assumption` |
The resolved edges populate a `## Edge Coverage` section in `SPEC.md`. Unresolved *applicable* edges trigger a soft gate (Resolve / Write-anyway-flagged / Keep-probing) rather than a hard block. Under `--auto`, the probe **never auto-dismisses** — it auto-covers where a defensible criterion exists, otherwise auto-backstops, and logs `[auto] edge coverage: C covered, B backstop, U unresolved`.
When a requirement's prose matches **no** shape cue, the probe does not silently drop it (#1110): it emits a single `unclassified — review manually` candidate so the zero-cue requirement is surfaced for the author to resolve like any other (specify / dismiss-with-reason / defer) — a manual-review nudge, not a hard block.
The resolved edges populate a `## Edge Coverage` section in `SPEC.md`. Unresolved *applicable* edges trigger a soft gate (Resolve / Write-anyway-flagged / Keep-probing) rather than a hard block. Under `--auto`, the probe **never auto-dismisses** — it auto-covers where a defensible criterion exists, otherwise auto-backstops, and logs `[auto] edge coverage: C covered, B backstop, U unresolved`. The one exception is an `unclassified` candidate: `--auto` leaves it **`unresolved`** (surfaced as a flagged assumption), never auto-`backstop` — a missing shape is not evidence an edge exists, so minting a held-out edge obligation would be a false claim.
The load-bearing wire is the `plan-phase` lift: `covered` and `backstop` edges become `must_haves.truths` the verifier can check, so the section is not merely documentation.
@@ -3120,6 +3122,7 @@ The load-bearing wire is the `plan-phase` lift: `covered` and `backstop` edges b
- REQ-EDGE-04: An unresolved applicable edge MUST trigger the soft gate; write-anyway stamps the row as a planner assumption.
- REQ-EDGE-05: `--auto` MUST never auto-dismiss — auto-cover or auto-backstop only.
- REQ-EDGE-06: `plan-phase` MUST lift `covered` criteria and `backstop` notes into `must_haves.truths`.
- REQ-EDGE-07: A requirement whose prose matches no shape cue MUST surface an `unclassified — review manually` candidate (never silently dropped); `--auto` MUST leave it `unresolved`, never auto-`backstop`.
**Reference:** [Edge Probe](../gsd-core/references/edge-probe.md)

View File

@@ -16,6 +16,8 @@ Each finding is one **applicable** edge for one requirement — a boundary the p
You must resolve each finding into exactly one of four states. Claude presents them as a numbered choice (or an `AskUserQuestion` menu).
You may also see an **`unclassified — review manually`** finding. The relevance filter is a heuristic over prose cues, so a requirement whose wording is edge-relevant but matched no cue surfaces this single soft candidate instead of being silently dropped. Resolve it like any other finding — specify a criterion, or **dismiss it with a reason** if the requirement is genuinely edge-free (e.g. "static asset, no input"). It is a manual-review nudge, never a hard block.
---
## Specify it — write an acceptance criterion

View File

@@ -77,6 +77,17 @@ Two rules keep the probe honest and prevent an "everything is N/A" failure mode:
2. **Dismissal requires a reason string.** "N/A — input is a bounded enum, no boundary
exists" is valid; silence is not. The reason string is the audit trail.
**Zero-classification surfaces an `unclassified` candidate (#1110).** The relevance filter is
a heuristic over prose cues, so a requirement whose wording *is* edge-relevant but matches no
shape cue would otherwise classify to zero shapes → zero edges and vanish from coverage with
no signal — the same silent blind spot the probe exists to catch. Instead, a requirement with
non-empty prose, no authored `shapes`, and zero matched shapes surfaces exactly one soft
`unclassified — review manually` candidate (`category: "unclassified"`, `status: "unresolved"`).
It is a dismissible nudge — resolve it, or dismiss it with a reason (e.g. a genuinely edge-free
static-asset requirement) — never a hard block. `unclassified` is a review signal, **not** a
ninth taxonomy category: the closed eight above are unchanged, and an explicit `shapes: []`
opt-out stays silent (the author's deliberate "no edge surface").
Each raised edge carries two orthogonal axes — a resolution **lifecycle** and, when
resolved, a **verification** tier (ADR-550 Decision 7, the shared probe-core model):

View File

@@ -295,6 +295,9 @@ For each Requirement gathered so far:
- **Dismiss (reason)** → mark `dismissed` with a required non-empty reason.
- **Backstop with a test** → mark `backstop`; note "held-out edge test" for plan-phase.
- **Defer** → leave `unresolved`.
- An `unclassified` row (probe `unclassified — review manually`) means the requirement's
prose matched no shape cue (#1110) — treat it like any other candidate (**Specify**,
**Dismiss (reason)**, or **Defer**). A manual-review nudge, not a hard block.
**Soft gate (after resolving):**
- All applicable edges resolved → proceed to Step 6.
@@ -310,6 +313,14 @@ For each Requirement gathered so far:
otherwise auto-`backstop` (never auto-dismiss — a wrong dismissal is the exact silent
failure being eliminated). Log: `[auto] edge coverage: C covered, B backstop, U unresolved`.
**`unclassified` exception (#1110):** `--auto` leaves an `unclassified` candidate
**`unresolved`** (the soft gate surfaces it as a flagged planner assumption) — it never
auto-`backstop`s it. A missing shape is not evidence an edge exists, so minting a held-out
edge obligation on a requirement that may be genuinely edge-free would be a false claim and
risks a vacuous edge test. Leaving it `unresolved` keeps the zero-cue requirement visible
(never a silent drop) without fabricating an edge — which is exactly #1110's purpose: surface
it for review, do not auto-handle it.
Populate the `## Edge Coverage` section of SPEC.md from the resolved edges.
## Step 5.6: Prohibition-Completeness Probe (must-NOT)

View File

@@ -104,8 +104,16 @@ export function applicableCategories(shapes: Shape[]): string[] {
* taxonomy; both verification tiers require a non-empty `resolution` (an explicit AC's text
* or a backstop note) so plan-phase has a criterion to lift.
*/
/**
* Pseudo-category for a requirement whose prose matched NO shape cue (#1110). It is a soft
* "review manually" signal, NOT a 9th taxonomy category: it stays out of `TAXONOMY` (the closed
* eight) and only joins `EDGE_VALIDATORS.categories` so `analyzeCoverage` accepts the item.
*/
export const UNCLASSIFIED_CATEGORY = 'unclassified';
const UNCLASSIFIED_PROBE = 'unclassified — review manually';
export const EDGE_VALIDATORS: Validators = {
categories: TAXONOMY.map((c) => c.id),
categories: [...TAXONOMY.map((c) => c.id), UNCLASSIFIED_CATEGORY],
verification: ['explicit', 'backstop'],
requiredFieldsByVerification: { explicit: ['resolution'], backstop: ['resolution'] },
};
@@ -162,6 +170,22 @@ export function proposeEdges(requirement: Requirement): Edge[] {
shapes = requirement.shapes;
} else {
shapes = classifyShape(requirement.text);
if (shapes.length === 0) {
// Prose present but no shape cue matched. Do NOT silently drop it (#1110): an
// edge-relevant requirement whose phrasing missed every cue would otherwise vanish from
// coverage with no signal — the exact blind spot this probe exists to catch. Surface ONE
// soft, dismissible "unclassified — review manually" candidate. The explicit `shapes: []`
// opt-out (handled above) stays silent — that is the author's deliberate "no edge surface".
return [{
requirement_id: requirement.id,
category: UNCLASSIFIED_CATEGORY,
status: 'unresolved',
verification: null,
resolution: null,
reason: null,
probe: UNCLASSIFIED_PROBE,
}];
}
}
return applicableCategories(shapes).map((catId): Edge => {
const cat = TAXONOMY.find((c) => c.id === catId);

View File

@@ -219,6 +219,57 @@ describe('edge-probe: proposeEdges — empty-shapes override (RR-06)', () => {
});
});
describe('edge-probe: proposeEdges — unclassified candidate for prose-zero-cue (#1110)', () => {
// A requirement with non-empty prose that matches NO shape cue must not be silently
// dropped (zero edges, no signal). It now surfaces ONE soft "unclassified — review
// manually" candidate instead. The explicit `shapes: []` opt-out stays silent.
test('prose with no shape cue surfaces a single unclassified candidate', () => {
const edges = ep.proposeEdges({ id: 'R1', text: 'Display the company logo' });
assert.equal(edges.length, 1, 'prose-zero-cue must surface exactly one unclassified candidate');
assert.deepEqual(edges[0], {
requirement_id: 'R1',
category: 'unclassified',
status: 'unresolved',
verification: null,
resolution: null,
reason: null,
probe: 'unclassified — review manually',
});
});
test('UNCLASSIFIED_CATEGORY is a valid item category but NOT a taxonomy category', () => {
assert.equal(ep.UNCLASSIFIED_CATEGORY, 'unclassified');
assert.ok(ep.EDGE_VALIDATORS.categories.includes('unclassified'), 'analyzeCoverage must accept the unclassified item category');
assert.ok(!ep.TAXONOMY.some((c) => c.id === 'unclassified'), 'unclassified must NOT pollute the closed 8-category taxonomy');
});
test('explicit shapes: [] stays silent (deliberate opt-out — NOT unclassified)', () => {
const edges = ep.proposeEdges({ id: 'R1', text: 'Display the company logo', shapes: [] });
assert.deepEqual(edges, []);
});
test('prose that DOES classify proposes real edges, never an unclassified candidate', () => {
const edges = ep.proposeEdges({ id: 'R1', text: 'Round a number to N decimal places' });
assert.ok(edges.length > 0);
assert.ok(!edges.some((e) => e.category === 'unclassified'), 'a classifiable requirement must not emit unclassified');
});
test('analyzeCoverage surfaces the unclassified candidate as unresolved (no throw)', () => {
const report = ep.analyzeCoverage([{ id: 'R1', text: 'Display the company logo' }]);
assert.equal(report.coverage.applicable, 1);
assert.equal(report.coverage.unresolved, 1);
assert.equal(report.items[0].category, 'unclassified');
});
test('an unclassified candidate can be dismissed with a reason (edge-probe parity)', () => {
const report = ep.analyzeCoverage(
[{ id: 'R1', text: 'Display the company logo' }],
[{ requirement_id: 'R1', category: 'unclassified', status: 'dismissed', reason: 'genuinely edge-free — static asset' }],
);
assert.equal(report.items[0].status, 'dismissed');
});
});
describe('edge-probe: proposeEdges — invalid authored shapes fail closed (re-review #3 High)', () => {
// A non-empty but INVALID shapes array must NOT silently suppress every probe.
// shapes:['numeric'] (typo for the locked 'numeric-range') previously passed

View File

@@ -72,7 +72,7 @@
"ship.md": 24388,
"sketch-wrap-up.md": 14223,
"sketch.md": 19960,
"spec-phase.md": 27589,
"spec-phase.md": 28438,
"spike-wrap-up.md": 15092,
"spike.md": 24517,
"stats.md": 6718,