feat(spec-phase): surface zero-classification edge-probe requirements as unclassified candidates (#1110) (#1117)
Edge-probe now surfaces a zero-classification requirement (non-empty prose, no shape cue matched, no `shapes` override) as a single soft `unclassified — review manually` candidate instead of silently dropping it — the exact blind spot the probe exists to catch. Dismissible like any edge; the `shapes: []` opt-out stays silent; `TAXONOMY` (the closed 8 categories) is unchanged. Under `--auto` the candidate is left `unresolved`, never auto-`backstop` (a missing shape is not evidence an edge exists). Closes #1110
This commit is contained in:
5
.changeset/serene-mice-swim.md
Normal file
5
.changeset/serene-mice-swim.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Changed
|
||||
pr: 1117
|
||||
---
|
||||
Edge-probe now surfaces a zero-classification requirement (non-empty prose, no shape cue matched, no `shapes` override) as a single soft `unclassified — review manually` candidate instead of silently dropping it. Dismissible like any edge; `shapes: []` opt-out stays silent; TAXONOMY unchanged.
|
||||
@@ -238,7 +238,7 @@ Runtime-neutral predicate evaluating `*-UAT.md` / `*-VERIFICATION.md` result fie
|
||||
Generic spec-phase probe resolution model — the shared seam underlying spec-completeness probes (ADR-550 Decision 7). Owns the `status × verification` model (`status: resolved | dismissed | unresolved` × a per-probe `verification` tier), structural validation (`validateResolution`, `validateRequirement` — fail-closed: `verification` must be null unless status is `resolved`, and an out-of-enum status, a `dismissed`-without-`reason`, or an `unresolved` carrying a `resolution`/`reason`/tier payload all throw rather than silently miscount), the `analyzeCoverage(items, resolutions?, validators)` merge/rollup/orphan-reject pipeline, the `byVerification` per-tier rollup, and the `runProbeCli` I/O scaffold (parse → validate → analyze → emit, structurally guarding the report shape before write — a malformed report fails closed with stderr + exit 2 instead of stringifying as green). Adapter-agnostic: consumed by the Edge Probe Module today and the Prohibition Probe Module (#644) next. Exports (generic surface): `VALID_STATUS`, `validateResolution`, `validateRequirement`, `analyzeCoverage`, `runProbeCli` — the prohibition adapter exports that also ship from this module (`projectProhibitions`, `PROHIBITION_VALIDATORS`, `validateProhibitionResolution`, `dispositionForProhibition`) are documented under the Prohibition Probe Module's own locked-surface line. Source of truth: `gsd-core/bin/lib/probe-core.cjs` (generated from `src/probe-core.cts`, gitignored per ADR-457). Tests: `tests/probe-core.test.cjs`. See ADR-550 and Edge Probe Module. Under ADR-857 (phase-6 boundary, settled 2026-06-12) this seam is classified **core verification substrate** on the *contract* side: its deterministic validators are the verifier↔predicate contract's CI-testable surface (ADR-550 Decision 5) — core and non-toggleable, never an off-by-default Feature Capability. (The recall-gapped *generator* is the probe adapters that propose predicates, not this resolution engine — see Edge Probe Module and Verification substrate (predicate boundary).)
|
||||
|
||||
### Edge Probe Module
|
||||
First adapter of the Probe Core Module (ADR-550 Decision 7): the spec-phase edge-completeness probe wired into spec-phase Step 5.5. Owns shape classification (`classifyShape`), the applicable-category relevance filter (`applicableCategories` over the 8-category edge `TAXONOMY`), edge proposal (`proposeEdges`), and the `{explicit, backstop}` verification validators; delegates merge/rollup/CLI to `probe-core`. Fail-closed input contract: an edge requirement with missing/empty `text` and no `shapes` override is rejected (a `{ id }`-only requirement no longer classifies to zero edges and silently drops), while the legitimate `shapes: []` opt-out is preserved. Downstream, the `plan-phase` planner lifts every `covered`/`backstop` edge from the SPEC `## Edge Coverage` section into `must_haves.truths`. Exports (locked surface): `classifyShape`, `applicableCategories`, `proposeEdges`, `analyzeCoverage`, `validateResolution`, `validateRequirement`, plus the constants `TAXONOMY` (the 8 edge categories), `VALID_SHAPES`, `SHAPE_CUES`, and `EDGE_VALIDATORS` (the `{explicit, backstop}` validators bundle injected into `probe-core`'s generic engine). Source of truth: `gsd-core/bin/lib/edge-probe.cjs` (generated from `src/edge-probe.cts`, gitignored per ADR-457). Tests: `tests/edge-probe.test.cjs`, `tests/edge-probe-spec-phase-contract.test.cjs`, `tests/edge-probe-planner-contract.test.cjs`. See ADR-550 and Probe Core Module. Per ADR-857's phase-6 boundary (2026-06-12) the predicates this module generates are **core verification substrate** (they set the verifier's reach), so it is wired onto the core predicate rail as a core-default module rather than migrated to an off-by-default `capabilities/edge-probe/` Feature Capability.
|
||||
First adapter of the Probe Core Module (ADR-550 Decision 7): the spec-phase edge-completeness probe wired into spec-phase Step 5.5. Owns shape classification (`classifyShape`), the applicable-category relevance filter (`applicableCategories` over the 8-category edge `TAXONOMY`), edge proposal (`proposeEdges`), and the `{explicit, backstop}` verification validators; delegates merge/rollup/CLI to `probe-core`. Fail-closed input contract: an edge requirement with missing/empty `text` and no `shapes` override is rejected (a `{ id }`-only requirement no longer classifies to zero edges and silently drops), while the legitimate `shapes: []` opt-out is preserved. Downstream, the `plan-phase` planner lifts every `covered`/`backstop` edge from the SPEC `## Edge Coverage` section into `must_haves.truths`. Exports (locked surface): `classifyShape`, `applicableCategories`, `proposeEdges`, `analyzeCoverage`, `validateResolution`, `validateRequirement`, plus the constants `TAXONOMY` (the **closed** 8 edge categories), `UNCLASSIFIED_CATEGORY` (the `unclassified` review-manually sentinel for zero-cue prose, #1110 — deliberately **not** a 9th taxonomy entry: it stays out of `TAXONOMY` but is present in `EDGE_VALIDATORS.categories`), `VALID_SHAPES`, `SHAPE_CUES`, and `EDGE_VALIDATORS` (the `{explicit, backstop}` validators bundle injected into `probe-core`'s generic engine; its `categories` = the `TAXONOMY` ids plus `UNCLASSIFIED_CATEGORY`). Source of truth: `gsd-core/bin/lib/edge-probe.cjs` (generated from `src/edge-probe.cts`, gitignored per ADR-457). Tests: `tests/edge-probe.test.cjs`, `tests/edge-probe-spec-phase-contract.test.cjs`, `tests/edge-probe-planner-contract.test.cjs`. See ADR-550 and Probe Core Module. Per ADR-857's phase-6 boundary (2026-06-12) the predicates this module generates are **core verification substrate** (they set the verifier's reach), so it is wired onto the core predicate rail as a core-default module rather than migrated to an off-by-default `capabilities/edge-probe/` Feature Capability.
|
||||
|
||||
### Verification substrate (predicate boundary)
|
||||
The ADR-857 classification (settled 2026-06-12, prompted by @davesienkowski's boundary analysis on #857) that predicate-generation — the must-NOT-have / edge predicates that set the verifier's reach — is **core**, not an off-by-default Feature Capability. Load-bearing premise: *verifier reach = spec reach* (the verifier can only catch what the spec concretely names). Decomposes into: the **verifier↔predicate contract** (the verifier always expects predicates and grades **exogenously** against them — core, non-toggleable, a stability contract alongside the Loop Extension Point names), and the **generator** (the probe *adapters* that propose predicates — edge-probe's `classifyShape`/`proposeEdges`, the prohibition probe's adversarial LLM-propose — core-default but independently versionable, kept their own module because of a measured recall gap; `probe-core`'s deterministic validators sit on the contract side, not the generator side). Altitude rule distinguishing it from `gate` hooks: a gate runs *against* the spec (hook); predicate-generation defines the spec's *reach* (core). The decision-#6 `produces`/`consumes` artifact flow is the internal rail from generation to the core verifier. See ADR-857 *Verification substrate vs. plug-in tier (the predicate boundary)* and the ADR-550 cross-reference.
|
||||
|
||||
@@ -3109,7 +3109,9 @@ explicit reviewer flags -> --all -> review.default_reviewers -> all detected rev
|
||||
| `backstop` | Intent recorded, needs a held-out/property-based test | Lifted into `must_haves.truths` as a non-inferable check |
|
||||
| `unresolved` | Deferred | Soft-gates the spec; row stamped `⚠ Edge unresolved — planner must treat as assumption` |
|
||||
|
||||
The resolved edges populate a `## Edge Coverage` section in `SPEC.md`. Unresolved *applicable* edges trigger a soft gate (Resolve / Write-anyway-flagged / Keep-probing) rather than a hard block. Under `--auto`, the probe **never auto-dismisses** — it auto-covers where a defensible criterion exists, otherwise auto-backstops, and logs `[auto] edge coverage: C covered, B backstop, U unresolved`.
|
||||
When a requirement's prose matches **no** shape cue, the probe does not silently drop it (#1110): it emits a single `unclassified — review manually` candidate so the zero-cue requirement is surfaced for the author to resolve like any other (specify / dismiss-with-reason / defer) — a manual-review nudge, not a hard block.
|
||||
|
||||
The resolved edges populate a `## Edge Coverage` section in `SPEC.md`. Unresolved *applicable* edges trigger a soft gate (Resolve / Write-anyway-flagged / Keep-probing) rather than a hard block. Under `--auto`, the probe **never auto-dismisses** — it auto-covers where a defensible criterion exists, otherwise auto-backstops, and logs `[auto] edge coverage: C covered, B backstop, U unresolved`. The one exception is an `unclassified` candidate: `--auto` leaves it **`unresolved`** (surfaced as a flagged assumption), never auto-`backstop` — a missing shape is not evidence an edge exists, so minting a held-out edge obligation would be a false claim.
|
||||
|
||||
The load-bearing wire is the `plan-phase` lift: `covered` and `backstop` edges become `must_haves.truths` the verifier can check, so the section is not merely documentation.
|
||||
|
||||
@@ -3120,6 +3122,7 @@ The load-bearing wire is the `plan-phase` lift: `covered` and `backstop` edges b
|
||||
- REQ-EDGE-04: An unresolved applicable edge MUST trigger the soft gate; write-anyway stamps the row as a planner assumption.
|
||||
- REQ-EDGE-05: `--auto` MUST never auto-dismiss — auto-cover or auto-backstop only.
|
||||
- REQ-EDGE-06: `plan-phase` MUST lift `covered` criteria and `backstop` notes into `must_haves.truths`.
|
||||
- REQ-EDGE-07: A requirement whose prose matches no shape cue MUST surface an `unclassified — review manually` candidate (never silently dropped); `--auto` MUST leave it `unresolved`, never auto-`backstop`.
|
||||
|
||||
**Reference:** [Edge Probe](../gsd-core/references/edge-probe.md)
|
||||
|
||||
|
||||
@@ -16,6 +16,8 @@ Each finding is one **applicable** edge for one requirement — a boundary the p
|
||||
|
||||
You must resolve each finding into exactly one of four states. Claude presents them as a numbered choice (or an `AskUserQuestion` menu).
|
||||
|
||||
You may also see an **`unclassified — review manually`** finding. The relevance filter is a heuristic over prose cues, so a requirement whose wording is edge-relevant but matched no cue surfaces this single soft candidate instead of being silently dropped. Resolve it like any other finding — specify a criterion, or **dismiss it with a reason** if the requirement is genuinely edge-free (e.g. "static asset, no input"). It is a manual-review nudge, never a hard block.
|
||||
|
||||
---
|
||||
|
||||
## Specify it — write an acceptance criterion
|
||||
|
||||
@@ -77,6 +77,17 @@ Two rules keep the probe honest and prevent an "everything is N/A" failure mode:
|
||||
2. **Dismissal requires a reason string.** "N/A — input is a bounded enum, no boundary
|
||||
exists" is valid; silence is not. The reason string is the audit trail.
|
||||
|
||||
**Zero-classification surfaces an `unclassified` candidate (#1110).** The relevance filter is
|
||||
a heuristic over prose cues, so a requirement whose wording *is* edge-relevant but matches no
|
||||
shape cue would otherwise classify to zero shapes → zero edges and vanish from coverage with
|
||||
no signal — the same silent blind spot the probe exists to catch. Instead, a requirement with
|
||||
non-empty prose, no authored `shapes`, and zero matched shapes surfaces exactly one soft
|
||||
`unclassified — review manually` candidate (`category: "unclassified"`, `status: "unresolved"`).
|
||||
It is a dismissible nudge — resolve it, or dismiss it with a reason (e.g. a genuinely edge-free
|
||||
static-asset requirement) — never a hard block. `unclassified` is a review signal, **not** a
|
||||
ninth taxonomy category: the closed eight above are unchanged, and an explicit `shapes: []`
|
||||
opt-out stays silent (the author's deliberate "no edge surface").
|
||||
|
||||
Each raised edge carries two orthogonal axes — a resolution **lifecycle** and, when
|
||||
resolved, a **verification** tier (ADR-550 Decision 7, the shared probe-core model):
|
||||
|
||||
|
||||
@@ -295,6 +295,9 @@ For each Requirement gathered so far:
|
||||
- **Dismiss (reason)** → mark `dismissed` with a required non-empty reason.
|
||||
- **Backstop with a test** → mark `backstop`; note "held-out edge test" for plan-phase.
|
||||
- **Defer** → leave `unresolved`.
|
||||
- An `unclassified` row (probe `unclassified — review manually`) means the requirement's
|
||||
prose matched no shape cue (#1110) — treat it like any other candidate (**Specify**,
|
||||
**Dismiss (reason)**, or **Defer**). A manual-review nudge, not a hard block.
|
||||
|
||||
**Soft gate (after resolving):**
|
||||
- All applicable edges resolved → proceed to Step 6.
|
||||
@@ -310,6 +313,14 @@ For each Requirement gathered so far:
|
||||
otherwise auto-`backstop` (never auto-dismiss — a wrong dismissal is the exact silent
|
||||
failure being eliminated). Log: `[auto] edge coverage: C covered, B backstop, U unresolved`.
|
||||
|
||||
**`unclassified` exception (#1110):** `--auto` leaves an `unclassified` candidate
|
||||
**`unresolved`** (the soft gate surfaces it as a flagged planner assumption) — it never
|
||||
auto-`backstop`s it. A missing shape is not evidence an edge exists, so minting a held-out
|
||||
edge obligation on a requirement that may be genuinely edge-free would be a false claim and
|
||||
risks a vacuous edge test. Leaving it `unresolved` keeps the zero-cue requirement visible
|
||||
(never a silent drop) without fabricating an edge — which is exactly #1110's purpose: surface
|
||||
it for review, do not auto-handle it.
|
||||
|
||||
Populate the `## Edge Coverage` section of SPEC.md from the resolved edges.
|
||||
|
||||
## Step 5.6: Prohibition-Completeness Probe (must-NOT)
|
||||
|
||||
@@ -104,8 +104,16 @@ export function applicableCategories(shapes: Shape[]): string[] {
|
||||
* taxonomy; both verification tiers require a non-empty `resolution` (an explicit AC's text
|
||||
* or a backstop note) so plan-phase has a criterion to lift.
|
||||
*/
|
||||
/**
|
||||
* Pseudo-category for a requirement whose prose matched NO shape cue (#1110). It is a soft
|
||||
* "review manually" signal, NOT a 9th taxonomy category: it stays out of `TAXONOMY` (the closed
|
||||
* eight) and only joins `EDGE_VALIDATORS.categories` so `analyzeCoverage` accepts the item.
|
||||
*/
|
||||
export const UNCLASSIFIED_CATEGORY = 'unclassified';
|
||||
const UNCLASSIFIED_PROBE = 'unclassified — review manually';
|
||||
|
||||
export const EDGE_VALIDATORS: Validators = {
|
||||
categories: TAXONOMY.map((c) => c.id),
|
||||
categories: [...TAXONOMY.map((c) => c.id), UNCLASSIFIED_CATEGORY],
|
||||
verification: ['explicit', 'backstop'],
|
||||
requiredFieldsByVerification: { explicit: ['resolution'], backstop: ['resolution'] },
|
||||
};
|
||||
@@ -162,6 +170,22 @@ export function proposeEdges(requirement: Requirement): Edge[] {
|
||||
shapes = requirement.shapes;
|
||||
} else {
|
||||
shapes = classifyShape(requirement.text);
|
||||
if (shapes.length === 0) {
|
||||
// Prose present but no shape cue matched. Do NOT silently drop it (#1110): an
|
||||
// edge-relevant requirement whose phrasing missed every cue would otherwise vanish from
|
||||
// coverage with no signal — the exact blind spot this probe exists to catch. Surface ONE
|
||||
// soft, dismissible "unclassified — review manually" candidate. The explicit `shapes: []`
|
||||
// opt-out (handled above) stays silent — that is the author's deliberate "no edge surface".
|
||||
return [{
|
||||
requirement_id: requirement.id,
|
||||
category: UNCLASSIFIED_CATEGORY,
|
||||
status: 'unresolved',
|
||||
verification: null,
|
||||
resolution: null,
|
||||
reason: null,
|
||||
probe: UNCLASSIFIED_PROBE,
|
||||
}];
|
||||
}
|
||||
}
|
||||
return applicableCategories(shapes).map((catId): Edge => {
|
||||
const cat = TAXONOMY.find((c) => c.id === catId);
|
||||
|
||||
@@ -219,6 +219,57 @@ describe('edge-probe: proposeEdges — empty-shapes override (RR-06)', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('edge-probe: proposeEdges — unclassified candidate for prose-zero-cue (#1110)', () => {
|
||||
// A requirement with non-empty prose that matches NO shape cue must not be silently
|
||||
// dropped (zero edges, no signal). It now surfaces ONE soft "unclassified — review
|
||||
// manually" candidate instead. The explicit `shapes: []` opt-out stays silent.
|
||||
test('prose with no shape cue surfaces a single unclassified candidate', () => {
|
||||
const edges = ep.proposeEdges({ id: 'R1', text: 'Display the company logo' });
|
||||
assert.equal(edges.length, 1, 'prose-zero-cue must surface exactly one unclassified candidate');
|
||||
assert.deepEqual(edges[0], {
|
||||
requirement_id: 'R1',
|
||||
category: 'unclassified',
|
||||
status: 'unresolved',
|
||||
verification: null,
|
||||
resolution: null,
|
||||
reason: null,
|
||||
probe: 'unclassified — review manually',
|
||||
});
|
||||
});
|
||||
|
||||
test('UNCLASSIFIED_CATEGORY is a valid item category but NOT a taxonomy category', () => {
|
||||
assert.equal(ep.UNCLASSIFIED_CATEGORY, 'unclassified');
|
||||
assert.ok(ep.EDGE_VALIDATORS.categories.includes('unclassified'), 'analyzeCoverage must accept the unclassified item category');
|
||||
assert.ok(!ep.TAXONOMY.some((c) => c.id === 'unclassified'), 'unclassified must NOT pollute the closed 8-category taxonomy');
|
||||
});
|
||||
|
||||
test('explicit shapes: [] stays silent (deliberate opt-out — NOT unclassified)', () => {
|
||||
const edges = ep.proposeEdges({ id: 'R1', text: 'Display the company logo', shapes: [] });
|
||||
assert.deepEqual(edges, []);
|
||||
});
|
||||
|
||||
test('prose that DOES classify proposes real edges, never an unclassified candidate', () => {
|
||||
const edges = ep.proposeEdges({ id: 'R1', text: 'Round a number to N decimal places' });
|
||||
assert.ok(edges.length > 0);
|
||||
assert.ok(!edges.some((e) => e.category === 'unclassified'), 'a classifiable requirement must not emit unclassified');
|
||||
});
|
||||
|
||||
test('analyzeCoverage surfaces the unclassified candidate as unresolved (no throw)', () => {
|
||||
const report = ep.analyzeCoverage([{ id: 'R1', text: 'Display the company logo' }]);
|
||||
assert.equal(report.coverage.applicable, 1);
|
||||
assert.equal(report.coverage.unresolved, 1);
|
||||
assert.equal(report.items[0].category, 'unclassified');
|
||||
});
|
||||
|
||||
test('an unclassified candidate can be dismissed with a reason (edge-probe parity)', () => {
|
||||
const report = ep.analyzeCoverage(
|
||||
[{ id: 'R1', text: 'Display the company logo' }],
|
||||
[{ requirement_id: 'R1', category: 'unclassified', status: 'dismissed', reason: 'genuinely edge-free — static asset' }],
|
||||
);
|
||||
assert.equal(report.items[0].status, 'dismissed');
|
||||
});
|
||||
});
|
||||
|
||||
describe('edge-probe: proposeEdges — invalid authored shapes fail closed (re-review #3 High)', () => {
|
||||
// A non-empty but INVALID shapes array must NOT silently suppress every probe.
|
||||
// shapes:['numeric'] (typo for the locked 'numeric-range') previously passed
|
||||
|
||||
@@ -72,7 +72,7 @@
|
||||
"ship.md": 24388,
|
||||
"sketch-wrap-up.md": 14223,
|
||||
"sketch.md": 19960,
|
||||
"spec-phase.md": 27589,
|
||||
"spec-phase.md": 28438,
|
||||
"spike-wrap-up.md": 15092,
|
||||
"spike.md": 24517,
|
||||
"stats.md": 6718,
|
||||
|
||||
Reference in New Issue
Block a user