* feat(spec-phase): spec-completeness edge-probe (#550) — relocated to gsd-core/
Rebased onto current next and relocated the whole feature from get-shit-done/ to
gsd-core/ per #615 (trek-e re-review #4, option 1). The artifact now builds to
gsd-core/bin/lib/edge-probe.cjs; all hard-coded path strings (tests, workflow
@-refs, run-tests.cjs sentinel, eslint ADR-457 ignore, .gitignore) updated.
Content conflicts in .gitignore / eslint.config.mjs / run-tests.cjs resolved.
Feature: Step 5.5 edge-completeness probe walks each SPEC requirement against a
closed 8-category edge taxonomy, proposes applicable candidate edges, and resolves
each (covered/dismissed/backstop/unresolved). covered/backstop criteria are lifted
by plan-phase into must_haves.truths, extending the goal-backward verifier's reach
to boundary edges no requirement was written for. Engine authored as strict TS
(src/edge-probe.cts, ADR-457), compiled to the gitignored gsd-core/bin/lib/edge-probe.cjs.
Folds in every prior review round on PR #584:
- RR-01..03: plan-phase resolves the phase *-SPEC.md and injects {SPEC_PATH} into the
planner; must_haves<->Edge-Coverage quality_gate; held-out planner-contract test.
- RR-04/11: Step 5.5 invokes the compiled engine at runtime (npm --prefix-pinned,
source-checkout-gated build fallback) instead of LLM re-derivation; the engine
capture is exit-checked and the report JSON-validated before use (fail closed).
- RR-05..10: all six fixtures embedded + count-equality; backstop/covered require a
resolution; Array.isArray(shapes); duplicate-resolution rejection; CLI JSON exit(2);
per-artifact build sentinel.
- Authored-shape validation: invalid (non-empty) shapes fail closed (VALID_SHAPES).
- Adversarial-review hardening: orphan/typo resolution rejection, requirement input
validation (id/text/shapes, duplicate id, non-array), zero-applicable guard.
Full suite 0 failures; npm run lint 0 errors; edge-probe suite 72/72.
* test(#550): RED — status×verification re-cut + probe-core engine specs
Re-cut the edge-probe resolution model onto two orthogonal axes per
ADR-550 Decision 7 (trek-e #644 comments 2026-06-03 14:36 + 14:44):
status: resolved | dismissed | unresolved (lifecycle, shared)
verification: explicit | backstop | null (only when resolved)
- tests/probe-core.test.cjs (new): behavioral specs for the generic engine
to be extracted — validateResolution(r, validators), validateRequirement,
analyzeCoverage(items, resolutions?, validators), byVerification rollup,
runProbeCli I/O scaffold (injected-io unit tests).
- tests/edge-probe.test.cjs: covered→{resolved,explicit}, backstop→
{resolved,backstop}; coverage gains byVerification.{explicit,backstop};
proposeEdges items gain verification:null.
- 6 fixtures re-genned + re-embedded in edge-probe.md; coverage.resolved
COUNT preserved on every fixture (closed set = resolved+dismissed; doc
line: 'adjacency=covered + ordering=dismissed' -> 2). edge-probe.md prose
rewritten to the two-axis model.
Fails as expected: probe-core.cjs has no source yet; edge-probe still
emits the old covered/backstop enum (27/61 edge specs red).
* feat(#550): extract probe-core seam + refactor edge-probe onto it (ADR-550 D7)
Extract the generic resolution model into src/probe-core.cts (the shared
seam the prohibition probe #644 is born on) and refactor edge-probe.cts
into its first adapter.
probe-core owns (probe-agnostic):
- the status×verification re-cut: status: resolved|dismissed|unresolved ×
verification: <probe-defined>|null
- validateResolution(r, validators) / validateRequirement (generic id+text)
- analyzeCoverage(items, resolutions?, validators) over ALREADY-PROPOSED
items[] (core never assumes propose is deterministic — edge resolves via
LLM, #644 proposes via LLM), with merge / dup-reject / orphan-reject
- byVerification rollup; coverage.resolved = closed set (resolved+dismissed),
count-preserved from the pre-re-cut engine
- runProbeCli I/O scaffold (injected io; one bin per probe)
- hybrid typing: generic params + injected runtime validators
{categories, verification, requiredFieldsByVerification} (ADR-550 #5)
edge-probe keeps ONLY the edge cluster: Shape/SHAPE_CUES/VALID_SHAPES/
classifyShape/TAXONOMY/applicableCategories/proposeEdges + EDGE_VALIDATORS
{explicit,backstop}; delegates merge/rollup/CLI to probe-core. Every shipped
#584 guarantee preserved (fail-closed shapes, orphan/dup rejection, input
validation, CLI exit 2). 104/104 edge+probe-core+docs+contract specs green.
* chore(#550): register probe-core.cjs artifact in ledgers + inventory
New gitignored build artifact gsd-core/bin/lib/probe-core.cjs (compiled
from src/probe-core.cts) needs registering in every artifact ledger:
- .gitignore + eslint.config.mjs ADR-457 ignore: lint the .cts source,
never the emitted .cjs.
- scripts/run-tests.cjs per-artifact build sentinel: build if probe-core.cjs
is missing on a clean checkout.
- docs/INVENTORY.md: CLI Modules 83 -> 84, new probe-core.cjs row, and the
edge-probe.cjs row updated to reflect it is now the first probe-core adapter.
- docs/INVENTORY-MANIFEST.json: regenerated (gen-inventory-manifest.cjs --write).
probe-core.test.cjs is a single test file (under the 2-file cap), so no
lint-test-file-count allowlist entry is needed.
* docs(adr-550): spec-phase probe pattern + prohibition contract [Accepted]
trek-e's final ADR-550 body, verbatim (open-gsd/gsd-core#644 comment
2026-06-03T15:23Z), Accepted by both maintainer and #550 author. Lands on
PR #584 alongside the probe-core extraction (Decision 7) it governs, so the
contract and its first implementation arrive together.
Decisions: probe packaging (3 layers); recall->precision protocol;
prohibition home = SPEC acceptance criteria + optional must_haves.prohibitions:
(truths untouched, no polarity); tiered verification test|judgment
(judgment = mode-dependent soft-gate-with-flags, never silent pass / never
hard-halt); CI tests the contract not the classifier; secure-phase ownership
seam; and Decision 7 — probe-core seam + status×verification re-cut (7a-7e),
which this PR implements.
* feat(#550): fail-closed probe-core across full status×verification + runProbeCli structural guard
Re-review #5 (trek-e) seam-hardening on the generic probe-core contract #644 inherits:
- validateResolution now enforces the 'verification is null unless resolved'
invariant for EVERY status (not just resolved): a dismissed/unresolved
resolution carrying a verification tier is rejected instead of merging verbatim.
- An unresolved resolution carrying a resolution/reason payload is rejected
(was silently dropped into the unresolved count).
- runProbeCli structurally validates the report an adapter returns before writing
it (was: any malformed object stringified as green output) — fails closed → exit 2.
- coverage.resolved kept count-preserved (closed set) per the blessed migration
contract; a new test locks that an all-dismissed run is NOT affirmatively covered
(byVerification is the honest gate).
Tests: probe-core 37/37; full edge-probe suite 113/113; full suite 1816/1816; lint 0.
* docs(adr-550): annotate Decision 5 #584/#644 scope + correct 7a coverage.resolved semantics
Re-review #5 (trek-e) clarity edits:
- Decision 5: annotate that only contract item (a) ships on #584 (the edge
adapter's parse+validate test); (b)–(d) are #644 scope, matching Consequences.
- Decision 7a: correct the 'coverage.resolved is preserved (status === resolved)'
parenthetical — the blessed/implemented semantics are count-preserved = the
CLOSED set (resolved + dismissed = applicable − unresolved), with byVerification
carrying the per-tier resolved-status breakdown. The old parenthetical
contradicted the shipped count.
* test(#550): cover runProbeCli structural-guard numeric-count branch
Second-pass coverage audit found the 'coverage object present but counts
non-numeric' branch of isValidReport (built probe-core.cjs:60-61) unexercised —
the {nope:true} malformed test fails earlier at the items[] check. Add a report
with well-formed items[] + a coverage object carrying non-numeric counts so the
numeric branch is hit. No source change; closes the line gap.
* fix(#550): reject edge requirement with missing/empty text when no shapes override (M2)
The edge adapter's `text` is the classification signal and a required field, but
core `validateRequirement` left it optional, so a `{ id }` requirement classified to
zero shapes -> zero edges -> was silently DROPPED from coverage with no signal -- the
exact fail-open this feature exists to eliminate. Reject missing/empty text unless an
authored `shapes` override (incl. `[]`) opts out of prose classification.
* fix(#550): validate verbatim items in analyzeCoverage shared seam (m1)
A proposed item with no matching author resolution is rolled up VERBATIM, but its own
status/fields were never validated -- an item carrying an out-of-enum status (the dropped
"covered") or `dismissed` without a reason would be counted closed. The edge adapter only
proposes `unresolved` items, but the prohibition adapter (#644) proposes LLM-generated
items that arrive populated. An Item is structurally a superset of a Resolution, so reuse
validateResolution to fail closed. ADR-550 Decision 5 hardens this shared seam.
* fix(#550): move edge-coverage lift instruction to runtime planner surface (M1)
templates/planner-subagent-prompt.md is loaded by nothing at runtime (no @-import in
agents/gsd-planner.md; plan-phase.md spawns the planner from its own inline
<planning_context>), so the precise covered/backstop -> must_haves.truths lift instruction
this PR added there never reached the planner -- and the RR-02 contract test asserted it in
that dead file, giving false green. Move the instruction into plan-phase.md's runtime
<downstream_consumer> block (where the rest of the wire already lives), revert the dead-template
edit, and retarget RR-02 to the loaded surface with a guard against re-orphaning.
* test(#550): lock machine<->SPEC vocabulary mapping against drift (m2)
The machine contract uses orthogonal status x verification; the SPEC table renders a flat
covered/dismissed/backstop/unresolved. The migration map (ADR-550 Decision 7a) was prose-only
with no test, so the layers could silently drift. The SPEC table is LLM-rendered (no JS
renderer to round-trip), so pin the canonical bijection as code AND ground it in every doc
surface that renders the vocabulary (ADR migration clause, spec.md legend, reference mapping
table) -- a rename or remap now fails the suite.
* docs(#550): add how-to for resolving edge-coverage findings (B1)
Feature shipped reference coverage (FEATURES.md, COMMANDS.md, references/edge-probe.md) but
no how-to -- reference-only does not satisfy the Diataxis docs standard for a user-facing
capability. Add a single-mode how-to (imperative, goal-directed) walking each resolution
state (specify/dismiss/backstop/defer), the soft gate, and --auto, with taxonomy/concepts
linked out to the reference. Register it in the docs/how-to index.
* docs(#550): add Probe Core + Edge Probe glossary entries to CONTEXT.md (N1)
trek-e re-review #7 N1 (Major): adding probe-core/edge-probe as src/*.cts-derived
seam modules (ADR-550 Decision 7) requires CONTEXT.md Domain-terms glossary entries
per the maintainer-enforced new-seam gate. Adds '### Probe Core Module' and
'### Edge Probe Module' with exports, generated source paths, and the ADR-550 seam
contract, placed beside the Research Module feature-seam entries.
* test(#550): add fast-check property suite for probe-core analyzeCoverage (N2)
trek-e re-review #7 N2 (RULESET.TESTS.property-based-testing): analyzeCoverage is a
transformation/rollup module, the class the property-testing predicate covers, and
fast-check is already a dependency with an established *.property.test.cjs pattern.
Adds 5 properties over the algebraic invariants: closed-set identity
(applicable === resolved + unresolved), byVerification sums ≤ resolved, per-tier
recount + resolved-status-only counting, rollup determinism, and stable orphan
rejection. 200 runs/seed 42 via helpers/fast-check-setup.cjs.
* test(#550): align allow-test-rule tokens to canonical runtime-contract-is-the-product (N3)
trek-e re-review #7 N3 (Nit): the // allow-test-rule: tokens (source-text-is-the-product,
docs-parity) differed from CONTEXT.md's canonical exemption category
'runtime-contract-is-the-product' (RULESET.TESTS.no-source-grep.exemption, CONTEXT.md:240).
All three tests assert deployed runtime-contract surfaces (spec-phase.md Step 5.5, the
plan-phase.md planner prompt, the rendered reference/SPEC/ADR vocabulary), so the canonical
category fits; each now carries a one-line justification per the ruleset format. Free-text
reason — lint behavior unchanged.
* test(#550): re-baseline plan-phase + spec-phase byte sizes for edge-probe
Rebased onto next (e4f0910d), which replaced the line-based tier-max
size ratchet (#597) with the byte-based per-file baseline guard (#1074).
The edge-probe feature legitimately grows two workflows:
- spec-phase.md 15131 -> 23094 (+7963): Step 5.5 Edge-Completeness Probe
- plan-phase.md 93135 -> 94253 (+1118): covered/backstop edge lift into
must_haves.truths (the live <downstream_consumer> block)
Both remain under their tier hard caps (plan-phase XL 98304, ~4KB
headroom; spec-phase DEFAULT 40960). Growth is real inline workflow
content the feature requires at that step — not eager @-import proxy
gaming. Drops the obsolete line-based XL_BUDGET 93000->94000 bump
(superseded by the byte baseline) via rebase.
* chore(#550): reconcile INVENTORY headline counts after rebase onto next
Rebase onto current next dropped the prior reconcile commit (stale counts
refs 68 / CLI 107). Current next + the edge-probe additions yield:
- References (68 -> 69 shipped): + gsd-core/references/edge-probe.md
- CLI Modules (107 -> 109 shipped): + edge-probe.cjs + probe-core.cjs
Rows for all three already present; only the headline counts were stale.
Caught by tests/inventory-counts.test.cjs (CI ubuntu-24 leg).
This commit is contained in:
Rezolv
217
src/edge-probe.cts
Normal file
217
src/edge-probe.cts
Normal file
@@ -0,0 +1,217 @@
|
||||
/**
|
||||
* Spec-completeness edge-probe — the FIRST adapter of the probe-core resolution model
|
||||
* (ADR-457 build model; ADR-550 Decision 7 seam).
|
||||
*
|
||||
* The generic resolution lifecycle, the status×verification re-cut, `validateResolution`,
|
||||
* `validateRequirement`, the `analyzeCoverage` merge/rollup/orphan-reject engine, and the
|
||||
* `runProbeCli` scaffold all live in `src/probe-core.cts`. This module keeps ONLY the
|
||||
* edge-specific cluster: the five data/behavior shapes, the closed 8-category edge taxonomy,
|
||||
* shape classification, edge proposal, and the `{ explicit, backstop }` verification validators.
|
||||
*
|
||||
* Authored as strict TypeScript (`src/edge-probe.cts`) and compiled by
|
||||
* `tsc -p tsconfig.build.json` to the gitignored runtime artifact
|
||||
* `gsd-core/bin/lib/edge-probe.cjs`. Do NOT hand-write the `.cjs`; it is emitted. Tests
|
||||
* `require()` the built artifact; `pretest` runs `build:lib` first.
|
||||
*
|
||||
* Pure and dependency-free: it classifies each requirement's data/behavior shape, filters
|
||||
* the closed 8-category edge taxonomy to applicable categories, proposes concrete candidate
|
||||
* edges, and (via probe-core) merges author resolutions into a coverage report.
|
||||
*/
|
||||
|
||||
import {
|
||||
type Item,
|
||||
type Resolution,
|
||||
type CoverageReport,
|
||||
type Validators,
|
||||
validateRequirement as coreValidateRequirement,
|
||||
validateResolution as coreValidateResolution,
|
||||
analyzeCoverage as coreAnalyzeCoverage,
|
||||
runProbeCli,
|
||||
} from './probe-core.cjs';
|
||||
|
||||
/** The five data/behavior shapes a requirement can exhibit. */
|
||||
export type Shape = 'numeric-range' | 'collection' | 'text' | 'stateful' | 'io';
|
||||
|
||||
/** The edge probe's verification tiers (the `verification` axis values for a resolved edge). */
|
||||
export type EdgeVerification = 'explicit' | 'backstop';
|
||||
|
||||
/** A single edge taxonomy category. */
|
||||
export interface TaxonomyEntry {
|
||||
id: string;
|
||||
name: string;
|
||||
shapes: Shape[];
|
||||
probe: string;
|
||||
}
|
||||
|
||||
/** A SPEC requirement; `shapes` is an optional authored override of classification. */
|
||||
export interface Requirement {
|
||||
id: string;
|
||||
text: string;
|
||||
shapes?: Shape[];
|
||||
}
|
||||
|
||||
/** An edge item — a probe-core `Item` specialized to the edge verification vocabulary. */
|
||||
export type Edge = Item<EdgeVerification>;
|
||||
|
||||
/**
|
||||
* Word-boundary cues mapping requirement prose -> data/behavior shape.
|
||||
* Heuristic and intentionally lossy; an authored `shapes` array overrides it.
|
||||
*/
|
||||
export const SHAPE_CUES: Record<Shape, RegExp> = {
|
||||
'numeric-range': /\b(round(ing|ed)?|threshold|max(imum)?|min(imum)?|limit|bound(ary)?|between|cap|percent|amount|price|count|number|score|rate|decimal)\b/i,
|
||||
'collection': /\b(lists?|arrays?|sets?|items?|collections?|each|every|all|sort(ed|ing)?|merge|dedupe|group|ranges?|intervals?|overlap(ping)?)\b/i,
|
||||
'text': /\b(string|text|names?|labels?|truncate|substring|char(acter)?s?|length|slug|message|unicode)\b/i,
|
||||
'stateful': /\b(save|persist|store|update|toggle|create|delete|remove|submit|retry|apply|register|insert)\b/i,
|
||||
'io': /\b(files?|requests?|fetch|upload|download|network|api|endpoints?|connections?|sockets?)\b/i,
|
||||
};
|
||||
|
||||
/** The locked shape vocabulary — exactly the keys of SHAPE_CUES (single source of truth). */
|
||||
export const VALID_SHAPES: ReadonlySet<string> = new Set(Object.keys(SHAPE_CUES));
|
||||
|
||||
/** Detect which shapes a requirement's prose matches (heuristic). */
|
||||
export function classifyShape(text: string): Shape[] {
|
||||
const shapes: Shape[] = [];
|
||||
const subject = String(text == null ? '' : text);
|
||||
for (const shape of Object.keys(SHAPE_CUES) as Shape[]) {
|
||||
if (SHAPE_CUES[shape].test(subject)) shapes.push(shape);
|
||||
}
|
||||
return shapes;
|
||||
}
|
||||
|
||||
/**
|
||||
* Closed taxonomy of 8 domain-boundary edge categories (established QA names).
|
||||
* `shapes` lists which requirement shapes make the category relevant.
|
||||
*/
|
||||
export const TAXONOMY: TaxonomyEntry[] = [
|
||||
{ id: 'boundary', name: 'Boundary values', shapes: ['numeric-range'], probe: 'What happens exactly at each min/max/threshold — and one step either side?' },
|
||||
{ id: 'adjacency', name: 'Adjacency / touching', shapes: ['collection'], probe: 'When two things are exactly equal or just touch, do they merge, collide, or separate?' },
|
||||
{ id: 'empty', name: 'Empty / degenerate', shapes: ['collection', 'text'], probe: 'What is the result for empty, single-element, or null input?' },
|
||||
{ id: 'encoding', name: 'Encoding / representation', shapes: ['text'], probe: 'Whose definition of length/equality applies — bytes, code points, grapheme clusters, or normalized form?' },
|
||||
{ id: 'ordering', name: 'Ordering / stability', shapes: ['collection'], probe: 'When elements compare equal, is output order specified and stable?' },
|
||||
{ id: 'precision', name: 'Precision / overflow', shapes: ['numeric-range'], probe: 'Where can precision loss or overflow occur, and what is the contract?' },
|
||||
{ id: 'idempotency', name: 'Idempotency / repetition', shapes: ['stateful'], probe: 'What happens if this runs twice on the same input?' },
|
||||
{ id: 'concurrency', name: 'Concurrency / effect ordering', shapes: ['stateful', 'io'], probe: 'If interrupted or run in parallel, what is guaranteed?' },
|
||||
];
|
||||
|
||||
/** Return taxonomy category ids whose applicable shapes intersect the input set. */
|
||||
export function applicableCategories(shapes: Shape[]): string[] {
|
||||
const set = new Set<Shape>(shapes);
|
||||
return TAXONOMY.filter((c) => c.shapes.some((s) => set.has(s))).map((c) => c.id);
|
||||
}
|
||||
|
||||
/**
|
||||
* The edge adapter's injected runtime validators (ADR-550 #5). `categories` is the closed
|
||||
* taxonomy; both verification tiers require a non-empty `resolution` (an explicit AC's text
|
||||
* or a backstop note) so plan-phase has a criterion to lift.
|
||||
*/
|
||||
export const EDGE_VALIDATORS: Validators = {
|
||||
categories: TAXONOMY.map((c) => c.id),
|
||||
verification: ['explicit', 'backstop'],
|
||||
requiredFieldsByVerification: { explicit: ['resolution'], backstop: ['resolution'] },
|
||||
};
|
||||
|
||||
/**
|
||||
* Validate a single requirement — the generic id/text checks (probe-core) plus the edge's
|
||||
* `shapes`-must-be-an-array check. A bare string like `shapes:"numeric-range"` would otherwise
|
||||
* fall through to prose classification, silently ignoring the authored override.
|
||||
*
|
||||
* The edge adapter's `text` is REQUIRED (the prose is the classification signal), so reject a
|
||||
* missing/empty `text` when no authored `shapes` override is present. Without this, a `{ id }`
|
||||
* requirement classifies to zero shapes → zero edges → it is silently DROPPED from coverage
|
||||
* with no signal — the exact fail-open this feature exists to eliminate. An explicit `shapes`
|
||||
* array (including `[]` for "no applicable categories") is the legitimate way to opt out of
|
||||
* prose classification, so `text` is only required when `shapes` is absent.
|
||||
*/
|
||||
export function validateRequirement(requirement: Requirement): void {
|
||||
coreValidateRequirement(requirement);
|
||||
const r = requirement as unknown as { shapes?: unknown; text?: unknown };
|
||||
if (r.shapes != null && !Array.isArray(r.shapes)) {
|
||||
throw new Error(`requirement ${requirement.id} shapes must be an array when present`);
|
||||
}
|
||||
if (r.shapes == null && !(typeof r.text === 'string' && r.text.trim())) {
|
||||
throw new Error(
|
||||
`requirement ${requirement.id} text must be a non-empty string when no shapes override is provided`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** Validate an edge resolution against the edge verification vocabulary. */
|
||||
export function validateResolution(resolution: Resolution<EdgeVerification>): true {
|
||||
return coreValidateResolution(resolution, EDGE_VALIDATORS);
|
||||
}
|
||||
|
||||
/**
|
||||
* Propose candidate edges for a requirement. Uses authored `shapes` when present, else
|
||||
* classifies from prose. Every proposed edge starts unresolved (verification null).
|
||||
*/
|
||||
export function proposeEdges(requirement: Requirement): Edge[] {
|
||||
validateRequirement(requirement);
|
||||
let shapes: Shape[];
|
||||
if (Array.isArray(requirement.shapes)) {
|
||||
// Fail closed: an authored array must contain only locked shape values. A non-empty
|
||||
// but invalid array (e.g. ['numeric'], a typo for 'numeric-range') would otherwise
|
||||
// intersect no category and silently suppress every probe — the gate reads green while
|
||||
// nothing was checked. An empty array stays a valid "no applicable categories" override.
|
||||
for (const s of requirement.shapes) {
|
||||
if (typeof s !== 'string' || !VALID_SHAPES.has(s)) {
|
||||
throw new Error(
|
||||
`invalid shape ${JSON.stringify(s)} for requirement ${requirement.id} — must be one of: ${[...VALID_SHAPES].join(', ')}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
shapes = requirement.shapes;
|
||||
} else {
|
||||
shapes = classifyShape(requirement.text);
|
||||
}
|
||||
return applicableCategories(shapes).map((catId): Edge => {
|
||||
const cat = TAXONOMY.find((c) => c.id === catId);
|
||||
return {
|
||||
requirement_id: requirement.id,
|
||||
category: catId,
|
||||
status: 'unresolved',
|
||||
verification: null,
|
||||
resolution: null,
|
||||
reason: null,
|
||||
probe: cat ? cat.probe : '',
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Propose edges for every requirement (deterministic propose), then delegate the
|
||||
* merge/rollup/orphan-reject to probe-core. Edge-specific pre-checks: requirements must be an
|
||||
* array, requirement ids must be unique. Throws on any invalid resolution.
|
||||
*/
|
||||
export function analyzeCoverage(
|
||||
requirements: Requirement[],
|
||||
resolutions: Resolution<EdgeVerification>[] = [],
|
||||
): CoverageReport<EdgeVerification> {
|
||||
if (!Array.isArray(requirements)) {
|
||||
throw new Error('requirements must be an array');
|
||||
}
|
||||
const items: Edge[] = [];
|
||||
const seenReqIds = new Set<string>();
|
||||
for (const req of requirements) {
|
||||
validateRequirement(req);
|
||||
if (seenReqIds.has(req.id)) {
|
||||
throw new Error(`duplicate requirement id ${JSON.stringify(req.id)}`);
|
||||
}
|
||||
seenReqIds.add(req.id);
|
||||
for (const edge of proposeEdges(req)) items.push(edge);
|
||||
}
|
||||
return coreAnalyzeCoverage(items, resolutions, EDGE_VALIDATORS);
|
||||
}
|
||||
|
||||
/*
|
||||
* CLI entry (EP-06 invokable surface): `edge-probe.cjs <requirements.json> [resolutions.json]`.
|
||||
* The generic I/O plumbing (parse, fail-closed exit 2, pretty-JSON out) lives in probe-core's
|
||||
* `runProbeCli`; the edge adapter supplies its `analyzeCoverage`. Guarded by
|
||||
* `require.main === module` so it runs only when the compiled `.cjs` is executed directly.
|
||||
*/
|
||||
if (require.main === module) {
|
||||
runProbeCli(
|
||||
(requirements, resolutions) =>
|
||||
analyzeCoverage(requirements as Requirement[], resolutions as Resolution<EdgeVerification>[]),
|
||||
{ usage: 'edge-probe.cjs <requirements.json> [resolutions.json]' },
|
||||
);
|
||||
}
|
||||
336
src/probe-core.cts
Normal file
336
src/probe-core.cts
Normal file
@@ -0,0 +1,336 @@
|
||||
/**
|
||||
* probe-core — generic spec-phase probe resolution model (ADR-550 Decision 7).
|
||||
*
|
||||
* Extracted from the edge-probe (the first adapter) once the prohibition probe (#644)
|
||||
* proved it the *second* adapter of the same model: one adapter is a hypothetical seam,
|
||||
* two is a real one. This module owns everything generic — the resolution lifecycle,
|
||||
* the status×verification re-cut, `validateResolution`/`validateRequirement`, the
|
||||
* `analyzeCoverage(items, resolutions?, validators)` merge/rollup/orphan-reject engine,
|
||||
* the `byVerification` rollup, and the `runProbeCli` I/O scaffold. Each probe is a thin
|
||||
* adapter: it supplies the proposal logic (deterministic for edge, LLM-recall for
|
||||
* prohibition) and its closed vocabularies via injected validators.
|
||||
*
|
||||
* Authored as strict TypeScript (`src/probe-core.cts`) and compiled by
|
||||
* `tsc -p tsconfig.build.json` to the gitignored runtime artifact
|
||||
* `gsd-core/bin/lib/probe-core.cjs`. Do NOT hand-write the `.cjs`; it is emitted.
|
||||
*
|
||||
* Two orthogonal axes (the re-cut):
|
||||
* - status: resolved | dismissed | unresolved — the resolution LIFECYCLE (shared)
|
||||
* - verification: <probe-defined> | null — HOW a resolved item is verified
|
||||
* The edge adapter declares `verification: explicit | backstop`; the prohibition adapter
|
||||
* (#644) will declare `test | judgment`. Splitting the axes keeps the lifecycle enum free
|
||||
* of a verification fact and lets a sibling probe add its own tiers without a parallel enum.
|
||||
*
|
||||
* Typing is hybrid (ADR-550 #5): generic type params for adapter DX, but enforcement runs
|
||||
* through injected runtime validators, because the CLI executes over JSON where TS types are
|
||||
* erased. The contract test pins the validators, not the types.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
|
||||
/** Resolution lifecycle — shared across every probe adapter. */
|
||||
export type Status = 'resolved' | 'dismissed' | 'unresolved';
|
||||
|
||||
/** The LOCKED set of valid lifecycle statuses (the re-cut: no covered/backstop). */
|
||||
export const VALID_STATUS: Status[] = ['resolved', 'dismissed', 'unresolved'];
|
||||
|
||||
/**
|
||||
* A proposed or resolved item for a requirement/category pair. Generic over the probe's
|
||||
* verification-tier vocabulary `V` (edge: `'explicit' | 'backstop'`). A freshly proposed
|
||||
* item is `{ status: 'unresolved', verification: null, resolution: null, reason: null }`.
|
||||
*/
|
||||
export interface Item<V extends string = string> {
|
||||
requirement_id: string;
|
||||
category: string;
|
||||
status: Status;
|
||||
verification: V | null;
|
||||
resolution: string | null;
|
||||
reason: string | null;
|
||||
probe: string;
|
||||
}
|
||||
|
||||
/** An author resolution merged onto a proposed item. */
|
||||
export interface Resolution<V extends string = string> {
|
||||
requirement_id: string;
|
||||
category: string;
|
||||
status: Status;
|
||||
verification?: V | null;
|
||||
resolution?: string | null;
|
||||
reason?: string | null;
|
||||
}
|
||||
|
||||
/** A coverage report: the merged items plus rollup counts (incl. the per-tier breakdown). */
|
||||
export interface CoverageReport<V extends string = string> {
|
||||
items: Item<V>[];
|
||||
coverage: {
|
||||
applicable: number;
|
||||
resolved: number;
|
||||
unresolved: number;
|
||||
byVerification: Record<string, number>;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The injected runtime enforcement contract (ADR-550 #5). The probe declares its closed
|
||||
* vocabularies so `analyzeCoverage`/`validateResolution` can enforce them at the JSON
|
||||
* boundary where TS types no longer exist.
|
||||
* - `categories` — the probe's valid category ids (a proposed item outside this set is an
|
||||
* adapter bug, caught rather than silently rolled up).
|
||||
* - `verification` — the valid verification tiers for a `resolved` item.
|
||||
* - `requiredFieldsByVerification` — for each tier, which resolution fields MUST be a
|
||||
* non-empty string (edge: every tier needs `resolution` text for plan-phase to lift).
|
||||
*/
|
||||
export interface Validators {
|
||||
categories: string[];
|
||||
verification: string[];
|
||||
requiredFieldsByVerification: Record<string, Array<'resolution' | 'reason'>>;
|
||||
}
|
||||
|
||||
/** A generic requirement — every probe ingests at least `{ id, text }`. */
|
||||
export interface Requirement {
|
||||
id: string;
|
||||
text?: string;
|
||||
}
|
||||
|
||||
function errMessage(e: unknown): string {
|
||||
return e instanceof Error ? e.message : String(e);
|
||||
}
|
||||
|
||||
/**
|
||||
* Structural guard for the report an adapter's `analyze` returns. The scaffold types `analyze`
|
||||
* loosely (it runs over JSON-parsed input the adapter `as`-casts), so a future adapter (#644)
|
||||
* that forgets to validate inside its closure could hand back a malformed object. Rather than
|
||||
* stringify garbage as green output, `runProbeCli` checks the report shape and fails closed.
|
||||
*/
|
||||
function isValidReport(report: unknown): report is CoverageReport {
|
||||
if (report == null || typeof report !== 'object') return false;
|
||||
const r = report as { items?: unknown; coverage?: unknown };
|
||||
if (!Array.isArray(r.items)) return false;
|
||||
const c = r.coverage as
|
||||
| { applicable?: unknown; resolved?: unknown; unresolved?: unknown; byVerification?: unknown }
|
||||
| undefined;
|
||||
if (c == null || typeof c !== 'object') return false;
|
||||
if (typeof c.applicable !== 'number' || typeof c.resolved !== 'number' || typeof c.unresolved !== 'number') {
|
||||
return false;
|
||||
}
|
||||
if (c.byVerification == null || typeof c.byVerification !== 'object') return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a requirement's generic structural fields — fail closed on malformed input rather
|
||||
* than coercing it. Probe-specific fields (e.g. the edge adapter's `shapes`) are validated by
|
||||
* the adapter. Typed loosely because the CLI casts arbitrary parsed JSON to `Requirement`.
|
||||
*/
|
||||
export function validateRequirement(requirement: Requirement): void {
|
||||
const r = requirement as unknown as { id?: unknown; text?: unknown };
|
||||
if (typeof r.id !== 'string' || !r.id.trim()) {
|
||||
throw new Error(`requirement id must be a non-empty string (got ${JSON.stringify(r.id)})`);
|
||||
}
|
||||
if (r.text != null && typeof r.text !== 'string') {
|
||||
throw new Error(`requirement ${r.id} text must be a string when present`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a resolution against the probe's injected validators. Rejects an unknown status,
|
||||
* a dismissal without a non-empty reason, a `resolved` item with a missing/unknown
|
||||
* verification tier, and a `resolved` item missing any field its tier requires (per
|
||||
* `requiredFieldsByVerification`). Returns true on success.
|
||||
*/
|
||||
export function validateResolution<V extends string>(r: Resolution<V>, validators: Validators): true {
|
||||
const key = `${r.requirement_id}::${r.category}`;
|
||||
if (!VALID_STATUS.includes(r.status)) {
|
||||
throw new Error(`invalid status "${r.status}" for ${key}`);
|
||||
}
|
||||
// Invariant (this module's header): `verification` is null unless `status` is `resolved`.
|
||||
// Enforce it for EVERY status — a dismissed/unresolved resolution carrying a verification
|
||||
// tier would otherwise merge verbatim (`analyzeCoverage` below) and silently break the
|
||||
// model for the second adapter (#644) that inherits this seam. Fail closed across the full
|
||||
// status×verification space, not just `resolved`.
|
||||
if (r.status !== 'resolved' && r.verification != null) {
|
||||
throw new Error(`verification must be null unless status is "resolved" (got "${r.verification}") for ${key}`);
|
||||
}
|
||||
// An `unresolved` resolution is an UNACTED item: it must carry no resolution/reason payload.
|
||||
// A populated payload is an authoring mistake (the author meant resolved/dismissed) that
|
||||
// would otherwise be silently dropped into the unresolved count with no error pointing at
|
||||
// it. Reject it so the mistake surfaces.
|
||||
if (r.status === 'unresolved') {
|
||||
if (r.resolution != null && String(r.resolution).trim()) {
|
||||
throw new Error(`unresolved must not carry a resolution (${key})`);
|
||||
}
|
||||
if (r.reason != null && String(r.reason).trim()) {
|
||||
throw new Error(`unresolved must not carry a reason (${key})`);
|
||||
}
|
||||
}
|
||||
if (r.status === 'dismissed' && !(r.reason && String(r.reason).trim())) {
|
||||
throw new Error(`dismissed requires a reason (${key})`);
|
||||
}
|
||||
if (r.status === 'resolved') {
|
||||
const tier = r.verification;
|
||||
if (tier == null) {
|
||||
throw new Error(`resolved requires a verification tier (one of: ${validators.verification.join(', ')}) for ${key}`);
|
||||
}
|
||||
if (!validators.verification.includes(tier)) {
|
||||
throw new Error(`invalid verification "${tier}" for ${key} — must be one of: ${validators.verification.join(', ')}`);
|
||||
}
|
||||
const required = validators.requiredFieldsByVerification[tier] ?? [];
|
||||
for (const field of required) {
|
||||
// field is 'resolution' | 'reason'; both are `string | null | undefined` on Resolution,
|
||||
// so the indexed access is string-typed (no unknown-to-string coercion).
|
||||
const value = r[field];
|
||||
if (!(value != null && String(value).trim())) {
|
||||
throw new Error(`${tier} requires a ${field} (${key})`);
|
||||
}
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge author resolutions onto ALREADY-PROPOSED items and roll up coverage counts.
|
||||
*
|
||||
* Core operates on `items[]`, never a `proposeFn`: probes have different deterministic
|
||||
* surfaces (edge = deterministic propose + LLM resolve; prohibition = LLM propose + deterministic
|
||||
* validate/merge), so proposal stays in each adapter and core must not assume it is deterministic.
|
||||
*
|
||||
* `coverage.resolved` is the COUNT of CLOSED items (`resolved` + `dismissed` status) =
|
||||
* `applicable - unresolved` — the pre-re-cut "covered + dismissed + backstop" set,
|
||||
* count-preserved. `byVerification` breaks the `resolved`-status items down by tier (each tier
|
||||
* initialized to 0). Throws on any invalid resolution, a duplicate, an orphan (a resolution
|
||||
* matching no proposed item), or a proposed item whose category is outside `validators.categories`.
|
||||
*/
|
||||
export function analyzeCoverage<V extends string>(
|
||||
items: Item<V>[],
|
||||
resolutions: Resolution<V>[] = [],
|
||||
validators: Validators,
|
||||
): CoverageReport<V> {
|
||||
if (!Array.isArray(items)) {
|
||||
throw new Error('items must be an array');
|
||||
}
|
||||
const key = (r: { requirement_id: string; category: string }): string => `${r.requirement_id}::${r.category}`;
|
||||
const resMap = new Map<string, Resolution<V>>();
|
||||
for (const r of resolutions) {
|
||||
validateResolution(r, validators);
|
||||
if (resMap.has(key(r))) {
|
||||
throw new Error(`duplicate resolution for ${key(r)}`);
|
||||
}
|
||||
resMap.set(key(r), r);
|
||||
}
|
||||
const validCategories = new Set(validators.categories);
|
||||
const merged: Item<V>[] = [];
|
||||
const itemKeys = new Set<string>();
|
||||
for (const item of items) {
|
||||
if (!validCategories.has(item.category)) {
|
||||
throw new Error(`item ${key(item)} has unknown category "${item.category}" — not one of: ${validators.categories.join(', ')}`);
|
||||
}
|
||||
itemKeys.add(key(item));
|
||||
const o = resMap.get(key(item));
|
||||
if (o) {
|
||||
merged.push({ ...item, status: o.status, verification: o.verification ?? null, resolution: o.resolution ?? null, reason: o.reason ?? null });
|
||||
} else {
|
||||
// No author resolution: the item is rolled up VERBATIM, so its own status/fields must be
|
||||
// valid too. The edge adapter only proposes `unresolved` items, but the prohibition adapter
|
||||
// (#644) proposes LLM-generated items that arrive already populated — one carrying an
|
||||
// out-of-enum status (e.g. the dropped "covered") or `dismissed` with no reason would
|
||||
// otherwise be counted closed without validation. An Item is structurally a superset of a
|
||||
// Resolution, so the same fail-closed check guards both. (ADR-550 Decision 5 hardens this
|
||||
// shared seam for the second adapter; m1.)
|
||||
validateResolution(item as unknown as Resolution<V>, validators);
|
||||
merged.push(item);
|
||||
}
|
||||
}
|
||||
// Reject orphan resolutions — a resolution whose (requirement_id, category) matches no
|
||||
// proposed item (typo'd category or a non-applicable one) would otherwise be silently
|
||||
// dropped, leaving the author believing an item is resolved while the report shows it
|
||||
// unresolved (adversarial-review HIGH; preserved from the edge-probe's original engine).
|
||||
for (const k of resMap.keys()) {
|
||||
if (!itemKeys.has(k)) {
|
||||
throw new Error(`unknown resolution for ${k} — no matching proposed item (typo'd category or non-applicable shape?)`);
|
||||
}
|
||||
}
|
||||
const unresolved = merged.filter((i) => i.status === 'unresolved').length;
|
||||
const applicable = merged.length;
|
||||
const resolved = applicable - unresolved; // closed set: resolved-status + dismissed
|
||||
const byVerification: Record<string, number> = {};
|
||||
for (const tier of validators.verification) byVerification[tier] = 0;
|
||||
for (const i of merged) {
|
||||
if (i.status === 'resolved' && i.verification != null) {
|
||||
byVerification[i.verification] = (byVerification[i.verification] ?? 0) + 1;
|
||||
}
|
||||
}
|
||||
return { items: merged, coverage: { applicable, resolved, unresolved, byVerification } };
|
||||
}
|
||||
|
||||
/*
|
||||
* CLI scaffold (the EP-06 invokable surface, generalized). Each probe ships one bin that
|
||||
* calls `runProbeCli` with its own `analyze` (closing over the adapter's propose + validators)
|
||||
* and usage string; a single dispatcher CLI is a deferred follow-on. The I/O dependencies are
|
||||
* injectable so the generic plumbing is unit-testable without spawning a process.
|
||||
*
|
||||
* `tsconfig.build.json` sets `"types": ["node"]`, so `process` and `node:fs` are typed.
|
||||
*/
|
||||
|
||||
/** Injectable I/O for `runProbeCli` (defaults wire to the real process). */
|
||||
export interface ProbeCliOptions {
|
||||
usage: string;
|
||||
argv?: string[];
|
||||
readFile?: (path: string) => string;
|
||||
write?: (s: string) => void;
|
||||
writeErr?: (s: string) => void;
|
||||
exit?: (code: number) => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the requirements file (and optional resolutions file), run the adapter's `analyze`,
|
||||
* and write the report as pretty JSON + newline. With no requirements path, writes the usage
|
||||
* line to stderr and exits 2. A JSON-parse failure or any `analyze` throw is a handled error:
|
||||
* stderr + exit 2, never an uncaught stack trace — so the engine's fail-closed validation
|
||||
* surfaces at the workflow boundary rather than failing open.
|
||||
*/
|
||||
export function runProbeCli(
|
||||
analyze: (requirements: unknown, resolutions: unknown) => CoverageReport,
|
||||
options: ProbeCliOptions,
|
||||
): void {
|
||||
const argv = options.argv ?? process.argv;
|
||||
const readFile = options.readFile ?? ((p: string) => fs.readFileSync(p, 'utf8'));
|
||||
const write = options.write ?? ((s: string) => { process.stdout.write(s); });
|
||||
const writeErr = options.writeErr ?? ((s: string) => { process.stderr.write(s); });
|
||||
const exit = options.exit ?? ((code: number) => { process.exit(code); });
|
||||
|
||||
const reqPath: string | undefined = argv[2];
|
||||
const resPath: string | undefined = argv[3];
|
||||
if (!reqPath) {
|
||||
writeErr(`usage: ${options.usage}\n`);
|
||||
exit(2);
|
||||
return;
|
||||
}
|
||||
let requirements: unknown;
|
||||
try {
|
||||
requirements = JSON.parse(readFile(reqPath));
|
||||
} catch (e: unknown) {
|
||||
writeErr(`error: cannot parse JSON from ${reqPath}: ${errMessage(e)}\n`);
|
||||
exit(2);
|
||||
return;
|
||||
}
|
||||
let resolutions: unknown = [];
|
||||
if (resPath) {
|
||||
try {
|
||||
resolutions = JSON.parse(readFile(resPath));
|
||||
} catch (e: unknown) {
|
||||
writeErr(`error: cannot parse JSON from ${resPath}: ${errMessage(e)}\n`);
|
||||
exit(2);
|
||||
return;
|
||||
}
|
||||
}
|
||||
try {
|
||||
const report = analyze(requirements, resolutions);
|
||||
if (!isValidReport(report)) {
|
||||
throw new Error('adapter returned a structurally-invalid coverage report (expected { items[], coverage{ applicable, resolved, unresolved, byVerification } })');
|
||||
}
|
||||
write(`${JSON.stringify(report, null, 2)}\n`);
|
||||
} catch (e: unknown) {
|
||||
writeErr(`error: ${errMessage(e)}\n`);
|
||||
exit(2);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user