Files
msd-core/src/edge-probe.cts
Tom Boucher bf4485ada2 enhance(#3717): make the edge probe's shape cues language-aware via an optional text_en field (#4156)
* test(#3717): add failing-first coverage for text_en language-aware classification

Adds unit tests for the not-yet-implemented text_en field on Requirement
(fallback selection, empty/whitespace/non-string rejection, shapes-override
precedence), a SHAPE_CUES/VALID_SHAPES parity guard (RULESET.GENERATIVE-FIX),
and workflow-prose contract tests asserting spec-phase.md Step 5.5 documents
populating text_en for response_language projects. All new tests are RED
until src/edge-probe.cts and the workflow docs are updated.

* feat(#3717): make edge-probe shape classification read an optional text_en field

Requirement gains an optional text_en; classifyShape's own signature stays
untouched (a locked, directly-tested export), and the text_en ?? text
selection is pushed to proposeEdges' single call site instead. text_en is
validated fail-closed: an empty or whitespace-only value throws rather than
silently winning the ?? fallback and degrading classification to zero shapes.

This makes the #2773 doc-only translation convention an explicit,
validatable field instead of an invisible instruction, per the approved
Form-1 scope on #3717.

* docs(#3717): document the text_en field across spec-phase, reference and how-to docs

Updates Step 5.5's response_language instructions, the edge-probe reference
Inputs contract, the FEATURES.md fragment, and the non-English how-to guide
to describe the new text_en field: text keeps the requirement's own wording
in all cases, text_en (when populated) is the engine-only English rendering
the classifier prefers.

* docs(#3717): record the text_en locked-surface change in CONTEXT.md and ADR-550

Updates the Edge Probe Module glossary entry to describe the text_en field
and its fail-closed validation, and appends an ADR-550 amendment recording
why this is additive and does not re-open the #652 LLM-classifier rejection
(text_en is a plain field read by the existing deterministic regex
classifier, not a new model-dependent surface).

* docs(#3717): add changeset fragment and regenerate FEATURES.md

pr:0 placeholder — backfilled with the real PR number after the PR opens.

* docs(#3717): attribute the text_en machine check to engine-level validation, not prose tests

Code-review (Spec axis) finding: the workflow-prose contract tests and the
ADR-550 amendment overclaimed themselves as "the machine check the #2773
doc-only stopgap lacked." That check is actually engine-level
(validateRequirement/classifyShape, covered in tests/edge-probe.test.cjs) —
the prose tests are the same style of assertion #2773 already used. Reworded
both to attribute the claim correctly.

* fix(#3717): rewrap spec-phase.md so the id-unchanged sentence stays on one line

The #3717 rewrite of Step 5.5's response_language paragraph moved a line
break so "requirement `id`s" ended one physical line and "are never
translated" started the next. The pre-existing #2773 regression test
(tests/edge-probe-spec-phase-contract.test.cjs) asserts id + "never
translated" on the SAME line (no \n in between, matching git's own
line-oriented prose), so the reflow silently broke it. Rewrapped so the
sentence lands on one line again, verified against every #2773/#3717
regex assertion in that test file.

Emitted-Drift-Ack-Growth: spec-phase.md — #3717 adds text_en documentation to Step 5.5 (response_language paragraph + REQS_JSON heredoc comment); this growth is this PR's own diff, not incidental drift.

* chore(#3717): backfill changeset PR number

pr:0 -> pr:4156 now that the PR exists.

---------

Co-authored-by: sim <sim@local>
2026-09-01 21:39:53 -04:00

274 lines
14 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* Spec-completeness edge-probe — the FIRST adapter of the probe-core resolution model
* (ADR-457 build model; ADR-550 Decision 7 seam).
*
* The generic resolution lifecycle, the status×verification re-cut, `validateResolution`,
* `validateRequirement`, the `analyzeCoverage` merge/rollup/orphan-reject engine, and the
* `runProbeCli` scaffold all live in `src/probe-core.cts`. This module keeps ONLY the
* edge-specific cluster: the five data/behavior shapes, the closed 8-category edge taxonomy,
* shape classification, edge proposal, and the `{ explicit, backstop }` verification validators.
*
* Authored as strict TypeScript (`src/edge-probe.cts`) and compiled by
* `tsc -p tsconfig.build.json` to the gitignored runtime artifact
* `gsd-core/bin/lib/edge-probe.cjs`. Do NOT hand-write the `.cjs`; it is emitted. Tests
* `require()` the built artifact; `pretest` runs `build:lib` first.
*
* Pure and dependency-free: it classifies each requirement's data/behavior shape, filters
* the closed 8-category edge taxonomy to applicable categories, proposes concrete candidate
* edges, and (via probe-core) merges author resolutions into a coverage report.
*/
import {
type Item,
type Resolution,
type CoverageReport,
type Validators,
validateRequirement as coreValidateRequirement,
validateResolution as coreValidateResolution,
analyzeCoverage as coreAnalyzeCoverage,
runProbeCli,
} from './probe-core.cjs';
// eslint-disable-next-line @typescript-eslint/no-require-imports
import cliExitModule = require('./cli-exit.cjs');
const { runMain } = cliExitModule;
/** The five data/behavior shapes a requirement can exhibit. */
export type Shape = 'numeric-range' | 'collection' | 'text' | 'stateful' | 'io';
/** The edge probe's verification tiers (the `verification` axis values for a resolved edge). */
export type EdgeVerification = 'explicit' | 'backstop';
/** A single edge taxonomy category. */
export interface TaxonomyEntry {
id: string;
name: string;
shapes: Shape[];
probe: string;
}
/**
* A SPEC requirement; `shapes` is an optional authored override of classification.
*
* `text_en` (#3717) is an optional English translation of `text`, read by shape
* classification in preference to `text` when present (`text_en ?? text`). `SHAPE_CUES`
* are English-only word-boundary patterns, so a non-English `text` (e.g. a project running
* with `response_language` set) classifies to zero shapes unless `text_en` supplies an
* English rendering. `text` itself is unaffected and keeps its own meaning (the
* requirement's own text, in whatever language the SPEC uses) — only classification reads
* `text_en` preferentially.
*/
export interface Requirement {
id: string;
text: string;
text_en?: string;
shapes?: Shape[];
}
/** An edge item — a probe-core `Item` specialized to the edge verification vocabulary. */
export type Edge = Item<EdgeVerification>;
/**
* Word-boundary cues mapping requirement prose -> data/behavior shape.
* Heuristic and intentionally lossy; an authored `shapes` array overrides it.
*/
export const SHAPE_CUES: Record<Shape, RegExp> = {
'numeric-range': /\b(round(ing|ed)?|threshold|max(imum)?|min(imum)?|limit|bound(ary)?|between|cap|percent|amount|price|count|number|score|rate|decimal)\b/i,
'collection': /\b(lists?|arrays?|sets?|items?|collections?|each|every|all|sort(ed|ing)?|merge|dedupe|group|ranges?|intervals?|overlap(ping)?)\b/i,
'text': /\b(string|text|names?|labels?|truncate|substring|char(acter)?s?|length|slug|message|unicode)\b/i,
'stateful': /\b(save|persist|store|update|toggle|create|delete|remove|submit|retry|apply|register|insert)\b/i,
'io': /\b(files?|requests?|fetch|upload|download|network|api|endpoints?|connections?|sockets?)\b/i,
};
/** The locked shape vocabulary — exactly the keys of SHAPE_CUES (single source of truth). */
export const VALID_SHAPES: ReadonlySet<string> = new Set(Object.keys(SHAPE_CUES));
/** Detect which shapes a requirement's prose matches (heuristic). */
export function classifyShape(text: string): Shape[] {
const shapes: Shape[] = [];
const subject = String(text == null ? '' : text);
for (const shape of Object.keys(SHAPE_CUES) as Shape[]) {
if (SHAPE_CUES[shape].test(subject)) shapes.push(shape);
}
return shapes;
}
/**
* Closed taxonomy of 8 domain-boundary edge categories (established QA names).
* `shapes` lists which requirement shapes make the category relevant.
*/
export const TAXONOMY: TaxonomyEntry[] = [
{ id: 'boundary', name: 'Boundary values', shapes: ['numeric-range'], probe: 'What happens exactly at each min/max/threshold — and one step either side?' },
{ id: 'adjacency', name: 'Adjacency / touching', shapes: ['collection'], probe: 'When two things are exactly equal or just touch, do they merge, collide, or separate?' },
{ id: 'empty', name: 'Empty / degenerate', shapes: ['collection', 'text'], probe: 'What is the result for empty, single-element, or null input?' },
{ id: 'encoding', name: 'Encoding / representation', shapes: ['text'], probe: 'Whose definition of length/equality applies — bytes, code points, grapheme clusters, or normalized form?' },
{ id: 'ordering', name: 'Ordering / stability', shapes: ['collection'], probe: 'When elements compare equal, is output order specified and stable?' },
{ id: 'precision', name: 'Precision / overflow', shapes: ['numeric-range'], probe: 'Where can precision loss, overflow, or rounding/tie-breaking occur — and what is the exact contract (e.g. half-up vs half-to-even, ceil/floor/truncate)?' },
{ id: 'idempotency', name: 'Idempotency / repetition', shapes: ['stateful'], probe: 'What happens if this runs twice on the same input?' },
{ id: 'concurrency', name: 'Concurrency / effect ordering', shapes: ['stateful', 'io'], probe: 'If interrupted or run in parallel, what is guaranteed?' },
];
/** Return taxonomy category ids whose applicable shapes intersect the input set. */
export function applicableCategories(shapes: Shape[]): string[] {
const set = new Set<Shape>(shapes);
return TAXONOMY.filter((c) => c.shapes.some((s) => set.has(s))).map((c) => c.id);
}
/**
* The edge adapter's injected runtime validators (ADR-550 #5). `categories` is the closed
* taxonomy; both verification tiers require a non-empty `resolution` (an explicit AC's text
* or a backstop note) so plan-phase has a criterion to lift.
*/
/**
* Pseudo-category for a requirement whose prose matched NO shape cue (#1110). It is a soft
* "review manually" signal, NOT a 9th taxonomy category: it stays out of `TAXONOMY` (the closed
* eight) and only joins `EDGE_VALIDATORS.categories` so `analyzeCoverage` accepts the item.
*/
export const UNCLASSIFIED_CATEGORY = 'unclassified';
const UNCLASSIFIED_PROBE = 'unclassified — review manually';
export const EDGE_VALIDATORS: Validators = {
categories: [...TAXONOMY.map((c) => c.id), UNCLASSIFIED_CATEGORY],
verification: ['explicit', 'backstop'],
requiredFieldsByVerification: { explicit: ['resolution'], backstop: ['resolution'] },
};
/**
* Validate a single requirement — the generic id/text checks (probe-core) plus the edge's
* `shapes`-must-be-an-array check. A bare string like `shapes:"numeric-range"` would otherwise
* fall through to prose classification, silently ignoring the authored override.
*
* The edge adapter's `text` is REQUIRED (the prose is the classification signal), so reject a
* missing/empty `text` when no authored `shapes` override is present. Without this, a `{ id }`
* requirement classifies to zero shapes → zero edges → it is silently DROPPED from coverage
* with no signal — the exact fail-open this feature exists to eliminate. An explicit `shapes`
* array (including `[]` for "no applicable categories") is the legitimate way to opt out of
* prose classification, so `text` is only required when `shapes` is absent.
*/
export function validateRequirement(requirement: Requirement): void {
coreValidateRequirement(requirement);
const r = requirement as unknown as { shapes?: unknown; text?: unknown; text_en?: unknown };
if (r.shapes != null && !Array.isArray(r.shapes)) {
throw new Error(`requirement ${requirement.id} shapes must be an array when present`);
}
if (r.shapes == null && !(typeof r.text === 'string' && r.text.trim())) {
throw new Error(
`requirement ${requirement.id} text must be a non-empty string when no shapes override is provided`,
);
}
// text_en (#3717) is optional, but when present it must be a non-empty string. An empty
// string is NOT caught by `??` (only null/undefined are), so an unvalidated `text_en: ''`
// would silently win `text_en ?? text` and classify against '' — the same fail-open shape
// #1110/#2773 already exist to eliminate, just moved one field over. Validated
// unconditionally (not gated on whether `shapes` will make it unused) so bad data fails
// closed even when it happens to be dead for this particular call.
if (r.text_en != null && !(typeof r.text_en === 'string' && r.text_en.trim())) {
throw new Error(`requirement ${requirement.id} text_en must be a non-empty string when present`);
}
}
/** Validate an edge resolution against the edge verification vocabulary. */
export function validateResolution(resolution: Resolution<EdgeVerification>): true {
return coreValidateResolution(resolution, EDGE_VALIDATORS);
}
/**
* Propose candidate edges for a requirement. Uses authored `shapes` when present, else
* classifies from prose. Every proposed edge starts unresolved (verification null).
*/
export function proposeEdges(requirement: Requirement): Edge[] {
validateRequirement(requirement);
let shapes: Shape[];
if (Array.isArray(requirement.shapes)) {
// Fail closed: an authored array must contain only locked shape values. A non-empty
// but invalid array (e.g. ['numeric'], a typo for 'numeric-range') would otherwise
// intersect no category and silently suppress every probe — the gate reads green while
// nothing was checked. An empty array stays a valid "no applicable categories" override.
for (const s of requirement.shapes) {
if (typeof s !== 'string' || !VALID_SHAPES.has(s)) {
throw new Error(
`invalid shape ${JSON.stringify(s)} for requirement ${requirement.id} — must be one of: ${[...VALID_SHAPES].join(', ')}`,
);
}
}
shapes = requirement.shapes;
} else {
// #3717: prefer the English translation when present — SHAPE_CUES are English-only
// word-boundary patterns, so a non-English `text` (e.g. response_language projects)
// would otherwise classify to zero shapes. validateRequirement (called above) has
// already guaranteed text_en, if present, is a non-empty string.
shapes = classifyShape(requirement.text_en ?? requirement.text);
if (shapes.length === 0) {
// Prose present but no shape cue matched. Do NOT silently drop it (#1110): an
// edge-relevant requirement whose phrasing missed every cue would otherwise vanish from
// coverage with no signal — the exact blind spot this probe exists to catch. Surface ONE
// soft, dismissible "unclassified — review manually" candidate. The explicit `shapes: []`
// opt-out (handled above) stays silent — that is the author's deliberate "no edge surface".
return [{
requirement_id: requirement.id,
category: UNCLASSIFIED_CATEGORY,
status: 'unresolved',
verification: null,
resolution: null,
reason: null,
probe: UNCLASSIFIED_PROBE,
}];
}
}
return applicableCategories(shapes).map((catId): Edge => {
const cat = TAXONOMY.find((c) => c.id === catId);
return {
requirement_id: requirement.id,
category: catId,
status: 'unresolved',
verification: null,
resolution: null,
reason: null,
probe: cat ? cat.probe : '',
};
});
}
/**
* Propose edges for every requirement (deterministic propose), then delegate the
* merge/rollup/orphan-reject to probe-core. Edge-specific pre-checks: requirements must be an
* array, requirement ids must be unique. Throws on any invalid resolution.
*/
export function analyzeCoverage(
requirements: Requirement[],
resolutions: Resolution<EdgeVerification>[] = [],
): CoverageReport<EdgeVerification> {
if (!Array.isArray(requirements)) {
throw new Error('requirements must be an array');
}
const items: Edge[] = [];
const seenReqIds = new Set<string>();
for (const req of requirements) {
validateRequirement(req);
if (seenReqIds.has(req.id)) {
throw new Error(`duplicate requirement id ${JSON.stringify(req.id)}`);
}
seenReqIds.add(req.id);
for (const edge of proposeEdges(req)) items.push(edge);
}
return coreAnalyzeCoverage(items, resolutions, EDGE_VALIDATORS);
}
/*
* CLI entry (EP-06 invokable surface): `edge-probe.cjs <requirements.json> [resolutions.json]`.
* The generic I/O plumbing (parse, fail-closed exit 2, pretty-JSON out) lives in probe-core's
* `runProbeCli`; the edge adapter supplies its `analyzeCoverage`. Guarded by
* `require.main === module` so it runs only when the compiled `.cjs` is executed directly.
*/
if (require.main === module) {
// runProbeCli's default `exit` now throws ExitError (src/probe-core.cts) rather
// than calling process.exit directly, so this entry point must run under
// runMain to translate that throw into process.exitCode.
runMain(() => {
runProbeCli(
(requirements, resolutions) =>
analyzeCoverage(requirements as Requirement[], resolutions as Resolution<EdgeVerification>[]),
{ usage: 'edge-probe.cjs <requirements.json> [resolutions.json]' },
);
});
}