Files
msd-core/src/edge-probe.cts
Tom Boucher d98b55562c enhance(#3910): the raw terminator is banned by construction (#3980)
* enhance(#3910): move the last src/ terminators onto the seam

Phase 6 bans the raw terminator by construction, which it cannot do while
violations stand. A census found 12 sites the rule would flag; nine of the ten
unsanctioned ones were owned by no phase of the epic at all — a coverage hole
in the decomposition, since P0-P2 are infra, P3 the gate modules, P4 the
scanners, P5 the fragments, P7 the hooks, P8 io.cts, and P6 itself only adds
the rule. `src/**/*.cts` now holds exactly 2 raw exits, both inside
`terminateNow`, the single sanctioned site.

`io.cts`'s `error()` is the interesting one. It was first called substantive on
"dozens of callers, contract risk" — asserted, not measured, and the
measurement refuted it: 289 call sites, zero inside a try whose catch would
swallow a throw. The real obstacle was structural instead: `terminateNow`
cannot emit exit 1, because ADR-3889 §1 makes 0 and 1 unallocatable and
`nameForExitCode(1)` throws. So the only route is `ExitError` under `runMain`,
which sets exitCode and writes stderr only when the error carries a user
message — keeping the existing stderr write and throwing a message-less
ExitError is observably identical.

That census was still too narrow, and running the CLI proved it. It asked
whether the CALL sits in a try/catch; the two regressions that surfaced were
interceptors elsewhere on the stack:

- `command-routing-hub.cts`'s `dispatch()` swallowed the ExitError into a
  HandlerFailure, so the caller emitted a duplicated, wrong stderr line on
  every Hub-routed path. It now rethrows ExitError explicitly — the same shape
  `gsd-tools.cjs` already used at two dispatch sites, so this follows an
  established idiom rather than inventing one.
- the profile-pipeline router's deliberately un-awaited `.catch(e => error(...))`
  turned an ExitError rejection into an uncaught exception; it now mirrors
  runMain's handling.

`edge-probe` and `ui-consideration-probe` gained `runMain` wrappers because
probe-core's new throwing default would otherwise have escaped them.

A follow-up sweep of every dispatcher — 19 command routers, the Hub, the
gsd-tools dispatch seams — found no further swallowing catch. The admitted
bound: ~1260 non-rethrowing catches repo-wide were scanned structurally but not
individually classified. Both real regressions were found by execution, not by
reading, so the suite is the detector that matters here.

`gsd-tools.cjs:253` stays a raw exit deliberately: it is the ensureRuntimeBuild
bootstrap, which runs before cli-exit is required, so the seam does not yet
exist. It needs a second allowlist entry, which means #3910's "single allowlist
entry" criterion is unachievable as written.

Verification runs on the remote runner.

Refs #3910

* enhance(#3910): ban the raw terminator by construction

Adds local/require-registered-exit and registers it on all four globs:
src/**/*.cts, scripts/**/*.cjs, hooks/**/*.js, gsd-core/bin/**/*.cjs.

Registering on the .cts glob is load-bearing, not redundant — the emitted .cjs
mirrors are globally eslint-ignored, so a rule registered only on the emitted
globs is blind to the sources. That is the #3496 lesson, and it is how the
previous guard became invisible: n/no-process-exit was 'error' in one block yet
fired zero times on all three surfaces that mattered.

The dead n/no-process-exit: 'off' block for hooks is deleted in the same PR.
Phase 7 migrated every hook, so the exemption now protects nothing.

Two allowlist entries, not the one #3910 anticipated. terminateNow's body is
detected STRUCTURALLY — a process.exit lexically inside a function of that name
— rather than by a path and line number that rots. The second is
gsd-tools.cjs's ensureRuntimeBuild bootstrap, an inline disable with its reason
at the call site: it runs before ./lib/cli-exit.cjs is required, so the seam
does not exist yet and no migration is possible. #3910's 'single allowlist
entry' criterion is therefore unachievable as written, and is amended with the
measurement rather than quietly missed.

The rule is proven able to FAIL, per glob: four positive controls, one for each
registered glob. A guard that cannot be shown to fire is not a guard. Four
matching negative controls pin process.exitCode as never-flagged — conflating
it with process.exit is what inflated this epic's original census 2x. An
allowlist case and a near-miss (same shape, different function name) fix the
structural detection in place.

Verification runs on the remote runner.

Refs #3910

* fix(#3910): stop the detached catch from throwing, and scope the allowlist

Review findings, one of them a regression the previous fix introduced.

_handlePipelineRejection called error() from inside a DETACHED .catch().
error() now throws, so that throw became an unhandled promise rejection — and
on Node >=15 with --unhandled-rejections=throw, Node dumps a raw stack trace
with absolute paths on top of the clean Error: line. That was impossible before
this branch, because process.exit(1) terminated synchronously before any
rejection machinery could observe it. The handler now writes byte-identical
stderr itself, in both plain and --json-errors form, and sets exitCode in
place. This was the THIRD interceptor found, and like the first two it surfaced
by running the CLI rather than by reading code.

The rule's terminateNow allowlist had no path constraint, so any function
anywhere named terminateNow across all four globs inherited it. It now requires
the structural nesting check AND a cli-exit.cts basename — still no line
numbers to rot.

The four per-glob positive controls only varied a filename inside RuleTester,
which never resolves eslint.config.mjs. Since the rule is filename-agnostic,
all four exercised identical logic and none proved the rule was WIRED — this
epic's own failure mode. A registration test now asserts the rule resolves for
a real path in each glob, and it is proven able to fail: removing one glob's
registration flips the resolved value from [2] to undefined.

Three evasions the rule cannot catch (computed member, aliasing, .call/.apply)
are documented in its header and pinned by tests, labelled as known limits
rather than endorsed, so a future change that starts catching them is a
deliberate diff.

Refs #3910

* docs(#3910): document the raw-terminator ban

Reference and Explanation via a new docs/features fragment (FEATURES.md is
generated from it, not hand-edited). How-To:
docs/how-to/resolve-a-raw-terminator-finding.md, indexed from docs/README.md —
a contributor whose code trips the rule picks among three replacements by
surface (runMain/ExitError for a CLI path, terminateNow for a hook,
process.exitCode where the process should drain), and needs to know why
process.exitCode is correct and never flagged, since conflating the two is what
inflated this epic's original census 2x.

The page also names the three patterns the rule cannot catch and says plainly
that using one to dodge it is a review finding, not a fix — documenting them
without that sentence would read as a sanctioned workaround.

docs/INVENTORY.md deliberately untouched: eslint-rules/ is not a tracked family
in the manifest (verified — a regen produced a zero diff), so a hand-written row
would desync the table from the family it claims to belong to.

Refs #3910

* fix(#3910): a catch that sniffs the message swallows an ExitError

The remote run returned 41 failures, and one of them was a live production
regression rather than a test artifact.

`cmdMilestoneComplete`'s unstarted-phase guard re-threw only when
`e.message.startsWith('Cannot mark milestone complete:')`. `error()` used to
`process.exit(1)`, uncatchable, so the guard always fired. It now throws an
ExitError carrying no message, the string test fails, and the ExitError was
silently swallowed — the guard stopped blocking milestone completion entirely.
Proven against the real CLI: pre-fix, a milestone with an unstarted phase
archived at exit 0; post-fix it is blocked at exit 1 with the intended message.

That is a guard that silently stopped guarding, which is this epic's thesis
appearing inside the phase meant to enforce it. Worth stating plainly: an
earlier census DID examine this site, saw a `throw e`, and classified it as
rethrowing. It was wrong — the rethrow is conditional, and a conditional
rethrow on an inspected message is indistinguishable from an unconditional one
unless you read the predicate.

So the class was swept rather than patched where it was tripped over. An AST
census of every CatchClause across src/, gsd-core/bin/ and scripts/ found 38
conditional rethrows. Two more had the same defect and are fixed the same way:
`config.cts`'s `'No config.json'` sniff and `gsd-tools.cjs`'s
`e.name === 'WindowsError'`. The remaining 25 are provably unreachable — every
one wraps a bare fs, YAML, manifest-require or git-exec primitive that cannot
throw ExitError — and two were scanner false positives, both explained. Each
fix is an unconditional `instanceof ExitError` rethrow placed BEFORE any
inspection, matching the idiom command-routing-hub and gsd-tools already used.

Residual bound, stated rather than implied: zero known-reachable unfixed sites,
contingent only on error() never later being called inside one of those 25
primitive try blocks.

The remaining failures were harness artifacts, and the harnesses were corrected
to the new contract rather than the assertions weakened. Tests that mocked
`process.exit` to observe termination now catch ExitError and assert its code;
tests parsing stderr as a single JSON object still assert exactly that, with
their ad-hoc `node -e` scripts wrapped in runMain so it is true. milestone and
phase-resolution-parity needed no test change — they were correctly written
against the real bug and are what caught it.

Verification runs on the remote runner.

Refs #3910

* chore(#3910): backfill the changeset PR number

Also reframes the fragment to lead with the user-visible change — the
milestone guard blocking again — rather than the narrowest of the three fixes.

Refs #3910

---------

Co-authored-by: sim <sim@local>
2026-08-28 03:15:39 -04:00

250 lines
12 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* Spec-completeness edge-probe — the FIRST adapter of the probe-core resolution model
* (ADR-457 build model; ADR-550 Decision 7 seam).
*
* The generic resolution lifecycle, the status×verification re-cut, `validateResolution`,
* `validateRequirement`, the `analyzeCoverage` merge/rollup/orphan-reject engine, and the
* `runProbeCli` scaffold all live in `src/probe-core.cts`. This module keeps ONLY the
* edge-specific cluster: the five data/behavior shapes, the closed 8-category edge taxonomy,
* shape classification, edge proposal, and the `{ explicit, backstop }` verification validators.
*
* Authored as strict TypeScript (`src/edge-probe.cts`) and compiled by
* `tsc -p tsconfig.build.json` to the gitignored runtime artifact
* `gsd-core/bin/lib/edge-probe.cjs`. Do NOT hand-write the `.cjs`; it is emitted. Tests
* `require()` the built artifact; `pretest` runs `build:lib` first.
*
* Pure and dependency-free: it classifies each requirement's data/behavior shape, filters
* the closed 8-category edge taxonomy to applicable categories, proposes concrete candidate
* edges, and (via probe-core) merges author resolutions into a coverage report.
*/
import {
type Item,
type Resolution,
type CoverageReport,
type Validators,
validateRequirement as coreValidateRequirement,
validateResolution as coreValidateResolution,
analyzeCoverage as coreAnalyzeCoverage,
runProbeCli,
} from './probe-core.cjs';
// eslint-disable-next-line @typescript-eslint/no-require-imports
import cliExitModule = require('./cli-exit.cjs');
const { runMain } = cliExitModule;
/** The five data/behavior shapes a requirement can exhibit. */
export type Shape = 'numeric-range' | 'collection' | 'text' | 'stateful' | 'io';
/** The edge probe's verification tiers (the `verification` axis values for a resolved edge). */
export type EdgeVerification = 'explicit' | 'backstop';
/** A single edge taxonomy category. */
export interface TaxonomyEntry {
id: string;
name: string;
shapes: Shape[];
probe: string;
}
/** A SPEC requirement; `shapes` is an optional authored override of classification. */
export interface Requirement {
id: string;
text: string;
shapes?: Shape[];
}
/** An edge item — a probe-core `Item` specialized to the edge verification vocabulary. */
export type Edge = Item<EdgeVerification>;
/**
* Word-boundary cues mapping requirement prose -> data/behavior shape.
* Heuristic and intentionally lossy; an authored `shapes` array overrides it.
*/
export const SHAPE_CUES: Record<Shape, RegExp> = {
'numeric-range': /\b(round(ing|ed)?|threshold|max(imum)?|min(imum)?|limit|bound(ary)?|between|cap|percent|amount|price|count|number|score|rate|decimal)\b/i,
'collection': /\b(lists?|arrays?|sets?|items?|collections?|each|every|all|sort(ed|ing)?|merge|dedupe|group|ranges?|intervals?|overlap(ping)?)\b/i,
'text': /\b(string|text|names?|labels?|truncate|substring|char(acter)?s?|length|slug|message|unicode)\b/i,
'stateful': /\b(save|persist|store|update|toggle|create|delete|remove|submit|retry|apply|register|insert)\b/i,
'io': /\b(files?|requests?|fetch|upload|download|network|api|endpoints?|connections?|sockets?)\b/i,
};
/** The locked shape vocabulary — exactly the keys of SHAPE_CUES (single source of truth). */
export const VALID_SHAPES: ReadonlySet<string> = new Set(Object.keys(SHAPE_CUES));
/** Detect which shapes a requirement's prose matches (heuristic). */
export function classifyShape(text: string): Shape[] {
const shapes: Shape[] = [];
const subject = String(text == null ? '' : text);
for (const shape of Object.keys(SHAPE_CUES) as Shape[]) {
if (SHAPE_CUES[shape].test(subject)) shapes.push(shape);
}
return shapes;
}
/**
* Closed taxonomy of 8 domain-boundary edge categories (established QA names).
* `shapes` lists which requirement shapes make the category relevant.
*/
export const TAXONOMY: TaxonomyEntry[] = [
{ id: 'boundary', name: 'Boundary values', shapes: ['numeric-range'], probe: 'What happens exactly at each min/max/threshold — and one step either side?' },
{ id: 'adjacency', name: 'Adjacency / touching', shapes: ['collection'], probe: 'When two things are exactly equal or just touch, do they merge, collide, or separate?' },
{ id: 'empty', name: 'Empty / degenerate', shapes: ['collection', 'text'], probe: 'What is the result for empty, single-element, or null input?' },
{ id: 'encoding', name: 'Encoding / representation', shapes: ['text'], probe: 'Whose definition of length/equality applies — bytes, code points, grapheme clusters, or normalized form?' },
{ id: 'ordering', name: 'Ordering / stability', shapes: ['collection'], probe: 'When elements compare equal, is output order specified and stable?' },
{ id: 'precision', name: 'Precision / overflow', shapes: ['numeric-range'], probe: 'Where can precision loss, overflow, or rounding/tie-breaking occur — and what is the exact contract (e.g. half-up vs half-to-even, ceil/floor/truncate)?' },
{ id: 'idempotency', name: 'Idempotency / repetition', shapes: ['stateful'], probe: 'What happens if this runs twice on the same input?' },
{ id: 'concurrency', name: 'Concurrency / effect ordering', shapes: ['stateful', 'io'], probe: 'If interrupted or run in parallel, what is guaranteed?' },
];
/** Return taxonomy category ids whose applicable shapes intersect the input set. */
export function applicableCategories(shapes: Shape[]): string[] {
const set = new Set<Shape>(shapes);
return TAXONOMY.filter((c) => c.shapes.some((s) => set.has(s))).map((c) => c.id);
}
/**
* The edge adapter's injected runtime validators (ADR-550 #5). `categories` is the closed
* taxonomy; both verification tiers require a non-empty `resolution` (an explicit AC's text
* or a backstop note) so plan-phase has a criterion to lift.
*/
/**
* Pseudo-category for a requirement whose prose matched NO shape cue (#1110). It is a soft
* "review manually" signal, NOT a 9th taxonomy category: it stays out of `TAXONOMY` (the closed
* eight) and only joins `EDGE_VALIDATORS.categories` so `analyzeCoverage` accepts the item.
*/
export const UNCLASSIFIED_CATEGORY = 'unclassified';
const UNCLASSIFIED_PROBE = 'unclassified — review manually';
export const EDGE_VALIDATORS: Validators = {
categories: [...TAXONOMY.map((c) => c.id), UNCLASSIFIED_CATEGORY],
verification: ['explicit', 'backstop'],
requiredFieldsByVerification: { explicit: ['resolution'], backstop: ['resolution'] },
};
/**
* Validate a single requirement — the generic id/text checks (probe-core) plus the edge's
* `shapes`-must-be-an-array check. A bare string like `shapes:"numeric-range"` would otherwise
* fall through to prose classification, silently ignoring the authored override.
*
* The edge adapter's `text` is REQUIRED (the prose is the classification signal), so reject a
* missing/empty `text` when no authored `shapes` override is present. Without this, a `{ id }`
* requirement classifies to zero shapes → zero edges → it is silently DROPPED from coverage
* with no signal — the exact fail-open this feature exists to eliminate. An explicit `shapes`
* array (including `[]` for "no applicable categories") is the legitimate way to opt out of
* prose classification, so `text` is only required when `shapes` is absent.
*/
export function validateRequirement(requirement: Requirement): void {
coreValidateRequirement(requirement);
const r = requirement as unknown as { shapes?: unknown; text?: unknown };
if (r.shapes != null && !Array.isArray(r.shapes)) {
throw new Error(`requirement ${requirement.id} shapes must be an array when present`);
}
if (r.shapes == null && !(typeof r.text === 'string' && r.text.trim())) {
throw new Error(
`requirement ${requirement.id} text must be a non-empty string when no shapes override is provided`,
);
}
}
/** Validate an edge resolution against the edge verification vocabulary. */
export function validateResolution(resolution: Resolution<EdgeVerification>): true {
return coreValidateResolution(resolution, EDGE_VALIDATORS);
}
/**
* Propose candidate edges for a requirement. Uses authored `shapes` when present, else
* classifies from prose. Every proposed edge starts unresolved (verification null).
*/
export function proposeEdges(requirement: Requirement): Edge[] {
validateRequirement(requirement);
let shapes: Shape[];
if (Array.isArray(requirement.shapes)) {
// Fail closed: an authored array must contain only locked shape values. A non-empty
// but invalid array (e.g. ['numeric'], a typo for 'numeric-range') would otherwise
// intersect no category and silently suppress every probe — the gate reads green while
// nothing was checked. An empty array stays a valid "no applicable categories" override.
for (const s of requirement.shapes) {
if (typeof s !== 'string' || !VALID_SHAPES.has(s)) {
throw new Error(
`invalid shape ${JSON.stringify(s)} for requirement ${requirement.id} — must be one of: ${[...VALID_SHAPES].join(', ')}`,
);
}
}
shapes = requirement.shapes;
} else {
shapes = classifyShape(requirement.text);
if (shapes.length === 0) {
// Prose present but no shape cue matched. Do NOT silently drop it (#1110): an
// edge-relevant requirement whose phrasing missed every cue would otherwise vanish from
// coverage with no signal — the exact blind spot this probe exists to catch. Surface ONE
// soft, dismissible "unclassified — review manually" candidate. The explicit `shapes: []`
// opt-out (handled above) stays silent — that is the author's deliberate "no edge surface".
return [{
requirement_id: requirement.id,
category: UNCLASSIFIED_CATEGORY,
status: 'unresolved',
verification: null,
resolution: null,
reason: null,
probe: UNCLASSIFIED_PROBE,
}];
}
}
return applicableCategories(shapes).map((catId): Edge => {
const cat = TAXONOMY.find((c) => c.id === catId);
return {
requirement_id: requirement.id,
category: catId,
status: 'unresolved',
verification: null,
resolution: null,
reason: null,
probe: cat ? cat.probe : '',
};
});
}
/**
* Propose edges for every requirement (deterministic propose), then delegate the
* merge/rollup/orphan-reject to probe-core. Edge-specific pre-checks: requirements must be an
* array, requirement ids must be unique. Throws on any invalid resolution.
*/
export function analyzeCoverage(
requirements: Requirement[],
resolutions: Resolution<EdgeVerification>[] = [],
): CoverageReport<EdgeVerification> {
if (!Array.isArray(requirements)) {
throw new Error('requirements must be an array');
}
const items: Edge[] = [];
const seenReqIds = new Set<string>();
for (const req of requirements) {
validateRequirement(req);
if (seenReqIds.has(req.id)) {
throw new Error(`duplicate requirement id ${JSON.stringify(req.id)}`);
}
seenReqIds.add(req.id);
for (const edge of proposeEdges(req)) items.push(edge);
}
return coreAnalyzeCoverage(items, resolutions, EDGE_VALIDATORS);
}
/*
* CLI entry (EP-06 invokable surface): `edge-probe.cjs <requirements.json> [resolutions.json]`.
* The generic I/O plumbing (parse, fail-closed exit 2, pretty-JSON out) lives in probe-core's
* `runProbeCli`; the edge adapter supplies its `analyzeCoverage`. Guarded by
* `require.main === module` so it runs only when the compiled `.cjs` is executed directly.
*/
if (require.main === module) {
// runProbeCli's default `exit` now throws ExitError (src/probe-core.cts) rather
// than calling process.exit directly, so this entry point must run under
// runMain to translate that throw into process.exitCode.
runMain(() => {
runProbeCli(
(requirements, resolutions) =>
analyzeCoverage(requirements as Requirement[], resolutions as Resolution<EdgeVerification>[]),
{ usage: 'edge-probe.cjs <requirements.json> [resolutions.json]' },
);
});
}