enhance(#3717): make the edge probe's shape cues language-aware via an optional text_en field (#4156)
* test(#3717): add failing-first coverage for text_en language-aware classification Adds unit tests for the not-yet-implemented text_en field on Requirement (fallback selection, empty/whitespace/non-string rejection, shapes-override precedence), a SHAPE_CUES/VALID_SHAPES parity guard (RULESET.GENERATIVE-FIX), and workflow-prose contract tests asserting spec-phase.md Step 5.5 documents populating text_en for response_language projects. All new tests are RED until src/edge-probe.cts and the workflow docs are updated. * feat(#3717): make edge-probe shape classification read an optional text_en field Requirement gains an optional text_en; classifyShape's own signature stays untouched (a locked, directly-tested export), and the text_en ?? text selection is pushed to proposeEdges' single call site instead. text_en is validated fail-closed: an empty or whitespace-only value throws rather than silently winning the ?? fallback and degrading classification to zero shapes. This makes the #2773 doc-only translation convention an explicit, validatable field instead of an invisible instruction, per the approved Form-1 scope on #3717. * docs(#3717): document the text_en field across spec-phase, reference and how-to docs Updates Step 5.5's response_language instructions, the edge-probe reference Inputs contract, the FEATURES.md fragment, and the non-English how-to guide to describe the new text_en field: text keeps the requirement's own wording in all cases, text_en (when populated) is the engine-only English rendering the classifier prefers. * docs(#3717): record the text_en locked-surface change in CONTEXT.md and ADR-550 Updates the Edge Probe Module glossary entry to describe the text_en field and its fail-closed validation, and appends an ADR-550 amendment recording why this is additive and does not re-open the #652 LLM-classifier rejection (text_en is a plain field read by the existing deterministic regex classifier, not a new model-dependent surface). * docs(#3717): add changeset fragment and regenerate FEATURES.md pr:0 placeholder — backfilled with the real PR number after the PR opens. * docs(#3717): attribute the text_en machine check to engine-level validation, not prose tests Code-review (Spec axis) finding: the workflow-prose contract tests and the ADR-550 amendment overclaimed themselves as "the machine check the #2773 doc-only stopgap lacked." That check is actually engine-level (validateRequirement/classifyShape, covered in tests/edge-probe.test.cjs) — the prose tests are the same style of assertion #2773 already used. Reworded both to attribute the claim correctly. * fix(#3717): rewrap spec-phase.md so the id-unchanged sentence stays on one line The #3717 rewrite of Step 5.5's response_language paragraph moved a line break so "requirement `id`s" ended one physical line and "are never translated" started the next. The pre-existing #2773 regression test (tests/edge-probe-spec-phase-contract.test.cjs) asserts id + "never translated" on the SAME line (no \n in between, matching git's own line-oriented prose), so the reflow silently broke it. Rewrapped so the sentence lands on one line again, verified against every #2773/#3717 regex assertion in that test file. Emitted-Drift-Ack-Growth: spec-phase.md — #3717 adds text_en documentation to Step 5.5 (response_language paragraph + REQS_JSON heredoc comment); this growth is this PR's own diff, not incidental drift. * chore(#3717): backfill changeset PR number pr:0 -> pr:4156 now that the PR exists. --------- Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
@@ -504,3 +504,59 @@ test('#2773: no Step 5.5 exit path leaks the $REQS_JSON temp file', () => {
|
||||
`every exit between the mktemp and the unconditional cleanup must rm -f "$REQS_JSON" first; leaking exits: ${JSON.stringify(leaks)}`,
|
||||
);
|
||||
});
|
||||
|
||||
// ─── #3717: durable text_en field (follow-up to the #2773 doc-only stopgap) ───
|
||||
//
|
||||
// #2773 shipped a documented convention: for a response_language project, Step 5.5's `text`
|
||||
// field silently carried the English TRANSLATION rather than the SPEC's own requirement
|
||||
// prose. #3717 (approved scope: Form 1 only, `text_en` field) makes this an explicit,
|
||||
// validatable field: `text` goes back to meaning "the requirement's own text" (in
|
||||
// response_language, when set), and `text_en` carries the translation the classifier reads.
|
||||
//
|
||||
// These three tests only check the WORKFLOW PROSE instructs populating text_en correctly —
|
||||
// the same prose-assertion pattern the #2773 tests above already use for Step 5.5, not a new
|
||||
// testing technique. The actual machine check the #2773 doc-only stopgap lacked is
|
||||
// ENGINE-level, not workflow-level: text_en is now a real, validated Requirement field —
|
||||
// validateRequirement fail-closes on an empty value, and classifyShape/proposeEdges
|
||||
// demonstrably prefer it over text — a property the #2773 prose-only convention had no way
|
||||
// to enforce. See tests/edge-probe.test.cjs for that engine-level coverage.
|
||||
|
||||
test('#3717: Step 5.5 documents populating text_en for response_language projects', () => {
|
||||
const block = extractStep55Block(readSpecPhase());
|
||||
assert.ok(block.length > 0, 'Step 5.5 block must be extractable from spec-phase.md');
|
||||
|
||||
assert.match(
|
||||
block,
|
||||
/text_en/,
|
||||
'Step 5.5 must name the text_en field — the durable #3717 fix for non-English requirement classification',
|
||||
);
|
||||
assert.match(
|
||||
block,
|
||||
/response_language/,
|
||||
'Step 5.5 must still name response_language as the setting that triggers populating text_en',
|
||||
);
|
||||
});
|
||||
|
||||
test('#3717: Step 5.5 documents that text stays the original SPEC-language requirement text', () => {
|
||||
const block = extractStep55Block(readSpecPhase());
|
||||
assert.ok(block.length > 0, 'Step 5.5 block must be extractable from spec-phase.md');
|
||||
|
||||
// The #2773-era stopgap had `text` secretly carry the English translation. The #3717
|
||||
// durable fix keeps `text` meaning the requirement's own text (original language) and
|
||||
// moves the translation into the new text_en field — this must be documented, not just
|
||||
// implemented, or the next reader re-introduces the #2773 overload by habit.
|
||||
assert.match(
|
||||
block,
|
||||
/`?text`?[^\n]*(original language|SPEC's own|requirement's own|unchanged)|(original language)[^\n]*`?text`?/i,
|
||||
'Step 5.5 must state that text carries the SPEC requirement in its own (original) language — not the translation',
|
||||
);
|
||||
});
|
||||
|
||||
test('#3717: the edge-probe reference documents the text_en field', () => {
|
||||
const ref = fs.readFileSync(EDGE_PROBE_REF_PATH, 'utf8');
|
||||
assert.match(
|
||||
ref,
|
||||
/text_en/,
|
||||
'the edge-probe reference `## Inputs` contract must document the optional text_en field and its text_en ?? text fallback',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -471,6 +471,115 @@ describe('edge-probe: analyzeCoverage — duplicate rejection (RR-09)', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('edge-probe: text_en language-aware classification (#3717)', () => {
|
||||
// The prose the #2773 stopgap test file already uses as its canonical Portuguese/English
|
||||
// pair, kept identical here so the two test files agree on the same fixture (no drift).
|
||||
const pt = 'O sistema mescla intervalos sobrepostos em uma lista ordenada';
|
||||
const en = 'The system merges overlapping intervals in a sorted list';
|
||||
|
||||
test('proposeEdges: text_en absent falls back to text (back-compat)', () => {
|
||||
const withoutTextEn = ep.proposeEdges({ id: 'R1', text: 'Round a number to N decimal places' });
|
||||
assert.deepEqual(withoutTextEn.map((e) => e.category).sort(), ['boundary', 'precision']);
|
||||
});
|
||||
|
||||
test('proposeEdges: text_en present is used for classification instead of text', () => {
|
||||
// text alone (non-English) classifies to zero shapes -> the unclassified sentinel.
|
||||
const nonEnglishOnly = ep.proposeEdges({ id: 'R1', text: pt });
|
||||
assert.deepEqual(nonEnglishOnly.map((e) => e.category), ['unclassified']);
|
||||
|
||||
// text_en present -> classification runs against the English translation.
|
||||
const withTextEn = ep.proposeEdges({ id: 'R1', text: pt, text_en: en });
|
||||
assert.deepEqual(withTextEn.map((e) => e.category).sort(), ['adjacency', 'empty', 'ordering']);
|
||||
});
|
||||
|
||||
test('#3717: a non-English requirement with text_en classifies identically to its English equivalent', () => {
|
||||
const englishOnly = ep.proposeEdges({ id: 'R1', text: en });
|
||||
const nonEnglishWithTranslation = ep.proposeEdges({ id: 'R1', text: pt, text_en: en });
|
||||
assert.deepEqual(
|
||||
nonEnglishWithTranslation.map((e) => e.category).sort(),
|
||||
englishOnly.map((e) => e.category).sort(),
|
||||
'a translated non-English requirement must raise the same categories as the English original',
|
||||
);
|
||||
});
|
||||
|
||||
test('validateRequirement: text_en: null is treated as absent (no throw)', () => {
|
||||
assert.doesNotThrow(() => ep.validateRequirement({ id: 'R1', text: 'Round a number', text_en: null }));
|
||||
});
|
||||
|
||||
test('proposeEdges: text_en: null falls back to text', () => {
|
||||
const edges = ep.proposeEdges({ id: 'R1', text: 'Round a number to N decimal places', text_en: null });
|
||||
assert.deepEqual(edges.map((e) => e.category).sort(), ['boundary', 'precision']);
|
||||
});
|
||||
|
||||
test('validateRequirement: rejects empty-string text_en (?? does not catch \'\')', () => {
|
||||
// Nullish coalescing only falls back on null/undefined — an empty string would
|
||||
// otherwise win `text_en ?? text` and silently classify against '', degrading to
|
||||
// zero shapes with no signal (the exact fail-open #1110/#2773 exist to eliminate).
|
||||
assert.throws(
|
||||
() => ep.validateRequirement({ id: 'R1', text: 'Round a number', text_en: '' }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('validateRequirement: rejects whitespace-only text_en', () => {
|
||||
assert.throws(
|
||||
() => ep.validateRequirement({ id: 'R1', text: 'Round a number', text_en: ' ' }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('validateRequirement: rejects non-string text_en (number/array/object)', () => {
|
||||
assert.throws(
|
||||
() => ep.validateRequirement({ id: 'R1', text: 'Round a number', text_en: 42 }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
assert.throws(
|
||||
() => ep.validateRequirement({ id: 'R1', text: 'Round a number', text_en: ['x'] }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
assert.throws(
|
||||
() => ep.validateRequirement({ id: 'R1', text: 'Round a number', text_en: {} }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('proposeEdges: authored shapes override still wins when text_en is also present', () => {
|
||||
const edges = ep.proposeEdges({ id: 'R9', text: pt, text_en: en, shapes: ['numeric-range'] });
|
||||
assert.deepEqual(edges.map((e) => e.category).sort(), ['boundary', 'precision']);
|
||||
});
|
||||
|
||||
test('validateRequirement: rejects empty text_en even when shapes override makes it unused', () => {
|
||||
// Validation is unconditional — it does not skip the text_en check just because the
|
||||
// classify branch would never run. Bad data fails closed regardless of whether it
|
||||
// happens to be dead for this particular call.
|
||||
assert.throws(
|
||||
() => ep.validateRequirement({ id: 'R1', text: 'x', text_en: '', shapes: ['collection'] }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('edge-probe: SHAPE_CUES/VALID_SHAPES/Shape vocabulary stay single-sourced (DEFECT.GENERATIVE-FIX)', () => {
|
||||
// RULESET.GENERATIVE-FIX (CONTEXT.md): parallel implementations diverge silently when no
|
||||
// parity test enforces equality at the test layer. VALID_SHAPES is derived from
|
||||
// Object.keys(SHAPE_CUES) in source, but that construction alone is not a regression
|
||||
// guard — this test fails if a future edit ever hardcodes one of them independently or
|
||||
// adds/removes a shape from only one side.
|
||||
const LOCKED_SHAPES = ['numeric-range', 'collection', 'text', 'stateful', 'io'];
|
||||
|
||||
test('SHAPE_CUES keys match the locked 5-shape vocabulary exactly', () => {
|
||||
assert.deepEqual(Object.keys(ep.SHAPE_CUES).sort(), [...LOCKED_SHAPES].sort());
|
||||
});
|
||||
|
||||
test('VALID_SHAPES matches SHAPE_CUES keys exactly (single source of truth)', () => {
|
||||
assert.deepEqual([...ep.VALID_SHAPES].sort(), Object.keys(ep.SHAPE_CUES).sort());
|
||||
});
|
||||
|
||||
test('VALID_SHAPES matches the locked 5-shape vocabulary exactly', () => {
|
||||
assert.deepEqual([...ep.VALID_SHAPES].sort(), [...LOCKED_SHAPES].sort());
|
||||
});
|
||||
});
|
||||
|
||||
describe('edge-probe: golden fixtures', () => {
|
||||
const root = path.join(__dirname, '..', 'gsd-core', 'references', 'edge-probe-fixtures');
|
||||
const fixtures = fs.readdirSync(root).filter((d) =>
|
||||
|
||||
Reference in New Issue
Block a user