* test(#4657): add failing-first coverage for the ui probe's text_en channel Mirrors the #3717/#4156 test shape onto the UI adapter: a failing-first proposeConsiderations regression (Danish text + English text_en must classify as its English equivalent, not land in the #1110 unclassified sentinel), proposeElements/analyzeCoverage/CLI end-to-end pairs, fail-closed text_en validation cases (empty/whitespace/non-string, unconditional under an elements override), a ui-phase.md Step 9.5 workflow-prose contract test, a reference-doc Inputs parity test, and a fast-check property proving any cue-matching prose classifies identically under a cue-free Danish rendering plus text_en. All new assertions are RED until src/ui-consideration-probe.cts and the workflow/reference docs are updated. * fix(#4657): give the ui consideration probe a text_en language channel Element gains an optional text_en; classifyElement's own signature stays untouched (a locked, directly-tested export) and the text_en ?? text selection is pushed to the two classification call sites (proposeConsiderations, proposeElements) instead. text_en is validated fail-closed: an empty or whitespace-only value throws rather than silently winning the ?? fallback and degrading classification to zero kinds. Mirrors #3717/#4156 onto the UI adapter: ui-phase.md Step 9.5 gains the Non-English projects section (mirroring spec-phase Step 5.5) and the ELEMENTS_JSON shape comment documents the field with both zero-applicable guard arms named; the reference doc's Inputs section, the PROBE.ui CONTEXT predicate (with both derived indexes regenerated), and the nav-override test expectation stay in sync. The ui-phase contract test carries the site-scoped allow-test-rule marker and its cluster is registered in the test-file-count allowlist ratchet. Emitted-Drift-Ack-Growth: ui-phase.md — Non-English text_en section, ELEMENTS_JSON shape comment, and two-arm guard wording (#4657) * docs(#4657): backfill changeset PR number --------- Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
5
.changeset/sturdy-lemurs-snooze.md
Normal file
5
.changeset/sturdy-lemurs-snooze.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Fixed
|
||||
pr: 4804
|
||||
---
|
||||
**The UI consideration probe reads an optional `text_en` field** — in a `response_language` project every UI element used to classify to zero categories and land in `unclassified`, dropping the whole state-coverage axis; a faithful English translation per element now classifies exactly like its English equivalent. Mirrors the edge probe's #3717 remedy; an empty or whitespace `text_en` fails closed. (#4657)
|
||||
@@ -614,6 +614,7 @@ The prompt-level data/instruction isolation seam for untrusted web/document ingr
|
||||
`PROBE.prohib.verification=test|judgment`
|
||||
`PROBE.ui.verification=explicit|backstop`
|
||||
`PROBE.ui.axis=MIXED — closed compiled shape-rooted 8 (empty/loading/error/populated/partial/overflow/zero-one-many/long-text) via ui-consideration-probe adapter; open UX (real-time/a11y/i18n-RTL) prose-owned in references/domain-probes.md, NOT compiled (#1867)`
|
||||
`PROBE.ui.text_en=optional English translation of Element.text read by classification (text_en ?? text), engine input never user-facing output; mirrors the edge adapter's #3717 field; empty/whitespace/non-string fails closed (#4657)`
|
||||
`PROBE.ui.seam=ui-phase Step 9.5 post-verification: element-cue classify -> propose-then-confirm (partial-cue mitigation, Goodhart) -> autoResolve --auto floor (never dismiss; unclassified stays unresolved #1110) -> ## UI Considerations write-back -> plan-phase `## UI Considerations` lift rule (#1867)`
|
||||
`PROBE.ci.surface=the contract (parse/validate, projection round-trip, fail-closed guards), NEVER the LLM judgment (ADR-550 D5)`
|
||||
`PROHIB.recall=LLM-prose; no compiled prohibition-probe recall engine (only the schema/projection layer is code, ADR-550 D7b)`
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"count": 285,
|
||||
"count": 286,
|
||||
"classes": {
|
||||
"ARCH": 1,
|
||||
"CI": 2,
|
||||
@@ -14,7 +14,7 @@
|
||||
"PLANNING": 3,
|
||||
"PR": 2,
|
||||
"PRED": 68,
|
||||
"PROBE": 11,
|
||||
"PROBE": 12,
|
||||
"PROC": 14,
|
||||
"PROHIB": 10,
|
||||
"RELEASE-NOTES": 31,
|
||||
@@ -646,6 +646,11 @@
|
||||
"klass": "PROBE",
|
||||
"value": "ui-phase Step 9.5 post-verification: element-cue classify -> propose-then-confirm (partial-cue mitigation, Goodhart) -> autoResolve --auto floor (never dismiss; unclassified stays unresolved #1110) -> ## UI Considerations write-back -> plan-phase `## UI Considerations` lift rule (#1867)"
|
||||
},
|
||||
{
|
||||
"id": "PROBE.ui.text_en",
|
||||
"klass": "PROBE",
|
||||
"value": "optional English translation of Element.text read by classification (text_en ?? text), engine input never user-facing output; mirrors the edge adapter's #3717 field; empty/whitespace/non-string fails closed (#4657)"
|
||||
},
|
||||
{
|
||||
"id": "PROBE.ui.verification",
|
||||
"klass": "PROBE",
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -15,11 +15,16 @@ here. Forcing them into a closed compiled taxonomy is the wrong model.
|
||||
|
||||
## Inputs
|
||||
|
||||
A list of UI elements, each a `{ id, text, elements? }` record where `text` is the
|
||||
researcher-authored description and `elements` is an optional author-supplied override of the
|
||||
element classification. The six element kinds are: `form`, `list-collection`, `nav`, `media`,
|
||||
`interactive-control`, `static-content`. When `elements` is absent, a heuristic classifier
|
||||
proposes kinds from the prose (propose-then-confirm) — the author may correct the kind.
|
||||
A list of UI elements, each a `{ id, text, text_en?, elements? }` record where `text` is the
|
||||
researcher-authored description, `elements` is an optional author-supplied override of the
|
||||
element classification, and `text_en` is an optional English translation of `text`
|
||||
(#4657). The element cues are English word-boundary patterns, so a non-English `text` (a
|
||||
project running with `response_language` set) classifies to zero kinds unless `text_en`
|
||||
carries a faithful English rendering — engine input, never user-facing output; `text`
|
||||
itself keeps the UI-SPEC's own language and is never translated. The six element kinds are:
|
||||
`form`, `list-collection`, `nav`, `media`, `interactive-control`, `static-content`. When
|
||||
`elements` is absent, a heuristic classifier proposes kinds from the prose
|
||||
(`text_en ?? text`, propose-then-confirm) — the author may correct the kind.
|
||||
|
||||
## Taxonomy (8 categories)
|
||||
|
||||
|
||||
@@ -305,6 +305,27 @@ section (re-run), the write-back is idempotent (it REPLACES that section, never
|
||||
runtime is non-Claude and the probe engine cannot be resolved, the shim FAILS LOUD (below) — it
|
||||
never silently no-ops (a silent skip would drop the whole state-coverage axis).
|
||||
|
||||
**Non-English projects — `text_en` carries the classifier-facing translation; the UI-SPEC is
|
||||
not.** The element cues the classifier matches are **English** word-boundary patterns, so
|
||||
element prose written in another language matches nothing, classifies to zero kinds, and lands
|
||||
every element in `unclassified` (#1110) — the taxonomy contributes nothing and `--auto` leaves
|
||||
it all `unresolved`. When this project has `response_language` set, add an optional `text_en`
|
||||
key to each `$ELEMENTS_JSON` entry: a faithful **English** translation of that element's
|
||||
`text`. `text_en` is **engine input, never user-facing output**, so the `response_language`
|
||||
rule at the top of this workflow does not govern it — but `text` itself is NOT translated:
|
||||
write it as the element's own wording, exactly as it appears in the UI-SPEC. The UI-SPEC keeps
|
||||
the original language — only `text_en` is translated, and element `id`s are never translated
|
||||
or renumbered (coverage rows join back on `id`). Populate `text_en` for **every** element, not
|
||||
only the ones that look UI-relevant: the zero-applicable guard below fires when `$APPLICABLE`
|
||||
is `0` and again on the all-unclassified case (`$UNCLASSIFIED = $APPLICABLE`, #4656), so a
|
||||
partly-classified surface slips through both arms with no signal at all. When
|
||||
`response_language` is unset (an English-language project), omit `text_en` —
|
||||
`text` is already English and the engine falls back to it automatically (`text_en ?? text`).
|
||||
If an element still classifies to zero kinds with `text_en` populated, it carries no cue in
|
||||
any language (the recorded recall gap — ADR-857 §98 / ADR-550 D7b, not a translation failure);
|
||||
author an explicit `elements` array on that element instead of relying on the prose classifier.
|
||||
(#4657)
|
||||
|
||||
**Runtime coverage compute — resolve and invoke ui-consideration-probe.cjs:**
|
||||
|
||||
```bash
|
||||
@@ -345,8 +366,9 @@ fi
|
||||
|
||||
# Element extraction (MANUAL BY DESIGN — not an oversight): the agent reads the researcher-authored
|
||||
# UI-SPEC prose (the described surfaces — the Design System / Copywriting rows and any element the
|
||||
# researcher named) and writes ONE object per UI element/surface: {"id","text"} where text is the
|
||||
# prose describing it. This mirrors spec-phase Step 5.5's edge-probe REQS_JSON step VERBATIM — a
|
||||
# researcher named) and writes ONE object per UI element/surface: {"id","text","text_en"?} where
|
||||
# text is the prose describing it. This mirrors spec-phase Step 5.5's edge-probe REQS_JSON step
|
||||
# VERBATIM — a
|
||||
# hand-populated heredoc guarded by the fail-loud <replace:> check below — the established, shipped
|
||||
# pattern for feeding a probe from a prose spec. It is NOT mechanized on purpose: a UI-SPEC has no
|
||||
# single machine-parseable "elements" column — surfaces are distributed across design-token tables
|
||||
@@ -355,7 +377,10 @@ fi
|
||||
# element). The agent-authored heredoc + fail-loud guard is the conservative choice, identical to the
|
||||
# requirement-side edge-probe path (RR-04). If a future UI-SPEC gains a canonical element table,
|
||||
# revisit to parse it. Populate the heredoc from the UI-SPEC; the guard below fails loud on a
|
||||
# forgotten substitution (never a no-op).
|
||||
# forgotten substitution (never a no-op). When `response_language` is set, ALSO add `text_en` — a
|
||||
# faithful ENGLISH translation of `text` (see the Non-English projects note above); the element
|
||||
# cues are English-only, so original-language `text` alone classifies to zero kinds. `text` itself
|
||||
# stays the UI-SPEC's own wording and is never translated.
|
||||
ELEMENTS_JSON=$(mktemp "${TMPDIR:-/tmp}/ui-probe-elements-XXXXXX") && mv "$ELEMENTS_JSON" "${ELEMENTS_JSON}.json" && ELEMENTS_JSON="${ELEMENTS_JSON}.json" || exit 1
|
||||
cat > "$ELEMENTS_JSON" <<'JSON'
|
||||
[
|
||||
|
||||
@@ -1,6 +1,15 @@
|
||||
{
|
||||
"_doc": "Baseline of modules currently exceeding the 2-test-file limit. Each entry locks in TODAY's exact test filenames as the allowlisted set (identity ratchet). Adding a NEW test file to a capped module fails (novel). Removing one requires pruning this list (stale, ratchet-down). When a cluster drops to \u2264 2, remove its entry entirely. New entries require justification in PR description.",
|
||||
"_doc": "Baseline of modules currently exceeding the 2-test-file limit. Each entry locks in TODAY's exact test filenames as the allowlisted set (identity ratchet). Adding a NEW test file to a capped module fails (novel). Removing one requires pruning this list (stale, ratchet-down). When a cluster drops to ≤ 2, remove its entry entirely. New entries require justification in PR description.",
|
||||
"modules": {
|
||||
"ui-consideration-probe": {
|
||||
"files": [
|
||||
"ui-consideration-probe-docs-fixtures.test.cjs",
|
||||
"ui-consideration-probe-ui-phase-contract.test.cjs",
|
||||
"ui-consideration-probe.test.cjs"
|
||||
],
|
||||
"issue": "4657",
|
||||
"justification": "The ui-consideration-probe cluster gains its workflow-prose contract file (tests/ui-consideration-probe-ui-phase-contract.test.cjs, #4657), mirroring the edge probe's split (edge-probe.test.cjs / edge-probe-docs-fixtures.test.cjs / edge-probe-spec-phase-contract.test.cjs). The contract file reads shipped gsd-core/workflows/ui-phase.md under the source-text-is-the-product exception, which the unit file deliberately does not carry (it asserts typed returns only)."
|
||||
},
|
||||
"adr-parser": {
|
||||
"files": [
|
||||
"adr-parser.property.test.cjs",
|
||||
@@ -40,7 +49,7 @@
|
||||
"frontmatter.unit.test.cjs"
|
||||
],
|
||||
"issue": "3881",
|
||||
"justification": "Grandfathered by #3227 for the original 4-file cluster (pre-existing over-cap, not new sprawl). #3881 (ADR-3473 \u00a78.1, the js-yaml parser migration) adds two more: frontmatter-golden-parity.test.cjs (D-series golden-corpus diff against the legacy parser, ~900 real documents \u2014 cannot be folded into an existing file without losing its independent-of-current-parser provenance) and frontmatter-roundtrip.property.test.cjs (fast-check property coverage the migration's bijective-contract rule requires). Both are migration-specific, not incidental sprawl."
|
||||
"justification": "Grandfathered by #3227 for the original 4-file cluster (pre-existing over-cap, not new sprawl). #3881 (ADR-3473 §8.1, the js-yaml parser migration) adds two more: frontmatter-golden-parity.test.cjs (D-series golden-corpus diff against the legacy parser, ~900 real documents — cannot be folded into an existing file without losing its independent-of-current-parser provenance) and frontmatter-roundtrip.property.test.cjs (fast-check property coverage the migration's bijective-contract rule requires). Both are migration-specific, not incidental sprawl."
|
||||
},
|
||||
"graphify": {
|
||||
"files": [
|
||||
@@ -140,7 +149,7 @@
|
||||
"verify-work-deferred-promotion.test.cjs",
|
||||
"verify.test.cjs"
|
||||
],
|
||||
"issue": "#3813 \u2014 the live-path #1883 suite drives buildPlanningSnapshot in-process with an fs fault; the verify module suites are CLI-driving and cannot host an fs monkeypatch",
|
||||
"issue": "#3813 — the live-path #1883 suite drives buildPlanningSnapshot in-process with an fs fault; the verify module suites are CLI-driving and cannot host an fs monkeypatch",
|
||||
"justification": "verify-command-grounding added by #2401/#3678; allowlist follow-up landed with #3606 (base was red on this lane). #4546 adds verify-work-deferred-promotion (deferred-follow-up promotion writer contract + parity)."
|
||||
},
|
||||
"install": {
|
||||
@@ -155,7 +164,7 @@
|
||||
"install-write-confinement.test.cjs",
|
||||
"install.test.cjs"
|
||||
],
|
||||
"issue": "#3798 \u2014 the profile-closure spawn test reads the real repo surface (commands+workflows+agents); consolidating it into install.test.cjs would put a whole-repo scan behind the suite's heaviest file"
|
||||
"issue": "#3798 — the profile-closure spawn test reads the real repo surface (commands+workflows+agents); consolidating it into install.test.cjs would put a whole-repo scan behind the suite's heaviest file"
|
||||
},
|
||||
"edge-probe": {
|
||||
"files": [
|
||||
@@ -253,8 +262,8 @@
|
||||
"docs-state-md-locale-parity.test.cjs",
|
||||
"docs-update.test.cjs"
|
||||
],
|
||||
"issue": "3873 | #3839 \u2014 the hook-event parity suite cross-checks the four docs hook tables against src/runtime-hooks-surface.cts registrations; a generated-docs concern distinct from content freshness and registry parity",
|
||||
"justification": "ADR-3473 \u00a78.8 makes docs/reference/state-md.md and its four locale siblings generated-and-committed and adds the repo's first locale-parity check (verified: no locale-parity lint precedent exists here). That check is a distinct concern from docs-update (content freshness) and docs-parity-live-registry (registry parity); folding it into either would place an unrelated subject inside them purely to satisfy a count. The section it guards is absent from all four translations today and documents the status enum behind #3853."
|
||||
"issue": "3873 | #3839 — the hook-event parity suite cross-checks the four docs hook tables against src/runtime-hooks-surface.cts registrations; a generated-docs concern distinct from content freshness and registry parity",
|
||||
"justification": "ADR-3473 §8.8 makes docs/reference/state-md.md and its four locale siblings generated-and-committed and adds the repo's first locale-parity check (verified: no locale-parity lint precedent exists here). That check is a distinct concern from docs-update (content freshness) and docs-parity-live-registry (registry parity); folding it into either would place an unrelated subject inside them purely to satisfy a count. The section it guards is absent from all four translations today and documents the status enum behind #3853."
|
||||
},
|
||||
"audit": {
|
||||
"files": [
|
||||
@@ -266,7 +275,7 @@
|
||||
"audit-uat-summary-segmentation.test.cjs",
|
||||
"audit-workstream-layouts.test.cjs"
|
||||
],
|
||||
"issue": "#3804/#3805 \u2014 audit-layout and acknowledged-marker suites drive the real audit-uat CLI against on-disk fixtures | #3817 \u2014 regression suite proving countReal counts the truncation remainder marker instead of treating it as one phantom item | #3783 \u2014 dedicated suite for the summary.current_milestone/archived segmentation, matching this module's one-file-per-feature-slice precedent (e.g. audit-uat-acknowledged.test.cjs) rather than growing the unrelated 6900+ line tests/uat.test.cjs (a different module, keyed off uat.cjs)"
|
||||
"issue": "#3804/#3805 — audit-layout and acknowledged-marker suites drive the real audit-uat CLI against on-disk fixtures | #3817 — regression suite proving countReal counts the truncation remainder marker instead of treating it as one phantom item | #3783 — dedicated suite for the summary.current_milestone/archived segmentation, matching this module's one-file-per-feature-slice precedent (e.g. audit-uat-acknowledged.test.cjs) rather than growing the unrelated 6900+ line tests/uat.test.cjs (a different module, keyed off uat.cjs)"
|
||||
},
|
||||
"shell-command-projection": {
|
||||
"files": [
|
||||
@@ -274,7 +283,16 @@
|
||||
"shell-command-projection-md-normalize.test.cjs",
|
||||
"shell-command-projection-path-sep.test.cjs"
|
||||
],
|
||||
"issue": "#3854 \u2014 the tight-list suite pins the markdown write-normalizer's blank-line policy through the exported normalizeContent/platformWriteSync seam; a distinct concern from dispatch routing and path separation"
|
||||
"issue": "#3854 — the tight-list suite pins the markdown write-normalizer's blank-line policy through the exported normalizeContent/platformWriteSync seam; a distinct concern from dispatch routing and path separation"
|
||||
},
|
||||
"ui-consideration-probe": {
|
||||
"files": [
|
||||
"ui-consideration-probe-docs-fixtures.test.cjs",
|
||||
"ui-consideration-probe-ui-phase-contract.test.cjs",
|
||||
"ui-consideration-probe.test.cjs"
|
||||
],
|
||||
"issue": "4657",
|
||||
"justification": "The ui-consideration-probe cluster gains its workflow-prose contract file (tests/ui-consideration-probe-ui-phase-contract.test.cjs, #4657), mirroring the edge probe's split (edge-probe.test.cjs / edge-probe-docs-fixtures.test.cjs / edge-probe-spec-phase-contract.test.cjs). The contract file reads shipped gsd-core/workflows/ui-phase.md under the source-text-is-the-product exception, which the unit file deliberately does not carry (it asserts typed returns only)."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -55,10 +55,22 @@ export interface TaxonomyEntry {
|
||||
consideration: string;
|
||||
}
|
||||
|
||||
/** A UI element to probe; `elements` is an optional authored override of classification. */
|
||||
/**
|
||||
* A UI element to probe; `elements` is an optional authored override of classification.
|
||||
*
|
||||
* `text_en` (#4657) is an optional English translation of `text`, read by element
|
||||
* classification in preference to `text` when present (`text_en ?? text`). `UI_CUES`
|
||||
* are English-only word-boundary patterns, so a non-English `text` (e.g. a project
|
||||
* running with `response_language` set) classifies to zero kinds and lands every
|
||||
* element in the `unclassified` soft signal unless `text_en` supplies an English
|
||||
* rendering. `text` itself is unaffected and keeps its own meaning (the element's own
|
||||
* description, in whatever language the UI-SPEC uses) — only classification reads
|
||||
* `text_en` preferentially. Mirrors the edge adapter's `Requirement.text_en` (#3717).
|
||||
*/
|
||||
export interface Element {
|
||||
id: string;
|
||||
text?: string;
|
||||
text_en?: string;
|
||||
elements?: UIElementKind[];
|
||||
}
|
||||
|
||||
@@ -146,7 +158,7 @@ export const UI_VALIDATORS: Validators = {
|
||||
*/
|
||||
export function validateRequirement(element: Element): void {
|
||||
coreValidateRequirement(element);
|
||||
const r = element as unknown as { elements?: unknown; text?: unknown };
|
||||
const r = element as unknown as { elements?: unknown; text?: unknown; text_en?: unknown };
|
||||
if (r.elements != null && !Array.isArray(r.elements)) {
|
||||
throw new Error(`element ${element.id} elements must be an array when present`);
|
||||
}
|
||||
@@ -155,6 +167,15 @@ export function validateRequirement(element: Element): void {
|
||||
`element ${element.id} text must be a non-empty string when no elements override is provided`,
|
||||
);
|
||||
}
|
||||
// text_en (#4657) is optional, but when present it must be a non-empty string. An empty
|
||||
// string is NOT caught by `??` (only null/undefined are), so an unvalidated `text_en: ''`
|
||||
// would silently win `text_en ?? text` and classify against '' — the same fail-open shape
|
||||
// #1110/#2773 already exist to eliminate, just moved one field over. Validated
|
||||
// unconditionally (not gated on whether `elements` will make it unused) so bad data fails
|
||||
// closed even when it happens to be dead for this particular call.
|
||||
if (r.text_en != null && !(typeof r.text_en === 'string' && r.text_en.trim())) {
|
||||
throw new Error(`element ${element.id} text_en must be a non-empty string when present`);
|
||||
}
|
||||
}
|
||||
|
||||
/** Validate a UI-consideration resolution against the UI verification vocabulary (delegated, D-06). */
|
||||
@@ -184,7 +205,11 @@ export function proposeConsiderations(element: Element): UIConsideration[] {
|
||||
}
|
||||
kinds = element.elements;
|
||||
} else {
|
||||
kinds = classifyElement(element.text as string);
|
||||
// #4657: prefer the English translation when present — UI_CUES are English-only
|
||||
// word-boundary patterns, so a non-English `text` (e.g. response_language projects)
|
||||
// would otherwise classify to zero kinds. validateRequirement (called above) has
|
||||
// already guaranteed text_en, if present, is a non-empty string.
|
||||
kinds = classifyElement((element.text_en ?? element.text) as string);
|
||||
if (kinds.length === 0) {
|
||||
// Prose present but no element cue matched. Do NOT silently drop it (#1110): a UI element
|
||||
// whose phrasing missed every cue would otherwise vanish from coverage with no signal — the
|
||||
@@ -270,7 +295,7 @@ export function proposeElements(elements: Element[]): ElementProposal[] {
|
||||
const considerations = proposeConsiderations(el);
|
||||
const kinds: UIElementKind[] = Array.isArray(el.elements)
|
||||
? el.elements // already validated inside proposeConsiderations
|
||||
: classifyElement(el.text as string);
|
||||
: classifyElement((el.text_en ?? el.text) as string); // #4657: text_en ?? text
|
||||
const unclassified = !Array.isArray(el.elements) && kinds.length === 0;
|
||||
const categories = unclassified ? [] : applicableCategories(kinds);
|
||||
return { id: el.id, kinds, categories, considerations, unclassified };
|
||||
|
||||
@@ -55,6 +55,15 @@ describe('ui-consideration-probe doc/code parity (ADPT-02)', () => {
|
||||
const md = fs.readFileSync(docPath, 'utf8');
|
||||
assert.match(md, /domain-probes\.md/);
|
||||
});
|
||||
|
||||
test('Inputs section documents the text_en translation channel (#4657)', () => {
|
||||
// PARSED from the `## Inputs` section only (never a whole-file substring) — a reformat
|
||||
// that preserves the data does not fail; a dropped field does.
|
||||
const md = fs.readFileSync(docPath, 'utf8');
|
||||
const inputs = md.split(/^## Inputs/m)[1].split(/^## /m)[0];
|
||||
assert.match(inputs, /text_en/, 'Inputs must document the optional text_en field');
|
||||
assert.match(inputs, /English/, 'Inputs must say text_en is the classifier-facing English rendering');
|
||||
});
|
||||
});
|
||||
|
||||
describe('UI-SPEC template `## UI Considerations` section (WIRE-02 SC3 de-dup)', () => {
|
||||
|
||||
104
tests/ui-consideration-probe-ui-phase-contract.test.cjs
Normal file
104
tests/ui-consideration-probe-ui-phase-contract.test.cjs
Normal file
@@ -0,0 +1,104 @@
|
||||
// ui-phase.md Step 9.5 is the deployed workflow runtime contract under assertion; its text IS
|
||||
// what the runtime loads. These checks lock the #4657 text_en wiring so the Non-English
|
||||
// guidance cannot silently rot the way it did for the edge adapter before #3717 — mirroring
|
||||
// tests/edge-probe-spec-phase-contract.test.cjs onto the ui-phase probe step.
|
||||
// Assertions scope to the extracted Step 9.5 block to avoid false positives from incidental
|
||||
// mentions elsewhere in the file.
|
||||
|
||||
'use strict';
|
||||
|
||||
process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const fc = require('./helpers/fast-check-setup.cjs');
|
||||
|
||||
const UI_PHASE_PATH = path.join(__dirname, '..', 'gsd-core', 'workflows', 'ui-phase.md');
|
||||
const { proposeConsiderations } = require(
|
||||
path.join(__dirname, '..', 'gsd-core', 'bin', 'lib', 'ui-consideration-probe.cjs'),
|
||||
);
|
||||
|
||||
function readUiPhase() {
|
||||
// allow-test-rule: source-text-is-the-product (#4657)
|
||||
// The workflow file's text IS the deployed runtime contract; assertions below match on it.
|
||||
return fs.readFileSync(UI_PHASE_PATH, 'utf8');
|
||||
}
|
||||
|
||||
// Slice the Step 9.5 block: from the "## 9.5." heading to the next "## " heading. Scopes
|
||||
// assertions to the probe step only.
|
||||
function extractStep95Block(content) {
|
||||
const startIdx = content.indexOf('## 9.5.');
|
||||
if (startIdx === -1) return '';
|
||||
const rest = content.slice(startIdx + '## 9.5.'.length);
|
||||
const nextHeading = rest.search(/\n## /);
|
||||
if (nextHeading === -1) return content.slice(startIdx);
|
||||
return content.slice(startIdx, startIdx + '## 9.5.'.length + nextHeading);
|
||||
}
|
||||
|
||||
describe('ui-consideration-probe ui-phase contract: text_en (#4657)', () => {
|
||||
test('Step 9.5 documents the Non-English text_en remedy (mirrors spec-phase Step 5.5, #3717)', () => {
|
||||
const block = extractStep95Block(readUiPhase());
|
||||
assert.ok(block.length > 0, 'Step 9.5 block must be extractable from ui-phase.md');
|
||||
assert.match(block, /Non-English/i, 'Step 9.5 must carry the Non-English projects guidance');
|
||||
assert.match(block, /text_en/, 'Step 9.5 must name text_en — the classifier-facing translation field');
|
||||
assert.match(block, /response_language/, 'Step 9.5 must tie the remedy to response_language projects');
|
||||
assert.match(block, /every element/i, 'Step 9.5 must instruct populating text_en for EVERY element, not only UI-obvious ones');
|
||||
assert.match(block, /engine input/i, 'text_en is engine input, never user-facing output (ADR-550)');
|
||||
});
|
||||
|
||||
test('Step 9.5 ELEMENTS_JSON shape comment includes the optional text_en key', () => {
|
||||
const block = extractStep95Block(readUiPhase());
|
||||
assert.match(block, /"text_en"\?/, 'the element record shape must document text_en? alongside id/text/elements');
|
||||
});
|
||||
|
||||
// Property (deterministic: pinned seed, bounded runs, counterexample printed on failure):
|
||||
// for English element prose built from the classifier's own cue vocabulary, a cue-free
|
||||
// Danish rendering as `text` plus the original English as `text_en` must classify
|
||||
// identically to the English original — and the Danish rendering alone must stay
|
||||
// unclassified. Document-shaped inputs: the English words ARE cue words UI_CUES matches;
|
||||
// the Danish translations are verified cue-free by the property itself on every run.
|
||||
test('property: text_en carries classification for any cue-matching prose under a cue-free Danish rendering', () => {
|
||||
const enToDa = [
|
||||
['form', 'formular'],
|
||||
['input', 'indtastning'],
|
||||
['list', 'liste'],
|
||||
['table', 'tabel'],
|
||||
['tabs', 'faner'],
|
||||
['pagination', 'sideinddeling'],
|
||||
['image', 'billede'],
|
||||
['gallery', 'galleri'],
|
||||
['button', 'knap'],
|
||||
['dropdown', 'rullemenu'],
|
||||
['heading', 'overskrift'],
|
||||
['title', 'titel'],
|
||||
];
|
||||
const enWords = enToDa.map(([en]) => en);
|
||||
const daOf = (w) => enToDa.find(([en]) => en === w)[1];
|
||||
|
||||
fc.assert(
|
||||
fc.property(
|
||||
fc.constantFrom(...enWords),
|
||||
fc.constantFrom(...enWords),
|
||||
fc.nat(2),
|
||||
(w1, w2, extra) => {
|
||||
const english = [w1, w2, ...Array(extra).fill(w2)].join(' ');
|
||||
const danish = [w1, w2, ...Array(extra).fill(w2)].map(daOf).join(' ');
|
||||
const cats = (el) => proposeConsiderations(el).map((c) => c.category);
|
||||
assert.deepEqual(
|
||||
cats({ id: 'P', text: danish }),
|
||||
['unclassified'],
|
||||
`the Danish rendering must be cue-free (counterexample text: ${JSON.stringify(danish)})`,
|
||||
);
|
||||
assert.deepEqual(
|
||||
cats({ id: 'P', text: danish, text_en: english }),
|
||||
cats({ id: 'P', text: english }),
|
||||
`translated element must classify as its English original (counterexample text_en: ${JSON.stringify(english)})`,
|
||||
);
|
||||
},
|
||||
),
|
||||
{ seed: 4657, numRuns: 200 },
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -19,7 +19,12 @@ process.env.GSD_TEST_MODE = '1';
|
||||
|
||||
const { test, describe } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const { runNode } = require('./helpers/process-seam.cjs');
|
||||
const { PROBE_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
|
||||
const { throwIfFailed } = require('./helpers/git-fixture.cjs');
|
||||
const { cleanup, createTempDir } = require('./helpers.cjs');
|
||||
|
||||
const BUILT_SCRIPT = path.join(__dirname, '..', 'gsd-core', 'bin', 'lib', 'ui-consideration-probe.cjs');
|
||||
const uc = require(BUILT_SCRIPT);
|
||||
@@ -307,3 +312,140 @@ describe('ui-consideration-probe WIRE-02: backward-compat + format-match + idemp
|
||||
assert.deepEqual(uc.proposeElements(els), uc.proposeElements(els));
|
||||
});
|
||||
});
|
||||
|
||||
// ══ #4657 — the text_en language channel (mirrors #3717/#4156 onto the UI adapter) ═════════
|
||||
// UI_CUES are English word-boundary patterns; a non-English element classifies to zero kinds
|
||||
// and lands in the #1110 unclassified soft signal. text_en carries the classifier-facing
|
||||
// English translation — engine input, never user-facing output. (The ADR-550 amendment's
|
||||
// record scopes to the edge adapter's Requirement; #4657 extends the same remedy shape to
|
||||
// the UI Element.) classifyElement's own signature stays untouched; the text_en ?? text
|
||||
// selection lives at the two classification call sites (proposeConsiderations, proposeElements).
|
||||
describe('ui-consideration-probe: text_en language-aware classification (#4657)', () => {
|
||||
// The reproduction pair from the issue: Danish UI-SPEC prose + faithful English translations.
|
||||
const daList = 'En liste over projektorer med knapper til at forbinde og afbryde.';
|
||||
const daForm = 'En formular hvor brugeren indtaster IP-adresse og adgangskode.';
|
||||
const enList = 'A list of projectors with buttons to connect and disconnect.';
|
||||
const enForm = 'A form where the user enters IP address and password.';
|
||||
|
||||
test('proposeConsiderations: text_en present is used for classification instead of text (failing-first regression)', () => {
|
||||
// text alone (non-English) classifies to zero kinds -> the unclassified sentinel.
|
||||
const nonEnglishOnly = uc.proposeConsiderations({ id: 'U1', text: daList });
|
||||
assert.deepEqual(nonEnglishOnly.map((c) => c.category), ['unclassified']);
|
||||
|
||||
// text_en present -> classification runs against the English translation (list-collection
|
||||
// from "list", interactive-control from "buttons").
|
||||
const withTextEn = uc.proposeConsiderations({ id: 'U1', text: daList, text_en: enList });
|
||||
assert.deepEqual(
|
||||
withTextEn.map((c) => c.category),
|
||||
['empty', 'loading', 'error', 'populated', 'partial', 'overflow', 'zero-one-many', 'long-text'],
|
||||
);
|
||||
});
|
||||
|
||||
test('#4657: a non-English element with text_en classifies identically to its English equivalent', () => {
|
||||
for (const [da, en] of [[daList, enList], [daForm, enForm]]) {
|
||||
const englishOnly = uc.proposeConsiderations({ id: 'U1', text: en });
|
||||
const nonEnglishWithTranslation = uc.proposeConsiderations({ id: 'U1', text: da, text_en: en });
|
||||
assert.deepEqual(
|
||||
nonEnglishWithTranslation.map((c) => c.category),
|
||||
englishOnly.map((c) => c.category),
|
||||
`a translated non-English element must raise the same categories as the English original (${da})`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('proposeConsiderations: text_en absent falls back to text (back-compat — English projects unchanged)', () => {
|
||||
const withoutTextEn = uc.proposeConsiderations({ id: 'U1', text: enForm });
|
||||
assert.deepEqual(withoutTextEn.map((c) => c.category), ['empty', 'loading', 'error', 'partial', 'long-text']);
|
||||
});
|
||||
|
||||
test('validateRequirement: text_en: null is treated as absent (no throw)', () => {
|
||||
assert.doesNotThrow(() => uc.validateRequirement({ id: 'U1', text: 'A results table', text_en: null }));
|
||||
});
|
||||
|
||||
test('proposeConsiderations: text_en: null falls back to text', () => {
|
||||
const viaNull = uc.proposeConsiderations({ id: 'U1', text: enForm, text_en: null });
|
||||
assert.deepEqual(viaNull.map((c) => c.category), ['empty', 'loading', 'error', 'partial', 'long-text']);
|
||||
});
|
||||
|
||||
test('validateRequirement: rejects empty-string text_en (?? does not catch \'\')', () => {
|
||||
// Nullish coalescing only falls back on null/undefined — an empty string would otherwise
|
||||
// win `text_en ?? text` and silently classify against '', degrading to zero kinds with no
|
||||
// signal (the exact fail-open #1110/#2773 exist to eliminate).
|
||||
assert.throws(
|
||||
() => uc.validateRequirement({ id: 'U1', text: daList, text_en: '' }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('validateRequirement: rejects whitespace-only text_en', () => {
|
||||
assert.throws(
|
||||
() => uc.validateRequirement({ id: 'U1', text: daList, text_en: ' ' }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('validateRequirement: rejects non-string text_en (number/array/object)', () => {
|
||||
for (const bad of [42, ['x'], {}]) {
|
||||
assert.throws(
|
||||
() => uc.validateRequirement({ id: 'U1', text: daList, text_en: bad }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('validateRequirement: rejects empty text_en even when an elements override makes it unused', () => {
|
||||
// Validation is unconditional — it does not skip the text_en check just because the
|
||||
// authored-elements branch would never classify prose. Bad data fails closed regardless.
|
||||
assert.throws(
|
||||
() => uc.validateRequirement({ id: 'U1', text: daList, text_en: '', elements: ['form'] }),
|
||||
/text_en must be a non-empty string when present/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('proposeConsiderations: authored elements override still wins when text_en is also present', () => {
|
||||
// nav is an applicable element kind for loading, error, overflow AND long-text.
|
||||
const override = uc.proposeConsiderations({ id: 'U9', text: daList, text_en: enList, elements: ['nav'] });
|
||||
assert.deepEqual(override.map((c) => c.category), ['loading', 'error', 'overflow', 'long-text']);
|
||||
});
|
||||
|
||||
test('proposeConsiderations: zero-cue text_en still surfaces the unclassified soft signal (#1110)', () => {
|
||||
const zeroCue = uc.proposeConsiderations({ id: 'U1', text: daForm, text_en: 'Purple elephant dreams.' });
|
||||
assert.deepEqual(zeroCue.map((c) => c.category), ['unclassified']);
|
||||
});
|
||||
|
||||
test('proposeElements: kinds derive from text_en when present (confirm surface sees the translation)', () => {
|
||||
const [translated] = uc.proposeElements([{ id: 'U1', text: daList, text_en: enList }]);
|
||||
assert.deepEqual(translated.kinds, ['list-collection', 'interactive-control']);
|
||||
assert.equal(translated.unclassified, false);
|
||||
// and without text_en the same element stays on the unclassified path
|
||||
const [plain] = uc.proposeElements([{ id: 'U1', text: daList }]);
|
||||
assert.equal(plain.unclassified, true);
|
||||
assert.deepEqual(plain.kinds, []);
|
||||
});
|
||||
|
||||
test('analyzeCoverage: the reproduction pair — Danish text + text_en yields the English report\'s categories', () => {
|
||||
const english = uc.analyzeCoverage([{ id: 'U1', text: enList }, { id: 'U2', text: enForm }]);
|
||||
const translated = uc.analyzeCoverage([{ id: 'U1', text: daList, text_en: enList }, { id: 'U2', text: daForm, text_en: enForm }]);
|
||||
assert.equal(translated.coverage.applicable, english.coverage.applicable);
|
||||
assert.equal(translated.coverage.unclassified, 0);
|
||||
assert.deepEqual(
|
||||
translated.items.map((c) => c.category),
|
||||
english.items.map((c) => c.category),
|
||||
);
|
||||
});
|
||||
|
||||
test('CLI: elements file with text_en classifies through the built engine (exit 0, no unclassified)', (t) => {
|
||||
const dir = createTempDir('ui-probe-4657-');
|
||||
t.after(() => cleanup(dir));
|
||||
const elementsPath = path.join(dir, 'elements.json');
|
||||
fs.writeFileSync(elementsPath, JSON.stringify([
|
||||
{ id: 'U1', text: daList, text_en: enList },
|
||||
{ id: 'U2', text: daForm, text_en: enForm },
|
||||
]));
|
||||
const r = runNode([BUILT_SCRIPT, elementsPath], { timeoutMs: PROBE_TIMEOUT_MS });
|
||||
throwIfFailed(r, `node ${BUILT_SCRIPT} ${elementsPath}`);
|
||||
const rep = JSON.parse(r.stdout);
|
||||
assert.equal(rep.coverage.unclassified, 0);
|
||||
assert.ok(rep.coverage.applicable > 0, 'translated elements must raise applicable categories');
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user