next
24 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jakub Zych
|
a9a7a328e6 |
refactor: hard-fork GSD -> MSD (Make Software Done)
Mechanical rename produced by scripts/msd-rename.cjs: gsd/Gsd/GSD -> msd/Msd/MSD across contents and paths, upstream package/repo coordinates -> @golem15/msd-core and golem15com/msd-core. Deep links into upstream history, sibling upstream packages, the GSD-2 import feature, CHANGELOG.md and .changeset/ are kept as-is. Hand edits on top: MSD block-letter banner and logos, LICENSE copyright line, package/plugin identity, regenerated lockfile, install-tree fixtures, derived registries and benchmark baseline; migration checksum baseline re-locked (MSD keeps its own install state, so no install had applied the old sums); sort-order and regex-escaped expectations in tests adjusted. |
||
|
Adnan
|
f4bf449296 |
fix(#3850): surface gaps_found VERIFICATION files in audit-uat (#3879)
* fix(#3850): surface gaps_found VERIFICATION files in audit-uat cmdAuditUat admits `human_needed` OR `gaps_found`, but parseVerificationItems had a body only for the first and returned an empty array for the second — standing on a comment deferring to `plan-phase --gaps`, a different command audit-uat never reaches. Since cmdAuditUat pushes a file into `results` only when `items.length > 0`, a `gaps_found` report did not under-report: it vanished, taking its phase's `by_phase` row with it, so a clean-looking total gave the reader no cue anything was skipped. Eligibility now has one owner (the caller) and parseVerificationItems reports what the file says. The closed-entry filter could not be built on extractFrontmatter: its array-item parser keeps only each `- ` entry's FIRST line and has no notion of nested key/value objects, so an entry's `status:`/ `resolution:` siblings never reach its output and a closed entry is indistinguishable from an open one downstream. Rather than grow a competing object-list parser — or change extractFrontmatter, whose blast radius is every frontmatter consumer in the repo — this reads the raw segment BEFORE the flattening, via the existing anchored sliceTopLevelFrontmatterSegments, and hands it to the `## Gaps` machinery that already parses exactly this `- `-opened, indentation- continued shape. The human_needed path is byte-for-byte unchanged: same reader, same display names, same numbering, no resolved-entry filtering — pinned by a test and verified by identical CLI output on base and head. parseGapsItems keeps its narrower `status: resolved` rule so no *-UAT.md behaviour moves. Closes #3850 * chore(#3850): backfill changeset pr number for #3879 * fix(#3850): one parse per entry, one fence parser, one resolved-entry rule Adversarial review on #3879: B1, B2, M3, m5, m8 and n9. B1 — `sliceFrontmatterArrayEntries` hand-rolled a second frontmatter fence regex, which re-asserted the byte-0 rule #2977 removed: a BOM'd file (PowerShell 5.1 `>`/`Out-File` writes one by default) sliced nothing, so a `gaps_found` report vanished from the audit exactly as it did before this fix — this issue's own symptom, on a platform the repo already has a named defect class for. `extractFrontmatter`'s BOM+fence logic is now factored out as `frontmatterRegion` and shared. One fence parser, not two. B2 — the resolved-entry skip paired two DIFFERENT parsers by array index: `parseYamlRegion` is indent-blind, `splitGapsEntries` is indent-anchored. A block sequence written at its key's indent — ordinary, legal YAML — makes them disagree about entry count, and from the first disagreement every index names a different entry, so an OPEN entry inherits a CLOSED one's resolution and is silently dropped. That is the defect this PR exists to fix, reintroduced inside the fix. Display name and sibling fields now come from ONE parse of the raw slice; `frontmatterEntryDisplayName` applies `parseQuotedScalar` exactly as `parseYamlRegion` does, so the string is byte-identical to what `extractFrontmatter` produced. The flattened array remains the #2286 GATE, but is no longer the source of items. `sliceFrontmatterArrayEntries` also takes the LAST duplicate key, matching `parseYamlRegion`'s last-wins assignment. M3 — `frontmatterEntryToUatItem` is the single entry->UatItem mapper both readers use, rather than two copies differing only in `result`. m8 — closed entries are skipped on BOTH statuses. The earlier asymmetry cited an acceptance criterion #3850 does not contain: the issue has no AC section, and its suggested fix (2) states the skip unconditionally, naming a file with 14 of 16 entries resolved. That file is `human_needed`, so the asymmetry left the reporter's own scenario over-reporting by 14. m5 — `sliceTopLevelFrontmatterSegments`' contract doc names both consumers and says the column-0 boundary rule is now a cross-module contract. n9 — the vestigial bare block is gone and its body de-indented. Tests: the B1 BOM case, B2's nested-sequence and bare-bullet repros, a CRLF fixture (M4 — it survived by accident, now pinned) and the unified skip rule. Fail-first verified by running the new tests against the pre-fix build: the BOM, nested-sequence and unified-skip cases are red there. * fix(#3850): read the entries as objects, not as re-parsed display text Rebased onto `next`, which changed the ground this fix stood on. ADR-3473 §8.1 (#3881) replaced the hand-rolled frontmatter scanner with the vendored js-yaml: `parseQuotedScalar` and `parseYamlRegion` no longer exist, and an object entry now flattens to `test: A, resolution: R` rather than to its first line. The original mechanism existed ONLY to work around that lossy first-line flattening — it sliced the raw frontmatter segment and re-parsed each entry by hand so a `resolution:` sibling was visible at all. With a real parser upstream that workaround is obsolete, so it is deleted rather than repaired: `sliceFrontmatterArrayEntries`, `frontmatterEntryDisplayName`, the `splitGapsEntries`/`extractGapEntryFields` reuse and the second fence regex are all gone. `frontmatter.cts` instead exposes `frontmatterObjectListEntries(content, key)` — the same parse `extractFrontmatter` runs (same BOM strip, same byte-0 fence, same anchor/alias and sentinel guards, same ambiguous-colon repair), stopping one step before the display flattening. `flattenObjectListItem` is exposed alongside it so a caller deriving a display name produces the byte-identical string `extractFrontmatter` would have. That collapses the review's blockers into properties of the parse rather than things this fix has to get right: - B1 (BOM) — shares `extractFrontmatter`'s strip; verified through the CLI. - B2 (index pairing) — there is no second reader. Display name and sibling fields come from one object. - M3 (duplicate mapper) — one `frontmatterEntryToUatItem` for both readers. - M4 (CRLF) — js-yaml's, not ours; verified through the CLI. Also confirmed on the rebased base, per review: #3850 still reproduces on `next` after #3707 landed (`total_files: 0`, `total_items: 0` on a `gaps_found` fixture), so this PR is still doing work #3707 did not do. Nothing was dropped as redundant. One behaviour note: `entryField` returns a present value verbatim and treats only whitespace-only as absent. Trimming would rewrite an author's `truth:` on its way to becoming the display name. * fix(#3850): keep every frontmatter list entry at its own row Review round 3's Blocker. `frontmatterObjectListEntries` filtered its result to objects, and filtering COMPACTS: `parseHumanVerificationItems` then numbered the survivors by their position in the compacted array. On a list mixing object and non-object entries the non-object rows disappeared outright and the rest were renumbered — #3850's own vanishing-row defect, reached through entry SHAPE instead of file STATUS. Base never had it: it walked the display array, so every row surfaced at its own position. Renamed to `frontmatterListEntries` and it no longer filters (the name now matches what it returns). Deciding what a non-object entry MEANS is a caller's judgement; dropping it is nobody's. Both readers now walk the DISPLAY array — one element per row, the array #2286 already gates on — and consult the parsed array only for "does this entry carry a closure field?". `parsedEntriesFor` owns that pairing and checks the two lengths agree before trusting an index; all-null is the correct degradation, since over-reporting a closed row is recoverable and closing the wrong one is not. Names stay byte-identical to base for every entry shape, including a nested sequence (`[nested]`, not `["nested"]`). Same class closed in the gaps reader: a non-object `gaps:` entry surfaced nothing at all and now surfaces as `unknown`, which is this module's documented fail-safe direction (`parseGapsItems`) on a false-negative bug. Also restores the shared fence parser round 2 accepted. The ADR-3473 rebase dropped `frontmatterRegion` and left the BOM strip and byte-0 fence rule inlined twice; `extractFrontmatter` now routes through it, so "one fence parser" is enforced rather than asserted in a comment. Minors: `frontmatterEntryToUatItem`'s dead `forcedResult` option deleted and its "shared by both readers" comment corrected — it has one call site, and the two readers differ deliberately, each mirroring its own established sibling (`parseGapsItems` vs #2286). Documented at the divergence. Tests: `B2` asserted a name substring, so it passed while the row was mis-numbered and would have passed through outright loss; it now asserts positions and count. B2b pins the reviewer's 6-entry mixed fixture verbatim, B2c the survivors' file positions across skipped rows, B2d the gaps reader. All four fail-first against the reviewed head; 332/332 green with the fix. * fix(#3850): make status authoritative, and let the two gaps readers agree Round 4 review, all five findings. Major. `isFrontmatterEntryResolved` treated a non-empty `resolution:` as closure regardless of `status:`, so `status: failed` + `resolution: "attempted retry, still failing"` vanished from the report — the silently-vanishing-item defect #3850 exists to close, reached by field combination instead of file status. Closure is now per key, because the two keys have different conventions and one rule cannot serve both: `gaps:` `status: resolved` only, byte-identical to the rule `parseGapsItems` applies to a `## Gaps` markdown section, so one authored entry cannot read closed in one reader and open in the other. `human_verification:` a bare `resolution:` still closes, since that is how verifier-written entries record it — but a readable `status:` that contradicts it wins. A single unified rule was the first draft and is wrong: it closes a frontmatter `gaps:` entry carrying `resolution:` and no `status:`, which `parseGapsItems` surfaces, and `parseVerificationGapsItems`' own docstring claims it mirrors that reader's fail-safe status handling. The contradiction guard is not a judgment call about YAML. It is the rule this codebase already applies to the same field pair: `validateResolution` (probe-core.cts) rejects a populated `resolution:` on a non-resolved status outright — "a populated payload is an authoring mistake ... Reject it so the mistake surfaces." A reporter cannot throw, so it surfaces the item. Minor 1. Direct unit tests for `frontmatterListEntries` and `flattenObjectListItem` in `tests/frontmatter.unit.test.cjs`, the file that historically co-changes with `frontmatter.cts`. They were reachable only through `uat.cts`' readers before. Minor 2. `parsedEntriesFor`'s degrade-to-all-null branch is asserted directly. Verified unreachable through content rather than assumed: both readers enter through `frontmatterRegion`, `extractFrontmatter`'s only extra argument gates a warning, and `normalizeParsedValue`'s `value.map` is 1:1. It is a drift alarm for a future edit to either parser, so the helper is exported for tests rather than left as the one unpinned branch. Minor 3. The vestigial `const skipResolved = true` and its dead conditional are gone. Minor 4. `frontmatterEntryToUatItem` no longer reads `test:`. A `gaps:` entry has no `test:` in its vocabulary — the template's entries carry truth/status/reason/artifacts/missing — so it was speculative support for a field the shape does not have, and it collided with the 1..N row numbers `parseHumanVerificationItems` assigns by array position. Not reading it makes the collision impossible; an offset would have rewritten an authored value, against `entryField`'s verbatim contract. Docs, changeset and the dispatcher docstring all stated the unconditional rule and are corrected — three prior rounds here were comment/code drift. Fail-first proven: restoring the universal rule reddens all three new unit tests and both rewritten properties. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H3eK225hgcnEDZsnmtaP1U --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Tom Boucher <trekkie@nomorestars.com> |
||
|
0xdhx
|
ef9ce3e598 |
fix(#3702): count asterisk, plus and ordered markers as deferred-items entries (#3739)
* fix(#3702): deferred-items counts `*`, `+` and ordered markers as list items
`deferred-items.md` has no template and no mandated shape, but its parser
recognised only the `- ` hyphen marker. Asterisk bullets, plus bullets and
dot-terminated ordered lists — all lists in CommonMark and GFM — contributed
ZERO entries on both the headless and the heading-delimited path, and a mixed
file dropped its non-hyphen entries while keeping their hyphenated siblings,
under-reporting without ever looking empty.
The restriction was a regex literal inherited from the Gaps seam, where the
template genuinely mandates the hyphen YAML-lite form; nothing in the module's
stated rationale distinguishes `*` from `-`.
Widened on the deferred path only:
- `splitGapsEntriesCore`'s entry opener, `extractGapEntryFields`' line-0 strip
and `rawGapEntryText`'s line-0 strip take a `BulletMarkers` parameter that
DEFAULTS to the hyphen-only set, so `## Gaps` keeps its template-mandated
grammar byte-for-byte and the module still has exactly one grouping pass.
- `splitDeferredHeadingEntries`' body-bullet test, `stripLeadingBulletMarker`
and `acknowledgeDeferredItem`'s status-field regexes move in lockstep —
widening what OPENS an entry without widening what is STRIPPED before field
extraction would surface an entry that can never resolve.
Unchanged, and pinned by tests: prose-only and bare headings still contribute
nothing ("prose is not an item"); a table under a leaf heading still yields
exactly its rows, since table lines are skipped before the body-marker flag can
be set and a `|` row is not a list marker; the paren-terminated ordered form
`1)` is out of this fix's scope.
* docs(#3702): changeset fragment (pr: 0 placeholder pre-create)
* fix(#3702): widen the forensic-audit prose entry rule to match the parser
Sibling site of the same defect class, found by a defect-class sweep of the
deferred-items consumers. `/gsd-progress` check 7 does NOT go through
`gsd-tools query` — it globs `deferred-items.md` and has the model read entries
by a prose rule that mandated "one entry per top-level `- ` line". Left as-is,
the marker widening would hold on the CLI path while the one consumer that
bypasses the parser kept reporting "No unresolved deferred items" for a file
written with `*`, `+` or an ordered marker: the same false negative, surviving
in the only place the fix could not reach by code.
Also pass DEFERRED_BULLET_MARKERS explicitly where the heading path extracts
fields. It was already correct — stripLeadingBulletMarker pre-strips the widened
set from every line, so the default hyphen strip is a no-op there — but relying
on that leaves a detection site and a strip site nominally on different marker
sets, which is exactly the asymmetry the BulletMarkers doc comment warns about.
Explicit is local; inferred is a trap for whoever edits the strip next.
Out of scope, noted rather than fixed: forensic-audit.md globs only
`.planning/phases/*/` and so misses archived milestone phases that
`scanDeferredItems` covers. Pre-existing, a different defect, and not this
issue's ruling.
* docs(#3702): note the milestone-close halt for heading-shape non-hyphen files in the changeset
A heading-delimited deferred-items.md written with */+/ordered markers
previously parsed to zero and closed silently; it now yields entries whose
heading shape acknowledgeDeferredItem refuses, halting complete-milestone
until hand-edited. User-visible, so the fragment states it.
* chore(#3702): set changeset fragment pr to 3739
* fix(#3702): CR-normalise the heading path and the acknowledge writer (review B1, M4, m2)
B1 — `splitDeferredHeadingEntries` stored RAW lines; on a CRLF file every
body line but the last still carried its `\r`, the `$`-anchored marker
strip failed on it, the marker survived into field extraction and the
field was lost — a `**Status:** resolved` that was not the file's final
line resurfaced its entry as open. The heading path now stores CR-stripped
lines like the headless path already did, and the strip regex tolerates a
trailing CR on its own. Round 1's CRLF test put `**Status:**` on the last
line, the one position `collectSection`'s `.trimEnd()` had already
de-CR'd; the new tests put it first and mid-body.
M4 (pre-existing on `next`) — `acknowledgeDeferredItem` found the status
line on a CR-stripped copy but rewrote the raw line with a `$`-anchored
`.*`, which cannot consume `\r`; `replace` returned its input, and the
writer reported `ok` over byte-identical content. The rewrite now runs on
a CR-stripped line. The comment that claimed `.*$` consumed the `\r` is
corrected — it was the bug, stated as the design.
m2 — the indent probe for an inserted `status:` line ran on the raw line
and fell back to indent 0 on CRLF; it is CR-stripped too.
* fix(#3702): derive every deferred-items marker regex from one source (review M3, N1, N2)
M3 — round 1 carried the marker alternation in FOUR places: the
`BulletMarkers` pair and two inline literals inside
`acknowledgeDeferredItem`, under a doc comment saying the interface
existed so a detection site and its strip site could not drift. All four
now derive from `DEFERRED_MARKER_ALT`; drift is impossible rather than
discouraged. A parity test pins the vocabulary against
`markdown-sectionizer`'s `iterateBullets` on everything the two grammars
are meant to agree on, and names the two points they deliberately differ.
N1 — the ordered marker is `\d{1,9}\.` (CommonMark §5.2), not `\d+\.`.
N2 — the marker is followed by `[ \t]`, not `\s`, which also accepted
`\r`; the tab remains accepted (CommonMark-legal) and the divergence from
`iterateBullets`' literal space is pinned rather than papered over.
The four regexes are exported for the parity test only.
* fix(#3702): an ordered marker opens an entry only from `1.` or inside a run (review B2, m1)
B2 — `\d+\.` alone read ordinary prose as a list: "2026. was a bad year
for this module" and, under `### Notes`, "3. is the number of retries we
settled on." both opened an entry on round 1, the second straight through
the "prose is not an item" contract that round's AC4 claimed to preserve.
CommonMark §5.3 faces the same ambiguity when an ordered list would
interrupt a paragraph and resolves it by requiring the list to start with
1; `matchListOpener` applies that rule wherever an ordered marker is seen,
with the run carried per list (headless) or per leaf-heading body. Numbers
after the first are ignored, as CommonMark ignores them. Stated cost,
pinned: a hand-numbered list starting at 2 reads as prose — every ordered
record in the #3702 scan starts at 1.
Both reviewer cases are pinned as prose; the ruling's `1. alpha / 2. beta`
shape still counts.
m1 — the 9-digit boundary is pinned at both sides (`999999999.` opens,
ten digits is not a marker), and the 3-vs-4-space indentation cliff is
pinned as deliberately NOT applied: the parser is indent-lenient because
surfacing a questionable hand-written entry beats dropping a real one.
* fix(#3702): thematic breaks close the list and fenced code never opens an entry (review M1, M2)
M1 — `- - -` was a phantom `"- -"` entry on base; widening the marker set
added `* * *` and `+ + +` to the class, and `* * *` is the separator an
author writing in the `*` style is most likely to use. A CommonMark §4.1
thematic break (plus the `+ + +` gesture, which is the same garbage as an
entry name) now closes the open entry on the headless path and is dropped
from the body on the heading path — neither an item nor a continuation.
M2 — neither splitter was fence-aware, so `+ `-prefixed diff lines and
`1.`-numbered repro steps inside a code block counted as entries; #3702's
wild records carry exactly those blocks. Both splitters now classify lines
by the sectionizer's own `scanFencedBlocks` (so `~~~`, indented and
unterminated fences behave as `stripFencedCode` would): fence content
never opens an entry, is continuation inside an open one — keeping the
span invariant `acknowledgeDeferredItem` re-verifies — and is discarded
before the first.
* test(#3702): range the #2287 deferred-items property over marker × shape × line ending (review B3)
The `#2287` property hard-coded `- ` and filtered `\r\n` out of its
arbitraries, so the widened marker set — an enumerated domain, exactly
what a property is for — was never under it. It now ranges over
`{-, *, +, ordered}` × `{headless, heading}` × `{LF, CRLF}`, with the
heading shape placing `**Status:**` first or last: the review's
prescription (markers × line endings) would not have reached B1, which
lives on the heading path only, so the shape axis is the load-bearing
addition. Ordered entries are numbered from 1, so the B2 run rule is
under the property too.
A second property drives `acknowledgeDeferredItem` over every unresolved
headless entry across the same marker × line-ending grid — the one that
reaches M4 (a CRLF rewrite that reported `ok` and wrote nothing) and m2.
* test(#3702): pin the milestone-close halt on a heading-delimited `*`/`+`/`1.` file (review m3)
A heading-delimited `deferred-items.md` written with a non-hyphen marker
previously parsed to zero entries and let `complete-milestone` close
silently; it now yields entries whose heading shape `acknowledgeDeferredItem`
refuses, which the milestone loop turns into `record_ack_failure` → exit 1.
The loop is prose in a workflow, so the test drives the two CLI calls it
makes: `audit-open --json` must list the entry, and
`audit-open acknowledge --text <the audit's own text>` must refuse with the
heading-delimited message and write nothing.
* docs(#3702): changeset and forensic-audit prose carry the round-2 grammar
The changeset names the CRLF fixes, the ordered start-at-1 rule, thematic
breaks and fences. The `/gsd-progress` forensic-audit step is the one
prose parser of this file and must state the same grammar the code has.
* fix(#3702): round-review refinements — run ends at a paragraph, rejected ordinals unstripped, breaks at any indent, fenced fields, `## Gaps` scope
Findings from the pre-push adversarial review of round 2, each pinned:
- An ordered run ENDS at a paragraph that follows a blank line (CommonMark
§5.3); a non-indented line with no blank before it is lazy continuation
and keeps the run open. `1. a` / blank / `paragraph` / blank / `5. x` is
one entry, not two.
- The heading path strips the marker off every body line before field
extraction (#3457); a line whose ordinal `matchListOpener` REJECTED must
not be stripped, or "3. status: resolved" as prose loses its `3. ` and
reads as a resolved field. `splitDeferredHeadingEntriesDetailed` now
carries a per-line opener flag and only accepted openers are stripped —
in headless regions of a heading-shaped file too.
- A thematic break is recognised at any indent, matching the parser's
indent-lenient reading of items; ` * * *` was a phantom `* *`.
- Fenced lines carry no FIELDS either: a `status: resolved` quoted inside a
code block no longer resolves its entry on either path.
- Block structure (breaks, fences) is a property of the GRAMMAR, carried as
`BulletMarkers.blockStructure`: the deferred set opts in, the Gaps set
does not, so `## Gaps` is byte-for-byte on its `next` behaviour — the
round-2 M1/M2 change had reached it through the shared splitter.
* test(#3702): the property exercises the rejected-ordinal branch; the N2 control is independent
Round review: the widened #2287 property numbered every ordered run from 1
and so never generated an ordinal the start-at-1 rule rejects — it could
not tell round 1 from round 2 on B2. Each entry may now carry a decoy prose
line beginning with a non-1 ordinal, placed where it cannot end a run
(before the first headless entry; first in a heading body), followed by a
`status: resolved` that must never become a field; and a decoy-only
heading body must yield no entry.
The N2 assertion accepted a tab, which round 1's `\s` accepted too, so a
`[ \t]` → `\s` revert alone stayed green. NBSP, form-feed and vertical-tab
are now asserted refused — the assertion that fails on that revert on its
own, and the disclosure that `[ \t]` narrows what round 1 accepted.
* fix(#3702): the splitter records its own opener flags; an opener clears the blank-line memory
Round-review continuation, two state defects in the ordered-run logic:
- `blankSeen` survived the headless splitter's opener branch, so an opener
followed by a lazy continuation line read as "paragraph after a blank" and
ended the run — `1. a` / blank / `2. b` / lazy / `3. c` folded `c` into `b`.
The opener branch now clears it.
- The heading path re-derived per-line opener flags for headless regions
without the paragraph reset, re-accepting a rejected `3. status: resolved`
under a stale run and stripping it into a field. `GapsEntrySpan` now
carries the flags the splitter itself computed, and the heading path reads
them; the re-derivation is deleted.
* fix(#3702): ordered-run memory is per indent — nested runs resolve, nested ordinals never inherit the top-level run
Round-review continuation 2: nested openers consulted the TOP-LEVEL run
flag and never wrote their own, so a nested `1. / 2.` run under a hyphen
entry rejected its `2. status: resolved` (round 1 resolved it), while a
nested `3. status: resolved` under a nested `- ` bullet inherited an open
top-level run and was stripped into a false field.
`OrderedRuns` keys the memory by indent: a new opener at indent d resets
every deeper level, a paragraph after a blank at indent d ends the runs at
d and deeper, a thematic break or a heading clears all. Both splitters use
it; the top level still decides entry boundaries, nested levels decide
only which continuation lines are accepted openers for field stripping.
Pinned for LF and CRLF.
* fix(#3702): run levels — one top level at or above the base, CommonMark column indents, a fence ends its level's runs
Round-review continuation 3:
- A dedenting top-level list (` 1.` / ` 2.` / `3.`) lost its entry
boundaries: the exact-indent run lookup rejected the shallower ordinals
before the boundary check ran. Every indent at or shallower than the
list's base is now ONE level, in both splitters.
- `indentOf` counted characters, so a tab and a space aliased to one level
and `\t1. nested` / ` 2. status: resolved` resolved falsely. Indent is now
measured in CommonMark columns (§2.2: a tab advances to the next multiple
of 4), for the run level and the entry-boundary check alike.
- A nested run survived a fenced block. A fence is a non-list block: its
opening delimiter ends the runs at its level and deeper, exactly as a
paragraph after a blank does.
* fix(#3702): the indent measure is grammar-scoped — Gaps keeps next's character count
`blockStructure: false` promised the Gaps grammar byte-for-byte parity with
`next`, but the CommonMark-column indent measure added for the deferred
grammar was shared by the whole splitter core, so tab-indented Gaps input
changed entry boundaries in BOTH directions:
`\t- a` / ` - b` — next folded into one entry, HEAD split into two
` - a` / `\t- b` — next split into two, HEAD folded into one
`indentWidth` now keys the measure on the grammar: columns for the deferred
set, raw character count for Gaps. The opt-out covers indent semantics, not
only fences and thematic breaks.
Four cases pin both halves — the two flipped Gaps pairs, the two Gaps pairs
that never moved, and the same tab/space pairs on the deferred path returning
the opposite (column-measured) verdict by design.
* fix(#3702): the acknowledge path reads and writes through one classifier
Round 3, Blockers 1 and 3, and Minors 7 and 8 — one mechanism, so one commit.
Every consumer of an entry's lines now reads the splitter's own per-line
verdict instead of a re-derivation of it.
B1. Round 2 widened the WRITER's status-line finder to the deferred marker set
while `extractGapEntryFields` still de-bulleted line 0 only. A nested
` * status: pending` was therefore selectable by the writer and invisible to
the reader: acknowledge rewrote it in place, returned `ok`, and the item stayed
outstanding on every later audit. Measured against a `next` build, `*`, `+` and
`1.` each resolved on base and stopped resolving at round 2's head — a
regression, not a gap in new behaviour. The hyphen form of the same shape was
already broken on `next` and is fixed here too: one classifier cannot be right
for three markers and wrong for the fourth.
`parseGapEntryFieldLine` is now the single place a line is classified as a
field, and it reports the offset at which the VALUE begins. The rewrite happens
at that offset rather than through a second regex, so a line the classifier can
select is one whose rewrite it has already located — the selection and the
rewrite cannot disagree. Both `DEFERRED_STATUS_FIELD_RE` and
`DEFERRED_STATUS_REWRITE_RE` are deleted rather than widened. A read-back guard
returns `rewrite_not_readable` rather than `ok`; it is unreachable by
construction today and is the fail-loud floor under the next divergence.
B3. This is the end state the round-3 review prescribed on both #3739 and
#3773: #3773's shared classifier, parameterised by this PR's marker set, with
this PR's two status regexes deleted. #3773 lands first. Its hyphen-only strip
is consistent with `next`'s hyphen-only splitter today, so the writer/reader
divergence is created by THIS merge, which is why widening every consumer
belongs to the PR that widens the domain.
m7. The heading path marker-stripped its lines before calling the reader, so
the reader's fence scan ran over text the splitter never saw: `- ```sh` is an
ordinary bullet to the splitter but strips to a fence opener, and a
`**Status:** resolved` after it was suppressed as fence content — a resolved
entry resurfaced as open. Stripping now happens inside the reader, after the
fence scan.
m8. `rawGapEntryText` stripped a marker off line 0 unconditionally, but on the
heading shape line 0 is the heading TEXT: `### 1. Race in the writer` was
silently renamed to `Race in the writer`, and the name is the key acknowledge
matches on. Line 0 is stripped only when the splitter accepted it as an opener.
Also removed: `splitDeferredHeadingEntries`, whose sole caller only null-checked
it (round 3, M4 — the claim was zero callers, which was wrong; the wrapper's
`.map` was waste at the one call site), and `stripLeadingBulletMarker`, which
this change leaves with no callers at all. The export surface narrows to the two
splitter regexes the behavioural parity test reads (M6).
[PEER-ASK pr-order-12d5]
q: Reviewer blocked both on merge order. I'm declaring #3773 lands first and
building the end-state shape into #3739 now (both my status regexes
deleted). Does that match your plan?
reply: CONFIRMED - same order, derived independently. #3773 cannot carry the
fold: `DEFERRED_BULLET_MARKERS`/`BulletMarkers` have zero occurrences at
`next` (verified), so the prescribed end state is not executable inside
#3773 without absorbing this PR's work.
deadline: 03:55 UTC (answered before it)
fallback: declare #3773 first, adopt end-state shape in #3739, push+comment
decision: proceeded as stated; #3773 lands first, this PR carries the widening
of every consumer.
Refs #3740
* test(#3702): pin the detect/strip symmetry, and drop a white-box test that could not reach it
Round 3, Blocker 2 and Minors 6 and 9.
B2. The regression shipped green because no fixture put a marker on a nested
status line. Four markers x {nested status line}, each asserting the entry
READS BACK as acknowledged rather than that acknowledge merely reported `ok` —
reporting `ok` over a line the reader skips is the whole defect. Plus the bare
capitalised `Status:` case (the reader stores it case-sensitively, so the
writer must not select it), and an idempotence test, which is the failure the
defect actually produced: the item resurfaces, is acknowledged again, and never
settles.
Each of these was run against the pre-fix build first: all five fail there and
pass here. Two further assertions in the block are labelled CONTROL because
they held pre-fix — they guard the new offset-based rewrite and the opener-flag
threading against regressing, and calling them regression tests for a reported
defect would overclaim.
M6. The round-2 parity test asserted that four writer-side regexes embedded the
same source string. That is true of a detect/read asymmetry too, so it could
not have caught B1 — and two of the four regexes were widened into `export =`
purely to let it read them. Replaced with a behavioural test that drives the
real seam: every marker that opens an entry must also resolve it through
acknowledge. The structural assertion is kept for the two splitter regexes,
which really are two copies of one alternation.
m9. `expectedResolved` was computed and immediately voided; the loop beneath it
already asserts both polarities.
m7/m8 coverage lands here too: a bullet whose content is a fence opener must
not suppress the entry's fields, and a heading beginning with a list marker
must keep it in the entry name.
* docs(#3702): document the deferred-items entry shape where the file is written
Round 3, Major 5, and #3702's own item 2. The widened grammar was documented in
the reader (`forensic-audit.md`) but not at the write site, where
`executor-examples.md` still said only "log to deferred-items.md" — so the
question the issue actually raised, which shapes count, remained unanswered
anywhere a human writes the file.
States what opens an entry (`-`, `*`, `+`, and `1.` when the list starts at
`1.`), that `1)` is not a marker here, that a separator closes the list and
fenced content is never an entry or a field, and that an entry without an
explicit `status: resolved` stays open by design.
* chore(#3702): regenerate the changeset through the generator
Round 3, Minor 10. The fragment was hand-named against 64 generated names on
`next`, and its body ran ~250 words against CONTRIBUTING's one-sentence form.
Regenerated via `npm run changeset`, which is also what the random three-word
name is for: concurrent PRs never collide.
* fix(#3702): the fence gate lives on the seam both sides call, not just the reader
Found by the pre-push adversarial review of this round, and it is a regression
this round introduced rather than a pre-existing one.
`extractGapEntryFields` applied `fencedLineSet` before classifying; the
acknowledge writer's status-line search did not. So a `status:` line inside a
fenced block was SELECTED by the writer and SKIPPED by the reader — the write
produced a line nothing reads, the read-back guard refused it, and the entry
became impossible to acknowledge at all: `audit acknowledge` raised an internal
error and `complete-milestone` halted on it.
Measured, `- alpha` / fence / ` status: pending` / fence:
next ack=ok -> reads back "acknowledged"
round-2 head ack=ok -> reads back "" (the B1 defect)
before this ack=rewrite_not_readable -> refuses entirely (worse than next)
`entryFieldLines` is now the seam — per line of an entry, the field it declares
or `null`, fences included — and the reader and the writer both go through it.
That makes "the writer cannot select a line the reader will not read back"
structural rather than asserted, which is what the previous commit's message
claimed while a second read-side filter still lived outside the classifier.
Two comments corrected with it. The read-back guard is NOT "unreachable by
construction": this round shipped a reachable path to it, which is precisely
what an invariant asserted in a comment is worth. And the M6 replacement test
put its marker only on the entry opener, so it passed against the defective
build — the exact weakness it was introduced to fix in round 2's test. It now
marks the nested status line too, and fails pre-fix like the rest.
Round-3 tests against the pre-fix build: 10 of 12 fail there, and the 2 that
hold are labelled CONTROL because they guard this round's new code rather than
pin a reported defect.
* fix(#3702): one end-of-file CRLF algorithm, adopting #3773's with its B4 closed
Round-4 M1. Two open PRs shipped two different answers to "what line ending
does an entry that ENDS THE FILE get?", and the review's ruling was that the
disagreement needs one answer, not two. Neither shipped answer was that one.
Measured on builds of both heads:
case #3739 r3 #3773 here
undelimited single entry, CRLF preamble pass FAIL pass
LF-dominant list, one stray CRLF at EOF FAIL pass pass
(the other five) pass pass pass
This PR's content.endsWith('\r\n', matchIndexInContent) reads the terminator of
the PREVIOUS line, so it propagated an isolated CRLF into an LF-dominant list --
refuted by #3773's own LF-dominant fixture, ported here. Withdrawn.
#3773's crlfAtEof asks the right question -- does anything before the entry,
within scope, contradict CRLF -- and fails closed. But its scope goes EMPTY for
an undelimited single-entry list, because the entry-list region runs from the
first entry's start to the insertion point and those coincide; crlfAtEof('') is
false by its own before.length > 0 guard, so 'preamble\r\n\r\n- alpha' gained a
bare \n in a CRLF document. That is #3773's B4, verified by driving its head.
Adopted here with the scope widened to everything preceding the insertion point
where the preferred region is empty, rather than asserting LF from no evidence.
That only ever loosens a scope carrying zero information, and the predicate
stays fail-closed over the wider one. An entry at offset 0 of an undelimited
document has no evidence under either scope and stays LF.
Tests: 10 added. Negative control, driven -- 1 of the 10 fails against this
branch's own pre-fix head (the stray-CRLF fixture); B4 fails against #3773's
head; the remaining 8 are the scope counterexamples ported with the function,
which were regression pins in #3773 and are guards here. Each still kills a
simpler algorithm: drop any one and a refuted scope passes again.
Four deferred-items suites 450/450, 0 skipped. npm run lint:ci exit 0.
* fix(#3702): drop the unreachable rewrite_not_readable guard (B3)
Round-4 B3: the status had zero test coverage in either file. The review
offered two branches -- drive it from a test, or delete it and stop carrying an
untested terminal status. Taking the second, with the reason stated rather than
assumed.
Why it cannot be driven. Round 3 added the guard after a fenced `status:` line
proved the writer could select a line the reader would not read back. Round 3
then closed that divergence STRUCTURALLY, by routing the writer's line selection
and the reader's field extraction through one entryFieldLines seam. The guard
now detects a state construction prevents: 21 document shapes were driven
against it -- fence openers on the bullet line for every marker in the widened
set, duplicate and triplicate status lines, bolded and nested variants, fences
between duplicates -- and none reached it. The only seam that would is routing
the internal call through the module's exports so a test could stub it, which
reshapes production surface for a test.
Why leaving it undriven is not free. RULESET.TESTS.mutation-score runs Stryker
incrementally over changed files at an 80% threshold and says to treat a
surviving mutant as a failing test specification. An undriven `if` on a changed
file is exactly that, on both the condition and the .toLowerCase() comparison.
What this gives up, stated rather than hidden: if a future change re-splits the
writer's selection from the reader's extraction, acknowledgeDeferredItem returns
ok over an item that stays outstanding -- the original #3702 defect class. One
correction to the review's framing: match_verification_failed does NOT backfill
it. That check runs BEFORE the write and compares the matched span to the
target, so it cannot see a post-write read-back failure. The protection against
re-splitting is the shared seam and the round-3 tests that pin it, not a runtime
assertion. A comment at the removal site records all of this.
Removing it also drops the union member from both files, which resolves the PR
body's internal contradiction (it claimed no type-signature changes while adding
one) and the duplicate-status surface #3773 collides on.
No test changed behaviour: 450/450 across the four deferred-items suites, 149/149
across the audit suites, npm run lint:ci exit 0 -- the same figures as before the
removal, which is itself the evidence that nothing exercised the branch.
* fix(#3702): the deferred fence gate is indent-unbounded, like the rest of the grammar (M2)
Round-4 M2. scanFencedBlocks is CommonMark, which caps a fence delimiter's
indent at three spaces -- a fourth makes it an indented code block instead. This
grammar had already opted out of that cliff for entry openers ([ \t]*) and for
THEMATIC_BREAK_RE (^[ \t]*), but not for fences. So a fence at four spaces was
not a fence to the gate, and a `status: resolved` line inside it RESOLVED the
entry containing it.
That is not an exotic shape. A fenced block written under a NESTED bullet sits
at four spaces, so ordinary hand-written deferred-items.md files reach it.
Driven before the fix at indents 4, 5, 8 and a leading tab: all four silently
resolved. It is the #3702 silent-resolution defect class in a new place.
gsd-core/references/executor-examples.md, added by this PR, states flatly that
"nothing inside a fenced code block is an entry or a field". The review offered
fixing the parser or bounding that claim in three places. Fixing it -- the claim
is the one users will rely on, and the grammar had already chosen unbounded
indent everywhere else.
NO second fence dialect (the rule blankIndentedFenceDelimiters states). The
classification is still done by scanFencedBlocks, the one exported CommonMark
state machine, over a de-indented VIEW of the same lines. Run lengths, backtick
vs tilde, closer-must-match-and-not-trail, info-string rules and the
unterminated-at-EOF case remain that engine's answers. Indent is the only
dimension hidden from it, and it is exactly the dimension this grammar has
already declared it does not measure. Index alignment is 1:1 -- map preserves
length -- so every returned line index still addresses the original line.
Scope is the deferred grammar only. Both marker-parameterised call sites gate on
markers.blockStructure, which the Gaps set does not set, so Gaps reaches an empty
set. Verified, not asserted: the 47-fixture Gaps differential (marker x
line-ending x separator x fence x break x key-shape x list-shape) is
BYTE-IDENTICAL across this change, 8033 bytes both sides.
Tests: 14 added, of which 8 fail against the pre-fix source and pass here; the
other 6 are the deliberate controls -- indents 0 through 3, which must NOT move,
and the Gaps opt-out guard.
Four deferred-items suites green; the 58 suites touching uat/deferred/sectionizer
run 6045 tests with an IDENTICAL failing set before and after this change (17
pre-existing environment failures -- installs and an unpinned GSD_EMITTED_BASE;
emitted-attribution passes 259/259 in isolation with its base pinned). lint:ci
exit 0.
* fix(#3702): changeset, both prose parsers, and the minors (M3, M4, m1-m3, m5, n1-n2)
M3 -- the changeset omitted a user-BREAKING change. Measured against next: a
heading-delimited deferred-items.md written with `*`, `+` or `1.` went from
"0 entries, so complete-milestone has nothing to acknowledge and closes" to
"1 entry, the CLI writer refuses the heading shape, ACK_FAILURES accumulates,
exit 1". The `-` form already halted and is unchanged. That is release-note
material: a close that used to succeed now fails, and the correct response is to
fix the file, not revert. Also names the fence-indent fix below, and adds #3740
so #3773's issue is attributed here as it is absorbed.
M4 -- gsd-core/workflows/progress/steps/forensic-audit.md is a SECOND,
model-executed parser of the same grammar, and prose cannot carry a parity test.
Its widened text stated the start-at-1 rule, fences and separators but not the
`1)` exclusion nor the nine-digit ordinal cap, both enforced in code with pinned
tests. Both stated now, along with the round-4 fence-indent rule. (No ack
fragment: the size ratchet's currentSizes does a NON-recursive readdirSync of
gsd-core/workflows and agents, so a file under workflows/progress/steps/ is
outside its scope -- verified by reading the helper, not by the green.)
n1 -- executor-examples.md documented that the BOLDED status key is matched
case-insensitively and left the bare key's rule to inference. Driven: bare
`Status: resolved` is NOT read, so the entry stays open with no warning, while
`**Status:**` is. Stated explicitly, with the digit cap and the any-indent fence
rule (n2).
m1 -- boundary coverage was 2/3. limit (999999999.) and limit+1 (1234567890.)
were pinned; limit-1 (12345678.) added, per RULESET.TESTS.boundary-coverage.
m2 -- THEMATIC_BREAK_RE and the tab-expanding indent counter are hand-rolled
CommonMark rules with no in-repo peer to compare against, so the parity
assertion is against the SPEC: eight positive and five negative fixtures, plus
the two DELIBERATE divergences pinned as deliberate (`+` is a separator here but
not in CommonMark, because `+` is a list marker in this grammar and `+ + +`
would otherwise be a phantom entry; indent is unbounded). One fixture was
initially wrong -- `-- -` IS a CommonMark break, since the spec allows free
spacing between the three characters -- and the parser was right.
m3 -- the result union is hand-duplicated in audit.cts as part of a deliberate
structural view of uat.cjs, so the fix is not to delete a copy but to make drift
observable. Every REACHABLE status is now driven from a fixture; four of the six
(ambiguous, unsupported_heading_shape, already_resolved, match_verification_failed)
had no assertion anywhere in the suite before this. match_verification_failed is
still undriven and the test says so rather than omitting it.
m5 -- DECLINED, with the measurement. The review is right that `(\s*)` in the
opener and `/^[ \t]*/` in the reader disagree about \f, \v and NBSP, but its
prescribed narrowing was implemented, driven and REVERTED: as shipped, an entry
indented with any of those surfaces, parses its status field, acknowledges, and
reads back acknowledged -- a complete round-trip. Narrowing turns all three into
SILENTLY DROPPED entries, which is the #3702 defect class itself and the opposite
of this file's stated fail-safe rule. A latent inconsistency in the safe
direction is not worth a live regression in the unsafe one. Pinned by three
round-trip tests so the prescription cannot be re-applied silently; if it is ever
closed, the direction is to make the readers agree with the opener, not to make
the opener reject lines it accepts today.
Four deferred-items suites 475/475, 0 skipped. lint:ci and lint:changeset exit 0.
The 47-fixture Gaps differential is byte-identical at 8033 bytes.
* fix(#3702): the pinned `## Gaps` phantom now cites its issue (m4)
Round-4 m4. The second assertion in the Gaps byte-for-byte test pins a real
defect as expected output: a spaced hyphen thematic break in `## Gaps` is read
as an ITEM, so `- - -` surfaces a phantom open gap named `- -`. Reproduced on
pristine next at
|
||
|
Tom Boucher
|
b431ae9f0d |
fix(#3898): a separator-shaped line in ## Gaps is skipped, not made an entry (#4057)
* test(#3898): a spaced-hyphen thematic break in ## Gaps is not an entry (failing first) * fix(#3898): a separator-shaped line in ## Gaps is skipped, not made an entry splitGapsEntriesCore's opener regex (/^(\s*)-\s/) matched a spaced hyphen thematic break, fabricating a gap named '- -' with result 'unknown' — surfaced by audit-uat as an outstanding finding that cannot be cleared by editing any entry, because there is no entry, only the separator the author wrote for readability. Five shapes were affected (- - -, -, and wider/indented variants); the unaffected ones (---, ----, * * *, ___) were safe only by accident — the path never matched them, not because it understood breaks. A line whose content after the opening marker is solely hyphens and whitespace (with at least one further hyphen) is now skipped entirely — neither an opener nor a continuation. Deliberately option 2 from the issue, not a full thematic-break concept: a break does not close the Gaps list, entries after it keep parsing, and the deliberately-frozen byte-for-byte Gaps behavior changes ONLY for documents carrying such a separator (which previously produced a phantom). A real entry whose truth begins with a hyphen (- truth: "-5 error budget...") is untouched — its remainder contains non-hyphen characters. * fix(#3898): review fold-ins — span-contiguous skip, property coverage The skip is narrowed to where the phantom came from: a separator-shaped line BETWEEN entries (nothing open, or it would open a top-level entry). One landing strictly inside a live entry (indent > baseIndent) folds back as a continuation line, so entry lines and the GapsEntrySpan agree byte-for-byte — the span invariant and the #3805 ack writer's identity re-verification both hold (the review traced the unconditional skip to a match_verification_failed refusal in that corner). Adds the parser- convention property test (arbitrary hyphen counts/indents/spacings) and a span-contiguity pin. * chore(#3898): changeset fragment (pr number backfilled after PR creation) * chore(#3898): backfill changeset PR number (4057) --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
2012e8cc7f |
fix(#3781): span-carrying heading walk unblocks acknowledge on heading-shaped deferred items (#3998)
* test(#3781): heading-shaped deferred entries must be acknowledgeable * fix(#3781): span-carrying heading walk unblocks acknowledge on heading-shaped deferred items * test(#3781): table fixture counts the row; BLOCKER 1 updated to the supported contract * chore(#3781): changeset fragment (pr number backfilled after PR creation) * chore(#3781): backfill changeset PR number (3998) --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
9f1996b8f9 |
fix(#3775): ack matches exactly the status-line case shapes the reader reads back (#3989)
* test(#3775): bare Title-case status lines must ack through the reader-visible path * fix(#3775): match exactly the status-line case shapes the reader reads back * chore(#3775): changeset fragment (pr number backfilled after PR creation) * chore(#3775): backfill changeset PR number (3989) --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
bbecc6a08b |
fix(#3740): ack writer searches the exact status-field shape the reader parses (#3940)
* test(#3740): acknowledging a nested-marker status line must clear the entry * fix(#3740): narrow the ack status-field search to the marker-free shape the reader parses * chore(#3740): changeset fragment (pr number backfilled after PR creation) * chore(#3740): backfill changeset PR number (3940) --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
4e8927b0b9 |
fix(#3707): degrade the fold for every UAT gap class, and stop line endings hiding rows from the audit and the acceptance gate (#3903)
* test(#3707): failing-first coverage for reverting the fence-shortfall fold shield Pins the post-revert contract: a phase whose only gap is a fence shortfall must degrade the fold and withhold the milestone percentages, like every other gap class. Five of the eight rows are CONTROLS that pass before the change, and they carry more weight than the failing row. The failure mode of this revert is degrading TOO MUCH: a revert that sets foldScope outside the headingsSeen > 0 branch would withhold every percentage in the project, and only the no-gap control catches that. Another control catches a revert that collapses the two scopes into one and loses the distinction between what a phase reports and what the fold folds -- uat.scope must stay TRUNCATED for every gap either way, which it already is. The row that pinned the shielded behavior is rewritten rather than deleted. Deleting a test because the behavior it asserts is being reversed leaves the reversal unguarded. * fix(#3707): degrade the fold for every UAT gap class, reverting the fence-shortfall shield Maintainer decision. The two orthogonal engines split on this during #3707 and neither filed it as blocking, so it shipped in the shape the engine that raised the objection endorsed after verifying seven fixtures. The call has now gone the other way, restoring the fail-safe direction chosen twice already on this issue. The shield exempted one gap class from the fold's teeth. It could not do that safely: shortfallBlocks is a single tally incremented at exactly one site and spans BOTH a harmless fenced documentation sample AND a genuinely fence-straddled result: blocked row. Exempting it therefore could not exempt only the harmless case -- it also published a milestone percentage over a real, unread outstanding row. SCOPE.TRUNCATED means the scan could not SEE part of the evidence, which is exactly that case. scope and foldScope now agree: every gap class degrades both. The accepted over-report documented in uat.cts is unchanged and still documented there; what changed is only that it no longer buys an exemption from the fold. The comment block above it argued FOR the shield and is rewritten, because a comment defending behavior the code no longer has is worse than no comment. shortfallBlocks leaves this function's destructure but is untouched upstream, where audit-uat still consumes it. * fix(#3707): correct the caller comment, add the changeset, and name what the order tests guard Review found a SECOND comment still documenting the removed shield -- the caller's, beside the worstScope fold, stating that foldScope differs from scope for exactly one case which must not raise phase_scope_degraded or withhold the milestone's percentages. That is now the opposite of what the code does. I rewrote the buildUatRows comment in the previous commit and asserted in its message that a comment defending behavior the code no longer has is worse than no comment, then left exactly that one standing a few hundred lines away. The change had no changeset. It is user-visible: a milestone's percentage goes from published to withheld whenever any phase has a fence-shortfall-only gap. PR gates hard-fail a user-facing code diff without one. The two scopes are now identical at every return site. They are NOT collapsed -- that would change the return shape and the caller on what is meant to be a one-condition revert, and the seam is worth keeping if the distinction is ever wanted again -- but the declaration now says plainly that they agree by decision rather than by accident, so a reader does not have to re-derive it. The two order-independence tests were renamed. foldScope is monotonic with no reset path, so file order is structurally irrelevant and those rows could never have failed for the ordering reason their names promised. They do guard something real -- a multi-file phase degrading when any one file has a shortfall-only gap -- so they now say that instead. * test(#3707): failing-first coverage for the lone-CR UAT false-clean The parser splits on newline only, and the heading tokenizer agrees with it, so a lone carriage return is not a line boundary anywhere in it. CommonMark treats a lone CR as a line ending, so such a row renders to a human reader while being invisible to BOTH sides of the parser's symmetry invariant: no item, no shortfall, no headingsSeen. A phase hiding a result: blocked row this way reports 100 percent with zero diagnostics. Found by the security review of the fold-shield revert. It is the one false-clean class that revert does not reach, and it is the same bug class this issue exists to fix -- an unreadable row reported as clean. Nine rows. The LF control is what proves this is a separator defect rather than a content defect: identical bodies, one separator apart, and only one of them hides the row. CRLF and CR-inside-a-fence controls guard the coming normalization against double-counting or tearing content that legitimately contains a carriage return. Two further manifestations turned up while writing them: a leading CR breaks column-0 anchoring of the first heading, and an all-CR document flags a shortfall it cannot attribute to any row. * fix(#3707): treat a lone carriage return as a line ending in the UAT parser A lone CR was not a line boundary anywhere in the parser -- it split on newline only, and the heading tokenizer agreed with it. CommonMark treats a lone CR as a line ending, so such a row rendered to a human reader while being invisible to BOTH sides of the parser's symmetry invariant: no item, no shortfall, no headingsSeen. A phase hiding a result: blocked row that way reported 100 percent with zero diagnostics. Line endings are now normalized once at document ingress -- CRLF and lone CR both to newline -- at the two independent entry points, rather than teaching each split site about CR. Every downstream scan, offset and span therefore reads one convention. That single-frame property is deliberate: this issue already cost a HIGH when two scans read the same document through different frames. MY OWN END-TO-END TEST WAS WRONG and is replaced rather than weakened. It asserted that a lone-CR document must withhold its percentage, which reasons from the pre-fix symptom: after the fix the row is not hidden, it is surfaced, and this module deliberately keeps visible outstanding UAT work separate from completion percentages -- only unreadable evidence degrades scope. The success of the fix is what made the assertion false. The implementing agent refused to satisfy both it and the architecture and asked instead of bending either; it was right. What replaces it is a stronger contract: a lone-CR document and its LF twin, built from one source, must produce identical audit output -- scope, percent, every unresolved row by identity, and the diagnostic set. That is what 'a line-ending convention must not change what the audit reports' actually means, and it carries a non-vacuity check so it cannot pass with both sides empty. shortfallBlocks keeps being returned, now documented as currently unconsumed. An earlier reviewer told me audit-uat still consumed it and I passed that on as an instruction; it was wrong, and it was caught by checking rather than by me. * fix(#3707): normalize at the document read boundary, not at two call sites The lone-CR fix was half-applied and both review engines caught it independently. cmdAuditUat has four document ingresses, not the two I normalized: VERIFICATION.md and deferred-items.md still handed raw text to newline-only splitters, and the frontmatter extract in the UAT loop read raw content while its parser read normalized -- one audit entry mixing the two frames the fix exists to unify. Measured: a phase written twice from one source gave total_files 2 / total_items 4 under LF and results [] / total_items 0 under lone CR, with zero diagnostics. Normalizing two call sites and declaring it done is exactly why two were missed, so this moves it to the read boundary: every document now enters through a helper that normalizes, in audit-uat, in planning-inspect's readDocument, and in the shared verification-status read. Future parsers downstream get normalized text by construction rather than because someone remembered. That last seam also fixes an under-reporting case of the same root: a lone-CR VERIFICATION.md saying status: passed was read as missing, telling the user a verify step that had completed never ran. The parity test's load-bearing assertion is now marked as such. Four of its five equality checks still pass with the bug present -- only the unresolved-row identity differs -- so trimming that one as redundant would make the row vacuous. Second changeset added: the CR fix is user-visible independently of the fold revert, and one fragment covering both would have described neither. * test(#3707): failing-first coverage for the U+2028 and duplicate-result false-cleans Two more of the same class, both found by the security review of this branch and both reproduced before writing a line. normalizeLineEndings folds only carriage returns, but a JS /m anchor also treats U+2028 and U+2029 as line terminators while split on newline does not. That is the identical asymmetry the carriage-return bug exploited, one separator over, and worse in one respect: these are not CommonMark line endings, so a reader still sees the column-0 result: blocked that the tool discards. Measured: a scalar-internal result: pass placed after U+2028 wins over the real blocked line and the row disappears with no gap raised. Separately, and independent of any separator, a block with two column-0 result: lines resolves to the first with no ambiguity signalled. Prepending result: pass to a block therefore deletes an outstanding row silently; reversing the order surfaces it. Order deciding meaning is the defect, so the pair of rows pins the contract as ambiguity-is-a-gap rather than last-one-wins, leaving the fix room to implement the gap sensibly. Four controls: an ordinary marker in the same position (proving separator not content), legitimate U+2028 inside prose that must not be torn, a single result line, and a result line inside a fence that must not count as a second occurrence. * fix(#3707): scan result lines by split, not by a multiline anchor Two more false-cleans from the security review, both closed by the same change. A JS /m anchor treats U+2028 and U+2029 as line terminators while split on newline does not. A scalar-internal result: pass placed after one of those separators therefore matched as a line start and beat the real column-0 result: blocked, and the row vanished at 100 percent with no gap. Worse than the carriage-return case in one respect: these are not CommonMark line endings, so a reader still saw the blocked row the tool discarded. Separately, the non-global match returned the leftmost hit, so a block with two column-0 result: lines silently resolved to the first. Prepending result: pass deleted an outstanding row; reversing the order surfaced it. Order deciding meaning was the defect. Both close by scanning lines produced by split rather than by anchoring a regex inside the whole document: each line is tested on its own, and a count other than exactly one is reported as a parse gap instead of resolved to either candidate. I asked for U+2028 to be folded in normalizeLineEndings and that was wrong. Folding is length-preserving, so it would have made the U+2028 fixture byte-identical to the genuine two-result-line fixture -- while one requires a confident item and the other requires an ambiguity gap. No implementation can satisfy both once the distinguishing character is erased. The agent proved that and deviated rather than forcing it, which is why normalizeLineEndings still folds only carriage returns, now with a comment saying why. * fix(#3707): bound the ambiguity scan at the next heading-shaped line The split-based result scan regressed four pre-existing #3078/#3707 guards, each off by exactly one gap. My diagnosis was wrong. I read the off-by-one as double counting -- zero-result blocks taking both the new path and the pre-existing one -- and said to change the ambiguity condition from not-equal-one to greater-than-one. The agent checked and refused: the zero path was never duplicated. The real cause is double ATTRIBUTION. A block is sliced to the next TOKENIZED heading, so when the next row is untokenized -- hidden by a straddling fence, or indented and already counted by the shortfall scan -- that row's own result: line is absorbed into the previous block. The scan then saw two result lines across what are really two rows and raised a second, redundant gap on top of the one already counted elsewhere. Had the greater-than-one change gone in, the counts would have matched while the double attribution stayed. That is the compensating-adjustment failure I had asked it to refuse, and it did. The scan is now bounded at the first following heading-shaped line, either indent class, so a genuine same-block ambiguity is untouched while spillover from a row counted elsewhere is excluded. * fix(#3707): keep the U+2028 immunity, revert the ambiguity detection The ambiguity half of this change regressed the suite twice and is coming out. Attempt one double-attributed: a block is sliced to the next TOKENIZED heading, so when the real next row is untokenized its result: line was absorbed into the previous block and raised a second gap on a row already counted elsewhere. Four guards broke. Attempt two bounded the scan at the next heading-shaped line and broke thirty. An indented ### N. inside a block scalar is legitimate scalar CONTENT, not a heading, and truncating there defeats every #3078 guard that exists to stop scalar bodies being read as rows. Telling a genuinely hidden indented row apart from indented scalar text is a classification countUnattributedIndentedRows already owns; a raw regex does not have that information. What survives is the half that is sound and was never implicated in either regression: the result scan tests each line produced by split rather than anchoring a regex with the multiline flag over the whole block. split never treats U+2028 or U+2029 as a delimiter, so those separators can no longer manufacture a line start and steal a row. Everything else returns to first-match-wins, byte-identical to origin/next. The two tests pinning ambiguity-as-a-gap are removed with it, since the contract is no longer implemented here. The defect they described is real, pre-existing and independent of any separator -- result: pass before result: blocked silently deletes an outstanding row -- and it needs its own change with a scalar-aware counter rather than being wedged into a branch already carrying three fixes. * fix(#3707): correct the shared-seam rationale and restore U+2028 trailing text The revert left a stale rationale in core-utils, justifying the decision not to fold U+2028 by claiming uat.cts must tell a fake line start apart from a real second column-0 result: declaration that gets flagged as ambiguous. Nothing flags ambiguity any more; that behavior was reverted and the same file says so a few lines away. The decision is still right, the stated reason was false. This is the third stale comment this branch has shipped and had to fix, and the worst placed of them: core-utils is a shared leaf that every future document consumer will read for guidance. Rewritten to the true reason -- the scan tests each split line individually rather than anchoring over the block, so an exotic separator cannot manufacture a line start and folding is unnecessary. Also a real behavior delta I had not noticed. Dropping the multiline flag left the pattern's trailing .*$ in place, and dot never matches U+2028, so a genuine column-0 result: blocked whose TRAILING text contained one stopped parsing entirely -- a visible parse gap rather than a false clean, so fail-safe, but a regression against origin/next that nothing pinned. The trailing portion now matches any character and a test pins it by identity against its plain-LF twin. Plus the JSDoc orphaned when normalizeLineEndings moved to core-utils, and the changeset, which described neither the separator fix nor planning-inspect surfacing lone-CR rows. * fix(#3707): harden the acceptance gate, which had both halves of the same bug uat-predicate is a SECOND, independent UAT parser, and it is the one that decides phase uat-passed. It read raw bytes and anchored a multiline regex over unsplit text -- exactly the two defects this branch closed one module away in uat.cts. The consequence is worse than the audit surface it mirrors. Measured on identical bytes: a U+2028 scalar injection made the gate return passed true while planning inspect reported the same row as blocked and outstanding. The hardened surface and the gate disagreed, and the gate was the permissive one -- so a phase could be accepted over a row the audit could see and the gate could not. Both raw reads now go through the shared normalize seam and both scans test lines produced by split rather than anchoring over the document. First-match-wins, matching uat.cts; no ambiguity counting is reintroduced. Tests assert the AGREEMENT between the two surfaces rather than each separately, because divergence is the defect. Also finishes the same root cause one module over: phase complete's advisory pre-scan read raw bytes, so a lone-CR VERIFICATION.md lost its human_needed or gaps_found warning -- the fix verification.cts already got on this branch. And narrows the core-utils rationale I reworded last commit, which claimed consumers already avoid multiline anchors. uat.cts still has five over unsplit text. That is the fourth comment on this branch to assert something the code does not do, so it now states only what is true of core-utils itself. * fix(#3707): give structure and attribution different line frames, normalize the close audit Two more from review, and the first was a regression I introduced one commit earlier. Converting the gate's heading scan to split-then-match removed a detection origin/next had: a ### N. heading delimited by U+2028 was found by the old multiline scan and was not found after. So hardening the result scan quietly weakened the heading scan, and the gate stopped blocking on rows origin/next blocked -- the permissive direction, on the surface that decides acceptance. The insight I had missed is that the two scans need DIFFERENT frames. Heading detection is structure: there is no distinction to preserve, so it splits on newline or either exotic separator and finds a heading however it is delimited. The result scan is attribution: the newline-only frame is exactly what stops a scalar-internal result: from being read as a column-0 line, so it stays. One frame applied uniformly was the error. Second, a THIRD unnormalized parser family: the milestone-close audit read every artifact raw. A lone-CR VERIFICATION.md degraded to status unknown and was skipped, and deferred entries vanished outright -- measured as three items requiring decisions under LF and one under CR, on identical bytes. All nine scanner reads now normalize; six of them had the identical defect beyond the three review named. The acknowledge path stays deliberately raw, since it splices by byte offset, and now says so. Also pins the cross-newline result: divergence, and replaces three raw U+2028 literals in test source with escapes. A raw separator in a fixture is one formatter away from becoming an ordinary-character control that still passes -- vacuous in the only test pinning the separator fix. * fix(#3707): share one frame between the acknowledge writer and the audit reader Normalizing the audit scanners left the writer and the reader on different frames. cmdAuditAcknowledge derives its stored snapshot values from raw content -- correct for the SPLICE, which rewrites by byte offset -- but scanUatGaps and scanContextQuestions now recompute those same values from normalized content. For a lone-CR artifact the two can never match, so an acknowledgement never suppresses its item and it resurfaces on every audit: acknowledge became a silent no-op. Fail-safe in direction, since the item stays visible rather than being wrongly suppressed, but it is the writer and reader disagreeing about what a line is -- the exact class this branch exists to eliminate, and the fourth instance of it here. The derive functions now read a normalized copy while the splice keeps raw bytes and raw offsets, so both sides share one frame and the byte-offset rewrite is untouched. Round trip pinned for lone-CR and LF, with an existing LF marker asserted still recognised so the change cannot silently invalidate acknowledgements already in users' files. Also tightens an assertion that pinned this branch's own heading fix with a proxy: notStrictEqual against 'passed' also passes on 'pass', which IS a passing token, so it could not have caught a regression attributing a passing result to the recovered heading. It now pins the exact token. * chore(#3707): backfill changeset pr numbers Both fragments still carried the pr: 0 placeholder, which failed changeset-lint and docs-lint on PR 3903. The review had flagged the backfill as pending and I opened the PR without doing it. --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
6b7df61938 |
enhance(#3881): one YAML parser — vendored js-yaml replaces the hand-rolled dialect (#3888)
* docs(#3881): answer §8.1's open question and correct three wrong premises ADR-3473 §8.1 carries a blocking open question with a forcing function: it must be answered before any implementation PR for the rule opens. Answered here as (a), a string-coercing adapter, with the measurement that settles it. The sequencing note bet that §8.8's schema would make (b) tractable. Measured against merged reality it does not: only 33 of extractFrontmatter's 78 non-test call sites read STATE.md, and two of the five compensating mechanisms §8.1 lists survive real types, leaving ~31 lines across 3 call sites as the actual prize. Also corrects three claims verified false while answering it. §8.1's justifying sentence names #3349 and #3360 as defects a real parser would fix; both are already fixed on next, confirmed by executing the compiled parser rather than reading it. The guard roster calls lint-frontmatter-scalar-broad-grep.cjs an expected casualty of this rule, but it guards shell grep idioms in workflow bash fences and never touches our parser. The same roster calls lint-vendored-deps.cjs reusable as-is; it is hardcoded to re2js throughout. The last two were caught by applying the rule this amendment records -- a factual claim in this ADR is a hypothesis until the implementing phase executes it -- on its first use. Refs #3881 * docs(#3881): record that §8.1's fork is ill-posed and (a) is not implementable An adversarial pass on the Phase 4 design established by execution that extractFrontmatter is not a YAML parser but a line-oriented scanner whose output is a function of raw source text. Four spellings of the same value collapse to one js-yaml tree but produce four distinct legacy strings, one of them mangled. No adapter over a tree can choose among outputs the tree does not distinguish, so fork (a) -- keep a string-coercing adapter so the existing contract holds -- cannot be built. For any document with a non-scalar value, (a) collapses into (b); about 26 percent of frontmatter-carrying documents have one. Also records three design defects and one new attack surface, all confirmed by execution: catching a parse failure and returning {} would delete the frontmatter block on the next write at eight call sites that conflate empty with unparseable; an empty value yields null where legacy yields {}, and reconstructFrontmatter omits null-valued keys, so the shipped state template's empty progress key would vanish; the #1882 truncation probe is parseYamlRegion itself rather than a pre-parse heuristic, so it cannot both stay unchanged and survive that deletion; and FAILSAFE_SCHEMA still resolves aliases, expanding seven lines to 22.8 MB. The rule is not deferred. The measurement is the deliverable and the re-scoping is recorded as an open question with a forcing function, per section 8's own rule. Refs #3881 * test(#3881): failing-first rows for block scalars, unicode keys and the missing #3594 matrix Creates tests/feat-3594-parser-adversarial-frontmatter.test.cjs, the file the fixture README instructs contributors to register fixtures in but which never existed. Section C: table-driven ownership check over tests/fixtures/adversarial/frontmatter/ so a fixture with no matrix entry fails loudly; six existing fixtures (duplicate-keys, crlf-mixed, unclosed-block, unicode-keys-and-values, null-byte-value, huge-bounded) each get the invariant its README states. B1 blockScalarValueIsNotTheBlockIndicator: parsing commands/gsd/add-tests.md must give argument-instructions the instruction text, not the literal '|'. RED today. B2 blockScalarDoesNotInventATopLevelKey: same parse must not produce a top-level Example key scraped from inside the block body. RED today. B3 unicodeKeyRoundTripsAsIs: the 相 key in unicode-keys-and-values.md must survive parsing; today it is silently dropped. RED today. Refs #3881 * chore(#3881): vendor js-yaml and generalize the vendored-deps guard to a manifest Packaging step for ADR-3473 §8.1: makes js-yaml available to gsd-core/bin/** without promoting it out of devDependencies (promoting broke every installed tree, #3496). gsd-core/bin/lib/vendor/js-yaml.cjs is a verbatim copy of node_modules/js-yaml/dist/js-yaml.js (the self-contained UMD dist bundle, not index.js), exposing load/dump/FAILSAFE_SCHEMA/YAMLException with zero require() calls of its own. src/vendor/js-yaml.d.cts is hand-authored, not copied, because js-yaml ships no upstream .d.ts and @types/js-yaml is not installed. It is deliberately narrow, declaring only the four symbols in use, so anchors/aliases/custom types/loadAll are unreachable from typed code -- a compile-time enforcement of ADR-3473 §8.1's refusal to expand alias resolution for security reasons. Because it has no upstream counterpart it is excluded from the byte-compare. scripts/lint-vendored-deps.cjs is refactored from a script hardcoded to re2js into a table-driven VENDORED manifest (one row per package: upstream/vendored .cjs paths, optional .d.cts paths, twin kind upstream-verbatim vs hand-authored) so a second vendored package does not require a second hardcoded check block, per ADR-3473 §8.3 'one implementation per rule'. The four existing re2js checks (vendored .cjs vs node_modules, vendored .d.cts vs node_modules, src/vendor twin vs bin-side twin, devDependency version pin vs installed version) are preserved unchanged; verified pass/fail identical before and after the refactor, and the guard's ability to fail was re-proven with a deliberate one-byte append to both re2js.cjs and js-yaml.cjs, then restored. docs/INVENTORY.md and docs/INVENTORY-MANIFEST.json (via gen-inventory-manifest.cjs --write, run after build:lib) register vendor/js-yaml.cjs. gsd-core/bin/lib/vendor/README.md documents both vendored packages and the two twin kinds. Refs #3881 * feat(#3881): parse .planning frontmatter with the vendored js-yaml ADR-3473 §8.1: extractFrontmatter's read path is no longer a hand-rolled line scanner. parseYamlRegion, escapeDoubleQuoted, unescapeDoubleQuoted and parseQuotedScalar are deleted (not patched); parsing now goes through the vendored js-yaml (./vendor/js-yaml.cjs) under { schema: FAILSAFE_SCHEMA, json: true }. Everything js-yaml does not do is layered on top, in one place, carrying the seven design-doc consequences: 1. Empty value: a null js-yaml value is coerced to {} (matching legacy's own empty-value contract) so reconstructFrontmatter — which omits null-valued keys — still round-trips a bare `key:` line instead of deleting it. Verified live: progress: with no value survives parse -> reconstruct -> re-parse. 2. Unparseable no longer collapses to a bare {}: a new FRONTMATTER_UNPARSEABLE Symbol (exported), keyed exactly like the existing #3257 FULL_LINE_COMMENTS channel, is carried on the {} returned for malformed/refused YAML. Invisible to Object.keys/entries/JSON.stringify/for-in, so the 70 call sites that never inspect it are unaffected; wiring the 8 hasFrontmatter sites to consult it is a separate change, not done here. 3. Non-scalar object-list items (the four spellings of `- test: a b` that js-yaml collapses into one tree shape) are rendered as a canonical `key: value[, key2: value2]` string per item, keeping the existing array-of-strings value SHAPE. A full corpus differential over all 1702 tracked markdown files found 11 residual divergences from the legacy parser (enumerated in the PR/report), most of them the parser now being MORE correct (a dropped quoted top-level key, the block-scalar/phantom-key defect, a dropped Unicode key). 4. The #1882 truncation probe still runs the one real parser, but derives its key count from js-yaml's own thrown error and mark.line when the whole region doesn't parse cleanly (the dominant real truncation shape: fence opened, well-formed keys, no closing fence). Verified against both the clean-parse and the exception-fallback path. 5. The #3257 comment channel now attributes each pending column-0 comment against js-yaml's own parsed top-level key list (matched by literal key text, in document order) instead of the legacy ASCII-only key regex, so a comment above a Unicode key attaches correctly. 6. Anchors, aliases and merge keys are refused outright (a raw-text pre-scan, since FAILSAFE_SCHEMA still resolves them) — corpus occurrences today: zero. A 7-line billion-laughs fixture is verified refused rather than expanded. 7. A literal U+0000 is swapped for a private-use sentinel before the parse and restored in every resulting string afterward, since js-yaml rejects NUL unconditionally under every schema. escapeDoubleQuoted is deleted and reimplemented via js-yaml's dump() (forced double-quoted style), with control-char hex escapes lowercased to keep serialized output byte-stable (#1779 emitted lowercase); it keeps its exported name and signature for its two other call sites (commands.cts, runtime-artifact-conversion.cts), which need no change. frontmatterDeepEqual, the comment channel, sliceTopLevelFrontmatterSegments, regenerateFrontmatterKey's guard, noOpObjectListSetError and parseMustHavesBlock are all unchanged — retiring them is fork (b) and is not this phase. Refs #3881 * fix(#3881): quote template placeholders and preserve unparseable frontmatter SECURITY.md/UI-SPEC.md/VALIDATION.md wrote frontmatter placeholders as bare {N}/{phase-slug}/{date}, which is valid YAML flow-mapping syntax under the vendored js-yaml parser, not the literal placeholder text intended. Quote them so they parse as strings. Wire the FRONTMATTER_UNPARSEABLE Symbol (exported but unused) at the 8 call sites in state.cts/state-transition.cts that compute hasFrontmatter via Object.keys(extractFrontmatter(...)).length > 0 and reassemble the document without a frontmatter block when false. That check conflated 'no frontmatter' with 'unparseable frontmatter' (both parse to {}), so a document with a merge-conflict marker or refused alias in its frontmatter had that block silently dropped on write. Each site now preserves the exact raw bytes stripFrontmatter removed when the marker is set, leaving the genuinely-empty case unchanged. Refs #3881 * test(#3881): consequence and boundary coverage for the js-yaml migration Rows: A1 emptyValuedKeySurvivesAWrite, A2 unparseableDocumentKeepsItsFrontmatterBlock, A3 unparseableIsDistinguishableFromEmpty, A4 nonScalarValuesCanonicalize, A5 truncationProbeStillFiresOnAnOpenFence, A6 commentsStayOnTheirOwnKey, A7 anchorsAndAliasesAreRefused, A8 aliasExpansionCannotExhaustMemory, F1 UNTERMINATED_KEY_THRESHOLD boundary, F2 alias/nesting refusal bound, F3 frontmatter size boundary (huge-bounded.md + larger). Adds tests/fixtures/adversarial/frontmatter/anchor-alias-bomb.md and its entry in the feat-3594 fixture matrix. Refs #3881 * docs(#3881): document the vendored parser, correct a stale rationale, add a vendoring how-to Refs #3881 * docs(#3881): correct the frontmatter glossary entry Two errors in the entry as first written: it named parseYamlRegion as part of the read path when that function is deleted, and it recorded the eight hasFrontmatter call sites as unwired follow-on work when they were wired in e35ac2a2c. Also records the scope caveat that the CLI write path rebuilds the frontmatter block independently, so the marker binds at the transform layer. Refs #3881 * docs(#3881): record the semantic-migration decision and the counted guard ledger The maintainer chose the full semantic migration over splitting the rule into its own epic or patching the scanner, so section 8.1 is answered as "the fork was ill-posed and the migration is semantic" rather than as (a) or (b). Also replaces the pre-implementation guess that this phase would shrink the guard surface with the counted result: excluding vendored third-party lines the hand-maintained surface is net +307, and frontmatter.cts grew by 68 lines despite four functions being deleted, because the compatibility layer over js-yaml is larger than the scanner it replaced. Section 8.1's stated benefit is therefore not delivered as written; what improved is the kind of code maintained, not the amount. Decision 6 requires recording that rather than netting it away. Refs #3881 * chore(#3881): changeset for the vendored YAML parser migration Refs #3881 * test(#3881): golden parity, round-trip property and packaging coverage Refs #3881 * fix(#3881): refuse anchors structurally and fold in review findings ADR-3473 §8.1 review findings, addressed inline: Finding 1 (BLOCKER): refuseAnchorsAndAliases was a raw-line regex that matched only the bare-key spelling (key: &x). A quoted key ("a": &x), a flow mapping ({b: &x}) and a flow sequence ([&x, *x]) all define/use the SAME anchor mechanics while never matching that line shape, so the exact expansion the guard exists to stop went straight through unrefused (a 303-byte quoted-key bomb expanded to ~35.8MB). Replaced with js-yaml's own `load` `listener` callback, which reports `state.anchor` for every event belonging to an anchored node in every spelling, and throws from inside the callback to abort before any expansion (~1-2ms vs full expand-then-discard). A merge key with an alias is still refused (merge always requires a previously anchored node, so the alias itself trips the listener); a bare merge key with NO alias is no longer separately refused, documented as intentional: FAILSAFE_SCHEMA never resolves `!!merge`, so it carries no expansion risk. Table-driven tests added for all four bypass spellings + merge key, plus a quoted-key-spelled billion-laughs fixture registered in the adversarial matrix and README. Finding 2: src/vendor/js-yaml.d.cts's docblock falsely claimed anchors/ aliases were "simply UNREACHABLE from typed code" through the twin. Corrected to state the truth: anchor/alias resolution is document-level `load` mechanics reachable through exactly the declared surface, and refusal is enforced at RUNTIME (Finding 1's listener), not by the type surface. Finding 3 (MAJOR): the null-byte sentinel (U+E000) round-trip was non-injective — restoreNullBytesDeep rewrote every U+E000 in the parsed tree back to NUL, including one the document author legitimately wrote, silently corrupting it. Now refuses outright whenever the raw region already contains U+E000 (consistent with the existing anchor/merge-key refusal path), making the substitution provably injective. Tests added for a real NUL alone (preserved), a pre-existing U+E000 alone (refused, not corrupted), and both together (refused, not merged into one byte). Finding 4 (MAJOR): scripts/lint-vendored-deps.cjs's `srcTwin` field was dead for a hand-authored row (only read inside the upstream-verbatim branch) — exactly how Finding 2's stale docblock drifted unnoticed. Added checkHandAuthoredTwin: every value-level export the twin DECLARES must be an actual own property of the vendored runtime module at require-time. Tests added, including a sensor that a declared-but-nonexistent export IS caught. Finding 5: the existingFm/hasFrontmatter/stripFrontmatter/fmPrefix/ unparseableFm/reassemble preamble, copy-pasted at 7 sites in state-transition.cts plus a sixth hand-inlined copy in state.cts's cmdStateCompletePhase, is now one exported helper (beginFrontmatterReassembly) every site routes through, including the hand-inlined one. Three call sites (beginPhaseCore, patchCore, updateCore) keep a literal `body = stripFrontmatter(content)` assignment alongside the helper call so scripts/lint-state-write-path-drift.cjs's single-hop backward scan (which does not chase aliases) still sees the strip; stripFrontmatter is pure/idempotent so the extra call changes nothing observable. Finding 6: corrected the frontmatter.cts docblock's stale "wiring is a separate change" claim (the 8 call sites are wired on this branch) and the changeset's backlink from (#3473) to (#3881). Finding 7: fixed the lint:ci failures blocking the gate — an @typescript-eslint/only-throw-error violation from throwing a bare Symbol as the anchor-detected signal (now a real Error subclass), unused-var warnings left over from the Finding 5 refactor, a lint-test-file-count cap exceeded by two migration-specific test files (allowlisted with justification), and the lint-state-write-path-drift false positive from Finding 5's helper (fixed above). tests/frontmatter-golden-parity.test.cjs:117's execFileSync already carried an explicit timeout; no change was needed there. Golden fixture: added a golden entry for the new anchor-alias-bomb-quoted.md fixture ({} — matches what the legacy line scanner would also produce, since it independently dropped every quoted top-level key). No other corpus document diverges: real .planning/ documents carry zero anchors/aliases/merge keys/U+E000 today. Refs #3881 * fix(#3881): fold in second-round review findings Finding 1 (BLOCKER): tests/frontmatter.test.cjs pinned the pre-migration ASCII-only key regex for the Unicode fixture; updated to require the 相 key's value now that js-yaml has no such restriction. Audited the rest of the file for other pre-migration pins (block scalars, quoted keys, flattened values, empty values, duplicate keys, unclosed blocks, null bytes) by execution against real fixtures; found none regressed. Finding 2: parseYamlRegion and escapeDoubleQuoted renamed to parseGuardedYamlRegion and escapeDoubleQuotedScalar in src/frontmatter.cts so no function still answers to the deleted hand-rolled scanner's name (ADR-3473 §8.1 "deleted, not patched"). escapeDoubleQuotedScalar's three external call sites (src/commands.cts, src/runtime-artifact-conversion.cts) updated in the same change — a mechanical rename, not an ADR-amendment matter. Finding 3 (BLOCKER): fixed a real crash and a silent data-loss bug found by execution. A top-level key named constructor/__proto__/toString/ valueOf/hasOwnProperty crashed reconstructFrontmatter (bracket read resolving an inherited Object.prototype member); a key literally named __proto__ was silently DROPPED entirely (bracket assignment on an ordinary {} invoked the inherited __proto__ setter instead of creating a data property). Fixed by building every parsed Frontmatter object with Object.create(null), and replacing an `in` check with hasOwnProperty.call in propagateCommentChannel. Added round-trip tests for all five hostile keys, each with its own leading comment. Finding 4 (MAJOR): escapeDoubleQuotedScalar's docstring falsely claimed full byte-stability across the migration. Verified by execution: BEL/NUL/ NEL/NBSP/LS/PS/BOM now emit YAML-named escapes instead of the old hex/raw- literal forms. Proved round-trip equivalence (each escape re-parses to the exact source codepoint) and corrected the docstring. Found and fixed a related real defect while verifying: a lone UTF-16 surrogate was emitted BARE (scalarNeedsDoubleQuoting didn't trigger), producing genuinely unparseable YAML that silently collapsed to {} on re-read — extended scalarNeedsDoubleQuoting to route surrogates through the quoted+escaped path. Finding 5 (MAJOR): countKeysBeforeTruncation went silent on 4 real truncation shapes (unquoted colon, open flow collection, mis-indented sibling key, refused anchor). Root cause: the mark-based prefix recovery excluded the very line whose key needed counting, and a mark-less refusal never entered the recovery branch at all. Fixed by taking the max of two lower bounds: the longest parser-verified line-prefix, and a raw-text count of key-shaped lines (reusing the same key-shape pattern this file already uses for isFrontmatterShaped). Extended test-matrix row A5 table-driven over all 4 regressed shapes. Finding 6: the design doc's claim that no test owned the #3594 adversarial fixture corpus was false — consolidation epic #1969 had already folded it into tests/frontmatter.test.cjs. An earlier commit on this branch re-created a standalone duplicate under that false premise; folded its genuinely-new coverage (fixture-ownership check, anchor-bomb fixtures, block-scalar B1/B2 rows) into frontmatter.test.cjs and deleted the duplicate file. Corrected the false claims in 40-design.md §3.3.1 and the ADR's §8.1 note, including the roadmap-sibling claim (no such file exists). Finding 7: the golden serializer sorted object keys, making it structurally blind to the key-order-parity invariant ADR-3473 §8.1 actually claims. Made it order-preserving and regenerated the golden fixture from a standalone compile of the legacy (pre-#3881) parser at ddde001af; the current parser matches it with zero undocumented divergences, confirming key-order parity genuinely holds. Extended row A2 table-driven across 6 of the remaining 7 transitionCore kinds (all pass) plus documented, by execution, a newly-discovered 8th-site regression: state.cts's cmdStateCompletePhase calls the same preservation helper but its result is clobbered by a later unconditional resync — filed as a distinct finding rather than fixed here (touches syncAndPreserveStateMd, outside this change's verified scope). Refs #3881 * fix(#3881): preserve unparseable frontmatter through the CLI write path Characterization (executed, before/after shown): case (b), not (a). The frontmatter FENCE survives — `state complete-phase` on a conflict-marked STATE.md returns success and a well-formed, freshly-derived frontmatter block, not a document with no frontmatter at all. But the block's actual content (the merge-conflict markers, and with them any signal to a human that the document was in conflict) is silently discarded and replaced. Root cause was two clobber sites, not one: 1. syncStateFrontmatter (src/state.cts) re-parses the already-preserved `transformedContent` from readModifyWriteStateMd, finds {} + the FRONTMATTER_UNPARSEABLE marker, and unconditionally rebuilt a fresh frontmatter block from the body anyway. 2. Even after (1) is fixed, applyPostSyncPreservation's own postFm/applyStatePreservation/authoritativeFm-reassertion machinery re-extracts frontmatter from syncedContent, restores curated fields from the pre-write snapshot, and reconstructs a NEW block again — confirmed live via `state begin-phase`, which still lost the markers after fixing (1) alone. Both are now guarded by the same predicate (isUnparseableFrontmatter, checking FRONTMATTER_UNPARSEABLE): when the ORIGINAL frontmatter did not parse and the caller is not on ADR-3408 §8.3's closed "body wins" list, both functions return their input content unchanged rather than re-deriving over it. The closed list (cmdStateSync #905, /gsd-health --repair's REGENERATE_STATE, both routed only through writeStateMd, which never reaches applyPostSyncPreservation and passes sanctionedPermanentEmptyFallback=true to syncStateFrontmatter) is untouched — neither widened nor narrowed; verified by execution that `state sync` still overwrites the conflict-marked block exactly as before. Other verbs sharing the same readModifyWriteStateMd path were checked and were equally affected before this fix: state update, query state.patch, and state begin-phase all lost the conflict markers (RED, shown by execution), and all three now preserve them (GREEN). Covered table-driven in tests/feat-3881-yaml-parser-consequences.test.cjs's new A2b describe block, which drives the real CLI verbs via runGsdTools — not just the pure transitionCore layer the earlier A2 rows exercised — plus a control asserting state sync's body-wins contract is unchanged. Refs #3881 * fix(#3881): restore the parse surface's prototype and fix remote-runner failures Root cause of the bulk of the 88 remote-runner failures: extractFrontmatter/parseGuardedYamlRegion handed back Object.create(null) trees for prototype-pollution safety, but assert.deepStrictEqual compares prototypes, so every assertion against a plain object literal failed (57 frontmatter.unit.test.cjs + 5 frontmatter.test.cjs + others). Fixed by keeping the internal construction null-prototype (unchanged) and converting to a plain-prototype tree via Object.defineProperty (never bracket assignment, so __proto__/constructor/toString keys stay safe) at the parseGuardedYamlRegion/unparseableResult return boundary only; the internal FULL_LINE_COMMENTS Symbol channel is copied by reference, not recursed, so its own __proto__-safety is untouched. Per-class fixes: (1) bomAcrossArtifactTypes was the same prototype bug, no separate code change needed. (2) frontmatter-cli #1660: added objectListFieldWouldLoseData, a broader lossy-field detector alongside the existing byte-identical noOpObjectListSetError -- js-yaml's flattenObjectListItem now correctly includes every sub-key of an object-list item (a real bug fix over the legacy scanner, which silently dropped every field but the first), so a set that drops that now-included data is no longer byte-identical to the original and needs its own guard. (3) uat.test.cjs: updated the pinned expectation for the human_verification quote-stripping artifact -- js-yaml resolves quoting correctly where the legacy regex left an unbalanced quote; documented as an intentional, non-lossy behavior change. (4) smart-entry: added a fallback-only loadWithAmbiguousColonRepair so a column-0 key: value line whose value itself contains an unquoted colon (the #2571 hand-edited-STATE.md shape) round-trips instead of failing the whole frontmatter block closed. (5) frontmatter.unit.test.cjs bracket-array leniency: added a second fallback, repairMalformedInlineArrays, restoring the legacy scanner's tolerant inline-array handling (consecutive/blank commas, unclosed bracket) -- both repairs run ONLY after the primary parse already threw, so well-formed documents are unaffected. (6) prompt-injection-scan: src/frontmatter.cts had a literal U+FEFF BOM embedded in a comment illustrating the #2977 fix; replaced with the U+FEFF text escape. (7) eslint-glob-coverage: allowlisted the new src/vendor/js-yaml.d.cts vendored type declaration, same precedent as the existing re2js.d.cts entry. (8) frontmatter-golden-parity: git ls-files *.md now runs with -c safe.directory=* (process-scoped) so it survives the remote runner's dubious-ownership check without a persistent git config write. Refs #3881 * chore(#3881): backfill changeset PR number Refs #3881 * test(#3881): make golden parity resistant to unrelated tree churn A corpus-wide snapshot keyed to every tracked *.md file was coupled to mutable-by-design files: .changeset/*.md's pr:0 -> real-PR-number backfill is a required workflow step, not a parser change, yet it turned this suite red. Training people to 'just regenerate the golden' on that kind of failure defeats the point of the snapshot. Exclude .changeset/** from the golden corpus entirely, tolerate tracked *.md files with no golden entry (they postdate the capture) instead of failing on them, keep hard failures for a golden entry whose file has vanished from the tree and for any real parity divergence, and add a coverage floor so the enumeration cannot quietly degrade to comparing a handful of files. Golden regenerated by recompiling the legacy pre-migration parser (git show ddde001af:src/frontmatter.cts) standalone, independent of the current parser, over the same non-changeset corpus. Refs #3881 * test(#3881): make the parser golden hermetic instead of tree-keyed This repo merges ~21 commits/day; a 14-day sample measured 937 touches of the exact files (commands/gsd/*.md, gsd-core/workflows/*.md, agents/*.md, docs/*.md) the prior golden pinned by tracked path. Any PR editing one of those files' frontmatter for reasons unrelated to the parser (an argument-hint addition, an allowed-tools tweak) turned the suite red, and the reflex fix -- "regenerate the golden" -- overwrote the very snapshot meant to catch a real regression. Excluding .changeset/** was not enough; the design itself was wrong: a regression fixture must not be keyed to mutable repo paths, and a single 376-entry JSON every such PR touches is also a guaranteed merge-conflict surface. Rebuilt the fixture to carry its own documents: each of 51 entries stores a stable id, literal documentText (shrunk from a real ddde001af-era corpus document), and an expectedParse captured independently from the pre-migration legacy parser (git show ddde001af:src/frontmatter.cts, compiled standalone against its byte-identical sibling modules). The test reads no tracked path, shells out to no git command, and enumerates no tree -- a PR editing commands/gsd/help.md cannot affect it. Every entry's reconstruction was verified at capture time to reproduce both the current and legacy parser's output on the original document; 0 of 51 candidates were dropped by that check (1, the deliberately-unterminated unclosed-block.md adversarial fixture, has no closing fence to truncate at and is stored unshrunk). Kept the 5 documented DIVERGENCES rows (now diverges:true entries) and the D2 order-preserving structural serializer that keeps the comparison from passing vacuously; dropped the tree-enumeration helpers, the coverage floor, the post-capture-skip logic, and the vanished-file check -- all artifacts of the path-keyed design. Refs #3881 * fix(#3881): resolve vendored-deps paths independently of cwd shape Five rows in tests/lint-vendored-deps-manifest.test.cjs failed on windows-latest CI: the test passed absolute scratch-file paths into compareFiles()/checkRow(), whose helpers joined every input onto ROOT via path.join(ROOT, rel), producing garbage when the input was already absolute. It surfaced on windows-latest specifically because GitHub's Windows runners checkout the repo on a different drive than TEMP, so path.relative(REPO_ROOT, tmpFile) returned the absolute path unchanged (no relative traversal is representable across drives) rather than the relative form the test assumed. The remote gsd-test runner this repo gates pushes on is Linux-only and could never have caught this; GitHub CI's windows-latest job is the only signal that does, and it did. Fixed the helper itself (scripts/lint-vendored-deps.cjs's new resolvePath()) to treat an already-absolute input as absolute-in, absolute-out instead of silently mis-joining it, and updated the test to pass the scratch file's absolute path directly rather than relying on a relative conversion that is not always representable. Kept every mutation-sensor assertion intact and added coverage proving resolvePath is a no-op for relative inputs and correctly passes absolute ones through unchanged. Refs #3881 * fix(#3881): warn when state sync regenerates over unparseable frontmatter state sync (ADR-3408 §8.3's sanctioned regenerate path) correctly overwrites an unparseable frontmatter block per its 'body wins' contract — that overwrite behavior is unchanged here. The defect was the silence: synced:true/exit 0 gave no signal that the existing block (including git merge-conflict markers) could not be parsed and was destroyed, per ADR-3473 §8.5 ('a derived conclusion may not be reported as authoritative when the derivation dropped input it could not resolve') and §8.4 ('failure is a value'). Adds a gsd: warning — ... (#3881) line on stderr, matching the existing #3573 precedent, and surfaces the same disclosure in the JSON result's existing changes[] array so a machine consumer sees it too. Exit code and synced:true are left unchanged — sync did what its contract says. REGENERATE_STATE (/gsd-health --repair's sibling on the same sanctioned-regenerate list) is DESTRUCTIVE-risk and unconditionally refused by applyRepairs's dispatcher before runRepairAction ever runs (src/health-diagnostic.cts), so it is not a live path today and is not in scope for this fix. Refs #3881 * fix(#3881): exit non-zero when a state command returns an error Refs #3881 * chore(#3881): changeset for the state exit-code fix Refs #3881 * fix(#3881): honor the documented --project-dir flag Refs #3881 * revert(#3881): restore exit-0 result envelopes for state errors Reverts 9638f2936 and its changeset. The change was wrong and the revert is the correction. This repo distinguishes two error mechanisms deliberately. error() in src/io.cts writes to stderr and calls process.exit(1) -- the hard-failure path. output({error: ...}) writes a JSON result envelope to stdout and returns normally with exit 0. The reverted commit converted 23 result-envelope sites into hard failures, which is a different contract, not a bug fix. tests/state-contract.test.cjs's errorPathDoesNotPublish asserts the envelope contract directly -- a failing command exits 0 with a JSON error envelope and must not publish state.json -- and the remote matrix run caught it along with four cases in the QA scenario walk. Thirteen tests in tests/state.test.cjs that the original commit rewrote were encoding that real contract, not the bug it claimed; they are restored. Whether an error envelope on stdout with exit 0 is the right CLI design is a genuine question, and it is section 8.4's rule ('failure is a value') with its own phase. It is not something to flip inside this PR. Refs #3881 * chore(#3881): backfill changeset PR number for the project-dir fix Refs #3881 * test(#3881): keep the frontmatter mutation shard inside its time budget The Stryker (frontmatter) shard hit the documented 15-minute (900s) shard cap. Root cause is NOT row-level spawn overhead (contrast the #2790/ core-utils precedent): the three shard test files' own logic runs in ~413ms total (356+30+27ms) with all 392 assertions passing. Instead, src/frontmatter.cts grew from ~825 to 1496 lines (+671/-187) migrating to the vendored YAML parser, proportionally growing the mutant count Stryker generates for gsd-core/bin/lib/frontmatter.cjs. Stryker's command runner bills the full 'node --test <3 files>' invocation once per mutant, and node:test's default per-file process isolation forks a child process for each of the three files on every one of those invocations — pure fork overhead multiplied by a much larger mutant population. Fix: scripts/mutation-matrix.cjs COVERED.frontmatter now declares isolation: 'none', and .github/workflows/mutation.yml passes --test-isolation=${{ matrix.isolation }} (defaulting to 'process' — i.e. unchanged behavior — for the other 8 shards, which were not individually audited for cross-file state leakage under shared-process execution). Measured locally via node:test's run() API on the exact 3-file set: isolation:'process' took ~593ms vs isolation:'none' ~478ms for the same 392 passing assertions. The true CI-shard number can only be confirmed on the GitHub Actions run (Stryker cannot run locally, and 'node --test' is hard-blocked in this environment). Refs #3881 * test(#3881): register the vendored-parser tests in the frontmatter mutation shard stryker.config.mjs's own rule ("Keep this list in sync with the tests arrays in scripts/mutation-matrix.cjs COVERED") was violated: #3881 grew src/frontmatter.cts from ~825 to 1496 lines but its new tests (tests/feat-3881-yaml-parser-consequences.test.cjs, tests/frontmatter-golden-parity.test.cjs, tests/frontmatter-roundtrip.property.test.cjs, and +167 lines in tests/frontmatter.test.cjs) were never added to the frontmatter shard's tests array, so Stryker's mutants in the new vendored-js-yaml adapter had nothing constraining them. PR #3888 measured 55.8% against the 65 floor (748 killed / 593 survived / 17 timeout) and the shard was separately cancelled at 15m04s against the 15-minute per-shard cap. Registers all four files (each earns its slot on evidence of a unique constraining assertion, documented inline), gives the shard a measured/projected 180-minute budget via a new per-module timeoutMinutes field threaded through mutation.yml's job-level timeout-minutes the same way isolation is threaded, and removes the prior isolation:'none' override (re-measured at this file-set size, its savings are within run-to-run noise, not worth the unaudited cross-file-state-leakage risk). Refs #3881 * feat(#3881): derive the mutation test list and ratchet the score floor Refs #3881 * test(#3881): ratchet five stale mutation floors and close the frontmatter gap Raised five module minScore floors per CI run 33012034388 (floor(achieved)-1): config-schema 75.51%->74, prompt-budget 88.95%->87, context-composer 79.92%->78, context-utilization 92.31%->91, active-workstream-store 87.42%->86. Updated both scripts/mutation-matrix.cjs COVERED entries and tests/mutation-matrix-ratchet.test.cjs RATCHET_BASELINE in the same diff per the ratchet's own contract. Closed the frontmatter shard's 63.03%-vs-65 gap with new behavioral tests in tests/feat-3881-yaml-parser-consequences.test.cjs, each paired with a documented near-miss: frontmatterDeepEqual's array-order/length/type-mismatch/key-order semantics (via spliceFrontmatter's no-op guard), scalarNeedsDoubleQuoting's leading/trailing-whitespace and dash/surrogate triggers (via reconstructFrontmatter), repairAmbiguousColonValues' already-quoted vs ambiguous-colon repair paths (via extractFrontmatter), and the null-byte sentinel round-trip surviving at region offset 1. Did not lower minScore. Refs #3881 * test(#3881): decouple the ratchet test from real module floors The CLI end-to-end rows in tests/mutation-score-ratchet.test.cjs hardcoded config-schema's real floor (52), which commit 973321541 legitimately ratcheted to 74 -- breaking a test pinned to the exact value the mechanism under test exists to change. Add an injectable --matrix seam to scripts/check-mutation-score-ratchet.cjs and point the CLI rows at a synthetic module + synthetic floor built via a temp fixture, so the rows are indifferent to any real module's floor moving while still exercising the same fail/pass behaviour. Refs #3881 * refactor(#3881): parse must_haves with the vendored parser and drop re-implemented leniency Refs #3881 * fix(#3881): restore the ambiguous-colon repair its hand-edited-STATE.md contract needs A tracked-document sweep of 910 *.md files cannot see this dependent: repairAmbiguousColonValues's one real caller is user hand-edited STATE.md content that never lives in this repo's tree, only on end users' machines, and is pinned by tests/smart-entry.unit.test.cjs. Restores the function plus its post-throw fallback path (loadWithAmbiguousColonRepair) only; repairMalformedInlineArrays and splitLegacyInlineArrayItems stay deleted, reverified against the full frontmatter test shard. Adds a frontmatter-level regression row in tests/feat-3881-yaml-parser-consequences.test.cjs so the dependency is visible where the function lives. Closes #2571 Refs #3881 --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
832dcbb751 |
fix(#3707): surface UAT rows audit-uat silently dropped, and never report a clean result for a file it could not read (#3887)
* test(#3707): failing-first coverage for the three parseUatItems false negatives Nine tests that must be red and three controls that must already be green. The controls are the point of the split. `result: pass` staying unsurfaced is what stops the fix inverting the filter so eagerly that every passing test becomes an outstanding item, and the classic single-line shape is the no-churn control for rewriting the adjacency regex. Both were confirmed green against the current build before being written down; a control that is red today would be a second bug, not a control. Each failing fixture was run through the built parser first and returns [] for its stated cause — the issue row matched then filtered, the block-scalar and wrapped rows never matched at all, the all-unparseable file vanishing whole. That evidence is in 50-test-matrix.md rather than asserted. Tests target ../gsd-core/bin/lib/uat.cjs, the built live module, and drive the real CLI through runGsdTools. #3706 lost a full RED/GREEN cycle to tests that imported a different copy of the function under test, so the import target was verified before anything was written. * fix(#3707): stop parseUatItems dropping outstanding UAT rows Three independent false negatives, all in the audit path, plus one the issue did not mention. The matcher no longer requires `expected:` and `result:` to be adjacent single lines. It slices each `### N.` block to the next heading and reads the first `result:` line within it, taking `expected:` from parseExpectedFromTestBlock — the seam that already parsed both the block-scalar and inline forms correctly and was sitting unused two hundred lines away. Two parsers in one module read the same field with different grammars; now there is one. The result filter is inverted from an inclusion list of three to an exclusion of a minimal PASS set. This was the issue's one open design question, which the reporter explicitly declined to answer for the maintainer; it was asked and decided deliberately. The fail-safe direction is what parseGapsItems documents seventy lines below for this same false-negative class (#2286): a token nobody recognised surfaces rather than vanishing. The trade is a visible, correctable false positive if a project invents a novel pass-word, against today's silent and invisible drop. `issue` also needed a category. It is template-sanctioned with its own `issues:` counter, but categorizeItem fell through to `unknown` — surfacing it in the wrong bucket would have been a half-fix. Finally, a file parsing to zero items no longer vanishes with its frontmatter `status:`. One with a non-terminal status is reported with `parse_gap: true`, so the reader gets a cue to look; a `complete` one stays omitted as before. That is what made the first two defects dangerous rather than merely lossy — the audit omitted the phase instead of under-counting it. * fix(#3707): close the review blockers, including a regression I introduced The remote suite was RED on the previous commit and both reviews found real defects. Everything below was verified by execution, not by reading. I introduced a regression against origin/next. The rewritten result matcher was END-anchored where the old one was not, so `result: pending (blocked on staging)`, `result: [skipped] # no device` and `result: blocked - waiting` all returned a row before this branch and returned nothing on it — me reproducing the exact defect class this issue exists to kill, in the fix for it. The anchor is gone and each shape has a regression test; trailing text now falls back to `reason` when the block has none. `parse_gap` was inferred from the wrong signal. It fired for ANY zero-item file whose status was not `complete`, which asserted something false about a perfectly-parsed all-pass file, swept in archived phases left at `testing`, and is what turned the #2286 Gaps tests red — a control this change was supposed to keep green. It now derives from headings SEEN BUT UNYIELDED, reported by a new parseUatItemsWithStats, so an all-pass file and a Gaps-only file are not parse gaps and a file whose blocks carry no `result:` line is. The fix was also invisible end to end, which both reviewers caught independently. parse_gap entries carry no items, and both audit-uat.md and progress.md gate on `total_items === 0` — so the headline symptom, the phase vanishing, still reproduced for a user and only the raw JSON had changed. There is now a `parse_gap_files` counter and both workflows gate and report on it. Also: categorizeItem compared case-sensitively while the new PASS check lowercased, so `result: PENDING` surfaced as `unknown`; blocks are bounded at the next heading of any level, so a trailing `## Gaps` entry no longer bleeds its `reason` onto the preceding test; dead unreachable fallbacks removed; and the all-pass control was strengthened, since asserting only `total_items === 0` let it stay green through the bogus parse_gap entry. * chore(#3707): acknowledge the workflow growth the fix required The emitted-attribution guard went red because audit-uat.md and progress.md grew, and it is right to ask: runtime-loaded workflow prose is the product, so growth there is a real change to what an executing agent reads. The growth is not incidental to this fix, it IS the fix reaching a user. Both reviewers found independently that emitting `parse_gap` in the JSON changed nothing observable, because both workflows gated their output on `total_items === 0` and parse-gap entries carry no items — so a phase whose rows could not be parsed still printed "All Clear" and still vanished from the progress report. The widened gates and the branches that name the unparsed files with their phase and path are what close that. Acks exactly the two paths the guard reported, keyed on the bare filename. The three spent acknowledgments it also listed are inert by its own description — the base already absorbs them — so they are left alone rather than swept up here, where they would just add unrelated churn to this diff. * fix(#3707): close the mixed-file blocker and the second false-clean surface The suite was GREEN and the isolated review still found a blocker, which is the useful part: none of this was covered by a test. A MIXED file dropped its unparseable rows silently. `parse_gap` sat behind an `else if` on `items.length > 0`, so one parseable row was enough to discard `headingsSeen` entirely — a file with one pending row and two unreadable blocks reported one item and no gap. That is the exact class this issue exists to kill, reappearing inside its own fix for the third time. The flag is now set independently of item count and the entry carries `unparsed_blocks`, so the count is quantified rather than merely flagged. A `result:` inside a fenced code block was being read as real, so a PASSING test could be reported as outstanding from a value in a code sample — another regression against origin/next, whose adjacency regex ignored it. Field scans now run against a fence-stripped copy while `expected:` still reads the raw block, since a block scalar may legitimately contain fenced-looking text. The workflow report was still unreachable whenever anything else was outstanding: the unparsed table lived in the all-clear branch, so a project with one pending row in phase 01 and an unreadable phase 02 rendered phase 02 nowhere. It now fires on `parse_gap_files > 0` from the `present` step. planning-inspect was the second surface making a false-clean claim — for exactly the files audit-uat now flags it emitted `scope: 'complete'` with an empty unresolved list, positively asserting completeness over a file it could not read. It consumes the stats now and reports SCOPE.TRUNCATED with a `uat_unreadable` diagnostic, reusing the vocabulary already used two lines above for an unreadable file rather than inventing a token. Also: headings with no name no longer vanish whole; the trailing-text-to-reason synthesis I had added is removed, since it was never required by the blocker and silently changed categorization for `result: [skipped] # no device`; the emitted `result` token is normalized to lower case so it agrees with `category` in a published contract; and an O(n^2) indexOf is gone from the heading loop. * fix(#3707): stop rows stealing each other's fields, on all three surfaces The suite was green when the security review found these. Two are blocker-severity and one of them is a direct hit on my own verification. A `### N.` line indented two spaces inside an `expected: |` scalar is a valid ATX heading, so it became a phantom row that STOLE the real row's result token while the real row vanished. I had probed this shape and declared it fixed — my probe asserted the item COUNT and the result token, both of which the phantom satisfied, so it passed for exactly the reason it should have failed. Block scalar bodies are now masked to blank lines (line count preserved, so offsets still line up) before headings are tokenized, and the tests assert row IDENTITY — number and name — not presence. Feeding parseExpectedFromTestBlock the raw slice let one row publish another row's `expected:` from inside a fence the stripped view had correctly excluded. Blocks handed to it are now clipped at the first fence opener. This was not cosmetic on the render-checkpoint path: a checkpoint banner a HUMAN reads and answers was rendering a different row's expected text. A balanced fence pair straddling a test block made that heading invisible, so an outstanding row disappeared with no item, no gap and no count — the exact false-clean this issue exists to close, and a regression against origin/next. Suppressed `### N.` lines now count toward headingsSeen so the file is flagged. An unterminated fence swallowed the rest of the document including `## Gaps`, producing a whole-file false clean. Such a file is now treated as a parse gap, following what uat-predicate already does. Found and fixed inline while there: parseExpectedFromTestBlock's scalar opener required a bare newline, so a CRLF `expected: |` fell through to the inline arm and published `expected: "|"`, silently discarding the entire value. The same fall-through hit `|-` and `|+`. parseFirstPendingTest had the identical exposure on the render-checkpoint path and now shares the same masking and clipping. Five legitimate fixtures — inline expected, a real block scalar, CRLF, bracketed pending, and a first-pending that is not the first test — are byte-identical before and after. Also from the code review: the admit condition disagreed with the terminal status guard, so a `status: complete` file with an unparseable block was emitted as an empty entry that rendered nowhere but inflated total_files; a control test was vacuous because its fixture filename did not match its phase dir, so #3511 scoping meant the file was never opened — and that vacuity is why the admit regression shipped green; an unterminated fence discarded the flag that would have caught it; `### 1.2.3` parsed as test 1; and planning-inspect did not share the terminal-status rule. * test(#3707): assert what the render-checkpoint fix actually does The suite went red on three of my own tests and the source was right — the assertions were wrong, in a way worth naming. One forbade the rendered checkpoint from containing `### 3. Fake Row`. But in that fixture the string IS row 1's legitimate `expected:` block-scalar value; a heading-shaped line inside a scalar is inert text and rendering it is correct. The test was forbidding correct output. It now asserts row IDENTITY — the checkpoint is for test 1 named Alpha and never test 3 named Fake Row — which is the property that actually distinguishes the fix from the bug. The other expected success where the correct outcome is a clean error: row 1 in that fixture has no `expected:` of its own and only ever appeared to have one by stealing row 2's from inside a fence. Depending on the bug to produce a pass is how a test ends up pinning the defect. The fixture now gives row 1 its own value and asserts the checkpoint carries it and never the fence-hidden text, and the error path gets its own test asserting it fails cleanly without leaking. All three were checked against the real rendered output before the assertion was written, and each was reasoned through for whether it can fail: the identity test breaks if a phantom row is parsed, the clipping test breaks if the raw block is read again, and the error test would pass-not-fail under the old stealing behavior. * fix(#3707): correct the scalar masking frame and cover every YAML block opener Two reviews independently found the same blocker, and it is the sharpest defect on this issue: maskBlockScalarBodies computed line offsets in UTF-16 units but spliced them into Array.from(content), a CODE POINT array. One emoji anywhere earlier in the file shifted every later mask write, so the mask blanked the wrong characters and spilled past line ends. Measured: at two astral characters a result token truncated `pending` to `pendi` and recategorized to unknown; at six the real row vanished; at twelve the FOLLOWING row's `result: blocked` disappeared and the file reported clean. That is the false-clean class this issue exists to close, reintroduced by the mitigation written to prevent it, and defeating both new detectors at once. The mask is rebuilt line by line now, which is frame-agnostic and length-preserving by construction. The opener grammar was also incomplete. YAML block scalar headers take an optional indentation indicator and an optional chomping indicator in either order, so `|2`, `|2-`, `|-2`, `>2`, `>2+` are all valid — and none were matched. An unmasked `expected: |2` body meant a `### N.` line inside the value became a real heading: reproduced, row 1 disappeared and a fabricated row 2 named "Phantom" took its identity. Fixing that exposed a third instance of the same family, found by my own probe rather than by review: the value extractor understood only the `|` openers, so every `>` folded scalar published the LITERAL OPENER as its value — `expected` came back as ">" or ">2+" and the whole scalar was discarded. The extractor now shares the opener grammar and implements real folding, joining paragraph lines with a space and turning a blank line into a newline, rather than pretending `>` means `|`. Also from the reviews: the shortfall counter scanned the masked copy but not a fence-stripped one, so a `### N.`-shaped line inside a properly closed documentation fence — the ordinary way to document the row format inside a UAT file — counted as a suppressed row and flagged the file against nothing; and clipping at the first fence discarded a legitimate `expected:` that appeared after a closed fence, which is silent field loss. Every opener now verified for both row identity and exact extracted value, in LF and CRLF, alongside the emoji fixtures at 1/2/6/12. * refactor(#3707): replace the scalar masking with a column-0 heading rule The fix had grown to five helpers whose only job was undoing one over-permissive rule: tokenizeHeadings treats a heading indented up to three spaces as real, so a `### N.` inside an `expected: |` body was parsed as a row and stole the real row's identity. Every blocker in the last three review rounds came out of that machinery rather than the reported bug — worst of all a UTF-16-versus-code-point frame mismatch that corrupted any document containing an emoji. A UAT test heading is at column 0. The shipped template puts all of them there, no `*UAT*.md` in the repo has an indented one, and the only indented `### N.` lines in the tree are the adversarial fixtures that must not parse. Requiring column 0 makes a scalar-interior heading a non-heading by construction, so maskBlockScalarBodies, indentWidthOf and BLOCK_SCALAR_OPENER_RE are gone along with the mask-invariant test that existed only to guard them. The frame bug is now structurally unreachable: no code-point array or offset splicing remains. The premise was incomplete and the reviewer caught it rather than forcing it through. Masking had been doing double duty — it also hid indented FENCE delimiters from the tokenizer, so removing it let a two-space fence inside a scalar body swallow a later column-0 row. The alternative on offer was to rewrite that test to assert the row is merely counted, which is a behavior regression dressed as a passing suite. Instead there is a small line-based pass that blanks only indented fence delimiters — same "column 0 is structure" rule extended consistently, no YAML knowledge, and line-based by construction so the frame bug cannot come back. It was proven load-bearing by a negative control: reverting the wiring reproduces the regression exactly. Kept, because they fix defects column-0 does not touch: the fence clipping that stops one row reading another's `expected:` from inside a fence, and the folded scalar handling that stopped `expected: >` publishing the literal ">". Also corrects a comment left pointing at a symbol this commit deletes. * fix(#3707): blank neutralized fence bodies, and give both parse paths one grammar Reviewing the simplification found two more, and the first is row theft again — the sixth time this class has surfaced on this issue, and the second time inside a mitigation written to stop it. Neutralizing an indented fence blanked only its DELIMITER lines. If the block's body held a column-0 `### N.`, un-hiding the delimiters made that line a real heading, which then took the preceding row's fields: an `### 1. Alpha` document came back as a single row 9 named Phantom, with Alpha gone. Neutralized blocks are now blanked open-to-close, body included, which is the honest reading of the intent — an indented fence inside a scalar is content, so nothing in it should be able to produce structure. The raw block is still what the expected extractor reads, so a legitimate `expected: |` carrying a fenced code sample keeps its full text. The two parse paths also disagreed about what a test row IS. parseFirstPendingTest filtered on `^\d+\.\s+` while parseUatItemsWithStats used `^\d+\.(?!\d)`, so `### 3.Foo` was a row when audited and not a row when resumed. Both now share one predicate and one extractor. The extractor mattered as much as the filter: the checkpoint path's name-mandatory pattern would have skipped exactly the shapes the widened filter admits, so fixing the filter alone would have moved the divergence down a line rather than closing it. Also from the security pass: parseUatItems had become an export with no callers and no direct test once both consumers moved to the stats form. It stays, since deleting an exported symbol from a shipped module is a contract change and not this issue's business, but it is now documented as the items-only wrapper and has a test. And the PASS check lowercased a value that extraction had already lowercased; normalization now happens once. * fix(#3707): revert the whole-block fence blanking, and pin the rule instead My previous commit over-corrected and the suite caught it. Blanking a neutralized fence block open-to-close destroys content legitimately living between the delimiters, and on an UNTERMINATED opener it blanks to EOF and deletes every later row. No framing makes that correct, and it is what turned two earlier tests red. The "blocker" that prompted it was my own misreading. This change adopted the rule that column 0 is structure and indentation is content. Under that rule an indented fence delimiter is not a fence, so a column-0 `### 9.` sitting between two indented delimiters genuinely IS a heading, and a `result:` after it genuinely belongs to it. That document is malformed and the parser reading it that way is consistent, not stealing. Nothing is silently lost either: the row whose result was taken surfaces as the parse gap. So the blanking is back to delimiters only, and rather than leaving the question open, the behavior is now pinned by a test asserting the rows by identity, with the rule stated at the site — so the next person does not oscillate the way I just did. Kept from the reverted commit: the shared row-heading grammar and extractor across both parse paths, the parseUatItems wrapper documentation and its test, and the single point of lowercasing. * fix(#3707): scope the shortfall scan, and stop neutralized content becoming structure The security review found a HIGH that is the earlier frame-mismatch bug wearing different clothes. The shortfall scan compared a SECTION-scoped raw line count — the `## Tests` body — against a DOCUMENT-wide token count. So a single legal `### N.` row anywhere outside `## Tests` decremented the shortfall and switched the fence-straddle detector off: an identical `## Tests` section went from `headingsSeen 1, parse_gap true` to `headingsSeen 0, no gap` purely because a `## Prior` section existed. A document whose rows render as ordinary blocked rows in any CommonMark renderer audited as totally clean. Both sides of the comparison now come from the same surface, by filtering tokens to the scan span rather than re-tokenizing, so there is no second offset basis to keep in step. Neutralizing a fence could also promote its former CONTENT into structure: a column-0 delimiter run inside an indented pair became an opener once the enclosing delimiters were blanked, hiding every later heading to EOF. Column-0 delimiter-shaped lines inside a neutralized block are now blanked too — and only those, so a column-0 heading between neutralized delimiters is still a heading (the pinned behaviour) and the field lines of a row living between two scalars still survive. The reviewer corrected my repro while fixing it: an even number of inner runs re-pairs and hides nothing, so the live shape needs an odd one, and both are now tests. Two more from that pass. A legal scalar header carrying a trailing comment (`expected: | # sample`) failed the end-anchored grammar, publishing the literal header and raising a false gap. And the indented-row counter walked backwards per row: 3.6 seconds at sixteen thousand rows, now 15ms, via one forward pass — though the reviewer also established my example was not the quadratic shape, which needs an uninterrupted scalar body. Carried in from the previous round: the indented-row counter keys on any block scalar rather than only `expected:`, so a template-sanctioned `reported: |` holding user prose with a heading-shaped line no longer raises a false gap; and `reason:`/`blocked_by:` read block scalars through the same shared extractor instead of publishing the literal `"|"`, which also means a multi-line reason can finally reach categorizeItem — a `reason:` mentioning a server now categorizes as server_blocked, which was impossible while the value was thrown away. * fix(#3707): make the shortfall scan whole-document on both sides Second HIGH in this area, and the diagnosis is the useful part: I closed the first one by making the two sides agree, but I did it by NARROWING the token side to the `## Tests` span while the parse side stayed whole-document. Rows outside that section are still parsed and surfaced when visible, so when a fence straddled one it fell through both sides of the comparison — no item, no gap, file never entered the results at all. A `## Regression Tests` section, or a second `## Tests` (collectSection takes the first), audited as totally clean while origin/next surfaced those rows. Both sides are whole-document now. Symmetry is the property that matters here; every attempt to be clever about which scope to compare has produced one of these, twice at HIGH severity. That reinstates a known over-report, deliberately: a `### N.`-shaped line inside a closed fence in a `## Notes` section — the ordinary way to document the row format — counts as a suppressed row and raises a gap on a file with nothing missing. Noisy, but visible and fail-safe, against two silent false-cleans on the other side of the trade. This issue exists to eliminate false cleans, so the trade goes that way, and the reasoning is written at the site so it does not get optimized back. Three existing tests encoded the retired scoping and are replaced rather than worked around: two now assert the accepted over-report, and one asserting a 4-space row is "not counted" was already contradicted by widening the counter to any indentation — refusing to PARSE a 4-space heading is right, refusing to COUNT it reopened the hole the counter exists to close. Also in this commit, from the same review round: the inner-delimiter sweep tested a column-0-anchored pattern, so an INDENTED delimiter inside a neutralized block was still promoted to structure and lost a row; it is indent-tolerant now. A refinement was identified and deliberately not taken — keying the documentation-sample exemption on the fence info string rather than on section scope. It is content-based and symmetric, so it would not reintroduce the asymmetry, but it belongs in its own change rather than riding this one. * docs(#3707): correct two claims in the over-report justification Both from review, both comment-only, and both matter because they would mislead the next person into "fixing" something correct. The over-report note called the triggering shape "the ordinary way to document the row format". It is narrower than that: the scan requires literal digits, so the conventional placeholder `### N. Name` does not trigger it at all — only a sample written with real numbers does, and no phase UAT file in-tree has one, only the shipped template, which selectPhaseUatFiles never scans. A maintainer who tested the documented placeholder form would find no over-report and could reasonably conclude the pin was stale. That is now stated, and it also makes the trade look better than I claimed: the real-world frequency is lower. The attribution guard is described as structural rather than positional. It is positional in one respect: the walk stops at the nearest column-0 line, so a block scalar nested inside a `## Gaps` bullet is transparent to it and a heading-shaped line in that value gets counted. Same accepted over-report, reached by a path the comment did not mention — recorded so it is not later mistaken for a new defect. * fix(#3707): a complete status no longer switches off the parse-gap detector The security review named this as the last silent-clean path in the change, and its phrase is the right one: a self-declared kill switch over the very detector this issue built. A file whose frontmatter said `status: complete` was omitted unconditionally, so one containing a fence-straddled `result: blocked` computed headingsSeen = 1 — the detector fired — and then emitted no entry at all. The audit reported nothing. The predicate is now status-independent: a file is surfaced when blocks were seen but yielded nothing, whatever it claims about itself. A terminal status is an assertion by the author, and an assertion is exactly what must not be allowed to suppress the signal that would contradict it. What does not change is the thing the status is actually for — a complete file with nothing to parse, and a complete file whose rows all parse and all pass, both stay silent, verified through the real CLI. I replaced a control test of mine, and it is worth saying why that is not a weakening: its name was already false. "A zero-item file with a complete status is still omitted" used a fixture with a `### 1.` block carrying no `result:` line, so headingsSeen was 1 — it was never a zero-item file, it was the kill switch itself, pinned. The intent it claimed is now covered by two stricter tests, one for a file with no blocks and one for a file where every row parses and passes, each asserting both that no entry exists and that no items are counted, where the old test asserted only the former. Everything else is byte-identical: 61 regression cases and the non-complete equivalents of all four shapes produce exactly the same output as before, with the delta confined to the two cells this change is meant to move. * fix(#3707): close the moved kill switch, and keep the archive out of the live gate Both reviewers independently found that closing the kill switch on one surface left it standing on the other. cmdAuditUat dropped the terminal-status guard, but buildUatRows in planning-inspect kept it — and its comment justified that by claiming to mirror a guard cmdAuditUat no longer had. One byte-identical file with `status: complete` and a fence-straddled `result: blocked` reported parse_gap through audit-uat while planning-inspect published `uat.scope: "complete"` with no diagnostic at all. That is the repo's own generative-fix-divergence class, and no test pinned that arm, which is why it survived. The clause and the false comment are gone and the arm now has tests. Removing it exposed a MAJOR the security pass had not reached: archived phase dirs are deliberately not milestone-filtered and archived UAT files are `complete` by definition, so status-independence newly admitted the entire project archive. One live pending row plus four signed-off milestones produced parse_gap_files 4 — and since progress.md gates Verification Debt on that counter, a mature project would have warned on every run, forever, about closed history no user action can clear. Warning fatigue that buries the next real gap is the feature defeating itself. So the counter is split rather than suppressed: `parse_gap_files` counts live phases only and remains the gate, `archived_parse_gap_files` carries the rest, and every archived entry stays in `results` with its parse_gap and its milestone. Nothing became silent; the live signal stayed actionable. Both workflows report the archived bucket as closed history rather than as something to act on. The scope cascade is also decoupled, on the security reviewer's advice that it is load-bearing here rather than a follow-up: `uat.scope` still reports TRUNCATED honestly so no completeness is claimed over an unread row, while the accepted fence-shortfall over-report no longer flips the aggregate fold that withholds a phase's percentage. A genuinely unreadable file degrades as before. Also corrects a frequency claim of mine: "no phase UAT file in-tree triggers this" was true over a sample of zero, since the only UAT file in the tree is the shipped template. The comment now says the shape is uncommon, which is what I can actually support. * fix(#3707): state that the live/archived split does not extend to outstanding_debt Review MINOR: the split's rationale read as though it governed every counter, but `summary.total_items` was never split — so a single archived `result: pending` row re-trips the same Verification Debt warning the split exists to stop. The asymmetry is deliberate: an archived parse gap is a row nobody can read, so the warning can never be cleared, whereas an archived pending row is legible work someone can still pay down by retesting. Debt that can be settled stays counted. The prose now says so at the point of the claim, instead of leaving the next reader to file it as a miss. Also rewrites the changeset, which described only the secondary fixes and omitted all three defects the issue actually reports: `result: issue` dropped, any wrapped or block-scalar `expected:` never matched at all, and the phase vanishing outright. * fix(#3707): count every parse gap, dropping the live/archived split The split had two regressions, both reproduced through the real CLI, and its premise was false. uat.cts carries #2766's rationale ~190 lines above the code I added: 'Outstanding UAT items do not stop mattering when a milestone closes: a deferred human-UAT scenario or a skipped live-stack test is exactly what gets archived still-open.' So 'archived UAT files are complete by definition' was never true, and the split rested on it. Regression 1: archived-ness was inferred from path shape alone. A phase in the CURRENT milestone, status in_progress, filed under .planning/milestones/v1.1-phases/ was classified archived and demoted out of the gate — live work reported as closed history that needs no action. Regression 2: the split was one-sided. total_items has no archived split, so an archived outstanding row that PARSES gates Verification Debt while the identical row that fails to parse was informational. The parse failure was what buried the debt — the exact bug class this issue exists to fix, re-created one surface over. parse_gap_files counts every parse_gap entry again, archived or not, so it agrees with total_items on what archived means. The pre-existing archived_milestone field and archived-phase scanning are untouched. Regression tests added for both cases. * fix(#3707): correct the changeset clause left behind by the split revert The changeset was rewritten before the split was removed, so its final clause still claimed archived parse gaps are counted separately because signed-off history is not work anyone can act on. There is one counter now, and that premise is the one uat.cts refutes and the revert was made over. This text lands in CHANGELOG.md verbatim, so it would have shipped a description of behavior the code does not have. * chore(#3707): backfill changeset pr number --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
cf15682d1c |
enhance(#3028): responsive Markdown separators instead of fixed-width rules (#3789)
* feat(#3028): responsive Markdown separators instead of fixed-width rules Stage banners, checkpoints, completion and error panels used fixed-width runs of box-drawing characters -- a 53-column heavy rule and a 62-column double-line box. Those runs are ordinary text to a Markdown-rendering host, so in a narrower pane they wrap and the border comes apart from the heading it framed. Shipped content now emits an ATX heading for a titled section and a blank-line-delimited --- for a break between sections, both of which adapt to the available width. The same convention is applied to the three code sites that built these strings at runtime: the UAT checkpoint renderer, the milestone-close audit report, and the TDD review checkpoint table. Removing the box also removes its only reason to exist -- the east-asian-width padding helpers that kept its right border aligned (checkpointBoxLine, displayWidth, isWideCodePoint, ZERO_WIDTH_MARK_RE, CHECKPOINT_BOX_WIDTH). RTL directional isolation is unchanged. The convention is specified in gsd-core/references/ui-brand.md and enforced across all shipped content by tests/responsive-separators.test.cjs. Refs #3028 * test(#3028): pin the heading form in checkpoint and audit-report assertions These suites asserted the exact box borders and the 62-column padded banner interior. With the box gone they assert the ### heading form, the --- break and the bolded instruction line, and each now carries a positive assertion that no box character remains -- which is what pins the fix rather than merely tolerating it. Language coverage is converted, not dropped: Japanese, Chinese, Korean, Hindi and Arabic all still assert their rendered banner, and the Arabic case still asserts the RTL directional isolates the box removal must not disturb. Adds a case for a banner longer than the old inner width, which previously produced a ragged border and now has none. Refs #3028 * chore(#3028): acknowledge execute-plan.md growth from the checkpoint display spec The checkpoint_protocol display spec described the drawn box; it now describes the heading, the --- break and the bolded action prompt, which costs 22 bytes (40111 -> 40133, 827 under the cap). Appended to the existing #3370 fragment rather than filed as a new one: a growth ack keys on the bare filename and #3370 already declares execute-plan.md, so a second source naming it would be a hard duplicate-key error. Same supersede-by-append route #3370 took for the spent #2652 fragment. Refs #3028 * docs(#3028): state the load-bearing half of the separator rule, and amend the zh-CN reference Review found three things. The rule as first written demanded a blank line above AND below every ---. Only the one above is load-bearing: it is what stops CommonMark reading the rule as a setext underline for the line above. The one below is cosmetic, because a thematic break is a leaf block. The rule now says that, with the reason, instead of asserting a stricter form the content does not keep. The zh-CN reference had received the mechanical box-to-heading swap but none of the prose behind it: it still claimed a 62-character checkpoint width and still listed --- among forbidden mixed banner styles, so it contradicted the convention it was translating. It now carries the separator section, the setext reasoning, the unconditional-vs-per-runtime rationale and a corrected anti-pattern list, in Chinese. The user guide asserted that a heading is not a degradation anywhere. That is an assertion, not a demonstration. It now says what was actually traded away in a plain terminal, points at the recorded rationale, and invites the report that would justify the capability flag instead. Refs #3028 * chore(#3028): backfill changeset PR number Refs #3028 --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
59e7a677fe | fix(#3511): scope every phase-directory scan to the phase it belongs to (#3535) | ||
|
Tom Boucher
|
08940c9071 |
fix(#3457): split deferred items on leaf headings, not bullets (#3488)
* fix(#3457): split deferred items on leaf headings, not bullets * fix(#3457): backfill changeset pr with 3488 --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
9341d8b8d3 |
test(#3334): fold the workflow-dispatch & review-lane fix-* cluster — Wave 2 (#3342)
Folds 15 tests/fix-*.test.cjs regression files (191 test() blocks) into their module's main suite, per the wave decomposition of #3315 (H3 of epic #3053). 187 blocks land in 8 existing suites (4 exact-duplicate cases dropped, documented inline); 4 blocks move via git mv into 2 new suite files with no prior coverage to merge into. Zero production behavior change. Also tightens two H1 (#3313) ratchets that the fold's own file-count reduction moved past their grace window, per the ratchets' documented dual failure mode (a stale/too-loose baseline fails exactly like a novel violation): - lint-test-file-count.allowlist.json: removes the stale "audit" entry (folding fix-2766 into tests/uat.test.cjs drops that module back to its 2-file cap). - lint-allow-test-rule-refs.ceiling.json: lowers maxFiles 314 -> 309, the real post-fold high-water mark (gsd-test's own repo-baseline test caught this — CI, not a human, found it). Two orthogonal review passes (Standards+Spec code-review, isolated security-review) found and this commit fixes two issues before push: a genuinely-distinct #2287 test case (file-absent vs. file-present- resolved) that a prior fold pass had wrongly dropped as a duplicate — restored verbatim into tests/uat.test.cjs; and a missing same-line allow-test-rule citation on the #2196 block in tests/debug-session-management.test.cjs, added for consistency with its sibling #2257 block. lint-removed-but-needed also caught two stale doc references to the now-folded-away fix-2285-claude-orchestration-wiring.test.cjs filename (docs/adr/1143-claude-orchestration-capability.md, gsd-core/references/execute-phase-response-language.md) — updated both to point at tests/claude-orchestration.test.cjs, its new home. Co-authored-by: sim <sim@local> |
||
|
JusticeWay
|
7b204ad2ac |
enhance(#2530): extend UAT checkpoint frame language pack (9 more languages) (#2564)
* feat: extend UAT checkpoint frame language pack (9 more languages) response_language is a free-form config value, but CHECKPOINT_FRAMES only covered 9 languages — any other configured language silently fell back to the English frame. Add Dutch, Polish, Russian, Ukrainian, Turkish, Hindi, Arabic, Vietnamese, and Indonesian frames plus their aliases, with a regression test asserting each resolves instead of falling back. Follow-up to #2402 (PR #2457). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: add changeset for #2527 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(#2530): list UAT checkpoint frame languages in CONFIGURATION.md Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(#2530): point changeset fragment at PR #2557 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(#2530): address Unicode language-pack review * fix: address checkpoint language review * fix: count spacing combining marks in checkpoint width * test: verify checkpoint aliases structurally * fix: isolate RTL checkpoint frames * fix: isolate RTL checkpoint frames correctly * test(#2530): assert checkpoint aliases neither collide nor go unreachable Review Minor #1. A duplicate alias key was invisible to the existing catalog tests: the runtime object is well-formed after JS collapses the literal, the self-alias assertion still holds, and the losing language just stops resolving. tsc catches the byte-equal case (TS1117), but not the two that survive compilation — an alias whose NFC-lowercase form already belongs to another language, and an alias not in lookup form at all, which resolveCheckpointFrame() can never produce. The check reads the source literal rather than the object, since the object no longer records what was written. Both assertions are independently load-bearing: an NFD twin of an existing alias trips the collision check, an uppercase alias trips the unreachability check. Review Minor #2: changeset retyped Changed -> Added. Nine wholly new supported response_language values are an addition under Keep a Changelog, not a modification of existing behavior. * test(#2530): check alias collisions on the catalog, not its source --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Tom Boucher <trekkie@nomorestars.com> Co-authored-by: Rezolv <dave@sienkowski.com> |
||
|
Tom Boucher
|
b6e6a22fce |
fix(#2402): honor response_language across orchestrator output + UAT checkpoint renderer (#2457)
* fix(#2402): honor response_language across orchestrator output + UAT checkpoint renderer Replays the in-flight bot branch fix/2402-response-language-orchestrator-coverage (seven commits, never pushed) onto current origin/next as a single squashed commit. The original work was substantial and correct; this commit preserves its full scope, trimmed where rebase conflicts + workflow size budgets required it. Three independent layers where response_language was being dropped are closed: Layer 1 — orchestrator-facing directives across workflows. Adds the strong "All user-facing output in this workflow MUST be presented in {response_language}; technical terms, code, paths, and subagent prompts stay in English" directive to ~40 workflows that previously either lacked it entirely (verify-work, new-project, new-milestone, quick, manager, and ~35 more) or carried only the weak subagent-prompt-only form (plan-phase, execute-phase). The directive covers narration between tool calls and banner output, not just the AskUserQuestion prompts. Layer 2 — UAT checkpoint renderer (src/uat.cts). buildCheckpoint now accepts an optional responseLanguage parameter and renders the frame strings ("CHECKPOINT: Verification Required", "Type `pass` or describe what's wrong.") in any of 9 languages (English/Spanish/French/German/Portuguese/Japanese/ Chinese/Korean/Italian) with an alias table covering ~30 input variants (en, es, español, ja, 日本語, etc.). cmdRenderCheckpoint reads config.response_language via loadConfig(cwd) and passes it through, so the byte-for-byte block verify-work.md reprints verbatim is already localized when written — preserving the anti-injection hygiene rule at verify-work.md (the model is forbidden to translate after the fact). CJK display width is computed by East Asian Width property ranges (W/F) so the right ║ border of the banner stays aligned for full-width characters. English fallback is byte-identical to the pre-fix behavior when response_language is unset or unrecognized. Layer 3 — literal English report templates in execute-phase. The top-of- workflow directive covers all template sites (templates are a structural source, not literal output). Inline render-language notes that previously sat at each template site were removed during the squash because they pushed execute-phase.md over its frozen pre-phase-6 byte ceiling (93600 — ADR-857 Phase 6 capstone). The single top directive covers the same surface with fewer bytes. Also extends src/docs.cts and src/init.cts to propagate response_language into the init JSON bundle of the additional workflows so the directive can read it. Tests added: - tests/uat.test.cjs: buildCheckpoint with unset/unrecognized language falls back to English default; recognized language swaps only the two frame strings while structural lines stay untouched; CJK display-width regression (independent recomputation of East Asian Width W/F ranges). - tests/workspace.test.cjs, tests/docs-update.test.cjs: response_language wiring through docs.cts/init.cts. References: #2402; reporter's three-layer triage + Layer-4 follow-up; the byte-for-byte anti-injection hygiene rule at verify-work.md (the reason Layer 2 must be renderer-side, not model-translated). This is a squash of the in-flight bot branch — seven commits representing the original implementation plus its subsequent fix/CJK-padding/test/ changeset/regen cycles, none of which were ever pushed or PR'd. The squash captures the final coherent state. * chore(#2402): backfill pr:2457 in .changeset/2402-response-language-orchestrator-coverage.md * chore(#2402): regen golden + size baseline after rebase against #2315 (PR #2451) Rebase conflicts were entirely in generated artifacts (golden-install-parity fixtures + workflow-size-baseline.json). After taking theirs during rebase, regenerated cleanly against the merged source tree. |
||
|
Tom Boucher
|
636316f720 |
fix(#2286): audit-uat surfaces Gaps section + frontmatter/heading verification items (#2317)
parseUatItems only scanned '### N.' expected/result blocks and parseVerificationItems only recognized table/bullet/numbered shapes, so audit-uat returned a false-clean total_items:0 when a file recorded open findings in a '## Gaps' section, declared items in a frontmatter human_verification: array, or used the '### N. <label>'+bold-paragraph verification shape. parseUatItems now also scans '## Gaps' (via collectSection + iterateBullets) and surfaces any entry whose status != resolved. parseVerificationItems now treats the frontmatter human_verification: array (via extractFrontmatter) as the primary source when present, and adds a tokenizeHeadings fallback for the '### N.'+bold-paragraph shape, preserving the existing table/bullet/numbered recognition (no double-count, no regression). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Tom Boucher
|
284dc7bc44 | fix: resume UAT checkpoint from paused placeholder (#1350) | ||
|
Tom Boucher
|
9f79cdc40a |
fix(security): neutralize spaced+closing injection markers; fix audit-uat resolved status (#2456)
* fix(security): neutralize spaced+closing injection markers; fix audit-uat resolved status scanForInjection recognizes — adds <user> tags, whitespace-padded tags (e.g. <user >), closing [/SYSTEM]/[/INST] markers, and closing <</SYS>> markers. Five new regression tests confirm each gap is closed. whose result column reads PASS or resolved, so items that were already confirmed do not appear as outstanding in audit-uat --raw. Two new regression tests cover item-level PASS and file-level status: passed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: add closing-tag assertion for spaced <user > sanitization The test for 'neutralizes spaced tags like <user >' only asserted that the opening token '<user' was removed. A spaced closing tag '</user >' could survive sanitization undetected. Added assert.ok(!result.includes('</user')) to the same test block so both sides of the tag are verified. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Tom Boucher
|
09e471188d |
fix(uat): accept bracketed result values and fix decimal phase renumber padding (#2283)
- uat.cjs: change result capture from \w+ to \[?(\w+)\]? so result: [pending], [blocked], [skipped] are parsed correctly (Closes #2273) - phase.cjs: capture zero-padded prefix in renameDecimalPhases so renamed dirs preserve original format (e.g. 06.3-slug → 06.2-slug, not 6.2-slug) - tests/uat.test.cjs: add regression test for bracketed result values (#2273) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Tom Boucher
|
2703422be8 |
refactor(tests): standardize to node:assert/strict and t.after() per CONTRIBUTING.md (#1675)
* refactor(tests): standardize to node:assert/strict and t.after() per CONTRIBUTING.md
- Replace require('node:assert') with require('node:assert/strict') across
all 73 test files to enforce strict equality (no type coercion)
- Replace try/finally cleanup blocks with t.after() hooks in core.test.cjs
and hooks-opt-in.test.cjs per the test lifecycle standards
- Utility functions in codex-config and security-scan retain try/finally
as that is appropriate for per-function resource guards, not lifecycle hooks
Closes #1674
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* perf(tests): add --test-concurrency=4 to test runner for parallel file execution
Node.js --test-concurrency controls how many test files run as parallel child
processes. Set to 4 by default, configurable via TEST_CONCURRENCY env var.
Fixes tests at a known level rather than inheriting os.availableParallelism()
which varies across CI environments.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(security): allowlist verify.test.cjs in prompt-injection scanner
tests/verify.test.cjs uses <human>...</human> as GSD phase task-type
XML (meaning "a human should verify this step"), which matches the
scanner's fake-message-boundary pattern for LLM APIs. This is a
false positive — add it to the allowlist alongside the other test files
that legitimately contain injection-adjacent patterns.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
Tom Boucher
|
e03a9edd44 |
fix: replace invalid \Z regex anchor and remove redundant pattern
The original PR (#1337) used \Z in a JavaScript regex, which is a Perl/Python/Ruby anchor — JavaScript interprets it as a literal match for the character 'Z', silently truncating expected text containing that letter. Replace with a two-pass approach: try next-key lookahead first, fall back to greedy match to end-of-string. Also remove the redundant `to=all:` pattern in sanitizeForDisplay() since it is a subset of the existing `to=[^:\s]+:` pattern. Add regression tests proving the Z-truncation bug and verifying expected blocks at end-of-section parse correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
3metaJun
|
aaaa8e96fe | Harden verify-work checkpoint rendering | ||
|
jecanore
|
60a76ae06e |
feat: add verification debt tracking and /gsd:audit-uat command
Prevent silent loss of UAT/verification items when projects advance. Surfaces outstanding items across all prior phases so nothing is forgotten. New command: - /gsd:audit-uat — cross-phase audit with categorized report and test plan New capabilities: - Cross-phase health check in /gsd:progress (Step 1.6) - status: partial for incomplete UAT sessions - result: blocked with blocked_by tag for dependency-gated tests - human_needed items persisted as trackable HUMAN-UAT.md files - Phase completion and transition warnings for verification debt Files: 4 new, 14 modified (9 feature + 5 docs) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |