fix(#3850): surface gaps_found VERIFICATION files in audit-uat (#3879)

* fix(#3850): surface gaps_found VERIFICATION files in audit-uat

cmdAuditUat admits `human_needed` OR `gaps_found`, but
parseVerificationItems had a body only for the first and returned an
empty array for the second — standing on a comment deferring to
`plan-phase --gaps`, a different command audit-uat never reaches. Since
cmdAuditUat pushes a file into `results` only when `items.length > 0`, a
`gaps_found` report did not under-report: it vanished, taking its
phase's `by_phase` row with it, so a clean-looking total gave the reader
no cue anything was skipped.

Eligibility now has one owner (the caller) and parseVerificationItems
reports what the file says.

The closed-entry filter could not be built on extractFrontmatter: its
array-item parser keeps only each `- ` entry's FIRST line and has no
notion of nested key/value objects, so an entry's `status:`/
`resolution:` siblings never reach its output and a closed entry is
indistinguishable from an open one downstream. Rather than grow a
competing object-list parser — or change extractFrontmatter, whose blast
radius is every frontmatter consumer in the repo — this reads the raw
segment BEFORE the flattening, via the existing anchored
sliceTopLevelFrontmatterSegments, and hands it to the `## Gaps`
machinery that already parses exactly this `- `-opened, indentation-
continued shape.

The human_needed path is byte-for-byte unchanged: same reader, same
display names, same numbering, no resolved-entry filtering — pinned by
a test and verified by identical CLI output on base and head.
parseGapsItems keeps its narrower `status: resolved` rule so no
*-UAT.md behaviour moves.

Closes #3850

* chore(#3850): backfill changeset pr number for #3879

* fix(#3850): one parse per entry, one fence parser, one resolved-entry rule

Adversarial review on #3879: B1, B2, M3, m5, m8 and n9.

B1 — `sliceFrontmatterArrayEntries` hand-rolled a second frontmatter fence
regex, which re-asserted the byte-0 rule #2977 removed: a BOM'd file (PowerShell
5.1 `>`/`Out-File` writes one by default) sliced nothing, so a `gaps_found`
report vanished from the audit exactly as it did before this fix — this issue's
own symptom, on a platform the repo already has a named defect class for.
`extractFrontmatter`'s BOM+fence logic is now factored out as
`frontmatterRegion` and shared. One fence parser, not two.

B2 — the resolved-entry skip paired two DIFFERENT parsers by array index:
`parseYamlRegion` is indent-blind, `splitGapsEntries` is indent-anchored. A
block sequence written at its key's indent — ordinary, legal YAML — makes them
disagree about entry count, and from the first disagreement every index names a
different entry, so an OPEN entry inherits a CLOSED one's resolution and is
silently dropped. That is the defect this PR exists to fix, reintroduced inside
the fix. Display name and sibling fields now come from ONE parse of the raw
slice; `frontmatterEntryDisplayName` applies `parseQuotedScalar` exactly as
`parseYamlRegion` does, so the string is byte-identical to what
`extractFrontmatter` produced. The flattened array remains the #2286 GATE, but
is no longer the source of items. `sliceFrontmatterArrayEntries` also takes the
LAST duplicate key, matching `parseYamlRegion`'s last-wins assignment.

M3 — `frontmatterEntryToUatItem` is the single entry->UatItem mapper both
readers use, rather than two copies differing only in `result`.

m8 — closed entries are skipped on BOTH statuses. The earlier asymmetry cited an
acceptance criterion #3850 does not contain: the issue has no AC section, and
its suggested fix (2) states the skip unconditionally, naming a file with 14 of
16 entries resolved. That file is `human_needed`, so the asymmetry left the
reporter's own scenario over-reporting by 14.

m5 — `sliceTopLevelFrontmatterSegments`' contract doc names both consumers and
says the column-0 boundary rule is now a cross-module contract.

n9 — the vestigial bare block is gone and its body de-indented.

Tests: the B1 BOM case, B2's nested-sequence and bare-bullet repros, a CRLF
fixture (M4 — it survived by accident, now pinned) and the unified skip rule.
Fail-first verified by running the new tests against the pre-fix build: the BOM,
nested-sequence and unified-skip cases are red there.

* fix(#3850): read the entries as objects, not as re-parsed display text

Rebased onto `next`, which changed the ground this fix stood on. ADR-3473 §8.1
(#3881) replaced the hand-rolled frontmatter scanner with the vendored js-yaml:
`parseQuotedScalar` and `parseYamlRegion` no longer exist, and an object entry
now flattens to `test: A, resolution: R` rather than to its first line.

The original mechanism existed ONLY to work around that lossy first-line
flattening — it sliced the raw frontmatter segment and re-parsed each entry by
hand so a `resolution:` sibling was visible at all. With a real parser upstream
that workaround is obsolete, so it is deleted rather than repaired:
`sliceFrontmatterArrayEntries`, `frontmatterEntryDisplayName`, the
`splitGapsEntries`/`extractGapEntryFields` reuse and the second fence regex are
all gone.

`frontmatter.cts` instead exposes `frontmatterObjectListEntries(content, key)` —
the same parse `extractFrontmatter` runs (same BOM strip, same byte-0 fence,
same anchor/alias and sentinel guards, same ambiguous-colon repair), stopping
one step before the display flattening. `flattenObjectListItem` is exposed
alongside it so a caller deriving a display name produces the byte-identical
string `extractFrontmatter` would have.

That collapses the review's blockers into properties of the parse rather than
things this fix has to get right:

- B1 (BOM) — shares `extractFrontmatter`'s strip; verified through the CLI.
- B2 (index pairing) — there is no second reader. Display name and sibling
  fields come from one object.
- M3 (duplicate mapper) — one `frontmatterEntryToUatItem` for both readers.
- M4 (CRLF) — js-yaml's, not ours; verified through the CLI.

Also confirmed on the rebased base, per review: #3850 still reproduces on `next`
after #3707 landed (`total_files: 0`, `total_items: 0` on a `gaps_found`
fixture), so this PR is still doing work #3707 did not do. Nothing was dropped
as redundant.

One behaviour note: `entryField` returns a present value verbatim and treats
only whitespace-only as absent. Trimming would rewrite an author's `truth:` on
its way to becoming the display name.

* fix(#3850): keep every frontmatter list entry at its own row

Review round 3's Blocker. `frontmatterObjectListEntries` filtered its result
to objects, and filtering COMPACTS: `parseHumanVerificationItems` then
numbered the survivors by their position in the compacted array. On a list
mixing object and non-object entries the non-object rows disappeared outright
and the rest were renumbered — #3850's own vanishing-row defect, reached
through entry SHAPE instead of file STATUS. Base never had it: it walked the
display array, so every row surfaced at its own position.

Renamed to `frontmatterListEntries` and it no longer filters (the name now
matches what it returns). Deciding what a non-object entry MEANS is a
caller's judgement; dropping it is nobody's.

Both readers now walk the DISPLAY array — one element per row, the array
#2286 already gates on — and consult the parsed array only for "does this
entry carry a closure field?". `parsedEntriesFor` owns that pairing and
checks the two lengths agree before trusting an index; all-null is the
correct degradation, since over-reporting a closed row is recoverable and
closing the wrong one is not. Names stay byte-identical to base for every
entry shape, including a nested sequence (`[nested]`, not `["nested"]`).

Same class closed in the gaps reader: a non-object `gaps:` entry surfaced
nothing at all and now surfaces as `unknown`, which is this module's
documented fail-safe direction (`parseGapsItems`) on a false-negative bug.

Also restores the shared fence parser round 2 accepted. The ADR-3473 rebase
dropped `frontmatterRegion` and left the BOM strip and byte-0 fence rule
inlined twice; `extractFrontmatter` now routes through it, so "one fence
parser" is enforced rather than asserted in a comment.

Minors: `frontmatterEntryToUatItem`'s dead `forcedResult` option deleted and
its "shared by both readers" comment corrected — it has one call site, and
the two readers differ deliberately, each mirroring its own established
sibling (`parseGapsItems` vs #2286). Documented at the divergence.

Tests: `B2` asserted a name substring, so it passed while the row was
mis-numbered and would have passed through outright loss; it now asserts
positions and count. B2b pins the reviewer's 6-entry mixed fixture verbatim,
B2c the survivors' file positions across skipped rows, B2d the gaps reader.
All four fail-first against the reviewed head; 332/332 green with the fix.

* fix(#3850): make status authoritative, and let the two gaps readers agree

Round 4 review, all five findings.

Major. `isFrontmatterEntryResolved` treated a non-empty `resolution:` as
closure regardless of `status:`, so `status: failed` + `resolution:
"attempted retry, still failing"` vanished from the report — the
silently-vanishing-item defect #3850 exists to close, reached by field
combination instead of file status.

Closure is now per key, because the two keys have different conventions
and one rule cannot serve both:

  `gaps:`               `status: resolved` only, byte-identical to the
                        rule `parseGapsItems` applies to a `## Gaps`
                        markdown section, so one authored entry cannot
                        read closed in one reader and open in the other.
  `human_verification:` a bare `resolution:` still closes, since that is
                        how verifier-written entries record it — but a
                        readable `status:` that contradicts it wins.

A single unified rule was the first draft and is wrong: it closes a
frontmatter `gaps:` entry carrying `resolution:` and no `status:`, which
`parseGapsItems` surfaces, and `parseVerificationGapsItems`' own docstring
claims it mirrors that reader's fail-safe status handling.

The contradiction guard is not a judgment call about YAML. It is the rule
this codebase already applies to the same field pair: `validateResolution`
(probe-core.cts) rejects a populated `resolution:` on a non-resolved status
outright — "a populated payload is an authoring mistake ... Reject it so
the mistake surfaces." A reporter cannot throw, so it surfaces the item.

Minor 1. Direct unit tests for `frontmatterListEntries` and
`flattenObjectListItem` in `tests/frontmatter.unit.test.cjs`, the file that
historically co-changes with `frontmatter.cts`. They were reachable only
through `uat.cts`' readers before.

Minor 2. `parsedEntriesFor`'s degrade-to-all-null branch is asserted
directly. Verified unreachable through content rather than assumed: both
readers enter through `frontmatterRegion`, `extractFrontmatter`'s only
extra argument gates a warning, and `normalizeParsedValue`'s `value.map`
is 1:1. It is a drift alarm for a future edit to either parser, so the
helper is exported for tests rather than left as the one unpinned branch.

Minor 3. The vestigial `const skipResolved = true` and its dead
conditional are gone.

Minor 4. `frontmatterEntryToUatItem` no longer reads `test:`. A `gaps:`
entry has no `test:` in its vocabulary — the template's entries carry
truth/status/reason/artifacts/missing — so it was speculative support for
a field the shape does not have, and it collided with the 1..N row numbers
`parseHumanVerificationItems` assigns by array position. Not reading it
makes the collision impossible; an offset would have rewritten an authored
value, against `entryField`'s verbatim contract.

Docs, changeset and the dispatcher docstring all stated the unconditional
rule and are corrected — three prior rounds here were comment/code drift.

Fail-first proven: restoring the universal rule reddens all three new unit
tests and both rewritten properties.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H3eK225hgcnEDZsnmtaP1U

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Tom Boucher <trekkie@nomorestars.com>

src/frontmatter.cts

@@ -665,47 +665,138 @@ function countTopLevelKeyShapedLines(region: string): number {
  *   key its deduplication. Optional because this function has 50-odd call sites and several
  *   hold only an in-memory string; those dedup on a content digest instead.
  */
-function extractFrontmatter(content: string, sourcePath?: string): Frontmatter {
+/**
+ * The frontmatter REGION of a document: the text between the opening `---`
+ * fence at byte 0 and its closing fence.
+ *
+ * One fence parser, not two (#3850 review round 2, B1). `extractFrontmatter`
+ * and `frontmatterListEntries` need the identical answer to "where does the
+ * frontmatter start and stop" — same BOM tolerance (#2977), same byte-0-only
+ * fence rule, same CR handling before the closing fence — and differ only in
+ * what they do with the region text afterwards. A second copy of this logic is
+ * the `DEFECT.GENERATIVE-FIX` shape: it silently stops agreeing the first time
+ * either is taught something.
+ *
+ * `terminated: false` is the fence-opened-but-never-closed case. It is not an
+ * error here because the two callers disagree about it: `extractFrontmatter`
+ * runs the #1882 truncation probe over the region and may warn, while
+ * `frontmatterListEntries` has no array to return and gives up. So the shape
+ * is reported and the decision is left to them.
+ *
+ * `content` is returned alongside because it is the BOM-STRIPPED text, and a
+ * caller reporting on the document (the truncation diagnostic) must describe
+ * the same bytes the offsets were computed against.
+ */
+function frontmatterRegion(
+  content: string,
+): { region: string; terminated: boolean; content: string } | null {
   // #2977: tolerate a single leading UTF-8 BOM (U+FEFF), which Windows tooling
-  // (PowerShell `>`/`Out-File` on PS 5.1, several editors) writes by default. Without this
-  // strip, the byte-0 `startsWith('---')` fence check below fails on the BOM and the whole
-  // parse collapses to {} — every frontmatter field silently disappears, and the engine
-  // proceeds as though the file had no frontmatter at all. The BOM is a single codepoint;
-  // stripping it here restores byte-0 alignment so the rest of the function is unchanged.
-  // Scope: BOM only. Arbitrary non-BOM content before the fence (leading whitespace/blank
-  // line/comment) is a separate product-intent decision (tolerate vs diagnose) left to a
-  // future change — this fix does not broaden the byte-0 fence rule beyond the BOM.
-  if (content.charCodeAt(0) === 0xFEFF) {
-    content = content.slice(1);
-  }
-  // Match frontmatter only at byte 0 — a `---` block later in the document
-  // body (YAML examples, horizontal rules) must never be treated as frontmatter.
+  // (PowerShell `>`/`Out-File` on PS 5.1, several editors) writes by default.
+  // Without this strip, the byte-0 `startsWith('---')` fence check below fails
+  // on the BOM and the whole parse collapses — every frontmatter field silently
+  // disappears, and the engine proceeds as though the file had no frontmatter
+  // at all. The BOM is a single codepoint; stripping it here restores byte-0
+  // alignment. Scope: BOM only. Arbitrary non-BOM content before the fence
+  // (leading whitespace/blank line/comment) is a separate product-intent
+  // decision (tolerate vs diagnose) left to a future change.
+  if (content.charCodeAt(0) === 0xFEFF) content = content.slice(1);
+  // Match frontmatter only at byte 0 — a `---` block later in the document body
+  // (YAML examples, horizontal rules) must never be treated as frontmatter.
   const headerEnd = content.startsWith('---\r\n') ? 5 : content.startsWith('---\n') ? 4 : -1;
-  if (headerEnd === -1) return {};
+  if (headerEnd === -1) return null;
 
   const closingLineStart = content.indexOf('\n---', headerEnd);
   if (closingLineStart === -1) {
-    const region = content.slice(headerEnd);
-    const keyCount = countKeysBeforeTruncation(region);
-    if (keyCount >= UNTERMINATED_KEY_THRESHOLD && isFrontmatterShaped(region)) {
+    return { region: content.slice(headerEnd), terminated: false, content };
+  }
+  const yamlEnd = content[closingLineStart - 1] === '\r' ? closingLineStart - 1 : closingLineStart;
+  return { region: content.slice(headerEnd, yamlEnd), terminated: true, content };
+}
+
+function extractFrontmatter(content: string, sourcePath?: string): Frontmatter {
+  // Fence location (BOM strip, byte-0 rule, CR handling) lives in
+  // `frontmatterRegion` so this and `frontmatterListEntries` cannot drift
+  // apart on where the frontmatter is.
+  const found = frontmatterRegion(content);
+  if (!found) return {};
+
+  if (!found.terminated) {
+    const keyCount = countKeysBeforeTruncation(found.region);
+    if (keyCount >= UNTERMINATED_KEY_THRESHOLD && isFrontmatterShaped(found.region)) {
       warnUnusableInput({
         reason: UNUSABLE_REASON.FRONTMATTER_UNTERMINATED,
         source: sourcePath,
-        content,
+        content: found.content,
       });
     }
     return {};
   }
 
-  const yamlEnd = content[closingLineStart - 1] === '\r' ? closingLineStart - 1 : closingLineStart;
-  const region = content.slice(headerEnd, yamlEnd);
   try {
-    return parseGuardedYamlRegion(region);
+    return parseGuardedYamlRegion(found.region);
   } catch {
     return unparseableResult();
   }
 }
 
+/**
+ * The entries of a top-level frontmatter ARRAY key, VERBATIM — as the values
+ * YAML actually describes rather than the display-flattened strings
+ * `extractFrontmatter` returns (#3850).
+ *
+ * `extractFrontmatter` renders each object entry for HUMANS —
+ * `normalizeParsedValue` maps `{test, resolution}` to `"test: …, resolution: …"`
+ * via `flattenObjectListItem`. That is the right contract for a reader printing
+ * a list, and the wrong one for a reader that needs to branch on a specific
+ * field: `status`, `resolution` and `reason` are recoverable from that string
+ * only by re-parsing prose, which cannot distinguish a real `resolution:` field
+ * from the same text inside a quoted `truth:`.
+ *
+ * This returns the same entries BEFORE that display step, off the same
+ * `extractFrontmatter` parse path — same BOM strip (#2977), same byte-0 fence
+ * rule (shared via `frontmatterRegion`), same anchor/alias and sentinel guards,
+ * same ambiguous-colon repair. It is deliberately NOT a second parser: a
+ * hand-rolled fence regex or entry slicer re-loses whatever the real one
+ * learned, which is the `DEFECT.GENERATIVE-FIX` shape.
+ *
+ * EVERY element is returned, at its own index, whatever its type (#3850 review
+ * round 3, Blocker). An earlier revision filtered to objects, which compacted
+ * the array: a caller numbering entries by array position then numbered the
+ * SURVIVORS, so a list mixing object and non-object entries lost rows outright
+ * and mis-numbered the rest — the same silently-vanishing-row defect this whole
+ * issue exists to close, triggered by entry shape instead of file status.
+ * Deciding what a non-object entry MEANS is a caller's judgement (the two
+ * readers in `uat.cts` answer it differently and both are right for their own
+ * vocabulary); dropping it is nobody's.
+ *
+ * Returns `null` when the document has no frontmatter, the frontmatter is
+ * unterminated or unparseable, the key is absent, or the key is not an array —
+ * "nothing to iterate" cases the caller should not have to tell apart.
+ */
+function frontmatterListEntries(content: string, key: string): unknown[] | null {
+  const found = frontmatterRegion(content);
+  if (!found || !found.terminated) return null;
+
+  let raw: unknown;
+  try {
+    refuseAnchorsAndAliases(found.region);
+    refuseIfSentinelPresent(found.region);
+    raw = restoreNullBytesDeep(loadWithAmbiguousColonRepair(escapeNullBytesForParse(found.region)));
+  } catch {
+    // Same posture as `extractFrontmatter`: an unparseable region is "no
+    // frontmatter", never a throw into a caller that was only reading a field.
+    return null;
+  }
+  if (!raw || typeof raw !== 'object' || Array.isArray(raw)) return null;
+  const value = (raw as Record<string, unknown>)[key];
+  // `Array.isArray` narrows an `unknown` to `any[]`, and returning that
+  // unchecked is how `any` escapes a guarded parser into every caller. The
+  // element type genuinely IS unknown here — that is the point of this
+  // function — so say so.
+  if (!Array.isArray(value)) return null;
+  return value as unknown[];
+}
+
 /**
  * Escape a string for emission inside a YAML double-quoted scalar (#1779). ADR-3473 §8.1
  * (#3881): routed through the vendored js-yaml's `dump()` (forced double-quoted style) rather
@@ -1468,6 +1559,13 @@ export = {
   stripFrontmatter,
   noOpObjectListSetError,
   parseMustHavesBlock,
+  // #3850: an array key's entries as parsed OBJECTS, for a caller that must
+  // branch on an entry's `status:`/`resolution:` rather than print it. Off the
+  // same parse path as `extractFrontmatter`, minus only the display flattening.
+  frontmatterListEntries,
+  // #3850: the display rendering itself, so a caller deriving a name from those
+  // objects produces the byte-identical string `extractFrontmatter` would have.
+  flattenObjectListItem,
   FRONTMATTER_SCHEMAS,
   cmdFrontmatterGet,
   cmdFrontmatterSet,