fix(#4546): deferred UAT follow-ups no longer block completion and promote to the backlog (#4769)

* test(#4546): failing-first tests for deferred uat follow-ups

* chore(#4546): regenerate derived lists for the deferred-promotion suite

The new verify-work-deferred-promotion suite changes the tests/ tree the
macOS conformance-tier classifier tracks and is a novel file under the
verify prefix in the test-file-count ratchet; both derived lists are
regenerated/registered per their own guards' instructions.

* fix(#4546): deferred uat follow-ups no longer block, and get promoted

Two halves of one disconnect (#1921's deferral design vs the completion
predicate):

- uat-predicate: the item parser now captures the block's reason: line
  alongside result:. A skipped item whose reason carries the
  verify-work writer's 'Deferred follow-up:' template is a deliberate
  deferral -- non-blocking, flagged deferred in the report. Quote-
  tolerant (the writer wraps the value) and case-insensitive. A
  reasonless skip, a non-deferral reason, pending/blocked/issue/
  failed/missing all still block, exactly as before.
- verify-work complete_session: when the Deferred Follow-Ups section is
  non-empty, offer to promote the items to a ROADMAP.md 999.x backlog
  entry reusing next.md's prior_phase_completeness entry shape, with a
  --files-scoped commit. Offer, not auto-mutation -- matches the
  workflow's interactive convention and next.md's own prompt style.

* chore(#4546): refresh compact-content benchmark baseline

verify-work.md grew (the #4546 deferred-follow-up promotion offer in
complete_session); the registered split's token counts moved with it.
Baseline recomputed with the script's own --write.

Emitted-Drift-Ack-Growth: verify-work.md — complete_session gained the deferred-follow-up promotion offer (detection, [P]/[K] choice, the next.md-shaped 999.x entry template, and the --files-scoped ROADMAP.md commit); the growth is the new contract text, not duplication

* fix(#4546): gate/audit agreement and review fixes for deferred follow-ups

- src/uat.cts categorizeItem: a skipped item carrying the deferred
  follow-up template reason now categorizes as 'deferred' (the category
  already existed for deferred-items.md entries) instead of being
  misfiled into the blocked families by keyword match -- the gate/audit
  agreement #3078-CR expects, restored in the permissive direction the
  #1921 design intends. Checked BEFORE the keyword families so '...
  on the release build next version' is not build_needed.
- verify-work.md promotion step: numbering scans for the smallest free
  999.n (count races + non-contiguous history), one backlog entry per
  deferred follow-up, ROADMAP.md-absent behavior specified, idea text
  newline-flattened, Deferred at placeholder harmonized with next.md.
- DEFERRED_REASON_RE: trust assumption documented (authoring contract,
  not a security boundary; non-matching spellings block fail-closed).
- tests: the property now drives evaluateUatPassed and derives
  expectations from the input spec (never restates the matcher),
  includes the no-result-line branch, and pins its seed; the parity
  test drops try/finally for the approved pattern, uses createTempDir,
  sites its allow-test-rule marker at the suppression site, and asserts
  the literal [P]/[K] choices.

* fix(#4546): close promotion-test docstring, drop fc replay-path misuse, refresh baseline

The final matrix run caught three defects in my own review-fix commit:
the parity test file's JSDoc was left unterminated (the whole file
parsed as one comment -- zero tests registered, hence the file-level
'test failed' the runner reported); fast-check's replay-path parameter
was misused as a label (invalid path at replay); and the workflow-text
ambiguity fixes re-drifted the compact-content benchmark baseline.

* docs(#4546): add Fixed changeset for deferred follow-up coverage

* docs(#4546): backfill changeset PR number

* fix(#4546): use the pattern seam escapeRegex for shape-marker matching

The hand-rolled metacharacter escape in the shape-marker assertion
tripped local/no-adhoc-regex-escape, whose named remedy this adopts.

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-09-15 07:30:38 -04:00
committed by GitHub
parent 0967358b8b
commit cbbde6786a
9 changed files with 353 additions and 10 deletions

View File

@@ -37,6 +37,13 @@ interface UatCheckItem {
name: string;
result: string;
passing: boolean;
/**
* #4546: true when this item is a `skipped` whose reason matches the
* verify-work writer's "Deferred follow-up:" template — a deliberately
* deferred follow-up (#1921) counts as passing and never blocks. Additive
* report field: consumers read `passed`/`blockers`.
*/
deferred: boolean;
}
interface UatPassedReport {
@@ -83,6 +90,22 @@ const BLOCKING_VERIFICATION_FM_STATUSES = new Set([
// UAT test-item `result` values that count as passing
const PASSING_RESULTS = new Set(['passed', 'pass']);
// #4546 — a `skipped` test-item whose reason carries the verify-work writer's
// deferral template prefix ("Deferred follow-up: …", #1921) is a deliberately
// deferred follow-up: non-blocking. Quote-tolerant (the writer wraps the value
// in double quotes) and case-insensitive (human-edited files vary). Anything
// else — a reasonless skip, a non-deferral reason — still blocks. PARITY: this
// matcher and the writer template in gsd-core/workflows/verify-work.md
// (process_response) are two halves of one contract, pinned together by
// tests/verify-work-deferred-promotion.test.cjs.
const DEFERRED_REASON_RE = /^["']?deferred follow-up\b/i;
// Trust note (#4546 review): the reason line is user-authored state — an
// author could equally write `result: passed` — so this prefix is an
// AUTHORING contract with the verify-work writer, not a security boundary.
// A hand-written deferral that skips the UAT file's ## Deferred Follow-Ups
// section also bypasses the complete_session promotion offer; the section is
// the durable project-level record. Variant spellings that do not match
// ("Deferred follow-ups:", "followup") block — fail-closed by design.
// ─── stripFalsePositiveContexts ───────────────────────────────────────────────
@@ -156,8 +179,8 @@ function analyzeMarkdown(raw: string): { unterminatedFence: boolean; unterminate
* - Support bracketed [passed] and bare passed (#2273).
* - Returns ALL items (both passing and non-passing).
*/
function parseUatResultItems(cleanContent: string): Array<{ test: number; name: string; result: string }> {
const items: Array<{ test: number; name: string; result: string }> = [];
function parseUatResultItems(cleanContent: string): Array<{ test: number; name: string; result: string; reason: string }> {
const items: Array<{ test: number; name: string; result: string; reason: string }> = [];
// Find all ### N. Name headings.
// #3078-CR MEDIUM (security review follow-up): STRUCTURE and ATTRIBUTION
@@ -212,16 +235,28 @@ function parseUatResultItems(cleanContent: string): Array<{ test: number; name:
// ambiguity counting, matching src/uat.cts's contract.
// Uses [ \t]* (not \s*) so the captured value must sit on the SAME line as result:.
// A result: key with the value on a subsequent line yields no match → 'missing' (blocker).
// #4546: the `reason:` line is captured with the same frame and FIRST-match
// rule — it is the deferral signal the evaluator needs (a `skipped` item
// whose reason is the verify-work writer's "Deferred follow-up:" template
// is non-blocking). Quoted values are captured with their quotes so the
// evaluator's matcher can tolerate them exactly as written.
const RESULT_LINE_RE = /^result:[ \t]*\[?([\w-]+)\]?/i;
const resultMatch = blockContent
.split('\n')
.map((line) => line.match(RESULT_LINE_RE))
.find((m): m is RegExpMatchArray => m !== null) ?? null;
const REASON_LINE_RE = /^reason:[ \t]*(.*)$/i;
const reasonMatch = blockContent
.split('\n')
.map((line) => line.match(REASON_LINE_RE))
.find((m): m is RegExpMatchArray => m !== null) ?? null;
const reason = reasonMatch ? reasonMatch[1].trim() : '';
if (resultMatch) {
items.push({
test: h.test,
name: h.name,
result: resultMatch[1].toLowerCase(),
reason,
});
} else {
// No column-0 result line → emit 'missing' (a non-passing state)
@@ -229,6 +264,7 @@ function parseUatResultItems(cleanContent: string): Array<{ test: number; name:
test: h.test,
name: h.name,
result: 'missing',
reason,
});
}
}
@@ -333,13 +369,20 @@ function evaluateUatPassed(
const items = parseUatResultItems(cleanContent);
for (const item of items) {
const passing = PASSING_RESULTS.has(item.result);
// #4546: a `skipped` item whose reason matches the verify-work writer's
// "Deferred follow-up:" template is a deliberately deferred follow-up
// (#1921) — non-blocking. Quote-tolerant because the writer emits the
// reason WITH its wrapping quotes. Everything else — pending, blocked,
// issue, missing, and a plain or non-deferral skipped — still blocks.
const deferred = item.result === 'skipped' && DEFERRED_REASON_RE.test(item.reason);
const passing = PASSING_RESULTS.has(item.result) || deferred;
checks.push({
file,
test: item.test,
name: item.name,
result: item.result,
passing,
deferred,
});
if (!passing) {
blockers.push(`${file}: test ${item.test} (${item.result})`);

View File

@@ -3973,6 +3973,13 @@ function categorizeItem(rawResult: string, reason?: string, blockedBy?: string):
return 'blocked';
}
if (result === 'skipped') {
// #4546 — a deliberately deferred follow-up (the verify-work writer's
// "Deferred follow-up:" template, #1921) is its own category, checked
// BEFORE the keyword families so e.g. "…on the release build next
// version" is not misfiled as build_needed. Must agree with
// uat-predicate.cts's DEFERRED_REASON_RE (gate/audit agreement,
// #3078-CR), pinned by tests/uat-predicate.test.cjs.
if (reason && /^["']?deferred follow-up\b/i.test(reason)) return 'deferred';
if (reason) {
if (/server|not running|not available/i.test(reason)) return 'server_blocked';
if (/simulator|physical|device/i.test(reason)) return 'device_needed';