* fix(#2766): scan archived phases and read table-shaped Gaps/deferred entries Three silent false negatives in cmdAuditUat and its readers, all failing in the reassuring (false-negative) direction — a UAT audit whose entire job is to catch leftover work reporting zero over real work: 1. Archived phases invisible (src/uat.cts cmdAuditUat): on milestone completion milestone.cts MOVES phase dirs into .planning/milestones/<version>-phases/ (archive-by-default since #1871), leaving .planning/phases/ empty or absent. Partial archive → false all-clear; full archive → hard error indistinguishable from a broken install. Fix: enumerate archived dirs via the canonical getArchivedPhaseDirs seam (phase-locator.cts); archived dirs deliberately bypass getMilestonePhaseFilter (which scopes to the CURRENT milestone — applying it to past-milestone dirs would discard every one and reinstate the bug). 2. Table-shaped deferred-items.md yielded zero items (splitGapsEntries keyed on bullet openers only; a GFM table row starts with |). Fix: union of bullet + numbered + table-row splits. 3. Table-shaped ## Gaps yielded zero items (same bullet-only splitter). Fix: same union walker. The table walker is deliberately NOT routed through parseMarkdownTable (ADR-2143 §3 — that reads only the first table and treats ragged/headerless shapes as errors, the wrong contract for a hand-written backstop table that must surface its rows). New additive archived_milestone field labels provenance. Fix authored by issue reporter gavin-ray and verified against the published tarball; maintainer triage (trek-e) confirmed all three findings. Cherry- picked onto fresh next after prior PR #2832 closed for staleness; re-verified under gsd-test + reviews. 21 regression tests including the negative direction (bullet-only unchanged, status: resolved still suppressed, empty phases dir still succeeds). * chore(#2766): backfill changeset PR number 3082 --------- Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
5
.changeset/mellow-eagles-purr.md
Normal file
5
.changeset/mellow-eagles-purr.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Fixed
|
||||
pr: 3082
|
||||
---
|
||||
**`/gsd-audit-uat` now sees archived phases and table-shaped artifacts** — three silent false negatives are fixed: (1) the audit scanned only `.planning/phases/`, so a project whose milestones had been archived to `.planning/milestones/<version>-phases/` silently omitted those phases, and one with ALL phases archived hard-errored with "No phases directory found" instead of reporting its outstanding items; (2) a `deferred-items.md` recording entries as a GFM table yielded zero items; (3) a table-shaped `## Gaps` section likewise yielded zero items. Results now carry `archived_milestone` so consumers can label provenance. Same false-negative family as #2286/#2287, one document shape further out. (#2766)
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"_doc": "Baseline of modules currently exceeding the 2-test-file limit. Each entry locks in TODAY's exact test filenames as the allowlisted set (identity ratchet). Adding a NEW test file to a capped module fails (novel). Removing one requires pruning this list (stale, ratchet-down). When a cluster drops to ≤ 2, remove its entry entirely. New entries require justification in PR description.",
|
||||
"_doc": "Baseline of modules currently exceeding the 2-test-file limit. Each entry locks in TODAY's exact test filenames as the allowlisted set (identity ratchet). Adding a NEW test file to a capped module fails (novel). Removing one requires pruning this list (stale, ratchet-down). When a cluster drops to \u2264 2, remove its entry entirely. New entries require justification in PR description.",
|
||||
"modules": {
|
||||
"config": {
|
||||
"files": [
|
||||
@@ -125,6 +125,14 @@
|
||||
"prompt-budget.test.cjs"
|
||||
],
|
||||
"issue": "2929"
|
||||
},
|
||||
"audit": {
|
||||
"files": [
|
||||
"audit-command-cutover.test.cjs",
|
||||
"audit-fix-command.test.cjs",
|
||||
"fix-2766-audit-uat-archived-and-table-shapes.test.cjs"
|
||||
],
|
||||
"issue": "2766"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
274
src/uat.cts
274
src/uat.cts
@@ -19,7 +19,7 @@ import markdownSectionizer = require('./markdown-sectionizer.cjs');
|
||||
const { collectSection, tokenizeHeadings } = markdownSectionizer;
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
import markdownTable = require('./markdown-table.cjs');
|
||||
const { splitTableRow } = markdownTable;
|
||||
const { splitTableRow, isDelimiterRow } = markdownTable;
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
import roadmapParser = require('./roadmap-parser.cjs');
|
||||
const { getMilestonePhaseFilter } = roadmapParser;
|
||||
@@ -35,6 +35,9 @@ const { extractFrontmatter } = frontmatter;
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
import phaseIdMod = require('./phase-id.cjs');
|
||||
const { PHASE_NUMBER_TOKEN_SOURCE } = phaseIdMod;
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||
import phaseLocator = require('./phase-locator.cjs');
|
||||
const { getArchivedPhaseDirs } = phaseLocator;
|
||||
import { requireSafePath, sanitizeForDisplay } from './security.cjs';
|
||||
// eslint-disable-next-line @typescript-eslint/no-require-imports -- config-loader.cjs is an export= CommonJS module
|
||||
import configLoader = require('./config-loader.cjs');
|
||||
@@ -62,6 +65,13 @@ interface UatFileResult {
|
||||
file_path: string;
|
||||
type: 'uat' | 'verification' | 'deferred';
|
||||
status: string;
|
||||
/**
|
||||
* Milestone version whose archive this phase dir was read from
|
||||
* (`.planning/milestones/<version>-phases/`), or undefined for a phase still
|
||||
* in the active `.planning/phases/` tree. Lets a consumer label provenance
|
||||
* instead of presenting archived and in-flight work identically.
|
||||
*/
|
||||
archived_milestone?: string;
|
||||
items: UatItem[];
|
||||
}
|
||||
|
||||
@@ -76,24 +86,56 @@ interface CurrentTest {
|
||||
|
||||
function cmdAuditUat(cwd: string, raw: boolean): void {
|
||||
const phasesDir = path.join(planningDir(cwd), 'phases');
|
||||
if (!fs.existsSync(phasesDir)) {
|
||||
const hasActivePhases = fs.existsSync(phasesDir);
|
||||
|
||||
// #2766: on milestone completion `milestone.cts` MOVES each phase dir into
|
||||
// `.planning/milestones/<version>-phases/` (archive-by-default since #1871),
|
||||
// leaving `.planning/phases/` empty or absent. Scanning only the active tree
|
||||
// meant a partly-archived project silently omitted the archived phases, and a
|
||||
// fully-archived one hard-errored with "No phases directory found" —
|
||||
// indistinguishable from a broken install. Outstanding UAT items do not stop
|
||||
// mattering when a milestone closes: a deferred human-UAT scenario or a
|
||||
// `skipped` live-stack test is exactly what gets archived still-open.
|
||||
//
|
||||
// Reuses the canonical `getArchivedPhaseDirs` seam (phase-locator.cts), which
|
||||
// `findPhaseInternal` already uses for this same fallback, so the archive
|
||||
// layout convention stays owned by one module.
|
||||
const archivedDirs = getArchivedPhaseDirs(cwd);
|
||||
if (!hasActivePhases && archivedDirs.length === 0) {
|
||||
error('No phases directory found in planning directory');
|
||||
}
|
||||
|
||||
const isDirInMilestone = getMilestonePhaseFilter(cwd);
|
||||
const results: UatFileResult[] = [];
|
||||
|
||||
// Scan all phase directories
|
||||
const dirs = fs.readdirSync(phasesDir, { withFileTypes: true })
|
||||
.filter(e => e.isDirectory())
|
||||
.map(e => e.name)
|
||||
.filter(isDirInMilestone)
|
||||
.sort();
|
||||
// Active dirs are milestone-filtered; archived dirs deliberately are NOT.
|
||||
// getMilestonePhaseFilter derives the CURRENT milestone's phase numbers from
|
||||
// ROADMAP.md, and archived phases belong to past milestones by definition — so
|
||||
// applying it to them discards every one and silently reinstates the bug.
|
||||
const scanTargets: { dir: string; phaseDir: string; milestone?: string }[] = [];
|
||||
|
||||
for (const dir of dirs) {
|
||||
if (hasActivePhases) {
|
||||
const dirs = fs.readdirSync(phasesDir, { withFileTypes: true })
|
||||
.filter(e => e.isDirectory())
|
||||
.map(e => e.name)
|
||||
.filter(isDirInMilestone)
|
||||
.sort();
|
||||
for (const dir of dirs) {
|
||||
scanTargets.push({ dir, phaseDir: path.join(phasesDir, dir) });
|
||||
}
|
||||
}
|
||||
|
||||
for (const archived of archivedDirs) {
|
||||
scanTargets.push({
|
||||
dir: archived.name,
|
||||
phaseDir: archived.fullPath,
|
||||
milestone: archived.milestone,
|
||||
});
|
||||
}
|
||||
|
||||
for (const { dir, phaseDir, milestone } of scanTargets) {
|
||||
const phaseMatch = dir.match(new RegExp(`^(${PHASE_NUMBER_TOKEN_SOURCE})`, 'i'));
|
||||
const phaseNum = phaseMatch ? phaseMatch[1] : dir;
|
||||
const phaseDir = path.join(phasesDir, dir);
|
||||
const files = fs.readdirSync(phaseDir);
|
||||
|
||||
// Process UAT files
|
||||
@@ -109,6 +151,7 @@ function cmdAuditUat(cwd: string, raw: boolean): void {
|
||||
file_path: toPosixPath(path.relative(cwd, path.join(phaseDir, file))),
|
||||
type: 'uat',
|
||||
status: (extractFrontmatter(content, uatFilePath).status as string || 'unknown'),
|
||||
archived_milestone: milestone,
|
||||
items,
|
||||
});
|
||||
}
|
||||
@@ -129,6 +172,7 @@ function cmdAuditUat(cwd: string, raw: boolean): void {
|
||||
file_path: toPosixPath(path.relative(cwd, path.join(phaseDir, file))),
|
||||
type: 'verification',
|
||||
status,
|
||||
archived_milestone: milestone,
|
||||
items,
|
||||
});
|
||||
}
|
||||
@@ -154,6 +198,7 @@ function cmdAuditUat(cwd: string, raw: boolean): void {
|
||||
file_path: toPosixPath(path.relative(cwd, path.join(phaseDir, deferredFile))),
|
||||
type: 'deferred',
|
||||
status: 'unresolved',
|
||||
archived_milestone: milestone,
|
||||
items,
|
||||
});
|
||||
}
|
||||
@@ -655,6 +700,170 @@ function parseGapsItems(content: string): UatItem[] {
|
||||
items.push(item);
|
||||
}
|
||||
|
||||
// #2766: union with the table form. A `|`-leading line is never a `- ` bullet
|
||||
// opener, so a section mixing bullet entries and a table surfaces both with no
|
||||
// double-counting.
|
||||
items.push(...parseGapsTableItems(gapsSection.body));
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
/**
|
||||
* Split a section body into its GFM pipe tables, one entry per table (#2766).
|
||||
*
|
||||
* Shared by `parseGapsTableItems` and `parseDeferredTableItems` so the
|
||||
* header/delimiter/table-boundary handling — the fiddly part — lives in exactly
|
||||
* one place, and the two consumers only decide what a data row MEANS.
|
||||
*
|
||||
* Header detection is lookahead-free: the last data-shaped row is held in
|
||||
* `pending` until the NEXT line decides its fate — a delimiter row
|
||||
* (`|---|---|`) proves the held row was a header, anything else promotes it to a
|
||||
* data row. So a conventional table drops exactly its header, a HEADERLESS table
|
||||
* keeps every row (hand-authored planning tables often omit the delimiter), and
|
||||
* a header with no data rows yields nothing. A prose or blank line ends the
|
||||
* current table, so two tables separated by text are read independently and each
|
||||
* drops its own header.
|
||||
*
|
||||
* Reuses the canonical `isDelimiterRow` shape check from markdown-table.cts
|
||||
* rather than re-deriving it. Deliberately NOT routed through
|
||||
* `parseMarkdownTable`, which reads only the FIRST table in a body and treats
|
||||
* ragged/headerless shapes as errors (ADR-2143 §3) — correct for the mandated
|
||||
* tables in STATE.md/ROADMAP.md, but the wrong contract here, where a malformed
|
||||
* hand-written table must still surface its rows rather than be dropped.
|
||||
*/
|
||||
function collectTableRows(sectionBody: string): { header: string[] | null; rows: string[][] }[] {
|
||||
const tables: { header: string[] | null; rows: string[][] }[] = [];
|
||||
let current: { header: string[] | null; rows: string[][] } | null = null;
|
||||
let pending: string[] | null = null;
|
||||
|
||||
const ensure = (): void => {
|
||||
if (!current) current = { header: null, rows: [] };
|
||||
};
|
||||
const flushPending = (): void => {
|
||||
if (pending) {
|
||||
ensure();
|
||||
current!.rows.push(pending);
|
||||
pending = null;
|
||||
}
|
||||
};
|
||||
const endTable = (): void => {
|
||||
flushPending();
|
||||
if (current) {
|
||||
tables.push(current);
|
||||
current = null;
|
||||
}
|
||||
};
|
||||
|
||||
for (const rawLine of sectionBody.split('\n')) {
|
||||
const line = rawLine.replace(/\r$/, '').trim();
|
||||
if (!line.startsWith('|')) {
|
||||
endTable();
|
||||
continue;
|
||||
}
|
||||
const cells = splitTableRow(line);
|
||||
if (cells.length === 0) continue;
|
||||
if (isDelimiterRow(cells)) {
|
||||
ensure();
|
||||
current!.header = pending; // may be null for a delimiter-first table
|
||||
pending = null;
|
||||
continue;
|
||||
}
|
||||
flushPending();
|
||||
pending = cells;
|
||||
}
|
||||
endTable();
|
||||
|
||||
return tables;
|
||||
}
|
||||
|
||||
/**
|
||||
* Header-name → canonical Gaps field (#2766).
|
||||
*
|
||||
* Anchored on the `## Gaps` field vocabulary `templates/UAT.md` mandates for the
|
||||
* YAML-lite bullet form (truth/status/reason/severity/test), plus the obvious
|
||||
* synonyms a human writing the same information as a table reaches for instead.
|
||||
*/
|
||||
const GAPS_COLUMN_ALIASES: Record<string, 'truth' | 'status' | 'reason' | 'severity' | 'test'> = {
|
||||
truth: 'truth', gap: 'truth', finding: 'truth', item: 'truth',
|
||||
description: 'truth', issue: 'truth', name: 'truth',
|
||||
status: 'status', result: 'status', state: 'status',
|
||||
reason: 'reason', note: 'reason', notes: 'reason',
|
||||
detail: 'reason', details: 'reason', evidence: 'reason',
|
||||
severity: 'severity',
|
||||
test: 'test', '#': 'test', 'test #': 'test', 'test number': 'test',
|
||||
};
|
||||
|
||||
function mapGapsHeader(header: string[] | null): Record<string, number> | null {
|
||||
if (!header) return null;
|
||||
const columns: Record<string, number> = {};
|
||||
header.forEach((cell, idx) => {
|
||||
const key = GAPS_COLUMN_ALIASES[cell.trim().toLowerCase().replace(/\*+/g, '')];
|
||||
if (key && !(key in columns)) columns[key] = idx;
|
||||
});
|
||||
return Object.keys(columns).length > 0 ? columns : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract gap entries from GFM pipe tables in a `## Gaps` section (#2766) — a
|
||||
* UNION with the YAML-lite bullet scan in `parseGapsItems`, for the same reason
|
||||
* `parseDeferredTableItems` exists: `splitGapsEntries` keys entirely on `- `
|
||||
* bullet openers, so a table-shaped `## Gaps` section yielded ZERO items and
|
||||
* every finding in it was silently invisible.
|
||||
*
|
||||
* Neither `templates/UAT.md` nor `templates/verification-report.md` documents a
|
||||
* table for this section (both mandate the bullet/numbered form), so a table
|
||||
* here is off-template hand-authoring — which is precisely why it must not fail
|
||||
* silently. Note `parseVerificationItems` in this same file already reads table
|
||||
* rows AND numbered AND bullet items as a union because the live sections mix
|
||||
* shapes; the Gaps and deferred parsers never got the same treatment.
|
||||
*
|
||||
* When a header row is present its columns are mapped by name against the
|
||||
* template's own field vocabulary (see GAPS_COLUMN_ALIASES) so a tabled gap
|
||||
* carries the same status/reason/test fields as its bullet equivalent and
|
||||
* `categorizeItem` classifies it identically. With no recognizable header, the
|
||||
* row degrades to a joined-cells name with status `unknown` — surfaced, not
|
||||
* dropped, matching this module's established fail-safe stance.
|
||||
*
|
||||
* Resolution follows the bullet path exactly: an entry is skipped ONLY on an
|
||||
* explicit resolved marker — the mapped `status` column reading `resolved`, or,
|
||||
* absent a status column, any cell reading exactly `resolved`. A gap with no
|
||||
* parseable status is NEVER treated as resolved.
|
||||
*/
|
||||
function parseGapsTableItems(sectionBody: string): UatItem[] {
|
||||
const items: UatItem[] = [];
|
||||
|
||||
for (const { header, rows } of collectTableRows(sectionBody)) {
|
||||
const columns = mapGapsHeader(header);
|
||||
for (const cells of rows) {
|
||||
const at = (key: string): string =>
|
||||
(columns && key in columns ? (cells[columns[key]] ?? '').trim() : '');
|
||||
|
||||
const rawStatus = at('status');
|
||||
if (rawStatus && rawStatus.toLowerCase() === 'resolved') continue;
|
||||
// No status column: fall back to an explicit resolved marker in any cell
|
||||
// (the headerless-table equivalent of `status: resolved`).
|
||||
if (!columns || !('status' in columns)) {
|
||||
if (cells.some(c => /^resolved$/i.test(c.trim()))) continue;
|
||||
}
|
||||
|
||||
const truth = at('truth');
|
||||
const reason = at('reason');
|
||||
const testNum = at('test');
|
||||
const name = truth || cells.filter(c => c !== '').join(' — ');
|
||||
if (!name) continue;
|
||||
|
||||
const status = rawStatus || 'unknown';
|
||||
const item: UatItem = {
|
||||
name,
|
||||
result: status,
|
||||
category: categorizeItem(status, reason || undefined, undefined),
|
||||
};
|
||||
if (testNum && /^\d+$/.test(testNum)) item.test = parseInt(testNum, 10);
|
||||
if (reason) item.reason = reason;
|
||||
items.push(item);
|
||||
}
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
@@ -711,6 +920,51 @@ function parseDeferredItems(content: string): UatItem[] {
|
||||
});
|
||||
}
|
||||
|
||||
// #2766: union with the table form — see parseDeferredTableItems. Executors
|
||||
// write this file by hand with no mandated shape, and a GFM table is a natural
|
||||
// choice for the common "test → failing seeds" case, which produced ZERO items.
|
||||
items.push(...parseDeferredTableItems(sectionBody));
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract deferred entries from GFM pipe tables in a deferred-items.md body
|
||||
* (#2766) — a UNION with the bullet scan in `parseDeferredItems`.
|
||||
*
|
||||
* Cells are joined with ` — ` rather than taking only the first: these tables
|
||||
* carry the useful detail in the later columns (the failing seeds, the reason,
|
||||
* the owner), and dropping them would surface a name with no context.
|
||||
*
|
||||
* A row is skipped when any cell reads exactly `resolved`/`done`/`pass`
|
||||
* (case-insensitive), mirroring the "explicit resolution only" convention
|
||||
* `parseGapsItems` uses for `status: resolved` and `parseVerificationItems` uses
|
||||
* for its `hasPassResult` cell scan — so a human can close a tabled deferred
|
||||
* item in place and keep deferred-items.md the single source of truth.
|
||||
*
|
||||
* Deliberately permissive: an unrelated table in a deferred-items.md (say a
|
||||
* table of environment notes) will surface as deferred entries. That is the
|
||||
* correct fail-safe direction for a false-NEGATIVE bug — the whole file exists to
|
||||
* record outstanding work, and this module's established stance (see
|
||||
* parseGapsItems' 'unknown'-status fallback) is to surface a questionable entry
|
||||
* rather than silently drop a real one.
|
||||
*/
|
||||
function parseDeferredTableItems(sectionBody: string): UatItem[] {
|
||||
const items: UatItem[] = [];
|
||||
|
||||
for (const { rows } of collectTableRows(sectionBody)) {
|
||||
for (const cells of rows) {
|
||||
if (cells.some(c => /^(resolved|done|pass)$/i.test(c))) continue;
|
||||
const name = cells.filter(c => c !== '').join(' — ');
|
||||
if (!name) continue;
|
||||
items.push({
|
||||
name,
|
||||
result: 'unresolved',
|
||||
category: 'deferred',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
|
||||
391
tests/fix-2766-audit-uat-archived-and-table-shapes.test.cjs
Normal file
391
tests/fix-2766-audit-uat-archived-and-table-shapes.test.cjs
Normal file
@@ -0,0 +1,391 @@
|
||||
/**
|
||||
* #2766 — three audit-uat false negatives.
|
||||
*
|
||||
* All three fail SILENTLY and in the reassuring direction: outstanding UAT work
|
||||
* is under-reported, so the command whose job is to be the backstop reports a
|
||||
* clean bill of health over real unresolved items.
|
||||
*
|
||||
* 1. `cmdAuditUat` scanned only `.planning/phases/`. On milestone completion
|
||||
* `milestone.cts` MOVES each phase dir into
|
||||
* `.planning/milestones/<version>-phases/` (archive-by-default since #1871),
|
||||
* so a partly-archived project silently omitted the archived phases and a
|
||||
* fully-archived one hard-errored with "No phases directory found",
|
||||
* indistinguishable from a broken install.
|
||||
* 2. `parseDeferredItems` delegates entry splitting to `splitGapsEntries`, which
|
||||
* keys entirely on `- ` bullet openers — so a `deferred-items.md` recording
|
||||
* entries as a GFM table produced ZERO items. The SCOPE BOUNDARY convention
|
||||
* mandates no shape, and a table is natural for "test → failing seeds".
|
||||
* 3. `parseGapsItems` has the identical blindness for the same shared reason, so
|
||||
* a table-shaped `## Gaps` section surfaced nothing.
|
||||
*
|
||||
* Same false-negative family as #2286 and #2287, which fixed the first two
|
||||
* shapes; these are the next ones out.
|
||||
*
|
||||
* This fix:
|
||||
* - `cmdAuditUat` collects scan targets from the active tree AND
|
||||
* `getArchivedPhaseDirs` (the canonical seam `findPhaseInternal` already uses),
|
||||
* erroring only when both are empty. Archived dirs deliberately bypass
|
||||
* `getMilestonePhaseFilter` — it derives the CURRENT milestone's phase numbers
|
||||
* from ROADMAP.md, so applying it to archived dirs discards every one and
|
||||
* silently reinstates the bug. Results gain `archived_milestone` for provenance.
|
||||
* - One shared `collectTableRows` walker (header/delimiter/boundary handling in
|
||||
* one place) feeds table scans in BOTH parsers as a UNION with the existing
|
||||
* bullet scan. A `|`-leading line is never a `- ` bullet opener, so mixed files
|
||||
* surface both with no double-counting. Resolution semantics are unchanged
|
||||
* from the bullet path: suppress only on an explicit marker.
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
|
||||
const { test, describe, beforeEach, afterEach } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const { runGsdTools, createTempProject, createTempDir, cleanup } = require('./helpers.cjs');
|
||||
const { parseDeferredItems } = require('../gsd-core/bin/lib/uat.cjs');
|
||||
|
||||
const UAT_ONE_PENDING = [
|
||||
'---',
|
||||
'status: partial',
|
||||
'phase: 01-foundation',
|
||||
'---',
|
||||
'',
|
||||
'## Current Test',
|
||||
'',
|
||||
'[awaiting human testing]',
|
||||
'',
|
||||
'## Tests',
|
||||
'',
|
||||
'### 1. A scenario nobody ever ran',
|
||||
'expected: something observable happens',
|
||||
'result: [pending]',
|
||||
'',
|
||||
'## Summary',
|
||||
'',
|
||||
'total: 1',
|
||||
'pending: 1',
|
||||
'',
|
||||
'## Gaps',
|
||||
'',
|
||||
].join('\n');
|
||||
|
||||
/** Write a UAT file whose `## Gaps` section holds `gapsBody`. */
|
||||
function uatWithGaps(gapsBody) {
|
||||
return [
|
||||
'---',
|
||||
'status: complete',
|
||||
'phase: 50-gaps',
|
||||
'---',
|
||||
'',
|
||||
'## Current Test',
|
||||
'',
|
||||
'[testing complete]',
|
||||
'',
|
||||
'## Tests',
|
||||
'',
|
||||
'### 1. A passing scenario',
|
||||
'expected: this one is fine',
|
||||
'result: pass',
|
||||
'',
|
||||
'## Summary',
|
||||
'',
|
||||
'total: 1',
|
||||
'passed: 1',
|
||||
'',
|
||||
'## Gaps',
|
||||
'',
|
||||
gapsBody,
|
||||
'',
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
// ─── Bug 1: archived phase dirs ───────────────────────────────────────────────
|
||||
|
||||
describe('#2766 cmdAuditUat: archived phase directories', () => {
|
||||
let tmpDir;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpDir = createTempProject();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
cleanup(tmpDir);
|
||||
});
|
||||
|
||||
test('phases ONLY in the archive → items surfaced, not a hard error', () => {
|
||||
const archiveDir = path.join(
|
||||
tmpDir, '.planning', 'milestones', 'v1.0-phases', '01-foundation',
|
||||
);
|
||||
fs.mkdirSync(archiveDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(archiveDir, '01-UAT.md'), UAT_ONE_PENDING);
|
||||
|
||||
const result = runGsdTools('audit-uat --raw', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.summary.total_items, 1);
|
||||
assert.strictEqual(output.results.length, 1);
|
||||
assert.strictEqual(output.results[0].phase, '01');
|
||||
assert.strictEqual(output.results[0].archived_milestone, 'v1.0');
|
||||
assert.match(output.results[0].file_path, /milestones\/v1\.0-phases\//);
|
||||
});
|
||||
|
||||
test('active and archived trees are both scanned', () => {
|
||||
const activeDir = path.join(tmpDir, '.planning', 'phases', '40-current');
|
||||
fs.mkdirSync(activeDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(activeDir, '40-UAT.md'), UAT_ONE_PENDING);
|
||||
|
||||
const archiveDir = path.join(
|
||||
tmpDir, '.planning', 'milestones', 'v1.0-phases', '01-foundation',
|
||||
);
|
||||
fs.mkdirSync(archiveDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(archiveDir, '01-UAT.md'), UAT_ONE_PENDING);
|
||||
|
||||
const result = runGsdTools('audit-uat --raw', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
const byPhase = new Map(output.results.map(r => [r.phase, r]));
|
||||
assert.ok(byPhase.has('01'), `archived phase missing: ${JSON.stringify([...byPhase.keys()])}`);
|
||||
assert.ok(byPhase.has('40'), `active phase missing: ${JSON.stringify([...byPhase.keys()])}`);
|
||||
assert.strictEqual(byPhase.get('01').archived_milestone, 'v1.0');
|
||||
assert.strictEqual(byPhase.get('40').archived_milestone, undefined);
|
||||
});
|
||||
|
||||
test('multiple archived milestones are all scanned', () => {
|
||||
for (const [version, phase] of [['v1.0', '01-foundation'], ['v2.0', '07-later']]) {
|
||||
const dir = path.join(tmpDir, '.planning', 'milestones', `${version}-phases`, phase);
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
fs.writeFileSync(path.join(dir, `${phase.slice(0, 2)}-UAT.md`), UAT_ONE_PENDING);
|
||||
}
|
||||
|
||||
const result = runGsdTools('audit-uat --raw', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
assert.strictEqual(output.summary.total_items, 2);
|
||||
assert.deepStrictEqual(
|
||||
output.results.map(r => r.archived_milestone).sort(),
|
||||
['v1.0', 'v2.0'],
|
||||
);
|
||||
});
|
||||
|
||||
test('an empty active phases dir still succeeds with no items (pre-existing behavior)', () => {
|
||||
// createTempProject() ships an empty `.planning/phases/`, so this is the
|
||||
// shape the existing uat.test.cjs "no UAT files" case covers — the archive
|
||||
// change must not turn it into an error.
|
||||
const result = runGsdTools('audit-uat --raw', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
assert.deepStrictEqual(output.results, []);
|
||||
assert.strictEqual(output.summary.total_items, 0);
|
||||
});
|
||||
|
||||
test('no phases dir AND no archive still errors — no false all-clear', () => {
|
||||
// A bare temp dir with a .planning/ that has NO phases subdir and no
|
||||
// milestones archive — built from createTempDir rather than by deleting
|
||||
// createTempProject's phases dir, so nothing is torn down mid-test.
|
||||
const bare = createTempDir();
|
||||
try {
|
||||
fs.mkdirSync(path.join(bare, '.planning'), { recursive: true });
|
||||
|
||||
const result = runGsdTools('audit-uat --raw', bare);
|
||||
assert.strictEqual(result.success, false, 'expected a failure when no phases exist at all');
|
||||
} finally {
|
||||
cleanup(bare);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Bug 2: table-shaped deferred-items.md ────────────────────────────────────
|
||||
|
||||
describe('#2766 parseDeferredItems: GFM table shape', () => {
|
||||
const names = (md) => parseDeferredItems(md).map(i => i.name);
|
||||
|
||||
test('header + delimiter → header dropped, data rows surfaced', () => {
|
||||
assert.deepStrictEqual(
|
||||
names([
|
||||
'## Discovered during 01-03',
|
||||
'',
|
||||
'| Test | Failing seeds |',
|
||||
'|------|---------------|',
|
||||
'| test_a | 0, 1 |',
|
||||
'| test_b | 424242 |',
|
||||
].join('\n')),
|
||||
['test_a — 0, 1', 'test_b — 424242'],
|
||||
);
|
||||
});
|
||||
|
||||
test('later columns are preserved, not truncated to the first cell', () => {
|
||||
const [name] = names('| T | seeds |\n|---|---|\n| test_a | 0, 1, 424242 |');
|
||||
assert.match(name, /0, 1, 424242/);
|
||||
});
|
||||
|
||||
test('headerless table → every row surfaced', () => {
|
||||
assert.deepStrictEqual(
|
||||
names('| test_a | 0 |\n| test_b | 1 |'),
|
||||
['test_a — 0', 'test_b — 1'],
|
||||
);
|
||||
});
|
||||
|
||||
test('row marked resolved/done/pass is suppressed', () => {
|
||||
assert.deepStrictEqual(
|
||||
names([
|
||||
'| Test | Seeds | Status |',
|
||||
'|---|---|---|',
|
||||
'| test_open | 0 | open |',
|
||||
'| test_fixed | 1 | resolved |',
|
||||
'| test_done | 2 | DONE |',
|
||||
].join('\n')),
|
||||
['test_open — 0 — open'],
|
||||
);
|
||||
});
|
||||
|
||||
test('two prose-separated tables → each drops its own header', () => {
|
||||
assert.deepStrictEqual(
|
||||
names([
|
||||
'| T1 | x |', '|---|---|', '| one | 1 |',
|
||||
'',
|
||||
'some prose in between',
|
||||
'',
|
||||
'| T2 | y |', '|---|---|', '| two | 2 |',
|
||||
].join('\n')),
|
||||
['one — 1', 'two — 2'],
|
||||
);
|
||||
});
|
||||
|
||||
test('bullets and a table in one file → union, no double-counting', () => {
|
||||
const got = names([
|
||||
'## Deferred Items',
|
||||
'',
|
||||
'- a bullet-shaped deferred entry',
|
||||
'',
|
||||
'| Test | Seeds |',
|
||||
'|---|---|',
|
||||
'| test_a | 0 |',
|
||||
].join('\n'));
|
||||
assert.strictEqual(got.length, 2, JSON.stringify(got));
|
||||
assert.ok(got.some(n => n.includes('bullet-shaped')));
|
||||
assert.ok(got.some(n => n.startsWith('test_a')));
|
||||
});
|
||||
|
||||
test('bullet-only file unchanged (no regression on #2287)', () => {
|
||||
assert.deepStrictEqual(
|
||||
names('## Deferred Items\n\n- entry one\n- entry two\n'),
|
||||
['entry one', 'entry two'],
|
||||
);
|
||||
});
|
||||
|
||||
test('explicit status: resolved bullet still suppressed (no regression on #2287)', () => {
|
||||
const got = names(
|
||||
'## Deferred Items\n\n- truth: "closed thing"\n status: resolved\n- truth: "open thing"\n',
|
||||
);
|
||||
assert.strictEqual(got.length, 1, JSON.stringify(got));
|
||||
assert.match(got[0], /open thing/);
|
||||
});
|
||||
|
||||
test('no table and no bullets → zero items, no throw', () => {
|
||||
assert.deepStrictEqual(names('# Notes\n\njust prose, nothing actionable.\n'), []);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Bug 3: table-shaped ## Gaps section ──────────────────────────────────────
|
||||
|
||||
describe('#2766 parseGapsItems: GFM table shape', () => {
|
||||
let tmpDir;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpDir = createTempProject();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
cleanup(tmpDir);
|
||||
});
|
||||
|
||||
/** Run audit-uat over a phase whose UAT file has `gapsBody` as its Gaps section. */
|
||||
function gapsItems(gapsBody) {
|
||||
const phaseDir = path.join(tmpDir, '.planning', 'phases', '50-gaps');
|
||||
fs.mkdirSync(phaseDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(phaseDir, '50-UAT.md'), uatWithGaps(gapsBody));
|
||||
|
||||
const result = runGsdTools('audit-uat --raw', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
const output = JSON.parse(result.output);
|
||||
const uat = output.results.find(r => r.type === 'uat');
|
||||
return uat ? uat.items : [];
|
||||
}
|
||||
|
||||
test('header-mapped table → truth/status/reason/test extracted', () => {
|
||||
const items = gapsItems([
|
||||
'| Truth | Status | Reason | Test |',
|
||||
'|-------|--------|--------|------|',
|
||||
'| Login should redirect | failed | User reported a 500 | 1 |',
|
||||
].join('\n'));
|
||||
|
||||
assert.strictEqual(items.length, 1, JSON.stringify(items));
|
||||
assert.strictEqual(items[0].name, 'Login should redirect');
|
||||
assert.strictEqual(items[0].result, 'failed');
|
||||
assert.strictEqual(items[0].reason, 'User reported a 500');
|
||||
assert.strictEqual(items[0].test, 1);
|
||||
});
|
||||
|
||||
test('status: resolved row suppressed, open row kept', () => {
|
||||
const items = gapsItems([
|
||||
'| Truth | Status |',
|
||||
'|-------|--------|',
|
||||
'| closed thing | resolved |',
|
||||
'| open thing | failed |',
|
||||
].join('\n'));
|
||||
|
||||
assert.strictEqual(items.length, 1, JSON.stringify(items.map(i => i.name)));
|
||||
assert.strictEqual(items[0].name, 'open thing');
|
||||
});
|
||||
|
||||
test('no status column → surfaced as unknown, not dropped', () => {
|
||||
const items = gapsItems('| Truth | Note |\n|---|---|\n| something is off | see logs |');
|
||||
|
||||
assert.strictEqual(items.length, 1, JSON.stringify(items));
|
||||
assert.strictEqual(items[0].result, 'unknown');
|
||||
assert.strictEqual(items[0].name, 'something is off');
|
||||
});
|
||||
|
||||
test('unrecognizable header → joined cells + unknown status', () => {
|
||||
const items = gapsItems('| Alpha | Beta |\n|---|---|\n| xxx | yyy |');
|
||||
|
||||
assert.strictEqual(items.length, 1, JSON.stringify(items));
|
||||
assert.strictEqual(items[0].result, 'unknown');
|
||||
assert.match(items[0].name, /xxx/);
|
||||
assert.match(items[0].name, /yyy/);
|
||||
});
|
||||
|
||||
test('headerless table → explicit resolved cell still suppressed', () => {
|
||||
const items = gapsItems('| open thing | failed |\n| closed thing | resolved |');
|
||||
|
||||
assert.strictEqual(items.length, 1, JSON.stringify(items.map(i => i.name)));
|
||||
assert.match(items[0].name, /open thing/);
|
||||
});
|
||||
|
||||
test('bullets and a table in one Gaps section → union, no double-counting', () => {
|
||||
const items = gapsItems([
|
||||
'- truth: "a bullet gap"',
|
||||
' status: failed',
|
||||
'',
|
||||
'| Truth | Status |',
|
||||
'|---|---|',
|
||||
'| a table gap | failed |',
|
||||
].join('\n'));
|
||||
|
||||
assert.strictEqual(items.length, 2, JSON.stringify(items.map(i => i.name)));
|
||||
assert.ok(items.some(i => i.name === 'a bullet gap'));
|
||||
assert.ok(items.some(i => i.name === 'a table gap'));
|
||||
});
|
||||
|
||||
test('bullet-only Gaps unchanged (no regression on #2286)', () => {
|
||||
const items = gapsItems('- truth: "only a bullet"\n status: failed\n reason: "because"\n');
|
||||
|
||||
assert.strictEqual(items.length, 1, JSON.stringify(items));
|
||||
assert.strictEqual(items[0].name, 'only a bullet');
|
||||
assert.strictEqual(items[0].reason, 'because');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user