golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

refactor(#651): consolidate verification-status routing into one queryable seam (#755)

Browse Source 
* refactor(#651): consolidate verification-status routing into one queryable seam

The passed/gaps_found/human_needed verification status was re-encoded as
bare strings across three prose surfaces (gsd-verifier emits, execute-phase
routes, ship gates), each independently deciding the per-status next action
with no parity coupling — the DEFECT.GENERATIVE-FIX class.

Give the enum one home: src/verification.cts (-> bin/lib/verification.cjs)
exposing `gsd_run query verification.status <phaseDir>` returning a typed
{status, next_action, next_command}. ship.md and execute-phase.md now consume
the query instead of re-deriving the routing in prose; gsd-verifier.md points
at the shared vocabulary as the single emitter (values unchanged).

Also fixes the latent broad-grep status misread (DEFECT.FRONTMATTER-SCALAR-
BROAD-GREP): execute-phase.md read `grep "^status:"` over the whole report, so
a body `status:` line could misroute a valid phase. Extraction is now
frontmatter-scoped in one place. A parity test fails if a verifier status
gains no route. Lands the two CONTEXT.md DEFECT entries captured on the issue.

Closes #651

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(#651): set changeset pr to 755

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-06-07 01:26:56 -04:00
committed by GitHub
parent f7e902f1cf
commit 4056d830bc
14 changed files with 679 additions and 128 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.changeset/651-verification-status-consolidation.md Normal file
View File

@@ -0,0 +1,5 @@
---
type: Changed
pr: 755
---
**Verification status routing is now owned by a single queryable seam** — `ship.md` and `execute-phase.md` both consume `gsd_run query verification.status` instead of re-deriving the `passed`/`gaps_found`/`human_needed` routing independently; the query returns `next_action` and `next_command` so per-status prose no longer needs to be kept in sync across files. This also fixes the broad-grep status misread in `execute-phase.md` where a body `status:` line (in a code block or copied artifact) could concatenate with the frontmatter value and misroute a valid passed phase; a parity test fails if a new verifier status value lacks a route. (#651)

2
.gitignore vendored
View File

@@ -106,6 +106,8 @@ build/
/gsd-core/bin/lib/state-document.cjs
/gsd-core/bin/lib/shell-command-projection.cjs
/gsd-core/bin/lib/security.cjs
/gsd-core/bin/lib/verification.cjs
/gsd-core/bin/lib/verification-command-router.cjs
/gsd-core/bin/lib/command-aliases.cjs
/gsd-core/bin/lib/config-schema.cjs
/gsd-core/bin/lib/model-profiles.cjs

9
CONTEXT.md
View File

@@ -526,6 +526,15 @@ The canonical lint infrastructure adopted in ADR 452 (`docs/adr/452-eslint-lint-
`DEFECT.GENERATIVE-FIX=for any new constant/array/parser shared between two parallel surfaces (two workflow surfaces, or a generated artifact and its hand-authored source), the same commit MUST add a parity assertion that fails when the two diverge`
`DEFECT.GENERATIVE-EXEMPLAR=tests/runtime-launcher-parity.test.cjs (asserts every workflow bash block uses the canonical gsd_run launcher — the in-repo pattern for enforcing equality across parallel surfaces)`
`DEFECT.FRONTMATTER-SCALAR-BROAD-GREP.symptom=a YAML-frontmatter scalar (e.g. VERIFICATION.md status) read with grep "^key:" over the WHOLE markdown report instead of the frontmatter block; a key: line in the body (code block, copied artifact, example) returns extra matches that concatenate after cut|tr into a value matching no expected token, so a valid state is misrouted`
`DEFECT.FRONTMATTER-SCALAR-BROAD-GREP.examples=#586/PR #650 ship.md verification gate — grep "^status:" also matched body status: lines, yielding passed+gaps_found+human_needed instead of passed and blocking a passed phase; the same broad-grep still lives in execute-phase.md (consolidation tracked by #651)`
`DEFECT.FRONTMATTER-SCALAR-BROAD-GREP.detect=grep "^<key>:" on a *.md whose result is compared to exact tokens, with no frontmatter scoping and no -m1; one body line beginning <key>: is enough to break it`
`DEFECT.FRONTMATTER-SCALAR-BROAD-GREP.fix-forward=scope to the leading frontmatter block and take the first match: sed -n '/^---$/,/^---$/p' "$f" | grep -m1 "^<key>:" | cut -d: -f2 | tr -d ' '; fix every parallel copy in the same change or consolidate behind one queryable seam (#651)`
`DEFECT.TEST-SHELL-PIPELINE-NONPORTABLE.symptom=a test that parses a workflow bash block out of a *.md and runs it via execFileSync('bash',...) breaks on Windows two ways: the fence regex uses a literal \n after the bash fence that will not match CRLF and trips windows-test-parity-guard (fenceRegexLiteralNewline); and git-bash exists so a bash-presence probe is true, but an os.tmpdir() Windows path (C:\...) is un-globbable in bash so the pipeline returns empty and assertions fail`
`DEFECT.TEST-SHELL-PIPELINE-NONPORTABLE.examples=#586/PR #650 tests/ship-586-verification-routing.test.cjs — the fence \n offender failed ubuntu-24/macos/coverage, then the Windows tmpdir-path glob failed full test (windows-latest,22) at fail 3; both were invisible to file-scoped gsd-test-both runs because the parity guard is only scanned by the full suite`
`DEFECT.TEST-SHELL-PIPELINE-NONPORTABLE.detect=test does readFileSync(md).match for a bash fence with literal \n, OR execFileSync('bash',...) gated only on a bash-presence probe; also verifying a new test with a file-scoped run instead of the full suite hides repo-wide static guards`
`DEFECT.TEST-SHELL-PIPELINE-NONPORTABLE.fix-forward=match the fence with \r?\n and normalize the captured block to LF; gate pipeline execution on process.platform !== 'win32' && hasBash since the extraction LOGIC is platform-independent and POSIX coverage suffices; run the full suite (or the parity/lint guards) before push when adding a test file`
---

2
agents/gsd-verifier.md
View File

@@ -576,6 +576,8 @@ Classify status using this decision tree IN ORDER (most restrictive first):
**passed is ONLY valid when the human verification section is empty.** If you identified items requiring human testing in Step 8, status MUST be human_needed.
> **Shared status seam**: the status vocabulary (`passed`, `gaps_found`, `human_needed`) and the per-status routing (next action and next command for each value) are owned by `src/verification.cts` via `gsd_run query verification.status`. This agent is the single emitter of the frontmatter status field; consumers (ship.md, execute-phase.md) read routing from that query instead of re-deriving it.
**Score:** `verified_truths / total_truths`
## Step 9b: Filter Deferred Items

2
docs/INVENTORY-MANIFEST.json
View File

@@ -344,6 +344,8 @@
"update-context.cjs",
"validate-command-router.cjs",
"validate.cjs",
"verification-command-router.cjs",
"verification.cjs",
"verify-command-router.cjs",
"verify.cjs",
"workstream-inventory-builder.cjs",

4
docs/INVENTORY.md
View File

@@ -370,7 +370,7 @@ The `gsd-planner` agent is decomposed into a core agent plus reference modules t
---
## CLI Modules (87 shipped)
## CLI Modules (89 shipped)
Full listing: `gsd-core/bin/lib/*.cjs`.
@@ -455,6 +455,8 @@ Full listing: `gsd-core/bin/lib/*.cjs`.
| `update-context.cjs` | Pure install-context resolver for `/gsd:update` — runtime/scope/config-dir/version detection (LOCAL/GLOBAL/UNKNOWN) ported from update.md bash; backs `gsd-tools update-context` (#498) |
| `validate-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools validate` |
| `validate.cjs` | Pure phase variant normalization helpers (`phaseVariants`, `buildRoadmapPhaseVariants`, `buildNotStartedPhaseVariants`) used by `verify.cjs` for W006/W007 checks; no I/O, no async |
| `verification-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools verification` |
| `verification.cjs` | Verification-status routing — consolidates pass/gaps_found/human_needed status from phase verifier-emitted VERIFICATION.md frontmatter (#651) |
| `verify-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools verify` |
| `verify.cjs` | Plan structure, phase completeness, reference, commit validation |
| `workstream-inventory-builder.cjs` | Pure workstream inventory projection builder |

2
eslint.config.mjs
View File

@@ -96,6 +96,8 @@ export default tseslint.config(
'gsd-core/bin/lib/config-types.cjs',
'gsd-core/bin/lib/phases-command-router.cjs',
'gsd-core/bin/lib/verify-command-router.cjs',
'gsd-core/bin/lib/verification.cjs',
'gsd-core/bin/lib/verification-command-router.cjs',
'gsd-core/bin/lib/init-command-router.cjs',
'gsd-core/bin/lib/agent-command-router.cjs',
'gsd-core/bin/lib/task-command-router.cjs',

22
gsd-core/bin/gsd-tools.cjs
View File

@@ -198,6 +198,8 @@ const learnings = require('./lib/learnings.cjs');
const gapChecker = require('./lib/gap-checker.cjs');
const { routeStateCommand } = require('./lib/state-command-router.cjs');
const { routeVerifyCommand } = require('./lib/verify-command-router.cjs');
const { routeVerificationCommand } = require('./lib/verification-command-router.cjs');
const verification = require('./lib/verification.cjs');
const { routeInitCommand } = require('./lib/init-command-router.cjs');
const { routePhaseCommand } = require('./lib/phase-command-router.cjs');
const { routePhasesCommand } = require('./lib/phases-command-router.cjs');
@@ -776,6 +778,26 @@ async function runCommand(command, args, cwd, raw, defaultValue, originalCommand
break;
}
// ─── Verification Status ───────────────────────────────────────────────
//
// verification status <phaseDir>
// Read the first *-VERIFICATION.md in phaseDir and return
// { status, next_action, next_command } routing result.
//
// Note: `verification` (reads verifier-emitted status) is distinct from
// `verify` (runs verification checks like plan-structure/artifacts).
case 'verification': {
routeVerificationCommand({
verification,
args,
cwd,
raw,
error,
});
break;
}
case 'generate-slug': {
// Phase 6 (#3575): dispatch via SDK executeForCjs when available.
// SDK handler: generateSlug in sdk/src/query/utils.ts.

13
gsd-core/workflows/execute-phase.md
View File

@@ -1425,16 +1425,15 @@ ${VERIFIER_SKILLS}",
> **ORCHESTRATOR RULE — CODEX RUNTIME**: After calling Agent() above, stop working on this task immediately. Do not read more files, edit code, or run tests related to this task while the subagent is active. Wait for the subagent to return its result. This prevents duplicate work, conflicting edits, and wasted context. Only resume when the subagent result is available.
Read status:
Read status via the canonical query (scoped to frontmatter, covers missing/unknown cases):
```bash
grep "^status:" "$PHASE_DIR"/*-VERIFICATION.md | cut -d: -f2 | tr -d ' '
VERIFICATION=$(gsd_run query verification.status "$PHASE_DIR" 2>/dev/null)
STATUS=$(printf '%s' "$VERIFICATION" | jq -r '.status' 2>/dev/null || echo "")
NEXT_ACTION=$(printf '%s' "$VERIFICATION" | jq -r '.next_action' 2>/dev/null || echo "")
NEXT_COMMAND=$(printf '%s' "$VERIFICATION" | jq -r '.next_command' 2>/dev/null || echo "")
```
| Status | Action |
|--------|--------|
| `passed` | → update_roadmap |
| `human_needed` | Persist and present human testing items; keep phase pending until verification reruns as `passed` |
| `gaps_found` | Present gap summary, offer `/gsd:plan-phase {phase} --gaps ${GSD_WS}` |
Route on `$STATUS`: if `passed`, proceed to update_roadmap. Otherwise keep the phase pending — present `$NEXT_ACTION` to the user and, when `$NEXT_COMMAND` is non-empty, show it as the next command to run. The query covers all cases including missing files (`missing`) and unexpected values (`unknown`), so no per-status arm needs to be listed here.
**If human_needed:**

13
gsd-core/workflows/ship.md
View File

@@ -41,15 +41,12 @@ Verify the work is ready to ship:
1. **Verification passed?**
```bash
VERIFICATION_FILE=$(ls ${PHASE_DIR}/*-VERIFICATION.md 2>/dev/null | head -1)
STATUS=$(sed -n '/^---$/,/^---$/p' "${VERIFICATION_FILE}" 2>/dev/null | grep -m1 "^status:" | cut -d: -f2 | tr -d ' ')
VERIFICATION=$(gsd_run query verification.status "${PHASE_DIR}" 2>/dev/null)
STATUS=$(printf '%s' "$VERIFICATION" | jq -r '.status' 2>/dev/null || echo "")
NEXT_ACTION=$(printf '%s' "$VERIFICATION" | jq -r '.next_action' 2>/dev/null || echo "")
NEXT_COMMAND=$(printf '%s' "$VERIFICATION" | jq -r '.next_command' 2>/dev/null || echo "")
```
The verifier emits exactly `passed`, `gaps_found`, or `human_needed` (see the status table in `execute-phase.md`); only `passed` may ship. Route on `${STATUS}` — on any non-`passed` value, block with `PHASE_VERIFICATION_INCOMPLETE` and state the matching next action:
- `passed` → verification complete; continue to the next preflight check.
- `gaps_found` → run `/gsd:plan-phase ${PHASE_NUMBER} --gaps` to plan the fixes, then re-run `/gsd:execute-phase` before shipping.
- `human_needed` → complete the manual tests in `${PHASE_DIR}/*-UAT.md`, then re-run the verify step until status is `passed`.
- empty (no `*-VERIFICATION.md`) → the verify step never completed; re-run `/gsd:execute-phase`.
- any other value → unexpected status `${STATUS}`; re-run `/gsd:execute-phase` verification.
Only `passed` may ship. If `$STATUS` is `passed`, verification is complete — continue to the next preflight check. Any other value (including `gaps_found`, `human_needed`, `missing`, and `unknown`) blocks with `PHASE_VERIFICATION_INCOMPLETE`: present `$NEXT_ACTION` to the user and, when `$NEXT_COMMAND` is non-empty, show it as the command to run next. The query already handles missing files and unexpected values, so no per-status arm is needed.
2. **Clean working tree?**
```bash

56
src/verification-command-router.cts Normal file
View File

@@ -0,0 +1,56 @@
/**
* Verification-status subcommand router.
* Routes `verification.status <phaseDir>` to verification.cmdVerificationStatus.
*
* Note: `verification` (reads verifier-emitted status) is distinct from `verify`
* (runs verification checks like plan-structure/artifacts). Keep them separate.
*
* ADR-457 build-at-publish: source in src/verification-command-router.cts,
* compiled to gsd-core/bin/lib/verification-command-router.cjs (gitignored).
*/
// eslint-disable-next-line @typescript-eslint/no-require-imports
import cjsCommandRouterAdapter = require('./cjs-command-router-adapter.cjs');
const { routeCjsCommandFamily } = cjsCommandRouterAdapter;
// ─── Types ────────────────────────────────────────────────────────────────────
interface VerificationModule {
cmdVerificationStatus(cwd: string, phaseDirArg: string | undefined, raw: boolean): void;
}
interface RouteVerificationCommandOptions {
verification: VerificationModule;
args: string[];
cwd: string;
raw: boolean;
error: (message: string) => void;
}
// ─── Implementation ───────────────────────────────────────────────────────────
const VERIFICATION_SUBCOMMANDS = ['status'];
function routeVerificationCommand({
verification,
args,
cwd,
raw,
error,
}: RouteVerificationCommandOptions): void {
routeCjsCommandFamily({
args,
subcommands: VERIFICATION_SUBCOMMANDS,
unsupported: {},
error,
unknownMessage: (_subcommand: string, available: string[]) =>
`Unknown verification subcommand. Available: ${available.join(', ')}`,
handlers: {
status: () => verification.cmdVerificationStatus(cwd, args[2], raw),
},
});
}
export = {
routeVerificationCommand,
};

234
src/verification.cts Normal file
View File

@@ -0,0 +1,234 @@
/**
* Verification Status — single queryable home for verification-status routing.
*
* Issue #651: consolidate the pass/gaps_found/human_needed routing that was
* previously scattered across ship.md and execute-phase.md into a single
* tested module. Both workflow files will later consume this module's routing
* table as the single source of truth.
*
* ADR-457 build-at-publish: source in src/verification.cts, compiled to
* gsd-core/bin/lib/verification.cjs (gitignored).
*
* DEFECT.FRONTMATTER-SCALAR-BROAD-GREP fix: status extraction is scoped to
* the leading YAML frontmatter block only. A `status:` line in the body (e.g.
* inside a fenced code block) is ignored — this is the exact failure mode that
* issue #586 / PR #650 identified. The shared extractFrontmatter parser anchors
* its regex at byte 0 of the document, which provides this guarantee.
*/
import fs from 'node:fs';
import path from 'node:path';
// eslint-disable-next-line @typescript-eslint/no-require-imports -- core.cjs is an export= CommonJS module
import core = require('./core.cjs');
// eslint-disable-next-line @typescript-eslint/no-require-imports -- frontmatter.cjs is an export= CommonJS module
import frontmatterMod = require('./frontmatter.cjs');
const { output, extractPhaseToken } = core;
const { extractFrontmatter } = frontmatterMod;
// ─── Constants ────────────────────────────────────────────────────────────────
/** The set of status values that the gsd-verifier agent emits. */
const VERIFIER_STATUSES: ReadonlyArray<string> = ['passed', 'gaps_found', 'human_needed'];
// ─── Routing table ────────────────────────────────────────────────────────────
interface VerificationRoute {
status: string;
next_action: string;
next_command: string;
}
/**
* Canonical routing table for verification statuses.
*
* This is the single source of truth — ship.md and execute-phase.md will
* later import from here instead of embedding their own message strings.
*
* INTERNAL SENTINELS: 'missing' and 'unknown' are operational states constructed
* internally — the verifier (gsd-verifier.md) never emits them. The verifier only
* emits values in VERIFIER_STATUSES (passed|gaps_found|human_needed). The guard in
* readVerificationStatus excludes 'missing' and 'unknown' from raw-status table
* lookup so they can only be reached via internal construction paths.
*
* For 'gaps_found', next_command is built at call time in readVerificationStatus
* by substituting the phase number — it is NOT stored as a function in the table.
*/
const VERIFICATION_ROUTING_TABLE: Record<string, VerificationRoute> = {
passed: {
status: 'passed',
next_action: 'Verification passed — continue.',
next_command: '',
},
gaps_found: {
status: 'gaps_found',
next_action: 'Gaps found. Plan the fixes, then re-run execute-phase before shipping.',
// next_command is computed at call time; this entry is never returned directly.
next_command: '',
},
human_needed: {
status: 'human_needed',
next_action: "Human verification required. Complete the manual tests in the phase's *-UAT.md, then re-run the verify step until status is passed.",
next_command: '',
},
// INTERNAL SENTINEL: constructed when no *-VERIFICATION.md file exists or when
// the file has no parseable frontmatter status. Never emitted by the verifier.
missing: {
status: 'missing',
next_action: 'No verification report found — the verify step never completed. Re-run execute-phase.',
next_command: '/gsd:execute-phase',
},
// INTERNAL SENTINEL: constructed when the file has a status value not in
// VERIFIER_STATUSES. Never emitted by the verifier.
unknown: {
status: 'unknown',
next_action: '', // filled in dynamically with the raw value
next_command: '/gsd:execute-phase',
},
};
// ─── Helpers ─────────────────────────────────────────────────────────────────
interface FsLike {
readdirSync(dir: string): string[];
readFileSync(filePath: string, encoding: 'utf-8'): string;
}
/**
* Build a 'missing' result from the routing table.
* Used for two early-return paths: no *-VERIFICATION.md file found, and
* file present but no parseable frontmatter status.
*/
function missingResult(): VerificationStatusResult {
const route = VERIFICATION_ROUTING_TABLE['missing'];
return {
status: route.status,
next_action: route.next_action,
next_command: route.next_command,
};
}
// ─── Public API ───────────────────────────────────────────────────────────────
interface ReadVerificationStatusOptions {
fs?: FsLike;
}
interface VerificationStatusResult {
status: string;
next_action: string;
next_command: string;
}
/**
* Read the verification status from the first `*-VERIFICATION.md` file in
* phaseDir and return the routing result.
*
* Behavior:
* 1. Find the first file matching `*-VERIFICATION.md` (sorted, take first).
* If none → status 'missing'.
* 2. Extract `status` from FRONTMATTER ONLY via the shared extractFrontmatter
* parser (DEFECT.FRONTMATTER-SCALAR-BROAD-GREP fix — parser anchors at byte 0).
* If no frontmatter block or no `status` key → status 'missing'.
* 3. Map to routing table. Unknown non-empty value → status 'unknown'.
*
* @param phaseDir - Absolute path to the phase directory.
* @param opts - Options. `opts.fs` allows test injection (defaults to node:fs).
*/
function readVerificationStatus(
phaseDir: string,
opts: ReadVerificationStatusOptions = {},
): VerificationStatusResult {
const fsImpl: FsLike = opts.fs ?? fs;
// Phase token for the gaps_found command
const baseName = path.basename(phaseDir);
const phaseToken = extractPhaseToken(baseName);
const phaseNumber = phaseToken.length > 0 ? phaseToken : baseName;
// 1. Find *-VERIFICATION.md
let verificationFile: string | null = null;
try {
const entries = fsImpl.readdirSync(phaseDir);
const candidates = entries.filter((f) => f.endsWith('-VERIFICATION.md')).sort();
verificationFile = candidates.length > 0 ? candidates[0] : null;
} catch {
// Directory unreadable → treat as missing
verificationFile = null;
}
if (!verificationFile) {
return missingResult();
}
// 2. Read and parse frontmatter using the shared parser.
// extractFrontmatter anchors at byte 0, so body `status:` lines are ignored.
const filePath = path.join(phaseDir, verificationFile);
let rawStatus: string | null = null;
try {
const content = fsImpl.readFileSync(filePath, 'utf-8');
const fm = extractFrontmatter(content);
const statusVal = fm['status'];
// status is always a scalar string in a well-formed VERIFICATION.md frontmatter;
// only accept string values — arrays and objects are not valid status values.
if (typeof statusVal === 'string') {
const trimmed = statusVal.trim();
rawStatus = trimmed.length > 0 ? trimmed : null;
}
} catch {
rawStatus = null;
}
if (!rawStatus) {
return missingResult();
}
// 3. Route — exclude internal sentinels from raw-file lookup (they are
// constructed internally above, never written by the verifier).
if (rawStatus in VERIFICATION_ROUTING_TABLE && rawStatus !== 'missing' && rawStatus !== 'unknown') {
const entry = VERIFICATION_ROUTING_TABLE[rawStatus];
// gaps_found: build the phase-specific command here rather than in the table.
const next_command =
rawStatus === 'gaps_found'
? `/gsd:plan-phase ${phaseNumber} --gaps`
: entry.next_command;
return {
status: entry.status,
next_action: entry.next_action,
next_command,
};
}
// Unknown value
const unknownRoute = VERIFICATION_ROUTING_TABLE['unknown'];
return {
status: unknownRoute.status,
next_action: `Unexpected verification status '${rawStatus}'. Re-run execute-phase verification.`,
next_command: unknownRoute.next_command,
};
}
/**
* CLI command handler: resolve phaseDir against cwd, call readVerificationStatus,
* emit via core.output().
*
* @param cwd - Current working directory (used to resolve phaseDirArg).
* @param phaseDirArg - Phase directory path (absolute or relative to cwd).
* @param raw - Whether to emit raw (non-JSON) output.
*/
function cmdVerificationStatus(cwd: string, phaseDirArg: string | undefined, raw: boolean): void {
if (!phaseDirArg) {
core.error('phase directory required for verification.status');
return;
}
const phaseDir = path.resolve(cwd, phaseDirArg);
const result = readVerificationStatus(phaseDir);
output(result, raw);
}
export = {
VERIFIER_STATUSES,
VERIFICATION_ROUTING_TABLE,
readVerificationStatus,
cmdVerificationStatus,
};

112
tests/ship-586-verification-routing.test.cjs
View File

@@ -1,112 +0,0 @@
'use strict';
// allow-test-rule: runtime-contract-is-the-product
// The ship.md verification gate is LLM-executed prose; its routing message text
// IS the user-facing product surface (RULESET.TESTS.no-source-grep.exemption:
// "reserved for tests where the file content IS the product surface ... agent .md").
// The behavioral tests below additionally EXECUTE the gate's own bash extraction
// pipeline (parsed out of ship.md) against fixture reports, so the extraction
// contract is verified, not just asserted as text.
const test = require('node:test');
const assert = require('node:assert');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');
const { execFileSync } = require('node:child_process');
const helpers = require('./helpers.cjs');
const SHIP_MD = path.join(__dirname, '..', 'gsd-core', 'workflows', 'ship.md');
const ship = fs.readFileSync(SHIP_MD, 'utf8');
const start = ship.indexOf('**Verification passed?**');
const end = ship.indexOf('**Clean working tree?**');
assert.ok(start !== -1 && end !== -1 && end > start, 'could not locate the verification gate block');
const gate = ship.slice(start, end);
// ---- content assertions (the routing message IS the product surface) ----
test('gate captures the status value with a single first-match grep', () => {
assert.match(gate, /grep -m1 "\^status:"/, 'gate must extract status via grep -m1 "^status:"');
});
test('gate scopes status extraction to the YAML frontmatter only', () => {
assert.match(gate, /sed -n '\/\^---\$\/,\/\^---\$\/p'/, 'gate must restrict extraction to the frontmatter block');
});
test('gate routes gaps_found to /gsd:plan-phase --gaps', () => {
assert.match(gate, /gaps_found/);
assert.match(gate, /\/gsd:plan-phase[^\n]*--gaps/);
});
test('gate routes human_needed to the UAT manual-test step', () => {
assert.match(gate, /human_needed/);
assert.match(gate, /UAT\.md/);
});
test('gate routes a missing VERIFICATION.md to re-running execute-phase', () => {
assert.match(gate, /\/gsd:execute-phase/);
});
test('gate still blocks with PHASE_VERIFICATION_INCOMPLETE', () => {
assert.match(gate, /PHASE_VERIFICATION_INCOMPLETE/);
});
test('the dead `pass` status arm is gone — only `passed` is accepted', () => {
assert.doesNotMatch(gate, /status:\s*pass(?!ed)/i, 'no bare `status: pass` arm may remain');
assert.doesNotMatch(gate, /`pass`\s*\/\s*`passed`/, 'the `pass` / `passed` either-arm must be removed');
assert.match(gate, /passed/);
});
// ---- behavioral tests: run the gate's OWN bash pipeline against fixtures ----
const bashBlock = (() => {
// `\r?\n` (not a literal `\n`) so the fence matches on Windows CRLF checkouts;
// normalize the captured block to LF before handing it to bash.
const m = gate.match(/```bash\r?\n([\s\S]*?)```/);
assert.ok(m, 'gate must contain a bash block');
return m[1].replace(/\r\n/g, '\n');
})();
const hasBash = (() => {
try { execFileSync('bash', ['-c', 'true'], { stdio: 'ignore' }); return true; }
catch { return false; }
})();
// The extraction logic is platform-independent; the bash *pipeline* is executed
// only where the gate's shell actually runs (POSIX). On Windows, git-bash exists
// but receives Windows-style tmpdir paths it cannot glob, so skip execution there.
const skipBashPipeline = (process.platform === 'win32' || !hasBash) && 'bash pipeline runs on POSIX only';
function runGateExtraction(verificationContents) {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ship586-'));
try {
if (verificationContents !== null) {
fs.writeFileSync(path.join(dir, '01-VERIFICATION.md'), verificationContents);
}
const script = `PHASE_DIR='${dir}'\n${bashBlock}\nprintf '%s' "$STATUS"`;
return execFileSync('bash', ['-c', script], { encoding: 'utf8' });
} finally {
helpers.cleanup(dir);
}
}
const FM = (status) =>
`---\nphase: 01-demo\nverified: 2026-01-01T00:00:00Z\nstatus: ${status}\nscore: 3/3 must-haves verified\n---\n\n# Verification\n`;
test('extraction yields passed for a passing frontmatter', { skip: skipBashPipeline }, () => {
assert.strictEqual(runGateExtraction(FM('passed')), 'passed');
});
test('extraction yields gaps_found / human_needed verbatim', { skip: skipBashPipeline }, () => {
assert.strictEqual(runGateExtraction(FM('gaps_found')), 'gaps_found');
assert.strictEqual(runGateExtraction(FM('human_needed')), 'human_needed');
});
test('REGRESSION: a body `status:` line does not corrupt a passing report (Codex PR #650 finding)', { skip: skipBashPipeline }, () => {
const withBodyStatus = FM('passed') +
'\n## Example\n\n```yaml\nstatus: gaps_found\n```\n\nstatus: human_needed\n';
assert.strictEqual(runGateExtraction(withBodyStatus), 'passed');
});
test('extraction yields empty when no VERIFICATION.md exists', { skip: skipBashPipeline }, () => {
assert.strictEqual(runGateExtraction(null), '');
});

331
tests/verification-status.test.cjs Normal file
View File

@@ -0,0 +1,331 @@
'use strict';
/**
* Tests for verification-status module (issue #651).
*
* Covers:
* 1. status: passed → routing
* 2. status: gaps_found with phase token extraction
* 3. status: human_needed → routing
* 4. No *-VERIFICATION.md → 'missing'
* 5. Frontmatter status present but unknown value → 'unknown'
* 6. BROAD-GREP REGRESSION: body `status:` lines ignored, frontmatter wins
* 7. PARITY: VERIFIER_STATUSES covered by routing table; gsd-verifier.md emitted statuses covered
* 8. CRLF line endings in frontmatter
* 9. Body-only file (no frontmatter block) → missing
* 10. Nonexistent phase directory → missing
* 11. Multiple *-VERIFICATION.md files → first by sort
* 12. ship.md PHASE_VERIFICATION_INCOMPLETE sentinel (contract anchor for #651 consolidation)
*
* PORTABILITY: pure JS — no shell-outs, no bash fences.
* Cross-platform (passes on Windows). Ref: DEFECT.TEST-SHELL-PIPELINE-NONPORTABLE.
*/
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const os = require('node:os');
const { cleanup } = require('./helpers.cjs');
const {
VERIFIER_STATUSES,
VERIFICATION_ROUTING_TABLE,
readVerificationStatus,
} = require('../gsd-core/bin/lib/verification.cjs');
// ─── Helpers ─────────────────────────────────────────────────────────────────
/**
* Create a temporary phase directory under os.tmpdir().
* Returns the absolute path; caller must clean up.
*/
function mkPhaseDir(suffix) {
return fs.mkdtempSync(path.join(os.tmpdir(), `gsd-651-${suffix}-`));
}
/**
* Write a *-VERIFICATION.md file with the given frontmatter status and
* optional body content.
*
* @param {string} dir - Phase directory path
* @param {string} filename - e.g. '01-review-VERIFICATION.md'
* @param {string} status - Frontmatter status value
* @param {string} [body] - Content after the closing `---`
*/
function writeVerificationMd(dir, filename, status, body = '') {
const frontmatter = `---\nstatus: ${status}\n---\n`;
fs.writeFileSync(path.join(dir, filename), frontmatter + body);
}
// ─── Tests ────────────────────────────────────────────────────────────────────
describe('verification-status', () => {
// ── Case 1: passed ────────────────────────────────────────────────────────
test('status: passed → next_command is empty, status is passed', () => {
const dir = mkPhaseDir('passed');
try {
writeVerificationMd(dir, '01-foo-VERIFICATION.md', 'passed');
const result = readVerificationStatus(dir);
assert.equal(result.status, 'passed', 'status must be passed');
assert.equal(result.next_command, '', 'next_command must be empty for passed');
assert.ok(result.next_action.length > 0, 'next_action must be non-empty');
} finally {
cleanup(dir);
}
});
// ── Case 2: gaps_found with phase token extraction ────────────────────────
test('status: gaps_found in "03-foo" dir → next_command includes phase token 03', () => {
// Phase dir basename starts with "03" — extractPhaseToken('03-foo') → '03'
const baseDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-651-parent-'));
const phaseDir = path.join(baseDir, '03-foo');
fs.mkdirSync(phaseDir);
try {
writeVerificationMd(phaseDir, '03-foo-VERIFICATION.md', 'gaps_found');
const result = readVerificationStatus(phaseDir);
assert.equal(result.status, 'gaps_found', 'status must be gaps_found');
assert.ok(
result.next_command.includes('03'),
`next_command should include phase token '03'; got: ${result.next_command}`,
);
assert.ok(
result.next_command.includes('--gaps'),
`next_command should include --gaps; got: ${result.next_command}`,
);
assert.equal(result.next_command, '/gsd:plan-phase 03 --gaps');
} finally {
cleanup(baseDir);
}
});
// ── Case 3: human_needed ──────────────────────────────────────────────────
test('status: human_needed → status human_needed, next_command is empty', () => {
const dir = mkPhaseDir('human-needed');
try {
writeVerificationMd(dir, '01-hn-VERIFICATION.md', 'human_needed');
const result = readVerificationStatus(dir);
assert.equal(result.status, 'human_needed');
assert.equal(result.next_command, '');
assert.ok(result.next_action.length > 0);
} finally {
cleanup(dir);
}
});
// ── Case 4: no *-VERIFICATION.md → missing ────────────────────────────────
test('no *-VERIFICATION.md file → status missing, next_command execute-phase', () => {
const dir = mkPhaseDir('missing');
try {
// write a non-matching file to confirm it is ignored
fs.writeFileSync(path.join(dir, 'README.md'), '# phase');
const result = readVerificationStatus(dir);
assert.equal(result.status, 'missing');
assert.equal(result.next_command, '/gsd:execute-phase');
assert.ok(result.next_action.includes('verify step never completed'));
} finally {
cleanup(dir);
}
});
// ── Case 5: unknown frontmatter status value ──────────────────────────────
test("frontmatter status 'bogus' → status unknown, next_command execute-phase", () => {
const dir = mkPhaseDir('unknown');
try {
writeVerificationMd(dir, '01-u-VERIFICATION.md', 'bogus');
const result = readVerificationStatus(dir);
assert.equal(result.status, 'unknown');
assert.equal(result.next_command, '/gsd:execute-phase');
assert.ok(
result.next_action.includes('bogus'),
`next_action should mention the raw value; got: ${result.next_action}`,
);
} finally {
cleanup(dir);
}
});
// ── Case 6: BROAD-GREP REGRESSION (critical) ──────────────────────────────
//
// Frontmatter: `status: passed`
// Body: a fenced code block containing `status: gaps_found` AND `status: human_needed`
// Result MUST be 'passed' — proving body lines are NOT matched.
// This is the exact failure mode that issue #586 / PR #650 hit.
//
test('BROAD-GREP REGRESSION: body status lines ignored, frontmatter status wins', () => {
const dir = mkPhaseDir('broad-grep');
try {
const bodyWithEmbeddedStatuses = [
'',
'## Section',
'',
'Some prose about the results.',
'',
'```yaml',
'status: gaps_found',
'gaps:',
' - fix the thing',
'```',
'',
'Another block:',
'',
'```',
'status: human_needed',
'```',
'',
'End of document.',
].join('\n');
writeVerificationMd(dir, '01-bg-VERIFICATION.md', 'passed', bodyWithEmbeddedStatuses);
const result = readVerificationStatus(dir);
assert.equal(
result.status,
'passed',
`Expected status 'passed' (frontmatter wins); got '${result.status}'. ` +
'Body status: lines must NOT be matched.',
);
assert.equal(result.next_command, '', 'next_command must be empty for passed');
} finally {
cleanup(dir);
}
});
// ── Case 7: PARITY ASSERTION ──────────────────────────────────────────────
//
// (a) Every value in VERIFIER_STATUSES has a corresponding key in VERIFICATION_ROUTING_TABLE.
// (b) Parse agents/gsd-verifier.md for emitted statuses via /→ \*\*status:\s*([a-z_]+)\*\*/g,
// collect the set, and assert every emitted status is a routing key.
//
test('PARITY: VERIFIER_STATUSES covered by routing table', () => {
for (const s of VERIFIER_STATUSES) {
assert.ok(
s in VERIFICATION_ROUTING_TABLE,
`VERIFIER_STATUS '${s}' has no entry in VERIFICATION_ROUTING_TABLE`,
);
}
});
test('PARITY: gsd-verifier.md emitted statuses all have routing table entries', () => {
const verifierPath = path.join(__dirname, '..', 'agents', 'gsd-verifier.md');
const content = fs.readFileSync(verifierPath, 'utf-8');
const emittedStatuses = new Set();
// Source (a): decision-tree arrow lines — `→ **status: <value>**`
// These are the per-branch emission points in Step 9 (the decision tree).
const reArrow = /→ \*\*status:\s*([a-z_]+)\*\*/g;
let m;
while ((m = reArrow.exec(content)) !== null) {
emittedStatuses.add(m[1]);
}
// Source (b): output-template line — `status: A | B | C` (pipe-delimited list
// of permitted values inside the frontmatter template block in the <output> section).
// Anchored to lines that start with `status:` and contain `|` to avoid false
// matches on prose sentences that happen to mention "status:".
const reTemplate = /^status:\s+([a-z_]+(?:\s*\|\s*[a-z_]+)+)\s*$/gm;
while ((m = reTemplate.exec(content)) !== null) {
for (const token of m[1].split('|')) {
const t = token.trim();
if (t) emittedStatuses.add(t);
}
}
assert.ok(
emittedStatuses.size > 0,
'No emitted statuses found in gsd-verifier.md — regex or file path may be wrong. ' +
'Checked: (a) → **status: X** arrow lines, (b) status: A | B | C template lines.',
);
for (const s of emittedStatuses) {
assert.ok(
s in VERIFICATION_ROUTING_TABLE,
`gsd-verifier.md emits status '${s}' but VERIFICATION_ROUTING_TABLE has no entry for it. ` +
'Add a route or remove/rename the status in gsd-verifier.md.',
);
}
});
// ── Edge cases ────────────────────────────────────────────────────────────
// CRLF line endings in frontmatter
test('CRLF line endings in frontmatter → correct status parsed', () => {
const dir = mkPhaseDir('crlf');
try {
// Construct a file with CRLF line endings throughout
const content = '---\r\nstatus: passed\r\nphase: 01-demo\r\n---\r\n\r\n# Body\r\n';
fs.writeFileSync(path.join(dir, '01-crlf-VERIFICATION.md'), content);
const result = readVerificationStatus(dir);
assert.equal(result.status, 'passed', 'CRLF frontmatter must parse to passed');
assert.equal(result.next_command, '');
} finally {
cleanup(dir);
}
});
// File with NO frontmatter block — body-only `status:` line must NOT be matched
test('body-only file with no frontmatter block (status: in body) → missing', () => {
const dir = mkPhaseDir('no-fm');
try {
// No opening `---` — this is a plain markdown file with a status: line in the body
const content = '# Phase Verification\n\nstatus: passed\n\nSome notes.\n';
fs.writeFileSync(path.join(dir, '01-nofm-VERIFICATION.md'), content);
const result = readVerificationStatus(dir);
assert.equal(
result.status,
'missing',
"A body-only status: line must NOT be read — result should be 'missing'",
);
} finally {
cleanup(dir);
}
});
// Missing / nonexistent phase directory → missing
test('nonexistent phase directory → missing', () => {
const nonexistent = path.join(os.tmpdir(), 'gsd-651-nonexistent-' + Date.now());
const result = readVerificationStatus(nonexistent);
assert.equal(result.status, 'missing', 'unreadable/nonexistent dir must return missing');
assert.equal(result.next_command, '/gsd:execute-phase');
});
// Multiple *-VERIFICATION.md files → deterministic pick (first by sort)
test('multiple *-VERIFICATION.md files in dir → first by sort order wins', () => {
const dir = mkPhaseDir('multi');
try {
// Write two files: alphabetically "01-a" comes before "02-b"
// "01-a" has passed; "02-b" has gaps_found — first by sort must win
const fm = (status) => `---\nstatus: ${status}\n---\n`;
fs.writeFileSync(path.join(dir, '01-a-VERIFICATION.md'), fm('passed'));
fs.writeFileSync(path.join(dir, '02-b-VERIFICATION.md'), fm('gaps_found'));
const result = readVerificationStatus(dir);
assert.equal(
result.status,
'passed',
'When multiple *-VERIFICATION.md files exist, the first by lexicographic sort must be used',
);
} finally {
cleanup(dir);
}
});
// ── Task 2 (B1): ship.md gate sentinel contract anchor ────────────────────
//
// The deleted tests/ship-586-verification-routing.test.cjs was the only
// thing asserting that ship.md emits the PHASE_VERIFICATION_INCOMPLETE block
// sentinel (its user-visible gate error key). This test re-anchors that contract.
//
test('ship.md still emits the PHASE_VERIFICATION_INCOMPLETE gate sentinel (contract anchor for #651 consolidation)', () => {
const shipMdPath = path.join(__dirname, '..', 'gsd-core', 'workflows', 'ship.md');
const content = fs.readFileSync(shipMdPath, 'utf-8');
assert.ok(
content.includes('PHASE_VERIFICATION_INCOMPLETE'),
'ship.md must contain the literal PHASE_VERIFICATION_INCOMPLETE gate sentinel. ' +
'If you renamed or removed it, update the verification routing and this contract test.',
);
});
});