Files
msd-core/tests/progress-forensic.test.cjs
Jeremy McSpadden 77c7b4fc9d fix(#1522): enforce canonical verification before phase transition (#1548)
* fix: require fresh phase verification before transition

* no-mistakes(review): Fix canonical verification closeout gates

* no-mistakes(review): Fix verify-work frontmatter promotion command

* no-mistakes(review): Fix stale verification gates

* no-mistakes(review): Fix canonical verification routing gates

* no-mistakes(review): Fix verification dependency and runtime routing gates

* no-mistakes(review): Block stale verification bypasses

* fix: handle large init manager outputs in verification workflows

* chore: update changeset pr number

* fix(verify-work): use fresh verification.status for stale gate

The stale check after UAT used phase_completion.verification_status from
session-start INIT while human_needed promotion already queried fresh
verification.status. Align the stale gate with the canonical query so
mid-session verification refresh is not ignored.

* fix(init): skip roadmap-checked phases when selecting next_phase

Roadmap-only phases without a disk directory were still promoted to
next_phase when their checkbox was already checked. Exclude
checkboxComplete phases so progress routing does not point at work the
roadmap already marks done.

* fix: gaps_found not overridden by stale, transition uses canonical verification

- verification.cts: check gaps_found before stale so gap-closure routing
  is not masked by a newer summary mtime
- phase.cts: remove redundant findStaleVerificationSummary — readVerificationStatus
  already handles stale detection
- transition.md: replace raw grep on file content with verification.status query
  to avoid false-positive blocks from body text matching

* ci: retrigger tests after rebase

* fix(transition): replace gsd_run advisory check with awk frontmatter extraction

The runtime launcher is not defined until the update_roadmap_and_state step
bash block (~line 165). The early verify_completion block used gsd_run to
query verification.status, which violated the runtime-launcher-parity test:
'preamble appears AFTER the first gsd_run reference'.

Replace the gsd_run call with an awk-based frontmatter extractor that reads
only the status: field between the two --- fences. This avoids both the
preamble-ordering constraint and the original false-positive grep bug where
body text like 'previous_status: gaps_found' would match a full-text regex.

The phase.complete gate at update_roadmap_and_state is the canonical
enforcement point; this early check is advisory only.

Also update workflow-size-baseline.json for the updated transition.md size.

Fixes: runtime-launcher-parity test (B)

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>

* fix: re-check verification under planning lock in phase complete

Move readVerificationStatus into withPlanningLock so stale verification
cannot slip through when a SUMMARY.md is written between the gate and
the roadmap/state mutation. Return the blocked status from the lock
callback and emit the error after release to avoid leaving .lock behind.

* fix(transition): gate on canonical verification.status including stale

Replace awk frontmatter read with verification.status query so transition
blocks when summaries are newer than VERIFICATION.md, matching phase.complete
and other workflows (autonomous, progress, verify-work).

* Fix workflow verification gates for yolo transition and stale routing

Require VERIFY_STATUS passed before yolo/interactive transition advance.
Route stale verification recovery to verify-work, matching canonical projection.

* fix(transition): use verification.status query for stale-aware advisory check

The awk-based check read raw frontmatter status: passed, which misses the
stale case where summaries are newer than the VERIFICATION.md file even
though the frontmatter still says passed. The stale status is computed from
file modification times, not stored in frontmatter.

Move the preamble to the verify_completion bash block (the first block with
a gsd_run call) so gsd_run query verification.status can be used for the
advisory check. This gives the full readVerificationStatus logic including
mtime-based staleness detection, matching the enforcement gate at phase.complete.

Capture full JSON (VERIFY_JSON) so next_action can be included in the
advisory output alongside the status.

Also update workflow-size-baseline.json for the updated transition.md size.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>

* ci: trigger test matrix for 525b946

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>

* fix(transition): restore awk frontmatter extraction for pre-shim verification check

The gsd_run launcher shim is not defined until line ~163 of transition.md,
so the verification debt check at line ~80 cannot use gsd_run. Restore the
awk-based frontmatter extraction that correctly reads status without needing
the runtime, and restore the shim at its proper location before
phase.complete.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(#1522): clarify transition verification gate wording

* fix(#1522): update transition workflow size baseline

* fix(#1522): update workflow-size-baseline after rebase onto next

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>

* fix(#1522): guard findStaleVerificationSummary FS calls + thread opts.fs seam (review)

Address review blocker B1 on #1548: findStaleVerificationSummary ran fs.readdirSync
and two fs.statSync calls unguarded between readVerificationStatus's try/catch sections,
so a TOCTOU race (a SUMMARY listed by scanPhasePlans then removed before statSync) or any
FS error threw uncaught into callers NOT under the planning lock (init.manager /
init.progress / uat-predicate). Wrap the body in try/catch degrading to 'not stale', and
thread the injectable opts.fs seam (add statSync to FsLike, pass fsImpl from the caller)
for parity with readVerificationStatus's no-throw contract and testability. Also adds the
Verification Module glossary entry to CONTEXT.md (review B3).

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Tom Boucher <trekkie@nomorestars.com>
2026-06-24 13:19:10 -04:00

244 lines
9.8 KiB
JavaScript

// allow-test-rule: source-text-is-the-product
// Reads .md/.json/.yml product files whose deployed text IS what the
// runtime loads — testing text content tests the deployed contract.
/**
* Tests for --forensic flag on /gsd-progress (#2189)
*
* The --forensic flag appends a 6-check integrity audit after the standard
* progress report. Default behavior (no flag) is unchanged.
*/
const { test, describe } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('fs');
const path = require('path');
describe('#2189: progress --forensic flag', () => {
test('progress command argument-hint includes --forensic', () => {
const command = fs.readFileSync(
path.join(__dirname, '..', 'commands', 'gsd', 'progress.md'), 'utf8'
);
assert.ok(command.includes('--forensic'), 'argument-hint should include --forensic');
});
test('progress workflow has a forensic_audit step', () => {
const workflow = fs.readFileSync(
path.join(__dirname, '..', 'gsd-core', 'workflows', 'progress.md'), 'utf8'
);
assert.ok(
workflow.includes('<step name="forensic_audit">'),
'workflow should have a forensic_audit step'
);
});
test('forensic_audit step is only triggered when --forensic is present', () => {
const workflow = fs.readFileSync(
path.join(__dirname, '..', 'gsd-core', 'workflows', 'progress.md'), 'utf8'
);
const forensicStep = workflow.slice(
workflow.indexOf('<step name="forensic_audit">'),
workflow.indexOf('</step>', workflow.indexOf('<step name="forensic_audit">'))
);
assert.ok(
forensicStep.includes('--forensic'),
'forensic_audit step should be gated on --forensic flag'
);
assert.ok(
forensicStep.includes('Skip') || forensicStep.includes('skip') || forensicStep.includes('exit'),
'forensic_audit step should skip when --forensic is not present'
);
});
test('forensic_audit step includes all 6 checks', () => {
const workflow = fs.readFileSync(
path.join(__dirname, '..', 'gsd-core', 'workflows', 'progress.md'), 'utf8'
);
const forensicStep = workflow.slice(
workflow.indexOf('<step name="forensic_audit">'),
workflow.indexOf('</step>', workflow.indexOf('<step name="forensic_audit">'))
);
// Check 1: STATE vs artifact consistency
assert.ok(
forensicStep.includes('STATE') && (forensicStep.includes('artifact') || forensicStep.includes('consistent')),
'forensic step should check STATE vs artifact consistency (check 1)'
);
// Check 2: Orphaned handoff files
assert.ok(
forensicStep.includes('HANDOFF') || forensicStep.includes('handoff'),
'forensic step should check for orphaned handoff files (check 2)'
);
// Check 3: Deferred scope drift
assert.ok(
forensicStep.includes('deferred') || forensicStep.includes('defer'),
'forensic step should check for deferred scope drift (check 3)'
);
// Check 4: Memory-flagged pending work
assert.ok(
forensicStep.includes('MEMORY') || forensicStep.includes('memory') || forensicStep.includes('pending'),
'forensic step should check memory-flagged pending work (check 4)'
);
// Check 5: Blocking todos
assert.ok(
forensicStep.includes('todo') || forensicStep.includes('Todo') || forensicStep.includes('TODO'),
'forensic step should check blocking operational todos (check 5)'
);
// Check 6: Uncommitted code
assert.ok(
forensicStep.includes('uncommitted') || forensicStep.includes('git status'),
'forensic step should check for uncommitted code (check 6)'
);
});
test('forensic_audit step produces a CLEAN or INTEGRITY ISSUE(S) FOUND verdict', () => {
const workflow = fs.readFileSync(
path.join(__dirname, '..', 'gsd-core', 'workflows', 'progress.md'), 'utf8'
);
const forensicStep = workflow.slice(
workflow.indexOf('<step name="forensic_audit">'),
workflow.indexOf('</step>', workflow.indexOf('<step name="forensic_audit">'))
);
assert.ok(
forensicStep.includes('CLEAN'),
'forensic step should produce a CLEAN verdict when all checks pass'
);
assert.ok(
forensicStep.includes('INTEGRITY ISSUE') || forensicStep.includes('integrity issue'),
'forensic step should surface INTEGRITY ISSUE when checks fail'
);
});
test('forensic_audit step does not change default progress behavior', () => {
const workflow = fs.readFileSync(
path.join(__dirname, '..', 'gsd-core', 'workflows', 'progress.md'), 'utf8'
);
// The forensic step must explicitly say default behavior is unchanged
const forensicStep = workflow.slice(
workflow.indexOf('<step name="forensic_audit">'),
workflow.indexOf('</step>', workflow.indexOf('<step name="forensic_audit">'))
);
assert.ok(
forensicStep.includes('unchanged') || forensicStep.includes('standard report'),
'forensic step should clarify that default behavior is unchanged'
);
});
test('COMMANDS.md documents --forensic flag for gsd-progress', () => {
const commands = fs.readFileSync(
path.join(__dirname, '..', 'docs', 'COMMANDS.md'), 'utf8'
);
assert.ok(
commands.includes('--forensic'),
'COMMANDS.md should document --forensic flag for gsd-progress'
);
});
});
/**
* Regression — issue #1107
*
* /gsd-progress reported a phase as complete and routed to the next phase even
* when its VERIFICATION.md ended `human_needed` / `gaps_found`, because routing
* derived completeness from plan/summary counts only and never consulted the
* `verification.status` query (built in #651). The fix adds a Step 1.7 consult
* and routing rows that send non-`passed` phases back to close the debt.
*/
describe('#1107: progress routing consults verification.status before reporting complete', () => {
function readWorkflow() {
return fs.readFileSync(
path.join(__dirname, '..', 'gsd-core', 'workflows', 'progress.md'), 'utf8'
);
}
test('workflow consults verification.status for the current phase', () => {
const workflow = readWorkflow();
assert.ok(
workflow.includes('verification.status'),
'progress workflow must query verification.status (the #651 seam)'
);
assert.ok(
workflow.includes('verification_status'),
'progress workflow must track a verification_status value for routing'
);
assert.ok(
workflow.includes('stale verification'),
'progress workflow must document that verification.status projects stale verification'
);
});
test('routing table has gaps_found and human_needed rows BEFORE the generic complete row', () => {
const workflow = readWorkflow();
const missingIdx = workflow.indexOf('verification_status = missing');
const unknownIdx = workflow.indexOf('verification_status = unknown');
const staleIdx = workflow.indexOf('verification_status = stale');
const gapsIdx = workflow.indexOf('verification_status = gaps_found');
const humanIdx = workflow.indexOf('verification_status = human_needed');
const completeIdx = workflow.indexOf('Phase complete (verification passed)');
assert.ok(missingIdx > -1, 'routing table must have a missing verification row');
assert.ok(unknownIdx > -1, 'routing table must have an unknown verification row');
assert.ok(staleIdx > -1, 'routing table must have a stale verification row');
assert.ok(gapsIdx > -1, 'routing table must have a gaps_found row');
assert.ok(humanIdx > -1, 'routing table must have a human_needed row');
assert.ok(completeIdx > -1, 'routing table must keep a generic complete row');
assert.ok(
missingIdx < completeIdx &&
unknownIdx < completeIdx &&
staleIdx < completeIdx &&
gapsIdx < completeIdx &&
humanIdx < completeIdx,
'verification rows must precede the generic "summaries = plans" complete row (first-match-wins)'
);
});
test('gaps_found routes to plan-phase --gaps (Route V.gaps)', () => {
const workflow = readWorkflow();
// Anchor on the definition heading (`**Route V.gaps:`), not the routing-table
// reference (`Go to **Route V.gaps**`).
assert.ok(workflow.includes('**Route V.gaps:'), 'must define a Route V.gaps section');
const route = workflow.slice(
workflow.indexOf('**Route V.gaps:'),
workflow.indexOf('**Route V.human:')
);
assert.ok(
route.includes('--gaps') && route.includes('plan-phase'),
'Route V.gaps must route to /gsd:plan-phase {phase} --gaps'
);
});
test('human_needed routes to verify-work (Route V.human)', () => {
const workflow = readWorkflow();
assert.ok(workflow.includes('**Route V.human:'), 'must define a Route V.human section');
const route = workflow.slice(
workflow.indexOf('**Route V.human:'),
workflow.indexOf('**Step 3', workflow.indexOf('**Route V.human:'))
);
assert.ok(
route.includes('verify-work'),
'Route V.human must route to /gsd:verify-work {phase}'
);
});
test('stale verification routes to verify-work (Route V.stale)', () => {
const workflow = readWorkflow();
assert.ok(workflow.includes('**Route V.stale:'), 'must define a Route V.stale section');
const route = workflow.slice(
workflow.indexOf('**Route V.stale:'),
workflow.indexOf('**Route V.gaps:')
);
assert.ok(
route.includes('verify-work'),
'Route V.stale must route to /gsd:verify-work {phase}'
);
});
test('missing and unknown verification do not route as complete', () => {
const workflow = readWorkflow();
assert.ok(
workflow.includes('Phase complete (verification passed)'),
'the generic complete row must only cover passed verification'
);
assert.ok(!workflow.includes('verification passed, missing, or n/a'),
'missing or unknown verification must not be documented as complete');
});
});