test: align pinned workflow text with the uat canonicalize seam
Some checks failed
Tests / conformance test (macos-latest, 24) (pull_request) Blocked by required conditions
Tests / conformance test (windows-latest, 24, shard 1/3) (pull_request) Blocked by required conditions
Tests / conformance test (windows-latest, 24, shard 2/3) (pull_request) Blocked by required conditions
Tests / conformance test (windows-latest, 24, shard 3/3) (pull_request) Blocked by required conditions
Tests / Required tests (pull_request) Blocked by required conditions
Validate Branch Name / PR mergeability (pull_request) Successful in 24s
Changeset Required / PR mergeability (pull_request) Successful in 10s
Default Flip Documentation / PR mergeability (pull_request) Successful in 9s
Dependabot Auto-Merge / auto-merge (pull_request) Has been skipped
Docs Required / PR mergeability (pull_request) Successful in 10s
Mutation Testing / PR mergeability (pull_request) Successful in 10s
Security Scan / PR mergeability (pull_request) Successful in 10s
Tests / PR mergeability (pull_request) Successful in 10s
Tests / Base branch health (pull_request) Successful in 11s
Tests / Detect test scope (pull_request) Successful in 16s
PR Target Validator / validate-target (pull_request_target) Successful in 14s
Validate Branch Name / check-branch (pull_request) Successful in 9s
Changeset Required / changeset-lint (pull_request) Successful in 27s
Default Flip Documentation / default-flip-documentation (pull_request) Successful in 27s
Docs Required / docs-lint (pull_request) Successful in 6m35s
Mutation Testing / Detect changed covered modules (pull_request) Successful in 20s
Security Scan / security (pull_request) Successful in 6m31s
Tests / lint-tests (pull_request) Failing after 1m56s
Tests / plugin-validate (pull_request) Successful in 1m2s
Tests / test (ubuntu-latest, 24, shard 1/3) (pull_request) Failing after 20s
Tests / test (ubuntu-latest, 24, shard 2/3) (pull_request) Failing after 19s
Tests / test (ubuntu-latest, 24, shard 3/3) (pull_request) Failing after 18s
Tests / test (ubuntu-latest, 24) (pull_request) Failing after 19s
Tests / test (inert CI) (pull_request) Has been skipped
Tests / QA loop walk (smell ratchet) (pull_request) Failing after 20s
Mutation Testing / Stryker (${{ matrix.name }}) (pull_request) Has been skipped
Tests / Coverage gate (merged shards) (pull_request) Has been skipped
Tests / Publish emitted-baseline artifact (pull_request) Has been skipped
Mutation Testing / Stryker mutation score (changed files only) (pull_request) Has been skipped
Some checks failed
Tests / conformance test (macos-latest, 24) (pull_request) Blocked by required conditions
Tests / conformance test (windows-latest, 24, shard 1/3) (pull_request) Blocked by required conditions
Tests / conformance test (windows-latest, 24, shard 2/3) (pull_request) Blocked by required conditions
Tests / conformance test (windows-latest, 24, shard 3/3) (pull_request) Blocked by required conditions
Tests / Required tests (pull_request) Blocked by required conditions
Validate Branch Name / PR mergeability (pull_request) Successful in 24s
Changeset Required / PR mergeability (pull_request) Successful in 10s
Default Flip Documentation / PR mergeability (pull_request) Successful in 9s
Dependabot Auto-Merge / auto-merge (pull_request) Has been skipped
Docs Required / PR mergeability (pull_request) Successful in 10s
Mutation Testing / PR mergeability (pull_request) Successful in 10s
Security Scan / PR mergeability (pull_request) Successful in 10s
Tests / PR mergeability (pull_request) Successful in 10s
Tests / Base branch health (pull_request) Successful in 11s
Tests / Detect test scope (pull_request) Successful in 16s
PR Target Validator / validate-target (pull_request_target) Successful in 14s
Validate Branch Name / check-branch (pull_request) Successful in 9s
Changeset Required / changeset-lint (pull_request) Successful in 27s
Default Flip Documentation / default-flip-documentation (pull_request) Successful in 27s
Docs Required / docs-lint (pull_request) Successful in 6m35s
Mutation Testing / Detect changed covered modules (pull_request) Successful in 20s
Security Scan / security (pull_request) Successful in 6m31s
Tests / lint-tests (pull_request) Failing after 1m56s
Tests / plugin-validate (pull_request) Successful in 1m2s
Tests / test (ubuntu-latest, 24, shard 1/3) (pull_request) Failing after 20s
Tests / test (ubuntu-latest, 24, shard 2/3) (pull_request) Failing after 19s
Tests / test (ubuntu-latest, 24, shard 3/3) (pull_request) Failing after 18s
Tests / test (ubuntu-latest, 24) (pull_request) Failing after 19s
Tests / test (inert CI) (pull_request) Has been skipped
Tests / QA loop walk (smell ratchet) (pull_request) Failing after 20s
Mutation Testing / Stryker (${{ matrix.name }}) (pull_request) Has been skipped
Tests / Coverage gate (merged shards) (pull_request) Has been skipped
Tests / Publish emitted-baseline artifact (pull_request) Has been skipped
Mutation Testing / Stryker mutation score (changed files only) (pull_request) Has been skipped
verify-work now calls verification.canonicalize-uat instead of hand-rolling the status flip, and progress's Route V.stale re-verifies in place. The flip's behavior is covered in tests/uat-evidence.test.cjs; these tests keep the workflow's side of the contract. Refreshes the compact-content benchmark baseline for the moved bytes.
This commit is contained in:
@@ -18,9 +18,9 @@
|
||||
"reductionPct": 16.74
|
||||
},
|
||||
"execute-phase": {
|
||||
"offTokens": 24960,
|
||||
"onTokens": 22854,
|
||||
"reductionPct": 8.44
|
||||
"offTokens": 25132,
|
||||
"onTokens": 23026,
|
||||
"reductionPct": 8.38
|
||||
},
|
||||
"new-project": {
|
||||
"offTokens": 14038,
|
||||
@@ -33,14 +33,14 @@
|
||||
"reductionPct": 11.06
|
||||
},
|
||||
"verify-work": {
|
||||
"offTokens": 13192,
|
||||
"onTokens": 11223,
|
||||
"reductionPct": 14.93
|
||||
"offTokens": 13136,
|
||||
"onTokens": 11167,
|
||||
"reductionPct": 14.99
|
||||
}
|
||||
},
|
||||
"aggregate": {
|
||||
"offTokens": 107146,
|
||||
"onTokens": 91020,
|
||||
"reductionPct": 15.05
|
||||
"offTokens": 107262,
|
||||
"onTokens": 91136,
|
||||
"reductionPct": 15.03
|
||||
}
|
||||
}
|
||||
|
||||
@@ -209,7 +209,7 @@ describe('#1107: progress routing consults verification.status before reporting
|
||||
);
|
||||
});
|
||||
|
||||
test('stale verification routes to verify-work (Route V.stale)', () => {
|
||||
test('stale verification is re-verified in place, as one action (Route V.stale)', () => {
|
||||
const workflow = readWorkflow();
|
||||
assert.ok(workflow.includes('**Route V.stale:'), 'must define a Route V.stale section');
|
||||
const route = workflow.slice(
|
||||
@@ -217,8 +217,16 @@ describe('#1107: progress routing consults verification.status before reporting
|
||||
workflow.indexOf('**Route V.gaps:')
|
||||
);
|
||||
assert.ok(
|
||||
route.includes('verify-work'),
|
||||
'Route V.stale must route to /msd:verify-work {phase}'
|
||||
route.includes('msd-core/references/stale-reverification.md'),
|
||||
'Route V.stale must follow the shared stale re-verification procedure'
|
||||
);
|
||||
assert.ok(
|
||||
route.includes('.reverify_phases'),
|
||||
'Route V.stale must batch every stale-but-executed phase from init.progress, not just the current one'
|
||||
);
|
||||
assert.ok(
|
||||
!/`\/msd:execute-phase \{phase\}/.test(route),
|
||||
'Route V.stale must not send the user to execute-phase for a phase whose plans are all summarized'
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -98,14 +98,18 @@ describe('verify-work.md — auto-transition after UAT passes with 0 issues', ()
|
||||
test('human_needed verification is promoted to passed only after successful human UAT', () => {
|
||||
const content = fs.readFileSync(VERIFY_WORK, 'utf-8');
|
||||
const statusIdx = content.indexOf('VERIFICATION_STATUS=$(msd_run query verification.status "$PHASE_DIR"');
|
||||
const humanNeededIdx = content.indexOf('if [ "$VERIFICATION_STATUS_VALUE" = "human_needed" ]; then');
|
||||
const setPassedIdx = content.indexOf('msd_run query frontmatter.set "$VERIFICATION_FILE" --field status --value passed');
|
||||
const staleIdx = content.indexOf('If `PHASE_VERIFICATION_STATUS` is `stale`');
|
||||
const canonIdx = content.indexOf('CANON=$(msd_run query verification.canonicalize-uat "$PHASE_DIR"');
|
||||
const predicateIdx = content.indexOf('PHASE_COMPLETE=$(msd_run phase uat-passed "{phase}" --require-verification)');
|
||||
|
||||
assert.ok(statusIdx !== -1, 'verify-work.md must inspect canonical verification status');
|
||||
assert.ok(humanNeededIdx > statusIdx, 'status=passed promotion must be restricted to human_needed');
|
||||
assert.ok(setPassedIdx > humanNeededIdx, 'human_needed verification must be promoted after status check');
|
||||
assert.ok(setPassedIdx < predicateIdx, 'verification must be canonicalized before the required predicate runs');
|
||||
assert.ok(canonIdx > statusIdx, 'the promotion must go through the shared canonicalize-uat seam, after the status read');
|
||||
assert.ok(staleIdx < canonIdx, 'a stale report must be re-verified BEFORE canonicalization, or a passed UAT can never be recorded');
|
||||
assert.ok(canonIdx < predicateIdx, 'verification must be canonicalized before the required predicate runs');
|
||||
assert.ok(
|
||||
!content.includes('frontmatter.set "$VERIFICATION_FILE" --field status --value passed'),
|
||||
'the workflow must not hand-roll the status flip — verification.canonicalize-uat owns it'
|
||||
);
|
||||
});
|
||||
|
||||
test('stale verification blocks before phase transition', () => {
|
||||
@@ -136,47 +140,33 @@ describe('verify-work.md — auto-transition after UAT passes with 0 issues', ()
|
||||
// phase-close uses in its --uat-only form (the verification-status blocker
|
||||
// is exactly what the flip removes, so the full predicate could never pass
|
||||
// at pre-check time); the flagged call stays the later transition gate.
|
||||
// The flip itself — gated on the uat-only row predicate, refused for any
|
||||
// status other than human_needed, fail-closed on missing evidence — is owned
|
||||
// by `verification.canonicalize-uat` and tested behaviorally in
|
||||
// tests/uat-evidence.test.cjs. What stays pinned here is the workflow's side
|
||||
// of that contract: it calls the seam, and it reports a refusal.
|
||||
describe('verify-work.md — canonicalize flip is gated by the UAT predicate (#4663)', () => {
|
||||
test('canonicalize flips to passed only when the uat-passed predicate reports passed (#4663)', () => {
|
||||
test('the flagged predicate remains the later transition gate (#4663)', () => {
|
||||
const content = fs.readFileSync(VERIFY_WORK, 'utf-8');
|
||||
const humanNeededIdx = content.indexOf('if [ "$VERIFICATION_STATUS_VALUE" = "human_needed" ]; then');
|
||||
const precheckIdx = content.indexOf('UAT_PRECHECK=$(msd_run phase uat-passed "{phase}" --uat-only 2>/dev/null)');
|
||||
const flipGuardIdx = content.indexOf('if [ "$UAT_PRECHECK_PASSED" = "true" ]; then');
|
||||
const setPassedIdx = content.indexOf('msd_run query frontmatter.set "$VERIFICATION_FILE" --field status --value passed');
|
||||
|
||||
assert.ok(precheckIdx !== -1, 'the canonicalize block must run the uat-passed predicate before flipping');
|
||||
assert.ok(humanNeededIdx !== -1 && precheckIdx > humanNeededIdx, 'the pre-check must sit inside the human_needed branch');
|
||||
assert.ok(content.includes(".passed // false"), 'the verdict must be extracted from the typed report with a false default');
|
||||
assert.ok(flipGuardIdx !== -1 && flipGuardIdx > precheckIdx, 'the flip must be guarded on the extracted passed verdict');
|
||||
assert.ok(setPassedIdx > flipGuardIdx, 'frontmatter.set must sit INSIDE the passed==true guard');
|
||||
});
|
||||
|
||||
test('the canonicalize pre-check runs uat-passed without --require-verification (#4663)', () => {
|
||||
const content = fs.readFileSync(VERIFY_WORK, 'utf-8');
|
||||
const precheckIdx = content.indexOf('UAT_PRECHECK=$(msd_run phase uat-passed "{phase}" --uat-only 2>/dev/null)');
|
||||
const canonIdx = content.indexOf('CANON=$(msd_run query verification.canonicalize-uat "$PHASE_DIR"');
|
||||
const flaggedIdx = content.indexOf('PHASE_COMPLETE=$(msd_run phase uat-passed "{phase}" --require-verification)');
|
||||
|
||||
assert.ok(precheckIdx !== -1, 'the --uat-only pre-check must exist');
|
||||
assert.ok(
|
||||
!content.slice(precheckIdx, precheckIdx + 120).includes('--require-verification'),
|
||||
'the pre-check is the unflagged predicate - requiring verification there would evaluate the very report being written'
|
||||
);
|
||||
assert.ok(flaggedIdx !== -1 && flaggedIdx > precheckIdx, 'the flagged predicate remains the later transition gate');
|
||||
assert.ok(canonIdx !== -1, 'the canonicalize call must exist');
|
||||
assert.ok(flaggedIdx !== -1 && flaggedIdx > canonIdx, 'the flagged predicate remains the later transition gate');
|
||||
});
|
||||
|
||||
test('refused canonicalization says the verification stays human_needed (#4663)', () => {
|
||||
const content = fs.readFileSync(VERIFY_WORK, 'utf-8');
|
||||
assert.match(content, /"\$CANON_REASON" = "uat_not_passed"/, 'the refusal must branch on the typed reason');
|
||||
assert.match(content, /stays human_needed/, 'the refusal message must say verification stays human_needed');
|
||||
assert.match(content, /blockers \| length/, 'the refusal must carry the blocking-row count');
|
||||
});
|
||||
|
||||
test('an indeterminate pre-check must not flip the report (fail closed) (#4663)', () => {
|
||||
test('an indeterminate canonicalize call reads as no reason, never as a flip (#4663)', () => {
|
||||
const content = fs.readFileSync(VERIFY_WORK, 'utf-8');
|
||||
// jq -r '.passed // false' with an `|| echo "false"` fallback: empty or
|
||||
// failed msd_run output must yield no-flip, never a flip.
|
||||
assert.ok(
|
||||
content.includes(`jq -r '.passed // false' 2>/dev/null || echo "false"`),
|
||||
'the extraction must default to false on empty/failed output'
|
||||
content.includes(`jq -r '.reason // empty' 2>/dev/null || echo ""`),
|
||||
'the extraction must default to empty on empty/failed output'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user