fix(#3956): require positive evidence for verify artifacts/key-links pass (#4004)

* fix(#3956): require positive evidence for verify artifacts/key-links pass

An all-string or path-less must_haves.artifacts / key_links block is
item-by-item skipped, leaving zero checked results, yet the pass verdict
was computed as `passed === results.length` (0 === 0), so all_passed /
all_verified read true with status valid and exit 0: a silent false GREEN
over zero acceptance evidence.

Add a positive-evidence floor (results.length > 0) to both verdicts,
mirroring the no-vacuous-pass rule at src/uat-predicate.cts. A well-formed
block, the fully-empty-block error, the parser's string tolerance, and
key-links pending (#1202) semantics are all unchanged.

Governing: ADR-3473 section 8 / 37C (absence, emptiness and failure must
not encode as success) and Decision 3 (failure is a value).

* chore(#3956): add changeset for verify vacuous-pass fix

* test(#3956): add mixed-block coverage and correct the key-links vacuous-pass comment

Addresses review on #4004:
- Correct the cmdVerifyKeyLinks positive-evidence-floor comment: only bare-string
  items are continue-skipped; a from:-less object is NOT skipped (it falls through
  to a verified:false hard failure), so it was never part of the vacuous-pass
  surface. The prior comment overclaimed symmetry with the artifacts side.
- Add a mixed-block regression test per verb (one bare-string prose bullet + one
  well-formed entry): the string is skipped, results.length === 1 > 0, and the
  verdict follows the single real entry — pinning that the floor does not
  over-reject a partial block.
- Tighten the changeset wording to match (all-bare-string, not "no path:/from: key").

---------

Co-authored-by: Tom Boucher <trekkie@nomorestars.com>
This commit is contained in:
Rezolv
2026-09-03 17:46:50 -04:00
committed by GitHub
parent a788afb120
commit 590edec7a7
3 changed files with 114 additions and 3 deletions

View File

@@ -1200,15 +1200,22 @@ function cmdVerifyArtifacts(cwd: string, planFilePath: string, raw: boolean): vo
}
const passed = results.filter((r) => r['passed']).length;
// Positive-evidence floor (#3956): a non-empty artifacts block whose items are
// all bare strings / path-less objects is item-by-item skipped, leaving results
// empty; `passed === results.length` would then be `0 === 0` → a vacuous GREEN
// over zero checks. Require at least one checked artifact, mirroring the
// no-vacuous-pass rule at src/uat-predicate.cts. The fully-empty block is still
// caught earlier by the `artifacts.length === 0` guard and returns its error.
const allPassed = results.length > 0 && passed === results.length;
output(
{
all_passed: passed === results.length,
all_passed: allPassed,
passed,
total: results.length,
artifacts: results,
},
raw,
passed === results.length ? 'valid' : 'invalid',
allPassed ? 'valid' : 'invalid',
);
}
@@ -1417,7 +1424,17 @@ function cmdVerifyKeyLinks(cwd: string, planFilePath: string, raw: boolean): voi
// A pending link (from: file promised by a same-or-later-wave plan) is not a
// hard failure — it should not count against the all_verified gate (#1202).
const hardFailed = results.filter((r) => !r['verified'] && !r['pending']).length;
const allVerified = hardFailed === 0;
// Positive-evidence floor (#3956): an all-bare-string key_links block skips
// every item (only `typeof link === 'string'` items are continue-skipped
// above), leaving results empty; `hardFailed === 0` would then be a vacuous
// GREEN over zero checks. Require at least one checked link. (A `from:`-less
// object is NOT skipped, unlike a path-less object on the artifacts side — it
// falls through to a `verified: false` result and hard-fails, so it was never
// part of the vacuous-pass surface; only the all-bare-string case is.) A
// pending link IS pushed to results (with pending: true), so an all-pending
// block still satisfies results.length > 0 and its #1202 non-hard-failing
// semantics are unchanged — the floor only rejects the zero-result case.
const allVerified = results.length > 0 && hardFailed === 0;
output(
{
all_verified: allVerified,