fix(#1905): normalize hand-authored backstop marker so it can't degrade to green (#1909)

* fix(#1905): normalize hand-authored backstop marker so it can't degrade to green

A hand-authored non-inferable `backstop` truth with a stray trailing space (or
surrounding quotes) silently graded {status:green} instead of abstaining — the exact
#1154 false-pass. `truthVerification` returned null for any value != 'backstop'/'explicit',
and the frontmatter continuation-KV parser preserved the stray whitespace captured inside
the quotes. Normalize the marker before comparison (Postel) AND trim the continuation-KV
value at the parser (durable root cause; also cleans the sibling check_target path).

Regression: a trailing-space/quoted backstop truth now abstains (insufficient_spec),
end-to-end from a hand-authored must_haves.truths block (#1820 rail).

Refs #1905, epic #1904.

* chore(changeset): Fixed fragment for #1909 (backstop marker normalize)
This commit is contained in:
Rezolv
2026-07-02 11:55:49 -04:00
committed by GitHub
parent 12b35eeeaf
commit a4bc04a5ff
5 changed files with 80 additions and 4 deletions

View File

@@ -366,6 +366,22 @@ Body content.`;
assert.strictEqual(result[1], 'Coverage exceeds 80%');
});
test('trims a continuation-KV value so a quoted trailing space does not survive (#1905, root cause of the #1154 false-pass)', () => {
// A quoted value like `"backstop "` captures the inner trailing space in group 2; left untrimmed,
// a hand-authored non-inferable `backstop` marker (#1820 spec-optional rail) degrades to `'backstop '`,
// which `truthVerification` no longer recognizes → the truth silently grades green instead of abstaining.
// Whitespace is never semantic in a scalar KV value, so the parser must trim it.
const content = `---
must_haves:
truths:
- statement: user data is never logged
verification: "backstop "
---
Body.`;
const result = parseMustHavesBlock(content, 'truths');
assert.strictEqual(result[0].verification, 'backstop', 'the captured value is trimmed, not left as "backstop "');
});
test('extracts artifacts as object array', () => {
const content = `---
phase: 01

View File

@@ -554,6 +554,19 @@ describe('probe-core: truthStatement / truthVerification normalizers (#1154, Hyr
assert.equal(pc.truthVerification('Overlapping intervals are merged'), null);
assert.equal(pc.truthVerification({ statement: 'x', verification: 'explicit' }), 'explicit');
});
test('normalizes a hand-authored marker with stray whitespace/surrounding quotes (#1905, the #1154 false-pass)', () => {
// A `must_haves` marker can be authored BY HAND (#1820 spec-optional predicate rail), and the
// frontmatter continuation-KV parser preserves stray surrounding whitespace/quotes. Failing to
// normalize means `'backstop '` → null → the non-inferable truth silently grades green (Postel:
// be liberal in what you accept).
assert.equal(pc.truthVerification({ statement: 'x', verification: 'backstop ' }), 'backstop', 'trailing space');
assert.equal(pc.truthVerification({ statement: 'x', verification: ' backstop' }), 'backstop', 'leading space');
assert.equal(pc.truthVerification({ statement: 'x', verification: '"backstop"' }), 'backstop', 'surrounding quotes');
assert.equal(pc.truthVerification({ statement: 'x', verification: 'explicit ' }), 'explicit', 'trailing space, explicit tier');
// An unrecoverably-corrupted marker stays unrecognized → null → graded normally (AC#3 over-abstention guard).
assert.equal(pc.truthVerification({ statement: 'x', verification: 'back"stop' }), null, 'an embedded quote is unrecoverable — no spurious tier');
});
});
describe('probe-core: dispositionForUnverifiableTruth (#1154, ADR-550 D4 truth-axis mirror)', () => {
@@ -588,6 +601,35 @@ describe('probe-core: dispositionForUnverifiableTruth (#1154, ADR-550 D4 truth-a
assert.equal(d.flagged, false);
});
test('a whitespace-mangled backstop truth still ABSTAINS, never silently greens (#1905 — the #1154 false-pass)', () => {
const d = pc.dispositionForUnverifiableTruth(
{ statement: 'user data is never logged', verification: 'backstop ' }, // stray trailing space
{ evidence: [] },
);
assert.equal(d.status, 'unverified', 'a mangled backstop marker must not degrade to a silent green');
assert.equal(d.flagged, true);
assert.equal(d.tier, 'backstop');
assert.equal(d.reason, 'insufficient_spec');
});
test('END-TO-END (#1905, #1820 hand-authoring path): a HAND-AUTHORED must_haves.truths trailing-space backstop marker parses to the tier and abstains, never greens', () => {
// A human authors the marker directly (the #1820 spec-optional predicate rail), NOT projectTruths.
// The frontmatter continuation-KV parser used to preserve the stray trailing space inside the quotes,
// so truthVerification saw 'backstop ' → null → the non-inferable truth graded green. It must parse
// clean and abstain — the exact honesty regression #1154 exists to eliminate (ADR-550 D4 truth axis).
const doc = [
'---', 'must_haves:', ' truths:',
' - statement: user data is never logged',
' verification: "backstop "',
'---', 'body',
].join('\n');
const parsed = fm.parseMustHavesBlock(doc, 'truths');
assert.equal(pc.truthVerification(parsed[0]), 'backstop', 'the mangled marker normalizes to the tier');
const d = pc.dispositionForUnverifiableTruth(parsed[0], { evidence: [] });
assert.equal(d.status, 'unverified', 'never a silent green (ADR-550 D4 truth-axis, #1154)');
assert.equal(d.reason, 'insufficient_spec');
});
test('over-abstention guard (AC#3): an explicit-tier truth NEVER abstains even with no evidence (only backstop triggers it)', () => {
const d = pc.dispositionForUnverifiableTruth({ statement: 'Symbol X is wired', verification: 'explicit' }, { evidence: [] });
assert.equal(d.status, 'green', 'an explicit (inferable) truth never abstains');