golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(#1905): normalize hand-authored backstop marker so it can't degrade to green (#1909)

Browse Source 
* fix(#1905): normalize hand-authored backstop marker so it can't degrade to green

A hand-authored non-inferable `backstop` truth with a stray trailing space (or
surrounding quotes) silently graded {status:green} instead of abstaining — the exact
#1154 false-pass. `truthVerification` returned null for any value != 'backstop'/'explicit',
and the frontmatter continuation-KV parser preserved the stray whitespace captured inside
the quotes. Normalize the marker before comparison (Postel) AND trim the continuation-KV
value at the parser (durable root cause; also cleans the sibling check_target path).

Regression: a trailing-space/quoted backstop truth now abstains (insufficient_spec),
end-to-end from a hand-authored must_haves.truths block (#1820 rail).

Refs #1905, epic #1904.

* chore(changeset): Fixed fragment for #1909 (backstop marker normalize)
This commit is contained in:
Rezolv
2026-07-02 11:55:49 -04:00
committed by GitHub
parent 12b35eeeaf
commit a4bc04a5ff
5 changed files with 80 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
src/frontmatter.cts
View File

@@ -482,7 +482,11 @@ function parseMustHavesBlock(content: string, blockName: string): unknown[] {
} else {
const kvMatch = trimmed.match(/^(\w+):\s*"?([^"]*)"?\s*$/);
if (kvMatch) {
const val = kvMatch[2];
// Trim: a quoted value like `"backstop "` captures the inner trailing space in group 2.
// Left untrimmed, a hand-authored `must_haves` marker degrades (a `backstop` truth silently
// grades green instead of abstaining — #1905, the #1154 false-pass; also the sibling
// check_target/violationFixture path). Whitespace is never semantic in a scalar KV value.
const val = kvMatch[2].trim();
// Try to parse as number
(current)[kvMatch[1]] = /^\d+$/.test(val) ? parseInt(val, 10) : val;
}

15
src/probe-core.cts
View File

@@ -527,12 +527,21 @@ export function truthStatement(truth: unknown): string {
/**
* Extract a truth's verification tier, or `null` when it carries none (a plain string, or an object
* with no/garbled marker). Failing toward `null` is the Postel-safe direction: an unrecognized marker
* grades NORMALLY (never a spurious abstention — the over-abstention guard, AC#3), and the marker is
* machine-emitted from validated edge data so garbling is not a live input path.
* grades NORMALLY (never a spurious abstention — the over-abstention guard, AC#3).
*
* The marker is NOT only machine-emitted: `must_haves` markers can be authored BY HAND (#1820's
* spec-optional predicate rail), and the frontmatter continuation-KV parser preserves stray
* surrounding whitespace/quotes on a hand-authored value. So we normalize before comparison
* (Postel: be liberal in what you accept) — `'backstop '`, `' backstop'`, `'"backstop"'` all
* recognize as the tier. Without this, a hand-authored non-inferable `backstop` truth with a stray
* trailing space silently grades green instead of abstaining — the exact #1154 false-pass (#1905).
* An unrecoverably-corrupted marker (e.g. an embedded quote) stays unrecognized → null → graded
* normally (AC#3): we cannot know its intent, and abstaining on it would be a spurious abstention.
*/
export function truthVerification(truth: unknown): TruthVerification | null {
if (truth == null || typeof truth !== 'object') return null;
const v = (truth as { verification?: unknown }).verification;
const raw = (truth as { verification?: unknown }).verification;
const v = typeof raw === 'string' ? raw.trim().replace(/^["']|["']$/g, '').trim() : raw;
return v === 'explicit' || v === 'backstop' ? v : null;
}