feat(#1154): honest verifier — abstain (insufficient_spec) on non-inferable backstop truths (#1738)

* feat(verify-phase): honest verifier — abstain (insufficient_spec) on non-inferable backstop truths (#1154)

Carry the edge-probe's existing `backstop` (non-inferable) tier through the
plan-phase projection as a structured flat-scalar marker instead of a prose
parenthetical, and make verify-phase abstain -> human_needed (never silent-pass)
on a backstop truth it cannot confirm with explicit evidence. Truth-axis mirror
of #644's prohibition judgment-tier (ADR-550 D4).

Engine (deterministic, CI-tested per ADR-550 D5 — never the LLM verdict):
- src/probe-core.cts: truthStatement/truthVerification normalizers, projectTruths
  (conservative serializer), dispositionForUnverifiableTruth (backstop+no-evidence
  -> unverified/flagged/insufficient_spec; backstop+evidence -> green; inferable
  -> green, the over-abstention guard).
- src/roadmap.cts: coerceTruthToString now reads `statement` first so an object-form
  backstop truth is surfaced, not dropped (Hyrum backward-compat for truth-readers).

Workflow/agent/docs: plan-phase emits the structured marker (flat scalar, ADR-550
#1278); verify-phase + gsd-verifier add the abstain arm; new references/honest-verifier.md;
FEATURES/COMMANDS document insufficient_spec; ADR-550 amended (truth-axis D4 mirror).

Decisions adopted (trek-e review): insufficient_spec feeds existing human_needed with a
distinguishable reason (no new VERIFIER_STATUS); changeset Changed; round-trip parity
test; abstain-on-unconfirmed-backstop regression test red-first.

Implementation notes (deviations from the issue's proposed file list, verified live):
- frontmatter.cts needs no change — its flat parser already round-trips object-form truths.
- verify.cts needs no change — it grades artifacts/key_links structurally; truths are
  LLM-graded at the workflow layer, so consumption lives there + the deterministic helper.
- No CJS<->SDK hand-sync — the SDK seam was retired (ADR-0174); src/*.cts is sole source.

Regenerated artifacts: golden-install-parity fixtures, INVENTORY-MANIFEST, size baselines.

* chore(#1154): add changeset (Changed) for honest verifier

User-facing changelog fragment for #1738. Typed `Changed` (not `Added`) per
trek-e review condition 3 — the verify behavior shifts for backstop-bearing specs
(a confident silent `passed` becomes `human_needed`), which is user-visible even
though the schema marker is additive.

* docs(#1154): score-formula also excludes abstained insufficient_spec truths (review nit-1)

trek-e review nit: the verify-phase score sentence said PRESENT_BEHAVIOR_UNVERIFIED
truths were "the only ones excluded" from verified_truths. Post-#1154 an abstained
`insufficient_spec` backstop truth is also excluded (it is not ✓ VERIFIED and routes
to human_needed). Behavior was already correct; this tightens the wording.
Regenerated golden-install-parity fixtures + workflow-size baseline for the touched
verify-phase.md. (Nit-2 — a dedicated insufficient_spec_items frontmatter list — is
intentionally not taken: the current design is ADR-550-D4-conformant, the abstain
cause rides as a distinguishable report reason, and adding it would exceed the
approved scope.)

---------

Co-authored-by: Tom Boucher <trekkie@nomorestars.com>

src/probe-core.cts

@@ -489,6 +489,140 @@ export function dispositionForProhibition(
   };
 }
 
+/* ─────────────────────────────────────────────────────────────────────────────
+ * Honest verifier (#1154) — the truth-axis abstention disposition.
+ *
+ * The verify-time MIRROR of the prohibition judgment-tier (ADR-550 D4), applied to the edge
+ * `backstop` truth tier (D7a). The edge probe already CLASSIFIES a non-inferable check as
+ * `verification: 'backstop'` and plan-phase lifts it into `must_haves.truths`. This gives that tier
+ * the same abstain-and-flag disposition D4 gave prohibitions: a `backstop` truth the verifier cannot
+ * confirm with explicit evidence disposes UNVERIFIED+flagged → `human_needed` (reason
+ * `insufficient_spec`), NEVER a silent green. An inferable (explicit/plain) truth never abstains (the
+ * over-abstention guard, AC#3). Exogenous, not endogenous: the trigger is the external `backstop`
+ * tag, not the verifier's self-judgment (ADR-550 Lineage / N17 — confidence-gating cannot reach a
+ * blind spot the model does not perceive).
+ * ───────────────────────────────────────────────────────────────────────────── */
+
+/** The truth verification tier — the SAME orthogonal axis the edge adapter uses (ADR-550 D7a). */
+export type TruthVerification = 'explicit' | 'backstop';
+
+/**
+ * A `must_haves.truths` item is EITHER a plain string (an inferable truth — today's shape and the
+ * overwhelmingly common case) OR a flat-scalar object carrying the non-inferable marker. Object form
+ * is additive and default-absent (Hyrum's Law): a reader that only ever sees strings behaves
+ * byte-identically. New readers MUST normalize via `truthStatement`/`truthVerification`.
+ */
+export type TruthItem = string | { statement: string; verification?: TruthVerification | null };
+
+/** Extract a truth's statement text from either the string or the object form (the Hyrum normalizer). */
+export function truthStatement(truth: unknown): string {
+  if (typeof truth === 'string') return truth;
+  if (truth != null && typeof truth === 'object') {
+    const s = (truth as { statement?: unknown }).statement;
+    if (typeof s === 'string') return s;
+  }
+  return '';
+}
+
+/**
+ * Extract a truth's verification tier, or `null` when it carries none (a plain string, or an object
+ * with no/garbled marker). Failing toward `null` is the Postel-safe direction: an unrecognized marker
+ * grades NORMALLY (never a spurious abstention — the over-abstention guard, AC#3), and the marker is
+ * machine-emitted from validated edge data so garbling is not a live input path.
+ */
+export function truthVerification(truth: unknown): TruthVerification | null {
+  if (truth == null || typeof truth !== 'object') return null;
+  const v = (truth as { verification?: unknown }).verification;
+  return v === 'explicit' || v === 'backstop' ? v : null;
+}
+
+/**
+ * Conservative serializer (Postel: "send well-formed, minimal data") for projecting truths into a
+ * `must_haves.truths` block — the truth-axis analogue of `projectProhibitions`. A `backstop` truth is
+ * emitted as a flat-scalar object `{ statement, verification: 'backstop' }` (ADR-550 #1278: flat
+ * scalars round-trip the existing `parseMustHavesBlock`; a nested object would mangle it). Every other
+ * truth collapses to a bare statement string — only the non-inferable tier needs a structured marker,
+ * so an `explicit`/inferable truth never carries one (no spurious markers). Empty statements are dropped.
+ */
+export function projectTruths(
+  items: unknown,
+): Array<string | { statement: string; verification: 'backstop' }> {
+  if (!Array.isArray(items)) return [];
+  const out: Array<string | { statement: string; verification: 'backstop' }> = [];
+  for (const item of items) {
+    const statement = truthStatement(item);
+    if (!statement) continue;
+    if (truthVerification(item) === 'backstop') {
+      out.push({ statement, verification: 'backstop' });
+    } else {
+      out.push(statement);
+    }
+  }
+  return out;
+}
+
+/**
+ * The structured verify-time disposition of a single truth. Same shape as `ProhibitionDisposition`
+ * (status the verifier reads + `flagged` for SUMMARY surfacing + `tier` echo + human-readable
+ * `reason`), but `reason` carries the STABLE token `insufficient_spec` on abstention so the
+ * `human_needed` outcome is distinguishable from an ordinary manual-UAT `human_needed` (review
+ * condition-1 caveat).
+ */
+export interface TruthDisposition {
+  status: 'green' | 'unverified';
+  flagged: boolean;
+  tier: TruthVerification | null;
+  reason: string;
+}
+
+/** Optional context: evidence that a `backstop` truth IS confirmable (a passing wired held-out/PBT test, or a directly-observed behavior). */
+export interface TruthDispositionContext {
+  evidence?: unknown[];
+}
+
+/** The stable, distinguishable verdict-reason token for an abstained non-inferable truth (review condition 1). */
+export const INSUFFICIENT_SPEC = 'insufficient_spec';
+
+/**
+ * Deterministic verify-time disposition for a single truth (ADR-550 D4 truth-axis mirror, #1154).
+ * PURE — no LLM judgment (ADR-550 D5); the LLM verifier's only job is to decide whether `evidence`
+ * exists, this helper owns the routing once that is known.
+ *
+ *   - A `backstop` (non-inferable) truth with NO explicit evidence → `{ unverified, flagged }`,
+ *     reason `insufficient_spec`. NEVER green — the verify-time companion to D4's never-silent-pass.
+ *   - A `backstop` truth WITH explicit evidence (a passing wired held-out/property test) → `green`.
+ *     Abstention is for the *unconfirmable*, not for every non-inferable check.
+ *   - Any non-`backstop` truth (explicit, or a plain inferable string) → `green`, never flagged.
+ *     This is the over-abstention guard (AC#3): abstention fires ONLY on the exogenous backstop tag.
+ */
+export function dispositionForUnverifiableTruth(
+  truth: unknown,
+  context: TruthDispositionContext = {},
+): TruthDisposition {
+  const tier = truthVerification(truth);
+  // Over-abstention guard (AC#3): only a backstop (non-inferable) truth is ever a candidate to abstain.
+  if (tier !== 'backstop') {
+    return {
+      status: 'green',
+      flagged: false,
+      tier,
+      reason: 'inferable truth — verified normally (no abstention; ADR-550 D4 over-abstention guard)',
+    };
+  }
+  const evidence = Array.isArray(context.evidence) ? context.evidence : [];
+  if (evidence.length === 0) {
+    // ABSTAIN: a non-inferable truth the verifier cannot confirm with explicit evidence. Routes to
+    // human_needed with the distinguishable insufficient_spec reason — never a silent pass (ADR-550 D4).
+    return { status: 'unverified', flagged: true, tier, reason: INSUFFICIENT_SPEC };
+  }
+  return {
+    status: 'green',
+    flagged: false,
+    tier,
+    reason: 'backstop truth confirmed by explicit evidence (a passing wired held-out/property test or directly-observed behavior)',
+  };
+}
+
 /*
  * CLI scaffold (the EP-06 invokable surface, generalized). Each probe ships one bin that
  * calls `runProbeCli` with its own `analyze` (closing over the adapter's propose + validators)

src/roadmap.cts

@@ -78,8 +78,11 @@ function coerceTruthToString(t: unknown): string {
     return String(t);
   }
   if (typeof t === 'object') {
-    // Prefer common title-bearing keys produced by parseMustHavesBlock
-    for (const k of ['title', 'text', 'name', 'rule', 'path', 'provides']) {
+    // Prefer common title-bearing keys produced by parseMustHavesBlock. `statement` is the canonical
+    // truth/prohibition payload field — and the carrier of #1154's object-form backstop truth
+    // `{ statement, verification: backstop }`, so it leads (a non-inferable truth must be coerced by
+    // its statement, never dropped — the Hyrum backward-compat guard for the new marker).
+    for (const k of ['statement', 'title', 'text', 'name', 'rule', 'path', 'provides']) {
       const v = (t as Record<string, unknown>)[k];
       if (typeof v === 'string' && v.trim()) return v;
       if (typeof v === 'number' || typeof v === 'boolean') return String(v);