From 6932fb16d7c0fa06b17e290becda47f49a1f4401 Mon Sep 17 00:00:00 2001 From: Rezolv Date: Tue, 28 Jul 2026 18:31:26 -0400 Subject: [PATCH] enhance(#1699): require read-and-cite provenance for in-repo discrete values (#2768) * test(#1699): failing-first contract tests for in-repo value provenance Nine assertions on the deployed gsd-phase-researcher contract: the discrete-value taxonomy, the same-session Read requirement, path-AND-line-range citation, grep-alone exclusion, the verbatim quote and paraphrase ban, the quote-is-the-checkable-artifact guard, [ASSUMED] routing for unquoted skeleton values, a no-regression guard on the pre-existing package name provenance rule, and a single-definition-site guard. Eight of the nine fail against the unmodified agent on origin/next; the ninth is the no-regression invariant and passes in both states, which is the intended enhancement shape. Added to tests/research-agent-profiles.test.cjs rather than a new file: that file already carries the allow-test-rule exemption research agent .md content is the governed surface, so no new allowlist entry and no change to the per-module test-file count. * enhance(#1699): require read-and-cite provenance for in-repo discrete values The claim-provenance system governed external facts (npm registry, official docs, Context7, package-name provenance). For an in-repo discrete value -- an enum, schema or type union, error code, status constant, or filesystem path -- [VERIFIED] could be earned from training memory or a bare codebase grep, which proves a string occurs, not that the definition was read. A drifted value passes into RESEARCH.md, is lifted by the planner into PLAN.md's context block, and is trusted by the executor, where it fails at parse()/typecheck as a mid-execution deviation -- the most expensive place to discover it. The rule lands beside its structural sibling, the package name provenance rule, since both say existence is not verification. The verbatim quote is named as the load-bearing artifact: a citation with no quote does not earn the tag, however precise the line range looks. That keeps the rule falsifiable against the file rather than a self-report, which is the Goodhart guard. Scope note: the quote goes in RESEARCH.md beside the claim, NOT in an block. appears zero times in this agent on next -- it is planner-side, defined at gsd-core/references/planner-interface-context.md:15 as a PLAN.md structure. The issue text and triage both said ; instructing the researcher to populate a block it does not emit would be undefined. Defined once at the definition site; the source-hierarchy recap is deliberately untouched, since restating it is the paraphrase-drift mode META.RULE.brief-no-paraphrase names. * test(#1699): regenerate agent size baseline and golden parity fixtures Regenerated via npm run size:baseline and npm run gen:golden, never hand-edited. agent-size-baseline gsd-phase-researcher.md 40866 to 42020 (LARGE tier, cap 49152, 7132 bytes headroom remaining, per ADR-1610's per-file baseline guard). 18 of 19 golden fixtures updated; pi.json is unchanged because the pi runtime ships zero agents. * chore(#1699): add changeset for the in-repo value citation rule User-facing behavior change in gsd-phase-researcher, so a Changed fragment is required. pr: 2768. * test(#1699): acknowledge the gsd-phase-researcher growth for the emitted-drift gate CI test (ubuntu-latest, 22) failed on emitted-attribution.test.cjs: gsd-phase-researcher.md grew 1154 bytes (40866 -> 42020) without an acknowledgment. The differential emitted-attribution gate landed on next in 9138271b (#2723) and requires intentional growth to be named AND explained in tests/emitted-drift-ack.json per ADR-2719 section 3. This creates the file; it did not previously exist on next. The reason records what grew and why the prose was added inline rather than relocated into an eagerly @-imported reference, which ADR-1610 Decision 4 names as gaming the size proxy. Verified locally against a freshly fetched origin/next: emitted-attribution 35/35, and the differential subtest runs rather than skips (skipped 0). --------- Co-authored-by: CI Rebase Check Co-authored-by: Tom Boucher --- .changeset/1699-in-repo-value-citation.md | 5 ++ agents/gsd-phase-researcher.md | 2 + docs/AGENTS.md | 1 + docs/COMMANDS.md | 3 + tests/emitted-drift-ack.json | 8 ++ tests/research-agent-profiles.test.cjs | 105 ++++++++++++++++++++++ 6 files changed, 124 insertions(+) create mode 100644 .changeset/1699-in-repo-value-citation.md create mode 100644 tests/emitted-drift-ack.json diff --git a/.changeset/1699-in-repo-value-citation.md b/.changeset/1699-in-repo-value-citation.md new file mode 100644 index 000000000..814a2e14e --- /dev/null +++ b/.changeset/1699-in-repo-value-citation.md @@ -0,0 +1,5 @@ +--- +type: Changed +pr: 2768 +--- +**The phase researcher must now read and cite in-repo values before calling them verified** — an enum, schema or type union, error code, status constant, or filesystem path earns a `[VERIFIED: path:line-range]` tag only if the researcher opened the source-of-truth file with `Read` during the run and quoted the values verbatim in the `` block; every value used in a code skeleton must appear in that quote, and anything else stays `[ASSUMED]`. Previously the tag could be earned from training memory or a web search alone, so a plausible-but-drifted enum could pass into RESEARCH.md, get copied into PLAN.md, and fail only at the executor's `parse()`/typecheck — a mid-execution deviation, the most expensive place to discover it. (#1699) diff --git a/agents/gsd-phase-researcher.md b/agents/gsd-phase-researcher.md index 00bd8ff21..d716c0fba 100644 --- a/agents/gsd-phase-researcher.md +++ b/agents/gsd-phase-researcher.md @@ -32,6 +32,8 @@ Spawned by `/gsd:plan-phase` (integrated) or `/gsd:plan-phase --research-phase < **Package name provenance rule:** A package name discovered via WebSearch, training data, or any non-authoritative source must be tagged `[ASSUMED]` regardless of whether `npm view` confirms it exists on the registry. Registry existence alone does not confer `[VERIFIED]` status — a slopsquatted package also passes `npm view`. Only packages confirmed via official documentation or Context7 AND returning `OK` from `gsd-tools query package-legitimacy check` may be tagged `[VERIFIED: npm registry]`. +**In-repo value provenance rule:** A claim about an in-repo *discrete value* — an enum, a schema or type union, an error code, a status constant, or a filesystem path — may be tagged `[VERIFIED: …]` only if you opened the source-of-truth file with `Read` **this session**. A codebase `grep` is not sufficient on its own: it confirms a string occurs, not that you read the definition. Cite the path **and line range** (`[VERIFIED: src/types/order.ts:14-22]`), and quote the values **verbatim** in RESEARCH.md beside the claim — paraphrase is forbidden. The quote is what makes the tag checkable — a citation with no quote beside it does not earn `[VERIFIED]`, however precise the line range looks. Every value appearing in a code example or skeleton must also appear in that verbatim quote; a value that does not is `[ASSUMED]`. For a filesystem path, cite the line in the script that creates it, not the location you expect it to occupy. Training memory and a web search are not substitutes for reading the file — a discrete value that merely looks right fails at the executor's `parse()`/typecheck, the most expensive place to discover it. + Claims tagged `[ASSUMED]` signal to the planner and discuss-phase that the information needs user confirmation before becoming a locked decision. Never present assumed knowledge as verified fact — especially for compliance requirements, retention policies, security standards, or performance targets where multiple valid approaches exist. diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 8d5c9c990..e093f5f62 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -69,6 +69,7 @@ GSD uses a multi-agent architecture where thin orchestrators (workflow files) sp - Reads CONTEXT.md to focus research on user's decisions - Investigates implementation patterns for the specific phase domain - Detects test infrastructure for Nyquist validation mapping +- Tags in-repo discrete values (enums, schema unions, error codes, status constants, paths) `[VERIFIED]` only after reading the source-of-truth file that run, citing path and line range, and quoting the values verbatim --- diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md index aeeb29982..bb36a5e47 100644 --- a/docs/COMMANDS.md +++ b/docs/COMMANDS.md @@ -237,6 +237,9 @@ Packages sourced from WebSearch are tagged `[ASSUMED]` (not `[VERIFIED]`) and tr See [Package Legitimacy Gate in the User Guide](USER-GUIDE.md#package-legitimacy-gate-v1421) for the full checkpoint format, verdict table, and troubleshooting. +**In-repo value citation:** +For any in-repo *discrete value* the researcher reports — an enum, a schema or type union, an error code, a status constant, or a filesystem path — a `[VERIFIED: …]` tag requires that it opened the source-of-truth file with `Read` during the run and cited the path **and line range** (`[VERIFIED: src/types/order.ts:14-22]`). The values are quoted verbatim in RESEARCH.md beside the claim, and any value used in a code example must also appear in that quote; anything else stays `[ASSUMED]`. A codebase `grep`, training memory, or a web search do not earn the tag on their own. This stops a plausible-but-drifted enum from reaching PLAN.md — where the planner lifts it into the plan's `` context block and the executor trusts it as ground truth — and surfacing only as a mid-execution deviation at typecheck. + ```bash /gsd-plan-phase 1 # Research + plan + verify phase 1 /gsd-plan-phase 3 --skip-research # Plan without research (familiar domain) diff --git a/tests/emitted-drift-ack.json b/tests/emitted-drift-ack.json new file mode 100644 index 000000000..d25bb9080 --- /dev/null +++ b/tests/emitted-drift-ack.json @@ -0,0 +1,8 @@ +{ + "version": 1, + "paths": { + "gsd-phase-researcher.md": { + "reason": "#1699 adds the in-repo value provenance rule to the claim-provenance section: an enum, schema/type-union, error code, status constant, or filesystem path earns [VERIFIED] only after a same-session Read, a path-and-line-range citation, and a verbatim quote in RESEARCH.md. Growth is inline prose in the agent body, deliberately NOT relocated into an eagerly @-imported reference, which ADR-1610 Decision 4 names as gaming the size proxy. 40866 -> 42020 bytes, LARGE tier, cap 49152." + } + } +} diff --git a/tests/research-agent-profiles.test.cjs b/tests/research-agent-profiles.test.cjs index f13e3d3f5..bab307129 100644 --- a/tests/research-agent-profiles.test.cjs +++ b/tests/research-agent-profiles.test.cjs @@ -519,3 +519,108 @@ describe('research agents do not inject year into web searches (#2559)', () => { }); }); } + +// --------------------------------------------------------------------------- +// In-repo value provenance rule (#1699) +// +// The claim-provenance block governed EXTERNAL facts only (npm registry, official +// docs, Context7, package-name provenance). An in-repo discrete value could earn +// [VERIFIED] from training memory or a bare grep, drift into RESEARCH.md, be lifted +// into PLAN.md's by the planner, and fail at the executor's typecheck. +// These assert the governed prose contract on the deployed agent definition. +// --------------------------------------------------------------------------- + +describe('gsd-phase-researcher in-repo value provenance rule (#1699)', () => { + const agentPath = path.join(ROOT, 'agents', 'gsd-phase-researcher.md'); + const read = () => fs.readFileSync(agentPath, 'utf-8'); + + test('names the in-repo discrete-value taxonomy the rule governs', () => { + const content = read(); + for (const term of ['enum', 'error code', 'status constant', 'filesystem path']) { + assert.ok( + content.includes(term), + `agent must name "${term}" as a governed in-repo discrete value` + ); + } + }); + + test('requires the source-of-truth file to be opened with Read this session', () => { + assert.match( + read(), + /opened the source-of-truth file with `Read` \*\*this session\*\*/, + 'agent must require a same-session Read of the source-of-truth file' + ); + }); + + test('requires both a path and a line range in the citation', () => { + const content = read(); + assert.match( + content, + /Cite the path \*\*and line range\*\*/, + 'agent must require path AND line range, not a bare path' + ); + assert.match( + content, + /\[VERIFIED: src\/types\/order\.ts:14-22\]/, + 'agent must carry a concrete path:line-range citation example' + ); + }); + + test('states that a codebase grep alone does not earn the tag', () => { + assert.match( + read(), + /codebase `grep` is not sufficient on its own/, + 'agent must exclude bare grep, which only proves a string occurs' + ); + }); + + test('requires a verbatim quote and forbids paraphrase', () => { + const content = read(); + assert.match( + content, + /quote the values \*\*verbatim\*\* in RESEARCH\.md beside the claim/, + 'the quote must land in RESEARCH.md, the researcher\'s own output' + ); + assert.ok( + content.includes('paraphrase is forbidden'), + 'agent must forbid paraphrase of in-repo discrete values' + ); + }); + + test('makes the quote the falsifiable artifact, not the citation (Goodhart guard)', () => { + assert.match( + read(), + /a citation with no quote beside it does not earn `\[VERIFIED\]`/, + 'a precise-looking line range without a quote must not earn the tag' + ); + }); + + test('routes an unquoted skeleton value to [ASSUMED]', () => { + assert.match( + read(), + /must also appear in that verbatim quote; a value that does not is `\[ASSUMED\]`/, + 'values used in examples but absent from the quote must degrade to [ASSUMED]' + ); + }); + + test('does not disturb the pre-existing package name provenance rule', () => { + const content = read(); + assert.ok( + content.includes('**Package name provenance rule:**'), + 'the sibling external-provenance rule must survive unchanged' + ); + assert.ok( + content.includes('a slopsquatted package also passes `npm view`'), + 'package-legitimacy reasoning must remain intact' + ); + }); + + test('defines the rule once — no paraphrased restatement (META.RULE.brief-no-paraphrase)', () => { + const occurrences = read().split('In-repo value provenance rule').length - 1; + assert.equal( + occurrences, + 1, + 'the rule must be defined at exactly one site; a second copy is the prose-drift mode' + ); + }); +});