* fix(#3724): stop advisory Dimension 3b findings from forcing the revision loop Dimension 3b (undeclared/temporal coupling, #1954) is spec'd "never a blocker" but tagged severity: warning — the tier plan-phase's revision loop counts as must-fix — and the planner is never taught the rule, so every multi-wave phase touching shared mutable state replans at least once, and intentionally coupled plans re-flag identically every iteration to the stall prompt. Three coordinated changes: - gsd-plan-checker: retag 3b to severity: info, the tier references/revision-loop.md already exempts by design; recognize a coupling_justified frontmatter declaration in the Do-NOT-flag list so deliberate pairs converge. Additions are offset by trimming 3b motivation prose — the checker sits 45 bytes under its LARGE hard cap. - plan-phase step 12: INFO-only accept — an issues block with zero BLOCKER/WARNING entries accepts the plan and surfaces the advisories instead of re-entering the revision loop. Real blockers and warnings still gate unconditionally. - gsd-planner: slim pointer in assign_waves to the new progressive-disclosure reference gsd-core/references/planner-coupling.md (the planner sits 19 chars under its own cap), which carries the shared-mutable-state rule and the coupling_justified escape hatch so first-pass plans avoid the finding when the coupling is unintentional. Documented the coupling_justified field in docs/reference/plan-md.md. Growth acks per #2914; inventory manifest and install-tree fixtures regenerated for the new reference file. Closes #3724 Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * test(#3724): pin Dimension 3b at severity: info The severity retag makes the old assertion (severity: warning) stale; lock the advisory tier from both directions — info must be present, warning must not — so a future edit cannot silently re-arm the revision-loop trigger. Refs #3724 Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * chore(#3724): changeset fragment for PR #3758 Refs #3724 Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * docs(#3724): roster planner-coupling.md in docs/INVENTORY.md The new reference was enumerated in the manifest and all 19 install-tree fixtures but missing its row in the Modular Planner Decomposition table — the roster half the manifest-sync test cannot check. (Review Blocker.) Refs #3724 Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * test(#3724): cover all four acceptance criteria (review round 1) - plan-checker-coupling: the 3b severity assertion is now a PARITY check deriving the exempt tier from revision-loop.md's flow instead of hardcoding info — editing either side alone reds the suite. New describe pins the other three criteria: plan-phase's INFO-only accept clause (proven failing-first), the BLOCKER + WARNING count staying intact, the coupling_justified Do-NOT-flag exemption + fix_hint, and the planner pointer + planner-coupling.md content. - ack fragment: $comment's plan-phase figure corrected to +79B; the 2775 pin note carried forward into the gsd-planner.md entry, updated for upstream's #3761/#3764 Rule-paragraph anchor (which this diff leaves verbatim). The parallel-dependent-plans re-anchor this commit originally carried was superseded by upstream #3764 during review; this branch no longer touches that file. Refs #3724 Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): review round 2 — align the stance enumeration, complete the template contract MAJOR: <adversarial_stance>'s severity enumeration gains the INFO bullet so it agrees with Dimension 3b's 'ALWAYS INFO' mandate instead of contradicting it. Funded by extracting the inline <examples> block to the new progressive- disclosure reference gsd-core/references/plan-checker-examples.md (@-inlined from the same spot; #1949 precedent), which also restores the 3b motivation clause round 1 traded away (Nit 4) and nets the agent file SMALLER than base (49107 -> 48486) — the extraction the byte pressure was owed. MINOR: gsd-core/templates/phase-prompt.md now carries coupling_justified, and the field's shape becomes one 'plan-id: reason' string per coupled peer so a plan justified against two peers can express it; docs/reference/plan-md.md's Type column names the shape. NIT: the 3409 ack's plan-phase entry no longer calls the #1168 workflow ratchet an 'XL tier'. Acks and derived artifacts updated accordingly (checker entry removed — a shrink needs no ack; INVENTORY roster row + regen:derived for the new file). Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * test(#3724): derive the 3b negative severity assertion (review round 2) Every severity token in the 3b span must BE the tier revision-loop.md exempts, replacing the hardcoded severity:warning negative — if the loop's exemption ever moves, the failure names the real conflict instead of blaming the agent file with a mutually-unsatisfiable pair. Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): refit the planner coupling pointer under the char cap Upstream #3299 (PR #3390) grew agents/gsd-planner.md to 49146 chars at the base, leaving 5 chars of headroom where the +16-char pointer was measured against 13 more. The pointer prose shortens to 'Non-file coupling:' — 49150 chars, back under the strict 49152-char cap — and the ack figures follow. The @-path the tests pin is unchanged. Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): re-home the plan-phase ack after the #3823 spent-fragment sweep Upstream #3078/#3823 deleted all fully-spent ack fragments, including 3409-unreachable-guard-arms.json, which carried this PR's plan-phase.md +79B append. Per the collision remedy that sweep added: take the deletion and home the still-live entry in this PR's own fragment. Figures re-measured at this merge base (90871 -> 90950 LF bytes). Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): absorb the spent #3172 plan-phase fragment into this PR's ack Upstream #3825 shipped 3172-stated-failing-direction.json naming only plan-phase.md, now spent at the base — colliding with this PR's live plan-phase entry. Per the #3003 pattern the fully-spent single-path fragment is deleted and this fragment stays the path's one source; figures re-measured at this base (93073 -> 93152 LF bytes). Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): review round 3 — true up the ack figures, restore the wave comment The fragment's absolute sizes are re-measured and anchored to basee40e9670(planner 47259 -> 47330 chars, checker 45537 -> 44916 B, plan-phase 91186 -> 91265 LF bytes), with a note that absolutes rot as next moves — the deltas are the durable claims. The round-1 removal of the '# Implicit dependency: files_modified overlap forces a later wave.' pseudocode comment offset headroom base drift had already returned, so it is restored (findings 2-3). Changeset gains the (#3724) backlink (finding 4). Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): review round 4 — close the verify-work surface, harden the boundaries BLOCKER: verify-work.md's verify_gap_plans is the second multi-plan consumer of the checker's sentinels, and its ISSUES FOUND handler entered revision_loop with zero severity parsing — the guaranteed replan #3724 fixed in plan-phase, alive on the gap-closure surface. The handler now counts BLOCKER + WARNING and accepts INFO-only returns with advisories displayed. The checker's INFO stance bullet is reworded to the claim that is true everywhere ('revision gates count only BLOCKER + WARNING'). Minor 1: plan-phase's iteration_count >= 3 arm recounts severities, so an INFO-only third check accepts instead of halting on a '0 issues remain' user gate. Minor 2: the coupling_justified exemption now requires the entry to NAME the other plan, closing the blanket-suppression reading. Nit 1: INVENTORY row states the extraction buys cap headroom, not context. Nit 2: the advisory display gains a concrete format on both surfaces. Ack fragment re-anchored at baseddde001a: verify-work.md +264B (new entry), plan-phase.md +395B, checker still net negative (-512B). Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * test(#3724): pin the verify-work accept and the iteration-cap boundary (review round 4) Two wiring assertions: verify_gap_plans' ISSUES FOUND handler gates on BLOCKER + WARNING and accepts INFO-only blocks, and plan-phase's iteration_count >= 3 arm recounts severities instead of gating advisories — the limit+1 boundary of the gate this PR fixes. Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): review round 5 — fail closed at the gates, surface the advisory Blocker 1: the checker's step-10 status rule routes an INFO-only result to ## ISSUES FOUND (with a new ### Advisories (info) template section and a severity-aware recommendation) so the orchestrator receives the block and displays the advisory instead of silently accepting a bare PASSED. Blockers 2+3: all three gate surfaces (plan-phase step 12 both arms, verify-work verify_gap_plans) carry one canonical clause verbatim — an entry whose severity is missing or unrecognized counts as a BLOCKER (fail closed) — making the accept condition an explicit-INFO whitelist while keeping issue_count coherent for stall math. Major 1: the INFO stance bullet scopes its claim to the plan-phase and verify-work gates (quick mode's loop still revises on any ISSUES FOUND). Major 2: INVENTORY row and ack $comment state the extraction's real trade (readability, +0.6 KB eager runtime context), not a cap remedy. Minor 1: plan-md.md marks coupling_justified as prompt convention, unvalidated. Nit 1: ack absolutes re-anchored at base 1e67ec97; checker now +120B and acked. Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * test(#3724): pin the round-5 contract — fail-closed parity, INFO-only return shape New: three-surface verbatim parity test for the fail-closed clause (Blockers 2+3); checker return-contract test for the INFO-only ## ISSUES FOUND route and advisories section (Blocker 1). All seven newly pinned tokens are absent at f3a5682d, so each new assertion fails pre-fix. Updated: accept-clause regexes track the explicit-INFO whitelist wording; the severity sweep scopes to the span's fenced yaml examples via yamlSeverityTiers (round-5 Minor 3, applied to the blocker negative too); the iteration-cap comment states it is a prose pin, not an executed boundary check (Minor 4); splitLines call sites document the line-pin coupling (Nit 2). Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): adopt next's line wrap in the 3b motivation clause — drops a wrap-only hunk from the diff Byte-identical content; the wrap difference was an artifact of the round-1 base adaptation predating upstream's #3003 landing. Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM * fix(#3724): review round 7 — gate every checker consumer, not just the two audited ones Blocker: quick/steps/plan-checker-loop.md (issue-named in #3724) gets the same canonical fail-closed clause and explicit-INFO whitelist accept as plan-phase/verify-work — an INFO-only result proceeds instead of entering quick mode's revision loop. Major: import.md plan_validate handles the checker return by severity (INFO-only never blocks an import) and is added to agent-contracts.md's consumer enumeration, which had omitted it. The checker's INFO stance bullet drops the quick-mode carve-out — the claim is universally true again now that every consuming gate is severity-aware. Minor: an applied coupling_justified exemption is surfaced as its own info advisory so a stale one-sided declaration stays observable. Nit: plan-phase's revision-iteration Display line is explicitly conditioned on not having already proceeded to step 13. Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM Emitted-Drift-Ack-Growth: import.md — #3724 round 7: the plan_validate step's checker-return handler becomes severity-aware — counts BLOCKER + WARNING failing closed and accepts an explicitly-INFO-only return with advisories displayed instead of blocking the import * test(#3724): pin the round-7 surfaces — five-gate parity, quick/import accepts, exemption visibility The verbatim fail-closed parity test extends to quick/steps/plan-checker-loop.md and import.md plan_validate; new assertions pin quick mode's INFO-only proceed, import's never-blocks accept, import.md's presence in agent-contracts.md's consumer row, and the surfaced coupling_justified exemption advisory. All four newly pinned token families are absent at the pre-fix head, so each new assertion fails first. Claude-Session: https://claude.ai/code/session_01GshUzpGjoxiw6uNRiFMHvM --------- Co-authored-by: Tom Boucher <trekkie@nomorestars.com>
579 lines
28 KiB
JavaScript
579 lines
28 KiB
JavaScript
// allow-test-rule: source-text-is-the-product — the plan-checker is a prompt; its .md text IS what the runtime loads (#1954)
|
|
|
|
/**
|
|
* Dimension 3b — undeclared / temporal coupling between same-wave plans (#1954).
|
|
*
|
|
* `gsd-plan-checker` proves plan dependencies resolve and are acyclic (Dimension 3),
|
|
* and `/gsd:execute-phase` separately proves same-wave plans do not overlap in
|
|
* `files_modified`. Neither axis sees coupling that is real but undeclared — plan A
|
|
* writes a config key / table / migration / global module that plan B reads, or B
|
|
* only works if A ran first. In parallel execution that surfaces as an intermittent
|
|
* failure the executor cannot attribute.
|
|
*
|
|
* ## What this suite locks
|
|
*
|
|
* The deployed contract AND the wiring between the three regions of the agent doc that
|
|
* must agree: the sub-check body, its severity rule, and the `<success_criteria>`
|
|
* checklist. A sub-check present in the body but absent from `success_criteria` is a
|
|
* check the agent is never told to run; the reverse is a checklist item with no rubric.
|
|
* Neither half is observable from the other, which is why both are asserted here.
|
|
*
|
|
* ## What it cannot prove
|
|
*
|
|
* That the model acts on the text. The subject is an LLM prompt — no test in this repo
|
|
* can prove behavior for any of the agent's twelve existing dimensions either. Stated
|
|
* so the coverage claim is honest rather than implied.
|
|
*
|
|
* Patterns are CRLF-tolerant (`\r?\n`): the runtime loads the file whole, including on
|
|
* a checkout that produced CRLF, the same case `scripts/workflow-size.cjs` defends.
|
|
*/
|
|
|
|
const { describe, test } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const { stripFencedCode } = require('../gsd-core/bin/lib/markdown-sectionizer.cjs');
|
|
|
|
const ROOT = path.join(__dirname, '..');
|
|
const AGENT_PATH = path.join(ROOT, 'agents', 'gsd-plan-checker.md');
|
|
const DOCS_AGENTS_PATH = path.join(ROOT, 'docs', 'AGENTS.md');
|
|
const REVISION_LOOP_PATH = path.join(ROOT, 'gsd-core', 'references', 'revision-loop.md');
|
|
const PLAN_PHASE_PATH = path.join(ROOT, 'gsd-core', 'workflows', 'plan-phase.md');
|
|
const PLANNER_PATH = path.join(ROOT, 'agents', 'gsd-planner.md');
|
|
const PLANNER_COUPLING_REF_PATH = path.join(ROOT, 'gsd-core', 'references', 'planner-coupling.md');
|
|
|
|
const VERIFY_WORK_PATH = path.join(ROOT, 'gsd-core', 'workflows', 'verify-work.md');
|
|
const QUICK_LOOP_PATH = path.join(ROOT, 'gsd-core', 'workflows', 'quick', 'steps', 'plan-checker-loop.md');
|
|
const IMPORT_PATH = path.join(ROOT, 'gsd-core', 'workflows', 'import.md');
|
|
const AGENT_CONTRACTS_PATH = path.join(ROOT, 'gsd-core', 'references', 'agent-contracts.md');
|
|
const { splitLines } = require('../gsd-core/bin/lib/text-lines.cjs');
|
|
|
|
const agentDoc = fs.readFileSync(AGENT_PATH, 'utf-8');
|
|
const docsAgents = fs.readFileSync(DOCS_AGENTS_PATH, 'utf-8');
|
|
|
|
// Severity tokens inside a span's fenced ```yaml examples only. Prose may
|
|
// legitimately contrast another tier ("this shape would be a blocker — see
|
|
// Dimension 9"); the yaml examples are the declaration the model copies, so
|
|
// severity assertions scope here (round-5 Minor 3).
|
|
function yamlSeverityTiers(span) {
|
|
const tiers = [];
|
|
let inYaml = false;
|
|
for (const line of splitLines(span)) {
|
|
if (line.trim() === '```yaml') { inYaml = true; continue; }
|
|
if (inYaml && line.trim() === '```') { inYaml = false; continue; }
|
|
const m = inYaml ? line.match(/severity:\s*(\w+)/) : null;
|
|
if (m) tiers.push(m[1].toLowerCase());
|
|
}
|
|
return tiers;
|
|
}
|
|
|
|
// ── Span helpers ───────────────────────────────────────────────────
|
|
// Offsets of the headings that bound each region. `indexOfHeading` returns -1 when
|
|
// absent so a missing heading fails as a named assertion rather than an off-by-one.
|
|
function indexOfHeading(content, pattern) {
|
|
const m = content.match(pattern);
|
|
return m && typeof m.index === 'number' ? m.index : -1;
|
|
}
|
|
|
|
const D3_HEADING = /^## Dimension 3: /m;
|
|
const D3B_HEADING = /^## Dimension 3b: /m;
|
|
const D4_HEADING = /^## Dimension 4: /m;
|
|
|
|
function sliceBetween(content, startPattern, endPattern) {
|
|
const start = indexOfHeading(content, startPattern);
|
|
const end = indexOfHeading(content, endPattern);
|
|
assert.ok(start >= 0, `start heading not found: ${startPattern}`);
|
|
assert.ok(end >= 0, `end heading not found: ${endPattern}`);
|
|
assert.ok(end > start, `end heading precedes start heading: ${startPattern} .. ${endPattern}`);
|
|
return content.slice(start, end);
|
|
}
|
|
|
|
/**
|
|
* Count top-level ordered-list items in a span. This is the trigger gate's arity —
|
|
* the "flag only when ALL N hold" conjunction. Widening it from 3 to 2 is what turns
|
|
* a precise heuristic into a noise generator, so the count is asserted, not the prose.
|
|
*/
|
|
function countOrderedItems(span) {
|
|
const matches = span.match(/^\d+\. /gm);
|
|
return matches ? matches.length : 0;
|
|
}
|
|
|
|
/**
|
|
* Remove fenced code blocks before scanning for headings. The agent's
|
|
* `### Dimension 8 Output` section embeds a literal `## Dimension 8: ...` line inside a
|
|
* fence as its output template, so a fence-blind scan reports Dimension 8 twice and any
|
|
* uniqueness or completeness check built on it is wrong before it starts.
|
|
*/
|
|
function stripFences(content) {
|
|
return stripFencedCode(content).text;
|
|
}
|
|
|
|
describe('gsd-plan-checker Dimension 3b — undeclared/temporal coupling (#1954)', () => {
|
|
describe('the sub-check exists and is scoped to Dimension 3', () => {
|
|
test('Dimension 3 carries an undeclared-coupling sub-check', () => {
|
|
const heading = agentDoc.match(/^## Dimension 3b: (.+)$/m);
|
|
assert.ok(heading, 'agents/gsd-plan-checker.md must define a "## Dimension 3b:" heading');
|
|
assert.match(
|
|
heading[1],
|
|
/coupling/i,
|
|
`Dimension 3b must name coupling as its subject, got: ${heading[1]}`
|
|
);
|
|
});
|
|
|
|
test('the sub-check sits inside Dimension 3, not after it', () => {
|
|
const d3 = indexOfHeading(agentDoc, D3_HEADING);
|
|
const d3b = indexOfHeading(agentDoc, D3B_HEADING);
|
|
const d4 = indexOfHeading(agentDoc, D4_HEADING);
|
|
assert.ok(d3 >= 0 && d3b >= 0 && d4 >= 0, 'Dimensions 3, 3b and 4 must all be present');
|
|
assert.ok(d3 < d3b, 'Dimension 3b must follow Dimension 3');
|
|
assert.ok(d3b < d4, 'Dimension 3b must precede Dimension 4 — it extends Dimension 3');
|
|
});
|
|
|
|
test('the sub-check scopes comparison to plan pairs', () => {
|
|
// Parallelism is per PLAN (execute-phase spawns one executor per plan per wave),
|
|
// so two tasks inside one plan run sequentially and cannot race. Scoping the
|
|
// comparison to task pairs would flag orderings that are guaranteed by construction.
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.match(
|
|
span,
|
|
/Scope:\s*PLAN pairs, not tasks/,
|
|
'Dimension 3b must state that the comparison is plan-pair scoped'
|
|
);
|
|
});
|
|
});
|
|
|
|
describe('the trigger gate is a three-way conjunction', () => {
|
|
test('the trigger gate enumerates exactly three conditions', () => {
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.strictEqual(
|
|
countOrderedItems(span),
|
|
3,
|
|
'Dimension 3b must gate on exactly three AND-ed conditions ' +
|
|
'(same wave, no declared edge, named shared mutable resource or produced-state ' +
|
|
'prerequisite). Dropping one widens the heuristic into noise; adding one ' +
|
|
'silently narrows what it can catch.'
|
|
);
|
|
});
|
|
|
|
test('condition counter fires at 2 / 3 / 4', () => {
|
|
// The assertion above can only ever observe the real doc's arity, so its
|
|
// inequality branch never executes. Exercise the counter at limit-1 / limit /
|
|
// limit+1 (RULESET.TESTS.boundary-coverage) through the SAME function the guard
|
|
// uses, in both LF and CRLF form, so a future edit cannot neuter it.
|
|
const item = (n) => `${n}. condition ${n}`;
|
|
for (const eol of ['\n', '\r\n']) {
|
|
const spanOf = (count) =>
|
|
['## Dimension 3b: heading', ...Array.from({ length: count }, (_, i) => item(i + 1))]
|
|
.join(eol);
|
|
assert.strictEqual(countOrderedItems(spanOf(2)), 2, `2 items must count as 2 (eol=${JSON.stringify(eol)})`);
|
|
assert.strictEqual(countOrderedItems(spanOf(3)), 3, `3 items must count as 3 (eol=${JSON.stringify(eol)})`);
|
|
assert.strictEqual(countOrderedItems(spanOf(4)), 4, `4 items must count as 4 (eol=${JSON.stringify(eol)})`);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('severity is advisory, and stays advisory', () => {
|
|
test('the sub-check severity is the tier the revision loop exempts (#3724 parity)', () => {
|
|
// #3724's defect was exactly this coming apart: 3b spec'd "advisory" but
|
|
// tagged `warning`, the tier the revision loop treats as must-fix. The
|
|
// exempt tier is therefore DERIVED from revision-loop.md's flow, never
|
|
// hardcoded — if the loop's exemption ever changes, this fails instead
|
|
// of silently re-opening the guaranteed-replan defect.
|
|
const loopDoc = fs.readFileSync(REVISION_LOOP_PATH, 'utf-8');
|
|
const exempt = loopDoc.match(/If PASSED or only (\w+)-level issues/);
|
|
assert.ok(exempt, 'revision-loop.md must state its exempt severity tier in the flow');
|
|
const tier = exempt[1].toLowerCase();
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.match(
|
|
span,
|
|
new RegExp(`severity:\\s*${tier}`),
|
|
`Dimension 3b's example issue must carry severity: ${tier} — the tier revision-loop.md exempts`
|
|
);
|
|
// The negative is derived too: every severity token in the span's yaml
|
|
// examples must BE the exempt tier. If revision-loop.md's exemption ever
|
|
// moves, this fails naming the real conflict instead of blaming the agent
|
|
// file with a stale hardcode.
|
|
const tiersInSpan = yamlSeverityTiers(span);
|
|
assert.ok(tiersInSpan.length > 0, 'Dimension 3b must carry at least one severity-tagged example');
|
|
for (const found of tiersInSpan) {
|
|
assert.strictEqual(
|
|
found,
|
|
tier,
|
|
`Dimension 3b carries severity: ${found}, but the only tier revision-loop.md exempts is ` +
|
|
`${tier} — a non-exempt tier re-arms the revision loop`
|
|
);
|
|
}
|
|
});
|
|
|
|
test('the sub-check forbids escalating to blocker', () => {
|
|
// The agent's own <adversarial_stance> penalises "issuing warnings for what are
|
|
// actually blockers", which biases the model to escalate. Issue #1954 rejected the
|
|
// hard-block alternative outright, so the prohibition has to be explicit in the span.
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.match(
|
|
span,
|
|
/never\s+(a\s+)?blocker/i,
|
|
'Dimension 3b must state that the finding is never a blocker'
|
|
);
|
|
assert.ok(
|
|
!yamlSeverityTiers(span).includes('blocker'),
|
|
'Dimension 3b must not contain a blocker-severity example — it is advisory only'
|
|
);
|
|
});
|
|
|
|
test('existing Dimension 3 blocker severities are unchanged', () => {
|
|
// Independence: 3b is additive. The circular-dependency finding above it must
|
|
// still block, or this change quietly downgraded a real gate.
|
|
const d3Body = sliceBetween(agentDoc, D3_HEADING, D3B_HEADING);
|
|
assert.match(
|
|
d3Body,
|
|
/severity:\s*blocker/,
|
|
'Dimension 3\'s own example issue must still be severity: blocker'
|
|
);
|
|
assert.match(
|
|
d3Body,
|
|
/Circular dependency/,
|
|
'Dimension 3 must still carry its circular-dependency example'
|
|
);
|
|
});
|
|
|
|
test('the finding reuses the dependency_correctness dimension key', () => {
|
|
// Hyrum: anything consuming the checker's structured issues keys on `dimension`.
|
|
// A new key would be a new observable contract; the issue asked for a finding
|
|
// "under Dimension 3", not a new dimension.
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.match(
|
|
span,
|
|
/dimension:\s*dependency_correctness/,
|
|
'Dimension 3b\'s example issue must reuse the dependency_correctness dimension key'
|
|
);
|
|
});
|
|
});
|
|
|
|
describe('negative space is enumerated', () => {
|
|
test('the sub-check enumerates its non-triggering cases', () => {
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.match(span, /Do NOT flag/, 'Dimension 3b must carry an explicit non-triggering list');
|
|
// Each token below is a distinct exclusion class from the design's negative space.
|
|
// Their absence is what produced false positives in the alternatives considered.
|
|
for (const [token, why] of [
|
|
[/files_modified/, 'the file-overlap axis is already checked elsewhere — report it once'],
|
|
[/depends_on/, 'a pair whose edge is already declared is not a finding'],
|
|
[/different wave/i, 'the wave itself already orders the pair'],
|
|
[/READ/, 'two readers of a shared resource are not coupled'],
|
|
]) {
|
|
assert.match(span, token, `Dimension 3b non-triggering list must cover: ${why}`);
|
|
}
|
|
});
|
|
|
|
test('the sub-check defers transform conflicts to Dimension 9', () => {
|
|
// Dimension 9 (Cross-Plan Data Contracts) owns incompatible transformations of a
|
|
// shared entity. Without this boundary the same plan pair is reported twice.
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.match(
|
|
span,
|
|
/Dimension 9/,
|
|
'Dimension 3b must defer incompatible-transform findings to Dimension 9'
|
|
);
|
|
});
|
|
});
|
|
|
|
describe('the sub-check is wired into the agent\'s completion checklist', () => {
|
|
test('success_criteria includes the coupling check', () => {
|
|
const span = sliceBetween(agentDoc, /<success_criteria>/, /<\/success_criteria>/);
|
|
assert.match(
|
|
span,
|
|
/^- \[ \] .*coupling.*$/im,
|
|
'the <success_criteria> checklist must carry a line for the coupling check — ' +
|
|
'a rubric the agent is never told to run is not a check'
|
|
);
|
|
});
|
|
});
|
|
|
|
describe('docs parity', () => {
|
|
// Parity against the agent's own headings is self-maintaining; a hand-typed count is
|
|
// not. The section previously claimed "8 Verification Dimensions" while enumerating
|
|
// names that matched no dimension in the agent at all — a stale count reads as
|
|
// authoritative, which is worse than no count.
|
|
function agentDimensionLabels() {
|
|
return [...stripFences(agentDoc).matchAll(/^## Dimension ([0-9]+[a-z]?): /gm)].map((m) => m[1]);
|
|
}
|
|
|
|
test('the agent defines a discoverable set of numbered dimensions', () => {
|
|
const labels = agentDimensionLabels();
|
|
assert.ok(
|
|
labels.length >= 12,
|
|
`expected the agent to define at least 12 numbered dimensions, found ${labels.length}`
|
|
);
|
|
assert.ok(labels.includes('3b'), 'Dimension 3b must be among the agent\'s numbered dimensions');
|
|
assert.strictEqual(
|
|
new Set(labels).size,
|
|
labels.length,
|
|
`duplicate dimension labels in the agent: ${labels.join(', ')}`
|
|
);
|
|
});
|
|
|
|
test('docs/AGENTS.md enumerates every dimension the agent defines', () => {
|
|
const section = sliceBetween(docsAgents, /^### gsd-plan-checker$/m, /^### gsd-integration-checker$/m);
|
|
const documented = new Set(
|
|
[...section.matchAll(/^\| ([0-9]+[a-z]?) \| /gm)].map((m) => m[1])
|
|
);
|
|
const missing = agentDimensionLabels().filter((label) => !documented.has(label));
|
|
assert.deepStrictEqual(
|
|
missing,
|
|
[],
|
|
`docs/AGENTS.md omits dimension(s): ${missing.join(', ')}`
|
|
);
|
|
});
|
|
|
|
test('docs/AGENTS.md documents the coupling check', () => {
|
|
const section = sliceBetween(docsAgents, /^### gsd-plan-checker$/m, /^### gsd-integration-checker$/m);
|
|
assert.match(
|
|
section,
|
|
/coupling/i,
|
|
'docs/AGENTS.md\'s gsd-plan-checker section must document the coupling check'
|
|
);
|
|
});
|
|
});
|
|
|
|
describe('#3724 — the advisory contract holds across the wiring', () => {
|
|
// The defect #3724 fixed lived in three places at once: the checker's tier,
|
|
// the orchestrator's loop gate, and the planner's ignorance of the rule.
|
|
// Each assertion here pins one side; reverting any one of them alone must
|
|
// red this suite, because #3237 shipping 3b with no orchestration-side
|
|
// assertion is exactly how the defect arrived.
|
|
|
|
test('plan-phase accepts an INFO-only issues block without entering the revision loop', () => {
|
|
const planPhase = fs.readFileSync(PLAN_PHASE_PATH, 'utf-8');
|
|
// Pins the full single-line paragraph: a reflow of that line in plan-phase.md
|
|
// reds this find() — update the startsWith prefix and the regexes together.
|
|
const paragraph = splitLines(planPhase).find((line) =>
|
|
line.startsWith('Parse issue count from checker return:')
|
|
);
|
|
assert.ok(paragraph, 'plan-phase.md step 12 must carry the parse-issue-count paragraph');
|
|
assert.match(
|
|
paragraph,
|
|
/likewise when every entry in the block is explicitly INFO \(display them as advisories\)/,
|
|
'step 12 must accept only an explicitly-INFO issues block, surfacing the advisories (#3724 criterion 1)'
|
|
);
|
|
});
|
|
|
|
test('plan-phase still counts BLOCKER + WARNING for the revision gate', () => {
|
|
// Criterion 2's orchestration half: severity-blindness must not invert.
|
|
// INFO joining the count would re-arm the loop; BLOCKER or WARNING
|
|
// leaving it would let real defects through.
|
|
const planPhase = fs.readFileSync(PLAN_PHASE_PATH, 'utf-8');
|
|
assert.match(
|
|
planPhase,
|
|
/count BLOCKER \+ WARNING entries in the YAML issues block/,
|
|
'step 12 must keep gating the revision loop on BLOCKER + WARNING counts'
|
|
);
|
|
});
|
|
|
|
test('the checker recognizes coupling_justified as a Do-NOT-flag exemption', () => {
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.match(
|
|
span,
|
|
/declared `coupling_justified` in either plan's frontmatter/,
|
|
'Dimension 3b must exempt a coupling_justified pair so intentional coupling can converge (#3724 criterion 4)'
|
|
);
|
|
assert.match(
|
|
span,
|
|
/fix_hint:[^\n]*coupling_justified/,
|
|
'the 3b fix_hint must name coupling_justified so the planner learns the escape hatch'
|
|
);
|
|
});
|
|
|
|
test('verify-work accepts an INFO-only issues block without entering its revision loop', () => {
|
|
// Round-4 Blocker: verify_gap_plans is the SECOND multi-plan consumer of the
|
|
// checker's sentinels (agent-contracts.md), spawning it over all phase plans with
|
|
// no dimension override — so 3b is live there and its handler must be
|
|
// severity-aware, or the guaranteed replan #3724 fixed survives on that surface.
|
|
const verifyWork = fs.readFileSync(VERIFY_WORK_PATH, 'utf-8');
|
|
// Pins the full single-line handler: a reflow of that line in verify-work.md
|
|
// reds this find() — update the startsWith prefix and the regexes together.
|
|
const handler = splitLines(verifyWork).find((line) =>
|
|
line.startsWith('- **ISSUES FOUND:**')
|
|
);
|
|
assert.ok(handler, 'verify-work.md must carry the ISSUES FOUND handler line');
|
|
assert.match(
|
|
handler,
|
|
/Count BLOCKER \+ WARNING/,
|
|
'the verify_gap_plans handler must gate its revision loop on BLOCKER + WARNING counts'
|
|
);
|
|
assert.match(
|
|
handler,
|
|
/every entry is explicitly INFO/,
|
|
'the verify_gap_plans handler must accept only an explicitly-INFO issues block (whitelist, not count-zero)'
|
|
);
|
|
});
|
|
|
|
test('plan-phase iteration cap recounts severities instead of gating on advisories', () => {
|
|
// Round-4 Minor 1: the INFO-only accept must hold at iteration_count >= 3 too,
|
|
// or an advisory-only third check halts the workflow on a "0 issues remain"
|
|
// user gate. This is a prose pin, not an executed boundary check: the >= 3 arm's
|
|
// text is what both the limit and limit+1 iterations land on, so one string
|
|
// match covers both — nothing here runs a counter (round-5 Minor 4).
|
|
const planPhase = fs.readFileSync(PLAN_PHASE_PATH, 'utf-8');
|
|
const lines = splitLines(planPhase);
|
|
const armIndex = lines.findIndex((line) => line.startsWith('**If iteration_count >= 3:**'));
|
|
assert.ok(armIndex >= 0, 'plan-phase.md must carry the iteration_count >= 3 arm');
|
|
const armWindow = lines.slice(armIndex, armIndex + 5).join(' ');
|
|
assert.match(
|
|
armWindow,
|
|
/Recount BLOCKER \+ WARNING/,
|
|
'the >= 3 arm must recount BLOCKER + WARNING before gating'
|
|
);
|
|
assert.match(
|
|
armWindow,
|
|
/explicitly INFO — display any advisories and proceed to step 13/,
|
|
'an INFO-only result at the iteration cap must accept, not halt on the user gate'
|
|
);
|
|
});
|
|
|
|
test('the fail-closed severity rule is verbatim-identical on all three gate surfaces', () => {
|
|
// Round-5 Blockers 2+3: a count-based accept ("BLOCKER + WARNING count is zero")
|
|
// is also true for an entry whose severity is missing, misspelled, or
|
|
// unrecognized — auto-accepting what base sent to the revision loop. All three
|
|
// gates carry one canonical clause, asserted verbatim, so the predicates cannot
|
|
// drift apart again (Generative Fix Divergence).
|
|
const CLAUSE = 'an entry whose severity is missing or unrecognized counts as a BLOCKER (fail closed)';
|
|
const planPhase = fs.readFileSync(PLAN_PHASE_PATH, 'utf-8');
|
|
const verifyWork = fs.readFileSync(VERIFY_WORK_PATH, 'utf-8');
|
|
const lines = splitLines(planPhase);
|
|
const parseLine = lines.find((line) => line.startsWith('Parse issue count from checker return:'));
|
|
assert.ok(
|
|
parseLine && parseLine.includes(CLAUSE),
|
|
'the plan-phase iteration_count < 3 arm must carry the fail-closed clause verbatim'
|
|
);
|
|
const armIndex = lines.findIndex((line) => line.startsWith('**If iteration_count >= 3:**'));
|
|
assert.ok(armIndex >= 0, 'plan-phase.md must carry the iteration_count >= 3 arm');
|
|
const armWindow = lines.slice(armIndex, armIndex + 5).join(' ');
|
|
assert.ok(
|
|
armWindow.includes(CLAUSE),
|
|
'the plan-phase iteration_count >= 3 arm must carry the fail-closed clause verbatim'
|
|
);
|
|
const handler = splitLines(verifyWork).find((line) => line.startsWith('- **ISSUES FOUND:**'));
|
|
assert.ok(
|
|
handler && handler.includes(CLAUSE),
|
|
'the verify-work verify_gap_plans handler must carry the fail-closed clause verbatim'
|
|
);
|
|
// Round-7 Blocker + Major: the checker's INFO-only ## ISSUES FOUND contract is
|
|
// dimension-agnostic and reaches every consumer, so the two remaining
|
|
// severity-blind handlers get the same clause — quick mode (issue-named in
|
|
// #3724) and import's plan_validate (absent even from agent-contracts.md
|
|
// until this round).
|
|
const quickLoop = fs.readFileSync(QUICK_LOOP_PATH, 'utf-8');
|
|
const quickHandler = splitLines(quickLoop).find((line) => line.startsWith('- **`## ISSUES FOUND`:**'));
|
|
assert.ok(
|
|
quickHandler && quickHandler.includes(CLAUSE),
|
|
'the quick-mode plan-checker-loop handler must carry the fail-closed clause verbatim'
|
|
);
|
|
const importDoc = fs.readFileSync(IMPORT_PATH, 'utf-8');
|
|
const importHandler = splitLines(importDoc).find((line) => line.startsWith('Handle the checker return by severity'));
|
|
assert.ok(
|
|
importHandler && importHandler.includes(CLAUSE),
|
|
'the import plan_validate handler must carry the fail-closed clause verbatim'
|
|
);
|
|
});
|
|
|
|
test('quick mode and import accept an explicitly-INFO-only issues block', () => {
|
|
const quickLoop = fs.readFileSync(QUICK_LOOP_PATH, 'utf-8');
|
|
const quickHandler = splitLines(quickLoop).find((line) => line.startsWith('- **`## ISSUES FOUND`:**'));
|
|
assert.ok(quickHandler, 'plan-checker-loop.md must carry the ISSUES FOUND handler line');
|
|
assert.match(
|
|
quickHandler,
|
|
/every entry is explicitly INFO/,
|
|
'quick mode must accept only an explicitly-INFO issues block (whitelist, not count-zero)'
|
|
);
|
|
assert.match(
|
|
quickHandler,
|
|
/proceed to step 6/,
|
|
'an INFO-only result in quick mode must proceed, not enter the revision loop'
|
|
);
|
|
const importDoc = fs.readFileSync(IMPORT_PATH, 'utf-8');
|
|
const importHandler = splitLines(importDoc).find((line) => line.startsWith('Handle the checker return by severity'));
|
|
assert.ok(importHandler, 'import.md plan_validate must carry the severity-aware handler paragraph');
|
|
assert.match(
|
|
importHandler,
|
|
/never blocks an import/,
|
|
'an INFO-only checker return must not block an import'
|
|
);
|
|
const contracts = fs.readFileSync(AGENT_CONTRACTS_PATH, 'utf-8');
|
|
const checkerRow = splitLines(contracts).find((line) => line.startsWith('| gsd-plan-checker |'));
|
|
assert.ok(
|
|
checkerRow && checkerRow.includes('gsd-core/workflows/import.md'),
|
|
'agent-contracts.md must list import.md as a gsd-plan-checker sentinel consumer'
|
|
);
|
|
});
|
|
|
|
test('an applied coupling_justified exemption stays observable', () => {
|
|
// Round-7 Minor: the exemption fires from a one-sided, schema-unvalidated
|
|
// declaration; without a surfaced note, a stale or copy-pasted entry
|
|
// suppresses the check silently and permanently.
|
|
const span = sliceBetween(agentDoc, D3B_HEADING, D4_HEADING);
|
|
assert.match(
|
|
span,
|
|
/note the applied\s+exemption as its own `info` advisory/,
|
|
'Dimension 3b must surface an applied coupling_justified exemption as an info advisory'
|
|
);
|
|
});
|
|
|
|
test('the checker returns ## ISSUES FOUND for an INFO-only result, with an advisories section', () => {
|
|
// Round-5 Blocker 1: with 3b at severity info, an INFO-only result satisfied
|
|
// the old step-10 `passed` rule and routed to ## VERIFICATION PASSED — a
|
|
// template with no issues block — so the advisory this fix exists to surface
|
|
// was dropped, and both orchestrator display clauses were unreachable.
|
|
assert.match(
|
|
agentDoc,
|
|
/An INFO-only result is NOT `passed`/,
|
|
'step 10 must exclude an INFO-only result from `passed`'
|
|
);
|
|
assert.match(
|
|
agentDoc,
|
|
/Return `## ISSUES FOUND` even when every issue is INFO/,
|
|
'step 10 must route an INFO-only result to ## ISSUES FOUND so the block reaches the orchestrator'
|
|
);
|
|
assert.match(
|
|
agentDoc,
|
|
/### Advisories \(info\)/,
|
|
'the ISSUES FOUND template must carry an advisories section so INFO entries render'
|
|
);
|
|
assert.match(
|
|
agentDoc,
|
|
/Advisory only — no revision required/,
|
|
'the recommendation must not claim a planner return for an INFO-only result'
|
|
);
|
|
});
|
|
|
|
test('the planner routes to the coupling reference, and the reference teaches the rule', () => {
|
|
const planner = fs.readFileSync(PLANNER_PATH, 'utf-8');
|
|
assert.match(
|
|
planner,
|
|
/@~\/\.claude\/gsd-core\/references\/planner-coupling\.md/,
|
|
'gsd-planner.md must point at the planner-coupling reference (#3724 criterion 3)'
|
|
);
|
|
assert.ok(
|
|
fs.existsSync(PLANNER_COUPLING_REF_PATH),
|
|
'gsd-core/references/planner-coupling.md must exist — the planner pointer routes there'
|
|
);
|
|
const ref = fs.readFileSync(PLANNER_COUPLING_REF_PATH, 'utf-8');
|
|
assert.match(
|
|
ref,
|
|
/mutable\s+resource/i,
|
|
'planner-coupling.md must state the shared-mutable-resource rule'
|
|
);
|
|
assert.match(
|
|
ref,
|
|
/coupling_justified/,
|
|
'planner-coupling.md must document the coupling_justified declaration'
|
|
);
|
|
assert.match(
|
|
ref,
|
|
/Dimension 3b/,
|
|
'planner-coupling.md must name the verifying side (Dimension 3b)'
|
|
);
|
|
});
|
|
});
|
|
});
|