* test(#1700): add failing-first regression test for spike MANIFEST.md idea scoping Pins the deployed-text contract for spike.md's create_manifest step and spike-wrap-up.md's gather/synthesize/write_skill steps before the fix lands, so the fix commit demonstrates RED to GREEN. * test(#1700): add issue ref to allow-test-rule exemption lint-allow-test-rule-refs.cjs (ADR-456) requires a tracking-issue reference on the same line as any new allow-test-rule: exemption. * fix(#1700): scope spike MANIFEST.md Idea/Requirements per idea key .planning/spikes/MANIFEST.md held one un-keyed ## Idea paragraph and one flat ## Requirements list, while spike.md's own read path (load_prior_context step d, frontier mode) assumed the tree holds several unrelated ideas across campaigns. spike-wrap-up.md then copied that flat Requirements list verbatim into every generated feature-area reference and the generated spike-findings skill, so one idea's requirements leaked into another idea's build as non-negotiable constraints. Scopes both sides by an explicit idea key: - spike.md's create_manifest step now writes MANIFEST.md as ## Ideas > ### {idea-key} subsections (idea paragraph + its own Requirements list), appends a new idea section instead of overwriting an existing one, and migrates a pre-fix flat-shape MANIFEST.md in place instead of discarding it. The ## Spikes table gains an Idea column. Spike README frontmatter carries the idea key. - spike-wrap-up.md's gather/synthesize/write_skill steps resolve each spike's idea key and pull Requirements only from the idea key(s) actually represented among the spikes being wrapped. - sketch.md's MANIFEST.md read (collateral of the same seam) is updated to match the new per-idea section shape. - references/artifact-types.md no longer describes the project-level, durable-index MANIFEST.md as "(per-spike)". Not a docs-vs-workflow conflict: docs/how-to/spike-and-sketch.md already matches this contract ("all spikes are indexed in MANIFEST.md") and needed no change; the defect was internal to spike.md (per-idea read assumption, single-idea write template). * test(#1700): make idea-frontmatter regex CRLF-safe CONTRIBUTING.md's cross-platform portability rules (docs/contributing/ cross-platform-portability-rules.md) ban a bare \n literal matched against fs.readFileSync'd content — Windows checkouts normalize to CRLF and would silently break the assertion. Use \r?\n instead. Found during self-review (code-review skill, Standards axis) before this branch was handed off for verification. * test(#1700): acknowledge emitted-size growth in spike.md, spike-wrap-up.md, sketch.md The differential attribution check (tests/emitted-attribution.test.cjs, ADR-2719) flagged unattributed byte growth in three emitted workflow files. All three grew as a direct, necessary consequence of the #1700 fix (idea-key scoping in spike.md's create_manifest, spike-wrap-up.md's gather/synthesize/ write_skill, and the matching one-line update to sketch.md's MANIFEST.md read) — none of it is incidental prose. Growth stays well under the DEFAULT tier byte cap (40960) for all three files; PRE_PHASE6 does not apply to them (only plan-phase.md/execute-phase.md). * chore(#1700): backfill changeset PR number to 3014 --------- Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"version": 1,
|
||||
"paths": {
|
||||
"spike.md": "#1700: create_manifest step now derives an explicit idea key and restructures the .planning/spikes/MANIFEST.md template from one flat top-level `## Idea` / `## Requirements` pair (single-idea only) to `## Ideas` > `### {idea-key}` subsections, each carrying its own scoped Requirements list — the fix for the leak where one idea's requirements bled into another idea's build via spike-wrap-up.md. Growth is: the idea-key derivation instructions and the expanded MANIFEST.md template (new/existing-idea/pre-#1700-flat-shape-migration branches) in create_manifest; an Idea column added to the `## Spikes` table header everywhere it's referenced; an `idea:` field added to the build_spikes README frontmatter template; the reground step scoping its established-requirements check to the current idea only; the frontier_mode load step and 'Get Alignment and Execute' step describing per-idea sections instead of one flat idea; and two success_criteria bullets updated to call out idea-key scoping. This is the write side of the fix, not incidental prose.",
|
||||
"spike-wrap-up.md": "#1700: gather/synthesize/write_skill steps now resolve each spike's idea key (from its README `idea:` frontmatter, falling back to the `## Spikes` table's Idea column for pre-#1700 spikes) and scope which Requirements they read and re-emit, so a feature-area reference or the generated spike-findings SKILL.md never pulls in an unrelated idea's requirements — the read/re-emit side of the #1700 fix. Growth is: the idea-key resolution instructions added to the gather step; the synthesize step's Requirements block rewritten to pull ONLY from the owning idea key(s) with an explicit 'never include a requirement from an idea key that has no spike in this group' rule; the write_skill step's <context> and <requirements> blocks scoped to the wrapped idea key(s) instead of 'the whole MANIFEST.md'; and one new success_criteria bullet forbidding blended requirements across ideas.",
|
||||
"sketch.md": "#1700: sketch.md's build_sketches step (b) reads spikes/MANIFEST.md's Requirements for non-negotiable design constraints; once spike.md restructures that file from one flat Requirements section to per-idea `### {idea-key}` sections (same PR), sketch.md's single-flat-section instruction would silently go stale — collateral of the seam this fix touches, required so sketch.md keeps reading the file correctly rather than a defect introduced by this change. Growth is the one clause distinguishing per-idea sections and instructing the reader to check the Requirements list of the relevant idea key (or all of them if none clearly matches)."
|
||||
}
|
||||
}
|
||||
221
tests/fix-1700-spike-manifest-idea-scoping.test.cjs
Normal file
221
tests/fix-1700-spike-manifest-idea-scoping.test.cjs
Normal file
@@ -0,0 +1,221 @@
|
||||
// allow-test-rule: source-text-is-the-product (#1700)
|
||||
// spike.md / spike-wrap-up.md / artifact-types.md / sketch.md are workflow and reference
|
||||
// markdown — their text IS the contract loaded by the agent at runtime. Testing text content
|
||||
// tests the deployed contract; there is no build step, CLI entrypoint, or generated code to
|
||||
// exercise for this class of file (confirmed via mcp__memtrace__find_code: only .md files and
|
||||
// unrelated `.planning/phases/*/REQUIREMENTS.md` symbols in gap-checker.cts/milestone.cts
|
||||
// reference this seam — a different artifact family, not this one).
|
||||
//
|
||||
// Regression test for #1700: `.planning/spikes/MANIFEST.md` held a single un-keyed `## Idea` /
|
||||
// `## Requirements` pair while its own read path (`spike.md:138`, frontier mode) assumed the
|
||||
// tree holds several unrelated ideas across campaigns. `spike-wrap-up.md` then copied that flat
|
||||
// Requirements list verbatim into every generated feature-area reference and the generated
|
||||
// spike-findings skill, so one idea's requirements leaked into another idea's build as binding
|
||||
// constraints. The fix scopes both the write side (spike.md's create_manifest) and the read/
|
||||
// re-emit side (spike-wrap-up.md's gather/synthesize/write_skill) by an explicit idea key,
|
||||
// carried through spike README frontmatter and a new Idea column in the `## Spikes` table.
|
||||
//
|
||||
// QA-matrix note: none of CONTRIBUTING.md's CLI / parser / filesystem-write / security matrices
|
||||
// apply — this is a workflow-prompt template edit with no runtime code path, parser, or write
|
||||
// seam of its own (the `.planning/spikes/` tree is written by an LLM agent following the
|
||||
// workflow prose, not by GSD's own code). No numeric limit is introduced, so the limit-1/limit/
|
||||
// limit+1 boundary-coverage rule and the fast-check property-test requirement do not apply
|
||||
// either — there is no parser, budget, or bijective contract in this change. See
|
||||
// `.gsd/bug/fix-1700-spike-manifest-idea-scoping/50-test-matrix.md` for the full row table.
|
||||
|
||||
'use strict';
|
||||
|
||||
const { test, describe } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
|
||||
const WORKFLOWS_DIR = path.join(__dirname, '..', 'gsd-core', 'workflows');
|
||||
const REFERENCES_DIR = path.join(__dirname, '..', 'gsd-core', 'references');
|
||||
|
||||
const spikeMd = () => fs.readFileSync(path.join(WORKFLOWS_DIR, 'spike.md'), 'utf-8');
|
||||
const spikeWrapUpMd = () => fs.readFileSync(path.join(WORKFLOWS_DIR, 'spike-wrap-up.md'), 'utf-8');
|
||||
const sketchMd = () => fs.readFileSync(path.join(WORKFLOWS_DIR, 'sketch.md'), 'utf-8');
|
||||
const artifactTypesMd = () => fs.readFileSync(path.join(REFERENCES_DIR, 'artifact-types.md'), 'utf-8');
|
||||
|
||||
describe('fix-1700: spike MANIFEST.md scopes Idea/Requirements per idea key', () => {
|
||||
// Row 1 — the failing-first regression test (50-test-matrix.md row 1).
|
||||
test('spike.md create_manifest step defines an explicit idea key and a scoped ## Ideas structure', () => {
|
||||
const content = spikeMd();
|
||||
assert.match(
|
||||
content,
|
||||
/idea key/i,
|
||||
'create_manifest step must introduce an explicit "idea key" concept — without one, ' +
|
||||
'the write side has no way to tell a continuation of the current idea from an unrelated new idea (#1700).'
|
||||
);
|
||||
assert.match(
|
||||
content,
|
||||
/##\s*Ideas\b/,
|
||||
'MANIFEST.md template must have a `## Ideas` section (plural) that can hold more than one ' +
|
||||
'idea, replacing the old single flat `## Idea` paragraph.'
|
||||
);
|
||||
assert.match(
|
||||
content,
|
||||
/###\s*\{idea-key\}/,
|
||||
"MANIFEST.md template must scope each idea under its own `### {idea-key}` subsection so " +
|
||||
"idea A's paragraph is never overwritten by idea B (#1700 acceptance criterion 1)."
|
||||
);
|
||||
});
|
||||
|
||||
test('create_manifest step instructs appending a new idea section instead of overwriting an existing one', () => {
|
||||
const content = spikeMd();
|
||||
assert.match(
|
||||
content,
|
||||
/new idea key[\s\S]{0,200}?append/i,
|
||||
'When the current idea is new (not a continuation), the workflow must append a new idea ' +
|
||||
'section rather than overwrite/merge into an existing one.'
|
||||
);
|
||||
assert.match(
|
||||
content,
|
||||
/[Nn]ever touch[\s\S]{0,80}?(?:another|other) idea/i,
|
||||
"The workflow must explicitly forbid touching another idea's section when adding to the current one."
|
||||
);
|
||||
});
|
||||
|
||||
test("create_manifest step migrates a pre-#1700 flat-shape MANIFEST.md instead of discarding it (Hyrum's Law)", () => {
|
||||
const content = spikeMd();
|
||||
assert.match(
|
||||
content,
|
||||
/old flat[\s\S]{0,40}?shape/i,
|
||||
'The workflow must handle an existing MANIFEST.md written before this fix (flat `## Idea` / ' +
|
||||
'`## Requirements`, no `## Ideas` heading) by migrating it in place, not discarding user data.'
|
||||
);
|
||||
});
|
||||
|
||||
test('## Spikes table gains an Idea column so every spike is attributable to an idea (acceptance criterion 3)', () => {
|
||||
const content = spikeMd();
|
||||
assert.match(
|
||||
content,
|
||||
/\|\s*#\s*\|\s*Idea\s*\|\s*Name\s*\|\s*Type\s*\|\s*Validates\s*\|\s*Verdict\s*\|\s*Tags\s*\|/,
|
||||
'The `## Spikes` table header must include an Idea column between # and Name, so the ' +
|
||||
'durable index still covers every spike across every idea while attributing each row.'
|
||||
);
|
||||
});
|
||||
|
||||
test('spike README frontmatter template carries an idea key field', () => {
|
||||
const content = spikeMd();
|
||||
assert.match(
|
||||
content,
|
||||
/spike:\s*NNN\r?\nidea:\s*\{idea-key\}/,
|
||||
'The build_spikes README frontmatter template must record the owning idea key directly ' +
|
||||
'on the spike, so spike-wrap-up can attribute requirements without re-deriving it.'
|
||||
);
|
||||
});
|
||||
|
||||
test("reground step checks the CURRENT idea's requirements, not all requirements", () => {
|
||||
const content = spikeMd();
|
||||
assert.match(
|
||||
content,
|
||||
/current idea's[\s\S]{0,80}?Requirements/i,
|
||||
'The reground step must scope its established-requirements check to the idea being spiked, ' +
|
||||
'not the whole MANIFEST.md, or it would falsely flag idea B as contradicting idea A.'
|
||||
);
|
||||
});
|
||||
|
||||
test("frontier_mode load step reads every idea's scoped requirements, not one flat section", () => {
|
||||
const content = spikeMd();
|
||||
assert.match(
|
||||
content,
|
||||
/every idea section/i,
|
||||
"Frontier mode's MANIFEST.md load step must describe reading per-idea sections, matching " +
|
||||
'the new ## Ideas structure it depends on.'
|
||||
);
|
||||
});
|
||||
|
||||
test("spike-wrap-up.md gather step resolves each spike's idea key before reading requirements", () => {
|
||||
const content = spikeWrapUpMd();
|
||||
assert.match(
|
||||
content,
|
||||
/idea key/i,
|
||||
"spike-wrap-up.md must resolve an idea key per spike (from frontmatter or the Spikes table) " +
|
||||
'before it can scope which Requirements apply — otherwise it still reads the whole ' +
|
||||
'MANIFEST.md Requirements section flat (#1700 root cause on the consumer side).'
|
||||
);
|
||||
});
|
||||
|
||||
test("spike-wrap-up.md synthesize step scopes Requirements to the feature area's owning idea key(s)", () => {
|
||||
const content = spikeWrapUpMd();
|
||||
assert.match(
|
||||
content,
|
||||
/ONLY from the Requirements list of the idea key/,
|
||||
"The synthesize step's Requirements block must pull only from the idea key(s) that own the " +
|
||||
'spikes in that feature-area group — never the whole MANIFEST.md (#1700 acceptance criterion 2).'
|
||||
);
|
||||
assert.match(
|
||||
content,
|
||||
/[Nn]ever include a requirement from an idea key that has no spike in this group/,
|
||||
"The synthesize step must explicitly forbid leaking an unrelated idea's requirement into a " +
|
||||
'feature-area reference it has no spike in.'
|
||||
);
|
||||
});
|
||||
|
||||
test("spike-wrap-up.md write_skill step unions only the wrapped idea keys' requirements", () => {
|
||||
const content = spikeWrapUpMd();
|
||||
assert.match(
|
||||
content,
|
||||
/never the whole MANIFEST\.md/,
|
||||
"The generated SKILL.md <requirements> block must be built from only the idea key(s) " +
|
||||
'represented among the spikes actually being wrapped in this session.'
|
||||
);
|
||||
});
|
||||
|
||||
test('artifact-types.md no longer labels the project-level MANIFEST.md "per-spike" (acceptance criterion 4)', () => {
|
||||
const content = artifactTypesMd();
|
||||
assert.doesNotMatch(
|
||||
content,
|
||||
/MANIFEST\.md \(per-spike/,
|
||||
'artifact-types.md must not describe the project-level, durable-index MANIFEST.md as ' +
|
||||
'"per-spike" — that mislabeling is the third independent statement of the #1700 ambiguity ' +
|
||||
"the maintainer's triage flagged."
|
||||
);
|
||||
assert.match(
|
||||
content,
|
||||
/MANIFEST\.md \(per-project index/,
|
||||
'artifact-types.md must instead label MANIFEST.md as the per-project durable index.'
|
||||
);
|
||||
});
|
||||
|
||||
test('artifact-types.md keeps the legacy /gsd:spike colon form untouched (out of scope, #2903)', () => {
|
||||
const content = artifactTypesMd();
|
||||
assert.match(
|
||||
content,
|
||||
/via \/gsd:spike/,
|
||||
'The colon-form command reference belongs to #2903 and must not be touched by this fix.'
|
||||
);
|
||||
});
|
||||
|
||||
test('sketch.md no longer assumes spikes/MANIFEST.md has a single flat Requirements section', () => {
|
||||
const content = sketchMd();
|
||||
assert.match(
|
||||
content,
|
||||
/separate `### \{idea-key\}` sections/,
|
||||
"sketch.md reads spikes/MANIFEST.md for design constraints; once MANIFEST.md nests " +
|
||||
"Requirements under per-idea sections, sketch.md's single-section instruction goes stale " +
|
||||
'unless updated in the same PR (collateral of this fix\'s seam, not a separate concern).'
|
||||
);
|
||||
});
|
||||
|
||||
test('success criteria in spike.md reflect idea-scoped requirement tracking', () => {
|
||||
const content = spikeMd();
|
||||
assert.match(
|
||||
content,
|
||||
/scoped to the idea key/,
|
||||
'spike.md success_criteria must call out that Requirements are tracked per idea key, not globally.'
|
||||
);
|
||||
});
|
||||
|
||||
test('success criteria in spike-wrap-up.md forbid blending requirements across ideas', () => {
|
||||
const content = spikeWrapUpMd();
|
||||
assert.match(
|
||||
content,
|
||||
/never blended with an unrelated idea/,
|
||||
'spike-wrap-up.md success_criteria must explicitly call out that requirements must never ' +
|
||||
'blend across idea keys.'
|
||||
);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user