* test(#2444): failing-first regression for checkpoint:* plan-structure validation Add acceptance-criteria tests covering the three canonical checkpoint task types (human-verify, decision, human-action) plus an unknown-subtype forward-compat case. Each canonical type must pass verify plan-structure when it carries its type-specific required fields (per gsd-core/references/checkpoints.md), and must be flagged when those fields are missing. Non-checkpoint tasks keep the existing <action>/<verify>/<done>/<files> requirements unchanged (AC3 regression guards). The existing 'errors when checkpoint task but autonomous is true' fixture is updated to use the canonical checkpoint:human-verify triple (<what-built>/<how-to-verify>/<resume-signal>) so it does not collide with the new per-type validator; the assertion (autonomous is not false) is unchanged. * fix(#2444): branch plan-structure validation on task type=checkpoint:* cmdVerifyPlanStructure unconditionally required <action>/<verify>/<done>/ <files> on every task, so every checkpoint:* task — which uses the checkpoint convention's type-specific fields instead — was reported as a structural error. Checkpoint-heavy phases produced walls of false findings. The fix introduces two pure helpers in verify.cts: - extractPlanTaskInfos(content): single ReDoS-safe pass over <task ...>...</task> blocks that captures BOTH the opening-tag attribute string (so the type= selector is not lost, as it is with extractTaggedBlocks) and the body, returning a typed PlanTaskInfo. - validatePlanTaskStructure(task): branches on the task's type. checkpoint:human-verify requires <what-built>/<how-to-verify>/ <resume-signal> (the canonical triple). checkpoint:decision requires <decision>/<options>/<resume-signal>. checkpoint:human-action requires <action>/<instructions>/ <verification>/<resume-signal>. Unknown checkpoint:* subtypes require only the universal <resume-signal> (forward-compat). All other types keep the historical <action>/<verify>/<done>/<files> requirements unchanged. Canonical reference: gsd-core/references/checkpoints.md. Per-type field sets validated against the documented templates in agents/gsd-planner.md and gsd-core/templates/phase-prompt.md. * fix(#2444): re-resolve body-parser to 2.3.0 in lockfile (GHSA-v422-hmwv-36x6) GHSA-v422-hmwv-36x6 (body-parser DoS via invalid limit value, low severity, published 2026-07-20T23:23:26Z) made tests/npm-integrity-gate.test.cjs (#3588: root workspace production tree has no advisories) fail any subsequent npm audit --omit=dev. The advisory affects body-parser >=2.0.0 <2.3.0 pulled transitively via @anthropic-ai/claude-agent-sdk -> @modelcontextprotocol/sdk -> express -> body-parser@2.2.2. express@5.2.1 already declares body-parser as ^2.2.1, so 2.3.0 is a valid re-resolution within express's own compatibility range — no override needed. Regenerated the lockfile via 'npm audit fix --omit=dev' which re-resolves transitive deps within their declared ranges; package.json is unchanged. Verified: npm audit --omit=dev reports 0/0/0/0/0 advisories; body-parser now reads as 2.3.0 in 'npm ls body-parser --omit=dev'. * test(#2444): close review gap-closure tests + harden type-attr charset Orthogonal review (code-review + security-review subagents) returned APPROVE on Standards and Spec. Per the playbook's zero-tolerance policy, address every Low finding: Spec gap-closures: - AC3 verbatim: add explicit <done> and <files> regression tests for non-checkpoint tasks (pre-existing tests only covered <action> and <verify>). - AC2: add checkpoint:decision missing <decision>, checkpoint:human-action missing <action>, checkpoint:human-action missing <verification> cases (the implementation enforces all of these; only one missing-field case per type was previously tested). - Remove the duplicate 'returns error for nonexistent file' test that leaked into the new describe block from the insertion edit. Security hardening (Low-sev, defense-in-depth): - Tighten the task type= attribute extractor in src/verify.cts from [^"'>\s]+ to [\w:-]+ so a hostile type= attribute cannot carry markup fragments (e.g. type=evil<fragment) into the verifier's typed JSON output. All legitimate type values (auto, tracer, manual, checkpoint:human-verify, checkpoint:decision, checkpoint:human-action, checkpoint:tdd-review) match the tighter charset. - Add adversarial regression test asserting type=evil<fragment surfaces as 'evil' (capture stops at '<'), with no markup chars (< > ( ) &) in the surfaced type field. * docs(changeset): add Fixed fragments for #2444 PR Two fragments: - sturdy-jays-tumble.md: the verify plan-structure checkpoint fix - witty-badgers-hum.md: the body-parser 2.3.0 re-resolution PR number backfilled to 0 placeholder per CLAUDE.md 'PR Number Handling'; will backfill to the real PR number immediately after gh pr create returns. * docs(changeset): backfill PR number to 2473 Per CLAUDE.md 'PR Number Handling': backfill the placeholder pr:0 with the real PR number returned by gh pr create.
This commit is contained in:
@@ -263,12 +263,11 @@ describe('verify plan-structure command', () => {
|
||||
' <verify><automated>echo ok</automated></verify>',
|
||||
' <done>Done</done>',
|
||||
'</task>',
|
||||
'<task type="checkpoint:human-verify">',
|
||||
'<task type="checkpoint:human-verify" gate="blocking">',
|
||||
' <name>Task 2: Verify UI</name>',
|
||||
' <files>some/file.ts</files>',
|
||||
' <action>Check the UI</action>',
|
||||
' <verify><human>Visit the app</human></verify>',
|
||||
' <done>UI verified</done>',
|
||||
' <what-built>UI at localhost:3000</what-built>',
|
||||
' <how-to-verify>Visit the app</how-to-verify>',
|
||||
' <resume-signal>Type "approved"</resume-signal>',
|
||||
'</task>',
|
||||
'</tasks>',
|
||||
].join('\n');
|
||||
@@ -299,6 +298,384 @@ describe('verify plan-structure command', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// verify plan-structure — checkpoint task types (#2444)
|
||||
// A checkpoint:* task uses type-specific required fields (per
|
||||
// gsd-core/references/checkpoints.md), NOT the auto-task <action>/<verify>/<done>.
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
describe('verify plan-structure — checkpoint task types (#2444)', () => {
|
||||
let tmpDir;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpDir = createTempProject();
|
||||
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-test'), { recursive: true });
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
cleanup(tmpDir);
|
||||
});
|
||||
|
||||
// Helper: wrap a task body in a complete valid PLAN.md scaffold.
|
||||
function planWithTask(taskBody, { autonomous = 'false' } = {}) {
|
||||
return [
|
||||
'---',
|
||||
'phase: 01-test',
|
||||
'plan: 01',
|
||||
'type: execute',
|
||||
'wave: 1',
|
||||
'depends_on: []',
|
||||
'files_modified: [some/file.ts]',
|
||||
`autonomous: ${autonomous}`,
|
||||
'must_haves:',
|
||||
' truths:',
|
||||
' - "something"',
|
||||
'---',
|
||||
'',
|
||||
'<tasks>',
|
||||
taskBody,
|
||||
'</tasks>',
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
function runVerify(planContent) {
|
||||
const planPath = path.join(tmpDir, '.planning', 'phases', '01-test', '01-01-PLAN.md');
|
||||
fs.writeFileSync(planPath, planContent);
|
||||
const result = runGsdTools('verify plan-structure .planning/phases/01-test/01-01-PLAN.md', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
return JSON.parse(result.output);
|
||||
}
|
||||
|
||||
// ── AC1: canonical checkpoint tasks pass with zero findings ────────────────
|
||||
|
||||
test('checkpoint:human-verify with canonical triple passes (AC1)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:human-verify" gate="blocking">',
|
||||
' <name>Checkpoint: verify UI</name>',
|
||||
' <what-built>Dashboard at localhost:3000</what-built>',
|
||||
' <how-to-verify>Visit /dashboard, check layout</how-to-verify>',
|
||||
' <resume-signal>Type "approved" or describe issues</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, true, `expected valid; errors: ${JSON.stringify(output.errors)}`);
|
||||
assert.deepStrictEqual(output.errors, [], `expected no errors; got: ${JSON.stringify(output.errors)}`);
|
||||
assert.strictEqual(output.task_count, 1, 'should count the checkpoint task');
|
||||
});
|
||||
|
||||
test('checkpoint:decision with canonical fields passes (AC1)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:decision" gate="blocking">',
|
||||
' <name>Checkpoint: pick auth provider</name>',
|
||||
' <decision>Select authentication provider</decision>',
|
||||
' <context>Need user authentication.</context>',
|
||||
' <options>',
|
||||
' <option id="supabase"><name>Supabase Auth</name><pros>Built-in</pros><cons>Lock-in</cons></option>',
|
||||
' <option id="clerk"><name>Clerk</name><pros>DX</pros><cons>Paid</cons></option>',
|
||||
' </options>',
|
||||
' <resume-signal>Select: supabase or clerk</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, true, `expected valid; errors: ${JSON.stringify(output.errors)}`);
|
||||
assert.deepStrictEqual(output.errors, [], `expected no errors; got: ${JSON.stringify(output.errors)}`);
|
||||
});
|
||||
|
||||
test('checkpoint:human-action with canonical fields passes (AC1)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:human-action" gate="blocking">',
|
||||
' <name>Checkpoint: complete email verification</name>',
|
||||
' <action>Click the verification link in your inbox</action>',
|
||||
' <instructions>I created the account; check your email.</instructions>',
|
||||
' <verification>API key works via curl</verification>',
|
||||
' <resume-signal>Type "done" when email verified</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, true, `expected valid; errors: ${JSON.stringify(output.errors)}`);
|
||||
assert.deepStrictEqual(output.errors, [], `expected no errors; got: ${JSON.stringify(output.errors)}`);
|
||||
});
|
||||
|
||||
test('unknown checkpoint:* subtype passes with just <resume-signal> (forward-compat)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:custom-future-type">',
|
||||
' <name>Checkpoint: future</name>',
|
||||
' <resume-signal>Type "ok"</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, true, `expected valid; errors: ${JSON.stringify(output.errors)}`);
|
||||
assert.deepStrictEqual(output.errors, [], `expected no errors; got: ${JSON.stringify(output.errors)}`);
|
||||
});
|
||||
|
||||
test('mixed plan: auto task + checkpoint:human-verify task passes (AC1 realistic)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="auto">',
|
||||
' <name>Task 1: build dashboard</name>',
|
||||
' <files>src/dashboard.ts</files>',
|
||||
' <action>Scaffold the dashboard</action>',
|
||||
' <verify><automated>npm test</automated></verify>',
|
||||
' <done>Dashboard renders</done>',
|
||||
'</task>',
|
||||
'<task type="checkpoint:human-verify" gate="blocking">',
|
||||
' <name>Checkpoint: visual review</name>',
|
||||
' <what-built>Dashboard at localhost:3000</what-built>',
|
||||
' <how-to-verify>Visit /dashboard, check responsive layout</how-to-verify>',
|
||||
' <resume-signal>Type "approved"</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, true, `expected valid; errors: ${JSON.stringify(output.errors)}`);
|
||||
assert.deepStrictEqual(output.errors, [], `expected no errors; got: ${JSON.stringify(output.errors)}`);
|
||||
assert.strictEqual(output.task_count, 2, 'should count both tasks');
|
||||
});
|
||||
|
||||
// ── AC2: checkpoint tasks missing required fields are still flagged ────────
|
||||
|
||||
test('checkpoint:human-verify missing <how-to-verify> is flagged (AC2)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:human-verify" gate="blocking">',
|
||||
' <name>Checkpoint: verify UI</name>',
|
||||
' <what-built>UI at localhost:3000</what-built>',
|
||||
' <resume-signal>Type "approved"</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <how-to-verify>')),
|
||||
`Expected "missing <how-to-verify>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('checkpoint:human-verify missing <what-built> is flagged (AC2)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:human-verify" gate="blocking">',
|
||||
' <name>Checkpoint: verify UI</name>',
|
||||
' <how-to-verify>Visit /dashboard</how-to-verify>',
|
||||
' <resume-signal>Type "approved"</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <what-built>')),
|
||||
`Expected "missing <what-built>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('checkpoint:decision missing <options> is flagged (AC2)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:decision" gate="blocking">',
|
||||
' <name>Checkpoint: pick</name>',
|
||||
' <decision>Select provider</decision>',
|
||||
' <resume-signal>Select: a or b</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <options>')),
|
||||
`Expected "missing <options>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('checkpoint:human-action missing <instructions> is flagged (AC2)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:human-action" gate="blocking">',
|
||||
' <name>Checkpoint: act</name>',
|
||||
' <action>Do the thing</action>',
|
||||
' <verification>curl returns 200</verification>',
|
||||
' <resume-signal>Type "done"</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <instructions>')),
|
||||
`Expected "missing <instructions>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('checkpoint:decision missing <decision> is flagged (AC2)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:decision" gate="blocking">',
|
||||
' <name>Checkpoint: pick</name>',
|
||||
' <options>',
|
||||
' <option id="a"><name>A</name><pros>p</pros><cons>c</cons></option>',
|
||||
' </options>',
|
||||
' <resume-signal>Select: a</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <decision>')),
|
||||
`Expected "missing <decision>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('checkpoint:human-action missing <action> is flagged (AC2)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:human-action" gate="blocking">',
|
||||
' <name>Checkpoint: act</name>',
|
||||
' <instructions>Do it.</instructions>',
|
||||
' <verification>curl returns 200</verification>',
|
||||
' <resume-signal>Type "done"</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <action>')),
|
||||
`Expected "missing <action>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('checkpoint:human-action missing <verification> is flagged (AC2)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:human-action" gate="blocking">',
|
||||
' <name>Checkpoint: act</name>',
|
||||
' <action>Do it</action>',
|
||||
' <instructions>Do it.</instructions>',
|
||||
' <resume-signal>Type "done"</resume-signal>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <verification>')),
|
||||
`Expected "missing <verification>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('any checkpoint:* missing <resume-signal> is flagged (AC2)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="checkpoint:human-verify" gate="blocking">',
|
||||
' <name>Checkpoint: verify UI</name>',
|
||||
' <what-built>UI</what-built>',
|
||||
' <how-to-verify>Visit</how-to-verify>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <resume-signal>')),
|
||||
`Expected "missing <resume-signal>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('checkpoint task without a type attribute still gets non-checkpoint rules (regression guard)', () => {
|
||||
// A bare <task> (no type=) is NOT treated as a checkpoint; current rules apply.
|
||||
const output = runVerify(planWithTask([
|
||||
'<task>',
|
||||
' <name>Bare task</name>',
|
||||
' <files>x.ts</files>',
|
||||
' <verify><automated>echo ok</automated></verify>',
|
||||
' <done>ok</done>',
|
||||
'</task>',
|
||||
].join('\n')));
|
||||
|
||||
assert.strictEqual(output.valid, false, 'should be invalid (missing <action>)');
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <action>')),
|
||||
`Expected "missing <action>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
// ── AC3: non-checkpoint tasks missing fields are still flagged (no regression) ──
|
||||
|
||||
test('non-checkpoint task missing <action> is still flagged (AC3)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="auto">',
|
||||
' <name>Task 1: no action</name>',
|
||||
' <verify><automated>echo ok</automated></verify>',
|
||||
' <done>Done</done>',
|
||||
'</task>',
|
||||
].join('\n'), { autonomous: 'true' }));
|
||||
|
||||
assert.ok(
|
||||
output.errors.some(e => e.includes('missing <action>')),
|
||||
`Expected "missing <action>" error: ${JSON.stringify(output.errors)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('non-checkpoint task missing <verify> still warns (AC3)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="auto">',
|
||||
' <name>Task 1: no verify</name>',
|
||||
' <files>x.ts</files>',
|
||||
' <action>Do it</action>',
|
||||
' <done>Done</done>',
|
||||
'</task>',
|
||||
].join('\n'), { autonomous: 'true' }));
|
||||
|
||||
assert.ok(
|
||||
output.warnings.some(w => w.includes('missing <verify>')),
|
||||
`Expected "missing <verify>" warning: ${JSON.stringify(output.warnings)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('non-checkpoint task missing <done> still warns (AC3)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="auto">',
|
||||
' <name>Task 1: no done</name>',
|
||||
' <files>x.ts</files>',
|
||||
' <action>Do it</action>',
|
||||
' <verify><automated>echo ok</automated></verify>',
|
||||
'</task>',
|
||||
].join('\n'), { autonomous: 'true' }));
|
||||
|
||||
assert.ok(
|
||||
output.warnings.some(w => w.includes('missing <done>')),
|
||||
`Expected "missing <done>" warning: ${JSON.stringify(output.warnings)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('non-checkpoint task missing <files> still warns (AC3)', () => {
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type="auto">',
|
||||
' <name>Task 1: no files</name>',
|
||||
' <action>Do it</action>',
|
||||
' <verify><automated>echo ok</automated></verify>',
|
||||
' <done>Done</done>',
|
||||
'</task>',
|
||||
].join('\n'), { autonomous: 'true' }));
|
||||
|
||||
assert.ok(
|
||||
output.warnings.some(w => w.includes('missing <files>')),
|
||||
`Expected "missing <files>" warning: ${JSON.stringify(output.warnings)}`
|
||||
);
|
||||
});
|
||||
|
||||
// ── Security: type-attribute charset is bounded (no markup injection) ──────
|
||||
|
||||
test('task type attribute with hostile markup fragment is not surfaced unsanitized', () => {
|
||||
// Per CONTRIBUTING.md §"Security and prompt-injection surfaces": a hostile
|
||||
// PLAN.md cannot inject unclosed-tag fragments into the verifier's typed
|
||||
// JSON output via the type= attribute. The charset [a-zA-Z0-9_:-] rejects
|
||||
// '<', '>', '(', '&', etc., so a payload like type=evil<fragment captures
|
||||
// only `evil` (the `<` terminates the match); the surfaced `type` field
|
||||
// carries no markup.
|
||||
const output = runVerify(planWithTask([
|
||||
'<task type=evil<fragment>',
|
||||
' <name>Hostile</name>',
|
||||
' <action>do</action>',
|
||||
' <verify><automated>echo ok</automated></verify>',
|
||||
' <done>ok</done>',
|
||||
'</task>',
|
||||
].join('\n'), { autonomous: 'true' }));
|
||||
|
||||
const hostile = output.tasks.find(t => t.name === 'Hostile');
|
||||
assert.ok(hostile, `Expected to find Hostile task in output.tasks: ${JSON.stringify(output.tasks)}`);
|
||||
assert.ok(
|
||||
!/[<>()&]/.test(hostile.type),
|
||||
`Expected type to contain no markup chars; got: ${JSON.stringify(hostile.type)}`
|
||||
);
|
||||
assert.strictEqual(hostile.type, 'evil', `Expected capture to stop at '<'; got: ${JSON.stringify(hostile.type)}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// verify phase-completeness command
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
Reference in New Issue
Block a user