* test(#4740): pin the per-step role-family partition Failing-first coverage for the Loop Host Contract role partition. At this commit crossCheckRoleFamilies does not exist, so the rows throw "crossCheckRoleFamilies is not a function" -- the RED proof they bind to behavior rather than restating it. ADR-894 section 3 assigns roles per step but parenthesises the assignment as "(illustrative roles)", and nothing enforced it. The only thing standing in the way was a single deepEqual in this same file, which is editable prose. Rows cover: each step's own family accepted; a strict subset accepted; a foreign role rejected at every step; an unknown role rejected; an unknown step failing CLOSED; capitalization not silently matched; every offending role reported rather than only the first; and purity, because buildContract puts the same array into the generated contract. Two rows exist because an earlier cut of this suite was vacuous. The purity fixture is deliberately UNSORTED -- an alphabetically-sorted fixture cannot fail an in-place sort(), and the mutant was being killed by three unrelated rows instead. A parity row asserts ROLE_FAMILY and ROLE_TO_AGENT cover the exact same role-name domain, both directions: they are parallel constants over one domain, so divergence is the generative-fix class CLAUDE.md names. Every negative row asserts the offending ROLE NAME and the STEP NAME appear in the message. A count-only assertion survives a mutant that reports the wrong role, which the 80% Stryker gate would surface only after a full CI round-trip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(#4740): reject a cross-family agent-role declaration Orchestration and execution are distinct functions of the loop and must not drift into one another. That partition was real but unenforced: ADR-894 section 3 calls its own role assignment "illustrative", and the generator accepted anything. Adding orchestrator to execute-phase.md's agent-roles line compiled, --check passed once regenerated, and capability-validator.cjs then began accepting into:"orchestrator" at every execute point. ROLE_FAMILY maps every role to one of orchestration, planning or execution. EXPECTED_FAMILY_BY_STEP gives each of the five steps exactly one family. crossCheckRoleFamilies rejects a cross-family role, a role outside the vocabulary, and an unknown step. It reports every offender, not the first. It fails CLOSED on an unknown step, deliberately diverging from assertPointsCoverage's "unknown step -- caught elsewhere". For points that is true: the canonical-set and duplicate checks catch it. For roles there is no second net, so failing open would leave an unknown step as the one input that bypasses the gate. crossCheckRoles' orchestrator exemption is untouched. ROLE_TO_AGENT maps roles to agent FILES and the orchestrator is the host, owning none -- admissibility and agent-file presence are separate concerns with separate checks. Additive to section 3's existing rule that contribution.into must be a member of the step's agentRoles, which is unchanged. That governs what a CAPABILITY may target; this governs what a WORKFLOW may declare. No capability is affected, and all five workflows already declare single-family sets, so the gate is green on the commit that introduces it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#4740): make the ADR-894 role assignment normative Section 3 parenthesises its per-step role assignment as "(illustrative roles)". That word was accurate about the list's PURPOSE -- it illustrated the shape of a generated contract entry -- and wrong about its STATUS, because the assignment was load-bearing from the moment the generator consumed it. Read literally it makes the partition an example rather than a rule. Appended as a dated in-place section per docs/contributor-standards.md, which records that an accepted ADR is never rewritten and names this the default pattern. Section 3's original body is untouched. The amendment states the three disjoint families, the one family each step admits, that a step may declare a strict subset but never outside it, and why this is a clarification rather than a new decision: the contract is generated from the workflow markers "so it cannot drift into a lie", and all five workflows have always declared single-family sets. What was absent was any statement that it is required, and any check that it holds. It also pins the distinction that is easy to re-merge: contribution.into being a member of agentRoles governs what a CAPABILITY may target and is unchanged; the family rule governs what a WORKFLOW may declare. The CONTEXT.md glossary entry for the Loop Host Contract records the same, beside the agent-reference drift guard it already documented. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(#4740): add changeset fragment pr:0 placeholder is backfilled with the real number once the PR exists. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(#4740): backfill changeset pr number Replaces the pr:0 placeholder with 4742 now that the PR exists. Verified with GITHUB_BASE_REF=next, the way CI runs them: changeset lint and lint:docs both go from invalid_pr(0) to ok. Without that env both report success without evaluating the branch at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4740): stop injecting the orchestrator procedure into executors claude-orchestration declared a contribution at execute:wave:pre with into:"executor". loop-hook-dispatch.md defines a contribution as "inject fragment.inline verbatim into the context for the role named in into", so its 267 lines were injected into EXECUTOR prompts whenever the capability was enabled. Those lines are orchestration end to end -- construct a wave manifest, resolve the dispatch backend, invoke the Workflow tool to spawn executors, bridge per-agent results into the merge chain. An executor can act on none of it. Retargeting to into:"orchestrator" would not have been a fix. ROLE_TO_AGENT carries no orchestrator entry by design: the orchestrator IS the host, and the host's procedure lives in execute-phase.md. A step's agentRoles enumerates agents a capability may inject context INTO, so adding orchestrator there would model the host as an injectable agent -- the same category error pointed the other way, and it would need an exception carved into the partition the same issue just made normative. So the defect is the mechanism, not the label. A contribution injects into an agent's context; "replace step 3's inline dispatch loop" is a change to what the HOST does. The contribution channel was serving as a host-behaviour directive because it was the only channel available at an execute point. The entry is removed. plan:post into:"planner" is correct and untouched. The procedure is preserved verbatim at docs/workflow-backend-dispatch.md inside the capability -- it is the only copy in the repo -- and is no longer injected anywhere. Consequence, not softened: the Workflow backend now has no loop wiring. Detection, emission and config remain and the design is intact, but nothing dispatches it. Under the separation ADR-1143 itself asserts it never had a legitimate channel; ADR-1143's own audit already records the end-to-end path has never been exercised. Wiring it properly needs a host-level mechanism that does not exist today. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#4740): invert the stale execute:wave:pre registry assertions Removing the contribution left four surfaces asserting or describing the old state. Caught by an isolated review before a verification run was spent, which is the point of reviewing first: the first of these was a guaranteed CI red. execute-wave-post-gate-pipeline-e2e asserted against the REAL generated registry that byLoopPoint['execute:wave:pre'] held exactly one contribution with capId claude-orchestration. It now holds zero. Inverted to assert exactly 0 -- not a vague >= 0 -- and the #2285 comment above it now explains the current state rather than the one it was written for. CONTEXT.md's Claude Orchestration entry claimed two contributions at wired points. It is now one, and the entry's execute:wave:post label was already wrong before this change: the manifest said execute:wave:pre. Rewritten to one plan:post contribution, why the execute-point one was removed, and where the procedure now lives. One assertion in claude-orchestration.test.cjs could not fail. It tested for the prose "(into the executor)" while the doc says "(`into: executor`)", so no plausible wording matched it and the paired plan:post assertion was carrying the row. Replaced with a check on the structural claim, and proved RED by restoring the two-contribution wording before reverting. The moved procedure keeps section headings that speak as a live contribution -- "When this contribution is active", "Why execute:wave:pre". Preserving the body verbatim was deliberate, so the headings stay and an editor's note under the header explains why they read that way. A sweep of all 17 files referencing byLoopPoint found no further siblings: the remaining hits are a synthetic capability fixture and an empty-points test that already expected no active hooks, both correct before and after. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
954 lines
39 KiB
JavaScript
954 lines
39 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* loop-host-contract.test.cjs — behavioral tests for gen-loop-host-contract.cjs.
|
|
*
|
|
* ADR-894 phase 3a-impl-2.
|
|
* Uses node:test + node:assert/strict.
|
|
* NO source-grep: tests use in-memory fixtures and real workflow files.
|
|
*/
|
|
|
|
const { describe, test } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('node:fs');
|
|
const os = require('node:os');
|
|
const path = require('node:path');
|
|
const fc = require('fast-check');
|
|
|
|
const { cleanup } = require('./helpers.cjs');
|
|
|
|
const {
|
|
parseLoopHostBlock,
|
|
crossCheckRoles,
|
|
crossCheckRoleFamilies,
|
|
assertPointsCoverage,
|
|
buildContract,
|
|
serializeContract,
|
|
normalizeLineEndings,
|
|
STEP_WORKFLOWS,
|
|
CANONICAL_POINTS,
|
|
EXPECTED_POINTS_BY_STEP,
|
|
ROLE_TO_AGENT,
|
|
ROLE_FAMILY,
|
|
} = require('../scripts/gen-loop-host-contract.cjs');
|
|
|
|
const { LOOP_HOST_CONTRACT } = require('../gsd-core/bin/lib/loop-host-contract.cjs');
|
|
|
|
const ROOT = path.resolve(__dirname, '..');
|
|
const CONTRACT_PATH = path.join(ROOT, 'gsd-core', 'bin', 'lib', 'loop-host-contract.cjs');
|
|
|
|
// ─── Helper: write a temporary workflows directory ────────────────────────────
|
|
|
|
function makeTempWorkflowsDir(files) {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'lhc-test-'));
|
|
for (const [name, content] of Object.entries(files)) {
|
|
fs.writeFileSync(path.join(tmpDir, name), content, 'utf8');
|
|
}
|
|
return tmpDir;
|
|
}
|
|
|
|
// ─── Minimal valid workflow content templates ─────────────────────────────────
|
|
|
|
function makeWorkflow(step, points, roles, produces, consumes, extraContent) {
|
|
const pointsList = points.join(', ');
|
|
const rolesList = roles.join(', ');
|
|
const producesList = produces.join(', ');
|
|
const consumesList = consumes.join(', ');
|
|
return (
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: ' + step + '\n' +
|
|
'points: ' + pointsList + '\n' +
|
|
'agent-roles: ' + rolesList + '\n' +
|
|
'produces: ' + producesList + '\n' +
|
|
'consumes: ' + consumesList + '\n' +
|
|
'-->\n' +
|
|
(extraContent || '')
|
|
);
|
|
}
|
|
|
|
// Minimal valid set of 5 workflows matching the canonical contract
|
|
function makeValidWorkflowFiles() {
|
|
return {
|
|
'discuss-phase.md': makeWorkflow('discuss', ['discuss:pre', 'discuss:post'], ['orchestrator'], ['CONTEXT.md'], []),
|
|
'plan-phase.md': makeWorkflow(
|
|
'plan', ['plan:pre', 'plan:post'],
|
|
['researcher', 'planner', 'checker'],
|
|
['PLAN.md'], ['CONTEXT.md'],
|
|
// Cross-check content: must contain the agent names
|
|
'gsd-phase-researcher gsd-planner gsd-plan-checker',
|
|
),
|
|
'quick.md': [
|
|
'PLAN_PRE_HOOKS_JSON=$(gsd_run loop render-hooks plan:pre --raw)',
|
|
'For each active entry where `kind == "contribution"` and `into == "planner"`: inject.',
|
|
'',
|
|
].join('\n'),
|
|
'execute-phase.md': makeWorkflow(
|
|
'execute', ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'],
|
|
['executor', 'verifier'],
|
|
['SUMMARY.md'], ['PLAN.md'],
|
|
'gsd-executor gsd-verifier',
|
|
),
|
|
'verify-work.md': makeWorkflow('verify', ['verify:pre', 'verify:post'], ['orchestrator'], ['UAT.md'], ['SUMMARY.md']),
|
|
'ship.md': makeWorkflow('ship', ['ship:pre', 'ship:post'], ['orchestrator'], [], ['UAT.md']),
|
|
};
|
|
}
|
|
|
|
// ─── 1. parseLoopHostBlock ────────────────────────────────────────────────────
|
|
|
|
describe('parseLoopHostBlock', () => {
|
|
test('parses a valid block correctly', () => {
|
|
const content = makeWorkflow(
|
|
'plan', ['plan:pre', 'plan:post'],
|
|
['researcher', 'planner', 'checker'],
|
|
['PLAN.md'], ['CONTEXT.md'],
|
|
);
|
|
const result = parseLoopHostBlock(content, 'plan-phase.md');
|
|
assert.strictEqual(result.step, 'plan');
|
|
assert.deepEqual(result.points, ['plan:pre', 'plan:post']);
|
|
assert.deepEqual(result.agentRoles, ['researcher', 'planner', 'checker']);
|
|
assert.deepEqual(result.coreArtifacts.produces, ['PLAN.md']);
|
|
assert.deepEqual(result.coreArtifacts.consumes, ['CONTEXT.md']);
|
|
});
|
|
|
|
test('parses empty produces field as empty array', () => {
|
|
const content = makeWorkflow(
|
|
'ship', ['ship:pre', 'ship:post'], ['orchestrator'], [], ['UAT.md'],
|
|
);
|
|
const result = parseLoopHostBlock(content, 'ship.md');
|
|
assert.deepEqual(result.coreArtifacts.produces, []);
|
|
assert.deepEqual(result.coreArtifacts.consumes, ['UAT.md']);
|
|
});
|
|
|
|
test('parses empty consumes field as empty array', () => {
|
|
const content = makeWorkflow(
|
|
'discuss', ['discuss:pre', 'discuss:post'], ['orchestrator'], ['CONTEXT.md'], [],
|
|
);
|
|
const result = parseLoopHostBlock(content, 'discuss-phase.md');
|
|
assert.deepEqual(result.coreArtifacts.consumes, []);
|
|
assert.deepEqual(result.coreArtifacts.produces, ['CONTEXT.md']);
|
|
});
|
|
|
|
test('throws when block is missing', () => {
|
|
assert.throws(
|
|
() => parseLoopHostBlock('no block here\n<purpose>hello</purpose>', 'test.md'),
|
|
/missing.*gsd:loop-host/,
|
|
);
|
|
});
|
|
|
|
test('throws when step field is missing from block', () => {
|
|
const content =
|
|
'<!-- gsd:loop-host\n' +
|
|
'points: plan:pre, plan:post\n' +
|
|
'agent-roles: orchestrator\n' +
|
|
'produces:\n' +
|
|
'consumes:\n' +
|
|
'-->\n';
|
|
assert.throws(
|
|
() => parseLoopHostBlock(content, 'test.md'),
|
|
/missing required field "step"/,
|
|
);
|
|
});
|
|
|
|
test('throws when points field is empty', () => {
|
|
const content =
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: plan\n' +
|
|
'points:\n' +
|
|
'agent-roles: orchestrator\n' +
|
|
'produces:\n' +
|
|
'consumes:\n' +
|
|
'-->\n';
|
|
assert.throws(
|
|
() => parseLoopHostBlock(content, 'test.md'),
|
|
/"points" must have at least one value/,
|
|
);
|
|
});
|
|
|
|
test('parses multi-value fields with spaces correctly', () => {
|
|
const content = makeWorkflow(
|
|
'execute',
|
|
['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'],
|
|
['executor', 'verifier'],
|
|
['SUMMARY.md'], ['PLAN.md'],
|
|
);
|
|
const result = parseLoopHostBlock(content, 'execute-phase.md');
|
|
assert.deepEqual(result.points, ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post']);
|
|
assert.deepEqual(result.agentRoles, ['executor', 'verifier']);
|
|
});
|
|
});
|
|
|
|
// ─── 2. crossCheckRoles ───────────────────────────────────────────────────────
|
|
|
|
describe('crossCheckRoles', () => {
|
|
test('passes for orchestrator-only roles (no agent file needed)', () => {
|
|
const errors = crossCheckRoles('anything', ['orchestrator'], 'discuss-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('passes when agent name is present in content', () => {
|
|
const content = 'Agent(subagent_type="gsd-planner") Agent(subagent_type="gsd-phase-researcher") gsd-plan-checker';
|
|
const errors = crossCheckRoles(content, ['researcher', 'planner', 'checker'], 'plan-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('fails when declared role has no agent reference in content', () => {
|
|
const content = 'gsd-phase-researcher gsd-plan-checker'; // planner missing
|
|
const errors = crossCheckRoles(content, ['researcher', 'planner', 'checker'], 'plan-phase.md');
|
|
assert.strictEqual(errors.length, 1, 'expected exactly 1 error for missing planner');
|
|
assert.match(errors[0], /planner.*gsd-planner/);
|
|
});
|
|
|
|
test('fails when declared role is unknown (not in ROLE_TO_AGENT)', () => {
|
|
const content = 'gsd-executor gsd-verifier';
|
|
const errors = crossCheckRoles(content, ['executor', 'nonexistent-role'], 'execute-phase.md');
|
|
assert.ok(errors.some((e) => e.includes('nonexistent-role') && e.includes('ROLE_TO_AGENT')));
|
|
});
|
|
});
|
|
|
|
// ─── 2b. crossCheckRoleFamilies (#4740) ──────────────────────────────────────
|
|
|
|
describe('crossCheckRoleFamilies (#4740)', () => {
|
|
test('accepts the execution family at execute', () => {
|
|
const errors = crossCheckRoleFamilies('execute', ['executor', 'verifier'], 'execute-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('accepts orchestrator at discuss', () => {
|
|
const errors = crossCheckRoleFamilies('discuss', ['orchestrator'], 'discuss-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('accepts the planning family at plan', () => {
|
|
const errors = crossCheckRoleFamilies('plan', ['researcher', 'planner', 'checker'], 'plan-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('accepts orchestrator at verify and ship', () => {
|
|
assert.deepEqual(crossCheckRoleFamilies('verify', ['orchestrator'], 'verify-work.md'), []);
|
|
assert.deepEqual(crossCheckRoleFamilies('ship', ['orchestrator'], 'ship.md'), []);
|
|
});
|
|
|
|
test('a strict subset of the family is legal', () => {
|
|
const errors = crossCheckRoleFamilies('execute', ['executor'], 'execute-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('rejects orchestrator added to execute', () => {
|
|
const errors = crossCheckRoleFamilies('execute', ['executor', 'verifier', 'orchestrator'], 'execute-phase.md');
|
|
assert.strictEqual(errors.length, 1, 'expected exactly 1 error');
|
|
assert.match(errors[0], /orchestrator/);
|
|
assert.match(errors[0], /execute/);
|
|
});
|
|
|
|
test('rejects an execution role at an orchestration step', () => {
|
|
const errors = crossCheckRoleFamilies('discuss', ['orchestrator', 'executor'], 'discuss-phase.md');
|
|
assert.strictEqual(errors.length, 1, 'expected exactly 1 error');
|
|
assert.match(errors[0], /executor/);
|
|
});
|
|
|
|
test('rejects an execution role at plan', () => {
|
|
const errors = crossCheckRoleFamilies('plan', ['planner', 'executor'], 'plan-phase.md');
|
|
assert.strictEqual(errors.length, 1, 'expected exactly 1 error');
|
|
assert.match(errors[0], /executor/);
|
|
});
|
|
|
|
test('rejects a planning role at execute', () => {
|
|
const errors = crossCheckRoleFamilies('execute', ['executor', 'planner'], 'execute-phase.md');
|
|
assert.strictEqual(errors.length, 1, 'expected exactly 1 error');
|
|
assert.match(errors[0], /planner/);
|
|
});
|
|
|
|
test('rejects a role with no family', () => {
|
|
const errors = crossCheckRoleFamilies('execute', ['executer'], 'execute-phase.md');
|
|
assert.ok(errors.length >= 1, 'expected at least 1 error');
|
|
assert.ok(errors.some((e) => e.includes('executer')));
|
|
});
|
|
|
|
test('fails closed on an unknown step', () => {
|
|
const errors = crossCheckRoleFamilies('audit', ['orchestrator'], 'audit.md');
|
|
assert.strictEqual(errors.length, 1, 'expected exactly 1 error for unknown step');
|
|
assert.match(errors[0], /audit/);
|
|
});
|
|
|
|
test('rejects a role whose capitalization does not match', () => {
|
|
const errors = crossCheckRoleFamilies('execute', ['Orchestrator'], 'execute-phase.md');
|
|
assert.ok(errors.length >= 1, 'capitalized role must not silently match');
|
|
assert.ok(errors.some((e) => e.includes('Orchestrator')));
|
|
});
|
|
|
|
test('a duplicated in-family role is not an error', () => {
|
|
const errors = crossCheckRoleFamilies('execute', ['executor', 'executor'], 'execute-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('reports every offending role, not just the first', () => {
|
|
const errors = crossCheckRoleFamilies('execute', ['executor', 'orchestrator', 'planner'], 'execute-phase.md');
|
|
assert.strictEqual(errors.length, 2, 'expected one error per offending role');
|
|
const combined = errors.join('\n');
|
|
assert.ok(combined.includes('orchestrator'));
|
|
assert.ok(combined.includes('planner'));
|
|
});
|
|
|
|
test('is pure and does not mutate its arguments', () => {
|
|
// Deliberately NOT alphabetically sorted — an in-place agentRoles.sort()
|
|
// must be caught by THIS test, not merely coincide with already-sorted
|
|
// input (#4740 review finding).
|
|
const agentRoles = ['verifier', 'executor', 'orchestrator'];
|
|
const snapshot = agentRoles.slice();
|
|
const first = crossCheckRoleFamilies('execute', agentRoles, 'execute-phase.md');
|
|
assert.deepEqual(agentRoles, snapshot, 'input array must not be mutated');
|
|
const second = crossCheckRoleFamilies('execute', agentRoles, 'execute-phase.md');
|
|
assert.deepEqual(first, second, 'repeated calls must produce identical results');
|
|
assert.deepEqual(agentRoles, snapshot, 'input array must still not be mutated after second call');
|
|
});
|
|
|
|
test('ROLE_FAMILY and ROLE_TO_AGENT cover the exact same role-name domain (#4740)', () => {
|
|
// Generative-fix-divergence guard: ROLE_FAMILY and ROLE_TO_AGENT are parallel
|
|
// constants over the same role-name domain (every non-orchestrator role, plus
|
|
// orchestrator which is family-only). Exact set equality both directions —
|
|
// not a subset check — so adding a role to only one map is caught here.
|
|
const familyKeys = new Set(Object.keys(ROLE_FAMILY));
|
|
const expectedKeys = new Set([...Object.keys(ROLE_TO_AGENT), 'orchestrator']);
|
|
assert.deepEqual(
|
|
[...familyKeys].sort(),
|
|
[...expectedKeys].sort(),
|
|
'ROLE_FAMILY keys must equal ROLE_TO_AGENT keys plus "orchestrator", exactly',
|
|
);
|
|
});
|
|
|
|
test('property: in-family subsets pass, any foreign role fails', () => {
|
|
const ROLE_FAMILY_LOCAL = {
|
|
orchestrator: 'orchestration',
|
|
researcher: 'planning', planner: 'planning', checker: 'planning',
|
|
executor: 'execution', verifier: 'execution',
|
|
};
|
|
const familyRoles = {
|
|
orchestration: ['orchestrator'],
|
|
planning: ['researcher', 'planner', 'checker'],
|
|
execution: ['executor', 'verifier'],
|
|
};
|
|
const stepsByFamily = {
|
|
discuss: 'orchestration', plan: 'planning', execute: 'execution',
|
|
verify: 'orchestration', ship: 'orchestration',
|
|
};
|
|
const steps = Object.keys(stepsByFamily);
|
|
|
|
fc.assert(
|
|
fc.property(
|
|
fc.constantFrom(...steps),
|
|
fc.subarray(Object.keys(ROLE_FAMILY_LOCAL), { minLength: 1 }),
|
|
(step, roles) => {
|
|
const family = stepsByFamily[step];
|
|
const inFamilySubset = familyRoles[family];
|
|
const errors = crossCheckRoleFamilies(step, inFamilySubset, 'x.md');
|
|
assert.deepEqual(errors, []);
|
|
|
|
const hasForeign = roles.some((r) => ROLE_FAMILY_LOCAL[r] !== family);
|
|
const errors2 = crossCheckRoleFamilies(step, roles, 'x.md');
|
|
if (hasForeign) {
|
|
assert.ok(errors2.length >= 1);
|
|
} else {
|
|
assert.deepEqual(errors2, []);
|
|
}
|
|
},
|
|
),
|
|
{ seed: 4740, numRuns: 200 },
|
|
);
|
|
});
|
|
});
|
|
|
|
// ─── 3. assertPointsCoverage ─────────────────────────────────────────────────
|
|
|
|
describe('assertPointsCoverage', () => {
|
|
test('passes when all 12 canonical points are covered', () => {
|
|
const entries = [
|
|
{ step: 'discuss', points: ['discuss:pre', 'discuss:post'] },
|
|
{ step: 'plan', points: ['plan:pre', 'plan:post'] },
|
|
{ step: 'execute', points: ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'] },
|
|
{ step: 'verify', points: ['verify:pre', 'verify:post'] },
|
|
{ step: 'ship', points: ['ship:pre', 'ship:post'] },
|
|
];
|
|
const errors = assertPointsCoverage(entries);
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('fails when a canonical point is missing', () => {
|
|
const entries = [
|
|
{ step: 'discuss', points: ['discuss:pre'] }, // missing discuss:post
|
|
{ step: 'plan', points: ['plan:pre', 'plan:post'] },
|
|
{ step: 'execute', points: ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'] },
|
|
{ step: 'verify', points: ['verify:pre', 'verify:post'] },
|
|
{ step: 'ship', points: ['ship:pre', 'ship:post'] },
|
|
];
|
|
const errors = assertPointsCoverage(entries);
|
|
assert.ok(errors.some((e) => e.includes('discuss:post') && e.includes('not declared')));
|
|
});
|
|
|
|
test('fails when an unknown point is declared', () => {
|
|
const entries = [
|
|
{ step: 'discuss', points: ['discuss:pre', 'discuss:post', 'discuss:extra'] },
|
|
{ step: 'plan', points: ['plan:pre', 'plan:post'] },
|
|
{ step: 'execute', points: ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'] },
|
|
{ step: 'verify', points: ['verify:pre', 'verify:post'] },
|
|
{ step: 'ship', points: ['ship:pre', 'ship:post'] },
|
|
];
|
|
const errors = assertPointsCoverage(entries);
|
|
assert.ok(errors.some((e) => e.includes('discuss:extra') && e.includes('not in the canonical')));
|
|
});
|
|
|
|
test('fails when a point is declared twice', () => {
|
|
const entries = [
|
|
{ step: 'discuss', points: ['discuss:pre', 'discuss:post'] },
|
|
{ step: 'plan', points: ['plan:pre', 'plan:post', 'discuss:pre'] }, // duplicate
|
|
{ step: 'execute', points: ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'] },
|
|
{ step: 'verify', points: ['verify:pre', 'verify:post'] },
|
|
{ step: 'ship', points: ['ship:pre', 'ship:post'] },
|
|
];
|
|
const errors = assertPointsCoverage(entries);
|
|
assert.ok(errors.some((e) => e.includes('discuss:pre') && e.includes('more than once')));
|
|
});
|
|
});
|
|
|
|
// ─── 4. buildContract — from real workflows ───────────────────────────────────
|
|
|
|
describe('buildContract from real workflows', () => {
|
|
test('produces a contract matching the inline LOOP_HOST_CONTRACT shape', () => {
|
|
const contract = buildContract(); // reads real gsd-core/workflows/
|
|
|
|
// Must be an array of 5 entries
|
|
assert.strictEqual(contract.length, 5, 'contract must have 5 step entries');
|
|
|
|
// Each entry must have step, points, agentRoles, coreArtifacts
|
|
for (const entry of contract) {
|
|
assert.ok(typeof entry.step === 'string', 'entry.step must be a string');
|
|
assert.ok(Array.isArray(entry.points), 'entry.points must be an array');
|
|
assert.ok(Array.isArray(entry.agentRoles), 'entry.agentRoles must be an array');
|
|
assert.ok(typeof entry.coreArtifacts === 'object', 'entry.coreArtifacts must be an object');
|
|
assert.ok(Array.isArray(entry.coreArtifacts.produces), 'entry.coreArtifacts.produces must be an array');
|
|
assert.ok(Array.isArray(entry.coreArtifacts.consumes), 'entry.coreArtifacts.consumes must be an array');
|
|
}
|
|
|
|
// Verify exact match with the committed loop-host-contract.cjs
|
|
assert.deepEqual(contract, LOOP_HOST_CONTRACT, 'built contract must match committed LOOP_HOST_CONTRACT');
|
|
});
|
|
|
|
test('covers exactly the 12 canonical points', () => {
|
|
const contract = buildContract();
|
|
const allPoints = contract.flatMap((e) => e.points);
|
|
assert.strictEqual(allPoints.length, 12, 'must cover exactly 12 points');
|
|
const pointSet = new Set(allPoints);
|
|
assert.strictEqual(pointSet.size, 12, 'all 12 points must be distinct');
|
|
for (const p of CANONICAL_POINTS) {
|
|
assert.ok(pointSet.has(p), 'canonical point "' + p + '" must be declared');
|
|
}
|
|
});
|
|
|
|
test('discuss step has orchestrator role and produces CONTEXT.md', () => {
|
|
const contract = buildContract();
|
|
const discuss = contract.find((e) => e.step === 'discuss');
|
|
assert.ok(discuss, 'discuss step must be present');
|
|
assert.deepEqual(discuss.agentRoles, ['orchestrator']);
|
|
assert.deepEqual(discuss.coreArtifacts.produces, ['CONTEXT.md']);
|
|
assert.deepEqual(discuss.coreArtifacts.consumes, []);
|
|
});
|
|
|
|
test('plan step has researcher/planner/checker roles and produces PLAN.md', () => {
|
|
const contract = buildContract();
|
|
const plan = contract.find((e) => e.step === 'plan');
|
|
assert.ok(plan, 'plan step must be present');
|
|
assert.deepEqual(plan.agentRoles, ['researcher', 'planner', 'checker']);
|
|
assert.deepEqual(plan.coreArtifacts.produces, ['PLAN.md']);
|
|
assert.deepEqual(plan.coreArtifacts.consumes, ['CONTEXT.md']);
|
|
});
|
|
|
|
test('execute step has executor/verifier roles and 4 points', () => {
|
|
const contract = buildContract();
|
|
const execute = contract.find((e) => e.step === 'execute');
|
|
assert.ok(execute, 'execute step must be present');
|
|
assert.deepEqual(execute.agentRoles, ['executor', 'verifier']);
|
|
assert.deepEqual(execute.points, ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post']);
|
|
assert.deepEqual(execute.coreArtifacts.produces, ['SUMMARY.md']);
|
|
assert.deepEqual(execute.coreArtifacts.consumes, ['PLAN.md']);
|
|
});
|
|
|
|
test('verify step has orchestrator role and produces UAT.md', () => {
|
|
const contract = buildContract();
|
|
const verify = contract.find((e) => e.step === 'verify');
|
|
assert.ok(verify, 'verify step must be present');
|
|
assert.deepEqual(verify.agentRoles, ['orchestrator']);
|
|
assert.deepEqual(verify.coreArtifacts.produces, ['UAT.md']);
|
|
assert.deepEqual(verify.coreArtifacts.consumes, ['SUMMARY.md']);
|
|
});
|
|
|
|
test('ship step has orchestrator role and empty produces', () => {
|
|
const contract = buildContract();
|
|
const ship = contract.find((e) => e.step === 'ship');
|
|
assert.ok(ship, 'ship step must be present');
|
|
assert.deepEqual(ship.agentRoles, ['orchestrator']);
|
|
assert.deepEqual(ship.coreArtifacts.produces, []);
|
|
assert.deepEqual(ship.coreArtifacts.consumes, ['UAT.md']);
|
|
});
|
|
|
|
test('the shipped workflows declare no cross-family role (#4740)', () => {
|
|
// Pins something real about THIS change, distinct from the per-step field
|
|
// assertions above: running crossCheckRoleFamilies directly against every
|
|
// entry of the REAL contract returns zero errors for every step, not merely
|
|
// that buildContract() didn't throw (which every other test in this describe
|
|
// block already exercises incidentally).
|
|
const contract = buildContract();
|
|
for (const entry of contract) {
|
|
const errors = crossCheckRoleFamilies(entry.step, entry.agentRoles, entry.step + '.md');
|
|
assert.deepEqual(errors, [], 'step "' + entry.step + '" must declare no cross-family role');
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── 5. cross-check rejects nonexistent agent-role ───────────────────────────
|
|
|
|
describe('buildContract cross-check drift guard', () => {
|
|
test('rejects a block declaring a nonexistent agent-role', () => {
|
|
// Build a temporary workflows dir where plan-phase.md declares a role
|
|
// that has no corresponding agent reference in the file content.
|
|
const files = makeValidWorkflowFiles();
|
|
// Override plan-phase.md to declare a "phantom" role with no agent reference
|
|
files['plan-phase.md'] =
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: plan\n' +
|
|
'points: plan:pre, plan:post\n' +
|
|
'agent-roles: researcher, planner, checker, phantom\n' +
|
|
'produces: PLAN.md\n' +
|
|
'consumes: CONTEXT.md\n' +
|
|
'-->\n' +
|
|
// Include real agents but NOT the phantom role's agent (phantom is not in ROLE_TO_AGENT)
|
|
'gsd-phase-researcher gsd-planner gsd-plan-checker\n';
|
|
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/ROLE_TO_AGENT|no entry/,
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
|
|
test('rejects a block declaring a role whose agent is absent from the workflow', () => {
|
|
// plan-phase.md declares 'researcher' but does NOT mention gsd-phase-researcher
|
|
const files = makeValidWorkflowFiles();
|
|
files['plan-phase.md'] =
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: plan\n' +
|
|
'points: plan:pre, plan:post\n' +
|
|
'agent-roles: researcher, planner, checker\n' +
|
|
'produces: PLAN.md\n' +
|
|
'consumes: CONTEXT.md\n' +
|
|
'-->\n' +
|
|
// Only planner and checker present, researcher's agent is absent
|
|
'gsd-planner gsd-plan-checker\n';
|
|
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/gsd-phase-researcher.*not referenced|researcher.*gsd-phase-researcher/,
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── 6. --check: CRLF-agnostic + committed-file staleness guard ──────────────
|
|
|
|
describe('normalizeLineEndings', () => {
|
|
test('normalizeLineEndings strips CR characters', () => {
|
|
const crlf = 'line1\r\nline2\r\nline3';
|
|
const lf = 'line1\nline2\nline3';
|
|
assert.strictEqual(normalizeLineEndings(crlf), lf);
|
|
assert.strictEqual(normalizeLineEndings(lf), lf);
|
|
});
|
|
// The committed-loop-host-contract.cjs freshness guard moved to
|
|
// `npm run lint:generated-sync` (gen-loop-host-contract.cjs --check), where it
|
|
// runs against the committed file in both local and CI lint lanes — instead of
|
|
// being masked by gsd-test's `npm run build` leg regenerating the artifact.
|
|
});
|
|
|
|
// ─── 7. STEP_WORKFLOWS and CANONICAL_POINTS exported constants ────────────────
|
|
|
|
describe('module exports', () => {
|
|
test('STEP_WORKFLOWS has 5 entries in pipeline order', () => {
|
|
assert.strictEqual(STEP_WORKFLOWS.length, 5);
|
|
assert.strictEqual(STEP_WORKFLOWS[0].step, 'discuss');
|
|
assert.strictEqual(STEP_WORKFLOWS[1].step, 'plan');
|
|
assert.strictEqual(STEP_WORKFLOWS[2].step, 'execute');
|
|
assert.strictEqual(STEP_WORKFLOWS[3].step, 'verify');
|
|
assert.strictEqual(STEP_WORKFLOWS[4].step, 'ship');
|
|
});
|
|
|
|
test('CANONICAL_POINTS has exactly 12 entries', () => {
|
|
assert.strictEqual(CANONICAL_POINTS.length, 12);
|
|
});
|
|
|
|
test('ROLE_TO_AGENT covers all non-orchestrator roles', () => {
|
|
// All non-orchestrator roles from the real contract
|
|
const allRoles = new Set(
|
|
LOOP_HOST_CONTRACT.flatMap((e) => e.agentRoles).filter((r) => r !== 'orchestrator'),
|
|
);
|
|
for (const role of allRoles) {
|
|
assert.ok(
|
|
ROLE_TO_AGENT[role] !== undefined,
|
|
'ROLE_TO_AGENT must cover non-orchestrator role "' + role + '"',
|
|
);
|
|
}
|
|
});
|
|
|
|
test('EXPECTED_POINTS_BY_STEP covers all 5 steps', () => {
|
|
assert.ok(EXPECTED_POINTS_BY_STEP, 'EXPECTED_POINTS_BY_STEP must be exported');
|
|
assert.strictEqual(Object.keys(EXPECTED_POINTS_BY_STEP).length, 5);
|
|
assert.ok(Array.isArray(EXPECTED_POINTS_BY_STEP.execute));
|
|
assert.strictEqual(EXPECTED_POINTS_BY_STEP.execute.length, 4);
|
|
});
|
|
});
|
|
|
|
// ─── #3778 — quick.md is auxiliary without becoming a 6th serialized step ────
|
|
|
|
describe('Quick auxiliary loop-host contract (#3778)', () => {
|
|
function makeQuickWorkflow({ includePoint = true, includeKind = true, into = 'planner' } = {}) {
|
|
const target = into === null ? '' : ` and \`into == "${into}"\``;
|
|
return [
|
|
includePoint ? 'PLAN_PRE_HOOKS_JSON=$(gsd_run loop render-hooks plan:pre --raw)' : '',
|
|
includeKind ? `For each active entry where \`kind == "contribution"\`${target}: inject.` : '',
|
|
'',
|
|
].join('\n');
|
|
}
|
|
|
|
test('Quick is a generator-owned plan auxiliary host with its own contribution expectation', () => {
|
|
const plan = STEP_WORKFLOWS.find((workflow) => workflow.step === 'plan');
|
|
assert.deepEqual(plan.auxiliaryHosts, [
|
|
{ file: 'quick.md', point: 'plan:pre', kinds: ['contribution'], into: 'planner' },
|
|
]);
|
|
|
|
const files = makeValidWorkflowFiles();
|
|
files['quick.md'] = makeQuickWorkflow();
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
const contract = buildContract(tmpDir);
|
|
assert.strictEqual(contract.length, 5, 'auxiliary hosts must not create a sixth serialized step');
|
|
assert.strictEqual(
|
|
new Set(contract.flatMap((entry) => entry.points)).size,
|
|
12,
|
|
'auxiliary hosts must not create a duplicate lifecycle point',
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
|
|
test('buildContract rejects Quick without its plan:pre render seam', () => {
|
|
const files = makeValidWorkflowFiles();
|
|
files['quick.md'] = makeQuickWorkflow({ includePoint: false });
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/quick\.md: auxiliary host missing expected point "plan:pre"/,
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
|
|
test('buildContract rejects Quick without its contribution dispatch', () => {
|
|
const files = makeValidWorkflowFiles();
|
|
files['quick.md'] = makeQuickWorkflow({ includeKind: false });
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/quick\.md: auxiliary host point "plan:pre" missing expected kind "contribution"/,
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
|
|
test('buildContract rejects a Quick contribution dispatch targeting the wrong role', () => {
|
|
const files = makeValidWorkflowFiles();
|
|
files['quick.md'] = makeQuickWorkflow({ into: 'orchestrator' });
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/quick\.md: auxiliary host point "plan:pre" missing expected kind "contribution"/,
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
|
|
test('buildContract rejects a Quick contribution dispatch without its planner target', () => {
|
|
const files = makeValidWorkflowFiles();
|
|
files['quick.md'] = makeQuickWorkflow({ into: null });
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/quick\.md: auxiliary host point "plan:pre" missing expected kind "contribution"/,
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── 8. Regression: FIX 1 — per-step point ownership ────────────────────────
|
|
|
|
describe('assertPointsCoverage per-step ownership (FIX 1)', () => {
|
|
test('fails when two steps swap a point (discuss declares plan:pre, plan declares discuss:pre)', () => {
|
|
const entries = [
|
|
{ step: 'discuss', points: ['discuss:post', 'plan:pre'] }, // wrong: has plan:pre instead of discuss:pre
|
|
{ step: 'plan', points: ['discuss:pre', 'plan:post'] }, // wrong: has discuss:pre instead of plan:pre
|
|
{ step: 'execute', points: ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'] },
|
|
{ step: 'verify', points: ['verify:pre', 'verify:post'] },
|
|
{ step: 'ship', points: ['ship:pre', 'ship:post'] },
|
|
];
|
|
const errors = assertPointsCoverage(entries);
|
|
assert.ok(errors.length > 0, 'expected per-step ownership errors');
|
|
const combined = errors.join('\n');
|
|
// Both steps should be named in the errors
|
|
assert.ok(combined.includes('discuss'), 'error must mention discuss step');
|
|
assert.ok(combined.includes('plan'), 'error must mention plan step');
|
|
});
|
|
|
|
test('fails when a step is missing one of its own points', () => {
|
|
const entries = [
|
|
{ step: 'discuss', points: ['discuss:pre'] }, // missing discuss:post
|
|
{ step: 'plan', points: ['plan:pre', 'plan:post'] },
|
|
{ step: 'execute', points: ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'] },
|
|
{ step: 'verify', points: ['verify:pre', 'verify:post'] },
|
|
{ step: 'ship', points: ['ship:pre', 'ship:post'] },
|
|
];
|
|
const errors = assertPointsCoverage(entries);
|
|
assert.ok(errors.length > 0, 'expected ownership error for missing point');
|
|
assert.ok(
|
|
errors.some((e) => e.includes('discuss') && e.includes('expected')),
|
|
'error must name the step and expected points',
|
|
);
|
|
});
|
|
|
|
test('fails when a step has an extra point beyond its own', () => {
|
|
const entries = [
|
|
{ step: 'discuss', points: ['discuss:pre', 'discuss:post', 'plan:pre'] }, // extra: plan:pre
|
|
{ step: 'plan', points: ['plan:post'] }, // missing: plan:pre
|
|
{ step: 'execute', points: ['execute:pre', 'execute:wave:pre', 'execute:wave:post', 'execute:post'] },
|
|
{ step: 'verify', points: ['verify:pre', 'verify:post'] },
|
|
{ step: 'ship', points: ['ship:pre', 'ship:post'] },
|
|
];
|
|
const errors = assertPointsCoverage(entries);
|
|
assert.ok(errors.length > 0, 'expected ownership errors for extra and missing points');
|
|
});
|
|
|
|
test('buildContract with swapped points across steps throws a per-step ownership error', () => {
|
|
const files = makeValidWorkflowFiles();
|
|
// discuss declares plan:pre instead of discuss:pre, plan declares discuss:pre instead of plan:pre
|
|
files['discuss-phase.md'] = makeWorkflow('discuss', ['discuss:post', 'plan:pre'], ['orchestrator'], ['CONTEXT.md'], []);
|
|
files['plan-phase.md'] = makeWorkflow(
|
|
'plan', ['discuss:pre', 'plan:post'],
|
|
['researcher', 'planner', 'checker'],
|
|
['PLAN.md'], ['CONTEXT.md'],
|
|
'gsd-phase-researcher gsd-planner gsd-plan-checker',
|
|
);
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
let cleaned = false;
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/step.*discuss.*expected|step.*plan.*expected/,
|
|
);
|
|
} finally {
|
|
if (!cleaned) {
|
|
cleanup(tmpDir);
|
|
cleaned = true;
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── 9. Regression: FIX 2 — multiple blocks + duplicate keys ─────────────────
|
|
|
|
describe('parseLoopHostBlock multiple-block and duplicate-key detection (FIX 2)', () => {
|
|
test('throws when a file has two gsd:loop-host marker blocks', () => {
|
|
const block =
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: discuss\n' +
|
|
'points: discuss:pre, discuss:post\n' +
|
|
'agent-roles: orchestrator\n' +
|
|
'produces: CONTEXT.md\n' +
|
|
'consumes:\n' +
|
|
'-->\n';
|
|
const content = block + '\nSome prose.\n\n' + block;
|
|
assert.throws(
|
|
() => parseLoopHostBlock(content, 'discuss-phase.md'),
|
|
/expected exactly one gsd:loop-host marker block, found 2/,
|
|
);
|
|
});
|
|
|
|
test('throws when a block has a duplicate "points" key', () => {
|
|
const content =
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: discuss\n' +
|
|
'points: discuss:pre, discuss:post\n' +
|
|
'points: discuss:pre\n' + // duplicate
|
|
'agent-roles: orchestrator\n' +
|
|
'produces: CONTEXT.md\n' +
|
|
'consumes:\n' +
|
|
'-->\n';
|
|
assert.throws(
|
|
() => parseLoopHostBlock(content, 'discuss-phase.md'),
|
|
/duplicate key 'points' in gsd:loop-host marker/,
|
|
);
|
|
});
|
|
|
|
test('throws when a block has a duplicate "step" key', () => {
|
|
const content =
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: discuss\n' +
|
|
'step: plan\n' + // duplicate
|
|
'points: discuss:pre, discuss:post\n' +
|
|
'agent-roles: orchestrator\n' +
|
|
'produces: CONTEXT.md\n' +
|
|
'consumes:\n' +
|
|
'-->\n';
|
|
assert.throws(
|
|
() => parseLoopHostBlock(content, 'discuss-phase.md'),
|
|
/duplicate key 'step' in gsd:loop-host marker/,
|
|
);
|
|
});
|
|
|
|
test('buildContract with a two-block file throws with "found 2" error', () => {
|
|
const files = makeValidWorkflowFiles();
|
|
const singleBlock =
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: discuss\n' +
|
|
'points: discuss:pre, discuss:post\n' +
|
|
'agent-roles: orchestrator\n' +
|
|
'produces: CONTEXT.md\n' +
|
|
'consumes:\n' +
|
|
'-->\n';
|
|
files['discuss-phase.md'] = singleBlock + '\nDoc example:\n\n' + singleBlock;
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/found 2/,
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── 10. Regression: FIX 3 — word-boundary agent cross-check ─────────────────
|
|
|
|
describe('crossCheckRoles word-boundary match (FIX 3)', () => {
|
|
test('gsd-plan-checker-v2 does NOT satisfy required gsd-plan-checker reference', () => {
|
|
// Content has gsd-plan-checker-v2 but NOT bare gsd-plan-checker
|
|
const content = 'Agent("gsd-phase-researcher") Agent("gsd-planner") gsd-plan-checker-v2';
|
|
const errors = crossCheckRoles(content, ['researcher', 'planner', 'checker'], 'plan-phase.md');
|
|
assert.strictEqual(errors.length, 1, 'expected exactly 1 error for checker missing bare reference');
|
|
assert.match(errors[0], /gsd-plan-checker/);
|
|
});
|
|
|
|
test('gsd-plan-checker (bare) still satisfies the checker role', () => {
|
|
const content = 'Agent("gsd-phase-researcher") Agent("gsd-planner") gsd-plan-checker something-else';
|
|
const errors = crossCheckRoles(content, ['researcher', 'planner', 'checker'], 'plan-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('gsd-plan-checker immediately followed by newline satisfies the checker role', () => {
|
|
const content = 'gsd-phase-researcher\ngsd-planner\ngsd-plan-checker\n';
|
|
const errors = crossCheckRoles(content, ['researcher', 'planner', 'checker'], 'plan-phase.md');
|
|
assert.deepEqual(errors, []);
|
|
});
|
|
|
|
test('buildContract rejects workflow referencing only -v2 agent variant', () => {
|
|
const files = makeValidWorkflowFiles();
|
|
// plan-phase.md refers to gsd-plan-checker-v2 but not gsd-plan-checker
|
|
files['plan-phase.md'] =
|
|
'<!-- gsd:loop-host\n' +
|
|
'step: plan\n' +
|
|
'points: plan:pre, plan:post\n' +
|
|
'agent-roles: researcher, planner, checker\n' +
|
|
'produces: PLAN.md\n' +
|
|
'consumes: CONTEXT.md\n' +
|
|
'-->\n' +
|
|
'gsd-phase-researcher gsd-planner gsd-plan-checker-v2\n';
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
try {
|
|
assert.throws(
|
|
() => buildContract(tmpDir),
|
|
/gsd-plan-checker.*not referenced|checker.*gsd-plan-checker/,
|
|
);
|
|
} finally {
|
|
cleanup(tmpDir);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── 11. Regression: FIX 4 — --check detects header/body tampering ───────────
|
|
|
|
describe('--check full-content comparison (FIX 4)', () => {
|
|
test('committed file with tampered header is detected as stale', () => {
|
|
const contract = buildContract();
|
|
const live = serializeContract(contract);
|
|
// Tamper: replace the DO-NOT-EDIT line with something else
|
|
const tampered = live.replace(
|
|
' * DO NOT EDIT BY HAND. Run: node scripts/gen-loop-host-contract.cjs --write',
|
|
' * TAMPERED HEADER LINE',
|
|
);
|
|
assert.notStrictEqual(
|
|
normalizeLineEndings(tampered),
|
|
normalizeLineEndings(live),
|
|
'tampered content must differ from live content (staleness detected)',
|
|
);
|
|
});
|
|
|
|
test('committed file with tampered body JSON is detected as stale', () => {
|
|
const contract = buildContract();
|
|
const live = serializeContract(contract);
|
|
// Tamper: add a phantom step name
|
|
const tampered = live.replace('"step": "discuss"', '"step": "discuss-tampered"');
|
|
assert.notStrictEqual(
|
|
normalizeLineEndings(tampered),
|
|
normalizeLineEndings(live),
|
|
'tampered body must differ from live content (staleness detected)',
|
|
);
|
|
});
|
|
|
|
test('un-tampered committed file passes full-content comparison', () => {
|
|
const contract = buildContract();
|
|
const live = serializeContract(contract);
|
|
const committed = fs.readFileSync(CONTRACT_PATH, 'utf8');
|
|
assert.strictEqual(
|
|
normalizeLineEndings(committed),
|
|
normalizeLineEndings(live),
|
|
'committed file must match live serialization exactly (full-content comparison)',
|
|
);
|
|
});
|
|
});
|
|
|
|
// ─── 12. Regression: FIX 5 — temp-dir cleanup in existing cross-check tests ──
|
|
// (cleanup is handled via try/finally in each test above that creates temp dirs;
|
|
// this suite documents and verifies the makeTempWorkflowsDir helper itself)
|
|
|
|
describe('temp-dir lifecycle', () => {
|
|
test('makeTempWorkflowsDir creates a directory that can be cleaned up', () => {
|
|
const files = { 'dummy.md': '<!-- gsd:loop-host\nstep: discuss\n-->' };
|
|
const tmpDir = makeTempWorkflowsDir(files);
|
|
assert.ok(fs.existsSync(tmpDir), 'temp dir must exist after creation');
|
|
cleanup(tmpDir);
|
|
assert.ok(!fs.existsSync(tmpDir), 'temp dir must not exist after cleanup');
|
|
});
|
|
});
|