Files
msd-core/tests/declarative-reference-zcode.test.cjs
Tom Boucher 3fac6e629f test(#3145): bound the installer/runtime cluster onto the process seam (#3176)
* test(#3145): bound the installer/runtime cluster onto the process seam

Migrates 156 unbounded sync spawn sites across 47 files. Allowlist 120 to 73.

Timeouts are sized from evidence already in the tree rather than a house
default, because this wave spawns installers rather than git plumbing and an
undersized bound does not catch a hang -- it manufactures CI flake, which is
worse, since a flake gets re-run instead of investigated. install.test.cjs
records a real spawnSync ETIMEDOUT at a 60000ms cap on a loaded bench while
another lane passed the same commit in 12.7s, so full installs are bound at
120000ms against that recorded incident.

Also adds an auditable escape to the guard's timeout ceiling. The 600000ms
cap was set in #3143 from partial evidence, but fragment-single-edit-
propagation carries a documented, load-tested 900000ms bound on a run that
chains a full build plus eight generators -- the guard would have rejected a
correct timeout the moment that file left the allowlist. A value above the
ceiling is now permitted only with an inline allow-spawn-timeout-ceiling
marker carrying a non-empty reason. It raises the ceiling; it never waives
the requirement for a bound, which is asserted directly.

install-shared.cjs keeps its hand-rolled assert rather than routing through
throwIfFailed: its message embeds both streams, and throwIfFailed carries
only a trimmed stderr. The message now also names the outcome, so a bounded
timeout reads as such across its 38 importers instead of as
expected null to equal 0.

* test(#3145): extract class-norm timeouts and correct the build-hooks sizing

A pre-PR review found 52 copies of four class-norm timeout constants across
this wave. These are not per-suite fixture bindings -- they are shared facts
about how long a class of subprocess takes, derived from a recorded bench
incident. That norm already moved once (60000 to 120000 after a real
ETIMEDOUT), and 52 copies would have drifted the next time it moved.

Extracts tests/helpers/timeouts.cjs, where each norm is justified once, and
converts the copies. A site that genuinely differs -- a real tsc compile, or
regen:derived -- keeps its own local constant with its own justification.

Also corrects a misclassification: scripts/build-hooks.js was sized as a
build at 120000 in twelve places and 60000 in another, but it compiles and
bundles nothing. Its own header says no bundling needed; it copies pre-built
files and syntax-checks them with vm. Three different values bounded one
script; now there is one.

* test(#3145): fix red CI — lint self-match and a Windows chunk overrun

Two failures on PR 3176.

lint-allow-test-rule-refs read a RuleTester fixture as a real exemption. The
fixture exists to prove an unrelated marker does NOT suppress the rule, so it
carries that marker's literal text as test data. Split via concatenation, the
same idiom no-unbounded-spawn-allowlist.test.cjs already uses for its own
self-match problem. The explanatory comment needed the same treatment.

The Windows shard 3/3 chunk was killed at its 600000ms budget. Output stopped
seven minutes before the kill, so this was an overrun rather than a slow
chunk: regenDerivedPropagatesSingleFragmentEditWithNoSecondSourceSurface runs
regen:derived bounded at 900000ms, which is larger than the whole chunk
budget, so the chunk killer always fires first and it can never complete
there. Both the test and that bound predate this change; modifying the file
pulled it into the Windows targeted set and exposed it. Skipped on Windows
with the reason recorded; the Linux lanes cover it. The 900000 bound and its
ceiling marker are unchanged -- they are correct.

* test(#3145): refresh the stale test-timings cost table

The Windows shard was killed at its 600000ms per-chunk budget. run-tests.cjs
packs chunks by measured duration from tests/test-timings.json, and an
unknown file falls back to the table's median weight -- advisory by design,
but it silently underweights exactly the files that matter.

Four of the failing chunk's 22 files were absent from the table, including
the two heaviest: fragment-single-edit-propagation.install.test.cjs at 230s
(it runs regen:derived) and agent-fragments-emission.install.test.cjs at 79s.
Both were weighted as average, so the chunk's total weight read 53.68 against
a budget of 60 and the packer produced a single chunk.

Regenerated from a passing full-suite run, per the remedy the script itself
documents. 700 to 770 entries, 70 added, 0 dropped -- verified, since
gen-test-timings.cjs replaces the table wholesale rather than merging.

Proven against the real packer: the same 22 files now weigh 103.91 and split
into two chunks. No logic, budget, or timeout was changed; raising a budget
to make a red gate pass is not a fix.

---------

Co-authored-by: sim <sim@local>
2026-08-07 15:18:18 -04:00

220 lines
12 KiB
JavaScript

// allow-test-rule: structural-regression-guard — AC2: assert no `runtime === 'zcode'` string-equality branch, no live `isZcode` read remains in bin/install.js, src/install-engine.cts, src/surface.cts, or src/runtime-artifact-conversion.cts — a source-text property, so source-grep is the faithful check (#2101)
'use strict';
/**
* Declarative reference host — ZCode (#2101 / ADR-1239 EoS).
*
* ZCode already installs through the descriptor-driven artifactLayout
* (nested skills/, flat commands/, flat agents/, each with a named
* `converter`), and its capability.json already declared `hostIntegration`
* axes. Issue #2101 found ZCode was already descriptor-driven except for one
* residual `isZcode` branch in bin/install.js: the shared-hooks-install
* exclusion (`&& !isZcode`), kept hardcoded because zcode's `hostBehaviors`
* block was previously empty. ZCode's golden install tree has ZERO hook
* files (verified), so folding this onto `hostBehaviors.skipSharedHooksInstall`
* is byte-parity — the same fold already done for
* windsurf/copilot/cursor/cline/kilo/trae (#2089/#2090/#2093/#2094/#2099/#2100).
*
* This test is the reference-host dogfood mirroring
* tests/declarative-reference-windsurf.test.cjs: it (1) classifies ZCode's
* profile via profileOf, (2) confirms the public declarative adapter
* classifies it as declarative, (3) round-trips a real install proving a
* gsd surface is emitted, (4) proves negotiation fails CLOSED on a corrupted
* descriptor, (5) proves the validator accepts the descriptor, and (6)
* source-greps the folded modules for the retired `isZcode` branch (AC2).
*
* Both capability upgrades anticipated by the issue (hook automation via
* ZCode's plugin Hook component; native MCP registration) remain BLOCKED —
* ZCode's docs do not publish the on-disk config format/location/schema for
* either surface (see docs/reference/host-integration-capability-matrix.md
* ## zcode for the cited doc URLs and rationale). No upgrade code is added
* here; implementing a guessed format would risk a false-green descriptor.
*/
const { test, before } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { runNode } = require('./helpers/process-seam.cjs');
const { throwIfFailed } = require('./helpers/git-fixture.cjs');
const {
profileOf,
negotiateHostCapabilities,
PROFILE_BASELINES,
UNDOCUMENTED,
} = require('../gsd-core/bin/lib/host-integration.cjs');
const { validateCapability } = require('../gsd-core/bin/lib/capability-validator.cjs');
const { createDeclarativeAdapter } = require('../gsd-core/bin/lib/adapter-declarative.cjs');
const { cleanup } = require('./helpers.cjs');
const { walk, runMinimalInstall, BUILD_SCRIPT } = require('./helpers/install-shared.cjs');
const DESC = path.join(__dirname, '..', 'capabilities', 'zcode', 'capability.json');
const ZCODE_CAP = JSON.parse(fs.readFileSync(DESC, 'utf8'));
const ZCODE_AXES = ZCODE_CAP.runtime.hostIntegration;
// scripts/build-hooks.js copies pre-built hook files into hooks/dist and
// syntax-checks them with vm — it does not compile/bundle anything. See
// tests/helpers/timeouts.cjs for the class-norm justification.
const { BUILD_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
// hooks/dist is gitignored and built (mirrors golden-install-parity harness).
before(() => {
throwIfFailed(
runNode([BUILD_SCRIPT], { timeoutMs: BUILD_TIMEOUT_MS }),
`node ${BUILD_SCRIPT}`,
);
});
test('ZCode classifies as the declarative-cli reference profile (profileOf)', () => {
const desc = JSON.parse(fs.readFileSync(DESC, 'utf8'));
const axes = desc.runtime.hostIntegration;
assert.ok(axes && axes.embeddingMode, 'zcode descriptor declares hostIntegration axes');
assert.equal(profileOf(axes), 'declarative-cli', 'ZCode is a Declarative-CLI host');
});
test('the public declarative adapter classifies ZCode as a declarative host', () => {
const adapter = createDeclarativeAdapter({ runtime: 'zcode' });
assert.equal(adapter.kind, 'declarative');
assert.equal(adapter.runtime, 'zcode');
assert.equal(typeof adapter.install, 'function');
assert.equal(typeof adapter.uninstall, 'function');
});
test('a real ZCode install emits an invocable gsd skill/command/agent surface', () => {
const { configDir, root } = runMinimalInstall({ runtime: 'zcode', scope: 'global' });
try {
const files = walk(configDir);
assert.ok(files.length > 0, 'install must emit artifacts');
const gsdSurface = files.filter((f) => /gsd/i.test(path.relative(configDir, f)));
assert.ok(gsdSurface.length > 0, 'install must emit a gsd surface (declarative reference)');
// ZCode's artifactLayout (capabilities/zcode/capability.json) declares
// nested skills/, flat commands/, and flat agents/ — verify all three.
const skillsDir = path.join(configDir, 'skills');
assert.ok(fs.existsSync(skillsDir), 'skills/ directory must exist');
const skillDirs = fs.readdirSync(skillsDir, { withFileTypes: true })
.filter((e) => e.isDirectory() && e.name.startsWith('gsd-'));
assert.ok(skillDirs.length > 0, 'skills/ must contain nested gsd-* skill directories');
const firstSkillFiles = fs.readdirSync(path.join(skillsDir, skillDirs[0].name));
assert.ok(firstSkillFiles.includes('SKILL.md'), 'each nested skill dir must contain SKILL.md');
const commandsDir = path.join(configDir, 'commands');
assert.ok(fs.existsSync(commandsDir), 'commands/ directory must exist');
const cmdFiles = fs.readdirSync(commandsDir).filter((f) => f.startsWith('gsd-') && f.endsWith('.md'));
assert.ok(cmdFiles.length > 0, 'commands/ must contain flat gsd-*.md slash commands');
const agentsDir = path.join(configDir, 'agents');
assert.ok(fs.existsSync(agentsDir), 'agents/ directory must exist');
const agentFiles = fs.readdirSync(agentsDir).filter((f) => f.startsWith('gsd-') && f.endsWith('.md'));
assert.ok(agentFiles.length > 0, 'agents/ must contain flat gsd-*.md agent files');
// #2101: zcode's shared-hooks exclusion is now descriptor-driven
// (hostBehaviors.skipSharedHooksInstall:true) — golden has zero hook
// files, so no hooks/ directory should be installed.
assert.ok(!fs.existsSync(path.join(configDir, 'hooks')), 'zcode install must not emit a hooks/ directory');
} finally {
cleanup(root);
}
});
// ---------------------------------------------------------------------------
// #2101 EoS/zcode — fail-closed negotiation + validator acceptance +
// the folded descriptor (mirrors codebuddy/windsurf/augment reference tests).
// ---------------------------------------------------------------------------
test('negotiateHostCapabilities never throws for zcode, even fully corrupted', () => {
assert.doesNotThrow(() => negotiateHostCapabilities({}));
assert.doesNotThrow(() => negotiateHostCapabilities({ ...ZCODE_AXES, embeddingMode: UNDOCUMENTED }));
assert.doesNotThrow(() => negotiateHostCapabilities({ ...ZCODE_AXES, embeddingMode: 'future-unknown' }));
assert.doesNotThrow(() => negotiateHostCapabilities({ ...ZCODE_AXES, dispatch: 'corrupted-not-an-object' }));
assert.doesNotThrow(() => negotiateHostCapabilities({ ...ZCODE_AXES, dispatch: { ...ZCODE_AXES.dispatch, maxDepth: 'not-a-number' } }));
});
test('a partial/empty zcode descriptor degrades to the safe floor, not the declarative-cli baseline', () => {
const result = negotiateHostCapabilities({});
assert.equal(result.effective.embeddingMode, 'declarative', 'omitted embeddingMode degrades closed');
assert.equal(result.effective.hookBus, 'none');
assert.notDeepEqual(result.effective, PROFILE_BASELINES['declarative-cli']);
assert.ok(result.warnings.length > 0);
});
// AC-style proof: the 2 still-`undocumented` dispatch sub-axes (nested/
// maxDepth) must degrade to the most-restrictive KNOWN value, never their
// optimistic value. Unlike augment (3 undocumented sub-axes) or antigravity
// (4), zcode documents namedDispatch/background/subagentToolkit/
// backgroundDispatch, leaving only nested + maxDepth undocumented. Real
// values confirmed via:
// node -e "const {negotiateHostCapabilities}=require('./gsd-core/bin/lib/host-integration.cjs');
// const cap=require('./capabilities/zcode/capability.json');
// console.log(negotiateHostCapabilities(cap.runtime.hostIntegration).effective.dispatch)"
// -> { namedDispatch:true, nested:false, maxDepth:0, background:false, subagentToolkit:'full', backgroundDispatch:false }
test("zcode's 2 still-undocumented dispatch sub-axes (nested/maxDepth) degrade to the most-restrictive known value, not their optimistic value", () => {
// Sanity: the descriptor itself still declares these 2 as the undocumented
// sentinel, while namedDispatch/background/subagentToolkit/backgroundDispatch
// are documented.
assert.equal(ZCODE_AXES.dispatch.nested, 'undocumented');
assert.equal(ZCODE_AXES.dispatch.maxDepth, 'undocumented');
assert.equal(ZCODE_AXES.dispatch.namedDispatch, true, 'sanity: namedDispatch is documented, not part of the undocumented set');
assert.equal(ZCODE_AXES.dispatch.background, false, 'sanity: background is documented, not part of the undocumented set');
assert.equal(ZCODE_AXES.dispatch.subagentToolkit, 'full', 'sanity: subagentToolkit is documented, not part of the undocumented set');
assert.equal(ZCODE_AXES.dispatch.backgroundDispatch, false, 'sanity: backgroundDispatch is documented, not part of the undocumented set');
const { effective, warnings } = negotiateHostCapabilities(ZCODE_AXES);
assert.equal(effective.dispatch.nested, false, 'undocumented nested must degrade to false, never true');
assert.equal(effective.dispatch.maxDepth, 0, 'undocumented maxDepth must degrade to 0, never -1/unbounded');
// namedDispatch/background/subagentToolkit/backgroundDispatch are documented
// — they are trusted and survive negotiation unchanged.
assert.equal(effective.dispatch.namedDispatch, true, "documented 'true' namedDispatch is trusted, unlike the undocumented sub-axes");
assert.equal(effective.dispatch.background, false, "documented 'false' background is trusted");
assert.equal(effective.dispatch.subagentToolkit, 'full', "documented 'full' subagentToolkit is trusted, unlike the undocumented sub-axes");
assert.equal(effective.dispatch.backgroundDispatch, false, "documented 'false' backgroundDispatch is trusted");
assert.ok(
warnings.some((w) => w.includes('dispatch.nested') && w.includes('undocumented')),
'a warning must be raised for the undocumented dispatch.nested axis',
);
assert.ok(
warnings.some((w) => w.includes('dispatch.maxDepth')),
'a warning must be raised for the undocumented dispatch.maxDepth axis (reported as missing/non-number)',
);
});
test('capabilities/zcode/capability.json validates — no errors', () => {
const errors = validateCapability(ZCODE_CAP, 'zcode');
assert.deepEqual(errors, [], `validateCapability must return no errors, got: ${JSON.stringify(errors)}`);
});
// -- AC2: the hardcoded branch is retired across all folded modules ---------
test('no `runtime === "zcode"` string-equality branch (nor live `isZcode` read) remains in bin/install.js, src/install-engine.cts, src/surface.cts, or src/runtime-artifact-conversion.cts (AC2)', () => {
const strip = (src) => src
.replace(/\/\*[\s\S]*?\*\//g, '')
.replace(/\/\/[^\r\n]*/g, '')
.replace(/`[^`]*`/g, '');
const repoRoot = path.join(__dirname, '..');
const files = [
path.join(repoRoot, 'bin', 'install.js'),
path.join(repoRoot, 'src', 'install-engine.cts'),
path.join(repoRoot, 'src', 'surface.cts'),
path.join(repoRoot, 'src', 'runtime-artifact-conversion.cts'),
];
for (const file of files) {
const src = fs.readFileSync(file, 'utf8');
const stripped = strip(src);
const eqOffenders = stripped.match(/runtime\s*[!=]==\s*["']zcode["']/g) || [];
assert.deepEqual(eqOffenders, [],
`AC2: no hardcoded runtime==='zcode' branch may remain in ${path.relative(repoRoot, file)}; found: ${eqOffenders.join(', ')}`);
// Excludes legit enumeration sites: --zcode CLI flag parsing, the numbered
// menu map ('16': 'zcode'), the allRuntimes set literal, help/usage text,
// and the `// #2101: isZcode dropped` comment (stripped above) — none of
// those contain the token `isZcode`, so a literal-word match is precise.
const isZcodeHits = stripped.match(/\bisZcode\b/g) || [];
assert.deepEqual(isZcodeHits, [],
`AC2: no live isZcode read may remain in ${path.relative(repoRoot, file)}; found ${isZcodeHits.length} occurrence(s)`);
}
});