* test(#4378): regression tests for collision-free seed ids * fix(#4378): mint collision-free SEED-YYMMDD-xxx ids, not a shared count plant-seed derived the next seed id from 'ls .planning/seeds/SEED-*.md | wc -l'. .planning/seeds/ is shared but each worktree only sees what has merged, so two workstreams planting before either merges computed the same id and git merged both files silently. The id is now the local date plus a 3-char random base36 suffix -- the shape .planning/quick/ already uses -- computed from knowledge one worktree has alone, with a same-day regen guard. deriveSeedIdentity learns the new canonical grammar alongside legacy SEED-NNN (whose parsing never changes), the --enrich parser and the filename-prefix fallback keep the full new-format id, and the docs that state the filename shape move to it. The prefix fallback previously truncated any non-pure-numeric id at 'SEED-<digits>' -- the same one-id-two-answers ambiguity the issue reports, reproduced one level down. * fix(#4378): harden seed id generation per adversarial review - parse-idea: anchor the --enrich extractor to the flag and capture the complete id, uppercase-tolerant; a leftmost 'SEED-[0-9]+' truncated an uppercase or malformed suffix to its date and enriched an arbitrary same-day seed via head -1. Ambiguous and unmatched targets now fail closed instead. - generate-seed-id: tolerate the expected SIGPIPE under pipefail, abort loudly when the suffix cannot be drawn (an empty suffix would collapse every seed's id to the bare date), and run the same-day regen guard as a find existence test (the 'ls <glob>' shape trips the #3409 drift guard and degenerates under a stray nullglob). - deriveSeedIdentity: document the theoretical legacy/new grammar ambiguity (6-digit counter + 3-char base36 slug, no frontmatter). - changeset: state the residual same-day collision bound instead of implying zero. Emitted-Drift-Ack-Growth: plant-seed.md — the counting step became hardened date+random generation with explicit failure modes; growth is the failure handling, not duplicated logic * fix(#4378): address standards and spec review findings - tests: move the allow-test-rule marker to its suppression site (the file-header placement was inert per CONTRIBUTING site-scoping); add width-boundary coverage (5/7-digit dates, 2/4-char suffixes pin the documented branch behavior); add a writer-to-reader parity property that parses the mint widths out of the shipped workflow so the two grammar owners cannot drift; cover uppercase ids end-to-end in the reader. - plant-seed.md: draw/retry restructured as one loop with a loud terminal failure; SEED_SUFX renamed SEED_SUFFIX; regen guard drops the redundant head -1; the ambiguity error no longer advises an impossible 'complete id' for duplicate legacy ids. - commands.cts: refresh the cmdListSeeds comment still describing SEED-NNN as the only canonical form. - changeset: drop the audit claim the spec axis showed to be an overstatement (audit's id display is filename-derived, pre-existing). - remove a stray untracked artifact file swept into the tree. * test(#4378): correct boundary expectations to the module's real branch behavior The first matrix run on the boundary tests caught my hand-trace of the regex branches, not a module defect: the slug regex's alternation backtracks to the legacy branch whenever the canonical branch cannot complete (so the slug is the remainder after the legacy numeric prefix), and the 7-digit case fails the canonical branch at its 7th digit before the dash. Pin the verified values. * docs(#4378): backfill changeset PR number * fix(#4378): audit seed identity uses the canonical grammar Review of this PR found the audit surface publishing a fused filename stem (SEED-081-region for SEED-081-region.md) where list-seeds reports the canonical id -- one id, two answers across surfaces, the same ambiguity class the issue files. scanSeeds now derives identity through the SAME deriveSeedIdentity the list-seeds gate uses (frontmatter id, then filename id-prefix, then stem), and audit-open acknowledge resolves --seed-id by scanning for the derived identity, falling back to the literal stem so callers scripted against pre-canonical output keep working. Roll-in per the fix-inline rule: found during this PR's review, same seed-identity seam. RED probe: pre-fix audit published seed_id SEED-081-region-becomes / slug 081-region-becomes for a legacy seeded file; post-fix SEED-081 / region-becomes, matching list-seeds. * test(#4378): probe timeout uses the class norm after windows-lane timeout The windows conformance shard failed its bounded sh -c probes at the local 5000ms bound (cold sh.exe spawn under shard load) while the identical code passed this PR's two earlier windows waves. The probe now uses PROBE_TIMEOUT_MS from the class-norm module instead of a local override, per the helpers/timeouts.cjs convention. --------- Co-authored-by: sim <sim@local>
396 lines
16 KiB
JavaScript
396 lines
16 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* Integration tests for the `check predicate` subcommand wiring (#2008).
|
|
*
|
|
* These exercise the PRODUCTION stack: the real `buildPredicateDeps()` binding
|
|
* (which wraps shell-command-projection.execTool → bounded `sh -c` spawnSync) and
|
|
* the `parsePredicateFlags` arg parser. The pure evaluator logic is covered by
|
|
* gate-predicate-evaluator.test.cjs; this file proves the wiring holds against
|
|
* real subprocess exit codes and a real timeout kill.
|
|
*
|
|
* Commands run are instant (`true` / `false` / `exit 3`) or tightly bounded
|
|
* (a 100ms timeout killing `sleep 1`), so there is no orphan/leak risk.
|
|
*/
|
|
|
|
const { describe, test, beforeEach, afterEach } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('fs');
|
|
const path = require('path');
|
|
const os = require('os');
|
|
|
|
const { evaluatePredicate } = require('../gsd-core/bin/lib/gate-predicate-evaluator.cjs');
|
|
const { buildPredicateDeps, parsePredicateFlags } = require('../gsd-core/bin/lib/check-command-router.cjs');
|
|
const { runGsdTools, createTempProject, cleanup } = require('./helpers.cjs');
|
|
const { PROBE_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
|
|
|
|
/**
|
|
* A real, bounded `sh -c` subprocess spawned via the production
|
|
* runBoundedShell dependency -- the describe block's own name is "real
|
|
* bounded sh -c subprocess."
|
|
*/
|
|
// #4378 (windows conformance lane): the local 5000ms bound timed out on a
|
|
// cold sh.exe spawn under windows-latest shard load while the identical code
|
|
// passed twice earlier the same day -- the probe now uses the class norm
|
|
// (tests/helpers/timeouts.cjs PROBE_TIMEOUT_MS) instead of a local override.
|
|
const BOUNDED_SHELL_PROBE_TIMEOUT_MS = PROBE_TIMEOUT_MS;
|
|
|
|
/**
|
|
* The same runBoundedShell call as BOUNDED_SHELL_PROBE_TIMEOUT_MS, but
|
|
* deliberately tiny (not generous headroom) to force a `sleep 1` command
|
|
* past the bound within this test's own lifetime, proving "timeout kills
|
|
* the subprocess (SIGTERM => timedOut:true)."
|
|
*/
|
|
const BOUNDED_SHELL_FORCED_TIMEOUT_MS = 100;
|
|
|
|
// ─── buildPredicateDeps: real subprocess exit mapping ─────────────────────────
|
|
|
|
describe('buildPredicateDeps — real bounded sh -c subprocess', () => {
|
|
const deps = buildPredicateDeps();
|
|
const cwd = process.cwd();
|
|
|
|
test('`true` => exitCode 0, not timed out', () => {
|
|
const r = deps.runBoundedShell({ command: 'true', cwd, timeoutMs: BOUNDED_SHELL_PROBE_TIMEOUT_MS });
|
|
assert.equal(r.exitCode, 0);
|
|
assert.equal(r.timedOut, false);
|
|
});
|
|
|
|
test('`false` => exitCode 1, not timed out', () => {
|
|
const r = deps.runBoundedShell({ command: 'false', cwd, timeoutMs: BOUNDED_SHELL_PROBE_TIMEOUT_MS });
|
|
assert.equal(r.exitCode, 1);
|
|
assert.equal(r.timedOut, false);
|
|
});
|
|
|
|
test('`exit 3` => exitCode 3', () => {
|
|
const r = deps.runBoundedShell({ command: 'exit 3', cwd, timeoutMs: BOUNDED_SHELL_PROBE_TIMEOUT_MS });
|
|
assert.equal(r.exitCode, 3);
|
|
});
|
|
|
|
test('stderr is captured from the subprocess', () => {
|
|
const r = deps.runBoundedShell({ command: 'echo oops >&2; exit 4', cwd, timeoutMs: BOUNDED_SHELL_PROBE_TIMEOUT_MS });
|
|
assert.equal(r.exitCode, 4);
|
|
assert.match(r.stderr, /oops/);
|
|
});
|
|
|
|
test('timeout kills the subprocess (SIGTERM => timedOut:true)', () => {
|
|
const r = deps.runBoundedShell({ command: 'sleep 1', cwd, timeoutMs: BOUNDED_SHELL_FORCED_TIMEOUT_MS });
|
|
assert.equal(r.timedOut, true);
|
|
assert.equal(r.signal, 'SIGTERM');
|
|
});
|
|
});
|
|
|
|
// ─── evaluatePredicate + production deps: end-to-end exit mapping ─────────────
|
|
|
|
describe('evaluatePredicate + production deps — command-exit-zero e2e', () => {
|
|
const deps = buildPredicateDeps();
|
|
const ctx = { cwd: process.cwd() };
|
|
|
|
test('command `true` => block:false', () => {
|
|
const res = evaluatePredicate({ kind: 'command-exit-zero', command: 'true' }, ctx, deps);
|
|
assert.equal(res.block, false);
|
|
});
|
|
|
|
test('command `false` => block:true', () => {
|
|
const res = evaluatePredicate({ kind: 'command-exit-zero', command: 'false' }, ctx, deps);
|
|
assert.equal(res.block, true);
|
|
assert.match(res.message, /1/);
|
|
});
|
|
|
|
test('interpolation reaches the real shell ($PHASE_NUMBER via flag context)', () => {
|
|
const res = evaluatePredicate(
|
|
{ kind: 'command-exit-zero', command: 'test "${PHASE_NUMBER}" = "07" && true || false' },
|
|
{ cwd: process.cwd(), phaseNumber: '07' },
|
|
deps,
|
|
);
|
|
assert.equal(res.block, false);
|
|
});
|
|
});
|
|
|
|
// ─── parsePredicateFlags ───────────────────────────────────────────────────────
|
|
|
|
describe('parsePredicateFlags', () => {
|
|
test('extracts --flag value pairs, skips positional + bare --flags', () => {
|
|
const out = parsePredicateFlags(['check', 'predicate', '--predicate', '{"kind":"x"}', '--phase-number', '03', '--raw']);
|
|
assert.deepEqual(out, { predicate: '{"kind":"x"}', 'phase-number': '03' });
|
|
});
|
|
|
|
test('last write wins for repeated flags', () => {
|
|
const out = parsePredicateFlags(['--phase-number', '01', '--phase-number', '02']);
|
|
assert.equal(out['phase-number'], '02');
|
|
});
|
|
|
|
test('value that starts with -- is not consumed (treated as a flag)', () => {
|
|
const out = parsePredicateFlags(['--predicate', '--phase-number']);
|
|
assert.equal('predicate' in out, false);
|
|
});
|
|
|
|
test('empty args => empty map', () => {
|
|
assert.deepEqual(parsePredicateFlags([]), {});
|
|
});
|
|
});
|
|
|
|
// ─── #4130 follow-up: partitionPredicateArgs (flags + positionals, one parser) ─
|
|
|
|
/**
|
|
* `partitionPredicateArgs` is the single pass behind `parsePredicateFlags`:
|
|
* it returns BOTH the --flag value map AND the non-consumed positional tokens
|
|
* under the exact same skip/consume/last-wins semantics. `check
|
|
* decision-coverage-plan --context <path>` uses it so the flag and the
|
|
* positional surface share one parser with `check predicate` — the two
|
|
* parsers cannot diverge because there is only one.
|
|
*/
|
|
describe('partitionPredicateArgs (#4130 follow-up)', () => {
|
|
const { partitionPredicateArgs } = require('../gsd-core/bin/lib/check-command-router.cjs');
|
|
|
|
test('splits --flag value pairs from positionals', () => {
|
|
const { flags, positionals } = partitionPredicateArgs(
|
|
['check', 'decision-coverage-plan', '--context', '/tmp/CONTEXT.md', 'phases/01-init'],
|
|
);
|
|
assert.deepEqual(flags, { context: '/tmp/CONTEXT.md' });
|
|
assert.deepEqual(positionals, ['check', 'decision-coverage-plan', 'phases/01-init']);
|
|
});
|
|
|
|
test('parsePredicateFlags is exactly the flags half (one source of truth)', () => {
|
|
const vectors = [
|
|
['check', 'predicate', '--predicate', '{"kind":"x"}', '--phase-number', '03', '--raw'],
|
|
['--phase-number', '01', '--phase-number', '02'],
|
|
['--predicate', '--phase-number'],
|
|
[],
|
|
['--context'],
|
|
['a', '--context', 'b', '--context', 'c', 'd'],
|
|
];
|
|
for (const v of vectors) {
|
|
assert.deepEqual(partitionPredicateArgs(v).flags, parsePredicateFlags(v),
|
|
`flags half must equal parsePredicateFlags for ${JSON.stringify(v)}`);
|
|
}
|
|
});
|
|
|
|
test('value that starts with -- is not consumed: both stay flags, neither becomes positional', () => {
|
|
const { flags, positionals } = partitionPredicateArgs(['--context', '--other']);
|
|
assert.deepEqual(flags, {});
|
|
assert.deepEqual(positionals, ['--context', '--other']);
|
|
});
|
|
|
|
test('last write wins; flag values never leak into positionals', () => {
|
|
const { flags, positionals } = partitionPredicateArgs(['p1', '--context', 'a', 'p2', '--context', 'b', 'p3']);
|
|
assert.equal(flags.context, 'b');
|
|
assert.deepEqual(positionals, ['p1', 'p2', 'p3']);
|
|
});
|
|
});
|
|
|
|
// ─── #4354: `check predicate --phase-dir` containment boundary ───────────────
|
|
//
|
|
// cmdCheckPredicate passes the `--phase-dir` flag VERBATIM into PredicateContext
|
|
// (src/check-command-router.cts) with no containment validation. Both predicate
|
|
// kinds read/interpolate that value: `artifact-frontmatter-equals` resolves it
|
|
// as `targetDir` for `findPhaseArtifact`, and `command-exit-zero` interpolates
|
|
// it into `${PHASE_DIR}` in the shelled-out command. These tests reproduce the
|
|
// issue's exact repro and prove the boundary is currently unconfined.
|
|
|
|
describe('check predicate --phase-dir — containment boundary (#4354)', () => {
|
|
let projDir;
|
|
let outsideDir;
|
|
|
|
beforeEach(() => {
|
|
projDir = createTempProject();
|
|
fs.mkdirSync(path.join(projDir, '.planning', 'phases', '05-x'), { recursive: true });
|
|
outsideDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-predicate-outside-'));
|
|
});
|
|
|
|
afterEach(() => {
|
|
cleanup(projDir);
|
|
cleanup(outsideDir);
|
|
});
|
|
|
|
test('[RED #4354] the issue\'s exact repro: artifact-frontmatter-equals against a foreign SECURITY.md via an outside --phase-dir must be rejected, not evaluated', () => {
|
|
fs.writeFileSync(
|
|
path.join(outsideDir, 'SECURITY.md'),
|
|
'---\nstatus: passed\n---\n# Security\n',
|
|
);
|
|
|
|
const predicate = JSON.stringify({
|
|
kind: 'artifact-frontmatter-equals',
|
|
artifact: 'SECURITY.md',
|
|
field: 'status',
|
|
equals: 'passed',
|
|
});
|
|
|
|
const result = runGsdTools(
|
|
['--json-errors', 'check', 'predicate', '--predicate', predicate, '--phase-dir', outsideDir, '--raw'],
|
|
projDir,
|
|
);
|
|
|
|
// CURRENT BUG (documented, not asserted as desired): this command today
|
|
// succeeds and prints {"block":false,...} — a BLOCKING gate passing on
|
|
// foreign evidence read from OUTSIDE the project. REQUIRED behavior:
|
|
// the outside --phase-dir must be rejected before evaluation.
|
|
assert.strictEqual(
|
|
result.success,
|
|
false,
|
|
`an outside --phase-dir must be rejected before evaluating the predicate ` +
|
|
`(currently: ${result.success ? `SUCCEEDED with output ${result.output}` : 'failed for an unrelated reason'})`,
|
|
);
|
|
});
|
|
|
|
test('[RED #4354] a command-exit-zero predicate interpolating ${PHASE_DIR} with an outside --phase-dir must also be rejected', () => {
|
|
fs.writeFileSync(path.join(outsideDir, 'marker.txt'), 'outside-marker\n');
|
|
|
|
const predicate = JSON.stringify({
|
|
kind: 'command-exit-zero',
|
|
command: 'test -f "${PHASE_DIR}/marker.txt"',
|
|
});
|
|
|
|
const result = runGsdTools(
|
|
['--json-errors', 'check', 'predicate', '--predicate', predicate, '--phase-dir', outsideDir, '--raw'],
|
|
projDir,
|
|
);
|
|
|
|
assert.strictEqual(
|
|
result.success,
|
|
false,
|
|
`a command-exit-zero predicate interpolating an outside --phase-dir must be rejected ` +
|
|
`(currently: ${result.success ? `SUCCEEDED with output ${result.output}` : 'failed for an unrelated reason'})`,
|
|
);
|
|
});
|
|
|
|
test('[regression] a valid in-project --phase-dir still evaluates', () => {
|
|
const phaseDir = path.join(projDir, '.planning', 'phases', '05-x');
|
|
fs.writeFileSync(
|
|
path.join(phaseDir, 'SECURITY.md'),
|
|
'---\nstatus: passed\n---\n# Security\n',
|
|
);
|
|
const predicate = JSON.stringify({
|
|
kind: 'artifact-frontmatter-equals',
|
|
artifact: 'SECURITY.md',
|
|
field: 'status',
|
|
equals: 'passed',
|
|
});
|
|
|
|
const result = runGsdTools(
|
|
['check', 'predicate', '--predicate', predicate, '--phase-dir', phaseDir, '--raw'],
|
|
projDir,
|
|
);
|
|
assert.ok(result.success, `Command failed: ${result.error}`);
|
|
const parsed = JSON.parse(result.output);
|
|
assert.strictEqual(parsed.block, false, 'in-project phase-dir evaluation must still pass');
|
|
});
|
|
|
|
test('[regression #4652] a relative --phase-dir must resolve against --cwd, not the real process cwd, and must not leak the outside file', () => {
|
|
// The real process cwd (outsideDir) contains a foreign SECURITY.md; the
|
|
// CLI is told --cwd projDir with a relative --phase-dir '.'. Before #4652,
|
|
// cmdCheckPredicate validated the joined (projDir + '.') path but passed
|
|
// the RAW, un-joined '.' into ctx.phaseDir, which findPhaseArtifact then
|
|
// resolved against the real process cwd (outsideDir) — leaking foreign
|
|
// frontmatter. The fix must reject this, and the leaked value must never
|
|
// appear in the output.
|
|
fs.writeFileSync(
|
|
path.join(outsideDir, 'SECURITY.md'),
|
|
'---\nstatus: LEAKED_VALUE\n---\n# Security\n',
|
|
);
|
|
|
|
const predicate = JSON.stringify({
|
|
kind: 'artifact-frontmatter-equals',
|
|
artifact: 'SECURITY.md',
|
|
field: 'status',
|
|
equals: 'NOPE',
|
|
});
|
|
|
|
const result = runGsdTools(
|
|
['--json-errors', 'check', 'predicate', '--cwd', projDir, '--predicate', predicate, '--phase-dir', '.', '--raw'],
|
|
outsideDir,
|
|
);
|
|
|
|
const combinedOutput = `${result.output || ''}${result.error || ''}`;
|
|
assert.ok(
|
|
!combinedOutput.includes('LEAKED_VALUE'),
|
|
`the outside file's frontmatter value must never leak into the output (got: ${combinedOutput})`,
|
|
);
|
|
assert.strictEqual(
|
|
result.success && JSON.parse(result.output).block === false,
|
|
false,
|
|
`a relative --phase-dir must not resolve against the real process cwd and must not pass ` +
|
|
`(currently: ${combinedOutput})`,
|
|
);
|
|
});
|
|
|
|
test('[#4652] a --phase-dir that is a symlink inside the project resolving outside the project is rejected', (t) => {
|
|
fs.writeFileSync(
|
|
path.join(outsideDir, 'SECURITY.md'),
|
|
'---\nstatus: passed\n---\n# Security\n',
|
|
);
|
|
const linkPath = path.join(projDir, '.planning', 'phases', 'linked-out');
|
|
try {
|
|
fs.symlinkSync(outsideDir, linkPath, 'dir');
|
|
} catch (e) {
|
|
if (e.code === 'EPERM') {
|
|
t.skip('symlink creation is not permitted on this platform (EPERM)');
|
|
return;
|
|
}
|
|
throw e;
|
|
}
|
|
|
|
const predicate = JSON.stringify({
|
|
kind: 'artifact-frontmatter-equals',
|
|
artifact: 'SECURITY.md',
|
|
field: 'status',
|
|
equals: 'passed',
|
|
});
|
|
|
|
const result = runGsdTools(
|
|
['--json-errors', 'check', 'predicate', '--predicate', predicate, '--phase-dir', linkPath, '--raw'],
|
|
projDir,
|
|
);
|
|
|
|
assert.strictEqual(
|
|
result.success,
|
|
false,
|
|
`a --phase-dir symlink resolving outside the project must be rejected ` +
|
|
`(currently: ${result.success ? `SUCCEEDED with output ${result.output}` : 'failed for an unrelated reason'})`,
|
|
);
|
|
});
|
|
|
|
test('[#4652] a relative --phase-dir interpolates ${PHASE_DIR} as the resolved ABSOLUTE path, not the relative value', () => {
|
|
const phaseDir = path.join(projDir, '.planning', 'phases', '05-x');
|
|
fs.writeFileSync(path.join(phaseDir, 'marker.txt'), 'marker\n');
|
|
|
|
const predicate = JSON.stringify({
|
|
kind: 'command-exit-zero',
|
|
command: 'echo "${PHASE_DIR}" > "${PHASE_DIR}/interpolated.txt"',
|
|
});
|
|
|
|
const result = runGsdTools(
|
|
['check', 'predicate', '--predicate', predicate, '--phase-dir', '.planning/phases/05-x', '--raw'],
|
|
projDir,
|
|
);
|
|
|
|
assert.ok(result.success, `Command failed: ${result.error}`);
|
|
const interpolated = fs.readFileSync(path.join(phaseDir, 'interpolated.txt'), 'utf-8').trim();
|
|
assert.strictEqual(
|
|
interpolated,
|
|
fs.realpathSync(phaseDir),
|
|
`${'${PHASE_DIR}'} must interpolate the resolved absolute path, not the relative --phase-dir value`,
|
|
);
|
|
});
|
|
|
|
test('[regression] no --phase-dir at all still falls back to cwd and evaluates', () => {
|
|
fs.writeFileSync(
|
|
path.join(projDir, 'SECURITY.md'),
|
|
'---\nstatus: passed\n---\n# Security\n',
|
|
);
|
|
const predicate = JSON.stringify({
|
|
kind: 'artifact-frontmatter-equals',
|
|
artifact: 'SECURITY.md',
|
|
field: 'status',
|
|
equals: 'passed',
|
|
});
|
|
|
|
const result = runGsdTools(
|
|
['check', 'predicate', '--predicate', predicate, '--raw'],
|
|
projDir,
|
|
);
|
|
assert.ok(result.success, `Command failed: ${result.error}`);
|
|
const parsed = JSON.parse(result.output);
|
|
assert.strictEqual(parsed.block, false, 'cwd-fallback evaluation must still pass');
|
|
});
|
|
});
|