Files
msd-core/tests/check-glossary-refs.test.cjs
Tom Boucher f2c077df38 chore(#2387): refactor CONTEXT.md legacy content + add glossary drift gate (#2391)
* chore(#2387): refactor CONTEXT.md legacy content + add glossary drift gate

Apply the audit-and-enforce concept from the ADR index (#2356) to CONTEXT.md:
correct stale facts, and add a CI gate so the machine-verifiable claims can't
silently re-rot.

CONTEXT.md was entirely hand-maintained with nothing checking its claims against
the shipped tree, so it had rotted. An audit against live code (Memtrace +
filesystem + gh), each finding adversarially re-verified, drove 38 factual
corrections + 1 surfaced by the new gate:

- Dead references: Package Identity named @opengsd/get-shit-done-redux (package
  is @opengsd/gsd-core); Shell Command Projection named run-git/run-npm/run-tool
  (real exports execGit/execNpm/execTool); a partial docs/adr/1606 ref; retired
  sdk/ framing.
- Superseded facts: allRuntimes 15 -> 17 (pi #2102, zcode); "seven nested-loader
  runtimes" -> five (claude reverted flat #924, antigravity flat); stacked-PR
  examples rebasing onto main -> next; QUOTA_SENTINELS precedence corrected to
  match src/agent-command-router.cts.
- Drifted CONTRIBUTING.md line citations refreshed.

Per CONTRIBUTING.md:179, only stale FACTS were corrected -- no maintainer intent,
lesson, or opinion was rewritten, and the append-only session log is untouched
except one dated in-place superseding note. The three tests that assert on
CONTEXT.md content (phase6-capstone-conformance, tracer-bullet,
external-job-waiting) keep all their anchors.

New scripts/check-glossary-refs.cjs (--check, wired into lint:generated-sync):
- Check A: every backticked file reference under a TRACKED_PREFIXES allowlist
  resolves on disk. Generated gsd-core/bin/lib/*.cjs (77 refs, gitignored),
  ~/-paths, .planning/, and bare filenames are deliberately skipped so a clean
  CI checkout never false-fails.
- Check B: the allRuntimes count + member set in the glossary prose match
  bin/install.js's allRuntimes literal (drifts on every runtime addition).
tests/check-glossary-refs.test.cjs covers both, including the false-positive
guard that a missing bin/lib/*.cjs ref does NOT trip the gate.

Closes #2387

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#2387): confine glossary-gate file refs to ROOT (no `..` traversal)

Pre-PR security review finding (low): extractTrackedRefs fed tokens straight to
fs.existsSync(path.join(ROOT, token)), and PATH_TOKEN_RE admits `.` in a segment,
so a CONTEXT.md token like `src/../../../etc/passwd` passed the `src/` prefix
check and normalized to an out-of-tree absolute path — turning the doc lint into
a filesystem-existence oracle on the CI host (existsSync only; CONTEXT.md is a
trusted committed file, hence low severity, but a defense-in-depth gap).

Add isWithinRoot() confinement in extractTrackedRefs: a token is dropped unless
path.resolve(ROOT, token) stays within ROOT. A CONTEXT.md reference is always a
plain in-repo path, so a `..` escape is never legitimate. Regression test asserts
a `..`-bearing token is skipped and never named in output.

Refs #2387

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(#2387): drop legacy `get-shit-done` name from a CONTEXT.md defect entry

CI lint-legacy-dir-name failed: the line-928 upstream-issue re-point I applied
wrote the historical provenance as "gsd-build/get-shit-done#3545", and
scripts/lint-legacy-dir-name.cjs forbids the legacy `get-shit-done` name. Reword
to "moved from #3545 in the predecessor repo" — same provenance, no legacy name.

Caught by `npm run lint:ci` (the CI lint chain), which I had not run locally —
lint:generated-sync + eslint do not include lint-legacy-dir-name.

Refs #2387

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 19:24:28 -04:00

194 lines
7.2 KiB
JavaScript

'use strict';
/**
* Behavioral tests for scripts/check-glossary-refs.cjs — the CONTEXT.md
* glossary drift gate (#2387).
*
* These drive the real CLI as a subprocess against synthetic CONTEXT.md /
* bin/install.js fixtures in a temp dir, asserting on exit code and emitted
* text. No source-grepping: the runtime behavior is the contract.
*/
const { test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { spawnSync } = require('node:child_process');
const { createTempDir, cleanup } = require('./helpers.cjs');
const REPO_ROOT = path.resolve(__dirname, '..');
const SCRIPT_REL = path.join('scripts', 'check-glossary-refs.cjs');
// The real bin/install.js allRuntimes array, mirrored here so fixtures can
// build both a matching and a deliberately-drifted CONTEXT.md against it.
const REAL_RUNTIMES = [
'claude', 'antigravity', 'augment', 'cline', 'codebuddy', 'codex', 'copilot',
'cursor', 'hermes', 'kimi', 'kilo', 'opencode', 'pi', 'qwen', 'trae', 'windsurf', 'zcode',
];
function allRuntimesSentence(count, members) {
return `Runtime enum: \`allRuntimes\` (${count} values: ${members.join(', ')})`;
}
/**
* Build a throwaway repo containing exactly what the gate reads: CONTEXT.md,
* bin/install.js, a real src/ file a clean fixture can legitimately reference,
* and a copy of the gate + its cli-exit dependency. A unique mkdtemp per call
* keeps parallel tests from colliding, and the dir is removed via `t.after()`
* so a failing assertion cannot leak it.
*/
function makeRepo(t, { contextBody, runtimes = REAL_RUNTIMES }) {
const root = createTempDir('gsd-glossary-refs-');
t.after(() => cleanup(root));
fs.mkdirSync(path.join(root, 'scripts', 'lib'), { recursive: true });
fs.mkdirSync(path.join(root, 'src'), { recursive: true });
fs.mkdirSync(path.join(root, 'bin'), { recursive: true });
fs.copyFileSync(path.join(REPO_ROOT, SCRIPT_REL), path.join(root, SCRIPT_REL));
fs.copyFileSync(
path.join(REPO_ROOT, 'scripts', 'lib', 'cli-exit.cjs'),
path.join(root, 'scripts', 'lib', 'cli-exit.cjs'),
);
fs.writeFileSync(path.join(root, 'CONTEXT.md'), contextBody);
fs.writeFileSync(
path.join(root, 'bin', 'install.js'),
`'use strict';\nconst allRuntimes = [${runtimes.map((r) => `'${r}'`).join(', ')}];\nmodule.exports = { allRuntimes };\n`,
);
// A real file a clean CONTEXT.md fixture can legitimately reference.
fs.writeFileSync(path.join(root, 'src', 'real-module.cts'), '// fixture\n');
return root;
}
/** Run the gate in `root`; never throws — returns {status, stdout, stderr}. */
function run(root, args = []) {
const res = spawnSync(process.execPath, [path.join(root, SCRIPT_REL), ...args], {
cwd: root,
encoding: 'utf8',
timeout: 30_000,
});
if (res.error) throw res.error;
return { status: res.status, stdout: res.stdout || '', stderr: res.stderr || '' };
}
test('a clean CONTEXT.md whose refs resolve and whose allRuntimes matches passes --check', (t) => {
const context = [
'# Context',
'',
'See `src/real-module.cts` for details.',
'',
`${allRuntimesSentence(17, REAL_RUNTIMES)}.`,
'',
].join('\n');
const root = makeRepo(t, { contextBody: context });
const res = run(root, ['--check']);
assert.equal(res.status, 0, `expected a clean pass: ${res.stderr}`);
assert.match(res.stdout, /glossary references are current/);
});
test('a reference to a nonexistent tracked file fails --check and names the token', (t) => {
const context = [
'# Context',
'',
'See `src/does-not-exist.cts` for details.',
'',
`${allRuntimesSentence(17, REAL_RUNTIMES)}.`,
'',
].join('\n');
const root = makeRepo(t, { contextBody: context });
const res = run(root, ['--check']);
assert.equal(res.status, 1);
assert.match(res.stderr, /src\/does-not-exist\.cts/);
});
test('allRuntimes count and membership drift is caught', (t) => {
// Mirrors the real-world case: CONTEXT.md says 15 while bin/install.js has 17.
const claimed15 = REAL_RUNTIMES.filter((r) => r !== 'pi' && r !== 'zcode');
assert.equal(claimed15.length, 15);
const context = [
'# Context',
'',
'See `src/real-module.cts` for details.',
'',
`${allRuntimesSentence(15, claimed15)}.`,
'',
].join('\n');
const root = makeRepo(t, { contextBody: context }); // bin/install.js defaults to the real 17
const res = run(root, ['--check']);
assert.equal(res.status, 1);
assert.match(res.stderr, /claims 15 values/);
assert.match(res.stderr, /has 17/);
assert.match(res.stderr, /pi/);
assert.match(res.stderr, /zcode/);
});
test('a reference to a nonexistent gsd-core/bin/lib/*.cjs path is skipped (generated, gitignored)', (t) => {
const context = [
'# Context',
'',
'Generated router lives at `gsd-core/bin/lib/does-not-exist.cjs`.',
'',
`${allRuntimesSentence(17, REAL_RUNTIMES)}.`,
'',
].join('\n');
const root = makeRepo(t, { contextBody: context });
const res = run(root, ['--check']);
assert.equal(res.status, 0, `generated bin/lib path must be skipped, not asserted missing: ${res.stderr}`);
});
test('a ~/-rooted path and a bare filename are both skipped', (t) => {
const context = [
'# Context',
'',
'See `~/.claude/x.md` and `core.cjs` for details.',
'',
`${allRuntimesSentence(17, REAL_RUNTIMES)}.`,
'',
].join('\n');
const root = makeRepo(t, { contextBody: context });
const res = run(root, ['--check']);
assert.equal(res.status, 0, `home path and bare filename must be skipped, not asserted missing: ${res.stderr}`);
});
test('a `..`-traversal token cannot escape ROOT into a filesystem-existence probe', (t) => {
// Security review finding: `src/../../../etc/passwd` passes PATH_TOKEN_RE (`.`
// is a legal segment char) and the `src/` prefix, so without a confinement
// check `path.join(ROOT, token)` normalizes out of the tree and existsSync
// probes it — a doc lint turned filesystem oracle. It must be skipped, so the
// gate neither errors nor reports a finding about an out-of-tree path.
const context = [
'# Context',
'',
'Escape attempt: `src/../../../../../../etc/passwd` and `src/../../etc/hosts`.',
'',
`${allRuntimesSentence(17, REAL_RUNTIMES)}.`,
'',
].join('\n');
const root = makeRepo(t, { contextBody: context });
const res = run(root, ['--check']);
assert.equal(res.status, 0, `..-traversal tokens must be confined to ROOT and skipped: ${res.stderr}`);
assert.doesNotMatch(res.stderr, /etc\/passwd|etc\/hosts/, 'a confined gate must not name out-of-tree paths');
});
test('the real script runs cleanly against the real repo without crashing', () => {
// Does NOT assert the exit code — CONTEXT.md may still be mid-edit — only
// that the gate itself runs to a real verdict (0 or 1), not an unhandled
// crash (no status / non-2-shaped exit).
const res = spawnSync(process.execPath, [path.join(REPO_ROOT, SCRIPT_REL), '--check'], {
cwd: REPO_ROOT,
encoding: 'utf8',
timeout: 30_000,
});
assert.equal(res.error, undefined, `spawn must not error: ${res.error}`);
assert.ok(res.status === 0 || res.status === 1, `expected exit 0 or 1, got ${res.status} (stderr: ${res.stderr})`);
});