Files
msd-core/tests/gen-health-docs.test.cjs
Tom Boucher 5f64d999dc fix(#3586): warn when .planning/ is gitignored but still tracked (#3598)
* feat(#3586): warn when .planning/ is gitignored but still tracked

git ignore rules have no effect on files git already tracks, so a project
that committed .planning/ before ignoring it keeps staging those files --
while commit_docs correctly resolves to false, which is exactly what makes
the contradiction invisible.

The probe lives in the SNAPSHOT BUILDER, not the rule: Rule.check may perform
no ambient I/O (ADR-3180 8.1 rule 1, enforced by lint-planning-snapshot-bypass).
buildPlanningTrackedField follows buildWorktreeHealthField's precedent --
injected execGit, bounded, degrading to UNREADABLE with a typed reason rather
than throwing. W024 went inline instead only because no snapshot field carried
its fact; that precondition does not apply here.

W029 fires only on COMPLETE scope with ignored and tracked both true, so a
degraded probe yields neither a finding nor a false all-clear, and the default
project (tracked, not ignored) stays silent. The remedy is ADVISE-only --
--repair never untracks anything.

* docs(#3586): document W029 and correct the health rule count

CONFIGURATION.md documented the gitignore auto-detect without the caveat that
ignore rules do not affect already-tracked files -- the very gap W029 exists to
surface. Adds the caveat, the warning, its remedy, and why --repair will not
act on it.

CONTEXT.md's rule count was stale at 31 before this change (actual 32 through
W028); corrected to 33 and pointed at the two other places the count is locked,
so the next editor updates all three together.

* fix(#3586): treat ls-files overflow as tracked, add CLI-level W029 tests

Review findings.

Security (minor, confirmed): execGit sets no maxBuffer, so Node's 1MB default
applies to git ls-files. A .planning/ tree large enough to overflow it failed
into git_list_failed and silenced W029 -- a false negative in exactly the
large-history case most likely to have the real bug. Overflow is now treated
as PROOF of tracking (the output was non-empty by definition) and resolves to
tracked:true, scope COMPLETE, reason ok_truncated.

Spec (major): test-matrix rows C1 and C2 were never implemented -- there was no
CLI-level integration test at all, only rule-level ones. Both now drive the real
validate-health dispatch and confirm W029 is reachable end-to-end.

Known limit documented, not papered over: a deliberate git add -f under an
otherwise-ignored .planning/ raises the same signal as the accidental case.
There is no reliable way to tell them apart, the finding is advisory-only, and
a heuristic that cannot actually distinguish them would be worse than the
honest caveat.

* test(#3586): update frozen health-doc counts and acknowledge health.md growth

The remote matrix caught three gates that lint:ci does not cover.

gen-health-docs.test.cjs froze a 35-row / 32-rule assertion; W029 makes it
36/33. Updated both the assertion and the test NAME, which embeds the counts --
a stale name is a lie even when the assertion passes. The second reported
failure was the same assertion surfacing at describe-rollup granularity, not a
distinct bug.

emitted-attribution's growth arm needed an ack for the generated health.md.
health.md was already named in 3309-health-docs-generated.json, and two ack
sources naming one path is a hard error -- so a new fragment was not an option.
That fragment's own history shows the pattern: #3309 created it, #2873 amended
it in place for W028. Amended again for W029, with a note recording why this
one file is amended rather than joined by a sibling.

* docs(#3586): add the private-planning how-to and fix a wrong link

docs/CONFIGURATION.md pointed 'Configure private planning' at
how-to/configure-model-profiles.md -- an unrelated page -- and no
private-planning how-to existed at all. Found while editing that section.

The how-to test genuinely fires here: going private is four steps and crosses
planning.search_gitignored, a setting owned by another concern, so a reference
table structurally cannot carry it. The new page walks the whole sequence and
leads with the step people miss -- .gitignore does not untrack what git already
tracks -- which is the exact state W029 now detects.

Also corrects 'artefacts' to 'artifacts' (repo house style is American).

* chore(#3586): backfill changeset pr number to 3598

---------

Co-authored-by: sim <sim@local>
2026-08-17 15:56:46 -04:00

259 lines
11 KiB
JavaScript

'use strict';
/**
* gen-health-docs.cjs regression tests (#3309, "health.md's tables are
* generated rather than hand-maintained, closing the 16-vs-30+ documentation
* gap structurally").
*
* Every CLI-level test spawns the real generator (execFileSync) against a
* temp copy of the shipped `gsd-core/workflows/health.md`, using the
* generator's `--target <path>` override — never mutates the real committed
* file. No fs monkeypatching is needed for these cases.
*/
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { execFileSync } = require('node:child_process');
const { createTempDir, cleanup } = require('./helpers.cjs');
const {
buildErrorCodeRows,
renderErrorCodesRegion,
renderRepairActionsRegion,
regenerateHealthMd,
spliceRegion,
compareCodes,
PRECHECK_CODES,
REMEDY_ACTION_ORDER,
ERROR_CODES_START,
ERROR_CODES_END,
} = require('../scripts/gen-health-docs.cjs');
const ROOT = path.resolve(__dirname, '..');
const SCRIPT = path.join(ROOT, 'scripts', 'gen-health-docs.cjs');
const SHIPPED_HEALTH_MD = path.join(ROOT, 'gsd-core', 'workflows', 'health.md');
const COMPILED_MODULE_PATH = path.join(ROOT, 'gsd-core', 'bin', 'lib', 'health-diagnostic.cjs');
function loadRealRules() {
// Real compiled RULES — build:lib is a pretest dependency for the whole
// suite (package.json `pretest`), so this is always present by the time
// node:test runs these files.
return require(COMPILED_MODULE_PATH).RULES;
}
/**
* @param {string[]} args
* @param {string} cwd
* @returns {{code: number, stdout: string, stderr: string}}
*/
function runGenHealthDocs(args, cwd = ROOT) {
try {
const stdout = execFileSync(process.execPath, [SCRIPT, ...args], {
cwd,
encoding: 'utf8',
stdio: ['pipe', 'pipe', 'pipe'],
timeout: 30000,
});
return { code: 0, stdout, stderr: '' };
} catch (err) {
return {
code: err.status ?? 1,
stdout: err.stdout ? err.stdout.toString() : '',
stderr: err.stderr ? err.stderr.toString() : '',
};
}
}
function copyShippedHealthMd(destDir) {
const dest = path.join(destDir, 'health.md');
fs.copyFileSync(SHIPPED_HEALTH_MD, dest);
return dest;
}
// ─── CLI: --check / --write round trip ─────────────────────────────────────
describe('gen-health-docs.cjs --check / --write (CLI, --target fixture)', () => {
test('--check passes on a freshly-written file', (t) => {
const tmpRoot = createTempDir('gen-health-docs-');
t.after(() => cleanup(tmpRoot));
const target = copyShippedHealthMd(tmpRoot);
const w = runGenHealthDocs(['--write', '--target', target]);
assert.equal(w.code, 0, `stderr: ${w.stderr}`);
const c = runGenHealthDocs(['--check', '--target', target]);
assert.equal(c.code, 0, `--check must be clean immediately after --write; stderr: ${c.stderr}`);
assert.match(c.stdout, /up to date/);
});
test('--check fails when the tagged region is stale (mutate a temp copy)', (t) => {
const tmpRoot = createTempDir('gen-health-docs-');
t.after(() => cleanup(tmpRoot));
const target = copyShippedHealthMd(tmpRoot);
// Mutate the committed, already-up-to-date table so it drifts from what
// the generator would produce — a single row edit is enough.
let content = fs.readFileSync(target, 'utf8');
assert.ok(content.includes('| E001 | error |'), 'sanity: shipped health.md must carry the E001 row');
content = content.replace('| E001 | error |', '| E001 | error-STALE-MUTATION |');
fs.writeFileSync(target, content, 'utf8');
const c = runGenHealthDocs(['--check', '--target', target]);
assert.equal(c.code, 1, 'a hand-mutated table must fail --check');
assert.match(c.stderr, /is stale/);
assert.match(c.stderr, /gen-health-docs\.cjs --write/);
});
test('--write on a stale copy regenerates it back to a clean --check', (t) => {
const tmpRoot = createTempDir('gen-health-docs-');
t.after(() => cleanup(tmpRoot));
const target = copyShippedHealthMd(tmpRoot);
let content = fs.readFileSync(target, 'utf8');
content = content.replace('| W010 |', '| W010-DRIFTED |');
fs.writeFileSync(target, content, 'utf8');
const failedCheck = runGenHealthDocs(['--check', '--target', target]);
assert.equal(failedCheck.code, 1, 'sanity: the mutated copy must fail --check first');
const w = runGenHealthDocs(['--write', '--target', target]);
assert.equal(w.code, 0, `stderr: ${w.stderr}`);
const c = runGenHealthDocs(['--check', '--target', target]);
assert.equal(c.code, 0, `stderr: ${c.stderr}`);
});
test('plain invocation (no flag) prints both tables to stdout and exits 0', () => {
const r = runGenHealthDocs([]);
assert.equal(r.code, 0, `stderr: ${r.stderr}`);
assert.match(r.stdout, /\| Code \| Severity \| Description \| Repairable \|/);
assert.match(r.stdout, /\| Action \| Effect \| Risk \|/);
});
test('an unrecognized flag exits 1 rather than silently falling through', () => {
const r = runGenHealthDocs(['--bogus']);
assert.equal(r.code, 1);
assert.match(r.stderr, /unknown flag/);
});
test('the shipped gsd-core/workflows/health.md already passes --check against the real repo', () => {
const r = runGenHealthDocs(['--check']);
assert.equal(r.code, 0, `the committed health.md must already be up to date; stderr: ${r.stderr}`);
});
});
// ─── Row content: representative codes, including previously-undocumented ─
describe('gen-health-docs.cjs row content (representative codes)', () => {
const rules = loadRealRules();
test('produces a 36-row <error_codes> table: 33 rules + 3 pre-checks (E001, E010, I010)', () => {
const rows = buildErrorCodeRows(rules);
assert.equal(rows.length, 36);
const codes = rows.map((r) => r.code);
for (const precheck of PRECHECK_CODES) {
assert.ok(codes.includes(precheck.code), `missing pre-check code ${precheck.code}`);
}
});
test('W010 (previously-undocumented, agent-install) renders with its Rule-sourced description and Repairable=No', () => {
const region = renderErrorCodesRegion(rules);
const row = region.split('\n').find((line) => line.startsWith('| W010 |'));
assert.ok(row, 'W010 row must be present');
const w010Rule = rules.find((r) => r.code === 'W010');
assert.ok(row.includes(w010Rule.description));
assert.match(row, /\| No \|$/);
});
test('W026 (previously-undocumented, new post-migration split code) renders with its Rule-sourced description', () => {
const region = renderErrorCodesRegion(rules);
const row = region.split('\n').find((line) => line.startsWith('| W026 |'));
assert.ok(row, 'W026 row must be present');
const w026Rule = rules.find((r) => r.code === 'W026');
assert.ok(row.includes(w026Rule.description));
});
test('E004 (already-documented, DESTRUCTIVE-risk remedy) renders with Repairable=No — --repair refuses to auto-apply regenerateState', () => {
const region = renderErrorCodesRegion(rules);
const row = region.split('\n').find((line) => line.startsWith('| E004 |'));
assert.ok(row);
assert.match(row, /\| No \|$/);
});
test('W018 renders the --backfill-qualified Repairable override, not a bare "Yes"', () => {
const region = renderErrorCodesRegion(rules);
const row = region.split('\n').find((line) => line.startsWith('| W018 |'));
assert.ok(row);
assert.match(row, /Yes \(`--backfill`\)/);
});
test('W025 (workflow-layer diagnostic, not a Rule) is absent from the generated table', () => {
const region = renderErrorCodesRegion(rules);
assert.ok(
!region.split('\n').some((line) => line.startsWith('| W025 |')),
'W025 must not appear as a generated row — it is documented in its own workflow step, not the RULES table',
);
});
test('<error_codes> rows are sorted E-codes, then W-codes numerically, then I-codes', () => {
const rows = buildErrorCodeRows(rules);
const sorted = [...rows].sort(compareCodes);
assert.deepEqual(rows, sorted, 'buildErrorCodeRows must already return its rows in sorted order');
// Spot-check the three-group boundary explicitly.
const codes = rows.map((r) => r.code);
const lastE = codes.lastIndexOf(codes.filter((c) => c.startsWith('E')).at(-1));
const firstW = codes.findIndex((c) => c.startsWith('W'));
const lastW = codes.lastIndexOf(codes.filter((c) => c.startsWith('W')).at(-1));
const firstI = codes.findIndex((c) => c.startsWith('I'));
assert.ok(lastE < firstW, 'every E-code must sort before every W-code');
assert.ok(lastW < firstI, 'every W-code must sort before every I-code');
});
test('renderRepairActionsRegion lists all 6 real repair actions, including the previously-undocumented addAiIntegrationPhaseKey', () => {
const region = renderRepairActionsRegion();
for (const action of REMEDY_ACTION_ORDER) {
assert.ok(region.includes(`| ${action} |`), `missing repair action row: ${action}`);
}
assert.equal(REMEDY_ACTION_ORDER.length, 6);
assert.ok(region.includes('addAiIntegrationPhaseKey'), '#3309: this action was "live in code, missing from docs"');
});
});
// ─── spliceRegion / regenerateHealthMd — pure-function edge cases ─────────
describe('gen-health-docs.cjs spliceRegion (pure function)', () => {
test('throws when a tag is missing', () => {
assert.throws(
() => spliceRegion('no tags here', ERROR_CODES_START, ERROR_CODES_END, 'x'),
/missing the .*tags/,
);
});
test('throws when a tag appears more than once', () => {
const text = `${ERROR_CODES_START}a${ERROR_CODES_END}${ERROR_CODES_START}b${ERROR_CODES_END}`;
assert.throws(() => spliceRegion(text, ERROR_CODES_START, ERROR_CODES_END, 'x'), /more than one/);
});
test('preserves content strictly outside the tags, byte-for-byte', () => {
const before = 'PROSE BEFORE\n';
const after = '\nPROSE AFTER';
const text = `${before}${ERROR_CODES_START}old inner${ERROR_CODES_END}${after}`;
const out = spliceRegion(text, ERROR_CODES_START, ERROR_CODES_END, 'new inner');
assert.ok(out.startsWith(before + ERROR_CODES_START));
assert.ok(out.endsWith(ERROR_CODES_END + after));
assert.ok(!out.includes('old inner'));
assert.ok(out.includes('new inner'));
});
test('regenerateHealthMd is idempotent: regenerating an already-generated document is a no-op', () => {
const rules = loadRealRules();
const shipped = fs.readFileSync(SHIPPED_HEALTH_MD, 'utf8');
const regenerated = regenerateHealthMd(rules, shipped);
assert.equal(regenerated, shipped);
});
});