golem15/msd-core
4
0
Fork 0
Code Issues 1 Pull Requests Actions 2 Packages Projects Releases Wiki Activity

fix: recover silently-excluded test dirs + test-architecture audit hardening (#1195)

Browse Source 
* fix: recurse test discovery so subdir test suites actually run

scripts/run-tests.cjs discovered tests with a flat readdirSync(testDir),
silently excluding tests/observability/ (4 files), tests/dispatch/ (1) and
tests/installer-migrations/ (1) — 94 passing tests — from `npm test` and all
CI lanes. Walk the tree recursively (relative subpaths preserved), classify
suites by basename, and add a fail-on-zero-executed guard for suite/default
runs (escape hatch GSD_ALLOW_EMPTY_SUITE=1) while preserving the empty
--files/--files-from path the CI inert lane relies on.

Unit suite 735 -> 741 files; surfaces ADR-227's observability/dispatch seam.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: retire 5 verified-worthless tests

Adversarial verification confirmed these 5 prove nothing — their coverage is
provided more strictly elsewhere:
- enh-2790 'has a name: field' spot-checks (command-contract enforces /^gsd[:-]/)
- command-routing-hub duplicate construct + duplicate ERROR_KINDS assertions
- no-cjs-sdk-handsync-tooling (guarded files that never existed on main; bug-190
  covers the real retired SDK artifacts)
- runtime-artifact-layout cline edge case (subsumed by the explicit-global test
  and bug-782-cline-skills-emission)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: add ADR-218 release version-validation coverage

ADR-218 (reject leading-zero versions like 1.01.0; npm duplicate pre-check) had
zero tests — the logic lived only in release.yml bash. Add a test that extracts
the actual rejection regexes from the workflow and exercises them against a
boundary table (leading-zero/malformed rejected, valid accepted) plus structural
wiring assertions. Goes red if the regex is reverted to [0-9]+.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: redesign weak tests into behavioral, deterministic assertions

Per the ADR test audit, rewrite 27 weak test files (test-only, no source
changes) so each can go red for the defect it guards:
- kill pass-always assert.ok(true) placeholders (research-cli, worktree-baseref,
  bug-260 security guard, eslint-rules x24, clusters '|| true')
- replace source-text grep with behavioral calls (install Kilo, sh-hook-paths,
  plan-review-convergence) and add a repo-layout governance test
- de-flake real-clock/Math.random coupling (phase last_updated, bug-3707 mtime,
  context-utilization property, feat-3594)
- fix independence/shared-state violations (bug-492 singleton, issue-844 tmpRoot,
  core reapStaleTempFiles, active-workstream TTY, feat-488 GSD_HOME)
- strengthen property/shape-only tests (research-provider/store classification +
  collision) and unconditional plugin.json schema validation (issue-766)

Verified: all 28 files run together 1220 pass / 0 fail / 1 skip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: add no-tautological-assert lint rule, error in test suite

New custom ESLint rule (eslint-rules/no-tautological-assert.cjs) bans asserts
that can never fail: assert(true)/assert.ok(<always-truthy literal>),
'cond || true' inside an assert, and equality asserts comparing two identical
literals. Wired as error on tests/**; full sweep confirmed zero existing
violations so the suite stays green. Prevents the placeholder-assert regressions
the audit redesigns just removed. RuleTester coverage added (6 valid, 8 invalid).

Note: no-only-tests was already enforced via eslint-plugin-no-only-tests, so no
duplicate rule was added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: gate new allow-test-rule exemptions to require an issue ref

ADR-456 requires any allow-test-rule exemption added after the ADR to carry a
tracking issue number, but nothing enforced it. New ratchet gate
(scripts/lint-allow-test-rule-refs.cjs, wired into lint:ci) fails when a NEW
allow-test-rule comment lacks a #NNN/URL reference; the 323 existing untracked
exemptions are grandfathered in an allowlist that ratchets down as they gain
refs. Red-green verified (novel untracked offender fails; compliant passes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs: add ADR test-audit evidence report (#1192)

Full risk-first qa-test-architect audit of the ADR portfolio (37 ADRs + 4
platform lenses, adversarial verification of retire verdicts) that drove the
P0 discovery fix, ADR-218 coverage, 5 retires, 27 redesigns, and the two new
lint gates. Filed as point-in-time evidence under docs/issueevidence/, named
for tracking issue #1192.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: replace pre-existing raw NUL byte with escape in feat-3594 fixture

feat-3594's null-byte parser fixture contained a literal NUL byte (pre-existing
on next at b10e5681 — confirmed: base blob has 1 NUL, this fix has 0), which
made git treat the file as binary and would break grep/editors. Switch to the
\x00 escape; the runtime string value (a real NUL in the parser input) is
unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: address adversarial-review findings

Codex adversarial pass over the branch:
- capability-registry drift test no longer mutates the committed generated
  capability-registry.cjs in place (concurrency hazard) — uses in-memory
  checkPipeline comparison instead.
- allow-test-rule ratchet now detects exemptions in ALL comment forms (block
  /* */ too, matching no-source-grep) so a block comment can't bypass it;
  one newly-surfaced pre-existing offender grandfathered (323->324).
- install.test Kilo case asserts on what install(false,'kilo') actually writes
  rather than manually calling configureKiloPermissions (masked the call site).
- issue-766 drops the undeclared transitive ajv dep for explicit structural
  assertions from the schema fixture.
- adr-218 test notes the hotfix leading-zero gap is tracked in #1186.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: address code-review findings (subdir discovery, rule + test gaps)

xhigh code review surfaced 15 confirmed issues, all fixed:
- run-tests.cjs --files now resolves subdir tests by bare basename + handles
  Windows backslash paths (ambiguous basenames error clearly).
- affected-tests-lib.cjs listTestFiles made recursive — the targeted CI lane was
  silently dropping changed subdir tests (same false-green class the audit fixed).
- no-tautological-assert now catches 'true || cond' and empty []/{}  equality.
- verify-test-quality: restore provenance-classification coverage, tighten the
  writeFile circular-detection check, guard the module-level file read.
- sh-hook-paths: cover the global-install .sh delegation branch (#2045 guard).
- active-workstream null-guard runs deterministically (no longer skipped on TTY).
- adr-218 structural guards tightened (major/minor leading-zero; needs: membership).
- repo-layout AGENTS.md guard no longer false-alarms on equivalent refactors.
- cross-ai ordering guard fails red when the step is missing.
- issue-766 parses required fields from the schema fixture (auto-enforced).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: stub USERPROFILE alongside HOME in feat-488 (Windows parity)

The feat-488 redesign stubbed process.env.HOME but not USERPROFILE; os.homedir()
resolves from USERPROFILE on Windows, so the home stub was not hermetic there —
caught by windows-test-parity-guard (stubsHomeNoUserProfile). Save/set/restore
USERPROFILE symmetrically with HOME (delete-if-originally-undefined).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: reconcile allow-test-rule allowlist after rebase onto next

Rebasing onto current next pulled in merged PR #1170, which added
inventory-headings-countfree.test.cjs (a baseline allow-test-rule exemption) and
deleted inventory-counts.test.cjs. Grandfather the former and prune the latter so
the ratchet matches the merged tree. No new debt from this PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-06-13 23:35:08 -04:00
committed by GitHub
parent 3ebd13c45d
commit 5fa4dcd78c
44 changed files with 3958 additions and 757 deletions
Download Patch File Download Diff File Expand all files Collapse all files

371
tests/eslint-rules.test.cjs
View File

@@ -18,6 +18,7 @@ const noSourceGrep = require('../eslint-rules/no-source-grep.cjs');
const noMagicSleepInTests = require('../eslint-rules/no-magic-sleep-in-tests.cjs');
const noElapsedAssertion = require('../eslint-rules/no-elapsed-assertion.cjs');
const noRawRmsyncInTests = require('../eslint-rules/no-raw-rmsync-in-tests.cjs');
const noTautologicalAssert = require('../eslint-rules/no-tautological-assert.cjs');
const ruleTester = new RuleTester({
languageOptions: {
@@ -29,6 +30,10 @@ const ruleTester = new RuleTester({
// ─── no-source-grep ──────────────────────────────────────────────────────────
describe('no-source-grep rule', () => {
test('rule module exports a create function', () => {
assert.strictEqual(typeof noSourceGrep.create, 'function');
});
test('valid: readFileSync on .md file is allowed', () => {
ruleTester.run('no-source-grep', noSourceGrep, {
valid: [
@@ -53,7 +58,6 @@ describe('no-source-grep rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-source-grep valid cases passed');
});
test('invalid: readFileSync on .cjs source file followed by .includes()', () => {
@@ -72,7 +76,6 @@ describe('no-source-grep rule', () => {
},
],
});
assert.ok(true, 'no-source-grep invalid case detected');
});
test('invalid: readFileSync on .cjs source file followed by .match()', () => {
@@ -91,7 +94,6 @@ describe('no-source-grep rule', () => {
},
],
});
assert.ok(true, 'no-source-grep match case detected');
});
test('valid: file with allow-test-rule annotation is exempt', () => {
@@ -111,7 +113,6 @@ describe('no-source-grep rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-source-grep allow-test-rule annotation works');
});
test('valid: require() of a .cjs file is allowed (not readFileSync)', () => {
@@ -127,13 +128,16 @@ describe('no-source-grep rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-source-grep require() is allowed');
});
});
// ─── no-magic-sleep-in-tests ─────────────────────────────────────────────────
describe('no-magic-sleep-in-tests rule', () => {
test('rule module exports a create function', () => {
assert.strictEqual(typeof noMagicSleepInTests.create, 'function');
});
test('valid: setTimeout used outside tests (no-op since rule only applies to *.test.cjs)', () => {
// Rule only applies to *.test.cjs files; a non-test filename is always valid
ruleTester.run('no-magic-sleep-in-tests', noMagicSleepInTests, {
@@ -147,7 +151,6 @@ describe('no-magic-sleep-in-tests rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-magic-sleep-in-tests does not apply outside test files');
});
test('invalid: Atomics.wait() in test file', () => {
@@ -165,7 +168,6 @@ describe('no-magic-sleep-in-tests rule', () => {
},
],
});
assert.ok(true, 'no-magic-sleep-in-tests flags Atomics.wait()');
});
test('invalid: setTimeout used for synchronization in Promise in test file', () => {
@@ -183,7 +185,6 @@ describe('no-magic-sleep-in-tests rule', () => {
},
],
});
assert.ok(true, 'no-magic-sleep-in-tests flags setTimeout in Promise');
});
test('valid: setTimeout with callback (not synchronization pattern) in test file', () => {
@@ -202,13 +203,16 @@ describe('no-magic-sleep-in-tests rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-magic-sleep-in-tests allows simple callback setTimeout');
});
});
// ─── no-elapsed-assertion ─────────────────────────────────────────────────────
describe('no-elapsed-assertion rule', () => {
test('rule module exports a create function', () => {
assert.strictEqual(typeof noElapsedAssertion.create, 'function');
});
test('valid: assert on non-timing property', () => {
ruleTester.run('no-elapsed-assertion', noElapsedAssertion, {
valid: [
@@ -230,7 +234,6 @@ describe('no-elapsed-assertion rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-elapsed-assertion valid cases passed');
});
test('invalid: assert on .elapsed property', () => {
@@ -248,7 +251,6 @@ describe('no-elapsed-assertion rule', () => {
},
],
});
assert.ok(true, 'no-elapsed-assertion flags assert on .elapsed');
});
test('invalid: assert on .duration property', () => {
@@ -265,7 +267,6 @@ describe('no-elapsed-assertion rule', () => {
},
],
});
assert.ok(true, 'no-elapsed-assertion flags assert on .duration');
});
test('invalid: assert on .took property', () => {
@@ -282,7 +283,6 @@ describe('no-elapsed-assertion rule', () => {
},
],
});
assert.ok(true, 'no-elapsed-assertion flags assert on .took');
});
test('invalid: assert on .ms property', () => {
@@ -299,7 +299,6 @@ describe('no-elapsed-assertion rule', () => {
},
],
});
assert.ok(true, 'no-elapsed-assertion flags assert on .ms');
});
test('invalid: assert.equal with timing comparison', () => {
@@ -316,13 +315,16 @@ describe('no-elapsed-assertion rule', () => {
},
],
});
assert.ok(true, 'no-elapsed-assertion flags assert.equal with timing comparison');
});
});
// ─── no-raw-rmsync-in-tests ──────────────────────────────────────────────────
describe('no-raw-rmsync-in-tests rule', () => {
test('rule module exports a create function', () => {
assert.strictEqual(typeof noRawRmsyncInTests.create, 'function');
});
// ── INVALID cases (must error) ────────────────────────────────────────────
test('invalid: fs.rmSync() in a test file', () => {
@@ -339,7 +341,6 @@ describe('no-raw-rmsync-in-tests rule', () => {
},
],
});
assert.ok(true, 'no-raw-rmsync-in-tests flags fs.rmSync() in test file');
});
test('invalid: computed member fs["rmSync"]() in a test file', () => {
@@ -356,7 +357,6 @@ describe('no-raw-rmsync-in-tests rule', () => {
},
],
});
assert.ok(true, 'no-raw-rmsync-in-tests flags fs["rmSync"]() in test file');
});
test('invalid: destructured rmSync from require("fs") in a test file', () => {
@@ -373,7 +373,6 @@ describe('no-raw-rmsync-in-tests rule', () => {
},
],
});
assert.ok(true, 'no-raw-rmsync-in-tests flags destructured rmSync from require("fs")');
});
test('invalid: aliased const del = fs.rmSync; del() in a test file', () => {
@@ -391,7 +390,6 @@ describe('no-raw-rmsync-in-tests rule', () => {
},
],
});
assert.ok(true, 'no-raw-rmsync-in-tests flags aliased fs.rmSync');
});
test('invalid: allow-test-rule annotation no longer suppresses this rule (Defect 1 fixed)', () => {
@@ -411,7 +409,6 @@ describe('no-raw-rmsync-in-tests rule', () => {
},
],
});
assert.ok(true, 'no-raw-rmsync-in-tests is NOT suppressed by allow-test-rule annotation');
});
// ── VALID cases (must NOT error) ──────────────────────────────────────────
@@ -429,7 +426,6 @@ describe('no-raw-rmsync-in-tests rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-raw-rmsync-in-tests allows helpers.cleanup()');
});
test('valid: bare rmSync() that is NOT fs-derived (local function) is not flagged', () => {
@@ -447,7 +443,6 @@ describe('no-raw-rmsync-in-tests rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-raw-rmsync-in-tests does not flag a locally-defined rmSync()');
});
// NOTE: The inline `// eslint-disable-next-line local/no-raw-rmsync-in-tests -- reason`
@@ -469,7 +464,6 @@ describe('no-raw-rmsync-in-tests rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-raw-rmsync-in-tests is inert in non-test files');
});
test('valid: member access / assignment without calling (not a CallExpression)', () => {
@@ -486,6 +480,335 @@ describe('no-raw-rmsync-in-tests rule', () => {
],
invalid: [],
});
assert.ok(true, 'no-raw-rmsync-in-tests ignores member access / assignment without call');
});
});
// ─── no-tautological-assert ──────────────────────────────────────────────────
describe('no-tautological-assert rule', () => {
test('rule module exports a create function', () => {
assert.strictEqual(typeof noTautologicalAssert.create, 'function');
});
// ── VALID cases (must NOT error) ──────────────────────────────────────────
test('valid: assert.ok with a non-literal identifier argument', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [
{
code: `
const assert = require('node:assert/strict');
assert.ok(result);
`,
filename: 'tests/foo.test.cjs',
},
],
invalid: [],
});
});
test('valid: assert.strictEqual with mixed literal/identifier arguments', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [
{
code: `
const assert = require('node:assert/strict');
assert.strictEqual(actual, true);
`,
filename: 'tests/foo.test.cjs',
},
],
invalid: [],
});
});
test('valid: assert.strictEqual with identifier and numeric literal', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [
{
code: `
const assert = require('node:assert/strict');
assert.strictEqual(x, 5);
`,
filename: 'tests/foo.test.cjs',
},
],
invalid: [],
});
});
test('valid: assert.ok with a CallExpression argument', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [
{
code: `
const assert = require('node:assert/strict');
assert.ok(fn());
`,
filename: 'tests/foo.test.cjs',
},
],
invalid: [],
});
});
test('valid: assert.deepStrictEqual with two identifier arguments', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [
{
code: `
const assert = require('node:assert/strict');
assert.deepStrictEqual(got, expected);
`,
filename: 'tests/foo.test.cjs',
},
],
invalid: [],
});
});
test('valid: assert.strictEqual with two different identifier arguments', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [
{
code: `
const assert = require('node:assert/strict');
assert.strictEqual(a, b);
`,
filename: 'tests/foo.test.cjs',
},
],
invalid: [],
});
});
// ── INVALID cases (must error) ────────────────────────────────────────────
test('invalid: assert.ok(true) — always-truthy boolean literal', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.ok(true);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalTruthiness' }],
},
],
});
});
test('invalid: assert(true) — bare assert with always-truthy boolean literal', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert');
assert(true);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalTruthiness' }],
},
],
});
});
test('invalid: assert.ok(1) — always-truthy non-zero numeric literal', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.ok(1);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalTruthiness' }],
},
],
});
});
test('invalid: assert.ok("always") — always-truthy non-empty string literal', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.ok('always');
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalTruthiness' }],
},
],
});
});
test('invalid: assert.ok([]) — always-truthy array literal', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.ok([]);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalTruthiness' }],
},
],
});
});
test('invalid: assert.ok(cond || true) — logical OR whose right side is true', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.ok(cond || true);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalTruthiness' }],
},
],
});
});
test('invalid: assert.strictEqual(true, true) — identical boolean literals', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.strictEqual(true, true);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalEquality' }],
},
],
});
});
test('invalid: assert.equal(1, 1) — identical numeric literals', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.equal(1, 1);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalEquality' }],
},
],
});
});
// ── Fix #3: true || cond (left-side true) ────────────────────────────────
test('invalid: assert.ok(true || x) — left side is literal true (always short-circuits)', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.ok(true || x);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalTruthiness' }],
},
],
});
});
test('invalid: assert(true || y) — bare assert, left side is literal true', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert');
assert(true || y);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalTruthiness' }],
},
],
});
});
// ── Fix #4: empty [] / {} deep-equality ──────────────────────────────────
test('invalid: assert.deepStrictEqual([], []) — two empty arrays are always deep-equal', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.deepStrictEqual([], []);
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalEquality' }],
},
],
});
});
test('invalid: assert.deepStrictEqual({}, {}) — two empty objects are always deep-equal', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [],
invalid: [
{
code: `
const assert = require('node:assert/strict');
assert.deepStrictEqual({}, {});
`,
filename: 'tests/foo.test.cjs',
errors: [{ messageId: 'tautologicalEquality' }],
},
],
});
});
// ── Conservative: non-empty arrays/objects must NOT be flagged ────────────
test('valid: assert.deepStrictEqual([1], [2]) — non-empty arrays with different content are not flagged', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [
{
code: `
const assert = require('node:assert/strict');
assert.deepStrictEqual([1], [2]);
`,
filename: 'tests/foo.test.cjs',
},
],
invalid: [],
});
});
test('valid: assert.deepStrictEqual(got, expected) — identifier arguments are not flagged', () => {
ruleTester.run('no-tautological-assert', noTautologicalAssert, {
valid: [
{
code: `
const assert = require('node:assert/strict');
assert.deepStrictEqual(got, expected);
`,
filename: 'tests/foo.test.cjs',
},
],
invalid: [],
});
});
});