Files
msd-core/tests/agent-required-reading-consistency.test.cjs
Tom Boucher 69e7afd0c7 chore(#3212): bounded quantifiers over document content — prohibition with teeth — Phase 4 (#3441)
* feat(#3415): ship local/no-unbounded-quantifier, burn down ReDoS class

Phase 4 of epic #3212 (ADR-3212 §5/§7, the final phase). New rule flags
an unbounded */+/{n,} quantifier over a broad character class
([\s\S], dotAll ., or a 1-2-unit negated class like [^\n]/[^)\n] — the
exact #2128-fixed shape) applied to a regex whose match target is
data-flow-traced to readFileSync content.

eslint-rules/lib/readfilesync-trace.cjs extracts the data-flow tracer
shared with no-crlf-fragile-split (Phase 2) rather than a second copy
— no-crlf-fragile-split refactored onto it with zero behavior change,
parity-tested.

Real triage, not 798 mechanical edits: the ADR's census (2026-08-08)
screened every unbounded quantifier in the tree unscoped. Correctly
scoped to readFileSync-derived content (matching Phase 2's own G2/G3
scoping), the rule found 162 real hits across two detection waves — the
second wave (93) surfaced only after a genuine off-by-one bug in this
rule's own first draft was caught while writing its RuleTester tests
and fixed (the bug silently missed every directly-quantified [\s\S]*
with no gap before the quantifier — exactly the class this rule exists
to catch). 3 hits landed in production src/ (commands.cts, milestone.cts,
roadmap.cts) and were each empirically timed against adversarial input
(matching #2128's own measured-not-assumed precedent) — all confirmed
linear-time/benign, left unbounded with a measured-evidence comment
rather than mechanically bounded. The remaining 159 are test-file
fixture parsing (test-author-controlled, fixed-size content, not
adversarial input) — each suppressed with a specific, non-generic
reason. Zero functional behavior changed anywhere in this diff.

tests/no-pending-3212-markers.test.cjs locks the epic's own closing
invariant (ADR §7: "assert zero pending #3212 markers remain") — ground
truth confirmed trivially true today (no phase left any such marker
behind), now regression-locked going forward.

Design: .gsd/phase/chore-3415-prohibition-with-teeth/40-design.md
Test matrix: .gsd/phase/chore-3415-prohibition-with-teeth/50-test-matrix.md

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#3415): correct rule category mislabel, add CI test-scope entry

An orthogonal Standards-axis review found eslint-rules/no-unbounded-quantifier.cjs
mistakenly carried meta.docs.category: 'Portability', copied from a sibling
rule without realizing what that implied: docs/contributing/cross-platform-
portability-rules.md governs an ADR-1703 rule family under a hard "zero
escape hatches" contract (tests/portability-rule-disable-ban.test.cjs's
PROTECTED_RULES bans eslint-disable for those rules entirely). This rule is
not part of that family — it's ADR-3212 (ReDoS/CWE-1333), a different epic —
and its eslint-disable-next-line suppressions (159 of them, added earlier
this same phase after empirical benign-verification) are an intentional,
correct design, not a bypass. Corrected to category: 'Best Practices',
matching the actual precedent (no-adhoc-regex-escape.cjs, Phase 1 of the
same epic, which is also correctly outside PROTECTED_RULES), and the rule's
own docstring now states this explicitly so a future reader doesn't have to
re-derive it.

Also registers a new scripts/ci-test-scope.cjs bucket so editing this rule
or the shared eslint-rules/lib/readfilesync-trace.cjs helper re-runs their
own test suites under targeted CI selection — was previously unregistered
and invisible to that fast-path (this PR's own gsd-test checkpoint runs the
full suite regardless, so this only affects future narrowly-scoped PRs).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#3415): bound no-unbounded-quantifier's own scanner (CWE-1333, ironic)

Security review found the rule meant to catch algorithmic-complexity bugs
had one of its own: hasUnboundedBroadQuantifier's negated-class inner
scan walked from each `[^` occurrence to the next `]` (or EOF) with no
bound, while the outer loop only ever advanced by one character — O(n²)
total work on a pattern with many unclosed `[^` runs. Runs unconditionally
inside checkPattern on any `new RegExp('literal string')` argument in any
linted file, before the (cheap) readFileSync data-flow gate — so a single
crafted string literal, no valid regex syntax required, could make
`npm run lint` / CI hang.

Empirically confirmed both the bug and the fix: pre-fix, n=4000/8000/
16000/32000 chars took 30.8/115.6/463.8/1874.3ms (~4x work per 2x n,
quadratic); extrapolated, the 300000-char repro from the finding would
run ~165s. Post-fix (bail the inner scan once units exceeds the rule's
own 1-2-unit scope, rather than continuing to hunt for a closing `]`),
the same 300000-char input runs in 8.7ms via the real rule module,
independently reconfirmed at 18ms via a fresh Linter.verify() call.

New regression row in tests/no-unbounded-quantifier.rule.test.cjs
asserts the RuleTester run on a 50000-char adversarial pattern
completes and returns a defined result — no wall-clock assertion
(CLAUDE.md Clock Seams / local/no-elapsed-assertion).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#3415): triage 3 new sites, re-raise ceiling after upstream batch

next merged 12 more PRs during this PR's review. Two consequences:

- tests/edit-phase.test.cjs (fix #3262, unrelated) added 3 new
  content.match(/<tag>([\s\S]*?)<\/tag>/) reads of this repo's own
  workflow .md content — the same Class A pattern as the ~159 sites
  already triaged elsewhere in this PR. Suppressed with the same
  established reason.
- lint-allow-test-rule-refs' ratchet ceiling needed re-raising again
  (301 -> 303) for the same reason as the two prior bumps: organic
  growth from unrelated, already-reviewed PRs landing concurrently,
  not a defect in this branch's own diff.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-14 10:02:28 -04:00

211 lines
7.9 KiB
JavaScript

// allow-test-rule: source-text-is-the-product
// Reads .md/.json/.yml product files whose deployed text IS what the
// runtime loads — testing text content tests the deployed contract.
/**
* GSD Agent Required Reading Consistency Tests
*
* Validates that all agent .md files use the standardized <required_reading>
* pattern and that no legacy <files_to_read> blocks remain.
*
* See: https://github.com/open-gsd/gsd-core/issues/2168
*/
const { test, describe } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('fs');
const path = require('path');
const { listAgentFiles } = require('./helpers/agent-roster.cjs');
const AGENTS_DIR = path.join(__dirname, '..', 'agents');
// Sorted basenames (without `.md`). Every use below generates an independent
// per-agent test and reads each file via `agent + '.md'`; nothing here depends
// on registration order, so the sorted helper roster is behaviorally identical.
const ALL_AGENTS = listAgentFiles(AGENTS_DIR);
// ─── No Legacy files_to_read Blocks ────────────────────────────────────────
describe('READING: no legacy <files_to_read> blocks remain', () => {
for (const agent of ALL_AGENTS) {
test(`${agent} does not contain <files_to_read>`, () => {
const content = fs.readFileSync(path.join(AGENTS_DIR, agent + '.md'), 'utf-8');
assert.ok(
!content.includes('<files_to_read>'),
`${agent} still has <files_to_read> opening tag — migrate to <required_reading>`
);
assert.ok(
!content.includes('</files_to_read>'),
`${agent} still has </files_to_read> closing tag — migrate to </required_reading>`
);
});
}
test('no backtick references to files_to_read in any agent', () => {
for (const agent of ALL_AGENTS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, agent + '.md'), 'utf-8');
assert.ok(
!content.includes('`<files_to_read>`'),
`${agent} still references \`<files_to_read>\` in prose — update to \`<required_reading>\``
);
}
});
});
// ─── Standardized required_reading Pattern ─────────────────────────────────
describe('READING: agents with reading blocks use <required_reading>', () => {
// Agents that have any kind of reading instruction should use required_reading
const AGENTS_WITH_READING = ALL_AGENTS.filter(name => {
const content = fs.readFileSync(path.join(AGENTS_DIR, name + '.md'), 'utf-8');
return content.includes('required_reading') || content.includes('files_to_read');
});
test('at least 20 agents have reading instructions', () => {
assert.ok(
AGENTS_WITH_READING.length >= 20,
`Expected at least 20 agents with reading instructions, found ${AGENTS_WITH_READING.length}`
);
});
for (const agent of AGENTS_WITH_READING) {
test(`${agent} uses required_reading (not files_to_read)`, () => {
const content = fs.readFileSync(path.join(AGENTS_DIR, agent + '.md'), 'utf-8');
assert.ok(
content.includes('required_reading'),
`${agent} has reading instructions but does not use required_reading`
);
assert.ok(
!content.includes('files_to_read'),
`${agent} still uses files_to_read — must be migrated to required_reading`
);
});
}
});
// ────────────────────────────────────────────────────────────────────────
// Folded from tests/bug-patterns-reference.test.cjs — consolidation epic #1969 (B8 #1977)
// ────────────────────────────────────────────────────────────────────────
{
const { describe: __foldDescribe } = require('node:test');
__foldDescribe("folded:bug-patterns-reference (consolidation epic #1969 B8 #1977)", () => {
// allow-test-rule: source-text-is-the-product
// Workflow .md / agent .md / command .md / reference .md files — their text
// IS what the runtime loads. Testing text content tests the deployed contract.
// Per CONTRIBUTING.md exception matrix.
/**
* Common Bug Patterns Reference Tests
*
* Structural tests for the common-bug-patterns.md reference file:
* - File exists at expected path
* - Contains expected bug pattern categories (at least 5 of 10)
* - Debugger agent references the file in required_reading
*/
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('fs');
const path = require('path');
const REFERENCE_PATH = path.join(
__dirname, '..', 'gsd-core', 'references', 'common-bug-patterns.md'
);
const DEBUGGER_AGENT_PATH = path.join(
__dirname, '..', 'agents', 'gsd-debugger.md'
);
const EXPECTED_CATEGORIES = [
'Off-by-One',
'Null',
'Async',
'State Management',
'Import',
'Environment',
'Data Shape',
'String Handling',
'File System',
'Error Handling',
];
describe('common-bug-patterns.md reference', () => {
test('reference file exists', () => {
assert.ok(
fs.existsSync(REFERENCE_PATH),
`Expected reference file at ${REFERENCE_PATH}`
);
});
test('has title and intro', () => {
const content = fs.readFileSync(REFERENCE_PATH, 'utf-8');
assert.ok(
content.startsWith('# Common Bug Patterns'),
'File should start with "# Common Bug Patterns" title'
);
assert.ok(
content.includes('---'),
'File should contain --- separator after intro'
);
});
test('contains at least 5 of 10 expected categories', () => {
const content = fs.readFileSync(REFERENCE_PATH, 'utf-8');
const found = EXPECTED_CATEGORIES.filter(cat =>
content.toLowerCase().includes(cat.toLowerCase())
);
assert.ok(
found.length >= 5,
`Expected at least 5 categories, found ${found.length}: ${found.join(', ')}`
);
});
test('each pattern category has at least one bold bullet item', () => {
const content = fs.readFileSync(REFERENCE_PATH, 'utf-8');
// Only check sections inside <patterns> block, not <usage>
const patternsBlock = (content.split('<patterns>')[1] || '').split('</patterns>')[0];
const sections = patternsBlock.split(/^## /m).slice(1);
assert.ok(sections.length >= 5, `Expected at least 5 pattern sections, got ${sections.length}`);
for (const section of sections) {
const title = section.split('\n')[0].trim();
const bullets = section.match(/^- \*\*/gm);
assert.ok(
bullets && bullets.length >= 1,
`Pattern section "${title}" should have at least one "- **" bullet item`
);
}
});
});
describe('debugger agent references bug patterns', () => {
test('gsd-debugger.md exists', () => {
assert.ok(
fs.existsSync(DEBUGGER_AGENT_PATH),
`Expected debugger agent at ${DEBUGGER_AGENT_PATH}`
);
});
test('gsd-debugger.md references common-bug-patterns.md', () => {
const content = fs.readFileSync(DEBUGGER_AGENT_PATH, 'utf-8');
assert.ok(
content.includes('common-bug-patterns.md'),
'Debugger agent should reference common-bug-patterns.md'
);
});
test('reference is inside <required_reading> block', () => {
const content = fs.readFileSync(DEBUGGER_AGENT_PATH, 'utf-8');
const reqReadMatch = content.match(
// eslint-disable-next-line local/no-unbounded-quantifier -- parses this repo's own agent .md content, fixed-size author-controlled content
/<required_reading>([\s\S]*?)<\/required_reading>/
);
assert.ok(reqReadMatch, 'Debugger agent should have a <required_reading> block');
assert.ok(
reqReadMatch[1].includes('common-bug-patterns.md'),
'common-bug-patterns.md should be inside <required_reading> block'
);
});
});
});
}