Files
msd-core/tests/feat-3594-parser-property-style.test.cjs
Tom Boucher b1317633db test(3594): adversarial parser fixtures + frontmatter/roadmap matrix + property-style suite (#3633)
* test(3594): adversarial parser fixtures + frontmatter/roadmap matrix + property-style suite

Lands the adversarial parser-input corpus that CONTRIBUTING.md
§"QA Matrix Requirements / Parser and project-file inputs" and
TEST-EXAMPLES.md §"Parser Adversarial Fixtures" describe.

New tests/fixtures/adversarial/ layout:

  frontmatter/
    duplicate-keys.md            — same key twice (collapses last-wins)
    crlf-mixed.md                — CRLF endings throughout
    unclosed-block.md            — `---` open with no close
    unicode-keys-and-values.md   — non-ASCII + emoji + Greek
    null-byte-value.md           — U+0000 in a value
    huge-bounded.md              — 2000-item array, ~30KB

  roadmap/
    phase-heading-inside-fenced-code.md   — #2787 fence shadowing
    nested-fenced-code.md                 — outer + inner ``` blocks
    unicode-phase-titles.md               — JP / Greek / emoji titles
    repeated-phase-ids.md                 — phase 1 declared twice
    decimal-phase-mixed.md                — 2 vs 2.1 vs 2.10 vs 21
    markdown-headings-inside-html-comment.md — comment shadowing

Test files (all node:test, no try/finally in test bodies, no source-grep,
no raw-text matching on stdout/file content):

  tests/feat-3594-parser-adversarial-frontmatter.test.cjs (12 tests)
    Loads each fixture, pins parser invariants on extractFrontmatter()
    return shape. Cross-corpus "does not throw on any fixture" sweep.

  tests/feat-3594-parser-adversarial-roadmap.test.cjs (18 tests)
    Loads each fixture into a temp project's .planning/ROADMAP.md and
    drives `gsd-tools roadmap get-phase <N>` via the runCli harness
    introduced by #3593. Asserts on the typed JSON payload.

  tests/feat-3594-parser-property-style.test.cjs (2 tests)
    Deterministic mulberry32 PRNG generates 500 malformed-ish
    frontmatter inputs per test. Pins (a) extractFrontmatter is total
    over the corpus (no null-deref TypeError, always returns a plain
    object on success), (b) the suite completes well under 2 seconds
    (quadratic-regression guard).

Known-open bugs surfaced and pinned (intentionally NOT fixed in this
PR — separate issues warranted):

  - CJS roadmap parser matches `## Phase N:` headings inside fenced
    code blocks (the SDK parser tracks fences per the #2787 comment in
    sdk/src/query/roadmap.ts but the CJS path has not caught up).
  - CJS roadmap parser matches `## Phase N:` headings inside HTML
    comments.

Both are documented in-test with the "currently STILL matches it
(open: needs <fix>)" naming pattern so the day the production fix
lands, flipping the assertion from `found: true` to `found: false` is
the regression guard.

Test totals:
  - 32 new feat-3594-* tests (12 frontmatter + 18 roadmap + 2 property)
  - 108/108 pass when running together with the pre-existing
    frontmatter.test.cjs + roadmap.test.cjs suites (76 of theirs).

Closes #3594

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(3594): use Fisher-Yates shuffle for deterministic seeded inputs

Replaces `arr.sort(() => rng() - 0.5)` with a Fisher-Yates shuffle
driven by the supplied PRNG. The sort-based shuffle is non-transitive:
V8's TimSort behavior on non-transitive comparators is engine-defined,
so the same seed produced different orderings across Node versions —
undermining the test's stated reproducibility guarantee.

Fisher-Yates is O(n), transitive (no comparator at all), and consumes
exactly n-1 RNG values in a fixed order. The mulberry32 seed now
determines the input sequence end-to-end.

Codex review on PR #3633.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 00:23:27 -04:00

136 lines
4.9 KiB
JavaScript
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* Deterministic property-style parser tests (#3594).
*
* Follows TEST-EXAMPLES.md §"Deterministic Property-Style Parser Test":
* a bounded, seeded loop generates many malformed inputs and asserts a
* single invariant against each. On failure the seed and case index
* are printed so the failing input can be reproduced exactly.
*
* The generator is a small mulberry32 PRNG so this file has zero
* external dependencies and is fully reproducible across Node versions.
* Each test pins its own seed and case count; bumping either is a
* deliberate test change, not a flake source.
*
* Invariant tested (frontmatter): for any random text the parser must
* either return a plain object or throw — never return null/undefined,
* never hang, never propagate "Cannot read properties of …" V8 prose.
*/
'use strict';
const { test } = require('node:test');
const assert = require('node:assert/strict');
const { extractFrontmatter } = require('../get-shit-done/bin/lib/frontmatter.cjs');
/**
* mulberry32 — small fast deterministic PRNG. Seed in, [0,1) out.
* Same input always produces the same sequence across Node versions.
*/
function mulberry32(seed) {
let a = seed >>> 0;
return function next() {
a = (a + 0x6D2B79F5) | 0;
let t = a;
t = Math.imul(t ^ (t >>> 15), t | 1);
t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
};
}
/**
* Build a single malformed-ish frontmatter input. Components are mixed
* deterministically by the supplied PRNG.
*/
function makeInput(rng) {
const fragments = [
'---\n',
'title: Generated\n',
'phase: 99\n',
'plans:\n - a\n - b\n',
'extra: \xff\xfe\xfd\n', // invalid UTF-8 bytes
'unicode: 日本語\n',
'crlf: ends\r\nin\rcr\n',
' indented_key: value\n',
'duplicate: first\nduplicate: second\n',
'sparse:\n\n\n',
'malformed_array: [a, "b", c\n', // unclosed inline array
'null_byte: beforeafter\n',
];
// Pick a random subset of fragments in random order. Always include
// the opening `---`. Closing `---` is included by 50% of cases so we
// exercise both well-formed and unclosed shapes.
const head = fragments[0];
const rest = shuffle(fragments.slice(1), rng).slice(0, 1 + Math.floor(rng() * 6));
const closing = rng() < 0.5 ? '---\n' : '';
return head + rest.join('') + closing + '\nBody.\n';
}
/**
* Fisher-Yates shuffle driven by the supplied PRNG. Returns a new
* array; does not mutate the input. Replaces the previous
* `arr.sort(() => rng() - 0.5)` which was non-transitive — the
* resulting order depended on V8's sort implementation, not only on
* the seed, so failing cases were unreproducible across Node versions.
* Fisher-Yates is O(n), transitive (no comparator), and depends only
* on the RNG output. Codex review on PR #3633 / #3594.
*/
function shuffle(arr, rng) {
const out = arr.slice();
for (let i = out.length - 1; i > 0; i--) {
const j = Math.floor(rng() * (i + 1));
const tmp = out[i];
out[i] = out[j];
out[j] = tmp;
}
return out;
}
test('extractFrontmatter is total over 500 deterministic random inputs (seed=1234)', () => {
const seed = 1234;
const rng = mulberry32(seed);
const count = 500;
for (let i = 0; i < count; i++) {
const input = makeInput(rng);
let result;
try {
result = extractFrontmatter(input);
} catch (err) {
// If the parser throws, the failure must be a controlled one —
// not a V8 "Cannot read properties of undefined" that signals a
// null-deref bug. Print the seed and case index so the input
// can be reproduced exactly.
const msg = String((err && err.message) || err);
assert.doesNotMatch(
msg,
/Cannot read propert/i,
`seed=${seed} case=${i}: parser must not propagate null-deref TypeError; input=${JSON.stringify(input)}`,
);
continue;
}
// No throw: result MUST be a plain object (not null, not array, not
// primitive). Print enough on failure to reproduce.
assert.equal(typeof result, 'object', `seed=${seed} case=${i}: result must be object, got ${typeof result}`);
assert.notEqual(result, null, `seed=${seed} case=${i}: result must not be null`);
assert.equal(Array.isArray(result), false, `seed=${seed} case=${i}: result must not be an array`);
}
});
test('extractFrontmatter completes 500 cases under 2 seconds (no quadratic regression guard)', () => {
const seed = 5678;
const rng = mulberry32(seed);
const count = 500;
const startedAt = Date.now();
for (let i = 0; i < count; i++) {
extractFrontmatter(makeInput(rng));
}
const elapsedMs = Date.now() - startedAt;
// Generous bound — a parser regression that becomes O(n^2) on input
// length would blow past this with ease while leaving room for
// legitimate Node-version variance.
assert.ok(
elapsedMs < 2000,
`500 random parses took ${elapsedMs}ms — should be < 2000ms (possible quadratic-in-input-length regression)`,
);
});