* test(3594): adversarial parser fixtures + frontmatter/roadmap matrix + property-style suite
Lands the adversarial parser-input corpus that CONTRIBUTING.md
§"QA Matrix Requirements / Parser and project-file inputs" and
TEST-EXAMPLES.md §"Parser Adversarial Fixtures" describe.
New tests/fixtures/adversarial/ layout:
frontmatter/
duplicate-keys.md — same key twice (collapses last-wins)
crlf-mixed.md — CRLF endings throughout
unclosed-block.md — `---` open with no close
unicode-keys-and-values.md — non-ASCII + emoji + Greek
null-byte-value.md — U+0000 in a value
huge-bounded.md — 2000-item array, ~30KB
roadmap/
phase-heading-inside-fenced-code.md — #2787 fence shadowing
nested-fenced-code.md — outer + inner ``` blocks
unicode-phase-titles.md — JP / Greek / emoji titles
repeated-phase-ids.md — phase 1 declared twice
decimal-phase-mixed.md — 2 vs 2.1 vs 2.10 vs 21
markdown-headings-inside-html-comment.md — comment shadowing
Test files (all node:test, no try/finally in test bodies, no source-grep,
no raw-text matching on stdout/file content):
tests/feat-3594-parser-adversarial-frontmatter.test.cjs (12 tests)
Loads each fixture, pins parser invariants on extractFrontmatter()
return shape. Cross-corpus "does not throw on any fixture" sweep.
tests/feat-3594-parser-adversarial-roadmap.test.cjs (18 tests)
Loads each fixture into a temp project's .planning/ROADMAP.md and
drives `gsd-tools roadmap get-phase <N>` via the runCli harness
introduced by #3593. Asserts on the typed JSON payload.
tests/feat-3594-parser-property-style.test.cjs (2 tests)
Deterministic mulberry32 PRNG generates 500 malformed-ish
frontmatter inputs per test. Pins (a) extractFrontmatter is total
over the corpus (no null-deref TypeError, always returns a plain
object on success), (b) the suite completes well under 2 seconds
(quadratic-regression guard).
Known-open bugs surfaced and pinned (intentionally NOT fixed in this
PR — separate issues warranted):
- CJS roadmap parser matches `## Phase N:` headings inside fenced
code blocks (the SDK parser tracks fences per the #2787 comment in
sdk/src/query/roadmap.ts but the CJS path has not caught up).
- CJS roadmap parser matches `## Phase N:` headings inside HTML
comments.
Both are documented in-test with the "currently STILL matches it
(open: needs <fix>)" naming pattern so the day the production fix
lands, flipping the assertion from `found: true` to `found: false` is
the regression guard.
Test totals:
- 32 new feat-3594-* tests (12 frontmatter + 18 roadmap + 2 property)
- 108/108 pass when running together with the pre-existing
frontmatter.test.cjs + roadmap.test.cjs suites (76 of theirs).
Closes #3594
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(3594): use Fisher-Yates shuffle for deterministic seeded inputs
Replaces `arr.sort(() => rng() - 0.5)` with a Fisher-Yates shuffle
driven by the supplied PRNG. The sort-based shuffle is non-transitive:
V8's TimSort behavior on non-transitive comparators is engine-defined,
so the same seed produced different orderings across Node versions —
undermining the test's stated reproducibility guarantee.
Fisher-Yates is O(n), transitive (no comparator at all), and consumes
exactly n-1 RNG values in a fixed order. The mulberry32 seed now
determines the input sequence end-to-end.
Codex review on PR #3633.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
136 lines
4.9 KiB
JavaScript
136 lines
4.9 KiB
JavaScript
/**
|
||
* Deterministic property-style parser tests (#3594).
|
||
*
|
||
* Follows TEST-EXAMPLES.md §"Deterministic Property-Style Parser Test":
|
||
* a bounded, seeded loop generates many malformed inputs and asserts a
|
||
* single invariant against each. On failure the seed and case index
|
||
* are printed so the failing input can be reproduced exactly.
|
||
*
|
||
* The generator is a small mulberry32 PRNG so this file has zero
|
||
* external dependencies and is fully reproducible across Node versions.
|
||
* Each test pins its own seed and case count; bumping either is a
|
||
* deliberate test change, not a flake source.
|
||
*
|
||
* Invariant tested (frontmatter): for any random text the parser must
|
||
* either return a plain object or throw — never return null/undefined,
|
||
* never hang, never propagate "Cannot read properties of …" V8 prose.
|
||
*/
|
||
|
||
'use strict';
|
||
|
||
const { test } = require('node:test');
|
||
const assert = require('node:assert/strict');
|
||
|
||
const { extractFrontmatter } = require('../get-shit-done/bin/lib/frontmatter.cjs');
|
||
|
||
/**
|
||
* mulberry32 — small fast deterministic PRNG. Seed in, [0,1) out.
|
||
* Same input always produces the same sequence across Node versions.
|
||
*/
|
||
function mulberry32(seed) {
|
||
let a = seed >>> 0;
|
||
return function next() {
|
||
a = (a + 0x6D2B79F5) | 0;
|
||
let t = a;
|
||
t = Math.imul(t ^ (t >>> 15), t | 1);
|
||
t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
|
||
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
|
||
};
|
||
}
|
||
|
||
/**
|
||
* Build a single malformed-ish frontmatter input. Components are mixed
|
||
* deterministically by the supplied PRNG.
|
||
*/
|
||
function makeInput(rng) {
|
||
const fragments = [
|
||
'---\n',
|
||
'title: Generated\n',
|
||
'phase: 99\n',
|
||
'plans:\n - a\n - b\n',
|
||
'extra: \xff\xfe\xfd\n', // invalid UTF-8 bytes
|
||
'unicode: 日本語\n',
|
||
'crlf: ends\r\nin\rcr\n',
|
||
' indented_key: value\n',
|
||
'duplicate: first\nduplicate: second\n',
|
||
'sparse:\n\n\n',
|
||
'malformed_array: [a, "b", c\n', // unclosed inline array
|
||
'null_byte: before |