test(3594): adversarial parser fixtures + frontmatter/roadmap matrix + property-style suite (#3633)

* test(3594): adversarial parser fixtures + frontmatter/roadmap matrix + property-style suite

Lands the adversarial parser-input corpus that CONTRIBUTING.md
§"QA Matrix Requirements / Parser and project-file inputs" and
TEST-EXAMPLES.md §"Parser Adversarial Fixtures" describe.

New tests/fixtures/adversarial/ layout:

  frontmatter/
    duplicate-keys.md            — same key twice (collapses last-wins)
    crlf-mixed.md                — CRLF endings throughout
    unclosed-block.md            — `---` open with no close
    unicode-keys-and-values.md   — non-ASCII + emoji + Greek
    null-byte-value.md           — U+0000 in a value
    huge-bounded.md              — 2000-item array, ~30KB

  roadmap/
    phase-heading-inside-fenced-code.md   — #2787 fence shadowing
    nested-fenced-code.md                 — outer + inner ``` blocks
    unicode-phase-titles.md               — JP / Greek / emoji titles
    repeated-phase-ids.md                 — phase 1 declared twice
    decimal-phase-mixed.md                — 2 vs 2.1 vs 2.10 vs 21
    markdown-headings-inside-html-comment.md — comment shadowing

Test files (all node:test, no try/finally in test bodies, no source-grep,
no raw-text matching on stdout/file content):

  tests/feat-3594-parser-adversarial-frontmatter.test.cjs (12 tests)
    Loads each fixture, pins parser invariants on extractFrontmatter()
    return shape. Cross-corpus "does not throw on any fixture" sweep.

  tests/feat-3594-parser-adversarial-roadmap.test.cjs (18 tests)
    Loads each fixture into a temp project's .planning/ROADMAP.md and
    drives `gsd-tools roadmap get-phase <N>` via the runCli harness
    introduced by #3593. Asserts on the typed JSON payload.

  tests/feat-3594-parser-property-style.test.cjs (2 tests)
    Deterministic mulberry32 PRNG generates 500 malformed-ish
    frontmatter inputs per test. Pins (a) extractFrontmatter is total
    over the corpus (no null-deref TypeError, always returns a plain
    object on success), (b) the suite completes well under 2 seconds
    (quadratic-regression guard).

Known-open bugs surfaced and pinned (intentionally NOT fixed in this
PR — separate issues warranted):

  - CJS roadmap parser matches `## Phase N:` headings inside fenced
    code blocks (the SDK parser tracks fences per the #2787 comment in
    sdk/src/query/roadmap.ts but the CJS path has not caught up).
  - CJS roadmap parser matches `## Phase N:` headings inside HTML
    comments.

Both are documented in-test with the "currently STILL matches it
(open: needs <fix>)" naming pattern so the day the production fix
lands, flipping the assertion from `found: true` to `found: false` is
the regression guard.

Test totals:
  - 32 new feat-3594-* tests (12 frontmatter + 18 roadmap + 2 property)
  - 108/108 pass when running together with the pre-existing
    frontmatter.test.cjs + roadmap.test.cjs suites (76 of theirs).

Closes #3594

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(3594): use Fisher-Yates shuffle for deterministic seeded inputs

Replaces `arr.sort(() => rng() - 0.5)` with a Fisher-Yates shuffle
driven by the supplied PRNG. The sort-based shuffle is non-transitive:
V8's TimSort behavior on non-transitive comparators is engine-defined,
so the same seed produced different orderings across Node versions —
undermining the test's stated reproducibility guarantee.

Fisher-Yates is O(n), transitive (no comparator at all), and consumes
exactly n-1 RNG values in a fixed order. The mulberry32 seed now
determines the input sequence end-to-end.

Codex review on PR #3633.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-05-16 00:23:27 -04:00
committed by GitHub
parent 5583b81f35
commit b1317633db
18 changed files with 2598 additions and 0 deletions

View File

@@ -0,0 +1,31 @@
# Adversarial Frontmatter Fixtures (#3594)
Reusable hostile inputs for `get-shit-done/bin/lib/frontmatter.cjs`
`extractFrontmatter()` and downstream consumers.
Each fixture is a single markdown file whose name encodes the abuse
category. Tests load them by relative path so the corpus can grow
without test code changes. Adding a new fixture means: drop the file
in here, add an entry to the matrix in
`tests/feat-3594-parser-adversarial-frontmatter.test.cjs`, decide
what invariant the parser must satisfy (typically "does not throw,
does not return half-parsed garbage, does not silently lose data").
Categories present:
- `duplicate-keys.md` — same key appears twice. Parser must produce
a deterministic result; tests document which value wins (last-wins
is the current behavior).
- `crlf-mixed.md` — CRLF endings throughout the frontmatter block.
Parser must handle the `\r` consistently and not bleed it into
values.
- `unclosed-block.md` — opening `---` with no closing `---`. Parser
must return empty frontmatter (or a clean error), never partial.
- `unicode-keys-and-values.md` — non-ASCII keys/values. Parser must
round-trip them as-is.
- `null-byte-value.md` — value contains a U+0000 null. Parser must
preserve or normalize it; must not crash and must not truncate
silently.
- `huge-bounded.md` — a deliberately-large but bounded frontmatter
block (~64KB of array items). Parser must complete in reasonable
time with a typed result, not OOM or hang.

View File

@@ -0,0 +1,9 @@
---
title: CRLF Title
phase: 02
plans:
- 02-01
- 02-02
---
Body with mixed CRLF endings.

View File

@@ -0,0 +1,13 @@
---
title: First
title: Second
status: active
status: blocked
phase: 01
---
Body content for duplicate-keys fixture.
When the parser encounters a key twice in the same block, the test pins
what is currently observed (last-wins) so a silent semantics change
becomes a test failure rather than a quiet data shift.

File diff suppressed because it is too large Load Diff

Binary file not shown.

View File

@@ -0,0 +1,6 @@
---
title: Unclosed Block
phase: 03
This frontmatter has no closing --- line, and the body starts here without
a delimiter. Parser must NOT consume body lines as frontmatter keys.

View File

@@ -0,0 +1,8 @@
---
title: 日本語のタイトル
相: 04
status: 🚧 in-flight
tags: [α, β, γ]
---
Body.

View File

@@ -0,0 +1,28 @@
# Adversarial Roadmap Fixtures (#3594)
Hostile / messy ROADMAP.md inputs for
`get-shit-done/bin/lib/roadmap.cjs` (`searchPhaseInContent`,
`cmdRoadmapGetPhase`, `cmdRoadmapAnalyze`) and the SDK roadmap parser
in `sdk/src/query/roadmap.ts`.
Tests in `tests/feat-3594-parser-adversarial-roadmap.test.cjs` and
`tests/feat-3594-parser-property-style.test.cjs` consume these.
Categories:
- `phase-heading-inside-fenced-code.md` — a ``` md ``` block contains
a `## Phase 999: fake` heading. The parser MUST ignore headings
inside fenced code blocks. Historical regression #2787.
- `nested-fenced-code.md` — outer ``` ``` ``` ``` ` block with inner
``` ``` ``` ``` ` block. Headings inside either layer must be
ignored.
- `unicode-phase-titles.md` — phase titles with non-ASCII characters.
Parser must preserve them in the returned `phase_name`.
- `repeated-phase-ids.md` — same integer phase number listed twice.
Parser behavior must be deterministic (first-wins or last-wins is
fine; the test pins whichever).
- `decimal-phase-mixed.md` — integer phase 2 and decimal phase 2.1
share a prefix. Parser must not return phase 2 when asked for 2.1
(or vice versa). Historical regression #3537.
- `markdown-headings-inside-html-comment.md` — `<!-- ## Phase 999
--> ` patterns. Parser must not be fooled by comments.

View File

@@ -0,0 +1,21 @@
# Roadmap
## Phase 2: integer phase two
**Goal:** the integer parent.
**Requirements:** [R2]
## Phase 2.1: decimal child of phase two
**Goal:** decimal subphase. Must not be returned when caller asks for phase 2.
**Requirements:** [R2.1]
## Phase 2.10: decimal phase 2.10
**Goal:** prefix-collision guard — "2.10" must NOT be matched by a "2.1" lookup.
**Requirements:** [R2.10]
## Phase 21: phase twenty-one
**Goal:** prefix-collision guard — "21" must NOT be matched by a "2" lookup.
**Requirements:** [R21]

View File

@@ -0,0 +1,14 @@
# Roadmap
<!--
## Phase 999: this heading is inside an HTML comment
The parser must NOT treat commented-out headings as real phases.
-->
## Phase 1: real phase
**Goal:** the only real phase.
**Requirements:** [R1]
<!-- ## Phase 998: single-line comment with a heading -->

View File

@@ -0,0 +1,20 @@
# Roadmap
````md
Outer block opened with four backticks.
```md
## Phase 998: nested fake phase
Inner block opened with three backticks.
```
## Phase 997: outer fake phase
Still inside the outer four-tick block — the three-tick close above
does NOT terminate it.
````
## Phase 1: real phase
**Goal:** the only real phase in this file.

View File

@@ -0,0 +1,18 @@
# Roadmap
Some intro prose.
```md
## Phase 999: this heading is inside a fenced code block
It must NOT be parsed as a real phase. Historical regression #2787.
```
## Phase 1: real phase one
**Goal:** Ship something.
**Requirements:** [R1]
## Phase 2: real phase two
**Goal:** Ship something else.

View File

@@ -0,0 +1,16 @@
# Roadmap
## Phase 1: first declaration of phase 1
**Goal:** the original.
**Requirements:** [R1]
## Phase 2: phase two
**Goal:** routine.
**Requirements:** [R2]
## Phase 1: duplicate declaration of phase 1
**Goal:** this should not silently shadow the first one.
**Requirements:** [R3]

View File

@@ -0,0 +1,15 @@
# Roadmap
## Phase 1: 日本語フェーズ — initial setup
**Goal:** ensure non-ASCII titles round-trip.
**Requirements:** [R1]
## Phase 2: 🚧 Émile's "phase" with smart quotes
**Goal:** mixed Unicode + ASCII + smart quotes survive parsing.
**Requirements:** [R2]
## Phase 3: αβγ δεζ ηθι
**Goal:** Greek letters in the title.