test(3594): adversarial parser fixtures + frontmatter/roadmap matrix + property-style suite (#3633)
* test(3594): adversarial parser fixtures + frontmatter/roadmap matrix + property-style suite
Lands the adversarial parser-input corpus that CONTRIBUTING.md
§"QA Matrix Requirements / Parser and project-file inputs" and
TEST-EXAMPLES.md §"Parser Adversarial Fixtures" describe.
New tests/fixtures/adversarial/ layout:
frontmatter/
duplicate-keys.md — same key twice (collapses last-wins)
crlf-mixed.md — CRLF endings throughout
unclosed-block.md — `---` open with no close
unicode-keys-and-values.md — non-ASCII + emoji + Greek
null-byte-value.md — U+0000 in a value
huge-bounded.md — 2000-item array, ~30KB
roadmap/
phase-heading-inside-fenced-code.md — #2787 fence shadowing
nested-fenced-code.md — outer + inner ``` blocks
unicode-phase-titles.md — JP / Greek / emoji titles
repeated-phase-ids.md — phase 1 declared twice
decimal-phase-mixed.md — 2 vs 2.1 vs 2.10 vs 21
markdown-headings-inside-html-comment.md — comment shadowing
Test files (all node:test, no try/finally in test bodies, no source-grep,
no raw-text matching on stdout/file content):
tests/feat-3594-parser-adversarial-frontmatter.test.cjs (12 tests)
Loads each fixture, pins parser invariants on extractFrontmatter()
return shape. Cross-corpus "does not throw on any fixture" sweep.
tests/feat-3594-parser-adversarial-roadmap.test.cjs (18 tests)
Loads each fixture into a temp project's .planning/ROADMAP.md and
drives `gsd-tools roadmap get-phase <N>` via the runCli harness
introduced by #3593. Asserts on the typed JSON payload.
tests/feat-3594-parser-property-style.test.cjs (2 tests)
Deterministic mulberry32 PRNG generates 500 malformed-ish
frontmatter inputs per test. Pins (a) extractFrontmatter is total
over the corpus (no null-deref TypeError, always returns a plain
object on success), (b) the suite completes well under 2 seconds
(quadratic-regression guard).
Known-open bugs surfaced and pinned (intentionally NOT fixed in this
PR — separate issues warranted):
- CJS roadmap parser matches `## Phase N:` headings inside fenced
code blocks (the SDK parser tracks fences per the #2787 comment in
sdk/src/query/roadmap.ts but the CJS path has not caught up).
- CJS roadmap parser matches `## Phase N:` headings inside HTML
comments.
Both are documented in-test with the "currently STILL matches it
(open: needs <fix>)" naming pattern so the day the production fix
lands, flipping the assertion from `found: true` to `found: false` is
the regression guard.
Test totals:
- 32 new feat-3594-* tests (12 frontmatter + 18 roadmap + 2 property)
- 108/108 pass when running together with the pre-existing
frontmatter.test.cjs + roadmap.test.cjs suites (76 of theirs).
Closes #3594
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(3594): use Fisher-Yates shuffle for deterministic seeded inputs
Replaces `arr.sort(() => rng() - 0.5)` with a Fisher-Yates shuffle
driven by the supplied PRNG. The sort-based shuffle is non-transitive:
V8's TimSort behavior on non-transitive comparators is engine-defined,
so the same seed produced different orderings across Node versions —
undermining the test's stated reproducibility guarantee.
Fisher-Yates is O(n), transitive (no comparator at all), and consumes
exactly n-1 RNG values in a fixed order. The mulberry32 seed now
determines the input sequence end-to-end.
Codex review on PR #3633.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
31
tests/fixtures/adversarial/frontmatter/README.md
vendored
Normal file
31
tests/fixtures/adversarial/frontmatter/README.md
vendored
Normal file
@@ -0,0 +1,31 @@
|
||||
# Adversarial Frontmatter Fixtures (#3594)
|
||||
|
||||
Reusable hostile inputs for `get-shit-done/bin/lib/frontmatter.cjs`
|
||||
`extractFrontmatter()` and downstream consumers.
|
||||
|
||||
Each fixture is a single markdown file whose name encodes the abuse
|
||||
category. Tests load them by relative path so the corpus can grow
|
||||
without test code changes. Adding a new fixture means: drop the file
|
||||
in here, add an entry to the matrix in
|
||||
`tests/feat-3594-parser-adversarial-frontmatter.test.cjs`, decide
|
||||
what invariant the parser must satisfy (typically "does not throw,
|
||||
does not return half-parsed garbage, does not silently lose data").
|
||||
|
||||
Categories present:
|
||||
|
||||
- `duplicate-keys.md` — same key appears twice. Parser must produce
|
||||
a deterministic result; tests document which value wins (last-wins
|
||||
is the current behavior).
|
||||
- `crlf-mixed.md` — CRLF endings throughout the frontmatter block.
|
||||
Parser must handle the `\r` consistently and not bleed it into
|
||||
values.
|
||||
- `unclosed-block.md` — opening `---` with no closing `---`. Parser
|
||||
must return empty frontmatter (or a clean error), never partial.
|
||||
- `unicode-keys-and-values.md` — non-ASCII keys/values. Parser must
|
||||
round-trip them as-is.
|
||||
- `null-byte-value.md` — value contains a U+0000 null. Parser must
|
||||
preserve or normalize it; must not crash and must not truncate
|
||||
silently.
|
||||
- `huge-bounded.md` — a deliberately-large but bounded frontmatter
|
||||
block (~64KB of array items). Parser must complete in reasonable
|
||||
time with a typed result, not OOM or hang.
|
||||
9
tests/fixtures/adversarial/frontmatter/crlf-mixed.md
vendored
Normal file
9
tests/fixtures/adversarial/frontmatter/crlf-mixed.md
vendored
Normal file
@@ -0,0 +1,9 @@
|
||||
---
|
||||
title: CRLF Title
|
||||
phase: 02
|
||||
plans:
|
||||
- 02-01
|
||||
- 02-02
|
||||
---
|
||||
|
||||
Body with mixed CRLF endings.
|
||||
13
tests/fixtures/adversarial/frontmatter/duplicate-keys.md
vendored
Normal file
13
tests/fixtures/adversarial/frontmatter/duplicate-keys.md
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
---
|
||||
title: First
|
||||
title: Second
|
||||
status: active
|
||||
status: blocked
|
||||
phase: 01
|
||||
---
|
||||
|
||||
Body content for duplicate-keys fixture.
|
||||
|
||||
When the parser encounters a key twice in the same block, the test pins
|
||||
what is currently observed (last-wins) so a silent semantics change
|
||||
becomes a test failure rather than a quiet data shift.
|
||||
2006
tests/fixtures/adversarial/frontmatter/huge-bounded.md
vendored
Normal file
2006
tests/fixtures/adversarial/frontmatter/huge-bounded.md
vendored
Normal file
File diff suppressed because it is too large
Load Diff
BIN
tests/fixtures/adversarial/frontmatter/null-byte-value.md
vendored
Normal file
BIN
tests/fixtures/adversarial/frontmatter/null-byte-value.md
vendored
Normal file
Binary file not shown.
6
tests/fixtures/adversarial/frontmatter/unclosed-block.md
vendored
Normal file
6
tests/fixtures/adversarial/frontmatter/unclosed-block.md
vendored
Normal file
@@ -0,0 +1,6 @@
|
||||
---
|
||||
title: Unclosed Block
|
||||
phase: 03
|
||||
|
||||
This frontmatter has no closing --- line, and the body starts here without
|
||||
a delimiter. Parser must NOT consume body lines as frontmatter keys.
|
||||
8
tests/fixtures/adversarial/frontmatter/unicode-keys-and-values.md
vendored
Normal file
8
tests/fixtures/adversarial/frontmatter/unicode-keys-and-values.md
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
---
|
||||
title: 日本語のタイトル
|
||||
相: 04
|
||||
status: 🚧 in-flight
|
||||
tags: [α, β, γ]
|
||||
---
|
||||
|
||||
Body.
|
||||
28
tests/fixtures/adversarial/roadmap/README.md
vendored
Normal file
28
tests/fixtures/adversarial/roadmap/README.md
vendored
Normal file
@@ -0,0 +1,28 @@
|
||||
# Adversarial Roadmap Fixtures (#3594)
|
||||
|
||||
Hostile / messy ROADMAP.md inputs for
|
||||
`get-shit-done/bin/lib/roadmap.cjs` (`searchPhaseInContent`,
|
||||
`cmdRoadmapGetPhase`, `cmdRoadmapAnalyze`) and the SDK roadmap parser
|
||||
in `sdk/src/query/roadmap.ts`.
|
||||
|
||||
Tests in `tests/feat-3594-parser-adversarial-roadmap.test.cjs` and
|
||||
`tests/feat-3594-parser-property-style.test.cjs` consume these.
|
||||
|
||||
Categories:
|
||||
|
||||
- `phase-heading-inside-fenced-code.md` — a ``` md ``` block contains
|
||||
a `## Phase 999: fake` heading. The parser MUST ignore headings
|
||||
inside fenced code blocks. Historical regression #2787.
|
||||
- `nested-fenced-code.md` — outer ``` ``` ``` ``` ` block with inner
|
||||
``` ``` ``` ``` ` block. Headings inside either layer must be
|
||||
ignored.
|
||||
- `unicode-phase-titles.md` — phase titles with non-ASCII characters.
|
||||
Parser must preserve them in the returned `phase_name`.
|
||||
- `repeated-phase-ids.md` — same integer phase number listed twice.
|
||||
Parser behavior must be deterministic (first-wins or last-wins is
|
||||
fine; the test pins whichever).
|
||||
- `decimal-phase-mixed.md` — integer phase 2 and decimal phase 2.1
|
||||
share a prefix. Parser must not return phase 2 when asked for 2.1
|
||||
(or vice versa). Historical regression #3537.
|
||||
- `markdown-headings-inside-html-comment.md` — `<!-- ## Phase 999
|
||||
--> ` patterns. Parser must not be fooled by comments.
|
||||
21
tests/fixtures/adversarial/roadmap/decimal-phase-mixed.md
vendored
Normal file
21
tests/fixtures/adversarial/roadmap/decimal-phase-mixed.md
vendored
Normal file
@@ -0,0 +1,21 @@
|
||||
# Roadmap
|
||||
|
||||
## Phase 2: integer phase two
|
||||
|
||||
**Goal:** the integer parent.
|
||||
**Requirements:** [R2]
|
||||
|
||||
## Phase 2.1: decimal child of phase two
|
||||
|
||||
**Goal:** decimal subphase. Must not be returned when caller asks for phase 2.
|
||||
**Requirements:** [R2.1]
|
||||
|
||||
## Phase 2.10: decimal phase 2.10
|
||||
|
||||
**Goal:** prefix-collision guard — "2.10" must NOT be matched by a "2.1" lookup.
|
||||
**Requirements:** [R2.10]
|
||||
|
||||
## Phase 21: phase twenty-one
|
||||
|
||||
**Goal:** prefix-collision guard — "21" must NOT be matched by a "2" lookup.
|
||||
**Requirements:** [R21]
|
||||
14
tests/fixtures/adversarial/roadmap/markdown-headings-inside-html-comment.md
vendored
Normal file
14
tests/fixtures/adversarial/roadmap/markdown-headings-inside-html-comment.md
vendored
Normal file
@@ -0,0 +1,14 @@
|
||||
# Roadmap
|
||||
|
||||
<!--
|
||||
## Phase 999: this heading is inside an HTML comment
|
||||
|
||||
The parser must NOT treat commented-out headings as real phases.
|
||||
-->
|
||||
|
||||
## Phase 1: real phase
|
||||
|
||||
**Goal:** the only real phase.
|
||||
**Requirements:** [R1]
|
||||
|
||||
<!-- ## Phase 998: single-line comment with a heading -->
|
||||
20
tests/fixtures/adversarial/roadmap/nested-fenced-code.md
vendored
Normal file
20
tests/fixtures/adversarial/roadmap/nested-fenced-code.md
vendored
Normal file
@@ -0,0 +1,20 @@
|
||||
# Roadmap
|
||||
|
||||
````md
|
||||
Outer block opened with four backticks.
|
||||
|
||||
```md
|
||||
## Phase 998: nested fake phase
|
||||
|
||||
Inner block opened with three backticks.
|
||||
```
|
||||
|
||||
## Phase 997: outer fake phase
|
||||
|
||||
Still inside the outer four-tick block — the three-tick close above
|
||||
does NOT terminate it.
|
||||
````
|
||||
|
||||
## Phase 1: real phase
|
||||
|
||||
**Goal:** the only real phase in this file.
|
||||
18
tests/fixtures/adversarial/roadmap/phase-heading-inside-fenced-code.md
vendored
Normal file
18
tests/fixtures/adversarial/roadmap/phase-heading-inside-fenced-code.md
vendored
Normal file
@@ -0,0 +1,18 @@
|
||||
# Roadmap
|
||||
|
||||
Some intro prose.
|
||||
|
||||
```md
|
||||
## Phase 999: this heading is inside a fenced code block
|
||||
|
||||
It must NOT be parsed as a real phase. Historical regression #2787.
|
||||
```
|
||||
|
||||
## Phase 1: real phase one
|
||||
|
||||
**Goal:** Ship something.
|
||||
**Requirements:** [R1]
|
||||
|
||||
## Phase 2: real phase two
|
||||
|
||||
**Goal:** Ship something else.
|
||||
16
tests/fixtures/adversarial/roadmap/repeated-phase-ids.md
vendored
Normal file
16
tests/fixtures/adversarial/roadmap/repeated-phase-ids.md
vendored
Normal file
@@ -0,0 +1,16 @@
|
||||
# Roadmap
|
||||
|
||||
## Phase 1: first declaration of phase 1
|
||||
|
||||
**Goal:** the original.
|
||||
**Requirements:** [R1]
|
||||
|
||||
## Phase 2: phase two
|
||||
|
||||
**Goal:** routine.
|
||||
**Requirements:** [R2]
|
||||
|
||||
## Phase 1: duplicate declaration of phase 1
|
||||
|
||||
**Goal:** this should not silently shadow the first one.
|
||||
**Requirements:** [R3]
|
||||
15
tests/fixtures/adversarial/roadmap/unicode-phase-titles.md
vendored
Normal file
15
tests/fixtures/adversarial/roadmap/unicode-phase-titles.md
vendored
Normal file
@@ -0,0 +1,15 @@
|
||||
# Roadmap
|
||||
|
||||
## Phase 1: 日本語フェーズ — initial setup
|
||||
|
||||
**Goal:** ensure non-ASCII titles round-trip.
|
||||
**Requirements:** [R1]
|
||||
|
||||
## Phase 2: 🚧 Émile's "phase" with smart quotes
|
||||
|
||||
**Goal:** mixed Unicode + ASCII + smart quotes survive parsing.
|
||||
**Requirements:** [R2]
|
||||
|
||||
## Phase 3: αβγ δεζ ηθι
|
||||
|
||||
**Goal:** Greek letters in the title.
|
||||
Reference in New Issue
Block a user