Final convergence review found the shared <tag> seam introduced 3 behavior
regressions; fixed all + locked with tests:
- #557 REGRESSION: stripTaggedBlocks's attribute-tolerance stripped `<details open>`
(the ACTIVE-milestone marker) that the old `<details>`-only regex preserved. The
seam now takes `allowAttributes` (default false) — details/decisions strip is
attr-INTOLERANT (preserves `<details open>`); only `<task type="…">` opts in.
Regression test added to roadmap-parser + markdown-sectionizer suites.
- verify.cts actionZones (negative-grep-echo security scan): reverted to a bounded
to-first-close scan `<action>([\s\S]{0,20000}?)</action>` so a grep-echo trick
can't hide behind an unterminated inner <action> (the seam's stop-at-next-open
would drop it). ReDoS-safe via the cap.
- check-command-router HTML-comment strip: `(?:-->|$)` fallback wiped to EOF
(fail-closed spurious gate block) — replaced with stop-at-next-open so an
unclosed `<!--` leaves downstream tags intact.
- Updated the extractTaggedBlocks nested-tag tests to the new (stop-at-next-open)
behavior: `<x><x>inner</x></x>` -> ['inner'].
All vectors still linear; every fix verified in-process.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A convergence audit showed the `<tag>[\s\S]*?</tag>` lazy-scan ReDoS was pervasive
(a dozen+ bespoke copies across roadmap-parser/check-command-router/verify), each
a distinct quadratic vector on a large document with unclosed tags. Rather than
whack-a-mole, single-source them (maintainer-directed):
- markdown-sectionizer: extractTaggedBlocks now shares one ReDoS-safe
`taggedBlockPattern` (stop-at-next-open, bounded optional attributes) and gains
a `stripTaggedBlocks` companion for block removal.
- roadmap-parser: 3 `<details>` strips -> stripTaggedBlocks (behavior-identical —
no <details> here carries attributes).
- verify: actionZones + both <task> loops + their nested <name>/<files>/gate/req
extractions -> extractTaggedBlocks (behavior byte-equivalent, verified).
- check-command-router: the objective|tasks?|action alternation hardened in place
(distinct multi-tag shape); HTML-comment strip gains a `$` fallback.
Every vector now linear (<3ms on 1.5MB adversarial); real content unchanged
(end-to-end verify/roadmap resolution + task extraction confirmed). The only
remaining `<!--…-->` scan (uat.cts:201) is anchored + non-global — one scan, safe.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A ReDoS-completeness audit surfaced a distinct class beyond the tag/bracket
clause: unbounded `[\s\S]*?` / `[^\]]*` lazy-scans searching for a literal
terminator that may never appear, driven quadratic by REPEATED structures in a
large PLAN.md/ROADMAP.md. Folded all 7 in at maintainer direction:
- files_modified `[^\]]*` -> `[^\]]{0,8000}` (commands.cts, verify.cts): 39.7s -> 0.9s.
- Plans-count `[\s\S]*?` -> section-local `(?:(?!\n#{1,4}\s)[\s\S])*?` — stops at the
next heading (semantically correct: Plans: belongs to the phase's own section)
(roadmap.cts x3, phase.cts): 36s -> 4ms.
- <tag> extraction `([\s\S]*?)` -> stop at the next same-tag opening
`((?:(?!<tag>)[\s\S])*?)` (verify.cts x3, markdown-sectionizer.cts): ~6s -> 2ms.
Every vector is now linear (comprehensively re-measured); real content matches
(end-to-end `roadmap get-phase` still resolves Plans-counted phases). Pre-existing;
byte-behavior preserved for realistic inputs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(#1373): add markdown-sectionizer seam (ADR-1372 T0)
Establishes the canonical markdown-structure parsing seam per ADR-1372.
No existing parsers are modified; this is the foundational T0 tier only.
- docs/adr/1372-markdown-sectionizer-seam.md: Accepted ADR defining the
seam interface, the tiered migration plan (T0-T7), and the prohibition
enforcement approach (no-adhoc-markdown-parsing ESLint rule in T7).
- src/markdown-sectionizer.cts: Pure module, Node built-ins only.
Exports: stripFencedCode (CommonMark-correct state machine ported from
uat-predicate.cts _stripFencedBlocks, CRLF-safe, unterminatedFence
signal), tokenizeHeadings (ATX headings outside fenced blocks),
collectSections (line-by-line predicate-driven section collection),
collectSection (single named section, levelBounded stop, optional
stripFences), iterateBullets (dash/checkbox/numbered + continuation).
- tests/markdown-sectionizer.test.cjs: 54-test behavioral suite covering
the parser QA matrix (LF/CRLF, Unicode headings, headings-inside-fences,
unterminated fences, nested levels, all bullet markers, continuation
lines, empty/non-string input) plus 4 fast-check property tests
(idempotence, output shape, never-throws, length monotonicity).
- CONTEXT.md: Markdown Sectionizer glossary entry added (PR review gate).
Tests: 54 pass, 0 fail. Existing adr-parser + uat-passed tests: 22 pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(#1373): add extractTaggedBlocks + replaceSection to seam; register inventory
- src/markdown-sectionizer.cts: extend Section type with bodyStart/bodyEnd offsets;
add extractTaggedBlocks(content, tagName) (inner text of <tag>…</tag> blocks,
tagName regex-escaped, caller decides fence-stripping) and replaceSection(content,
section, newBody) (pure character-offset splice for read-modify-write callers);
update collectSections/collectSection to populate bodyStart/bodyEnd.
- tests/markdown-sectionizer.test.cjs: add 33 new behavioral tests for
extractTaggedBlocks, replaceSection, and a DEFECT.GENERATIVE-FIX parity guard
that asserts stripFencedCode and uat-predicate's _stripFencedBlocks agree on a
shared 9-item corpus; documents the known 4-space-indent divergence.
- docs/adr/1372-markdown-sectionizer-seam.md: list extractTaggedBlocks and
replaceSection in §"The seam".
- CONTEXT.md: update ### Markdown Sectionizer glossary entry with the two new exports.
- docs/INVENTORY.md: add markdown-sectionizer.cjs row (alphabetically between
loop-resolver and milestone).
- docs/INVENTORY-MANIFEST.json: regenerated via gen-inventory-manifest --write.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(#1373): clear no-unsafe-assignment + unused-var lint in markdown-sectionizer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(#1373): correct section offset/round-trip + CommonMark heading/fence edges; register eslint coverage
FIX 1 (CRITICAL): Enforce content.slice(bodyStart,bodyEnd) === body invariant in both
collectSection and collectSections. bodyEnd is now bodyStart + body.length instead of
the raw stop-line offset, eliminating the trailing-newline overcounting that caused
replaceSection to drop separator newlines (## A\nbody## B gluing bug).
FIX 2 (MED): tokenizeHeadings now accepts ≤3-space indent (CommonMark §4.5) and empty
ATX headings (## / ## ), text=''. 4-space indent correctly excluded.
FIX 3 (MED): collectSection gains stopAtLevel option — stops at the next heading whose
level ≤ stopAtLevel, independent of the opener's level. Enables state.cts ## sections
that also stop at ### without abusing levelBounded.
FIX 4 (MED): Backtick fence opener info string must not contain a backtick (CommonMark).
Applied in both stripFencedCode and tokenizeHeadings fence state machines. Tilde fences
unaffected.
FIX 5 (LOW): "byte offset" → "character (string-index) offset" in HeadingToken / Section
doc comments.
FIX 6 (LOW): extractTaggedBlocks doc comment documents nested-tag non-support; test locks
the non-greedy close-at-first-</tag> behavior.
FIX 7: Add gsd-core/bin/lib/markdown-sectionizer.cjs to eslint.config.mjs ignores so
tests/551-eslint-bin-lib-coverage.test.cjs passes (3/3).
Tests: 107 pass / 0 fail (was 87; +20 new tests for FIX 1–4, 6).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(#1373): gitignore tsc-built markdown-sectionizer.cjs (ADR-457 build-at-publish)
The seam's compiled artifact must be a build-at-publish output like every other
src/*.cts->bin/lib/*.cjs module (decisions, core, state, ...), not a committed
file. Add it to the ADR-457 ignore list and untrack it; build:lib/CI regenerate it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
