bfcc7b2acb71ccd7a996800c331df20d6d09b839
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tom Boucher
|
6eea00b707 |
enhance(#3301): tell reviewers the plan ids and total count, grade coverage (#4084)
* test(#3301): add failing-first plan coverage manifest tests Failing-first regression tests for the plan-id manifest, the updated Review Instructions, and the mechanical per-reviewer coverage check, ahead of the review.md implementation. RED baseline before the fix lands. * test(#3301): raise allow-test-rule-refs unverified ceiling for new marker Adding tests/review-plan-coverage-manifest.test.cjs's source-text-is-the-product marker grows the unverified-exemption pool by one (282 -> 283), the same documented growth path scripts/lint-allow-test-rule-refs.cjs's own failure output names. Confirmed clean via 'npm run lint:allow-test-rule-refs' locally. * feat(#3301): tell reviewers the plan ids and total count, grade coverage build_prompt now derives a plan-id manifest from each *-PLAN.md filename (stripping the -PLAN.md suffix) and appends it, with the total plan count, to both gsd-review-instructions.md and gsd-review-prompt.md. The Review Instructions prose requires one heading-verbatim section per id before any cross-plan or overall-risk content. write_reviews grades each dispatched lane's real (non-stub, non-empty) review against that same manifest and records an optional plan_coverage: frontmatter block, present only when a lane is incomplete. The match escapes regex metacharacters in the id and excludes a preceding/trailing hyphen or word character as a boundary, closing the two traps named in the issue (a decimal phase like 12.6 satisfied by 12X6-01; a threat id like T-04-07 registering as coverage of plan 04-07). CodeRabbit is exempt, since it never receives the source-grounding prompt carrying the manifest. This closes the gap where a review that silently covers only some plans in a multi-plan phase is indistinguishable from one that covers all of them. * docs(#3301): add changeset fragment * test(#3301): use t.after() instead of try/finally for cleanup CONTRIBUTING.md bans try/finally inside test bodies. Code review caught this in the new coverage-manifest test file; switch every fixture-cleanup site to the approved t.after() pattern. * test: use t.after() instead of try/finally in #3300's build_prompt tests Pre-existing try/finally-for-cleanup pattern in this file (landed for #3300) violates CONTRIBUTING.md's explicit ban on try/finally inside test bodies. Surfaced incidentally while reviewing #3301's diff, which cites this file as its extraction-pattern precedent; fixed inline per the no-defer rule rather than deferred to a separate PR. * test(#3301): anchor coverage-check extraction on the fence line, not prose `.plans-manifest.md` also appears in write_reviews' own prose ahead of the ```bash fence, so indexOf found that occurrence first and the backward-walk-to-fence-open landed on the earlier, unrelated gate-check block instead. gsd-test caught this: coverage-check tests expecting a real verdict got null, because the wrong block ran and never writes .plan-coverage-<slug>.json. Anchor on the fence-only bash assignment line instead. Emitted-Drift-Ack-Growth: review.md — #3301 adds the plan-coverage manifest and mechanical coverage check to build_prompt/write_reviews. * fix(#3301): route id escaping through the canonical pattern seam ADR-3212 (epic #3212) consolidated ~44 hand-rolled regex-escape copies into one owner, src/pattern.cts's escapeRegex, specifically to stop this exact class of duplication. My coverage-check node -e script hand-rolled the identical metachar-escape regex — invisible to eslint-rules/no-adhoc-regex-escape.cjs only because it lives inside a workflow markdown file, not a .cts/.cjs source file the shape-matching guard scans. Require the compiled seam (gsd-core/bin/lib/pattern.cjs) instead, matching the established node -e-requires-a-compiled-lib idiom already used elsewhere in this workflow (code-review.md's code-review-flags.cjs/code-review-depth.cjs calls). Verified both named traps from the issue still resolve correctly under escapeRegex's RegExp.escape-backed implementation, which differs in escaped-text shape (hex-escapes hyphens/leading chars) but not match result. * test(#3301): run coverage-check block with cwd at the repo root The block's node -e now requires ./gsd-core/bin/lib/pattern.cjs, a path relative to the repo root (correct for production, which always runs from there). The test harness ran it with cwd at the fixture's own temp dir instead, so the require failed. Add an optional cwd param to runScript (default: root, unchanged for the plan-copy-block tests) and pass the real repo root for every coverage-check call site. Manually verified end-to-end before spending another remote run: the extracted block now produces the expected {complete:true} verdict. * docs(#3301): backfill changeset pr number (pr:0 -> pr:4084) --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
fd63889d1f |
fix(#3854): write normalization preserves tight multi-line lists (#4049)
* test(#3854): write normalization must preserve tight multi-line lists (failing first) * fix(#3854): no blank before a bullet whose previous line is an indented continuation _normalizeMd's 'separate a list from a preceding paragraph' rule inserted a blank before any bullet whose previous line wasn't a bullet — but an INDENTED CONTINUATION of the previous multi-line item also isn't a bullet. Every .md write (phase.complete in the report, but any write through platformWriteSync) therefore converted tight lists to loose ones: +61 blank lines on the reporter's 1015-line ROADMAP, one before each bullet following a wrapped item. Tight and loose lists render differently, so this was a rendering change plus misleading diff noise; one-shot (idempotent afterwards), which is why integrity checks on headings/content passed. The guard is the mirror image of the after-a-bullet rule two lines below, which already excludes indented next lines. Paragraph→list and heading→list separations — the rule's purpose — are pinned unchanged by the new suite. * fix(#3854): review fold-ins — ceiling tracks next's 281 + this branch's marker (282), header/require nits The ceiling is not ratcheted but must track the tree: origin/next raised it to 281 (sibling branch's marker file); this tree adds one more (shell-command-projection-md-normalize), so 282/282. Also fixes the test header's stale pre-rename filename and hoists the inline require to the file's single import. * chore(#3854): changeset fragment (pr number backfilled after PR creation) * chore(#3854): backfill changeset PR number (4049) --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
519ac23ebb |
fix(#3839): hook tables say PreToolUse (validate-commit) and SessionStart (session-state) (#4041)
* test(#3839): docs hook tables must match surface registrations (failing first) * docs(#3839): hook tables say PreToolUse for validate-commit, SessionStart for session-state gsd-validate-commit.sh is registered PreToolUse (src/runtime-hooks-surface.cts; its exit-2 block IS the contract — a post-tool hook cannot prevent a commit) and gsd-session-state.sh is registered SessionStart (session orientation, not post-tool tracking). Both rows said PostToolUse in ARCHITECTURE.md and the three INVENTORY locales; the issue asked for a neighbouring-row scan, which is how the session-state row was found. All other rows in the four tables verify against the surface. * fix(#3839): review fold-ins — 10 more wrong rows in ko-KR/pt-BR/zh-CN, parser authority + drift pins Adversarial review found the same two wrong rows shipped in five more files the issue's table missed (ko-KR ARCHITECTURE+INVENTORY, pt-BR ARCHITECTURE+INVENTORY, zh-CN ARCHITECTURE) — all fixed; DOC_TABLES now covers all ten shipped tables. The parity parser unioned only the Kimi mirror list, silently exempting agent-isolation-guard (registered via the dynamic preToolEvent push): probes are now parsed too, with bare hook names resolved against hooks/ ground truth and dynamic event variables resolved to their canonical (non-Gemini) events; an exact-set pin replaces the loose size guard. allow-test-rule marker carries the issue ref; unverified-ceiling 280→281 (audited: the new marker is legitimate — the suite reads product docs whose text is the contract). * fix(#3839): register the hook-table parity suite in the docs-guard lane The new suite reads ten docs/ paths, so lint-docs-guard-registration requires it in the docs-guard registry — the first GREEN bench run caught the omission (the RED run's docs-guard failures were the same signal, previously misread as marker fallout). * chore(#3839): changeset fragment (pr number backfilled after PR creation) * chore(#3839): backfill changeset PR number (4041) --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
bc557f6876 |
chore(#3520): ratchet on effective exemptions, track unverified separately (#3529)
Phase 5 of #3464, following #3465, #3466, #3502 and #3508. Those cut the
ceiling 305 -> 278, made the rule accurate, and ended file-wide amnesty. This
one fixes the number itself.
scripts/lint-allow-test-rule-refs.cjs counted FILES CONTAINING MARKER TEXT.
Only 5 of those files carry a marker that actually suppresses a violation the
rule detects, across 10 sites. The ratcheted number was ~98% noise, which is
exactly why bumping it was frictionless: the metric was never coupled to the
thing it claimed to govern. That is the whole complaint this epic opened with,
stated precisely.
Verified directly rather than assumed: only eslint-rules/no-source-grep.cjs
functionally honors the marker. Four other rule files mention allow-test-rule
in prose only, and no-raw-rmsync-in-tests.cjs:24 explicitly states it does not
apply. So the large count was not legitimately large because several rules
share the annotation.
Now two numbers, only the first ratcheted:
EFFECTIVE EXEMPTIONS -- markers that actually suppress a detected violation.
10 sites across 5 files. Tightly ratcheted in both directions, as before:
over the ceiling fails, and slack beyond grace fails.
UNVERIFIED MARKERS -- marker-bearing files with no detectable violation. 273
files. Reported and given a loose ceiling so the pool cannot silently
balloon, but deliberately NOT tightly ratcheted, because shrinking it is a
rule-coverage problem and not a delete-the-markers problem.
A file with at least one effective site counts as effective and is not also
counted as unverified; the two numbers never double-count.
Reporting ONLY the effective count was considered and rejected. It would say
five files and look excellent while being falsely reassuring, because "no
detectable violation" is not "no violation". This phase's own measurement found
two genuine source-greps that are unsuppressed AND undetected --
tests/security-prompt-injection.security.test.cjs:852 and
tests/check-update-config-dir.test.cjs:91 -- each reading a real shipped file
and text-searching it, invisible only because the path is bound to a separate
const the rule never resolves back to its literal. Markers guarding that class
count as zero-effective and would look vestigial. Trading a number that is too
big and meaningless for one that is too small and falsely reassuring is not
progress, so the script prints the known-limit caveat alongside the numbers and
the two undetected violations are filed separately rather than lost.
Single source of truth is structural, not a matter of discipline. The script
does not re-implement detection or the site-scoped adjacency predicate -- that
is the generative-fix-divergence class this repo has shipped before. The rule
now exports MAX_MARKER_LOOKAHEAD_LINES, MARKER_COMMENT_RE,
collectMarkerAndCommentLines and isSuppressedAt (extracted verbatim, no logic
change), plus a default-off neutralizeSuppression option so the counter can
enumerate every site through the real rule via ESLint's Linter API and then
classify each with the rule's own predicate, replicating reportUnlessSuppressed's
search-line-OR-read-line check exactly. Default rule behavior is byte-identical:
tests/eslint-rules.test.cjs passes 168/168 unchanged. A parity test asserts the
script's suppressed/not verdict equals the rule's own report/no-report outcome
for every site in a fixture corpus.
A real silent-failure bug surfaced and was fixed while building this: ESLint's
flat-config Linter reports "No matching configuration" and returns ZERO messages
for any filename resolving outside its cwd. That would have quietly
misclassified every sandboxed test fixture as having no violations -- a test
suite that passes while asserting nothing. Fixed by anchoring the Linter to the
tests dir, with a defensive throw if it ever recurs.
Two earlier claims of mine are corrected by this phase's measurement. Widening
the source-dir allowlist to include hooks/ -- the "fifth blind spot" recorded in
#3508 -- rescues ZERO sites; it is real in principle and has no practical
effect, because the hooks reads that exist are missed for other reasons
(.sh extension, identifier-indirection, dynamic filenames). And the #3508
correction that attributed those reads to the hooks/ gap rather than to variable
indirection was itself incomplete: both are independently sufficient, so fixing
either alone changes nothing. I accepted the reviewer's causal claim as
uncritically as I had made my own.
The unverified count is 273, not the ~289 in the phase design doc. That is
legitimate drift -- the baseline was measured at
|