f7df920681f233ae0fe064ee659550bdf41ff708
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tom Boucher
|
fb9823e1e1 |
fix(#3689): refuse a ledger write when the rendered table disagrees with its JSON (#3828)
* test(#3689): failing-first coverage for the ledger table/JSON agreement guard `.planning/WINDOWS.md` renders its markdown table from the fenced JSON that is its source of truth, but nothing checks the two still agree before a write overwrites the table. `windows append` / `waive` / `fixed` therefore discard a drifted cell silently, and erase a table-only row entirely, both at exit 0. Adds to tests/broken-windows.test.cjs: - five refusal cases that fail today, covering all three write commands, a drifted cell, a table-only row, and drift on a non-first row; each asserts the typed reason via GSD_JSON_ERRORS and that the file is byte-identical after the refusal, so a guard that refuses only after writing cannot pass - six anti-tightening pins that must stay green: an agreeing ledger, the first-write ENOENT path, #2893 trailing-prose preservation, #3657 3-backtick fence tolerance, escaped pipes and backslashes in a description, and the zero-entry placeholder table - a fast-check property pinning the round trip the guard depends on — extractTableRegion(renderLedger(l)) === renderTable(l.entries) — because a false refusal on a clean ledger would be worse than the bug Fixtures are built by running the real CLI and then perturbing only the table, so frontmatter and JSON stay consistent and the pre-existing counts cross-check still passes; a hand-written ledger would pass these for the wrong reason. Refs #3689 * fix(#3689): refuse a ledger write when the rendered table disagrees with its JSON `.planning/WINDOWS.md` renders its markdown table from the fenced JSON that is its source of truth, and `writeLedgerAtomic` regenerated that table on every `windows append` / `waive` / `fixed` without ever checking the two still agreed. A hand-edited cell was silently reverted; a row that existed only in the table vanished entirely. Both at exit 0, with nothing on stdout to say so. The write seam now compares the on-disk table against `renderTable(<entries parsed from the on-disk JSON>)` before regenerating anything, and refuses with a typed `windows_ledger_table_drift` error naming the drifted row ids and the remedy. Because the check sits at the single write seam, all three commands inherit it, and the file is left byte-identical on refusal. Deliberately not enforced in `parseLedger`: hardening the read would break `windows status` and the ship gate on exactly the ledgers an operator needs to inspect to diagnose the drift. Two hazards handled explicitly, both discovered in review of the first draft: - The pre-image read now distinguishes ENOENT from every other errno, per the #1950-H2 fail-closed-on-unreadable invariant `readLedgerOrNull` already honors. A bare catch would have let an unreadable pre-image skip the guard and write anyway. - Both the entries baseline and the table extraction pass the pre-image's own frontmatter `total_count` to `locateJsonBlock`. Without that hint the no-expectation fallback binds to the LATEST fenced JSON array in the file, which is the operator's prose block whenever that prose contains one — the exact case #2893 exists for — refusing every write on a ledger that never drifted. A regression test covers it. Also extends the CONTEXT.md Broken Windows Ledger glossary entry: the table is a third projection of the same source, cross-checked at the write seam, and the frozen REASON enum gains WINDOWS_LEDGER_TABLE_DRIFT. Fixes #3689 * fix(#3689): bind prose preservation to the pre-image's own ledger block Found while reviewing the table drift guard: the #2893 trailing-prose preservation in `writeLedgerAtomic` passed `ledger.total_count` — the POST-mutation count — as the disambiguation hint for a lookup over the PRE-image. On an append the pre-image holds N entries while the hint says N+1, so the hint can never match and `locateJsonBlock` falls through to its last-array-shaped-span fallback. When the operator's trailing prose itself contains a fenced JSON array — the ordinary case #2893 was written to protect — that prose block wins the fallback. The preserved region is then computed from the prose fence rather than the ledger fence, and everything between them, including the operator's own text above the array, is silently dropped on the next write. Reproduced against the real CLI: a prose block reading "Operator notes above the array, IMPORTANT DO NOT LOSE THIS TEXT." plus a fenced 3-element array came back empty after one `windows append`. Both the prose lookup and the drift guard now share one pre-image-derived `preImageExpectedTotal`, taken from the pre-image's own frontmatter, so they bind to the same and correct block. The existing trailing-prose regression test is strengthened to assert the prose survives byte-for-byte rather than merely that the command exited 0 — asserting only the exit code is why this was invisible. Refs #3689 * fix(#3689): anchor table extraction on the header row, not a line-prefix scan Independent review found the drift guard could brick a ledger nobody had hand-edited. `validateDescription` accepts a description containing a raw newline, and `renderTable`'s cell escaping covers backslash and pipe but not newlines — so such a description renders a row that physically spans two file lines, the second of which does not begin with `|`. `extractTableRegion` bounded the table by walking backward over the contiguous run of `|`-prefixed lines, so it stopped at that split. In the common case where the row's tail is the last line before the fence it returned null, and every subsequent append/waive/fixed was refused with "table region could not be located" — permanently, with no CLI recovery path, on a ledger that never drifted. A false refusal is worse than the bug this guard exists to fix. The region is now anchored on the header row `renderTable` always emits, running from its last line-start occurrence to the end of the pre-fence text. The boundary is the fence rather than a line prefix, so a multi-line row is captured whole, re-renders byte-identically, and compares equal. The header literal is hoisted to one constant both `renderTable` branches and the extractor share, so the two surfaces cannot drift apart. Deliberately unchanged: `cell()` and `validateDescription`. The cosmetic corruption a newline causes in the rendered table is pre-existing, and either escaping it or rejecting the input would change what existing ledgers render to or what input is accepted. Also closes a coverage gap the standards review raised: the non-ENOENT pre-image read branch — the one that stops an unreadable file from bypassing the guard — now has a behavioral test that injects EACCES by monkeypatching `fs.readFileSync` for that one path and restoring it in a `finally`, never by `chmod 0o000` (root ignores mode bits, so that would pass with zero coverage). The #3689 property generator no longer strips newlines out of descriptions, which is why this was invisible to it. Refs #3689 * chore(changeset): backfill PR number for #3689 fragment * chore(changeset): backfill PR number for #3689 fragment * fix(#3689): terminate the header scan when the match sits at index 0 `extractTableRegion`'s backward search for the table header could loop forever. On a rejected match at index 0 it set `searchFrom = idx - 1`, i.e. `-1`; `String.prototype.lastIndexOf` clamps its position argument into `[0, length]`, so the next iteration searched from 0, found the same match, rejected it identically, and set `-1` again. The loop made no progress. Reachable only through the exported `extractTableRegion` — `writeLedgerAtomic` reaches it after `parseFrontmatterStrict` has already succeeded, so the candidate region begins with the `---` frontmatter fence and a match at index 0 is impossible. Latent rather than live, but an exported `for(;;)` that can fail to advance is not something to ship. Confirmed by running the pre-fix compiled function on `TABLE_HEADER_LINE + 'X\n' + <a valid json fence>` as a backgrounded child: it was still alive after five seconds having printed nothing, and had to be killed. Post-fix the same input returns `null` promptly — correct, since the sole header occurrence fails the end-of-line test and no valid header exists. A regression here would stall the suite rather than fail it, so the new test also asserts the returned value rather than relying on termination alone. No wall-clock assertion is involved. Refs #3689 * test(#3034): publish the lane trace before the done-file that releases dependents `preservesSelectionOrderParallelDespiteCompletionOrder` forces a reverse completion order with a dependency chain rather than sleeps: each stub lane waits on `done-<dep>` before finishing. It then ended with touch "$RUN_DIR/done-$slug" echo "end:$slug" >> "$TRACE" Those are two unsynchronized operations in separate shell processes. A dependent's `wait_for_file` unblocks the instant the upstream's `touch` lands, but the upstream's own `echo` has not necessarily run — so if the upstream is descheduled between the two, the dependent can run its whole body and append its `end:` line first. The done-file was published before the state it signals. Observed on the remote runner as `[end:claude, end:codex, end:gemini]` where selection order demands `[end:claude, end:gemini, end:codex]`. The failure was in the fixture's own self-check, before it reached the assertion #3034 exists to make. Not a flake and not a wall-clock margin: this branch passed the full suite twice at 14f494644 and 90c5d7a03, and the only delta in the failing run was one added test in tests/broken-windows.test.cjs — an unrelated module. Adding load elsewhere in the suite was enough to invert it, which is what a real race does. Swapping the pair establishes a genuine happens-before: anything a dependent can observe is written before the file that releases it. A comment records why, so the order is not tidied back. The production path is unaffected and was independently confirmed correct — `invoke_reviewers` joins every lane with `wait`, then aggregates by iterating DISPATCH_SLUGS in selection order, reading per-slug result files. It consumes no completion-order signal at all. Refs #3034 --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
1f76861202 |
fix(#3657): tolerate commonmark fence widths in ledger readers (#3733)
* test(#3657): fence-width tolerance regression rows * test(#3657): fix pure-row fixtures to use appendWindow result shape * fix(#3657): tolerate commonmark fence widths in ledger readers * fix(#3657): restore throw-block indentation in parseJsonBlock * chore(#3657): backfill changeset pr number --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
a731a45cd6 |
fix(#3116): strip trailing CR per line in parseFrontmatterStrict for CRLF WINDOWS.md (#3137)
* test(#3116): failing-first — parseLedger throws on CRLF WINDOWS.md On repos with core.autocrlf=true (Windows default), .planning/WINDOWS.md is checked out CRLF. The \n--- close-fence scan leaves the last frontmatter line's CR attached, and the key:value regex's . doesn't match CR, so the parser throws WINDOWS_LEDGER_MALFORMED on the last key. * fix(#3116): strip trailing CR per line in parseFrontmatterStrict The \n--- close-fence scan lands on the LF of the last frontmatter line's CRLF, so yamlBody ends with a bare \r. split(/\r?\n/) strips CR from interior lines but the last line's \r survives. The key:value regex fails because . doesn't match CR. Fix: strip \r per line (rawLine.replace(/\r$/, '')) rather than normalizing raw — the writer round-trips raw byte-exact. * chore(#3116): add changeset fragment * chore(#3116): backfill changeset PR number 3137 --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
34633fa4ec |
fix(#2893): preserve prose below the JSON ledger on windows append/waive/fixed (#2975)
* test(#2893): add regression for append destroying prose below JSON ledger writeLedgerAtomic overwrites the entire file with renderLedger(ledger), dropping any prose below the JSON closing fence. The test creates a WINDOWS.md with prose sections below the ledger, appends an entry, and asserts the prose survives. * fix(#2893): preserve prose below the JSON ledger on append/waive/fixed writeLedgerAtomic was overwriting the entire WINDOWS.md with renderLedger(ledger), which reconstructs only the frontmatter + header + table + JSON block — silently destroying any prose a user wrote below the JSON closing fence. Now the writer reads the existing file before overwriting, extracts content after the closing fence, and appends it to the rendered ledger. First-write (no existing file) proceeds normally with no prose to preserve. * fix(#2893): address review — correct fence search + idempotency test BLOCKER from isolated adversarial review: indexOf(JSON_FENCE_CLOSE) matched the OPENING fence ('json' starts with ''), duplicating the entire JSON body as prose on every write. Now searches for the closing fence starting AFTER the opening fence, mirroring parseJsonBlock. Test hardened: non-empty initial ledger, second append (idempotency — prose appears exactly once, exactly one JSON fence open), parseLedger round-trip. * chore(#2893): add changeset fragment * chore(#2893): backfill changeset PR number 2975 --------- Co-authored-by: sim <sim@local> |
||
|
Tom Boucher
|
d16a66479a |
feat(#1950): broken-windows ledger — cross-phase defect register gating ship (#2441)
* feat(#1950): broken-windows ledger — cross-phase defect register gating ship Adds a new capability (#1950) that operationalizes GSD's no-defer discipline as a tracked, enforced artifact: accumulates stubs, TODOs, skipped tests, unrun verifies, and unmet truths across phases, and /gsd-ship blocks while any entry is open. Implementation: - src/broken-windows.cts → gsd-core/bin/lib/broken-windows.cjs: typed IR + I/O entry points (parseLedger/renderLedger/appendWindow/markWaived/markFixed + cmdWindowsStatus/Append/Waive/MarkFixed). Frozen REASON enum for typed error assertions. Windows-safe atomic rename with retry on transient EPERM/EBUSY/EACCES. - gsd-tools.cjs: new subcommand (status | append | waive | fixed), wired via routeWindows + HOST_COMMAND_ROUTERS.windows. - capabilities/broken-windows/capability.json: one ship:pre gate with artifact-frontmatter-equals predicate on WINDOWS.md open_count == 0. activationKey windows.enabled (default true) + sibling windows.enforce (default true, separate so tracking can precede enforcement). - gsd-core/workflows/ship.md: capId==broken-windows branch in preflight, sibling to security — reads gsd_run windows status --raw, fails closed on open_count > 0 or unreadable ledger. - agents/gsd-executor.md: extends the existing ## Known Stubs instruction to also append to WINDOWS.md via gsd_run windows append (best-effort, never blocks execution). - agents/gsd-verifier.md: new Step 8b — record unmet truths + human-verify items in WINDOWS.md. - gsd-core/workflows/progress.md: surfaces open + waived counts. - docs/COMMANDS.md + CONTEXT.md glossary entry + docs/INVENTORY.md: document the gate, waiver mechanism, and new module. - tests/broken-windows.test.cjs: pure + CLI behavioral coverage + fast-check roundtrip property; fail-closed on malformed ledger; security boundary on path traversal in --file. Backward-compatible: a project with no .planning/WINDOWS.md reports open_count: 0 and ships cleanly. Disable enforcement per-project with gsd config-set windows.enforce false (tracking continues, gate stays open). * chore(#1950): ratchet size baselines, defer verifier integration - Workflow size baseline: ship.md 25575→27928, progress.md 31789→32632 (broken-windows preflight branch + open-windows surface). - Agent size baseline: gsd-executor.md 46644→47951 (Known Stubs → also appends to WINDOWS.md). gsd-verifier.md unchanged. - LARGE_CAP (49152) preempted the planned verifier integration (gsd-verifier.md was at 49140 pre-PR — 12 bytes of headroom, not the documented 'real headroom'). Verifier integration deferred to a follow-up PR that extracts the VERIFICATION.md template (lines 739-859) to gsd-core/references/ — a pre-existing cap-tightness defect this PR exposed but does not expand scope to fix. Verifier integration is not in the issue's acceptance criteria (executor writes is; unmet-truths recording was an enhancement, not a gate). * fix(#1950): gate default-off, rename to workflow.windows_enforce, regen goldens Test-failure-driven fixes after first gsd-test run on db8733c8f failed 44 cases (pre-existing structural tests encoded 'ship:pre has 1 gate' / 'all caps off → empty hooks'): - capability manifest: rename windows.enabled+windows.enforce (default true) → single federated key workflow.windows_enforce (default FALSE, opt-in). Matches security's workflow.security_enforce convention and makes the adr857 all-caps-off test pass without modification (the test's buildAllFalseConfig handles workflow.* out of the box). Default-OFF keeps the gate out of the registry's default ship:pre resolution so existing loop-hooks-ship-pre-e2e structural assertions (exactly 1 gate, capId 'security') stay valid; users opt in via gsd config-set workflow.windows_enforce true. - drop activationKey (security doesn't have one either; workflow.* key doubles as the activation toggle). - regenerate docs/reference/capability-matrix.md to include broken-windows (capability-matrix-sync test). - regenerate tests/fixtures/golden-install-parity/*.json (18 runtimes) — installer now emits the new capability + lib file. - update CONTEXT.md, docs/COMMANDS.md, docs/FEATURES.md, ship.md, agents/gsd-executor.md to use the new key name and /gsd:colon slash syntax (slash-command-namespace test). - restore accidentally-regressed /gsd:capture in progress.md. Tracking-only by default; enforcement is opt-in. Acceptance criterion '/gsd-ship fails while any ledger entry is open' is met when workflow.windows_enforce=true (test fixture enables it). * test(#1950): update ship:pre structural invariants for 2-gate registry - loop-hooks-ship-pre-e2e: the registry now declares 2 gates at ship:pre (security + broken-windows), regardless of activation. Activation tests above still pin security-only or empty behavior via fixtures; these structural tests pin the REGISTRY shape, which has 2 gates as of #1950. - workflow-size-baseline: ship.md 27928→27945 (workflow.windows_enforce rename added 17 bytes). * fix(#1950): review H1+H2+M1+M2+M3 — fence-injection, EACCES fail-closed, cleanup, strict line, stryker Adversarial isolated review (Step 6.3) found 2 HIGH findings that block the PR and 3 mediums. All addressed: H1 (HIGH): description containing the markdown 3-backtick fence would terminate the ledger's JSON code block early inside JSON.stringify output (JSON doesn't escape backticks), corrupting the file and bricking the next parse. Fix: use a 4-backtick fence (json ... ) which JSON.stringify cannot produce on its own, AND validate that no entry text field contains a 4-backtick run (reject at append time with new WINDOWS_INVALID_TEXT reason code). Locked by a regression test. H2 (HIGH): readLedgerOrNull swallowed ALL fs errors as 'no ledger', silently returning open_count:0 on EACCES/EPERM/EIO. The ship gate would then pass on an unreadable ledger — the precise vector the workflow doc claims is impossible. Fix: only ENOENT returns null; every other fs error propagates as WINDOWS_LEDGER_MALFORMED so the gate blocks and the operator sees a real diagnostic. Locked by a regression test that chmod 000s a ledger with open_count=1 and asserts the result is never a false-green 0. M1: writeLedgerAtomic left an orphaned .tmp file on rename failure. Wrapped renameWithRetry in try/catch with best-effort unlink. M2: validateLine silently coerced 'abc' → NaN → null, hiding type drift. Removed the line === 0 special case (was undocumented) and made the error message match the strict check. Now any non-positive- integer line value throws, including strings. M3: tests/broken-windows.test.cjs (with its fast-check property test) was not in stryker.config.mjs DEFAULT_TEST_CMD — Stryker would mutate src/broken-windows.cts but no test would catch the mutations, producing false surviving-mutant scores. Added to the list. L1 (dead throw e after error()), L7 (line boundary tests, H1/H2 regression tests, 4-backtick CLI test) also addressed. * docs(#1950): inline concurrency + busy-wait notes (review L2+L3) * fix(#1950): regen goldens against latest gsd-tools; correct --line 0 boundary test gsd-test v4 caught two issues: - goldens I regenerated earlier (commit 526682084) predated the L1 routeWindows catch-block cleanup (commit dd844d565). Regenerated via 'npm run gen:golden' against current HEAD so the install parity hash for gsd-tools.cjs matches. - 'append --line boundary' test expected --line 0 to succeed with null entry.line, but the M2 fix correctly rejects 0 (lines are 1-indexed; 0 is not a valid source line). Updated the boundary test to assert --line 0 fails alongside -1 and 'abc'. * chore(#1950): regen goldens after rebase onto next * chore(#1950): quick.md baseline 50699→50993 (correct resolution from next rebase) * chore(changeset): backfill pr:2441 in .changeset/broken-windows-ledger.md * fix(#1950): renderTable escapes backslash before pipe (CodeQL incomplete-sanitization) CodeQL flagged the markdown-table cell escaper: String(s ?? '').replace(/\|/g, '\\|') — it escapes pipe but not backslash first. A description containing '\|' would render as '\\|' which markdown parses as 'literal backslash' + 'cell separator', splitting the column. Fix: escape backslash FIRST (each \ → \\), then pipe (each | → \|). Now a description with '\|' renders as '\\\\|' (literal '\\' + escaped pipe), which markdown renders as a single '\|' inside the cell. The JSON code block (the parse source-of-truth) was already correctly escaped via JSON.stringify; only the display-only table was affected. Locked by a regression test that: 1. Verifies the JSON block reparses with the description intact. 2. Walks the rendered table row counting unescaped pipes — must be exactly 11 (the row separators for 10 cells), proving no in-cell pipe added a split. |