42f8c192a8eafd96a00216e263bed83dc034f708
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tom Boucher
|
afba2afbe7 | fix(#229): normalize gsd-hook-version substitution in verify-reapply (#282) | ||
|
Tom Boucher
|
a7abc6df2f |
fix(3657): skip false-fail when pristine hash drifts after GSD update (#3767)
* test(3657): RED+shape-lock for verify-reapply-patches pristine-drift Adds tests/bug-3657-verify-reapply-patches-pristine-drift.test.cjs: - Core regression: exits 0 with reason=OK_PRISTINE_DRIFT_DETECTED when on-disk gsd-pristine/ hash does not match backup-meta.json.pristine_hashes - Counter-tests: real FAIL_USER_LINES_MISSING still caught when hashes match; over-broad mode unchanged when backup-meta.json is absent; clean run reports 0 failures when everything matches - Multi-file: drift + real-failure handled independently per file Updates tests/bug-2969-verify-reapply-patches.test.cjs REASON shape-lock to include OK_PRISTINE_DRIFT_DETECTED (added by the #3657 fix). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(3657): verify-reapply-patches skips pristine when hash drifts When gsd-pristine/ is refreshed to a newer GSD version after a backup is captured, the on-disk pristine's SHA-256 no longer matches the hash recorded in backup-meta.json.pristine_hashes. Using the wrong-version pristine as the diff baseline inverts the delta: every line the upstream removed between the two versions appears as a "user-added line that must survive", producing spurious FAIL_USER_LINES_MISSING false positives (Bug #3657). Fix: - Add sha256() and readPristineHashes() helpers to verify-reapply-patches.cjs - In verifyFile(), when a pristine_hashes entry exists for the file, compare the on-disk pristine's SHA-256 against it before accepting the baseline - On hash mismatch, return immediately with status=ok and the new REASON.OK_PRISTINE_DRIFT_DETECTED code, skipping the diff rather than false-failing - When no pristine_hashes entry exists (older installer / absent backup-meta), fall through to the pre-fix behaviour (use on-disk pristine as-is) - Export sha256, readPristineHashes, and OK_PRISTINE_DRIFT_DETECTED New REASON code OK_PRISTINE_DRIFT_DETECTED is added to the frozen enum. Exit code contract is unchanged: 0 for gate pass (including skipped-due-drift files), 1 for real user-content failures, 2 for structural errors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(3657): update changeset to reference PR #3767 * fix(3657): surface drifted_files in verify-reapply-patches JSON report Extend the top-level JSON report shape with two additive fields: - `drifted: N` — count of files skipped due to pristine-snapshot drift - `drifted_files: [...]` — relative paths of those files Per-file shape is unchanged (status:'ok' + reason:OK_PRISTINE_DRIFT_DETECTED) for backward compat. Drift still exits 0; `failures` count is unaffected. This gives workflow Step 5a structured data to gate on so drifted files are no longer silently treated as a full PASS (codex adversarial-review finding 1). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(3657): reapply-patches workflow halts on drifted files instead of silent pass Insert a "Step 5a: drift check" block between the exit-code check and the failures check in workflows/reapply-patches.md Step 5a. The new block: 1. Parses `drifted` + `drifted_files` from the JSON report (added in the companion prod-code commit). 2. When DRIFTED_COUNT > 0, emits a formatted HALT message naming each drifted file and instructs the user to re-baseline before re-running. 3. Sets DRIFT_DETECTED=true and exits non-zero so subsequent steps cannot execute while drift is unresolved. Drift is a distinct third state: it is not a failure (no missing user lines were proven) but it is also not a clean pass (the diff was skipped entirely). Existing pass/fail logic for VERIFY_STATUS and failures count is unchanged. Closes the silent-skip gap identified in codex adversarial-review finding 1. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(3657): assert drifted_files report shape + workflow Step 5a drift check Finding 1 (BLOCKER) — three new tests in bug-3657 test file: - Single drifted file: drifted=1, drifted_files contains the file path, failures=0, per-file shape unchanged (backward compat). - Multi-file drift: drifted=2, drifted_files lists both paths, clean file absent from array. - No-drift baseline: drifted=0, drifted_files=[] always present in output. Finding 2 (WARNING) — structural test on workflow source: - Asserts Step 5a contains "Step 5a: drift check" heading. - Asserts DRIFTED_COUNT, drifted_files, and DRIFT_DETECTED are referenced (confirming the gate exists and uses the structured report fields). - Asserts drift-check block appears before VERIFY_STATUS check (exit-code is 0 for drift, so the drift check must precede the non-zero gate). Also updates bug-2969 shape-lock to include the two new additive fields (drifted, drifted_files) per the contract change in the prod-code commit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(3657): address pr-review-toolkit + codex review + CI failures - lint-tests: add // allow-test-rule: source-text-is-the-product at file top of bug-3657 test file; the inline comment at line 477 was a prose sentence, not a file-level annotation, so the lint scanner did not recognise it as the bypass token - Windows test failures (4 subtests): normalize relPath to forward slashes before pristineHashes key lookup in verifyFile(); on Windows path.join produces backslash-separated relPath values but backup-meta.json stores keys with forward slashes, causing the hash lookup to silently return undefined, falling through to use-as-is mode and producing the same false FAIL_USER_LINES_MISSING that the fix was meant to prevent Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tests): bump verify allowlist ceiling 9→10 for bug-3657 test file Adding tests/bug-3657-verify-reapply-patches-pristine-drift.test.cjs in the previous commit pushed the verify module from 9 to 10 test files. The lint-test-file-count gate (added in #6313baad on main) enforces that modules cannot exceed their allowlist ceiling, so all 6 test platforms plus lint-tests and coverage failed with: FAIL_EXCEEDS_ALLOWLIST: verify count=10 ceiling=9 The fix is to raise the ceiling from 9 to 10 and set issue=3767. This file does not exist on this branch yet (introduced on main after the branch diverged) so we add it here. The merged CI state will see the bumped ceiling and pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Tom Boucher
|
ca3be82f71 |
fix(3597): clear residual Windows test failures + add ratchet lint guard
Two more diagnostic passes (clusters J: residuals in already-touched files,
K: 12 untouched files) plus a production-code path fix and a new
ratchet-style lint guard.
## Test-only fixes (14 files)
bug-1736, bug-2248, bug-2698 — replace inline 1s-budget rmSync with the
shared cleanup() helper (5s budget, 20×250ms retries). The earlier
inline maxRetries:10 / retryDelay:100 wasn't enough to absorb Windows
Defender's deferred-scan handle hold on cold runners.
bug-2256, skill-manifest — also override USERPROFILE alongside HOME in
beforeEach/runGsdTools calls. os.homedir() reads USERPROFILE on win32,
so HOME-only stubs leak the runner's real home into the SUT.
bug-2784, bug-3608, enh-2500, enh-2790, few-shot-calibration,
gsd-settings-advanced — CRLF tolerance: literal \n in regexes against
file content (frontmatter anchors, bash-fence regex, multi-line
numbered-list captures, awk-block extractors) becomes \r?\n; split('\n')
becomes split(/\r?\n/). Windows checkout with autocrlf=true puts \r
before every \n; .+ doesn't match \r in JS regex by default.
bug-2966 — three-part fix to extractStepRun (CRLF split), awk regex
(\r?\n), and conflict-marker parser (rawLine + \r$ strip).
bug-2969, config — normalize separators on test assertions where the
SUT correctly emits \ on win32 but the test compares against /.
prompt-injection-scan — normalize relPath via replace(/\\/g, '/') before
ALLOWLIST.has() lookup. ALLOWLIST keys are POSIX; path.relative returns
backslashes on win32 → falsely scans allowlisted security module → trips
the boundary-tag detector on its own legitimate detection code.
prune-orphaned-worktrees — use the existing canonicalPath +
listedWorktreePaths(repoDir).has(...) helpers instead of substring
matching the raw path. git stores long-form canonical paths
(runneradmin), but mkdtempSync returns 8.3 short-form (RUNNER~1) on
Windows runners; plain string compare misses every entry.
## Production-code fix (1 file)
get-shit-done/bin/lib/init.cjs — bug-3491 in_nested_subdir computation
canonicalizes both worktreeRoot and cwd via fs.realpathSync.native +
path.relative before declaring "nested." Windows runner cwd (8.3 short
name) vs git's --show-toplevel (long form, forward slashes) made the
raw string compare always say true even at the worktree root, breaking
the "init new-project at worktree root" subtest.
## New ratchet lint guard
tests/windows-test-parity-guard.test.cjs — scans tests/ for 7
anti-patterns that drove the Windows failure clusters. Each rule has a
baseline count snapshot from this PR; the test fails when a new
occurrence appears (count grows above baseline), ratcheting down as
existing offenders are fixed. Patterns covered:
G1 split('\n') after readFileSync (use /\r?\n/)
G2 ```bash\n fence regex (use ```bash\r?\n)
G3 ^---\n frontmatter anchor (use ^---\r?\n)
G4 hardcoded "/tmp/..." literal passed to fs.* (use os.tmpdir())
G5 bare 'npm' to execFileSync without {shell:true} on win32
G6 process.env.HOME stub with no USERPROFILE
G7 fs.rmSync({recursive,force}) without maxRetries
Future Windows-parity regressions get caught at PR time rather than
five iterations into a CI loop.
Validated: holodeck (ubuntu docker) 11232/0 pass (count +8 = the 7
new ratchet tests + parent describe).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Tom Boucher
|
4277f7d7e8 |
fix(#2994): move verify-reapply-patches.cjs to get-shit-done/bin/ so it ships to user installs (#3000)
* fix(#2994): move verify-reapply-patches.cjs to get-shit-done/bin/ so installer ships it scripts/verify-reapply-patches.cjs (added in #2972 to close the verified-yes-without-checking gap from #2969) shipped in the npm tarball but never reached user installs: bin/install.js copies get-shit-done/ recursively but does not copy the top-level scripts/ directory. Effect: every fresh install hit `Cannot find module …/scripts/verify-reapply-patches.cjs` on Step 5 of /gsd-reapply-patches. The whole point of moving verification out of LLM-driven prose into a deterministic script is undone if the script does not resolve at runtime. Fix: move the script to get-shit-done/bin/verify-reapply-patches.cjs (same pattern as gsd-tools.cjs and other runtime bin scripts that the installer ships) and update reapply-patches.md Step 5 to invoke ${GSD_HOME}/get-shit-done/bin/verify-reapply-patches.cjs. Tests: - bug-2969 SCRIPT path updated to the new location - New bug-2994-verify-reapply-patches-installed-path.test.cjs parses reapply-patches.md into structured invocation records and asserts every node ${GSD_HOME}/... reference lives under get-shit-done/ (the installed tree). Catches future regressions where someone moves a runtime-needed script back to scripts/. Closes #2994 * chore(#2994): add changeset fragment for PR #3000 * chore(#2994): add changeset fragment for PR #3000 * docs(#2994): update verifier-script-location comment to reflect new path (CR) CodeRabbit on PR #3000: the parenthetical at line 278 still said the script ships under scripts/, but this PR moved it to get-shit-done/bin/. Updated the prose to reference the new location and the installer target path. * chore(#3000): drop direct CHANGELOG.md edit; release entry now lives in .changeset/ The changeset-fragment workflow (#2975) renders fragments into CHANGELOG.md at release time. Direct edits to [Unreleased] on each PR caused merge conflicts on every concurrent PR. This commit restores CHANGELOG.md to match origin/main; the release entry for this fix is preserved in the .changeset/*.md fragment(s) on this branch, which the release workflow consolidates. |
||
|
Tom Boucher
|
ef43f5161f |
fix(#2969): deterministic Step 5 verification gate for /gsd-reapply-patches (#2972)
* fix(#2969): deterministic Step 5 verification gate for /gsd-reapply-patches The prior Step 5 "Hunk Verification Gate" was prescribed correctly in the workflow text — but executed laxly by the LLM, which filled in `verified: yes` without actually checking content presence. The reporter observed three distinct files (skills/gsd-discuss-phase/SKILL.md, skills/gsd-autonomous/ SKILL.md, get-shit-done/workflows/new-project.md) where archives contained substantive user-added blocks that did not survive into the merged result, yet the gate reported clean. Move verification from LLM-driven prose into a deterministic Node script the workflow calls. The script can't be shortcut. Changes: - scripts/verify-reapply-patches.cjs (new): pure Node, no external deps. For each file in the patches dir, computes user-added significant lines as the line-set diff between backup and pristine baseline (when available; falls back to "every significant backup line" when no pristine — over-broad but the safe direction for this bug class). Asserts each line appears literally in the merged installed file via String.prototype.includes. Filters trivial lines (length < 12 chars, pure punctuation, decorative comments) so harmless drift doesn't trigger false failures. Exits 0 on pass, 1 on any miss with per-file diagnostic, 2 on usage error. Supports --json for workflow consumption. - get-shit-done/workflows/reapply-patches.md: rewrite Step 5 to call the script and parse its JSON output. The Step 4 Hunk Verification Table remains as advisory Claude-readable summary, but the gate is now the script's exit code. - tests/bug-2969-verify-reapply-patches.test.cjs (new): 6 tests covering (a) pass when every line survives, (b) fail when a line is missing, (c) fail when the merged file is deleted entirely, (d) --json structured report shape, (e) backup-meta.json is correctly skipped as metadata, (f) no-pristine-dir fallback exercises the safe over-broad path. All pass. Out of scope: the manifest-baseline tightening described in #2969 Failure 1 (saveLocalPatches comparing against the wrong baseline so prior silent wipes poison subsequent updates). That's a separate, bigger architectural change involving pristine-content infrastructure; this PR addresses the gate fidelity half so users at least see the diagnostic when content goes missing. Closes #2969 (partial — Failure 2 only) * fix(#2969): preserve #1999 Hunk Verification Table assertions alongside new script gate CI failure on PR #2972 surfaced that tests/reapply-patches.test.cjs (the #1999 contract) asserts Step 5 references: - "Hunk Verification Table" - `verified: no` failure condition - explicit STOP/halt/abort directive - "table absent / missing" halt path My initial Step 5 rewrite for #2969 substituted the deterministic script for the table-based gate entirely, stripping those references. The script is the strictly stronger gate, but the existing #1999 test enforces the table-based safety net as a defense-in-depth contract. Restore both gates as a layered Step 5: - 5a (binding): deterministic verifier script — script gate, exits non-zero on any miss, cannot be shortcut by the LLM - 5b (advisory): Hunk Verification Table review — preserved as redundant safety net for the case where the script has a bug or the pristine baseline is unavailable Both gates must pass. Verified: tests/reapply-patches.test.cjs (5 tests in the #1999 suite) and tests/bug-2969-verify-reapply-patches.test.cjs (6 tests in the #2969 suite) all pass — 21/21 total in this fixture. * fix(#2969): address CodeRabbit findings on workflow + script Five CR findings on PR #2972, all valid; addressed in this commit: 1. (Major) Stderr was merged into VERIFY_OUTPUT via `2>&1`, so any Node warning, deprecation notice, or stack trace would corrupt the JSON parse downstream. Capture stdout only; stderr remains on the controlling terminal for operator visibility. 2. (Major) verifyFile() crashed with EISDIR/EACCES instead of producing a structured diagnostic when the installed path was a directory or unreadable. Wrap statSync/readFileSync in try/catch and emit a per-file fail row; the whole-run gate continues with structured output. Added test case asserting the directory-at-installed-path case fails with `not a regular file` diagnostic instead of crashing. 3. (Minor) PRISTINE_FLAG built as a single string + unquoted expansion would split paths with spaces. Switched to a bash array (VERIFY_ARGS) that preserves whitespace through expansion. 4. (Minor) Fenced code block missing language tag (markdownlint MD040). Added `text` tag to the error message block. 5. (Minor) Usage comment said pristine fallback was "backup-meta lookup" but the actual code path falls back to significant-line checks from backup content. Corrected the comment to match implementation. Verified all 21 tests in tests/reapply-patches.test.cjs (#1999 contract) + tests/bug-2969-verify-reapply-patches.test.cjs (now 7 tests with the new directory case) pass. * test(#2969): structured JSON assertions, no substring matching on script output Replace every assert.match(r.stdout, /pattern/) call with structured assertions on the parsed JSON report from the script's own --json mode. The script's --json contract IS the structured shape we test against — the test author should never depend on the human-readable formatter output, just as no test should depend on substring presence in source. Changes: - All 7 tests now run the verifier with --json (via a runVerifier() helper) and parse the resulting JSON document into { status, report, stderr }. Diagnostic stderr is preserved as a separate channel for debug output but is not used for assertions. - Each previously substring-matched diagnostic ("Failures: 1", "not a regular file", "installed file missing after merge", file path, dropped line) is now a deepEqual / equal / Array.includes against typed report fields: report.failures, report.results[i].status, report.results[i].reason, report.results[i].file, report.results[i].missing[]. - Added an explicit "documented shape" test asserting the JSON output has exactly the keys { file, missing, reason, status } per result — locks the public contract of the --json mode. - DRY'd up fixture reset into a resetFixture() helper since every test starts with a fresh patches/installed/pristine triple. Linter: scripts/lint-no-source-grep.cjs reports 0 violations across 348 test files. Combined run of bug-2969-...test.cjs (7 tests) + reapply-patches.test.cjs (5 tests in the #1999 suite) all pass — 22/22 in the relevant fixture. * fix(#2969): typed REASON enum + raw-text-matching rule shipped repo-wide This commit closes the loop on the no-source-grep discipline: 1. scripts/verify-reapply-patches.cjs: - Frozen REASON enum exposes the diagnostic surface as stable codes: OK_NO_USER_LINES_VS_PRISTINE, OK_NO_SIGNIFICANT_BACKUP_LINES, FAIL_INSTALLED_MISSING, FAIL_INSTALLED_NOT_REGULAR_FILE, FAIL_READ_ERROR, FAIL_USER_LINES_MISSING. - Each result.reason is now a code from this enum, not free text. Tests assert via REASON.X equality, not regex on prose. - REASON exported from module.exports. 2. tests/bug-2969-verify-reapply-patches.test.cjs: - Full rewrite. Every assertion on typed structured fields: report.results[0].status === 'fail', report.results[0].reason === REASON.FAIL_INSTALLED_NOT_REGULAR_FILE, report.results[0].missing.includes(droppedLine) (Array set membership, not String substring). - Locks the REASON enum surface via Object.keys(REASON).sort() deepEqual. - Locks the JSON report shape via Object.keys(report).sort() deepEqual. - Zero regex, zero String#includes, zero startsWith/endsWith on text. 3. CONTRIBUTING.md: - New section "Prohibited: Raw Text Matching on Test Outputs" with concrete BAD/GOOD examples (substring on file content; assert.match on stdout; "structured parser" hiding string ops; regex on free-form reason fields). - The rule statement: "Tests assert on typed structured values. If the code under test produces text, the code under test must also expose a structured intermediate representation, and the test must assert on that IR — never on the rendered text." - Required structured-surface table: file IR, --json mode, frozen enum, fs facts. - "Hiding grep behind a function is still grep" callout — the parser-wrapper anti-pattern. - New `pre-existing-text-matching` exemption category for the 8 grandfathered files. Marked Transitional; new tests cannot use it. 4. scripts/lint-no-source-grep.cjs: - Three new patterns enforced (in addition to the existing .cjs-source readFileSync rule): - assert.match/doesNotMatch on .stdout/.stderr - .stdout/.stderr.<includes|startsWith|endsWith>( - readFileSync(...).<includes|startsWith|endsWith>( - Aggregated violations per file (multiple findings now report together). - Updated diagnostic message references both CONTRIBUTING.md sections. 5. 8 pre-existing tests annotated with `// allow-test-rule: pre-existing-text-matching` so the lint passes on this commit; each carries the prose "Tracked for migration to typed-IR assertions; do not copy this pattern." Files: bug-2649, bug-2687, bug-2796, bug-2838, bug-2943, graphify, hooks-opt-in, security-scan. Verification: lint 0 violations across 348 test files; full suite passes. * fix(#2969): rename exemption category to pending-migration-to-typed-ir + cite tracking issue Per maintainer feedback: 1. "Grandfathered" / "legacy" framing is wrong — both terms imply permanent or condoned exemption. The 8 files are tracked for correction, not exempted. 2. Each annotated file must cite the tracking issue so the migration work is auditable. Changes: - CONTRIBUTING.md: rename exemption category from `pre-existing-text-matching` to `pending-migration-to-typed-ir`. Update prose to "Tracked for correction, not exempted" and require each annotation to cite the open migration issue (e.g. `// allow-test-rule: pending-migration-to-typed-ir [#NNNN]`). - 8 test files: update annotation to cite #2974 (the tracking issue opened for migrating these files to typed-IR assertions). |