2 Commits

Author	SHA1	Message	Date
Tom Boucher	1178c5f995	test(#3108): make the overlay ENOENT tolerance and hooks/dist readiness check honest (#3772) ... * test(#3108): failing-first suite for the overlay vanish-retry and hooks/dist staleness RED by construction, and deliberately narrower than the issue. #3108 reports a bare `ENOENT ... link '/work/hooks/dist/gsd-session-state.sh'` and attributes it to hooks/dist never having been built. That mechanism cannot produce that error: buildOverlayRepo enumerates with readdirSync and links what it enumerated, so a directory that never existed yields no names and no link is ever attempted. The error requires the file to have existed at readdir and vanished before the link -- which is the atomic-replace race placeVanishableLeaf was written for in #3285, three days AFTER this issue was filed. What is still genuinely broken, and what these tests bind to: placeVanishableLeaf's retry is unguarded. On ENOENT it re-checks existsSync and calls attempt() once more, bare. A second atomic replace inside that window throws an unhandled ENOENT of exactly the reported shape. Rows 1/2/3/5/6/7 fence the surrounding contract -- most already pass, which is the point: they are what stops the fix from widening into "swallow every error". Row 6 in particular covers a non-ENOENT on the RETRY, the exact path the new code will live on. ensureHooksDist's staleness predicate is extension-blind. It rebuilds only when hooks/dist is absent or holds zero .js files, while build-hooks.js also ships .sh -- including gsd-session-state.sh, the very file in the report. A dist with .js present and every .sh missing reads as populated and the rebuild is skipped. Rows 10 and 11 are mirrored on purpose: asserting only the .sh direction would permit swapping one extension heuristic for another, so both directions force the predicate to be about the expected set (HOOKS_TO_COPY, which build-hooks.js already exports) rather than about counting an extension. Rows 12 and 13 are cost guards. ensureHooksDist runs per suite and its rebuild is a real subprocess, so a predicate that over-triggers turns a correctness fix into a throughput regression nobody attributes to it; and hooks/dist legitimately carries files the expected list does not name, so an exact-set match would rebuild forever. The warning-text row is t.skip()'d rather than faked: the only ways to assert it were a source-grep (banned by local/no-source-grep) or a full overlay build, and a test that cannot be written honestly is better skipped visibly than written vacuously. Filename note: this started as fix-3108-*.test.cjs and tripped lint-regression-test-names, which bans new fix/bug/issue-NNNN files, then as install-overlay-helpers.test.cjs and tripped lint-test-file-count, whose `install` bucket is already at its limit. Module-named under the overlay bucket satisfies both. The allowlists were left untouched -- both are empty, so nothing here is grandfathered and adding an entry would have been the wrong instinct. * fix(#3108): guard the overlay retry and make the hooks/dist check see .sh Two holes, both reachable from the failure #3108 reports, neither of them the cause it names. placeVanishableLeaf's retry was bare. On ENOENT it re-checked existsSync and called attempt() once more with no catch, so a second atomic replace landing inside that window threw an unhandled ENOENT -- exactly the reported `ENOENT ... link '/work/hooks/dist/gsd-session-state.sh'`. Two vanishes inside the window means the same thing one does: the path is going away and is not part of the snapshot. It now returns false and skips the leaf, reaching the conclusion the single-vanish case already reached. Still ONE retry. No loop, no backoff, no sleep -- the existing comment argues that a timing-based wait here would be the flake rather than the fix, and that reasoning did not change. Non-ENOENT still propagates from either attempt, which is the invariant a careless widening would eat; the suite pins it on the retry path specifically, because a fix that guarded only the first attempt would look right and be wrong. ensureHooksDist could not see the file class that caused the report. It rebuilt only when hooks/dist was absent or held zero .js files, while build-hooks.js also ships .sh -- including gsd-session-state.sh itself. A dist with .js present and every .sh missing read as populated and the rebuild was skipped. The predicate is now membership against build-hooks.js's own exported HOOKS_TO_COPY, so it asks "is everything expected present" instead of counting an extension, and it cannot be blind to a file class again. It is extracted as isHooksDistStale(dir) with ensureHooksDist calling it, so there is one predicate rather than two that can drift. Extra unexpected entries are explicitly not stale -- hooks/dist legitimately accumulates subdirectory and hooks/lib output, and an exact-set match would rebuild forever. One readdirSync into a Set, no per-entry existsSync, no stat: it runs per suite and its rebuild is a real subprocess, so an over-triggering predicate would turn this into a throughput regression nobody would attribute to it. The skipped-leaf warning now names `npm run build:hooks`. It already named the cause; a reader still had to know what produces that directory. Deliberately NOT done: nothing here makes an absent hooks/dist fail. Absence is a legitimate package shape that bin/install.js:11191 treats as "nothing to verify", and six of the seven install suites never read the directory at all. * fix(#3108): count hooks/dist subdirectories, and never throw out of the predicate Two gaps found reviewing the predicate I had just written. It ignored HOOKS_SUBDIRS_TO_COPY. That is ["lib"], and hooks/dist/lib carries gsd-graphify-rebuild.sh, so a dist with all 27 top-level files but no lib/ read as populated. That is precisely the blindness the .js-count heuristic had, one level down: a whole file class invisible to the check. Fixing the extension case and leaving the subdirectory case would have been half a fix, and the half left behind is the one nobody would look at again. Subdir names are bare (no slashes), so they slot into the same top-level readdir Set — no second readdirSync, no stat. Whether lib is really a directory is not checked; that would cost a stat per entry and buys nothing, since the build owns that. It could also throw. existsSync passing does not make readdirSync safe: the path may be a regular file (ENOTDIR), unreadable (EACCES), or retired in the race between the two calls. This helper runs in every install suite's before(), so an unhandled throw there fails a suite on a condition it cannot act on. Unreadable is indistinguishable from unusable for this question, and rebuilding is idempotent, so it now reports stale instead. Both are the same shape as the original bug and the same shape as each other: a guard that answers "is this ready" must not have a blind spot or a hard edge, because every caller treats a false negative as "carry on". Two existing tests asserted a "complete" dist without lib and had to be corrected to keep meaning what their names claim, rather than being left passing against a definition of complete that no longer holds. * test(#3108): close the review findings, including a half-closed subdir check An isolated correctness reviewer found no blockers and four real gaps. The wiring was untested. Every Group-2 test exercised the pure predicate; none called ensureHooksDist. So restoring the old inline .js-count check INSIDE ensureHooksDist -- keeping isHooksDistStale exported and correct -- left the whole suite green, and that wiring is the actual #3108 defect. Two tests now drive ensureHooksDist itself through the process seam: build invoked exactly once when stale, never when fresh. The second is the one a permissive revert fails. Reaching that seam meant requiring process-seam as a module object rather than destructuring runNode, so a test can replace it in place. That is a testability affordance in a test helper, not a production change, and it is commented as such so it does not read as an accident later. The subdir check was only half closed, and the half left open was the important one. It required `lib` to be PRESENT in the top-level readdir, never looked inside -- so an EMPTY dist/lib, missing gsd-graphify-rebuild.sh, still read as populated. That is precisely the missing-file-class case the subdir check was added to catch, which made the fix a gesture at the problem rather than a fix. Each subdir entry must now be a readable, NON-EMPTY directory. A stray regular file named `lib` throws ENOTDIR into the same try/catch and reads stale too. Cost stayed honest: one extra readdirSync total (there is exactly one subdir entry), no stat, no per-expected-file syscall. Probed against the real hooks/dist -- still reports fresh, so no suite gains a rebuild. Two nits, both real: the error-code sweep re-tested EACCES already covered standalone, and the property ignored presentAtFinalAttempt whenever vanishCount was not 1, making roughly half the 200 runs duplicates. The flag now varies meaningfully across the whole range and the assertions depend on it. One reviewer finding was already stale: the subdir and ENOTDIR work was uncommitted when the reviewer snapshotted the tree, and had landed in a67aefb9c before the report arrived. Verified rather than assumed. * fix(#3108): stop the vanish tolerance from swallowing a dest-side ENOENT A defect this PR introduced, caught by an isolated security reviewer. linkSync(src, dest) throws ENOENT for the DESTINATION path too, not only for a vanished source. The widened retry caught that, saw the source still present, retried, got the same dest-side ENOENT, and returned false -- recording the leaf as "vanished mid-walk" and printing a warning that tells the reader to run `npm run build:hooks`. A remedy with nothing to do with the actual cause, an overlay quietly short a file, and the install under test proceeding against an incomplete tree. It also falsified the function's own documented invariant, which says in as many words: "Returns false only when the path left the source tree entirely." Widening the tolerance without re-reading the sentence above it is how that happens. The retry now re-checks existsSync(srcPath) before tolerating: source still present means the ENOENT was about something else and it propagates untouched. Chose the existsSync re-check over comparing retryErr.path to srcPath -- err.path normalization is not guaranteed across platforms, and a path-equality test is a subtler thing to get wrong later. The FIRST catch was probed and is already correct: for a dest-side ENOENT the source is present, so it falls through to the retry rather than returning false. Left unchanged rather than "fixed" symmetrically. Two regression pins, deliberately opposed: a dest-side ENOENT with the source present must THROW, and a genuinely absent source must still return false. The second exists because the obvious over-correction -- always rethrow on the retry -- passes the first and silently undoes what this PR set out to fix. Also closed the skipped placeholder. It claimed no non-flaky seam existed for asserting the warning text; the reviewer pointed out an injectable `warn` param is trivial, and they were right. buildOverlayRepo now takes opts.warn defaulting to console.warn (byte-identical for every existing caller) and the skip is replaced by real tests: fires with the remedy named on a skipped leaf, silent on a clean walk. "No seam exists" was a design choice presented as a constraint. Recorded the sequential-only constraint at the two sites that monkeypatch fs process-wide: adding { concurrency: true } to this file would cross-contaminate every other suite in the process. Better written down than rediscovered. Known limit, disclosed rather than fixed here: a legitimately dropped leaf can still pass vacuously downstream -- agent-fragments-emission asserts a negative over filesContaining, and mcp-catalog-parity has only an anti-vacuity floor of one. That is a pre-existing property of those suites and the tolerance #3285 already chose; this change narrows which drops are possible rather than adding the completeness assertion those suites lack. * fix(#3108): discriminate ENOENT by the dest parent, not by re-checking the source The previous commit's dest-side guard was wrong, and the remote run said so: "a leaf that vanishes again during the retry is skipped, not a bare ENOENT" Got unwanted exception. Actual message: "ENOENT: no such file or directory" That test was right and the guard was wrong. It rethrew when existsSync(srcPath) was still true, on the theory that a present source means the ENOENT was about the destination. But in the genuine race the source is being atomically REPLACED, so it is legitimately present again at the re-check while the ENOENT was entirely source-side. The gate therefore threw on precisely the race #3285 exists to tolerate -- trading one misclassification for a worse one, since the old bug was a bare crash and the new one broke the working tolerance. The security reviewer's alternative discriminator does not work either, and a probe settles it. Node populates BOTH `path` and `dest` on a link ENOENT, and `err.path` is the SOURCE in both directions: linkSync(existingSrc, missingDir/a.txt) -> ENOENT path=<source> dest=<dest> linkSync(missingSrc, validDest) -> ENOENT path=<source> dest=<dest> So the error object cannot tell you which side failed. What CAN: the dest parent. buildOverlayRepo builds its own dest tree -- place() mkdirSync's recursively into a private mkdtempSync root no other process touches -- so a missing dest parent is always a bug (Windows MAX_PATH, a concurrent cleanup, a bad dest), never the replace race. A present dest parent means the ENOENT was about the source, which is the case we tolerate. placeVanishableLeaf therefore takes an optional destPath and uses the dest parent as the sole discriminator when it has one; with no destPath it behaves exactly as before. linkOrCopyFile and the copy-mode call site both pass it, because those are the two places that actually know the destination. The doc comment now records BOTH failed discriminators and why each fails -- existsSync because the source is legitimately replaced mid-race, err.path because it names the source either way. Those are the two things a future reader reaches for first, and both look correct until they are not. The dest-side regression pin was rewritten to drive the real mechanism: a real temp source and a dest whose parent does not exist, through linkOrCopyFile. Previously it forced a throw through a present source, which is what encoded the wrong theory into a test and made it look verified. --------- Co-authored-by: sim <sim@local>	2026-08-22 23:04:44 -04:00
Tom Boucher	2e2b8ba4a7	enhance(#2704): resolve documentation links and compare H1 status brackets in the ADR gate (#3266) ... * test(#2704): failing-first coverage for ADR link resolution and H1 status brackets Binds the gate to two assertions it does not yet make: every relative markdown link under docs/adr/ must resolve, and an H1 trailing status bracket must agree with the Status: field instead of being silently stripped. Covers all 51 rows of the phase test matrix across two altitudes - the pure extractLinks/maskCode IR for fence and inline-code-span boundaries, hostile input and the fast-check totality properties, and the real CLI verdict for the end-to-end classes. Includes the DEFECT.GENERATIVE-FIX parity test that iterates the exported STATUSES array so a sixth status is covered the day it is added. * feat(#2704): resolve ADR documentation links and compare H1 status brackets The ADR gate validated naming, relation symmetry and index freshness but never resolved a link target, and it stripped an ADR's trailing H1 status bracket for display rather than comparing it against that ADR's own Status: field. Both classes were structurally invisible: #2691 found five dangling references by manual audit roughly a year after they were introduced, one of which reached the published npm payload, while CI reported green throughout. Both are now assertions on the same --check path, using only node:fs and node:path - no dependency and no subprocess. Fenced blocks and inline code spans are masked before scanning, because markdown does not render a link inside code. That is not a policy choice: the corpus contains exactly two such sequences today and both are ordinary JavaScript. Masking preserves length and column positions so findings still name a real line. Resolution is case-exact on every platform - a link that resolves only through macOS or Windows case-folding still 404s on github.com and still fails the Linux lane - and a destination resolving outside the repository is reported before any filesystem call is made. Also single-sources two duplicated surfaces this change would otherwise have extended: the H1 bracket vocabulary (a second hand-written copy of STATUSES with nothing asserting agreement, a DEFECT.GENERATIVE-FIX instance) and the docs/adr directory traversal. Two tests added by #2691 that reimplemented link resolution and bracket comparison inside the test file are removed for the same reason; the corpus assertion is now made by running the real gate against the real corpus. * fix(#2704): reject symlinks that leave the repository and linearize code masking Four defects from the isolated adversarial security review, plus one it noted. BLOCKER - a symlink defeated path containment. path.relative(ROOT, abs) is purely lexical, but the case-exact walk then calls readdirSync, which follows symlinks at the OS level: a contributor-committed docs/adr/x -> /etc together with a link through it passed containment and listed the real external directory, and a wrong-case probe echoed a real external filename through the "Did you mean" hint into publicly-readable fork-PR logs. Every segment is now lstat'd before descent; a symlink is realpathed and re-checked against realpath(ROOT) - realpath on both sides, so a root under /var does not produce false escapes - and an escape emits no hint and reads nothing further. The same rule now governs which FILES are read: an ADR entry that is a symlink out of the repository is excluded and reported rather than parsed, closing the vector this change had widened by newly reading README.md, naming-violation files, and full bodies rather than only header fields. MAJOR - inline-span masking rescanned the line remainder per backtick run, roughly O(n^1.6) on adversarial input: 1.76s for an 800KB line. Rewritten as a single linear pass pairing runs through forward-only per-length cursors. Same input now takes 3.31ms, with behavior unchanged. MINOR - an unreadable or broken entry threw, and the generic handler wrote a raw stack trace carrying absolute CI paths to stderr. The scan is now fault-tolerant and reports excluded entries as ordinary violations. The status vocabulary is escaped before being interpolated into a dynamic RegExp - defence-in-depth, not a live bug. The containment predicate had reached three hand-written copies while fixing this; it is now the single escapesRoot() helper used by all four call sites. * feat(#2704): add a --json report so the gate's tests assert on typed values Maintainer-directed addition. CONTRIBUTING.md's "Prohibited: Raw Text Matching on Test Outputs" requires that a system under test producing text also expose a structured intermediate representation, and that tests assert on that IR rather than on rendered prose. This gate had no such surface, so its verdict tests matched on stderr. --json runs exactly the same validation as --check and writes a report to stdout with the same exit code, following the frozen-REASON-enum pattern already used by verify-reapply-patches.cjs. Every violation carries a stable reason code plus the fields a consumer needs, so nothing has to pattern-match an error message. Adding a reason stays three coordinated changes - the enum, the emitting site, and the test locking Object.keys(REASON).sort(). The human output is unchanged, deliberately: a large pre-existing suite asserts on it and migrating that is not this PR's concern. Verified by running the pre-change and post-change scripts against an identical violating corpus and diffing their stderr - character-for-character identical. This PR's own verdict tests now assert on parsed --json. Absence checks improve the most: "no bracket violation" is now a reason-code predicate rather than a negative regex over prose, which could pass for the wrong reason. The security assertions were strengthened rather than translated - no leaked filename may appear in ANY field of the serialized report. Unknown flags are now rejected instead of silently falling through to printing the index. * test(#2704): fix the status-parity fixture and guard hooks/dist before overlay builds Two failures from the matrix run of 79b29909. The status-parity fixture was mine. It built, per status token, an ADR whose H1 bracket and Status field both carried that token - but Superseded carries an obligation beyond the bracket: it must name its successor as a file link and be symmetric with it. The fixture declared a bare Superseded, tripped that unrelated invariant, and the test reported a bracket-parity failure for a reason that had nothing to do with bracket parity. The fixture now satisfies each token's own obligations in both the agreeing and contradicting corpora, derived from the status actually declared rather than special-cased on one name, so a future token carrying obligations is handled rather than silently skipped. The second failure was not mine but is fixed here rather than deferred. mcp-catalog-parity.install.test.cjs hardlinks hooks/dist/* while building its overlay, but hooks/dist is a gitignored build artifact produced only by build:hooks. The suite had no guard, so it passed only when some other suite happened to build it first - an execution-order dependency, which is why it failed on node22 and passed on node24 for identical code. install.test.cjs already documents this exact hazard and guards it. Six behaviorally identical copies of that guard existed across three files. Rather than add a seventh, they are now one canonical tests/helpers/hooks-dist.cjs - idempotent and bounded by the shared BUILD_TIMEOUT_MS class norm - which is the same single-sourcing this PR applies to the ADR gate itself. * docs(#2704): add a how-to for contributors the ADR gate rejects The reference and explanation quadrants were covered by Lifecycle rules 5 and 6, but the task-oriented one was thin: a contributor meets this gate because it failed on their PR, under pressure, and the rules told them what is checked without telling them what to do about it. Adds the command to reproduce the CI failure locally and a message-to-remedy table covering every reason code that can be hit - unresolved target, wrong case with the did-you-mean hint, repository escape, symlinked ADR file, bracket contradiction - plus the backtick escape hatch for illustrative links and the caveat that indented code blocks are not skipped. The table is itself written in backticked inline code, so the gate skips it: the escape hatch demonstrated on the page that documents it. * chore(#2704): backfill changeset PR number pr:0 placeholder replaced with the real PR number now that #3266 exists. --------- Co-authored-by: sim <sim@local>	2026-08-09 17:08:46 -04:00