4 Commits

Author SHA1 Message Date
Jakub Zych
6cfa0c55d2 refactor: drop 12 runtimes, keep Claude, Codex, OpenCode, Cursor, ZCode, Antigravity
Removes kilo, kimi, kimi-code, copilot, windsurf, augment, trae, qwen, hermes,
cline, codebuddy and pi end to end: capability descriptors, installer branches
and converters (bin/install.js 14.9k -> 11.2k lines), TypeScript converters,
hook surfaces and runtime homes, review lanes qwen/kimi-code, the two pi
migrations, Kimi payload normalization in the hook guards, dead hostBehaviors
vocabulary, launcher home probes, fixtures, runtime-specific tests and the
prose that presented them as supported.

Installer output for the six kept runtimes is byte-identical to before the
prune. The Kimi tool-vocabulary tests in workflow-guard, read-guard and
read-injection-scanner are left in place pending a decision.
2026-10-06 20:02:40 +02:00
Tom Boucher
2131fe13f3 enhance(#3464): exec() detection widening, citation-debt cleanup — Phase 8 (#4171)
* feat(#3464): widen no-source-grep to detect regex.exec() on tracked text

Adds an execCall kind alongside the existing regexTest detection --
regex.exec(tracked) was invisible to the rule while regex.test(tracked)
was already caught, despite both reading a source-derived string through
a regex. Measured: 4 previously-invisible sites across 2 files.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(#3464): migrate 4 sites newly flagged by the exec() widening

docs-hooks-table-parity.test.cjs's three regex-extraction loops are
site-scoped marked (source-text-is-the-product) -- the dynamic
preToolEvent/postToolEvent dialect branching they mirror is explicitly
documented as not statically parseable, so a literal-pattern mirror is
the practical minimum-cost check.

no-bare-gsd-tools-command-position.test.cjs's readRouterVerbs() now
requires HOST_COMMAND_ROUTERS directly instead of regex-walking
gsd-tools.cjs's source text -- the same accessor three other suites
already use.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#3464): pay down 6 grandfathered uncited allow-test-rule markers

Two were genuinely load-bearing (suppressing a real detected violation)
and just needed a citation added -- phase6-capstone-conformance.test.cjs,
runtime-name-policy.test.cjs, both now (#3464).

Four were dead-weight file-header markers suppressing nothing -- each
file's real effective sites are covered by separate, already-cited
markers elsewhere in the same file. Deleted outright rather than cited,
per Phase 1's own precedent (remove non-load-bearing markers instead of
grandfathering them forever) -- codex-config.test.cjs (two copies),
gsd-check-update-worker-platform-gate.test.cjs, orphaned-hooks.test.cjs,
settings-jsonc.test.cjs.

allowlist.json: 134 -> 128 entries.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(#3464): re-baseline effective-exemption ceiling to 84

The exec() widening's 3 newly-marked sites are now suppressed and
counted; ceiling rises 81 -> 84, the exact measured high-water mark.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#3464): correct citation and restore a wrongly-deleted marker

Two review corrections, both found by the orthogonal review pass:

- docs-hooks-table-parity.test.cjs's 3 new exec() markers cited #3464
  (mechanically "the phase that widened the rule") when the file's own
  established, correct reference is #3839 (the issue this whole test
  exists to enforce, already cited in its file header) -- fixed to match.

- gsd-check-update-worker-platform-gate.test.cjs's deleted file-header
  marker was NOT dead weight: its codeOnly() helper wraps readFileSync
  and is called inline as an assert argument, a genuine source-grep
  pattern on real .cjs/.js source that the rule cannot currently see
  (helper-function indirection is a distinct blind spot from anything
  Phase 7/8 measured) -- CONTRIBUTING.md is explicit that "unverified"
  is not the same as "vestigial." Restored, site-scoped this time
  (directly above codeOnly(), not as an inert file-header comment) and
  cited (#3103, the issue the file's own docstring already references).

codex-config.test.cjs's two deletions and orphaned-hooks.test.cjs's /
settings-jsonc.test.cjs's deletions were independently re-verified and
stand: their flagged lines read generated .toml/.json OUTPUT, not
source, or have no residual pattern at all.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-02 08:11:23 -04:00
Tom Boucher
f16ff7d1b3 enhance(#3545): widen no-source-grep with fold+hooks, migrate 76 sites (#4161)
* feat(#3545): widen no-source-grep with one-hop path-fold and hooks dir

Resolve a readFileSync() path argument that is a bare Identifier one hop
back to its VariableDeclarator initializer before classification, and
recognize `hooks` as a source directory alongside bin/lib/gsd-core/src.

Measured (epic #3464 phase 7): fold+hooks together newly flag 76
unsuppressed sites across 18 files that were previously invisible to
identifier-indirected or hooks/-rooted source reads. Neither widening
alone is sufficient — hooks-only surfaces 0 new sites, confirming #3520's
prior finding that the identifier-indirection gap must close first.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(#3545): migrate 76 sites newly flagged by the fold+hooks widening

Per-site classification: rewrite behaviorally (require() the real module,
assert on its actual exported behavior) wherever the read was a proxy for
code behavior; add a site-scoped `// allow-test-rule: <reason> (#3545)`
marker only where the raw source text genuinely is the product under test
(codex-config.test.cjs's adapter-header-contract checks, install.js
structural-wiring guards with no exported symbol, AST-parse fixture
inputs, etc.) — each marker cites an existing repo-sanctioned category
from CONTRIBUTING.md's allow-test-rule exception table.

Also converts two try/finally test bodies (introduced during this same
migration) to the required t.after() cleanup pattern per CONTRIBUTING.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* chore(#3545): re-baseline effective-exemption ceiling to 81

The fold+hooks widening's own newly-detected sites are now suppressed by
site-scoped markers, moving them from invisible into the tightly-ratcheted
effective-exemption count. Ceiling rises from 10 to 81 (the exact measured
high-water mark, grace unchanged at 2) — a deliberate, measured re-baseline
per the widening working as intended, not an ordinary ceiling bump.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#3545): use canonical allow-test-rule category tokens

4 markers added during migration cited an issue ref correctly but didn't
use one of CONTRIBUTING.md's seven recognized category tokens, unlike
every other marker in this change. Cosmetic only — same suppression
lines, same effective/live counts (81/81, 0 live).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(#3545): correct stale phase-artifact path in test comment

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-01 21:40:38 -04:00
Tom Boucher
bc557f6876 chore(#3520): ratchet on effective exemptions, track unverified separately (#3529)
Phase 5 of #3464, following #3465, #3466, #3502 and #3508. Those cut the
ceiling 305 -> 278, made the rule accurate, and ended file-wide amnesty. This
one fixes the number itself.

scripts/lint-allow-test-rule-refs.cjs counted FILES CONTAINING MARKER TEXT.
Only 5 of those files carry a marker that actually suppresses a violation the
rule detects, across 10 sites. The ratcheted number was ~98% noise, which is
exactly why bumping it was frictionless: the metric was never coupled to the
thing it claimed to govern. That is the whole complaint this epic opened with,
stated precisely.

Verified directly rather than assumed: only eslint-rules/no-source-grep.cjs
functionally honors the marker. Four other rule files mention allow-test-rule
in prose only, and no-raw-rmsync-in-tests.cjs:24 explicitly states it does not
apply. So the large count was not legitimately large because several rules
share the annotation.

Now two numbers, only the first ratcheted:

  EFFECTIVE EXEMPTIONS -- markers that actually suppress a detected violation.
  10 sites across 5 files. Tightly ratcheted in both directions, as before:
  over the ceiling fails, and slack beyond grace fails.

  UNVERIFIED MARKERS -- marker-bearing files with no detectable violation. 273
  files. Reported and given a loose ceiling so the pool cannot silently
  balloon, but deliberately NOT tightly ratcheted, because shrinking it is a
  rule-coverage problem and not a delete-the-markers problem.

A file with at least one effective site counts as effective and is not also
counted as unverified; the two numbers never double-count.

Reporting ONLY the effective count was considered and rejected. It would say
five files and look excellent while being falsely reassuring, because "no
detectable violation" is not "no violation". This phase's own measurement found
two genuine source-greps that are unsuppressed AND undetected --
tests/security-prompt-injection.security.test.cjs:852 and
tests/check-update-config-dir.test.cjs:91 -- each reading a real shipped file
and text-searching it, invisible only because the path is bound to a separate
const the rule never resolves back to its literal. Markers guarding that class
count as zero-effective and would look vestigial. Trading a number that is too
big and meaningless for one that is too small and falsely reassuring is not
progress, so the script prints the known-limit caveat alongside the numbers and
the two undetected violations are filed separately rather than lost.

Single source of truth is structural, not a matter of discipline. The script
does not re-implement detection or the site-scoped adjacency predicate -- that
is the generative-fix-divergence class this repo has shipped before. The rule
now exports MAX_MARKER_LOOKAHEAD_LINES, MARKER_COMMENT_RE,
collectMarkerAndCommentLines and isSuppressedAt (extracted verbatim, no logic
change), plus a default-off neutralizeSuppression option so the counter can
enumerate every site through the real rule via ESLint's Linter API and then
classify each with the rule's own predicate, replicating reportUnlessSuppressed's
search-line-OR-read-line check exactly. Default rule behavior is byte-identical:
tests/eslint-rules.test.cjs passes 168/168 unchanged. A parity test asserts the
script's suppressed/not verdict equals the rule's own report/no-report outcome
for every site in a fixture corpus.

A real silent-failure bug surfaced and was fixed while building this: ESLint's
flat-config Linter reports "No matching configuration" and returns ZERO messages
for any filename resolving outside its cwd. That would have quietly
misclassified every sandboxed test fixture as having no violations -- a test
suite that passes while asserting nothing. Fixed by anchoring the Linter to the
tests dir, with a defensive throw if it ever recurs.

Two earlier claims of mine are corrected by this phase's measurement. Widening
the source-dir allowlist to include hooks/ -- the "fifth blind spot" recorded in
#3508 -- rescues ZERO sites; it is real in principle and has no practical
effect, because the hooks reads that exist are missed for other reasons
(.sh extension, identifier-indirection, dynamic filenames). And the #3508
correction that attributed those reads to the hooks/ gap rather than to variable
indirection was itself incomplete: both are independently sufficient, so fixing
either alone changes nothing. I accepted the reviewer's causal claim as
uncritically as I had made my own.

The unverified count is 273, not the ~289 in the phase design doc. That is
legitimate drift -- the baseline was measured at fba7c9032 and other merged work
has since removed markers. Left as measured rather than adjusted to match the
document.

Adversarial review found a BLOCKER in the first revision and it is fixed here.
The counter walked only tests/**/*.test.cjs, but the rule is registered on
tests/**/*.cjs -- every .cjs, not just test files -- plus scripts/**,
eslint-rules/**, bin/lib/**, pi/**, examples/**, gsd-core/bin/** and three
plugin globs. 35 non-.test.cjs files under tests/ were never walked, and
tests/helpers/live-command-registry.cjs:1 carries a real marker that appeared in
NEITHER reported number. A counter that undercounts is worse than the
meaningless one it replaces, because it will be trusted.

The scan set is now derived programmatically: the script dynamically imports
eslint.config.mjs and extracts the `files` globs from every config block that
enables local/no-source-grep. There is no hardcoded list to drift, which is the
same divergence class the parity test already guards.

Two further defects surfaced while fixing it. The silent-clean catch around
linter.verify() was worse than reported -- ESLint signals a parse error by
returning a message with fatal:true rather than throwing, so the original catch
would not even have fired for the common case. Both paths now throw with the
file path. That silent swallow was actively hiding a broken fixture in this
suite's own tests: 'no marker here\n' is not valid JS and the case only
"passed" because the parse failure was discarded. Fixed.

And once the scan widened, the raw-text marker scanner started matching this
tooling's own doc comments and RuleTester fixture strings, so marker extraction
moved to AST comment nodes. That corrected three long-standing FALSE POSITIVES:
allowlist entries for tests/eslint-rules.test.cjs that were never real markers,
only fixture payload. Allowlist 137 -> 135: three false positives pruned, one
real entry added for live-command-registry.

The headline numbers are coincidentally unchanged (10/10 effective across 5
files, 273/280 unverified) but the composition is corrected: one real file
gained, one phantom dropped. Verified by direct diff rather than inferred from
the totals matching.

The first remote run of this branch came back RED with 23 failures, all one
cause, and it is fixed here. Classification drove ESLint's flat-config Linter,
which resolves configuration relative to a cwd and refuses to lint any file
outside it. The test harness writes fixtures into an OS temp dir, so every
sandbox row hit "No matching configuration found" and tripped the defensive
throw. It surfaced only on the container because the repo lives at /work there
and the fixtures at /tmp, making the mismatch unmissable; a local run had
reported the suite green, which it was not.

Fixed by not depending on config *resolution* at all: the script now builds an
eslintrc-format Linter and registers the rule directly with defineRule, since it
already knows exactly which rule and options it wants. That removes the
cwd-ancestor constraint entirely and makes repo files and out-of-tree fixtures
classify identically. Verified out-of-tree explicitly, not just in-repo, because
the local temp path shape is what hid the bug the first time. Parse failures
still throw loudly -- that behavior is required and tested. Classifications are
unchanged for real repo files (10/10 effective across 5, 273/280 unverified,
0 live), which is the check that the config swap did not quietly alter results.

The second remote run cut the failures from 23 to 2, and the survivors were a
genuinely different and more interesting defect:
tests/packaging-shipped-scripts-require-only-shipped.test.cjs caught that
scripts/ SHIPS in the published package while eslint-rules/ does not, so
importing the rule for single-source-of-truth would MODULE_NOT_FOUND in a real
consumer's install. That test statically extracts require() calls, so hiding the
import inside a function would have dodged the check without fixing the problem.

Resolved along the grain of existing convention rather than by weakening
anything: package.json already excludes several repo-internal lint gates from
the shipped set via `!scripts/...`, including
`!scripts/lint-no-adhoc-regex-escape.cjs`, which is the same situation. This
gate is CI-only and has no meaning in a consumer install, so it joins them.
Verified with `npm pack --dry-run` that the .cjs is genuinely absent from the
tarball (its inert JSON config files remain, and carry no requires).

The third remote run failed on shard 3/3 across all three OSes with exit code
NULL and empty output -- the spawned gate was killed by a timeout, not failing
an assertion. Cause: replacing a raw text scan with a full ESLint Linter pass
over every file in every registered glob took the gate from 0.57s to ~7-12s,
and GitHub's runners are slower than the bench that had just passed it green.

Fixed by narrowing the work rather than raising the timeout to hide it. Both
numbers the gate computes are properties of files that CONTAIN a marker, so
only those (~294) need linting; the repo-wide byte walk that finds them stays,
since that was the undercount fix. Live violations in files carrying NO marker
are already enforced by npm run lint over exactly these globs, so re-detecting
them here was redundant. 2.87s now, from ~7s measured locally.

That narrowing changes what one reported number means, so the wording changed
with it: "live violations in marker-bearing files: 0 (unmarked files are
enforced separately by npm run lint)". A number that quietly covers less than
it reads is the failure this whole epic is about, so it is stated rather than
left implicit, and the test row that asserted the old broader behavior was
split -- an unmarked live violation now passes this gate (and is caught by
eslint), while a live violation in a marker-bearing file still fails it.

The repo-baseline test's timeout was also raised to 30s with a comment, since a
gate that legitimately takes ~3s must not sit at a timeout close to its own
runtime. Every other row keeps the shorter sandbox-scoped timeout.

Closes #3520

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-15 00:33:31 -04:00