b51cbf96cfbfad2fa4acac3d3c6d29c04aaf72d6
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tom Boucher
|
5fa4dcd78c |
fix: recover silently-excluded test dirs + test-architecture audit hardening (#1195)
* fix: recurse test discovery so subdir test suites actually run
scripts/run-tests.cjs discovered tests with a flat readdirSync(testDir),
silently excluding tests/observability/ (4 files), tests/dispatch/ (1) and
tests/installer-migrations/ (1) — 94 passing tests — from `npm test` and all
CI lanes. Walk the tree recursively (relative subpaths preserved), classify
suites by basename, and add a fail-on-zero-executed guard for suite/default
runs (escape hatch GSD_ALLOW_EMPTY_SUITE=1) while preserving the empty
--files/--files-from path the CI inert lane relies on.
Unit suite 735 -> 741 files; surfaces ADR-227's observability/dispatch seam.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: retire 5 verified-worthless tests
Adversarial verification confirmed these 5 prove nothing — their coverage is
provided more strictly elsewhere:
- enh-2790 'has a name: field' spot-checks (command-contract enforces /^gsd[:-]/)
- command-routing-hub duplicate construct + duplicate ERROR_KINDS assertions
- no-cjs-sdk-handsync-tooling (guarded files that never existed on main; bug-190
covers the real retired SDK artifacts)
- runtime-artifact-layout cline edge case (subsumed by the explicit-global test
and bug-782-cline-skills-emission)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: add ADR-218 release version-validation coverage
ADR-218 (reject leading-zero versions like 1.01.0; npm duplicate pre-check) had
zero tests — the logic lived only in release.yml bash. Add a test that extracts
the actual rejection regexes from the workflow and exercises them against a
boundary table (leading-zero/malformed rejected, valid accepted) plus structural
wiring assertions. Goes red if the regex is reverted to [0-9]+.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: redesign weak tests into behavioral, deterministic assertions
Per the ADR test audit, rewrite 27 weak test files (test-only, no source
changes) so each can go red for the defect it guards:
- kill pass-always assert.ok(true) placeholders (research-cli, worktree-baseref,
bug-260 security guard, eslint-rules x24, clusters '|| true')
- replace source-text grep with behavioral calls (install Kilo, sh-hook-paths,
plan-review-convergence) and add a repo-layout governance test
- de-flake real-clock/Math.random coupling (phase last_updated, bug-3707 mtime,
context-utilization property, feat-3594)
- fix independence/shared-state violations (bug-492 singleton, issue-844 tmpRoot,
core reapStaleTempFiles, active-workstream TTY, feat-488 GSD_HOME)
- strengthen property/shape-only tests (research-provider/store classification +
collision) and unconditional plugin.json schema validation (issue-766)
Verified: all 28 files run together 1220 pass / 0 fail / 1 skip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: add no-tautological-assert lint rule, error in test suite
New custom ESLint rule (eslint-rules/no-tautological-assert.cjs) bans asserts
that can never fail: assert(true)/assert.ok(<always-truthy literal>),
'cond || true' inside an assert, and equality asserts comparing two identical
literals. Wired as error on tests/**; full sweep confirmed zero existing
violations so the suite stays green. Prevents the placeholder-assert regressions
the audit redesigns just removed. RuleTester coverage added (6 valid, 8 invalid).
Note: no-only-tests was already enforced via eslint-plugin-no-only-tests, so no
duplicate rule was added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: gate new allow-test-rule exemptions to require an issue ref
ADR-456 requires any allow-test-rule exemption added after the ADR to carry a
tracking issue number, but nothing enforced it. New ratchet gate
(scripts/lint-allow-test-rule-refs.cjs, wired into lint:ci) fails when a NEW
allow-test-rule comment lacks a #NNN/URL reference; the 323 existing untracked
exemptions are grandfathered in an allowlist that ratchets down as they gain
refs. Red-green verified (novel untracked offender fails; compliant passes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: add ADR test-audit evidence report (#1192)
Full risk-first qa-test-architect audit of the ADR portfolio (37 ADRs + 4
platform lenses, adversarial verification of retire verdicts) that drove the
P0 discovery fix, ADR-218 coverage, 5 retires, 27 redesigns, and the two new
lint gates. Filed as point-in-time evidence under docs/issueevidence/, named
for tracking issue #1192.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: replace pre-existing raw NUL byte with escape in feat-3594 fixture
feat-3594's null-byte parser fixture contained a literal NUL byte (pre-existing
on next at
|
||
|
Tom Boucher
|
b8c33647d8 |
refactor(tests): retire output-grep & source-grep via typed surfaces (finish #2974) (#462)
* refactor(#455): implement typed surfaces to retire grep tests Production surfaces added: - hooks/managed-hooks-registry.cjs: new CJS module exporting MANAGED_HOOKS as a typed array; gsd-check-update-worker.js now requires it instead of declaring an inline array - bin/install.js: elevate inline gsdHooks to module-level GSD_UNINSTALL_HOOKS, export it alongside runtimeMap/allRuntimes (already exported) - scripts/build-hooks.js: export HOOKS_TO_COPY; guard build() behind require.main===module so tests can require the file without triggering a build - get-shit-done/bin/lib/init.cjs: add --json mode to agent-skills command, emitting typed IR { agent_type, block, skills_count } for test assertions - get-shit-done/bin/gsd-tools.cjs: wire --json flag for agent-skills dispatch Category-B source-grep migrations: - tests/managed-hooks.test.cjs: require MANAGED_HOOKS from registry, drop fs.readFileSync+regex - tests/orphaned-hooks.test.cjs: require MANAGED_HOOKS+HOOKS_TO_COPY as typed exports - tests/hooks-opt-in.test.cjs: replace gsdHooks regex-parse with GSD_UNINSTALL_HOOKS import - tests/install-minimal-hooks.test.cjs: replace gsdHooks regex-parse with GSD_UNINSTALL_HOOKS - tests/copilot-install.test.cjs: replace src.includes() checks with typed assertions on runtimeMap, allRuntimes, parseRuntimeInput, buildRuntimePromptText - tests/agent-skills.test.cjs: migrate to --json typed IR assertions pending-migration-to-typed-ir token cleared (87 of 87 files): - 78 files already had source-text-is-the-product; removed duplicate token - 5 files already used typed assertions; reclassified or annotated - 4 files required individual reclassification to source-text-is-the-product or architectural-invariant Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#455): update workflow-guard test to typed GSD_UNINSTALL_HOOKS import; isolate HOME in runtime-launcher (D) test Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#455): guard install.js main() behind require.main===module so the typed export is require-safe Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(#455): document --json typed surfaces for agent-skills, progress, validate context Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(#455): add changeset fragment for new --json surfaces Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#455): complete grep migration for files flagged by lint-tests The branch commit 4e630d99 stripped `allow-test-rule: pending-migration-to-typed-ir` from ~80 test files without replacing their assertions or adding the correct exemption annotation. The files were NOT source-grep tests — they read .md workflow/agent/command/reference files (source-text-is-the-product) or hook source files for structural invariants (structural-regression-guard). No assertion logic was changed; only the correct allow-test-rule annotation was added to each file per CONTRIBUTING.md exception matrix. 73 files: `source-text-is-the-product` — workflow/agent/command/reference .md 7 files: `structural-regression-guard` — hook .js / bin/install.js structural checks Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: CI Rebase Check <ci@gsd-redux> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Tom Boucher
|
918f987a19 |
feat(#2982): extend no-source-grep lint to catch var-binding readFileSync.includes() (#2985)
* feat(#2982): extend no-source-grep lint to catch var-binding readFileSync.includes() The base lint (scripts/lint-no-source-grep.cjs) only catches readFileSync(...).<text-method>() chained directly. The much more common var-binding form escapes it: const src = fs.readFileSync(p, 'utf8'); // 50 lines later if (src.includes('foo')) {} // ← still grep, lint missed it Scan of the test suite found ~141 files using this pattern. Implementation built TDD per #2982 with structured-IR assertions: scripts/lint-no-source-grep-extras.cjs - detectVarBindingViolations(src) — pure detector, two passes: pass 1 collects vars bound from readFileSync, pass 2 finds any <var>.<includes|startsWith|endsWith|match|search>( on those vars. - detectWrappedAssertOkMatch(src) — flags assert.ok(<expr>.match(...)) which escapes the assert.match rule. - VIOLATION enum exposes stable codes for tests to assert on. scripts/lint-no-source-grep.cjs - Wires the new detectors into the existing per-file check; one additional violation row per file with the first 3 sample tokens. tests/bug-2982-lint-var-binding.test.cjs - 13 tests, all assertions on typed VIOLATION enum / structured records. Covers all 5 text-match methods, multi-var, no-bind, string literal (must NOT trigger), wrapped assert.ok(.match), and assert.match (must NOT double-flag). Migration backlog (#2974 expanded scope): - 42 files annotated `// allow-test-rule: source-text-is-the-product` (legitimate — they read .md/.json/.yml files whose deployed text IS the product) - 3 files annotated `// allow-test-rule: pending-migration-to-typed-ir [#2974]` (read .cjs/.js source — clear migration debt) - 95 files annotated `pending-migration-to-typed-ir [#2974]` with `Per-file review may reclassify as source-text-is-the-product during migration` (mixed — manual review under #2974) After this lands the lint reports 0 violations on main; new violations in PRs surface immediately. Closes #2982 Refs #2974 * test(#2982): fix truncated test name per CR The label ended with a bare '(' from a copy-paste mishap. Now reads 'does NOT flag .matchAll(...) — matchAll is not match, so assert.ok(.matchAll(...)) is not flagged'. * chore(#2982): add changeset fragment for PR #2985 * chore(#2982): add changeset fragment for PR #2985 |
||
|
gg
|
759ff38d44 |
feat: add test quality audit step to verify-phase workflow
Adds a new `audit_test_quality` step between `scan_antipatterns` and `identify_human_verification` that catches test-level deceptions: - Disabled tests (it.skip/xit/test.todo) covering phase requirements - Circular tests (system generating its own expected values) - Weak assertions (existence-only when value-level proof needed) - Expected value provenance tracking for parity/migration phases Any blocker from this audit forces `gaps_found` status, preventing phases from being marked complete with inadequate test evidence. Fixes #1457 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |