golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

fix: recover silently-excluded test dirs + test-architecture audit hardening (#1195)

Browse Source 
* fix: recurse test discovery so subdir test suites actually run

scripts/run-tests.cjs discovered tests with a flat readdirSync(testDir),
silently excluding tests/observability/ (4 files), tests/dispatch/ (1) and
tests/installer-migrations/ (1) — 94 passing tests — from `npm test` and all
CI lanes. Walk the tree recursively (relative subpaths preserved), classify
suites by basename, and add a fail-on-zero-executed guard for suite/default
runs (escape hatch GSD_ALLOW_EMPTY_SUITE=1) while preserving the empty
--files/--files-from path the CI inert lane relies on.

Unit suite 735 -> 741 files; surfaces ADR-227's observability/dispatch seam.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: retire 5 verified-worthless tests

Adversarial verification confirmed these 5 prove nothing — their coverage is
provided more strictly elsewhere:
- enh-2790 'has a name: field' spot-checks (command-contract enforces /^gsd[:-]/)
- command-routing-hub duplicate construct + duplicate ERROR_KINDS assertions
- no-cjs-sdk-handsync-tooling (guarded files that never existed on main; bug-190
  covers the real retired SDK artifacts)
- runtime-artifact-layout cline edge case (subsumed by the explicit-global test
  and bug-782-cline-skills-emission)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: add ADR-218 release version-validation coverage

ADR-218 (reject leading-zero versions like 1.01.0; npm duplicate pre-check) had
zero tests — the logic lived only in release.yml bash. Add a test that extracts
the actual rejection regexes from the workflow and exercises them against a
boundary table (leading-zero/malformed rejected, valid accepted) plus structural
wiring assertions. Goes red if the regex is reverted to [0-9]+.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: redesign weak tests into behavioral, deterministic assertions

Per the ADR test audit, rewrite 27 weak test files (test-only, no source
changes) so each can go red for the defect it guards:
- kill pass-always assert.ok(true) placeholders (research-cli, worktree-baseref,
  bug-260 security guard, eslint-rules x24, clusters '|| true')
- replace source-text grep with behavioral calls (install Kilo, sh-hook-paths,
  plan-review-convergence) and add a repo-layout governance test
- de-flake real-clock/Math.random coupling (phase last_updated, bug-3707 mtime,
  context-utilization property, feat-3594)
- fix independence/shared-state violations (bug-492 singleton, issue-844 tmpRoot,
  core reapStaleTempFiles, active-workstream TTY, feat-488 GSD_HOME)
- strengthen property/shape-only tests (research-provider/store classification +
  collision) and unconditional plugin.json schema validation (issue-766)

Verified: all 28 files run together 1220 pass / 0 fail / 1 skip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: add no-tautological-assert lint rule, error in test suite

New custom ESLint rule (eslint-rules/no-tautological-assert.cjs) bans asserts
that can never fail: assert(true)/assert.ok(<always-truthy literal>),
'cond || true' inside an assert, and equality asserts comparing two identical
literals. Wired as error on tests/**; full sweep confirmed zero existing
violations so the suite stays green. Prevents the placeholder-assert regressions
the audit redesigns just removed. RuleTester coverage added (6 valid, 8 invalid).

Note: no-only-tests was already enforced via eslint-plugin-no-only-tests, so no
duplicate rule was added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: gate new allow-test-rule exemptions to require an issue ref

ADR-456 requires any allow-test-rule exemption added after the ADR to carry a
tracking issue number, but nothing enforced it. New ratchet gate
(scripts/lint-allow-test-rule-refs.cjs, wired into lint:ci) fails when a NEW
allow-test-rule comment lacks a #NNN/URL reference; the 323 existing untracked
exemptions are grandfathered in an allowlist that ratchets down as they gain
refs. Red-green verified (novel untracked offender fails; compliant passes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs: add ADR test-audit evidence report (#1192)

Full risk-first qa-test-architect audit of the ADR portfolio (37 ADRs + 4
platform lenses, adversarial verification of retire verdicts) that drove the
P0 discovery fix, ADR-218 coverage, 5 retires, 27 redesigns, and the two new
lint gates. Filed as point-in-time evidence under docs/issueevidence/, named
for tracking issue #1192.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: replace pre-existing raw NUL byte with escape in feat-3594 fixture

feat-3594's null-byte parser fixture contained a literal NUL byte (pre-existing
on next at b10e5681 — confirmed: base blob has 1 NUL, this fix has 0), which
made git treat the file as binary and would break grep/editors. Switch to the
\x00 escape; the runtime string value (a real NUL in the parser input) is
unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: address adversarial-review findings

Codex adversarial pass over the branch:
- capability-registry drift test no longer mutates the committed generated
  capability-registry.cjs in place (concurrency hazard) — uses in-memory
  checkPipeline comparison instead.
- allow-test-rule ratchet now detects exemptions in ALL comment forms (block
  /* */ too, matching no-source-grep) so a block comment can't bypass it;
  one newly-surfaced pre-existing offender grandfathered (323->324).
- install.test Kilo case asserts on what install(false,'kilo') actually writes
  rather than manually calling configureKiloPermissions (masked the call site).
- issue-766 drops the undeclared transitive ajv dep for explicit structural
  assertions from the schema fixture.
- adr-218 test notes the hotfix leading-zero gap is tracked in #1186.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: address code-review findings (subdir discovery, rule + test gaps)

xhigh code review surfaced 15 confirmed issues, all fixed:
- run-tests.cjs --files now resolves subdir tests by bare basename + handles
  Windows backslash paths (ambiguous basenames error clearly).
- affected-tests-lib.cjs listTestFiles made recursive — the targeted CI lane was
  silently dropping changed subdir tests (same false-green class the audit fixed).
- no-tautological-assert now catches 'true || cond' and empty []/{}  equality.
- verify-test-quality: restore provenance-classification coverage, tighten the
  writeFile circular-detection check, guard the module-level file read.
- sh-hook-paths: cover the global-install .sh delegation branch (#2045 guard).
- active-workstream null-guard runs deterministically (no longer skipped on TTY).
- adr-218 structural guards tightened (major/minor leading-zero; needs: membership).
- repo-layout AGENTS.md guard no longer false-alarms on equivalent refactors.
- cross-ai ordering guard fails red when the step is missing.
- issue-766 parses required fields from the schema fixture (auto-enforced).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: stub USERPROFILE alongside HOME in feat-488 (Windows parity)

The feat-488 redesign stubbed process.env.HOME but not USERPROFILE; os.homedir()
resolves from USERPROFILE on Windows, so the home stub was not hermetic there —
caught by windows-test-parity-guard (stubsHomeNoUserProfile). Save/set/restore
USERPROFILE symmetrically with HOME (delete-if-originally-undefined).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: reconcile allow-test-rule allowlist after rebase onto next

Rebasing onto current next pulled in merged PR #1170, which added
inventory-headings-countfree.test.cjs (a baseline allow-test-rule exemption) and
deleted inventory-counts.test.cjs. Grandfather the former and prune the latter so
the ratchet matches the merged tree. No new debt from this PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-06-13 23:35:08 -04:00
committed by GitHub
parent 3ebd13c45d
commit 5fa4dcd78c
44 changed files with 3958 additions and 757 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
scripts/affected-tests-lib.cjs
View File

@@ -259,10 +259,21 @@ function shouldRunFullSuite(changedFiles) {
}
function listTestFiles(repoRoot) {
return readdirSync(path.join(repoRoot, 'tests'))
.filter(file => file.endsWith('.test.cjs'))
.map(file => `tests/${file}`)
.sort();
const results = [];
function walk(dir, relBase) {
for (const entry of readdirSync(dir, { withFileTypes: true })) {
if (entry.isDirectory()) {
if (entry.name === 'node_modules') continue;
const nextRel = relBase ? `${relBase}/${entry.name}` : entry.name;
walk(path.join(dir, entry.name), nextRel);
} else if (entry.name.endsWith('.test.cjs')) {
const rel = relBase ? `${relBase}/${entry.name}` : entry.name;
results.push(`tests/${rel}`);
}
}
}
walk(path.join(repoRoot, 'tests'), '');
return results.sort();
}
/**
@@ -531,6 +542,7 @@ module.exports = {
buildForwardGraph,
buildReverseIndex,
buildTransitiveReverseIndex,
listTestFiles,
parseRelativeSpecifiers,
pickAffectedTests,
resolveBaseRef,

326
scripts/lint-allow-test-rule-refs.allowlist.json Normal file
View File

@@ -0,0 +1,326 @@
[
"tests/agent-classification-parity.test.cjs :: runtime-contract-is-the-product — docs/AGENTS.md section layout + docs/INVENTORY.md table ARE the classification surface being validated",
"tests/agent-frontmatter.test.cjs :: source-text-is-the-product",
"tests/agent-required-reading-consistency.test.cjs :: source-text-is-the-product",
"tests/agent-size-budget.test.cjs :: source-text-is-the-product",
"tests/agent-skills-awareness.test.cjs :: source-text-is-the-product",
"tests/ai-evals.test.cjs :: source-text-is-the-product",
"tests/analyze-dependencies.test.cjs :: source-text-is-the-product",
"tests/anti-pattern-enforcement.test.cjs :: source-text-is-the-product",
"tests/ask-user-questions-fallback.test.cjs :: source-text-is-the-product",
"tests/audit-fix-command.test.cjs :: source-text-is-the-product",
"tests/autonomous-allowed-tools.test.cjs :: source-text-is-the-product",
"tests/autonomous-converge.test.cjs :: source-text-is-the-product",
"tests/autonomous-decomposition.test.cjs :: source-text-is-the-product",
"tests/autonomous-interactive.test.cjs :: source-text-is-the-product",
"tests/autonomous-to-flag.test.cjs :: source-text-is-the-product",
"tests/bug-131-release-tarball-smoke-explicit-home.test.cjs :: integration-test-input",
"tests/bug-14-progress-auto-flag-dropped.test.cjs :: source-text-is-the-product",
"tests/bug-17-askuserquestion-option-cap.test.cjs :: source-text-is-the-product",
"tests/bug-170-workflow-fallback-install-hint.test.cjs :: workflow markdown is shipped product text; this test validates fallback hint literals across all workflow files",
"tests/bug-1891-file-resolution.test.cjs :: structural-implementation-guard",
"tests/bug-1924-preserve-user-artifacts.test.cjs :: source-text-is-the-product",
"tests/bug-1967-cache-invalidation.test.cjs :: source-text-is-the-product",
"tests/bug-211-launcher-home-fallback.test.cjs :: structural/behavioral regression for the ~/.claude fallback arm in",
"tests/bug-2136-sh-hook-version.test.cjs :: structural-regression-guard",
"tests/bug-214-phase-researcher-write-truncation-contract.test.cjs :: source-text-is-the-product",
"tests/bug-214-writer-agents-write-truncation-contract.test.cjs :: source-text-is-the-product",
"tests/bug-222-research-synthesizer-write-contract.test.cjs :: source-text-is-the-product",
"tests/bug-224-pick-stdout-capture.test.cjs :: structural-implementation-guard",
"tests/bug-2346-agent-read-loop-guards.test.cjs :: source-text-is-the-product",
"tests/bug-2388-plan-phase-no-branch-rename.test.cjs :: source-text-is-the-product",
"tests/bug-2396-makefile-test-priority.test.cjs :: source-text-is-the-product",
"tests/bug-2399-commit-docs-plan-phase.test.cjs :: source-text-is-the-product",
"tests/bug-2410-stream-checkpoint-heartbeats.test.cjs :: source-text-is-the-product",
"tests/bug-2419-project-researcher-agent.test.cjs :: source-text-is-the-product",
"tests/bug-2421-planner-grep-gate-hygiene.test.cjs :: source-text-is-the-product",
"tests/bug-2424-reapply-patches-baseline-detection.test.cjs :: source-text-is-the-product",
"tests/bug-2470-update-md-claude-path.test.cjs :: source-text-is-the-product",
"tests/bug-2492-context-coverage-gate.test.cjs :: source-text-is-the-product",
"tests/bug-2502-insert-phase-state-update.test.cjs :: source-text-is-the-product",
"tests/bug-2516-inherit-model-execute-phase.test.cjs :: source-text-is-the-product",
"tests/bug-2543-gsd-slash-namespace.test.cjs :: structural-regression-guard",
"tests/bug-2549-2550-2552-discuss-phase-context.test.cjs :: source-text-is-the-product",
"tests/bug-2559-stale-search-year.test.cjs :: source-text-is-the-product",
"tests/bug-2661-roadmap-sync-parallel.test.cjs :: source-text-is-the-product",
"tests/bug-2686-review-fix-worktree.test.cjs :: source-text-is-the-product",
"tests/bug-2698-crlf-install.test.cjs :: source-text-is-the-product",
"tests/bug-2770-annotate-deps-int-coerce.test.cjs :: source-text-is-the-product",
"tests/bug-2772-gitmodules-path-intersection.test.cjs :: source-text-is-the-product",
"tests/bug-2784-update-cache-clear-path.test.cjs :: structural-regression-guard",
"tests/bug-279-codex-agent-mapping.test.cjs :: source-text-is-the-product [adapter header contract in bin/install.js]",
"tests/bug-2808-skill-hyphen-name.test.cjs :: source-text-is-the-product",
"tests/bug-2831-opencode-home-path-prefix.test.cjs :: source-text-is-the-product",
"tests/bug-2839-review-fix-transactional-cleanup.test.cjs :: source-text-is-the-product",
"tests/bug-2948-spike-wrap-up-dispatch.test.cjs :: source-text-is-the-product",
"tests/bug-2949-sketch-wrap-up-dispatch.test.cjs :: source-text-is-the-product",
"tests/bug-2954-help-md-slash-command-stubs.test.cjs :: source-text-is-the-product",
"tests/bug-2973-profile-user-skills-path.test.cjs :: source-text-is-the-product. profile-user.md IS the",
"tests/bug-2990-code-fixer-worktree-branch.test.cjs :: source-text-is-the-product",
"tests/bug-3086-git-create-tag-config-gate.test.cjs :: workflow-markdown-is-the-runtime-contract",
"tests/bug-3096-ai-integration-phase-parallel-race.test.cjs :: reads product workflow markdown (ai-integration-phase.md) to verify structural ordering contract — not a source-grep test",
"tests/bug-3097-3099-executor-worktree-path-safety.test.cjs :: reads markdown product files (gsd-executor.md, worktree-path-safety.md) to verify structural protocol — not source-grep",
"tests/bug-3120-secure-phase-empty-register.test.cjs :: reads product workflow markdown (secure-phase.md) to verify structural guard contract — not a source-grep test",
"tests/bug-3126-global-skills-base-runtime-path.test.cjs :: last three tests read init.cjs source to verify delegation contract to runtime-homes.cjs — structural guard, no behavioral IR exposed",
"tests/bug-3127-state-begin-phase-idempotent.test.cjs :: reads runtime STATE.md written to temp dir — behavioral output test, not source-grep",
"tests/bug-3128-roadmap-plan-count-slug-layout.test.cjs :: reads roadmap.cjs source to verify isPlanFile pattern was adopted — structural contract prevents silent regression to old filter",
"tests/bug-3129-validate-commit-git-bypass.test.cjs :: reads hook shell script to verify delegation pattern — structural contract test, not source-grep",
"tests/bug-3130-update-npx-robust-invocation.test.cjs :: reads product workflow markdown (update.md) to verify structural invocation contract — not a source-grep test",
"tests/bug-3135-capture-backlog-workflow.test.cjs :: source-text-is-the-product — workflow and command .md files",
"tests/bug-3156-plan-phase-opencode-dispatch.test.cjs :: source-text-is-the-product",
"tests/bug-3168-task-to-agent-rename.test.cjs :: source-text-is-the-product",
"tests/bug-3236-capture-seed-one-shot.test.cjs :: source-text-is-the-product — workflow and command .md files",
"tests/bug-3258-no-stale-gsd-intel-references.test.cjs :: source-text-is-the-product — workflow, reference, and docs .md files",
"tests/bug-3290-intel-updater-layout-block.test.cjs :: source-text-is-the-product — agents/gsd-intel-updater.md IS",
"tests/bug-33-settings-model-profile-adaptive.test.cjs :: source-text-is-the-product",
"tests/bug-3381-verify-work-workstream.test.cjs :: source-text-is-the-product — verify-work.md is a runtime workflow contract.",
"tests/bug-3384-secondary-defects.test.cjs :: source-text-is-the-product",
"tests/bug-3418-progress-flag-routing.test.cjs :: source-text-is-the-product",
"tests/bug-3430-planner-phase-contract.test.cjs :: source-text-is-the-product",
"tests/bug-3431-debug-command-yaml.test.cjs :: source-text-is-the-product",
"tests/bug-3446-resume-continue-here-discovery.test.cjs :: source-text-is-the-product",
"tests/bug-3489-complete-phase-idempotent.test.cjs :: source-text-is-the-product",
"tests/bug-3491-nested-git-worktree.test.cjs :: source-text-is-the-product",
"tests/bug-3516-reapply-patches-gsd-update-filter.test.cjs :: source-text-is-the-product",
"tests/bug-3521-quick-cleanup-cwd-pin.test.cjs :: source-text-is-the-product",
"tests/bug-3523-cjs-loadconfig-branching-strategy-warning.test.cjs :: validates runtime CLI stdout/stderr warning behavior, not source grep",
"tests/bug-3542-executor-git-stash-prohibition.test.cjs :: source-text-is-the-product",
"tests/bug-3582-codex-skills-materialized.test.cjs :: source-text-is-the-product",
"tests/bug-3605-stale-research-insert-phase-agent-refs.test.cjs :: source-text-is-the-product",
"tests/bug-3628-bundled-hook-classifier-whitelist.test.cjs :: architectural-invariant",
"tests/bug-3657-verify-reapply-patches-pristine-drift.test.cjs :: source-text-is-the-product.",
"tests/bug-3657-verify-reapply-patches-pristine-drift.test.cjs :: source-text-is-the-product — Finding 2 reads reapply-patches.md to",
"tests/bug-3677-agent-colon-namespace-leak.test.cjs :: source-text-is-the-product",
"tests/bug-3678-executor-commit-docs-respect.test.cjs :: source-text-is-the-product",
"tests/bug-3683-command-colon-namespace-leak.test.cjs :: source-text-is-the-product",
"tests/bug-3683-command-cross-reference-invariant.test.cjs :: source-text-is-the-product",
"tests/bug-3683-workflow-colon-namespace-leak.test.cjs :: source-text-is-the-product",
"tests/bug-3689-resume-glob-nomatch.test.cjs :: source-text-is-the-product",
"tests/bug-3691-annotate-deps-plans-block-variants.test.cjs :: source-text-is-the-product",
"tests/bug-3706-ui-safety-gate-false-positives.test.cjs :: source-text-is-the-product",
"tests/bug-3707-locked-worktree-cleanup.test.cjs :: source-text-is-the-product",
"tests/bug-3727-code-review-fix-flag-dispatch.test.cjs :: source-text-is-the-product",
"tests/bug-378-update-check-scoped-name.test.cjs :: structural assertion on hook delegation; the behavior being",
"tests/bug-3784-gsd-settings-model-profile-ui-omits-adaptive.test.cjs :: source-text-is-the-product",
"tests/bug-3805-fast-md-log-to-state-schema.test.cjs :: source-text-is-the-product",
"tests/bug-3810-no-gsd-sdk-runtime-refs.test.cjs :: source-text-is-the-product",
"tests/bug-397-state-preserve-executor-authored.test.cjs :: reads runtime STATE.md written to temp dir — behavioral output test, not source-grep",
"tests/bug-444-resolver-local-claude-install.test.cjs :: structural/behavioral regression for the repo-local .claude/ install",
"tests/bug-474-clock-seam-date-determinism.test.cjs :: source-text-is-the-product",
"tests/bug-503-update-agent-antigravity-detection.test.cjs :: source-text-is-the-product",
"tests/bug-570-codex-leak-scanner.test.cjs :: source-text-is-the-product",
"tests/bug-571-doc-writer-fix-mode-edit-only.test.cjs :: source-text-is-the-product",
"tests/bug-619-codebase-drift-gate-shim.test.cjs :: source-text-is-the-product",
"tests/bug-621-plan-phase-gap-analysis-gsd-run.test.cjs :: source-text-is-the-product",
"tests/bug-622-graphify-optional-graph-html.test.cjs :: source-text-is-the-product",
"tests/bug-630-wave-cleanup-orchestrator-root.test.cjs :: source-text-is-the-product",
"tests/bug-637-workflow-no-hardcoded-home-tool.test.cjs :: source-text-is-the-product",
"tests/bug-685-windowshide-spawn.test.cjs :: source-text-is-the-product",
"tests/bug-687-agy-timeout.test.cjs :: source-text-is-the-product",
"tests/bug-704-codex-launcher-path-corruption.test.cjs :: source-text-is-the-product",
"tests/bug-851-codex-quick-adapter-agent-type-fallback.test.cjs :: source-text-is-the-product",
"tests/bug-891-non-claude-runtime-home-fallback.test.cjs :: structural/behavioral regression for non-Claude runtime-home",
"tests/bug-924-claude-flat-skill-layout.test.cjs :: source-text-is-the-product",
"tests/bug-936-no-nested-spawner-wrap.test.cjs :: source-text-is-the-product",
"tests/bug-947-hermes-gsd-prefix.test.cjs :: source-text-is-the-product",
"tests/bug-950-quick-summary-status-complete.test.cjs :: source-text-is-the-product",
"tests/bug-967-verify-key-links-strict-paths.test.cjs :: <runtime-contract-is-the-product> the plan-md.md reference",
"tests/bug-967-verify-key-links-strict-paths.test.cjs :: <runtime-contract-is-the-product> the plan-md.md reference example IS the documented authoring surface for key_links; asserting it uses a file path (not an endpoint) directly tests the documented contract.",
"tests/bug-983-trae-windsurf-claude-path-leak.test.cjs :: source-text-is-the-product",
"tests/bug-patterns-reference.test.cjs :: source-text-is-the-product",
"tests/chain-flag-plan-phase.test.cjs :: source-text-is-the-product",
"tests/changeset-cli.test.cjs :: reads a product workflow .md file (not CJS source) to verify",
"tests/check-update-config-dir.test.cjs :: structural-regression-guard",
"tests/ci-test-scope.test.cjs :: CLI usage banner presence is a user-facing contract.",
"tests/ci-test-scope.test.cjs :: CLI usage failure text is user-facing contract for this parser guard.",
"tests/claude-md.test.cjs :: source-text-is-the-product",
"tests/claude-skills-migration.test.cjs :: source-text-is-the-product",
"tests/cleanup-branch-pruning.test.cjs :: source-text-is-the-product",
"tests/cline-install.test.cjs :: source-text-is-the-product",
"tests/cline-support.test.cjs :: source-text-is-the-product",
"tests/clock-seam.test.cjs :: line 159 reads the STATE.md temp file written by readModifyWriteStateMd — this is a runtime output file assertion, not a source-grep; the API returns void so a file read-back is the only way to verify the transform was applied",
"tests/code-review-agent-skills.test.cjs :: source-text-is-the-product",
"tests/code-review-command.test.cjs :: source-text-is-the-product",
"tests/code-review-pipeline-regression.test.cjs :: source-text-is-the-product",
"tests/code-review.test.cjs :: source-text-is-the-product",
"tests/codebuddy-install.test.cjs :: source-text-is-the-product",
"tests/codex-config.test.cjs :: source-text-is-the-product",
"tests/command-contract.test.cjs :: source-text-is-the-product — commands/gsd/*.md files ARE the",
"tests/commands.test.cjs :: source-text-is-the-product",
"tests/concurrency-safety.test.cjs :: source-text-is-the-product",
"tests/config-field-docs.test.cjs :: docs-parity",
"tests/context-enrichment.test.cjs :: source-text-is-the-product",
"tests/contributor-standards.test.cjs :: source-text-is-the-product",
"tests/copilot-install.test.cjs :: integration-test-input",
"tests/core.test.cjs :: architectural-invariant",
"tests/core.test.cjs :: structural-regression-guard",
"tests/cursor-hooks.test.cjs :: source-text-is-the-product",
"tests/cursor-reviewer.test.cjs :: source-text-is-the-product",
"tests/debug-session-management.test.cjs :: source-text-is-the-product",
"tests/discuss-all-flag.test.cjs :: source-text-is-the-product",
"tests/discuss-checkpoint.test.cjs :: source-text-is-the-product",
"tests/discuss-mode.test.cjs :: structural-implementation-guard",
"tests/discuss-phase-power.test.cjs :: source-text-is-the-product",
"tests/docs-parity-live-registry.test.cjs :: source-text-is-the-product",
"tests/drift-detection.test.cjs :: source-text-is-the-product",
"tests/edge-probe-docs-fixtures.test.cjs :: runtime-contract-is-the-product — the rendered reference/SPEC/ADR vocab surfaces are the runtime contract; this pins their bijection to the code (docs-parity)",
"tests/edge-probe-planner-contract.test.cjs :: runtime-contract-is-the-product — plan-phase.md's planner prompt is the deployed runtime contract under assertion",
"tests/edge-probe-spec-phase-contract.test.cjs :: runtime-contract-is-the-product — spec-phase.md Step 5.5 is the deployed workflow runtime contract under assertion",
"tests/edit-phase.test.cjs :: source-text-is-the-product",
"tests/enh-2310-chunked-plan-phase.test.cjs :: source-text-is-the-product",
"tests/enh-2380-sync-skills.test.cjs :: source-text-is-the-product",
"tests/enh-2415-claude-md-link-mode.test.cjs :: source-text-is-the-product",
"tests/enh-2430-learnings-consumption.test.cjs :: source-text-is-the-product",
"tests/enh-2433-todo-phase-linking.test.cjs :: source-text-is-the-product",
"tests/enh-2446-milestones-drift.test.cjs :: source-text-is-the-product",
"tests/enh-2447-roadmap-wave-deps.test.cjs :: source-text-is-the-product",
"tests/enh-2448-artifact-registry.test.cjs :: source-text-is-the-product",
"tests/enh-2500-codebase-mapper-arch-rich-format.test.cjs :: source-text-is-the-product",
"tests/enh-2789-description-budget.test.cjs :: source-text-is-the-product",
"tests/enh-2790-skill-consolidation.test.cjs :: source-text-is-the-product",
"tests/enh-2792-namespace-skills.test.cjs :: source-text-is-the-product",
"tests/enh-3209-plan-phase-ingest-adr.test.cjs :: source-text-is-the-product",
"tests/enh-48-cwd-drift-guard-e2e.test.cjs :: integration-test-input",
"tests/enh-72-business-context.test.cjs :: source-text-is-the-product",
"tests/enh-769-context-fork-effort.install.test.cjs :: integration-test-input",
"tests/enh-770-claude-hook-events.test.cjs :: runtime-contract-is-the-product — hooks.json IS the",
"tests/enh-770-claude-hook-events.test.cjs :: runtime-contract-is-the-product — the hookEventName is",
"tests/enh-770-claude-hook-events.test.cjs :: runtime-contract-is-the-product — the stamp template token",
"tests/enh-770-claude-hook-events.test.cjs :: runtime-contract-is-the-product — the stdin-read and",
"tests/enh-773-codex-exec-automation-flags.test.cjs :: source-text-is-the-product",
"tests/enh-778-cross-runtime-command-enrichment.test.cjs :: source-text-is-the-product",
"tests/enh-789-codebuddy-commands.test.cjs :: source-text-is-the-product",
"tests/eslint-rules.test.cjs :: <source-grep reason> must still error",
"tests/eslint-rules.test.cjs :: pending migration",
"tests/eslint-rules.test.cjs :: source-text-is-the-product",
"tests/execute-phase-active-flags.test.cjs :: source-text-is-the-product",
"tests/execute-phase-step-5-5-deviation-doc.test.cjs :: source-text-is-the-product",
"tests/execute-phase-wave.test.cjs :: behavioral — calls gsd-tools and asserts structured output",
"tests/execute-phase-wave.test.cjs :: behavioral — exercises config-set validation, not source text",
"tests/execute-phase-wave.test.cjs :: behavioral — exercises gsd-tools wave-defaulting logic",
"tests/execute-phase-wave.test.cjs :: source-text-is-the-product",
"tests/execute-phase-worktree-artifacts.test.cjs :: source-text-is-the-product",
"tests/explore-command.test.cjs :: source-text-is-the-product",
"tests/extract-learnings.test.cjs :: source-text-is-the-product",
"tests/feat-22-surfacing-docs.test.cjs :: docs-parity",
"tests/feat-2840-issue-driven-orchestration-guide.test.cjs :: structural-IR parser for a docs guide. The .includes()",
"tests/feat-3039-help-tiered.test.cjs :: source-text-is-the-product",
"tests/feat-3167-ship-pr-body-sections.test.cjs :: source-text-is-the-product",
"tests/feat-3210-fallow-integration.test.cjs :: source-text-is-the-product",
"tests/feat-3210-fallow-integration.test.cjs :: source-text-is-the-product — code-review.md IS the workflow the orchestrator",
"tests/feat-3309-human-verify-mode.test.cjs :: source-text-is-the-product",
"tests/feat-443-effort-install-wiring.install.test.cjs :: integration-test-input",
"tests/feat-488-effort-sync.test.cjs :: structural-regression-guard — readFileSync asserts on installed agent .md files (the product under mutation) to verify dry-run safety and apply correctness; stderr.includes guards the CLI argument-rejection contract.",
"tests/fix-3722-execute-phase-human-needed-checkpoint.test.cjs :: source-text-is-the-product",
"tests/forensics.test.cjs :: source-text-is-the-product",
"tests/frontmatter-cli.test.cjs :: source-text-is-the-product",
"tests/gates-taxonomy.test.cjs :: source-text-is-the-product",
"tests/gsd-check-update-worker-platform-gate.test.cjs :: structural assertion on spawn-options shape; the behavior",
"tests/gsd-researcher-app-aware.test.cjs :: source-text-is-the-product",
"tests/gsd-researcher-flow-diagram.test.cjs :: source-text-is-the-product",
"tests/gsd-settings-advanced.test.cjs :: source-text-is-the-product",
"tests/gsd2-import.test.cjs :: source-text-is-the-product",
"tests/hermes-skills-migration.test.cjs :: source-text-is-the-product",
"tests/import-command.test.cjs :: source-text-is-the-product",
"tests/ingest-docs.test.cjs :: source-text-is-the-product",
"tests/inline-plan-threshold.test.cjs :: source-text-is-the-product",
"tests/install-minimal-hooks.test.cjs :: source-text-is-the-product",
"tests/install-nested-layout.test.cjs :: source-text-is-the-product",
"tests/install-runtime-artifacts.test.cjs :: source-text-is-the-product",
"tests/install.test.cjs :: runtime-contract-is-the-product",
"tests/install.test.cjs :: source-text-is-the-product",
"tests/intel.test.cjs :: source-text-is-the-product — agents/gsd-intel-updater.md IS the",
"tests/intel.test.cjs :: source-text-is-the-product — readFileSync assertions target API-SURFACE.md, which is the generated product of intelApiSurface; asserting on its text content is the only way to verify correct generation.",
"tests/inventory-headings-countfree.test.cjs :: runtime-contract-is-the-product — INVENTORY.md heading format is the shipped doc surface being locked",
"tests/ios-scaffold-safety.test.cjs :: source-text-is-the-product",
"tests/issue-2639-codex-toml-neutralization.test.cjs :: source-text-is-the-product",
"tests/issue-429-comment-text-gate.test.cjs :: source-text-is-the-product",
"tests/issue-498-package-identity.test.cjs :: architectural-invariant",
"tests/issue-498-update-backup-runtime-dir.test.cjs :: structural assertion on the deployed update.md backup bash;",
"tests/issue-57-runtime-install-no-drift.test.cjs :: delegation-presence guard. Catches wholesale removal of the registry",
"tests/issue-57-runtime-install-no-drift.test.cjs :: structural guard over bin/install.js source. Behavioral assertions",
"tests/issue-607-installer-dry-run.install.test.cjs :: integration-test-input",
"tests/issue-607-legacy-cleanup.test.cjs :: integration-test-input",
"tests/issue-787-cline-hooks-agents.test.cjs :: source-text-is-the-product",
"tests/issue-815-update-next-channel.test.cjs :: reads product workflow/command markdown to verify the --next RC channel contract — not a source-grep test",
"tests/locking-bugs-1909-1916-1925-1927.test.cjs :: architectural-invariant",
"tests/mcp-tool-inheritance.test.cjs :: source-text-is-the-product",
"tests/milestone-summary.test.cjs :: source-text-is-the-product",
"tests/milestone.test.cjs :: source-text-is-the-product",
"tests/milestone.test.cjs :: structural-regression-guard",
"tests/model-catalog-runtime-defaults.test.cjs :: source-text-is-the-product",
"tests/next-safety-gates.test.cjs :: source-text-is-the-product",
"tests/next-up-clear-order.test.cjs :: source-text-is-the-product",
"tests/no-hardcoded-home-gsd-tools.test.cjs :: source-text-is-the-product",
"tests/opencode-permissions.test.cjs :: architectural-invariant",
"tests/orphan-worktree-detection.test.cjs :: architectural-invariant",
"tests/orphaned-hooks.test.cjs :: structural-regression-guard",
"tests/package-legitimacy-gate.test.cjs :: source-text-is-the-product",
"tests/parallel-dependent-plans.test.cjs :: source-text-is-the-product",
"tests/path-replacement.test.cjs :: source-text-is-the-product",
"tests/phase-dependency-levels.test.cjs :: source-text-is-the-product",
"tests/phase.test.cjs :: source-text-is-the-product",
"tests/phase.test.cjs :: state-md-is-the-runtime-contract — regression tests for",
"tests/phase6-capability-docs.test.cjs :: source-text-is-the-product",
"tests/phase6-capstone-conformance.test.cjs :: source-text-is-the-product",
"tests/phase6-planning-capabilities.test.cjs :: source-text-is-the-product",
"tests/plan-bounce.test.cjs :: source-text-is-the-product",
"tests/plan-phase-drift-guard.test.cjs :: source-text-is-the-product",
"tests/plan-phase-ui-redirect.test.cjs :: source-text-is-the-product",
"tests/plan-review-convergence.test.cjs :: source-text-is-the-product",
"tests/planner-decomposition.test.cjs :: source-text-is-the-product",
"tests/planner-language-regression.test.cjs :: source-text-is-the-product",
"tests/playwright-ui-verify.test.cjs :: source-text-is-the-product",
"tests/policy-160-route0-resume.test.cjs :: source-text-is-the-product",
"tests/policy-release-no-npm-self-upgrade.test.cjs :: source-text-is-the-product",
"tests/product-name-purity.test.cjs :: source-text-is-the-product",
"tests/profile-output.test.cjs :: source-text-is-the-product",
"tests/progress-forensic.test.cjs :: source-text-is-the-product",
"tests/prompt-budget-cli.test.cjs :: prompt-content-is-the-product",
"tests/prompt-thinning.test.cjs :: source-text-is-the-product",
"tests/quick-session-management.test.cjs :: source-text-is-the-product",
"tests/qwen-skills-migration.test.cjs :: source-text-is-the-product",
"tests/read-guard.test.cjs :: source-text-is-the-product",
"tests/reapply-patches.test.cjs :: source-text-is-the-product",
"tests/reapply-verify-hunks.test.cjs :: source-text-is-the-product",
"tests/release-coverage-scope.test.cjs :: source-text-is-the-product",
"tests/release-tarball-smoke-workflow.test.cjs :: source-text-is-the-product",
"tests/release-tarball-smoke.install.test.cjs :: integration-test-input",
"tests/research-agent-profiles.test.cjs :: <runtime-contract-is-the-product> research agent .md content is the governed surface",
"tests/review-default-reviewers-workflow.test.cjs :: source-text-is-the-product",
"tests/roadmap.test.cjs :: source-text-is-the-product",
"tests/roadmapper-granularity.test.cjs :: source-text-is-the-product",
"tests/run-tests-harness.test.cjs :: run-tests.cjs is a CLI test harness whose only IR is its",
"tests/runtime-launcher-parity.test.cjs :: structural parity/drift guard — asserts literal presence/absence of the canonical gsd_run launcher and the retired $GSD_SDK / `/gsd-tools` tokens across workflow markdown; there is no typed IR for \"this source file does not contain substring X\".",
"tests/runtime-name-policy.test.cjs :: runtime-contract-is-the-product — FALLBACK_ALIASES source text IS the",
"tests/scan-command.test.cjs :: source-text-is-the-product",
"tests/secret-scan-lint.security.test.cjs :: source-text-is-the-product",
"tests/secure-phase.test.cjs :: source-text-is-the-product",
"tests/security-prompt-injection.security.test.cjs :: structural-regression-guard",
"tests/security-scan.security.test.cjs :: source-text-is-the-product",
"tests/seed-scan-new-milestone.test.cjs :: source-text-is-the-product",
"tests/settings-integrations.test.cjs :: source-text-is-the-product",
"tests/settings-jsonc.test.cjs :: structural-regression-guard",
"tests/skill-frontmatter-contract.test.cjs :: source-text-is-the-product",
"tests/spawn-liveness-banner.test.cjs :: source-text-is-the-product",
"tests/state-acquirestatelock-non-eexist.test.cjs :: architectural-invariant",
"tests/state.test.cjs :: source-text-is-the-product",
"tests/subagent-timeout.test.cjs :: source-text-is-the-product",
"tests/template.test.cjs :: source-text-is-the-product",
"tests/thinking-partner.test.cjs :: source-text-is-the-product",
"tests/thread-session-management.test.cjs :: source-text-is-the-product",
"tests/ultraplan-phase.test.cjs :: source-text-is-the-product",
"tests/verify-health.test.cjs :: source-text-is-the-product",
"tests/verify-test-quality.test.cjs :: source-text-is-the-product",
"tests/verify-work-auto-transition.test.cjs :: source-text-is-the-product",
"tests/windows-robustness.test.cjs :: source-text-is-the-product",
"tests/windows-test-parity-guard.test.cjs :: structural-regression-guard",
"tests/workflow-compat.test.cjs :: source-text-is-the-product",
"tests/workflow-guard-registration.test.cjs :: structural-regression-guard",
"tests/workflow-maintainer-skip.test.cjs :: source-text-is-the-product",
"tests/workflow-shell-pinning.test.cjs :: file-scope prefilter, not a test assertion — we need to",
"tests/workflow-size-budget.test.cjs :: source-text-is-the-product",
"tests/workspace.test.cjs :: source-text-is-the-product",
"tests/worktree-cleanup.test.cjs :: source-text-is-the-product",
"tests/worktree.test.cjs :: source-text-is-the-product"
]

162
scripts/lint-allow-test-rule-refs.cjs Normal file
View File

@@ -0,0 +1,162 @@
#!/usr/bin/env node
'use strict';
/**
* lint-allow-test-rule-refs.cjs — enforce that NEW `allow-test-rule:` exemption
* comments carry a tracking-issue reference.
*
* ## Why
*
* `allow-test-rule:` is an inline comment that disables the `no-source-grep`
* ESLint rule for a whole test file. Today many such comments exist with no
* issue reference, making it impossible to audit or revisit them. Per ADR-456
* (docs/adr/456-test-rigor-architecture.md) every NEW exemption must carry a
* `#NNN` issue reference or an https:// URL so the decision is traceable.
*
* ## What "compliant" means
*
* A compliant `allow-test-rule:` comment is one whose reason text (everything
* after the colon) contains either:
* - a `#\d+` token (e.g. `// allow-test-rule: see #1234`)
* - an https?:// URL
*
* Any other comment is an OFFENDER.
*
* ## Grandfathering
*
* All pre-existing untracked exemptions are recorded in
* scripts/lint-allow-test-rule-refs.allowlist.json (seeded at gate introduction
* time). The identity ratchet (scripts/lib/allowlist-ratchet.cjs) means:
* - A NEW non-compliant comment not in the allowlist → gate fails.
* - A previously-offending comment that is now compliant → allowlist entry is
* STALE and must be pruned (ratchet-down; the baseline only ever shrinks).
*
* ## Offender identifiers
*
* Identifiers are stable cross-rename-safe strings of the form:
* `<repo-relative-path> :: <trimmed-reason>`
*
* e.g. `tests/foo.test.cjs :: source-text-is-the-product`
*
* If a file has multiple non-compliant comments with the SAME reason text, only
* one identifier is recorded (deduped via Set).
*
* See docs/adr/456-test-rigor-architecture.md for the full policy.
*/
const fs = require('fs');
const path = require('path');
const { assertWithinAllowlist } = require('./lib/allowlist-ratchet.cjs');
const { ExitError, runMain } = require('./lib/cli-exit.cjs');
const ROOT = path.join(__dirname, '..');
const TESTS_DIR = process.env.GSD_LINT_ALLOW_TEST_RULE_TESTS_DIR || path.join(ROOT, 'tests');
const ALLOWLIST_PATH =
process.env.GSD_LINT_ALLOW_TEST_RULE_ALLOWLIST ||
path.join(__dirname, 'lint-allow-test-rule-refs.allowlist.json');
/**
* Extracts the reason text after `allow-test-rule:` from a single line of source
* text in any comment form that the no-source-grep ESLint rule honours.
*
* The ESLint rule tests `c.value` (AST comment node value, delimiters stripped)
* with /allow-test-rule:\s*\S/, which fires on BOTH:
* // allow-test-rule: <reason> (line comment)
* /* allow-test-rule: <reason> * / (block comment, single-line)
*
* By scanning line-by-line and extracting everything after `allow-test-rule:` on
* each line, we cover both forms without a cross-line regex (which was previously
* matching arbitrary `/* ... * /` pairs spanning hundreds of lines, causing false
* positives).
*
* The trailing `*\/` and whitespace are stripped so block-comment closers don't
* bleed into the extracted reason.
*/
const ALLOW_TEST_RULE_LINE_RE = /allow-test-rule:\s*(.+)/;
/** Matches a compliant issue reference or URL */
const ISSUE_REF_RE = /#\d+|https?:\/\//;
/**
* Recursively collect offender identifiers from all *.test.cjs files under dir.
*
* @param {string} dir absolute path to scan
* @returns {string[]} sorted, deduped list of `<relpath> :: <reason>` strings
*/
function collectOffenders(dir) {
const offenders = new Set();
function scan(current) {
for (const entry of fs.readdirSync(current, { withFileTypes: true })) {
const full = path.join(current, entry.name);
if (entry.isDirectory()) {
scan(full);
} else if (entry.isFile() && entry.name.endsWith('.test.cjs')) {
const relpath = path.relative(ROOT, full).split(path.sep).join('/');
let content;
try {
content = fs.readFileSync(full, 'utf8');
} catch {
// skip unreadable files (e.g. binary)
continue;
}
// Scan line-by-line. By testing each line for `allow-test-rule:` we
// cover BOTH comment forms without a cross-line regex:
// // allow-test-rule: <reason> ← line comment
// /* allow-test-rule: <reason> */ ← single-line block comment
//
// For each matching line we extract the reason (everything after the
// colon), then strip any trailing block-comment closer `*/` and
// whitespace so the identifier stays clean.
for (const line of content.split('\n')) {
const m = ALLOW_TEST_RULE_LINE_RE.exec(line);
if (!m) continue;
// Strip trailing block-comment closer and whitespace if present
const reason = m[1].replace(/\s*\*\/\s*$/, '').trim();
if (!reason) continue;
if (ISSUE_REF_RE.test(reason)) continue; // compliant — skip
offenders.add(`${relpath} :: ${reason}`);
}
}
}
}
scan(dir);
return [...offenders].sort();
}
function main() {
const args = process.argv.slice(2);
const unknown = args.filter((a) => a !== '--help');
if (unknown.length > 0) {
throw new ExitError(2, `lint-allow-test-rule-refs: unknown argument(s): ${unknown.join(', ')}`);
}
const current = collectOffenders(TESTS_DIR);
const known = JSON.parse(fs.readFileSync(ALLOWLIST_PATH, 'utf8'));
const failures = [];
const { novel } = assertWithinAllowlist({
label: 'allow-test-rule-refs',
current,
known,
fail: (msg) => failures.push(msg),
pruneHint: 'edit scripts/lint-allow-test-rule-refs.allowlist.json',
});
if (failures.length > 0) {
for (const msg of failures) process.stderr.write(`${msg}\n`);
if (novel.length > 0) {
process.stderr.write(
'\nNew allow-test-rule exemption without an issue ref — add `see #NNN` per ADR-456' +
' (docs/adr/456-test-rigor-architecture.md).\n'
);
}
throw new ExitError(1);
}
console.log(
`ok lint-allow-test-rule-refs: ${current.length} grandfathered exemption(s) tracked, no novel untracked offenders`
);
}
runMain(main);

80
scripts/run-tests.cjs
View File

@@ -21,7 +21,7 @@
'use strict';
const { readdirSync } = require('fs');
const { join } = require('path');
const { join, basename } = require('path');
const { execFileSync } = require('child_process');
const { ExitError, runMain } = require('./lib/cli-exit.cjs');
@@ -112,6 +112,21 @@ function ensureBuiltArtifacts(overrides = {}) {
}
const MARKED_SUITES = ['integration', 'install', 'security', 'slow'];
// Recursively collect *.test.cjs files under dir, returning paths relative to dir.
// Skips node_modules to avoid accidentally picking up decoy files.
function walkTestFiles(dir, relBase) {
const results = [];
for (const entry of readdirSync(dir, { withFileTypes: true })) {
if (entry.isDirectory()) {
if (entry.name === 'node_modules') continue;
results.push(...walkTestFiles(join(dir, entry.name), relBase ? `${relBase}/${entry.name}` : entry.name));
} else if (entry.name.endsWith('.test.cjs')) {
results.push(relBase ? `${relBase}/${entry.name}` : entry.name);
}
}
return results;
}
function parseArgs(argv) {
let suite = null;
let seen = false;
@@ -188,9 +203,12 @@ function parseArgs(argv) {
// Return the marked suite name embedded in a filename, or null if it's unmarked.
// foo.security.test.cjs -> "security"
// foo.test.cjs -> null (unit)
// Accepts either a bare filename or a relative subdir path; classification is
// based on the basename only so subdir paths classify identically to root files.
function suiteOf(filename) {
if (!filename.endsWith('.test.cjs')) return null;
const base = filename.slice(0, -'.test.cjs'.length);
const name = basename(filename);
if (!name.endsWith('.test.cjs')) return null;
const base = name.slice(0, -'.test.cjs'.length);
const lastDot = base.lastIndexOf('.');
if (lastDot === -1) return null;
const marker = base.slice(lastDot + 1);
@@ -213,7 +231,8 @@ function splitFileList(value) {
.split(/[,\s]+/)
.map(v => v.trim())
.filter(Boolean)
.map(v => v.replace(/^tests[\\/]/, ''));
.map(v => v.replace(/\\/g, '/')) // normalize Windows backslashes
.map(v => v.replace(/^tests\//, ''));
}
function selectExplicitFiles(allFiles, filesValue, filesFrom) {
@@ -222,8 +241,19 @@ function selectExplicitFiles(allFiles, filesValue, filesFrom) {
? splitFileList(fs.readFileSync(filesFrom, 'utf8'))
: splitFileList(filesValue);
const available = new Set(allFiles);
// Build a basename -> [relpath, ...] index for bare-basename resolution.
// A bare basename (no directory separator) may match exactly one subdir file.
const basenameIndex = new Map();
for (const f of allFiles) {
const b = basename(f);
if (!basenameIndex.has(b)) basenameIndex.set(b, []);
basenameIndex.get(b).push(f);
}
const selected = [];
const missing = [];
const errors = [];
for (const file of requested) {
// If the token is a bare suite name (e.g. "unit" written by ci-test-scope
// as the #408 fallback sentinel), delegate to the existing suite resolver
@@ -234,11 +264,27 @@ function selectExplicitFiles(allFiles, filesValue, filesFrom) {
selected.push(f);
}
} else if (available.has(file)) {
// Exact relpath match (e.g. "installer-migrations/001-legacy-orphan-files.test.cjs").
selected.push(file);
} else if (!file.includes('/')) {
// Bare basename (no directory separator): resolve via index.
const candidates = basenameIndex.get(file);
if (!candidates || candidates.length === 0) {
missing.push(file);
} else if (candidates.length > 1) {
errors.push(
`ambiguous basename "${file}" matches multiple files: ${candidates.join(', ')} — pass the subdir path instead`,
);
} else {
selected.push(candidates[0]);
}
} else {
missing.push(file);
}
}
if (errors.length > 0) {
return { error: errors.join('; ') };
}
if (missing.length > 0) {
return {
error: `requested test file(s) not found: ${missing.join(', ')}`,
@@ -266,17 +312,16 @@ function main() {
? process.env.GSD_TEST_DIR
: join(__dirname, '..', 'tests');
const allFiles = readdirSync(testDir)
.filter(f => f.endsWith('.test.cjs'))
.sort();
const allFiles = walkTestFiles(testDir, '').sort();
if (allFiles.length === 0) {
console.error(`No test files found in ${testDir}`);
throw new ExitError(1);
}
const usingExplicitFiles = parsed.files !== null || parsed.filesFrom !== null;
let selectedNames;
if (parsed.files !== null || parsed.filesFrom !== null) {
if (usingExplicitFiles) {
const explicit = selectExplicitFiles(allFiles, parsed.files, parsed.filesFrom);
if (explicit.error) {
console.error(`run-tests: ${explicit.error}`);
@@ -289,11 +334,20 @@ function main() {
const selected = selectedNames.map(f => join(testDir, f));
if (selected.length === 0) {
// Empty suite: report and exit 0 so empty lanes (e.g. `security` before
// adversarial tests land) don't gate CI. CI consumers wanting strictness
// can grep stderr for "no tests in suite".
console.error(`run-tests: no tests in suite "${suite || 'all'}"`);
return 0;
if (usingExplicitFiles) {
// Empty file list from --files/--files-from: allowed (e.g. CI passes an
// empty .ci-selected-tests.txt on docs-only/inert PRs). Exit 0 silently.
console.error(`run-tests: no tests in suite "${suite || 'all'}"`);
return 0;
}
// Empty suite/default run: this means discovery or the suite filter is broken.
// Allow GSD_ALLOW_EMPTY_SUITE=1 as an escape hatch (downgrades to a warning).
if (process.env.GSD_ALLOW_EMPTY_SUITE === '1') {
console.error(`run-tests: WARNING: 0 test files selected for suite "${suite || 'all'}" — discovery or suite filter may be broken (GSD_ALLOW_EMPTY_SUITE=1 suppressed the error)`);
return 0;
}
console.error(`run-tests: ERROR: 0 test files selected for suite "${suite || 'all'}" — discovery or suite filter is broken`);
throw new ExitError(1);
}
// Build the gitignored bin/lib artifact if absent, before any test requires it.