The windows-test-parity ratchet greps test source for fs.rmSync-without-
maxRetries (and six other Windows-portability anti-patterns), failing when an
integer offender COUNT exceeds a frozen baseline (rmSync: 95). A count ratchet
is a Goodhart metric: fixing one offender and adding another keeps the count
constant, so a new defect slips through green. Replace it — and every other
count ratchet in the repo — with a layered, masking-proof design.
Behavioral seam test
- tests/helpers-cleanup.test.cjs proves helpers.cleanup() carries the Windows
EBUSY retry budget. cleanup() delegates retries to Node's fs.rmSync via
maxRetries (it owns no loop), so the test asserts the option contract
(recursive/force/maxRetries>0/retryDelay>0) + real-FS removal + the cwd-guard,
rather than a loop that does not exist. The EBUSY risk is now tested ONCE at
the helper, not approximated textually at every call site.
Write-time ESLint rule (AST-accurate, replaces the grep)
- eslint-rules/no-raw-rmsync-in-tests.cjs (error in tests/**/*.test.cjs) bans
raw fs.rmSync, steering to cleanup(). Catches member, computed (fs['rmSync']),
destructured and aliased forms; escape hatch is inline
`// eslint-disable-next-line local/no-raw-rmsync-in-tests -- <reason>` only.
- Migrated 336 raw fs.rmSync teardown calls across ~116 test files to cleanup().
~18 genuinely load-bearing sites (mid-test SUT/fault-injection removals,
error-swallowing or name-colliding local teardown helpers) keep the raw call
with an inline eslint-disable + reason.
Shared anti-ratchet primitive
- scripts/lib/allowlist-ratchet.cjs:
- assertWithinAllowlist: fails on NOVEL ids (new offender introduced) AND on
STALE ids (a known offender was fixed but not pruned) — identity, not count,
and a ratchet DOWN toward zero.
- assertTightCeiling: a size/length budget whose ceiling must stay within a
grace band of the high-water mark, so budgets may only tighten, never creep.
Ratchets converted onto the primitive
- windows-test-parity-guard.test.cjs: rmSync rule deleted (now ESLint-enforced);
the remaining six patterns moved from integer baselines to named-set
allowlists with ratchet-down.
- scripts/lint-test-file-count.{cjs,allowlist.json}: per-module integer counts →
named filename sets (closes the swap-a-file-keep-the-count blind spot); a
module dropping under cap now FAILS to force pruning its allowlist entry.
- enh-2790 skill-count `<= 63` → named skill allowlist (ratchets toward ~58).
Size budgets hardened (tighten-only)
- agent-size / workflow-size / feat-3039 help-tiered: ceilings lowered to the
current high-water mark and an assertTightCeiling anti-creep check added per
tier. Fixed external-contract limits (description ≤100 chars, agent ≤100 KB)
are intentionally left as-is — they are not grandfathered creeping budgets.
No user-facing behavior change (tests + tooling only); no USER_FACING_PREFIXES
touched, so no changeset fragment is required.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
136
scripts/lib/allowlist-ratchet.cjs
Normal file
136
scripts/lib/allowlist-ratchet.cjs
Normal file
@@ -0,0 +1,136 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* @file allowlist-ratchet.cjs
|
||||
*
|
||||
* Reusable "better than a count ratchet" primitives for CI guards.
|
||||
*
|
||||
* ## Motivation (issue #597)
|
||||
*
|
||||
* A count ratchet (`assert(offenders.length <= N)`) has a masking blind spot:
|
||||
* fixing one offender and introducing a new one keeps the count constant, so a
|
||||
* novel defect slips through green. These helpers enforce on IDENTITY instead,
|
||||
* making every individual offender visible and requiring monotonic progress
|
||||
* toward zero.
|
||||
*
|
||||
* ## Design
|
||||
*
|
||||
* Both functions are pure (no I/O, no global state). The `fail` callback is
|
||||
* injected by the caller so the same logic can be used with `node:assert.fail`,
|
||||
* a custom throw, or a message-collector in unit tests.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Assert that `current` offenders are all within the known allowlist, and that
|
||||
* every entry in the allowlist still offends (forcing the allowlist to shrink
|
||||
* as defects are fixed).
|
||||
*
|
||||
* Fails when:
|
||||
* - Any id in `current` is NOT in `known` → novel offender introduced.
|
||||
* - Any id in `known` is NOT in `current` → stale allowlist entry must be
|
||||
* pruned so the guard ratchets toward zero (the ratchet-DOWN direction).
|
||||
*
|
||||
* ## Masking blind spot this prevents (issue #597)
|
||||
*
|
||||
* A count ratchet (`assert(count <= N)`) allows one offender to be silently
|
||||
* replaced by another while the count stays at N. By asserting on identity
|
||||
* instead, every new offender is caught by name, and every fixed offender
|
||||
* forces the allowlist to shrink.
|
||||
*
|
||||
* @param {object} opts
|
||||
* @param {string} opts.label - Human-readable name for the guard (used
|
||||
* in failure messages).
|
||||
* @param {Iterable<string>} opts.current - The offending ids found in the
|
||||
* current run.
|
||||
* @param {Iterable<string>} opts.known - The allowlisted ids (baseline).
|
||||
* @param {function(string): void} opts.fail - Callback invoked with a
|
||||
* descriptive message on any violation.
|
||||
* Pass `require('node:assert').fail`, a
|
||||
* custom thrower, or a collector. The
|
||||
* function is NOT imported here so callers
|
||||
* control the failure mode.
|
||||
* @param {string} [opts.pruneHint] - Optional hint appended to the stale-
|
||||
* entry failure message (e.g. the name of
|
||||
* the allowlist file to edit).
|
||||
* @returns {{ novel: string[], stale: string[] }} Sorted arrays of novel ids
|
||||
* (in current but not known) and stale ids (in known but not current).
|
||||
*/
|
||||
function assertWithinAllowlist({ label, current, known, fail, pruneHint }) {
|
||||
const currentSet = new Set(current);
|
||||
const knownSet = new Set(known);
|
||||
|
||||
const novel = [...currentSet].filter((id) => !knownSet.has(id)).sort();
|
||||
const stale = [...knownSet].filter((id) => !currentSet.has(id)).sort();
|
||||
|
||||
if (novel.length > 0) {
|
||||
const list = novel.map((id) => ` - ${id}`).join('\n');
|
||||
fail(
|
||||
`[${label}] ${novel.length} NEW offender(s) introduced — fix at the source; do not just add to the allowlist.\n${list}`
|
||||
);
|
||||
}
|
||||
|
||||
if (stale.length > 0) {
|
||||
const list = stale.map((id) => ` - ${id}`).join('\n');
|
||||
const hint = pruneHint ? `\n(${pruneHint})` : '';
|
||||
fail(
|
||||
`[${label}] ${stale.length} allowlisted id(s) no longer offend and MUST be pruned so the guard ratchets toward zero.${hint}\n${list}`
|
||||
);
|
||||
}
|
||||
|
||||
return { novel, stale };
|
||||
}
|
||||
|
||||
/**
|
||||
* Assert that an artifact's measured maximum stays within a declared ceiling,
|
||||
* and that the ceiling itself does not creep above the high-water mark (budgets
|
||||
* may only decrease, not increase over time).
|
||||
*
|
||||
* Fails when:
|
||||
* - `actualMax > ceiling` → regression: artifact exceeds budget.
|
||||
* - `ceiling - actualMax > grace` → ceiling sits too far above the measured
|
||||
* value; tighten it toward `actualMax`.
|
||||
*
|
||||
* ## Masking blind spot this prevents (issue #597)
|
||||
*
|
||||
* A plain `assert(size <= ceiling)` with a ceiling set generously high allows
|
||||
* the artifact to grow unchecked as long as it stays under the ceiling. The
|
||||
* `grace` band forces the ceiling to stay close to the high-water mark,
|
||||
* ensuring that any upward creep is immediately visible.
|
||||
*
|
||||
* @param {object} opts
|
||||
* @param {string} opts.label - Human-readable name for the guard (used
|
||||
* in failure messages).
|
||||
* @param {number} opts.actualMax - The measured value (e.g. bundle size in
|
||||
* bytes, line count).
|
||||
* @param {number} opts.ceiling - The declared budget ceiling.
|
||||
* @param {number} opts.grace - Maximum allowed slack (`ceiling -
|
||||
* actualMax`) before the ceiling is
|
||||
* considered too loose.
|
||||
* @param {function(string): void} opts.fail - Callback invoked with a
|
||||
* descriptive message on any violation.
|
||||
* @returns {{ ok: boolean, slack: number }} Whether both checks passed and the
|
||||
* current slack value.
|
||||
*/
|
||||
function assertTightCeiling({ label, actualMax, ceiling, grace, fail }) {
|
||||
const slack = ceiling - actualMax;
|
||||
let ok = true;
|
||||
|
||||
if (actualMax > ceiling) {
|
||||
ok = false;
|
||||
fail(
|
||||
`[${label}] Regression: artifact value ${actualMax} exceeds budget ceiling ${ceiling}. ` +
|
||||
`Raise the ceiling to at most ${actualMax} only if the increase is justified.`
|
||||
);
|
||||
} else if (slack > grace) {
|
||||
ok = false;
|
||||
fail(
|
||||
`[${label}] Ceiling ${ceiling} sits too far above the high-water mark ${actualMax} ` +
|
||||
`(slack ${slack} > grace ${grace}). Tighten the ceiling toward ${actualMax}. ` +
|
||||
`Budgets may only decrease.`
|
||||
);
|
||||
}
|
||||
|
||||
return { ok, slack };
|
||||
}
|
||||
|
||||
module.exports = { assertWithinAllowlist, assertTightCeiling };
|
||||
@@ -1,36 +1,131 @@
|
||||
{
|
||||
"_doc": "Baseline of modules currently exceeding the 2-test-file limit. Each entry locks in TODAY's count as the ceiling. Reductions are ratcheted automatically — when a cluster drops to ≤ 2, remove its entry. New entries require justification in PR description.",
|
||||
"_doc": "Baseline of modules currently exceeding the 2-test-file limit. Each entry locks in TODAY's exact test filenames as the allowlisted set (identity ratchet). Adding a NEW test file to a capped module fails (novel). Removing one requires pruning this list (stale, ratchet-down). When a cluster drops to ≤ 2, remove its entry entirely. New entries require justification in PR description.",
|
||||
"modules": {
|
||||
"phase": { "current": 5, "issue": "3788" },
|
||||
"worktree": { "current": 13, "issue": "TBD" },
|
||||
"milestone": { "current": 10, "issue": "TBD" },
|
||||
"roadmap": { "current": 9, "issue": "TBD" },
|
||||
"verify": { "current": 10, "issue": "3767" },
|
||||
"install": { "current": 9, "issue": "TBD" },
|
||||
"init": { "current": 8, "issue": "TBD" },
|
||||
"state": { "current": 11, "issue": "180" },
|
||||
"config": { "current": 8, "issue": "TBD" },
|
||||
"graphify": { "current": 7, "issue": "TBD" },
|
||||
"progress": { "current": 6, "issue": "14" },
|
||||
"cli": { "current": 5, "issue": "TBD" },
|
||||
"surface": { "current": 5, "issue": "TBD" },
|
||||
"commit": { "current": 4, "issue": "TBD" },
|
||||
"frontmatter": { "current": 4, "issue": "TBD" },
|
||||
"index": { "current": 5, "issue": "180" },
|
||||
"intel": { "current": 4, "issue": "TBD" },
|
||||
"mvp": { "current": 4, "issue": "TBD" },
|
||||
"install-profiles": { "current": 4, "issue": "TBD" },
|
||||
"audit": { "current": 4, "issue": "192" },
|
||||
"audit-open": { "current": 3, "issue": "TBD" },
|
||||
"config-schema": { "current": 3, "issue": "TBD" },
|
||||
"profile": { "current": 3, "issue": "TBD" },
|
||||
"prompt-budget": { "current": 3, "issue": "TBD" },
|
||||
"uat": { "current": 3, "issue": "TBD" },
|
||||
"validate": { "current": 3, "issue": "TBD" },
|
||||
"workstream": { "current": 3, "issue": "TBD" },
|
||||
"gsd-tools": { "current": 3, "issue": "TBD" },
|
||||
"runtime-artifact-layout":{ "current": 3, "issue": "TBD" },
|
||||
"security": { "current": 3, "issue": "TBD" },
|
||||
"gsd-sdk": { "current": 3, "issue": "TBD" }
|
||||
"audit": {
|
||||
"files": [
|
||||
"audit-fix-command.test.cjs",
|
||||
"bug-2659-audit-open-crash.test.cjs",
|
||||
"bug-2836-audit-open-summary-uat-drift.test.cjs",
|
||||
"bug-2911-audit-open-output-shape.test.cjs"
|
||||
],
|
||||
"issue": "192"
|
||||
},
|
||||
"config": {
|
||||
"files": [
|
||||
"bug-2943-config-get-context-window-default.test.cjs",
|
||||
"bug-321-config-defaults-clone-strategy.test.cjs",
|
||||
"bug-3227-config-set-model-overrides.test.cjs",
|
||||
"bug-442-config-dir-equals-in-path.test.cjs",
|
||||
"config-field-docs.test.cjs",
|
||||
"config-get-default.test.cjs",
|
||||
"config-schema.property.test.cjs",
|
||||
"config.test.cjs"
|
||||
],
|
||||
"issue": "TBD"
|
||||
},
|
||||
"graphify": {
|
||||
"files": [
|
||||
"graphify-auto-update.test.cjs",
|
||||
"graphify-query.test.cjs",
|
||||
"graphify-visualization.test.cjs",
|
||||
"graphify.test.cjs"
|
||||
],
|
||||
"issue": "TBD"
|
||||
},
|
||||
"intel": {
|
||||
"files": [
|
||||
"bug-2351-intel-kilo-layout.test.cjs",
|
||||
"bug-3290-intel-updater-layout-block.test.cjs",
|
||||
"intel.test.cjs"
|
||||
],
|
||||
"issue": "TBD"
|
||||
},
|
||||
"milestone": {
|
||||
"files": [
|
||||
"milestone-archive.test.cjs",
|
||||
"milestone-helper.test.cjs",
|
||||
"milestone-prefixed-convention.test.cjs",
|
||||
"milestone-summary.test.cjs",
|
||||
"milestone.test.cjs"
|
||||
],
|
||||
"issue": "TBD"
|
||||
},
|
||||
"phase": {
|
||||
"files": [
|
||||
"bug-214-phase-researcher-write-truncation-contract.test.cjs",
|
||||
"phase-dependency-levels.test.cjs",
|
||||
"phase.test.cjs"
|
||||
],
|
||||
"issue": "597"
|
||||
},
|
||||
"roadmap": {
|
||||
"files": [
|
||||
"bug-2661-roadmap-sync-parallel.test.cjs",
|
||||
"bug-3128-roadmap-plan-count-slug-layout.test.cjs",
|
||||
"bug-3599-roadmap-get-phase-project-code-prefix.test.cjs",
|
||||
"enh-2447-roadmap-wave-deps.test.cjs",
|
||||
"roadmap-mode-field.test.cjs",
|
||||
"roadmap-phase-fallback.test.cjs",
|
||||
"roadmap.test.cjs"
|
||||
],
|
||||
"issue": "TBD"
|
||||
},
|
||||
"runtime-artifact-layout": {
|
||||
"files": [
|
||||
"runtime-artifact-layout-install-profiles.test.cjs",
|
||||
"runtime-artifact-layout-surface.test.cjs",
|
||||
"runtime-artifact-layout.test.cjs"
|
||||
],
|
||||
"issue": "TBD"
|
||||
},
|
||||
"security": {
|
||||
"files": [
|
||||
"security-prompt-injection.test.cjs",
|
||||
"security-scan.test.cjs",
|
||||
"security.test.cjs"
|
||||
],
|
||||
"issue": "TBD"
|
||||
},
|
||||
"state": {
|
||||
"files": [
|
||||
"bug-21-state-md-template-frontmatter.test.cjs",
|
||||
"bug-2630-state-frontmatter-milestone-switch.test.cjs",
|
||||
"bug-3127-state-begin-phase-idempotent.test.cjs",
|
||||
"bug-3242-state-update-progress-trample.test.cjs",
|
||||
"bug-3286-state-write-routing.test.cjs",
|
||||
"bug-3454-state-dollar-backreference-growth.test.cjs",
|
||||
"bug-397-state-preserve-executor-authored.test.cjs",
|
||||
"state-acquirestatelock-non-eexist.test.cjs",
|
||||
"state-prune.test.cjs",
|
||||
"state.test.cjs"
|
||||
],
|
||||
"issue": "180"
|
||||
},
|
||||
"verify": {
|
||||
"files": [
|
||||
"bug-2969-verify-reapply-patches.test.cjs",
|
||||
"bug-2994-verify-reapply-patches-installed-path.test.cjs",
|
||||
"bug-3381-verify-work-workstream.test.cjs",
|
||||
"bug-3657-verify-reapply-patches-pristine-drift.test.cjs",
|
||||
"verify-health.test.cjs",
|
||||
"verify-mvp-uat.test.cjs",
|
||||
"verify-test-quality.test.cjs",
|
||||
"verify-work-auto-transition.test.cjs",
|
||||
"verify.test.cjs"
|
||||
],
|
||||
"issue": "3767"
|
||||
},
|
||||
"install": {
|
||||
"files": [
|
||||
"bug-410-install-defaults-test-mode-guard.test.cjs",
|
||||
"install-minimal-hooks.test.cjs",
|
||||
"install-path-detection.test.cjs",
|
||||
"install-regressions.test.cjs",
|
||||
"install-runtime-artifacts.test.cjs",
|
||||
"install-update-marker.test.cjs",
|
||||
"install.test.cjs"
|
||||
],
|
||||
"issue": "TBD"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,16 +4,19 @@
|
||||
*
|
||||
* Scans sdk/src/query/, sdk/src/, get-shit-done/bin/lib/, bin/ for production
|
||||
* modules, then counts matching test files in tests/ and sdk/src (recursive). Cap is 2
|
||||
* (primary + one integration). Over-limit clusters must be in the allowlist at
|
||||
* their frozen count (ratchet: may only decrease). --json emits structured output.
|
||||
* (primary + one integration). Over-limit clusters must be in the allowlist with the
|
||||
* EXACT set of test filenames grandfathered (identity ratchet via allowlist-ratchet.cjs).
|
||||
* Adding a new test file to a capped module is a novel offender; removing one requires
|
||||
* pruning the allowlist entry (stale). --json emits structured output.
|
||||
*
|
||||
* Verdicts: OK_UNDER_LIMIT | OK_IN_ALLOWLIST | FAIL_EXCEEDS_LIMIT |
|
||||
* FAIL_EXCEEDS_ALLOWLIST | HINT_CAN_REMOVE_FROM_ALLOWLIST
|
||||
* FAIL_NOVEL_FILES | FAIL_STALE_ALLOWLIST
|
||||
*/
|
||||
'use strict';
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { assertWithinAllowlist } = require('./lib/allowlist-ratchet.cjs');
|
||||
|
||||
const ROOT = path.join(__dirname, '..');
|
||||
const PROD_DIRS = [
|
||||
@@ -30,11 +33,11 @@ const ALLOWLIST_PATH = path.join(__dirname, 'lint-test-file-count.allowlist.json
|
||||
const MAX_FILES = 2;
|
||||
|
||||
const Verdict = Object.freeze({
|
||||
OK_UNDER_LIMIT: 'OK_UNDER_LIMIT',
|
||||
OK_IN_ALLOWLIST: 'OK_IN_ALLOWLIST',
|
||||
FAIL_EXCEEDS_LIMIT: 'FAIL_EXCEEDS_LIMIT',
|
||||
FAIL_EXCEEDS_ALLOWLIST: 'FAIL_EXCEEDS_ALLOWLIST',
|
||||
HINT_CAN_REMOVE_FROM_ALLOWLIST: 'HINT_CAN_REMOVE_FROM_ALLOWLIST',
|
||||
OK_UNDER_LIMIT: 'OK_UNDER_LIMIT',
|
||||
OK_IN_ALLOWLIST: 'OK_IN_ALLOWLIST',
|
||||
FAIL_EXCEEDS_LIMIT: 'FAIL_EXCEEDS_LIMIT',
|
||||
FAIL_NOVEL_FILES: 'FAIL_NOVEL_FILES',
|
||||
FAIL_STALE_ALLOWLIST: 'FAIL_STALE_ALLOWLIST',
|
||||
});
|
||||
|
||||
function isTestFile(name) {
|
||||
@@ -120,17 +123,62 @@ function loadAllowlist() {
|
||||
catch (_) { return {}; }
|
||||
}
|
||||
|
||||
/**
|
||||
* Evaluate one module's test files against the allowlist.
|
||||
*
|
||||
* For modules under the default cap (≤ MAX_FILES): simple count check.
|
||||
* For modules with an allowlist entry: identity check via assertWithinAllowlist.
|
||||
* - count now ≤ MAX_FILES → FAIL_STALE_ALLOWLIST (all known files are stale; prune entry)
|
||||
* - novel files (in current but not in known) → FAIL_NOVEL_FILES
|
||||
* - stale files (in known but not in current) → FAIL_STALE_ALLOWLIST
|
||||
* - exact match → OK_IN_ALLOWLIST
|
||||
* For modules over the cap with no allowlist entry: FAIL_EXCEEDS_LIMIT.
|
||||
*
|
||||
* Returns { verdict, prefix, count, knownFiles, novel, stale, files }
|
||||
*/
|
||||
function evaluateLint({ prefix, testFiles, allowlist }) {
|
||||
const count = testFiles.length;
|
||||
const entry = allowlist[prefix];
|
||||
const ceiling = entry ? entry.current : null;
|
||||
const currentNames = testFiles.map(f => path.basename(f));
|
||||
|
||||
if (entry !== undefined) {
|
||||
if (count <= MAX_FILES) return { verdict: Verdict.HINT_CAN_REMOVE_FROM_ALLOWLIST, prefix, count, ceiling, files: testFiles };
|
||||
if (count <= ceiling) return { verdict: Verdict.OK_IN_ALLOWLIST, prefix, count, ceiling, files: testFiles };
|
||||
return { verdict: Verdict.FAIL_EXCEEDS_ALLOWLIST, prefix, count, ceiling, files: testFiles };
|
||||
const knownFiles = Array.isArray(entry.files) ? entry.files : [];
|
||||
// If the module is now at or under the default cap, the entire allowlist entry is
|
||||
// stale and must be removed — this is a ratchet-DOWN failure, not a hint.
|
||||
// All known files are stale (the whole entry can go).
|
||||
if (count <= MAX_FILES) {
|
||||
return {
|
||||
verdict: Verdict.FAIL_STALE_ALLOWLIST,
|
||||
prefix, count,
|
||||
knownFiles,
|
||||
novel: [],
|
||||
stale: knownFiles.slice().sort(),
|
||||
files: testFiles,
|
||||
};
|
||||
}
|
||||
// Identity check via assertWithinAllowlist
|
||||
const messages = [];
|
||||
const { novel, stale } = assertWithinAllowlist({
|
||||
label: prefix,
|
||||
current: currentNames,
|
||||
known: knownFiles,
|
||||
fail: (msg) => messages.push(msg),
|
||||
pruneHint: 'scripts/lint-test-file-count.allowlist.json',
|
||||
});
|
||||
|
||||
if (novel.length > 0) {
|
||||
return { verdict: Verdict.FAIL_NOVEL_FILES, prefix, count, knownFiles, novel, stale, files: testFiles };
|
||||
}
|
||||
if (stale.length > 0) {
|
||||
return { verdict: Verdict.FAIL_STALE_ALLOWLIST, prefix, count, knownFiles, novel, stale, files: testFiles };
|
||||
}
|
||||
return { verdict: Verdict.OK_IN_ALLOWLIST, prefix, count, knownFiles, novel: [], stale: [], files: testFiles };
|
||||
}
|
||||
if (count <= MAX_FILES) return { verdict: Verdict.OK_UNDER_LIMIT, prefix, count, ceiling: null, files: testFiles };
|
||||
return { verdict: Verdict.FAIL_EXCEEDS_LIMIT, prefix, count, ceiling: null, files: testFiles };
|
||||
|
||||
if (count <= MAX_FILES) {
|
||||
return { verdict: Verdict.OK_UNDER_LIMIT, prefix, count, knownFiles: null, novel: [], stale: [], files: testFiles };
|
||||
}
|
||||
return { verdict: Verdict.FAIL_EXCEEDS_LIMIT, prefix, count, knownFiles: null, novel: [], stale: [], files: testFiles };
|
||||
}
|
||||
|
||||
function run() {
|
||||
@@ -147,35 +195,42 @@ function run() {
|
||||
}
|
||||
|
||||
const failures = results.filter(r =>
|
||||
r.verdict === Verdict.FAIL_EXCEEDS_LIMIT || r.verdict === Verdict.FAIL_EXCEEDS_ALLOWLIST);
|
||||
const hints = results.filter(r => r.verdict === Verdict.HINT_CAN_REMOVE_FROM_ALLOWLIST);
|
||||
r.verdict === Verdict.FAIL_EXCEEDS_LIMIT ||
|
||||
r.verdict === Verdict.FAIL_NOVEL_FILES ||
|
||||
r.verdict === Verdict.FAIL_STALE_ALLOWLIST);
|
||||
|
||||
if (jsonMode) {
|
||||
console.log(JSON.stringify({ ok: failures.length === 0, results, failures, hints }, null, 2));
|
||||
console.log(JSON.stringify({ ok: failures.length === 0, results, failures, hints: [] }, null, 2));
|
||||
process.exit(failures.length > 0 ? 1 : 0);
|
||||
}
|
||||
|
||||
if (failures.length === 0) {
|
||||
const inAllowlist = results.filter(r => r.verdict === Verdict.OK_IN_ALLOWLIST).length;
|
||||
console.log(`ok lint-test-file-count: ${results.length} module(s) checked, 0 failures` +
|
||||
(inAllowlist > 0 ? `, ${inAllowlist} allowlisted` : '') +
|
||||
(hints.length > 0 ? `, ${hints.length} hint(s)` : ''));
|
||||
for (const h of hints) {
|
||||
console.log(` hint: "${h.prefix}" is allowlisted at ${h.ceiling} but now has ${h.count} — remove from allowlist`);
|
||||
}
|
||||
(inAllowlist > 0 ? `, ${inAllowlist} allowlisted` : ''));
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
process.stderr.write(`\nERROR lint-test-file-count: ${failures.length} module(s) exceed the test-file limit\n\n`);
|
||||
for (const f of failures) {
|
||||
const tag = f.verdict === Verdict.FAIL_EXCEEDS_LIMIT
|
||||
? `${f.count} files (limit ${MAX_FILES})`
|
||||
: `${f.count} files (allowlist ceiling ${f.ceiling})`;
|
||||
process.stderr.write(` ${f.prefix}: ${tag}\n`);
|
||||
for (const tf of f.files) process.stderr.write(` ${path.relative(ROOT, tf)}\n`);
|
||||
if (f.verdict === Verdict.FAIL_EXCEEDS_LIMIT) {
|
||||
process.stderr.write(` ${f.prefix}: ${f.count} files (limit ${MAX_FILES}) — not in allowlist\n`);
|
||||
for (const tf of f.files) process.stderr.write(` ${path.relative(ROOT, tf)}\n`);
|
||||
} else if (f.verdict === Verdict.FAIL_NOVEL_FILES) {
|
||||
process.stderr.write(` ${f.prefix}: ${f.novel.length} NEW test file(s) not in allowlist\n`);
|
||||
for (const n of f.novel) process.stderr.write(` + ${n}\n`);
|
||||
} else if (f.verdict === Verdict.FAIL_STALE_ALLOWLIST) {
|
||||
if (f.count <= MAX_FILES) {
|
||||
const staleList = f.stale.join(', ');
|
||||
process.stderr.write(` "${f.prefix}": now at ${f.count} file(s) (≤ ${MAX_FILES}) — remove its entry from the allowlist (stale: ${staleList})\n`);
|
||||
} else {
|
||||
process.stderr.write(` ${f.prefix}: ${f.stale.length} allowlisted file(s) no longer present — prune allowlist\n`);
|
||||
for (const s of f.stale) process.stderr.write(` - ${s}\n`);
|
||||
}
|
||||
}
|
||||
}
|
||||
process.stderr.write('\nFix: consolidate test files per module (one primary + one integration).\n');
|
||||
process.stderr.write('Or add the module to scripts/lint-test-file-count.allowlist.json with PR justification.\n\n');
|
||||
process.stderr.write('Or update scripts/lint-test-file-count.allowlist.json with PR justification.\n\n');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user