Merge pull request #3392 from open-gsd/chore/3331-no-elapsed-assertion-error
chore(#3331): promote local/no-elapsed-assertion warn->error
This commit is contained in:
@@ -467,7 +467,7 @@ A test that generates many adversarial inputs automatically (via `fast-check`) a
|
||||
Stryker injects small code mutations (e.g., flipping a `>` to `>=`, deleting a `return` statement) and reruns the test suite for each. A mutation is "killed" if at least one test fails; "surviving" if all tests pass despite the mutation. Mutation score = killed / total. Score below 80 % on the changed scope blocks PR merge. See `RULESET.TESTS.mutation-score`.
|
||||
|
||||
### ESLint harness
|
||||
The canonical lint infrastructure adopted in ADR 452 (`docs/adr/452-eslint-lint-harness.md`): ESLint flat config (`eslint.config.mjs`) with `typescript-eslint`, `eslint-plugin-n`, `eslint-plugin-no-only-tests`, and a local AST-rule plugin at `eslint-rules/`. Replaces the homegrown `scripts/lint-*.cjs` regex scanners. The three custom test-rigor rules (`local/no-source-grep`, `local/no-magic-sleep-in-tests`, `local/no-elapsed-assertion`) initially ship at `warn`; they become `error` after the cleanup sweep tracked at issue #453 merges.
|
||||
The canonical lint infrastructure adopted in ADR 452 (`docs/adr/452-eslint-lint-harness.md`): ESLint flat config (`eslint.config.mjs`) with `typescript-eslint`, `eslint-plugin-n`, `eslint-plugin-no-only-tests`, and a local AST-rule plugin at `eslint-rules/`. Replaces the homegrown `scripts/lint-*.cjs` regex scanners. All three custom test-rigor rules (`local/no-source-grep`, `local/no-magic-sleep-in-tests`, `local/no-elapsed-assertion`) now ship at `error` (promoted by #3313 and #3331 respectively, superseding the original #453 cleanup-sweep handoff).
|
||||
|
||||
### External-job-waiting half-state
|
||||
A legal deferred state of an Execute step (`external_job_waiting`): the executor has dispatched a long-running async external job and committed an async-job manifest at `.planning/async-jobs/<job>.json` instead of a SUMMARY.md. Distinct from the synchronous "mid-production-commits" half-state and from an illegal partial-plan state. The core loop's step-completion + safe-resume/pause contract treats a non-terminal manifest as legal and reconciles against it (never re-dispatching the plan, which would duplicate the external job); SUMMARY.md is deferred until the job reaches a terminal state and its `expected_artifacts` are verified. The manifest is a versioned stability contract (`docs/reference/planning-artifacts.md`); core *consumes* it while a default-off scheduler-adapter Capability (#1164) *produces* it at `execute:wave:post` — the contract-is-core / producer-is-capability seam mirrors ADR-857's verification-substrate decision. Status enum is closed and scheduler-agnostic: `submitted`, `running`, `completed-unverified`, `failed`, `cancelled`, `timeout`.
|
||||
@@ -533,12 +533,12 @@ The prompt-level data/instruction isolation seam for untrusted web/document ingr
|
||||
`RULESET.TESTS.boundary-coverage.anti-pattern=test suites that pair budget:1_000_000 (trivially fits) with budget:1 (trivially overflows) and skip the boundary region; failure mode that shipped PR #3708 UNNEEDED_TRIM + FALSE_HARDFAIL regressions (commit 2df566ed, fixed bde1ae8f)`
|
||||
`LEARNING.prompt-budget.boundary-gap=PR #3708 commit 2df566ed reserved NOTE_RESERVE_TOKENS in pressure-threshold AND in minSet pre-check; both buggy paths only fire when baseTokens ∈ (effectiveBudget - NOTE_RESERVE_TOKENS, effectiveBudget]; original test suite used budgets far from that band so neither path was exercised; fix bde1ae8f confines NOTE_RESERVE accounting to post-trim assembly path only; future budget/limit code MUST add boundary fixtures per RULESET.TESTS.boundary-coverage.fixtures`
|
||||
|
||||
`RULESET.TESTS.no-timing-assertion=do not assert on wall-clock elapsed time (Date.now() delta, performance.now(), process.hrtime() comparison); such assertions test the host machine not the SUT and flake on loaded CI runners; enforcement: local/no-elapsed-assertion ESLint rule, currently warn (promotion to error tracked under open epic #1885, not #453 which already merged without completing it); canonical replacement: clock-seam pattern with node:test mock.timers`
|
||||
`RULESET.TESTS.no-timing-assertion=do not assert on wall-clock elapsed time (Date.now() delta, performance.now(), process.hrtime() comparison); such assertions test the host machine not the SUT and flake on loaded CI runners; enforcement: local/no-elapsed-assertion ESLint rule, error (promoted by #3331 once #3314 delivered the ADR-456 §(a) reachability rule + deterministic backfill precondition); canonical replacement: clock-seam pattern with node:test mock.timers`
|
||||
`RULESET.TESTS.clock-seam=concurrency logic must accept an optional {clock=Date} parameter; tests control time via t.mock.timers.enable(['Date']) + t.mock.timers.setTime(0) + t.mock.timers.tick(N); real OS scheduler races are not a permitted test pattern after ADR 456 (2026-05-28); real-race tests are deleted once deterministic seam tests cover the same logical path; clock.cjs realClock adds nowIso() (→ new Date(this.now()).toISOString()) and today() (→ nowIso().split('T')[0]) so all date-stamping in state.cjs routes through the seam; subprocess time-pin adapter: set GSD_TEST_MODE=1 + GSD_NOW_MS=<epoch-ms> in runGsdTools env to pin the date written by the SUT without touching real wall-clock (issue #474)`
|
||||
`RULESET.TESTS.property-based-testing=modules implementing parsing / transformation / budget-limit / bijective contracts must include at least one fast-check (fc) property test asserting a domain invariant; invariant categories: round-trip, monotonicity, boundary-containment, idempotency; property tests live in *.test.cjs alongside unit tests; CI signal: Stryker mutation score below 80% blocks merge`
|
||||
`RULESET.TESTS.mutation-score=Stryker runs incremental (--since origin/next) on ubuntu-latest/Node24 CI leg; default threshold 80% killed/total; surviving mutants in scope block merge unless path is listed in stryker.config.mjs with documented reason; treat surviving mutant as a failing test specification`
|
||||
`RULESET.TESTS.delete-bad-tests=pass-always / vacuous-truth / source-grep / elapsed-time / real-race / permanent-allow-test-rule tests are DELETED and replaced with compliant tests in the same PR; not skipped, not commented out, not permanently exempted; replacement must cover the same logical path via typed-surface assertion or clock-seam pattern`
|
||||
`RULESET.TESTS.eslint-harness=ADR 452 (2026-05-28): ESLint flat config + typescript-eslint + eslint-plugin-n + eslint-plugin-no-only-tests + local plugin at eslint-rules/ (repo root, NOT scripts/eslint-rules/); replaces scripts/lint-*.cjs regex scanners (fully removed in #632); of the three test-rigor rules, local/no-source-grep and local/no-magic-sleep-in-tests are already promoted to error in tests/**/*.test.cjs scope (post-cleanup), local/no-elapsed-assertion remains at warn pending open epic #1885 (its dedicated ratchet issue #453 already merged without completing this promotion; follow-up #1888 was closed not-planned and folded into #1885)`
|
||||
`RULESET.TESTS.eslint-harness=ADR 452 (2026-05-28): ESLint flat config + typescript-eslint + eslint-plugin-n + eslint-plugin-no-only-tests + local plugin at eslint-rules/ (repo root, NOT scripts/eslint-rules/); replaces scripts/lint-*.cjs regex scanners (fully removed in #632); all three test-rigor rules now ship at error in tests/**/*.test.cjs scope: local/no-source-grep and local/no-magic-sleep-in-tests promoted by #3313, local/no-elapsed-assertion promoted by #3331 once #3314 delivered its ADR-456 §(a) precondition (epic #1885 was subsumed into epic #3053 and closed stale before this promotion landed)`
|
||||
|
||||
`RULESET.AUDIT.search-source-not-generated=verify an invariant/validation EXISTS by searching the AUTHORED source (src/*.cts OR the scripts/gen-*.cjs generator), never the generated bin/lib/*.cjs (gitignored, ADR-457); gen-time checks live in gen-*.cjs not the .cts it consumes → search BOTH before declaring absent; read generated .cjs only for output drift. Repro: grep src/*.cts for VALID_CONVERTER_NAMES → false "5e ConverterName unenforced"; actually enforced in gen-capability-registry.cjs. cf RULESET.TESTS.no-source-grep`
|
||||
|
||||
|
||||
@@ -137,7 +137,7 @@ await doWork();
|
||||
assert(Date.now() - start < 200, 'must complete in 200ms');
|
||||
```
|
||||
|
||||
**Enforcement:** `local/no-elapsed-assertion` (ESLint, currently `warn`; promotion to `error` was sequenced behind #3314, now delivered — ADR-456 §(a) is amended with a reachability-based selection rule covering all three clock-control mechanisms this repo uses, and the direct-use modules carrying real time-gating logic (`commands.cts`, `init.cts`, `io.cts`) have deterministic backfill coverage. The actual `warn`→`error` promotion is tracked at [#3331](https://github.com/open-gsd/gsd-core/issues/3331), since #3314's own precondition-handoff target, #1885, closed before this issue landed). Also `no-restricted-syntax` ban on `performance.now()` comparisons in assertions.
|
||||
**Enforcement:** `local/no-elapsed-assertion` (ESLint, `error` — promoted by [#3331](https://github.com/open-gsd/gsd-core/issues/3331) once #3314 delivered its precondition: ADR-456 §(a) amended with a reachability-based selection rule covering all three clock-control mechanisms this repo uses, and the direct-use modules carrying real time-gating logic (`commands.cts`, `init.cts`, `io.cts`) backfilled with deterministic coverage). Also `no-restricted-syntax` ban on `performance.now()` comparisons in assertions.
|
||||
|
||||
### Clock-seam pattern for concurrency
|
||||
|
||||
@@ -209,12 +209,12 @@ Real multi-process race tests are deleted once the corresponding deterministic c
|
||||
|---|---|---|
|
||||
| `local/no-source-grep` | `error` (promoted by #3313) | `readFileSync` on source files + text assertions; `assert.match`/`doesNotMatch` on raw stdout/stderr |
|
||||
| `local/no-magic-sleep-in-tests` | `error` | `setTimeout`/`sleep`/`delay` calls inside `test()`/`it()`/`describe()` bodies |
|
||||
| `local/no-elapsed-assertion` | `warn` → `error` (#3314) | Assertions on `Date.now()` delta, `process.hrtime()`, `performance.now()` comparisons |
|
||||
| `local/no-elapsed-assertion` | `error` (promoted by #3331, precondition delivered by #3314) | Assertions on `Date.now()` delta, `process.hrtime()`, `performance.now()` comparisons |
|
||||
| `no-only-tests/no-only-tests` | `error` | `test.only`/`describe.only`/`it.only` committed to non-scratch files |
|
||||
| `no-restricted-syntax` (ban 1) | `error` | Top-level `setTimeout` in `ExpressionStatement` |
|
||||
| `no-restricted-syntax` (ban 2) | `error` | `.only` member access on `test`/`it`/`describe` (belt-and-suspenders) |
|
||||
|
||||
`local/no-source-grep` and `local/no-magic-sleep-in-tests` now ship at `error` (promoted by [#3313](https://github.com/open-gsd/gsd-core/issues/3313), absorbing the cleanup sweep originally tracked at #453). `local/no-elapsed-assertion` remains `warn` — [#3314](https://github.com/open-gsd/gsd-core/issues/3314) delivered its precondition (ADR-456 §(a) amended with a reachability-based 3-mechanism rule; `commands.cts`/`init.cts`/`io.cts` backfilled with deterministic coverage), but does not itself own the `warn`→`error` promotion (mirroring the same handover boundary the epic draws for its other items) — that promotion is tracked at [#3331](https://github.com/open-gsd/gsd-core/issues/3331). New violations added after the acceptance of ADR 456 are out of policy regardless of the current ESLint severity.
|
||||
`local/no-source-grep` and `local/no-magic-sleep-in-tests` ship at `error` (promoted by [#3313](https://github.com/open-gsd/gsd-core/issues/3313), absorbing the cleanup sweep originally tracked at #453). `local/no-elapsed-assertion` now also ships at `error` (promoted by [#3331](https://github.com/open-gsd/gsd-core/issues/3331)) — [#3314](https://github.com/open-gsd/gsd-core/issues/3314) delivered its precondition first (ADR-456 §(a) amended with a reachability-based 3-mechanism rule; `commands.cts`/`init.cts`/`io.cts` backfilled with deterministic coverage), mirroring the same handover boundary the epic draws for its other items. New violations added after the acceptance of ADR 456 are out of policy regardless of ESLint severity.
|
||||
|
||||
ESLint harness details: [`docs/adr/452-eslint-lint-harness.md`](docs/adr/452-eslint-lint-harness.md).
|
||||
|
||||
|
||||
@@ -1082,7 +1082,7 @@
|
||||
{
|
||||
"id": "RULESET.TESTS.eslint-harness",
|
||||
"klass": "RULESET",
|
||||
"value": "ADR 452 (2026-05-28): ESLint flat config + typescript-eslint + eslint-plugin-n + eslint-plugin-no-only-tests + local plugin at eslint-rules/ (repo root, NOT scripts/eslint-rules/); replaces scripts/lint-*.cjs regex scanners (fully removed in #632); of the three test-rigor rules, local/no-source-grep and local/no-magic-sleep-in-tests are already promoted to error in tests/**/*.test.cjs scope (post-cleanup), local/no-elapsed-assertion remains at warn pending open epic #1885 (its dedicated ratchet issue #453 already merged without completing this promotion; follow-up #1888 was closed not-planned and folded into #1885)"
|
||||
"value": "ADR 452 (2026-05-28): ESLint flat config + typescript-eslint + eslint-plugin-n + eslint-plugin-no-only-tests + local plugin at eslint-rules/ (repo root, NOT scripts/eslint-rules/); replaces scripts/lint-*.cjs regex scanners (fully removed in #632); all three test-rigor rules now ship at error in tests/**/*.test.cjs scope: local/no-source-grep and local/no-magic-sleep-in-tests promoted by #3313, local/no-elapsed-assertion promoted by #3331 once #3314 delivered its ADR-456 §(a) precondition (epic #1885 was subsumed into epic #3053 and closed stale before this promotion landed)"
|
||||
},
|
||||
{
|
||||
"id": "RULESET.TESTS.feedback-loop-convergence",
|
||||
@@ -1132,7 +1132,7 @@
|
||||
{
|
||||
"id": "RULESET.TESTS.no-timing-assertion",
|
||||
"klass": "RULESET",
|
||||
"value": "do not assert on wall-clock elapsed time (Date.now() delta, performance.now(), process.hrtime() comparison); such assertions test the host machine not the SUT and flake on loaded CI runners; enforcement: local/no-elapsed-assertion ESLint rule, currently warn (promotion to error tracked under open epic #1885, not #453 which already merged without completing it); canonical replacement: clock-seam pattern with node:test mock.timers"
|
||||
"value": "do not assert on wall-clock elapsed time (Date.now() delta, performance.now(), process.hrtime() comparison); such assertions test the host machine not the SUT and flake on loaded CI runners; enforcement: local/no-elapsed-assertion ESLint rule, error (promoted by #3331 once #3314 delivered the ADR-456 §(a) reachability rule + deterministic backfill precondition); canonical replacement: clock-seam pattern with node:test mock.timers"
|
||||
},
|
||||
{
|
||||
"id": "RULESET.TESTS.property-based-testing",
|
||||
|
||||
@@ -450,7 +450,9 @@ export default tseslint.config(
|
||||
'no-only-tests/no-only-tests': 'error',
|
||||
// Timing anti-patterns — ratcheted to error after cleanup (all violations fixed)
|
||||
'local/no-magic-sleep-in-tests': 'error',
|
||||
'local/no-elapsed-assertion': 'warn',
|
||||
// Promoted warn->error by #3331 once #3314 delivered its precondition (ADR-456 §(a) amended,
|
||||
// direct-use modules backfilled with deterministic time control) — see TESTING-STANDARDS.md.
|
||||
'local/no-elapsed-assertion': 'error',
|
||||
// Ban raw fs.rmSync in tests — use helpers.cleanup() for Windows-EBUSY retry budget
|
||||
'local/no-raw-rmsync-in-tests': 'error',
|
||||
// Ban tautological assertions (always-truthy arg or identical-literal equality)
|
||||
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user