fix(1259-01): close adversarial review findings — genuine enforcement, honest fail-first scope
Adversarial pre-submission review found the injected-runCheck tests masked a non-functional real runner. Fixes: - BL-01 (false green on vacuous test): the node-test runner now parses the TAP summary and requires a NON-VACUOUS pass (>=1 test, >=1 pass, 0 fail) AND a reported test named distinctly from the file — node --test counts an empty file as one passing test, so counts alone could not catch it. - SF-01 (lint anchor never greened): the lint-rule runner now runs the project eslint as --format json and filters by ruleId, so plugin rules (local/*) load via the flat config — bare --rule cannot load a plugin. local/no-source-grep now genuinely greens (covered by a real, non-injected test). - BL-02 (tautological fail-first): the runner no longer echoes the caller's failFirst as if confirmed. failFirst is documented as caller-ATTESTED; the producer requires attestation + a genuine non-vacuous pass. Machine-proven fail-first (needs a violation fixture) is flagged as a tracked follow-up in ADR-550, the changeset, FEATURES, the reference doc, and verify-phase. - SF-02: added real-runner end-to-end tests (no injected runCheck) + pure, exported parse/filter helpers (parseNodeTestSummary, tapTestNames, eslintJsonHasRule, eslintFileResultCount) so the shipping branches are mutation-pinned. - NIT-01/02: LOCATE guard rejects empty-string rule and unknown kinds. - Hardening: spawn checks with NODE_TEST_CONTEXT/NODE_OPTIONS scrubbed so an ambient test-runner context cannot corrupt a verify-time result. - Docs reconciled to the shipped behavior (no 'confirms fail-first' overclaim).
This commit is contained in:
@@ -113,11 +113,14 @@ lifecycle is identical to the edge-probe, the verification tiers differ):
|
||||
At verify time these tiers are routed differently (ADR-550 D4):
|
||||
- A **test**-tier prohibition is enforced + hard-gates via the deterministic
|
||||
`check prohibition-enforcement` sub-command (#1259, ADR-550 D5d): it locates the wired
|
||||
mechanical check (a `node --test` negative test OR a lint/AST rule), confirms it is
|
||||
fail-first, runs it, and emits the `dispositionForProhibition()` verdict. A passing wired
|
||||
check disposes **green** (satisfiable → can reach `passed`); a missing, failing, or
|
||||
non-fail-first check **hard-gates** (flagged, never green → `gaps_found`) in BOTH
|
||||
interactive and autonomous modes — never a silent pass.
|
||||
mechanical check (a `node --test` negative test OR a lint/AST rule run as
|
||||
`eslint --format json` and filtered by `ruleId`), requires the caller-attested `failFirst`
|
||||
marker, runs it for a genuine **non-vacuous** pass, and emits the
|
||||
`dispositionForProhibition()` verdict. A passing wired check disposes **green** (satisfiable
|
||||
→ can reach `passed`); a missing, non-attested, or genuinely-non-passing check **hard-gates**
|
||||
(flagged, never green → `gaps_found`) in BOTH interactive and autonomous modes — never a
|
||||
silent pass. (`failFirst` is caller-attested, not yet machine-proven against a violation
|
||||
fixture — a tracked follow-up; see ADR-550 D5d.)
|
||||
- A **judgment**-tier prohibition routes to a never-silent / never-hard-halt soft gate
|
||||
(autonomous emits an `unverified-prohibition — human review recommended` flag).
|
||||
|
||||
|
||||
@@ -76,9 +76,9 @@ Aggregate all must_haves across plans for phase-level verification.
|
||||
gsd_run check prohibition-enforcement <request.json>
|
||||
```
|
||||
|
||||
where `<request.json>` carries `{ prohibition, check, mode }` — `check` being the wired mechanical-check descriptor `{ kind: 'node-test' | 'lint-rule', target, rule?, failFirst: true }`. For `node-test`, `target` is the negative-test file path; for `lint-rule`, `target` is the PATH to lint and `rule` is the eslint rule id (e.g. `local/no-source-grep`) — both required (a lint-rule without `rule` is not a valid wired check). The producer LOCATES the wired check, CONFIRMS it is fail-first (`regression-must-fail-first`), RUNS it, builds `enforcementEvidence`, and emits the `dispositionForProhibition()` verdict (#1259, ADR-550 D5d). Route the result by its typed fields:
|
||||
- **`status: 'green'`, `flagged: false`** (a passing wired negative test / lint rule, `located: true`, non-empty `evidence`) → the item is satisfiable → it can reach **passed**.
|
||||
- **missing, failing, or non-fail-first check** (`located: false` OR `status: 'unverified'`, `flagged: true`) → **hard-gate**: disposes flagged-unverified, NEVER green, routing to `gaps_found` in BOTH interactive and autonomous modes (a failing mechanical check blocks even AFK; ADR-550 D4 / D3). The deterministic fail-closed default backing every miss/fail is `dispositionForProhibition()` in probe-core (`status: 'unverified'`, `flagged: true` on empty `enforcementEvidence`).
|
||||
where `<request.json>` carries `{ prohibition, check, mode }` — `check` being the wired mechanical-check descriptor `{ kind: 'node-test' | 'lint-rule', target, rule?, failFirst: true }`. For `node-test`, `target` is the negative-test file path; for `lint-rule`, `target` is the PATH to lint and `rule` is the eslint rule id (e.g. `local/no-source-grep`) — both required (a lint-rule without `rule` is not a valid wired check). The producer LOCATES the wired check, requires the caller-attested `failFirst` marker, RUNS it for a genuine non-vacuous pass, builds `enforcementEvidence`, and emits the `dispositionForProhibition()` verdict (#1259, ADR-550 D5d). (`failFirst` is caller-attested, not yet machine-proven against a violation fixture — a tracked follow-up.) Route the result by its typed fields:
|
||||
- **`status: 'green'`, `flagged: false`** (a genuinely-passing wired negative test / lint rule, `located: true`, non-empty `evidence`) → the item is satisfiable → it can reach **passed**.
|
||||
- **missing, non-attested, or genuinely-non-passing check** (`located: false` OR `status: 'unverified'`, `flagged: true`) → **hard-gate**: disposes flagged-unverified, NEVER green, routing to `gaps_found` in BOTH interactive and autonomous modes (a failing mechanical check blocks even AFK; ADR-550 D4 / D3). The deterministic fail-closed default backing every miss/fail is `dispositionForProhibition()` in probe-core (`status: 'unverified'`, `flagged: true` on empty `enforcementEvidence`).
|
||||
|
||||
**Option B: Use Success Criteria from ROADMAP.md**
|
||||
|
||||
|
||||
Reference in New Issue
Block a user