enhance(verify-phase): node-test causation control — prove the RED is content-caused (#1346)
The #1279 node-test machine-proof confirmed a known-bad subject drives the negative test RED, but could not distinguish a genuine content-violation from a deceptive test that reds merely because GSD_PROHIB_SUBJECT is set. Add an optional fifth flat scalar `check_clean_fixture` (-> CheckDescriptor.cleanFixture) threading a KNOWN-CLEAN control subject through projectProhibitions + descriptorFromProjection. When present, the prover also runs the check against the clean subject and requires GREEN, so fail-first is proven only when the check is RED on the violation AND GREEN on the clean subject (content-dependent). Opt-in and additive: absent a clean fixture the prover behaves exactly as post-#1314 (no control, documented residual), preserving the zero-authoring compose path; the lint-rule kind needs no analog (its subject IS the linted file, no env indirection). Coverage: RED-first deceptive case, positive, missing-clean fail-closed, round-trip read-back/emit, fast-check property extended to the 5th scalar, and an end-to-end COMPOSE capstone (honest vs deceptive). Docs: ADR-550 dated addendum, prohibition-probe reference, spec-phase + verify-phase workflows. Closes #1346 Claude-Session: https://claude.ai/code/session_01GsPRb8zvpcT7Eat6vZw8PX
This commit is contained in:
@@ -114,9 +114,19 @@ This addendum ratifies three contract points:
|
||||
Net effect on D4: the *guarantee* ("a `test`-tier prohibition is never a silent pass") was preserved at every step — fail-closed-now (#644), genuine-execution (#1259), and now **machine-proven fail-first (#1279)**. A `test`-tier prohibition reaches `green`/`passed` ONLY when the wired check both genuinely, non-vacuously passes AND is independently proven to fail on a violation; every miss/fail/un-provable hard-gates. The decision also lives in `src/prohibition-enforcement.cts` comments, `gsd-core/references/prohibition-probe.md`, `gsd-core/workflows/verify-phase.md`, and the #1279 changeset.
|
||||
|
||||
**Review corrections (#1314 maintainer review) — two soundness items:**
|
||||
- **node-test fixture-existence guard (was fail-OPEN) — FIXED.** The node-test prover originally guarded only `if (!fixture)`. A missing/typo'd/stale `violationFixture` path made `GSD_PROHIB_SUBJECT` point at a non-existent file; an honest negative test then threw ENOENT *inside its callback* — a failing test named distinctly from the file — which `isNonVacuousNodeTestRed` accepted as proof, **forging a green from a setup crash** (asymmetric with the lint-rule path, which fail-CLOSES on `< 1` file result). Fixed by requiring `fs.existsSync(path.resolve(cwd, fixture))` before spawning (symmetric fail-closed; resolved against the producer's `cwd` to match the child's resolution). **Documented residual (#1346):** existence is necessary but not sufficient — a deceptive test that reds merely *because* `GSD_PROHIB_SUBJECT` is set (not because the subject's CONTENT violates) is still accepted; proving causation generically for an arbitrary author-supplied test is not possible, so it is recorded as a constraint, not implied-solved.
|
||||
- **node-test fixture-existence guard (was fail-OPEN) — FIXED.** The node-test prover originally guarded only `if (!fixture)`. A missing/typo'd/stale `violationFixture` path made `GSD_PROHIB_SUBJECT` point at a non-existent file; an honest negative test then threw ENOENT *inside its callback* — a failing test named distinctly from the file — which `isNonVacuousNodeTestRed` accepted as proof, **forging a green from a setup crash** (asymmetric with the lint-rule path, which fail-CLOSES on `< 1` file result). Fixed by requiring `fs.existsSync(path.resolve(cwd, fixture))` before spawning (symmetric fail-closed; resolved against the producer's `cwd` to match the child's resolution). **Residual (#1346) — now MITIGATED by an optional control; see the 2026-06-21 addendum below:** existence is necessary but not sufficient — a deceptive test that reds merely *because* `GSD_PROHIB_SUBJECT` is set (not because the subject's CONTENT violates) was still accepted; a generic always-on proof is impossible, so #1346 adds an **opt-in clean-subject control** that proves content-dependence when the author supplies one (and the residual remains, documented, only for checks with no control fixture).
|
||||
- **`violationFixture` projection source (#1278 ↔ #1279 now COMPOSE) — DELIVERED.** Initially `descriptorFromProjection` reconstructed only `{ kind, target, rule? }` and the projection carried no fixture, so a prohibition wired purely through the deterministic path always hard-gated. This PR threads a **fourth flat scalar `check_violation_fixture`** through `projectProhibitions` + `descriptorFromProjection` (rides both kinds; mirrors `CheckDescriptor.violationFixture`). A prohibition authored with all four scalars now **machine-proves fail-first and greens end-to-end through the projection alone** (zero hand-authoring) — the round-trip is pinned by a fast-check property + CHK-03(D) + an end-to-end COMPOSE capstone. Fail-closed is preserved: a descriptor with no `check_violation_fixture` (or a blank one) projects absent and hard-gates. The remaining work under #1346 is now just the node-test causation residual above.
|
||||
|
||||
## Addendum (2026-06-21, #1346) — node-test causation control: prove the RED is CONTENT-caused
|
||||
|
||||
The #1314 review left one tracked residual (above): the node-test prover confirms the violation fixture exists and that the negative test goes a non-vacuous RED, but could not prove the RED was caused by the subject's **content** rather than by `GSD_PROHIB_SUBJECT` merely being *set*. A deceptive content-independent test (`assert.ok(!process.env.GSD_PROHIB_SUBJECT)`) was still accepted. A general always-on proof is impossible for an arbitrary author-supplied test, so #1346 closes the gap with an **opt-in control** rather than a forced one.
|
||||
|
||||
This addendum ratifies one contract point:
|
||||
|
||||
- **(d) `CheckDescriptor.cleanFixture?` / `check_clean_fixture` — the causation control (the 5th flat scalar).** An OPTIONAL author-supplied path to a KNOWN-CLEAN control subject. When present, the node-test prover runs the SAME negative test a second time with `GSD_PROHIB_SUBJECT=<cleanFixture>` and requires it to stay a **non-vacuous GREEN**. Fail-first is then proven ONLY when the check is **RED on the violation AND GREEN on the clean subject** — i.e. the red is content-dependent. A deceptive test that reds whenever the env var is set reds on the clean subject too → the control fails → not proven (fail-closed). The scalar rides both kinds through `projectProhibitions` + `descriptorFromProjection` exactly as `check_violation_fixture` does (round-trip pinned by the fast-check property + an end-to-end COMPOSE capstone exercising both the honest and deceptive subjects).
|
||||
|
||||
**Why opt-in, not required:** making the control mandatory would regress the #1314 zero-authoring compose path — every existing node-test prohibition (which carries no clean fixture) would suddenly hard-gate. So **absent `cleanFixture` → no control runs and behavior is byte-identical to post-#1314**; the residual remains a documented permanent constraint *only* for checks whose author did not supply a clean control. An author opts into the stronger machine guarantee by supplying one. The lint-rule kind needs no analog: its "subject" *is* the linted file (no `GSD_PROHIB_SUBJECT` indirection), so the "reds because the env var is set" gap does not exist there. Net effect on D4 is unchanged — every miss/fail/un-provable still hard-gates; this only *tightens* what counts as proven. The mechanism lives in `src/prohibition-enforcement.cts` (`defaultProveFailFirst` node-test branch + the `runNodeTestWithSubject` helper) and `src/probe-core.cts` (`projectProhibitions`), compiled by `build:lib`.
|
||||
|
||||
## Addendum (2026-06-15): optional `check` descriptor on the prohibition item — D3 shape extension (#1278)
|
||||
|
||||
This ratifies the **deterministic SOURCE** for the test-tier `CheckDescriptor` that #1259 (PR #1273) left caller/verifier-supplied. #1259 shipped the PRODUCER (`check prohibition-enforcement`) that *runs* a wired check given a `{kind, target, rule?}` descriptor, but the descriptor itself was invented by the verify-phase LLM each run (the "locate" half). #1278 makes that locate half **deterministic**: an optional `check` descriptor is authored at spec-phase on the resolved `test`-tier prohibition, projected by `projectProhibitions`, and read back by verify-phase — so a wired, passing test closes the gap with **zero manual authoring**. This extends the **Decision 3 prohibition-item shape** (it adds optional keys to that item), so it is ratified here rather than rewriting D3 in place.
|
||||
|
||||
Reference in New Issue
Block a user