fix(#3770): require intentional RED evidence before GREEN (#4279)

* test(3770): add failing tests for intentional RED evidence gate

RED: classifyRedEvidence / buildRedEvidenceRecord / check tdd-red-evidence do
not exist yet; every row fails on require. Per #3770 only an intentional
target-test failure may authorize GREEN; zero-test discovery, fixture crashes,
unrelated failures, and unexpected green are INVALID_RED.

* fix(3770): require intentional RED evidence before GREEN

Only an intentional failure of the TARGET test (distinctly named, TAP-reported
assertion failure) classifies as RED_EVIDENCE_OK and authorizes GREEN. Zero-test
discovery, fixture/load crashes (file-named failures), nonzero exits without a
failing test, unrelated failures, unexpected greens, and malformed/missing
records are INVALID_RED and block GREEN.

- src/tdd-red-evidence.cts: pure classifier + persisted record builder (reuses
  the prohibition-enforcement TAP primitives; fail-closed, never throws)
- check tdd-red-evidence <record.json>: validates the persisted record
  (command, exit code, failing test, expected, actual)
- gsd-executor.md / references/tdd.md / references/execute-mvp-tdd.md: RED now
  requires the evidence record + gate verdict, not a nonzero exit or a RED: tag

* chore(3770): regenerate inventory manifest for tdd-red-evidence.cjs

* fix(3770): fit executor fail-fast under size cap, fix unrelated-failure fixture, ignore generated lib

- gsd-executor.md: compress the #3770 fail-fast rule to one line (49149 B <
  49152 cap; line-count parity keeps the #2751 PROSE_ALLOWLIST line 816 valid)
- tests: the row-6 fixture used String.replace (first-occurrence), so the
  `not ok` line still named the target test and the classifier was right to
  accept it; replaceAll makes the failure genuinely unrelated
- eslint.config.mjs: ignore tsc-generated bin/lib/tdd-red-evidence.cjs
  (lint the src/*.cts source, per ADR-457 migration rule)

Emitted-Drift-Ack-Growth: gsd-executor.md — the #3770 fail-fast rule now requires intentional RED evidence (check tdd-red-evidence) before GREEN; +172 bytes, kept under the LARGE cap and on one line

* chore(3770): add changeset

* chore(3770): backfill PR number in changeset

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-09-04 16:44:38 -04:00
committed by GitHub
parent 2f4f7538e9
commit 8249ebcf6e
10 changed files with 560 additions and 14 deletions

View File

@@ -15,12 +15,15 @@ If any of these is false, the gate is inactive — execution proceeds normally.
For each task gated by TDD, the executor MUST verify (before running the implementation step):
1. **A failing-test commit exists.** Search git log on the current branch for a commit matching `test({phase}-{plan})` whose subject mentions the same plan as the current task. The commit must touch a test file (`*.test.*`, `*.spec.*`, `tests/**`).
2. **The test was actually red.** The commit message body or the executor's recent shell history must show the test failed when first run. Acceptable evidence:
- Commit message contains `RED:` prefix or `(RED)` tag
- Recent terminal output shows `FAIL` or non-zero exit on the new test before any implementation commit
2. **The test was actually red — INTENTIONALLY (#3770).** A nonzero exit is not RED by itself: syntax errors, zero-test discovery, fixture crashes, parser errors, and unrelated assertions are INVALID_RED. The executor must persist the RED evidence record (command, exit code, failing test, expected result from `<behavior>`, actual result) and verify it:
```bash
gsd_run check tdd-red-evidence <record.json> --raw
```
- `RED_EVIDENCE_OK` (reason `target_test_failed`): the TARGET test named by the plan failed on a real assertion — the ONLY verdict that authorizes GREEN.
- `INVALID_RED` (reasons `unexpected_green`, `zero_tests_discovered`, `nonzero_exit_without_test_failure`, `fixture_or_load_failure`, `no_target_test_failure`, `invalid_record`, `unreadable_record`): the gate trips — halt, fix the RED phase (test identity, fixture, discovery), and re-verify before any implementation step. A `RED:` prefix or `(RED)` tag in the commit message is NOT sufficient evidence on its own.
3. **No implementation commit yet.** No `feat({phase}-{plan})` commit may exist for the same plan ID before the failing-test commit.
If any check fails, the gate trips.
If any check fails, the gate trips. For check 2, an INVALID_RED verdict (`check tdd-red-evidence`) trips the gate — the executor MUST halt and block the implementation step.
## What "behavior-adding task" means
@@ -41,7 +44,7 @@ The executor MUST:
```
### TDD GATE TRIPPED — Plan {plan_id}, Task {task_id}
Reason: {missing_red_commit | red_commit_not_failing | feat_before_test}
Reason: {missing_red_commit | red_commit_not_failing | feat_before_test | invalid_red}
Behavior expected to be tested:
- {first behavior bullet}
@@ -69,7 +72,7 @@ The `--force-mvp-gate` flag is documented but not introduced by this plan — it
## What this gate does NOT do
- It does not enforce REFACTOR commits. REFACTOR remains optional (per `gsd-core/references/tdd.md`).
- It does not check test quality (the test could be trivially passing). That's the planner's job.
- It does not check test quality (the test could be trivially weak). That's the planner's job. It DOES check that the RED failure was intentional — the target test failing an assertion (#3770).
- It does not run tests. The executor only inspects git log + file system. Running tests is the implementation step's job.
- It does not gate config-only or doc-only tasks (see "behavior-adding task" definition).

View File

@@ -94,9 +94,10 @@ After completion, create SUMMARY.md with:
**RED - Write failing test:**
1. Create test file following project conventions
2. Write test describing expected behavior (from `<behavior>` element)
3. Run test - it MUST fail
4. If test passes: feature exists or test is wrong. Investigate.
5. Commit: `test({phase}-{plan}): add failing test for [feature]`
3. Run test - it MUST fail **intentionally** (#3770): the TARGET test you named must be the test that fails, on an assertion for the planned behavior. A nonzero exit alone is NOT RED — syntax errors, zero-test discovery, fixture crashes, parser errors, and unrelated assertions are INVALID_RED and must not authorize GREEN.
4. Persist the RED evidence record (command, exit code, failing test, expected result, actual result) and verify it: `gsd_run check tdd-red-evidence <record.json>`. Only verdict `RED_EVIDENCE_OK` satisfies the RED gate; `INVALID_RED` blocks GREEN until the RED phase is fixed.
5. If test passes: feature exists or test is wrong. Investigate.
6. Commit: `test({phase}-{plan}): add failing test for [feature]`
**GREEN - Implement to pass:**
1. Write minimal code to make test pass
@@ -256,15 +257,16 @@ When `workflow.tdd_mode` is enabled in config, the RED/GREEN/REFACTOR gate seque
| Gate | Required | Commit Pattern | Validation |
|------|----------|---------------|------------|
| RED | Yes | `test({phase}-{plan}): ...` | Test exists AND fails before implementation |
| RED | Yes | `test({phase}-{plan}): ...` | Test exists AND fails before implementation — intentionally: `check tdd-red-evidence` returns `RED_EVIDENCE_OK` (target test failed on an assertion for the behavior; anything else is INVALID_RED) |
| GREEN | Yes | `feat({phase}-{plan}): ...` | Test passes after implementation |
| REFACTOR | No | `refactor({phase}-{plan}): ...` | Tests still pass after cleanup |
### Fail-Fast Rules
1. **Unexpected GREEN in RED phase:** If the test passes before any implementation code is written, STOP. The feature may already exist or the test is wrong. Investigate before proceeding.
2. **Missing RED commit:** If no `test(...)` commit precedes the `feat(...)` commit, the TDD discipline was violated. Flag in SUMMARY.md.
3. **REFACTOR breaks tests:** Undo the refactor immediately. Commit was premature — refactor in smaller steps.
2. **INVALID_RED in RED phase (#3770):** A nonzero exit is not RED by itself. Zero-test discovery, fixture/load crashes, nonzero exits with no failing test, unrelated failing tests, and unexpected greens all classify as INVALID_RED (`gsd_run check tdd-red-evidence`). STOP and fix the RED phase — do NOT proceed to GREEN.
3. **Missing RED commit:** If no `test(...)` commit precedes the `feat(...)` commit, the TDD discipline was violated. Flag in SUMMARY.md.
4. **REFACTOR breaks tests:** Undo the refactor immediately. Commit was premature — refactor in smaller steps.
### Executor Gate Validation