feat(#1602): deterministic coverage-metadata UAT routing for verify-work
Add an optional structured `coverage:` block to SUMMARY.md frontmatter and a deterministic classifier that `verify-work` consumes to route deliverables to auto-pass vs human-UAT — replacing the rejected #1598/#1599 post-hoc heuristic. - New `src/coverage.cts` (→ bin/lib/coverage.cjs) parses the nested coverage block (extractFrontmatter can't — its `-` items are scalars-only; this is a focused parser, sibling of parseMustHavesBlock), validates each entry, and classifies into auto_passed vs present. Frozen MODE/PRESENT_REASON/ERROR_CODE typed-IR surface. Exposed via `uat classify-coverage --summary <f>`. - Auto-pass is the narrow proven case only: strict-boolean human_judgment:false AND non-empty all-`pass` verification AND zero validation errors. Everything else — judgment, empty/failing verification, malformed entry — routes to the human (fail-safe). A malformed block falls back to legacy prose extraction and surfaces an error; an absent block is byte-identical to pre-#1602. - execute-plan create_summary populates the block (fail-safe default human_judgment:true); verify-work extract_tests consumes it; create_uat_file marks auto-passed entries `source: automated`. - Templates (summary + 3 variants), CONTEXT.md predicate + glossary, INVENTORY, eslint/gitignore registration, and Diataxis docs (COMMANDS reference + USER-GUIDE explanation) updated. - Behavioral tests via the CLI (no source-grep); parser-robustness regressions for the null-entry/comment-header/mis-indent cases found in adversarial review. Closes #1602 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -313,6 +313,29 @@ For browser-backed UAT, use a configured browser MCP server. The current Open GS
|
||||
/gsd-verify-work 1 # UAT for phase 1
|
||||
```
|
||||
|
||||
**Coverage-aware UAT routing (#1602).** When a SUMMARY.md carries a `coverage:` frontmatter block, `verify-work` classifies each deliverable deterministically instead of prompting for every prose bullet: deliverables proven by passing tests are auto-passed (recorded with `source: automated`, no prompt) and only judgment-dependent deliverables are presented for human sign-off. SUMMARYs without a `coverage:` block fall back to the previous prose-based extraction unchanged. See the [`coverage:` block reference](#summary-coverage-block) below.
|
||||
|
||||
#### SUMMARY `coverage:` block
|
||||
|
||||
A SUMMARY.md may carry an optional `coverage:` frontmatter block — a list of per-deliverable entries that joins requirements → tests → verification status:
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `id` | Stable identifier (`D1`, `D2`…), unique within the SUMMARY |
|
||||
| `description` | The deliverable in human-readable form |
|
||||
| `requirement` | Optional REQ-ID linking to REQUIREMENTS.md |
|
||||
| `verification[].kind` | `unit` \| `integration` \| `e2e` \| `automated_ui` \| `manual_procedural` \| `other` |
|
||||
| `verification[].ref` | Test path + descriptor, screenshot ref, or command |
|
||||
| `verification[].status` | `pass` \| `fail` \| `unknown` |
|
||||
| `human_judgment` | Required boolean. `true` always routes to a human |
|
||||
| `rationale` | Required when `human_judgment: true` |
|
||||
|
||||
A deliverable is auto-passed **only** when `human_judgment: false`, its `verification` list is non-empty, and every entry's `status` is `pass`. Anything else — `human_judgment: true`, an empty `verification`, a non-`pass` status, or a schema error — is presented to a human (fail-safe). Inspect the classification directly with:
|
||||
|
||||
```bash
|
||||
node gsd-tools.cjs uat classify-coverage --summary .planning/phases/01-foundation/01-01-SUMMARY.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
@@ -312,6 +312,7 @@
|
||||
"configuration.cjs",
|
||||
"context-utilization.cjs",
|
||||
"core-utils.cjs",
|
||||
"coverage.cjs",
|
||||
"decisions.cjs",
|
||||
"docs.cjs",
|
||||
"drift.cjs",
|
||||
|
||||
@@ -422,6 +422,7 @@ Full listing: `gsd-core/bin/lib/*.cjs`.
|
||||
| `context-utilization.cjs` | Pure classifier for `gsd-health --context` — turns (tokensUsed, contextWindow) into a `{ percent, state }` triage result against the 60%/70% fracture-point thresholds (#2792) |
|
||||
| `core-utils.cjs` | Shared low-level utilities — POSIX path normalization, sub-repo/subdirectory scanning, phase file stats, slug/one-liner/plan-id helpers, time-ago (extracted from `core.cjs`, ADR-857) |
|
||||
| `core.cjs` | Shared utilities and runtime fallbacks; compatibility re-exports for planning-workspace and I/O (`io.cjs`) helpers |
|
||||
| `coverage.cjs` | Deterministic SUMMARY `coverage:` block parser/validator/classifier for `gsd-tools uat classify-coverage`; routes deliverables to auto-pass vs human-UAT with a fail-safe default (#1602) |
|
||||
| `decisions.cjs` | Parses CONTEXT.md `<decisions>` blocks; accepts numeric (D-42) and alphanumeric (D-INFRA-01) IDs; returns `{id, text, category, tags, trackable}` |
|
||||
| `docs.cjs` | Docs-update workflow init, Markdown scanning, monorepo detection |
|
||||
| `drift.cjs` | Post-execute codebase structural drift detector (#2003): classifies file changes into new-dir/barrel/migration/route categories and round-trips `last_mapped_commit` frontmatter |
|
||||
|
||||
@@ -462,6 +462,19 @@ The review step slots in after execution and before UAT:
|
||||
|
||||
---
|
||||
|
||||
## Coverage-Aware UAT Routing
|
||||
|
||||
Historically, `/gsd-verify-work` turned every `## Accomplishments` bullet in a SUMMARY into a manual checkpoint — even deliverables already covered one-to-one by a passing unit test. With a green test suite you were still asked to re-confirm things the tests had already proven, every phase.
|
||||
|
||||
GSD now lets the executor record, at authoring time, *how each deliverable was verified*. When a SUMMARY.md carries a `coverage:` frontmatter block (see [the `coverage:` block reference](COMMANDS.md#summary-coverage-block)), `/gsd-verify-work` routes deterministically:
|
||||
|
||||
- **Auto-passed** — a deliverable marked `human_judgment: false` whose `verification` list is non-empty and entirely `pass` is recorded as passed (`source: automated`) and never prompted.
|
||||
- **Presented** — everything else is shown to you for sign-off: anything flagged `human_judgment: true` (visual adequacy, multi-device behaviour, subjective quality), anything with no verification, anything not fully passing, and any malformed entry.
|
||||
|
||||
The asymmetry is deliberate. The worst outcome is auto-passing something broken that UAT existed to catch, so auto-pass is the narrow, fully-proven case and *uncertainty always routes back to you*. Flipping the flag alone cannot skip a prompt — a passing test reference is also required. SUMMARYs without a `coverage:` block behave exactly as before (prose-based checkpoints), so nothing changes for existing or un-migrated phases.
|
||||
|
||||
---
|
||||
|
||||
## Command And Configuration Reference
|
||||
|
||||
- **Command Reference:** see [`docs/COMMANDS.md`](COMMANDS.md) for every stable command's flags, subcommands, and examples.
|
||||
|
||||
Reference in New Issue
Block a user