* feat(#247): runtime-neutral phase uat-passed predicate from HUMAN-UAT results Wire the already-reserved `phase.uat-passed` alias (subcommand `uat-passed`, mutation:false) into the phase command router with a new markdown-aware predicate that evaluates HUMAN-UAT results and reports pass only when every required check passes. Post-SDK-retirement (ADR-0174/#174) successor to the SDK-framed #70, with no SDK-specific API surface. New pure module src/uat-predicate.cts: - stripFalsePositiveContexts: frontmatter -> HTML-comment -> CommonMark-style fenced-block state machine (tracks delimiter char+length) -> blockquote, each a small composable step, so a `result: passed` inside frontmatter, a fenced/~~~ block (incl. ~~~ nested in a ``` fence), a comment, or a blockquote is never counted. - parseUatResultItems: heading-block parser, column-0-anchored same-line result; a heading with no result -> `missing` (fail-closed). - analyzeMarkdown: unterminated fence/comment detection (malformed -> blocker). - evaluateUatPassed: allowlist pass/verification semantics; passed = no blockers && >=1 check && all passing; no_uat_artifacts discriminator (no vacuous pass); optional requireVerification policy hook. Thin cmdPhaseUatPassed handler in phase.cts; router closure rejects unknown flags via makeInvalidArgs. Hardened across two Codex adversarial passes (vacuous pass, dropped failing tests, permissive verification status, nested-fence escape, cross-line result value, masked unterminated comment) — all fixed fail-closed. New unit + CLI-integration suites incl. a fast-check property test; docs, CONTEXT glossary, inventory, and changeset updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(#247): backfill changeset PR number (#1063) * fix(#247): indexOf paired-scan for unterminated-comment detection CodeQL js/incomplete-multi-character-sanitization (high) flagged the `raw.replace(/<!--[\s\S]*?-->/g,'')`-then-`.includes('<!--')` detection in analyzeMarkdown as incomplete sanitization (a single regex pass can leave a residual `<!--`). Replace it with a paired left-to-right indexOf scan that contains no `.replace()` of the comment token — CodeQL-clean and strictly more correct (a closed earlier comment can never mask a later unterminated one). Behaviour unchanged; 98 predicate tests + scoped docker run green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
@@ -118,6 +118,10 @@ node gsd-tools.cjs phase remove <phase> [--force]
|
||||
# Mark phase complete, update state + roadmap
|
||||
node gsd-tools.cjs phase complete <phase>
|
||||
|
||||
# Evaluate HUMAN-UAT results for a phase (markdown-aware; ignores false-positive contexts)
|
||||
# Returns JSON: { passed, uat_files[], verification_files[], checks[], blockers[], policy }
|
||||
node gsd-tools.cjs phase uat-passed <phase> [--require-verification]
|
||||
|
||||
# Index plans with waves and status
|
||||
node gsd-tools.cjs phase-plan-index <phase>
|
||||
|
||||
|
||||
@@ -490,6 +490,37 @@ Retroactively audit and fill Nyquist validation gaps.
|
||||
|
||||
---
|
||||
|
||||
### `phase uat-passed <N> [--require-verification]`
|
||||
|
||||
Runtime-neutral predicate that evaluates HUMAN-UAT results for a phase and reports whether all required checks passed. Uses markdown-aware parsing that ignores false-positive contexts (YAML frontmatter, fenced code blocks, HTML comments, and blockquotes), so incomplete checkbox fragments in prose sections never trigger a false pass.
|
||||
|
||||
| Argument | Required | Description |
|
||||
|----------|----------|-------------|
|
||||
| `N` | **Yes** | Phase number to evaluate |
|
||||
| `--require-verification` | No | Require at least one `*-VERIFICATION.md` file alongside UAT results; fails if none are found |
|
||||
|
||||
**Output fields (JSON):**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `passed` | `boolean` | `true` only when at least one check exists AND all checks pass AND no blockers — fail-closed (no vacuous pass) |
|
||||
| `uat_files` | `string[]` | Filenames of `*-UAT.md` files evaluated |
|
||||
| `verification_files` | `string[]` | Filenames of `*-VERIFICATION.md` files evaluated |
|
||||
| `checks[]` | `{ file, test, name, result, passing }[]` | Per-item evaluation results parsed from heading blocks |
|
||||
| `blockers[]` | `string[]` | Human-readable reasons for failure (frontmatter issues, failing/missing test items, policy violations, malformed markdown) — NOT a subset of `checks[]` |
|
||||
| `no_uat_artifacts` | `boolean` | `true` when no real UAT test items were parsed (no `*-UAT.md` files, unreadable dir, or files with no test blocks); when `true`, `passed` is always `false` |
|
||||
| `policy.require_verification` | `boolean` | Whether `--require-verification` was active |
|
||||
|
||||
**Programmatic access:** `node gsd-tools.cjs phase uat-passed <N> [--require-verification] [--raw]` — see [CLI Tools Reference](CLI-TOOLS.md)
|
||||
|
||||
```bash
|
||||
node gsd-tools.cjs phase uat-passed 3 # Evaluate UAT for phase 3
|
||||
node gsd-tools.cjs phase uat-passed 3 --require-verification # Also require VERIFICATION.md
|
||||
node gsd-tools.cjs phase uat-passed 3 --raw # Machine-readable JSON output
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Navigation Commands
|
||||
|
||||
### `/gsd-progress`
|
||||
|
||||
@@ -164,6 +164,7 @@
|
||||
- [Statusline Context Position](#140-statusline-context-position)
|
||||
- [Milestone Tag Creation Toggle](#141-milestone-tag-creation-toggle)
|
||||
- [Structured JSON Error Mode](#142-structured-json-error-mode)
|
||||
- [UAT-Passed Predicate](#143-uat-passed-predicate)
|
||||
|
||||
---
|
||||
|
||||
@@ -3046,6 +3047,38 @@ explicit reviewer flags -> --all -> review.default_reviewers -> all detected rev
|
||||
|
||||
---
|
||||
|
||||
### 143. UAT-Passed Predicate
|
||||
|
||||
**CLI:** `node gsd-tools.cjs phase uat-passed <N> [--require-verification]`
|
||||
|
||||
**Purpose:** Provide a runtime-neutral, automatable predicate that evaluates HUMAN-UAT results for a phase and returns a structured pass/fail verdict with full diagnostic detail.
|
||||
|
||||
**Behavior:** Locates `*-UAT.md` and optionally `*-VERIFICATION.md` files for the given phase, parses UAT test blocks (heading-block parser, column-0 result lines) with a markdown-aware stripper that removes false-positive contexts (YAML frontmatter, fenced code blocks, HTML comments, and blockquotes). Returns `passed: true` only when at least one check exists AND all checks pass AND no blockers — fail-closed, no vacuous pass. The `--require-verification` flag requires at least one `*-VERIFICATION.md` with an allowlisted passing status; the command fails without one.
|
||||
|
||||
**Output envelope:** `{ passed, uat_files[], verification_files[], checks[], blockers[], no_uat_artifacts, policy: { require_verification } }`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `passed` | `boolean` | `true` only when ≥1 check exists AND all passing AND no blockers |
|
||||
| `uat_files` | `string[]` | Filenames of `*-UAT.md` files evaluated |
|
||||
| `verification_files` | `string[]` | Filenames of `*-VERIFICATION.md` files evaluated |
|
||||
| `checks[]` | `{ file, test, name, result, passing }[]` | Per-item results from heading blocks |
|
||||
| `blockers[]` | `string[]` | Human-readable failure reasons (frontmatter, failing/missing items, policy, malformed markdown) |
|
||||
| `no_uat_artifacts` | `boolean` | `true` when no test items were parsed; `passed` is always `false` when `true` |
|
||||
| `policy.require_verification` | `boolean` | Whether `--require-verification` was active |
|
||||
|
||||
**Requirements:**
|
||||
- REQ-UAT-PRED-01: The predicate MUST ignore result lines inside YAML frontmatter, fenced code blocks, HTML comments, and blockquotes.
|
||||
- REQ-UAT-PRED-02: `passed: true` MUST require at least one check AND all checks passing AND no blockers (fail-closed, no vacuous pass).
|
||||
- REQ-UAT-PRED-03: `--require-verification` MUST cause the command to fail when no `*-VERIFICATION.md` file with an allowlisted passing status is found.
|
||||
- REQ-UAT-PRED-04: `blockers[]` contains all human-readable failure reasons including frontmatter issues, policy violations, and malformed markdown — NOT limited to a subset of `checks[]`.
|
||||
- REQ-UAT-PRED-05: The module MUST be runtime-neutral (no runtime-specific env checks or exit shortcuts).
|
||||
- REQ-UAT-PRED-06: A heading block with no column-0 `result:` line emits `result:'missing'` (blocker); test items are never silently dropped.
|
||||
|
||||
**Reference:** [Phase Management Commands](COMMANDS.md#phase-uat-passed-n---require-verification)
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- [Commands](COMMANDS.md)
|
||||
|
||||
@@ -356,6 +356,7 @@
|
||||
"surface.cjs",
|
||||
"task-command-router.cjs",
|
||||
"template.cjs",
|
||||
"uat-predicate.cjs",
|
||||
"uat.cjs",
|
||||
"ui-safety-gate.cjs",
|
||||
"update-context.cjs",
|
||||
|
||||
@@ -371,7 +371,7 @@ The `gsd-planner` agent is decomposed into a core agent plus reference modules t
|
||||
|
||||
---
|
||||
|
||||
## CLI Modules (105 shipped)
|
||||
## CLI Modules (106 shipped)
|
||||
|
||||
Full listing: `gsd-core/bin/lib/*.cjs`.
|
||||
|
||||
@@ -468,6 +468,7 @@ Full listing: `gsd-core/bin/lib/*.cjs`.
|
||||
| `task-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools task` |
|
||||
| `template.cjs` | Template selection and filling with variable substitution |
|
||||
| `uat.cjs` | UAT file parsing, verification debt tracking, audit-uat support |
|
||||
| `uat-predicate.cjs` | UAT-passed predicate — markdown-aware evaluation of HUMAN-UAT results; returns pass only when all required checks pass; ignores false-positive contexts (frontmatter, fenced code, blockquotes, HTML comments) |
|
||||
| `ui-safety-gate.cjs` | Shell-free word-boundary UI token detector (#3706, #3718); reads phase-section text from stdin, exits 0 (UI found) or 1 (no UI); also deployed to `gsd-core/bin/lib/` so the GSD installer ships it to `$RUNTIME_DIR` (#448) |
|
||||
| `update-context.cjs` | Pure install-context resolver for `/gsd:update` — runtime/scope/config-dir/version detection (LOCAL/GLOBAL/UNKNOWN) ported from update.md bash; backs `gsd-tools update-context` (#498) |
|
||||
| `validate-command-router.cjs` | Thin CJS subcommand router adapter for `gsd-tools validate` |
|
||||
|
||||
Reference in New Issue
Block a user