* test(#2856): add failing-first suite for the live-dom-uat capability Binds the approved triage shape before any of it exists: - containment — the execute:wave:post hook must not render unless workflow.live_dom_uat is true AND the capability resolves active (fail-closed on a missing state entry, and on a non-boolean value) - criterion 4 — agents/gsd-executor.md carries no browser MCP family; asserted as an absence, which is the only way it is observable - Hyrum guard — the pre-existing mcp__playwright__* branch must stay outside the key-gated block, or upgrading silently removes working automated UI verification for every current Playwright-MCP user - parity — the browser glob list now lives in two surfaces (agent frontmatter + workflow detection block); the assertion fails if either gains or loses a family without the other Red by construction: the capability, agent and workflow block do not exist yet. Verified on the remote runner. Refs #2856 * enhance(#2856): add default-off live-DOM UAT capability A phase whose acceptance criteria needed a live DOM could not be finished by the agent that executed it: gsd-executor carries no browser tools, so it correctly returned checkpoint:human-action even though the work was not human-only, just tool-less. Every such phase degraded to "executed, then finished by hand in the orchestrator", and autonomous: false could not distinguish "a human must judge this" from "the executor lacks the tool". Implements the shape approved at triage, not the one reported. The executor's tools: line is NOT widened, in any configuration: for a first-party agent the static list is the only control that exists (ADR-1244 D2, ADR-857 D4, no per-dispatch override). Instead one default-off capability owns the key, the agent, and the step: - capabilities/live-dom-uat/ — activationKey workflow.live_dom_uat (boolean, default false), one additive step at execute:wave:post (onError: skip, gates: []), so it can never halt a wave - agents/gsd-dom-verifier.md — the only GSD agent carrying browser MCP globs, in its own tools: line, with no Bash - verify-work automated_ui_verification — a gsd:live-dom-families block naming both new families AND the key; presence alone never activates Two independent fail-closed gates: isCapabilityActive renders a hook only on state.active === true, plus the step's own `when`. The pre-existing mcp__playwright__* branch keeps the gating it already had and stays outside the new block. Pulling it behind a default-off key would have silently removed working automated UI verification from every current Playwright-MCP user on upgrade. Also closes a host gap this surfaced: execute:wave:post dispatched only contribution + gate, so ANY registered step was declared and silently never run — exactly the single-kind hand-roll loop-hook-dispatch.md names. Step 5.75 now dispatches every kind == "step". The browser-profile lock is tolerated, not coordinated: --isolated is a flag on the operator's own MCP-server registration that GSD neither launches nor parameterizes, so the verifier reports could_not_look / profile_locked, names the flag, and stops. DOM-VERIFY.md keeps could_not_look and nothing_to_report distinct behind a closed reason enum — collapsing them is the ambiguous-run-notes defect reported. Verified on the remote runner. Closes #2856 * fix(#2856): apply review findings from the orthogonal passes Correctness pass (blocker): - delete detectionBlockIsCrlfSafe. It was pass-always: it read the file, replaced LF with CRLF, then indexOf'd marker strings that contain no newline, so the replacement could not change the result and the assertion could never fail for the reason it stated. There is no real CRLF risk on this surface either — the gsd:live-dom-families block has no parser, only human and agent readers. Deleted rather than replaced, per the repo's pass-always-test rule. Isolated security pass (two minors, both real): - execute-phase.md step 5.75: this change is what first activates kind == "step" dispatch at execute:wave:post, which newly opens the ref.command shell path at that loop point. Our own step uses ref.agent and never touches it, but the door is now open, so the step-dispatch line carries the same in-context validate-before-shell warning the sibling gate-dispatch line directly below it already carries. - gsd-dom-verifier: quoted page text in DOM-VERIFY.md is attacker influenced. Require it wrapped in inline code or a fence, kept short, and never left reading as a directive to the next reader. Verified on the remote runner. Refs #2856 * fix(#2856): settle the new-agent roster ripple Checkpoint 2 returned 28 failures, none in the new suite — all of them the guards that exist to make adding an agent a deliberate act. Each is a real boundary that had to move: - docs/AGENTS.md: Tools row must copy the frontmatter verbatim (#2526), so the browser globs lose their backticks; primary-agent counts 21->22, roster 33/34->34/35, Verifiers category 1->2 - docs/INVENTORY.md: roster completeness requires every agents/gsd-*.md to be classified exactly once - gsd-dom-verifier: add the anti-heredoc instruction and the commented hooks: frontmatter pattern both agent gates require - gsd-core/bin/shared/model-catalog.json: every shipped agent needs a profile entry (#3229) - copilot-install / kilo-upgrades / qwen-upgrades: expected agent list and the 34->35 roster boundary - execute-wave-post-gate-pipeline-e2e: execute:wave:post legitimately carries one step now. Asserted as an exact shape — one step, capId live-dom-uat, ref.agent gsd-dom-verifier, onError skip — so it stays a real guard against accidental change rather than being relaxed Two findings worth naming: mcp-tool-inheritance (#2526) rejected the agent for documenting mcp__playwright__* while its tools: line withholds it — a dead instruction that invites the agent to claim a path it cannot take. The prose now names the Playwright MCP family without the dispatchable token, in both the agent and the capability fragment. runtime-launcher-parity rejected the new gsd_run call: each fenced block is its own shell, so a workflow step file invoking gsd_run needs its own canonical preamble. Propagated with scripts/sync-runtime-launcher.cjs. That script also normalizes explore.md, which is unrelated pre-existing drift the parity check tolerates, so it is reverted to keep this diff scoped. The emitted-drift ack supersedes the spent #3370 entry for execute-phase.md — it is merged into next, so its ripple is absorbed at the base and it can no longer clear anything. That is the same supersede the #3370 entry itself performed on the spent #3324 fragment. Its unrelated execute-plan.md entry is untouched. Verified on the remote runner. Refs #2856 * fix(#2856): drop the stale emitted-drift ack entry The automated-ui-verification.md entry was written speculatively rather than from a reported growth, and the check names that precisely: an ack "written or reworded in THIS diff, but nothing here needed it, so it explains nothing". The growth tier keys on the bare filename as it appears under gsd-core/workflows/ or agents/. automated-ui-verification.md is nested under verify-work/steps/, so it was never in the tracked set — only execute-phase.md was ever reported, both before and after the launcher preamble landed. Only ack what the check actually reports. Verified on the remote runner. Refs #2856 * chore(#2856): backfill changeset pr number pr:0 -> 3716. The placeholder fails both changeset-lint (fail_invalid_fragment) and docs-lint (fail_malformed_fragment) by design and can only be resolved once the PR number exists. Both now report ok against GITHUB_BASE_REF=next. Refs #2856 --------- Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
@@ -1,6 +1,6 @@
|
||||
# GSD Agent Reference
|
||||
|
||||
> Full role cards for 21 primary agents plus concise stubs for 12 advanced/specialized agents (33 shipped agents total). The `agents/` directory and [`docs/INVENTORY.md`](INVENTORY.md) are the authoritative roster; see [Architecture](ARCHITECTURE.md) for context.
|
||||
> Full role cards for 22 primary agents plus concise stubs for 12 advanced/specialized agents (34 shipped agents total). The `agents/` directory and [`docs/INVENTORY.md`](INVENTORY.md) are the authoritative roster; see [Architecture](ARCHITECTURE.md) for context.
|
||||
|
||||
---
|
||||
|
||||
@@ -12,7 +12,7 @@ GSD uses a multi-agent architecture where thin orchestrators (workflow files) sp
|
||||
|
||||
### Agent Categories
|
||||
|
||||
> The table below covers the **21 primary agents** detailed in this section. Thirteen additional shipped agents (pattern-mapper, debug-session-manager, code-reviewer, code-fixer, ai-researcher, domain-researcher, eval-planner, eval-auditor, framework-selector, intel-updater, doc-classifier, doc-synthesizer, mempalace-curator) have concise stubs in the [Advanced and Specialized Agents](#advanced-and-specialized-agents) section below. For the authoritative 34-agent roster, see [`docs/INVENTORY.md`](INVENTORY.md) and the `agents/` directory.
|
||||
> The table below covers the **22 primary agents** detailed in this section. Thirteen additional shipped agents (pattern-mapper, debug-session-manager, code-reviewer, code-fixer, ai-researcher, domain-researcher, eval-planner, eval-auditor, framework-selector, intel-updater, doc-classifier, doc-synthesizer, mempalace-curator) have concise stubs in the [Advanced and Specialized Agents](#advanced-and-specialized-agents) section below. For the authoritative 35-agent roster, see [`docs/INVENTORY.md`](INVENTORY.md) and the `agents/` directory.
|
||||
|
||||
| Category | Count | Agents |
|
||||
|----------|-------|--------|
|
||||
@@ -23,7 +23,7 @@ GSD uses a multi-agent architecture where thin orchestrators (workflow files) sp
|
||||
| Roadmappers | 1 | roadmapper |
|
||||
| Executors | 1 | executor |
|
||||
| Checkers | 3 | plan-checker, integration-checker, ui-checker |
|
||||
| Verifiers | 1 | verifier |
|
||||
| Verifiers | 2 | verifier, dom-verifier |
|
||||
| Auditors | 3 | nyquist-auditor, ui-auditor, security-auditor |
|
||||
| Mappers | 1 | codebase-mapper |
|
||||
| Debuggers | 1 | debugger |
|
||||
@@ -372,6 +372,37 @@ Two further dimensions carry no number: **Verify Command Format Sanity** and
|
||||
|
||||
---
|
||||
|
||||
### gsd-dom-verifier
|
||||
|
||||
**Role:** Observes a live DOM and reports which of a wave's stated UI acceptance criteria hold. Additive — never blocks.
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Spawned by** | `live-dom-uat` capability step at `execute:wave:post` |
|
||||
| **Parallelism** | One per wave |
|
||||
| **Tools** | Read, Write, Glob, Grep, mcp__chrome-devtools__*, mcp__claude-in-chrome__* |
|
||||
| **Disallowed Tools** | Edit, Bash, the Playwright MCP family |
|
||||
| **Model (balanced)** | Sonnet |
|
||||
| **Color** | Cyan |
|
||||
| **Produces** | `{phase}-DOM-VERIFY.md` |
|
||||
| **Gated by** | `workflow.live_dom_uat` (default `false`) |
|
||||
|
||||
This is the **only** GSD agent carrying browser MCP tools. `gsd-executor` is deliberately not widened — for a first-party agent the static `tools:` list is the only control that exists ([ADR-1244](adr/1244-capability-ecosystem.md) D2, [ADR-857](adr/857-capability-system.md) D4). It carries no `Bash`: it does not start dev servers or shell out.
|
||||
|
||||
**Outcome codes** (`nothing_to_report` and `could_not_look` are never conflated):
|
||||
|
||||
| `outcome` | `reason` | Meaning |
|
||||
|---|---|---|
|
||||
| `verified` | `ok` | Criteria existed and were observed |
|
||||
| `nothing_to_report` | `no_criteria` | The wave stated no UI acceptance criteria |
|
||||
| `could_not_look` | `no_browser_mcp` | No browser MCP answered |
|
||||
| `could_not_look` | `profile_locked` | Another instance holds the browser profile |
|
||||
| `could_not_look` | `target_unreachable` | Nothing serving the target |
|
||||
|
||||
**Reference:** [Enable live-DOM verification](how-to/enable-live-dom-verification.md) · [Explanation](explanation/live-dom-uat-capability.md)
|
||||
|
||||
---
|
||||
|
||||
### gsd-codebase-mapper
|
||||
|
||||
**Role:** Explores codebase and writes structured analysis documents.
|
||||
@@ -789,7 +820,7 @@ Twelve additional agents ship under `agents/gsd-*.md` and are used by specialty
|
||||
|
||||
## Agent Tool Permissions Summary
|
||||
|
||||
> **Scope:** this table covers the 21 primary agents only. The 13 advanced/specialized agents listed above carry their own tool surfaces in their `agents/gsd-*.md` frontmatter (summarized in the per-agent stubs above and in [`docs/INVENTORY.md`](INVENTORY.md)).
|
||||
> **Scope:** this table covers the 22 primary agents only. The 13 advanced/specialized agents listed above carry their own tool surfaces in their `agents/gsd-*.md` frontmatter (summarized in the per-agent stubs above and in [`docs/INVENTORY.md`](INVENTORY.md)).
|
||||
|
||||
| Agent | Read | Write | Edit | Bash | Grep | Glob | WebSearch | WebFetch | MCP |
|
||||
|-------|------|-------|------|------|------|------|-----------|----------|-----|
|
||||
|
||||
@@ -357,6 +357,7 @@ All workflow toggles follow the **absent = enabled** pattern. If a key is missin
|
||||
| `workflow.ui_safety_gate` | boolean | `true` | Prompt to run /gsd-ui-phase for frontend phases during plan-phase |
|
||||
| `workflow.assumption_delta` | boolean | `true` | Advisory architecture checkpoint during planning. When a phase makes something **plural, optional, or chosen** that used to be **singular, required, or derived** (e.g. a second auth method, a required field becoming optional, a constant becoming a parameter), the planner is prompted to re-ask whether the primary key / identity model still names the right thing (promote the new general representation vs. add it alongside). Non-blocking; fires only on a detected signal. Bare "or" is intentionally excluded (prose false-positives). Inspect a phase with `gsd_run query assumption-delta scan <phase>`. Added in #1561 |
|
||||
| `workflow.ui_review` | boolean | `true` | Run visual quality audit (`/gsd-ui-review`) after phase execution in autonomous mode. When `false`, the UI audit step is skipped. |
|
||||
| `workflow.live_dom_uat` | boolean | `false` | **Default-off.** Enable live-DOM verification (#2856). When `true`, a `gsd-dom-verifier` step runs after each execution wave and writes `{phase}-DOM-VERIFY.md`, and the orchestrator's automated UI verification will additionally consider `mcp__chrome-devtools__*` / `mcp__claude-in-chrome__*` when present. Browser reach is confined to `gsd-dom-verifier` — `gsd-executor`'s tool surface is unchanged in every configuration. Presence of a browser MCP server is **not** sufficient on its own: a server configured for unrelated work is never driven unless this key is on. The pre-existing `mcp__playwright__*` path is unaffected by this key. Note `chrome-devtools-mcp` holds an exclusive browser-profile lock, so concurrent waves need `--isolated` on **your** MCP server registration — GSD cannot pass it. See [Enable live-DOM verification](how-to/enable-live-dom-verification.md). |
|
||||
| `workflow.node_repair` | boolean | `true` | Autonomous task repair on verification failure |
|
||||
| `workflow.node_repair_budget` | number | `2` | Max repair attempts per failed task |
|
||||
| `workflow.smart_zone_tokens` | number | `100000` | Smart-zone token budget for phase-effort estimation (#2630, [ADR-2629](adr/2629-phase-effort-estimation-calibration.md)). A phase whose estimate exceeds this is flagged with a split recommendation — **advisory only, never a block**. This is a *policy default, not a benchmark constant*: LLM output quality degrades before the advertised context window is full, but the effective ceiling is model-, task-, and distractor-dependent, so no universal number exists. Lower it for models that degrade early; the estimate-vs-actual calibration loop corrects the figure per project over time. Must be a positive integer. |
|
||||
|
||||
@@ -3496,3 +3496,25 @@ See [Resolve verify-command path findings](how-to/resolve-verify-command-path-fi
|
||||
**Known limit — task-scoped file provenance.** A `<task>` declares the files it plans to touch, but `SUMMARY.md`'s `## Files Created/Modified` describes the whole plan. Spreading that list across a plan's tasks would be inference, so a task's `changed_files` is populated only where the summary attributes files to that specific task; otherwise it is `null` with `provenance: "plan_scoped"`. Closing this needs a change to the SUMMARY format, not to the reader.
|
||||
|
||||
**Reference:** [CLI Tools](CLI-TOOLS.md#planning-inspect) · [Consume the planning snapshot](how-to/consume-the-planning-snapshot.md)
|
||||
|
||||
### 164. Live-DOM UAT Capability
|
||||
|
||||
**Config key:** `workflow.live_dom_uat` (default `false`)
|
||||
|
||||
**Purpose:** A phase with a live-UI acceptance criterion could not be finished by the agent that executed it. `gsd-executor` carries no browser tools, so it correctly returned a `checkpoint:human-action` — even though the work was not human-only, just tool-less. Every such phase quietly degraded from *executed by the executor* to *executed, then finished by hand in the orchestrator*, and the plan's `autonomous: false` marker could not distinguish "a human must judge this" from "the executor lacks the tool" (#2856).
|
||||
|
||||
**Behavior:** A default-off capability owns one boolean key, one agent, and one additive step. When the key is on, `gsd-dom-verifier` runs at `execute:wave:post` and writes `{phase}-DOM-VERIFY.md`; the orchestrator's `automated_ui_verification` step additionally considers `mcp__chrome-devtools__*` / `mcp__claude-in-chrome__*` when present.
|
||||
|
||||
**The executor's tool surface is unchanged in every configuration.** Widening it was the reported proposal and was refused: for a first-party agent the static `tools:` list is the only control that exists — no capability can grant tools to one ([ADR-1244](adr/1244-capability-ecosystem.md) D2), no hook kind grants tool permissions ([ADR-857](adr/857-capability-system.md) D4), and there is no per-dispatch override. Browser reach lives in one purpose-built agent that carries no `Bash`.
|
||||
|
||||
**Two independent gates, both fail-closed.** The capability's `activationKey` makes it resolve inactive when the key is off — `resolveLoopHooks` renders a hook only on `state.active === true` — and the step carries its own `when` guard. Tool presence alone never activates it: a browser MCP configured for unrelated work is not driven by default.
|
||||
|
||||
**The pre-existing Playwright path is untouched.** `mcp__playwright__*` keeps the gating it already had (presence plus an active UI phase). Pulling it behind a new default-off key would have silently removed working behavior from current users on upgrade; the key gates only the newly added families.
|
||||
|
||||
**It tolerates the browser-profile lock rather than coordinating it.** `chrome-devtools-mcp` holds an exclusive lock on its profile, so parallel waves collide. `--isolated` is a flag on the operator's own MCP server registration — GSD neither launches that server nor passes its arguments — so the verifier reports `could_not_look` / `profile_locked`, names the flag, and stops. No retry, no held-up wave.
|
||||
|
||||
**`nothing_to_report` is never conflated with `could_not_look`.** A report claiming no issues when it never opened a browser is worse than no report; the artifact carries a closed reason enum so the two are always distinguishable.
|
||||
|
||||
**Known limits:** no sandbox — once enabled, nothing constrains which origins are reached ([ADR-1244](adr/1244-capability-ecosystem.md) D5); DOM observation only, no screenshot diffing, accessibility audit, or performance tracing.
|
||||
|
||||
**Reference:** [Configuration](CONFIGURATION.md) · [Enable live-DOM verification](how-to/enable-live-dom-verification.md) · [Explanation](explanation/live-dom-uat-capability.md) · [Agents](AGENTS.md)
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
"gsd-doc-synthesizer",
|
||||
"gsd-doc-verifier",
|
||||
"gsd-doc-writer",
|
||||
"gsd-dom-verifier",
|
||||
"gsd-domain-researcher",
|
||||
"gsd-eval-auditor",
|
||||
"gsd-eval-planner",
|
||||
|
||||
@@ -33,6 +33,7 @@ Full roster at `agents/gsd-*.md`. The "Primary doc" column flags whether [`docs/
|
||||
| gsd-verifier | Verifies phase goal achievement through goal-backward analysis. | `/gsd-execute-phase` | primary |
|
||||
| gsd-nyquist-auditor | Fills Nyquist validation gaps by generating tests. | `/gsd-validate-phase` | primary |
|
||||
| gsd-ui-auditor | Retroactive 6-pillar visual audit of implemented frontend code. | `/gsd-ui-review` | primary |
|
||||
| gsd-dom-verifier | Observes a live DOM and reports which of a wave's stated UI acceptance criteria hold. Additive; never blocks. | `execute:wave:post` step hook (`live-dom-uat` capability) | primary |
|
||||
| gsd-codebase-mapper | Explores codebase and writes structured analysis documents. | `/gsd-map-codebase` | primary |
|
||||
| gsd-debugger | Investigates bugs using scientific method with persistent state. | `/gsd-debug`, `/gsd-verify-work` | primary |
|
||||
| gsd-user-profiler | Scores developer behavior across 8 dimensions. | `/gsd-profile-user` | primary |
|
||||
|
||||
@@ -51,6 +51,7 @@ Language versions: [English](README.md) · [Português (pt-BR)](pt-BR/README.md)
|
||||
- [Interpret `state validate` results](how-to/interpret-state-validate-results.md) — read the `scope` reason codes and tell "nothing to report" apart from "could not look"
|
||||
- [Spike and sketch](how-to/spike-and-sketch.md) — use `/gsd-spike` and `/gsd-sketch` for exploratory work before committing to a plan
|
||||
- [Design a UI phase](how-to/design-a-ui-phase.md) — use the UI phase loop for frontend and visual work
|
||||
- [Enable live-DOM verification](how-to/enable-live-dom-verification.md) — opt a project into browser-backed UI acceptance checks during execution, handle the browser-profile lock, and tell "nothing to report" apart from "could not look"
|
||||
- [Develop a Capability for GSD 1.5+](how-to/develop-a-capability.md) — add feature Capabilities, hook fragments, and registry entries
|
||||
- [Ship a reviewer lane in your capability](how-to/ship-a-reviewer-lane.md) — declare a `reviewer` body so `/gsd-review` discovers, invokes, and renders your external review CLI or model endpoint
|
||||
- [List your reviewer lane in the registry](how-to/list-your-reviewer-lane.md) — publish a lane you have built to the Reviewer Lane Registry so other people can find and install it
|
||||
|
||||
105
docs/explanation/live-dom-uat-capability.md
Normal file
105
docs/explanation/live-dom-uat-capability.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# The live-DOM UAT capability
|
||||
|
||||
Why GSD can drive a browser during execution, and why it does that from a purpose-built agent
|
||||
instead of from the plan executor.
|
||||
|
||||
## The problem
|
||||
|
||||
A phase with a live-UI acceptance criterion could not be finished by the agent that executed
|
||||
it. `gsd-executor` carries no browser tools, so it did the correct thing — returned a
|
||||
`checkpoint:human-action` at the gate — even though the work was not actually human-only. An
|
||||
orchestrator with a browser MCP server could do it unattended.
|
||||
|
||||
The practical result was that every phase with a DOM-level acceptance criterion quietly
|
||||
degraded from *executed by the executor* to *executed by the executor, then finished by hand in
|
||||
the orchestrator*. On a UI-heavy project that is routine, not an edge case. Worse, the plan's
|
||||
own `autonomous: false` marker could not distinguish **"a human must judge this"** from
|
||||
**"the executor lacks the tool"** — so the run notes had to explain the deviation every time.
|
||||
|
||||
## The shape that was not taken
|
||||
|
||||
The obvious fix is to add browser globs to `agents/gsd-executor.md`'s `tools:` line. It is one
|
||||
line, and an absent MCP server simply means the tool is not offered, so it is inert for anyone
|
||||
without one.
|
||||
|
||||
That reasoning is correct and it is not the concern. The concern is the user who **does** have
|
||||
a browser MCP configured for unrelated work. For a first-party agent, the static `tools:` list
|
||||
is the only control that exists:
|
||||
|
||||
- **A capability cannot grant tools to a first-party agent.** [ADR-1244](../adr/1244-capability-ecosystem.md)
|
||||
D2 rejects an overlay whose `id` collides with a first-party id or that claims an agent stem
|
||||
already owned. A `contribution` hook ([ADR-857](../adr/857-capability-system.md) D4) injects
|
||||
prose into a step's prompt; no hook kind grants tool permissions.
|
||||
- **There is no per-dispatch tool override.** The executor is spawned with `subagent_type`,
|
||||
`description`, `model`, and `prompt`. The agent definition file is the sole authority on its
|
||||
tool surface.
|
||||
- **Nothing else contains it.** ADR-1244 D5 is explicit: *"there is no sandbox… Consent +
|
||||
integrity + reversibility are the barrier."* There is no domain allowlist and no gate that
|
||||
inspects what a browser call fetched.
|
||||
|
||||
The codebase already reflected this instinct. `gsd-ui-auditor` — the one subagent that produces
|
||||
UI screenshots — captures via CLI rather than taking an MCP tool grant. The executor carrying
|
||||
only `mcp__context7__*` while the researcher agents carry the full web-reaching set is a
|
||||
deliberate separation, not an oversight.
|
||||
|
||||
So widening the executor would trade a narrow, auditable surface for a permanent broad one, to
|
||||
solve a problem that a narrower surface also solves.
|
||||
|
||||
## The shape that was taken
|
||||
|
||||
One default-off capability owns everything:
|
||||
|
||||
- **The key.** `workflow.live_dom_uat`, `boolean`, default `false`, declared as the
|
||||
capability's `activationKey`. With it off the capability resolves **inactive**, and
|
||||
`resolveLoopHooks` is fail-closed on `state.active === true` — the hook does not render at
|
||||
all. The step's own `when` guard is a second, independent gate.
|
||||
- **The agent.** `gsd-dom-verifier` carries `mcp__chrome-devtools__*` and
|
||||
`mcp__claude-in-chrome__*` in **its own** `tools:` line, and carries no `Bash`. Browser reach
|
||||
is confined to one agent that only exists to look at a DOM.
|
||||
- **The step.** Registered at `execute:wave:post` as a `step` hook with `onError: skip`. A step
|
||||
is additive by construction — it never halts the host. Blocking preconditions are `gate`s,
|
||||
and this capability declares none.
|
||||
|
||||
`gsd-executor`'s tool surface is unchanged in every configuration. That is a tested invariant,
|
||||
asserted as an absence, because that is the only way it is observable.
|
||||
|
||||
## Why the existing Playwright path was left alone
|
||||
|
||||
The orchestrator's `automated_ui_verification` step already used `mcp__playwright__*`, gated on
|
||||
tool presence **plus** an active UI phase. Pulling that branch behind a new default-off key
|
||||
would have silently removed working behaviour from every current Playwright-MCP user on
|
||||
upgrade — a regression wearing an enhancement's clothes.
|
||||
|
||||
So the key gates only the **newly added** families. Playwright keeps the gating it already had.
|
||||
The instruction to gate on "presence AND the config key, not presence alone" is what gives the
|
||||
new families the same two-condition shape Playwright already possessed.
|
||||
|
||||
## Why there is no browser-lock coordination
|
||||
|
||||
`chrome-devtools-mcp` holds an exclusive lock on its browser profile, so parallel execution
|
||||
waves will collide. The tempting design is a lease or queue around the profile.
|
||||
|
||||
GSD cannot enforce one. `--isolated` is a flag on the **user's** MCP server registration; GSD
|
||||
neither launches that server nor passes its arguments. Coordination over a resource you do not
|
||||
own is theatre — it adds machinery, and the lock still happens.
|
||||
|
||||
So the verifier tolerates the lock instead: it reports `could_not_look` / `profile_locked`,
|
||||
names `--isolated` so the operator knows the remedy, and stops. No retry, no wait, no held-up
|
||||
wave. The docs carry the flag; the code does not pretend to.
|
||||
|
||||
## The distinction the artifact must preserve
|
||||
|
||||
`DOM-VERIFY.md` separates **`nothing_to_report`** (there were no UI criteria) from
|
||||
**`could_not_look`** (there were criteria, and the check did not happen — with a reason code
|
||||
saying which). Collapsing those two is what produced the ambiguous run notes that opened the
|
||||
original report. A report claiming *no issues* when it never opened a browser is worse than no
|
||||
report.
|
||||
|
||||
## Known limits
|
||||
|
||||
- No sandbox. Once the key is on, nothing constrains which origins a browser call reaches.
|
||||
This capability narrows *who* can reach a browser, not *where* it may go.
|
||||
- Concurrent waves still collide on a shared profile unless the operator passes `--isolated`.
|
||||
- DOM observation only — no screenshot diffing, accessibility audit, or performance tracing.
|
||||
- `chrome-devtools` and `claude-in-chrome` are detected but not feature-normalized; the
|
||||
verifier uses whichever answers and does not paper over differences between them.
|
||||
133
docs/how-to/enable-live-dom-verification.md
Normal file
133
docs/how-to/enable-live-dom-verification.md
Normal file
@@ -0,0 +1,133 @@
|
||||
# How to enable live-DOM verification
|
||||
|
||||
Let GSD open a real browser and check a phase's UI acceptance criteria against the live DOM —
|
||||
during execution, not only after it — without widening what the plan executor can reach.
|
||||
|
||||
> **Default-off, and deliberately so.** A browser MCP server you configured for unrelated work
|
||||
> must not start driving your project's UI on its own. You opt in per project with one key. See
|
||||
> the [explanation](../explanation/live-dom-uat-capability.md) for why the executor's own tool
|
||||
> surface was left alone.
|
||||
|
||||
**What you need:**
|
||||
|
||||
- GSD installed with the `full` profile (the capability is `tier: full`).
|
||||
- A browser MCP server registered in your runtime — either
|
||||
[`chrome-devtools-mcp`](https://github.com/ChromeDevTools/chrome-devtools-mcp) (exposes
|
||||
`mcp__chrome-devtools__*`) or Claude-in-Chrome (exposes `mcp__claude-in-chrome__*`).
|
||||
- Something serving your UI — a dev server, a preview deployment, any reachable URL.
|
||||
- A phase whose plan actually states UI acceptance criteria. The verifier will not invent them.
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Turn the key on
|
||||
|
||||
```bash
|
||||
gsd-tools query config-set workflow.live_dom_uat true
|
||||
```
|
||||
|
||||
Verify it took:
|
||||
|
||||
```bash
|
||||
gsd-tools query config-get workflow.live_dom_uat
|
||||
# → true
|
||||
```
|
||||
|
||||
That one key gates both halves: the `gsd-dom-verifier` step that runs after each execution
|
||||
wave, and the extra browser families the orchestrator's own UI-verification step will consider.
|
||||
With it off, neither reaches a browser.
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — Make the browser reachable to more than one wave
|
||||
|
||||
`chrome-devtools-mcp` keeps an **exclusive lock** on its browser profile at
|
||||
`$HOME/.cache/chrome-devtools-mcp/chrome-profile`. A second instance fails with:
|
||||
|
||||
```
|
||||
The browser is already running for <dir>. Use --isolated to run multiple browser instances.
|
||||
```
|
||||
|
||||
GSD runs execution waves in parallel, so two verifiers can reach for one profile. **GSD cannot
|
||||
fix this for you** — `--isolated` is a flag on *your* MCP server registration, not something
|
||||
GSD passes. Add it there:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"mcpServers": {
|
||||
"chrome-devtools": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "chrome-devtools-mcp@latest", "--isolated"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`--isolated` gives each instance a throwaway profile. If you would rather share one server
|
||||
across concurrent agents, `--experimentalPageIdRouting` routes tools per page instead.
|
||||
|
||||
Skipping this step is safe — you just get `could_not_look` / `profile_locked` on the waves that
|
||||
lost the race, never a failed wave.
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — Run a phase and read the report
|
||||
|
||||
Execute normally. After each wave, `gsd-dom-verifier` writes
|
||||
`.planning/phases/<phase>/<n>-DOM-VERIFY.md`:
|
||||
|
||||
```markdown
|
||||
---
|
||||
schema_version: 1
|
||||
wave: 2
|
||||
outcome: verified
|
||||
reason: ok
|
||||
checked: 4
|
||||
passed: 3
|
||||
failed: 0
|
||||
needs_review: 1
|
||||
---
|
||||
```
|
||||
|
||||
The body lists one line per criterion with the observation behind its verdict.
|
||||
|
||||
---
|
||||
|
||||
## Reading the outcome — "nothing to report" is not "could not look"
|
||||
|
||||
This is the part worth learning, because a report that says *no issues* when it never opened a
|
||||
browser is worse than no report at all.
|
||||
|
||||
| `outcome` | `reason` | What actually happened | What to do |
|
||||
|---|---|---|---|
|
||||
| `verified` | `ok` | Criteria existed and were observed | Read the per-criterion lines |
|
||||
| `nothing_to_report` | `no_criteria` | The wave's plan stated no UI acceptance criteria | Nothing. This is a clean result |
|
||||
| `could_not_look` | `no_browser_mcp` | Key is on, but no browser MCP answered | Check your MCP server is registered and running |
|
||||
| `could_not_look` | `profile_locked` | Another instance holds the browser profile | Add `--isolated` — see Step 2 |
|
||||
| `could_not_look` | `target_unreachable` | Nothing was serving the criterion's URL | Start your dev server before executing |
|
||||
|
||||
Only `could_not_look` means the check did not happen. `nothing_to_report` means it happened and
|
||||
found nothing to check.
|
||||
|
||||
---
|
||||
|
||||
## What this does not do
|
||||
|
||||
- **It never blocks.** The step is advisory by construction — it cannot fail a task, fail a
|
||||
wave, or stop a phase. Findings are findings; the executor still owns task outcomes.
|
||||
- **It does not widen the executor.** `gsd-executor` carries no browser tools in any
|
||||
configuration. The browser reach lives in `gsd-dom-verifier` alone.
|
||||
- **It does not sandbox the browser.** Once the key is on there is no domain allowlist and
|
||||
nothing inspects what a page fetched. Turn it on for projects where that is acceptable.
|
||||
- **It observes the DOM only.** No screenshot diffing, no accessibility audit, no performance
|
||||
tracing. A criterion needing one of those comes back `needs_review` with the reason named.
|
||||
|
||||
---
|
||||
|
||||
## Turning it back off
|
||||
|
||||
```bash
|
||||
gsd-tools query config-set workflow.live_dom_uat false
|
||||
```
|
||||
|
||||
The capability resolves inactive immediately and the hook stops rendering. See
|
||||
[Turn a capability off (and keep it off)](turn-a-capability-off.md) for removing it entirely.
|
||||
@@ -44,7 +44,7 @@ Core package and are stamped with the package version at release (per
|
||||
ADR-1244 D6). They are not subject to the consent or integrity-pin flow applied
|
||||
to third-party capabilities.
|
||||
|
||||
### Feature capabilities (role: feature) — 21
|
||||
### Feature capabilities (role: feature) — 22
|
||||
|
||||
Feature capabilities extend what the loop does — contributing research,
|
||||
planning, execution, verification, or ship artefacts at the loop extension
|
||||
@@ -63,6 +63,7 @@ points.
|
||||
| `gap-analysis` | feature | standard | `>=1.6.0` | `plan:post` | gate | first-party |
|
||||
| `graphify` | feature | full | `>=1.6.0` | — | — | first-party |
|
||||
| `intel` | feature | full | `>=1.6.0` | `plan:pre` | step | first-party |
|
||||
| `live-dom-uat` | feature | full | `>=1.11.0` | `execute:wave:post` | step | first-party |
|
||||
| `mempalace` | feature | full | `>=1.6.0` | `discuss:pre`, `discuss:post`, `plan:pre`, `plan:post`, `execute:wave:post`, `verify:post`, `ship:post` | step, contribution | first-party |
|
||||
| `nyquist` | feature | full | `>=1.6.0` | `verify:post` | step | first-party |
|
||||
| `pattern-mapper` | feature | full | `>=1.6.0` | `plan:pre` | step | first-party |
|
||||
|
||||
Reference in New Issue
Block a user