* test(#2856): add failing-first suite for the live-dom-uat capability Binds the approved triage shape before any of it exists: - containment — the execute:wave:post hook must not render unless workflow.live_dom_uat is true AND the capability resolves active (fail-closed on a missing state entry, and on a non-boolean value) - criterion 4 — agents/gsd-executor.md carries no browser MCP family; asserted as an absence, which is the only way it is observable - Hyrum guard — the pre-existing mcp__playwright__* branch must stay outside the key-gated block, or upgrading silently removes working automated UI verification for every current Playwright-MCP user - parity — the browser glob list now lives in two surfaces (agent frontmatter + workflow detection block); the assertion fails if either gains or loses a family without the other Red by construction: the capability, agent and workflow block do not exist yet. Verified on the remote runner. Refs #2856 * enhance(#2856): add default-off live-DOM UAT capability A phase whose acceptance criteria needed a live DOM could not be finished by the agent that executed it: gsd-executor carries no browser tools, so it correctly returned checkpoint:human-action even though the work was not human-only, just tool-less. Every such phase degraded to "executed, then finished by hand in the orchestrator", and autonomous: false could not distinguish "a human must judge this" from "the executor lacks the tool". Implements the shape approved at triage, not the one reported. The executor's tools: line is NOT widened, in any configuration: for a first-party agent the static list is the only control that exists (ADR-1244 D2, ADR-857 D4, no per-dispatch override). Instead one default-off capability owns the key, the agent, and the step: - capabilities/live-dom-uat/ — activationKey workflow.live_dom_uat (boolean, default false), one additive step at execute:wave:post (onError: skip, gates: []), so it can never halt a wave - agents/gsd-dom-verifier.md — the only GSD agent carrying browser MCP globs, in its own tools: line, with no Bash - verify-work automated_ui_verification — a gsd:live-dom-families block naming both new families AND the key; presence alone never activates Two independent fail-closed gates: isCapabilityActive renders a hook only on state.active === true, plus the step's own `when`. The pre-existing mcp__playwright__* branch keeps the gating it already had and stays outside the new block. Pulling it behind a default-off key would have silently removed working automated UI verification from every current Playwright-MCP user on upgrade. Also closes a host gap this surfaced: execute:wave:post dispatched only contribution + gate, so ANY registered step was declared and silently never run — exactly the single-kind hand-roll loop-hook-dispatch.md names. Step 5.75 now dispatches every kind == "step". The browser-profile lock is tolerated, not coordinated: --isolated is a flag on the operator's own MCP-server registration that GSD neither launches nor parameterizes, so the verifier reports could_not_look / profile_locked, names the flag, and stops. DOM-VERIFY.md keeps could_not_look and nothing_to_report distinct behind a closed reason enum — collapsing them is the ambiguous-run-notes defect reported. Verified on the remote runner. Closes #2856 * fix(#2856): apply review findings from the orthogonal passes Correctness pass (blocker): - delete detectionBlockIsCrlfSafe. It was pass-always: it read the file, replaced LF with CRLF, then indexOf'd marker strings that contain no newline, so the replacement could not change the result and the assertion could never fail for the reason it stated. There is no real CRLF risk on this surface either — the gsd:live-dom-families block has no parser, only human and agent readers. Deleted rather than replaced, per the repo's pass-always-test rule. Isolated security pass (two minors, both real): - execute-phase.md step 5.75: this change is what first activates kind == "step" dispatch at execute:wave:post, which newly opens the ref.command shell path at that loop point. Our own step uses ref.agent and never touches it, but the door is now open, so the step-dispatch line carries the same in-context validate-before-shell warning the sibling gate-dispatch line directly below it already carries. - gsd-dom-verifier: quoted page text in DOM-VERIFY.md is attacker influenced. Require it wrapped in inline code or a fence, kept short, and never left reading as a directive to the next reader. Verified on the remote runner. Refs #2856 * fix(#2856): settle the new-agent roster ripple Checkpoint 2 returned 28 failures, none in the new suite — all of them the guards that exist to make adding an agent a deliberate act. Each is a real boundary that had to move: - docs/AGENTS.md: Tools row must copy the frontmatter verbatim (#2526), so the browser globs lose their backticks; primary-agent counts 21->22, roster 33/34->34/35, Verifiers category 1->2 - docs/INVENTORY.md: roster completeness requires every agents/gsd-*.md to be classified exactly once - gsd-dom-verifier: add the anti-heredoc instruction and the commented hooks: frontmatter pattern both agent gates require - gsd-core/bin/shared/model-catalog.json: every shipped agent needs a profile entry (#3229) - copilot-install / kilo-upgrades / qwen-upgrades: expected agent list and the 34->35 roster boundary - execute-wave-post-gate-pipeline-e2e: execute:wave:post legitimately carries one step now. Asserted as an exact shape — one step, capId live-dom-uat, ref.agent gsd-dom-verifier, onError skip — so it stays a real guard against accidental change rather than being relaxed Two findings worth naming: mcp-tool-inheritance (#2526) rejected the agent for documenting mcp__playwright__* while its tools: line withholds it — a dead instruction that invites the agent to claim a path it cannot take. The prose now names the Playwright MCP family without the dispatchable token, in both the agent and the capability fragment. runtime-launcher-parity rejected the new gsd_run call: each fenced block is its own shell, so a workflow step file invoking gsd_run needs its own canonical preamble. Propagated with scripts/sync-runtime-launcher.cjs. That script also normalizes explore.md, which is unrelated pre-existing drift the parity check tolerates, so it is reverted to keep this diff scoped. The emitted-drift ack supersedes the spent #3370 entry for execute-phase.md — it is merged into next, so its ripple is absorbed at the base and it can no longer clear anything. That is the same supersede the #3370 entry itself performed on the spent #3324 fragment. Its unrelated execute-plan.md entry is untouched. Verified on the remote runner. Refs #2856 * fix(#2856): drop the stale emitted-drift ack entry The automated-ui-verification.md entry was written speculatively rather than from a reported growth, and the check names that precisely: an ack "written or reworded in THIS diff, but nothing here needed it, so it explains nothing". The growth tier keys on the bare filename as it appears under gsd-core/workflows/ or agents/. automated-ui-verification.md is nested under verify-work/steps/, so it was never in the tracked set — only execute-phase.md was ever reported, both before and after the launcher preamble landed. Only ack what the check actually reports. Verified on the remote runner. Refs #2856 * chore(#2856): backfill changeset pr number pr:0 -> 3716. The placeholder fails both changeset-lint (fail_invalid_fragment) and docs-lint (fail_malformed_fragment) by design and can only be resolved once the PR number exists. Both now report ok against GITHUB_BASE_REF=next. Refs #2856 --------- Co-authored-by: sim <sim@local>
3.7 KiB
This step is ADDITIVE. It never halts the wave, never fails the phase, and never rewrites SUMMARY.md. If you cannot look, say so and finish.
<required_reading>
- {phase_dir}/{phase_num}-PLAN.md (the wave's tasks and their acceptance criteria)
- {phase_dir}/{phase_num}-UI-SPEC.md if it exists (the design contract, when the phase has one) </required_reading>
<browser_surface>
You carry exactly two browser MCP families: mcp__chrome-devtools__* and
mcp__claude-in-chrome__*. Use whichever responds. Do not assume they expose the same
tool names — probe, then use what is there. Do not paper over differences between them.
You do NOT carry the Playwright MCP family. That path belongs to the orchestrator's own verification step and is not yours. </browser_surface>
<profile_lock>
chrome-devtools-mcp holds an exclusive lock on its browser profile
($HOME/.cache/chrome-devtools-mcp/chrome-profile). A second concurrent instance fails with:
The browser is already running for <dir>. Use --isolated to run multiple browser instances.
If you see that, or any equivalent lock error:
- Record
outcome: could_not_lookandreason: profile_locked. - Name
--isolatedin the notes, so the operator knows the remedy is a flag on THEIR MCP server registration. - Stop. Do not retry, do not loop, do not wait for the lock. GSD cannot pass
--isolated— it is not GSD's flag — and a retry loop here just holds up the wave.
Parallel execution waves sharing one profile WILL hit this. It is an expected condition, not a defect, and it is not a reason to fail anything. </profile_lock>
For each UI acceptance criterion you can identify in the wave's plan:- Resolve its target URL. If no dev server or target is reachable, that criterion is
could_not_look/target_unreachable— not a failure. - Open it with the browser family that responded.
- Observe the DOM for the specific, stated condition. Assert on structure and content — an element's presence, its text, its attributes, its computed state.
- Record
passedwhen the stated condition is observably true,needs_reviewwhen it is ambiguous or requires human judgement (subjective aesthetics, content accuracy).
Scope limit for this version: DOM observation against stated criteria only. No screenshot
diffing, no accessibility audit, no performance tracing. If a criterion needs one of those,
mark it needs_review and say which.
Never invent a criterion. If the plan states no UI acceptance criteria, that is
outcome: nothing_to_report / reason: no_criteria, and it is a perfectly good result.
Frontmatter carries scalars only, so a reader can get the verdict without parsing prose:
---
schema_version: 1
wave: {wave_number}
outcome: verified | nothing_to_report | could_not_look
reason: ok | no_criteria | no_browser_mcp | profile_locked | target_unreachable
checked: <integer>
passed: <integer>
needs_review: <integer>
---
Then a short body: one line per criterion with its verdict, and — when outcome is
could_not_look — exactly what stopped you and what the operator would change.
nothing_to_report and could_not_look are different outcomes and must never be
conflated. "There were no UI criteria in this wave" and "there were criteria but I had no
browser" look identical in a summary that collapses them, and that ambiguity is the reported
problem this capability exists to remove.