Files

0xdhx 88b5775dc8 enhance(#4223 ): default-off interaction capture for gsd-ui-auditor via the chrome-devtools CLI (#4477 )

* enhance(#4223): default-off interaction capture for gsd-ui-auditor via the chrome-devtools CLI

gsd-ui-auditor is chartered to audit interaction and handed a capture
driver with no interaction verb: `npx playwright screenshot` cannot
click, fill, hover, press or snapshot, so a hover state, an open menu,
a focus ring or a form's validation state never appears in its
evidence and every Experience Design finding degrades to code reading.

Implements the shape approved at triage, not a new capability:

- capabilities/ui/capability.json declares `workflow.ui_interaction_capture`
  (boolean, default false) on the capability that already owns the
  auditor (ADR-894 one-owner invariant); capability-registry.cjs regenerated.
- gsd-core/workflows/ui-review.md reads the key through gsd_run and hands
  it to the auditor as `interaction_capture:` in the spawn <config> block —
  the auditor carries no gsd_run resolver, so the key travels by value.
- agents/gsd-ui-auditor.md gains an anchored interaction-capture section
  AFTER the static block. With the key on and a Chrome binary resolved it
  starts the `chrome-devtools` CLI (chrome-devtools-mcp, floor ^1.8.0) on
  an --isolated profile, opens the dev URL the static block reached,
  takes the a11y snapshot for element uids, captures the baseline and a
  Tab focus-ring state, drives the UI-SPEC's interactive components, saves
  console output, and stops the daemon unconditionally. Key off, no dev
  server, or no Chrome: one status line, and the Playwright-only static
  path runs exactly as before — the static fence is untouched.

Needs only Bash: no MCP server, no tools: change. Chromium-only by
nature; Firefox/WebKit stay on Playwright. `wait_for` is MCP-only, so
readiness is polled through evaluate_script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* test(#4223): bind the interaction-capture shape and containment

- manifest, generated registry, config schema and config-set/loadConfig
  all know workflow.ui_interaction_capture as a default-off boolean, and
  hand-written non-booleans fall to the slice default
- the orchestrator reads the key and hands it down; the auditor never
  grows a gsd_run dependency
- the static fence stays Playwright-only and the interaction fence
  chrome-devtools-only, so key-off is today's path
- the interaction fence runs under bash with a stub driver on PATH: key
  off / absent / no dev server / no Chrome invoke nothing; the happy path
  starts first and stops last on the [selected] pageId with the documented
  flags; a failed capture is removed and not counted; new_page and start
  failures still honour the stop-only-if-started rule; CHROME_BIN and
  CHROME_DEVTOOLS_MCP_VERSION overrides flow through
- docs/CONFIGURATION.md row shape; registered in the docs-guard lane

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* docs(#4223): document workflow.ui_interaction_capture and its how-to

- docs/CONFIGURATION.md: one row in the workflow.* table, default-off
- docs/AGENTS.md: the gsd-ui-auditor entry names the key and what the
  interaction-capture section adds, skips and never claims
- docs/how-to/enable-ui-interaction-capture.md: turn it on, read the
  `**Interaction captures:**` outcomes, what it does not do, turn it off
- docs/README.md: index the how-to beside live-DOM verification

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* chore(#4223): add changeset

Added-type fragment; pr: carries the issue number until the PR exists.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* enhance(#4223): use the /gsd:ui-review namespace form in the auditor's prose

Claude-facing source (agents/, workflows/) uses the /gsd:<cmd> namespace;
the hyphen form is retired there and the slash-command-namespace guard
rejects it. docs/ keep the hyphen form by convention.

Emitted-Drift-Ack-Growth: gsd-ui-auditor.md — #4223: the anchored default-off interaction-capture section (prose + one bash fence) appended after the static Playwright block inside <screenshot_approach>, plus one `**Interaction captures:**` line in each of the two report templates, one completion-checklist line and one Step-3 sentence. The static fence is byte-identical to next; nothing was removed or reordered.
Emitted-Drift-Ack-Growth: ui-review.md — #4223: a two-line config-get read + true/false normalisation in step 0 and one `interaction_capture:` line in the spawn <config> block with a three-line note on why the value travels by prompt. No step, gate, or dispatch shape changed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* enhance(#4223): per-run daemon session, bounded navigation, and step failures that count

Three findings from the pre-file adversarial review of the interaction
fence, folded in:

- `--sessionId <epoch>-<pid>` on every driver call. `start` restarts
  whatever daemon shares its session and --isolated isolates only the
  browser profile, so two concurrent audits — or an audit beside the
  operator's own CLI daemon — would otherwise stop each other. The CLI
  accepts hex and dashes only; the id is validated by the test stub.
- `new_page --timeout 30000`: the one verb that takes a bound, placed
  before every verb that does not, so a hung page is caught first.
- a failed take_snapshot or press_key now increments the failure count
  and is named on stdout; two clean screenshots can no longer read as
  `0 failed` after the step that gives the interactions their uids failed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* enhance(#4223): subshell-unique session id, CRLF-safe page-id parse, stale-snapshot removal

Second review round, both reviewers:

- session id is `<epoch>-<BASHPID>-<RANDOM>`: `$$` is inherited by a
  subshell, so two audits forked from one parent in the same second
  shared an id and could stop each other's daemon (driven by the reviewer)
- `tr -d '\r'` before the `[selected]` parse so a CRLF-emitting driver
  under Git Bash still matches the `$` anchor, and `|| true` on the
  assignment so a failed new_page cannot abort the block under
  `set -e -o pipefail` before the unconditional stop
- a failed take_snapshot removes any snapshot.txt it left or inherited
  from a reused directory, so stale uids never drive the interactions
- `<config>` placeholder is `{interaction_capture}`, lowercase like its
  `{phase_dir}` / `{padded_phase}` siblings — the block is a prompt
  template, not a bash heredoc
- how-to: the `not captured` row no longer claims the daemon started

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* enhance(#4223): check new_page's exit status before parsing its output; regression cases for the edges

Third review round:

- a new_page that prints a page line and then exits non-zero is a failed
  navigation, not a page id: the exit status is checked in an `if` before
  the output is parsed (driven by the reviewer against the previous
  `|| true`, which masked exactly that)
- regression cases for what the last two rounds added: CRLF driver
  output, a stale snapshot removed on failure, partial-output new_page
  failure, and the whole fence under `set -e -o pipefail` (both the
  failed-navigation path and the happy path)
- the harness whitelist gains `date`; the session-id assertion now
  requires all three parts, so a silently empty epoch cannot hide again

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* enhance(#4223): keep gsd-ui-auditor under the DEFAULT-tier size cap; changeset pr placeholder

- the three review folds pushed agents/gsd-ui-auditor.md to 25179 bytes,
  over the 24576-byte hard cap tests/agent-size-budget.test.cjs enforces;
  the interaction section's comments are tightened to the same content
  in fewer bytes (23559 now). No bash changed — the fence's own tests and
  the real-browser run are unchanged.
- .changeset/vivid-yaks-fly.md carries the policy placeholder `pr: 0`, which
  the post-create backfill rewrites to the PR's own number.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* chore(#4223): set changeset fragment pr to 4477

* test(#4223): compare the fence's status path with the separator the fence uses

On the windows-latest lane the happy-path case failed on `\interaction` vs
`/interaction` alone: the fence joins "$SCREENSHOT_DIR/interaction" with a
literal slash, and the assertion built its expectation with path.join. Every
other case in the file passed on that lane, including the CRLF and
errexit/pipefail ones.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6

* test(#4223): drop the inert file-header allow-test-rule marker

Review round 1 on #4477: the `source-text-is-the-product` marker sat at
line 2, outside no-source-grep's 8-line lookahead of every readFileSync
site (the first is ~60 lines down), so it suppressed nothing. It was also
unnecessary: every read in this file is a .md/.json path, which the rule
does not trigger on. Deleted rather than relocated — there is no site to
relocate it to. Negative control: `eslint` on the file is clean without it.

* chore(#4223): regenerate the platform-conformance-tier lists for the new test

Review round 3 on #4477. `next` gained chore(#4591)'s platform-conformance-tier
gate after this branch opened; its two committed lists must name every file
under tests/, and this PR's tests/ui-interaction-capture.test.cjs had never
been in them. Once the branch was updated against next the lists were stale
and three jobs went red on head 575667dd: lint-tests (gen-platform-conformance-tier
--check), conformance test (macos-latest) at 546 !== 547, and shard 1/3's
fragment-single-edit-propagation, which sees the same staleness as regen:derived
touching files beyond the fragment edit under test.

Regenerated with the repo's own generators, no hand-editing. The general tier
goes 546 -> 547 and the macOS tier 196 -> 197, each by exactly this one entry;
both --check arms are clean. Verified the red is this PR's own file and not
base drift: at upstream/next both generators report "list matches" (546 / 196),
and our committed copies were byte-identical to next's before this commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015CBQTeGX1JYHF5DRWp4wvZ

* fix(#4223): bound, confine and trap the chrome-devtools driver fence

Round 4 — three findings in one fence, interleaved on the same lines, so one
commit:

- Every driver call is time-bounded. `cdt <ceiling> <verb>` runs the client as a
  background job in its own process group (`set -m`) under a watchdog that kills
  the whole group at the ceiling — TERM, then KILL two seconds later. One pid is
  not enough: npm forwards SIGTERM only to its direct child, so killing `npx`
  alone leaves the client holding the fence's stdout and a `$(cdt … new_page)`
  capture blocked past the ceiling (driven against a real npx tree by the
  round's adversarial review; the pid-only first cut of this commit had exactly
  that hole). The watchdog is an exec'd bash (`"$BASH" -c`), never a `( … )`
  subshell: a subshell inherits bash's saved copies of the caller's stdio (the
  fds ≥10 a function-level `>/dev/null` redirect leaves behind) and holds them
  open, so a runner waiting for EOF waited out the whole 60 s ceiling whenever a
  watchdog outlived its kill — measured as the intermittent 30 s test run the
  review flagged; 0/60 after. It polls the job's process GROUP (`kill -0 --
  -pgid`, every 0.1 s) and stands down by itself once the group is empty;
  nothing ever signals it. The group, not the leader pid: a child can outlive
  the leader while holding the `$(cdt … new_page)` pipe, and a leader-pid poll
  stood down at once and left the substitution open-ended (driven by the
  round's adversarial review at 6× the ceiling; a pgid cannot be reused while
  any member lives, which a bare pid can). The daemon `start` launches is
  spawned detached (its own session) and never in that group. Two platforms
  forced the never-signalled shape. Under bash 3.2.57 the earlier `trap … TERM;
  sleep & wait $!` form ignored its TERM in 3 of 300 fast calls and slept out
  the whole ceiling — CI's macos job hanging 30 s right after `start`. On Git
  Bash a signal to a watchdog still starting up hung the fence's `wait` for it:
  18 of 20 fence tests at the harness's 30 s cap in 3 of 3 full-file runs,
  while a fence slowed by xtrace, or three tests run alone, never hit it (a
  startup race; the mechanism is not pinned further). Polling: 0/300 slow calls
  and 0 orphaned sleeps under 3.2.57 and 5.2, the fence suite 20/20 in 3 of 3
  full-file runs on Git Bash 5.2.37 (fractional `sleep 0.1`: driven on GNU,
  msys and busybox sleep; BSD sleep documents it). A clock that cannot launch
  (`sleep … || exit 0`) stands the watchdog down rather than firing at once
  and killing a healthy call — by design that leaves a hung call unbounded,
  the pre-round-4 behaviour, instead of failing a healthy one. A hung call
  returns once its group is gone: at the ceiling, plus up to the 2 s
  TERM-to-KILL grace. The KILL after the grace is sent only to a group that
  is still alive: a pgid freed during the grace can be reused, and an
  unconditional KILL could hit an unrelated group (the round's review).
  `start` (npx fetch + Chrome launch) gets CHROME_DEVTOOLS_START_TIMEOUT
  (180 s), every verb CHROME_DEVTOOLS_STEP_TIMEOUT (60 s). timeout(1) is absent
  on macOS and this agent carries no gsd-tools resolver, hence a bash watchdog
  rather than either.
- --allowUnrestrictedPaths -> --workspace "$INTERACTION_DIR": the driver may
  write under the run's interaction/ directory and nowhere else. Relative, like
  every --filePath (unchanged from rounds 1-3): the daemon resolves both against
  one cwd (chrome-devtools-mcp 1.9.0 spawns it with cwd: process.cwd() and
  path.resolve()s both), and a relative path needs no dialect translation — an
  absolute `pwd -P` path is an msys path on Git Bash, which a Windows-native
  daemon cannot resolve (CI's windows conformance shard caught the first cut).
  --workspace is a 1.9.0 flag (absent from 1.8.0's `start --help`, verified),
  so the documented floor moves from ^1.8.0 to ^1.9.0, where
  --allowUnrestrictedPaths is deprecated.
- `stop` is owed by an EXIT trap after a successful `start`, not by position
  (it replaces any earlier EXIT trap — none exists in this file); the explicit
  call keeps it in order, a flag makes the trap a no-op afterwards, and only the
  shell that installed the trap may act: a subshell copy of the fence state
  carries CDT_STARTED=1 and, under a timing race CI's ubuntu job hit (reproduced
  locally at 3/40 under load: the second `stop` came from a subshell pid, never
  main), issued a second `stop`. The identity is `$(exec /bin/sh -c 'echo
  "$PPID"')`, not $BASHPID — macOS ships bash 3.2, where BASHPID does not exist
  and CI's macos conformance job showed the guard comparing empty to empty. The
  fence was driven under bash 3.2.57 for the injected-subshell, errexit
  failed-new_page, errexit failed-resize, hung-start, hung-new_page and happy
  paths. A failed resize_page is a counted failed step now, not the one bare
  command an errexit runner could abort on.

Prose in the section is tightened to pay for the mechanism: 23559 -> 24517
bytes against the 24576 DEFAULT-tier cap.

Tests: the stub driver hangs as a real child tree (sh waiting on a child that
holds stdout — never an exec), so a pid-only kill fails the new
aHungNewPageWhoseChildHoldsStdoutIsStillCutOffAtTheCeiling test (negative-
controlled: it blocks for the harness's whole cap on the old wrapper). A hung
start and a hung capture are cut off within ceiling + grace + slack and still
reach stop; an injected bare failure under errexit reaches stop through the
trap, exactly once; an injected subshell call of cdt_stop issues nothing; the
happy path issues exactly one stop; every driver call site names a ceiling and
the only bare $CDT is the wrapper's own spawn; the start line carries
--workspace with the capture directory, every --filePath lies under it, and no
code line carries --allowUnrestrictedPaths. A driver whose leader exits at
once while a child keeps holding the capture pipe is still cut off at the
ceiling (negative-controlled: a leader-pid poll blocks for the harness's whole
cap). A watchdog whose clock cannot launch leaves a 300 ms driver call alone
(negative-controlled: the trap form kills `start` in under 20 ms). The harness
EXPORTS its stub-only PATH — unexported, the exec'd
watchdog fell through to bash's compiled-in default PATH and never saw the stub
dir — and ships `sleep` there as an exec-wrapper script (portable to Git Bash,
pid-preserving).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LgNMb2G67rAJfFQRHEBTAj

* fix(#4223): gitignore gate covers the capture directory, and upgrades an existing file

Round 4 Blocker. The gate enumerated image extensions, so snapshot.txt (the
accessibility tree, with entered form values) and console.txt (which can carry
tokens) were committable by `git add .`. The gate now ignores `interaction/` as
a directory — the next artifact type is covered by construction — and it appends
whatever an existing .gitignore lacks instead of writing once. The write-once
form was the same defect one step later: every project that had already run an
audit would never have received the new pattern at all.

Tests run the gate fence under bash: a fresh file carries every pattern; an
image-only file from an earlier audit gains interaction/ and keeps its own
header without duplicating present lines; a second run appends nothing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LgNMb2G67rAJfFQRHEBTAj

* test(#4223): declare the interaction-capture anchor as a comment marker

The #4324 colon-token gate (slash-command-namespace) landed on next after this
branch was opened and reads `<!-- gsd:ui-interaction-capture -->` as an
unconvertible /gsd: command token. It is a section anchor of the same family as
gsd:live-dom-families and gsd:write-continue, so it is declared in
COMMENT_MARKER_TOKENS rather than renamed. Found by running the base-added
gates against the merged tree; CI at ca8d2508 predates the gate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LgNMb2G67rAJfFQRHEBTAj

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Tom Boucher <trekkie@nomorestars.com>
Co-authored-by: CI Rebase Check <ci@gsd-redux>

2026-09-20 05:45:15 -04:00

5.5 KiB

Raw Blame History

How to enable UI interaction capture

Let /gsd-ui-review's auditor capture what a page looks like after an interaction — a hover state, a focus ring, an open menu, a filled form's validation state — instead of only the first paint, without configuring an MCP server or widening the agent's tool surface.

Default-off, and deliberately so. The static capture path is unchanged in every configuration; this key adds captures on top of it. You opt in per project with one key. The gap it closes is #4223: the auditor is chartered to audit interaction, and npx playwright screenshot has no interaction verb.

What you need:

An installed Chrome or Chromium. The driver launches the system browser (Puppeteer channel: 'chrome'); it does not download one. Discovery tries google-chrome, google-chrome-stable, chromium, chromium-browser and chrome on PATH, then the standard macOS and Windows install paths. CHROME_BIN=/path/to/chrome overrides it.
npx able to fetch chrome-devtools-mcp (the package that ships the chrome-devtools CLI). It is resolved at the documented floor ^1.9.0 (the release that added --workspace, which confines the driver's file writes to the capture directory); CHROME_DEVTOOLS_MCP_VERSION overrides it.
Nothing else. Every driver call runs under a ceiling — CHROME_DEVTOOLS_START_TIMEOUT (default 180 s, for the npx fetch plus the Chrome launch) and CHROME_DEVTOOLS_STEP_TIMEOUT (default 60 s, per capture verb). At the ceiling the client's whole process group is killed (TERM, then KILL two seconds later) and the step is counted as failed, so a cold npm cache or a Chrome that never comes up costs a failed step rather than an open-ended wait. The one exception is a watchdog whose own clock (sleep) cannot launch: it stands down instead of killing a healthy call, and that call is then unbounded, as every call was before the ceilings existed.
A dev server the static capture already reaches — interaction capture runs against the same URL and skips itself when the static block reached nothing.

Step 1 — Turn the key on

gsd-tools query config-set workflow.ui_interaction_capture true

Verify it took:

gsd-tools query config-get workflow.ui_interaction_capture
# → true

/gsd-ui-review reads the key and passes interaction_capture: true in the auditor's <config> block — the auditor itself never reads config.

Step 2 — Run a review and read the report

/gsd-ui-review 3

The audit's static captures land where they always did. With the key on and a Chrome resolved, an interaction/ directory beside them holds baseline.png, focus-first.png (focus ring on the first focusable element), one capture per interaction the auditor drove from your UI-SPEC's interactive components, the accessibility snapshot it used for element ids, and the page's console output. The audit's .gitignore gate covers that interaction/ directory as a whole — the snapshot carries whatever was typed into forms and the console output can carry tokens — so a git add . never commits it, and a project whose .gitignore predates the directory is upgraded on the next audit. UI-REVIEW.md carries the outcome on its own line:

**Interaction captures:** captured (4 state(s), 0 failed) in .planning/ui-reviews/03-.../interaction

The other values it can hold are honest, not decorative:

`Interaction captures:`	Meaning
`off`	The key is `false`. Nothing else changed.
`skipped (no dev server reached)`	The static capture found no dev server; there was nothing to interact with.
`skipped (no Chrome binary resolved)`	The key is on but no browser resolved. Set `CHROME_BIN`.
`not captured (driver or capture failure)`	The key was on and Chrome resolved, but no state landed on disk — `npx` could not fetch the driver, Chrome did not launch, the page never opened, or every capture failed. The audit output names the failing step.

An interaction state that does not appear in that directory is not reported as observed; the Experience Design pillar says when its findings are code-derived.

What this does not do

It does not replace the static captures. npx playwright screenshot stays the driver for the three viewport shots; it has --wait-for-selector, --device, --color-scheme and cross-engine -b firefox|webkit, none of which the CLI driver offers. Firefox and WebKit needs stay on Playwright.
It does not touch gsd-dom-verifier or workflow.live_dom_uat. That capability drives a browser MCP server at execute time; this one drives a CLI from Bash at review time. They are independent keys.
It does not share the chrome-devtools-mcp browser profile. The auditor starts the daemon with --isolated, so it never contends for the profile lock a registered MCP server holds, and stops it when the capture ends — under an EXIT trap, so an aborted audit still stops it.
It does not let the driver write outside the capture directory. The daemon starts with --workspace set to the run's interaction/ directory; that is the only place its file-writing verbs may land.
It does not wait on selectors. wait_for is MCP-only; the auditor polls document.readyState through evaluate_script where a state needs settling.

Turning it back off

gsd-tools query config-set workflow.ui_interaction_capture false

The next review runs the Playwright-only path and reports **Interaction captures:** off.

5.5 KiB Raw Blame History