enhance(#4223): default-off interaction capture for gsd-ui-auditor via the chrome-devtools CLI (#4477)
* enhance(#4223): default-off interaction capture for gsd-ui-auditor via the chrome-devtools CLI gsd-ui-auditor is chartered to audit interaction and handed a capture driver with no interaction verb: `npx playwright screenshot` cannot click, fill, hover, press or snapshot, so a hover state, an open menu, a focus ring or a form's validation state never appears in its evidence and every Experience Design finding degrades to code reading. Implements the shape approved at triage, not a new capability: - capabilities/ui/capability.json declares `workflow.ui_interaction_capture` (boolean, default false) on the capability that already owns the auditor (ADR-894 one-owner invariant); capability-registry.cjs regenerated. - gsd-core/workflows/ui-review.md reads the key through gsd_run and hands it to the auditor as `interaction_capture:` in the spawn <config> block — the auditor carries no gsd_run resolver, so the key travels by value. - agents/gsd-ui-auditor.md gains an anchored interaction-capture section AFTER the static block. With the key on and a Chrome binary resolved it starts the `chrome-devtools` CLI (chrome-devtools-mcp, floor ^1.8.0) on an --isolated profile, opens the dev URL the static block reached, takes the a11y snapshot for element uids, captures the baseline and a Tab focus-ring state, drives the UI-SPEC's interactive components, saves console output, and stops the daemon unconditionally. Key off, no dev server, or no Chrome: one status line, and the Playwright-only static path runs exactly as before — the static fence is untouched. Needs only Bash: no MCP server, no tools: change. Chromium-only by nature; Firefox/WebKit stay on Playwright. `wait_for` is MCP-only, so readiness is polled through evaluate_script. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * test(#4223): bind the interaction-capture shape and containment - manifest, generated registry, config schema and config-set/loadConfig all know workflow.ui_interaction_capture as a default-off boolean, and hand-written non-booleans fall to the slice default - the orchestrator reads the key and hands it down; the auditor never grows a gsd_run dependency - the static fence stays Playwright-only and the interaction fence chrome-devtools-only, so key-off is today's path - the interaction fence runs under bash with a stub driver on PATH: key off / absent / no dev server / no Chrome invoke nothing; the happy path starts first and stops last on the [selected] pageId with the documented flags; a failed capture is removed and not counted; new_page and start failures still honour the stop-only-if-started rule; CHROME_BIN and CHROME_DEVTOOLS_MCP_VERSION overrides flow through - docs/CONFIGURATION.md row shape; registered in the docs-guard lane Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * docs(#4223): document workflow.ui_interaction_capture and its how-to - docs/CONFIGURATION.md: one row in the workflow.* table, default-off - docs/AGENTS.md: the gsd-ui-auditor entry names the key and what the interaction-capture section adds, skips and never claims - docs/how-to/enable-ui-interaction-capture.md: turn it on, read the `**Interaction captures:**` outcomes, what it does not do, turn it off - docs/README.md: index the how-to beside live-DOM verification Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * chore(#4223): add changeset Added-type fragment; pr: carries the issue number until the PR exists. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * enhance(#4223): use the /gsd:ui-review namespace form in the auditor's prose Claude-facing source (agents/, workflows/) uses the /gsd:<cmd> namespace; the hyphen form is retired there and the slash-command-namespace guard rejects it. docs/ keep the hyphen form by convention. Emitted-Drift-Ack-Growth: gsd-ui-auditor.md — #4223: the anchored default-off interaction-capture section (prose + one bash fence) appended after the static Playwright block inside <screenshot_approach>, plus one `**Interaction captures:**` line in each of the two report templates, one completion-checklist line and one Step-3 sentence. The static fence is byte-identical to next; nothing was removed or reordered. Emitted-Drift-Ack-Growth: ui-review.md — #4223: a two-line config-get read + true/false normalisation in step 0 and one `interaction_capture:` line in the spawn <config> block with a three-line note on why the value travels by prompt. No step, gate, or dispatch shape changed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * enhance(#4223): per-run daemon session, bounded navigation, and step failures that count Three findings from the pre-file adversarial review of the interaction fence, folded in: - `--sessionId <epoch>-<pid>` on every driver call. `start` restarts whatever daemon shares its session and --isolated isolates only the browser profile, so two concurrent audits — or an audit beside the operator's own CLI daemon — would otherwise stop each other. The CLI accepts hex and dashes only; the id is validated by the test stub. - `new_page --timeout 30000`: the one verb that takes a bound, placed before every verb that does not, so a hung page is caught first. - a failed take_snapshot or press_key now increments the failure count and is named on stdout; two clean screenshots can no longer read as `0 failed` after the step that gives the interactions their uids failed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * enhance(#4223): subshell-unique session id, CRLF-safe page-id parse, stale-snapshot removal Second review round, both reviewers: - session id is `<epoch>-<BASHPID>-<RANDOM>`: `$$` is inherited by a subshell, so two audits forked from one parent in the same second shared an id and could stop each other's daemon (driven by the reviewer) - `tr -d '\r'` before the `[selected]` parse so a CRLF-emitting driver under Git Bash still matches the `$` anchor, and `|| true` on the assignment so a failed new_page cannot abort the block under `set -e -o pipefail` before the unconditional stop - a failed take_snapshot removes any snapshot.txt it left or inherited from a reused directory, so stale uids never drive the interactions - `<config>` placeholder is `{interaction_capture}`, lowercase like its `{phase_dir}` / `{padded_phase}` siblings — the block is a prompt template, not a bash heredoc - how-to: the `not captured` row no longer claims the daemon started Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * enhance(#4223): check new_page's exit status before parsing its output; regression cases for the edges Third review round: - a new_page that prints a page line and then exits non-zero is a failed navigation, not a page id: the exit status is checked in an `if` before the output is parsed (driven by the reviewer against the previous `|| true`, which masked exactly that) - regression cases for what the last two rounds added: CRLF driver output, a stale snapshot removed on failure, partial-output new_page failure, and the whole fence under `set -e -o pipefail` (both the failed-navigation path and the happy path) - the harness whitelist gains `date`; the session-id assertion now requires all three parts, so a silently empty epoch cannot hide again Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * enhance(#4223): keep gsd-ui-auditor under the DEFAULT-tier size cap; changeset pr placeholder - the three review folds pushed agents/gsd-ui-auditor.md to 25179 bytes, over the 24576-byte hard cap tests/agent-size-budget.test.cjs enforces; the interaction section's comments are tightened to the same content in fewer bytes (23559 now). No bash changed — the fence's own tests and the real-browser run are unchanged. - .changeset/vivid-yaks-fly.md carries the policy placeholder `pr: 0`, which the post-create backfill rewrites to the PR's own number. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * chore(#4223): set changeset fragment pr to 4477 * test(#4223): compare the fence's status path with the separator the fence uses On the windows-latest lane the happy-path case failed on `\interaction` vs `/interaction` alone: the fence joins "$SCREENSHOT_DIR/interaction" with a literal slash, and the assertion built its expectation with path.join. Every other case in the file passed on that lane, including the CRLF and errexit/pipefail ones. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PanfY8KaLb4RVVubcUoGP6 * test(#4223): drop the inert file-header allow-test-rule marker Review round 1 on #4477: the `source-text-is-the-product` marker sat at line 2, outside no-source-grep's 8-line lookahead of every readFileSync site (the first is ~60 lines down), so it suppressed nothing. It was also unnecessary: every read in this file is a .md/.json path, which the rule does not trigger on. Deleted rather than relocated — there is no site to relocate it to. Negative control: `eslint` on the file is clean without it. * chore(#4223): regenerate the platform-conformance-tier lists for the new test Review round 3 on #4477. `next` gained chore(#4591)'s platform-conformance-tier gate after this branch opened; its two committed lists must name every file under tests/, and this PR's tests/ui-interaction-capture.test.cjs had never been in them. Once the branch was updated against next the lists were stale and three jobs went red on head 575667dd: lint-tests (gen-platform-conformance-tier --check), conformance test (macos-latest) at 546 !== 547, and shard 1/3's fragment-single-edit-propagation, which sees the same staleness as regen:derived touching files beyond the fragment edit under test. Regenerated with the repo's own generators, no hand-editing. The general tier goes 546 -> 547 and the macOS tier 196 -> 197, each by exactly this one entry; both --check arms are clean. Verified the red is this PR's own file and not base drift: at upstream/next both generators report "list matches" (546 / 196), and our committed copies were byte-identical to next's before this commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015CBQTeGX1JYHF5DRWp4wvZ * fix(#4223): bound, confine and trap the chrome-devtools driver fence Round 4 — three findings in one fence, interleaved on the same lines, so one commit: - Every driver call is time-bounded. `cdt <ceiling> <verb>` runs the client as a background job in its own process group (`set -m`) under a watchdog that kills the whole group at the ceiling — TERM, then KILL two seconds later. One pid is not enough: npm forwards SIGTERM only to its direct child, so killing `npx` alone leaves the client holding the fence's stdout and a `$(cdt … new_page)` capture blocked past the ceiling (driven against a real npx tree by the round's adversarial review; the pid-only first cut of this commit had exactly that hole). The watchdog is an exec'd bash (`"$BASH" -c`), never a `( … )` subshell: a subshell inherits bash's saved copies of the caller's stdio (the fds ≥10 a function-level `>/dev/null` redirect leaves behind) and holds them open, so a runner waiting for EOF waited out the whole 60 s ceiling whenever a watchdog outlived its kill — measured as the intermittent 30 s test run the review flagged; 0/60 after. It polls the job's process GROUP (`kill -0 -- -pgid`, every 0.1 s) and stands down by itself once the group is empty; nothing ever signals it. The group, not the leader pid: a child can outlive the leader while holding the `$(cdt … new_page)` pipe, and a leader-pid poll stood down at once and left the substitution open-ended (driven by the round's adversarial review at 6× the ceiling; a pgid cannot be reused while any member lives, which a bare pid can). The daemon `start` launches is spawned detached (its own session) and never in that group. Two platforms forced the never-signalled shape. Under bash 3.2.57 the earlier `trap … TERM; sleep & wait $!` form ignored its TERM in 3 of 300 fast calls and slept out the whole ceiling — CI's macos job hanging 30 s right after `start`. On Git Bash a signal to a watchdog still starting up hung the fence's `wait` for it: 18 of 20 fence tests at the harness's 30 s cap in 3 of 3 full-file runs, while a fence slowed by xtrace, or three tests run alone, never hit it (a startup race; the mechanism is not pinned further). Polling: 0/300 slow calls and 0 orphaned sleeps under 3.2.57 and 5.2, the fence suite 20/20 in 3 of 3 full-file runs on Git Bash 5.2.37 (fractional `sleep 0.1`: driven on GNU, msys and busybox sleep; BSD sleep documents it). A clock that cannot launch (`sleep … || exit 0`) stands the watchdog down rather than firing at once and killing a healthy call — by design that leaves a hung call unbounded, the pre-round-4 behaviour, instead of failing a healthy one. A hung call returns once its group is gone: at the ceiling, plus up to the 2 s TERM-to-KILL grace. The KILL after the grace is sent only to a group that is still alive: a pgid freed during the grace can be reused, and an unconditional KILL could hit an unrelated group (the round's review). `start` (npx fetch + Chrome launch) gets CHROME_DEVTOOLS_START_TIMEOUT (180 s), every verb CHROME_DEVTOOLS_STEP_TIMEOUT (60 s). timeout(1) is absent on macOS and this agent carries no gsd-tools resolver, hence a bash watchdog rather than either. - --allowUnrestrictedPaths -> --workspace "$INTERACTION_DIR": the driver may write under the run's interaction/ directory and nowhere else. Relative, like every --filePath (unchanged from rounds 1-3): the daemon resolves both against one cwd (chrome-devtools-mcp 1.9.0 spawns it with cwd: process.cwd() and path.resolve()s both), and a relative path needs no dialect translation — an absolute `pwd -P` path is an msys path on Git Bash, which a Windows-native daemon cannot resolve (CI's windows conformance shard caught the first cut). --workspace is a 1.9.0 flag (absent from 1.8.0's `start --help`, verified), so the documented floor moves from ^1.8.0 to ^1.9.0, where --allowUnrestrictedPaths is deprecated. - `stop` is owed by an EXIT trap after a successful `start`, not by position (it replaces any earlier EXIT trap — none exists in this file); the explicit call keeps it in order, a flag makes the trap a no-op afterwards, and only the shell that installed the trap may act: a subshell copy of the fence state carries CDT_STARTED=1 and, under a timing race CI's ubuntu job hit (reproduced locally at 3/40 under load: the second `stop` came from a subshell pid, never main), issued a second `stop`. The identity is `$(exec /bin/sh -c 'echo "$PPID"')`, not $BASHPID — macOS ships bash 3.2, where BASHPID does not exist and CI's macos conformance job showed the guard comparing empty to empty. The fence was driven under bash 3.2.57 for the injected-subshell, errexit failed-new_page, errexit failed-resize, hung-start, hung-new_page and happy paths. A failed resize_page is a counted failed step now, not the one bare command an errexit runner could abort on. Prose in the section is tightened to pay for the mechanism: 23559 -> 24517 bytes against the 24576 DEFAULT-tier cap. Tests: the stub driver hangs as a real child tree (sh waiting on a child that holds stdout — never an exec), so a pid-only kill fails the new aHungNewPageWhoseChildHoldsStdoutIsStillCutOffAtTheCeiling test (negative- controlled: it blocks for the harness's whole cap on the old wrapper). A hung start and a hung capture are cut off within ceiling + grace + slack and still reach stop; an injected bare failure under errexit reaches stop through the trap, exactly once; an injected subshell call of cdt_stop issues nothing; the happy path issues exactly one stop; every driver call site names a ceiling and the only bare $CDT is the wrapper's own spawn; the start line carries --workspace with the capture directory, every --filePath lies under it, and no code line carries --allowUnrestrictedPaths. A driver whose leader exits at once while a child keeps holding the capture pipe is still cut off at the ceiling (negative-controlled: a leader-pid poll blocks for the harness's whole cap). A watchdog whose clock cannot launch leaves a 300 ms driver call alone (negative-controlled: the trap form kills `start` in under 20 ms). The harness EXPORTS its stub-only PATH — unexported, the exec'd watchdog fell through to bash's compiled-in default PATH and never saw the stub dir — and ships `sleep` there as an exec-wrapper script (portable to Git Bash, pid-preserving). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LgNMb2G67rAJfFQRHEBTAj * fix(#4223): gitignore gate covers the capture directory, and upgrades an existing file Round 4 Blocker. The gate enumerated image extensions, so snapshot.txt (the accessibility tree, with entered form values) and console.txt (which can carry tokens) were committable by `git add .`. The gate now ignores `interaction/` as a directory — the next artifact type is covered by construction — and it appends whatever an existing .gitignore lacks instead of writing once. The write-once form was the same defect one step later: every project that had already run an audit would never have received the new pattern at all. Tests run the gate fence under bash: a fresh file carries every pattern; an image-only file from an earlier audit gains interaction/ and keeps its own header without duplicating present lines; a second run appends nothing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LgNMb2G67rAJfFQRHEBTAj * test(#4223): declare the interaction-capture anchor as a comment marker The #4324 colon-token gate (slash-command-namespace) landed on next after this branch was opened and reads `<!-- gsd:ui-interaction-capture -->` as an unconvertible /gsd: command token. It is a section anchor of the same family as gsd:live-dom-families and gsd:write-continue, so it is declared in COMMENT_MARKER_TOKENS rather than renamed. Found by running the base-added gates against the merged tree; CI at ca8d2508 predates the gate. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LgNMb2G67rAJfFQRHEBTAj --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Tom Boucher <trekkie@nomorestars.com> Co-authored-by: CI Rebase Check <ci@gsd-redux>
This commit is contained in:
0xdhx
5
.changeset/vivid-yaks-fly.md
Normal file
5
.changeset/vivid-yaks-fly.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Added
|
||||
pr: 4477
|
||||
---
|
||||
**`/gsd-ui-review` can capture post-interaction UI states** — a new default-off `workflow.ui_interaction_capture` key on the `ui` capability lets `gsd-ui-auditor` add hover, focus-ring, open-menu and filled-form captures through the `chrome-devtools` CLI, driven from Bash with no MCP server and no tool-surface change. Requires an installed Chrome; when off, or when none resolves, the Playwright-only static capture runs exactly as before. (#4223)
|
||||
@@ -78,29 +78,21 @@ If no UI-SPEC exists: audit against abstract 6-pillar standards.
|
||||
|
||||
## Screenshot Storage Safety
|
||||
|
||||
**MUST run before any screenshot capture.** Prevents binary files from reaching git history.
|
||||
**MUST run before any screenshot capture.** Prevents capture output from reaching git history.
|
||||
|
||||
```bash
|
||||
# Ensure directory exists
|
||||
mkdir -p .planning/ui-reviews
|
||||
|
||||
# Write .gitignore if not present
|
||||
if [ ! -f .planning/ui-reviews/.gitignore ]; then
|
||||
cat > .planning/ui-reviews/.gitignore << 'GITIGNORE'
|
||||
# Screenshot files — never commit binary assets
|
||||
*.png
|
||||
*.webp
|
||||
*.jpg
|
||||
*.jpeg
|
||||
*.gif
|
||||
*.bmp
|
||||
*.tiff
|
||||
GITIGNORE
|
||||
echo "Created .planning/ui-reviews/.gitignore"
|
||||
fi
|
||||
# Append any pattern the file lacks — an older .gitignore is still covered; never rewritten.
|
||||
[ -f .planning/ui-reviews/.gitignore ] \
|
||||
|| { printf '# UI-audit captures — never commit\n' > .planning/ui-reviews/.gitignore; echo "Created .planning/ui-reviews/.gitignore"; }
|
||||
for p in '*.png' '*.webp' '*.jpg' '*.jpeg' '*.gif' '*.bmp' '*.tiff' 'interaction/'; do
|
||||
grep -qxF -- "$p" .planning/ui-reviews/.gitignore || printf '%s\n' "$p" >> .planning/ui-reviews/.gitignore
|
||||
done
|
||||
```
|
||||
|
||||
This gate runs unconditionally on every audit. The .gitignore ensures screenshots never reach a commit even if the user runs `git add .` before cleanup.
|
||||
It keeps capture output out of a commit even after `git add .`: static screenshots by extension, and the `interaction/` directory as a whole (its snapshot carries form values; its console output can carry tokens); a directory pattern covers the next artifact type by construction.
|
||||
|
||||
</gitignore_gate>
|
||||
|
||||
@@ -141,6 +133,149 @@ If dev server not detected: audit runs on code review only (Tailwind class audit
|
||||
|
||||
Try port 3000 first, then 5173 (Vite default), then 8080.
|
||||
|
||||
<!-- gsd:ui-interaction-capture -->
|
||||
|
||||
### Interaction capture (default-off — `workflow.ui_interaction_capture`)
|
||||
|
||||
The static captures show the first paint of `/` and nothing after it: `npx playwright screenshot` has no click, fill, hover, press, snapshot or console verb, yet the Experience Design pillar is scored on exactly that. When the `<config>` block carries `interaction_capture: true` (the `workflow.ui_interaction_capture` key, read by `/gsd:ui-review`) **and** a Chrome binary resolves, the `chrome-devtools` CLI (`chrome-devtools-mcp`) adds post-interaction captures over `Bash` alone: no MCP server, no `tools:` change. Key off, or no Chrome: one status line, then the audit as before.
|
||||
|
||||
```bash
|
||||
# INTERACTION_CAPTURE: the <config> block's `interaction_capture` (absent = off); SCREENSHOT_DIR/DEV_URL: above.
|
||||
INTERACTION_CAPTURE="${INTERACTION_CAPTURE:-false}"
|
||||
INTERACTION_STATUS="off"
|
||||
|
||||
# An installed Chrome, never a download; CHROME_BIN overrides.
|
||||
CHROME_BIN="${CHROME_BIN:-}"
|
||||
if [ -z "$CHROME_BIN" ]; then
|
||||
for _c in google-chrome google-chrome-stable chromium chromium-browser chrome; do
|
||||
if command -v "$_c" >/dev/null 2>&1; then CHROME_BIN=$(command -v "$_c"); break; fi
|
||||
done
|
||||
fi
|
||||
if [ -z "$CHROME_BIN" ] && [ -x "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" ]; then
|
||||
CHROME_BIN="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
|
||||
fi
|
||||
if [ -z "$CHROME_BIN" ] && [ -x "${PROGRAMFILES:-/nonexistent}/Google/Chrome/Application/chrome.exe" ]; then
|
||||
CHROME_BIN="${PROGRAMFILES}/Google/Chrome/Application/chrome.exe"
|
||||
fi
|
||||
|
||||
# Floor, not a pin (--workspace needs 1.9.0); -y answers npx's prompt. --sessionId (hex/dashes) keys the
|
||||
# daemon socket; concurrent audits need their own: BASHPID not $$ (subshells share $$) + $RANDOM.
|
||||
CDT_SESSION="$(date +%s)-${BASHPID:-$$}-$RANDOM"
|
||||
CDT="npx -y -p chrome-devtools-mcp@${CHROME_DEVTOOLS_MCP_VERSION:-^1.9.0} chrome-devtools --sessionId $CDT_SESSION"
|
||||
# cdt <ceiling-s> <verb> [args...]: every driver call is bounded (no timeout(1) on macOS, no gsd-tools here) by a
|
||||
# watchdog killing the job's process group at the ceiling (TERM, KILL 2 s later; npm forwards SIGTERM only to its
|
||||
# direct child). An exec'd bash (a subshell keeps the caller's saved stdio open) polling the job's GROUP (a child
|
||||
# can outlive the leader holding stdout), standing down once it is empty — never signalled: bash 3.2 may not
|
||||
# interrupt `wait` for a trapped signal; Git Bash hangs on a signal to a process still starting up. No sleep, no fire.
|
||||
CDT_T_START="${CHROME_DEVTOOLS_START_TIMEOUT:-180}"
|
||||
CDT_T_STEP="${CHROME_DEVTOOLS_STEP_TIMEOUT:-60}"
|
||||
cdt() {
|
||||
local ceiling="$1" pid wd rc=0; shift
|
||||
set -m; $CDT "$@" & pid=$!; set +m # -m: job = own process group
|
||||
"${BASH:-bash}" -c 'n=$(($1 * 10)); while kill -0 -- "-$2" 2>/dev/null; do [ "$n" -gt 0 ] || { kill -TERM -- "-$2" 2>/dev/null; sleep 2; kill -0 -- "-$2" 2>/dev/null && kill -KILL -- "-$2" 2>/dev/null; exit 0; }; sleep 0.1 || exit 0; n=$((n - 1)); done' _ "$ceiling" "$pid" >/dev/null 2>&1 & wd=$!
|
||||
wait "$pid" || rc=$?
|
||||
wait "$wd" 2>/dev/null || true
|
||||
return "$rc"
|
||||
}
|
||||
|
||||
if [ "$INTERACTION_CAPTURE" != "true" ]; then
|
||||
echo "Interaction capture: off (workflow.ui_interaction_capture is false)"
|
||||
elif [ -z "${SCREENSHOT_DIR:-}" ] || [ ! -d "$SCREENSHOT_DIR" ]; then
|
||||
INTERACTION_STATUS="skipped (no dev server reached)"
|
||||
echo "Interaction capture: skipped — static capture reached no dev server"
|
||||
elif [ -z "$CHROME_BIN" ]; then
|
||||
INTERACTION_STATUS="skipped (no Chrome binary resolved)"
|
||||
echo "Interaction capture: skipped — no Chrome binary resolved (set CHROME_BIN)"
|
||||
else
|
||||
DEV_URL="${DEV_URL:-http://localhost:3000}"
|
||||
INTERACTION_DIR="$SCREENSHOT_DIR/interaction"
|
||||
mkdir -p "$INTERACTION_DIR"
|
||||
ICAPTURED=0
|
||||
IFAILED=0
|
||||
PAGE_ID=""
|
||||
# cdt_me: this shell's pid (bash 3.2 has no BASHPID).
|
||||
cdt_me() { exec /bin/sh -c 'echo "$PPID"'; }
|
||||
CDT_STARTED=0; CDT_SHELL=$(cdt_me)
|
||||
|
||||
# ishot <label>: count a non-empty file, else remove.
|
||||
ishot() {
|
||||
if cdt "$CDT_T_STEP" take_screenshot "$PAGE_ID" --filePath "$INTERACTION_DIR/$1.png" >/dev/null 2>&1 \
|
||||
&& [ -s "$INTERACTION_DIR/$1.png" ]; then
|
||||
ICAPTURED=$((ICAPTURED + 1))
|
||||
else
|
||||
rm -f "$INTERACTION_DIR/$1.png"
|
||||
IFAILED=$((IFAILED + 1))
|
||||
echo " interaction capture FAILED: $1"
|
||||
fi
|
||||
}
|
||||
|
||||
# cdt_stop: stop is owed once after a successful start (no self-reap): trapped on EXIT (replaces any earlier
|
||||
# trap), in order, flag-deduped, by the installing shell only (never a subshell copy).
|
||||
cdt_stop() { [ "$CDT_STARTED" = 1 ] && [ "$(cdt_me)" = "$CDT_SHELL" ] || return 0; CDT_STARTED=0; cdt "$CDT_T_STEP" stop >/dev/null 2>&1 || true; }
|
||||
|
||||
# --isolated: throwaway profile. --workspace: writes under the capture dir only — relative like every
|
||||
# --filePath (one cwd, dialect-free on Git Bash); --allowUnrestrictedPaths is deprecated.
|
||||
if cdt "$CDT_T_START" start -e "$CHROME_BIN" --isolated --workspace "$INTERACTION_DIR" --usageStatistics=false >/dev/null 2>&1; then
|
||||
CDT_STARTED=1; trap cdt_stop EXIT
|
||||
# new_page marks the page `[selected]`: the pageId every later verb takes. --timeout (ms) bounds the
|
||||
# navigation inside the ceiling; exit status checked before parsing; tr: CRLF.
|
||||
PAGE_ID=""
|
||||
if NEW_PAGE_OUT=$(cdt "$CDT_T_STEP" new_page "$DEV_URL" --timeout 30000 2>/dev/null); then
|
||||
PAGE_ID=$(printf '%s\n' "$NEW_PAGE_OUT" | tr -d '\r' | sed -n 's/^\([0-9][0-9]*\): .*\[selected\]$/\1/p' | head -1)
|
||||
fi
|
||||
if [ -n "$PAGE_ID" ]; then
|
||||
# A failed resize: a failed step, no abort.
|
||||
if ! cdt "$CDT_T_STEP" resize_page "$PAGE_ID" 1440 900 >/dev/null 2>&1; then
|
||||
IFAILED=$((IFAILED + 1))
|
||||
echo " interaction step FAILED: resize_page"
|
||||
fi
|
||||
# uids are per-snapshot: re-take after each interaction. A failure counts.
|
||||
if ! cdt "$CDT_T_STEP" take_snapshot "$PAGE_ID" --filePath "$INTERACTION_DIR/snapshot.txt" >/dev/null 2>&1; then
|
||||
# Remove what it left, or a stale one (reused dir)
|
||||
rm -f "$INTERACTION_DIR/snapshot.txt"
|
||||
IFAILED=$((IFAILED + 1))
|
||||
echo " interaction step FAILED: take_snapshot"
|
||||
fi
|
||||
ishot baseline
|
||||
# Focus ring: first focusable.
|
||||
if cdt "$CDT_T_STEP" press_key "$PAGE_ID" Tab >/dev/null 2>&1; then
|
||||
ishot focus-first
|
||||
else
|
||||
IFAILED=$((IFAILED + 1))
|
||||
echo " interaction step FAILED: press_key Tab"
|
||||
fi
|
||||
# --- Drive each interactive component UI-SPEC.md declares (or the snapshot shows): real
|
||||
# calls, a uid from the latest snapshot, one capture each:
|
||||
# cdt "$CDT_T_STEP" hover "$PAGE_ID" <uid> && ishot hover-<label>
|
||||
# cdt "$CDT_T_STEP" click "$PAGE_ID" <uid> && ishot <label>-open
|
||||
# cdt "$CDT_T_STEP" fill "$PAGE_ID" <uid> "<value>" && ishot <label>-filled
|
||||
# cdt "$CDT_T_STEP" press_key "$PAGE_ID" Enter && ishot <label>-submitted
|
||||
# cdt "$CDT_T_STEP" take_snapshot "$PAGE_ID" --filePath "$INTERACTION_DIR/snapshot.txt"
|
||||
# Console output since navigation.
|
||||
cdt "$CDT_T_STEP" list_console_messages "$PAGE_ID" > "$INTERACTION_DIR/console.txt" 2>/dev/null || true
|
||||
else
|
||||
echo " new_page FAILED: $DEV_URL"
|
||||
fi
|
||||
cdt_stop
|
||||
else
|
||||
echo " start FAILED (npx fetch, Chrome at $CHROME_BIN, or the ${CDT_T_START}s ceiling)"
|
||||
fi
|
||||
|
||||
if [ "$ICAPTURED" -gt 0 ]; then
|
||||
INTERACTION_STATUS="captured ($ICAPTURED state(s), $IFAILED failed) in $INTERACTION_DIR"
|
||||
else
|
||||
INTERACTION_STATUS="not captured (driver or capture failure)"
|
||||
fi
|
||||
echo "Interaction capture: $INTERACTION_STATUS"
|
||||
fi
|
||||
```
|
||||
|
||||
`wait_for` is MCP-only: where a state needs settling, poll `cdt "$CDT_T_STEP" evaluate_script "() => document.readyState" --pageId "$PAGE_ID"` for `complete`. The driver is Chromium-only; Firefox and WebKit stay on `npx playwright screenshot -b firefox|webkit`.
|
||||
|
||||
Carry `$INTERACTION_STATUS` into the report's `**Interaction captures:**` field. **Never report an interaction state you did not capture** — key off or section skipped, interaction findings are code-derived and say so.
|
||||
|
||||
<!-- /gsd:ui-interaction-capture -->
|
||||
|
||||
</screenshot_approach>
|
||||
|
||||
<audit_pillars>
|
||||
@@ -300,6 +435,7 @@ Write to: `$PHASE_DIR/$PADDED_PHASE-UI-REVIEW.md`
|
||||
**Audited:** {date}
|
||||
**Baseline:** {UI-SPEC.md / abstract standards}
|
||||
**Screenshots:** {captured / not captured (no dev server)}
|
||||
**Interaction captures:** {$INTERACTION_STATUS — off / skipped (reason) / captured (N states) / not captured (reason)}
|
||||
|
||||
---
|
||||
|
||||
@@ -366,7 +502,7 @@ Run the gitignore gate from `<gitignore_gate>`. This MUST happen before step 3.
|
||||
|
||||
## Step 3: Detect Dev Server and Capture Screenshots
|
||||
|
||||
Run the screenshot approach from `<screenshot_approach>`. Record whether screenshots were captured.
|
||||
Run the screenshot approach from `<screenshot_approach>`. Record whether screenshots were captured. Then run its interaction-capture section with `INTERACTION_CAPTURE` set from the `<config>` block's `interaction_capture` value, and record `$INTERACTION_STATUS` verbatim — it is `off` unless `workflow.ui_interaction_capture` is on and a Chrome binary resolved.
|
||||
|
||||
## Step 4: Scan Implemented Files
|
||||
|
||||
@@ -407,6 +543,7 @@ Use output format from `<output_format>`. If registry audit produced flags, add
|
||||
**Phase:** {phase_number} - {phase_name}
|
||||
**Overall Score:** {total}/24
|
||||
**Screenshots:** {captured / not captured}
|
||||
**Interaction captures:** {$INTERACTION_STATUS}
|
||||
|
||||
### Pillar Summary
|
||||
| Pillar | Score |
|
||||
@@ -441,6 +578,7 @@ UI audit is complete when:
|
||||
- [ ] .gitignore gate executed before any screenshot capture
|
||||
- [ ] Dev server detection attempted
|
||||
- [ ] Screenshots captured (or noted as unavailable)
|
||||
- [ ] Interaction-capture outcome recorded from `$INTERACTION_STATUS` (off, skipped with reason, or captured)
|
||||
- [ ] All 6 pillars scored with evidence
|
||||
- [ ] Registry safety audit executed (if shadcn + third-party registries present)
|
||||
- [ ] Top 3 priority fixes identified with concrete solutions
|
||||
|
||||
@@ -39,6 +39,11 @@
|
||||
"type": "boolean",
|
||||
"default": true,
|
||||
"description": "Block execution on unmet UI-SPEC contracts."
|
||||
},
|
||||
"workflow.ui_interaction_capture": {
|
||||
"type": "boolean",
|
||||
"default": false,
|
||||
"description": "Default-off. Let gsd-ui-auditor add post-interaction captures (hover, focus, open menus, filled forms) through the chrome-devtools CLI, driven from Bash — no MCP server and no tools: change (#4223). Requires an installed Chrome; when off, or when none resolves, the auditor's Playwright-only static capture is unchanged."
|
||||
}
|
||||
},
|
||||
"steps": [
|
||||
|
||||
@@ -378,6 +378,7 @@ Three further dimensions carry no number: **Verify Command Format Sanity**,
|
||||
| **Model (balanced)** | Sonnet |
|
||||
| **Color** | Pink |
|
||||
| **Produces** | `{phase}-UI-REVIEW.md` with scores |
|
||||
| **Interaction capture** | `workflow.ui_interaction_capture` (default `false`) |
|
||||
|
||||
**6 Audit Pillars (scored 1-4):**
|
||||
1. Copywriting
|
||||
@@ -387,6 +388,19 @@ Three further dimensions carry no number: **Verify Command Format Sanity**,
|
||||
5. Spacing
|
||||
6. Experience Design
|
||||
|
||||
**Interaction capture (default-off).** The static screenshots are three viewport captures of
|
||||
the first paint, taken with `npx playwright screenshot`, which has no interaction verb — so a
|
||||
hover state, an open menu, a focus ring or a form's validation state never appears in them.
|
||||
With `workflow.ui_interaction_capture` on, `/gsd-ui-review` passes `interaction_capture: true`
|
||||
in the auditor's `<config>` block and the auditor adds post-interaction captures through the
|
||||
`chrome-devtools` CLI (the second binary in the `chrome-devtools-mcp` package), driven from
|
||||
`Bash` against a throwaway `--isolated` profile: no MCP server, no `tools:` change. It needs an
|
||||
installed Chrome (`CHROME_BIN` overrides discovery). With the key off, or no Chrome resolved,
|
||||
the section prints one line and the Playwright-only path runs exactly as before. The report's
|
||||
`**Interaction captures:**` field carries the outcome — off, skipped with its reason, or the
|
||||
number of states captured — and an interaction state that was not captured is never reported as
|
||||
observed. See [Enable UI interaction capture](how-to/enable-ui-interaction-capture.md).
|
||||
|
||||
---
|
||||
|
||||
### gsd-dom-verifier
|
||||
|
||||
@@ -506,6 +506,7 @@ All workflow toggles follow the **absent = enabled** pattern. If a key is missin
|
||||
| `workflow.assumption_delta` | boolean | `true` | Advisory architecture checkpoint during planning. When a phase makes something **plural, optional, or chosen** that used to be **singular, required, or derived** (e.g. a second auth method, a required field becoming optional, a constant becoming a parameter), the planner is prompted to re-ask whether the primary key / identity model still names the right thing (promote the new general representation vs. add it alongside). Non-blocking; fires only on a detected signal. Bare "or" is intentionally excluded (prose false-positives). Inspect a phase with `gsd_run query assumption-delta scan <phase>`. Added in #1561. A phase section that cannot be resolved returns `{"skipped":true,"reason":"phase_unresolved"}` rather than a fabricated `detected:false` (#3909) |
|
||||
| `workflow.ui_review` | boolean | `true` | Run visual quality audit (`/gsd-ui-review`) after phase execution in autonomous mode. When `false`, the UI audit step is skipped. |
|
||||
| `workflow.live_dom_uat` | boolean | `false` | **Default-off.** Enable live-DOM verification (#2856). When `true`, a `gsd-dom-verifier` step runs after each execution wave and writes `{phase}-DOM-VERIFY.md`, and the orchestrator's automated UI verification will additionally consider `mcp__chrome-devtools__*` / `mcp__claude-in-chrome__*` when present. Browser reach is confined to `gsd-dom-verifier` — `gsd-executor`'s tool surface is unchanged in every configuration. Presence of a browser MCP server is **not** sufficient on its own: a server configured for unrelated work is never driven unless this key is on. The pre-existing `mcp__playwright__*` path is unaffected by this key. Note `chrome-devtools-mcp` holds an exclusive browser-profile lock, so concurrent waves need `--isolated` on **your** MCP server registration — GSD cannot pass it. See [Enable live-DOM verification](how-to/enable-live-dom-verification.md). |
|
||||
| `workflow.ui_interaction_capture` | boolean | `false` | **Default-off.** Let `gsd-ui-auditor` add post-interaction captures — hover, focus ring, open menus, filled forms — to its static screenshots (#4223). The driver is the `chrome-devtools` CLI from the `chrome-devtools-mcp` package, run from `Bash`, so no MCP server is configured and the agent's tool surface is unchanged. Requires an installed Chrome (`CHROME_BIN` overrides discovery; Chromium-only — Firefox/WebKit stay on Playwright). When `false`, or when no Chrome resolves, the Playwright-only static capture runs exactly as before. Read by `/gsd-ui-review` and handed to the auditor. See [Enable UI interaction capture](how-to/enable-ui-interaction-capture.md). |
|
||||
| `workflow.node_repair` | boolean | `true` | Autonomous task repair on verification failure |
|
||||
| `workflow.node_repair_budget` | number | `2` | Max repair attempts per failed task |
|
||||
| `workflow.smart_zone_tokens` | number | `100000` | Smart-zone token budget for phase-effort estimation (#2630, [ADR-2629](adr/2629-phase-effort-estimation-calibration.md)). A phase whose estimate exceeds this is flagged with a split recommendation — **advisory only, never a block**. This is a *policy default, not a benchmark constant*: LLM output quality degrades before the advertised context window is full, but the effective ceiling is model-, task-, and distractor-dependent, so no universal number exists. Lower it for models that degrade early; the estimate-vs-actual calibration loop corrects the figure per project over time. Must be a positive integer. |
|
||||
|
||||
@@ -66,6 +66,7 @@ Language versions: [English](README.md) · [Português (pt-BR)](pt-BR/README.md)
|
||||
- [Spike and sketch](how-to/spike-and-sketch.md) — use `/gsd-spike` and `/gsd-sketch` for exploratory work before committing to a plan
|
||||
- [Design a UI phase](how-to/design-a-ui-phase.md) — use the UI phase loop for frontend and visual work
|
||||
- [Enable live-DOM verification](how-to/enable-live-dom-verification.md) — opt a project into browser-backed UI acceptance checks during execution, handle the browser-profile lock, and tell "nothing to report" apart from "could not look"
|
||||
- [Enable UI interaction capture](how-to/enable-ui-interaction-capture.md) — let `/gsd-ui-review`'s auditor capture hover, focus, open-menu and filled-form states through the `chrome-devtools` CLI from Bash, with no MCP server
|
||||
- [Develop a Capability for GSD 1.5+](how-to/develop-a-capability.md) — add feature Capabilities, hook fragments, and registry entries
|
||||
- [Develop a task-content resolver capability](how-to/develop-a-task-content-resolver-capability.md) — declare a `taskContentResolver` so `execute-plan.md` resolves per-task content from your external issue tracker instead of `PLAN.md`
|
||||
- [Ship a reviewer lane in your capability](how-to/ship-a-reviewer-lane.md) — declare a `reviewer` body so `/gsd-review` discovers, invokes, and renders your external review CLI or model endpoint
|
||||
|
||||
111
docs/how-to/enable-ui-interaction-capture.md
Normal file
111
docs/how-to/enable-ui-interaction-capture.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# How to enable UI interaction capture
|
||||
|
||||
Let `/gsd-ui-review`'s auditor capture what a page looks like *after* an interaction — a hover
|
||||
state, a focus ring, an open menu, a filled form's validation state — instead of only the first
|
||||
paint, without configuring an MCP server or widening the agent's tool surface.
|
||||
|
||||
> **Default-off, and deliberately so.** The static capture path is unchanged in every
|
||||
> configuration; this key adds captures on top of it. You opt in per project with one key.
|
||||
> The gap it closes is [#4223](https://github.com/open-gsd/gsd-core/issues/4223): the auditor
|
||||
> is chartered to audit interaction, and `npx playwright screenshot` has no interaction verb.
|
||||
|
||||
**What you need:**
|
||||
|
||||
- An installed Chrome or Chromium. The driver launches the system browser (Puppeteer
|
||||
`channel: 'chrome'`); it does not download one. Discovery tries `google-chrome`,
|
||||
`google-chrome-stable`, `chromium`, `chromium-browser` and `chrome` on `PATH`, then the
|
||||
standard macOS and Windows install paths. `CHROME_BIN=/path/to/chrome` overrides it.
|
||||
- `npx` able to fetch `chrome-devtools-mcp` (the package that ships the `chrome-devtools` CLI).
|
||||
It is resolved at the documented floor `^1.9.0` (the release that added `--workspace`, which
|
||||
confines the driver's file writes to the capture directory); `CHROME_DEVTOOLS_MCP_VERSION`
|
||||
overrides it.
|
||||
- Nothing else. Every driver call runs under a ceiling — `CHROME_DEVTOOLS_START_TIMEOUT` (default
|
||||
180 s, for the npx fetch plus the Chrome launch) and `CHROME_DEVTOOLS_STEP_TIMEOUT` (default
|
||||
60 s, per capture verb). At the ceiling the client's whole process group is killed (TERM, then
|
||||
KILL two seconds later) and the step is counted as failed, so a cold npm cache or a Chrome that
|
||||
never comes up costs a failed step rather than an open-ended wait. The one exception is a
|
||||
watchdog whose own clock (`sleep`) cannot launch: it stands down instead of killing a healthy
|
||||
call, and that call is then unbounded, as every call was before the ceilings existed.
|
||||
- A dev server the static capture already reaches — interaction capture runs against the same
|
||||
URL and skips itself when the static block reached nothing.
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Turn the key on
|
||||
|
||||
```bash
|
||||
gsd-tools query config-set workflow.ui_interaction_capture true
|
||||
```
|
||||
|
||||
Verify it took:
|
||||
|
||||
```bash
|
||||
gsd-tools query config-get workflow.ui_interaction_capture
|
||||
# → true
|
||||
```
|
||||
|
||||
`/gsd-ui-review` reads the key and passes `interaction_capture: true` in the auditor's
|
||||
`<config>` block — the auditor itself never reads config.
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — Run a review and read the report
|
||||
|
||||
```bash
|
||||
/gsd-ui-review 3
|
||||
```
|
||||
|
||||
The audit's static captures land where they always did. With the key on and a Chrome
|
||||
resolved, an `interaction/` directory beside them holds `baseline.png`, `focus-first.png`
|
||||
(focus ring on the first focusable element), one capture per interaction the auditor drove
|
||||
from your UI-SPEC's interactive components, the accessibility snapshot it used for element
|
||||
ids, and the page's console output. The audit's `.gitignore` gate covers that `interaction/`
|
||||
directory as a whole — the snapshot carries whatever was typed into forms and the console output
|
||||
can carry tokens — so a `git add .` never commits it, and a project whose `.gitignore` predates
|
||||
the directory is upgraded on the next audit. `UI-REVIEW.md` carries the outcome on its own line:
|
||||
|
||||
```
|
||||
**Interaction captures:** captured (4 state(s), 0 failed) in .planning/ui-reviews/03-.../interaction
|
||||
```
|
||||
|
||||
The other values it can hold are honest, not decorative:
|
||||
|
||||
| `**Interaction captures:**` | Meaning |
|
||||
|---|---|
|
||||
| `off` | The key is `false`. Nothing else changed. |
|
||||
| `skipped (no dev server reached)` | The static capture found no dev server; there was nothing to interact with. |
|
||||
| `skipped (no Chrome binary resolved)` | The key is on but no browser resolved. Set `CHROME_BIN`. |
|
||||
| `not captured (driver or capture failure)` | The key was on and Chrome resolved, but no state landed on disk — `npx` could not fetch the driver, Chrome did not launch, the page never opened, or every capture failed. The audit output names the failing step. |
|
||||
|
||||
An interaction state that does not appear in that directory is not reported as observed; the
|
||||
Experience Design pillar says when its findings are code-derived.
|
||||
|
||||
---
|
||||
|
||||
## What this does not do
|
||||
|
||||
- **It does not replace the static captures.** `npx playwright screenshot` stays the driver for
|
||||
the three viewport shots; it has `--wait-for-selector`, `--device`, `--color-scheme` and
|
||||
cross-engine `-b firefox|webkit`, none of which the CLI driver offers. Firefox and WebKit needs
|
||||
stay on Playwright.
|
||||
- **It does not touch `gsd-dom-verifier` or `workflow.live_dom_uat`.** That capability drives a
|
||||
browser MCP server at execute time; this one drives a CLI from Bash at review time. They are
|
||||
independent keys.
|
||||
- **It does not share the `chrome-devtools-mcp` browser profile.** The auditor starts the daemon
|
||||
with `--isolated`, so it never contends for the profile lock a registered MCP server holds, and
|
||||
stops it when the capture ends — under an `EXIT` trap, so an aborted audit still stops it.
|
||||
- **It does not let the driver write outside the capture directory.** The daemon starts with
|
||||
`--workspace` set to the run's `interaction/` directory; that is the only place its file-writing
|
||||
verbs may land.
|
||||
- **It does not wait on selectors.** `wait_for` is MCP-only; the auditor polls
|
||||
`document.readyState` through `evaluate_script` where a state needs settling.
|
||||
|
||||
---
|
||||
|
||||
## Turning it back off
|
||||
|
||||
```bash
|
||||
gsd-tools query config-set workflow.ui_interaction_capture false
|
||||
```
|
||||
|
||||
The next review runs the Playwright-only path and reports `**Interaction captures:** off`.
|
||||
@@ -3858,6 +3858,11 @@ const capabilities = {
|
||||
"type": "boolean",
|
||||
"default": true,
|
||||
"description": "Block execution on unmet UI-SPEC contracts."
|
||||
},
|
||||
"workflow.ui_interaction_capture": {
|
||||
"type": "boolean",
|
||||
"default": false,
|
||||
"description": "Default-off. Let gsd-ui-auditor add post-interaction captures (hover, focus, open menus, filled forms) through the chrome-devtools CLI, driven from Bash — no MCP server and no tools: change (#4223). Requires an installed Chrome; when off, or when none resolves, the auditor's Playwright-only static capture is unchanged."
|
||||
}
|
||||
},
|
||||
"steps": [
|
||||
@@ -4896,7 +4901,8 @@ const configKeys = {
|
||||
"workflow.tdd_mode": "tdd",
|
||||
"workflow.ui_phase": "ui",
|
||||
"workflow.ui_review": "ui",
|
||||
"workflow.ui_safety_gate": "ui"
|
||||
"workflow.ui_safety_gate": "ui",
|
||||
"workflow.ui_interaction_capture": "ui"
|
||||
};
|
||||
|
||||
const configSchema = {
|
||||
@@ -5446,6 +5452,12 @@ const configSchema = {
|
||||
"type": "boolean",
|
||||
"default": true,
|
||||
"description": "Block execution on unmet UI-SPEC contracts."
|
||||
},
|
||||
"workflow.ui_interaction_capture": {
|
||||
"owner": "ui",
|
||||
"type": "boolean",
|
||||
"default": false,
|
||||
"description": "Default-off. Let gsd-ui-auditor add post-interaction captures (hover, focus, open menus, filled forms) through the chrome-devtools CLI, driven from Bash — no MCP server and no tools: change (#4223). Requires an installed Chrome; when off, or when none resolves, the auditor's Playwright-only static capture is unchanged."
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -21,6 +21,10 @@ RESPONSE_LANGUAGE=$(gsd_run query config-get response_language --raw --default "
|
||||
INIT=$(gsd_run query init.phase-op "${PHASE_ARG}")
|
||||
if [[ "$INIT" == @file:* ]]; then INIT=$(cat "${INIT#@file:}"); fi
|
||||
AGENT_SKILLS_UI_REVIEWER=$(gsd_run query agent-skills gsd-ui-auditor)
|
||||
# workflow.ui_interaction_capture (default false): read here and handed to the auditor
|
||||
# through its <config> block — the agent carries no gsd_run resolver of its own.
|
||||
INTERACTION_CAPTURE=$(gsd_run query config-get workflow.ui_interaction_capture --raw 2>/dev/null || echo "false")
|
||||
[ "$INTERACTION_CAPTURE" = "true" ] || INTERACTION_CAPTURE="false"
|
||||
```
|
||||
|
||||
**If `response_language` is set:** All user-facing output of this workflow — narration between tool calls, status updates, progress notes, findings, questions, prompts, and explanations — MUST be presented in `{response_language}`. Technical terms, code, file paths, and subagent prompts stay in English — only user-facing output is translated.
|
||||
@@ -94,10 +98,13 @@ ${AGENT_SKILLS_UI_REVIEWER}
|
||||
<config>
|
||||
phase_dir: {phase_dir}
|
||||
padded_phase: {padded_phase}
|
||||
interaction_capture: {interaction_capture}
|
||||
</config>
|
||||
```
|
||||
|
||||
Omit null file paths.
|
||||
Omit null file paths. `interaction_capture` is always present — the `$INTERACTION_CAPTURE` value read in step 0, `true` only when
|
||||
`workflow.ui_interaction_capture` is on; the auditor's `<screenshot_approach>` branches on it and
|
||||
falls back to its Playwright-only static capture when it is `false` or no Chrome binary resolves.
|
||||
|
||||
<!-- #2508 runtime-aware-dispatch -->
|
||||
|
||||
|
||||
@@ -402,6 +402,7 @@ const DOCS_GUARD_TESTS = {
|
||||
'docs/how-to/plan-a-phase.md',
|
||||
'docs/AGENTS.md',
|
||||
],
|
||||
'tests/ui-interaction-capture.test.cjs': ['docs/CONFIGURATION.md'],
|
||||
'tests/ui-spec-inventory-provenance.test.cjs': [
|
||||
'docs/FEATURES.md',
|
||||
'docs/how-to/design-a-ui-phase.md',
|
||||
|
||||
@@ -198,6 +198,7 @@ module.exports = {
|
||||
"tests/tracer-bullet.test.cjs",
|
||||
"tests/uat-predicate.test.cjs",
|
||||
"tests/uat.test.cjs",
|
||||
"tests/ui-interaction-capture.test.cjs",
|
||||
"tests/ui-safety-gate.test.cjs",
|
||||
"tests/ui-spec-inventory-provenance.test.cjs",
|
||||
"tests/unreachable-guard-drift.test.cjs",
|
||||
|
||||
@@ -257,6 +257,7 @@ module.exports = {
|
||||
"tests/teams-status.test.cjs",
|
||||
"tests/test-gate-watch-mode.test.cjs",
|
||||
"tests/todos-workstream-scope.test.cjs",
|
||||
"tests/ui-interaction-capture.test.cjs",
|
||||
"tests/undo-commit-selection-4465.test.cjs",
|
||||
"tests/unreachable-guard-drift.test.cjs",
|
||||
"tests/unreachable-shell-guard.test.cjs",
|
||||
|
||||
@@ -1171,6 +1171,7 @@ describe('bug #3683 — workflow/reference colon-namespace leak (Claude local in
|
||||
'plan-revision-conflicts', // <!-- gsd:plan-revision-conflicts:begin --> / :end
|
||||
'live-dom-families', // <!-- gsd:live-dom-families -->
|
||||
'write-continue', // <!-- gsd:write-continue … -->
|
||||
'ui-interaction-capture', // <!-- gsd:ui-interaction-capture --> / <!-- /gsd:ui-interaction-capture -->
|
||||
]);
|
||||
const BARE_MARKER_TOKENS = new Set([
|
||||
'guard', // `# gsd:guard=orchestrator-cwd-drift`
|
||||
|
||||
778
tests/ui-interaction-capture.test.cjs
Normal file
778
tests/ui-interaction-capture.test.cjs
Normal file
@@ -0,0 +1,778 @@
|
||||
'use strict';
|
||||
// Nothing here reads a source module of the kinds no-source-grep tracks
|
||||
// (.cjs/.cts/.js/.mjs/.mts/.ts), so there is no site to suppress and the file
|
||||
// carries no allow-test-rule marker.
|
||||
|
||||
/**
|
||||
* workflow.ui_interaction_capture — #4223
|
||||
*
|
||||
* Enhancement shape approved at triage: a new default-off key on the EXISTING
|
||||
* `ui` capability (ADR-894's one-owner invariant — `ui` already owns
|
||||
* gsd-ui-auditor, so no new capability directory may claim it), with the
|
||||
* auditor's <screenshot_approach> branching on it and falling back to today's
|
||||
* Playwright-only path when the key is off or no Chrome binary resolves.
|
||||
*
|
||||
* Risk zone under test (in order):
|
||||
* 1. Containment — with the key off, or no Chrome, or no dev server, the
|
||||
* driver is never invoked and the static path is untouched.
|
||||
* 2. The key must not parse-and-do-nothing: the orchestrator reads it and
|
||||
* hands it down; the registry, schema and config layers all know it.
|
||||
* 3. Lifecycle honesty — the daemon is stopped whenever it was started (trapped,
|
||||
* not merely placed last), a failed capture is never counted, and the status
|
||||
* line says what happened.
|
||||
* 4. Bounded and confined — every driver call runs under a ceiling (a hung
|
||||
* npx fetch or Chrome launch cannot wedge the audit), and the driver may
|
||||
* write only under the capture directory (--workspace, never
|
||||
* --allowUnrestrictedPaths).
|
||||
* 5. The gitignore gate covers the capture directory as a whole, and an
|
||||
* existing .gitignore is upgraded rather than left as written.
|
||||
*
|
||||
* The auditor carries no gsd_run resolver, so the key travels through the
|
||||
* <config> block /gsd-ui-review builds; the fence under test consumes the
|
||||
* INTERACTION_CAPTURE, SCREENSHOT_DIR and DEV_URL variables the surrounding
|
||||
* prose defines and is executed here under bash with a stub driver on PATH.
|
||||
*/
|
||||
|
||||
const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const os = require('node:os');
|
||||
const path = require('node:path');
|
||||
const { spawnSync } = require('node:child_process');
|
||||
const { splitLines } = require('../gsd-core/bin/lib/text-lines.cjs');
|
||||
const { createTempProject, cleanup, runGsdTools } = require('./helpers.cjs');
|
||||
const { PROBE_TIMEOUT_MS } = require('./helpers/timeouts.cjs');
|
||||
|
||||
// The stub driver never blocks, so one fence run is a handful of sh spawns; distinct
|
||||
// from PROBE (a single probe) and BUILD (a compiler) — its own class, bounded generously.
|
||||
const FENCE_RUN_TIMEOUT_MS = 30000;
|
||||
|
||||
const realRegistry = require('../gsd-core/bin/lib/capability-registry.cjs');
|
||||
const { isValidConfigKey } = require('../gsd-core/bin/lib/config-schema.cjs');
|
||||
const { loadConfig } = require('../gsd-core/bin/lib/config-loader.cjs');
|
||||
|
||||
const REPO_ROOT = path.join(__dirname, '..');
|
||||
const CAP_ID = 'ui';
|
||||
const KEY = 'workflow.ui_interaction_capture';
|
||||
const AGENT = 'gsd-ui-auditor';
|
||||
|
||||
const AUDITOR_PATH = path.join(REPO_ROOT, 'agents', `${AGENT}.md`);
|
||||
const UI_REVIEW_PATH = path.join(REPO_ROOT, 'gsd-core', 'workflows', 'ui-review.md');
|
||||
const MANIFEST_PATH = path.join(REPO_ROOT, 'capabilities', CAP_ID, 'capability.json');
|
||||
const DOCS_CONFIG_PATH = path.join(REPO_ROOT, 'docs', 'CONFIGURATION.md');
|
||||
const HOWTO_PATH = path.join(REPO_ROOT, 'docs', 'how-to', 'enable-ui-interaction-capture.md');
|
||||
|
||||
const OPEN_ANCHOR = '<!-- gsd:ui-interaction-capture -->';
|
||||
const CLOSE_ANCHOR = '<!-- /gsd:ui-interaction-capture -->';
|
||||
const FENCE = '`'.repeat(3);
|
||||
|
||||
function readManifest() {
|
||||
return JSON.parse(fs.readFileSync(MANIFEST_PATH, 'utf8'));
|
||||
}
|
||||
|
||||
/**
|
||||
* The bash fences inside <screenshot_approach>, split into the static block
|
||||
* (before the interaction anchor) and the interaction block (inside it).
|
||||
* Line-based on purpose: no unbounded regex over file content, no ad-hoc
|
||||
* markdown parsing, and splitLines() handles CRLF checkouts.
|
||||
*/
|
||||
function screenshotApproachFences() {
|
||||
const lines = splitLines(fs.readFileSync(AUDITOR_PATH, 'utf8'));
|
||||
const out = { static: [], interaction: [] };
|
||||
let inSection = false;
|
||||
let inInteraction = false;
|
||||
let inFence = false;
|
||||
for (const line of lines) {
|
||||
if (!inSection) {
|
||||
if (line.includes('<screenshot_approach>')) inSection = true;
|
||||
continue;
|
||||
}
|
||||
if (line.includes('</screenshot_approach>')) break;
|
||||
if (line.includes(OPEN_ANCHOR)) { inInteraction = true; continue; }
|
||||
if (line.includes(CLOSE_ANCHOR)) { inInteraction = false; continue; }
|
||||
if (!inFence) {
|
||||
if (line.trim() === `${FENCE}bash`) inFence = true;
|
||||
continue;
|
||||
}
|
||||
if (line.trim() === FENCE) { inFence = false; continue; }
|
||||
(inInteraction ? out.interaction : out.static).push(line);
|
||||
}
|
||||
assert.ok(out.static.length > 0, '<screenshot_approach> must keep its static bash fence');
|
||||
assert.ok(out.interaction.length > 0, `${OPEN_ANCHOR} must wrap a bash fence`);
|
||||
return out;
|
||||
}
|
||||
|
||||
describe('ui capability owns workflow.ui_interaction_capture', () => {
|
||||
test('manifestDeclaresTheKeyAsADefaultOffBoolean', () => {
|
||||
const cap = readManifest();
|
||||
const slice = cap.config[KEY];
|
||||
assert.ok(slice, `${MANIFEST_PATH} must declare ${KEY}`);
|
||||
assert.equal(slice.type, 'boolean');
|
||||
assert.equal(slice.default, false, 'the approved shape is default-off');
|
||||
assert.ok(typeof slice.description === 'string' && slice.description.length > 0);
|
||||
});
|
||||
|
||||
test('theKeyLivesOnTheCapabilityThatAlreadyOwnsTheAuditor', () => {
|
||||
// ADR-894 one-owner invariant: a NEW capability directory could not also
|
||||
// claim gsd-ui-auditor, which is why triage required the existing manifest.
|
||||
const cap = readManifest();
|
||||
assert.ok(cap.agents.includes(AGENT), `${CAP_ID} must own ${AGENT}`);
|
||||
assert.equal(realRegistry.byAgent[AGENT], CAP_ID, 'the generated registry must agree on the owner');
|
||||
assert.equal(cap.activationKey, undefined,
|
||||
'the key gates one section of one agent, never the whole ui capability');
|
||||
});
|
||||
|
||||
test('generatedRegistryCarriesTheSliceUnderTheUiOwner', () => {
|
||||
const entry = realRegistry.configSchema[KEY];
|
||||
assert.ok(entry, 'capability-registry.cjs must be regenerated after the manifest change');
|
||||
assert.equal(entry.owner, CAP_ID);
|
||||
assert.equal(entry.type, 'boolean');
|
||||
assert.equal(entry.default, false);
|
||||
assert.deepEqual(realRegistry.capabilities[CAP_ID].config[KEY], readManifest().config[KEY]);
|
||||
});
|
||||
|
||||
test('configSchemaAcceptsTheKey', () => {
|
||||
assert.equal(isValidConfigKey(KEY), true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('config layer round-trips the key', () => {
|
||||
test('configSetAcceptsAndPersistsTrue', (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
const result = runGsdTools(`config-set ${KEY} true`, tmpDir);
|
||||
assert.ok(result.success, `config-set must accept ${KEY}: ${result.error}`);
|
||||
const cfg = JSON.parse(fs.readFileSync(path.join(tmpDir, '.planning', 'config.json'), 'utf8'));
|
||||
assert.equal(cfg.workflow?.ui_interaction_capture, true);
|
||||
});
|
||||
|
||||
test('configSetRejectsANonBoolean', (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
const result = runGsdTools(`config-set ${KEY} banana`, tmpDir);
|
||||
assert.ok(!result.success, 'a boolean slice must reject a non-boolean');
|
||||
});
|
||||
|
||||
test('absentKeyResolvesToFalse', (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
assert.equal(loadConfig(tmpDir).workflow?.ui_interaction_capture, false);
|
||||
});
|
||||
|
||||
for (const [label, value] of [['stringTrue', '"true"'], ['numberOne', '1'], ['nullValue', 'null']]) {
|
||||
test(`handWrittenNonBooleanResolvesToFalse_${label}`, (t) => {
|
||||
const tmpDir = createTempProject();
|
||||
t.after(() => cleanup(tmpDir));
|
||||
fs.writeFileSync(path.join(tmpDir, '.planning', 'config.json'),
|
||||
`{"workflow":{"ui_interaction_capture":${value}}}`);
|
||||
assert.equal(loadConfig(tmpDir).workflow?.ui_interaction_capture, false,
|
||||
`${label} must fall to the slice default, not survive as truthy`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('/gsd-ui-review hands the key to the auditor', () => {
|
||||
test('orchestratorReadsTheKeyAndPassesItInTheConfigBlock', () => {
|
||||
const src = fs.readFileSync(UI_REVIEW_PATH, 'utf8');
|
||||
assert.ok(src.includes(`config-get ${KEY}`), 'ui-review.md must read the key through gsd_run');
|
||||
// Lowercase placeholder, like the block's `{phase_dir}` / `{padded_phase}` siblings —
|
||||
// the block is a prompt template the orchestrator fills, not a bash heredoc.
|
||||
assert.ok(src.includes('interaction_capture: {interaction_capture}'),
|
||||
'the spawn prompt <config> block must carry interaction_capture');
|
||||
// Normalised to a literal true/false before it is handed down, so the
|
||||
// auditor's fence only ever compares against "true".
|
||||
assert.ok(src.includes('[ "$INTERACTION_CAPTURE" = "true" ] || INTERACTION_CAPTURE="false"'));
|
||||
});
|
||||
|
||||
test('auditorNeverReadsConfigItself', () => {
|
||||
// The auditor carries no gsd_run resolver; the key must arrive by value.
|
||||
const src = fs.readFileSync(AUDITOR_PATH, 'utf8');
|
||||
// Prose may NAME gsd_run (this section explains why it is absent); an invocation
|
||||
// or a resolver definition is what must not appear.
|
||||
assert.ok(!/gsd_run (query|runtime-identity)|gsd_run\(\)/.test(src),
|
||||
'gsd-ui-auditor.md must not grow a gsd_run dependency for this');
|
||||
assert.ok(src.includes('interaction_capture'), 'the auditor must name the <config> field it consumes');
|
||||
});
|
||||
});
|
||||
|
||||
describe('<screenshot_approach> keeps the two paths apart', () => {
|
||||
test('staticFenceIsPlaywrightOnlyAndInteractionFenceIsChromeDevtoolsOnly', () => {
|
||||
const { static: staticFence, interaction } = screenshotApproachFences();
|
||||
const staticText = staticFence.join('\n');
|
||||
const interactionText = interaction.join('\n');
|
||||
assert.ok(staticText.includes('npx playwright screenshot'), 'the static path stays on Playwright');
|
||||
assert.ok(!staticText.includes('chrome-devtools'),
|
||||
'the static fence must not reference the driver — key off means today\'s path, byte for byte');
|
||||
assert.ok(interactionText.includes('chrome-devtools'));
|
||||
assert.ok(!interactionText.includes('playwright'),
|
||||
'the interaction fence adds captures; it never replaces the Playwright ones');
|
||||
});
|
||||
|
||||
test('interactionFenceGatesOnTheHandedDownValueAndAChromeBinary', () => {
|
||||
const text = screenshotApproachFences().interaction.join('\n');
|
||||
assert.ok(text.includes('if [ "$INTERACTION_CAPTURE" != "true" ]'));
|
||||
assert.ok(text.includes('elif [ -z "$CHROME_BIN" ]'));
|
||||
assert.ok(text.includes('--isolated'), 'a throwaway profile, never the shared chrome-devtools-mcp one');
|
||||
assert.ok(text.includes('--workspace "$INTERACTION_DIR"'), 'the driver is confined to the capture directory');
|
||||
const code = screenshotApproachFences().interaction.filter((l) => !/^\s*#/.test(l)).join('\n');
|
||||
assert.ok(!code.includes('--allowUnrestrictedPaths'), 'deprecated in 1.9.0, and it lifts every path restriction');
|
||||
assert.ok(text.includes('--usageStatistics=false'));
|
||||
assert.ok(text.includes('--sessionId $CDT_SESSION'), 'a per-run daemon session, so audits never stop each other');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Behavioural: run the interaction fence under bash with a stub driver on PATH.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const HAS_BASH = (() => {
|
||||
const r = spawnSync('bash', ['-c', 'exit 0'], { encoding: 'utf8', timeout: PROBE_TIMEOUT_MS });
|
||||
return !r.error && r.status === 0;
|
||||
})();
|
||||
|
||||
/** Absolute paths of the coreutils the fence needs, so PATH can hold only the stubs. */
|
||||
function coreutilPaths() {
|
||||
const r = spawnSync('bash', ['-c', 'for c in sed head mkdir rm tr date sleep; do command -v "$c" || exit 1; done'],
|
||||
{ encoding: 'utf8', timeout: PROBE_TIMEOUT_MS });
|
||||
assert.equal(r.status, 0, `coreutils must resolve: ${r.stderr}`);
|
||||
const [sed, head, mkdir, rm, tr, date, sleep] = r.stdout.trim().split(/\r?\n/);
|
||||
return { sed, head, mkdir, rm, tr, date, sleep };
|
||||
}
|
||||
|
||||
const STUB_NPX = `#!/bin/sh
|
||||
# argv: -y -p chrome-devtools-mcp@<range> chrome-devtools --sessionId <hex-id> <cmd> [args...]
|
||||
printf '%s\\n' "$*" >> "$STUB_LOG"
|
||||
[ "$1" = "-y" ] && [ "$2" = "-p" ] && [ "$4" = "chrome-devtools" ] && [ "$5" = "--sessionId" ] || { echo "stub npx: unexpected argv: $*" >&2; exit 66; }
|
||||
case "$6" in *[!0-9a-fA-F-]*|"") echo "stub npx: sessionId not hex/dashes: $6" >&2; exit 68 ;; esac
|
||||
cmd="$7"
|
||||
shift 7
|
||||
# STUB_HANG=<verb>: that verb never returns — the fence's ceiling is what ends it. Deliberately a
|
||||
# CHILD holding stdout under this sh (the npx -> sh -> client shape), never an exec: a kill that
|
||||
# reaches only this pid leaves the child blocking the fence's $(...) capture.
|
||||
[ "\${STUB_HANG:-}" = "$cmd" ] && { "$STUB_SLEEP" 300; exit 0; }
|
||||
[ "\${STUB_ORPHAN:-}" = "$cmd" ] && { "$STUB_SLEEP" 300 & exit 0; }
|
||||
[ -n "\${STUB_DELAY:-}" ] && "$STUB_SLEEP" "$STUB_DELAY"
|
||||
case "$cmd" in
|
||||
resize_page)
|
||||
[ "\${STUB_FAIL_RESIZE:-0}" = 1 ] && exit 1
|
||||
exit 0 ;;
|
||||
start)
|
||||
[ "\${STUB_FAIL_START:-0}" = 1 ] && exit 1
|
||||
exit 0 ;;
|
||||
new_page)
|
||||
if [ "\${STUB_FAIL_NEWPAGE:-0}" = 1 ]; then printf '## Pages\\n1: about:blank\\n'; exit 0; fi
|
||||
nl='\\n'; [ "\${STUB_CRLF:-0}" = 1 ] && nl='\\r\\n'
|
||||
printf "## Pages\${nl}1: about:blank\${nl}2: %s [selected]\${nl}" "$1"
|
||||
exit "\${STUB_NEWPAGE_RC:-0}" ;;
|
||||
press_key)
|
||||
[ "\${STUB_FAIL_PRESS:-0}" = 1 ] && exit 1
|
||||
exit 0 ;;
|
||||
take_screenshot|take_snapshot)
|
||||
f=""
|
||||
while [ $# -gt 0 ]; do [ "$1" = "--filePath" ] && f="$2"; shift; done
|
||||
[ -n "$f" ] || { echo "stub: $cmd without --filePath" >&2; exit 67; }
|
||||
if [ "$cmd" = take_screenshot ] && [ "\${STUB_FAIL_SCREENSHOT:-0}" = 1 ]; then : > "$f"; exit 1; fi
|
||||
if [ "$cmd" = take_snapshot ] && [ "\${STUB_FAIL_SNAPSHOT:-0}" = 1 ]; then exit 1; fi
|
||||
printf 'STUB' > "$f"; exit 0 ;;
|
||||
*) exit 0 ;;
|
||||
esac
|
||||
`;
|
||||
|
||||
/**
|
||||
* Run the interaction fence. `opts.chrome` puts a stub google-chrome on PATH;
|
||||
* `opts.env` is merged over the minimal environment. Returns stdout, the
|
||||
* driver invocation log (one argv line per call) and the final status value.
|
||||
*/
|
||||
function runInteractionFence(t, opts = {}) {
|
||||
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ui-interaction-'));
|
||||
t.after(() => cleanup(tmp));
|
||||
const bin = path.join(tmp, 'bin');
|
||||
fs.mkdirSync(bin);
|
||||
fs.writeFileSync(path.join(bin, 'npx'), STUB_NPX, { mode: 0o755 });
|
||||
if (opts.chrome) fs.writeFileSync(path.join(bin, 'google-chrome'), '#!/bin/sh\nexit 0\n', { mode: 0o755 });
|
||||
const log = path.join(tmp, 'driver.log');
|
||||
fs.writeFileSync(log, '');
|
||||
const cu = coreutilPaths();
|
||||
// The fence's watchdog is an exec'd bash that resolves `sleep` through the EXPORTED PATH, where a
|
||||
// shell function is invisible — so PATH is exported below, and the stub dir carries `sleep` as a
|
||||
// portable exec-wrapper script (an exec keeps the pid, so a kill aimed at it reaches the real sleep).
|
||||
// `opts.brokenSleep`: that clock cannot launch (stands in for an msys fork failure under load).
|
||||
fs.writeFileSync(path.join(bin, 'sleep'),
|
||||
opts.brokenSleep ? '#!/bin/sh\nexit 1\n' : `#!/bin/sh\nexec ${JSON.stringify(cu.sleep)} "$@"\n`, { mode: 0o755 });
|
||||
// Functions are looked up before PATH on every platform, so the fence sees
|
||||
// real coreutils while PATH holds nothing but the stubs (#4176's harness note).
|
||||
const script = [
|
||||
'#!/bin/bash',
|
||||
`export PATH=${JSON.stringify(bin)}`,
|
||||
`sed() { ${JSON.stringify(cu.sed)} "$@"; }`,
|
||||
`head() { ${JSON.stringify(cu.head)} "$@"; }`,
|
||||
`mkdir() { ${JSON.stringify(cu.mkdir)} "$@"; }`,
|
||||
`rm() { ${JSON.stringify(cu.rm)} "$@"; }`,
|
||||
`tr() { ${JSON.stringify(cu.tr)} "$@"; }`,
|
||||
`date() { ${JSON.stringify(cu.date)} "$@"; }`,
|
||||
// Opt-in strict mode: an agent runner MAY execute the fence under errexit + pipefail,
|
||||
// and the unconditional stop must still be reached on a failed navigation.
|
||||
...(opts.strict ? ['set -e -o pipefail'] : []),
|
||||
// `opts.failAfter`: a bare failing command right after the first fence line containing
|
||||
// it — the future edit the EXIT trap exists for, since no shipped line fails bare today.
|
||||
// `opts.injectAfter`: {marker, line} — an arbitrary line after the first fence line
|
||||
// containing the marker (the subshell-copy control below).
|
||||
...screenshotApproachFences().interaction.flatMap((line) =>
|
||||
(opts.failAfter && line.includes(opts.failAfter)) ? [line, 'false']
|
||||
: (opts.injectAfter && line.includes(opts.injectAfter.marker)) ? [line, opts.injectAfter.line]
|
||||
: [line]),
|
||||
'printf "FINAL_STATUS=%s\\n" "$INTERACTION_STATUS"',
|
||||
'',
|
||||
].join('\n');
|
||||
const scriptPath = path.join(tmp, 'fence.sh');
|
||||
fs.writeFileSync(scriptPath, script);
|
||||
const env = {
|
||||
STUB_LOG: log,
|
||||
STUB_SLEEP: cu.sleep,
|
||||
HOME: tmp,
|
||||
TMPDIR: tmp,
|
||||
...(opts.env || {}),
|
||||
};
|
||||
const r = spawnSync('bash', [scriptPath], { encoding: 'utf8', timeout: FENCE_RUN_TIMEOUT_MS, env, cwd: tmp });
|
||||
const calls = fs.readFileSync(log, 'utf8').split(/\r?\n/).filter(Boolean);
|
||||
const status = (r.stdout.match(/^FINAL_STATUS=(.*)$/m) || [])[1];
|
||||
return { tmp, r, calls, status, stdout: r.stdout };
|
||||
}
|
||||
|
||||
function shotsDir(t) {
|
||||
const d = fs.mkdtempSync(path.join(os.tmpdir(), 'ui-shots-'));
|
||||
t.after(() => cleanup(d));
|
||||
return d;
|
||||
}
|
||||
|
||||
describe('interaction fence (bash, stub driver)', { skip: HAS_BASH ? false : 'bash not on PATH' }, () => {
|
||||
test('keyOffInvokesNothingAndSaysSo', (t) => {
|
||||
const dir = shotsDir(t);
|
||||
const out = runInteractionFence(t, { chrome: true, env: { INTERACTION_CAPTURE: 'false', SCREENSHOT_DIR: dir } });
|
||||
assert.equal(out.r.status, 0, out.r.stderr);
|
||||
assert.deepEqual(out.calls, [], 'the driver must never run with the key off');
|
||||
assert.match(out.stdout, /Interaction capture: off/);
|
||||
assert.equal(out.status, 'off');
|
||||
assert.ok(!fs.existsSync(path.join(dir, 'interaction')), 'no directory is created with the key off');
|
||||
});
|
||||
|
||||
test('absentValueMeansOff', (t) => {
|
||||
const out = runInteractionFence(t, { chrome: true, env: { SCREENSHOT_DIR: shotsDir(t) } });
|
||||
assert.deepEqual(out.calls, []);
|
||||
assert.equal(out.status, 'off');
|
||||
});
|
||||
|
||||
test('noDevServerSkipsBeforeTouchingTheDriver', (t) => {
|
||||
// The static block sets SCREENSHOT_DIR only when it reached a dev server.
|
||||
const out = runInteractionFence(t, { chrome: true, env: { INTERACTION_CAPTURE: 'true' } });
|
||||
assert.deepEqual(out.calls, []);
|
||||
assert.equal(out.status, 'skipped (no dev server reached)');
|
||||
assert.match(out.stdout, /reached no dev server/);
|
||||
});
|
||||
|
||||
const HOST_HAS_FIXED_PATH_CHROME =
|
||||
fs.existsSync('/Applications/Google Chrome.app/Contents/MacOS/Google Chrome') ||
|
||||
(process.env.PROGRAMFILES && fs.existsSync(path.join(process.env.PROGRAMFILES, 'Google', 'Chrome', 'Application', 'chrome.exe')));
|
||||
|
||||
test('noChromeSkipsWithTheRemedyAndInvokesNothing', {
|
||||
skip: HOST_HAS_FIXED_PATH_CHROME ? 'host has Chrome at a fixed install path the fence probes' : false,
|
||||
}, (t) => {
|
||||
const dir = shotsDir(t);
|
||||
const out = runInteractionFence(t, { chrome: false, env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: dir } });
|
||||
assert.deepEqual(out.calls, [], 'no Chrome means the driver is never fetched or started');
|
||||
assert.equal(out.status, 'skipped (no Chrome binary resolved)');
|
||||
assert.match(out.stdout, /set CHROME_BIN/);
|
||||
});
|
||||
|
||||
test('chromeBinOverrideWinsOverDiscovery', (t) => {
|
||||
const dir = shotsDir(t);
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: false,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: dir, CHROME_BIN: '/opt/custom/chrome', DEV_URL: 'http://localhost:5173' },
|
||||
});
|
||||
const start = out.calls.find((c) => c.includes(' start '));
|
||||
assert.ok(start, `start must run: ${out.calls.join(' | ')}`);
|
||||
assert.ok(start.includes('-e /opt/custom/chrome'), start);
|
||||
});
|
||||
|
||||
test('happyPathDrivesTheLifecycleAndCountsOnlyRealFiles', (t) => {
|
||||
const dir = shotsDir(t);
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: dir, DEV_URL: 'http://localhost:3999/' },
|
||||
});
|
||||
assert.equal(out.r.status, 0, out.r.stderr);
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.equal(verbs[0], 'start', 'the daemon starts first');
|
||||
assert.equal(verbs[verbs.length - 1], 'stop', 'and is stopped last — it does not self-reap');
|
||||
// The workspace and every --filePath are the same path the fence joins with a literal `/`
|
||||
// (relative when SCREENSHOT_DIR is — dialect-free on Git Bash, where an absolute msys path
|
||||
// would mean nothing to a Windows-native daemon).
|
||||
assert.ok(out.calls[0].includes(`--isolated --workspace ${dir}/interaction --usageStatistics=false`), out.calls[0]);
|
||||
assert.ok(!out.calls[0].includes('--allowUnrestrictedPaths'), out.calls[0]);
|
||||
assert.ok(out.calls[0].includes('-p chrome-devtools-mcp@^1.9.0 '), 'documented floor by default — --workspace needs 1.9.0');
|
||||
// Every file the driver is asked to write lies inside the workspace it was confined to.
|
||||
for (const c of out.calls.filter((x) => x.includes('--filePath '))) {
|
||||
const target = c.split('--filePath ')[1].split(' ')[0];
|
||||
assert.ok(target.startsWith(`${dir}/interaction/`), `write outside the workspace: ${c}`);
|
||||
}
|
||||
assert.equal(verbs.filter((v) => v === 'stop').length, 1, 'stop is issued exactly once — never from a subshell copy');
|
||||
assert.ok(out.calls.some((c) => c.includes(' new_page http://localhost:3999/ --timeout 30000')), 'navigates to the resolved dev URL, time-bounded');
|
||||
const sessions = new Set(out.calls.map((c) => c.split(' ')[5]));
|
||||
assert.equal(sessions.size, 1, `every call must address one daemon session: ${[...sessions].join(',')}`);
|
||||
assert.match([...sessions][0], /^[0-9]+-[0-9]+-[0-9]+$/,
|
||||
'epoch-BASHPID-RANDOM, every part present — the CLI accepts hex and dashes only');
|
||||
// Every later command addresses the page new_page marked [selected].
|
||||
for (const c of out.calls.filter((x) => / (resize_page|take_snapshot|take_screenshot|press_key|list_console_messages) /.test(x))) {
|
||||
assert.ok(/ (resize_page|take_snapshot|take_screenshot|press_key|list_console_messages) 2( |$)/.test(c), `pageId 2 expected: ${c}`);
|
||||
}
|
||||
assert.ok(out.calls.some((c) => c.includes(' resize_page 2 1440 900')));
|
||||
assert.ok(out.calls.some((c) => c.includes(' press_key 2 Tab')), 'the focus-ring interaction always runs');
|
||||
const idir = path.join(dir, 'interaction');
|
||||
for (const f of ['baseline.png', 'focus-first.png', 'snapshot.txt', 'console.txt']) {
|
||||
assert.ok(fs.existsSync(path.join(idir, f)), `${f} must exist`);
|
||||
}
|
||||
// The fence joins with a literal `/` ("$SCREENSHOT_DIR/interaction"); path.join would
|
||||
// put a backslash there on Windows and the strings would differ by separator alone.
|
||||
assert.equal(out.status, `captured (2 state(s), 0 failed) in ${dir}/interaction`);
|
||||
});
|
||||
|
||||
test('versionOverrideFlowsIntoTheNpxSpec', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), CHROME_DEVTOOLS_MCP_VERSION: '9.9.9' },
|
||||
});
|
||||
assert.ok(out.calls[0].includes('-p chrome-devtools-mcp@9.9.9 '), out.calls[0]);
|
||||
});
|
||||
|
||||
test('devUrlDefaultsToTheStaticBlocksPort', (t) => {
|
||||
const out = runInteractionFence(t, { chrome: true, env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t) } });
|
||||
assert.ok(out.calls.some((c) => c.includes(' new_page http://localhost:3000')), out.calls.join(' | '));
|
||||
});
|
||||
|
||||
test('failedCaptureIsNotCountedLeavesNoFileAndStillStops', (t) => {
|
||||
const dir = shotsDir(t);
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: dir, STUB_FAIL_SCREENSHOT: '1' },
|
||||
});
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.equal(verbs[verbs.length - 1], 'stop');
|
||||
assert.equal(out.status, 'not captured (driver or capture failure)');
|
||||
assert.ok(!fs.existsSync(path.join(dir, 'interaction', 'baseline.png')), 'a zero-byte capture is removed, not counted');
|
||||
assert.match(out.stdout, /interaction capture FAILED: baseline/);
|
||||
});
|
||||
|
||||
test('snapshotFailureCountsAsAFailedStepAndRemovesAStaleSnapshot', (t) => {
|
||||
const dir = shotsDir(t);
|
||||
// A reused directory can hold a snapshot from an earlier run; its uids belong to a
|
||||
// page this run never saw.
|
||||
fs.mkdirSync(path.join(dir, 'interaction'));
|
||||
fs.writeFileSync(path.join(dir, 'interaction', 'snapshot.txt'), 'uid=9_9 stale');
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: dir, STUB_FAIL_SNAPSHOT: '1' },
|
||||
});
|
||||
assert.match(out.stdout, /interaction step FAILED: take_snapshot/);
|
||||
assert.ok(!fs.existsSync(path.join(dir, 'interaction', 'snapshot.txt')), 'stale uids must not survive a failed snapshot');
|
||||
assert.match(out.status, /^captured \(2 state\(s\), 1 failed\)/, 'two clean screenshots must not read as 0 failed');
|
||||
});
|
||||
|
||||
test('crlfDriverOutputStillYieldsThePageId', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_CRLF: '1' },
|
||||
});
|
||||
assert.ok(out.calls.some((c) => c.includes(' resize_page 2 1440 900')), `page id must parse from CRLF output: ${out.calls.join(' | ')}`);
|
||||
assert.match(out.status, /^captured \(2 state\(s\), 0 failed\)/);
|
||||
});
|
||||
|
||||
test('newPageThatPrintsAPageLineButExitsNonZeroIsAFailedNavigation', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_NEWPAGE_RC: '7' },
|
||||
});
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.deepEqual(verbs, ['start', 'new_page', 'stop'], 'partial output must not be mistaken for a page id');
|
||||
assert.match(out.stdout, /new_page FAILED/);
|
||||
assert.equal(out.status, 'not captured (driver or capture failure)');
|
||||
});
|
||||
|
||||
test('underErrexitAndPipefailAFailedNewPageStillReachesStop', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
strict: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_NEWPAGE_RC: '7' },
|
||||
});
|
||||
assert.equal(out.r.status, 0, `the block must not abort: ${out.r.stderr}`);
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.deepEqual(verbs, ['start', 'new_page', 'stop']);
|
||||
});
|
||||
|
||||
test('underErrexitAndPipefailTheHappyPathIsUnchanged', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
strict: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t) },
|
||||
});
|
||||
assert.equal(out.r.status, 0, out.r.stderr);
|
||||
assert.match(out.status, /^captured \(2 state\(s\), 0 failed\)/);
|
||||
});
|
||||
|
||||
test('pressKeyFailureCountsAndSkipsTheFocusCapture', (t) => {
|
||||
const dir = shotsDir(t);
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: dir, STUB_FAIL_PRESS: '1' },
|
||||
});
|
||||
assert.match(out.stdout, /interaction step FAILED: press_key Tab/);
|
||||
assert.ok(!fs.existsSync(path.join(dir, 'interaction', 'focus-first.png')));
|
||||
assert.match(out.status, /^captured \(1 state\(s\), 1 failed\)/);
|
||||
});
|
||||
|
||||
test('newPageWithoutASelectedPageStopsTheDaemonAndCapturesNothing', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_FAIL_NEWPAGE: '1' },
|
||||
});
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.deepEqual(verbs, ['start', 'new_page', 'stop'], verbs.join(','));
|
||||
assert.match(out.stdout, /new_page FAILED/);
|
||||
assert.equal(out.status, 'not captured (driver or capture failure)');
|
||||
});
|
||||
|
||||
test('failedStartNeverIssuesStop', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_FAIL_START: '1' },
|
||||
});
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.deepEqual(verbs, ['start'], 'stop only follows a start that succeeded');
|
||||
assert.match(out.stdout, /start FAILED/);
|
||||
assert.equal(out.status, 'not captured (driver or capture failure)');
|
||||
});
|
||||
|
||||
test('everyDriverInvocationGoesThroughTheBoundedWrapper', () => {
|
||||
// Static: the only bare `$CDT` is the wrapper's own spawn; every call site names a ceiling.
|
||||
const lines = screenshotApproachFences().interaction;
|
||||
const bare = lines.filter((l) => /\$CDT /.test(l) && !/\$CDT "\$@" &/.test(l) && !/^\s*#/.test(l));
|
||||
assert.deepEqual(bare, [], `unbounded driver call(s): ${bare.join(' | ')}`);
|
||||
const calls = lines.filter((l) => /(^|[\s(!])cdt /.test(l) && !/^\s*#/.test(l) && !/^\s*cdt\(\)/.test(l));
|
||||
assert.ok(calls.length >= 7, `expected the seven driver verbs plus stop through the wrapper, saw ${calls.length}`);
|
||||
for (const c of calls) assert.match(c, /cdt "\$CDT_T_(START|STEP)" /, `call without a ceiling: ${c}`);
|
||||
});
|
||||
|
||||
test('aHungStartIsKilledAtItsCeilingAndReportedAsAFailedStart', (t) => {
|
||||
const started = Date.now();
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_HANG: 'start', CHROME_DEVTOOLS_START_TIMEOUT: '1' },
|
||||
});
|
||||
// 1 s ceiling + 2 s KILL grace, with slack — never the stub's 300 s or the harness's 30 s cap.
|
||||
assert.ok(Date.now() - started < 8000, `the ceiling, not the stub, ended the call (${Date.now() - started} ms)`);
|
||||
assert.deepEqual(out.calls.map((c) => c.split(' ')[6]), ['start'], 'a start that never returned is a failed start: no stop');
|
||||
assert.match(out.stdout, /start FAILED/);
|
||||
assert.equal(out.status, 'not captured (driver or capture failure)');
|
||||
});
|
||||
|
||||
test('aHungCaptureIsKilledAtItsCeilingAndStopIsStillReached', (t) => {
|
||||
const dir = shotsDir(t);
|
||||
const started = Date.now();
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: dir, STUB_HANG: 'take_screenshot', CHROME_DEVTOOLS_STEP_TIMEOUT: '1' },
|
||||
});
|
||||
// two hung captures: 2 x (1 s ceiling + 2 s KILL grace), with slack
|
||||
assert.ok(Date.now() - started < 12000, `two hung captures at a 1 s ceiling (${Date.now() - started} ms)`);
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.equal(verbs[verbs.length - 1], 'stop', 'a timed-out step still reaches the stop');
|
||||
assert.match(out.stdout, /interaction capture FAILED: baseline/);
|
||||
assert.ok(!fs.existsSync(path.join(dir, 'interaction', 'baseline.png')));
|
||||
assert.equal(out.status, 'not captured (driver or capture failure)');
|
||||
});
|
||||
|
||||
test('aHungNewPageWhoseChildHoldsStdoutIsStillCutOffAtTheCeiling', (t) => {
|
||||
// new_page is the one verb whose output the fence captures with $(...): the substitution ends
|
||||
// only when every writer closes stdout, so killing the driver's parent alone would block here
|
||||
// for the stub's full 300 s (measured on the pre-fix wrapper against a real npx tree).
|
||||
const started = Date.now();
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_HANG: 'new_page', CHROME_DEVTOOLS_STEP_TIMEOUT: '1' },
|
||||
});
|
||||
assert.ok(Date.now() - started < 8000, `the process-group kill must release the capture (${Date.now() - started} ms)`);
|
||||
assert.deepEqual(out.calls.map((c) => c.split(' ')[6]), ['start', 'new_page', 'stop'], out.calls.join(' | '));
|
||||
assert.match(out.stdout, /new_page FAILED/);
|
||||
assert.equal(out.status, 'not captured (driver or capture failure)');
|
||||
});
|
||||
|
||||
test('aNewPageWhoseLeaderExitsWhileAChildHoldsStdoutIsStillCutOffAtTheCeiling', (t) => {
|
||||
// The driver's leader exits at once but leaves a child holding the $(...) pipe: the
|
||||
// substitution stays open-ended unless the watchdog keeps watching the process GROUP
|
||||
// rather than the leader pid it was handed. Negative-controlled: a leader-pid poll stands
|
||||
// down the moment the leader is gone and this blocks for the stub's full 300 s.
|
||||
const started = Date.now();
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_ORPHAN: 'new_page', CHROME_DEVTOOLS_STEP_TIMEOUT: '1' },
|
||||
});
|
||||
assert.ok(Date.now() - started < 8000, `the group must be watched, not the leader (${Date.now() - started} ms)`);
|
||||
assert.deepEqual(out.calls.map((c) => c.split(' ')[6]), ['start', 'new_page', 'stop'], out.calls.join(' | '));
|
||||
assert.match(out.stdout, /new_page FAILED/);
|
||||
assert.equal(out.status, 'not captured (driver or capture failure)');
|
||||
});
|
||||
|
||||
test('aWatchdogWhoseClockCannotLaunchStandsDownInsteadOfKillingTheStep', (t) => {
|
||||
// The watchdog's clock is `sleep`. If it cannot launch (an msys fork failure under load, an
|
||||
// absent binary), the watchdog must stand down — never fire at once and kill a healthy driver
|
||||
// call. The stub takes a real driver call's shape (a few hundred ms, not the instant exit a
|
||||
// premature kill would miss). Negative-controlled: a `sleep & wait $!` watchdog (the trap form
|
||||
// this replaced) fires at once here and `start` reads as failed.
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
brokenSleep: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_DELAY: '0.3' },
|
||||
});
|
||||
assert.equal(out.r.status, 0, out.r.stderr);
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.deepEqual([verbs[0], verbs[verbs.length - 1]], ['start', 'stop'], verbs.join(','));
|
||||
assert.match(out.status, /^captured \(2 state\(s\), 0 failed\)/, 'a watchdog without a clock must not kill the step');
|
||||
});
|
||||
|
||||
test('underErrexitAnAbortAfterStartStillReachesStopThroughTheTrap', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
strict: true,
|
||||
failAfter: 'ishot baseline',
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t) },
|
||||
});
|
||||
assert.notEqual(out.r.status, 0, 'the injected bare failure must abort the block under errexit');
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.equal(verbs[verbs.length - 1], 'stop', `the EXIT trap owes the stop: ${verbs.join(',')}`);
|
||||
assert.equal(verbs.filter((v) => v === 'stop').length, 1, 'once — the flag makes the trap a no-op after an in-order stop');
|
||||
});
|
||||
|
||||
test('aSubshellCopyOfTheFenceStateNeverIssuesAStop', (t) => {
|
||||
// A driver call can run in a subshell that inherits CDT_STARTED=1 (a $(...) capture).
|
||||
// A cdt_stop reached in one of them — CI's ubuntu job hit that through a timing race — must be
|
||||
// inert: only the shell that installed the EXIT trap may issue the stop. Deterministic stand-in
|
||||
// for the race: call cdt_stop from a subshell right after the trap is installed.
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
injectAfter: { marker: 'trap cdt_stop EXIT', line: '( cdt_stop )' },
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t) },
|
||||
});
|
||||
assert.equal(out.r.status, 0, out.r.stderr);
|
||||
const verbs = out.calls.map((c) => c.split(' ')[6]);
|
||||
assert.equal(verbs.filter((v) => v === 'stop').length, 1, `one stop, from the installing shell only: ${verbs.join(',')}`);
|
||||
assert.equal(verbs[1], 'new_page', 'the subshell call must not have stopped the daemon before the capture ran');
|
||||
assert.match(out.status, /^captured \(2 state\(s\), 0 failed\)/);
|
||||
});
|
||||
|
||||
test('aFailedResizeIsCountedAndTheCapturesStillRun', (t) => {
|
||||
const out = runInteractionFence(t, {
|
||||
chrome: true,
|
||||
strict: true,
|
||||
env: { INTERACTION_CAPTURE: 'true', SCREENSHOT_DIR: shotsDir(t), STUB_FAIL_RESIZE: '1' },
|
||||
});
|
||||
assert.equal(out.r.status, 0, out.r.stderr);
|
||||
assert.match(out.stdout, /interaction step FAILED: resize_page/);
|
||||
assert.match(out.status, /^captured \(2 state\(s\), 1 failed\)/, 'the resize is a step, not the audit');
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// The gitignore gate: run it under bash against a fresh and a pre-existing file.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function gitignoreGateFence() {
|
||||
const lines = splitLines(fs.readFileSync(AUDITOR_PATH, 'utf8'));
|
||||
const out = [];
|
||||
let inSection = false;
|
||||
let inFence = false;
|
||||
for (const line of lines) {
|
||||
if (!inSection) { if (line.includes('<gitignore_gate>')) inSection = true; continue; }
|
||||
if (line.includes('</gitignore_gate>')) break;
|
||||
if (!inFence) { if (line.trim() === `${FENCE}bash`) inFence = true; continue; }
|
||||
if (line.trim() === FENCE) break;
|
||||
out.push(line);
|
||||
}
|
||||
assert.ok(out.length > 0, '<gitignore_gate> must carry a bash fence');
|
||||
return out;
|
||||
}
|
||||
|
||||
function runGitignoreGate(t, seed) {
|
||||
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ui-gitignore-'));
|
||||
t.after(() => cleanup(tmp));
|
||||
if (seed !== undefined) {
|
||||
fs.mkdirSync(path.join(tmp, '.planning', 'ui-reviews'), { recursive: true });
|
||||
fs.writeFileSync(path.join(tmp, '.planning', 'ui-reviews', '.gitignore'), seed);
|
||||
}
|
||||
const script = path.join(tmp, 'gate.sh');
|
||||
fs.writeFileSync(script, ['#!/bin/bash', 'set -e', ...gitignoreGateFence(), ''].join('\n'));
|
||||
const run = () => spawnSync('bash', [script], { encoding: 'utf8', timeout: FENCE_RUN_TIMEOUT_MS, cwd: tmp });
|
||||
const read = () => splitLines(fs.readFileSync(path.join(tmp, '.planning', 'ui-reviews', '.gitignore'), 'utf8')).filter(Boolean);
|
||||
return { tmp, run, read };
|
||||
}
|
||||
|
||||
describe('gitignore gate (bash)', { skip: HAS_BASH ? false : 'bash not on PATH' }, () => {
|
||||
const REQUIRED = ['*.png', '*.webp', '*.jpg', '*.jpeg', '*.gif', '*.bmp', '*.tiff', 'interaction/'];
|
||||
|
||||
test('aFreshFileCoversTheImagesAndTheCaptureDirectory', (t) => {
|
||||
const g = runGitignoreGate(t);
|
||||
const r = g.run();
|
||||
assert.equal(r.status, 0, r.stderr);
|
||||
assert.match(r.stdout, /Created \.planning\/ui-reviews\/\.gitignore/);
|
||||
const lines = g.read();
|
||||
for (const p of REQUIRED) assert.ok(lines.includes(p), `${p} must be ignored`);
|
||||
assert.ok(lines.includes('interaction/'), 'snapshot.txt and console.txt are covered by the directory, not by an extension');
|
||||
});
|
||||
|
||||
test('anExistingImageOnlyFileGainsTheCaptureDirectoryAndKeepsItsOwnLines', (t) => {
|
||||
// The shape every project that ran an audit before interaction capture existed has on disk.
|
||||
const seed = '# Screenshot files — never commit binary assets\n*.png\n*.webp\n*.jpg\n*.jpeg\n*.gif\n*.bmp\n*.tiff\n';
|
||||
const g = runGitignoreGate(t, seed);
|
||||
const r = g.run();
|
||||
assert.equal(r.status, 0, r.stderr);
|
||||
assert.doesNotMatch(r.stdout, /Created/, 'an existing file is upgraded in place, never recreated');
|
||||
const lines = g.read();
|
||||
assert.equal(lines[0], '# Screenshot files — never commit binary assets', 'the user\'s file keeps its own header');
|
||||
assert.ok(lines.includes('interaction/'), 'a pre-existing file must not stay image-only');
|
||||
assert.equal(lines.filter((l) => l === '*.png').length, 1, 'present patterns are not duplicated');
|
||||
});
|
||||
|
||||
test('theGateIsIdempotent', (t) => {
|
||||
const g = runGitignoreGate(t);
|
||||
assert.equal(g.run().status, 0);
|
||||
const once = g.read();
|
||||
assert.equal(g.run().status, 0);
|
||||
assert.deepEqual(g.read(), once, 'a second run appends nothing');
|
||||
});
|
||||
});
|
||||
|
||||
describe('documentation', () => {
|
||||
test('configurationMdDocumentsTheKeyAsDefaultOff', () => {
|
||||
const row = splitLines(fs.readFileSync(DOCS_CONFIG_PATH, 'utf8'))
|
||||
.find((l) => l.startsWith(`| \`${KEY}\``));
|
||||
assert.ok(row, `docs/CONFIGURATION.md must carry a row for ${KEY}`);
|
||||
assert.ok(row.includes('| boolean |'), row);
|
||||
assert.ok(row.includes('| `false` |'), row);
|
||||
assert.ok(row.includes('enable-ui-interaction-capture.md'), 'the row must link the how-to');
|
||||
});
|
||||
|
||||
test('howToExistsAndNamesBothOffSwitches', () => {
|
||||
const src = fs.readFileSync(HOWTO_PATH, 'utf8');
|
||||
assert.ok(src.includes(`config-set ${KEY} true`));
|
||||
assert.ok(src.includes(`config-set ${KEY} false`));
|
||||
assert.ok(src.includes('CHROME_BIN'));
|
||||
});
|
||||
|
||||
test('howToNamesTheFloorTheCeilingsAndTheGitignoreCoverage', () => {
|
||||
const src = fs.readFileSync(HOWTO_PATH, 'utf8');
|
||||
assert.ok(src.includes('`^1.9.0`'), 'the floor the fence resolves by default');
|
||||
assert.ok(!src.includes('^1.8.0'), 'no stale floor');
|
||||
assert.ok(src.includes('CHROME_DEVTOOLS_START_TIMEOUT'));
|
||||
assert.ok(src.includes('CHROME_DEVTOOLS_STEP_TIMEOUT'));
|
||||
assert.ok(src.includes('`interaction/`'), 'the directory the gitignore gate covers');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user