From 67b064d53407fa07d0a06997210c33de29dcb234 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?T=C3=82CHES?= Date: Wed, 21 Jan 2026 10:52:30 -0600 Subject: [PATCH] Reduce manual verification in checkpoint system (#220) * docs: enforce automation-first checkpoint verification Checkpoints should never ask users to run CLI commands that Claude Code can execute. This update reinforces the automation-first principle: Key changes: - Add golden rules: Claude runs CLI, users only visit URLs - Add dev server automation patterns (start before checkpoint) - Add environment variable CLI patterns (Convex, Vercel, etc.) - Add anti-patterns: asking user to run npm, add dashboard env vars - Update all examples to show Claude starting servers - Add comprehensive "Never Ask Users To" and "Users Only Do" lists - Update gsd-executor with pre-checkpoint automation requirements The core principle: if Claude CAN automate it, Claude MUST automate it. Users only do what requires human judgment (visual verification, UX). * refactor: DRY checkpoint automation with server lifecycle and error handling Changes: - checkpoints.md is now single source of truth for automation-first patterns - Added server lifecycle protocol (start, port conflicts, cleanup) - Added CLI installation handling (auto-install matrix) - Added pre-checkpoint failure handling (fix before checkpoint) - Removed ~93 lines of duplication from verification-patterns.md - Replaced inline examples in phase-prompt.md with references - Slimmed gsd-executor.md checkpoint section to reference checkpoints.md Net effect: -23 lines while adding 3 new capabilities (server lifecycle, CLI install, error handling). Single place to update automation patterns. Co-Authored-By: Claude Opus 4.5 --------- Co-authored-by: Claude --- agents/gsd-executor.md | 15 + get-shit-done/references/checkpoints.md | 346 ++++++++++++++++-- .../references/verification-patterns.md | 17 + get-shit-done/templates/phase-prompt.md | 45 +-- 4 files changed, 368 insertions(+), 55 deletions(-) diff --git a/agents/gsd-executor.md b/agents/gsd-executor.md index e65a96b96..a86e76e45 100644 --- a/agents/gsd-executor.md +++ b/agents/gsd-executor.md @@ -347,6 +347,21 @@ Type "done" when authenticated. + +**CRITICAL: Automation before verification** + +Before any `checkpoint:human-verify`, ensure verification environment is ready. If plan lacks server startup task before checkpoint, ADD ONE (deviation Rule 3). + +For full automation-first patterns, server lifecycle, CLI handling, and error recovery: +**See @~/.claude/get-shit-done/references/checkpoints.md** + +**Quick reference:** +- Users NEVER run CLI commands - Claude does all automation +- Users ONLY visit URLs, click UI, evaluate visuals, provide secrets +- Claude starts servers, seeds databases, configures env vars + +--- + When encountering `type="checkpoint:*"`: **STOP immediately.** Do not continue to next task. diff --git a/get-shit-done/references/checkpoints.md b/get-shit-done/references/checkpoints.md index b678ea874..89af06584 100644 --- a/get-shit-done/references/checkpoints.md +++ b/get-shit-done/references/checkpoints.md @@ -2,6 +2,12 @@ Plans execute autonomously. Checkpoints formalize the interaction points where human verification or decisions are needed. **Core principle:** Claude automates everything with CLI/API. Checkpoints are for verification and decisions, not manual work. + +**Golden rules:** +1. **If Claude can run it, Claude runs it** - Never ask user to execute CLI commands, start servers, or run builds +2. **Claude sets up the verification environment** - Start dev servers, seed databases, configure env vars +3. **User only does what requires human judgment** - Visual checks, UX evaluation, "does this feel right?" +4. **Secrets come from user, automation comes from Claude** - Ask for API keys, then Claude uses them via CLI @@ -67,20 +73,28 @@ Plans execute autonomously. Checkpoints formalize the interaction points where h Dashboard component builds without errors + + Start dev server for verification + Run `npm run dev` in background, wait for "ready" message, capture port + curl http://localhost:3000 returns 200 + Dev server running at http://localhost:3000 + + - Responsive dashboard layout at /dashboard + Responsive dashboard layout - dev server running at http://localhost:3000 - 1. Run: npm run dev - 2. Visit: http://localhost:3000/dashboard - 3. Desktop (>1024px): Verify sidebar left, content right, header top - 4. Tablet (768px): Verify sidebar collapses to hamburger - 5. Mobile (375px): Verify single column, bottom nav - 6. Check: No layout shift, no horizontal scroll + Visit http://localhost:3000/dashboard and verify: + 1. Desktop (>1024px): Sidebar left, content right, header top + 2. Tablet (768px): Sidebar collapses to hamburger menu + 3. Mobile (375px): Single column layout, bottom nav appears + 4. No layout shift or horizontal scroll at any size Type "approved" or describe layout issues ``` +**Key pattern:** Claude starts the dev server BEFORE the checkpoint. User only needs to visit the URL. + **Example: Xcode Build** ```xml @@ -466,6 +480,8 @@ Task 3 complete. Continuing to task 4... **The rule:** If it has CLI/API, Claude does it. Never ask human to perform automatable work. +## Service CLI Reference + | Service | CLI/API | Key Commands | Auth Gate | |---------|---------|--------------|-----------| | Vercel | `vercel` | `--yes`, `env add`, `--prod`, `ls` | `vercel login` | @@ -476,13 +492,174 @@ Task 3 complete. Continuing to task 4... | Upstash | `upstash` | `redis create`, `redis get` | `upstash auth login` | | PlanetScale | `pscale` | `database create`, `branch create` | `pscale auth login` | | GitHub | `gh` | `repo create`, `pr create`, `secret set` | `gh auth login` | -| Node | `npm`/`pnpm` | `install`, `run build`, `test` | N/A | +| Node | `npm`/`pnpm` | `install`, `run build`, `test`, `run dev` | N/A | | Xcode | `xcodebuild` | `-project`, `-scheme`, `build`, `test` | N/A | -| Convex | `npx convex` | `dev`, `deploy`, `import` | `npx convex login` | +| Convex | `npx convex` | `dev`, `deploy`, `env set`, `env get` | `npx convex login` | + +## Environment Variable Automation **Env files:** Use Write/Edit tools. Never ask human to create .env manually. -**Quick reference:** +**Dashboard env vars via CLI:** + +| Platform | CLI Command | Example | +|----------|-------------|---------| +| Convex | `npx convex env set` | `npx convex env set OPENAI_API_KEY sk-...` | +| Vercel | `vercel env add` | `vercel env add STRIPE_KEY production` | +| Railway | `railway variables set` | `railway variables set API_KEY=value` | +| Fly | `fly secrets set` | `fly secrets set DATABASE_URL=...` | +| Supabase | `supabase secrets set` | `supabase secrets set MY_SECRET=value` | + +**Pattern for secret collection:** +```xml + + + Add OPENAI_API_KEY to Convex dashboard + Go to dashboard.convex.dev → Settings → Environment Variables → Add + + + + + Provide your OpenAI API key + + I need your OpenAI API key to configure the Convex backend. + Get it from: https://platform.openai.com/api-keys + Paste the key (starts with sk-) + + I'll add it via `npx convex env set` and verify it's configured + Paste your API key + + + + Configure OpenAI key in Convex + Run `npx convex env set OPENAI_API_KEY {user-provided-key}` + `npx convex env get OPENAI_API_KEY` returns the key (masked) + +``` + +## Dev Server Automation + +**Claude starts servers, user visits URLs:** + +| Framework | Start Command | Ready Signal | Default URL | +|-----------|---------------|--------------|-------------| +| Next.js | `npm run dev` | "Ready in" or "started server" | http://localhost:3000 | +| Vite | `npm run dev` | "ready in" | http://localhost:5173 | +| Convex | `npx convex dev` | "Convex functions ready" | N/A (backend only) | +| Express | `npm start` | "listening on port" | http://localhost:3000 | +| Django | `python manage.py runserver` | "Starting development server" | http://localhost:8000 | + +### Server Lifecycle Protocol + +**Starting servers:** +```bash +# Run in background, capture PID for cleanup +npm run dev & +DEV_SERVER_PID=$! + +# Wait for ready signal (max 30s) +timeout 30 bash -c 'until curl -s localhost:3000 > /dev/null 2>&1; do sleep 1; done' +``` + +**Port conflicts:** +If default port is in use, check what's running and either: +1. Kill the existing process if it's stale: `lsof -ti:3000 | xargs kill` +2. Use alternate port: `npm run dev -- --port 3001` + +**Server stays running** for the duration of the checkpoint. After user approves, server continues running for subsequent tasks. Only kill explicitly if: +- Plan is complete and no more verification needed +- Switching to production deployment +- Port needed for different service + +**Pattern:** +```xml + + + Start dev server + Run `npm run dev` in background, wait for ready signal + curl http://localhost:3000 returns 200 + Dev server running + + + + + Feature X - dev server running at http://localhost:3000 + + Visit http://localhost:3000/feature and verify: + 1. [Visual check 1] + 2. [Visual check 2] + + +``` + +## CLI Installation Handling + +**When a required CLI is not installed:** + +| CLI | Auto-install? | Command | +|-----|---------------|---------| +| npm/pnpm/yarn | No - ask user | User chooses package manager | +| vercel | Yes | `npm i -g vercel` | +| gh (GitHub) | Yes | `brew install gh` (macOS) or `apt install gh` (Linux) | +| stripe | Yes | `npm i -g stripe` | +| supabase | Yes | `npm i -g supabase` | +| convex | No - use npx | `npx convex` (no install needed) | +| fly | Yes | `brew install flyctl` or curl installer | +| railway | Yes | `npm i -g @railway/cli` | + +**Protocol:** +1. Try the command +2. If "command not found", check if auto-installable +3. If yes: install silently, retry command +4. If no: create checkpoint asking user to install + +```xml + + + Install Vercel CLI + Run `npm i -g vercel` + `vercel --version` succeeds + Vercel CLI installed + +``` + +## Pre-Checkpoint Automation Failures + +**When setup fails before checkpoint:** + +| Failure | Response | +|---------|----------| +| Server won't start | Check error output, fix issue, retry (don't proceed to checkpoint) | +| Port in use | Kill stale process or use alternate port | +| Missing dependency | Run `npm install`, retry | +| Build error | Fix the error first (this is a bug, not a checkpoint issue) | +| Auth error | Create auth gate checkpoint | +| Network timeout | Retry with backoff, then checkpoint if persistent | + +**Key principle:** Never present a checkpoint with broken verification environment. If `curl localhost:3000` fails, don't ask user to "visit localhost:3000". + +```xml + + + Dashboard (server failed to start) + Visit http://localhost:3000... + + + + + Fix server startup issue + Investigate error, fix root cause, restart server + curl http://localhost:3000 returns 200 + Server running correctly + + + + Dashboard - server running at http://localhost:3000 + Visit http://localhost:3000/dashboard... + +``` + +## Quick Reference | Action | Automatable? | Claude does it? | |--------|--------------|-----------------| @@ -491,9 +668,15 @@ Task 3 complete. Continuing to task 4... | Write .env file | Yes (Write tool) | YES | | Create Upstash DB | Yes (`upstash`) | YES | | Run tests | Yes (`npm test`) | YES | +| Start dev server | Yes (`npm run dev`) | YES | +| Add env vars to Convex | Yes (`npx convex env set`) | YES | +| Add env vars to Vercel | Yes (`vercel env add`) | YES | +| Seed database | Yes (CLI/API) | YES | | Click email verification link | No | NO | | Enter credit card with 3DS | No | NO | | Complete OAuth in browser | No | NO | +| Visually verify UI looks correct | No | NO | +| Test interactive user flows | No | NO | @@ -643,17 +826,23 @@ Task 3 complete. Continuing to task 4... npm run build succeeds - + + Start dev server for auth testing + Run `npm run dev` in background, wait for ready signal + curl http://localhost:3000 returns 200 + Dev server running at http://localhost:3000 + + + - Complete authentication flow (schema + API + UI) + Complete authentication flow - dev server running at http://localhost:3000 - 1. Run: npm run dev - 2. Visit: http://localhost:3000/login - 3. Click "Sign in with GitHub" - 4. Complete GitHub OAuth flow - 5. Verify: Redirected to /dashboard, user name displayed - 6. Refresh page: Session persists - 7. Click logout: Session cleared + 1. Visit: http://localhost:3000/login + 2. Click "Sign in with GitHub" + 3. Complete GitHub OAuth flow + 4. Verify: Redirected to /dashboard, user name displayed + 5. Refresh page: Session persists + 6. Click logout: Session cleared Type "approved" or describe issues @@ -662,7 +851,77 @@ Task 3 complete. Continuing to task 4... -### ❌ BAD: Asking human to automate +### ❌ BAD: Asking user to start dev server + +```xml + + Dashboard component + + 1. Run: npm run dev + 2. Visit: http://localhost:3000/dashboard + 3. Check layout is correct + + +``` + +**Why bad:** Claude can run `npm run dev`. User should only visit URLs, not execute commands. + +### ✅ GOOD: Claude starts server, user visits + +```xml + + Start dev server + Run `npm run dev` in background + curl localhost:3000 returns 200 + + + + Dashboard at http://localhost:3000/dashboard (server running) + + Visit http://localhost:3000/dashboard and verify: + 1. Layout matches design + 2. No console errors + + +``` + +### ❌ BAD: Asking user to add env vars in dashboard + +```xml + + Add environment variables to Convex + + 1. Go to dashboard.convex.dev + 2. Select your project + 3. Navigate to Settings → Environment Variables + 4. Add OPENAI_API_KEY with your key + + +``` + +**Why bad:** Convex has `npx convex env set`. Claude should ask for the key value, then run the CLI command. + +### ✅ GOOD: Claude collects secret, adds via CLI + +```xml + + Provide your OpenAI API key + + I need your OpenAI API key. Get it from: https://platform.openai.com/api-keys + Paste the key below (starts with sk-) + + I'll configure it via CLI + Paste your key + + + + Add OpenAI key to Convex + Run `npx convex env set OPENAI_API_KEY {key}` + `npx convex env get` shows OPENAI_API_KEY configured + +``` + +### ❌ BAD: Asking human to deploy ```xml @@ -750,23 +1009,54 @@ Task 3 complete. Continuing to task 4... **Why bad:** No specifics. User doesn't know what to test or what "works" means. -### ✅ GOOD: Specific verification steps +### ✅ GOOD: Specific verification steps (server already running) ```xml - Responsive dashboard at /dashboard + Responsive dashboard - server running at http://localhost:3000 - 1. Run: npm run dev - 2. Visit: http://localhost:3000/dashboard - 3. Desktop (>1024px): Sidebar visible, content area fills remaining space - 4. Tablet (768px): Sidebar collapses to icons - 5. Mobile (375px): Sidebar hidden, hamburger menu in header - 6. Check: No horizontal scroll at any size + Visit http://localhost:3000/dashboard and verify: + 1. Desktop (>1024px): Sidebar visible, content area fills remaining space + 2. Tablet (768px): Sidebar collapses to icons + 3. Mobile (375px): Sidebar hidden, hamburger menu in header + 4. No horizontal scroll at any size Type "approved" or describe layout issues ``` +### ❌ BAD: Asking user to run any CLI command + +```xml + + Run database migrations + + 1. Run: npx prisma migrate deploy + 2. Run: npx prisma db seed + 3. Verify tables exist + + +``` + +**Why bad:** Claude can run these commands. User should never execute CLI commands. + +### ❌ BAD: Asking user to copy values between services + +```xml + + Configure webhook URL in Stripe + + 1. Copy the deployment URL from terminal + 2. Go to Stripe Dashboard → Webhooks + 3. Add endpoint with URL + /api/webhooks + 4. Copy webhook signing secret + 5. Add to .env file + + +``` + +**Why bad:** Stripe has an API. Claude should create the webhook via API and write to .env directly. + diff --git a/get-shit-done/references/verification-patterns.md b/get-shit-done/references/verification-patterns.md index fde7bdeca..0afb5e63a 100644 --- a/get-shit-done/references/verification-patterns.md +++ b/get-shit-done/references/verification-patterns.md @@ -593,3 +593,20 @@ Some things can't be verified programmatically. Flag these for human testing: ``` + + + +## Pre-Checkpoint Automation + +For automation-first checkpoint patterns, server lifecycle management, CLI installation handling, and error recovery protocols, see: + +**@~/.claude/get-shit-done/references/checkpoints.md** → `` section + +Key principles: +- Claude sets up verification environment BEFORE presenting checkpoints +- Users never run CLI commands (visit URLs only) +- Server lifecycle: start before checkpoint, handle port conflicts, keep running for duration +- CLI installation: auto-install where safe, checkpoint for user choice otherwise +- Error handling: fix broken environment before checkpoint, never present checkpoint with failed setup + + diff --git a/get-shit-done/templates/phase-prompt.md b/get-shit-done/templates/phase-prompt.md index 607a97b4f..d1f6c9515 100644 --- a/get-shit-done/templates/phase-prompt.md +++ b/get-shit-done/templates/phase-prompt.md @@ -75,33 +75,23 @@ Output: [What artifacts will be created] [Acceptance criteria] + + + [What needs deciding] [Why this decision matters] - - + + - [How to indicate choice - "Select: option-a or option-b"] + Select: option-a or option-b - [What Claude just built that needs verification] - - 1. Run: [command to start dev server/app] - 2. Visit: [URL to check] - 3. Test: [Specific interactions] - 4. Confirm: [Expected behaviors] - - Type "approved" to continue, or describe issues to fix + [What Claude built] - server running at [URL] + Visit [URL] and verify: [visual checks only, NO CLI commands] + Type "approved" or describe issues @@ -403,15 +393,16 @@ Output: Working dashboard component. Dashboard renders without errors + + + Start dev server + Run `npm run dev` in background, wait for ready + curl localhost:3000 returns 200 + + - Responsive dashboard with user and product sections - - 1. Run: npm run dev - 2. Visit: http://localhost:3000/dashboard - 3. Desktop: Verify two-column grid - 4. Mobile: Verify stacked layout - 5. Check: No layout shift, no scroll issues - + Dashboard - server at http://localhost:3000 + Visit localhost:3000/dashboard. Check: desktop grid, mobile stack, no scroll issues. Type "approved" or describe issues