feat: add phase verification system
- verify-phase.md workflow: validates phase goal achievement vs task completion - subagent-verify-prompt.md: template for spawning verification agents - verification-report.md: VERIFICATION.md output template - verification-patterns.md: common anti-patterns and verification strategies Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Lex Christopherson
595
get-shit-done/references/verification-patterns.md
Normal file
595
get-shit-done/references/verification-patterns.md
Normal file
@@ -0,0 +1,595 @@
|
||||
# Verification Patterns
|
||||
|
||||
How to verify different types of artifacts are real implementations, not stubs or placeholders.
|
||||
|
||||
<core_principle>
|
||||
**Existence ≠ Implementation**
|
||||
|
||||
A file existing does not mean the feature works. Verification must check:
|
||||
1. **Exists** - File is present at expected path
|
||||
2. **Substantive** - Content is real implementation, not placeholder
|
||||
3. **Wired** - Connected to the rest of the system
|
||||
4. **Functional** - Actually works when invoked
|
||||
|
||||
Levels 1-3 can be checked programmatically. Level 4 often requires human verification.
|
||||
</core_principle>
|
||||
|
||||
<stub_detection>
|
||||
|
||||
## Universal Stub Patterns
|
||||
|
||||
These patterns indicate placeholder code regardless of file type:
|
||||
|
||||
**Comment-based stubs:**
|
||||
```bash
|
||||
# Grep patterns for stub comments
|
||||
grep -E "(TODO|FIXME|XXX|HACK|PLACEHOLDER)" "$file"
|
||||
grep -E "implement|add later|coming soon|will be" "$file" -i
|
||||
grep -E "// \.\.\.|/\* \.\.\. \*/|# \.\.\." "$file"
|
||||
```
|
||||
|
||||
**Placeholder text in output:**
|
||||
```bash
|
||||
# UI placeholder patterns
|
||||
grep -E "placeholder|lorem ipsum|coming soon|under construction" "$file" -i
|
||||
grep -E "sample|example|test data|dummy" "$file" -i
|
||||
grep -E "\[.*\]|<.*>|\{.*\}" "$file" # Template brackets left in
|
||||
```
|
||||
|
||||
**Empty or trivial implementations:**
|
||||
```bash
|
||||
# Functions that do nothing
|
||||
grep -E "return null|return undefined|return \{\}|return \[\]" "$file"
|
||||
grep -E "pass$|\.\.\.|\bnothing\b" "$file"
|
||||
grep -E "console\.(log|warn|error).*only" "$file" # Log-only functions
|
||||
```
|
||||
|
||||
**Hardcoded values where dynamic expected:**
|
||||
```bash
|
||||
# Hardcoded IDs, counts, or content
|
||||
grep -E "id.*=.*['\"].*['\"]" "$file" # Hardcoded string IDs
|
||||
grep -E "count.*=.*\d+|length.*=.*\d+" "$file" # Hardcoded counts
|
||||
grep -E "\\\$\d+\.\d{2}|\d+ items" "$file" # Hardcoded display values
|
||||
```
|
||||
|
||||
</stub_detection>
|
||||
|
||||
<react_components>
|
||||
|
||||
## React/Next.js Components
|
||||
|
||||
**Existence check:**
|
||||
```bash
|
||||
# File exists and exports component
|
||||
[ -f "$component_path" ] && grep -E "export (default |)function|export const.*=.*\(" "$component_path"
|
||||
```
|
||||
|
||||
**Substantive check:**
|
||||
```bash
|
||||
# Returns actual JSX, not placeholder
|
||||
grep -E "return.*<" "$component_path" | grep -v "return.*null" | grep -v "placeholder" -i
|
||||
|
||||
# Has meaningful content (not just wrapper div)
|
||||
grep -E "<[A-Z][a-zA-Z]+|className=|onClick=|onChange=" "$component_path"
|
||||
|
||||
# Uses props or state (not static)
|
||||
grep -E "props\.|useState|useEffect|useContext|\{.*\}" "$component_path"
|
||||
```
|
||||
|
||||
**Stub patterns specific to React:**
|
||||
```javascript
|
||||
// RED FLAGS - These are stubs:
|
||||
return <div>Component</div>
|
||||
return <div>Placeholder</div>
|
||||
return <div>{/* TODO */}</div>
|
||||
return <p>Coming soon</p>
|
||||
return null
|
||||
return <></>
|
||||
|
||||
// Also stubs - empty handlers:
|
||||
onClick={() => {}}
|
||||
onChange={() => console.log('clicked')}
|
||||
onSubmit={(e) => e.preventDefault()} // Only prevents default, does nothing
|
||||
```
|
||||
|
||||
**Wiring check:**
|
||||
```bash
|
||||
# Component imports what it needs
|
||||
grep -E "^import.*from" "$component_path"
|
||||
|
||||
# Props are actually used (not just received)
|
||||
# Look for destructuring or props.X usage
|
||||
grep -E "\{ .* \}.*props|\bprops\.[a-zA-Z]+" "$component_path"
|
||||
|
||||
# API calls exist (for data-fetching components)
|
||||
grep -E "fetch\(|axios\.|useSWR|useQuery|getServerSideProps|getStaticProps" "$component_path"
|
||||
```
|
||||
|
||||
**Functional verification (human required):**
|
||||
- Does the component render visible content?
|
||||
- Do interactive elements respond to clicks?
|
||||
- Does data load and display?
|
||||
- Do error states show appropriately?
|
||||
|
||||
</react_components>
|
||||
|
||||
<api_routes>
|
||||
|
||||
## API Routes (Next.js App Router / Express / etc.)
|
||||
|
||||
**Existence check:**
|
||||
```bash
|
||||
# Route file exists
|
||||
[ -f "$route_path" ]
|
||||
|
||||
# Exports HTTP method handlers (Next.js App Router)
|
||||
grep -E "export (async )?(function|const) (GET|POST|PUT|PATCH|DELETE)" "$route_path"
|
||||
|
||||
# Or Express-style handlers
|
||||
grep -E "\.(get|post|put|patch|delete)\(" "$route_path"
|
||||
```
|
||||
|
||||
**Substantive check:**
|
||||
```bash
|
||||
# Has actual logic, not just return statement
|
||||
wc -l "$route_path" # More than 10-15 lines suggests real implementation
|
||||
|
||||
# Interacts with data source
|
||||
grep -E "prisma\.|db\.|mongoose\.|sql|query|find|create|update|delete" "$route_path" -i
|
||||
|
||||
# Has error handling
|
||||
grep -E "try|catch|throw|error|Error" "$route_path"
|
||||
|
||||
# Returns meaningful response
|
||||
grep -E "Response\.json|res\.json|res\.send|return.*\{" "$route_path" | grep -v "message.*not implemented" -i
|
||||
```
|
||||
|
||||
**Stub patterns specific to API routes:**
|
||||
```typescript
|
||||
// RED FLAGS - These are stubs:
|
||||
export async function POST() {
|
||||
return Response.json({ message: "Not implemented" })
|
||||
}
|
||||
|
||||
export async function GET() {
|
||||
return Response.json([]) // Empty array with no DB query
|
||||
}
|
||||
|
||||
export async function PUT() {
|
||||
return new Response() // Empty response
|
||||
}
|
||||
|
||||
// Console log only:
|
||||
export async function POST(req) {
|
||||
console.log(await req.json())
|
||||
return Response.json({ ok: true })
|
||||
}
|
||||
```
|
||||
|
||||
**Wiring check:**
|
||||
```bash
|
||||
# Imports database/service clients
|
||||
grep -E "^import.*prisma|^import.*db|^import.*client" "$route_path"
|
||||
|
||||
# Actually uses request body (for POST/PUT)
|
||||
grep -E "req\.json\(\)|req\.body|request\.json\(\)" "$route_path"
|
||||
|
||||
# Validates input (not just trusting request)
|
||||
grep -E "schema\.parse|validate|zod|yup|joi" "$route_path"
|
||||
```
|
||||
|
||||
**Functional verification (human or automated):**
|
||||
- Does GET return real data from database?
|
||||
- Does POST actually create a record?
|
||||
- Does error response have correct status code?
|
||||
- Are auth checks actually enforced?
|
||||
|
||||
</api_routes>
|
||||
|
||||
<database_schema>
|
||||
|
||||
## Database Schema (Prisma / Drizzle / SQL)
|
||||
|
||||
**Existence check:**
|
||||
```bash
|
||||
# Schema file exists
|
||||
[ -f "prisma/schema.prisma" ] || [ -f "drizzle/schema.ts" ] || [ -f "src/db/schema.sql" ]
|
||||
|
||||
# Model/table is defined
|
||||
grep -E "^model $model_name|CREATE TABLE $table_name|export const $table_name" "$schema_path"
|
||||
```
|
||||
|
||||
**Substantive check:**
|
||||
```bash
|
||||
# Has expected fields (not just id)
|
||||
grep -A 20 "model $model_name" "$schema_path" | grep -E "^\s+\w+\s+\w+"
|
||||
|
||||
# Has relationships if expected
|
||||
grep -E "@relation|REFERENCES|FOREIGN KEY" "$schema_path"
|
||||
|
||||
# Has appropriate field types (not all String)
|
||||
grep -A 20 "model $model_name" "$schema_path" | grep -E "Int|DateTime|Boolean|Float|Decimal|Json"
|
||||
```
|
||||
|
||||
**Stub patterns specific to schemas:**
|
||||
```prisma
|
||||
// RED FLAGS - These are stubs:
|
||||
model User {
|
||||
id String @id
|
||||
// TODO: add fields
|
||||
}
|
||||
|
||||
model Message {
|
||||
id String @id
|
||||
content String // Only one real field
|
||||
}
|
||||
|
||||
// Missing critical fields:
|
||||
model Order {
|
||||
id String @id
|
||||
// No: userId, items, total, status, createdAt
|
||||
}
|
||||
```
|
||||
|
||||
**Wiring check:**
|
||||
```bash
|
||||
# Migrations exist and are applied
|
||||
ls prisma/migrations/ 2>/dev/null | wc -l # Should be > 0
|
||||
npx prisma migrate status 2>/dev/null | grep -v "pending"
|
||||
|
||||
# Client is generated
|
||||
[ -d "node_modules/.prisma/client" ]
|
||||
```
|
||||
|
||||
**Functional verification:**
|
||||
```bash
|
||||
# Can query the table (automated)
|
||||
npx prisma db execute --stdin <<< "SELECT COUNT(*) FROM $table_name"
|
||||
```
|
||||
|
||||
</database_schema>
|
||||
|
||||
<hooks_utilities>
|
||||
|
||||
## Custom Hooks and Utilities
|
||||
|
||||
**Existence check:**
|
||||
```bash
|
||||
# File exists and exports function
|
||||
[ -f "$hook_path" ] && grep -E "export (default )?(function|const)" "$hook_path"
|
||||
```
|
||||
|
||||
**Substantive check:**
|
||||
```bash
|
||||
# Hook uses React hooks (for custom hooks)
|
||||
grep -E "useState|useEffect|useCallback|useMemo|useRef|useContext" "$hook_path"
|
||||
|
||||
# Has meaningful return value
|
||||
grep -E "return \{|return \[" "$hook_path"
|
||||
|
||||
# More than trivial length
|
||||
[ $(wc -l < "$hook_path") -gt 10 ]
|
||||
```
|
||||
|
||||
**Stub patterns specific to hooks:**
|
||||
```typescript
|
||||
// RED FLAGS - These are stubs:
|
||||
export function useAuth() {
|
||||
return { user: null, login: () => {}, logout: () => {} }
|
||||
}
|
||||
|
||||
export function useCart() {
|
||||
const [items, setItems] = useState([])
|
||||
return { items, addItem: () => console.log('add'), removeItem: () => {} }
|
||||
}
|
||||
|
||||
// Hardcoded return:
|
||||
export function useUser() {
|
||||
return { name: "Test User", email: "test@example.com" }
|
||||
}
|
||||
```
|
||||
|
||||
**Wiring check:**
|
||||
```bash
|
||||
# Hook is actually imported somewhere
|
||||
grep -r "import.*$hook_name" src/ --include="*.tsx" --include="*.ts" | grep -v "$hook_path"
|
||||
|
||||
# Hook is actually called
|
||||
grep -r "$hook_name()" src/ --include="*.tsx" --include="*.ts" | grep -v "$hook_path"
|
||||
```
|
||||
|
||||
</hooks_utilities>
|
||||
|
||||
<environment_config>
|
||||
|
||||
## Environment Variables and Configuration
|
||||
|
||||
**Existence check:**
|
||||
```bash
|
||||
# .env file exists
|
||||
[ -f ".env" ] || [ -f ".env.local" ]
|
||||
|
||||
# Required variable is defined
|
||||
grep -E "^$VAR_NAME=" .env .env.local 2>/dev/null
|
||||
```
|
||||
|
||||
**Substantive check:**
|
||||
```bash
|
||||
# Variable has actual value (not placeholder)
|
||||
grep -E "^$VAR_NAME=.+" .env .env.local 2>/dev/null | grep -v "your-.*-here|xxx|placeholder|TODO" -i
|
||||
|
||||
# Value looks valid for type:
|
||||
# - URLs should start with http
|
||||
# - Keys should be long enough
|
||||
# - Booleans should be true/false
|
||||
```
|
||||
|
||||
**Stub patterns specific to env:**
|
||||
```bash
|
||||
# RED FLAGS - These are stubs:
|
||||
DATABASE_URL=your-database-url-here
|
||||
STRIPE_SECRET_KEY=sk_test_xxx
|
||||
API_KEY=placeholder
|
||||
NEXT_PUBLIC_API_URL=http://localhost:3000 # Still pointing to localhost in prod
|
||||
```
|
||||
|
||||
**Wiring check:**
|
||||
```bash
|
||||
# Variable is actually used in code
|
||||
grep -r "process\.env\.$VAR_NAME|env\.$VAR_NAME" src/ --include="*.ts" --include="*.tsx"
|
||||
|
||||
# Variable is in validation schema (if using zod/etc for env)
|
||||
grep -E "$VAR_NAME" src/env.ts src/env.mjs 2>/dev/null
|
||||
```
|
||||
|
||||
</environment_config>
|
||||
|
||||
<wiring_verification>
|
||||
|
||||
## Wiring Verification Patterns
|
||||
|
||||
Wiring verification checks that components actually communicate. This is where most stubs hide.
|
||||
|
||||
### Pattern: Component → API
|
||||
|
||||
**Check:** Does the component actually call the API?
|
||||
|
||||
```bash
|
||||
# Find the fetch/axios call
|
||||
grep -E "fetch\(['\"].*$api_path|axios\.(get|post).*$api_path" "$component_path"
|
||||
|
||||
# Verify it's not commented out
|
||||
grep -E "fetch\(|axios\." "$component_path" | grep -v "^.*//.*fetch"
|
||||
|
||||
# Check the response is used
|
||||
grep -E "await.*fetch|\.then\(|setData|setState" "$component_path"
|
||||
```
|
||||
|
||||
**Red flags:**
|
||||
```typescript
|
||||
// Fetch exists but response ignored:
|
||||
fetch('/api/messages') // No await, no .then, no assignment
|
||||
|
||||
// Fetch in comment:
|
||||
// fetch('/api/messages').then(r => r.json()).then(setMessages)
|
||||
|
||||
// Fetch to wrong endpoint:
|
||||
fetch('/api/message') // Typo - should be /api/messages
|
||||
```
|
||||
|
||||
### Pattern: API → Database
|
||||
|
||||
**Check:** Does the API route actually query the database?
|
||||
|
||||
```bash
|
||||
# Find the database call
|
||||
grep -E "prisma\.$model|db\.query|Model\.find" "$route_path"
|
||||
|
||||
# Verify it's awaited
|
||||
grep -E "await.*prisma|await.*db\." "$route_path"
|
||||
|
||||
# Check result is returned
|
||||
grep -E "return.*json.*data|res\.json.*result" "$route_path"
|
||||
```
|
||||
|
||||
**Red flags:**
|
||||
```typescript
|
||||
// Query exists but result not returned:
|
||||
await prisma.message.findMany()
|
||||
return Response.json({ ok: true }) // Returns static, not query result
|
||||
|
||||
// Query not awaited:
|
||||
const messages = prisma.message.findMany() // Missing await
|
||||
return Response.json(messages) // Returns Promise, not data
|
||||
```
|
||||
|
||||
### Pattern: Form → Handler
|
||||
|
||||
**Check:** Does the form submission actually do something?
|
||||
|
||||
```bash
|
||||
# Find onSubmit handler
|
||||
grep -E "onSubmit=\{|handleSubmit" "$component_path"
|
||||
|
||||
# Check handler has content
|
||||
grep -A 10 "onSubmit.*=" "$component_path" | grep -E "fetch|axios|mutate|dispatch"
|
||||
|
||||
# Verify not just preventDefault
|
||||
grep -A 5 "onSubmit" "$component_path" | grep -v "only.*preventDefault" -i
|
||||
```
|
||||
|
||||
**Red flags:**
|
||||
```typescript
|
||||
// Handler only prevents default:
|
||||
onSubmit={(e) => e.preventDefault()}
|
||||
|
||||
// Handler only logs:
|
||||
const handleSubmit = (data) => {
|
||||
console.log(data)
|
||||
}
|
||||
|
||||
// Handler is empty:
|
||||
onSubmit={() => {}}
|
||||
```
|
||||
|
||||
### Pattern: State → Render
|
||||
|
||||
**Check:** Does the component render state, not hardcoded content?
|
||||
|
||||
```bash
|
||||
# Find state usage in JSX
|
||||
grep -E "\{.*messages.*\}|\{.*data.*\}|\{.*items.*\}" "$component_path"
|
||||
|
||||
# Check map/render of state
|
||||
grep -E "\.map\(|\.filter\(|\.reduce\(" "$component_path"
|
||||
|
||||
# Verify dynamic content
|
||||
grep -E "\{[a-zA-Z_]+\." "$component_path" # Variable interpolation
|
||||
```
|
||||
|
||||
**Red flags:**
|
||||
```tsx
|
||||
// Hardcoded instead of state:
|
||||
return <div>
|
||||
<p>Message 1</p>
|
||||
<p>Message 2</p>
|
||||
</div>
|
||||
|
||||
// State exists but not rendered:
|
||||
const [messages, setMessages] = useState([])
|
||||
return <div>No messages</div> // Always shows "no messages"
|
||||
|
||||
// Wrong state rendered:
|
||||
const [messages, setMessages] = useState([])
|
||||
return <div>{otherData.map(...)}</div> // Uses different data
|
||||
```
|
||||
|
||||
</wiring_verification>
|
||||
|
||||
<verification_checklist>
|
||||
|
||||
## Quick Verification Checklist
|
||||
|
||||
For each artifact type, run through this checklist:
|
||||
|
||||
### Component Checklist
|
||||
- [ ] File exists at expected path
|
||||
- [ ] Exports a function/const component
|
||||
- [ ] Returns JSX (not null/empty)
|
||||
- [ ] No placeholder text in render
|
||||
- [ ] Uses props or state (not static)
|
||||
- [ ] Event handlers have real implementations
|
||||
- [ ] Imports resolve correctly
|
||||
- [ ] Used somewhere in the app
|
||||
|
||||
### API Route Checklist
|
||||
- [ ] File exists at expected path
|
||||
- [ ] Exports HTTP method handlers
|
||||
- [ ] Handlers have more than 5 lines
|
||||
- [ ] Queries database or service
|
||||
- [ ] Returns meaningful response (not empty/placeholder)
|
||||
- [ ] Has error handling
|
||||
- [ ] Validates input
|
||||
- [ ] Called from frontend
|
||||
|
||||
### Schema Checklist
|
||||
- [ ] Model/table defined
|
||||
- [ ] Has all expected fields
|
||||
- [ ] Fields have appropriate types
|
||||
- [ ] Relationships defined if needed
|
||||
- [ ] Migrations exist and applied
|
||||
- [ ] Client generated
|
||||
|
||||
### Hook/Utility Checklist
|
||||
- [ ] File exists at expected path
|
||||
- [ ] Exports function
|
||||
- [ ] Has meaningful implementation (not empty returns)
|
||||
- [ ] Used somewhere in the app
|
||||
- [ ] Return values consumed
|
||||
|
||||
### Wiring Checklist
|
||||
- [ ] Component → API: fetch/axios call exists and uses response
|
||||
- [ ] API → Database: query exists and result returned
|
||||
- [ ] Form → Handler: onSubmit calls API/mutation
|
||||
- [ ] State → Render: state variables appear in JSX
|
||||
|
||||
</verification_checklist>
|
||||
|
||||
<automated_verification_script>
|
||||
|
||||
## Automated Verification Approach
|
||||
|
||||
For the verification subagent, use this pattern:
|
||||
|
||||
```bash
|
||||
# 1. Check existence
|
||||
check_exists() {
|
||||
[ -f "$1" ] && echo "EXISTS: $1" || echo "MISSING: $1"
|
||||
}
|
||||
|
||||
# 2. Check for stub patterns
|
||||
check_stubs() {
|
||||
local file="$1"
|
||||
local stubs=$(grep -c -E "TODO|FIXME|placeholder|not implemented" "$file" 2>/dev/null || echo 0)
|
||||
[ "$stubs" -gt 0 ] && echo "STUB_PATTERNS: $stubs in $file"
|
||||
}
|
||||
|
||||
# 3. Check wiring (component calls API)
|
||||
check_wiring() {
|
||||
local component="$1"
|
||||
local api_path="$2"
|
||||
grep -q "$api_path" "$component" && echo "WIRED: $component → $api_path" || echo "NOT_WIRED: $component → $api_path"
|
||||
}
|
||||
|
||||
# 4. Check substantive (more than N lines, has expected patterns)
|
||||
check_substantive() {
|
||||
local file="$1"
|
||||
local min_lines="$2"
|
||||
local pattern="$3"
|
||||
local lines=$(wc -l < "$file" 2>/dev/null || echo 0)
|
||||
local has_pattern=$(grep -c -E "$pattern" "$file" 2>/dev/null || echo 0)
|
||||
[ "$lines" -ge "$min_lines" ] && [ "$has_pattern" -gt 0 ] && echo "SUBSTANTIVE: $file" || echo "THIN: $file ($lines lines, $has_pattern matches)"
|
||||
}
|
||||
```
|
||||
|
||||
Run these checks against each must-have artifact. Aggregate results into VERIFICATION.md.
|
||||
|
||||
</automated_verification_script>
|
||||
|
||||
<human_verification_triggers>
|
||||
|
||||
## When to Require Human Verification
|
||||
|
||||
Some things can't be verified programmatically. Flag these for human testing:
|
||||
|
||||
**Always human:**
|
||||
- Visual appearance (does it look right?)
|
||||
- User flow completion (can you actually do the thing?)
|
||||
- Real-time behavior (WebSocket, SSE)
|
||||
- External service integration (Stripe, email sending)
|
||||
- Error message clarity (is the message helpful?)
|
||||
- Performance feel (does it feel fast?)
|
||||
|
||||
**Human if uncertain:**
|
||||
- Complex wiring that grep can't trace
|
||||
- Dynamic behavior depending on state
|
||||
- Edge cases and error states
|
||||
- Mobile responsiveness
|
||||
- Accessibility
|
||||
|
||||
**Format for human verification request:**
|
||||
```markdown
|
||||
## Human Verification Required
|
||||
|
||||
### 1. Chat message sending
|
||||
**Test:** Type a message and click Send
|
||||
**Expected:** Message appears in list, input clears
|
||||
**Check:** Does message persist after refresh?
|
||||
|
||||
### 2. Error handling
|
||||
**Test:** Disconnect network, try to send
|
||||
**Expected:** Error message appears, message not lost
|
||||
**Check:** Can retry after reconnect?
|
||||
```
|
||||
|
||||
</human_verification_triggers>
|
||||
216
get-shit-done/templates/subagent-verify-prompt.md
Normal file
216
get-shit-done/templates/subagent-verify-prompt.md
Normal file
@@ -0,0 +1,216 @@
|
||||
# Subagent Verify Prompt Template
|
||||
|
||||
Template for spawning a verification subagent from execute-phase.md.
|
||||
|
||||
---
|
||||
|
||||
## Template
|
||||
|
||||
```markdown
|
||||
<objective>
|
||||
Verify Phase {phase_number} achieved its GOAL, not just completed its TASKS.
|
||||
|
||||
Your job: Goal-backward verification. Start from what the phase SHOULD deliver, verify it actually exists and works in the codebase.
|
||||
|
||||
**Critical mindset:** Do NOT trust SUMMARY.md claims. SUMMARYs document what Claude SAID it did. You verify what ACTUALLY exists in the code. These often differ.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@~/.claude/get-shit-done/workflows/verify-phase.md
|
||||
@~/.claude/get-shit-done/templates/verification-report.md
|
||||
@~/.claude/get-shit-done/references/verification-patterns.md
|
||||
@~/.claude/get-shit-done/references/goal-backward.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
**Phase being verified:**
|
||||
- Phase: {phase_number} - {phase_name}
|
||||
- Phase goal: {phase_goal_from_roadmap}
|
||||
- Phase directory: .planning/phases/{phase_dir}/
|
||||
|
||||
**Must-haves to verify:**
|
||||
{must_haves_yaml_from_plan_frontmatter}
|
||||
|
||||
**If must_haves not in frontmatter, derive from:**
|
||||
- Phase goal (above)
|
||||
- @.planning/ROADMAP.md (phase description)
|
||||
- @.planning/REQUIREMENTS.md (requirements mapped to this phase)
|
||||
|
||||
**Context for verification:**
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/REQUIREMENTS.md (if exists)
|
||||
@.planning/phases/{phase_dir}/*-SUMMARY.md (claims to verify against)
|
||||
</context>
|
||||
|
||||
<verification_approach>
|
||||
|
||||
## Step 1: Establish Must-Haves
|
||||
|
||||
**If must_haves provided in context above:** Use them directly.
|
||||
|
||||
**If must_haves NOT provided:** Derive using goal-backward process:
|
||||
1. State the phase goal
|
||||
2. Ask: "What must be TRUE for this goal to be achieved?" (3-7 observable truths)
|
||||
3. For each truth: "What must EXIST?" (concrete artifacts)
|
||||
4. For each artifact: "What must be WIRED?" (connections between artifacts)
|
||||
5. Identify key links (critical connections most likely to be stubs)
|
||||
|
||||
Document your derived must-haves before proceeding.
|
||||
|
||||
## Step 2: Verify Observable Truths
|
||||
|
||||
For each truth, determine if it's achievable given the codebase state.
|
||||
|
||||
A truth is:
|
||||
- ✓ VERIFIED: Codebase makes this truth achievable
|
||||
- ✗ FAILED: Codebase prevents this truth (missing/stub code)
|
||||
- ? UNCERTAIN: Can't verify programmatically (needs human)
|
||||
|
||||
## Step 3: Verify Required Artifacts
|
||||
|
||||
For each artifact, check three levels:
|
||||
|
||||
**Level 1 - Exists:**
|
||||
```bash
|
||||
[ -f "{artifact_path}" ] && echo "EXISTS" || echo "MISSING"
|
||||
```
|
||||
|
||||
**Level 2 - Substantive:**
|
||||
- Is it more than a stub? (check line count, grep for stub patterns)
|
||||
- Does it have expected exports/structure?
|
||||
- Are there TODO/placeholder comments?
|
||||
|
||||
Use patterns from verification-patterns.md.
|
||||
|
||||
**Level 3 - Wired:**
|
||||
- Is it imported somewhere?
|
||||
- Is it called/used?
|
||||
- Does data flow through it?
|
||||
|
||||
## Step 4: Verify Key Links
|
||||
|
||||
Key links are the critical connections. For each:
|
||||
|
||||
```bash
|
||||
# Check if A calls B
|
||||
grep -E "{pattern_for_call}" "{file_A}"
|
||||
|
||||
# Check if the call uses the response
|
||||
grep -A 5 "{call_pattern}" "{file_A}" | grep -E "await|then|set"
|
||||
```
|
||||
|
||||
Key links are where stubs hide. A component might exist and an API might exist, but the component doesn't actually call the API.
|
||||
|
||||
## Step 5: Check Requirements Coverage
|
||||
|
||||
If REQUIREMENTS.md exists and has requirements mapped to this phase:
|
||||
- For each requirement, determine if the codebase satisfies it
|
||||
- A requirement is BLOCKED if any supporting artifact is missing/stub
|
||||
|
||||
## Step 6: Scan for Anti-Patterns
|
||||
|
||||
Run stub detection across all phase-related files:
|
||||
|
||||
```bash
|
||||
# Find all files modified in this phase (from SUMMARYs)
|
||||
# Then scan for anti-patterns
|
||||
|
||||
grep -r -E "TODO|FIXME|placeholder|not implemented" {phase_files} -i
|
||||
grep -r -E "return null|return \{\}|return \[\]" {phase_files}
|
||||
grep -r -E "console\.log\(.*\)" {phase_files} | grep -v "error\|warn"
|
||||
```
|
||||
|
||||
## Step 7: Generate Report
|
||||
|
||||
Create `.planning/phases/{phase_dir}/{phase}-VERIFICATION.md` using the verification-report.md template.
|
||||
|
||||
Include:
|
||||
- All truths with status and evidence
|
||||
- All artifacts with status and details
|
||||
- All key links with status and details
|
||||
- Requirements coverage
|
||||
- Anti-patterns found
|
||||
- Human verification needs
|
||||
- Gap summary with fix recommendations
|
||||
|
||||
</verification_approach>
|
||||
|
||||
<critical_rules>
|
||||
|
||||
**DO NOT trust SUMMARY claims.** SUMMARYs say "implemented chat component" — you verify the component actually renders messages, not a placeholder.
|
||||
|
||||
**DO NOT assume existence = implementation.** A file existing is level 1. You need level 2 (substantive) and level 3 (wired) verification.
|
||||
|
||||
**DO NOT skip key link verification.** This is where 80% of stubs hide. The pieces exist but aren't connected.
|
||||
|
||||
**DO generate fix plans if gaps found.** Don't just report "this is broken" — recommend specific fix plans with tasks.
|
||||
|
||||
**DO flag for human verification when uncertain.** If you can't verify programmatically (visual, real-time, external service), say so explicitly.
|
||||
|
||||
**DO keep verification fast.** Use grep/file checks, not running the app. Goal is structural verification, not functional testing.
|
||||
|
||||
</critical_rules>
|
||||
|
||||
<output>
|
||||
Create: `.planning/phases/{phase_dir}/{phase}-VERIFICATION.md`
|
||||
|
||||
Use the verification-report.md template structure.
|
||||
|
||||
Return to orchestrator with:
|
||||
- Status: passed | gaps_found | human_needed
|
||||
- Score: N/M must-haves verified
|
||||
- If gaps_found: Summary of gaps and recommended fix plans
|
||||
- If human_needed: List of items requiring human verification
|
||||
</output>
|
||||
|
||||
<success_criteria>
|
||||
- [ ] Must-haves established (from frontmatter or derived)
|
||||
- [ ] All truths verified with evidence
|
||||
- [ ] All artifacts checked (exists + substantive + wired)
|
||||
- [ ] All key links traced
|
||||
- [ ] Requirements coverage assessed
|
||||
- [ ] Anti-patterns scanned
|
||||
- [ ] VERIFICATION.md created with full report
|
||||
- [ ] Fix plans recommended if gaps found
|
||||
- [ ] Human verification items listed if uncertain
|
||||
</success_criteria>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
The execute-phase orchestrator fills this template and spawns a subagent:
|
||||
|
||||
```typescript
|
||||
Task(
|
||||
prompt: filled_template,
|
||||
subagent_type: "general-purpose",
|
||||
description: "Verify phase {X} goal achievement"
|
||||
)
|
||||
```
|
||||
|
||||
The subagent:
|
||||
1. Loads the verification workflow and references
|
||||
2. Establishes must-haves (from frontmatter or derived)
|
||||
3. Runs verification checks against codebase
|
||||
4. Creates VERIFICATION.md report
|
||||
5. Returns status to orchestrator
|
||||
|
||||
The orchestrator then:
|
||||
- If `passed`: Proceeds to update_roadmap
|
||||
- If `gaps_found`: Creates fix plans, executes them, re-verifies
|
||||
- If `human_needed`: Presents human verification items to user
|
||||
|
||||
---
|
||||
|
||||
## Template Variables
|
||||
|
||||
| Variable | Source | Example |
|
||||
|----------|--------|---------|
|
||||
| `{phase_number}` | From execute-phase | `03` |
|
||||
| `{phase_name}` | From ROADMAP.md | `chat-interface` |
|
||||
| `{phase_goal_from_roadmap}` | From ROADMAP.md phase description | `Working chat interface where users can send and receive messages` |
|
||||
| `{phase_dir}` | From filesystem | `03-chat-interface` |
|
||||
| `{must_haves_yaml_from_plan_frontmatter}` | From PLAN.md frontmatter | YAML block with truths, artifacts, key_links |
|
||||
322
get-shit-done/templates/verification-report.md
Normal file
322
get-shit-done/templates/verification-report.md
Normal file
@@ -0,0 +1,322 @@
|
||||
# Verification Report Template
|
||||
|
||||
Template for `.planning/phases/XX-name/{phase}-VERIFICATION.md` — phase goal verification results.
|
||||
|
||||
---
|
||||
|
||||
## File Template
|
||||
|
||||
```markdown
|
||||
---
|
||||
phase: XX-name
|
||||
verified: YYYY-MM-DDTHH:MM:SSZ
|
||||
status: passed | gaps_found | human_needed
|
||||
score: N/M must-haves verified
|
||||
---
|
||||
|
||||
# Phase {X}: {Name} Verification Report
|
||||
|
||||
**Phase Goal:** {goal from ROADMAP.md}
|
||||
**Verified:** {timestamp}
|
||||
**Status:** {passed | gaps_found | human_needed}
|
||||
|
||||
## Goal Achievement
|
||||
|
||||
### Observable Truths
|
||||
|
||||
| # | Truth | Status | Evidence |
|
||||
|---|-------|--------|----------|
|
||||
| 1 | {truth from must_haves} | ✓ VERIFIED | {what confirmed it} |
|
||||
| 2 | {truth from must_haves} | ✗ FAILED | {what's wrong} |
|
||||
| 3 | {truth from must_haves} | ? UNCERTAIN | {why can't verify} |
|
||||
|
||||
**Score:** {N}/{M} truths verified
|
||||
|
||||
### Required Artifacts
|
||||
|
||||
| Artifact | Expected | Status | Details |
|
||||
|----------|----------|--------|---------|
|
||||
| `src/components/Chat.tsx` | Message list component | ✓ EXISTS + SUBSTANTIVE | Exports ChatList, renders Message[], no stubs |
|
||||
| `src/app/api/chat/route.ts` | Message CRUD | ✗ STUB | File exists but POST returns placeholder |
|
||||
| `prisma/schema.prisma` | Message model | ✓ EXISTS + SUBSTANTIVE | Model defined with all fields |
|
||||
|
||||
**Artifacts:** {N}/{M} verified
|
||||
|
||||
### Key Link Verification
|
||||
|
||||
| From | To | Via | Status | Details |
|
||||
|------|----|----|--------|---------|
|
||||
| Chat.tsx | /api/chat | fetch in useEffect | ✓ WIRED | Line 23: `fetch('/api/chat')` with response handling |
|
||||
| ChatInput | /api/chat POST | onSubmit handler | ✗ NOT WIRED | onSubmit only calls console.log |
|
||||
| /api/chat POST | database | prisma.message.create | ✗ NOT WIRED | Returns hardcoded response, no DB call |
|
||||
|
||||
**Wiring:** {N}/{M} connections verified
|
||||
|
||||
## Requirements Coverage
|
||||
|
||||
| Requirement | Status | Blocking Issue |
|
||||
|-------------|--------|----------------|
|
||||
| {REQ-01}: {description} | ✓ SATISFIED | - |
|
||||
| {REQ-02}: {description} | ✗ BLOCKED | API route is stub |
|
||||
| {REQ-03}: {description} | ? NEEDS HUMAN | Can't verify WebSocket programmatically |
|
||||
|
||||
**Coverage:** {N}/{M} requirements satisfied
|
||||
|
||||
## Anti-Patterns Found
|
||||
|
||||
| File | Line | Pattern | Severity | Impact |
|
||||
|------|------|---------|----------|--------|
|
||||
| src/app/api/chat/route.ts | 12 | `// TODO: implement` | ⚠️ Warning | Indicates incomplete |
|
||||
| src/components/Chat.tsx | 45 | `return <div>Placeholder</div>` | 🛑 Blocker | Renders no content |
|
||||
| src/hooks/useChat.ts | - | File missing | 🛑 Blocker | Expected hook doesn't exist |
|
||||
|
||||
**Anti-patterns:** {N} found ({blockers} blockers, {warnings} warnings)
|
||||
|
||||
## Human Verification Required
|
||||
|
||||
{If no human verification needed:}
|
||||
None — all verifiable items checked programmatically.
|
||||
|
||||
{If human verification needed:}
|
||||
|
||||
### 1. {Test Name}
|
||||
**Test:** {What to do}
|
||||
**Expected:** {What should happen}
|
||||
**Why human:** {Why can't verify programmatically}
|
||||
|
||||
### 2. {Test Name}
|
||||
**Test:** {What to do}
|
||||
**Expected:** {What should happen}
|
||||
**Why human:** {Why can't verify programmatically}
|
||||
|
||||
## Gaps Summary
|
||||
|
||||
{If no gaps:}
|
||||
**No gaps found.** Phase goal achieved. Ready to proceed.
|
||||
|
||||
{If gaps found:}
|
||||
|
||||
### Critical Gaps (Block Progress)
|
||||
|
||||
1. **{Gap name}**
|
||||
- Missing: {what's missing}
|
||||
- Impact: {why this blocks the goal}
|
||||
- Fix: {what needs to happen}
|
||||
|
||||
2. **{Gap name}**
|
||||
- Missing: {what's missing}
|
||||
- Impact: {why this blocks the goal}
|
||||
- Fix: {what needs to happen}
|
||||
|
||||
### Non-Critical Gaps (Can Defer)
|
||||
|
||||
1. **{Gap name}**
|
||||
- Issue: {what's wrong}
|
||||
- Impact: {limited impact because...}
|
||||
- Recommendation: {fix now or defer}
|
||||
|
||||
## Recommended Fix Plans
|
||||
|
||||
{If gaps found, generate fix plan recommendations:}
|
||||
|
||||
### {phase}-{next}-PLAN.md: {Fix Name}
|
||||
|
||||
**Objective:** {What this fixes}
|
||||
|
||||
**Tasks:**
|
||||
1. {Task to fix gap 1}
|
||||
2. {Task to fix gap 2}
|
||||
3. {Verification task}
|
||||
|
||||
**Estimated scope:** {Small / Medium}
|
||||
|
||||
---
|
||||
|
||||
### {phase}-{next+1}-PLAN.md: {Fix Name}
|
||||
|
||||
**Objective:** {What this fixes}
|
||||
|
||||
**Tasks:**
|
||||
1. {Task}
|
||||
2. {Task}
|
||||
|
||||
**Estimated scope:** {Small / Medium}
|
||||
|
||||
---
|
||||
|
||||
## Verification Metadata
|
||||
|
||||
**Verification approach:** Goal-backward (derived from phase goal)
|
||||
**Must-haves source:** {PLAN.md frontmatter | derived from ROADMAP.md goal}
|
||||
**Automated checks:** {N} passed, {M} failed
|
||||
**Human checks required:** {N}
|
||||
**Total verification time:** {duration}
|
||||
|
||||
---
|
||||
*Verified: {timestamp}*
|
||||
*Verifier: Claude (subagent)*
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Guidelines
|
||||
|
||||
**Status values:**
|
||||
- `passed` — All must-haves verified, no blockers
|
||||
- `gaps_found` — One or more critical gaps found
|
||||
- `human_needed` — Automated checks pass but human verification required
|
||||
|
||||
**Evidence types:**
|
||||
- For EXISTS: "File at path, exports X"
|
||||
- For SUBSTANTIVE: "N lines, has patterns X, Y, Z"
|
||||
- For WIRED: "Line N: code that connects A to B"
|
||||
- For FAILED: "Missing because X" or "Stub because Y"
|
||||
|
||||
**Severity levels:**
|
||||
- 🛑 Blocker: Prevents goal achievement, must fix
|
||||
- ⚠️ Warning: Indicates incomplete but doesn't block
|
||||
- ℹ️ Info: Notable but not problematic
|
||||
|
||||
**Fix plan generation:**
|
||||
- Only generate if gaps_found
|
||||
- Group related fixes into single plans
|
||||
- Keep to 2-3 tasks per plan
|
||||
- Include verification task in each plan
|
||||
|
||||
---
|
||||
|
||||
## Example
|
||||
|
||||
```markdown
|
||||
---
|
||||
phase: 03-chat
|
||||
verified: 2025-01-15T14:30:00Z
|
||||
status: gaps_found
|
||||
score: 2/5 must-haves verified
|
||||
---
|
||||
|
||||
# Phase 3: Chat Interface Verification Report
|
||||
|
||||
**Phase Goal:** Working chat interface where users can send and receive messages
|
||||
**Verified:** 2025-01-15T14:30:00Z
|
||||
**Status:** gaps_found
|
||||
|
||||
## Goal Achievement
|
||||
|
||||
### Observable Truths
|
||||
|
||||
| # | Truth | Status | Evidence |
|
||||
|---|-------|--------|----------|
|
||||
| 1 | User can see existing messages | ✗ FAILED | Component renders placeholder, not message data |
|
||||
| 2 | User can type a message | ✓ VERIFIED | Input field exists with onChange handler |
|
||||
| 3 | User can send a message | ✗ FAILED | onSubmit handler is console.log only |
|
||||
| 4 | Sent message appears in list | ✗ FAILED | No state update after send |
|
||||
| 5 | Messages persist across refresh | ? UNCERTAIN | Can't verify - send doesn't work |
|
||||
|
||||
**Score:** 1/5 truths verified
|
||||
|
||||
### Required Artifacts
|
||||
|
||||
| Artifact | Expected | Status | Details |
|
||||
|----------|----------|--------|---------|
|
||||
| `src/components/Chat.tsx` | Message list component | ✗ STUB | Returns `<div>Chat will be here</div>` |
|
||||
| `src/components/ChatInput.tsx` | Message input | ✓ EXISTS + SUBSTANTIVE | Form with input, submit button, handlers |
|
||||
| `src/app/api/chat/route.ts` | Message CRUD | ✗ STUB | GET returns [], POST returns { ok: true } |
|
||||
| `prisma/schema.prisma` | Message model | ✓ EXISTS + SUBSTANTIVE | Message model with id, content, userId, createdAt |
|
||||
|
||||
**Artifacts:** 2/4 verified
|
||||
|
||||
### Key Link Verification
|
||||
|
||||
| From | To | Via | Status | Details |
|
||||
|------|----|----|--------|---------|
|
||||
| Chat.tsx | /api/chat GET | fetch | ✗ NOT WIRED | No fetch call in component |
|
||||
| ChatInput | /api/chat POST | onSubmit | ✗ NOT WIRED | Handler only logs, doesn't fetch |
|
||||
| /api/chat GET | database | prisma.message.findMany | ✗ NOT WIRED | Returns hardcoded [] |
|
||||
| /api/chat POST | database | prisma.message.create | ✗ NOT WIRED | Returns { ok: true }, no DB call |
|
||||
|
||||
**Wiring:** 0/4 connections verified
|
||||
|
||||
## Requirements Coverage
|
||||
|
||||
| Requirement | Status | Blocking Issue |
|
||||
|-------------|--------|----------------|
|
||||
| CHAT-01: User can send message | ✗ BLOCKED | API POST is stub |
|
||||
| CHAT-02: User can view messages | ✗ BLOCKED | Component is placeholder |
|
||||
| CHAT-03: Messages persist | ✗ BLOCKED | No database integration |
|
||||
|
||||
**Coverage:** 0/3 requirements satisfied
|
||||
|
||||
## Anti-Patterns Found
|
||||
|
||||
| File | Line | Pattern | Severity | Impact |
|
||||
|------|------|---------|----------|--------|
|
||||
| src/components/Chat.tsx | 8 | `<div>Chat will be here</div>` | 🛑 Blocker | No actual content |
|
||||
| src/app/api/chat/route.ts | 5 | `return Response.json([])` | 🛑 Blocker | Hardcoded empty |
|
||||
| src/app/api/chat/route.ts | 12 | `// TODO: save to database` | ⚠️ Warning | Incomplete |
|
||||
|
||||
**Anti-patterns:** 3 found (2 blockers, 1 warning)
|
||||
|
||||
## Human Verification Required
|
||||
|
||||
None needed until automated gaps are fixed.
|
||||
|
||||
## Gaps Summary
|
||||
|
||||
### Critical Gaps (Block Progress)
|
||||
|
||||
1. **Chat component is placeholder**
|
||||
- Missing: Actual message list rendering
|
||||
- Impact: Users see "Chat will be here" instead of messages
|
||||
- Fix: Implement Chat.tsx to fetch and render messages
|
||||
|
||||
2. **API routes are stubs**
|
||||
- Missing: Database integration in GET and POST
|
||||
- Impact: No data persistence, no real functionality
|
||||
- Fix: Wire prisma calls in route handlers
|
||||
|
||||
3. **No wiring between frontend and backend**
|
||||
- Missing: fetch calls in components
|
||||
- Impact: Even if API worked, UI wouldn't call it
|
||||
- Fix: Add useEffect fetch in Chat, onSubmit fetch in ChatInput
|
||||
|
||||
## Recommended Fix Plans
|
||||
|
||||
### 03-04-PLAN.md: Implement Chat API
|
||||
|
||||
**Objective:** Wire API routes to database
|
||||
|
||||
**Tasks:**
|
||||
1. Implement GET /api/chat with prisma.message.findMany
|
||||
2. Implement POST /api/chat with prisma.message.create
|
||||
3. Verify: API returns real data, POST creates records
|
||||
|
||||
**Estimated scope:** Small
|
||||
|
||||
---
|
||||
|
||||
### 03-05-PLAN.md: Implement Chat UI
|
||||
|
||||
**Objective:** Wire Chat component to API
|
||||
|
||||
**Tasks:**
|
||||
1. Implement Chat.tsx with useEffect fetch and message rendering
|
||||
2. Wire ChatInput onSubmit to POST /api/chat
|
||||
3. Verify: Messages display, new messages appear after send
|
||||
|
||||
**Estimated scope:** Small
|
||||
|
||||
---
|
||||
|
||||
## Verification Metadata
|
||||
|
||||
**Verification approach:** Goal-backward (derived from phase goal)
|
||||
**Must-haves source:** 03-01-PLAN.md frontmatter
|
||||
**Automated checks:** 2 passed, 8 failed
|
||||
**Human checks required:** 0 (blocked by automated failures)
|
||||
**Total verification time:** 2 min
|
||||
|
||||
---
|
||||
*Verified: 2025-01-15T14:30:00Z*
|
||||
*Verifier: Claude (subagent)*
|
||||
```
|
||||
629
get-shit-done/workflows/verify-phase.md
Normal file
629
get-shit-done/workflows/verify-phase.md
Normal file
@@ -0,0 +1,629 @@
|
||||
<purpose>
|
||||
Verify phase goal achievement through goal-backward analysis. Check that the codebase actually delivers what the phase promised, not just that tasks were completed.
|
||||
|
||||
This workflow is executed by a verification subagent spawned from execute-phase.md.
|
||||
</purpose>
|
||||
|
||||
<core_principle>
|
||||
**Task completion ≠ Goal achievement**
|
||||
|
||||
A task "create chat component" can be marked complete when the component is a placeholder. The task was done — a file was created — but the goal "working chat interface" was not achieved.
|
||||
|
||||
Goal-backward verification starts from the outcome and works backwards:
|
||||
1. What must be TRUE for the goal to be achieved?
|
||||
2. What must EXIST for those truths to hold?
|
||||
3. What must be WIRED for those artifacts to function?
|
||||
|
||||
Then verify each level against the actual codebase.
|
||||
</core_principle>
|
||||
|
||||
<required_reading>
|
||||
**Load these references:**
|
||||
- ~/.claude/get-shit-done/references/goal-backward.md (derivation process)
|
||||
- ~/.claude/get-shit-done/references/verification-patterns.md (detection patterns)
|
||||
- ~/.claude/get-shit-done/templates/verification-report.md (output format)
|
||||
</required_reading>
|
||||
|
||||
<process>
|
||||
|
||||
<step name="load_context" priority="first">
|
||||
**Gather all verification context:**
|
||||
|
||||
```bash
|
||||
# Phase directory
|
||||
PHASE_DIR=$(ls -d .planning/phases/${PHASE_ARG}* 2>/dev/null | head -1)
|
||||
|
||||
# Phase goal from ROADMAP
|
||||
grep -A 5 "Phase ${PHASE_NUM}" .planning/ROADMAP.md
|
||||
|
||||
# Requirements mapped to this phase
|
||||
grep -E "^| ${PHASE_NUM}" .planning/REQUIREMENTS.md 2>/dev/null
|
||||
|
||||
# All SUMMARY files (claims to verify)
|
||||
ls "$PHASE_DIR"/*-SUMMARY.md 2>/dev/null
|
||||
|
||||
# All PLAN files (for must_haves in frontmatter)
|
||||
ls "$PHASE_DIR"/*-PLAN.md 2>/dev/null
|
||||
```
|
||||
|
||||
**Extract phase goal:** Parse ROADMAP.md for this phase's goal/description. This is the outcome to verify, not the tasks.
|
||||
|
||||
**Extract requirements:** If REQUIREMENTS.md exists, find requirements mapped to this phase. These become additional verification targets.
|
||||
</step>
|
||||
|
||||
<step name="establish_must_haves">
|
||||
**Determine what must be verified.**
|
||||
|
||||
**Option A: Must-haves in PLAN frontmatter**
|
||||
|
||||
Check if any PLAN.md has `must_haves` in frontmatter:
|
||||
|
||||
```bash
|
||||
grep -l "must_haves:" "$PHASE_DIR"/*-PLAN.md 2>/dev/null
|
||||
```
|
||||
|
||||
If found, extract and use:
|
||||
```yaml
|
||||
must_haves:
|
||||
truths:
|
||||
- "User can see existing messages"
|
||||
- "User can send a message"
|
||||
artifacts:
|
||||
- path: "src/components/Chat.tsx"
|
||||
provides: "Message list rendering"
|
||||
key_links:
|
||||
- from: "Chat.tsx"
|
||||
to: "api/chat"
|
||||
via: "fetch in useEffect"
|
||||
```
|
||||
|
||||
**Option B: Derive from phase goal**
|
||||
|
||||
If no must_haves in frontmatter, derive using goal-backward process:
|
||||
|
||||
1. **State the goal:** Take phase goal from ROADMAP.md
|
||||
|
||||
2. **Derive truths:** Ask "What must be TRUE for this goal to be achieved?"
|
||||
- List 3-7 observable behaviors from user perspective
|
||||
- Each truth should be testable by a human using the app
|
||||
|
||||
3. **Derive artifacts:** For each truth, ask "What must EXIST?"
|
||||
- Map truths to concrete files (components, routes, schemas)
|
||||
- Be specific: `src/components/Chat.tsx`, not "chat component"
|
||||
|
||||
4. **Derive key links:** For each artifact, ask "What must be CONNECTED?"
|
||||
- Identify critical wiring (component calls API, API queries DB)
|
||||
- These are where stubs hide
|
||||
|
||||
5. **Document derived must-haves** before proceeding to verification.
|
||||
|
||||
See ~/.claude/get-shit-done/references/goal-backward.md for detailed derivation guidance.
|
||||
</step>
|
||||
|
||||
<step name="verify_truths">
|
||||
**For each observable truth, determine if codebase enables it.**
|
||||
|
||||
A truth is achievable if the supporting artifacts exist, are substantive, and are wired correctly.
|
||||
|
||||
**Verification status:**
|
||||
- ✓ VERIFIED: All supporting artifacts pass all checks
|
||||
- ✗ FAILED: One or more supporting artifacts missing, stub, or unwired
|
||||
- ? UNCERTAIN: Can't verify programmatically (needs human)
|
||||
|
||||
**For each truth:**
|
||||
|
||||
1. Identify supporting artifacts (which files make this truth possible?)
|
||||
2. Check artifact status (see verify_artifacts step)
|
||||
3. Check wiring status (see verify_wiring step)
|
||||
4. Determine truth status based on supporting infrastructure
|
||||
|
||||
**Example:**
|
||||
|
||||
Truth: "User can see existing messages"
|
||||
|
||||
Supporting artifacts:
|
||||
- Chat.tsx (renders messages)
|
||||
- /api/chat GET (provides messages)
|
||||
- Message model (defines schema)
|
||||
|
||||
If Chat.tsx is a stub → Truth FAILED
|
||||
If /api/chat GET returns hardcoded [] → Truth FAILED
|
||||
If Chat.tsx exists, is substantive, calls API, renders response → Truth VERIFIED
|
||||
</step>
|
||||
|
||||
<step name="verify_artifacts">
|
||||
**For each required artifact, verify three levels:**
|
||||
|
||||
### Level 1: Existence
|
||||
|
||||
```bash
|
||||
check_exists() {
|
||||
local path="$1"
|
||||
if [ -f "$path" ]; then
|
||||
echo "EXISTS"
|
||||
elif [ -d "$path" ]; then
|
||||
echo "EXISTS (directory)"
|
||||
else
|
||||
echo "MISSING"
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
If MISSING → artifact fails, record and continue to next artifact.
|
||||
|
||||
### Level 2: Substantive
|
||||
|
||||
Check that the file has real implementation, not a stub.
|
||||
|
||||
**Line count check:**
|
||||
```bash
|
||||
check_length() {
|
||||
local path="$1"
|
||||
local min_lines="$2"
|
||||
local lines=$(wc -l < "$path" 2>/dev/null || echo 0)
|
||||
[ "$lines" -ge "$min_lines" ] && echo "SUBSTANTIVE ($lines lines)" || echo "THIN ($lines lines)"
|
||||
}
|
||||
```
|
||||
|
||||
Minimum lines by type:
|
||||
- Component: 15+ lines
|
||||
- API route: 10+ lines
|
||||
- Hook/util: 10+ lines
|
||||
- Schema model: 5+ lines
|
||||
|
||||
**Stub pattern check:**
|
||||
```bash
|
||||
check_stubs() {
|
||||
local path="$1"
|
||||
|
||||
# Universal stub patterns
|
||||
local stubs=$(grep -c -E "TODO|FIXME|placeholder|not implemented|coming soon" "$path" 2>/dev/null || echo 0)
|
||||
|
||||
# Empty returns
|
||||
local empty=$(grep -c -E "return null|return undefined|return \{\}|return \[\]" "$path" 2>/dev/null || echo 0)
|
||||
|
||||
# Placeholder content
|
||||
local placeholder=$(grep -c -E "will be here|placeholder|lorem ipsum" "$path" 2>/dev/null || echo 0)
|
||||
|
||||
local total=$((stubs + empty + placeholder))
|
||||
[ "$total" -gt 0 ] && echo "STUB_PATTERNS ($total found)" || echo "NO_STUBS"
|
||||
}
|
||||
```
|
||||
|
||||
**Export check (for components/hooks):**
|
||||
```bash
|
||||
check_exports() {
|
||||
local path="$1"
|
||||
grep -E "^export (default )?(function|const|class)" "$path" && echo "HAS_EXPORTS" || echo "NO_EXPORTS"
|
||||
}
|
||||
```
|
||||
|
||||
**Combine level 2 results:**
|
||||
- SUBSTANTIVE: Adequate length + no stubs + has exports
|
||||
- STUB: Too short OR has stub patterns OR no exports
|
||||
- PARTIAL: Mixed signals (length OK but has some stubs)
|
||||
|
||||
### Level 3: Wired
|
||||
|
||||
Check that the artifact is connected to the system.
|
||||
|
||||
**Import check (is it used?):**
|
||||
```bash
|
||||
check_imported() {
|
||||
local artifact_name="$1"
|
||||
local search_path="${2:-src/}"
|
||||
|
||||
# Find imports of this artifact
|
||||
local imports=$(grep -r "import.*$artifact_name" "$search_path" --include="*.ts" --include="*.tsx" 2>/dev/null | wc -l)
|
||||
|
||||
[ "$imports" -gt 0 ] && echo "IMPORTED ($imports times)" || echo "NOT_IMPORTED"
|
||||
}
|
||||
```
|
||||
|
||||
**Usage check (is it called?):**
|
||||
```bash
|
||||
check_used() {
|
||||
local artifact_name="$1"
|
||||
local search_path="${2:-src/}"
|
||||
|
||||
# Find usages (function calls, component renders, etc.)
|
||||
local uses=$(grep -r "$artifact_name" "$search_path" --include="*.ts" --include="*.tsx" 2>/dev/null | grep -v "import" | wc -l)
|
||||
|
||||
[ "$uses" -gt 0 ] && echo "USED ($uses times)" || echo "NOT_USED"
|
||||
}
|
||||
```
|
||||
|
||||
**Combine level 3 results:**
|
||||
- WIRED: Imported AND used
|
||||
- ORPHANED: Exists but not imported/used
|
||||
- PARTIAL: Imported but not used (or vice versa)
|
||||
|
||||
### Final artifact status
|
||||
|
||||
| Exists | Substantive | Wired | Status |
|
||||
|--------|-------------|-------|--------|
|
||||
| ✓ | ✓ | ✓ | ✓ VERIFIED |
|
||||
| ✓ | ✓ | ✗ | ⚠️ ORPHANED |
|
||||
| ✓ | ✗ | - | ✗ STUB |
|
||||
| ✗ | - | - | ✗ MISSING |
|
||||
|
||||
Record status and evidence for each artifact.
|
||||
</step>
|
||||
|
||||
<step name="verify_wiring">
|
||||
**Verify key links between artifacts.**
|
||||
|
||||
Key links are critical connections. If broken, the goal fails even with all artifacts present.
|
||||
|
||||
### Pattern: Component → API
|
||||
|
||||
Check if component actually calls the API:
|
||||
|
||||
```bash
|
||||
verify_component_api_link() {
|
||||
local component="$1"
|
||||
local api_path="$2"
|
||||
|
||||
# Check for fetch/axios call to the API
|
||||
local has_call=$(grep -E "fetch\(['\"].*$api_path|axios\.(get|post).*$api_path" "$component" 2>/dev/null)
|
||||
|
||||
if [ -n "$has_call" ]; then
|
||||
# Check if response is used
|
||||
local uses_response=$(grep -A 5 "fetch\|axios" "$component" | grep -E "await|\.then|setData|setState" 2>/dev/null)
|
||||
|
||||
if [ -n "$uses_response" ]; then
|
||||
echo "WIRED: $component → $api_path (call + response handling)"
|
||||
else
|
||||
echo "PARTIAL: $component → $api_path (call exists but response not used)"
|
||||
fi
|
||||
else
|
||||
echo "NOT_WIRED: $component → $api_path (no call found)"
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern: API → Database
|
||||
|
||||
Check if API route queries database:
|
||||
|
||||
```bash
|
||||
verify_api_db_link() {
|
||||
local route="$1"
|
||||
local model="$2"
|
||||
|
||||
# Check for Prisma/DB call
|
||||
local has_query=$(grep -E "prisma\.$model|db\.$model|$model\.(find|create|update|delete)" "$route" 2>/dev/null)
|
||||
|
||||
if [ -n "$has_query" ]; then
|
||||
# Check if result is returned
|
||||
local returns_result=$(grep -E "return.*json.*\w+|res\.json\(\w+" "$route" 2>/dev/null)
|
||||
|
||||
if [ -n "$returns_result" ]; then
|
||||
echo "WIRED: $route → database ($model)"
|
||||
else
|
||||
echo "PARTIAL: $route → database (query exists but result not returned)"
|
||||
fi
|
||||
else
|
||||
echo "NOT_WIRED: $route → database (no query for $model)"
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern: Form → Handler
|
||||
|
||||
Check if form submission does something:
|
||||
|
||||
```bash
|
||||
verify_form_handler_link() {
|
||||
local component="$1"
|
||||
|
||||
# Find onSubmit handler
|
||||
local has_handler=$(grep -E "onSubmit=\{|handleSubmit" "$component" 2>/dev/null)
|
||||
|
||||
if [ -n "$has_handler" ]; then
|
||||
# Check if handler has real implementation
|
||||
local handler_content=$(grep -A 10 "onSubmit.*=" "$component" | grep -E "fetch|axios|mutate|dispatch" 2>/dev/null)
|
||||
|
||||
if [ -n "$handler_content" ]; then
|
||||
echo "WIRED: form → handler (has API call)"
|
||||
else
|
||||
# Check for stub patterns
|
||||
local is_stub=$(grep -A 5 "onSubmit" "$component" | grep -E "console\.log|preventDefault\(\)$|\{\}" 2>/dev/null)
|
||||
if [ -n "$is_stub" ]; then
|
||||
echo "STUB: form → handler (only logs or empty)"
|
||||
else
|
||||
echo "PARTIAL: form → handler (exists but unclear implementation)"
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "NOT_WIRED: form → handler (no onSubmit found)"
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern: State → Render
|
||||
|
||||
Check if state is actually rendered:
|
||||
|
||||
```bash
|
||||
verify_state_render_link() {
|
||||
local component="$1"
|
||||
local state_var="$2"
|
||||
|
||||
# Check if state variable exists
|
||||
local has_state=$(grep -E "useState.*$state_var|\[$state_var," "$component" 2>/dev/null)
|
||||
|
||||
if [ -n "$has_state" ]; then
|
||||
# Check if state is used in JSX
|
||||
local renders_state=$(grep -E "\{.*$state_var.*\}|\{$state_var\." "$component" 2>/dev/null)
|
||||
|
||||
if [ -n "$renders_state" ]; then
|
||||
echo "WIRED: state → render ($state_var displayed)"
|
||||
else
|
||||
echo "NOT_WIRED: state → render ($state_var exists but not displayed)"
|
||||
fi
|
||||
else
|
||||
echo "N/A: state → render (no state var $state_var)"
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
### Aggregate key link results
|
||||
|
||||
For each key link in must_haves:
|
||||
- Run appropriate verification function
|
||||
- Record status and evidence
|
||||
- WIRED / PARTIAL / STUB / NOT_WIRED
|
||||
</step>
|
||||
|
||||
<step name="verify_requirements">
|
||||
**Check requirements coverage if REQUIREMENTS.md exists.**
|
||||
|
||||
```bash
|
||||
# Find requirements mapped to this phase
|
||||
grep -E "Phase ${PHASE_NUM}" .planning/REQUIREMENTS.md 2>/dev/null
|
||||
```
|
||||
|
||||
For each requirement:
|
||||
1. Parse requirement description
|
||||
2. Identify which truths/artifacts support it
|
||||
3. Determine status based on supporting infrastructure
|
||||
|
||||
**Requirement status:**
|
||||
- ✓ SATISFIED: All supporting truths verified
|
||||
- ✗ BLOCKED: One or more supporting truths failed
|
||||
- ? NEEDS HUMAN: Can't verify requirement programmatically
|
||||
</step>
|
||||
|
||||
<step name="scan_antipatterns">
|
||||
**Scan for anti-patterns across phase files.**
|
||||
|
||||
Identify files modified in this phase:
|
||||
```bash
|
||||
# Extract files from SUMMARY.md
|
||||
grep -E "^\- \`" "$PHASE_DIR"/*-SUMMARY.md | sed 's/.*`\([^`]*\)`.*/\1/' | sort -u
|
||||
```
|
||||
|
||||
Run anti-pattern detection:
|
||||
```bash
|
||||
scan_antipatterns() {
|
||||
local files="$@"
|
||||
|
||||
echo "## Anti-Patterns Found"
|
||||
echo ""
|
||||
|
||||
for file in $files; do
|
||||
[ -f "$file" ] || continue
|
||||
|
||||
# TODO/FIXME comments
|
||||
grep -n -E "TODO|FIXME|XXX|HACK" "$file" 2>/dev/null | while read line; do
|
||||
echo "| $file | $(echo $line | cut -d: -f1) | TODO/FIXME | ⚠️ Warning |"
|
||||
done
|
||||
|
||||
# Placeholder content
|
||||
grep -n -E "placeholder|coming soon|will be here" "$file" -i 2>/dev/null | while read line; do
|
||||
echo "| $file | $(echo $line | cut -d: -f1) | Placeholder | 🛑 Blocker |"
|
||||
done
|
||||
|
||||
# Empty implementations
|
||||
grep -n -E "return null|return \{\}|return \[\]|=> \{\}" "$file" 2>/dev/null | while read line; do
|
||||
echo "| $file | $(echo $line | cut -d: -f1) | Empty return | ⚠️ Warning |"
|
||||
done
|
||||
|
||||
# Console.log only implementations
|
||||
grep -n -B 2 -A 2 "console\.log" "$file" 2>/dev/null | grep -E "^\s*(const|function|=>)" | while read line; do
|
||||
echo "| $file | - | Log-only function | ⚠️ Warning |"
|
||||
done
|
||||
done
|
||||
}
|
||||
```
|
||||
|
||||
Categorize findings:
|
||||
- 🛑 Blocker: Prevents goal achievement (placeholder renders, empty handlers)
|
||||
- ⚠️ Warning: Indicates incomplete (TODO comments, console.log)
|
||||
- ℹ️ Info: Notable but not problematic
|
||||
</step>
|
||||
|
||||
<step name="identify_human_verification">
|
||||
**Flag items that need human verification.**
|
||||
|
||||
Some things can't be verified programmatically:
|
||||
|
||||
**Always needs human:**
|
||||
- Visual appearance (does it look right?)
|
||||
- User flow completion (can you do the full task?)
|
||||
- Real-time behavior (WebSocket, SSE updates)
|
||||
- External service integration (payments, email)
|
||||
- Performance feel (does it feel fast?)
|
||||
- Error message clarity
|
||||
|
||||
**Needs human if uncertain:**
|
||||
- Complex wiring that grep can't trace
|
||||
- Dynamic behavior depending on state
|
||||
- Edge cases and error states
|
||||
|
||||
**Format for human verification:**
|
||||
```markdown
|
||||
## Human Verification Required
|
||||
|
||||
### 1. {Test Name}
|
||||
**Test:** {What to do}
|
||||
**Expected:** {What should happen}
|
||||
**Why human:** {Why can't verify programmatically}
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="determine_status">
|
||||
**Calculate overall verification status.**
|
||||
|
||||
**Status: passed**
|
||||
- All truths VERIFIED
|
||||
- All artifacts pass level 1-3
|
||||
- All key links WIRED
|
||||
- No blocker anti-patterns
|
||||
- (Human verification items are OK — will be prompted)
|
||||
|
||||
**Status: gaps_found**
|
||||
- One or more truths FAILED
|
||||
- OR one or more artifacts MISSING/STUB
|
||||
- OR one or more key links NOT_WIRED
|
||||
- OR blocker anti-patterns found
|
||||
|
||||
**Status: human_needed**
|
||||
- All automated checks pass
|
||||
- BUT items flagged for human verification
|
||||
- Can't determine goal achievement without human
|
||||
|
||||
**Calculate score:**
|
||||
```
|
||||
score = (verified_truths / total_truths)
|
||||
```
|
||||
</step>
|
||||
|
||||
<step name="generate_fix_plans">
|
||||
**If gaps_found, recommend fix plans.**
|
||||
|
||||
Group related gaps into fix plans:
|
||||
|
||||
1. **Identify gap clusters:**
|
||||
- API stub + component not wired → "Wire frontend to backend"
|
||||
- Multiple artifacts missing → "Complete core implementation"
|
||||
- Wiring issues only → "Connect existing components"
|
||||
|
||||
2. **Generate plan recommendations:**
|
||||
|
||||
```markdown
|
||||
### {phase}-{next}-PLAN.md: {Fix Name}
|
||||
|
||||
**Objective:** {What this fixes}
|
||||
|
||||
**Tasks:**
|
||||
1. {Task to fix gap 1}
|
||||
- Files: {files to modify}
|
||||
- Action: {specific fix}
|
||||
- Verify: {how to confirm fix}
|
||||
|
||||
2. {Task to fix gap 2}
|
||||
- Files: {files to modify}
|
||||
- Action: {specific fix}
|
||||
- Verify: {how to confirm fix}
|
||||
|
||||
3. Re-verify phase goal
|
||||
- Run verification again
|
||||
- Confirm all must-haves pass
|
||||
|
||||
**Estimated scope:** {Small / Medium}
|
||||
```
|
||||
|
||||
3. **Keep plans focused:**
|
||||
- 2-3 tasks per plan
|
||||
- Single concern per plan
|
||||
- Include verification task
|
||||
|
||||
4. **Order by dependency:**
|
||||
- Fix missing artifacts before wiring
|
||||
- Fix stubs before integration
|
||||
- Verify after all fixes
|
||||
</step>
|
||||
|
||||
<step name="create_report">
|
||||
**Generate VERIFICATION.md using template.**
|
||||
|
||||
```bash
|
||||
REPORT_PATH="$PHASE_DIR/${PHASE_NUM}-VERIFICATION.md"
|
||||
```
|
||||
|
||||
Fill template sections:
|
||||
1. **Frontmatter:** phase, verified timestamp, status, score
|
||||
2. **Goal Achievement:** Truth verification table
|
||||
3. **Required Artifacts:** Artifact verification table
|
||||
4. **Key Link Verification:** Wiring verification table
|
||||
5. **Requirements Coverage:** If REQUIREMENTS.md exists
|
||||
6. **Anti-Patterns Found:** Scan results table
|
||||
7. **Human Verification Required:** Items needing human
|
||||
8. **Gaps Summary:** Critical and non-critical gaps
|
||||
9. **Recommended Fix Plans:** If gaps_found
|
||||
10. **Verification Metadata:** Approach, timing, counts
|
||||
|
||||
See ~/.claude/get-shit-done/templates/verification-report.md for complete template.
|
||||
</step>
|
||||
|
||||
<step name="return_to_orchestrator">
|
||||
**Return results to execute-phase orchestrator.**
|
||||
|
||||
**Return format:**
|
||||
|
||||
```markdown
|
||||
## Verification Complete
|
||||
|
||||
**Status:** {passed | gaps_found | human_needed}
|
||||
**Score:** {N}/{M} must-haves verified
|
||||
**Report:** .planning/phases/{phase_dir}/{phase}-VERIFICATION.md
|
||||
|
||||
{If passed:}
|
||||
All must-haves verified. Phase goal achieved. Ready to proceed.
|
||||
|
||||
{If gaps_found:}
|
||||
### Gaps Found
|
||||
|
||||
{N} critical gaps blocking goal achievement:
|
||||
1. {Gap 1 summary}
|
||||
2. {Gap 2 summary}
|
||||
|
||||
### Recommended Fixes
|
||||
|
||||
{N} fix plans recommended:
|
||||
1. {phase}-{next}-PLAN.md: {name}
|
||||
2. {phase}-{next+1}-PLAN.md: {name}
|
||||
|
||||
{If human_needed:}
|
||||
### Human Verification Required
|
||||
|
||||
{N} items need human testing:
|
||||
1. {Item 1}
|
||||
2. {Item 2}
|
||||
|
||||
Automated checks passed. Awaiting human verification.
|
||||
```
|
||||
|
||||
The orchestrator will:
|
||||
- If `passed`: Continue to update_roadmap
|
||||
- If `gaps_found`: Create and execute fix plans, then re-verify
|
||||
- If `human_needed`: Present items to user, collect responses
|
||||
</step>
|
||||
|
||||
</process>
|
||||
|
||||
<success_criteria>
|
||||
- [ ] Must-haves established (from frontmatter or derived)
|
||||
- [ ] All truths verified with status and evidence
|
||||
- [ ] All artifacts checked at all three levels
|
||||
- [ ] All key links verified
|
||||
- [ ] Requirements coverage assessed (if applicable)
|
||||
- [ ] Anti-patterns scanned and categorized
|
||||
- [ ] Human verification items identified
|
||||
- [ ] Overall status determined
|
||||
- [ ] Fix plans generated (if gaps_found)
|
||||
- [ ] VERIFICATION.md created with complete report
|
||||
- [ ] Results returned to orchestrator
|
||||
</success_criteria>
|
||||
Reference in New Issue
Block a user