* feat: /gsd:ai-phase + /gsd:eval-review — AI evals and framework selection layer Adds a structured AI development layer to GSD with 5 new agents, 2 new commands, 2 new workflows, 2 reference files, and 1 template. Commands: - /gsd:ai-phase [N] — pre-planning AI design contract (inserts between discuss-phase and plan-phase). Orchestrates 4 agents in sequence: framework-selector → ai-researcher → domain-researcher → eval-planner. Output: AI-SPEC.md with framework decision, implementation guidance, domain expert context, and evaluation strategy. - /gsd:eval-review [N] — retroactive eval coverage audit. Scores each planned eval dimension as COVERED/PARTIAL/MISSING. Output: EVAL-REVIEW.md with 0-100 score, verdict, and remediation plan. Agents: - gsd-framework-selector: interactive decision matrix (6 questions) → scored framework recommendation for CrewAI, LlamaIndex, LangChain, LangGraph, OpenAI Agents SDK, Claude Agent SDK, AutoGen/AG2, Haystack - gsd-ai-researcher: fetches official framework docs + writes AI systems best practices (Pydantic structured outputs, async-first, prompt discipline, context window management, cost/latency budget) - gsd-domain-researcher: researches business domain and use-case context — surfaces domain expert evaluation criteria, industry failure modes, regulatory constraints, and practitioner rubric ingredients before eval-planner writes measurable criteria - gsd-eval-planner: designs evaluation strategy grounded in domain context; defaults to Arize Phoenix (tracing) + RAGAS (RAG eval) with detect-first guard for existing tooling - gsd-eval-auditor: retroactive codebase scan → scores eval coverage Integration points: - plan-phase: non-blocking nudge (step 4.5) when AI keywords detected and no AI-SPEC.md present - settings: new workflow.ai_phase toggle (default on) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: refine ai-integration-phase layer — rename, house style, consistency fixes Amends the ai-evals framework layer (df8cb6c) with post-review improvements before opening upstream PR. Rename /gsd:ai-phase → /gsd:ai-integration-phase: - Renamed commands/gsd/ai-phase.md → ai-integration-phase.md - Renamed get-shit-done/workflows/ai-phase.md → ai-integration-phase.md - Updated config key: workflow.ai_phase → workflow.ai_integration_phase - Updated repair action: addAiPhaseKey → addAiIntegrationPhaseKey - Updated all 84 cross-references across agents, workflows, templates, tests Consistency fixes (same class as PR #1380 review): - commands/gsd: objective described 3-agent chain, missing gsd-domain-researcher - workflows/ai-integration-phase: purpose tag described 3-agent chain + "locks three things" — updated to 4 agents + 4 outputs - workflows/ai-integration-phase: missing DOMAIN_MODEL resolve-model call in step 1 (domain-researcher was spawned in step 7.5 with no model variable) - workflows/ai-integration-phase: fractional step ## 7.5 renumbered to integers (steps 8–12 shifted) Agent house style (GSD meta-prompting conformance): - All 5 new agents refactored to execution_flow + step name="" structure - Role blocks compressed to 2 lines (removed verbose "Core responsibilities") - Added skills: frontmatter to all 5 agents (agent-frontmatter tests) - Added # hooks: commented pattern to file-writing agents - Added ALWAYS use Write tool anti-heredoc instruction to file-writing agents - Line reductions: ai-researcher −41%, domain-researcher −25%, eval-planner −26%, eval-auditor −25%, framework-selector −9% Test coverage (tests/ai-evals.test.cjs — 48 tests): - CONFIG: workflow.ai_integration_phase defaults and config-set/get - HEALTH: W010 warning emission and addAiIntegrationPhaseKey repair - TEMPLATE: AI-SPEC.md section completeness (10 sections) - COMMAND: ai-integration-phase + eval-review frontmatter validity - AGENTS: all 5 new agent files exist - REFERENCES: ai-evals.md + ai-frameworks.md exist and are non-empty - WORKFLOW: plan-phase nudge integration, workflow files exist + agent coverage 603/603 tests passing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add Google ADK to framework selector and reference matrix Google ADK (released March 2025) was missing from the framework options. Adds Python + Java multi-agent framework optimised for Gemini / Vertex AI. - get-shit-done/references/ai-frameworks.md: add Google ADK profile (type, language, model support, best for, avoid if, strengths, weaknesses, eval concerns); update Quick Picks, By System Type, and By Model Commitment tables - agents/gsd-framework-selector.md: add "Google (Gemini)" to model provider interview question - agents/gsd-ai-researcher.md: add Google ADK docs URL to documentation_sources Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: adapt to upstream conventions post-rebase - Remove skills: frontmatter from all 5 new agents (upstream changed convention — skills: breaks Gemini CLI and must not be present) - Add workflow.ai_integration_phase to VALID_CONFIG_KEYS whitelist in config.cjs (config-set blocked unknown keys) - Add ai_integration_phase: true to CONFIG_DEFAULTS in core.cjs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: rephrase 4b.1 line to avoid false-positive in prompt-injection scan "contract as a Pydantic model" matched the `act as a` pattern case-insensitively. Rephrased to "output schema using a Pydantic model". Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: adapt to upstream conventions (W016, colon refs, config docs) - Replace verify.cjs from upstream to restore W010-W015 + cmdValidateAgents, lost when rebase conflict was resolved with --theirs - Add W016 (workflow.ai_integration_phase absent) inside the config try block, avoids collision with upstream's W010 agent-installation check - Add addAiIntegrationPhaseKey repair case mirroring addNyquistKey pattern - Replace /gsd: colon format with /gsd- hyphen format across all new files (agents, workflows, templates, verify.cjs) per stale-colon-refs guard (#1748) - Add workflow.ai_integration_phase to planning-config.md reference table - Add ai_integration_phase → workflow.ai_integration_phase to NAMESPACE_MAP in config-field-docs.test.cjs so CONFIG_DEFAULTS coverage check passes - Update ai-evals tests to use W016 instead of W010 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: add 5 new agents to E2E Copilot install expected list gsd-ai-researcher, gsd-domain-researcher, gsd-eval-auditor, gsd-eval-planner, gsd-framework-selector added to the hardcoded expected agent list in copilot-install.test.cjs (#1890). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
12 KiB
12 KiB
Interactive configuration of GSD workflow agents (research, plan_check, verifier) and model profile selection via multi-question prompt. Updates .planning/config.json with user preferences. Optionally saves settings as global defaults (~/.gsd/defaults.json) for future projects.
Merge new settings into existing config.json:
Display:
<required_reading> Read all files referenced by the invoking prompt's execution_context before starting. </required_reading>
Ensure config exists and load current state:node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" config-ensure-section
INIT=$(node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" state load)
if [[ "$INIT" == @file:* ]]; then INIT=$(cat "${INIT#@file:}"); fi
Creates .planning/config.json with defaults if missing and loads current config values.
Parse current values (default to true if not present):
workflow.research— spawn researcher during plan-phaseworkflow.plan_check— spawn plan checker during plan-phaseworkflow.verifier— spawn verifier during execute-phaseworkflow.nyquist_validation— validation architecture research during plan-phase (default: true if absent)workflow.ui_phase— generate UI-SPEC.md design contracts for frontend phases (default: true if absent)workflow.ui_safety_gate— prompt to run /gsd-ui-phase before planning frontend phases (default: true if absent)workflow.ai_integration_phase— framework selection + eval strategy for AI phases (default: true if absent)model_profile— which model each agent uses (default:balanced)git.branching_strategy— branching approach (default:"none")workflow.use_worktrees— whether parallel executor agents run in worktree isolation (default:true)
AskUserQuestion([
{
question: "Which model profile for agents?",
header: "Model",
multiSelect: false,
options: [
{ label: "Quality", description: "Opus everywhere except verification (highest cost)" },
{ label: "Balanced (Recommended)", description: "Opus for planning, Sonnet for research/execution/verification" },
{ label: "Budget", description: "Sonnet for writing, Haiku for research/verification (lowest cost)" },
{ label: "Inherit", description: "Use current session model for all agents (best for OpenRouter, local models, or runtime model switching)" }
]
},
{
question: "Spawn Plan Researcher? (researches domain before planning)",
header: "Research",
multiSelect: false,
options: [
{ label: "Yes", description: "Research phase goals before planning" },
{ label: "No", description: "Skip research, plan directly" }
]
},
{
question: "Spawn Plan Checker? (verifies plans before execution)",
header: "Plan Check",
multiSelect: false,
options: [
{ label: "Yes", description: "Verify plans meet phase goals" },
{ label: "No", description: "Skip plan verification" }
]
},
{
question: "Spawn Execution Verifier? (verifies phase completion)",
header: "Verifier",
multiSelect: false,
options: [
{ label: "Yes", description: "Verify must-haves after execution" },
{ label: "No", description: "Skip post-execution verification" }
]
},
{
question: "Auto-advance pipeline? (discuss → plan → execute automatically)",
header: "Auto",
multiSelect: false,
options: [
{ label: "No (Recommended)", description: "Manual /clear + paste between stages" },
{ label: "Yes", description: "Chain stages via Task() subagents (same isolation)" }
]
},
{
question: "Enable Nyquist Validation? (researches test coverage during planning)",
header: "Nyquist",
multiSelect: false,
options: [
{ label: "Yes (Recommended)", description: "Research automated test coverage during plan-phase. Adds validation requirements to plans. Blocks approval if tasks lack automated verify." },
{ label: "No", description: "Skip validation research. Good for rapid prototyping or no-test phases." }
]
},
// Note: Nyquist validation depends on research output. If research is disabled,
// plan-phase automatically skips Nyquist steps (no RESEARCH.md to extract from).
{
question: "Enable UI Phase? (generates UI-SPEC.md design contracts for frontend phases)",
header: "UI Phase",
multiSelect: false,
options: [
{ label: "Yes (Recommended)", description: "Generate UI design contracts before planning frontend phases. Locks spacing, typography, color, and copywriting." },
{ label: "No", description: "Skip UI-SPEC generation. Good for backend-only projects or API phases." }
]
},
{
question: "Enable UI Safety Gate? (prompts to run /gsd-ui-phase before planning frontend phases)",
header: "UI Gate",
multiSelect: false,
options: [
{ label: "Yes (Recommended)", description: "plan-phase asks to run /gsd-ui-phase first when frontend indicators detected." },
{ label: "No", description: "No prompt — plan-phase proceeds without UI-SPEC check." }
]
},
{
question: "Enable AI Phase? (framework selection + eval strategy for AI phases)",
header: "AI Phase",
multiSelect: false,
options: [
{ label: "Yes (Recommended)", description: "Run /gsd-ai-phase before planning AI system phases. Surfaces the right framework, researches its docs, and designs the evaluation strategy." },
{ label: "No", description: "Skip AI design contract. Good for non-AI phases or when framework is already decided." }
]
},
{
question: "Git branching strategy?",
header: "Branching",
multiSelect: false,
options: [
{ label: "None (Recommended)", description: "Commit directly to current branch" },
{ label: "Per Phase", description: "Create branch for each phase (gsd/phase-{N}-{name})" },
{ label: "Per Milestone", description: "Create branch for entire milestone (gsd/{version}-{name})" }
]
},
{
question: "Enable context window warnings? (injects advisory messages when context is getting full)",
header: "Ctx Warnings",
multiSelect: false,
options: [
{ label: "Yes (Recommended)", description: "Warn when context usage exceeds 65%. Helps avoid losing work." },
{ label: "No", description: "Disable warnings. Allows Claude to reach auto-compact naturally. Good for long unattended runs." }
]
},
{
question: "Research best practices before asking questions? (web search during new-project and discuss-phase)",
header: "Research Qs",
multiSelect: false,
options: [
{ label: "No (Recommended)", description: "Ask questions directly. Faster, uses fewer tokens." },
{ label: "Yes", description: "Search web for best practices before each question group. More informed questions but uses more tokens." }
]
},
{
question: "Skip discuss-phase in autonomous mode? (use ROADMAP phase goals as spec)",
header: "Skip Discuss",
multiSelect: false,
options: [
{ label: "No (Recommended)", description: "Run smart discuss before each phase — surfaces gray areas and captures decisions." },
{ label: "Yes", description: "Skip discuss in /gsd-autonomous — chain directly to plan. Best for backend/pipeline work where phase descriptions are the spec." }
]
},
{
question: "Use git worktrees for parallel agent isolation?",
header: "Worktrees",
multiSelect: false,
options: [
{ label: "Yes (Recommended)", description: "Each parallel executor runs in its own worktree branch — no conflicts between agents." },
{ label: "No", description: "Disable worktree isolation. Use on platforms where EnterWorktree is broken (e.g. Windows with feature branches). Agents run sequentially on the main working tree." }
]
}
])
{
...existing_config,
"model_profile": "quality" | "balanced" | "budget" | "adaptive" | "inherit",
"workflow": {
"research": true/false,
"plan_check": true/false,
"verifier": true/false,
"auto_advance": true/false,
"nyquist_validation": true/false,
"ui_phase": true/false,
"ui_safety_gate": true/false,
"ai_integration_phase": true/false,
"text_mode": true/false,
"research_before_questions": true/false,
"discuss_mode": "discuss" | "assumptions",
"skip_discuss": true/false,
"use_worktrees": true/false
},
"git": {
"branching_strategy": "none" | "phase" | "milestone",
"quick_branch_template": <string|null>
},
"hooks": {
"context_warnings": true/false,
"workflow_guard": true/false
}
}
Write updated config to .planning/config.json.
AskUserQuestion([
{
question: "Save these as default settings for all new projects?",
header: "Defaults",
multiSelect: false,
options: [
{ label: "Yes", description: "New projects start with these settings (saved to ~/.gsd/defaults.json)" },
{ label: "No", description: "Only apply to this project" }
]
}
])
If "Yes": write the same config object (minus project-specific fields like brave_search) to ~/.gsd/defaults.json:
mkdir -p ~/.gsd
Write ~/.gsd/defaults.json with:
{
"mode": <current>,
"granularity": <current>,
"model_profile": <current>,
"commit_docs": <current>,
"parallelization": <current>,
"branching_strategy": <current>,
"quick_branch_template": <current>,
"workflow": {
"research": <current>,
"plan_check": <current>,
"verifier": <current>,
"auto_advance": <current>,
"nyquist_validation": <current>,
"ui_phase": <current>,
"ui_safety_gate": <current>,
"ai_integration_phase": <current>,
"skip_discuss": <current>
}
}
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
GSD ► SETTINGS UPDATED
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
| Setting | Value |
|----------------------|-------|
| Model Profile | {quality/balanced/budget/inherit} |
| Plan Researcher | {On/Off} |
| Plan Checker | {On/Off} |
| Execution Verifier | {On/Off} |
| Auto-Advance | {On/Off} |
| Nyquist Validation | {On/Off} |
| UI Phase | {On/Off} |
| UI Safety Gate | {On/Off} |
| AI Integration Phase | {On/Off} |
| Git Branching | {None/Per Phase/Per Milestone} |
| Skip Discuss | {On/Off} |
| Context Warnings | {On/Off} |
| Saved as Defaults | {Yes/No} |
These settings apply to future /gsd-plan-phase and /gsd-execute-phase runs.
Quick commands:
- /gsd-set-profile <profile> — switch model profile
- /gsd-plan-phase --research — force research
- /gsd-plan-phase --skip-research — skip research
- /gsd-plan-phase --skip-verify — skip plan check
<success_criteria>
- Current config read
- User presented with 14 settings (profile + 11 workflow toggles + git branching + ctx warnings)
- Config updated with model_profile, workflow, and git sections
- User offered to save as global defaults (~/.gsd/defaults.json)
- Changes confirmed to user </success_criteria>