Tom Boucher
f19d0327b2
feat(agents): sycophancy hardening for 9 audit-class agents ( #2489 )
...
* fix(tests): update 5 source-text tests to read config-schema.cjs
VALID_CONFIG_KEYS moved from config.cjs to config-schema.cjs in the
drift-prevention companion PR. Tests that read config.cjs source text
and checked for key literal includes() now point to the correct file.
Closes #2480
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
* feat(agents): sycophancy hardening for 9 audit-class agents (#2427 )
Add adversarial reviewer posture to gsd-plan-checker, gsd-code-reviewer,
gsd-security-auditor, gsd-verifier, gsd-eval-auditor, gsd-nyquist-auditor,
gsd-ui-auditor, gsd-integration-checker, and gsd-doc-verifier.
Four changes per agent:
- Third-person framing: <role> opens with submission framing, not "You are a GSD X"
- FORCE stance: explicit starting hypothesis that the submission is flawed
- Failure modes: agent-specific list of how each reviewer type goes soft
- BLOCKER/WARNING classification: every finding must carry an explicit severity
Also applies to sdk/prompts/agents variants of gsd-plan-checker and gsd-verifier.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-20 18:20:08 -04:00
Tom Boucher
20fe395064
feat(2149,2150): add project skills awareness to 9 GSD agents ( #2152 )
...
- gsd-debugger: add Project skills block after required_reading
- gsd-integration-checker, gsd-security-auditor, gsd-nyquist-auditor,
gsd-codebase-mapper, gsd-roadmapper, gsd-eval-auditor, gsd-intel-updater,
gsd-doc-writer: add Project skills block at context-load step
- Add context budget note to 8 quality/audit agents
- gsd-doc-writer: add security note for user-supplied doc_assignment content
- Add tests/agent-skills-awareness.test.cjs validation suite
2026-04-12 09:40:20 -04:00
Fana
33575ba91d
feat: /gsd-ai-integration-phase + /gsd-eval-review — AI framework selection and eval coverage layer ( #1971 )
...
* feat: /gsd:ai-phase + /gsd:eval-review — AI evals and framework selection layer
Adds a structured AI development layer to GSD with 5 new agents, 2 new
commands, 2 new workflows, 2 reference files, and 1 template.
Commands:
- /gsd:ai-phase [N] — pre-planning AI design contract (inserts between
discuss-phase and plan-phase). Orchestrates 4 agents in sequence:
framework-selector → ai-researcher → domain-researcher → eval-planner.
Output: AI-SPEC.md with framework decision, implementation guidance,
domain expert context, and evaluation strategy.
- /gsd:eval-review [N] — retroactive eval coverage audit. Scores each
planned eval dimension as COVERED/PARTIAL/MISSING. Output: EVAL-REVIEW.md
with 0-100 score, verdict, and remediation plan.
Agents:
- gsd-framework-selector: interactive decision matrix (6 questions) →
scored framework recommendation for CrewAI, LlamaIndex, LangChain,
LangGraph, OpenAI Agents SDK, Claude Agent SDK, AutoGen/AG2, Haystack
- gsd-ai-researcher: fetches official framework docs + writes AI systems
best practices (Pydantic structured outputs, async-first, prompt
discipline, context window management, cost/latency budget)
- gsd-domain-researcher: researches business domain and use-case context —
surfaces domain expert evaluation criteria, industry failure modes,
regulatory constraints, and practitioner rubric ingredients before
eval-planner writes measurable criteria
- gsd-eval-planner: designs evaluation strategy grounded in domain context;
defaults to Arize Phoenix (tracing) + RAGAS (RAG eval) with detect-first
guard for existing tooling
- gsd-eval-auditor: retroactive codebase scan → scores eval coverage
Integration points:
- plan-phase: non-blocking nudge (step 4.5) when AI keywords detected and
no AI-SPEC.md present
- settings: new workflow.ai_phase toggle (default on)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
* fix: refine ai-integration-phase layer — rename, house style, consistency fixes
Amends the ai-evals framework layer (df8cb6c) with post-review improvements
before opening upstream PR.
Rename /gsd:ai-phase → /gsd:ai-integration-phase:
- Renamed commands/gsd/ai-phase.md → ai-integration-phase.md
- Renamed get-shit-done/workflows/ai-phase.md → ai-integration-phase.md
- Updated config key: workflow.ai_phase → workflow.ai_integration_phase
- Updated repair action: addAiPhaseKey → addAiIntegrationPhaseKey
- Updated all 84 cross-references across agents, workflows, templates, tests
Consistency fixes (same class as PR #1380 review):
- commands/gsd: objective described 3-agent chain, missing gsd-domain-researcher
- workflows/ai-integration-phase: purpose tag described 3-agent chain + "locks
three things" — updated to 4 agents + 4 outputs
- workflows/ai-integration-phase: missing DOMAIN_MODEL resolve-model call in
step 1 (domain-researcher was spawned in step 7.5 with no model variable)
- workflows/ai-integration-phase: fractional step ## 7.5 renumbered to integers
(steps 8–12 shifted)
Agent house style (GSD meta-prompting conformance):
- All 5 new agents refactored to execution_flow + step name="" structure
- Role blocks compressed to 2 lines (removed verbose "Core responsibilities")
- Added skills: frontmatter to all 5 agents (agent-frontmatter tests)
- Added # hooks: commented pattern to file-writing agents
- Added ALWAYS use Write tool anti-heredoc instruction to file-writing agents
- Line reductions: ai-researcher −41%, domain-researcher −25%, eval-planner −26%,
eval-auditor −25%, framework-selector −9%
Test coverage (tests/ai-evals.test.cjs — 48 tests):
- CONFIG: workflow.ai_integration_phase defaults and config-set/get
- HEALTH: W010 warning emission and addAiIntegrationPhaseKey repair
- TEMPLATE: AI-SPEC.md section completeness (10 sections)
- COMMAND: ai-integration-phase + eval-review frontmatter validity
- AGENTS: all 5 new agent files exist
- REFERENCES: ai-evals.md + ai-frameworks.md exist and are non-empty
- WORKFLOW: plan-phase nudge integration, workflow files exist + agent coverage
603/603 tests passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
* feat: add Google ADK to framework selector and reference matrix
Google ADK (released March 2025) was missing from the framework options.
Adds Python + Java multi-agent framework optimised for Gemini / Vertex AI.
- get-shit-done/references/ai-frameworks.md: add Google ADK profile (type,
language, model support, best for, avoid if, strengths, weaknesses, eval
concerns); update Quick Picks, By System Type, and By Model Commitment tables
- agents/gsd-framework-selector.md: add "Google (Gemini)" to model provider
interview question
- agents/gsd-ai-researcher.md: add Google ADK docs URL to documentation_sources
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
* fix: adapt to upstream conventions post-rebase
- Remove skills: frontmatter from all 5 new agents (upstream changed
convention — skills: breaks Gemini CLI and must not be present)
- Add workflow.ai_integration_phase to VALID_CONFIG_KEYS whitelist in
config.cjs (config-set blocked unknown keys)
- Add ai_integration_phase: true to CONFIG_DEFAULTS in core.cjs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
* fix: rephrase 4b.1 line to avoid false-positive in prompt-injection scan
"contract as a Pydantic model" matched the `act as a` pattern case-insensitively.
Rephrased to "output schema using a Pydantic model".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
* fix: adapt to upstream conventions (W016, colon refs, config docs)
- Replace verify.cjs from upstream to restore W010-W015 + cmdValidateAgents,
lost when rebase conflict was resolved with --theirs
- Add W016 (workflow.ai_integration_phase absent) inside the config try block,
avoids collision with upstream's W010 agent-installation check
- Add addAiIntegrationPhaseKey repair case mirroring addNyquistKey pattern
- Replace /gsd: colon format with /gsd- hyphen format across all new files
(agents, workflows, templates, verify.cjs) per stale-colon-refs guard (#1748 )
- Add workflow.ai_integration_phase to planning-config.md reference table
- Add ai_integration_phase → workflow.ai_integration_phase to NAMESPACE_MAP
in config-field-docs.test.cjs so CONFIG_DEFAULTS coverage check passes
- Update ai-evals tests to use W016 instead of W010
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
* fix: add 5 new agents to E2E Copilot install expected list
gsd-ai-researcher, gsd-domain-researcher, gsd-eval-auditor,
gsd-eval-planner, gsd-framework-selector added to the hardcoded
expected agent list in copilot-install.test.cjs (#1890 ).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com >
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-04-10 10:49:00 -04:00
