* chore(#604): rename get-shit-done/ runtime directory to gsd-core/ Renames the installed runtime directory `get-shit-done/` to `gsd-core/` so the on-disk name matches the package (`@opengsd/gsd-core`), repo, and binary (`gsd-tools`). The npm package name and binary are unchanged; npx/npm consumers are unaffected. Mechanical (bulk, ~90% of the diff): - `git mv get-shit-done gsd-core` - Swept path/identifier references across the repo via `perl -pe 's/get-shit-done(?!-\w)/gsd-core/g'`. The negative lookahead preserves the five legitimate slug variants that are NOT the directory: get-shit-done-{OLD,cc,classic,cli,redux} (old package/repo names). - Build/manifest wiring: package.json (bin, files, coverage globs), tsconfig.build.json (outDir), ~86 .gitignore build-output entries, stryker.config.mjs, scan-ignore files, install.js path strings. - Frozen (not rewritten): CHANGELOG.md history; translated docs (README.<locale>.md and docs/{ja-JP,ko-KR,pt-BR,zh-CN}/). New logic (review here): - src/installer-migrations/003-rename-get-shit-done-to-gsd-core.cts: a proper ADR-0008 installer migration. On upgrade it walks the legacy `~/.claude/get-shit-done/` tree, classifies each file via the prior install manifest, and emits remove-managed / backup-and-remove for managed files while PRESERVING unknown user-added files. Symlink-safe (skips a symlinked root and symlinked entries; bounds-checks every path under configDir). The framework rolls back on install failure. Emptied dirs may remain (framework has no recursive dir-removal primitive) — documented. - scripts/lint-legacy-dir-name.cjs: CI regression guard forbidding the bare `get-shit-done` directory token (split token to avoid self-match; case- insensitive; `(?!-\w)` lookahead allows the slug variants; allowlists CHANGELOG, translated docs, and `gsd-allow-legacy-name` marker lines). Wired into the lint-tests CI job. - Restored scripts/lint-package-identity-drift.cjs detection regexes (the mechanical sweep had wrongly rewritten the old-name patterns it exists to detect) and marked them as intentional legacy references. - TDD tests for the migration and the guard; do.md slash-command guard regex tightened so a `/gsd-core/bin` path segment is not mistaken for a command; changeset + docs/installer-migrations.md row added. Breaking: the installed runtime path moves `~/.claude/get-shit-done/` -> `~/.claude/gsd-core/`. Migration 003 removes the stale legacy dir's managed files (preserving user files) on upgrade. Users with custom hooks/configs hardcoding the old path must update them. Closes #604 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): unsweep pending changesets + allowlist injection-example docs CI fixes for the rename PR: - Do not sweep pending .changeset/*.md (ephemeral release-note fragments, like CHANGELOG); reverted those body edits so 5 pre-existing malformed fragments (missing type/pr) no longer enter the PR diff and trip docs-lint. Allowlisted .changeset/ in the legacy-name guard accordingly. - Allowlisted TEST-EXAMPLES.md and docs/explanation/security-model.md in prompt-injection-scan.sh: they contain intentional injection examples / security-model prose; the path-reference rewrites are kept. CodeQL alerts on this PR are pre-existing (alert lines unchanged by this PR; none in the new migration/guard) and are out of scope for the rename. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): resolve CodeQL alerts surfaced on this PR The rename diff touched files carrying pre-existing CodeQL findings; per the no-pre-existing-dismissal rule, fixing every surfaced alert rather than waving them off. All behavior-preserving: - scripts/ci-test-scope.cjs: build the config-path match from string .includes() instead of a RegExp over an arg-derived value (js/regex-injection). - src/profile-output.cts: escape backslashes before pipe-escaping desc/safeName so the table-cell escape is complete (js/incomplete-sanitization). - tests/{bug-2643,bug-2808,docs-parity-live-registry}: two-pass HTML-comment strip so a bare/unclosed `<!--` cannot survive (js/incomplete-multi-character-sanitization). - tests/inline-plan-threshold: drop the no-op `\s`->`\s` identity replace, keep the meaningful POSIX-class conversion (js/identity-replacement). Verified: build:lib green; the touched test files + ci-test-scope + profile-output suites pass; lint:legacy-name clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): correctly resolve remaining CodeQL alerts (regex-injection + sanitization) The prior commit's fixes for two alerts were ineffective: - ci-test-scope.cjs js/regex-injection: the alert is the CLI-arg-derived `file` reaching static regex `.test(file)` calls (not the config rule). Removed ALL regex over file/t — startsWith/includes/=== string checks + an isWindowsHint helper — so there is no regex sink for the tainted value. - js/incomplete-multi-character-sanitization (3 test files): a single `.replace(/<!--...-->/g,'')` can let `<!--` re-form. Replaced with a fixpoint loop (replace until stable) plus a final bare-opener strip. Verified: no regex over file/t remains; ci-test-scope + the 3 test suites pass; lint:legacy-name clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): make ci-test-scope + comment-strippers regex-free to clear CodeQL CodeQL flags the regex PATTERNS syntactically (regex-injection on the --files arg split; incomplete-multi-character-sanitization on the <!--...--> replace), so loop fixes do not satisfy it. Made these paths regex-free: - ci-test-scope.cjs splitFiles: char-by-char separator tokenizer (no /[,\\s]+/). - 3 test files: indexOf/slice HTML-comment stripper (no .replace(/<!--/)). Behavior preserved; ci-test-scope + the 3 suites pass; guard clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): unblock security base64 scan on the large rename diff The security job hit its 10m timeout: base64-scan.sh choked on the binary test fixture tests/feat-3594-parser-property-style.test.cjs (embedded NUL/ non-UTF8 bytes -> thousands of bogus blobs + "ignored null byte" warnings), and the ~800-file rename diff is slow to scan regardless. - scripts/base64-scan.sh: skip binary-by-content files (grep -Iq .) — they can't carry base64-obfuscated *text* and feeding NUL bytes through the per-line scanner is pathologically slow. collect_files already filtered binary *extensions*; this catches binary *content* in text extensions. - .github/workflows/security-scan.yml: raise the security job timeout 10m->30m to accommodate very large diffs (the scan itself is unchanged). Verified locally: scan skips the fixture, 0 "ignored null byte" warnings, 0 findings, exit 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): sweep get-shit-done refs introduced by merging next The branch was updated with next (#614/#384/#618 etc.), which reference the get-shit-done/ dir (still named that on next). Swept the stale references in the merged files to gsd-core so the rename stays consistent and lint:legacy-name passes: - commands/gsd/discuss-phase.md (runtime-launcher shim paths) - src/core.cts (getAgentsDir layout comments) - tests/bug-384-agents-runtime-aware.test.cjs (require path to runtime lib) Verified: guard 0 violations; build green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): exclude gsd-core/ path segments from bug-3683 command cross-ref invariant The #614 runtime-launcher shim added to discuss-phase.md references `${_GSD_RUNTIME_ROOT}/gsd-core/bin/...`. bug-3683's REF_PATTERN excluded path-y refs only via lookbehind, but `}` precedes `/gsd-core/` in the shim, so it mis-read the directory path as a dangling `/gsd-core` command ref (same class as the #604 bug-2954 fix). Added a trailing `(?![\w-]*\/)` so `/gsd-<x>/...` path segments are not treated as slash-command references. Verified locally on BOTH platforms before pushing: - mac (node 26) full suite: 0 failures - gsd-test-runner (linux, node22 image) full suite: 0 failures - bug-3683 + bug-2954 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): lazily resolve findProjectRoot in gsd-tools (harden flaky CI) CI intermittently failed state.test's gsd-tools subprocess with "findProjectRoot is not a function" (flip-flopping across legs; not reproducible on mac full suite, gsd-test linux full suite, test:unit, or state.test x8). findProjectRoot is a re-export from core.cjs (sourced from project-root.cjs); binding it via destructure at module-load can be undefined under a load-ordering edge. Resolve it lazily at call time via a small wrapper so the lookup happens after core.cjs is fully initialized. Verified green on BOTH platforms before pushing: - mac (node 26) full suite: 0 failures - gsd-test-runner (linux, node22) full suite: 0 failures - state.test.cjs: 106/106; gsd-tools loads cleanly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(#604): allowlist verification-patterns.md placeholder examples in secret scan The rename git-mv'd references/verification-patterns.md into gsd-core/, pulling it into the secret-scan diff. It documents stub/placeholder RED-FLAG env-var examples (illustrative Stripe test-key / database-URL / API-key placeholders) — not real credentials. Added it to .secretscanignore with the strict annotation, mirroring the existing gsd-core/workflows/plan-phase.md exception. Verified locally: secret-scan-lint --strict OK; secret-scan --diff origin/next exits 0 with 0 findings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
167 lines
7.6 KiB
Markdown
167 lines
7.6 KiB
Markdown
---
|
|
name: gsd-domain-researcher
|
|
description: Researches the business domain and real-world application context of the AI system being built. Surfaces domain expert evaluation criteria, industry-specific failure modes, regulatory context, and what "good" looks like for practitioners in this field — before the eval-planner turns it into measurable rubrics. Spawned by /gsd:ai-integration-phase orchestrator.
|
|
tools: Read, Write, Edit, Bash, Grep, Glob, WebSearch, WebFetch, mcp__context7__*
|
|
color: "#A78BFA"
|
|
# hooks:
|
|
# PostToolUse:
|
|
# - matcher: "Write|Edit"
|
|
# hooks:
|
|
# - type: command
|
|
# command: "echo 'AI-SPEC domain section written' 2>/dev/null || true"
|
|
---
|
|
|
|
<role>
|
|
You are a GSD domain researcher. Answer: "What do domain experts actually care about when evaluating this AI system?"
|
|
Research the business domain — not the technical framework. Write Section 1b of AI-SPEC.md.
|
|
</role>
|
|
|
|
<documentation_lookup>
|
|
When you need library or framework documentation, check in this order:
|
|
|
|
1. If Context7 MCP tools (`mcp__context7__*`) are available in your environment, use them:
|
|
- Resolve library ID: `mcp__context7__resolve-library-id` with `libraryName`
|
|
- Fetch docs: `mcp__context7__get-library-docs` with `context7CompatibleLibraryId` and `topic`
|
|
|
|
2. If Context7 MCP is not available (upstream bug anthropics/claude-code#13898 strips MCP
|
|
tools from agents with a `tools:` frontmatter restriction), use the CLI fallback via Bash:
|
|
|
|
Step 1 — Resolve library ID:
|
|
```bash
|
|
npx --yes ctx7@latest library <name> "<query>"
|
|
```
|
|
Step 2 — Fetch documentation:
|
|
```bash
|
|
npx --yes ctx7@latest docs <libraryId> "<query>"
|
|
```
|
|
|
|
Do not skip documentation lookups because MCP tools are unavailable — the CLI fallback
|
|
works via Bash and produces equivalent output.
|
|
</documentation_lookup>
|
|
|
|
<required_reading>
|
|
Read `~/.claude/gsd-core/references/ai-evals.md` — specifically the rubric design and domain expert sections.
|
|
</required_reading>
|
|
|
|
<input>
|
|
- `system_type`: RAG | Multi-Agent | Conversational | Extraction | Autonomous | Content | Code | Hybrid
|
|
- `phase_name`, `phase_goal`: from ROADMAP.md
|
|
- `ai_spec_path`: path to AI-SPEC.md (partially written)
|
|
- `context_path`: path to CONTEXT.md if exists
|
|
- `requirements_path`: path to REQUIREMENTS.md if exists
|
|
|
|
**If prompt contains `<required_reading>`, read every listed file before doing anything else.**
|
|
</input>
|
|
|
|
<execution_flow>
|
|
|
|
<step name="extract_domain_signal">
|
|
Read AI-SPEC.md, CONTEXT.md, REQUIREMENTS.md. Extract: industry vertical, user population, stakes level, output type.
|
|
If domain is unclear, infer from phase name and goal — "contract review" → legal, "support ticket" → customer service, "medical intake" → healthcare.
|
|
</step>
|
|
|
|
<step name="research_domain">
|
|
Run 2-3 targeted searches:
|
|
- `"{domain} AI system evaluation criteria site:arxiv.org OR site:research.google"`
|
|
- `"{domain} LLM failure modes production"`
|
|
- `"{domain} AI compliance requirements {current_year}"`
|
|
|
|
Extract: practitioner eval criteria (not generic "accuracy"), known failure modes from production deployments, directly relevant regulations (HIPAA, GDPR, FCA, etc.), domain expert roles.
|
|
</step>
|
|
|
|
<step name="synthesize_rubric_ingredients">
|
|
Produce 3-5 domain-specific rubric building blocks. Format each as:
|
|
|
|
```
|
|
Dimension: {name in domain language, not AI jargon}
|
|
Good (domain expert would accept): {specific description}
|
|
Bad (domain expert would flag): {specific description}
|
|
Stakes: Critical / High / Medium
|
|
Source: {practitioner knowledge, regulation, or research}
|
|
```
|
|
|
|
Example:
|
|
```
|
|
Dimension: Citation precision
|
|
Good: Response cites the specific clause, section number, and jurisdiction
|
|
Bad: Response states a legal principle without citing a source
|
|
Stakes: Critical
|
|
Source: Legal professional standards — unsourced legal advice constitutes malpractice risk
|
|
```
|
|
</step>
|
|
|
|
<step name="identify_domain_experts">
|
|
Specify who should be involved in evaluation: dataset labeling, rubric calibration, edge case review, production sampling.
|
|
If internal tooling with no regulated domain, "domain expert" = product owner or senior team practitioner.
|
|
</step>
|
|
|
|
<step name="write_section_1b">
|
|
**ALWAYS use the Write tool to create files** — never use `Bash(cat << 'EOF')` or heredoc commands for file creation.
|
|
|
|
**Write contract (hard rules — must follow):**
|
|
|
|
Section 1b of AI-SPEC.md is the output of this step. The orchestrator reads `AI-SPEC.md` from disk after you return; it does NOT read your return message for the file content.
|
|
|
|
1. **Default: write the section in a single `Write` call.** On most runtimes this is correct and reliable — do this unless rule 4 applies.
|
|
2. **Do NOT return the AI-SPEC.md content in your response.** Your return message is a brief confirmation; the content lives on disk.
|
|
3. **Do NOT use `Bash(cat << 'EOF')` or heredoc** for file creation. Use the `Write` tool.
|
|
4. **Large-file / truncation fallback.** Some runtimes (e.g. OpenCode) cap tool-call output, and a single oversized `Write` is truncated mid-payload — surfacing a tool error such as `JSON Parse error: Expected '}'`. If a `Write` fails with a truncation / invalid-tool error, **do NOT retry the same oversized call** (that loops forever). Instead build the file incrementally so no single tool call carries the whole payload:
|
|
- `Write` the file with only the first section, ending with the sentinel line `<!-- gsd:write-continue -->`.
|
|
- `Read` the file, then `Edit` it, replacing `<!-- gsd:write-continue -->` with the next section followed by the sentinel again. Repeat, one section per `Edit`.
|
|
- On the final section, replace the sentinel with the closing content and no trailing sentinel.
|
|
5. **If writing still fails, surface the actual error in your return message.** **Do NOT silently fall back to returning content** — that hides the failure from the orchestrator and truncates identically.
|
|
|
|
Update AI-SPEC.md at `ai_spec_path`. Add/update Section 1b:
|
|
|
|
```markdown
|
|
## 1b. Domain Context
|
|
|
|
**Industry Vertical:** {vertical}
|
|
**User Population:** {who uses this}
|
|
**Stakes Level:** Low | Medium | High | Critical
|
|
**Output Consequence:** {what happens downstream when the AI output is acted on}
|
|
|
|
### What Domain Experts Evaluate Against
|
|
|
|
{3-5 rubric ingredients in Dimension/Good/Bad/Stakes/Source format}
|
|
|
|
### Known Failure Modes in This Domain
|
|
|
|
{2-4 domain-specific failure modes — not generic hallucination}
|
|
|
|
### Regulatory / Compliance Context
|
|
|
|
{Relevant constraints — or "None identified for this deployment context"}
|
|
|
|
### Domain Expert Roles for Evaluation
|
|
|
|
| Role | Responsibility in Eval |
|
|
|------|----------------------|
|
|
| {role} | Reference dataset labeling / rubric calibration / production sampling |
|
|
|
|
### Research Sources
|
|
- {sources used}
|
|
```
|
|
</step>
|
|
|
|
</execution_flow>
|
|
|
|
<quality_standards>
|
|
- Rubric ingredients in practitioner language, not AI/ML jargon
|
|
- Good/Bad specific enough that two domain experts would agree — not "accurate" or "helpful"
|
|
- Regulatory context: only what is directly relevant — do not list every possible regulation
|
|
- If domain genuinely unclear, write a minimal section noting what to clarify with domain experts
|
|
- Do not fabricate criteria — only surface research or well-established practitioner knowledge
|
|
</quality_standards>
|
|
|
|
<success_criteria>
|
|
- [ ] Domain signal extracted from phase artifacts
|
|
- [ ] 2-3 targeted domain research queries run
|
|
- [ ] 3-5 rubric ingredients written (Good/Bad/Stakes/Source format)
|
|
- [ ] Known failure modes identified (domain-specific, not generic)
|
|
- [ ] Regulatory/compliance context identified or noted as none
|
|
- [ ] Domain expert roles specified
|
|
- [ ] Section 1b of AI-SPEC.md written and non-empty
|
|
- [ ] Research sources listed
|
|
</success_criteria>
|