Files
msd-core/get-shit-done/workflows/map-codebase.md
Tom Boucher 1a694fcac3 feat: auto-remap codebase after significant phase execution (closes #2003) (#2605)
* feat: auto-remap codebase after significant phase execution (#2003)

Adds a post-phase structural drift detector that compares the committed tree
against `.planning/codebase/STRUCTURE.md` and either warns or auto-remaps
the affected subtrees when drift exceeds a configurable threshold.

## Summary
- New `bin/lib/drift.cjs` — pure detector covering four drift categories:
  new directories outside mapped paths, new barrel exports at
  `(packages|apps)/*/src/index.*`, new migration files, and new route
  modules. Prioritizes the most-specific category per file.
- New `verify codebase-drift` CLI subcommand + SDK handler, registered as
  `gsd-sdk query verify.codebase-drift`.
- New `codebase_drift_gate` step in `execute-phase` between
  `schema_drift_gate` and `verify_phase_goal`. Non-blocking by contract —
  any error logs and the phase continues.
- Two new config keys: `workflow.drift_threshold` (int, default 3) and
  `workflow.drift_action` (`warn` | `auto-remap`, default `warn`), with
  enum/integer validation in `config-set`.
- `gsd-codebase-mapper` learns an optional `--paths <p1,p2,...>` scope hint
  for incremental remapping; agent/workflow docs updated.
- `last_mapped_commit` lives in YAML frontmatter on each
  `.planning/codebase/*.md` file; `readMappedCommit`/`writeMappedCommit`
  round-trip helpers ship in `drift.cjs`.

## Tests
- 55 new tests in `tests/drift-detection.test.cjs` covering:
  classification, threshold gating at 2/3/4 elements, warn vs. auto-remap
  routing, affected-path scoping, `--paths` sanitization (traversal,
  absolute, shell metacharacter rejection), frontmatter round-trip,
  defensive paths (missing STRUCTURE.md, malformed input, non-git repos),
  CLI JSON output, and documentation parity.
- Full suite: 5044 pass / 0 fail.

## Documentation
- `docs/CONFIGURATION.md` — rows for both new keys.
- `docs/ARCHITECTURE.md` — section on the post-execute drift gate.
- `docs/AGENTS.md` — `--paths` flag on `gsd-codebase-mapper`.
- `docs/USER-GUIDE.md` — user-facing behavior note + toggle commands.
- `docs/FEATURES.md` — new 27a section with REQ-DRIFT-01..06.
- `docs/INVENTORY.md` + `docs/INVENTORY-MANIFEST.json` — drift.cjs listed.
- `get-shit-done/workflows/execute-phase.md` — `codebase_drift_gate` step.
- `get-shit-done/workflows/map-codebase.md` — `parse_paths_flag` step.
- `agents/gsd-codebase-mapper.md` — `--paths` directive under parse_focus.

## Design decisions
- **Frontmatter over sidecar JSON** for `last_mapped_commit`: keeps the
  baseline attached to the file, survives git moves, survives per-doc
  regeneration, no extra file lifecycle.
- **Substring match against STRUCTURE.md** for `isPathMapped`: the map is
  free-form markdown, not a structured manifest; any mention of a path
  prefix counts as "mapped territory". Cheap, no parser, zero false
  negatives on reasonable maps.
- **Category priority migration > route > barrel > new_dir** so a file
  matching multiple rules counts exactly once at the most specific level.
- **Empty-tree SHA fallback** (`4b825dc6…`) when `last_mapped_commit` is
  absent — semantically correct (no baseline means everything is drift)
  and deterministic across repos.
- **Four layers of non-blocking** — detector try/catch, CLI try/catch, SDK
  handler try/catch, and workflow `|| echo` shell fallback. Any single
  layer failing still returns a valid skipped result.
- **SDK handler delegates to `gsd-tools.cjs`** rather than re-porting the
  detector to TypeScript, keeping drift logic in one canonical place.

Closes #2003

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(mapper): tag --paths fenced block as text (CodeRabbit MD040)

Comment 3127255172.

* docs(config): use /gsd- dash command syntax in drift_action row (CodeRabbit)

Comment 3127255180. Matches the convention used by every other command
reference in docs/CONFIGURATION.md.

* fix(execute-phase): initialize AGENT_SKILLS_MAPPER + tag fenced blocks

Two CodeRabbit findings on the auto-remap branch of the drift gate:

- 3127255186 (must-fix): the mapper Task prompt referenced
  ${AGENT_SKILLS_MAPPER} but only AGENT_SKILLS (for gsd-executor) is
  loaded at init_context (line 72). Without this fix the literal
  placeholder string would leak into the spawned mapper's prompt.
  Add an explicit gsd-sdk query agent-skills gsd-codebase-mapper step
  right before the Task spawn.
- 3127255183: tag the warn-message and Task() fenced code blocks as
  text to satisfy markdownlint MD040.

* docs(map-codebase): wire PATH_SCOPE_HINT through every mapper prompt

CodeRabbit (review id 4158286952, comment 3127255190) flagged that the
parse_paths_flag step defined incremental-remap semantics but did not
inject a normalized variable into the spawn_agents and sequential_mapping
mapper prompts, so incremental remap could silently regress to a
whole-repo scan.

- Define SCOPED_PATHS / PATH_SCOPE_HINT in parse_paths_flag.
- Inject ${PATH_SCOPE_HINT} into all four spawn_agents Task prompts.
- Document the same scope contract for sequential_mapping mode.

* fix(drift): writeMappedCommit tolerates missing target file

CodeRabbit (review id 4158286952, drift.cjs:349-355 nitpick) noted that
readMappedCommit returns null on ENOENT but writeMappedCommit threw — an
asymmetry that breaks first-time stamping of a freshly produced doc that
the caller has not yet written.

- Catch ENOENT on the read; treat absent file as empty content.
- Add a regression test that calls writeMappedCommit on a non-existent
  path and asserts the file is created with correct frontmatter.
  Test was authored to fail before the fix (ENOENT) and passes after.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 21:21:44 -04:00

15 KiB

Orchestrate parallel codebase mapper agents to analyze codebase and produce structured documents in .planning/codebase/

Each agent has fresh context, explores a specific focus area, and writes documents directly. The orchestrator only receives confirmation + line counts, then writes a summary.

Output: .planning/codebase/ folder with 7 structured documents about the codebase state.

<available_agent_types> Valid GSD subagent types (use exact names — do not fall back to 'general-purpose'):

  • gsd-codebase-mapper — Maps project structure and dependencies </available_agent_types>
**Why dedicated mapper agents:** - Fresh context per domain (no token contamination) - Agents write documents directly (no context transfer back to orchestrator) - Orchestrator only summarizes what was created (minimal context usage) - Faster execution (agents run simultaneously)

Document quality over length: Include enough detail to be useful as reference. Prioritize practical examples (especially code patterns) over arbitrary brevity.

Always include file paths: Documents are reference material for Claude when planning/executing. Always include actual file paths formatted with backticks: src/services/user.ts.

Parse an optional `--paths ` argument. When supplied (by the post-execute codebase-drift gate in `/gsd:execute-phase` or by a user running `/gsd:map-codebase --paths apps/accounting,packages/ui`), the workflow operates in **incremental-remap mode**:
  • Pass --paths <p1>,<p2>,... through to each spawned gsd-codebase-mapper agent's prompt. Agents scope their Glob/Grep/Bash exploration to the listed repo-relative prefixes only — no whole-repo scan.
  • Reject path values that contain .., start with /, or include shell metacharacters (;, `, $, &, |, <, >). If all provided paths are invalid, fall back to a normal whole-repo run.
  • On write, each mapper stamps last_mapped_commit: <HEAD sha> into the YAML frontmatter of every document it produces (see bin/lib/drift.cjs:writeMappedCommit).

Explicit contract — propagate --paths through a single normalized variable. Downstream steps (spawn_agents, sequential_mapping, and any Task-mode prompt construction) MUST use ${PATH_SCOPE_HINT} to ensure every mapper receives the same deterministic scope. Without this contract incremental-remap can silently regress to a whole-repo scan.

# Validated, comma-separated paths (empty if --paths absent or all rejected):
SCOPED_PATHS="<validated paths or empty>"
if [ -n "$SCOPED_PATHS" ]; then
  PATH_SCOPE_HINT="--paths $SCOPED_PATHS"
else
  PATH_SCOPE_HINT=""
fi

All mapper prompts built later in this workflow MUST include ${PATH_SCOPE_HINT} (expanded to empty when full-repo mode is in effect).

When --paths is absent, behave exactly as before: full-repo scan, all 7 documents refreshed.

Load codebase mapping context:
INIT=$(gsd-sdk query init.map-codebase)
if [[ "$INIT" == @file:* ]]; then INIT=$(cat "${INIT#@file:}"); fi
AGENT_SKILLS_MAPPER=$(gsd-sdk query agent-skills gsd-codebase-mapper 2>/dev/null)

Extract from init JSON: mapper_model, commit_docs, codebase_dir, existing_maps, has_maps, codebase_dir_exists, subagent_timeout, date.

Check if .planning/codebase/ already exists using `has_maps` from init context.

If codebase_dir_exists is true:

ls -la .planning/codebase/

If exists:

.planning/codebase/ already exists with these documents:
[List files found]

What's next?
1. Refresh - Delete existing and remap codebase
2. Update - Keep existing, only update specific documents
3. Skip - Use existing codebase map as-is

Wait for user response.

If "Refresh": Delete .planning/codebase/, continue to create_structure If "Update": Ask which documents to update, continue to spawn_agents (filtered) If "Skip": Exit workflow

If doesn't exist: Continue to create_structure.

Create .planning/codebase/ directory:
mkdir -p .planning/codebase

Expected output files:

  • STACK.md (from tech mapper)
  • INTEGRATIONS.md (from tech mapper)
  • ARCHITECTURE.md (from arch mapper)
  • STRUCTURE.md (from arch mapper)
  • CONVENTIONS.md (from quality mapper)
  • TESTING.md (from quality mapper)
  • CONCERNS.md (from concerns mapper)

Continue to spawn_agents.

Before spawning agents, detect whether the current runtime supports the `Task` tool for subagent delegation.

How to detect: Check if you have access to a Task tool (may be capitalized as Task or lowercase as task depending on runtime). If you do NOT have a Task/task tool (or only have tools like browser_subagent which is for web browsing, NOT code analysis):

→ Skip spawn_agents and collect_confirmations — go directly to sequential_mapping instead.

CRITICAL: Never use browser_subagent or Explore as a substitute for Task. The browser_subagent tool is exclusively for web page interaction and will fail for codebase analysis. If Task is unavailable, perform the mapping sequentially in-context.

Spawn 4 parallel gsd-codebase-mapper agents.

Use Task tool with subagent_type="gsd-codebase-mapper", model="{mapper_model}", and run_in_background=true for parallel execution.

CRITICAL: Use the dedicated gsd-codebase-mapper agent, NOT Explore or browser_subagent. The mapper agent writes documents directly.

Agent 1: Tech Focus

Task(
  subagent_type="gsd-codebase-mapper",
  model="{mapper_model}",
  run_in_background=true,
  description="Map codebase tech stack",
  prompt="Focus: tech
Today's date: {date}

Analyze this codebase for technology stack and external integrations.

Write these documents to .planning/codebase/:
- STACK.md - Languages, runtime, frameworks, dependencies, configuration
- INTEGRATIONS.md - External APIs, databases, auth providers, webhooks

IMPORTANT: Use {date} for all [YYYY-MM-DD] date placeholders in documents.

Scope: ${PATH_SCOPE_HINT:-(full repo)} — when --paths is supplied, restrict exploration to those prefixes only.

Explore thoroughly. Write documents directly using templates. Return confirmation only.
${AGENT_SKILLS_MAPPER}"
)

Agent 2: Architecture Focus

Task(
  subagent_type="gsd-codebase-mapper",
  model="{mapper_model}",
  run_in_background=true,
  description="Map codebase architecture",
  prompt="Focus: arch
Today's date: {date}

Analyze this codebase architecture and directory structure.

Write these documents to .planning/codebase/:
- ARCHITECTURE.md - Pattern, layers, data flow, abstractions, entry points
- STRUCTURE.md - Directory layout, key locations, naming conventions

IMPORTANT: Use {date} for all [YYYY-MM-DD] date placeholders in documents.

Scope: ${PATH_SCOPE_HINT:-(full repo)} — when --paths is supplied, restrict exploration to those prefixes only.

Explore thoroughly. Write documents directly using templates. Return confirmation only.
${AGENT_SKILLS_MAPPER}"
)

Agent 3: Quality Focus

Task(
  subagent_type="gsd-codebase-mapper",
  model="{mapper_model}",
  run_in_background=true,
  description="Map codebase conventions",
  prompt="Focus: quality
Today's date: {date}

Analyze this codebase for coding conventions and testing patterns.

Write these documents to .planning/codebase/:
- CONVENTIONS.md - Code style, naming, patterns, error handling
- TESTING.md - Framework, structure, mocking, coverage

IMPORTANT: Use {date} for all [YYYY-MM-DD] date placeholders in documents.

Scope: ${PATH_SCOPE_HINT:-(full repo)} — when --paths is supplied, restrict exploration to those prefixes only.

Explore thoroughly. Write documents directly using templates. Return confirmation only.
${AGENT_SKILLS_MAPPER}"
)

Agent 4: Concerns Focus

Task(
  subagent_type="gsd-codebase-mapper",
  model="{mapper_model}",
  run_in_background=true,
  description="Map codebase concerns",
  prompt="Focus: concerns
Today's date: {date}

Analyze this codebase for technical debt, known issues, and areas of concern.

Write this document to .planning/codebase/:
- CONCERNS.md - Tech debt, bugs, security, performance, fragile areas

IMPORTANT: Use {date} for all [YYYY-MM-DD] date placeholders in documents.

Scope: ${PATH_SCOPE_HINT:-(full repo)} — when --paths is supplied, restrict exploration to those prefixes only.

Explore thoroughly. Write document directly using template. Return confirmation only.
${AGENT_SKILLS_MAPPER}"
)

Continue to collect_confirmations.

Wait for all 4 agents to complete using TaskOutput tool.

For each agent task_id returned by the Agent tool calls above:

TaskOutput tool:
  task_id: "{task_id from Agent result}"
  block: true
  timeout: {subagent_timeout from init context, default 300000}

The timeout is configurable via workflow.subagent_timeout in .planning/config.json (milliseconds). Default: 300000 (5 minutes). Increase for large codebases or slower models.

Call TaskOutput for all 4 agents in parallel (single message with 4 TaskOutput calls).

Once all TaskOutput calls return, read each agent's output file to collect confirmations.

Expected confirmation format from each agent:

## Mapping Complete

**Focus:** {focus}
**Documents written:**
- `.planning/codebase/{DOC1}.md` ({N} lines)
- `.planning/codebase/{DOC2}.md` ({N} lines)

Ready for orchestrator summary.

What you receive: Just file paths and line counts. NOT document contents.

If any agent failed, note the failure and continue with successful documents.

Continue to verify_output.

When the `Task` tool is unavailable, perform codebase mapping sequentially in the current context. This replaces `spawn_agents` and `collect_confirmations`.

IMPORTANT: Do NOT use browser_subagent, Explore, or any browser-based tool. Use only file system tools (Read, Bash, Write, Grep, Glob, list_dir, view_file, grep_search, or equivalent tools available in your runtime).

IMPORTANT: Use {date} from init context for all [YYYY-MM-DD] date placeholders in documents. NEVER guess the date.

SCOPE: When ${PATH_SCOPE_HINT} is non-empty (i.e. --paths was supplied), restrict every pass below to the validated path prefixes in ${SCOPED_PATHS}. Do NOT scan files outside those prefixes. When ${PATH_SCOPE_HINT} is empty, perform a full-repo scan.

Perform all 4 mapping passes sequentially:

Pass 1: Tech Focus

  • Explore package.json/Cargo.toml/go.mod/requirements.txt, config files, dependency trees
  • Write .planning/codebase/STACK.md — Languages, runtime, frameworks, dependencies, configuration
  • Write .planning/codebase/INTEGRATIONS.md — External APIs, databases, auth providers, webhooks

Pass 2: Architecture Focus

  • Explore directory structure, entry points, module boundaries, data flow
  • Write .planning/codebase/ARCHITECTURE.md — Pattern, layers, data flow, abstractions, entry points
  • Write .planning/codebase/STRUCTURE.md — Directory layout, key locations, naming conventions

Pass 3: Quality Focus

  • Explore code style, error handling patterns, test files, CI config
  • Write .planning/codebase/CONVENTIONS.md — Code style, naming, patterns, error handling
  • Write .planning/codebase/TESTING.md — Framework, structure, mocking, coverage

Pass 4: Concerns Focus

  • Explore TODOs, known issues, fragile areas, security patterns
  • Write .planning/codebase/CONCERNS.md — Tech debt, bugs, security, performance, fragile areas

Use the same document templates as the gsd-codebase-mapper agent. Include actual file paths formatted with backticks.

Continue to verify_output.

Verify all documents created successfully:
ls -la .planning/codebase/
wc -l .planning/codebase/*.md

Verification checklist:

  • All 7 documents exist
  • No empty documents (each should have >20 lines)

If any documents missing or empty, note which agents may have failed.

Continue to scan_for_secrets.

**CRITICAL SECURITY CHECK:** Scan output files for accidentally leaked secrets before committing.

Run secret pattern detection:

# Check for common API key patterns in generated docs
grep -E '(sk-[a-zA-Z0-9]{20,}|sk_live_[a-zA-Z0-9]+|sk_test_[a-zA-Z0-9]+|ghp_[a-zA-Z0-9]{36}|gho_[a-zA-Z0-9]{36}|glpat-[a-zA-Z0-9_-]+|AKIA[A-Z0-9]{16}|xox[baprs]-[a-zA-Z0-9-]+|-----BEGIN.*PRIVATE KEY|eyJ[a-zA-Z0-9_-]+\.eyJ[a-zA-Z0-9_-]+\.)' .planning/codebase/*.md 2>/dev/null && SECRETS_FOUND=true || SECRETS_FOUND=false

If SECRETS_FOUND=true:

⚠️  SECURITY ALERT: Potential secrets detected in codebase documents!

Found patterns that look like API keys or tokens in:
[show grep output]

This would expose credentials if committed.

**Action required:**
1. Review the flagged content above
2. If these are real secrets, they must be removed before committing
3. Consider adding sensitive files to Claude Code "Deny" permissions

Pausing before commit. Reply "safe to proceed" if the flagged content is not actually sensitive, or edit the files first.

Wait for user confirmation before continuing to commit_codebase_map.

If SECRETS_FOUND=false:

Continue to commit_codebase_map.

Commit the codebase map:
gsd-sdk query commit "docs: map existing codebase" .planning/codebase/*.md

Continue to offer_next.

Present completion summary and next steps.

Get line counts:

wc -l .planning/codebase/*.md

Output format:

Codebase mapping complete.

Created .planning/codebase/:
- STACK.md ([N] lines) - Technologies and dependencies
- ARCHITECTURE.md ([N] lines) - System design and patterns
- STRUCTURE.md ([N] lines) - Directory layout and organization
- CONVENTIONS.md ([N] lines) - Code style and patterns
- TESTING.md ([N] lines) - Test structure and practices
- INTEGRATIONS.md ([N] lines) - External services and APIs
- CONCERNS.md ([N] lines) - Technical debt and issues


---

## ▶ Next Up — [${PROJECT_CODE}] ${PROJECT_TITLE}

**Initialize project** — use codebase context for planning

`/clear` then:

`/gsd:new-project`

---

**Also available:**
- Re-run mapping: `/gsd:map-codebase`
- Review specific file: `cat .planning/codebase/STACK.md`
- Edit any document before proceeding

---

End workflow.

<success_criteria>

  • .planning/codebase/ directory created
  • If Task tool available: 4 parallel gsd-codebase-mapper agents spawned with run_in_background=true
  • If Task tool NOT available: 4 sequential mapping passes performed inline (never using browser_subagent)
  • All 7 codebase documents exist
  • No empty documents (each should have >20 lines)
  • Clear completion summary with line counts
  • User offered clear next steps in GSD style </success_criteria>