Files
msd-core/get-shit-done/bin/lib/config.cjs
Tom Boucher 1a694fcac3 feat: auto-remap codebase after significant phase execution (closes #2003) (#2605)
* feat: auto-remap codebase after significant phase execution (#2003)

Adds a post-phase structural drift detector that compares the committed tree
against `.planning/codebase/STRUCTURE.md` and either warns or auto-remaps
the affected subtrees when drift exceeds a configurable threshold.

## Summary
- New `bin/lib/drift.cjs` — pure detector covering four drift categories:
  new directories outside mapped paths, new barrel exports at
  `(packages|apps)/*/src/index.*`, new migration files, and new route
  modules. Prioritizes the most-specific category per file.
- New `verify codebase-drift` CLI subcommand + SDK handler, registered as
  `gsd-sdk query verify.codebase-drift`.
- New `codebase_drift_gate` step in `execute-phase` between
  `schema_drift_gate` and `verify_phase_goal`. Non-blocking by contract —
  any error logs and the phase continues.
- Two new config keys: `workflow.drift_threshold` (int, default 3) and
  `workflow.drift_action` (`warn` | `auto-remap`, default `warn`), with
  enum/integer validation in `config-set`.
- `gsd-codebase-mapper` learns an optional `--paths <p1,p2,...>` scope hint
  for incremental remapping; agent/workflow docs updated.
- `last_mapped_commit` lives in YAML frontmatter on each
  `.planning/codebase/*.md` file; `readMappedCommit`/`writeMappedCommit`
  round-trip helpers ship in `drift.cjs`.

## Tests
- 55 new tests in `tests/drift-detection.test.cjs` covering:
  classification, threshold gating at 2/3/4 elements, warn vs. auto-remap
  routing, affected-path scoping, `--paths` sanitization (traversal,
  absolute, shell metacharacter rejection), frontmatter round-trip,
  defensive paths (missing STRUCTURE.md, malformed input, non-git repos),
  CLI JSON output, and documentation parity.
- Full suite: 5044 pass / 0 fail.

## Documentation
- `docs/CONFIGURATION.md` — rows for both new keys.
- `docs/ARCHITECTURE.md` — section on the post-execute drift gate.
- `docs/AGENTS.md` — `--paths` flag on `gsd-codebase-mapper`.
- `docs/USER-GUIDE.md` — user-facing behavior note + toggle commands.
- `docs/FEATURES.md` — new 27a section with REQ-DRIFT-01..06.
- `docs/INVENTORY.md` + `docs/INVENTORY-MANIFEST.json` — drift.cjs listed.
- `get-shit-done/workflows/execute-phase.md` — `codebase_drift_gate` step.
- `get-shit-done/workflows/map-codebase.md` — `parse_paths_flag` step.
- `agents/gsd-codebase-mapper.md` — `--paths` directive under parse_focus.

## Design decisions
- **Frontmatter over sidecar JSON** for `last_mapped_commit`: keeps the
  baseline attached to the file, survives git moves, survives per-doc
  regeneration, no extra file lifecycle.
- **Substring match against STRUCTURE.md** for `isPathMapped`: the map is
  free-form markdown, not a structured manifest; any mention of a path
  prefix counts as "mapped territory". Cheap, no parser, zero false
  negatives on reasonable maps.
- **Category priority migration > route > barrel > new_dir** so a file
  matching multiple rules counts exactly once at the most specific level.
- **Empty-tree SHA fallback** (`4b825dc6…`) when `last_mapped_commit` is
  absent — semantically correct (no baseline means everything is drift)
  and deterministic across repos.
- **Four layers of non-blocking** — detector try/catch, CLI try/catch, SDK
  handler try/catch, and workflow `|| echo` shell fallback. Any single
  layer failing still returns a valid skipped result.
- **SDK handler delegates to `gsd-tools.cjs`** rather than re-porting the
  detector to TypeScript, keeping drift logic in one canonical place.

Closes #2003

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(mapper): tag --paths fenced block as text (CodeRabbit MD040)

Comment 3127255172.

* docs(config): use /gsd- dash command syntax in drift_action row (CodeRabbit)

Comment 3127255180. Matches the convention used by every other command
reference in docs/CONFIGURATION.md.

* fix(execute-phase): initialize AGENT_SKILLS_MAPPER + tag fenced blocks

Two CodeRabbit findings on the auto-remap branch of the drift gate:

- 3127255186 (must-fix): the mapper Task prompt referenced
  ${AGENT_SKILLS_MAPPER} but only AGENT_SKILLS (for gsd-executor) is
  loaded at init_context (line 72). Without this fix the literal
  placeholder string would leak into the spawned mapper's prompt.
  Add an explicit gsd-sdk query agent-skills gsd-codebase-mapper step
  right before the Task spawn.
- 3127255183: tag the warn-message and Task() fenced code blocks as
  text to satisfy markdownlint MD040.

* docs(map-codebase): wire PATH_SCOPE_HINT through every mapper prompt

CodeRabbit (review id 4158286952, comment 3127255190) flagged that the
parse_paths_flag step defined incremental-remap semantics but did not
inject a normalized variable into the spawn_agents and sequential_mapping
mapper prompts, so incremental remap could silently regress to a
whole-repo scan.

- Define SCOPED_PATHS / PATH_SCOPE_HINT in parse_paths_flag.
- Inject ${PATH_SCOPE_HINT} into all four spawn_agents Task prompts.
- Document the same scope contract for sequential_mapping mode.

* fix(drift): writeMappedCommit tolerates missing target file

CodeRabbit (review id 4158286952, drift.cjs:349-355 nitpick) noted that
readMappedCommit returns null on ENOENT but writeMappedCommit threw — an
asymmetry that breaks first-time stamping of a freshly produced doc that
the caller has not yet written.

- Catch ENOENT on the read; treat absent file as empty content.
- Add a regression test that calls writeMappedCommit on a non-existent
  path and asserts the file is created with correct frontmatter.
  Test was authored to fail before the fix (ENOENT) and passes after.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 21:21:44 -04:00

477 lines
16 KiB
JavaScript

/**
* Config — Planning config CRUD operations
*/
const fs = require('fs');
const path = require('path');
const { output, error, planningDir, withPlanningLock, CONFIG_DEFAULTS, atomicWriteFileSync } = require('./core.cjs');
const {
VALID_PROFILES,
getAgentToModelMapForProfile,
formatAgentToModelMapAsTable,
} = require('./model-profiles.cjs');
const { VALID_CONFIG_KEYS, isValidConfigKey } = require('./config-schema.cjs');
const CONFIG_KEY_SUGGESTIONS = {
'workflow.nyquist_validation_enabled': 'workflow.nyquist_validation',
'agents.nyquist_validation_enabled': 'workflow.nyquist_validation',
'nyquist.validation_enabled': 'workflow.nyquist_validation',
'hooks.research_questions': 'workflow.research_before_questions',
'workflow.research_questions': 'workflow.research_before_questions',
'workflow.codereview': 'workflow.code_review',
'workflow.review_command': 'workflow.code_review_command',
'workflow.review': 'workflow.code_review',
'workflow.code_review_level': 'workflow.code_review_depth',
'workflow.review_depth': 'workflow.code_review_depth',
'review.model': 'review.models.<cli-name>',
'sub_repos': 'planning.sub_repos',
'plan_checker': 'workflow.plan_check',
};
function validateKnownConfigKeyPath(keyPath) {
const suggested = CONFIG_KEY_SUGGESTIONS[keyPath];
if (suggested) {
error(`Unknown config key: ${keyPath}. Did you mean ${suggested}?`);
}
}
/**
* Build a fully-materialized config object for a new project.
*
* Merges (increasing priority):
* 1. Hardcoded defaults — every key that loadConfig() resolves, plus mode/granularity
* 2. User-level defaults from ~/.gsd/defaults.json (if present)
* 3. userChoices — the settings the user explicitly selected during /gsd:new-project
*
* Uses the canonical `git` namespace for branching keys (consistent with VALID_CONFIG_KEYS
* and the settings workflow). loadConfig() handles both flat and nested formats, so this
* is backward-compatible with existing projects that have flat keys.
*
* Returns a plain object — does NOT write any files.
*/
function buildNewProjectConfig(userChoices) {
const choices = userChoices || {};
const homedir = require('os').homedir();
// Detect API key availability
const braveKeyFile = path.join(homedir, '.gsd', 'brave_api_key');
const hasBraveSearch = !!(process.env.BRAVE_API_KEY || fs.existsSync(braveKeyFile));
const firecrawlKeyFile = path.join(homedir, '.gsd', 'firecrawl_api_key');
const hasFirecrawl = !!(process.env.FIRECRAWL_API_KEY || fs.existsSync(firecrawlKeyFile));
const exaKeyFile = path.join(homedir, '.gsd', 'exa_api_key');
const hasExaSearch = !!(process.env.EXA_API_KEY || fs.existsSync(exaKeyFile));
// Load user-level defaults from ~/.gsd/defaults.json if available
const globalDefaultsPath = path.join(homedir, '.gsd', 'defaults.json');
let userDefaults = {};
try {
if (fs.existsSync(globalDefaultsPath)) {
userDefaults = JSON.parse(fs.readFileSync(globalDefaultsPath, 'utf-8'));
// Migrate deprecated "depth" key to "granularity"
if ('depth' in userDefaults && !('granularity' in userDefaults)) {
const depthToGranularity = { quick: 'coarse', standard: 'standard', comprehensive: 'fine' };
userDefaults.granularity = depthToGranularity[userDefaults.depth] || userDefaults.depth;
delete userDefaults.depth;
try {
fs.writeFileSync(globalDefaultsPath, JSON.stringify(userDefaults, null, 2), 'utf-8');
} catch { /* intentionally empty */ }
}
}
} catch {
// Ignore malformed global defaults
}
const hardcoded = {
model_profile: CONFIG_DEFAULTS.model_profile,
commit_docs: CONFIG_DEFAULTS.commit_docs,
parallelization: CONFIG_DEFAULTS.parallelization,
search_gitignored: CONFIG_DEFAULTS.search_gitignored,
brave_search: hasBraveSearch,
firecrawl: hasFirecrawl,
exa_search: hasExaSearch,
git: {
branching_strategy: CONFIG_DEFAULTS.branching_strategy,
phase_branch_template: CONFIG_DEFAULTS.phase_branch_template,
milestone_branch_template: CONFIG_DEFAULTS.milestone_branch_template,
quick_branch_template: CONFIG_DEFAULTS.quick_branch_template,
},
workflow: {
research: true,
plan_check: true,
verifier: true,
nyquist_validation: true,
auto_advance: false,
node_repair: true,
node_repair_budget: 2,
ui_phase: true,
ui_safety_gate: true,
ai_integration_phase: true,
tdd_mode: false,
text_mode: false,
research_before_questions: false,
discuss_mode: 'discuss',
skip_discuss: false,
code_review: true,
code_review_depth: 'standard',
code_review_command: null,
pattern_mapper: true,
plan_bounce: false,
plan_bounce_script: null,
plan_bounce_passes: 2,
auto_prune_state: false,
security_enforcement: CONFIG_DEFAULTS.security_enforcement,
security_asvs_level: CONFIG_DEFAULTS.security_asvs_level,
security_block_on: CONFIG_DEFAULTS.security_block_on,
},
hooks: {
context_warnings: true,
},
project_code: null,
phase_naming: 'sequential',
agent_skills: {},
claude_md_path: './CLAUDE.md',
};
// Three-level deep merge: hardcoded <- userDefaults <- choices
return {
...hardcoded,
...userDefaults,
...choices,
git: {
...hardcoded.git,
...(userDefaults.git || {}),
...(choices.git || {}),
},
workflow: {
...hardcoded.workflow,
...(userDefaults.workflow || {}),
...(choices.workflow || {}),
},
hooks: {
...hardcoded.hooks,
...(userDefaults.hooks || {}),
...(choices.hooks || {}),
},
agent_skills: {
...hardcoded.agent_skills,
...(userDefaults.agent_skills || {}),
...(choices.agent_skills || {}),
},
};
}
/**
* Command: create a fully-materialized .planning/config.json for a new project.
*
* Accepts user-chosen settings as a JSON string (the keys the user explicitly
* configured during /gsd:new-project). All remaining keys are filled from
* hardcoded defaults and optional ~/.gsd/defaults.json.
*
* Idempotent: if config.json already exists, returns { created: false }.
*/
function cmdConfigNewProject(cwd, choicesJson, raw) {
const planningBase = planningDir(cwd);
const configPath = path.join(planningBase, 'config.json');
// Idempotent: don't overwrite existing config
if (fs.existsSync(configPath)) {
output({ created: false, reason: 'already_exists' }, raw, 'exists');
return;
}
// Parse user choices
let userChoices = {};
if (choicesJson && choicesJson.trim() !== '') {
try {
userChoices = JSON.parse(choicesJson);
} catch (err) {
error('Invalid JSON for config-new-project: ' + err.message);
}
}
// Ensure .planning directory exists
try {
if (!fs.existsSync(planningBase)) {
fs.mkdirSync(planningBase, { recursive: true });
}
} catch (err) {
error('Failed to create .planning directory: ' + err.message);
}
const config = buildNewProjectConfig(userChoices);
try {
atomicWriteFileSync(configPath, JSON.stringify(config, null, 2), 'utf-8');
output({ created: true, path: '.planning/config.json' }, raw, 'created');
} catch (err) {
error('Failed to write config.json: ' + err.message);
}
}
/**
* Ensures the config file exists (creates it if needed).
*
* Does not call `output()`, so can be used as one step in a command without triggering `exit(0)` in
* the happy path. But note that `error()` will still `exit(1)` out of the process.
*/
function ensureConfigFile(cwd) {
const planningBase = planningDir(cwd);
const configPath = path.join(planningBase, 'config.json');
// Ensure .planning directory exists
try {
if (!fs.existsSync(planningBase)) {
fs.mkdirSync(planningBase, { recursive: true });
}
} catch (err) {
error('Failed to create .planning directory: ' + err.message);
}
// Check if config already exists
if (fs.existsSync(configPath)) {
return { created: false, reason: 'already_exists' };
}
const config = buildNewProjectConfig({});
try {
atomicWriteFileSync(configPath, JSON.stringify(config, null, 2), 'utf-8');
return { created: true, path: '.planning/config.json' };
} catch (err) {
error('Failed to create config.json: ' + err.message);
}
}
/**
* Command to ensure the config file exists (creates it if needed).
*
* Note that this exits the process (via `output()`) even in the happy path; use
* `ensureConfigFile()` directly if you need to avoid this.
*/
function cmdConfigEnsureSection(cwd, raw) {
const ensureConfigFileResult = ensureConfigFile(cwd);
if (ensureConfigFileResult.created) {
output(ensureConfigFileResult, raw, 'created');
} else {
output(ensureConfigFileResult, raw, 'exists');
}
}
/**
* Sets a value in the config file, allowing nested values via dot notation (e.g.,
* "workflow.research").
*
* Does not call `output()`, so can be used as one step in a command without triggering `exit(0)` in
* the happy path. But note that `error()` will still `exit(1)` out of the process.
*/
function setConfigValue(cwd, keyPath, parsedValue) {
const configPath = path.join(planningDir(cwd), 'config.json');
return withPlanningLock(cwd, () => {
// Load existing config or start with empty object
let config = {};
try {
if (fs.existsSync(configPath)) {
config = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
}
} catch (err) {
error('Failed to read config.json: ' + err.message);
}
// Set nested value using dot notation (e.g., "workflow.research")
const keys = keyPath.split('.');
let current = config;
for (let i = 0; i < keys.length - 1; i++) {
const key = keys[i];
if (current[key] === undefined || typeof current[key] !== 'object') {
current[key] = {};
}
current = current[key];
}
const previousValue = current[keys[keys.length - 1]]; // Capture previous value before overwriting
current[keys[keys.length - 1]] = parsedValue;
// Write back
try {
atomicWriteFileSync(configPath, JSON.stringify(config, null, 2), 'utf-8');
return { updated: true, key: keyPath, value: parsedValue, previousValue };
} catch (err) {
error('Failed to write config.json: ' + err.message);
}
});
}
/**
* Command to set a value in the config file, allowing nested values via dot notation (e.g.,
* "workflow.research").
*
* Note that this exits the process (via `output()`) even in the happy path; use `setConfigValue()`
* directly if you need to avoid this.
*/
function cmdConfigSet(cwd, keyPath, value, raw) {
if (!keyPath) {
error('Usage: config-set <key.path> <value>');
}
validateKnownConfigKeyPath(keyPath);
if (!isValidConfigKey(keyPath)) {
error(`Unknown config key: "${keyPath}". Valid keys: ${[...VALID_CONFIG_KEYS].sort().join(', ')}, agent_skills.<agent-type>, features.<feature_name>`);
}
// Parse value (handle booleans, numbers, and JSON arrays/objects)
let parsedValue = value;
if (value === 'true') parsedValue = true;
else if (value === 'false') parsedValue = false;
else if (!isNaN(value) && value !== '') parsedValue = Number(value);
else if (typeof value === 'string' && (value.startsWith('[') || value.startsWith('{'))) {
try { parsedValue = JSON.parse(value); } catch { /* keep as string */ }
}
const VALID_CONTEXT_VALUES = ['dev', 'research', 'review'];
if (keyPath === 'context' && !VALID_CONTEXT_VALUES.includes(String(parsedValue))) {
error(`Invalid context value '${value}'. Valid values: ${VALID_CONTEXT_VALUES.join(', ')}`);
}
// Codebase drift detector (#2003)
const VALID_DRIFT_ACTIONS = ['warn', 'auto-remap'];
if (keyPath === 'workflow.drift_action' && !VALID_DRIFT_ACTIONS.includes(String(parsedValue))) {
error(`Invalid workflow.drift_action '${value}'. Valid values: ${VALID_DRIFT_ACTIONS.join(', ')}`);
}
if (keyPath === 'workflow.drift_threshold') {
if (typeof parsedValue !== 'number' || !Number.isInteger(parsedValue) || parsedValue < 1) {
error(`Invalid workflow.drift_threshold '${value}'. Must be a positive integer.`);
}
}
const setConfigValueResult = setConfigValue(cwd, keyPath, parsedValue);
output(setConfigValueResult, raw, `${keyPath}=${parsedValue}`);
}
function cmdConfigGet(cwd, keyPath, raw, defaultValue) {
const configPath = path.join(planningDir(cwd), 'config.json');
const hasDefault = defaultValue !== undefined;
if (!keyPath) {
error('Usage: config-get <key.path> [--default <value>]');
}
let config = {};
try {
if (fs.existsSync(configPath)) {
config = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
} else if (hasDefault) {
output(defaultValue, raw, String(defaultValue));
return;
} else {
error('No config.json found at ' + configPath);
}
} catch (err) {
if (err.message.startsWith('No config.json')) throw err;
error('Failed to read config.json: ' + err.message);
}
// Traverse dot-notation path (e.g., "workflow.auto_advance")
const keys = keyPath.split('.');
let current = config;
for (const key of keys) {
if (current === undefined || current === null || typeof current !== 'object') {
if (hasDefault) { output(defaultValue, raw, String(defaultValue)); return; }
error(`Key not found: ${keyPath}`);
}
current = current[key];
}
if (current === undefined) {
if (hasDefault) { output(defaultValue, raw, String(defaultValue)); return; }
error(`Key not found: ${keyPath}`);
}
output(current, raw, String(current));
}
/**
* Command to set the model profile in the config file.
*
* Note that this exits the process (via `output()`) even in the happy path.
*/
function cmdConfigSetModelProfile(cwd, profile, raw) {
if (!profile) {
error(`Usage: config-set-model-profile <${VALID_PROFILES.join('|')}>`);
}
const normalizedProfile = profile.toLowerCase().trim();
if (!VALID_PROFILES.includes(normalizedProfile)) {
error(`Invalid profile '${profile}'. Valid profiles: ${VALID_PROFILES.join(', ')}`);
}
// Ensure config exists (create if needed)
ensureConfigFile(cwd);
// Set the model profile in the config
const { previousValue } = setConfigValue(cwd, 'model_profile', normalizedProfile, raw);
const previousProfile = previousValue || 'balanced';
// Build result value / message and return
const agentToModelMap = getAgentToModelMapForProfile(normalizedProfile);
const result = {
updated: true,
profile: normalizedProfile,
previousProfile,
agentToModelMap,
};
const rawValue = getCmdConfigSetModelProfileResultMessage(
normalizedProfile,
previousProfile,
agentToModelMap
);
output(result, raw, rawValue);
}
/**
* Returns the message to display for the result of the `config-set-model-profile` command when
* displaying raw output.
*/
function getCmdConfigSetModelProfileResultMessage(
normalizedProfile,
previousProfile,
agentToModelMap
) {
const agentToModelTable = formatAgentToModelMapAsTable(agentToModelMap);
const didChange = previousProfile !== normalizedProfile;
const paragraphs = didChange
? [
`✓ Model profile set to: ${normalizedProfile} (was: ${previousProfile})`,
'Agents will now use:',
agentToModelTable,
'Next spawned agents will use the new profile.',
]
: [
`✓ Model profile is already set to: ${normalizedProfile}`,
'Agents are using:',
agentToModelTable,
];
return paragraphs.join('\n\n');
}
/**
* Print the resolved config.json path (workstream-aware). Used by settings.md
* so the workflow writes/reads the correct file when a workstream is active (#2282).
*/
function cmdConfigPath(cwd) {
// Always emit as plain text — a file path is used via shell substitution,
// never consumed as JSON. Passing raw=true forces plain-text output.
const configPath = path.join(planningDir(cwd), 'config.json');
output(configPath, true, configPath);
}
module.exports = {
VALID_CONFIG_KEYS,
cmdConfigEnsureSection,
cmdConfigSet,
cmdConfigGet,
cmdConfigSetModelProfile,
cmdConfigNewProject,
cmdConfigPath,
};