diff --git a/.changeset/patient-ibex-wake.md b/.changeset/patient-ibex-wake.md new file mode 100644 index 000000000..8ee84fd82 --- /dev/null +++ b/.changeset/patient-ibex-wake.md @@ -0,0 +1,5 @@ +--- +type: Added +pr: 487 +--- +Added a plan-vs-codebase drift guard. /gsd:plan-review-convergence now verifies (default on, plan_review.source_grounding) that symbols a plan cites actually exist in your source, flagging hallucinated names as needs-acknowledgement before execution instead of at runtime; UNCHECKABLE cases are logged, never silently passed. With intel enabled, the planner also receives a rendered API-SURFACE.md hint (gsd-tools intel api-surface). The resolver authority is configurable via plan_review.source_grounding_authority (grep|intel|...). diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 4556731ca..25cac74df 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -624,6 +624,8 @@ The migration module also owns the gated first-time baseline scan for legacy installs, classifying known runtime install surfaces before later migrations remove or rewrite anything. +The plan drift guard (`plan_review.source_grounding`) — which verifies symbol references in generated plans against live source before execution — is specified in [ADR 22](adr/22-plan-drift-guard.md). + ### Platform Handling - **Windows:** `windowsHide` on child processes, EPERM/EACCES protection on protected directories, path separator normalization diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md index 6e4bf1d3a..fffaa3d39 100644 --- a/docs/COMMANDS.md +++ b/docs/COMMANDS.md @@ -1042,6 +1042,18 @@ Build, query, and inspect the project knowledge graph stored in `.planning/graph **Programmatic access:** `node gsd-tools.cjs graphify ` — see [CLI Tools Reference](CLI-TOOLS.md). +### `gsd-tools intel api-surface` + +Render the `.planning/intel/api-map.json` index (built by `/gsd-map-codebase`) into a human-readable `API-SURFACE.md` in `.planning/intel/`. Gated on `intel.enabled: true` in `config.json`; when Intel is disabled the command prints an activation hint and exits. The output path is always `.planning/intel/API-SURFACE.md` — there is no `--out` or `--format` flag. When `api-map.json` is absent or empty the command still writes the file with an explicit "incomplete" banner so consumers never mistake silence for "nothing exists". + +**Produces:** `.planning/intel/API-SURFACE.md` + +```bash +node gsd-tools.cjs intel api-surface # Render api-map.json → API-SURFACE.md +``` + +The `API-SURFACE.md` output lists exported symbols (functions, classes, decorators, constants) grouped by source file with their signatures and detected visibility. When `plan_review.source_grounding_authority` is set to `intel`, the plan drift guard reads `api-map.json` directly rather than invoking the `api-surface` renderer. + --- ## AI Integration Commands diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 65a3180fe..29723f08e 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -457,6 +457,16 @@ Toggle optional capabilities via the `features.*` config namespace. Feature flag | `learnings.max_inject` | number | `10` | Maximum number of cross-project learnings injected into each planner prompt. Lower values reduce prompt size; higher values provide broader historical context | | `intel.enabled` | boolean | `false` | Enable queryable codebase intelligence system. When `true`, `/gsd-map-codebase --query` commands build and query a JSON index in `.planning/intel/`. Added in v1.34 | + +### Plan Review Settings + +The `plan_review.*` namespace controls the plan drift guard, which verifies that symbols cited in generated plans (decorators, classes, functions, CLI flags) actually exist in your source code at review time. This catches hallucinated names before execution begins. + +| Setting | Type | Default | Description | +|---------|------|---------|-------------| +| `plan_review.source_grounding` | boolean | `true` | Enable the plan drift guard. When `true` (the default), plan review resolves every symbol reference cited in a PLAN.md against the live source tree. Plans that cite a non-existent function, class, decorator, or CLI flag produce a `needs-acknowledgement` notice before the plan is approved. Disable with `false` to skip symbol verification entirely. Toggle during setup (`/gsd:new-project`) or at any time via `/gsd:settings`. | +| `plan_review.source_grounding_authority` | enum | `grep` | Selects the resolver adapter used to verify symbol existence. Allowed values: `grep` (default — ripgrep/grep search of source files, works in any project without additional tooling), `intel` (query the `.planning/intel/api-map.json` index built by `/gsd:map-codebase`; requires `intel.enabled: true`), `treesitter` (reserved for future tree-sitter adapter), `lsp` (reserved for future LSP adapter), `scip` (reserved for future SCIP/LSIF adapter). Use `intel` when you have run `/gsd:map-codebase` and want the faster, pre-indexed lookup. All other values beyond `grep` and `intel` are reserved and have no effect in the current release. | + ### Graphify Settings diff --git a/docs/USER-GUIDE.md b/docs/USER-GUIDE.md index af7606319..c5481e5e2 100644 --- a/docs/USER-GUIDE.md +++ b/docs/USER-GUIDE.md @@ -992,6 +992,33 @@ invocation to refresh just the affected subtrees. Flip the behavior with: The gate is non-blocking: any internal failure logs and the phase continues. +### Plan Drift Guard + +**Default-on.** The plan drift guard (`plan_review.source_grounding: true`) runs during plan review and verifies that every symbol your plans cite — decorators, classes, functions, CLI flags — actually exists in your source tree at review time. This catches hallucinated names (symbols the planner invented but that don't exist yet) before any execution agent runs. + +**What it catches:** + +- Functions referenced in a PLAN.md step that don't exist in source +- Class or decorator names that were renamed or removed since the plan was written +- CLI flags documented in a plan that are not defined in the argument parser +- Module paths cited in implementation steps that resolve to no files + +**Needs-acknowledgement behavior.** When the guard finds a missing symbol, it emits a `needs-acknowledgement` notice in the plan review output rather than hard-blocking. You can acknowledge and proceed (the symbol may be intentionally new) or request a plan revision. The guard does not auto-reject plans — it surfaces signal for human decision. + +**Works without intel.** By default the guard uses `grep`/`ripgrep` to search source files — no pre-indexing required. If you have run `/gsd:map-codebase` with `intel.enabled: true`, set `plan_review.source_grounding_authority: intel` to use the faster pre-built `api-map.json` index instead. + +```bash +# Enable/disable (default: on) +/gsd-settings plan_review.source_grounding true +/gsd-settings plan_review.source_grounding false + +# Switch resolver authority +/gsd-settings plan_review.source_grounding_authority grep # live grep (default) +/gsd-settings plan_review.source_grounding_authority intel # pre-indexed api-map.json +``` + +Toggle at project setup (`/gsd:new-project` asks during workflow preferences) or any time via `/gsd:settings` (Planning section → Drift Guard). + ### Quick Bug Fix ```bash diff --git a/get-shit-done/bin/gsd-tools.cjs b/get-shit-done/bin/gsd-tools.cjs index 83c472c36..f6c9302af 100755 --- a/get-shit-done/bin/gsd-tools.cjs +++ b/get-shit-done/bin/gsd-tools.cjs @@ -83,6 +83,7 @@ * intel patch-meta Update _meta.updated_at in an intel file * intel validate Validate intel file structure * intel extract-exports Extract exported symbols from a source file + * intel api-surface Render api-map.json into API-SURFACE.md * * Scaffolding: * scaffold context --phase Create CONTEXT.md template @@ -1266,8 +1267,11 @@ async function runCommand(command, args, cwd, raw, defaultValue, originalCommand } else if (subcommand === 'update') { const planningDir = path.join(cwd, '.planning'); core.output(intel.intelUpdate(planningDir), raw); + } else if (subcommand === 'api-surface') { + const planningDir = path.join(cwd, '.planning'); + core.output(intel.intelApiSurface(planningDir), raw); } else { - error('Unknown intel subcommand. Available: query, status, update, diff, snapshot, patch-meta, validate, extract-exports', ERROR_REASON.SDK_UNKNOWN_COMMAND); + error('Unknown intel subcommand. Available: query, status, update, diff, snapshot, patch-meta, validate, extract-exports, api-surface', ERROR_REASON.SDK_UNKNOWN_COMMAND); } break; } diff --git a/get-shit-done/bin/lib/config.cjs b/get-shit-done/bin/lib/config.cjs index 87bf5ca9e..c89135948 100644 --- a/get-shit-done/bin/lib/config.cjs +++ b/get-shit-done/bin/lib/config.cjs @@ -208,6 +208,10 @@ function buildNewProjectConfig(userChoices) { phase_naming: 'sequential', agent_skills: {}, claude_md_path: './CLAUDE.md', + plan_review: { + source_grounding: true, + source_grounding_authority: 'grep', + }, }; // Three-level deep merge: hardcoded <- userDefaults <- choices @@ -240,6 +244,11 @@ function buildNewProjectConfig(userChoices) { ...(userDefaults.agent_skills || {}), ...(choices.agent_skills || {}), }, + plan_review: { + ...hardcoded.plan_review, + ...(userDefaults.plan_review || {}), + ...(choices.plan_review || {}), + }, }; validateShipPrBodySections(config.ship.pr_body_sections); @@ -476,6 +485,19 @@ function cmdConfigSet(cwd, keyPath, value, raw) { error(`Invalid code_quality.fallow.profile '${value}'. Valid values: ${VALID_FALLOW_PROFILES.join(', ')}`); } + // plan_review.source_grounding (#22) — boolean only + if (keyPath === 'plan_review.source_grounding') { + if (typeof parsedValue !== 'boolean') { + error(`Invalid plan_review.source_grounding '${value}'. Must be a boolean (true or false).`); + } + } + + // plan_review.source_grounding_authority (#22) — enum + const VALID_SOURCE_GROUNDING_AUTHORITIES = ['grep', 'intel', 'treesitter', 'lsp', 'scip']; + if (keyPath === 'plan_review.source_grounding_authority' && !VALID_SOURCE_GROUNDING_AUTHORITIES.includes(String(parsedValue))) { + error(`Invalid plan_review.source_grounding_authority '${value}'. Valid values: ${VALID_SOURCE_GROUNDING_AUTHORITIES.join(', ')}`); + } + if (keyPath === 'review.default_reviewers') { const normalized = normalizeConfiguredDefaultReviewers(parsedValue); if (normalized.errors.length > 0) { diff --git a/get-shit-done/bin/lib/intel.cjs b/get-shit-done/bin/lib/intel.cjs index d33affa42..c7e523fa0 100644 --- a/get-shit-done/bin/lib/intel.cjs +++ b/get-shit-done/bin/lib/intel.cjs @@ -456,6 +456,70 @@ function intelValidate(planningDir) { return { valid: errors.length === 0, errors, warnings }; } +/** + * Render .planning/intel/api-map.json into a human-readable API-SURFACE.md. + * Always writes the file — even when api-map.json is absent or empty, the + * surface will contain an explicit "incomplete" banner so consumers never + * mistake silence for "nothing exists". + * + * @param {string} planningDir - Path to .planning directory + * @returns {{ written: string, symbolCount: number, stale: boolean } | { disabled: true, message: string }} + */ +function intelApiSurface(planningDir) { + if (!isIntelEnabled(planningDir)) return disabledResponse(); + + const intelPath = ensureIntelDir(planningDir); + const apiMapPath = path.join(intelPath, INTEL_FILES.apis); + const outputPath = path.join(intelPath, 'API-SURFACE.md'); + + const data = safeReadJson(apiMapPath); + const entries = (data && data.entries && typeof data.entries === 'object') + ? Object.entries(data.entries) + : []; + const symbolCount = entries.length; + + // Staleness: reuse the _meta.updated_at field if present + const STALE_MS = 24 * 60 * 60 * 1000; + let stale = true; + if (data && data._meta && data._meta.updated_at) { + const age = Date.now() - new Date(data._meta.updated_at).getTime(); + stale = age > STALE_MS; + } + + const lines = []; + lines.push('# API Surface'); + lines.push(''); + lines.push('> Generated from `.planning/intel/api-map.json`. Do not edit by hand.'); + lines.push(''); + + if (symbolCount === 0) { + lines.push('> **Incomplete:** api-map.json has no entries (intel extraction is regex/JS-only or not yet populated).'); + lines.push('> Treat absence here as "unknown", not "does not exist".'); + lines.push(''); + } else { + if (stale) { + lines.push('> **Warning:** api-map.json is stale (>24 hours old). Data below may be out of date.'); + lines.push(''); + } + + for (const [symbol, info] of entries) { + lines.push(`## \`${symbol}\``); + lines.push(''); + if (info && typeof info === 'object') { + for (const [field, val] of Object.entries(info)) { + const display = Array.isArray(val) ? val.join(', ') : String(val); + lines.push(`- **${field}:** ${display}`); + } + } + lines.push(''); + } + } + + platformWriteSync(outputPath, lines.join('\n')); + + return { written: outputPath, symbolCount, stale }; +} + /** * Patch _meta.updated_at in a JSON intel file to the current timestamp. * Reads the file, updates _meta.updated_at, increments version, writes back. @@ -632,6 +696,7 @@ module.exports = { intelValidate, intelExtractExports, intelPatchMeta, + intelApiSurface, // Utilities ensureIntelDir, diff --git a/get-shit-done/bin/shared/config-defaults.manifest.json b/get-shit-done/bin/shared/config-defaults.manifest.json index 4e6590a6d..3939fed8e 100644 --- a/get-shit-done/bin/shared/config-defaults.manifest.json +++ b/get-shit-done/bin/shared/config-defaults.manifest.json @@ -89,5 +89,9 @@ "heavy": false }, "agent_overrides": {} + }, + "plan_review": { + "source_grounding": true, + "source_grounding_authority": "grep" } } diff --git a/get-shit-done/bin/shared/config-schema.manifest.json b/get-shit-done/bin/shared/config-schema.manifest.json index 35d10ef06..bac2c81b2 100644 --- a/get-shit-done/bin/shared/config-schema.manifest.json +++ b/get-shit-done/bin/shared/config-schema.manifest.json @@ -98,7 +98,9 @@ "runtime", "resolve_model_ids", "effort.default", - "fast_mode.enabled" + "fast_mode.enabled", + "plan_review.source_grounding", + "plan_review.source_grounding_authority" ], "runtimeStateKeys": [ "workflow._auto_chain_active" diff --git a/get-shit-done/workflows/new-project.md b/get-shit-done/workflows/new-project.md index ba99f0147..f8b94a76e 100644 --- a/get-shit-done/workflows/new-project.md +++ b/get-shit-done/workflows/new-project.md @@ -222,6 +222,15 @@ AskUserQuestion([ { label: "No", description: "Trust execution, skip verification" } ] }, + { + header: "Drift Guard", + question: "Enable the plan drift-guard? It verifies that symbols your plans cite (decorators, classes, functions, CLI flags) actually exist in your source at review time, catching hallucinated names before execution. [Y/n]", + multiSelect: false, + options: [ + { label: "Yes (Recommended)", description: "Resolve symbol references against live source during plan review — catches hallucinated names before execution" }, + { label: "No", description: "Skip symbol grounding — plan review proceeds without source verification" } + ] + }, { header: "AI Models", question: "Which AI models for planning agents?", @@ -264,7 +273,7 @@ Create `.planning/config.json` with all settings (CLI fills in remaining default ```bash mkdir -p .planning -gsd_run query config-new-project '{"mode":"yolo","granularity":"[selected]","parallelization":true|false,"commit_docs":true|false,"model_profile":"quality|balanced|budget|inherit","workflow":{"research":true|false,"plan_check":true|false,"verifier":true|false,"nyquist_validation":true|false,"auto_advance":true},"ship":{"pr_body_sections":[{"heading":"User Stories & Acceptance Criteria","enabled":true|false,"source":"REQUIREMENTS.md ## User Stories || REQUIREMENTS.md ## Acceptance Criteria","fallback":"- Acceptance criteria are covered by the linked requirements and verification evidence."},{"heading":"Risks & Dependencies","enabled":true|false,"source":"PLAN.md ## Risks || PLAN.md ## Dependencies","fallback":"- No known high-risk rollout dependencies."},{"heading":"Success Metrics & Release Criteria","enabled":true|false,"source":"REQUIREMENTS.md ## Definition of Done || VERIFICATION.md ## Release Criteria","fallback":"- Release when automated verification and required manual checks pass."},{"heading":"Stakeholder Review & Approval","enabled":true|false,"template":"- Product owner approval pending for {phase_name}."}]}}' +gsd_run query config-new-project '{"mode":"yolo","granularity":"[selected]","parallelization":true|false,"commit_docs":true|false,"model_profile":"quality|balanced|budget|inherit","workflow":{"research":true|false,"plan_check":true|false,"verifier":true|false,"nyquist_validation":true|false,"auto_advance":true},"plan_review":{"source_grounding":true|false},"ship":{"pr_body_sections":[{"heading":"User Stories & Acceptance Criteria","enabled":true|false,"source":"REQUIREMENTS.md ## User Stories || REQUIREMENTS.md ## Acceptance Criteria","fallback":"- Acceptance criteria are covered by the linked requirements and verification evidence."},{"heading":"Risks & Dependencies","enabled":true|false,"source":"PLAN.md ## Risks || PLAN.md ## Dependencies","fallback":"- No known high-risk rollout dependencies."},{"heading":"Success Metrics & Release Criteria","enabled":true|false,"source":"REQUIREMENTS.md ## Definition of Done || VERIFICATION.md ## Release Criteria","fallback":"- Release when automated verification and required manual checks pass."},{"heading":"Stakeholder Review & Approval","enabled":true|false,"template":"- Product owner approval pending for {phase_name}."}]}}' ``` **If commit_docs = No:** Add `.planning/` to `.gitignore`. @@ -506,6 +515,7 @@ Format the JSON into human-readable bullets using these label mappings: - `workflow.research` → "Research" (`true` → "Yes", `false` → "No") - `workflow.plan_check` → "Plan Check" (`true` → "Yes", `false` → "No") - `workflow.verifier` → "Verifier" (`true` → "Yes", `false` → "No") +- `plan_review.source_grounding` → "Drift Guard" (`true` → "Yes", `false` → "No") Display above the prompt: @@ -519,6 +529,7 @@ Your saved defaults (~/.gsd/defaults.json): • Research: [Yes|No] • Plan Check: [Yes|No] • Verifier: [Yes|No] + • Drift Guard: [Yes|No] ``` Then ask: @@ -555,6 +566,7 @@ Which settings do you want to change? (enter numbers, comma-separated) 6. Research — Currently: [Yes|No] 7. Plan Check — Currently: [Yes|No] 8. Verifier — Currently: [Yes|No] + 9. Drift Guard — Currently: [Yes|No] ``` **Otherwise** (Claude runtime with AskUserQuestion): use a two-block split @@ -626,6 +638,22 @@ AskUserQuestion([ ]) ``` +Then ask: + +```text +AskUserQuestion([ + { + question: "Do you want to change the Drift Guard setting (plan-review source-grounding)?", + header: "Drift Guard", + multiSelect: false, + options: [ + { label: "Yes", description: "Toggle Drift Guard (currently: [Yes|No])" }, + { label: "No", description: "Keep current Drift Guard setting" } + ] + } +]) +``` + For each selected setting across both blocks, ask only that question using the option set from Round 1 / Round 2 below. Merge user answers over the saved defaults — unchanged settings retain their saved values. Then skip to @@ -745,7 +773,7 @@ Create `.planning/config.json` with all settings (CLI fills in remaining default ```bash mkdir -p .planning -gsd_run query config-new-project '{"mode":"[yolo|interactive]","granularity":"[selected]","parallelization":true|false,"commit_docs":true|false,"model_profile":"quality|balanced|budget|inherit","workflow":{"research":true|false,"plan_check":true|false,"verifier":true|false,"nyquist_validation":[false if granularity=coarse, true otherwise]},"ship":{"pr_body_sections":[{"heading":"User Stories & Acceptance Criteria","enabled":true|false,"source":"REQUIREMENTS.md ## User Stories || REQUIREMENTS.md ## Acceptance Criteria","fallback":"- Acceptance criteria are covered by the linked requirements and verification evidence."},{"heading":"Risks & Dependencies","enabled":true|false,"source":"PLAN.md ## Risks || PLAN.md ## Dependencies","fallback":"- No known high-risk rollout dependencies."},{"heading":"Success Metrics & Release Criteria","enabled":true|false,"source":"REQUIREMENTS.md ## Definition of Done || VERIFICATION.md ## Release Criteria","fallback":"- Release when automated verification and required manual checks pass."},{"heading":"Stakeholder Review & Approval","enabled":true|false,"template":"- Product owner approval pending for {phase_name}."}]}}' +gsd_run query config-new-project '{"mode":"[yolo|interactive]","granularity":"[selected]","parallelization":true|false,"commit_docs":true|false,"model_profile":"quality|balanced|budget|inherit","workflow":{"research":true|false,"plan_check":true|false,"verifier":true|false,"nyquist_validation":[false if granularity=coarse, true otherwise]},"plan_review":{"source_grounding":true|false},"ship":{"pr_body_sections":[{"heading":"User Stories & Acceptance Criteria","enabled":true|false,"source":"REQUIREMENTS.md ## User Stories || REQUIREMENTS.md ## Acceptance Criteria","fallback":"- Acceptance criteria are covered by the linked requirements and verification evidence."},{"heading":"Risks & Dependencies","enabled":true|false,"source":"PLAN.md ## Risks || PLAN.md ## Dependencies","fallback":"- No known high-risk rollout dependencies."},{"heading":"Success Metrics & Release Criteria","enabled":true|false,"source":"REQUIREMENTS.md ## Definition of Done || VERIFICATION.md ## Release Criteria","fallback":"- Release when automated verification and required manual checks pass."},{"heading":"Stakeholder Review & Approval","enabled":true|false,"template":"- Product owner approval pending for {phase_name}."}]}}' ``` **Note:** Run `/gsd:settings` anytime to update model profile, workflow agents, branching strategy, and other preferences. diff --git a/get-shit-done/workflows/plan-phase.md b/get-shit-done/workflows/plan-phase.md index 8d2a9a356..9428e7dc5 100644 --- a/get-shit-done/workflows/plan-phase.md +++ b/get-shit-done/workflows/plan-phase.md @@ -857,6 +857,18 @@ After pattern mapper completes, update the path variable: PATTERNS_PATH="${PHASE_DIR}/${PADDED_PHASE}-PATTERNS.md" ``` +## 7.9. Regenerate API-SURFACE.md (intel gate) + +```bash +INTEL_CFG=$(gsd_run query config-get intel.enabled 2>/dev/null || echo "false") +# false (absent = false) → API_SURFACE_PATH stays empty; step-8 planner entry omitted +if [ "$INTEL_CFG" = "true" ]; then + gsd_run intel api-surface + API_SURFACE_PATH=".planning/intel/API-SURFACE.md" + echo "✓ API surface regenerated: ${API_SURFACE_PATH}" # injected into step 8 as HINT +fi +``` + ## 8. Spawn gsd-planner Agent Display banner: @@ -888,6 +900,7 @@ Planner prompt: - {UI_SPEC_PATH} (UI Design Contract — visual/interaction specs, if exists) - {SPIKE_FINDINGS_PATH} (Spike Findings — validated patterns, constraints, landmines from experiments, if exists) - {SKETCH_FINDINGS_PATH} (Sketch Findings — validated design decisions, CSS patterns, visual direction, if exists) +- {API_SURFACE_PATH} (API Surface — HINT ONLY, if intel.enabled; see below) ${CONTEXT_WINDOW >= 500000 ? ` **Cross-phase context (1M model enrichment):** - CONTEXT.md files from the 3 most recent completed phases (locked decisions — maintain consistency) @@ -897,7 +910,11 @@ ${CONTEXT_WINDOW >= 500000 ? ` - Skip all other prior phases to stay within context budget ` : ''} - +${API_SURFACE_PATH ? ` + +**API Surface (HINT — may be incomplete):** When \`intel.enabled\` is true, \`.planning/intel/API-SURFACE.md\` lists symbols extracted from the codebase by regex/JS analysis. Prefer symbols listed there when referencing existing code. This surface is regex/JS-derived and MAY BE INCOMPLETE — a symbol's absence means *unknown*, not *nonexistent*. Never treat the surface as exhaustive. If you reference a symbol that is not in the surface and this phase creates it, list it under "Artifacts this phase produces". + +` : ''} ${AGENT_SKILLS_PLANNER} **Phase requirement IDs (every ID MUST appear in a plan's `requirements` field):** {phase_req_ids} @@ -932,6 +949,7 @@ Output consumed by /gsd:execute-phase. Plans need: - Tasks in XML format with read_first and acceptance_criteria fields (MANDATORY on every task) - Verification criteria - must_haves for goal-backward verification +- **"Artifacts this phase produces" section (MANDATORY)** — list every symbol this phase creates: decorators, classes, functions, CLI flags, struct/dataclass fields, new file paths. The plan-review-convergence source-grounding pass reads this section to exclude newly-created symbols from drift verification; omitting it causes new symbols to be flagged for acknowledgement. @@ -975,6 +993,7 @@ Every task MUST include these fields — they are NOT optional: - [ ] Dependencies correctly identified - [ ] Waves assigned for parallel execution - [ ] must_haves derived from phase goal +- [ ] Every PLAN.md includes an "Artifacts this phase produces" section listing symbols created by this phase (decorators, classes, functions, CLI flags, struct/dataclass fields, new file paths) ``` diff --git a/get-shit-done/workflows/plan-review-convergence.md b/get-shit-done/workflows/plan-review-convergence.md index 517b774cd..9e1653558 100644 --- a/get-shit-done/workflows/plan-review-convergence.md +++ b/get-shit-done/workflows/plan-review-convergence.md @@ -176,6 +176,22 @@ Your final response MUST also include this section immediately after the CYCLE_S ) ``` +### Source-grounding pass (config: `plan_review.source_grounding`, default on) + +Run this pass unless `plan_review.source_grounding` is `false`. It verifies every symbol the plan cites against the project source before approval, catching hallucinated symbols at review time instead of execution time. + +1. **Enumerate cited symbols.** List every referenced symbol by kind, quoting the plan line for each (coverage must be auditable): decorators (`@name`), classes/methods (`Class.method`), functions (`module.function`), CLI flags (`--name`), file paths, dataclass/struct fields. +2. **Exclude new artifacts.** Do NOT verify symbols the plan declares under its "Artifacts this phase produces" section — those are created by this phase, not references to existing code. +3. **Resolve each remaining symbol** using the adapter named by `plan_review.source_grounding_authority` (default `grep`): + - `grep` — ripgrep / Read the source; confirm the name appears as a real declaration. + - `intel` — consult `.planning/intel/API-SURFACE.md` / `api-map.json` (only when `intel.enabled`). + Record one verdict per symbol: **VERIFIED** (quote `file:line`), **MISSING** (adapter can check this language/kind and the symbol is absent), **AMBIGUOUS** (multiple candidates), or **UNCHECKABLE** (adapter cannot analyze this language/kind — e.g. non-JS under `intel`, or any signature under `grep`). Never treat UNCHECKABLE as verified or missing. +4. **Severity & gating:** + - **MISSING** at authority `grep`/`intel` → `needs-acknowledgement`: the plan proceeds only if the author confirms the symbol is genuinely new or dynamically resolved, and that acknowledgement is recorded. A hard block is reserved for higher-authority adapters (LSP/SCIP) that can prove absence. + - **AMBIGUOUS** → MEDIUM. **UNCHECKABLE** → INFO. + - Signature mismatches cannot be asserted under `grep`/`intel`; report the signature as UNCHECKABLE. +5. **Coverage block.** Append a "Verification coverage" section to `REVIEWS.md` listing every UNCHECKABLE/skipped symbol and why — a clean review must never silently mean "nothing was checked." + After agent returns, verify REVIEWS.md exists: ```bash REVIEWS_FILE=$(ls ${phase_dir}/${padded_phase}-REVIEWS.md 2>/dev/null) diff --git a/get-shit-done/workflows/settings.md b/get-shit-done/workflows/settings.md index 3512b0586..7f367b0fd 100644 --- a/get-shit-done/workflows/settings.md +++ b/get-shit-done/workflows/settings.md @@ -40,6 +40,7 @@ Parse current values (default to `true` if not present): - `workflow.research` — spawn researcher during plan-phase - `workflow.plan_check` — spawn plan checker during plan-phase - `workflow.verifier` — spawn verifier during execute-phase +- `plan_review.source_grounding` — verify plan symbols against live source during plan review (default: true if absent; set `plan_review.source_grounding_authority` to select the resolver adapter: `grep` (default), `intel`, `treesitter`, `lsp`, or `scip`) - `workflow.nyquist_validation` — validation architecture research during plan-phase (default: true if absent) - `workflow.pattern_mapper` — run gsd-pattern-mapper between research and planning (default: true if absent) - `workflow.ui_phase` — generate UI-SPEC.md design contracts for frontend phases (default: true if absent) @@ -83,7 +84,7 @@ Use AskUserQuestion with current values pre-selected. Questions are grouped into Section layout: ### Planning -Research, Plan Checker, Pattern Mapper, Nyquist, UI Phase, UI Gate, AI Phase +Research, Plan Checker, Drift Guard, Pattern Mapper, Nyquist, UI Phase, UI Gate, AI Phase ### Execution Verifier, TDD Mode, Code Review, Code Review Depth _(conditional — only when code_review=on)_, UI Review @@ -176,6 +177,15 @@ AskUserQuestion([ { label: "No", description: "Skip post-execution verification" } ] }, + { + question: "Enable Plan Drift Guard? (verifies that symbols cited in plans exist in source at review time)", + header: "Drift Guard", + multiSelect: false, + options: [ + { label: "Yes (Recommended)", description: "Resolve symbol references (decorators, classes, functions, CLI flags) against live source — catches hallucinated names before execution. Authority controlled by plan_review.source_grounding_authority (default: grep)." }, + { label: "No", description: "Skip symbol grounding. Plan review proceeds without source verification." } + ] + }, { question: "Enable TDD Mode? (RED/GREEN/REFACTOR gates for eligible tasks)", header: "TDD", @@ -395,6 +405,9 @@ Merge new settings into existing config.json: "skip_discuss": true/false, "use_worktrees": true/false }, + "plan_review": { + "source_grounding": true/false + }, "intel": { "enabled": true/false }, @@ -468,6 +481,9 @@ Write `~/.gsd/defaults.json` with: "ui_review": , "skip_discuss": }, + "plan_review": { + "source_grounding": + }, "intel": { "enabled": }, @@ -496,6 +512,7 @@ Display: | Execution Verifier | {On/Off} | | TDD Mode | {On/Off} | | Code Review | {On/Off} | +| Plan Drift Guard | {On/Off} | | Code Review Depth | {quick/standard/deep} | | UI Review | {On/Off} | | Commit Docs | {On/Off} | @@ -528,7 +545,7 @@ Quick commands: - [ ] Current config read -- [ ] User presented with 23 settings (profile + workflow toggles + features + git branching + git tagging + ctx warnings), grouped into six sections: Planning, Execution, Docs & Output, Features, Model & Pipeline, Misc. `code_review_depth` is conditional on `code_review=on`. Model profile uses a two-question split (Q1: Adaptive / Standard tier / Inherit; Q2: Quality / Balanced / Budget — only when Standard tier chosen) to stay within the 4-option AskUserQuestion cap while exposing all 5 valid profiles (#3784). +- [ ] User presented with 24 settings (profile + workflow toggles + features + git branching + git tagging + ctx warnings), grouped into six sections: Planning, Execution, Docs & Output, Features, Model & Pipeline, Misc. `code_review_depth` is conditional on `code_review=on`. Model profile uses a two-question split (Q1: Adaptive / Standard tier / Inherit; Q2: Quality / Balanced / Budget — only when Standard tier chosen) to stay within the 4-option AskUserQuestion cap while exposing all 5 valid profiles (#3784). Drift Guard (`plan_review.source_grounding`) is in the Planning section. - [ ] Config updated with model_profile, workflow, and git sections - [ ] User offered to save as global defaults (~/.gsd/defaults.json) - [ ] Changes confirmed to user diff --git a/tests/config.test.cjs b/tests/config.test.cjs index 44f7ab65c..7fef7b286 100644 --- a/tests/config.test.cjs +++ b/tests/config.test.cjs @@ -992,3 +992,108 @@ describe('config-path command (#2282)', () => { assert.strictEqual(configContent.model_profile, 'quality', 'config-path should point to the file config-set wrote'); }); }); + +// ─── plan_review.source_grounding + _authority (#22) ───────────────────────── + +describe('plan_review.source_grounding and plan_review.source_grounding_authority (#22)', () => { + let tmpDir; + + beforeEach(() => { + tmpDir = createTempProject(); + runGsdTools('config-ensure-section', tmpDir, { HOME: tmpDir, USERPROFILE: tmpDir }); + }); + + afterEach(() => { + cleanup(tmpDir); + }); + + // (a) Default of plan_review.source_grounding is true + test('plan_review.source_grounding defaults to true when not set in config.json', () => { + const config = readConfig(tmpDir); + assert.strictEqual( + config.plan_review?.source_grounding, + true, + 'plan_review.source_grounding must default to true' + ); + }); + + // (b) Default of plan_review.source_grounding_authority is "grep" + test('plan_review.source_grounding_authority defaults to "grep" when not set in config.json', () => { + const config = readConfig(tmpDir); + assert.strictEqual( + config.plan_review?.source_grounding_authority, + 'grep', + 'plan_review.source_grounding_authority must default to "grep"' + ); + }); + + // (c) Both keys are recognized as valid config keys + test('plan_review.source_grounding is a valid config key accepted by config-set', () => { + const result = runGsdTools('config-set plan_review.source_grounding false', tmpDir); + assert.ok(result.success, `config-set plan_review.source_grounding failed: ${result.error}`); + + const config = readConfig(tmpDir); + assert.strictEqual(config.plan_review.source_grounding, false); + }); + + test('plan_review.source_grounding_authority is a valid config key accepted by config-set', () => { + const result = runGsdTools('config-set plan_review.source_grounding_authority intel', tmpDir); + assert.ok(result.success, `config-set plan_review.source_grounding_authority failed: ${result.error}`); + + const config = readConfig(tmpDir); + assert.strictEqual(config.plan_review.source_grounding_authority, 'intel'); + }); + + // Enum positive: all valid authority values + test('plan_review.source_grounding_authority accepts all valid enum values', () => { + const validValues = ['grep', 'intel', 'treesitter', 'lsp', 'scip']; + for (const v of validValues) { + const result = runGsdTools(`config-set plan_review.source_grounding_authority ${v}`, tmpDir); + assert.ok(result.success, `config-set plan_review.source_grounding_authority ${v} failed: ${result.error}`); + const config = readConfig(tmpDir); + assert.strictEqual(config.plan_review.source_grounding_authority, v); + } + }); + + // (d) NEGATIVE MATRIX — invalid enum values are rejected + test('plan_review.source_grounding_authority rejects invalid value "bogus"', () => { + const result = runGsdTools('config-set plan_review.source_grounding_authority bogus', tmpDir); + assert.strictEqual(result.success, false, 'bogus should be rejected'); + assert.ok( + result.error.includes('Invalid plan_review.source_grounding_authority'), + `Expected "Invalid plan_review.source_grounding_authority" in error: ${result.error}` + ); + }); + + test('plan_review.source_grounding_authority rejects flag-looking value "--grep"', () => { + const result = runGsdTools(['config-set', 'plan_review.source_grounding_authority', '--grep'], tmpDir); + assert.strictEqual(result.success, false, '--grep should be rejected'); + assert.ok( + result.error.includes('Invalid plan_review.source_grounding_authority'), + `Expected "Invalid plan_review.source_grounding_authority" in error: ${result.error}` + ); + }); + + test('plan_review.source_grounding_authority rejects empty string', () => { + const result = runGsdTools(['config-set', 'plan_review.source_grounding_authority', ''], tmpDir); + assert.strictEqual(result.success, false, 'empty string should be rejected'); + }); + + test('plan_review.source_grounding rejects non-boolean value "yes"', () => { + const result = runGsdTools('config-set plan_review.source_grounding yes', tmpDir); + assert.strictEqual(result.success, false, '"yes" should be rejected as non-boolean'); + assert.ok( + result.error.includes('Invalid plan_review.source_grounding'), + `Expected "Invalid plan_review.source_grounding" in error: ${result.error}` + ); + }); + + test('plan_review.source_grounding rejects numeric value 1', () => { + const result = runGsdTools('config-set plan_review.source_grounding 1', tmpDir); + assert.strictEqual(result.success, false, 'numeric 1 should be rejected as non-boolean'); + assert.ok( + result.error.includes('Invalid plan_review.source_grounding'), + `Expected "Invalid plan_review.source_grounding" in error: ${result.error}` + ); + }); +}); diff --git a/tests/docs-parity-live-registry.test.cjs b/tests/docs-parity-live-registry.test.cjs index f0c218369..b0adeedb9 100644 --- a/tests/docs-parity-live-registry.test.cjs +++ b/tests/docs-parity-live-registry.test.cjs @@ -159,6 +159,13 @@ const INTERNAL_COMPONENT_SLUGS = new Set([ // The regex captures "/gsd-test-runner" from the URL path component. This is // an external tool repo, not a user-typable slash command in this product. 'test-runner', + + // gsd-core — GitHub repository name: "open-gsd/gsd-core". + // docs/adr/22-plan-drift-guard.md references it as an issue tracker link: + // open-gsd/gsd-core#22 + // The regex captures "/gsd-core" from the org/repo path separator. This is + // the canonical repo name, not a user-typable slash command in this product. + 'core', ]); /** diff --git a/tests/feat-22-surfacing-docs.test.cjs b/tests/feat-22-surfacing-docs.test.cjs new file mode 100644 index 000000000..a947e386d --- /dev/null +++ b/tests/feat-22-surfacing-docs.test.cjs @@ -0,0 +1,198 @@ +// allow-test-rule: docs-parity +// Verifies that issue #22 drift-guard surfacing changes are present: +// - new-project workflow mentions plan_review.source_grounding +// - CONFIGURATION.md documents both new config keys +// - COMMANDS.md mentions gsd-tools intel api-surface + +const { describe, test } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('fs'); +const path = require('path'); + +const ROOT = path.join(__dirname, '..'); + +const NEW_PROJECT_PATH = path.join(ROOT, 'get-shit-done', 'workflows', 'new-project.md'); +const SETTINGS_PATH = path.join(ROOT, 'get-shit-done', 'workflows', 'settings.md'); +const CONFIGURATION_PATH = path.join(ROOT, 'docs', 'CONFIGURATION.md'); +const COMMANDS_PATH = path.join(ROOT, 'docs', 'COMMANDS.md'); +const USER_GUIDE_PATH = path.join(ROOT, 'docs', 'USER-GUIDE.md'); +const ARCHITECTURE_PATH = path.join(ROOT, 'docs', 'ARCHITECTURE.md'); + +describe('feat-22-surfacing-docs', () => { + // ── A1: new-project workflow ───────────────────────────────────────────── + + test('new-project workflow mentions source_grounding', () => { + const content = fs.readFileSync(NEW_PROJECT_PATH, 'utf-8'); + assert.ok( + content.includes('source_grounding'), + 'new-project.md must mention source_grounding' + ); + }); + + test('new-project workflow has Drift Guard question', () => { + const content = fs.readFileSync(NEW_PROJECT_PATH, 'utf-8'); + assert.ok( + content.includes('Drift Guard'), + 'new-project.md must include a "Drift Guard" question header' + ); + }); + + test('new-project workflow wires source_grounding into config-new-project call', () => { + const content = fs.readFileSync(NEW_PROJECT_PATH, 'utf-8'); + assert.ok( + content.includes('"plan_review":{"source_grounding":'), + 'new-project.md config-new-project call must include plan_review.source_grounding' + ); + }); + + test('new-project workflow has Drift Guard default-yes option', () => { + const content = fs.readFileSync(NEW_PROJECT_PATH, 'utf-8'); + // Both question blocks (auto and interactive) should have the yes option + const count = (content.match(/Yes \(Recommended\).*catches hallucinated names/g) || []).length; + assert.ok( + count >= 1, + 'new-project.md must have at least one Drift Guard "Yes (Recommended)" option' + ); + }); + + // ── A2: settings workflow ──────────────────────────────────────────────── + + test('settings workflow mentions source_grounding in read_current step', () => { + const content = fs.readFileSync(SETTINGS_PATH, 'utf-8'); + assert.ok( + content.includes('plan_review.source_grounding'), + 'settings.md must mention plan_review.source_grounding in the read_current step' + ); + }); + + test('settings workflow has Drift Guard AskUserQuestion toggle', () => { + const content = fs.readFileSync(SETTINGS_PATH, 'utf-8'); + assert.ok( + content.includes('Drift Guard'), + 'settings.md must include a "Drift Guard" question header' + ); + }); + + test('settings workflow update_config includes plan_review.source_grounding', () => { + const content = fs.readFileSync(SETTINGS_PATH, 'utf-8'); + assert.ok( + content.includes('"source_grounding": true/false'), + 'settings.md update_config block must include source_grounding: true/false' + ); + }); + + test('settings workflow mentions source_grounding_authority', () => { + const content = fs.readFileSync(SETTINGS_PATH, 'utf-8'); + assert.ok( + content.includes('source_grounding_authority'), + 'settings.md must mention source_grounding_authority' + ); + }); + + test('settings confirm table includes Plan Drift Guard row', () => { + const content = fs.readFileSync(SETTINGS_PATH, 'utf-8'); + assert.ok( + content.includes('Plan Drift Guard'), + 'settings.md confirm table must include a "Plan Drift Guard" row' + ); + }); + + // ── B1: CONFIGURATION.md ───────────────────────────────────────────────── + + test('CONFIGURATION.md documents plan_review.source_grounding', () => { + const content = fs.readFileSync(CONFIGURATION_PATH, 'utf-8'); + assert.ok( + content.includes('`plan_review.source_grounding`'), + 'CONFIGURATION.md must document plan_review.source_grounding' + ); + }); + + test('CONFIGURATION.md documents plan_review.source_grounding_authority', () => { + const content = fs.readFileSync(CONFIGURATION_PATH, 'utf-8'); + assert.ok( + content.includes('`plan_review.source_grounding_authority`'), + 'CONFIGURATION.md must document plan_review.source_grounding_authority' + ); + }); + + test('CONFIGURATION.md documents grep as default authority', () => { + const content = fs.readFileSync(CONFIGURATION_PATH, 'utf-8'); + assert.ok( + content.includes('`grep`') && content.includes('source_grounding_authority'), + 'CONFIGURATION.md must document grep as the default source_grounding_authority' + ); + }); + + test('CONFIGURATION.md lists all five authority enum values', () => { + const content = fs.readFileSync(CONFIGURATION_PATH, 'utf-8'); + const authorities = ['grep', 'intel', 'treesitter', 'lsp', 'scip']; + const missing = authorities.filter(a => !content.includes(a)); + assert.deepStrictEqual( + missing, + [], + `CONFIGURATION.md must list all authority values; missing: ${missing.join(', ')}` + ); + }); + + // ── B2: COMMANDS.md ─────────────────────────────────────────────────────── + + test('COMMANDS.md mentions intel api-surface', () => { + const content = fs.readFileSync(COMMANDS_PATH, 'utf-8'); + assert.ok( + content.includes('intel api-surface'), + 'COMMANDS.md must document the gsd-tools intel api-surface command' + ); + }); + + test('COMMANDS.md documents api-surface gating on intel.enabled', () => { + const content = fs.readFileSync(COMMANDS_PATH, 'utf-8'); + assert.ok( + content.includes('intel.enabled'), + 'COMMANDS.md intel api-surface section must mention the intel.enabled gate' + ); + }); + + test('COMMANDS.md mentions API-SURFACE.md output', () => { + const content = fs.readFileSync(COMMANDS_PATH, 'utf-8'); + assert.ok( + content.includes('API-SURFACE.md'), + 'COMMANDS.md must mention the API-SURFACE.md output file' + ); + }); + + // ── B3: USER-GUIDE.md ───────────────────────────────────────────────────── + + test('USER-GUIDE.md has Plan Drift Guard subsection', () => { + const content = fs.readFileSync(USER_GUIDE_PATH, 'utf-8'); + assert.ok( + content.includes('### Plan Drift Guard'), + 'USER-GUIDE.md must have a "### Plan Drift Guard" subsection' + ); + }); + + test('USER-GUIDE.md mentions needs-acknowledgement behavior', () => { + const content = fs.readFileSync(USER_GUIDE_PATH, 'utf-8'); + assert.ok( + content.includes('needs-acknowledgement'), + 'USER-GUIDE.md drift guard section must describe needs-acknowledgement behavior' + ); + }); + + test('USER-GUIDE.md explains drift guard works without intel', () => { + const content = fs.readFileSync(USER_GUIDE_PATH, 'utf-8'); + assert.ok( + content.includes('without intel') || content.includes('Works without intel'), + 'USER-GUIDE.md must explain that the drift guard works without intel' + ); + }); + + // ── B4: ARCHITECTURE.md ─────────────────────────────────────────────────── + + test('ARCHITECTURE.md links to ADR 22', () => { + const content = fs.readFileSync(ARCHITECTURE_PATH, 'utf-8'); + assert.ok( + content.includes('adr/22-plan-drift-guard.md') || content.includes('ADR 22'), + 'ARCHITECTURE.md must link to ADR 22 (adr/22-plan-drift-guard.md)' + ); + }); +}); diff --git a/tests/intel.test.cjs b/tests/intel.test.cjs index 10e55c3eb..2aced5a69 100644 --- a/tests/intel.test.cjs +++ b/tests/intel.test.cjs @@ -21,6 +21,7 @@ const { intelSnapshot, intelPatchMeta, intelExtractExports, + intelApiSurface, ensureIntelDir, isIntelEnabled, INTEL_FILES, @@ -703,4 +704,145 @@ describe('gsd-tools intel subcommands', () => { assert.strictEqual(output.valid, false); assert.ok(output.errors.length > 0); }); + + test('unknown intel subcommand error lists api-surface', () => { + const result = runGsdTools(['intel', 'nonexistent-subcmd'], tmpDir); + assert.strictEqual(result.success, false); + const errorText = result.error || ''; + assert.ok(errorText.includes('api-surface'), 'error message must list api-surface'); + }); + + test('flag-looking intel subcommand treated as unknown, not crash', () => { + const result = runGsdTools(['intel', '--api-surface'], tmpDir); + assert.strictEqual(result.success, false); + const errorText = result.error || ''; + assert.ok(errorText.includes('Unknown intel subcommand'), 'must emit typed unknown-subcommand error'); + }); + + test('intel api-surface returns disabled message when not enabled', () => { + const result = runGsdTools(['intel', 'api-surface'], tmpDir); + assert.strictEqual(result.success, true); + const output = JSON.parse(result.output); + assert.strictEqual(output.disabled, true); + }); + + test('intel api-surface writes API-SURFACE.md when enabled with populated api-map.json', () => { + const planningDir = path.join(tmpDir, '.planning'); + enableIntel(planningDir); + writeIntelJson(planningDir, 'api-map.json', { + _meta: { updated_at: new Date().toISOString() }, + entries: { + 'intelQuery': { method: 'function', handler: 'intelQuery', role: 'query intel files' }, + 'intelStatus': { method: 'function', handler: 'intelStatus', role: 'report freshness' }, + }, + }); + const result = runGsdTools(['intel', 'api-surface'], tmpDir); + assert.strictEqual(result.success, true); + const output = JSON.parse(result.output); + assert.ok(output.written, 'result must include written path'); + assert.strictEqual(output.symbolCount, 2); + const mdContent = fs.readFileSync(output.written, 'utf8'); + assert.ok(mdContent.includes('intelQuery'), 'API-SURFACE.md must list intelQuery symbol'); + assert.ok(mdContent.includes('intelStatus'), 'API-SURFACE.md must list intelStatus symbol'); + }); +}); + +// ─── intelApiSurface ──────────────────────────────────────────────────────── + +describe('intelApiSurface', () => { + let tmpDir; + let planningDir; + + beforeEach(() => { + tmpDir = createTempProject(); + planningDir = path.join(tmpDir, '.planning'); + }); + + afterEach(() => { + cleanup(tmpDir); + }); + + test('returns disabled response when intel is off', () => { + const result = intelApiSurface(planningDir); + assert.strictEqual(result.disabled, true); + assert.ok(result.message.includes('disabled')); + }); + + test('writes API-SURFACE.md with symbol entries from api-map.json', () => { + enableIntel(planningDir); + writeIntelJson(planningDir, 'api-map.json', { + _meta: { updated_at: new Date().toISOString() }, + entries: { + 'authenticate': { method: 'POST', handler: 'authController', role: 'user login' }, + 'createUser': { method: 'POST', handler: 'userController', role: 'user registration' }, + }, + }); + + const result = intelApiSurface(planningDir); + assert.strictEqual(result.symbolCount, 2); + assert.ok(result.written.endsWith('API-SURFACE.md')); + + const content = fs.readFileSync(result.written, 'utf8'); + assert.ok(content.includes('authenticate'), 'must include symbol name authenticate'); + assert.ok(content.includes('createUser'), 'must include symbol name createUser'); + assert.ok(content.includes('authController'), 'must include field value authController'); + }); + + test('writes API-SURFACE.md with incomplete banner when api-map.json is absent', () => { + enableIntel(planningDir); + // No api-map.json written + + const result = intelApiSurface(planningDir); + assert.strictEqual(result.symbolCount, 0); + assert.ok(result.written.endsWith('API-SURFACE.md')); + + const content = fs.readFileSync(result.written, 'utf8'); + assert.ok(content.includes('Incomplete'), 'must contain Incomplete banner when no entries'); + assert.ok(content.includes('unknown'), 'must say treat absence as "unknown"'); + }); + + test('writes API-SURFACE.md with incomplete banner when entries is empty object', () => { + enableIntel(planningDir); + writeIntelJson(planningDir, 'api-map.json', { + _meta: { updated_at: new Date().toISOString() }, + entries: {}, + }); + + const result = intelApiSurface(planningDir); + assert.strictEqual(result.symbolCount, 0); + + const content = fs.readFileSync(result.written, 'utf8'); + assert.ok(content.includes('Incomplete'), 'empty entries must still emit incomplete banner'); + }); + + test('returns stale=false for fresh api-map.json', () => { + enableIntel(planningDir); + writeIntelJson(planningDir, 'api-map.json', { + _meta: { updated_at: new Date().toISOString() }, + entries: { 'myFunc': { method: 'function' } }, + }); + + const result = intelApiSurface(planningDir); + assert.strictEqual(result.stale, false); + }); + + test('returns stale=true for old api-map.json', () => { + enableIntel(planningDir); + const oldDate = new Date(Date.now() - 25 * 60 * 60 * 1000).toISOString(); + writeIntelJson(planningDir, 'api-map.json', { + _meta: { updated_at: oldDate }, + entries: { 'myFunc': { method: 'function' } }, + }); + + const result = intelApiSurface(planningDir); + assert.strictEqual(result.stale, true); + }); + + test('return shape has written, symbolCount, stale fields', () => { + enableIntel(planningDir); + const result = intelApiSurface(planningDir); + assert.ok('written' in result, 'result must have written field'); + assert.ok('symbolCount' in result, 'result must have symbolCount field'); + assert.ok('stale' in result, 'result must have stale field'); + }); }); diff --git a/tests/plan-phase-drift-guard.test.cjs b/tests/plan-phase-drift-guard.test.cjs new file mode 100644 index 000000000..7d7ecfee9 --- /dev/null +++ b/tests/plan-phase-drift-guard.test.cjs @@ -0,0 +1,137 @@ +/** + * Drift guard for gsd:plan-phase workflow (#22) + * + * Validates that the plan-phase workflow contains the key structural elements + * added for issue #22 Change #1: + * + * (A) intel.enabled gate — when intel.enabled is true, plan-phase regenerates + * API-SURFACE.md via `gsd-tools intel api-surface` and injects it into the + * planner's required reading as a HINT (prefer symbols, may be incomplete, + * absence = unknown, never exhaustive). + * + * (B) "Artifacts this phase produces" section — every PLAN.md must include + * this section so the plan-review-convergence source-grounding pass can + * exclude newly-created symbols from drift verification. + */ + +// allow-test-rule: source-text-is-the-product +// The workflow markdown IS the runtime instruction. Testing its text content +// tests the deployed contract — if the intel gate or Artifacts section +// requirement is absent, the drift-guard feature is absent from defenses too. + +const { test, describe } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('fs'); +const path = require('path'); + +const WORKFLOW_PATH = path.join( + __dirname, + '..', + 'get-shit-done', + 'workflows', + 'plan-phase.md' +); + +// ─── Fixture ────────────────────────────────────────────────────────────────── + +const workflow = fs.readFileSync(WORKFLOW_PATH, 'utf8'); + +// ─── (A) intel.enabled gate ─────────────────────────────────────────────────── + +describe('plan-phase workflow: intel.enabled gate for API-SURFACE injection (#22)', () => { + test('workflow reads intel.enabled config before planner spawn', () => { + assert.ok( + workflow.includes('intel.enabled'), + 'workflow must gate API-SURFACE generation on intel.enabled config key' + ); + }); + + test('workflow runs gsd-tools intel api-surface to regenerate surface', () => { + assert.ok( + workflow.includes('intel api-surface'), + 'workflow must call `gsd_run intel api-surface` (or equivalent) to regenerate API-SURFACE.md' + ); + }); + + test('workflow injects API-SURFACE.md into planner files_to_read when intel.enabled', () => { + assert.ok( + workflow.includes('API-SURFACE.md') && workflow.includes('API_SURFACE_PATH'), + 'workflow must pass API_SURFACE_PATH into the planner prompt files_to_read block' + ); + }); + + test('workflow labels the surface as a HINT (not a hard rule)', () => { + assert.ok( + workflow.includes('HINT') || workflow.includes('intel_surface_hint'), + 'API-SURFACE.md must be annotated as a HINT, never a hard rule' + ); + }); + + test('workflow documents that surface absence means unknown not nonexistent', () => { + assert.ok( + workflow.includes("absence means *unknown*, not *nonexistent*") || + workflow.includes("absence = unknown") || + workflow.includes("absence means unknown"), + "workflow must state that a symbol's absence from the surface means unknown, not nonexistent" + ); + }); + + test('workflow states the surface may be incomplete', () => { + assert.ok( + workflow.includes('MAY BE INCOMPLETE') || workflow.includes('may be incomplete'), + 'workflow must warn that the API surface may be incomplete' + ); + }); + + test('workflow skips surface injection when intel.enabled is false', () => { + // The gate must have an explicit false/skip branch + assert.ok( + workflow.includes("INTEL_CFG") && + (workflow.includes("'false'") || workflow.includes('"false"') || workflow.includes('false')), + 'workflow must skip the intel step when intel.enabled is false (config defaults to false)' + ); + }); +}); + +// ─── (B) "Artifacts this phase produces" requirement ───────────────────────── + +describe('plan-phase workflow: Artifacts this phase produces section (#22)', () => { + test('downstream_consumer block requires Artifacts this phase produces section', () => { + assert.ok( + workflow.includes('Artifacts this phase produces'), + 'downstream_consumer must list "Artifacts this phase produces" as a required plan section' + ); + }); + + test('quality_gate checklist includes Artifacts this phase produces item', () => { + // Find the quality_gate block and confirm the checklist item is there + const qualityGateMatch = workflow.match(/([\s\S]*?)<\/quality_gate>/); + assert.ok( + qualityGateMatch, + 'workflow must have a block' + ); + assert.ok( + qualityGateMatch[1].includes('Artifacts this phase produces'), + ' checklist must include an "Artifacts this phase produces" item' + ); + }); + + test('workflow explains why Artifacts section is needed (source-grounding reviewer)', () => { + assert.ok( + workflow.includes('source-grounding') || workflow.includes('plan-review-convergence'), + 'workflow must explain that the Artifacts section is consumed by the source-grounding pass' + ); + }); + + test('workflow lists symbol kinds for Artifacts section (decorators, classes, functions, CLI flags)', () => { + // Must enumerate concrete symbol kinds so planner knows what to list + const hasDecorators = workflow.includes('decorators'); + const hasClasses = workflow.includes('classes'); + const hasFunctions = workflow.includes('functions'); + const hasCliFlags = workflow.includes('CLI flags'); + assert.ok( + hasDecorators && hasClasses && hasFunctions && hasCliFlags, + 'workflow must enumerate symbol kinds: decorators, classes, functions, CLI flags (needed for Artifacts section guidance)' + ); + }); +}); diff --git a/tests/plan-review-convergence.test.cjs b/tests/plan-review-convergence.test.cjs index 1c25c31c1..f623219d1 100644 --- a/tests/plan-review-convergence.test.cjs +++ b/tests/plan-review-convergence.test.cjs @@ -566,6 +566,47 @@ describe('plan-review-convergence local model config schema registration (#2306- }); }); +// ─── Workflow: source-grounding pass (#22) ─────────────────────────────────── + +describe('plan-review-convergence workflow: source-grounding reviewer pass (#22)', () => { + const workflow = fs.readFileSync(WORKFLOW_PATH, 'utf8'); + + test('workflow documents plan_review.source_grounding config key (default on)', () => { + assert.ok( + workflow.includes('plan_review.source_grounding'), + 'workflow must document the plan_review.source_grounding config key that gates the source-grounding pass (#22)' + ); + }); + + test('workflow defines all four symbol verdicts: VERIFIED, MISSING, AMBIGUOUS, UNCHECKABLE', () => { + assert.ok(workflow.includes('VERIFIED'), 'workflow must define VERIFIED verdict'); + assert.ok(workflow.includes('MISSING'), 'workflow must define MISSING verdict'); + assert.ok(workflow.includes('AMBIGUOUS'), 'workflow must define AMBIGUOUS verdict'); + assert.ok(workflow.includes('UNCHECKABLE'), 'workflow must define UNCHECKABLE verdict'); + }); + + test('workflow specifies needs-acknowledgement gating for MISSING symbols', () => { + assert.ok( + workflow.includes('needs-acknowledgement'), + 'workflow must specify needs-acknowledgement (not hard block) for MISSING at grep/intel authority (#22)' + ); + }); + + test('workflow instructs reviewer to exclude symbols declared under "Artifacts this phase produces"', () => { + assert.ok( + workflow.includes('Artifacts this phase produces'), + 'workflow must exclude new artifacts declared by the plan from symbol verification (#22)' + ); + }); + + test('workflow requires "Verification coverage" section appended to REVIEWS.md', () => { + assert.ok( + workflow.includes('Verification coverage'), + 'workflow must require a Verification coverage section in REVIEWS.md listing every UNCHECKABLE/skipped symbol (#22)' + ); + }); +}); + describe('plan-review-convergence local model CONFIGURATION.md documentation (#2306-local)', () => { const configDoc = fs.readFileSync(CONFIG_DOC_PATH, 'utf8');