chore: remove project-specific planning files
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Lex Christopherson
@@ -1,63 +0,0 @@
|
||||
# GSD Context Optimization
|
||||
|
||||
## What This Is
|
||||
|
||||
Optimizing GSD's context loading so agents start lean and stay in their peak quality zone. Reduces prompt bloat through lazy loading, tiered prompts, and compiled artifacts while preserving instructional density.
|
||||
|
||||
## Core Value
|
||||
|
||||
Agents execute at peak quality by starting at 8-12% context instead of 15-25%.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Validated
|
||||
|
||||
(None yet — ship to validate)
|
||||
|
||||
### Active
|
||||
|
||||
- [ ] History loads as structured digest instead of full SUMMARY.md files
|
||||
- [ ] Summary templates match plan complexity (minimal/standard/complex)
|
||||
- [ ] State operations are atomic patches, not full file read/write cycles
|
||||
- [ ] Executor loads references on-demand based on task type
|
||||
- [ ] Planner loads extensions based on planning mode
|
||||
- [ ] Plans can be pre-compiled before execution
|
||||
|
||||
### Out of Scope
|
||||
|
||||
- Enhanced semantic queries (Phase 4) — defer unless Phases 1-3 prove insufficient
|
||||
- Compressing instructional content — core methodology must remain intact
|
||||
- Complex cache invalidation — simple mtime checks are sufficient
|
||||
|
||||
## Context
|
||||
|
||||
**Problem:** Claude's quality degrades predictably with context load:
|
||||
- 0-30% context: Peak quality
|
||||
- 30-50%: Good, occasional shortcuts
|
||||
- 50-70%: Degrading, efficiency mode
|
||||
- 70%+: Poor, rushed, misses requirements
|
||||
|
||||
Complex phases currently start agents at 15-25% context from prompt loading alone—before any codebase reading.
|
||||
|
||||
**Prior work:** v1.12.x shipped compound init commands (4,245 net line reduction), proving the direction works.
|
||||
|
||||
**Target state:** Agents start at 8-12% context, preserving peak quality zone for execution.
|
||||
|
||||
## Constraints
|
||||
|
||||
- **Instructional density**: Teaching content cannot be compressed—it's GSD's value
|
||||
- **Adaptive intelligence**: Agents must still handle mid-execution surprises
|
||||
- **Maintainability**: Clear separation between core and extensions
|
||||
- **Debuggability**: Easy to trace which modules loaded for any execution
|
||||
- **TDD**: Use test-driven development for gsd-tools.js changes — write tests first, then implementation
|
||||
|
||||
## Key Decisions
|
||||
|
||||
| Decision | Rationale | Outcome |
|
||||
|----------|-----------|---------|
|
||||
| Lazy loading over railroad architecture | Preserves adaptive intelligence while reducing context | — Pending |
|
||||
| Build on compound-init foundation | v1.12.x proved the pattern works | — Pending |
|
||||
| Three-phase implementation | Incremental, reversible changes | — Pending |
|
||||
|
||||
---
|
||||
*Last updated: 2025-02-07 after initialization*
|
||||
@@ -1,114 +0,0 @@
|
||||
# Requirements: GSD Context Optimization
|
||||
|
||||
**Defined:** 2025-02-07
|
||||
**Core Value:** Agents execute at peak quality by starting at 8-12% context instead of 15-25%
|
||||
|
||||
## v1 Requirements
|
||||
|
||||
### History Digest
|
||||
|
||||
- [ ] **HIST-01**: `gsd-tools history-digest` generates structured JSON from SUMMARY.md frontmatter
|
||||
- [ ] **HIST-02**: Digest includes phases, provides, patterns, affects, decisions, and tech_stack
|
||||
- [ ] **HIST-03**: Planner uses digest instead of reading full SUMMARY.md files
|
||||
- [ ] **HIST-04**: Tests verify digest output matches expected schema
|
||||
|
||||
### Summary Variants
|
||||
|
||||
- [ ] **SUMM-01**: Three summary templates exist: minimal (~30 lines), standard (~60 lines), complex (~100 lines)
|
||||
- [ ] **SUMM-02**: `gsd-tools select-template` returns appropriate template based on plan complexity
|
||||
- [ ] **SUMM-03**: Selection logic considers task count, decision tasks, and file count
|
||||
- [ ] **SUMM-04**: Executor uses selected template for summary creation
|
||||
- [ ] **SUMM-05**: Tests verify template selection logic
|
||||
|
||||
### Atomic State Operations
|
||||
|
||||
- [ ] **STATE-01**: `gsd-tools state get <field>` returns specific STATE.md field value
|
||||
- [ ] **STATE-02**: `gsd-tools state patch --field value` updates specific fields atomically
|
||||
- [ ] **STATE-03**: Agents use atomic ops instead of full STATE.md read/write
|
||||
- [ ] **STATE-04**: Tests verify get/patch operations
|
||||
|
||||
### Lazy-Load Executor
|
||||
|
||||
- [ ] **EXEC-01**: `gsd-executor-core.md` contains base executor (~150 lines)
|
||||
- [ ] **EXEC-02**: `references/executor/` contains modular references (deviation-rules, tdd, checkpoint, continuation, summary-creation)
|
||||
- [ ] **EXEC-03**: Executor loads TDD reference only when task.tdd="true"
|
||||
- [ ] **EXEC-04**: Executor loads checkpoint reference only when task.type="checkpoint:*"
|
||||
- [ ] **EXEC-05**: Executor loads continuation reference only when <completed_tasks> present
|
||||
- [ ] **EXEC-06**: Executor loads summary-creation reference at plan completion
|
||||
|
||||
### Tiered Planner
|
||||
|
||||
- [ ] **PLAN-01**: `gsd-planner-core.md` contains base planner (~300 lines)
|
||||
- [ ] **PLAN-02**: `gsd-planner-ext/` contains extensions (gap-closure, revision, tdd, checkpoints)
|
||||
- [ ] **PLAN-03**: Orchestrator builds prompt dynamically based on flags and context
|
||||
- [ ] **PLAN-04**: Gap-closure extension loads only with --gaps flag
|
||||
- [ ] **PLAN-05**: Revision extension loads only when checker issues exist
|
||||
- [ ] **PLAN-06**: TDD extension loads only when TDD candidates detected
|
||||
|
||||
### Compiled Plans
|
||||
|
||||
- [ ] **COMP-01**: `gsd-tools compile-plan` inlines all @ references
|
||||
- [ ] **COMP-02**: Compilation strips irrelevant sections based on task types
|
||||
- [ ] **COMP-03**: Compiled plans saved as .compiled.md
|
||||
- [ ] **COMP-04**: Executor uses compiled version when available
|
||||
- [ ] **COMP-05**: Staleness detection via mtime comparison
|
||||
- [ ] **COMP-06**: Tests verify compilation output
|
||||
|
||||
## v2 Requirements
|
||||
|
||||
### Enhanced Semantic Queries
|
||||
|
||||
- **SEM-01**: gsd-memory MCP supports structured queries (what_uses, pattern_for, decisions_affecting)
|
||||
|
||||
## Out of Scope
|
||||
|
||||
| Feature | Reason |
|
||||
|---------|--------|
|
||||
| Compressing instructional content | Core methodology must remain intact |
|
||||
| Complex cache invalidation | Simple mtime checks are sufficient |
|
||||
| Railroad architecture | Loses adaptive intelligence |
|
||||
|
||||
## Traceability
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| HIST-01 | Phase 1 | Pending |
|
||||
| HIST-02 | Phase 1 | Pending |
|
||||
| HIST-03 | Phase 1 | Pending |
|
||||
| HIST-04 | Phase 1 | Pending |
|
||||
| SUMM-01 | Phase 2 | Pending |
|
||||
| SUMM-02 | Phase 2 | Pending |
|
||||
| SUMM-03 | Phase 2 | Pending |
|
||||
| SUMM-04 | Phase 2 | Pending |
|
||||
| SUMM-05 | Phase 2 | Pending |
|
||||
| STATE-01 | Phase 3 | Pending |
|
||||
| STATE-02 | Phase 3 | Pending |
|
||||
| STATE-03 | Phase 3 | Pending |
|
||||
| STATE-04 | Phase 3 | Pending |
|
||||
| EXEC-01 | Phase 4 | Pending |
|
||||
| EXEC-02 | Phase 4 | Pending |
|
||||
| EXEC-03 | Phase 4 | Pending |
|
||||
| EXEC-04 | Phase 4 | Pending |
|
||||
| EXEC-05 | Phase 4 | Pending |
|
||||
| EXEC-06 | Phase 4 | Pending |
|
||||
| PLAN-01 | Phase 5 | Pending |
|
||||
| PLAN-02 | Phase 5 | Pending |
|
||||
| PLAN-03 | Phase 5 | Pending |
|
||||
| PLAN-04 | Phase 5 | Pending |
|
||||
| PLAN-05 | Phase 5 | Pending |
|
||||
| PLAN-06 | Phase 5 | Pending |
|
||||
| COMP-01 | Phase 6 | Pending |
|
||||
| COMP-02 | Phase 6 | Pending |
|
||||
| COMP-03 | Phase 6 | Pending |
|
||||
| COMP-04 | Phase 6 | Pending |
|
||||
| COMP-05 | Phase 6 | Pending |
|
||||
| COMP-06 | Phase 6 | Pending |
|
||||
|
||||
**Coverage:**
|
||||
- v1 requirements: 28 total
|
||||
- Mapped to phases: 28
|
||||
- Unmapped: 0 ✓
|
||||
|
||||
---
|
||||
*Requirements defined: 2025-02-07*
|
||||
*Last updated: 2025-02-07 after initial definition*
|
||||
@@ -1,183 +0,0 @@
|
||||
# Roadmap: GSD Context Optimization
|
||||
|
||||
**Created:** 2025-02-07
|
||||
**Phases:** 6
|
||||
**Depth:** Standard
|
||||
**Core Value:** Agents execute at peak quality by starting at 8-12% context instead of 15-25%
|
||||
|
||||
## Overview
|
||||
|
||||
| # | Phase | Goal | Requirements | Status |
|
||||
|---|-------|------|--------------|--------|
|
||||
| 1 | History Digest | Planner loads structured digest instead of full SUMMARY.md | HIST-01, HIST-02, HIST-03, HIST-04 | Pending |
|
||||
| 2 | Summary Variants | Executor produces right-sized summaries | SUMM-01, SUMM-02, SUMM-03, SUMM-04, SUMM-05 | Pending |
|
||||
| 3 | Atomic State | Agents update STATE.md fields atomically | STATE-01, STATE-02, STATE-03, STATE-04 | Pending |
|
||||
| 4 | Lazy Executor | Executor loads references on-demand | EXEC-01, EXEC-02, EXEC-03, EXEC-04, EXEC-05, EXEC-06 | Pending |
|
||||
| 5 | Tiered Planner | Planner prompt built dynamically | PLAN-01, PLAN-02, PLAN-03, PLAN-04, PLAN-05, PLAN-06 | Pending |
|
||||
| 6 | Compiled Plans | Plans pre-compiled for minimal execution context | COMP-01, COMP-02, COMP-03, COMP-04, COMP-05, COMP-06 | Pending |
|
||||
|
||||
## Dependencies
|
||||
|
||||
```
|
||||
Phase 1 ─┐
|
||||
Phase 2 ─┼─► Phase 4 ─► Phase 5 ─► Phase 6
|
||||
Phase 3 ─┘
|
||||
```
|
||||
|
||||
- Phases 1, 2, 3 can run in parallel (no dependencies)
|
||||
- Phase 4 depends on Phase 2 (summary templates) and Phase 3 (state ops)
|
||||
- Phase 5 depends on Phase 4 (executor patterns inform planner structure)
|
||||
- Phase 6 depends on Phase 4 and 5 (compilation targets both agents)
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: History Digest
|
||||
|
||||
**Goal:** Planner loads structured digest instead of full SUMMARY.md files
|
||||
|
||||
**Requirements:** HIST-01, HIST-02, HIST-03, HIST-04
|
||||
|
||||
### Success Criteria
|
||||
|
||||
1. `gsd-tools history-digest` produces valid JSON with frontmatter fields
|
||||
2. Digest includes: phases, provides, patterns, affects, decisions, tech_stack
|
||||
3. Planner startup no longer reads full SUMMARY.md files
|
||||
4. Tests pass for digest schema validation
|
||||
|
||||
### Approach
|
||||
|
||||
- Add `history-digest` command to gsd-tools.js
|
||||
- Parse SUMMARY.md frontmatter (YAML between `---` markers)
|
||||
- Extract key fields into structured JSON
|
||||
- Update planner to consume digest instead of full files
|
||||
- TDD: Write tests first for digest output schema
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Summary Variants
|
||||
|
||||
**Goal:** Executor produces right-sized summaries based on plan complexity
|
||||
|
||||
**Requirements:** SUMM-01, SUMM-02, SUMM-03, SUMM-04, SUMM-05
|
||||
|
||||
### Success Criteria
|
||||
|
||||
1. Three template files exist: summary-minimal.md (~30 lines), summary-standard.md (~60 lines), summary-complex.md (~100 lines)
|
||||
2. `gsd-tools select-template` returns appropriate template path
|
||||
3. Selection considers: task count, decision tasks, file count
|
||||
4. Executor uses selected template for summary creation
|
||||
|
||||
### Approach
|
||||
|
||||
- Create three summary templates with increasing detail
|
||||
- Add `select-template` command with selection logic
|
||||
- Update executor workflow to call select-template
|
||||
- TDD: Write tests first for template selection logic
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Atomic State Operations
|
||||
|
||||
**Goal:** Agents update STATE.md fields atomically without full file read/write
|
||||
|
||||
**Requirements:** STATE-01, STATE-02, STATE-03, STATE-04
|
||||
|
||||
### Success Criteria
|
||||
|
||||
1. `gsd-tools state get <field>` returns specific STATE.md field value
|
||||
2. `gsd-tools state patch --field value` updates specific fields atomically
|
||||
3. Agent workflows updated to use atomic ops
|
||||
4. Tests verify get/patch operations preserve file integrity
|
||||
|
||||
### Approach
|
||||
|
||||
- Add `state get` subcommand to read specific fields
|
||||
- Add `state patch` subcommand for atomic field updates
|
||||
- Parse STATE.md as markdown, update in place
|
||||
- Update executor/planner workflows to use atomic ops
|
||||
- TDD: Write tests first for get/patch operations
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Lazy-Load Executor
|
||||
|
||||
**Goal:** Executor loads references on-demand based on task type
|
||||
|
||||
**Requirements:** EXEC-01, EXEC-02, EXEC-03, EXEC-04, EXEC-05, EXEC-06
|
||||
|
||||
### Success Criteria
|
||||
|
||||
1. `gsd-executor-core.md` contains base executor (~150 lines)
|
||||
2. `references/executor/` contains: deviation-rules.md, tdd-execution.md, checkpoint-protocol.md, continuation.md, summary-creation.md
|
||||
3. TDD reference loads only when task.tdd="true"
|
||||
4. Checkpoint reference loads only for task.type="checkpoint:*"
|
||||
5. Continuation reference loads only when <completed_tasks> present
|
||||
|
||||
### Approach
|
||||
|
||||
- Extract current executor into core + modular references
|
||||
- Core contains: role, task execution loop, basic flow
|
||||
- References contain: specialized protocols for specific situations
|
||||
- Add conditional @ references based on task attributes
|
||||
- Measure context reduction vs current executor
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Tiered Planner
|
||||
|
||||
**Goal:** Planner prompt built dynamically based on planning mode
|
||||
|
||||
**Requirements:** PLAN-01, PLAN-02, PLAN-03, PLAN-04, PLAN-05, PLAN-06
|
||||
|
||||
### Success Criteria
|
||||
|
||||
1. `gsd-planner-core.md` contains base planner (~300 lines)
|
||||
2. `gsd-planner-ext/` contains: gap-closure.md, revision.md, tdd.md, checkpoints.md
|
||||
3. Gap-closure loads only with --gaps flag
|
||||
4. Revision loads only when checker issues exist
|
||||
5. TDD loads only when TDD candidates detected
|
||||
|
||||
### Approach
|
||||
|
||||
- Extract current planner into core + extensions
|
||||
- Core contains: role, philosophy, task breakdown, plan format
|
||||
- Extensions contain: specialized planning modes
|
||||
- Update orchestrator to build prompt dynamically
|
||||
- Measure context reduction vs current planner
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Compiled Plans
|
||||
|
||||
**Goal:** Plans pre-compiled for minimal execution context
|
||||
|
||||
**Requirements:** COMP-01, COMP-02, COMP-03, COMP-04, COMP-05, COMP-06
|
||||
|
||||
### Success Criteria
|
||||
|
||||
1. `gsd-tools compile-plan` inlines all @ references
|
||||
2. Compilation strips irrelevant sections based on task types in plan
|
||||
3. Compiled plans saved as .compiled.md alongside original
|
||||
4. Executor uses compiled version when fresh (mtime check)
|
||||
5. Tests verify compilation output matches expected structure
|
||||
|
||||
### Approach
|
||||
|
||||
- Add `compile-plan` command to gsd-tools.js
|
||||
- Resolve all @ references recursively
|
||||
- Strip sections not relevant to this plan's tasks
|
||||
- Save as PLAN.compiled.md
|
||||
- Update executor to prefer compiled version
|
||||
- TDD: Write tests first for compilation logic
|
||||
|
||||
---
|
||||
|
||||
## Coverage
|
||||
|
||||
**v1 Requirements:** 28 total
|
||||
**Mapped to phases:** 28
|
||||
**Unmapped:** 0 ✓
|
||||
|
||||
---
|
||||
*Roadmap created: 2025-02-07*
|
||||
*Last updated: 2025-02-07 after initial creation*
|
||||
@@ -1,45 +0,0 @@
|
||||
# Project State: GSD Context Optimization
|
||||
|
||||
## Current Position
|
||||
|
||||
**Phase:** 1 - History Digest
|
||||
**Status:** Not Started
|
||||
**Plan:** None
|
||||
|
||||
## Project Reference
|
||||
|
||||
See: .planning/PROJECT.md (updated 2025-02-07)
|
||||
|
||||
**Core value:** Agents execute at peak quality by starting at 8-12% context instead of 15-25%
|
||||
**Current focus:** Phase 1 - History Digest
|
||||
|
||||
## Phase Progress
|
||||
|
||||
| Phase | Status | Plans |
|
||||
|-------|--------|-------|
|
||||
| 1 - History Digest | ○ Pending | 0/0 |
|
||||
| 2 - Summary Variants | ○ Pending | 0/0 |
|
||||
| 3 - Atomic State | ○ Pending | 0/0 |
|
||||
| 4 - Lazy Executor | ○ Pending | 0/0 |
|
||||
| 5 - Tiered Planner | ○ Pending | 0/0 |
|
||||
| 6 - Compiled Plans | ○ Pending | 0/0 |
|
||||
|
||||
## Recent Activity
|
||||
|
||||
- 2025-02-07: Project initialized
|
||||
- 2025-02-07: Requirements defined (28 total)
|
||||
- 2025-02-07: Roadmap created (6 phases)
|
||||
|
||||
## Blockers
|
||||
|
||||
None
|
||||
|
||||
## Key Decisions
|
||||
|
||||
| Decision | Phase | Rationale |
|
||||
|----------|-------|-----------|
|
||||
| TDD for gsd-tools changes | All | Write tests first to ensure behavior matches expectations |
|
||||
| Lazy loading over railroad | All | Preserves adaptive intelligence while reducing context |
|
||||
|
||||
---
|
||||
*Last updated: 2025-02-07 after initialization*
|
||||
@@ -1,12 +0,0 @@
|
||||
{
|
||||
"mode": "yolo",
|
||||
"depth": "standard",
|
||||
"parallelization": true,
|
||||
"commit_docs": true,
|
||||
"model_profile": "quality",
|
||||
"workflow": {
|
||||
"research": false,
|
||||
"plan_check": true,
|
||||
"verifier": true
|
||||
}
|
||||
}
|
||||
@@ -1,245 +0,0 @@
|
||||
# GSD Context Optimization: Action Plan
|
||||
|
||||
## Consensus Across All Syntheses
|
||||
|
||||
All three proposals agree on the first two phases. Divergence begins at Phase 3 (how far to push deterministic orchestration). This plan executes the consensus first, then evaluates before committing to architectural changes.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Quick Wins (Do First)
|
||||
**Why first:** Highest ROI, lowest risk, validates the approach before bigger changes.
|
||||
|
||||
### 1.1 History Digest
|
||||
**What:** Add `gsd-tools history-digest` command that extracts frontmatter from SUMMARY.md files into a structured JSON.
|
||||
|
||||
**Why:** 10 summaries = 500-1000 lines → 2KB JSON. 80-90% reduction in history context.
|
||||
|
||||
**Implementation:**
|
||||
```javascript
|
||||
// bin/gsd-tools.js
|
||||
case 'history-digest':
|
||||
const summaries = glob.sync('.planning/phases/*/*-SUMMARY.md');
|
||||
const digest = { phases: {}, decisions: [], tech_stack: new Set() };
|
||||
|
||||
for (const file of summaries) {
|
||||
const frontmatter = extractFrontmatter(file);
|
||||
// Extract: provides, patterns, affects, decisions, tech-stack
|
||||
}
|
||||
|
||||
console.log(JSON.stringify(digest, null, 2));
|
||||
```
|
||||
|
||||
**Acceptance:** Planner uses digest instead of reading full summaries.
|
||||
|
||||
---
|
||||
|
||||
### 1.2 Atomic State Operations
|
||||
**What:** Add `gsd-tools state get <section>` and `gsd-tools state patch --key value`.
|
||||
|
||||
**Why:** Agents currently hold full STATE.md (50-100 lines) throughout execution. Patch operations read/write only what's needed.
|
||||
|
||||
**Implementation:**
|
||||
```javascript
|
||||
case 'state':
|
||||
if (args[1] === 'get') {
|
||||
// Return specific section as JSON
|
||||
}
|
||||
if (args[1] === 'patch') {
|
||||
// Parse flags, apply atomic updates
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance:** Executor uses `state patch` instead of rewriting full file.
|
||||
|
||||
---
|
||||
|
||||
### 1.3 Summary Template Variants
|
||||
**What:** Create three templates: `summary-minimal.md` (~30 lines), `summary-standard.md` (~60 lines), `summary-complex.md` (~100 lines).
|
||||
|
||||
**Why:** A 2-task config change doesn't need a 114-line summary template.
|
||||
|
||||
**Selection logic:**
|
||||
- Minimal: ≤2 tasks, ≤3 files, no decisions
|
||||
- Complex: Has decisions OR >6 files
|
||||
- Standard: Everything else
|
||||
|
||||
**Acceptance:** Executor selects template based on plan metadata.
|
||||
|
||||
---
|
||||
|
||||
### 1.4 Finish Compound Init Sweep
|
||||
**What:** Ensure all major workflows use compound init calls.
|
||||
|
||||
**Why:** Already proven in v1.12.x. Consolidates 5-10 atomic calls into single payload.
|
||||
|
||||
**Targets:** `new-project.md`, `complete-milestone.md`, `verify-work.md`, `transition.md`, `help.md`, `discuss-phase.md`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Tiered Agent Architecture (Do Second)
|
||||
**Why second:** Addresses the biggest context hogs (planner: 55KB, executor: 19KB) but requires more careful refactoring.
|
||||
|
||||
### 2.1 Split Executor
|
||||
**What:** Extract protocols into separate reference files, load conditionally.
|
||||
|
||||
**Before:**
|
||||
```
|
||||
agents/gsd-executor.md (382 lines, ~19KB)
|
||||
- deviation_rules (60 lines) - always loaded
|
||||
- checkpoint_protocol (40 lines) - always loaded
|
||||
- tdd_execution (30 lines) - always loaded
|
||||
- continuation_handling (20 lines) - always loaded
|
||||
```
|
||||
|
||||
**After:**
|
||||
```
|
||||
agents/gsd-executor-core.md (~150 lines)
|
||||
|
||||
references/executor/
|
||||
deviation-rules.md # Always loaded (core identity)
|
||||
tdd-execution.md # Loaded if task.tdd="true"
|
||||
checkpoint-protocol.md # Loaded if task.type="checkpoint:*"
|
||||
continuation.md # Loaded if <completed_tasks> present
|
||||
```
|
||||
|
||||
**Loading logic:** Executor uses `@` references conditionally based on task metadata.
|
||||
|
||||
**Acceptance:** Executor base context drops from ~19KB to ~8KB.
|
||||
|
||||
---
|
||||
|
||||
### 2.2 Split Planner
|
||||
**What:** Extract mode-specific logic into extensions.
|
||||
|
||||
**Before:**
|
||||
```
|
||||
agents/gsd-planner.md (1,116 lines, ~55KB)
|
||||
```
|
||||
|
||||
**After:**
|
||||
```
|
||||
agents/gsd-planner-core.md (~300 lines)
|
||||
- role, philosophy, context_fidelity
|
||||
- task_breakdown (core)
|
||||
- plan_format
|
||||
- execution_flow (standard path)
|
||||
|
||||
agents/gsd-planner-ext/
|
||||
gap-closure.md # Loaded if --gaps flag
|
||||
revision.md # Loaded if checker issues
|
||||
tdd.md # Loaded if TDD candidates detected
|
||||
checkpoints.md # Loaded if phase has checkpoints
|
||||
```
|
||||
|
||||
**Orchestrator assembles prompt dynamically based on flags and phase characteristics.**
|
||||
|
||||
**Acceptance:** Planner base context drops from ~55KB to ~20KB.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Compiled Plans (Do Third)
|
||||
**Why third:** Depends on stable tiered agents. Provides incremental gains on top of Phase 2.
|
||||
|
||||
### 3.1 Implement compile-plan
|
||||
**What:** Add `gsd-tools compile-plan <path>` that:
|
||||
1. Inlines all `@` references
|
||||
2. Strips sections unused by this plan (no TDD tasks → remove TDD sections)
|
||||
3. Outputs `.compiled.md` ready for execution
|
||||
|
||||
**Why:** Eliminates runtime `@` resolution. Pre-strips unused protocols.
|
||||
|
||||
**Staleness handling:**
|
||||
- Check mtime before execution
|
||||
- `.compiled.md` in .gitignore
|
||||
- Recompile on GSD update
|
||||
|
||||
**Acceptance:** Plans execute from compiled form, ~30-40% context reduction.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Evaluate Railroad (Decide Later)
|
||||
**Why later:** High effort, architectural risk. Phases 1-3 may provide sufficient improvement.
|
||||
|
||||
**The question:** Should `gsd-tools` become a state machine that tells Claude "do this next" instead of Claude reading workflow files?
|
||||
|
||||
**Arguments for:**
|
||||
- Maximum context reduction
|
||||
- Deterministic execution (can't skip steps)
|
||||
- LLM focuses purely on task, not process
|
||||
|
||||
**Arguments against:**
|
||||
- Loses adaptive intelligence
|
||||
- Can't handle mid-execution surprises
|
||||
- Debugging split across code and prompts
|
||||
- Near-total rewrite
|
||||
|
||||
**Decision point:** After Phase 3, measure context usage. If still >40% at task start, consider Railroad. If <40%, skip.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Semantic Queries (Optional)
|
||||
**Why optional:** gsd-memory MCP exists. Only invest here if grep-based lookups prove insufficient after other optimizations.
|
||||
|
||||
**What:** Extend gsd-memory with structured queries:
|
||||
```javascript
|
||||
gsd_memory_what_uses({ symbol: "User", type: "model" })
|
||||
gsd_memory_pattern_for({ domain: "auth" })
|
||||
```
|
||||
|
||||
**Decision point:** After Phase 3, evaluate if history/pattern lookups are still a bottleneck.
|
||||
|
||||
---
|
||||
|
||||
## Execution Order Summary
|
||||
|
||||
```
|
||||
IMMEDIATE (Phase 1) - 1-2 days
|
||||
├── 1.1 history-digest command
|
||||
├── 1.2 state get/patch commands
|
||||
├── 1.3 Summary template variants
|
||||
└── 1.4 Compound init sweep completion
|
||||
|
||||
NEXT (Phase 2) - 3-5 days
|
||||
├── 2.1 Split gsd-executor into core + references
|
||||
└── 2.2 Split gsd-planner into core + extensions
|
||||
|
||||
THEN (Phase 3) - 3-5 days
|
||||
└── 3.1 compile-plan command
|
||||
|
||||
EVALUATE (Phase 4) - Decision point
|
||||
└── Measure: Is context <40%?
|
||||
└── Yes → Done
|
||||
└── No → Consider Railroad architecture
|
||||
|
||||
OPTIONAL (Phase 5)
|
||||
└── Semantic MCP queries (only if needed)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
| Metric | Current | Phase 1 | Phase 2 | Phase 3 | Target |
|
||||
|--------|---------|---------|---------|---------|--------|
|
||||
| Executor base | ~19KB | ~19KB | ~8KB | ~6KB | <8KB |
|
||||
| Planner base | ~55KB | ~55KB | ~20KB | ~18KB | <20KB |
|
||||
| History (10 phases) | ~20KB | ~2KB | ~2KB | ~2KB | <3KB |
|
||||
| Task start context | ~25% | ~18% | ~12% | ~10% | <15% |
|
||||
|
||||
---
|
||||
|
||||
## What NOT To Do
|
||||
|
||||
1. **Don't implement Railroad before Phase 3 measurement.** The architectural cost is high; validate simpler solutions first.
|
||||
|
||||
2. **Don't compress instructional content.** The teaching content is GSD's value. Optimize *when* it loads, not *what* it says.
|
||||
|
||||
3. **Don't over-engineer caching.** Simple mtime checks work. No cache invalidation logic until proven necessary.
|
||||
|
||||
4. **Don't parallelize phases.** Each phase validates assumptions for the next. Sequential execution reduces risk.
|
||||
|
||||
---
|
||||
|
||||
## Next Action
|
||||
|
||||
Start Phase 1.1: Implement `gsd-tools history-digest`.
|
||||
@@ -1,580 +0,0 @@
|
||||
# GSD Context Optimization Strategy
|
||||
|
||||
> Reducing context load while maintaining (or improving) execution quality.
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### Context Budget Reality
|
||||
|
||||
| File Type | Total Size | Files | Loaded When |
|
||||
|-----------|-----------|-------|-------------|
|
||||
| Agents | ~197KB | 11 | Per subagent spawn |
|
||||
| Workflows | ~266KB | 30 | Per command invocation |
|
||||
| References | ~88KB | ~10 | Inlined in agents/workflows |
|
||||
| Templates | ~114KB | ~8 | Per document creation |
|
||||
|
||||
**Heaviest subagents:**
|
||||
- `gsd-planner`: 1,116 lines (~55KB)
|
||||
- `gsd-executor`: 382 lines (~19KB)
|
||||
- `gsd-phase-researcher`: ~400 lines (~20KB)
|
||||
|
||||
**Problem:** A simple 2-task plan execution loads the full 382-line executor prompt, including TDD flows, gap closure protocols, and checkpoint handling it won't use.
|
||||
|
||||
### The Quality Degradation Curve
|
||||
|
||||
| Context Usage | Quality | Behavior |
|
||||
|---------------|---------|----------|
|
||||
| 0-30% | PEAK | Thorough, comprehensive, follows all instructions |
|
||||
| 30-50% | GOOD | Solid work, occasional shortcuts |
|
||||
| 50-70% | DEGRADING | Efficiency mode, skips optional steps |
|
||||
| 70%+ | POOR | Rushed, minimal, misses requirements |
|
||||
|
||||
**Current reality:** Complex phases start agents at 15-25% context just from prompt loading, before any codebase reading. This leaves less headroom for actual work.
|
||||
|
||||
---
|
||||
|
||||
## Optimization Principles
|
||||
|
||||
### 1. Demand Loading Over Eager Loading
|
||||
|
||||
Load what IS needed when it's needed, not what MIGHT be needed upfront.
|
||||
|
||||
**Compiler analogy:**
|
||||
- Dead code elimination → Don't load TDD protocol if no TDD tasks
|
||||
- Lazy evaluation → Load checkpoint protocol when checkpoint encountered
|
||||
- Incremental compilation → Compile plans once, reuse compiled form
|
||||
|
||||
### 2. Separation of Concerns
|
||||
|
||||
Base prompts define WHAT to do. Extension modules define HOW for specific scenarios.
|
||||
|
||||
### 3. Semantic Over Syntactic
|
||||
|
||||
Query for meaning ("what patterns exist for auth?") instead of text ("grep for auth in all summaries").
|
||||
|
||||
---
|
||||
|
||||
## Optimization Opportunities
|
||||
|
||||
### O1: Lazy-Load Reference Sections
|
||||
|
||||
**Priority:** 1 (High impact, medium effort)
|
||||
|
||||
**Current state:** Agents inline all protocols regardless of need.
|
||||
|
||||
```markdown
|
||||
# gsd-executor.md currently has:
|
||||
<deviation_rules> # 60 lines - always loaded
|
||||
<checkpoint_protocol> # 40 lines - always loaded
|
||||
<tdd_execution> # 30 lines - always loaded
|
||||
<continuation_handling> # 20 lines - always loaded
|
||||
```
|
||||
|
||||
**Proposed state:** Base agent with conditional loading.
|
||||
|
||||
```markdown
|
||||
# gsd-executor-core.md (~150 lines)
|
||||
<role>...</role>
|
||||
<execution_flow>
|
||||
<step name="execute_tasks">
|
||||
For each task:
|
||||
1. If `tdd="true"`: @~/.claude/get-shit-done/references/tdd-execution.md
|
||||
2. If `type="checkpoint:*"`: @~/.claude/get-shit-done/references/checkpoint-protocol.md
|
||||
3. Execute task...
|
||||
</step>
|
||||
</execution_flow>
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```
|
||||
agents/
|
||||
gsd-executor.md → gsd-executor-core.md (base, ~150 lines)
|
||||
|
||||
references/
|
||||
executor/
|
||||
deviation-rules.md # Loaded always (core to executor identity)
|
||||
tdd-execution.md # Loaded if task.tdd="true"
|
||||
checkpoint-protocol.md # Loaded if task.type starts with "checkpoint:"
|
||||
continuation.md # Loaded if <completed_tasks> in prompt
|
||||
summary-creation.md # Loaded at plan completion
|
||||
```
|
||||
|
||||
**Savings:** 40-50% reduction in executor base context. TDD, checkpoint, continuation references load only when task type requires them.
|
||||
|
||||
**Risk:** @ resolution adds file I/O. Mitigated by keeping references small (<50 lines each).
|
||||
|
||||
---
|
||||
|
||||
### O2: Tiered Agent Prompts
|
||||
|
||||
**Priority:** 2 (High impact, medium effort)
|
||||
|
||||
**Current state:** Planner has 1,116 lines covering all modes.
|
||||
|
||||
```markdown
|
||||
# gsd-planner.md sections:
|
||||
<context_fidelity> # 55 lines
|
||||
<philosophy> # 40 lines
|
||||
<discovery_levels> # 50 lines
|
||||
<task_breakdown> # 120 lines
|
||||
<dependency_graph> # 80 lines
|
||||
<scope_estimation> # 60 lines
|
||||
<plan_format> # 110 lines
|
||||
<goal_backward> # 100 lines
|
||||
<checkpoints> # 90 lines
|
||||
<tdd_integration> # 50 lines
|
||||
<gap_closure_mode> # 80 lines
|
||||
<revision_mode> # 100 lines
|
||||
<execution_flow> # 200 lines
|
||||
<structured_returns> # 60 lines
|
||||
```
|
||||
|
||||
**Proposed state:** Base + extensions.
|
||||
|
||||
```
|
||||
agents/
|
||||
gsd-planner-core.md # ~300 lines (always loaded)
|
||||
- role, philosophy, context_fidelity
|
||||
- task_breakdown (core)
|
||||
- plan_format
|
||||
- execution_flow (standard path)
|
||||
- structured_returns
|
||||
|
||||
gsd-planner-ext/
|
||||
discovery.md # ~50 lines (if new dependencies detected)
|
||||
goal-backward.md # ~100 lines (always for now, core methodology)
|
||||
gap-closure.md # ~80 lines (if --gaps flag)
|
||||
revision.md # ~100 lines (if checker issues provided)
|
||||
tdd.md # ~50 lines (if TDD candidates detected)
|
||||
checkpoints.md # ~90 lines (if checkpoints in phase)
|
||||
```
|
||||
|
||||
**Orchestrator logic:**
|
||||
|
||||
```bash
|
||||
PLANNER_PROMPT="@~/.claude/agents/gsd-planner-core.md"
|
||||
|
||||
if [[ "$FLAGS" == *"--gaps"* ]]; then
|
||||
PLANNER_PROMPT+="\n@~/.claude/agents/gsd-planner-ext/gap-closure.md"
|
||||
fi
|
||||
|
||||
if [[ -n "$CHECKER_ISSUES" ]]; then
|
||||
PLANNER_PROMPT+="\n@~/.claude/agents/gsd-planner-ext/revision.md"
|
||||
fi
|
||||
|
||||
# Detect TDD candidates from phase description
|
||||
if echo "$PHASE_DESC" | grep -qiE "business logic|validation|algorithm|transform"; then
|
||||
PLANNER_PROMPT+="\n@~/.claude/agents/gsd-planner-ext/tdd.md"
|
||||
fi
|
||||
```
|
||||
|
||||
**Savings:** ~60% reduction in typical planning context. Gap closure and revision modes loaded only when triggered.
|
||||
|
||||
---
|
||||
|
||||
### O3: Frontmatter-Only History Digest
|
||||
|
||||
**Priority:** 3 (Medium impact, low effort)
|
||||
|
||||
**Current state:** Planner reads full SUMMARY.md files to understand project history.
|
||||
|
||||
```bash
|
||||
# gsd-planner step: read_project_history
|
||||
for f in .planning/phases/*/*-SUMMARY.md; do
|
||||
cat "$f" # Full file, 50-100 lines each
|
||||
done
|
||||
```
|
||||
|
||||
**Problem:** 10 prior summaries = 500-1000 lines of context. Most is prose, deviation docs, commit lists — not needed for dependency analysis.
|
||||
|
||||
**Proposed state:** gsd-tools generates digest.
|
||||
|
||||
```bash
|
||||
node ~/.claude/get-shit-done/bin/gsd-tools.js history-digest
|
||||
```
|
||||
|
||||
**Output:**
|
||||
|
||||
```json
|
||||
{
|
||||
"phases": {
|
||||
"01-setup": {
|
||||
"plans": ["01-01", "01-02"],
|
||||
"provides": ["Next.js app", "Prisma schema", "Auth utils"],
|
||||
"patterns": ["App Router", "Server Actions", "jose for JWT"],
|
||||
"affects": ["api", "components", "lib"]
|
||||
},
|
||||
"02-core": {
|
||||
"plans": ["02-01", "02-02", "02-03"],
|
||||
"provides": ["User CRUD", "Project CRUD", "Dashboard"],
|
||||
"patterns": ["Zod validation", "React Query"],
|
||||
"affects": ["api", "components", "hooks"]
|
||||
}
|
||||
},
|
||||
"decisions": [
|
||||
{"phase": "01-01", "decision": "Use jose over jsonwebtoken for Edge compatibility"},
|
||||
{"phase": "02-01", "decision": "Server components for data fetching, client for interactivity"}
|
||||
],
|
||||
"tech_stack": ["next.js", "prisma", "tailwind", "jose", "zod", "react-query"]
|
||||
}
|
||||
```
|
||||
|
||||
**Planner loads:**
|
||||
- 2KB digest instead of 20KB of full summaries
|
||||
- Full summary only for directly-dependent phases (from `affects` field)
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```javascript
|
||||
// bin/gsd-tools.js
|
||||
case 'history-digest':
|
||||
const summaries = glob.sync('.planning/phases/*/*-SUMMARY.md');
|
||||
const digest = { phases: {}, decisions: [], tech_stack: new Set() };
|
||||
|
||||
for (const file of summaries) {
|
||||
const frontmatter = extractFrontmatter(file);
|
||||
const phase = frontmatter.phase;
|
||||
|
||||
digest.phases[phase] = digest.phases[phase] || { plans: [], provides: [], patterns: [], affects: [] };
|
||||
digest.phases[phase].plans.push(frontmatter.plan);
|
||||
digest.phases[phase].provides.push(...(frontmatter['dependency-graph']?.provides || []));
|
||||
digest.phases[phase].patterns.push(...(frontmatter['patterns-established'] || []));
|
||||
digest.phases[phase].affects.push(...(frontmatter['dependency-graph']?.affects || []));
|
||||
|
||||
if (frontmatter['key-decisions']) {
|
||||
digest.decisions.push(...frontmatter['key-decisions'].map(d => ({ phase, decision: d })));
|
||||
}
|
||||
|
||||
(frontmatter['tech-stack']?.added || []).forEach(t => digest.tech_stack.add(t.name));
|
||||
}
|
||||
|
||||
digest.tech_stack = [...digest.tech_stack];
|
||||
console.log(JSON.stringify(digest, null, 2));
|
||||
break;
|
||||
```
|
||||
|
||||
**Savings:** 80-90% reduction in history context for complex projects.
|
||||
|
||||
---
|
||||
|
||||
### O4: Compiled Plans
|
||||
|
||||
**Priority:** 4 (Medium impact, medium effort)
|
||||
|
||||
**Current state:** Every plan has `@` references resolved at runtime.
|
||||
|
||||
```markdown
|
||||
# 03-01-PLAN.md
|
||||
<execution_context>
|
||||
@~/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@~/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/STATE.md
|
||||
</context>
|
||||
```
|
||||
|
||||
**Problem:** Every executor agent re-resolves the same references. execute-plan.md alone is 354 lines.
|
||||
|
||||
**Proposed state:** Pre-compile plans before execution.
|
||||
|
||||
```bash
|
||||
node ~/.claude/get-shit-done/bin/gsd-tools.js compile-plan .planning/phases/03-auth/03-01-PLAN.md
|
||||
# Produces .planning/phases/03-auth/03-01-PLAN.compiled.md
|
||||
```
|
||||
|
||||
**Compilation steps:**
|
||||
|
||||
1. Inline all `@` references
|
||||
2. Strip sections irrelevant to this plan's tasks:
|
||||
- No TDD tasks? Remove TDD sections
|
||||
- No checkpoints? Remove checkpoint sections
|
||||
- No external services? Remove user_setup handling
|
||||
3. Minify prose (optional): Remove example sections, reduce explanatory text
|
||||
4. Output ready-to-execute prompt
|
||||
|
||||
**Executor spawning:**
|
||||
|
||||
```bash
|
||||
# Before: Agent resolves references
|
||||
PLAN_CONTENT=$(cat "$PLAN_PATH")
|
||||
|
||||
# After: Pre-compiled, ready to execute
|
||||
if [[ -f "${PLAN_PATH%.md}.compiled.md" ]]; then
|
||||
PLAN_CONTENT=$(cat "${PLAN_PATH%.md}.compiled.md")
|
||||
else
|
||||
node ~/.claude/get-shit-done/bin/gsd-tools.js compile-plan "$PLAN_PATH"
|
||||
PLAN_CONTENT=$(cat "${PLAN_PATH%.md}.compiled.md")
|
||||
fi
|
||||
```
|
||||
|
||||
**Savings:** Eliminates file I/O and @-resolution overhead per agent. Pre-strips unused protocols. Estimate: 30-40% reduction in executor context for simple plans.
|
||||
|
||||
**Trade-off:** Compiled plans become stale if source references change. Mitigated by:
|
||||
- Compile on-demand (check mtime)
|
||||
- `.compiled.md` files in .gitignore
|
||||
- Recompile on GSD update
|
||||
|
||||
---
|
||||
|
||||
### O5: Atomic State Operations
|
||||
|
||||
**Priority:** 5 (Medium impact, low effort)
|
||||
|
||||
**Current state:** Agents read full STATE.md, hold in context, modify at end.
|
||||
|
||||
```bash
|
||||
# Start of execution
|
||||
STATE=$(cat .planning/STATE.md) # ~50-100 lines in context
|
||||
|
||||
# ... 50% context later ...
|
||||
|
||||
# End of execution
|
||||
# Modify STATE in memory, write back
|
||||
```
|
||||
|
||||
**Problem:**
|
||||
- Full STATE.md in context throughout execution
|
||||
- Risk of stale reads if parallel agents update
|
||||
- Merge conflicts on concurrent writes
|
||||
|
||||
**Proposed state:** Patch-based operations.
|
||||
|
||||
```bash
|
||||
# Read only what's needed
|
||||
POSITION=$(node ~/.claude/get-shit-done/bin/gsd-tools.js state get position)
|
||||
DECISIONS=$(node ~/.claude/get-shit-done/bin/gsd-tools.js state get decisions)
|
||||
|
||||
# Atomic updates
|
||||
node ~/.claude/get-shit-done/bin/gsd-tools.js state patch \
|
||||
--position "Phase: 03, Plan: 02, Status: Complete" \
|
||||
--add-decision "Used jose for JWT per Edge runtime constraints" \
|
||||
--set-session "Last: 2024-01-15, Stopped: 03-02, Resume: 03-03-PLAN.md"
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```javascript
|
||||
// bin/gsd-tools.js
|
||||
case 'state':
|
||||
const subcommand = args[1]; // 'get' or 'patch'
|
||||
const statePath = '.planning/STATE.md';
|
||||
|
||||
if (subcommand === 'get') {
|
||||
const section = args[2]; // 'position', 'decisions', 'session', etc.
|
||||
const content = fs.readFileSync(statePath, 'utf8');
|
||||
const parsed = parseStateSection(content, section);
|
||||
console.log(JSON.stringify(parsed));
|
||||
}
|
||||
|
||||
if (subcommand === 'patch') {
|
||||
// Parse --position, --add-decision, --set-session flags
|
||||
// Read file, apply patches, write back atomically
|
||||
// Use file locking for concurrent safety
|
||||
}
|
||||
break;
|
||||
```
|
||||
|
||||
**Savings:** Agents don't hold full STATE.md in context. Reduces context by ~1-2KB per agent. Prevents concurrent update conflicts.
|
||||
|
||||
---
|
||||
|
||||
### O6: Smart Summary Templates
|
||||
|
||||
**Priority:** 6 (Low impact, low effort)
|
||||
|
||||
**Current state:** One summary template for all plan types.
|
||||
|
||||
```markdown
|
||||
# summary.md template - 114 lines
|
||||
# Includes: frontmatter, title, overview, task details, deviations,
|
||||
# auth gates, verification, self-check, key files, next steps
|
||||
```
|
||||
|
||||
**Problem:** A simple config change produces the same verbose summary as a complex auth implementation.
|
||||
|
||||
**Proposed state:** Template variants.
|
||||
|
||||
```
|
||||
templates/
|
||||
summary-minimal.md # ~30 lines - config, simple CRUD
|
||||
summary-standard.md # ~60 lines - typical features
|
||||
summary-complex.md # ~100 lines - architectural changes, decisions
|
||||
```
|
||||
|
||||
**Selection heuristic:**
|
||||
|
||||
```javascript
|
||||
function selectSummaryTemplate(plan) {
|
||||
const taskCount = plan.tasks.length;
|
||||
const hasDecisions = plan.tasks.some(t => t.type.includes('decision'));
|
||||
const hasDeviations = plan.deviations?.length > 0;
|
||||
const fileCount = plan.files_modified?.length || 0;
|
||||
|
||||
if (taskCount <= 2 && fileCount <= 3 && !hasDecisions) {
|
||||
return 'summary-minimal.md';
|
||||
}
|
||||
if (hasDecisions || hasDeviations || fileCount > 6) {
|
||||
return 'summary-complex.md';
|
||||
}
|
||||
return 'summary-standard.md';
|
||||
}
|
||||
```
|
||||
|
||||
**Savings:** Reduces summary creation context by 30-60% for simple plans. Executor doesn't load 114-line template for a 2-task config change.
|
||||
|
||||
---
|
||||
|
||||
### O7: MCP Semantic Queries
|
||||
|
||||
**Priority:** 7 (Medium impact, high effort)
|
||||
|
||||
**Current state:** Context loading via grep and file reads.
|
||||
|
||||
```bash
|
||||
# Find what uses User model
|
||||
grep -r "User" .planning/phases/*/*-SUMMARY.md
|
||||
|
||||
# Find auth patterns
|
||||
grep -r "auth\|jwt\|session" .planning/phases/*/*-SUMMARY.md
|
||||
```
|
||||
|
||||
**Problem:** Text matching returns noise. "User" matches "user experience", "user-facing", etc.
|
||||
|
||||
**Proposed state:** Extend gsd-memory MCP with semantic queries.
|
||||
|
||||
```javascript
|
||||
// New MCP tools
|
||||
gsd_memory_what_uses({ symbol: "User", type: "model" })
|
||||
// Returns: ["03-01-SUMMARY.md", "04-02-SUMMARY.md"] with context
|
||||
|
||||
gsd_memory_pattern_for({ domain: "auth" })
|
||||
// Returns: { library: "jose", approach: "httpOnly cookies", refresh: "7d rotation" }
|
||||
|
||||
gsd_memory_decisions_affecting({ subsystem: "api" })
|
||||
// Returns: [{ phase: "01-02", decision: "Server Actions over API routes" }]
|
||||
```
|
||||
|
||||
**Implementation approach:**
|
||||
|
||||
1. Index summaries on write (post-commit hook or gsd-tools trigger)
|
||||
2. Store in SQLite with FTS5 for text search + JSON fields for structured data
|
||||
3. Expose via MCP tools
|
||||
4. Agents query instead of grep
|
||||
|
||||
**Savings:** Precise context retrieval. Agent gets exactly what it needs, no noise. Estimate: 50% reduction in history-loading context for large projects.
|
||||
|
||||
**Trade-off:** Significant implementation effort. Index maintenance. SQLite dependency.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Roadmap
|
||||
|
||||
### Phase 1: Quick Wins (1-2 days)
|
||||
|
||||
1. **O3: Frontmatter digest** - Add `gsd-tools history-digest`
|
||||
2. **O5: State patch operations** - Add `gsd-tools state get/patch`
|
||||
3. **O6: Summary template variants** - Create minimal/standard/complex
|
||||
|
||||
### Phase 2: Architecture Changes (3-5 days)
|
||||
|
||||
4. **O1: Lazy-load references** - Restructure executor into core + references
|
||||
5. **O2: Tiered agent prompts** - Restructure planner into core + extensions
|
||||
|
||||
### Phase 3: Build System (3-5 days)
|
||||
|
||||
6. **O4: Compiled plans** - Add `gsd-tools compile-plan` with smart stripping
|
||||
|
||||
### Phase 4: Semantic Layer (5-10 days)
|
||||
|
||||
7. **O7: MCP semantic queries** - Extend gsd-memory with structured queries
|
||||
|
||||
---
|
||||
|
||||
## Metrics
|
||||
|
||||
### Before/After Tracking
|
||||
|
||||
| Metric | Current | Target | How to Measure |
|
||||
|--------|---------|--------|----------------|
|
||||
| Executor base context | ~19KB | ~8KB | `wc -c agents/gsd-executor*.md` |
|
||||
| Planner base context | ~55KB | ~20KB | `wc -c agents/gsd-planner*.md` |
|
||||
| History loading (10 phases) | ~20KB | ~2KB | Digest size vs full summaries |
|
||||
| Plan execution start context | ~25% | ~12% | Log context % at first task |
|
||||
|
||||
### Quality Indicators
|
||||
|
||||
- Plans completing within 50% context (target: 95%+)
|
||||
- Verification pass rate (should stay same or improve)
|
||||
- Deviation rate (should stay same)
|
||||
- User checkpoint fatigue (fewer "skip" responses)
|
||||
|
||||
---
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Lazy loading adds latency | Keep references <50 lines, cache resolved content |
|
||||
| Tiered prompts miss edge cases | Comprehensive testing, fallback to full prompt |
|
||||
| Compiled plans become stale | Mtime checks, recompile on source change |
|
||||
| Semantic queries require maintenance | Auto-index on summary creation, periodic reindex |
|
||||
| Breaking changes during refactor | Feature flags, A/B testing old vs new paths |
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. **Context efficiency:** Average plan execution uses <40% context (down from ~55%)
|
||||
2. **Quality maintenance:** Verification pass rate stays ≥95%
|
||||
3. **Speed:** No measurable latency increase from lazy loading
|
||||
4. **Maintainability:** Clear separation between core and extensions
|
||||
5. **Debuggability:** Easy to trace which modules were loaded for any execution
|
||||
|
||||
---
|
||||
|
||||
## Appendix: File Structure After Optimization
|
||||
|
||||
```
|
||||
agents/
|
||||
gsd-executor-core.md # Base executor (~150 lines)
|
||||
gsd-planner-core.md # Base planner (~300 lines)
|
||||
gsd-planner-ext/
|
||||
discovery.md
|
||||
gap-closure.md
|
||||
revision.md
|
||||
tdd.md
|
||||
checkpoints.md
|
||||
gsd-verifier.md # Already lean
|
||||
gsd-phase-researcher.md # Candidate for similar treatment
|
||||
...
|
||||
|
||||
references/
|
||||
executor/
|
||||
deviation-rules.md
|
||||
tdd-execution.md
|
||||
checkpoint-protocol.md
|
||||
continuation.md
|
||||
summary-creation.md
|
||||
planner/
|
||||
goal-backward.md # Core methodology, always loaded
|
||||
checkpoints.md # Existing, used by multiple agents
|
||||
tdd.md # Existing
|
||||
|
||||
templates/
|
||||
summary-minimal.md
|
||||
summary-standard.md
|
||||
summary-complex.md
|
||||
...
|
||||
|
||||
bin/
|
||||
gsd-tools.js
|
||||
# Existing commands
|
||||
+ history-digest # O3
|
||||
+ state get/patch # O5
|
||||
+ compile-plan # O4
|
||||
+ select-template # O6
|
||||
```
|
||||
@@ -1,103 +0,0 @@
|
||||
# The "Railroad" Architecture: GSD Framework Evolution
|
||||
|
||||
## Executive Summary
|
||||
This document outlines the strategic evolution of the Get Shit Done (GSD) framework from a "Prompt Engineering" reliance to a **"Software Engineering"** foundation. The core objective is to reduce token consumption ("context tax") and increase reliability by shifting workflow logic from LLM context to deterministic code.
|
||||
|
||||
## 1. The Core Shift: "Railroad" State Machine
|
||||
|
||||
### Current State
|
||||
Currently, the LLM acts as the **Workflow Engine**. It reads a verbose Markdown file (e.g., `new-project.md`) containing logic like:
|
||||
> "If the user says X, then ask Y. If they say Z, execute tool A."
|
||||
|
||||
This relies on "soft state management" where the LLM must perfectly recall and adhere to written instructions.
|
||||
|
||||
### Proposed State ("Railroad")
|
||||
The LLM becomes an **Intelligent Worker**, while `gsd-tools` becomes the **Workflow Engine**.
|
||||
|
||||
* **Logic in Code:** Complex branching logic is moved to `gsd-tools`.
|
||||
* **The "Next Step" Pattern:**
|
||||
Instead of reading a map, the Agent simply asks: "What is next?"
|
||||
|
||||
```bash
|
||||
# Agent calls:
|
||||
node gsd-tools.js next-step --workflow new-project --state-file .planning/state.json
|
||||
```
|
||||
|
||||
**Tool Returns:**
|
||||
```json
|
||||
{
|
||||
"action": "ask_user",
|
||||
"prompt": "What do you want to build?",
|
||||
"context_files": []
|
||||
}
|
||||
```
|
||||
|
||||
* **Benefit:** The Agent needs *zero* knowledge of the overall workflow. It focuses 100% of its attention on executing the single, immediate task. It cannot "skip" steps because it doesn't know they exist until the tool assigns them.
|
||||
|
||||
## 2. Just-in-Time (JIT) Context Loading
|
||||
|
||||
### Current State
|
||||
We load all templates, references, and instructions at the start of a session.
|
||||
* *Example:* `new-project` loads `questioning.md`, `project.md` template, `requirements.md` template, etc.
|
||||
|
||||
### Proposed State
|
||||
Context is injected only when needed for the specific step.
|
||||
|
||||
* **Step 1 (Questioning):** Load `questioning.md`. Do NOT load `requirements.md` template.
|
||||
* **Step 2 (Requirements):** Unload `questioning.md`. Load `requirements.md` template.
|
||||
* **Step 3 (Roadmap):** Load `ROADMAP.md` template.
|
||||
|
||||
**Mechanism:**
|
||||
The `gsd-tools next-step` response includes a `context_files` array. The wrapper script or system prompt dynamically reads these files into the context window for that turn only.
|
||||
|
||||
## 3. Native Interventions (Bypassing the LLM)
|
||||
|
||||
### Current State
|
||||
The LLM acts as a text-based menu system.
|
||||
> "User, please choose: 1. YOLO Mode, 2. Interactive Mode."
|
||||
|
||||
This burns tokens on input tokens (instructions on how to ask), output tokens (asking the question), and input tokens (processing the user's "1").
|
||||
|
||||
### Proposed State
|
||||
Standardize discrete choices into CLI interactions managed by `gsd-tools` or the wrapper.
|
||||
|
||||
* **Configuration Wizard:** `gsd-tools init new-project` runs an interactive CLI wizard (using `inquirer` or `readline`) to gather `mode`, `depth`, `parallelization` preferences *before* the LLM is even invoked.
|
||||
* **Structured Output:** The LLM receives the result as a finalized JSON configuration, not a conversation transcript.
|
||||
|
||||
## 4. Micro-Agents & Chained Execution
|
||||
|
||||
### Current State
|
||||
Monolithic workflows (`new-project`) maintain a single context window that grows indefinitely. By the time the agent reaches "Roadmap Creation" (Step 9), the context is polluted with "Questioning" (Step 3) transcripts.
|
||||
|
||||
### Proposed State
|
||||
Break workflows into atomic, chainable units.
|
||||
|
||||
1. **Process:** `gsd:new-project`
|
||||
2. **Chain:**
|
||||
* **Agent A (Scope):** Interviews user $\rightarrow$ Outputs `PROJECT.md` $\rightarrow$ **Exits.**
|
||||
* *Context Flush*
|
||||
* **Agent B (Research):** Reads `PROJECT.md` $\rightarrow$ Outputs `research/` $\rightarrow$ **Exits.**
|
||||
* *Context Flush*
|
||||
* **Agent C (Roadmap):** Reads `PROJECT.md` + `research/` $\rightarrow$ Outputs `ROADMAP.md`.
|
||||
|
||||
**Implementation:**
|
||||
The `gsd-tools` utility manages the hand-off. When Agent A finishes, it returns a specific exit code or signal. The outer loop detects this, clears context, and spawns Agent B with the specific inputs generated by A.
|
||||
|
||||
## Implementation Roadmap
|
||||
|
||||
1. **Expand `gsd-tools`:**
|
||||
* Implement `next-step` state machine logic.
|
||||
* Add interactive CLI prompts for configuration.
|
||||
2. **Refactor Workflows:**
|
||||
* Strip `workflows/*.md` to bare essentials (or remove them entirely in favor of code definitions).
|
||||
3. **Update Agent Definitions:**
|
||||
* Direct agents to rely on `gsd-tools` for guidance rather than internal reasoning about the process.
|
||||
|
||||
## Summary of Benefits
|
||||
|
||||
| Metric | Current | Railroad Architecture |
|
||||
| :--- | :--- | :--- |
|
||||
| **Context Usage** | High (Full workflow + History) | Minimal (Current Step + Immediate Inputs) |
|
||||
| **Reliability** | Variable (LLM can hallucinate process) | Deterministic (Code enforces process) |
|
||||
| **Speed** | Slower (Reading/generating verbose text) | Faster (Code execution + concise LLM tasks) |
|
||||
| **Maintenance** | Distributed (Markdown files) | Centralized (JS Logic) |
|
||||
@@ -1,123 +0,0 @@
|
||||
# GSD Context Optimization Proposal
|
||||
Goal: Reduce unnecessary context load while preserving or improving execution quality and speed.
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
GSD is already moving toward leaner orchestration and centralized tooling. The next step is to systematically shift logic out of prompts and into deterministic tooling, while loading only the instructions needed for the current step. This proposal delivers that outcome through a phased approach:
|
||||
|
||||
1. Immediate context diet with fast ROI and minimal risk.
|
||||
2. Prompt modularization to reduce baseline agent size.
|
||||
3. Precompiled execution artifacts to strip unused protocols.
|
||||
4. Railroad (state machine) architecture for long-term determinism.
|
||||
|
||||
This balances speed, quality, and maintainability while protecting GSD's core instructional content.
|
||||
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
GSD's effectiveness depends on rich instructional content, but its current context load includes large, often unused sections. As context usage grows, quality degrades and speed drops. The system needs a precision-loading strategy: only bring into context what's needed for the next step, and let deterministic tooling enforce the process.
|
||||
|
||||
---
|
||||
|
||||
## Design Principles
|
||||
- Demand-load over eager-load: Load instructions only when needed.
|
||||
- Deterministic orchestration: Move branching logic into code.
|
||||
- Stable core, modular extensions: Keep minimal core prompts and load extensions conditionally.
|
||||
- Measurable quality: Reductions must not degrade verification pass rates.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Strategy
|
||||
|
||||
### 1) Immediate Context Diet (Quick Wins)
|
||||
- History digest: Load structured summaries instead of full history.
|
||||
- State patch ops: Read only the state sections required.
|
||||
- Summary template variants: Choose minimal or standard templates based on complexity.
|
||||
- Compound init sweep: Consolidate context loading into a single structured payload.
|
||||
|
||||
Impact: Faster startup, lower context load, minimal refactor risk.
|
||||
|
||||
---
|
||||
|
||||
### 2) Prompt Modularity (High Impact, Moderate Effort)
|
||||
- Split large agents into core + extensions.
|
||||
- Load extensions only when triggered by plan characteristics or flags.
|
||||
- Add a context budget mode (min|std|full) with escalation when needed.
|
||||
|
||||
Impact: Large baseline prompt reduction without behavior regression.
|
||||
|
||||
---
|
||||
|
||||
### 3) Compiled Plans (Performance + Precision)
|
||||
- Pre-compile plans by resolving references and stripping unused protocols.
|
||||
- Cache compiled outputs; recompile on source change.
|
||||
|
||||
Impact: Significant runtime context reduction and faster execution start.
|
||||
|
||||
---
|
||||
|
||||
### 4) Railroad Architecture (Long-Term Determinism)
|
||||
- gsd-tools becomes a state machine, returning the next action step.
|
||||
- Agents only see current step + immediate context.
|
||||
- Enables JIT context and prevents step skipping.
|
||||
|
||||
Impact: Maximum context efficiency and reliability, higher implementation cost.
|
||||
|
||||
---
|
||||
|
||||
## Phased Roadmap
|
||||
|
||||
### Phase 1: Quick Wins (1-2 days)
|
||||
1. Add history-digest to gsd-tools.
|
||||
2. Add state get/patch operations.
|
||||
3. Add summary template variants.
|
||||
4. Finish compound init sweep.
|
||||
|
||||
### Phase 2: Prompt Modularity (3-5 days)
|
||||
1. Split gsd-executor into core + conditional references.
|
||||
2. Split gsd-planner into core + extensions.
|
||||
3. Add context budget mode.
|
||||
|
||||
### Phase 3: Compiled Plans (3-5 days)
|
||||
1. Implement compile-plan.
|
||||
2. Strip unused protocols based on plan metadata.
|
||||
3. Cache compiled outputs with mtime checks.
|
||||
|
||||
### Phase 4: Railroad Architecture (5-10 days)
|
||||
1. Implement next-step state machine.
|
||||
2. Migrate workflows to deterministic code.
|
||||
3. Add micro-agent chaining with context flush.
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
- Average context usage under 40% at task start.
|
||||
- Verification pass rate >= 95%.
|
||||
- No measurable latency increase from dynamic loading.
|
||||
- Stable or improved deviation rate.
|
||||
- Reduced user "skip" frequency during checkpoints.
|
||||
|
||||
---
|
||||
|
||||
## Risks and Mitigations
|
||||
- Missing edge-case instructions
|
||||
Mitigation: fallback to full prompt under --context=full.
|
||||
- Compiled plan staleness
|
||||
Mitigation: mtime checks and auto-recompile.
|
||||
- Railroad refactor complexity
|
||||
Mitigation: incremental rollout with feature flags.
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
Start with Phase 1 and Phase 2 immediately. These deliver the largest context reductions per unit of risk and pave the way for compiled plans and the Railroad architecture. Once stable, proceed to Phase 3 and Phase 4.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Next Steps
|
||||
1. Approve Phase 1 scope and timeline.
|
||||
2. Decide acceptable tolerance for architectural refactors.
|
||||
3. Choose quality metrics and thresholds for go/no-go.
|
||||
|
||||
If you want, I can turn this into actionable tickets with acceptance criteria and an initial implementation plan for gsd-tools and prompt splits.
|
||||
@@ -1,72 +0,0 @@
|
||||
# GSD Performance Architecture: The "Hydra" Engine
|
||||
> **Objective:** Transition GSD from a context-taxed "Prompt-Driven" system to a "Software-Engineered" Agentic Framework.
|
||||
|
||||
## 1. The Core Philosophy: "Attention Density"
|
||||
In LLM systems, **Context is Noise**. Every irrelevant line of TDD instruction loaded during a simple documentation task isn't just a "token cost"—it is **attention dilution**. The "Hydra" architecture ensures that for any given turn, the LLM has **100% relevance density**.
|
||||
|
||||
---
|
||||
|
||||
## 2. Structural Synthesis: The Three Pillars
|
||||
|
||||
### Pillar I: The "Railroad" Spine (Deterministic Logic)
|
||||
We will move workflow management out of the LLM's head and into `gsd-tools.js`.
|
||||
- **The Shift:** Instead of an Agent reading a 300-line `execute-phase.md` to figure out what to do next, it calls `gsd-tools next-step`.
|
||||
- **The Result:** The LLM's context window is freed from "Process Overhead" and reserved entirely for "Task Logic."
|
||||
|
||||
### Pillar II: "Lazy-Load" Context (The JIT Memory)
|
||||
We will implement **Just-In-Time (JIT)** injection for all instructional content.
|
||||
- **Tiered Agents:** `gsd-planner-core` (The what) + `gsd-planner-tdd` (The how, loaded only when code changes are detected).
|
||||
- **Frontmatter Digests:** Instead of reading 10 full summaries (20KB), the Agent reads a single JSON digest (2KB) of dependencies and patterns.
|
||||
- **Compiled Plans:** Plans are "pre-baked" by `gsd-tools` to strip unused protocols (e.g., removing Checkpoint rules if the plan has no checkpoints).
|
||||
|
||||
### Pillar III: Chained Execution (Context Flushing)
|
||||
We will move away from monolithic sessions.
|
||||
- **Micro-Agents:** Break `new-project` into: `Interviewer` → `Architect` → `Roadmapper`.
|
||||
- **Hand-offs:** Each agent performs its task, writes to disk, and **terminates**. The next agent starts with a **clean context window**, reading only the necessary outputs from the previous step. This eliminates "Transcript Bloat."
|
||||
|
||||
---
|
||||
|
||||
## 3. The Target State Execution Flow
|
||||
*Example: Running `gsd execute-phase`*
|
||||
|
||||
1. **State Check:** `gsd-tools` reads `.planning/state.json`.
|
||||
2. **Context Assembly:**
|
||||
- Loads `gsd-executor-core.md` (Base identity).
|
||||
- Detects "TDD" flag in task: Injects `references/tdd.md`.
|
||||
- Detects "API" change: Injects `history-digest.json` filtered for "API" tags.
|
||||
3. **Execution:** Agent executes the task with **< 15% context usage**.
|
||||
4. **Atomic Update:** Agent reports success. `gsd-tools` patches `STATE.md` and `SUMMARY.md` via deterministic code, not LLM rewriting.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation Roadmap (The 4-Phase Sprint)
|
||||
|
||||
### Phase 1: The "Low-Hanging" Pruning (Quick Wins)
|
||||
* **Action:** Implement `gsd-tools history-digest` and `gsd-tools state patch`.
|
||||
* **Impact:** Immediate 40% reduction in history/state overhead.
|
||||
* **Timeline:** 1 Day.
|
||||
|
||||
### Phase 2: The "Lazy-Load" Restructuring
|
||||
* **Action:** Split `gsd-executor` and `gsd-planner` into `-core` and `-ext` modules.
|
||||
* **Action:** Update wrapper scripts to dynamically assemble prompts based on task metadata.
|
||||
* **Impact:** Massive reduction in "Instructional Noise."
|
||||
* **Timeline:** 3 Days.
|
||||
|
||||
### Phase 3: The "Railroad" Migration
|
||||
* **Action:** Convert `workflows/*.md` logic into a JSON-based state machine in `gsd-tools`.
|
||||
* **Action:** Implement the `next-step` pattern.
|
||||
* **Impact:** Elimination of "Process Hallucination" and workflow skipping.
|
||||
* **Timeline:** 5 Days.
|
||||
|
||||
### Phase 4: Semantic Intelligence (The MCP Layer)
|
||||
* **Action:** Extend `gsd-memory` MCP to index summaries.
|
||||
* **Action:** Replace `grep` searches with semantic queries.
|
||||
* **Impact:** Precise, noise-free context retrieval for large codebases.
|
||||
* **Timeline:** 7 Days.
|
||||
|
||||
---
|
||||
|
||||
## 5. Risk Assessment: "The Instruction Guardrail"
|
||||
You noted that **instructional content is crucial**.
|
||||
- **Mitigation:** We do not "slim" the instructions; we **modularize** them.
|
||||
- **Example:** The TDD protocol remains exactly as detailed as it is today, but it simply *does not exist* in the LLM's mind when it is renaming a file or updating a README. This actually **improves** TDD fidelity because the LLM isn't distracted by other protocols.
|
||||
Reference in New Issue
Block a user