1b317dec45
* feat(gsd-tools): add history-digest, atomic state operations, and summary variants Adds new performance-focused commands to gsd-tools and introduces specialized summary templates to reduce context tax: - history-digest: Compiles phase summaries into structured JSON for JIT loading - state get/patch: Enables atomic STATE.md operations instead of full rewrites - template select: Automatically chooses optimal summary template based on plan complexity - Adds minimal, standard, and complex summary templates Part of the "Hydra" architecture for GSD context optimization. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add project config * docs: define v1 requirements * docs: create roadmap (6 phases) * feat(history-digest): fix nested YAML parsing and add tests - Fix extractFrontmatter() to handle nested YAML structures like dependency-graph.provides, tech-stack.added using stack-based parsing - Add test infrastructure with Node test runner (npm test) - Update gsd-planner to use digest fields directly instead of reading full SUMMARY.md files - Add 6 schema validation tests covering nested fields, merging, malformed files, and backward compatibility Closes: HIST-01, HIST-02, HIST-03, HIST-04 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(planner): use digest for selection, full SUMMARY for understanding The previous commit went too far by eliminating full SUMMARY reads. The digest is an index for smart selection, not a replacement for understanding what actually happened. Two-step approach: 1. Digest to score/select relevant phases (2-4 typically) 2. Full SUMMARY read for selected phases (implementation details) 3. Digest-level context retained for unselected phases Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(gsd-tools): add phases, roadmap, and phase commands TDD implementation of three new commands to replace repetitive bash: phases list [--type plans|summaries] [--phase N] - Lists phase directories sorted numerically (handles decimals) - Filter by file type or specific phase - Replaces: ls -d .planning/phases/*/ | sort -V (22 occurrences) roadmap get-phase <N> - Extracts phase section from ROADMAP.md - Returns name, goal, full section content - Replaces: grep -A20 "Phase X:" ROADMAP.md (19 occurrences) phase next-decimal <N> - Calculates next decimal phase (06 → 06.1, 06.2 → 06.3) - Handles gaps, normalizes input - Replaces: complex bash math in insert-phase (3 occurrences) 16 new tests, all passing. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: migrate agents/workflows to use gsd-tools commands Replace inline bash patterns with centralized gsd-tools commands: phases list: - audit-milestone.md: ls -d .planning/phases/*/ | sort -V - plan-milestone-gaps.md: ls -d ... | sort -V | tail -1 roadmap get-phase: - plan-phase.md: grep -A5 "Phase X:" ROADMAP.md (2 occurrences) - research-phase.md: grep patterns (2 occurrences) - verify-phase.md: grep -A5 pattern - gsd-verifier.md: grep -A5 pattern - gsd-plan-checker.md: grep -A10 pattern - commands/gsd/research-phase.md: grep patterns (2 occurrences) phase next-decimal: - insert-phase.md: complex bash decimal calculation - decimal-phase-calculation.md: reference doc rewritten phase-argument-parsing.md: updated to reference gsd-tools Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
4.5 KiB
4.5 KiB
GSD Context Optimization Proposal
Goal: Reduce unnecessary context load while preserving or improving execution quality and speed.
Executive Summary
GSD is already moving toward leaner orchestration and centralized tooling. The next step is to systematically shift logic out of prompts and into deterministic tooling, while loading only the instructions needed for the current step. This proposal delivers that outcome through a phased approach:
- Immediate context diet with fast ROI and minimal risk.
- Prompt modularization to reduce baseline agent size.
- Precompiled execution artifacts to strip unused protocols.
- Railroad (state machine) architecture for long-term determinism.
This balances speed, quality, and maintainability while protecting GSD's core instructional content.
Problem Statement
GSD's effectiveness depends on rich instructional content, but its current context load includes large, often unused sections. As context usage grows, quality degrades and speed drops. The system needs a precision-loading strategy: only bring into context what's needed for the next step, and let deterministic tooling enforce the process.
Design Principles
- Demand-load over eager-load: Load instructions only when needed.
- Deterministic orchestration: Move branching logic into code.
- Stable core, modular extensions: Keep minimal core prompts and load extensions conditionally.
- Measurable quality: Reductions must not degrade verification pass rates.
Proposed Strategy
1) Immediate Context Diet (Quick Wins)
- History digest: Load structured summaries instead of full history.
- State patch ops: Read only the state sections required.
- Summary template variants: Choose minimal or standard templates based on complexity.
- Compound init sweep: Consolidate context loading into a single structured payload.
Impact: Faster startup, lower context load, minimal refactor risk.
2) Prompt Modularity (High Impact, Moderate Effort)
- Split large agents into core + extensions.
- Load extensions only when triggered by plan characteristics or flags.
- Add a context budget mode (min|std|full) with escalation when needed.
Impact: Large baseline prompt reduction without behavior regression.
3) Compiled Plans (Performance + Precision)
- Pre-compile plans by resolving references and stripping unused protocols.
- Cache compiled outputs; recompile on source change.
Impact: Significant runtime context reduction and faster execution start.
4) Railroad Architecture (Long-Term Determinism)
- gsd-tools becomes a state machine, returning the next action step.
- Agents only see current step + immediate context.
- Enables JIT context and prevents step skipping.
Impact: Maximum context efficiency and reliability, higher implementation cost.
Phased Roadmap
Phase 1: Quick Wins (1-2 days)
- Add history-digest to gsd-tools.
- Add state get/patch operations.
- Add summary template variants.
- Finish compound init sweep.
Phase 2: Prompt Modularity (3-5 days)
- Split gsd-executor into core + conditional references.
- Split gsd-planner into core + extensions.
- Add context budget mode.
Phase 3: Compiled Plans (3-5 days)
- Implement compile-plan.
- Strip unused protocols based on plan metadata.
- Cache compiled outputs with mtime checks.
Phase 4: Railroad Architecture (5-10 days)
- Implement next-step state machine.
- Migrate workflows to deterministic code.
- Add micro-agent chaining with context flush.
Success Metrics
- Average context usage under 40% at task start.
- Verification pass rate >= 95%.
- No measurable latency increase from dynamic loading.
- Stable or improved deviation rate.
- Reduced user "skip" frequency during checkpoints.
Risks and Mitigations
- Missing edge-case instructions Mitigation: fallback to full prompt under --context=full.
- Compiled plan staleness Mitigation: mtime checks and auto-recompile.
- Railroad refactor complexity Mitigation: incremental rollout with feature flags.
Recommendation
Start with Phase 1 and Phase 2 immediately. These deliver the largest context reductions per unit of risk and pave the way for compiled plans and the Railroad architecture. Once stable, proceed to Phase 3 and Phase 4.
Proposed Next Steps
- Approve Phase 1 scope and timeline.
- Decide acceptable tolerance for architectural refactors.
- Choose quality metrics and thresholds for go/no-go.
If you want, I can turn this into actionable tickets with acceptance criteria and an initial implementation plan for gsd-tools and prompt splits.