docs(01-04): complete kernel unit and integration test plan
- Record coverage, check-phase1.sh result, and measured hello rebuild - Document race-buffer and pact-test deviations
This commit is contained in:
Jakub Zych
179
.planning/phases/01-framework-kernel-foundation/01-04-SUMMARY.md
Normal file
179
.planning/phases/01-framework-kernel-foundation/01-04-SUMMARY.md
Normal file
@@ -0,0 +1,179 @@
|
|||||||
|
---
|
||||||
|
phase: 01-framework-kernel-foundation
|
||||||
|
plan: 04
|
||||||
|
subsystem: testing
|
||||||
|
tags: [go, unit-tests, race, watch, cli, validation]
|
||||||
|
|
||||||
|
requires:
|
||||||
|
- phase: 01-framework-kernel-foundation
|
||||||
|
provides: Kernel packages, hello workspace, summer build/dev, bonfire output
|
||||||
|
provides:
|
||||||
|
- Behavioral unit and race coverage for KERN-01 through KERN-09 and CLI-01
|
||||||
|
- Repeatable scripts/check-phase1.sh across root, hello, base, greeter, optional
|
||||||
|
- Nyquist-compliant VALIDATION.md with passing automated checks
|
||||||
|
affects: [02, 03, verify-work]
|
||||||
|
|
||||||
|
tech-stack:
|
||||||
|
added: []
|
||||||
|
patterns:
|
||||||
|
- Table-driven contract tests over implementation-structure asserts
|
||||||
|
- Nested-module check script runs vet/test/race per go.work member
|
||||||
|
- Watch tests inject Build/Start plus one real hello workspace rebuild
|
||||||
|
|
||||||
|
key-files:
|
||||||
|
created:
|
||||||
|
- pact/capabilities_test.go
|
||||||
|
- scripts/check-phase1.sh
|
||||||
|
modified:
|
||||||
|
- compass/config_test.go
|
||||||
|
- party/registry_test.go
|
||||||
|
- backpack/services_test.go
|
||||||
|
- festival/bus_test.go
|
||||||
|
- towel/context_test.go
|
||||||
|
- bonfire/output_test.go
|
||||||
|
- bonfire/prompts_test.go
|
||||||
|
- internal/build/build_test.go
|
||||||
|
- internal/dev/watch_test.go
|
||||||
|
- examples/hello/hello_test.go
|
||||||
|
- .planning/phases/01-framework-kernel-foundation/01-VALIDATION.md
|
||||||
|
|
||||||
|
key-decisions:
|
||||||
|
- "Cover contract branches rather than an arbitrary coverage percentage"
|
||||||
|
- "Watch tests use a mutex-safe buffer so go test -race can read captured output"
|
||||||
|
- "VALIDATION.md is a docs commit separate from test code per CLAUDE.md"
|
||||||
|
|
||||||
|
patterns-established:
|
||||||
|
- "Pattern: scripts/check-phase1.sh is the phase-wide verification command"
|
||||||
|
- "Pattern: hello copies rewrite framework replace to an absolute path for temp-dir builds"
|
||||||
|
|
||||||
|
requirements-completed: [KERN-01, KERN-02, KERN-03, KERN-04, KERN-05, KERN-06, KERN-07, KERN-08, KERN-09, CLI-01]
|
||||||
|
|
||||||
|
duration: 11min
|
||||||
|
completed: 2026-09-16
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 1 Plan 4: Kernel Unit and Integration Tests Summary
|
||||||
|
|
||||||
|
**Phase-wide `scripts/check-phase1.sh` with race-enabled contract tests for config precedence, plugin lifecycle, typed events, non-TTY CLI, and a measured hello workspace rebuild**
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
|
||||||
|
- **Duration:** 11 min
|
||||||
|
- **Started:** 2026-09-16T12:03:03Z
|
||||||
|
- **Completed:** 2026-09-16T12:14:21Z
|
||||||
|
- **Tasks:** 2
|
||||||
|
- **Files modified:** 13
|
||||||
|
|
||||||
|
## Accomplishments
|
||||||
|
|
||||||
|
- Expanded kernel smoke tests into source-to-observable assertions for all six compass layers, malformed YAML, snake_case env keys, `.env` non-override, Set/Persist/Reload, concurrent reads, plugin duplicate/missing/cycle, HasConfig/HasCommands discovery, isolated backpack services, three festival dispatch modes, and nested towel context
|
||||||
|
- Covered namespaced command flags/args, injected output, non-TTY spinner/progress/table/prompts, `NO_COLOR`/`FORCE_COLOR`/`TERM=dumb`, closed stdin, secret non-leak, `make:plugin`/`plugin:add`, malicious IDs, invalid Requires on a built hello binary, debounce/ignore/fail/reap watch hooks, and one real fsnotify hello rebuild that prints `rebuild: <duration>`
|
||||||
|
- Added `scripts/check-phase1.sh` (root, hello app, base, greeter, optional) and signed VALIDATION.md `nyquist_compliant: true` after the script passed
|
||||||
|
|
||||||
|
## Task Commits
|
||||||
|
|
||||||
|
Each task was committed atomically:
|
||||||
|
|
||||||
|
1. **Task 1: Cover config, lifecycle, services, events and context** - `a43c734` (test)
|
||||||
|
2. **Task 2: Cover tool, output, watch loop and every workspace module** - `dc7997e` (test)
|
||||||
|
3. **Task 2 docs: Record phase 1 validation evidence** - `bee03bd` (docs)
|
||||||
|
|
||||||
|
**Plan metadata:** pending (this file)
|
||||||
|
|
||||||
|
## Files Created/Modified
|
||||||
|
|
||||||
|
- `compass/config_test.go` - table-driven precedence, malformed YAML, snake_case env, Persist/Reload, concurrent reads
|
||||||
|
- `party/registry_test.go` - reordered manifest, duplicate/missing IDs, HasConfig/HasCommands type assertions
|
||||||
|
- `backpack/services_test.go` - two-app isolation, nil publish, concurrent lookup
|
||||||
|
- `festival/bus_test.go` - concurrent buses, panic payload omission, collect/until-handled edges
|
||||||
|
- `towel/context_test.go` - nested isolation, nil context, concurrent accessors
|
||||||
|
- `pact/capabilities_test.go` - HasConfig/HasCommands/OptionalMessage discovery
|
||||||
|
- `bonfire/output_test.go` / `prompts_test.go` - flags/args, injected streams, secret non-leak, closed stdin
|
||||||
|
- `internal/build/build_test.go` - duplicate modules, path/shell-meta IDs, built hello invalid Requires
|
||||||
|
- `internal/dev/watch_test.go` - debounce, ignored bin/tmp, real hello rebuild latency line
|
||||||
|
- `examples/hello/hello_test.go` - built binary unknown-command non-zero exit
|
||||||
|
- `scripts/check-phase1.sh` - vet/test/race per workspace module plus `./bin/hello greeter:hello`
|
||||||
|
- `.planning/phases/01-framework-kernel-foundation/01-VALIDATION.md` - passing map, `nyquist_compliant: true`
|
||||||
|
|
||||||
|
## Per-package coverage (contract branches)
|
||||||
|
|
||||||
|
| Package | Coverage |
|
||||||
|
|---------|----------|
|
||||||
|
| compass | 80.3% |
|
||||||
|
| party | 79.3% |
|
||||||
|
| backpack | 84.0% |
|
||||||
|
| festival | 94.5% |
|
||||||
|
| towel | 100.0% |
|
||||||
|
| pact | no statements (interfaces only; tests assert type discovery) |
|
||||||
|
|
||||||
|
## Measured rebuild
|
||||||
|
|
||||||
|
- `scripts/check-phase1.sh` `summer build` of examples/hello: **built hello in 80ms** (warm cache).
|
||||||
|
- `TestWatchHelloWorkspaceRebuildLatency` observed `rebuild: <duration>` after a greeter source edit and a child restart; no latency threshold was asserted. Generated `main.go` / `plugins.gen.go` edits did not loop.
|
||||||
|
|
||||||
|
## Decisions Made
|
||||||
|
|
||||||
|
- Cover observable contracts (precedence winners, error IDs, captured CLI bytes) rather than chasing a percentage
|
||||||
|
- Keep watch production writers as-is; serialize test buffers so `-race` can inspect output
|
||||||
|
- Split VALIDATION.md into a docs commit because CLAUDE.md forbids mixing planning docs with code
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
### Auto-fixed Issues
|
||||||
|
|
||||||
|
**1. [Rule 1 - Bug] Watch tests raced on bytes.Buffer under -race**
|
||||||
|
- **Found during:** Task 2 (`go test -race ./internal/dev`)
|
||||||
|
- **Issue:** The watch loop writes captured output while the test goroutine called `Buffer.String()`
|
||||||
|
- **Fix:** Added a mutex-wrapped `safeBuffer` used by all concurrent watch tests
|
||||||
|
- **Files modified:** `internal/dev/watch_test.go`
|
||||||
|
- **Verification:** `go test -race ./internal/dev` exits 0
|
||||||
|
- **Committed in:** `dc7997e` (Task 2)
|
||||||
|
|
||||||
|
**2. [Rule 2 - Missing Critical] Pact capability discovery tests**
|
||||||
|
- **Found during:** Task 1 (KERN-03 HasConfig/HasCommands assertions; verify includes `./pact`)
|
||||||
|
- **Issue:** Plan file list omitted `pact/capabilities_test.go` while KERN-03 requires type-assertion discovery
|
||||||
|
- **Fix:** Added compile-time/runtime discovery tests for HasConfig, HasCommands, OptionalMessage
|
||||||
|
- **Files modified:** `pact/capabilities_test.go`
|
||||||
|
- **Verification:** `go test -race ./pact` exits 0
|
||||||
|
- **Committed in:** `a43c734` (Task 1)
|
||||||
|
|
||||||
|
**3. [CLAUDE.md - Commit split] VALIDATION.md committed separately from test code**
|
||||||
|
- **Found during:** Task 2
|
||||||
|
- **Issue:** Task files mixed tests with a planning doc; CLAUDE.md requires planning docs and code in separate commits
|
||||||
|
- **Fix:** `dc7997e` for tests/script, `bee03bd` for VALIDATION.md
|
||||||
|
- **Files modified:** `.planning/phases/01-framework-kernel-foundation/01-VALIDATION.md`
|
||||||
|
- **Verification:** two commits, no co-author trailers
|
||||||
|
- **Committed in:** `bee03bd`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Total deviations:** 3 auto-fixed (1 bug, 1 missing critical, 1 commit-policy)
|
||||||
|
**Impact on plan:** Required for green `-race`, KERN-03 evidence, and CLAUDE.md. No production API change.
|
||||||
|
|
||||||
|
## Issues Encountered
|
||||||
|
|
||||||
|
- Nested plugin modules `base` and `optional` have no test files; `go test` reports `[no test files]` and still exits 0. Greeter has `plugin_test.go`.
|
||||||
|
- `go.work.sum` remains untracked in this worktree (pre-existing, not part of the plan).
|
||||||
|
|
||||||
|
## Authentication Gates
|
||||||
|
|
||||||
|
None.
|
||||||
|
|
||||||
|
## User Setup Required
|
||||||
|
|
||||||
|
None - no external service configuration required.
|
||||||
|
|
||||||
|
## Next Phase Readiness
|
||||||
|
|
||||||
|
Phase 1 kernel contracts have a repeatable race-enabled check. Ready for verify-work and later phases that depend on compass/party/backpack/festival/bonfire/`summer build`/`summer dev`. Do not add HTTP/DB/auth here.
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
|
|
||||||
|
- Created files exist: `pact/capabilities_test.go`, `scripts/check-phase1.sh`, this SUMMARY
|
||||||
|
- Commits `a43c734`, `dc7997e`, `bee03bd` exist on `worktree-agent-01-04`
|
||||||
|
- `bash scripts/check-phase1.sh` exited 0 and named root, hello app, base, greeter, optional
|
||||||
|
- STATE.md and ROADMAP.md were not updated in this worktree
|
||||||
|
|
||||||
|
---
|
||||||
|
*Phase: 01-framework-kernel-foundation*
|
||||||
|
*Completed: 2026-09-16*
|
||||||
Reference in New Issue
Block a user