docs(phase-19.7): add validation strategy
This commit is contained in:
93
.planning/phases/19.7-generic-mcp-module/19.7-VALIDATION.md
Normal file
93
.planning/phases/19.7-generic-mcp-module/19.7-VALIDATION.md
Normal file
@@ -0,0 +1,93 @@
|
||||
---
|
||||
phase: "19.7"
|
||||
slug: "generic-mcp-module"
|
||||
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
|
||||
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
wave_0_complete: false
|
||||
created: "2026-10-21"
|
||||
---
|
||||
|
||||
# Phase 19.7 — Validation Strategy
|
||||
|
||||
> Per-phase validation contract for feedback sampling during execution. Source: `19.7-RESEARCH.md` § Validation Architecture.
|
||||
|
||||
---
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Framework** | Go `testing` + testcontainers-go (Postgres) + miniredis + go-sdk client; vitest for the admin SPA seam |
|
||||
| **Config file** | none (Go); `admin/vitest.config.*` for the SPA |
|
||||
| **Quick run command** | `go test -short ./modules/pier ./modules/surf ./modules/bouncer ./modules/cabana ./modules/wristband ./modules/pact` |
|
||||
| **Full suite command** | `go vet ./... && go test ./...` (framework) + `go test ./...` in each touched plugin + `npm --prefix admin run test` + `scripts/check-phase19.7.sh` |
|
||||
| **Estimated runtime** | ~120 seconds (quick ~20 s) |
|
||||
|
||||
---
|
||||
|
||||
## Sampling Rate
|
||||
|
||||
- **After every task commit:** quick run command for the touched module(s)
|
||||
- **After every plan wave:** framework `go vet ./... && go test ./...` plus the touched plugin suites
|
||||
- **Before `/msd-verify-work`:** `scripts/check-phase19.7.sh` green
|
||||
- **Max feedback latency:** 120 seconds
|
||||
|
||||
---
|
||||
|
||||
## Per-Task Verification Map
|
||||
|
||||
Filled in by the planner per task; the behaviour → test map is:
|
||||
|
||||
| Ref | Behavior | Test Type | Automated Command | File Exists | Status |
|
||||
|-----|----------|-----------|-------------------|-------------|--------|
|
||||
| SC-1/D-30 | disabled → nothing mounted; enabled → `/mcp` mounted | unit | `go test -short ./modules/surf -run TestPierMount` | ❌ W0 | ⬜ pending |
|
||||
| SC-1 | SSE stream flushes through the streaming route kind | unit | `go test -short ./modules/surf -run TestStreamingRoute` | ❌ W0 | ⬜ pending |
|
||||
| SC-2/D-10..D-12 | binding/duplicate/name boot errors; prefixing | unit | `go test -short ./modules/pier -run TestCatalog` | ❌ W0 | ⬜ pending |
|
||||
| D-11/D-13/D-14 | arg split, isError with route body, annotations | unit | `go test -short ./modules/pier -run TestDispatch` | ❌ W0 | ⬜ pending |
|
||||
| D-15/D-16 | non-token routes refused; requirement union | unit | `go test -short ./modules/pier -run TestGuardRules` | ❌ W0 | ⬜ pending |
|
||||
| D-17 | marker honoured in-process only; not forgeable from HTTP; frontend marker refused by backend guard | unit | `go test -short ./modules/bouncer ./modules/cabana -run TestDispatchMarker` | ❌ W0 | ⬜ pending |
|
||||
| SC-3/D-20/D-23 | per-principal tools/list; read grant hides write tools | integration | `go test -short ./modules/pier -run TestToolsList` | ❌ W0 | ⬜ pending |
|
||||
| D-25 | session binding, DELETE, cross-instance rehydration, TTL | integration | `go test -short ./modules/pier -run TestSessions` | ❌ W0 | ⬜ pending |
|
||||
| D-26/D-27 | pub/sub list_changed; revocation ends session | integration | `go test -short ./modules/pier -run TestFanOut` | ❌ W0 | ⬜ pending |
|
||||
| D-09 | plugin frontend permission codes in editors | Postgres | user plugin `go test ./... -run TestFrontendPermissionCatalog` | ❌ W0 | ⬜ pending |
|
||||
| D-03..D-08 | link proof, one-shot, 1:1, admin set, per-request lookup | Postgres | user plugin `go test ./... -run TestBackendLink` | ❌ W0 | ⬜ pending |
|
||||
| D-04 | account contribution seam; existing payloads unchanged | unit + vitest | `go test -short ./modules/cabana -run TestAccountContributions`; `npm --prefix admin run test` | ❌ W0 | ⬜ pending |
|
||||
| D-21 | `backend` scope and consent line | unit + Postgres | `go test -short ./modules/wristband -run TestMCPScopes`; plugin `-run TestMCPConsent` | ❌ W0 | ⬜ pending |
|
||||
| SC-4 | `oas_at_` verifier refuses OIDC/revoked/expired/foreign tokens | Postgres | oauthserver plugin `go test ./... -run TestMCPTokenVerifier` | ❌ W0 | ⬜ pending |
|
||||
| SC-4/D-18 | end-to-end in sm-demo-app incl. unlink removes tools | e2e | `cd ../sm-demo-app && go test ./... -run TestMCPEndToEnd` | ❌ W0 | ⬜ pending |
|
||||
| D-31 | consumers build and pass at bumped pins | build | `scripts/check-phase19.7.sh` stage `consumers` | ❌ W0 | ⬜ pending |
|
||||
|
||||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||||
|
||||
---
|
||||
|
||||
## Wave 0 Requirements
|
||||
|
||||
- [ ] `scripts/check-phase19.7.sh` — phase gate (pattern: `scripts/check-phase19.4.sh`)
|
||||
- [ ] pier test helpers: fake guard provider, fake verifier, two-instance harness on one miniredis, SDK client with a bearer-injecting `http.Client`
|
||||
- [ ] user-plugin Postgres fixtures for links and frontend permission rows
|
||||
- [ ] additive-API checks for sm-user-plugin, sm-journal-plugin, sm-oauthserver-plugin
|
||||
|
||||
---
|
||||
|
||||
## Manual-Only Verifications
|
||||
|
||||
| Behavior | Requirement | Why Manual | Test Instructions |
|
||||
|----------|-------------|------------|-------------------|
|
||||
| Real MCP client (Claude) connects once via OAuth and sees both tool sets | SC-4 | external client and browser consent | Add the demo's `/mcp` URL to a client, sign in, link an admin, request `backend`, call a Journal tool |
|
||||
| go-sdk legitimacy check before `go get` | D-24 | supply-chain human check | Confirm module path, version and maintainers in the decision note |
|
||||
|
||||
---
|
||||
|
||||
## Validation Sign-Off
|
||||
|
||||
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
|
||||
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
|
||||
- [ ] Wave 0 covers all MISSING references
|
||||
- [ ] No watch-mode flags
|
||||
- [ ] Feedback latency < 120s
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter
|
||||
|
||||
**Approval:** pending
|
||||
Reference in New Issue
Block a user