From 39c0c42db71d4f1c83efff90981059c3f20b1c81 Mon Sep 17 00:00:00 2001 From: Jakub Zych Date: Wed, 21 Oct 2026 17:38:00 +0200 Subject: [PATCH] docs(phase-19.7): add validation strategy --- .../19.7-VALIDATION.md | 93 +++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 .planning/phases/19.7-generic-mcp-module/19.7-VALIDATION.md diff --git a/.planning/phases/19.7-generic-mcp-module/19.7-VALIDATION.md b/.planning/phases/19.7-generic-mcp-module/19.7-VALIDATION.md new file mode 100644 index 00000000..ff8eb141 --- /dev/null +++ b/.planning/phases/19.7-generic-mcp-module/19.7-VALIDATION.md @@ -0,0 +1,93 @@ +--- +phase: "19.7" +slug: "generic-mcp-module" +# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6) +# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117) +status: draft +nyquist_compliant: false +wave_0_complete: false +created: "2026-10-21" +--- + +# Phase 19.7 — Validation Strategy + +> Per-phase validation contract for feedback sampling during execution. Source: `19.7-RESEARCH.md` § Validation Architecture. + +--- + +## Test Infrastructure + +| Property | Value | +|----------|-------| +| **Framework** | Go `testing` + testcontainers-go (Postgres) + miniredis + go-sdk client; vitest for the admin SPA seam | +| **Config file** | none (Go); `admin/vitest.config.*` for the SPA | +| **Quick run command** | `go test -short ./modules/pier ./modules/surf ./modules/bouncer ./modules/cabana ./modules/wristband ./modules/pact` | +| **Full suite command** | `go vet ./... && go test ./...` (framework) + `go test ./...` in each touched plugin + `npm --prefix admin run test` + `scripts/check-phase19.7.sh` | +| **Estimated runtime** | ~120 seconds (quick ~20 s) | + +--- + +## Sampling Rate + +- **After every task commit:** quick run command for the touched module(s) +- **After every plan wave:** framework `go vet ./... && go test ./...` plus the touched plugin suites +- **Before `/msd-verify-work`:** `scripts/check-phase19.7.sh` green +- **Max feedback latency:** 120 seconds + +--- + +## Per-Task Verification Map + +Filled in by the planner per task; the behaviour → test map is: + +| Ref | Behavior | Test Type | Automated Command | File Exists | Status | +|-----|----------|-----------|-------------------|-------------|--------| +| SC-1/D-30 | disabled → nothing mounted; enabled → `/mcp` mounted | unit | `go test -short ./modules/surf -run TestPierMount` | ❌ W0 | ⬜ pending | +| SC-1 | SSE stream flushes through the streaming route kind | unit | `go test -short ./modules/surf -run TestStreamingRoute` | ❌ W0 | ⬜ pending | +| SC-2/D-10..D-12 | binding/duplicate/name boot errors; prefixing | unit | `go test -short ./modules/pier -run TestCatalog` | ❌ W0 | ⬜ pending | +| D-11/D-13/D-14 | arg split, isError with route body, annotations | unit | `go test -short ./modules/pier -run TestDispatch` | ❌ W0 | ⬜ pending | +| D-15/D-16 | non-token routes refused; requirement union | unit | `go test -short ./modules/pier -run TestGuardRules` | ❌ W0 | ⬜ pending | +| D-17 | marker honoured in-process only; not forgeable from HTTP; frontend marker refused by backend guard | unit | `go test -short ./modules/bouncer ./modules/cabana -run TestDispatchMarker` | ❌ W0 | ⬜ pending | +| SC-3/D-20/D-23 | per-principal tools/list; read grant hides write tools | integration | `go test -short ./modules/pier -run TestToolsList` | ❌ W0 | ⬜ pending | +| D-25 | session binding, DELETE, cross-instance rehydration, TTL | integration | `go test -short ./modules/pier -run TestSessions` | ❌ W0 | ⬜ pending | +| D-26/D-27 | pub/sub list_changed; revocation ends session | integration | `go test -short ./modules/pier -run TestFanOut` | ❌ W0 | ⬜ pending | +| D-09 | plugin frontend permission codes in editors | Postgres | user plugin `go test ./... -run TestFrontendPermissionCatalog` | ❌ W0 | ⬜ pending | +| D-03..D-08 | link proof, one-shot, 1:1, admin set, per-request lookup | Postgres | user plugin `go test ./... -run TestBackendLink` | ❌ W0 | ⬜ pending | +| D-04 | account contribution seam; existing payloads unchanged | unit + vitest | `go test -short ./modules/cabana -run TestAccountContributions`; `npm --prefix admin run test` | ❌ W0 | ⬜ pending | +| D-21 | `backend` scope and consent line | unit + Postgres | `go test -short ./modules/wristband -run TestMCPScopes`; plugin `-run TestMCPConsent` | ❌ W0 | ⬜ pending | +| SC-4 | `oas_at_` verifier refuses OIDC/revoked/expired/foreign tokens | Postgres | oauthserver plugin `go test ./... -run TestMCPTokenVerifier` | ❌ W0 | ⬜ pending | +| SC-4/D-18 | end-to-end in sm-demo-app incl. unlink removes tools | e2e | `cd ../sm-demo-app && go test ./... -run TestMCPEndToEnd` | ❌ W0 | ⬜ pending | +| D-31 | consumers build and pass at bumped pins | build | `scripts/check-phase19.7.sh` stage `consumers` | ❌ W0 | ⬜ pending | + +*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky* + +--- + +## Wave 0 Requirements + +- [ ] `scripts/check-phase19.7.sh` — phase gate (pattern: `scripts/check-phase19.4.sh`) +- [ ] pier test helpers: fake guard provider, fake verifier, two-instance harness on one miniredis, SDK client with a bearer-injecting `http.Client` +- [ ] user-plugin Postgres fixtures for links and frontend permission rows +- [ ] additive-API checks for sm-user-plugin, sm-journal-plugin, sm-oauthserver-plugin + +--- + +## Manual-Only Verifications + +| Behavior | Requirement | Why Manual | Test Instructions | +|----------|-------------|------------|-------------------| +| Real MCP client (Claude) connects once via OAuth and sees both tool sets | SC-4 | external client and browser consent | Add the demo's `/mcp` URL to a client, sign in, link an admin, request `backend`, call a Journal tool | +| go-sdk legitimacy check before `go get` | D-24 | supply-chain human check | Confirm module path, version and maintainers in the decision note | + +--- + +## Validation Sign-Off + +- [ ] All tasks have `` verify or Wave 0 dependencies +- [ ] Sampling continuity: no 3 consecutive tasks without automated verify +- [ ] Wave 0 covers all MISSING references +- [ ] No watch-mode flags +- [ ] Feedback latency < 120s +- [ ] `nyquist_compliant: true` set in frontmatter + +**Approval:** pending