23 KiB
23 KiB
phase, slug, status, nyquist_compliant, wave_0_complete, created
| phase | slug | status | nyquist_compliant | wave_0_complete | created |
|---|---|---|---|---|---|
| 19.7 | generic-mcp-module | validated | true | true | 2026-10-21 |
Phase 19.7 — Validation Strategy
Per-phase validation contract for feedback sampling during execution. Source:
19.7-RESEARCH.md§ Validation Architecture.
Test Infrastructure
| Property | Value |
|---|---|
| Framework | Go testing + testcontainers-go (Postgres) + miniredis + go-sdk client; vitest for the admin SPA seam |
| Config file | none (Go); admin/vitest.config.* for the SPA |
| Quick run command | go test -short ./modules/pier ./modules/surf ./modules/bouncer ./modules/cabana ./modules/wristband ./modules/pact |
| Full suite command | go vet ./... && go test ./... (framework) + go test ./... in each touched plugin + npm --prefix admin run test + scripts/check-phase19.7.sh |
| Estimated runtime | ~120 seconds (quick ~20 s) |
Sampling Rate
- After every task commit: quick run command for the touched module(s)
- After every plan wave: framework
go vet ./... && go test ./...plus the touched plugin suites - Before
/msd-verify-work:bash scripts/check-phase19.7.sh --allgreen on a Docker host (every stage PASS, no NOT RUN docker) - Max feedback latency: 120 seconds
Per-Task Verification Map
Filled in by the planner per task; the behaviour → test map is:
| Ref | Behavior | Test Type | Automated Command | File Exists | Status |
|---|---|---|---|---|---|
| SC-1/D-30/D-33 | disabled → nothing mounted; enabled without the tag → boot error; tagged → /mcp stream routes and PRM documents; collision names the owner |
unit | go test -short ./modules/surf -run '^TestPierMount$' and go test -short -tags summercms_mcp ./modules/surf -run '^TestPierMount$' |
✅ modules/surf/pier_mount_test.go |
✅ green |
| SC-1 | SSE stream flushes through the streaming route kind; throttle, CORS profile and access-log pattern kept; buffered Flush a no-op | unit | go test -short ./modules/surf -run '^TestStreamingRoute$' |
✅ modules/surf/streaming_test.go |
✅ green |
| SC-2/D-10..D-12/D-14 | binding/duplicate/name boot errors; prefixing; annotations by method | unit | go test -short ./modules/pier -run '^TestCatalog$' |
✅ modules/pier/catalog_test.go |
✅ green |
| D-11/D-13 | arg split, omitted fields absent, no outer credentials, isError with the route body, {"data":…}, size cap, call timeout |
unit | go test -short ./modules/pier -run '^TestDispatch$' |
✅ modules/pier/dispatch_test.go |
✅ green |
| D-15/D-16/D-22 | non-token routes refused; requirement union; method-aware provider; admin token scope per method | unit | go test -short ./modules/pier -run '^TestGuardRules$'; go test -short ./modules/cabana -run '^TestScopeLimitForMethods$' |
✅ modules/pier/guard_rules_test.go, modules/cabana/scope_limit_for_test.go |
✅ green |
| D-17 | marker honoured in-process only; not forgeable from HTTP; frontend marker refused by the backend guard and ResolveBackend |
unit + Postgres | go test -short ./modules/bouncer ./modules/cabana -run '^TestDispatchMarker$'; Postgres part go test ./modules/cabana -run '^TestDispatchMarker$'; gate stage marker |
✅ modules/bouncer/dispatch_test.go, modules/cabana/dispatch_marker_test.go |
✅ green |
| SC-3/D-20/D-23 | per-principal tools/list; read grant hides write tools; no subject, no tool; unlisted call never reaches the route; union of guards | integration | go test -short ./modules/pier -run '^TestToolsList$' |
✅ modules/pier/tools_list_test.go |
✅ green |
| D-25 | session binding, DELETE, cross-instance rehydration, TTL, per-binding cap (Redis and memory) | integration | go test -short ./modules/pier -run '^TestSessions$' |
✅ modules/pier/sessions_test.go |
✅ green |
| D-26/D-27 | pub/sub list_changed; link change and cache bypass; revocation and deletion end sessions and evict stateless servers | integration | go test -short ./modules/pier -run '^TestFanOut$' |
✅ modules/pier/fanout_test.go |
✅ green |
| D-09 | plugin frontend permission codes in editors; duplicates dropped; saved codes granted | Postgres | user plugin GOWORK=off go test . -run '^TestFrontendPermissionCatalog$' |
✅ plugins/golem15/user/frontend_permission_catalog_test.go |
✅ green |
| D-03..D-08 | link proof, one-shot, fresh sign-in at the second, 1:1, admin set, events, per-request lookup | Postgres | user plugin GOWORK=off go test . -run '^TestBackendLink$' |
✅ plugins/golem15/user/backend_link_test.go |
✅ green |
| D-15/D-35 | user guard: parser refusals, deleted user, expired token, dispatch credential against user.scope |
Postgres | user plugin GOWORK=off go test . -run '^TestUserMCPGuard$' |
✅ plugins/golem15/user/mcp_guard_test.go |
✅ green |
| D-04 | account contribution seam; Applies and permission filtering; 409; interactive only; transaction; existing payloads unchanged | unit + Postgres + vitest | go test -short ./modules/cabana -run '^TestAccountContributions$' (Postgres part without -short); npm --prefix admin run test; gate check account.go unchanged since 51d088a |
✅ modules/cabana/account_contributions_test.go, admin/tests/account/AccountContributions.test.ts |
✅ green |
| D-34 | prompted record actions: localized prompt, trim and cap, refusals, empty Value for a plain action | unit + Postgres + vitest | go test -short ./modules/cabana -run '^TestRecordActionPrompt$'; go test ./modules/cabana -run '^TestRecordActionPromptRun$'; npm --prefix admin run test |
✅ modules/cabana/record_action_prompt_test.go, admin/tests/form/RecordActionsPrompt.test.ts |
✅ green |
| D-21 | backend scope and consent line |
unit + Postgres | go test -short ./modules/wristband -run '^TestMCPScopes$'; oauthserver plugin go test . -run '^TestMCPConsent$' |
✅ modules/wristband/mcp_scopes_test.go, sm-oauthserver-plugin/mcp_consent_test.go |
✅ green |
| SC-4 | oas_at_ verifier refuses OIDC/revoked/expired/foreign/inactive-client/deleted-user tokens; conditional stamp |
Postgres | oauthserver plugin go test . -run '^TestMCPTokenVerifier$' |
✅ sm-oauthserver-plugin/mcp_verifier_test.go |
✅ green |
| D-15/D-16 | Journal tools compile through pier; input keys are the routes' keys; prompt | unit | blog plugin go test -short . -run '^TestJournalMCP$' |
✅ sm-journal-plugin/mcp_test.go |
✅ green |
| SC-4/D-08/D-18/D-20/D-26 | end-to-end in sm-demo-app incl. unlink list_changed, read grant, revoked grant, personal tokens off | e2e | cd ../sm-demo-app && go test -tags summercms_mcp . -run '^TestMCPEndToEnd$' |
✅ sm-demo-app/mcp_e2e_test.go |
✅ green |
| D-31 | plugins additive against their pre-phase commits; consumers build and pass at bumped pins | build | bash scripts/check-phase19.7.sh --plugins and --consumers |
✅ scripts/check-phase19.7.sh |
✅ green |
| A-CR-01/A-IN-01/A-IN-06 | reaper closes expired copies (memory, Redis per instance), store error keeps the copy, GET stream ends when its record is gone, ephemeral label |
integration | go test -short ./modules/pier -run '^TestSessionReaper$' (also -tags summercms_mcp) |
✅ modules/pier/reaper_test.go |
✅ green |
| A-WR-03 | 50 concurrent initializes open exactly max_per_binding (memory, two instances on miniredis); a failed initialize releases its slot; Count/Put fallback |
integration | go test -short ./modules/pier -run '^TestSessionReserve$' |
✅ modules/pier/reserve_test.go |
✅ green |
| A-WR-07 | a call, a DELETE and a stateless request finish while an initialize blocks in a guard check | integration | go test -short -race ./modules/pier -run '^TestLockScope$' |
✅ modules/pier/lock_scope_test.go |
✅ green |
| A-WR-04 | a returning binding with read write, or with a backend subject, lists the new tools on both paths; narrowing hides them | integration | go test -short ./modules/pier -run '^TestScopeWidening$' |
✅ modules/pier/widening_test.go |
✅ green |
| A-WR-05/A-WR-06 | a stateless-only Redis instance applies a revocation; a listen stream ends with its bearer; LRU eviction closes the stream | integration | go test -short ./modules/pier -run '^TestStatelessFanOut$' |
✅ modules/pier/stateless_fanout_test.go |
✅ green |
| A-WR-09 | a subject without TokenID is 401; two users' personal tokens get distinct bindings | integration | go test -short ./modules/pier -run '^TestPersonalTokenBinding$' |
✅ modules/pier/personal_binding_test.go |
✅ green |
| A-WR-01/A-WR-02/A-WR-08/A-IN-04/A-IN-05/A-IN-08 | traversal and empty segments refused before the route; missing path value refused; fixed provider error text; large integers keep their digits; outer Host and forwarding headers; contradicting dispatch refused | unit | go test -short ./modules/pier -run '^TestDispatchHardening$' |
✅ modules/pier/dispatch_hardening_test.go |
✅ green |
| A-WR-02/A-IN-07 | an omitempty path field and an embedded struct on a GET tool fail boot | unit | go test -short ./modules/pier -run '^TestCatalogPathFields$' |
✅ modules/pier/catalog_path_fields_test.go |
✅ green |
| C-IN-01 | a Writes prompt is listed only for read and write grants |
integration | go test -short ./modules/pier -run '^TestPromptWriteCeiling$'; blog plugin go test -short . -run '^TestJournalPromptWrites$' |
✅ modules/pier/prompt_ceiling_test.go, sm-journal-plugin/mcp_prompt_writes_test.go |
✅ green |
| B-WR-04/C-IN-02 | a TokenID event ends only that token's sessions, a ClientID event only that client's, neither all; through Redis between instances | integration | go test -short ./modules/pier -run '^TestTokenScopedRevoke$' |
✅ modules/pier/token_events_test.go |
✅ green |
| B-IN-04 | a resolver sees towel.ClientIP (RemoteAddr host, or the host's trusted-proxy answer); towel round trip; ResolveBackendToken logs remote |
unit + Postgres | go test -short ./modules/pier -run '^TestResolveClientAddress$'; go test -short ./modules/towel -run '^TestClientIP$'; go test ./modules/cabana -run '^TestResolveBackendTokenRemote$' |
✅ modules/pier/token_events_test.go, modules/towel/client_ip_test.go, modules/cabana/token_resolver_remote_test.go |
✅ green |
| C-CR-02/D-05 | auth_time at mint (both audiences), unchanged through every refresh path, absent for legacy tokens; VerifyAuthTime refusals |
unit | go test -short ./modules/bouncer -run '^TestAuthTime$' |
✅ modules/bouncer/auth_time_test.go |
✅ green |
| E-2 (framework) | ValidTokenScopeName table; HasFrontendTokenScopes discovered; cabana's scope validation agrees with pact |
unit | go test -short ./modules/pact -run '^TestFrontendTokenScopes$'; go test -short ./modules/cabana -run '^TestTokenScopeNameParity$' |
✅ modules/pact/frontend_token_scopes_test.go, modules/cabana/token_scope_name_test.go |
✅ green |
| B-WR-03 | javascript:, data:, vbscript: and // action URLs dropped and logged without the URL; https and same-origin kept |
unit | go test -short ./modules/cabana -run '^TestAccountContributionURL$' |
✅ modules/cabana/account_contributions_hardening_test.go |
✅ green |
| B-WR-06/B-IN-03 | a failing Status marks only its section, a failing Applies drops only its action, 200; dynamic status text not looked up | unit | go test -short ./modules/cabana -run '^TestAccountContributionsIsolation$' |
✅ modules/cabana/account_contributions_hardening_test.go |
✅ green |
| B-IN-02 | a non-string prompt payload answers the localized action_prompt_invalid text (en, pl) |
unit | go test -short ./modules/cabana -run '^TestRecordActionPromptLocalized$' |
✅ modules/cabana/record_action_prompt_localized_test.go |
✅ green |
| B-WR-04 | revoking one g15_ token fires one revoked event with its TokenID |
Postgres | go test ./modules/cabana -run '^TestAccountTokenRevokeEvent$' |
✅ modules/cabana/account_token_events_test.go |
✅ green |
| B-WR-01/B-WR-02 | plain keys ignored from an open shadow root and a custom field element in a form; Ctrl+S and Ctrl+Enter save the taglist draft | vitest | npm --prefix admin run test (tests/app/hotkeysEditing.test.ts required by the gate) |
✅ admin/tests/app/hotkeysEditing.test.ts |
✅ green |
| B-WR-03/B-WR-06/B-IN-01 | no link and no window.open for a javascript: URL; section error text; toast and kept sections on a failed reload; My account confirm and stale texts |
vitest | npm --prefix admin run test |
✅ admin/tests/account/AccountContributions.test.ts |
✅ green |
| E-2 | E-2's acceptance list item by item (mint and middleware.Scope, the 403 message, 422, listing, discovery in en and pl), per-application catalog, legacy fallback |
Postgres | user plugin GOWORK=off go test . -run '^TestMemberTokenScopes$' |
✅ plugins/golem15/user/member_token_scopes_test.go |
✅ green |
| E-2 | duplicate (both plugins named), redeclared built-ins, bad names, empty label and a missing phrase key fail boot | unit | user plugin GOWORK=off go test -short . -run '^TestMemberTokenScopesBoot$' |
✅ plugins/golem15/user/member_token_scopes_test.go |
✅ green |
| C-CR-02 | the link proof against login, request, refresh and re-login orders, no auth_time, another user's session token |
Postgres | user plugin GOWORK=off go test . -run '^TestBackendLinkAuthTime$' |
✅ plugins/golem15/user/backend_link_auth_time_test.go |
✅ green |
| C-WR-01/C-IN-06 | POST show answers like GET with no-store; fragment URL; backend_links.page_path |
Postgres | user plugin GOWORK=off go test . -run '^TestBackendLinkShowPost$' |
✅ plugins/golem15/user/backend_link_show_test.go |
✅ green |
| C-CR-01 | AccessRevokedEvent from ban, reset, change and admin-set password (ids, Before, reason, transaction, subject event after commit, rollback); banned user loses the link |
Postgres | user plugin GOWORK=off go test . -run '^TestAccessRevocation$' |
✅ plugins/golem15/user/access_revocation_test.go |
✅ green |
| C-IN-04 | MCP personal-token resolve stamps last_used_ip |
Postgres | user plugin GOWORK=off go test . -run '^TestUserMCPGuardClientIP$' |
✅ plugins/golem15/user/mcp_guard_client_ip_test.go |
✅ green |
| C-IN-02 | DELETE /tokens/{id} fires one revoked event with that TokenID |
Postgres | user plugin GOWORK=off go test . -run '^TestAPITokenRevokeEvent$' |
✅ plugins/golem15/user/api_token_events_test.go |
✅ green |
| E-1 | logo fileupload after the avatar; upload stored with field logo; OrganisationLogoFields with and without a logo; delete removes both attachments |
Postgres | user plugin GOWORK=off go test . -run '^TestOrganisationLogo$' |
✅ plugins/golem15/user/organisation_logo_test.go |
✅ green |
| C-CR-01 | the verifier and refresh refuse pre-cutoff (same-second), banned and deleted users without a revoke listener; untruncated Before |
Postgres | oauthserver plugin go test . -run '^TestMCPVerifierRevocation$' |
✅ sm-oauthserver-plugin/mcp_revocation_test.go |
✅ green |
| C-CR-01/C-WR-02/C-IN-02 | MCP and OIDC grants before Before revoked, later kept; consents and unused codes deleted; link changes revoke only backend tokens with client events after commit; authorized-apps revoke event |
Postgres | oauthserver plugin go test . -run '^TestMCPGrantRevocation$' |
✅ sm-oauthserver-plugin/mcp_revocation_test.go |
✅ green |
| SC-4/C-CR-01/C-WR-02/C-IN-01/C-IN-02 | demo revocation matrix through the SDK client: ban via the bulk route, unlink and a new consent, read-only prompts, one usr_ token revoked |
e2e | cd ../sm-demo-app && go test -tags summercms_mcp . -run '^TestMCPRevocationEndToEnd$' |
✅ sm-demo-app/mcp_revocation_e2e_test.go |
✅ green |
| C-WR-01 | readLinkCode and isLinkCode |
vitest | pnpm --dir ../sm-demo-app/vue-demo-app run test (app/lib/linkCode.test.ts required by the gate) |
✅ sm-demo-app/vue-demo-app/app/lib/linkCode.test.ts |
✅ green |
| T-19.7-64 (P-40) | workspace check, consumers built against the unreleased user plugin, unreleased-pin policy and PHASE197_RELEASED strict mode |
gate | bash scripts/check-phase19.7.sh --self-test, --plugins, --consumers; PHASE197_RELEASED=1 bash scripts/check-phase19.7.sh --consumers exits 1 before the release |
✅ scripts/check-phase19.7.sh |
✅ green |
| A-IN-02 | cooler's KeyPrefix and keyPrefix carry their own doc comments |
docs | bash scripts/check-phase19.7.sh --docs |
✅ scripts/check-phase19.7.sh |
✅ green |
| RA-WR-01 | fan-out never waits on a session lock under the router lock; re-filter guard checks bounded by mcp.call_timeout |
integration | go test -short -race ./modules/pier -run '^TestLockScopeUnderWidening$' |
✅ modules/pier/lock_scope_test.go |
✅ green |
| RA-WR-02 | the stateless fingerprint ignores the OAuth token id; a narrow token does not flip a shared server's tool set mid-call | unit + integration | go test -short ./modules/pier -run '^(TestFingerprintIgnoresTokenID|TestStatelessSharedBinding)$' |
✅ modules/pier/fingerprint_test.go, modules/pier/stateless_fanout_test.go |
✅ green |
| RA-WR-03 | a guard error while widening or recomputing (stateless, stateful, on an event) does not narrow the session | integration | go test -short ./modules/pier -run '^TestWideningGuardError$' |
✅ modules/pier/widening_test.go |
✅ green |
| RA-WR-04 | LRU eviction spares a server with a listen stream; a revocation still ends the stream | integration | go test -short ./modules/pier -run '^TestStatelessFanOut$' |
✅ modules/pier/stateless_fanout_test.go |
✅ green |
| RA-WR-05 | prompt Render errors stay out of the client's answer (stateful and stateless) | integration | go test -short ./modules/pier -run '^TestPromptRenderError$' |
✅ modules/pier/prompt_ceiling_test.go |
✅ green |
| RA-IN-02 | stream bearer re-checks carry the client address | integration | go test -short ./modules/pier -run '^TestStreamRecheckClientIP$' |
✅ modules/pier/token_events_test.go |
✅ green |
| RA-IN-04 | the non-reserving fallback writes the record before building the server; a failed initialize deletes it (Count-then-Put race deferred) | integration | go test -short ./modules/pier -run '^TestSessionReserve$' |
✅ modules/pier/reserve_test.go |
✅ green |
| RA-IN-05 | an admin personal token whose scopes all resolve to nothing gets "the token's scopes no longer grant this route", not "account unknown"; the listing check is a refusal; known scope, missing admin and OAuth subject unchanged | Postgres | go test ./modules/pier -run '^TestBackendScopeUnavailable$' |
✅ modules/pier/backend_scope_unavailable_test.go |
✅ green |
| RA-WR-06/RA-IN-06 | the verdict is qualified while demo pins are unreleased; a silent failing pin check is refused | gate | bash scripts/check-phase19.7.sh --self-test (pins-test-only-unrecorded, pins-silent-failure) |
✅ scripts/check-phase19.7.sh |
✅ green |
| RB-WR-01/RB-IN-01 | Ctrl+S and Ctrl+Enter commit a draft inside an open or closed field-type shadow root; control-character and bare https:// URLs render no link and open no window |
vitest | npm --prefix admin run test |
✅ admin/tests/app/hotkeysEditing.test.ts, admin/tests/account/AccountContributions.test.ts |
✅ green |
| RB-WR-02 | a padded or whitespace-carrying token scope name fails boot | unit | go test -short ./modules/cabana -run '^TestTokenScopeNameParity$' |
✅ modules/cabana/token_scope_name_test.go |
✅ green |
| RB-IN-02 | a failing Applies marks its My account section | unit | go test -short ./modules/cabana -run '^TestAccountContributionsAppliesFailureFlagsSection$' |
✅ modules/cabana/account_contributions_hardening_test.go |
✅ green |
| RC-WR-01 | a backend grant acts only as the administrator it was consented for: decide 409 matrix, binding through code, exchange, rotation and verifier, link change, revoke racing a rotation | Postgres + integration + e2e | oauthserver plugin go test . -run '^TestMCPBackendBinding$'; go test -short ./modules/pier -run '^(TestToolsList|TestFanOut)$'; demo TestMCPEndToEnd |
✅ sm-oauthserver-plugin, modules/pier/tools_list_test.go, modules/pier/fanout_test.go, sm-demo-app/mcp_e2e_test.go |
✅ green |
| RC-CR-01/RC-WR-02 | a self-service password change revokes grants rotated after the session's iat and keeps the session; a ban alone refuses the verifier and rotation | Postgres | user plugin GOWORK=off go test . -run '^TestAccessRevocation$'; oauthserver plugin go test . -run '^(TestMCPGrantRevocation|TestMCPVerifierRevocation)$' |
✅ plugins/golem15/user/access_revocation_test.go, sm-oauthserver-plugin/mcp_revocation_test.go |
✅ green |
| RC-IN-01/RC-IN-03 | a link code in the query string is ignored on POST; avatar and logo fallbacks use the public path | Postgres | user plugin GOWORK=off go test . -run '^(TestBackendLinkShowPost|TestAvatarURL|TestOrganisationLogo)$' |
✅ plugins/golem15/user/backend_link_show_test.go, plugins/golem15/user/organisation_logo_test.go |
✅ green |
Status legend: ✅ green · ❌ red · ⚠️ flaky. Every row is green as of the plan 09 gate run (bash scripts/check-phase19.7.sh --all, every stage PASS). The rows after D-31 are the gap closure (plans 06-09); the RA-/RB-/RC- rows are the re-review fixes (19.7-REREVIEW-FIX.md), added by the 2026-10-24 validation audit.
Wave 0 Requirements
scripts/check-phase19.7.sh— phase gate (pattern:scripts/check-phase19.4.sh)- pier test helpers: fake guard provider, fake verifier, two-instance harness on one miniredis, SDK client with a bearer-injecting
http.Client(modules/pier/helpers_test.go) - user-plugin Postgres fixtures for links and frontend permission rows (the admin harness,
startLink/confirmLink, the seeded permission rows) - additive-API checks for sm-user-plugin, sm-journal-plugin, sm-oauthserver-plugin (gate stage
plugins)
Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|---|---|---|---|
| Real MCP client (Claude) connects once via OAuth and sees both tool sets | SC-4 | external client and browser consent | Add the demo's /mcp URL to a client, sign in, link an admin, request backend, call a Journal tool |
go-sdk legitimacy check before go get |
D-24 | supply-chain human check | Confirm module path, version and maintainers in the decision note |
Validation Sign-Off
- All tasks have
<automated>verify or Wave 0 dependencies - Sampling continuity: no 3 consecutive tasks without automated verify
- Wave 0 covers all MISSING references
- No watch-mode flags
- Feedback latency < 120s for the quick run (the full gate with Docker takes several minutes)
- Nyquist compliance recorded in the frontmatter
Approval: plan 19.7-05 (the gate passes in full); extended by plan 19.7-09 for the gap closure (the gate passes in full)
Validation Audit 2026-10-24
The audit compared the map with the re-review fixes (19.7-REREVIEW-FIX.md). Fifteen behaviours had tests but no row, and they are now in the map. RA-IN-01, RA-IN-03, RB-IN-04 and RC-IN-02 changed only documentation, so they get no row. RA-IN-05 had no test, and TestBackendScopeUnavailable (49a3df6) now covers it.
| Metric | Count |
|---|---|
| Gaps found | 1 |
| Resolved | 1 |
| Escalated | 0 |