feat(19.7-01): streaming routes, MCP mount hook, configurable authorize scopes

- surf.Router.MountStream and recoverStream: a raw, unbuffered route kind
  whose http.Flusher reaches the client; panic after the first byte only
  aborts the body
- surf.MCPBuildTag and the mountPier hook: mcp.enabled is inert without the
  summercms_mcp tag until it is true, then boot fails naming the tag
  (pier_off.go is the untagged side)
- wristband parseAuthorizeScopes takes the server's allowed set built from
  Options.ScopesSupported; the default set is unchanged
- cooler.KeyPrefix exported for the MCP module's Redis keys
- smoke tests TestStreamingRouteSmoke, TestPierMountSmoke, TestMCPScopesSmoke
- surf, wristband and cooler READMEs, docs/services/{routing,oauth-server}.md

modules/surf/README.md

@@ -40,7 +40,9 @@ Use cases:
 - Opt-in API documentation page: with `http.api_docs.enabled` true, `surf.BuildRouter` mounts `GET /api-docs` (a Swagger UI page) and `GET /api-docs.json` (owned by `summercms.apidocs`, public, no middleware). The spec comes from the one plugin implementing `surf.APIDocsProvider`, as YAML or JSON, converted once at boot to compact JSON in source key order; zero or several providers, or a spec that does not convert, fail boot. The page loads swagger-ui-dist `surf.SwaggerUIVersion` from jsDelivr with Subresource Integrity hashes on all three assets and a `Content-Security-Policy` whose only script source is a per-request nonce; every script tag, the two jsDelivr scripts included, carries it, so no other jsDelivr script can run.
 - Request log, on by default: every request writes one Info record `http request` from the [diary](../diary/README.md) component `http` after it is served, with `method`, `pattern` (the matched route, such as `GET /posts/{id}`), `status`, `duration_ms` and `request_id` (the `X-Request-Id` header, trimmed and cut to 128 bytes, left out when empty). A request whose client went away adds `client_gone=true`; its 5xx is logged as 499 (client closed request) only when the client had already gone when the status was written and the handler did not panic, so a 5xx written before the cancellation and a recovered panic keep their real status. An unmatched request (a 404 or 405 from the mux, or a CORS preflight answered before routing) has an empty `pattern`. The URL path, the query string, headers and the client address are never logged, for an unmatched request too: a token route requested with another method answers 405 before any route handler runs. Records go to the app's published `*slog.Logger`, else the `http` component logger, whose channels all redact through [sunscreen](../sunscreen/README.md). Silence it with `log.components.http: warn` or turn it off with `http.access_log: false`.
 - Startup summary and shutdown report for `serve`: once it listens, a block on a terminal (version, environment, URL and admin URL, database host and name, pending migrations, plugins, queue, scheduler, mail, cache, realtime and search drivers; `-v` adds one line per plugin) and, off a terminal, the same facts as a `startup summary` record and a `listening on <addr>` record of component `serve`; after SIGINT or SIGTERM, a report of requests served, drained and aborted, worker jobs, uptime and drain errors. Never a DSN user or password.
-- A read-only route table (`surf.Router.Routes`), the admin mount status (`surf.Router.AdminPrefix`) and the `serve` and `route:list` commands.
+- Streaming routes: `surf.Router.MountStream` registers a raw route whose response is not buffered, so the handler's `http.Flusher` reaches the client while it is still running (server-sent events, long-lived streams). Named middleware, `body.limit`, locale, the CORS profile and the access-log pattern apply as on a raw route; a panic before the first byte answers an opaque 500 and after it only cuts the body short. Used by the MCP module; plugin route groups stay buffered.
+- MCP module hook: with `mcp.enabled` true, `surf.BuildRouter` calls the mount hook the MCP module sets in a binary built with the `surf.MCPBuildTag` tag (`summercms_mcp` in `summer.yaml` `build.tags`), after every plugin route and the admin exist. Off (the default) mounts nothing and changes nothing; on without the tag fails boot with an error naming the tag.
+- A read-only route table (`surf.Router.Routes`), one route's compiled chain (`surf.Router.Handler`), the admin mount status (`surf.Router.AdminPrefix`) and the `serve` and `route:list` commands.
 
 ## Usage
 
@@ -127,6 +129,8 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur
 | `surf.Router.RegisterMiddlewareFactory` | Stores a parameterized middleware used as `name:param`. |
 | `surf.Router.Routes` | Returns a copy of the registered routes as `surf.RouteInfo` values. |
 | `surf.Router.Handler` | Returns the compiled middleware chain of one registered route by method and pattern, for an in-process dispatch that must run the same guards, throttles, body limit, locale and recovery as an HTTP request. False for an unknown route or before `surf.BuildRouter` compiled the routes. |
+| `surf.Router.MountStream` | Registers one streaming route (raw, unbuffered, `http.Flusher` passed through) under the bound plugin with the given middleware names; a route another plugin registered for the method and path fails naming it. |
+| `surf.MCPBuildTag` | `summercms_mcp`: the build tag that links the MCP module; without it `mcp.enabled` true fails boot. |
 | `surf.Router.AdminPrefix` | The admin mount path and true when `surf.BuildRouter` mounted the cabana admin; false when no plugin registers an admin controller. |
 | `surf.RouteInfo` | Method, pattern, owning plugin, middleware and raw flag of one route. |
 | `surf.Use` | Builds a middleware name list for a group. |
@@ -173,6 +177,7 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur
 | `http.api_docs.enabled` | `false` | Mounts `GET <http.api_docs.path>` (Swagger UI page) and `GET <http.api_docs.path>.json` (the spec of the one `surf.APIDocsProvider` plugin). Both are public. Environment: `SUMMER_HTTP__API_DOCS__ENABLED`. |
 | `http.api_docs.path` | `/api-docs` | Page path; the JSON route adds `.json`. Must be an absolute URL path; a plugin route on either path fails boot. |
 | `http.api_docs.title` | `API documentation` | Page title. |
+| `mcp.enabled` | `false` | Read here only to call the MCP module's mount hook after every plugin route and the admin are registered. `false` mounts nothing; `true` in a binary built without the `summercms_mcp` tag fails boot with an error naming the tag. The other `mcp.*` keys belong to the MCP module. Environment: `SUMMER_MCP__ENABLED`. |
 | `http.access_log` | `true` | Logs one `http request` record per request under the `http` component (method, matched pattern, status, duration in milliseconds, request id; never the path or query). Set to `false` to turn it off, or raise `log.components.http` to `warn` to silence it. Environment: `SUMMER_HTTP__ACCESS_LOG`. |
 | `http.error_pages.enabled` | `true` | Answers browser navigations that hit a framework-generated 404, 405 or 500 with the HTML error page (or a published `surf.ErrorPageRenderer`'s page). `false` restores the plain stdlib and JSON bodies for every request. Environment: `SUMMER_HTTP__ERROR_PAGES__ENABLED`. |
 | `storage.uploads.serve` | `false` | Mounts the uploads route `GET <storage.uploads.public_path_prefix>/{path...}` (prefix default `/storage/uploads`; it must then be an absolute URL path, or start-up fails) on `attach.StaticHandlerPublic` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Prefer a separate origin for uploads in production. Environment: `SUMMER_STORAGE__UPLOADS__SERVE`. |

modules/surf/pier_mount_smoke_test.go

@@ -0,0 +1,54 @@
+package surf
+
+import (
+	"reflect"
+	"strings"
+	"testing"
+
+	"git.golem15.com/golem15/summercms/modules/party"
+)
+
+// TestPierMountSmoke covers the MCP mount hook of phase 19.7 in a binary
+// built without the summercms_mcp tag: mcp.enabled unset mounts nothing and
+// changes nothing (D-30); mcp.enabled true fails boot naming the tag (D-33).
+func TestPierMountSmoke(t *testing.T) {
+	if mountPier != nil {
+		t.Skip("mountPier is set: this binary carries the summercms_mcp tag")
+	}
+	plugins := []party.Plugin{dispatchPlugin{}}
+	base, err := BuildRouter(transportApp(t, "development", map[string]string{"http.yaml": transportBodyLimits}), plugins)
+	if err != nil {
+		t.Fatal(err)
+	}
+	for name, mcp := range map[string]string{
+		"unset":         "",
+		"enabled false": "enabled: false\n",
+		"only the path": "path: /mcp\n",
+	} {
+		t.Run(name, func(t *testing.T) {
+			files := map[string]string{"http.yaml": transportBodyLimits}
+			if mcp != "" {
+				files["mcp.yaml"] = mcp
+			}
+			router, err := BuildRouter(transportApp(t, "development", files), plugins)
+			if err != nil {
+				t.Fatal(err)
+			}
+			for _, ri := range router.Routes() {
+				if strings.HasPrefix(ri.Pattern, "/mcp") || strings.HasPrefix(ri.Pattern, "/.well-known/oauth-protected-resource") {
+					t.Fatalf("route mounted while mcp is off: %+v", ri)
+				}
+			}
+			if !reflect.DeepEqual(router.Routes(), base.Routes()) {
+				t.Fatalf("route table differs from a build without the key:\n%+v\n%+v", router.Routes(), base.Routes())
+			}
+		})
+	}
+	_, err = BuildRouter(transportApp(t, "development", map[string]string{
+		"http.yaml": transportBodyLimits,
+		"mcp.yaml":  "enabled: true\n",
+	}), plugins)
+	if err == nil || !strings.Contains(err.Error(), "summercms_mcp") {
+		t.Fatalf("mcp.enabled without the tag: err = %v, want an error naming summercms_mcp", err)
+	}
+}

modules/surf/pier_off.go

@@ -0,0 +1,9 @@
+//go:build !summercms_mcp
+
+package surf
+
+// Without the summercms_mcp tag (MCPBuildTag) the MCP module is not linked:
+// the mountPier hook declared in router.go stays nil, so BuildRouter mounts
+// nothing for mcp.* and refuses mcp.enabled: true with an error naming the
+// tag. The tagged counterpart, pier_on.go, imports the pier module and sets
+// the hook; this file declares nothing else so both builds compile.

modules/surf/router.go

@@ -2,6 +2,7 @@ package surf
 
 import (
 	"bytes"
+	"errors"
 	"fmt"
 	"log/slog"
 	"math"
@@ -46,8 +47,18 @@ type route struct {
 	// preflight marks an OPTIONS route addCORSPreflights registered for a
 	// cors:<profile> path: wrapped only in recovery and recordPattern.
 	preflight bool
+	// stream marks a MountStream route: raw, and served without the
+	// response buffer so the handler can flush to the client (streaming.go).
+	stream bool
 }
 
+// mountPier is the hook the MCP module (pier) sets when a binary is built
+// with the summercms_mcp tag (pier_on.go); without the tag it stays nil
+// (pier_off.go). BuildRouter calls it after every plugin route and the admin
+// exist, when mcp.enabled is true. pier throttles its endpoint with the
+// inline throttle:<max>,<minutes> spelling, so no bucket is registered here.
+var mountPier func(r *Router, app *backpack.App, plugins []party.Plugin) error
+
 // Router compiles group declarations onto net/http ServeMux.
 type Router struct {
 	pluginID     string
@@ -503,9 +514,12 @@ func (r *Router) wrap(rt route) (http.Handler, error) {
 		h = bodyLimit(limit)(h)
 	}
 	h = r.locale(h)
-	if rt.raw {
+	switch {
+	case rt.stream:
+		h = recoverStream(r.pages, h)
+	case rt.raw:
 		h = recoverBare(r.pages, h)
-	} else {
+	default:
 		h = recoverJSON(r.pages, h)
 	}
 	// The CORS profile wraps recovery, so a refusal from a guard listed
@@ -680,6 +694,16 @@ func BuildRouter(app *backpack.App, plugins []party.Plugin) (*Router, error) {
 		r.pages.setAdminPrefix(admin.Prefix)
 		r.adminMounted = true
 	}
+	// The MCP module mounts last, so its tool bindings can see every plugin
+	// route and the admin (D-30: nothing when off; D-33: tag required).
+	if app != nil && app.Config != nil && app.Config.Bool("mcp.enabled") {
+		if mountPier == nil {
+			return nil, errors.New("surf: mcp.enabled is true but this binary was built without the summercms_mcp tag (summer.yaml build.tags)")
+		}
+		if err := mountPier(r, app, plugins); err != nil {
+			return nil, err
+		}
+	}
 	if err := r.compileHandlers(); err != nil {
 		return nil, err
 	}

modules/surf/streaming.go

@@ -0,0 +1,108 @@
+package surf
+
+import (
+	"fmt"
+	"net/http"
+	"strings"
+
+	"git.golem15.com/golem15/summercms/modules/wire"
+)
+
+// MCPBuildTag is the Go build tag that links the MCP module (pier) and its
+// SDK into a binary. List it under build.tags in summer.yaml so summer build
+// passes it to go build; a raw go build needs -tags summercms_mcp. Without
+// the tag mcp.enabled: true fails the start-up with an error naming the tag.
+const MCPBuildTag = "summercms_mcp"
+
+// MountStream registers one streaming route under the currently bound plugin
+// (BindPlugin): a raw route (no house middleware, no default body limit)
+// whose response is not buffered, so the handler's http.Flusher reaches the
+// client while it is still running, for server-sent events and other
+// long-lived responses. Everything else is as for a raw route: the named
+// middleware (guards, throttles), a body.limit, the locale, the CORS profile
+// and the access-log pattern. A panic before the first byte answers an opaque
+// 500; after it the body is only cut short. A route another plugin already
+// registered for method and path fails with an error naming that plugin.
+func (r *Router) MountStream(method, path string, h http.Handler, middleware ...string) error {
+	if r == nil {
+		return fmt.Errorf("surf: router is nil")
+	}
+	method = strings.ToUpper(strings.TrimSpace(method))
+	full := joinPath(r.prefix, path)
+	if method == "" || strings.TrimSpace(path) == "" || h == nil {
+		return fmt.Errorf("surf: plugin %q registered an empty stream route", r.pluginID)
+	}
+	if strings.ContainsAny(full, " \t\r\n") {
+		return fmt.Errorf("surf: plugin %q registered stream route %s %q with whitespace in its path", r.pluginID, method, full)
+	}
+	key := method + " " + full
+	if prev, ok := r.seen[key]; ok {
+		return fmt.Errorf("surf: route %s registered by %s collides with the stream route plugin %q mounts", key, prev, r.pluginID)
+	}
+	r.seen[key] = r.pluginID
+	mw := append([]string{}, r.middleware...)
+	mw = append(mw, middleware...)
+	r.routes = append(r.routes, route{
+		pluginID:   r.pluginID,
+		method:     method,
+		path:       full,
+		handler:    h,
+		middleware: mw,
+		raw:        true,
+		stream:     true,
+	})
+	return nil
+}
+
+// recoverStream is the recovery layer of a stream route. It serves next on
+// the real ResponseWriter (no bufferedResponse, so Flush reaches the client)
+// and recovers a panic by writing wire's opaque 500, or the HTML error page
+// for a browser navigation, only while nothing has been sent yet; after the
+// first header, byte or flush a panic only aborts the body and marks the
+// request panicked for the access log. The panic value is never written.
+func recoverStream(pages *errorPages, next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		sw := &headerSentWriter{ResponseWriter: w}
+		defer func() {
+			if rec := recover(); rec != nil {
+				markPanicked(r)
+				if sw.sent {
+					return
+				}
+				if pages.wants(r) {
+					pages.write(w, r, http.StatusInternalServerError)
+					return
+				}
+				wire.WriteOpaque500(w)
+			}
+		}()
+		next.ServeHTTP(sw, r)
+	})
+}
+
+// headerSentWriter records whether the response has started (a WriteHeader,
+// Write or Flush) and passes Flush and Unwrap through to the real writer.
+type headerSentWriter struct {
+	http.ResponseWriter
+	sent bool
+}
+
+func (w *headerSentWriter) WriteHeader(status int) {
+	w.sent = true
+	w.ResponseWriter.WriteHeader(status)
+}
+
+func (w *headerSentWriter) Write(p []byte) (int, error) {
+	w.sent = true
+	return w.ResponseWriter.Write(p)
+}
+
+// Flush sends the headers (200 when none were written) and the bytes so far.
+func (w *headerSentWriter) Flush() {
+	w.sent = true
+	if f, ok := w.ResponseWriter.(http.Flusher); ok {
+		f.Flush()
+	}
+}
+
+func (w *headerSentWriter) Unwrap() http.ResponseWriter { return w.ResponseWriter }

modules/surf/streaming_smoke_test.go

@@ -0,0 +1,184 @@
+package surf
+
+import (
+	"bufio"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+
+	"git.golem15.com/golem15/summercms/modules/backpack"
+	"git.golem15.com/golem15/summercms/modules/pact"
+	"git.golem15.com/golem15/summercms/modules/party"
+)
+
+// Smoke test for the streaming route kind of phase 19.7 (RESEARCH Pitfall 1):
+// a MountStream route flushes to the client while the handler still runs, a
+// buffered route does not, and a panic is recovered without rewriting a
+// response that has started.
+
+// streamPlugin registers one buffered route; its stream routes are mounted
+// through the mountPier hook, as the MCP module would.
+type streamPlugin struct {
+	release chan struct{}
+}
+
+func (streamPlugin) ID() string                   { return "acme.stream" }
+func (streamPlugin) Requires() []string           { return nil }
+func (streamPlugin) Register(*backpack.App) error { return nil }
+func (streamPlugin) Boot(*backpack.App) error     { return nil }
+
+func (p streamPlugin) events(w http.ResponseWriter, _ *http.Request) {
+	w.Header().Set("Content-Type", "text/event-stream")
+	_, _ = io.WriteString(w, "event: a\n\n")
+	if f, ok := w.(http.Flusher); ok {
+		f.Flush()
+	}
+	<-p.release
+	_, _ = io.WriteString(w, "event: b\n\n")
+}
+
+func (p streamPlugin) Routes(r pact.Router) error {
+	r.Get("/acme/buffered", p.events)
+	return nil
+}
+
+// firstLine reads one line of the body in the background and reports it on
+// the returned channel.
+func firstLine(t *testing.T, res *http.Response) <-chan string {
+	t.Helper()
+	out := make(chan string, 1)
+	go func() {
+		line, err := bufio.NewReader(res.Body).ReadString('\n')
+		if err != nil {
+			line = "error: " + err.Error()
+		}
+		out <- strings.TrimSpace(line)
+	}()
+	return out
+}
+
+func TestStreamingRouteSmoke(t *testing.T) {
+	plugin := streamPlugin{release: make(chan struct{})}
+	panicAfterFlush := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
+		w.Header().Set("Content-Type", "text/event-stream")
+		_, _ = io.WriteString(w, "event: a\n\n")
+		w.(http.Flusher).Flush()
+		panic("after flush")
+	})
+	panicBeforeWrite := http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
+		panic("before write")
+	})
+	mountPier = func(r *Router, _ *backpack.App, _ []party.Plugin) error {
+		r.BindPlugin("summercms.pier")
+		if err := r.MountStream("GET", "/mcp/events", http.HandlerFunc(plugin.events)); err != nil {
+			return err
+		}
+		if err := r.MountStream("GET", "/mcp/panic-late", panicAfterFlush); err != nil {
+			return err
+		}
+		if err := r.MountStream("GET", "/mcp/panic-early", panicBeforeWrite); err != nil {
+			return err
+		}
+		// A stream route may not replace a plugin route.
+		if err := r.MountStream("GET", "/acme/buffered", panicBeforeWrite); err == nil || !strings.Contains(err.Error(), "acme.stream") {
+			t.Errorf("MountStream over a plugin route: %v", err)
+		}
+		return nil
+	}
+	defer func() { mountPier = nil }()
+
+	app := transportApp(t, "development", map[string]string{
+		"http.yaml": transportBodyLimits,
+		"mcp.yaml":  "enabled: true\n",
+	})
+	h, err := Assemble(app, []party.Plugin{plugin})
+	if err != nil {
+		t.Fatal(err)
+	}
+	srv := httptest.NewServer(h)
+	defer srv.Close()
+	client := srv.Client()
+
+	// The stream route delivers the first event before the handler returns.
+	stream, err := client.Get(srv.URL + "/mcp/events")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer stream.Body.Close()
+	if stream.StatusCode != http.StatusOK {
+		t.Fatalf("stream status = %d", stream.StatusCode)
+	}
+	select {
+	case line := <-firstLine(t, stream):
+		if line != "event: a" {
+			t.Fatalf("stream first line = %q", line)
+		}
+	case <-time.After(5 * time.Second):
+		t.Fatal("stream route did not flush its first event while the handler was running")
+	}
+
+	// The buffered route delivers nothing, not even its headers, until the
+	// handler returns: the whole GET stays pending until the release.
+	type bufferedResult struct {
+		status int
+		body   string
+		err    error
+	}
+	bufferedDone := make(chan bufferedResult, 1)
+	go func() {
+		res, err := client.Get(srv.URL + "/acme/buffered")
+		if err != nil {
+			bufferedDone <- bufferedResult{err: err}
+			return
+		}
+		defer res.Body.Close()
+		body, err := io.ReadAll(res.Body)
+		bufferedDone <- bufferedResult{status: res.StatusCode, body: string(body), err: err}
+	}()
+	select {
+	case got := <-bufferedDone:
+		t.Fatalf("buffered route answered before the handler returned: %+v", got)
+	case <-time.After(300 * time.Millisecond):
+	}
+	close(plugin.release)
+	select {
+	case got := <-bufferedDone:
+		if got.err != nil || got.status != http.StatusOK || got.body != "event: a\n\nevent: b\n\n" {
+			t.Fatalf("buffered route after release = %+v", got)
+		}
+	case <-time.After(5 * time.Second):
+		t.Fatal("buffered route never answered")
+	}
+	rest, err := io.ReadAll(stream.Body)
+	if err != nil || !strings.Contains(string(rest), "event: b") {
+		t.Fatalf("stream rest = %q, %v", rest, err)
+	}
+
+	// A panic after the first flush ends the body; the 200 already sent stays.
+	late, err := client.Get(srv.URL + "/mcp/panic-late")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer late.Body.Close()
+	if late.StatusCode != http.StatusOK {
+		t.Fatalf("panic after flush status = %d", late.StatusCode)
+	}
+	body, _ := io.ReadAll(late.Body)
+	if !strings.HasPrefix(string(body), "event: a") || strings.Contains(string(body), "500") || strings.Contains(string(body), "after flush") {
+		t.Fatalf("panic after flush body = %q", body)
+	}
+
+	// A panic before any write answers the opaque 500.
+	early, err := client.Get(srv.URL + "/mcp/panic-early")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer early.Body.Close()
+	body, _ = io.ReadAll(early.Body)
+	if early.StatusCode != http.StatusInternalServerError || strings.Contains(string(body), "before write") {
+		t.Fatalf("panic before write = %d %q", early.StatusCode, body)
+	}
+}