diff --git a/docs/architecture/request-lifecycle.md b/docs/architecture/request-lifecycle.md index c8fef616..fb281be7 100644 --- a/docs/architecture/request-lifecycle.md +++ b/docs/architecture/request-lifecycle.md @@ -38,14 +38,16 @@ surf wraps every non-raw route in the same layers. From the outside in: 1. Optional locale-prefix rewrite, when a `surf.LocaleResolver` is published: an enabled locale as the first path segment is stripped so later layers and routing see the unprefixed path. 2. CORS, only for the paths configured under `http.cors.paths`, including preflight requests. 3. The route's CORS profile, when it or its group names `cors:`. It sits here whatever its position in the middleware list, so refusals and recovered panics carry its headers too. -4. Panic recovery. A panic becomes an opaque JSON 500 from [wire](../../modules/wire/README.md), and because the response is buffered until the handler returns, the client never receives half a body. +4. Panic recovery. A panic becomes an opaque JSON 500 from [wire](../../modules/wire/README.md), or the HTML error page with status 500 when the request is a browser navigation (see [Error pages](../services/routing.md#error-pages)). Because the response is buffered until the handler returns, the client never receives half a body. 5. The request locale, taken from the published `surf.LocaleResolver` when one exists, or from the `Accept-Language` header otherwise, and stored in the context. 6. The body limit: `http.body_limits.default_bytes`, or the route's own `body.limit:`. 7. The route's middleware, in the order you listed them: group middleware first, then the route's own (`cors:` excepted, see step 3). 8. The path constraints. A request whose parameter fails `pact.Router.Where` or `pact.Router.WhereIn` gets a 404 before your handler runs. 9. Your handler. -Raw groups, declared with `pact.Router.GroupRaw`, are for webhooks and file streams. They skip the default body limit, refuse house middleware, and a panic in them returns a bare 500. +Raw groups, declared with `pact.Router.GroupRaw`, are for webhooks and file streams. They skip the default body limit, refuse house middleware, and a panic in them returns a bare 500, or the HTML error page for a browser navigation. + +A request that no route matches, or that uses a method its path does not accept, is answered by the ServeMux and negotiated the same way: a browser navigation gets the HTML error page with status 404 or 405, every other caller the stdlib text. ## Request context values diff --git a/docs/services/routing.md b/docs/services/routing.md index 536be38b..38b6ee37 100644 --- a/docs/services/routing.md +++ b/docs/services/routing.md @@ -56,7 +56,7 @@ Laravel matches routes in the order `routes.php` declares them and checks `->whe surf accepts such routes. When the application starts, it groups every set of routes that ServeMux would refuse side by side into an overlap family and registers the family once, under a generated pattern with a wildcard wherever its routes differ. A request that reaches the family is tried against its routes in registration order. The first route whose literal segments and constraints match runs, with its own path values, middleware, body limit and panic recovery. Declare overlapping routes in the order the PHP file declares them. -- **No match:** a request whose path no route of the family matches gets the same bare 404 as any unknown path, for every method. +- **No match:** a request whose path no route of the family matches gets the same 404 as any unknown path, for every method: the bare stdlib text for an API client, the [error page](#error-pages) for a browser navigation. - **Wrong method:** a request whose path a route matches, but not with its method, gets the 405 and `Allow` header that ServeMux gives for the same routes without the overlap. - **More specific routes** stay outside the family: a literal `GET /shelves/items/similar` keeps answering its own path. - **Route table:** `surf.Router.Routes` and `route:list` list every route of a family on its own, with its own pattern and middleware. @@ -138,6 +138,76 @@ fmt.Println(rec.Code, rec.Body.String()) `wire.WriteOpaque500` writes the fixed 500 body that panic recovery also uses; it reveals nothing about the failure. +## Error pages + +When the framework itself answers with an error, an API client and a browser get different bodies. An API client, curl, `fetch` or an XHR keeps exactly the response it always got: the stdlib `404 page not found` and `Method Not Allowed` text (with `Allow`), the opaque JSON 500 from `wire.WriteOpaque500`, or the empty 500 of a raw group. A person who opens the address in a browser gets a self-contained SummerCMS HTML page with the same status, a short explanation and a link to `/`. + +A request gets the HTML page only when all of these hold: + +- `Sec-Fetch-Mode` is `navigate` when the browser sends it; without the header, the method is GET or HEAD. +- There is no `X-Requested-With` header. +- `Accept` names `text/html` or `application/xhtml+xml` with a q above 0 and above the q of every JSON type it names (`application/json`, `text/json`, `application/*+json`). `*/*` and `text/*` count for neither side, and only the first 32 media ranges are read. +- The path is not `/api`, not under `/api/`, and its first segment does not start with `_` (the `/_/` plugin prefixes). +- The path is not the admin prefix or under it, so the admin keeps its own behaviour. + +The path checked is the one routing sees, after a locale prefix is stripped. + +The page covers the errors surf writes itself: an unmatched path (404), a method the path does not accept (405, with the `Allow` header), a parameter that fails `pact.Router.Where` or `pact.Router.WhereIn`, an overlap-family miss or wrong method, the uploads route while its bucket or database is not published (404), and a recovered panic in a normal or raw group (500). The panic value is never shown. A handler's own error body, middleware refusals (401, 403, 413, 422, 429, 503) and the 404 for a missing upload row are not changed. + +Method, path and the `X-Request-Id` request header (up to 128 bytes) are listed on the page only when `app.debug` is true or the environment is set and is not `production`. Every HTML error response carries `Cache-Control: no-store`, `X-Content-Type-Options: nosniff`, `Vary: Accept, Sec-Fetch-Mode, X-Requested-With` and a `Content-Security-Policy` without a script source, so no script runs on an error page. + +Set `http.error_pages.enabled: false` to send the plain bodies to every client, browsers included. + +An application or theme plugin replaces the page by publishing a `surf.ErrorPageRenderer` from `Register`. surf hands it a `surf.ErrorPage` for every error it would render and keeps the status and headers; returning `ok=false`, an empty body or panicking keeps the built-in page: + +```go src=modules/surf/example_test.go#themePages.RenderErrorPage +// RenderErrorPage returns the theme's own 404 and keeps the built-in page +// for every other status. +func (themePages) RenderErrorPage(r *http.Request, page surf.ErrorPage) ([]byte, bool) { + if page.Status != http.StatusNotFound { + return nil, false // keep the built-in page + } + body := `Not found

Nothing here

Home` + return []byte(body), true +} +``` + +```go src=modules/surf/example_test.go#ThemePlugin.Register +// Register publishes the theme's error page for surf to look up. +func (ThemePlugin) Register(app *backpack.App) error { + return app.Publish[surf.ErrorPageRenderer](themePages{}) +} +``` + +A browser navigation to an unknown path then gets the theme's page, and curl still gets the stdlib text: + +```go src=modules/surf/example_test.go#ExampleErrorPageRenderer +app := backpack.New(nil) +plugin := ThemePlugin{} +if err := plugin.Register(app); err != nil { // the runtime calls Register + fmt.Println(err) + return +} +h, err := surf.Assemble(app, []party.Plugin{plugin}) +if err != nil { + fmt.Println(err) + return +} +browser := httptest.NewRequest("GET", "/nowhere", nil) +browser.Header.Set("Accept", "text/html,application/xhtml+xml,*/*;q=0.8") +browser.Header.Set("Sec-Fetch-Mode", "navigate") +curl := httptest.NewRequest("GET", "/nowhere", nil) +curl.Header.Set("Accept", "*/*") +for _, req := range []*http.Request{browser, curl} { + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + fmt.Println(rec.Code, rec.Header().Get("Content-Type"), strings.TrimSpace(rec.Body.String())) +} +// Output: +// 404 text/html; charset=utf-8 Not found

Nothing here

Home +// 404 text/plain; charset=utf-8 404 page not found +``` + ## Testing and listing routes `surf.Assemble` builds the complete handler from the application and its plugins, so a test can drive it with `net/http/httptest`: diff --git a/docs/setup/configuration.md b/docs/setup/configuration.md index b7dd1fcb..9be9fa8a 100644 --- a/docs/setup/configuration.md +++ b/docs/setup/configuration.md @@ -60,6 +60,7 @@ These are the keys most applications set. Each module's reference page lists all | `site_cache.purge_url`, `site_cache.purge_token` | The host site's page-cache purge route and its bearer token (at least 32 characters). Set both or neither; http only to a loopback host. Keep the token in `SUMMER_SITE_CACHE__PURGE_TOKEN`. | [cooler](../../modules/cooler/README.md) | | `redis.url` | Redis URL (`redis://`, `rediss://` or `unix://`), required when a Redis cache or rate-limit store is selected. Keep it in `SUMMER_REDIS__URL` when it carries a password. | [cooler](../../modules/cooler/README.md) | | `http.access_log` | Logs one record per request with method, matched route pattern, status and duration. Defaults to `false`. | [surf](../../modules/surf/README.md) | +| `http.error_pages.enabled` | HTML error pages for browser navigations (defaults to `true`); request details appear when `app.debug` is true or outside production. | [surf](../../modules/surf/README.md) | | `storage.uploads.bucket_url` | Uploads bucket, `file://`, `mem://` or `s3://` (the last only in a binary built with the `summercms_s3` tag from `build.tags` in `summer.yaml`; credentials from the AWS environment). Required by `serve`. | [lagoon](../../modules/lagoon/README.md) | | `storage.uploads.public_url` | Absolute `http` or `https` base of a public bucket or CDN; file and thumbnail URLs become that base plus the partition path. Unset keeps `storage.uploads.public_path_prefix`. Only for a bucket holding public files alone. | [lagoon](../../modules/lagoon/README.md) | | `storage.uploads.serve` | Mounts `attach.StaticHandlerPublic` under `storage.uploads.public_path_prefix` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Defaults to `false`. | [surf](../../modules/surf/README.md) | diff --git a/modules/surf/README.md b/modules/surf/README.md index 9c580609..5848913c 100644 --- a/modules/surf/README.md +++ b/modules/surf/README.md @@ -32,6 +32,7 @@ Use cases: - Fixed-window rate limiting (`surf.FixedWindowLimiter`): named buckets from plugins that implement `surf.BucketProvider`, or inline limits keyed by the signed-in user, or by client IP for guests. Rejected requests get a 429 with `Retry-After` and `X-RateLimit-*` headers. The in-process `surf.MemoryStore` sits behind the `surf.Store` interface; `surf.BuildRouter` uses a `surf.Store` published on the app instead when there is one, which is how `serve` shares counters across instances through [cooler](../cooler/README.md)'s Redis store (`http.rate_limit.store: redis`). - Client IP resolution for limiter keys (`surf.ClientIP`) that only trusts `X-Forwarded-For` hops when the direct peer is inside a configured trusted proxy range (`surf.TrustedProxies`). - Every non-raw route runs inside JSON panic recovery (an opaque 500 via [wire](../wire/README.md)), gets the request locale from a published `surf.LocaleResolver` when one exists or from the `Accept-Language` header otherwise (see [towel](../towel/README.md)), and a request body cap. Responses are buffered until the handler returns, so a panic never leaves a half-written body. +- Branded error pages for browsers: a top-level browser navigation (`Sec-Fetch-Mode: navigate` when the header is sent, otherwise GET or HEAD; no `X-Requested-With`; an `Accept` header that names `text/html` or `application/xhtml+xml` with a higher q than any JSON type; not `/api`, `/api/...`, `/_/...` or the admin prefix) that hits a framework-generated error gets a self-contained SummerCMS HTML page with the same status. Covered: unmatched routes (404), wrong methods (405, `Allow` kept), failed `Where`/`WhereIn` constraints and overlap-family misses (404, 405), the uploads route while its bucket or database is unpublished (404), and recovered panics in normal and raw groups (500; the panic value is never shown). Every other request keeps the exact stdlib or [wire](../wire/README.md) body. Method, path and `X-Request-Id` appear on the page only when `app.debug` is true or outside the production environment. The page sends `Cache-Control: no-store`, `X-Content-Type-Options: nosniff`, `Vary: Accept, Sec-Fetch-Mode, X-Requested-With` and a `Content-Security-Policy` that allows no script. An application or plugin replaces the page by publishing a `surf.ErrorPageRenderer`; `http.error_pages.enabled: false` turns the feature off. Handlers' own error bodies and middleware refusals are not touched. - Path-scoped CORS configured with the same keys as Laravel's `config/cors.php` (`surf.CORSConfig`), including preflight handling. `surf.LoadCORSConfig` refuses credentials with a `*` origin and origin patterns that do not compile; an allowed origin is echoed exactly, with `Vary: Origin` on every response whose answer depends on the request origin. Every `OPTIONS` request on a CORS path is answered with 204 before routing, with the headers Laravel's `HandleCors` sends: `Cache-Control: no-cache, private` always and, on a preflight (an `Origin` and an `Access-Control-Request-Method`), the requested method (upper-cased) and headers echoed in `Access-Control-Allow-Methods` and `Access-Control-Allow-Headers` when `*` allows any, `Vary` on the request headers and PHP's default `Content-Type: text/html; charset=UTF-8`. - Optional `surf.LocaleResolver`: a compiled plugin publishes the interface through backpack; surf looks it up without importing the plugin. When present, it may strip an enabled locale prefix before routing and writes a validated code with `towel.WithLocale`. When absent, hosts keep the raw `Accept-Language` header and `surf.LocaleFromPrincipal` overlay. - `surf.LocaleFromPrincipal` switches the request locale to the signed-in user's preferred locale. @@ -141,6 +142,8 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur | `surf.SwaggerUIVersion` | The exact swagger-ui-dist release the api-docs page loads, pinned so its assets carry SRI hashes. | | `surf.LocaleResolver` | Optional backpack-published request locale; `Resolve` returns a validated code and `Rewrite` strips an enabled prefix before routing. | | `surf.LocaleFromPrincipal` | Middleware that applies the signed-in user's preferred locale. | +| `surf.ErrorPage` | What a browser error page shows: status, title, message, home link, and (only with `ShowDetails`) method, path and request id. | +| `surf.ErrorPageRenderer` | Optional backpack-published override of the HTML error page; `RenderErrorPage` returns the full document, or `ok=false` to keep the built-in page. | | `surf.ServeCommand` | The `serve` console command. | | `surf.RouteListCommand` | The `route:list` console command. | @@ -169,6 +172,7 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur | `http.api_docs.path` | `/api-docs` | Page path; the JSON route adds `.json`. Must be an absolute URL path; a plugin route on either path fails boot. | | `http.api_docs.title` | `API documentation` | Page title. | | `http.access_log` | `false` | Logs one `http request` record per request (method, matched pattern, status, duration in milliseconds; never the path or query). Environment: `SUMMER_HTTP__ACCESS_LOG`. | +| `http.error_pages.enabled` | `true` | Answers browser navigations that hit a framework-generated 404, 405 or 500 with the HTML error page (or a published `surf.ErrorPageRenderer`'s page). `false` restores the plain stdlib and JSON bodies for every request. Environment: `SUMMER_HTTP__ERROR_PAGES__ENABLED`. | | `storage.uploads.serve` | `false` | Mounts the uploads route `GET /{path...}` (prefix default `/storage/uploads`; it must then be an absolute URL path, or start-up fails) on `attach.StaticHandlerPublic` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Prefer a separate origin for uploads in production. Environment: `SUMMER_STORAGE__UPLOADS__SERVE`. | ```yaml @@ -198,7 +202,7 @@ The `serve` command also opens the database through [lagoon](../lagoon/README.md - SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [cabana](../cabana/README.md), [compass](../compass/README.md), [conga](../conga/README.md) (the in-process job worker of `serve`), [cooler](../cooler/README.md) (the shared rate-limit store `serve` installs), [lagoon](../lagoon/README.md) (including `lagoon/attach`), [pact](../pact/README.md), [party](../party/README.md), [towel](../towel/README.md), [wire](../wire/README.md). - Third-party: `gocloud.dev/blob` (uploads bucket opened by `serve`), `github.com/goccy/go-yaml` (api-docs spec conversion). -- Standard library: `bytes`, `context`, `crypto/rand`, `encoding/base64`, `encoding/json`, `errors`, `fmt`, `html/template`, `log/slog`, `math`, `mime`, `net`, `net/http`, `net/netip`, `net/url`, `os`, `os/signal`, `regexp`, `sort`, `strconv`, `strings`, `sync`, `syscall`, `time`. +- Standard library: `bytes`, `context`, `crypto/rand`, `embed`, `encoding/base64`, `encoding/json`, `errors`, `fmt`, `html/template`, `log/slog`, `math`, `mime`, `net`, `net/http`, `net/netip`, `net/url`, `os`, `os/signal`, `regexp`, `sort`, `strconv`, `strings`, `sync`, `syscall`, `time`, `unicode/utf8`. - Tests additionally use `github.com/testcontainers/testcontainers-go` (Redis container). ## Testing diff --git a/modules/surf/accesslog_test.go b/modules/surf/accesslog_test.go index d1469e80..644bbe13 100644 --- a/modules/surf/accesslog_test.go +++ b/modules/surf/accesslog_test.go @@ -168,19 +168,19 @@ func TestAccessLogClientClosedRequest(t *testing.T) { }) }, outcome{200, true}}, {"5xx through the JSON recovery after the client went away", func(cancel context.CancelFunc) http.Handler { - return recordPattern("GET /blog/posts/{id}", recoverJSON(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + return recordPattern("GET /blog/posts/{id}", recoverJSON(nil, http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { cancel() w.WriteHeader(http.StatusInternalServerError) }))) }, outcome{499, true}}, {"panic after the client went away (JSON group)", func(cancel context.CancelFunc) http.Handler { - return recordPattern("GET /blog/posts/{id}", recoverJSON(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { + return recordPattern("GET /blog/posts/{id}", recoverJSON(nil, http.HandlerFunc(func(http.ResponseWriter, *http.Request) { cancel() panic("blog: broken") }))) }, outcome{500, true}}, {"panic after the client went away (raw group)", func(cancel context.CancelFunc) http.Handler { - return recordPattern("GET /blog/posts/{id}", recoverBare(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { + return recordPattern("GET /blog/posts/{id}", recoverBare(nil, http.HandlerFunc(func(http.ResponseWriter, *http.Request) { cancel() panic("blog: broken") }))) diff --git a/modules/surf/errorpage.go b/modules/surf/errorpage.go new file mode 100644 index 00000000..cd0c48f3 --- /dev/null +++ b/modules/surf/errorpage.go @@ -0,0 +1,381 @@ +package surf + +import ( + "bytes" + _ "embed" + "encoding/base64" + "html/template" + "net/http" + "strconv" + "strings" + "unicode/utf8" + + "git.golem15.com/golem15/summercms/modules/backpack" +) + +// ErrorPage describes a framework-generated error answered with an HTML page +// to a browser navigation: an unmatched route, a wrong method, a failed +// Where/WhereIn constraint, the uploads route while it is unavailable, or a +// recovered panic. An ErrorPageRenderer receives it to build its own page. +type ErrorPage struct { + // Status is the HTTP status the response carries (404, 405 or 500). + Status int + // Title is the short English heading for Status, such as "Page not found". + Title string + // Message is one English sentence explaining Status. + Message string + // Home is the link back to the site, always "/". + Home string + // ShowDetails is true when app.debug is true or the environment is set + // and is not production. Method, Path and RequestID are empty otherwise. + ShowDetails bool + // Method is the request method; empty unless ShowDetails is true. + Method string + // Path is the request path routing saw; empty unless ShowDetails is true. + Path string + // RequestID is the X-Request-Id request header, trimmed and cut to 128 + // bytes; empty unless ShowDetails is true or when the header is absent. + RequestID string +} + +// ErrorPageRenderer is an optional backpack-published service that replaces +// the built-in HTML error page. surf looks it up for every page it renders, +// so an application or plugin publishes it in Register: +// +// app.Publish[surf.ErrorPageRenderer](myRenderer{}) +// +// surf still sets the status and the page headers (Content-Type, nosniff, +// no-store, Vary and the error-page Content-Security-Policy, which allows no +// script). Returning ok=false or an empty body keeps the built-in page, and +// so does a panic inside RenderErrorPage. +type ErrorPageRenderer interface { + // RenderErrorPage returns the complete HTML document for page. + RenderErrorPage(r *http.Request, page ErrorPage) (body []byte, ok bool) +} + +// errorPageCSP goes on every HTML error response. There is no script-src, +// so no script runs on an error page, an override's included. +const errorPageCSP = "default-src 'none'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self'; base-uri 'none'; form-action 'self'; frame-ancestors 'none'" + +// errorPageMaxRanges caps how many Accept media ranges negotiation reads. +const errorPageMaxRanges = 32 + +// errorPageMaxRequestID caps the X-Request-Id shown on the page, in bytes. +const errorPageMaxRequestID = 128 + +//go:embed errorpage.html +var errorPageHTML string + +//go:embed errorpage_sun.webp +var errorPageLogo []byte + +var ( + errorPageTemplate = template.Must(template.New("errorpage").Parse(errorPageHTML)) + // errorPageLogoURI is a constant built from the embedded file, so it is + // safe to hand html/template as a URL. + errorPageLogoURI = template.URL("data:image/webp;base64," + base64.StdEncoding.EncodeToString(errorPageLogo)) +) + +// errorPageView is what the built-in template renders. +type errorPageView struct { + ErrorPage + Logo template.URL +} + +// errorPages negotiates and writes the HTML error page. A nil *errorPages +// keeps every call site's pre-error-page behaviour. +type errorPages struct { + app *backpack.App + adminPrefix string +} + +// newErrorPages returns the error-page writer for app, or nil when +// http.error_pages.enabled is present and false. A nil app, a nil config or +// an absent key keep error pages on. +func newErrorPages(app *backpack.App) *errorPages { + if app != nil && app.Config != nil { + if _, ok := app.Config.Lookup("http.error_pages.enabled"); ok && !app.Config.Bool("http.error_pages.enabled") { + return nil + } + } + return &errorPages{app: app} +} + +// setAdminPrefix excludes the mounted admin prefix (and every path under it) +// from negotiation. +func (p *errorPages) setAdminPrefix(prefix string) { + if p == nil { + return + } + p.adminPrefix = strings.TrimSuffix(prefix, "/") +} + +// wants reports whether r gets the HTML page instead of today's response. +func (p *errorPages) wants(r *http.Request) bool { + if p == nil || r == nil || r.URL == nil { + return false + } + path := r.URL.Path + if apiStylePath(path) { + return false + } + if p.adminPrefix != "" && (path == p.adminPrefix || strings.HasPrefix(path, p.adminPrefix+"/")) { + return false + } + return browserNavigation(r) +} + +// browserNavigation reports whether r looks like a top-level browser +// navigation that prefers HTML: Sec-Fetch-Mode navigate when the header is +// present (GET or HEAD when it is absent), no X-Requested-With, and an Accept +// header that names an HTML type with a q above every JSON type it names. +func browserNavigation(r *http.Request) bool { + if modes := r.Header.Values("Sec-Fetch-Mode"); len(modes) > 0 { + if !strings.EqualFold(strings.TrimSpace(modes[0]), "navigate") { + return false + } + } else if r.Method != http.MethodGet && r.Method != http.MethodHead { + return false + } + if len(r.Header.Values("X-Requested-With")) > 0 { + return false + } + return acceptPrefersHTML(r.Header.Values("Accept")) +} + +// acceptPrefersHTML reads at most errorPageMaxRanges media ranges. text/html +// or application/xhtml+xml must be named with q > 0 and a q strictly above +// that of every JSON type named (application/json, text/json, +// application/*+json). Wildcards count for neither side; a range with a +// malformed q is skipped. +func acceptPrefersHTML(values []string) bool { + htmlQ, jsonQ := -1.0, -1.0 + read := 0 + for _, line := range values { + for _, rng := range strings.Split(line, ",") { + if read >= errorPageMaxRanges { + break + } + read++ + media, params, _ := strings.Cut(rng, ";") + media = strings.ToLower(strings.TrimSpace(media)) + q, ok := acceptQ(params) + if !ok { + continue + } + switch { + case media == "text/html" || media == "application/xhtml+xml": + htmlQ = max(htmlQ, q) + case media == "application/json" || media == "text/json" || + (strings.HasPrefix(media, "application/") && strings.HasSuffix(media, "+json")): + jsonQ = max(jsonQ, q) + } + } + } + return htmlQ > 0 && htmlQ > jsonQ +} + +// acceptQ returns the q parameter of a media range (1 when absent) and false +// when it is malformed or outside 0..1. +func acceptQ(params string) (float64, bool) { + for _, param := range strings.Split(params, ";") { + key, value, found := strings.Cut(param, "=") + if !found || !strings.EqualFold(strings.TrimSpace(key), "q") { + continue + } + q, err := strconv.ParseFloat(strings.TrimSpace(value), 64) + if err != nil || q < 0 || q > 1 { + return 0, false + } + return q, true + } + return 1, true +} + +// apiStylePath reports whether path is an API path that never gets the page: +// /api, anything under /api/, and the /_/ prefixes. +func apiStylePath(path string) bool { + return path == "/api" || strings.HasPrefix(path, "/api/") || strings.HasPrefix(path, "/_") +} + +// pageFor returns the English title and message for status. +func pageFor(status int) (string, string) { + switch status { + case http.StatusNotFound: + return "Page not found", "The page you are looking for does not exist or has moved." + case http.StatusMethodNotAllowed: + return "Method not allowed", "This address does not accept that kind of request." + case http.StatusInternalServerError: + return "Something went wrong", "The server hit an unexpected error. Please try again in a moment." + } + title := http.StatusText(status) + if title == "" { + title = "Error" + } + return title, "The server could not complete this request." +} + +// showDetails reports whether the page lists method, path and request id: +// app.debug is true, or the environment is set and is not production. A nil +// app or config, or an empty environment, counts as production. +func showDetails(app *backpack.App) bool { + if app == nil || app.Config == nil { + return false + } + if app.Config.Bool("app.debug") { + return true + } + env := strings.TrimSpace(app.Config.Environment()) + return env != "" && !strings.EqualFold(env, "production") +} + +// requestID returns the X-Request-Id header trimmed and cut to +// errorPageMaxRequestID bytes on a rune boundary. +func requestID(r *http.Request) string { + id := strings.ToValidUTF8(strings.TrimSpace(r.Header.Get("X-Request-Id")), "") + if len(id) <= errorPageMaxRequestID { + return id + } + cut := errorPageMaxRequestID + for cut > 0 && !utf8.RuneStart(id[cut]) { + cut-- + } + return id[:cut] +} + +// pageData fills the ErrorPage for status and r. +func (p *errorPages) pageData(r *http.Request, status int) ErrorPage { + title, message := pageFor(status) + page := ErrorPage{Status: status, Title: title, Message: message, Home: "/"} + if showDetails(p.app) { + page.ShowDetails = true + page.Method = r.Method + page.Path = r.URL.Path + page.RequestID = requestID(r) + } + return page +} + +// render returns the published renderer's page, or nil when there is none, +// it declines, returns an empty body or panics. +func (p *errorPages) render(r *http.Request, page ErrorPage) (body []byte) { + if p.app == nil { + return nil + } + renderer, ok := p.app.Lookup[ErrorPageRenderer]() + if !ok || renderer == nil { + return nil + } + defer func() { + if recover() != nil { + body = nil + } + }() + out, ok := renderer.RenderErrorPage(r, page) + if !ok || len(out) == 0 { + return nil + } + return out +} + +// write answers r with the HTML error page for status. The recovered panic +// value, if any, never reaches it. +func (p *errorPages) write(w http.ResponseWriter, r *http.Request, status int) { + page := p.pageData(r, status) + body := p.render(r, page) + if body == nil { + var buf bytes.Buffer + if err := errorPageTemplate.Execute(&buf, errorPageView{ErrorPage: page, Logo: errorPageLogoURI}); err != nil { + http.Error(w, http.StatusText(status), status) + return + } + body = buf.Bytes() + } + h := w.Header() + h.Set("Content-Type", "text/html; charset=utf-8") + h.Set("X-Content-Type-Options", "nosniff") + h.Set("Cache-Control", "no-store") + h.Set("Content-Security-Policy", errorPageCSP) + addVary(h, "Accept") + addVary(h, "Sec-Fetch-Mode") + addVary(h, "X-Requested-With") + h.Del("Content-Length") + w.WriteHeader(status) + if r.Method != http.MethodHead { + _, _ = w.Write(body) + } +} + +// notFound answers 404: the page for a browser navigation, else http.NotFound. +func (p *errorPages) notFound(w http.ResponseWriter, r *http.Request) { + if p.wants(r) { + p.write(w, r, http.StatusNotFound) + return + } + http.NotFound(w, r) +} + +// methodNotAllowed answers 405 after the caller has set Allow: the page for a +// browser navigation, else the stdlib text. +func (p *errorPages) methodNotAllowed(w http.ResponseWriter, r *http.Request) { + if p.wants(r) { + p.write(w, r, http.StatusMethodNotAllowed) + return + } + http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed) +} + +// fallback wraps mux so the 404 and 405 that ServeMux writes itself become +// the page for a browser navigation. Every other request, and every request +// a route matches, goes straight to mux. A nil receiver returns mux. +func (p *errorPages) fallback(mux *http.ServeMux) http.Handler { + if p == nil { + return mux + } + return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) { + if !p.wants(req) { + mux.ServeHTTP(w, req) + return + } + h, pattern := mux.Handler(req) + if pattern != "" { + mux.ServeHTTP(w, req) + return + } + capture := &statusCapture{header: make(http.Header)} + h.ServeHTTP(capture, req) + switch capture.status { + case http.StatusNotFound, http.StatusMethodNotAllowed: + default: + // A path-cleaning redirect also has no pattern: let mux answer. + mux.ServeHTTP(w, req) + return + } + if allow := capture.header.Get("Allow"); allow != "" { + w.Header().Set("Allow", allow) + } + p.write(w, req, capture.status) + }) +} + +// statusCapture records the header and status of ServeMux's own 404/405 +// handler and discards the body. +type statusCapture struct { + header http.Header + status int +} + +func (c *statusCapture) Header() http.Header { return c.header } + +func (c *statusCapture) WriteHeader(status int) { + if c.status == 0 { + c.status = status + } +} + +func (c *statusCapture) Write(b []byte) (int, error) { + if c.status == 0 { + c.status = http.StatusOK + } + return len(b), nil +} diff --git a/modules/surf/errorpage.html b/modules/surf/errorpage.html new file mode 100644 index 00000000..50ce78f6 --- /dev/null +++ b/modules/surf/errorpage.html @@ -0,0 +1,121 @@ + + + + + + +{{.Status}} · {{.Title}} · SummerCMS + + + + +
+
+SummerCMS +SummerCMS +
+

{{.Status}}

+

{{.Title}}

+

{{.Message}}

+Go to the home page +{{- if .ShowDetails}} +
+
Status
{{.Status}}
+
Method
{{.Method}}
+
Path
{{.Path}}
+{{- if .RequestID}} +
Request ID
{{.RequestID}}
+{{- end}} +
+{{- end}} +

Powered by SummerCMS

+
+ + diff --git a/modules/surf/errorpage_sun.webp b/modules/surf/errorpage_sun.webp new file mode 100644 index 00000000..b86ffacf Binary files /dev/null and b/modules/surf/errorpage_sun.webp differ diff --git a/modules/surf/errorpage_test.go b/modules/surf/errorpage_test.go new file mode 100644 index 00000000..699cf898 --- /dev/null +++ b/modules/surf/errorpage_test.go @@ -0,0 +1,125 @@ +package surf + +import ( + "net/http" + "net/http/httptest" + "strings" + "testing" + + "git.golem15.com/golem15/summercms/modules/pact" + "git.golem15.com/golem15/summercms/modules/party" +) + +const chromeAccept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8" + +// TestErrorPageTracer: a browser navigation to an unmatched path gets the +// branded 404 through surf.Assemble; curl keeps the stdlib text. +func TestErrorPageTracer(t *testing.T) { + app := transportApp(t, "production", map[string]string{"http.yaml": transportBodyLimits}) + plugin := routesPluginFunc{id: "acme.ping", routes: func(r pact.Router) { + r.Get("/api/ping", func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte("pong")) }) + }} + h, err := Assemble(app, []party.Plugin{plugin}) + if err != nil { + t.Fatal(err) + } + + req := httptest.NewRequest(http.MethodGet, "/nowhere", nil) + req.Header.Set("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") + req.Header.Set("Sec-Fetch-Mode", "navigate") + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + if rec.Code != http.StatusNotFound { + t.Fatalf("browser status = %d, want 404", rec.Code) + } + if got := rec.Header().Get("Content-Type"); got != "text/html; charset=utf-8" { + t.Fatalf("browser Content-Type = %q", got) + } + if got := rec.Header().Get("Cache-Control"); got != "no-store" { + t.Fatalf("Cache-Control = %q", got) + } + if got := rec.Header().Get("Content-Security-Policy"); !strings.HasPrefix(got, "default-src 'none'") { + t.Fatalf("Content-Security-Policy = %q", got) + } + body := rec.Body.String() + for _, want := range []string{"404", `href="/"`, "data:image/webp;base64,"} { + if !strings.Contains(body, want) { + t.Fatalf("browser body lacks %q:\n%s", want, body) + } + } + if strings.Contains(body, "Not found

Nothing here

Home` + return []byte(body), true +} + +// ThemePlugin is the acme.theme plugin; Register publishes its error page. +type ThemePlugin struct{} + +func (ThemePlugin) ID() string { return "acme.theme" } +func (ThemePlugin) Requires() []string { return nil } +func (ThemePlugin) Boot(*backpack.App) error { return nil } + +// Register publishes the theme's error page for surf to look up. +func (ThemePlugin) Register(app *backpack.App) error { + return app.Publish[surf.ErrorPageRenderer](themePages{}) +} + +func ExampleErrorPageRenderer() { + app := backpack.New(nil) + plugin := ThemePlugin{} + if err := plugin.Register(app); err != nil { // the runtime calls Register + fmt.Println(err) + return + } + h, err := surf.Assemble(app, []party.Plugin{plugin}) + if err != nil { + fmt.Println(err) + return + } + browser := httptest.NewRequest("GET", "/nowhere", nil) + browser.Header.Set("Accept", "text/html,application/xhtml+xml,*/*;q=0.8") + browser.Header.Set("Sec-Fetch-Mode", "navigate") + curl := httptest.NewRequest("GET", "/nowhere", nil) + curl.Header.Set("Accept", "*/*") + for _, req := range []*http.Request{browser, curl} { + rec := httptest.NewRecorder() + h.ServeHTTP(rec, req) + fmt.Println(rec.Code, rec.Header().Get("Content-Type"), strings.TrimSpace(rec.Body.String())) + } + // Output: + // 404 text/html; charset=utf-8 Not found

Nothing here

Home + // 404 text/plain; charset=utf-8 404 page not found +} + // TestDocsDeclarations checks the plugin declarations the Routing and Rate -// limiting pages show. +// limiting pages show, the error-page override included. func TestDocsDeclarations(t *testing.T) { p := &BlogPlugin{} if _, ok := p.Middlewares()["acme.auth"]; !ok { @@ -247,4 +299,11 @@ func TestDocsDeclarations(t *testing.T) { if spec, err := p.APIDocsSpec(); err != nil || !strings.HasPrefix(string(spec), "openapi:") { t.Fatalf("APIDocsSpec() = %q, %v", spec, err) } + req := httptest.NewRequest("GET", "/nowhere", nil) + if body, ok := (themePages{}).RenderErrorPage(req, surf.ErrorPage{Status: http.StatusNotFound, Home: "/"}); !ok || !strings.Contains(string(body), `href="/"`) { + t.Fatalf("RenderErrorPage(404) = %q, %v", body, ok) + } + if _, ok := (themePages{}).RenderErrorPage(req, surf.ErrorPage{Status: http.StatusInternalServerError}); ok { + t.Fatal("RenderErrorPage(500) should keep the built-in page") + } } diff --git a/modules/surf/overlap.go b/modules/surf/overlap.go index e5e50a33..4bb45fd3 100644 --- a/modules/surf/overlap.go +++ b/modules/surf/overlap.go @@ -391,6 +391,7 @@ type familyDispatch struct { members []familyMember methods []string // every method in the router's table, for Allow mux *http.ServeMux + pages *errorPages // nil keeps the stdlib 404/405 text } func (f *familyDispatch) ServeHTTP(w http.ResponseWriter, r *http.Request) { @@ -421,16 +422,16 @@ func (f *familyDispatch) ServeHTTP(w http.ResponseWriter, r *http.Request) { return } if methodSeen { - http.NotFound(w, r) + f.pages.notFound(w, r) return } allow := f.allowed(r, vals) if len(allow) == 0 { - http.NotFound(w, r) + f.pages.notFound(w, r) return } w.Header().Set("Allow", strings.Join(allow, ", ")) - http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed) + f.pages.methodNotAllowed(w, r) } // allowed lists the methods ServeMux would put in Allow for this path if the @@ -466,7 +467,7 @@ func (f *familyDispatch) allowed(r *http.Request, vals []string) []string { // registerOverlapFamilies registers each family pattern on mux, then checks // that no other route takes a member's requests away from the family. -func registerOverlapFamilies(mux *http.ServeMux, routes []route, handlers []http.Handler, fams []overlapFamily) error { +func registerOverlapFamilies(mux *http.ServeMux, pages *errorPages, routes []route, handlers []http.Handler, fams []overlapFamily) error { if len(fams) == 0 { return nil } @@ -480,7 +481,7 @@ func registerOverlapFamilies(mux *http.ServeMux, routes []route, handlers []http } sort.Strings(methods) for _, fam := range fams { - d := &familyDispatch{pattern: fam.pattern, wild: fam.wild, methods: methods, mux: mux} + d := &familyDispatch{pattern: fam.pattern, wild: fam.wild, methods: methods, mux: mux, pages: pages} d.lits = make([]string, len(fam.wild)) for _, idx := range fam.members { rt := routes[idx] diff --git a/modules/surf/params.go b/modules/surf/params.go index f15e9fd8..7fb33f0b 100644 --- a/modules/surf/params.go +++ b/modules/surf/params.go @@ -78,14 +78,16 @@ func (c Constraint) match(value string) bool { return false } -func constrain(next http.Handler, cs []Constraint) http.Handler { +// constrain answers 404 (the error page for a browser navigation, pages +// permitting) when a Where/WhereIn constraint fails. +func constrain(pages *errorPages, next http.Handler, cs []Constraint) http.Handler { if len(cs) == 0 { return next } return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { for _, c := range cs { if !c.match(r.PathValue(c.param)) { - http.NotFound(w, r) + pages.notFound(w, r) return } } diff --git a/modules/surf/router.go b/modules/surf/router.go index 6c4d2b56..a787ce9d 100644 --- a/modules/surf/router.go +++ b/modules/surf/router.go @@ -68,6 +68,9 @@ type Router struct { app *backpack.App adminPrefix string adminMounted bool + // pages writes the HTML error page for browser navigations; nil keeps + // every error response as it was (errorpage.go). + pages *errorPages } var ( @@ -381,10 +384,10 @@ func (r *Router) compile() (http.Handler, error) { return nil, err } } - if err := registerOverlapFamilies(mux, r.routes, handlers, families); err != nil { + if err := registerOverlapFamilies(mux, r.pages, r.routes, handlers, families); err != nil { return nil, err } - h := r.stripLocalePrefix(pathScopedCORS(r.corsCfg, mux)) + h := r.stripLocalePrefix(pathScopedCORS(r.corsCfg, r.pages.fallback(mux))) if accessLogEnabled(r.app) { h = accessLog(r.app, h) } @@ -405,10 +408,10 @@ func handleRoute(mux *http.ServeMux, rt route, h http.Handler) (err error) { func (r *Router) wrap(rt route) (http.Handler, error) { if rt.preflight { // Preflights carry no credentials: no auth, throttle, body or locale. - h := recoverBare(constrain(rt.handler, rt.constraints)) + h := recoverBare(r.pages, constrain(r.pages, rt.handler, rt.constraints)) return recordPattern(rt.method+" "+rt.path, h), nil } - h := constrain(rt.handler, rt.constraints) + h := constrain(r.pages, rt.handler, rt.constraints) limit, err := routeBodyLimit(rt, r.defaultBytes) if err != nil { return nil, fmt.Errorf("surf: plugin %q: %w", rt.pluginID, err) @@ -473,9 +476,9 @@ func (r *Router) wrap(rt route) (http.Handler, error) { } h = r.locale(h) if rt.raw { - h = recoverBare(h) + h = recoverBare(r.pages, h) } else { - h = recoverJSON(h) + h = recoverJSON(r.pages, h) } // The CORS profile wraps recovery, so a refusal from a guard listed // before it, a 413 from the body limit and a recovered 500 all carry @@ -502,6 +505,7 @@ func Assemble(app *backpack.App, plugins []party.Plugin) (http.Handler, error) { func BuildRouter(app *backpack.App, plugins []party.Plugin) (*Router, error) { r := New(corsOrigins(app)) r.app = app + r.pages = newErrorPages(app) trusted := TrustedProxies(nil) if app != nil { trusted = TrustedProxies(app.Config) @@ -645,6 +649,7 @@ func BuildRouter(app *backpack.App, plugins []party.Plugin) (*Router, error) { return nil, err } r.adminPrefix = admin.Prefix + r.pages.setAdminPrefix(admin.Prefix) r.adminMounted = true } for _, rt := range r.routes { @@ -775,12 +780,19 @@ func corsOrigins(app *backpack.App) []string { } } -func recoverJSON(next http.Handler) http.Handler { +// recoverJSON turns a panic into wire's opaque JSON 500, or the HTML error +// page for a browser navigation (pages permitting). The panic value is never +// written. +func recoverJSON(pages *errorPages, next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { buffered := newBufferedResponse() defer func() { if rec := recover(); rec != nil { markPanicked(r) + if pages.wants(r) { + pages.write(w, r, http.StatusInternalServerError) + return + } wire.WriteOpaque500(w) return } @@ -790,12 +802,18 @@ func recoverJSON(next http.Handler) http.Handler { }) } -func recoverBare(next http.Handler) http.Handler { +// recoverBare turns a panic into a bare 500, or the HTML error page for a +// browser navigation (pages permitting). +func recoverBare(pages *errorPages, next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { buffered := newBufferedResponse() defer func() { if rec := recover(); rec != nil { markPanicked(r) + if pages.wants(r) { + pages.write(w, r, http.StatusInternalServerError) + return + } w.WriteHeader(http.StatusInternalServerError) return } diff --git a/modules/surf/uploads.go b/modules/surf/uploads.go index c22af914..80ef382a 100644 --- a/modules/surf/uploads.go +++ b/modules/surf/uploads.go @@ -66,7 +66,7 @@ func mountUploads(r *Router, app *backpack.App) error { return fmt.Errorf("surf: route %s %s (plugin %q) is under the uploads prefix storage.uploads.public_path_prefix %s, which storage.uploads.serve mounts", rt.method, rt.path, rt.pluginID, prefix) } } - handler := uploadsHandler(app, prefix) + handler := uploadsHandler(app, r.pages, prefix) r.BindPlugin(uploadsPluginID) r.GroupRaw(prefix, nil, func(g pact.Router) { g.Get("/{path...}", handler) @@ -103,18 +103,19 @@ func uploadsPrefix(raw string) (string, error) { } // uploadsHandler resolves the published bucket and database on every request -// and answers 404 while either is missing. -func uploadsHandler(app *backpack.App, prefix string) http.HandlerFunc { +// and answers 404 (the error page for a browser navigation) while either is +// missing. +func uploadsHandler(app *backpack.App, pages *errorPages, prefix string) http.HandlerFunc { return func(w http.ResponseWriter, req *http.Request) { serveUploads(w, req, http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) { bucket, ok := app.Lookup[*blob.Bucket]() if !ok || bucket == nil { - http.NotFound(w, req) + pages.notFound(w, req) return } gdb, ok := app.Lookup[*gorm.DB]() if !ok || gdb == nil { - http.NotFound(w, req) + pages.notFound(w, req) return } attach.StaticHandlerPublic(bucket, prefix, gdb).ServeHTTP(w, req)