diff --git a/README.md b/README.md index dee9c85..d87cf1c 100644 --- a/README.md +++ b/README.md @@ -109,6 +109,8 @@ The pipeline runs normalisation, then redirects on the normalised path (followed {"action": "redirect", "status": 301, "location": "https://example.com/pages/b", "url": "https://example.com/pages/a", "locale": "", "staging": false, "seo": null} ``` +**Behind the frontend server.** The resolve calls and the 404 reports come from the frontend (SSR) server, not from the visitor, so the throttle buckets see that server's address unless it is listed in `http.trusted_proxies` and forwards the visitor's address in `X-Forwarded-For` (the rightmost hop outside the trusted list counts). Without that, `seo-resolve` and `seo-notfound` are caps for the whole site, 600 requests a minute each (about 10 rendered pages a second), and resolve answers `429` beyond them. Put the frontend server in `http.trusted_proxies` and have it forward `X-Forwarded-For`. + `status` is 302 when a temporary redirect takes part. A 410 rule answers `"action": "gone"`, `"status": 410`. A redirect cycle met at runtime (stored past the save check) stops after 10 hops with `not_found` and is logged at most once an hour per cycle (keyed by its rules, not by the requested URL). A path that no published record owns and no URL rule covers answers `"action": "not_found"`, `"status": 404` and `"seo": null` (the home page `/` is always known). A `url` that is neither a path nor an http(s) URL is a `422`. Absolute URLs are built from `golem15.seo.site_url`, or from the request URL when it names the site's host or its `www` twin (for the scheme and host rules), never from the request's Host header. ### The 404 report endpoint diff --git a/routes.go b/routes.go index ac4ae5c..8795042 100644 --- a/routes.go +++ b/routes.go @@ -87,7 +87,10 @@ func noindex(h http.HandlerFunc) http.HandlerFunc { // trusted proxies: seo-resolve 600 a minute (the frontend server resolves // every page it renders), seo-notfound 600 a minute (one report per 404 // page), seo-robots 120 a minute and seo-api 300 a minute (an MCP agent -// working through an audit). +// working through an audit). The resolve calls and 404 reports come from +// the frontend server, so that server must be a trusted proxy forwarding +// X-Forwarded-For, or the first two buckets cap the whole site (README, +// "Behind the frontend server"). func (p *Plugin) Buckets() map[string]surf.Bucket { trusted := surf.TrustedProxies(nil) if p.app != nil {