feat(19.3-01): share rate limits across instances through cooler's Redis store

- new module cooler: Install, Handle, OpenRedis (no URL in errors), RateLimitStore
  (atomic fixed-window Lua script with MemoryStore semantics, per-instance fallback)
- serve installs cooler and publishes the Redis surf.Store before BuildRouter;
  BuildRouter looks up a published Store, route:list keeps the memory store
- config keys http.rate_limit.store and redis.url; go-redis v9.23.0
- cooler README, root table row, surf README and rate-limiting, configuration
  and scaling docs
This commit is contained in:
Jakub Zych
2026-10-15 17:34:39 +02:00
parent 77dd11c804
commit fa4d89d0b9
15 changed files with 777 additions and 10 deletions

View File

@@ -99,6 +99,7 @@ The `migrate`, `migrate:status`, `migrate:rollback`, `serve` and admin commands
| [cabana](modules/cabana/README.md) | Schema-driven admin backend that compiles WinterCMS-style YAML list, form, filter and relation definitions at boot and serves them as a JSON admin API next to the embedded admin SPA. | The beach hut where the staff run the beach; it is the admin backend. |
| [compass](modules/compass/README.md) | Layered YAML configuration with per-environment directories, `SUMMER_` environment overrides, embedded plugin defaults and dot-path access. | Points you the right way; configuration tells the application how to behave in each environment. |
| [conga](modules/conga/README.md) | Background jobs on River over the shared Postgres pool: transactional dispatch, a `summer_jobs` progress record, in-process or dedicated workers, and a wall-clock scheduler. | A line of dancers moving one after another; jobs queue up and are worked through in order. |
| [cooler](modules/cooler/README.md) | Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, and a Redis-backed rate-limit store. | The cool box you bring to the beach; it keeps things fresh and at hand for everyone in the group. |
| [festival](modules/festival/README.md) | Typed, synchronous event bus with listener priorities, payload collection and stop-when-handled dispatch. | A summer festival announces events and the crowd reacts; it is the event bus. |
| [fetchguard](modules/fetchguard/README.md) | Guarded outbound HTTP client and fetcher that blocks private and reserved addresses, enforces host, size and timeout limits, and offers an explicit trusted mode for operator-configured endpoints. | A lifeguard for outbound fetches; it watches every request the application sends out and stops the dangerous ones. |
| [flare](modules/flare/README.md) | Web Push delivery with VAPID (RFC 8292) and aes128gcm payload encryption (RFC 8291) behind a small Pusher interface. | A signal flare reaches people far away; it sends Web Push notifications to browsers. |

View File

@@ -61,7 +61,7 @@ Some costs come over with the port unchanged, and a few framework behaviours nee
- Garbage collection pauses are sub-millisecond and do not matter at this scale.
> [!WARNING]
> Rate limits are counted per process. The throttle middleware that `serve` builds keeps its counters in memory (`surf.MemoryStore`, behind the `surf.Store` interface), and `serve` offers no way to supply a different store yet. With N replicas the effective limit is N times the configured one. Until a shared store exists, divide the per-route limits by the replica count, or enforce the limit at the load balancer. [Rate limiting](../services/rate-limiting.md#named-buckets) describes the buckets.
> Rate limits are counted per process by default. The throttle middleware keeps its counters in memory (`surf.MemoryStore`, behind the `surf.Store` interface), so with N replicas the effective limit is N times the configured one. Set `http.rate_limit.store: redis` and `redis.url`, and `serve` counts in Redis through [cooler](../../modules/cooler/README.md)'s `cooler.RateLimitStore`, shared by every replica; see [Sharing limits across instances](../services/rate-limiting.md#sharing-limits-across-instances).
## Scaling and operations

View File

@@ -33,7 +33,32 @@ Each `surf.Bucket` has the maximum number of requests, the window length (`Decay
A request over the limit gets a 429 response with the body `{"message":"Too Many Attempts."}` and a `Retry-After` header. Every throttled response carries `X-RateLimit-Limit` and `X-RateLimit-Remaining`, and a rejected one also `X-RateLimit-Reset`.
The limiter is a fixed window: the counter resets when the window ends, as Laravel's cache limiter does. Counters live in the process (`surf.MemoryStore`, behind the `surf.Store` interface), so each application instance counts on its own. Behind a load balancer with several instances, the effective limit is the configured limit times the number of instances.
The limiter is a fixed window: the counter resets when the window ends, as Laravel's cache limiter does. By default counters live in the process (`surf.MemoryStore`, behind the `surf.Store` interface). With this memory store each application instance counts on its own, so behind a load balancer with several instances the effective limit is the configured limit times the number of instances. [Sharing limits across instances](#sharing-limits-across-instances) moves the counters to Redis.
## Sharing limits across instances
Laravel shares its limiter counters through the cache store, usually Redis. SummerCMS does the same through [cooler](../../modules/cooler/README.md): set `http.rate_limit.store` to `redis` (the default is `memory`) and point `redis.url` at a Redis every instance reaches.
```yaml
# config/http.yaml
rate_limit:
store: redis
```
```yaml
# config/redis.yaml
url: redis://10.0.0.7:6379/0
```
Keep a URL that carries a password in the environment instead: `SUMMER_REDIS__URL=redis://:secret@10.0.0.7:6379/0`. `rediss://` connects over TLS and `unix://` through a socket.
When `serve` starts, it calls `cooler.Install`, which builds the Redis client and pings it, and publishes a `cooler.RateLimitStore` as the `surf.Store` before the router is built. Routes do not change: `throttle:60,1` and `throttle:<bucket>` count in Redis, with the same fixed-window rules as the memory store. Each attempt is one atomic Lua script, so two instances answering requests from the same client count one window together: with `throttle:2,1`, the third request in the minute gets a 429 whichever instance answers it.
Keys are `<prefix>rl:<key>`. The prefix is `cache.prefix`, or by default `app.name` lower-cased with other characters than letters and digits replaced by `_`, followed by `:`, so several applications can share one Redis.
When Redis does not answer, each attempt waits at most `cooler.AttemptTimeout` (500 ms) and then counts in a per-instance `surf.MemoryStore`, as if the memory store were selected, until Redis answers again. A warning is logged at most once a minute. Redis being down at start-up is a warning too, not a failed start; a missing or malformed `redis.url` fails the start with an error that names the key and never shows the URL.
Only `serve` connects to Redis. `route:list` builds the router with the memory store, and so does `surf.Assemble` in tests, so neither needs a running Redis.
## Client IP and trusted proxies

View File

@@ -53,6 +53,8 @@ These are the keys most applications set. Each module's reference page lists all
| `app.locale`, `app.fallback_locale` | Default and fallback translation locales. | [phrasebook](../../modules/phrasebook/README.md) |
| `http.body_limits.default_bytes`, `http.body_limits.upload_bytes` | Request body limits. Required by `serve` and `route:list`. | [surf](../../modules/surf/README.md) |
| `http.cors.*`, `http.trusted_proxies` | CORS and the proxies whose forwarded client IP is trusted. | [surf](../../modules/surf/README.md) |
| `http.rate_limit.store` | Where `throttle` counters live: `memory` (per instance, the default) or `redis` (shared by every instance). | [cooler](../../modules/cooler/README.md) |
| `redis.url` | Redis URL (`redis://`, `rediss://` or `unix://`), required when a Redis store is selected. Keep it in `SUMMER_REDIS__URL` when it carries a password. | [cooler](../../modules/cooler/README.md) |
| `http.access_log` | Logs one record per request with method, matched route pattern, status and duration. Defaults to `false`. | [surf](../../modules/surf/README.md) |
| `storage.uploads.bucket_url` | Uploads bucket, `file://` or `mem://`. Required by `serve`. | [lagoon](../../modules/lagoon/README.md) |
| `storage.uploads.serve` | Mounts `attach.StaticHandlerPublic` under `storage.uploads.public_path_prefix` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Defaults to `false`. | [surf](../../modules/surf/README.md) |

2
go.mod
View File

@@ -21,6 +21,7 @@ require (
github.com/knadh/koanf/v2 v2.3.6
github.com/microcosm-cc/bluemonday v1.0.27
github.com/nicksnyder/go-i18n/v2 v2.6.1
github.com/redis/go-redis/v9 v9.23.0
github.com/riverqueue/river v0.47.0
github.com/riverqueue/river/riverdriver/riverdatabasesql v0.47.0
github.com/riverqueue/river/rivertype v0.47.0
@@ -114,6 +115,7 @@ require (
go.opentelemetry.io/otel/sdk v1.44.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.44.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.uber.org/atomic v1.12.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect

12
go.sum
View File

@@ -76,6 +76,10 @@ github.com/aws/smithy-go v1.26.0 h1:9ouqbi+NyKP7fV3Te7UElCwdAb6Y8uk7LGwPE5tVe/s=
github.com/aws/smithy-go v1.26.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
@@ -181,6 +185,8 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw=
github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU=
github.com/knadh/koanf/maps v0.1.2 h1:RBfmAW5CnZT+PJ1CVc1QSJKf4Xu9kxfQgYVQSu8hpbo=
github.com/knadh/koanf/maps v0.1.2/go.mod h1:npD/QZY3V6ghQDdcQzl1W4ICNVTkohC8E73eI2xW4yI=
github.com/knadh/koanf/parsers/yaml v1.1.1 h1:u70vV5IyaM0HvONh8HoqBC97oTgO33KcpZbTLiKVinU=
@@ -240,6 +246,8 @@ github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/redis/go-redis/v9 v9.23.0 h1:/TTVdJa8BRNsybwwGGnmiYktHNlUXtbOQxlWpHBPL3s=
github.com/redis/go-redis/v9 v9.23.0/go.mod h1:EWP2UUk+XDaI1s5nQB5mablZN6vccXlmBQTyfnF87D0=
github.com/riverqueue/river v0.47.0 h1:j8HOEyiOE8gRRhVS2wllKams372TH1WWiH6xCakXHqc=
github.com/riverqueue/river v0.47.0/go.mod h1:Wgmwx475ZBd8lQnNrJgyG2DWH7BfyNiSAtv0rC9bJBQ=
github.com/riverqueue/river/riverdriver v0.47.0 h1:qU8VkjdMl9plqeRg57SxsDUM/i/eECaSYejZ7HynC60=
@@ -297,6 +305,8 @@ github.com/yuin/goldmark v1.8.6 h1:d0VcaP1sx9GkFVkoW+KtggpGi2KZ965i14b0+bDQST4=
github.com/yuin/goldmark v1.8.6/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/detectors/gcp v1.42.0 h1:kpt2PEJuOuqYkPcktfJqWWDjTEd/FNgrxcniL7kQrXQ=
@@ -317,6 +327,8 @@ go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRk
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.uber.org/atomic v1.12.0 h1:BvcXdFKuviU4fTL/f+SxdQ5qJX/Jix8pAkgdUcb3XOE=
go.uber.org/atomic v1.12.0/go.mod h1:I6c4cg+6HCxRjfjSsYtApoFILnpc0CGUdGkXVqbYVNk=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=

117
modules/cooler/README.md Normal file
View File

@@ -0,0 +1,117 @@
# cooler
Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, and a Redis-backed rate-limit store.
`import "git.golem15.com/golem15/summercms/modules/cooler"`
**Name:** The cool box you bring to the beach; it keeps things fresh and at hand for everyone in the group.
## Overview
cooler holds the state that several application instances share. Its first piece is a Redis-backed rate-limit store: with `http.rate_limit.store: redis`, the `serve` command of [surf](../surf/README.md) calls `cooler.Install`, which builds a Redis client from `redis.url`, and publishes the store so the `throttle` middleware counts in Redis instead of in the process. Two instances behind one load balancer then enforce one limit together. It is the counterpart of Laravel's Redis cache store behind `RateLimiter` and the `throttle` middleware.
## Role in the framework
A service module on `backpack` and `compass`, below `surf`: surf imports cooler, never the reverse. `cooler.RateLimitStore` has the method set of `surf.Store` (through `cooler.Attempter`), so surf takes no Redis dependency and cooler imports neither surf nor cabana. The framework wires cooler at boot; applications import nothing.
Use cases:
- Keep `throttle` limits exact when an application runs as several instances behind a load balancer.
- Share one Redis connection setting (`redis.url`) between the framework services that need Redis.
## Features
- `cooler.Install` reads `http.rate_limit.store` (`memory`, the default, or `redis`). With `redis` it builds the client from `redis.url` and pings it within `cooler.PingTimeout` (3 s).
- Start-up never fails because Redis is down: an unanswered ping is one warning that names the endpoint (host and port, never the URL), and the limiter counts per instance until Redis answers. A missing or malformed `redis.url`, or an unknown store name, fails start-up with an error that names the key and never contains the URL or its password.
- `cooler.RateLimitStore` runs one atomic Lua script per attempt with the semantics of `surf.MemoryStore`: the first attempt opens the window, a denied attempt does not increment, the window never extends, and the retry time is what is left of the window. Redis keeps the clock, so instance clock skew does not matter.
- Each attempt is bounded by `cooler.AttemptTimeout` (500 ms). When Redis fails, the attempt is counted in the fallback store surf passes in (a per-instance `surf.MemoryStore`) and a warning is logged at most once per minute.
- Keys carry a per-application prefix: `cache.prefix`, or by default `app.name` lower-cased with every run of other characters than `a-z` and `0-9` replaced by `_`, then `:` (`summer:` when `app.name` is empty). Rate-limit keys are `<prefix>rl:<key>`.
- `cooler.OpenRedis` accepts `redis://`, `rediss://` (TLS) and `unix://` URLs, does not dial at construction, and skips the `CLIENT SETINFO` and maintenance-notification handshakes.
- Only `serve` installs cooler: `route:list` and `surf.Assemble` in tests keep the in-process store and never dial Redis.
## Usage
Select the shared store in the application's config and keep the URL, which may carry a password, in the environment (`SUMMER_REDIS__URL`):
```yaml
# config/http.yaml
rate_limit:
store: redis
```
```yaml
# config/redis.yaml (or SUMMER_REDIS__URL=redis://:secret@10.0.0.7:6379/0)
url: redis://10.0.0.7:6379/0
```
Nothing else changes: routes keep `throttle:60,1` or `throttle:<bucket>`, and `serve` publishes the Redis store before it builds the router.
A tool outside `serve` can use the store directly:
```go
package blog
import (
"context"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/cooler"
)
// AllowExport admits five exports a minute per user across all instances.
func AllowExport(ctx context.Context, app *backpack.App, fallback cooler.Attempter, userKey string) (bool, error) {
client, err := cooler.OpenRedis(ctx, app.Config.String(cooler.RedisURLKey))
if err != nil {
return false, err
}
defer client.Close()
store := cooler.NewRateLimitStore(client, "acme:", fallback)
allowed, _, _ := store.Attempt("export|"+userKey, 5, time.Minute)
return allowed, nil
}
```
## API reference
| Identifier | Description |
|------------|-------------|
| `cooler.Install` | Reads cooler's config, opens and pings Redis when a Redis store is selected, and returns a `cooler.Handle`. Called by surf's `serve`. |
| `cooler.Handle` | What `cooler.Install` set up. |
| `cooler.Handle.Close` | Closes the Redis client `cooler.Install` opened, if any. |
| `cooler.Handle.RateLimitRedis` | Reports whether `http.rate_limit.store` selects the Redis store. |
| `cooler.Handle.RateLimitStore` | Returns the Redis rate-limit store with the given fallback, or the fallback itself when Redis is not selected. |
| `cooler.Attempter` | One atomic fixed-window admission; the method set of `surf.Store`. |
| `cooler.RateLimitStore` | Redis fixed-window store shared by all instances, with a fallback while Redis fails. |
| `cooler.NewRateLimitStore` | Builds a `cooler.RateLimitStore` from a go-redis client, a key prefix and a fallback. |
| `cooler.RateLimitStore.Attempt` | Admits and counts one request in one Lua script; falls back on a Redis error. |
| `cooler.OpenRedis` | Builds a go-redis client from a `redis://`, `rediss://` or `unix://` URL without dialing; errors never contain the URL. |
| `cooler.RedisURLKey` | `redis.url`. |
| `cooler.RateLimitStoreKey` | `http.rate_limit.store`. |
| `cooler.CachePrefixKey` | `cache.prefix`. |
| `cooler.PingTimeout` | Bound of the start-up ping (3 s). |
| `cooler.AttemptTimeout` | Bound of one rate-limit attempt on Redis (500 ms). |
## Configuration
`cooler.Install` reads these keys from the app's [compass](../compass/README.md) config:
| Key | Default | Controls |
|-----|---------|----------|
| `http.rate_limit.store` | `memory` | Where `throttle` counters live: `memory` (per instance) or `redis` (shared). Any other value fails at boot. Environment: `SUMMER_HTTP__RATE_LIMIT__STORE`. |
| `redis.url` | none | Redis URL (`redis://`, `rediss://` or `unix://`). Required when a Redis store is selected; keep it in the environment when it carries a password. Environment: `SUMMER_REDIS__URL`. |
| `cache.prefix` | from `app.name` | Prefix of every cooler key. Environment: `SUMMER_CACHE__PREFIX`. |
## Dependencies
- SummerCMS modules: [backpack](../backpack/README.md), [compass](../compass/README.md).
- Third-party: `github.com/redis/go-redis/v9` (Redis client).
- Standard library: `context`, `errors`, `fmt`, `log/slog`, `strings`, `sync`, `time`.
## Testing
```sh
go test ./modules/cooler/...
```
The smoke tests need no external services. The cross-instance rate-limit test lives in surf (`go test ./modules/surf -run TestRateLimitAcrossInstancesSmoke`), starts a Redis container through testcontainers-go and needs Docker; `-short` skips it.

216
modules/cooler/cooler.go Normal file
View File

@@ -0,0 +1,216 @@
// Package cooler holds the shared-state services of a SummerCMS application:
// a Redis-backed fixed-window rate-limit store that keeps limits across
// application instances, with the Redis client built from the one redis.url
// key. The framework installs it at boot (surf's serve command), so
// applications import nothing.
//
// cooler imports neither surf nor cabana: surf imports cooler, and the
// rate-limit store satisfies surf.Store structurally through Attempter.
package cooler
import (
"context"
"fmt"
"log/slog"
"strings"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/compass"
"github.com/redis/go-redis/v9"
)
// Config keys and defaults cooler reads.
const (
// RedisURLKey is the shared Redis URL (redis://, rediss:// or unix://),
// required when a Redis driver is selected. Environment: SUMMER_REDIS__URL.
RedisURLKey = "redis.url"
// CacheDriverKey selects the cache driver: memory (default) or redis.
CacheDriverKey = "cache.driver"
// CachePrefixKey is the per-application key prefix. The default derives
// from app.name.
CachePrefixKey = "cache.prefix"
// CacheMaxEntriesKey bounds the memory driver's entry count.
CacheMaxEntriesKey = "cache.memory.max_entries"
// RateLimitStoreKey selects where rate-limit counters live: memory
// (default, per instance) or redis (shared).
RateLimitStoreKey = "http.rate_limit.store"
// DefaultMaxEntries is the memory driver's size bound when
// cache.memory.max_entries is unset.
DefaultMaxEntries = 10000
// PingTimeout bounds the start-up Ping that Install sends to Redis.
PingTimeout = 3 * time.Second
// AttemptTimeout bounds one rate-limit attempt on Redis before the
// fallback store answers.
AttemptTimeout = 500 * time.Millisecond
)
// Driver names accepted by cache.driver and http.rate_limit.store.
const (
driverMemory = "memory"
driverRedis = "redis"
)
// defaultPrefix is the key prefix when neither cache.prefix nor app.name is set.
const defaultPrefix = "summer:"
// Attempter is one atomic fixed-window admission. It has exactly the method
// set of surf.Store, so any surf.Store is an Attempter and a RateLimitStore
// is a surf.Store.
type Attempter interface {
Attempt(key string, max int, decay time.Duration) (allowed bool, attempts int, retryAfter time.Duration)
}
// Handle is what Install set up. Close releases the Redis client Install
// opened, if any.
type Handle struct {
client *redis.Client
prefix string
rateRedis bool
log *slog.Logger
}
// Close closes the Redis client Install opened. It is safe on a nil Handle
// and when no Redis driver was selected.
func (h *Handle) Close() error {
if h == nil || h.client == nil {
return nil
}
err := h.client.Close()
h.client = nil
return err
}
// RateLimitRedis reports whether http.rate_limit.store selects the Redis store.
func (h *Handle) RateLimitRedis() bool {
return h != nil && h.rateRedis && h.client != nil
}
// RateLimitStore returns the Redis rate-limit store, falling back to fallback
// while Redis does not answer. When the Redis store is not selected it
// returns fallback itself.
func (h *Handle) RateLimitStore(fallback Attempter) Attempter {
if !h.RateLimitRedis() {
return fallback
}
s := NewRateLimitStore(h.client, h.prefix, fallback)
s.log = h.log
return s
}
// Install reads cooler's configuration and opens what it selects. With
// http.rate_limit.store set to redis it builds a client from redis.url and
// pings it within PingTimeout. An unreachable Redis is logged as a warning
// (the endpoint only, never the URL) and start-up continues: the rate limiter
// falls back to per-instance memory until Redis answers. A missing or invalid
// redis.url, or an unknown store name, is a boot error naming the key.
//
// Only the serve command calls Install; route:list and tests never dial Redis.
func Install(ctx context.Context, app *backpack.App) (*Handle, error) {
if app == nil {
return nil, fmt.Errorf("cooler: app is nil")
}
if ctx == nil {
ctx = context.Background()
}
cfg := app.Config
h := &Handle{prefix: keyPrefix(cfg), log: loggerFromApp(app)}
store, err := driverName(cfg, RateLimitStoreKey)
if err != nil {
return nil, err
}
h.rateRedis = store == driverRedis
if h.rateRedis {
client, err := openFromConfig(ctx, cfg, RateLimitStoreKey)
if err != nil {
return nil, err
}
h.client = client
ping(ctx, client, h.log)
}
return h, nil
}
// openFromConfig builds the Redis client from redis.url, naming the key that
// selected Redis when the URL is missing.
func openFromConfig(ctx context.Context, cfg *compass.Config, selectedBy string) (*redis.Client, error) {
raw := ""
if cfg != nil {
raw = strings.TrimSpace(cfg.String(RedisURLKey))
}
if raw == "" {
return nil, fmt.Errorf("cooler: %s is required when %s is %s (set it in config/redis.yaml or SUMMER_REDIS__URL)", RedisURLKey, selectedBy, driverRedis)
}
return OpenRedis(ctx, raw)
}
// ping checks Redis once within PingTimeout and logs a warning on failure.
func ping(ctx context.Context, client *redis.Client, log *slog.Logger) {
pctx, cancel := context.WithTimeout(ctx, PingTimeout)
defer cancel()
if err := client.Ping(pctx).Err(); err != nil {
log.Warn("cooler: redis did not answer at start-up; rate limits count per instance until it does",
"endpoint", redisEndpoint(client), "error", err)
}
}
// driverName reads a memory|redis selector. Empty means memory.
func driverName(cfg *compass.Config, key string) (string, error) {
if cfg == nil {
return driverMemory, nil
}
v := strings.ToLower(strings.TrimSpace(cfg.String(key)))
switch v {
case "", driverMemory:
return driverMemory, nil
case driverRedis:
return driverRedis, nil
}
return "", fmt.Errorf("cooler: %s must be %s or %s, got %q", key, driverMemory, driverRedis, v)
}
// keyPrefix returns the key prefix every cooler key starts with: cache.prefix
// when set, otherwise app.name lower-cased with every run of characters
// outside [a-z0-9] replaced by "_" (trimmed at both ends) and a trailing ":",
// or "summer:" when app.name is empty too. Cache keys are <prefix>cache:<key>,
// rate-limit keys <prefix>rl:<key>.
func keyPrefix(cfg *compass.Config) string {
if cfg == nil {
return defaultPrefix
}
if p := strings.TrimSpace(cfg.String(CachePrefixKey)); p != "" {
return p
}
name := slug(cfg.String("app.name"))
if name == "" {
return defaultPrefix
}
return name + ":"
}
func slug(s string) string {
var b strings.Builder
under := false
for _, r := range strings.ToLower(s) {
if (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') {
b.WriteRune(r)
under = false
continue
}
if !under {
b.WriteByte('_')
under = true
}
}
return strings.Trim(b.String(), "_")
}
func loggerFromApp(app *backpack.App) *slog.Logger {
if app != nil {
if log, ok := app.Lookup[*slog.Logger](); ok && log != nil {
return log
}
}
return slog.Default()
}

121
modules/cooler/ratelimit.go Normal file
View File

@@ -0,0 +1,121 @@
package cooler
import (
"context"
"fmt"
"log/slog"
"sync"
"time"
"github.com/redis/go-redis/v9"
)
// attemptScript is one atomic fixed-window admission with the semantics of
// surf.MemoryStore: a missing or expired key opens a window of decay
// milliseconds, a count at max denies without incrementing, otherwise the
// count is incremented. INCR keeps the TTL, so the window never extends.
// It returns {allowed (0|1), count, remaining window in ms}.
//
// KEYS[1] = <prefix>rl:<key>; ARGV[1] = max; ARGV[2] = decay in ms (>= 1).
var attemptScript = redis.NewScript(`
local max = tonumber(ARGV[1])
local decay = tonumber(ARGV[2])
local count = tonumber(redis.call('GET', KEYS[1]) or '0')
local ttl = redis.call('PTTL', KEYS[1])
if ttl < 0 then
count = 0
redis.call('SET', KEYS[1], 0, 'PX', decay)
ttl = decay
end
if count >= max then
return {0, count, ttl}
end
count = redis.call('INCR', KEYS[1])
return {1, count, ttl}
`)
// warnEvery bounds how often a Redis failure is logged.
const warnEvery = time.Minute
// RateLimitStore is a fixed-window rate-limit store on Redis, shared by every
// application instance that points at the same Redis. Its Attempt has the
// exact signature of surf.Store, so surf uses it without cooler importing
// surf. When Redis does not answer, Attempt counts in the fallback store
// (per instance) and logs a warning at most once per minute.
type RateLimitStore struct {
client *redis.Client
prefix string
fallback Attempter
log *slog.Logger
mu sync.Mutex
lastWarn time.Time
}
// NewRateLimitStore returns a store that keeps its counters under
// <prefix>rl:<key> on client. fallback answers while Redis fails; surf passes
// a surf.MemoryStore. A nil fallback admits every request during an outage.
func NewRateLimitStore(client *redis.Client, prefix string, fallback Attempter) *RateLimitStore {
return &RateLimitStore{client: client, prefix: prefix, fallback: fallback, log: slog.Default()}
}
// Attempt admits and counts one request when key is below max, in one
// atomic Redis script bounded by AttemptTimeout. The first attempt opens the
// window, a denied attempt does not increment, and retryAfter is the time
// left in the window.
func (s *RateLimitStore) Attempt(key string, max int, decay time.Duration) (allowed bool, attempts int, retryAfter time.Duration) {
ms := decay.Milliseconds()
if ms < 1 {
ms = 1
}
ctx, cancel := context.WithTimeout(context.Background(), AttemptTimeout)
defer cancel()
res, err := s.run(ctx, s.prefix+"rl:"+key, max, ms)
if err != nil {
s.warn(err)
if s.fallback == nil {
return true, 0, 0
}
return s.fallback.Attempt(key, max, decay)
}
ttl := time.Duration(res[2]) * time.Millisecond
if ttl < 0 {
ttl = 0
}
return res[0] == 1, int(res[1]), ttl
}
func (s *RateLimitStore) run(ctx context.Context, key string, max int, ms int64) ([]int64, error) {
if s == nil || s.client == nil {
return nil, fmt.Errorf("cooler: rate-limit store has no Redis client")
}
res, err := attemptScript.Run(ctx, s.client, []string{key}, max, ms).Int64Slice()
if err != nil {
return nil, err
}
if len(res) != 3 {
return nil, fmt.Errorf("cooler: rate-limit script returned %d values, want 3", len(res))
}
return res, nil
}
// warn logs a Redis failure at most once per warnEvery.
func (s *RateLimitStore) warn(err error) {
if s == nil {
return
}
s.mu.Lock()
now := time.Now()
if !s.lastWarn.IsZero() && now.Sub(s.lastWarn) < warnEvery {
s.mu.Unlock()
return
}
s.lastWarn = now
s.mu.Unlock()
log := s.log
if log == nil {
log = slog.Default()
}
log.Warn("cooler: redis rate-limit store failed; counting per instance until it answers",
"endpoint", redisEndpoint(s.client), "error", err)
}

46
modules/cooler/redis.go Normal file
View File

@@ -0,0 +1,46 @@
package cooler
import (
"context"
"errors"
"strings"
"github.com/redis/go-redis/v9"
"github.com/redis/go-redis/v9/maintnotifications"
)
// errInvalidRedisURL is the only error OpenRedis returns for a URL that does
// not parse. It never wraps or quotes the parser's error: url.Parse echoes the
// whole URL, password included.
var errInvalidRedisURL = errors.New("cooler: " + RedisURLKey + " is not a valid redis://, rediss:// or unix:// URL")
// OpenRedis builds a go-redis client from a redis:// (plain), rediss:// (TLS)
// or unix:// URL, such as the value of redis.url. It does not dial: go-redis
// connects on the first command. A URL that does not parse yields an error that
// names redis.url and never contains the URL or its credentials.
//
// The client skips the CLIENT SETINFO and maintenance-notification handshakes,
// which older servers and test doubles do not know.
func OpenRedis(ctx context.Context, rawURL string) (*redis.Client, error) {
_ = ctx // reserved: go-redis does not dial at construction
rawURL = strings.TrimSpace(rawURL)
if rawURL == "" {
return nil, errors.New("cooler: " + RedisURLKey + " is empty")
}
opt, err := redis.ParseURL(rawURL)
if err != nil {
return nil, errInvalidRedisURL
}
opt.DisableIdentity = true
opt.MaintNotificationsConfig = &maintnotifications.Config{Mode: maintnotifications.ModeDisabled}
return redis.NewClient(opt), nil
}
// redisEndpoint is the part of a client's address that may be logged: the
// host and port (or socket path), never the URL, user or password.
func redisEndpoint(c *redis.Client) string {
if c == nil {
return ""
}
return c.Options().Addr
}

View File

@@ -0,0 +1,36 @@
package cooler
import (
"strings"
"testing"
)
// Smoke tests only; plan 05 of phase 19.3 owns full coverage.
func TestOpenRedisRedactsSmoke(t *testing.T) {
ctx := t.Context()
_, err := OpenRedis(ctx, "redis://alice:s3cretpw@cache.example.com:6379/0\x7f")
if err == nil {
t.Fatal("a URL with a control character must not parse")
}
msg := err.Error()
if strings.Contains(msg, "alice") || strings.Contains(msg, "s3cretpw") {
t.Fatalf("error leaks credentials: %q", msg)
}
if !strings.Contains(msg, RedisURLKey) {
t.Fatalf("error should name %s: %q", RedisURLKey, msg)
}
c, err := OpenRedis(ctx, "rediss://user:pw@cache.example.com:6380/1")
if err != nil {
t.Fatalf("rediss URL: %v", err)
}
defer c.Close()
opt := c.Options()
if opt.TLSConfig == nil || opt.Addr != "cache.example.com:6380" || opt.DB != 1 {
t.Fatalf("rediss options = addr %q db %d tls %v", opt.Addr, opt.DB, opt.TLSConfig != nil)
}
if !opt.DisableIdentity {
t.Fatal("OpenRedis must disable the CLIENT SETINFO handshake")
}
}

View File

@@ -12,7 +12,7 @@ surf turns the routes that plugins declare through `pact.HasRoutes` into one `ht
## Role in the framework
The HTTP entry point, at the top of the stack: it depends on `backpack`, `bonfire`, `bouncer`, `cabana`, `compass`, `conga`, `lagoon`, `pact`, `party`, `towel` and `wire`, and provides the serve command. Plugins contribute routes through `pact`; `surf` collects them into one standard library ServeMux.
The HTTP entry point, at the top of the stack: it depends on `backpack`, `bonfire`, `bouncer`, `cabana`, `compass`, `conga`, `cooler`, `lagoon`, `pact`, `party`, `towel` and `wire`, and provides the serve command. Plugins contribute routes through `pact`; `surf` collects them into one standard library ServeMux.
Use cases:
@@ -28,7 +28,7 @@ Use cases:
- Named middleware from plugins (`pact.HasMiddleware`), parameterized middleware used as `name:param` (`pact.HasMiddlewareFactories`) and house middleware for the JSON envelope and error handling (`pact.HasHouseMiddleware`). Duplicate or unknown names fail boot.
- Raw groups (`surf.Router.GroupRaw`) for routes that must not be wrapped in house middleware, such as webhooks or file streams: house middleware is refused there, the default body limit is skipped and a panic returns a bare 500.
- Built-in middleware names: `throttle:<bucket>` or `throttle:<max>,<minutes>`, `body.limit:<bytes>`, `locale.from-principal`, plus `backend` (the admin guard) when the admin is enabled.
- Fixed-window rate limiting (`surf.FixedWindowLimiter`): named buckets from plugins that implement `surf.BucketProvider`, or inline limits keyed by the signed-in user, or by client IP for guests. Rejected requests get a 429 with `Retry-After` and `X-RateLimit-*` headers. The in-process `surf.MemoryStore` sits behind the `surf.Store` interface.
- Fixed-window rate limiting (`surf.FixedWindowLimiter`): named buckets from plugins that implement `surf.BucketProvider`, or inline limits keyed by the signed-in user, or by client IP for guests. Rejected requests get a 429 with `Retry-After` and `X-RateLimit-*` headers. The in-process `surf.MemoryStore` sits behind the `surf.Store` interface; `surf.BuildRouter` uses a `surf.Store` published on the app instead when there is one, which is how `serve` shares counters across instances through [cooler](../cooler/README.md)'s Redis store (`http.rate_limit.store: redis`).
- Client IP resolution for limiter keys (`surf.ClientIP`) that only trusts `X-Forwarded-For` hops when the direct peer is inside a configured trusted proxy range (`surf.TrustedProxies`).
- Every non-raw route runs inside JSON panic recovery (an opaque 500 via [wire](../wire/README.md)), gets the request locale from a published `surf.LocaleResolver` when one exists or from the `Accept-Language` header otherwise (see [towel](../towel/README.md)), and a request body cap. Responses are buffered until the handler returns, so a panic never leaves a half-written body.
- Path-scoped CORS configured with the same keys as Laravel's `config/cors.php` (`surf.CORSConfig`), including preflight handling. Every `OPTIONS` request on a CORS path is answered with 204 before routing, with the headers Laravel's `HandleCors` sends: `Cache-Control: no-cache, private` always and, on a preflight (an `Origin` and an `Access-Control-Request-Method`), the requested method (upper-cased) and headers echoed in `Access-Control-Allow-Methods` and `Access-Control-Allow-Headers` when `*` allows any, `Vary` on the request headers and PHP's default `Content-Type: text/html; charset=UTF-8`.
@@ -156,6 +156,7 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur
| `http.cors.exposed_headers` | empty | Headers sent in `Access-Control-Expose-Headers`. |
| `http.cors.max_age` | `0` | Preflight cache time in seconds. |
| `http.cors.supports_credentials` | `false` | Sends `Access-Control-Allow-Credentials: true`. |
| `http.rate_limit.store` | `memory` | Where `throttle` counters live: `memory` (per instance) or `redis` (shared by every instance through [cooler](../cooler/README.md), which then needs `redis.url`). Only `serve` installs the Redis store; `route:list` keeps the memory store and never dials Redis. Any other value fails at boot. Environment: `SUMMER_HTTP__RATE_LIMIT__STORE`. |
| `http.access_log` | `false` | Logs one `http request` record per request (method, matched pattern, status, duration in milliseconds; never the path or query). Environment: `SUMMER_HTTP__ACCESS_LOG`. |
| `storage.uploads.serve` | `false` | Mounts the uploads route `GET <storage.uploads.public_path_prefix>/{path...}` (prefix default `/storage/uploads`; it must then be an absolute URL path, or start-up fails) on `attach.StaticHandlerPublic` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Prefer a separate origin for uploads in production. Environment: `SUMMER_STORAGE__UPLOADS__SERVE`. |
@@ -173,20 +174,21 @@ http:
supports_credentials: true
```
The `serve` command also opens the database through [lagoon](../lagoon/README.md) and the uploads bucket through `attach.OpenBucket`, so their settings must be present as well. It starts the background job worker of [conga](../conga/README.md) in the same process unless `queue.work_in_serve` is `false` (default `true`); set it to `false` when a separate `queue:work` process runs the jobs. The other `queue.*` keys are documented in the conga README.
The `serve` command also opens the database through [lagoon](../lagoon/README.md) and the uploads bucket through `attach.OpenBucket`, so their settings must be present as well. It then installs [cooler](../cooler/README.md) (`cooler.Install`) before it builds the router: with `http.rate_limit.store: redis` it publishes cooler's Redis-backed store as the `surf.Store`, falling back to a per-instance `surf.MemoryStore` while Redis does not answer. A Redis that is down at start-up is a warning, not a boot failure; a missing or malformed `redis.url` is. It starts the background job worker of [conga](../conga/README.md) in the same process unless `queue.work_in_serve` is `false` (default `true`); set it to `false` when a separate `queue:work` process runs the jobs. The other `queue.*` keys are documented in the conga README.
## CLI commands
| Command | Flags | Description |
|---------|-------|-------------|
| `serve` | `--addr` (default `:8080`) | Opens the database and uploads bucket, assembles the router, binds the address, starts the in-process job worker (see `queue.work_in_serve`) and serves HTTP until SIGINT or SIGTERM, then shuts the server and the worker down gracefully within 10 seconds. After binding it prints the listen address with a browser URL (an unspecified host shows as 127.0.0.1) and an admin status line: the admin URL, with a warning naming `summer admin:create` when no administrator exists yet, or `admin: off` with the steps to add an admin controller. |
| `serve` | `--addr` (default `:8080`) | Opens the database and uploads bucket, installs cooler (the shared rate-limit store when `http.rate_limit.store` is `redis`), assembles the router, binds the address, starts the in-process job worker (see `queue.work_in_serve`) and serves HTTP until SIGINT or SIGTERM, then shuts the server and the worker down gracefully within 10 seconds. After binding it prints the listen address with a browser URL (an unspecified host shows as 127.0.0.1) and an admin status line: the admin URL, with a warning naming `summer admin:create` when no administrator exists yet, or `admin: off` with the steps to add an admin controller. |
| `route:list` | none | Builds the router the same way `serve` does, without opening the database or listening, and prints a table of method, pattern, plugin, middleware and raw flag for every route, followed by the admin status line (`admin: mounted at <prefix>` or `admin: off`). |
## Dependencies
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [cabana](../cabana/README.md), [compass](../compass/README.md), [conga](../conga/README.md) (the in-process job worker of `serve`), [lagoon](../lagoon/README.md) (including `lagoon/attach`), [pact](../pact/README.md), [party](../party/README.md), [towel](../towel/README.md), [wire](../wire/README.md).
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [cabana](../cabana/README.md), [compass](../compass/README.md), [conga](../conga/README.md) (the in-process job worker of `serve`), [cooler](../cooler/README.md) (the shared rate-limit store `serve` installs), [lagoon](../lagoon/README.md) (including `lagoon/attach`), [pact](../pact/README.md), [party](../party/README.md), [towel](../towel/README.md), [wire](../wire/README.md).
- Third-party: `gocloud.dev/blob` (uploads bucket opened by `serve`).
- Standard library: `bytes`, `context`, `fmt`, `math`, `net`, `net/http`, `net/netip`, `os`, `os/signal`, `regexp`, `strconv`, `strings`, `sync`, `syscall`, `time`.
- Tests additionally use `github.com/testcontainers/testcontainers-go` (Redis container).
## Testing
@@ -194,4 +196,4 @@ The `serve` command also opens the database through [lagoon](../lagoon/README.md
go test ./modules/surf/...
```
The tests use `net/http/httptest` and in-memory stores and need no external services.
The tests use `net/http/httptest` and in-memory stores. `TestRateLimitAcrossInstancesSmoke` starts a Redis container through testcontainers-go and needs Docker; `go test -short ./modules/surf/...` skips it.

View File

@@ -472,8 +472,19 @@ func BuildRouter(app *backpack.App, plugins []party.Plugin) (*Router, error) {
if app != nil {
trusted = TrustedProxies(app.Config)
}
// longest bucket decay is 1 minute; sweep at 2x
lim := NewFixedWindowLimiter(NewMemoryStore(2*time.Minute), trusted)
// serve publishes a shared Store (cooler's Redis store) before building
// the router; route:list and tests get the in-process MemoryStore.
var store Store
if app != nil {
if s, ok := app.Lookup[Store](); ok && s != nil {
store = s
}
}
if store == nil {
// longest bucket decay is 1 minute; sweep at 2x
store = NewMemoryStore(2 * time.Minute)
}
lim := NewFixedWindowLimiter(store, trusted)
r.limiter = lim
if err := r.RegisterMiddlewareFactory("surf", "throttle", func(param string) pact.Middleware {
return lim.Middleware(param)

View File

@@ -16,6 +16,7 @@ import (
"git.golem15.com/golem15/summercms/modules/bonfire"
"git.golem15.com/golem15/summercms/modules/cabana"
"git.golem15.com/golem15/summercms/modules/conga"
"git.golem15.com/golem15/summercms/modules/cooler"
"git.golem15.com/golem15/summercms/modules/lagoon"
"git.golem15.com/golem15/summercms/modules/lagoon/attach"
"git.golem15.com/golem15/summercms/modules/party"
@@ -53,6 +54,11 @@ func ServeCommand(app *backpack.App, plugins []party.Plugin) bonfire.Command {
return err
}
defer bucket.Close()
coolerHandle, err := installCooler(ctx, app)
if err != nil {
return err
}
defer coolerHandle.Close()
r, err := BuildRouter(app, plugins)
if err != nil {
return err
@@ -139,6 +145,25 @@ func publishUploads(ctx context.Context, app *backpack.App) (*blob.Bucket, error
return bucket, nil
}
// installCooler installs cooler and, when http.rate_limit.store is redis,
// publishes its Redis-backed Store (falling back to a per-instance
// MemoryStore while Redis does not answer) so BuildRouter's limiter shares
// counters across instances. Only serve calls it: route:list never dials Redis.
func installCooler(ctx context.Context, app *backpack.App) (*cooler.Handle, error) {
h, err := cooler.Install(ctx, app)
if err != nil {
return nil, err
}
if h.RateLimitRedis() {
// longest bucket decay is 1 minute; sweep at 2x (as BuildRouter)
if err := app.Publish[Store](h.RateLimitStore(NewMemoryStore(2 * time.Minute))); err != nil {
_ = h.Close()
return nil, fmt.Errorf("surf: %w", err)
}
}
return h, nil
}
// adminCreateCommand is the command the startup hints name for creating the
// first backend administrator.
const adminCreateCommand = "summer admin:create --email you@example.com --superuser"

View File

@@ -0,0 +1,151 @@
package surf
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"sync/atomic"
"testing"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/compass"
"git.golem15.com/golem15/summercms/modules/party"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/wait"
)
// Smoke tests only; plan 05 of phase 19.3 owns full coverage.
const storeHTTPYAML = `body_limits:
default_bytes: 1048576
upload_bytes: 1048576
rate_limit:
store: redis
`
// redisStoreApp builds an app whose config selects the Redis rate-limit store
// at redisURL.
func redisStoreApp(t *testing.T, redisURL string) *backpack.App {
t.Helper()
dir := t.TempDir()
files := map[string]string{
"app.yaml": "name: store-test\n",
"http.yaml": storeHTTPYAML,
"redis.yaml": fmt.Sprintf("url: %q\n", redisURL),
}
for name, body := range files {
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
cfg, err := compass.Open(compass.Options{Dir: dir, Environ: []string{"SUMMER_ENV=development"}})
if err != nil {
t.Fatal(err)
}
return backpack.New(cfg)
}
type countingStore struct{ n atomic.Int64 }
func (s *countingStore) Attempt(_ string, max int, decay time.Duration) (bool, int, time.Duration) {
n := int(s.n.Add(1))
return n <= max, n, decay
}
func TestBuildRouterStoreSmoke(t *testing.T) {
// Redis selected but never installed (route:list's path): BuildRouter
// must not dial and keeps the in-process store.
app := redisStoreApp(t, "redis://127.0.0.1:1")
start := time.Now()
r, err := BuildRouter(app, nil)
if err != nil {
t.Fatal(err)
}
if d := time.Since(start); d > time.Second {
t.Fatalf("BuildRouter took %s; it must not dial Redis", d)
}
if _, ok := r.limiter.store.(*MemoryStore); !ok {
t.Fatalf("limiter store = %T, want *MemoryStore", r.limiter.store)
}
// A published Store is what the throttle counts in.
app = redisStoreApp(t, "redis://127.0.0.1:1")
fake := &countingStore{}
if err := app.Publish[Store](fake); err != nil {
t.Fatal(err)
}
p := assemblePlugin{id: "golem15.demo", path: "/ping", use: []string{"throttle:2,1"}}
h, err := Assemble(app, []party.Plugin{p})
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/ping", nil))
if rec.Code != http.StatusOK || fake.n.Load() != 1 {
t.Fatalf("status %d, fake attempts %d; want 200 through the published store", rec.Code, fake.n.Load())
}
}
func TestRateLimitAcrossInstancesSmoke(t *testing.T) {
if testing.Short() {
t.Skip("requires testcontainers redis")
}
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Minute)
defer cancel()
ctr, err := testcontainers.Run(ctx, "redis:7-alpine",
testcontainers.WithExposedPorts("6379/tcp"),
testcontainers.WithWaitStrategy(wait.ForListeningPort("6379/tcp")),
)
if err != nil {
t.Fatalf("redis: testcontainers: %v", err)
}
t.Cleanup(func() { _ = testcontainers.TerminateContainer(ctr) })
host, err := ctr.Host(ctx)
if err != nil {
t.Fatal(err)
}
port, err := ctr.MappedPort(ctx, "6379/tcp")
if err != nil {
t.Fatal(err)
}
url := fmt.Sprintf("redis://%s:%s/0", host, port.Port())
instance := func() http.Handler {
app := redisStoreApp(t, url)
hd, err := installCooler(ctx, app)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = hd.Close() })
p := assemblePlugin{id: "golem15.demo", path: "/ping", use: []string{"throttle:2,1"}}
h, err := Assemble(app, []party.Plugin{p})
if err != nil {
t.Fatal(err)
}
return h
}
a, b := instance(), instance()
get := func(h http.Handler) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodGet, "/api/ping", nil)
req.RemoteAddr = "192.0.2.10:4000"
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
return rec
}
for i, tc := range []struct {
h http.Handler
want int
}{{a, http.StatusOK}, {b, http.StatusOK}, {a, http.StatusTooManyRequests}} {
rec := get(tc.h)
if rec.Code != tc.want {
t.Fatalf("request %d: status %d, want %d", i+1, rec.Code, tc.want)
}
if tc.want == http.StatusTooManyRequests && rec.Header().Get("Retry-After") == "" {
t.Fatal("429 without Retry-After")
}
}
}