feat(19.3-04): cooler site-cache purge service with site_cache.purge_url and purge_token
- cooler.SiteCachePurger moved from the Journal plugin: loopback-only http, 32-character token, 2 s detached request, no token in logs or errors - NewSiteCachePurgerFor names caller keys (for plugin alias keys) - PurgeSiteCacheAfterCommit through lagoon.AfterCommit, one POST per save - SiteCacheFromApp for hooks outside serve; Install publishes the purger unless a plugin already did, and fails boot on an invalid config - caching docs section, cooler README, configuration row
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# cooler
|
||||
|
||||
Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, and a Redis-backed rate-limit store.
|
||||
Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, a Redis-backed rate-limit store, and the site-cache purge after admin saves.
|
||||
|
||||
`import "git.golem15.com/golem15/summercms/modules/cooler"`
|
||||
|
||||
@@ -8,7 +8,7 @@ Cache drivers and shared counters for SummerCMS: memory and Redis caches with a
|
||||
|
||||
## Overview
|
||||
|
||||
cooler holds the state that several application instances share. `cooler.Cache` is a Laravel-core cache (get, set, has, forget, remember, increment, flush) with per-entry TTLs under a per-application key prefix, kept in process by an otter cache or shared through Redis. The second piece is a Redis-backed rate-limit store: with `http.rate_limit.store: redis`, the `serve` command of [surf](../surf/README.md) calls `cooler.Install`, which builds a Redis client from `redis.url`, and publishes the store so the `throttle` middleware counts in Redis instead of in the process. Two instances behind one load balancer then enforce one limit together. It is the counterpart of Laravel's `Cache` facade with its array and Redis stores, and of the Redis store behind `RateLimiter` and the `throttle` middleware.
|
||||
cooler holds the state that several application instances share. `cooler.Cache` is a Laravel-core cache (get, set, has, forget, remember, increment, flush) with per-entry TTLs under a per-application key prefix, kept in process by an otter cache or shared through Redis. The second piece is a Redis-backed rate-limit store: with `http.rate_limit.store: redis`, the `serve` command of [surf](../surf/README.md) calls `cooler.Install`, which builds a Redis client from `redis.url`, and publishes the store so the `throttle` middleware counts in Redis instead of in the process. Two instances behind one load balancer then enforce one limit together. The third piece is the site-cache purge: with `site_cache.purge_url` and `site_cache.purge_token` set, an admin save in a plugin sends one bearer-token POST to the host site's purge route after the save commits, so a cached page shows the change on its next view. It is the counterpart of Laravel's `Cache` facade with its array and Redis stores, and of the Redis store behind `RateLimiter` and the `throttle` middleware.
|
||||
|
||||
## Role in the framework
|
||||
|
||||
@@ -19,6 +19,7 @@ Use cases:
|
||||
- Cache an expensive query or an upstream answer for a few minutes, in a plugin or a job.
|
||||
- Keep `throttle` limits exact when an application runs as several instances behind a load balancer.
|
||||
- Share one Redis connection setting (`redis.url`) between the framework services that need Redis.
|
||||
- Drop a server-rendered site's page cache right after an editor saves a FAQ, a menu or a post in the admin, instead of waiting for the cache's TTL.
|
||||
|
||||
## Features
|
||||
|
||||
@@ -35,6 +36,10 @@ Use cases:
|
||||
- Keys carry a per-application prefix: `cache.prefix`, or by default `app.name` lower-cased with every run of other characters than `a-z` and `0-9` replaced by `_`, then `:` (`summer:` when `app.name` is empty). Cache keys are `<prefix>cache:<key>`, rate-limit keys `<prefix>rl:<key>`.
|
||||
- `cooler.OpenRedis` accepts `redis://`, `rediss://` (TLS) and `unix://` URLs, does not dial at construction, and skips the `CLIENT SETINFO` and maintenance-notification handshakes.
|
||||
- Only `serve` installs cooler: `route:list` and `surf.Assemble` in tests keep the in-process store and never dial Redis.
|
||||
- Site-cache purge: `cooler.SiteCachePurger` sends `POST {"kind":…,"slug":…}` with `Authorization: Bearer <token>` to `site_cache.purge_url`. `cooler.PurgeSiteCacheAfterCommit` registers that request on the save transaction through `lagoon.AfterCommit`, so it runs only after the save commits and never for a save that rolled back. One request per committed save, no debounce.
|
||||
- A purge never fails a save: each request is bounded by `cooler.SiteCachePurgeTimeout` (2 s) and detached from the request's cancellation, and a failure, an error status or a timeout is one warning (`cooler: site cache purge failed`) with the kind, slug and status. A success is one info line (`cooler: site cache purged`) with the purged count the site answered. Log lines and errors never carry the token or the full URL.
|
||||
- The purge URL must be http or https with a host and without user info or a fragment; plain http is accepted only for a loopback host (127.0.0.0/8, `::1`, `localhost`), so the token never crosses a network in clear. The token needs at least `cooler.SiteCachePurgeMinTokenLength` (32) characters, the same minimum the site's route applies. Setting only one of the two keys, or an invalid value, fails `serve` at boot with an error that names the key.
|
||||
- `cooler.Install` publishes the purger at boot unless a plugin already published one (a plugin that still reads its own deprecated keys), and `cooler.SiteCacheFromApp` gives admin hooks the same purger lazily in commands and tests that do not run `serve`. Plugins take the save transaction from `cabana.TxFromContext` themselves; cooler imports neither surf nor cabana.
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -85,6 +90,40 @@ url: redis://10.0.0.7:6379/0
|
||||
|
||||
Nothing else changes: routes keep `throttle:60,1` or `throttle:<bucket>`, and `serve` publishes the Redis store before it builds the router.
|
||||
|
||||
An admin controller purges the host site's page cache after its saves. The hook runs inside the save transaction, and the purge waits for the commit:
|
||||
|
||||
```go
|
||||
package blog
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/cabana"
|
||||
"git.golem15.com/golem15/summercms/modules/cooler"
|
||||
)
|
||||
|
||||
// Article is the blog's article model.
|
||||
type Article struct {
|
||||
Slug string
|
||||
}
|
||||
|
||||
type articlesController struct {
|
||||
app func() *backpack.App
|
||||
}
|
||||
|
||||
// FormAfterUpdate purges the site cache once the save commits.
|
||||
func (c articlesController) FormAfterUpdate(ctx context.Context, model any) error {
|
||||
tx, ok := cabana.TxFromContext(ctx)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
purger := cooler.SiteCacheFromApp(c.app())
|
||||
cooler.PurgeSiteCacheAfterCommit(ctx, tx, purger, "article", model.(*Article).Slug)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
A tool outside `serve` can use the store directly:
|
||||
|
||||
```go
|
||||
@@ -133,6 +172,17 @@ func AllowExport(ctx context.Context, app *backpack.App, fallback cooler.Attempt
|
||||
| `cooler.NewRateLimitStore` | Builds a `cooler.RateLimitStore` from a go-redis client, a key prefix and a fallback. |
|
||||
| `cooler.RateLimitStore.Attempt` | Admits and counts one request in one Lua script; falls back on a Redis error. |
|
||||
| `cooler.OpenRedis` | Builds a go-redis client from a `redis://`, `rediss://` or `unix://` URL without dialing; errors never contain the URL. |
|
||||
| `cooler.SiteCachePurger` | Sends the site-cache purge request; a nil purger does nothing. |
|
||||
| `cooler.NewSiteCachePurger` | Builds the purger from the values of `site_cache.purge_url` and `site_cache.purge_token`; both empty returns nil; errors name those keys and never the token. |
|
||||
| `cooler.NewSiteCachePurgerFor` | The same with the config keys its errors name passed in, for a plugin that reads deprecated keys of its own. |
|
||||
| `cooler.SiteCachePurger.Purge` | Sends one purge request (2 s timeout, detached from cancellation) and logs the outcome; never fails. |
|
||||
| `cooler.SiteCachePurger.Endpoint` | The purge URL's host and path, for log lines. |
|
||||
| `cooler.PurgeSiteCacheAfterCommit` | Registers one purge to run after the given transaction commits (at once outside a transaction). |
|
||||
| `cooler.SiteCacheFromApp` | Returns the published purger, or builds it from the framework keys and publishes it; nil when unset or invalid. |
|
||||
| `cooler.SiteCachePurgeURLKey` | `site_cache.purge_url`. |
|
||||
| `cooler.SiteCachePurgeTokenKey` | `site_cache.purge_token`. |
|
||||
| `cooler.SiteCachePurgeTimeout` | Bound of one purge request (2 s). |
|
||||
| `cooler.SiteCachePurgeMinTokenLength` | Shortest accepted purge token (32). |
|
||||
| `cooler.RedisURLKey` | `redis.url`. |
|
||||
| `cooler.RateLimitStoreKey` | `http.rate_limit.store`. |
|
||||
| `cooler.CacheDriverKey` | `cache.driver`. |
|
||||
@@ -153,12 +203,15 @@ func AllowExport(ctx context.Context, app *backpack.App, fallback cooler.Attempt
|
||||
| `http.rate_limit.store` | `memory` | Where `throttle` counters live: `memory` (per instance) or `redis` (shared). Any other value fails at boot. Environment: `SUMMER_HTTP__RATE_LIMIT__STORE`. |
|
||||
| `redis.url` | none | Redis URL (`redis://`, `rediss://` or `unix://`). Required when a Redis cache or rate-limit store is selected; keep it in the environment when it carries a password. Environment: `SUMMER_REDIS__URL`. |
|
||||
| `cache.prefix` | from `app.name` | Prefix of every cooler key. Environment: `SUMMER_CACHE__PREFIX`. |
|
||||
| `site_cache.purge_url` | none | The host site's purge route, for example `http://127.0.0.1:3000/api/__site__/cache/purge`. http only to a loopback host, https otherwise. Set together with the token, or neither (no purge). Environment: `SUMMER_SITE_CACHE__PURGE_URL`. |
|
||||
| `site_cache.purge_token` | none | Bearer token of the purge request, at least 32 characters, the same value as the site's `NUXT_SITE_CACHE_PURGE_TOKEN`. Keep it in the environment. Environment: `SUMMER_SITE_CACHE__PURGE_TOKEN`. |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: [backpack](../backpack/README.md), [compass](../compass/README.md).
|
||||
- SummerCMS modules: [backpack](../backpack/README.md), [compass](../compass/README.md), [lagoon](../lagoon/README.md) (`lagoon.AfterCommit` for the after-commit purge).
|
||||
- Third-party: `github.com/redis/go-redis/v9` (Redis client), `github.com/maypok86/otter/v2` (memory driver).
|
||||
- Standard library: `context`, `errors`, `fmt`, `log/slog`, `strconv`, `strings`, `sync`, `time`.
|
||||
- Third-party (purge): `gorm.io/gorm` (the save transaction handle).
|
||||
- Standard library: `bytes`, `context`, `encoding/json`, `errors`, `fmt`, `io`, `log/slog`, `net`, `net/http`, `net/url`, `strconv`, `strings`, `sync`, `time`.
|
||||
- Tests additionally use `github.com/testcontainers/testcontainers-go` (Redis container).
|
||||
|
||||
## Testing
|
||||
@@ -167,4 +220,4 @@ func AllowExport(ctx context.Context, app *backpack.App, fallback cooler.Attempt
|
||||
go test ./modules/cooler/...
|
||||
```
|
||||
|
||||
`TestRedisCacheSmoke` starts a Redis container through testcontainers-go and needs Docker; `go test -short ./modules/cooler/...` skips it, and the other tests need no external services. The cross-instance rate-limit test lives in surf (`go test ./modules/surf -run TestRateLimitAcrossInstancesSmoke`), starts a Redis container through testcontainers-go and needs Docker; `-short` skips it.
|
||||
`TestSiteCachePurgeConfig` and `TestSiteCachePurgeRequest` cover the purge configuration and the request against an `httptest` receiver without external services. `TestRedisCacheSmoke` starts a Redis container through testcontainers-go and needs Docker; `go test -short ./modules/cooler/...` skips it, and the other tests need no external services. The cross-instance rate-limit test lives in surf (`go test ./modules/surf -run TestRateLimitAcrossInstancesSmoke`), starts a Redis container through testcontainers-go and needs Docker; `-short` skips it.
|
||||
|
||||
Reference in New Issue
Block a user