feat(19.3-04): cooler site-cache purge service with site_cache.purge_url and purge_token
- cooler.SiteCachePurger moved from the Journal plugin: loopback-only http, 32-character token, 2 s detached request, no token in logs or errors - NewSiteCachePurgerFor names caller keys (for plugin alias keys) - PurgeSiteCacheAfterCommit through lagoon.AfterCommit, one POST per save - SiteCacheFromApp for hooks outside serve; Install publishes the purger unless a plugin already did, and fails boot on an invalid config - caching docs section, cooler README, configuration row
This commit is contained in:
@@ -99,7 +99,7 @@ The `migrate`, `migrate:status`, `migrate:rollback`, `serve` and admin commands
|
||||
| [cabana](modules/cabana/README.md) | Schema-driven admin backend that compiles WinterCMS-style YAML list, form, filter and relation definitions at boot and serves them as a JSON admin API next to the embedded admin SPA. | The beach hut where the staff run the beach; it is the admin backend. |
|
||||
| [compass](modules/compass/README.md) | Layered YAML configuration with per-environment directories, `SUMMER_` environment overrides, embedded plugin defaults and dot-path access. | Points you the right way; configuration tells the application how to behave in each environment. |
|
||||
| [conga](modules/conga/README.md) | Background jobs on River over the shared Postgres pool: transactional dispatch, a `summer_jobs` progress record, in-process or dedicated workers, and a wall-clock scheduler. | A line of dancers moving one after another; jobs queue up and are worked through in order. |
|
||||
| [cooler](modules/cooler/README.md) | Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, and a Redis-backed rate-limit store. | The cool box you bring to the beach; it keeps things fresh and at hand for everyone in the group. |
|
||||
| [cooler](modules/cooler/README.md) | Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, a Redis-backed rate-limit store, and the site-cache purge after admin saves. | The cool box you bring to the beach; it keeps things fresh and at hand for everyone in the group. |
|
||||
| [festival](modules/festival/README.md) | Typed, synchronous event bus with listener priorities, payload collection and stop-when-handled dispatch. | A summer festival announces events and the crowd reacts; it is the event bus. |
|
||||
| [fetchguard](modules/fetchguard/README.md) | Guarded outbound HTTP client and fetcher that blocks private and reserved addresses, enforces host, size and timeout limits, and offers an explicit trusted mode for operator-configured endpoints. | A lifeguard for outbound fetches; it watches every request the application sends out and stops the dangerous ones. |
|
||||
| [flare](modules/flare/README.md) | Web Push delivery with VAPID (RFC 8292) and aes128gcm payload encryption (RFC 8291) behind a small Pusher interface. | A signal flare reaches people far away; it sends Web Push notifications to browsers. |
|
||||
|
||||
@@ -94,3 +94,77 @@ The `serve` command installs cooler at start-up (`cooler.Install`) and publishes
|
||||
## Flush
|
||||
|
||||
`cooler.Cache.Flush` removes this application's cache keys and nothing else. The Redis driver walks `SCAN MATCH <prefix>cache:*` in batches of 500 and removes each batch with `UNLINK`; it never empties the whole Redis database, which would also drop rate-limit counters and other applications' keys. The memory driver empties the process cache.
|
||||
|
||||
## Site cache purge
|
||||
|
||||
A server-rendered site in front of the API usually caches its pages for a few minutes (a Nuxt `routeRules` `swr` window, for example). Without help, an edit in the admin shows on the site only when that window ends. cooler closes the gap: after an admin save commits, it sends the site one purge request, and the site drops its route cache, so the next page view renders the change.
|
||||
|
||||
Configure the target once for the application:
|
||||
|
||||
| Key | Environment | Meaning |
|
||||
|-----|-------------|---------|
|
||||
| `site_cache.purge_url` | `SUMMER_SITE_CACHE__PURGE_URL` | The site's purge route, for example `http://127.0.0.1:3000/api/__site__/cache/purge`. |
|
||||
| `site_cache.purge_token` | `SUMMER_SITE_CACHE__PURGE_TOKEN` | The bearer token, at least 32 characters (`cooler.SiteCachePurgeMinTokenLength`). |
|
||||
|
||||
Set both or neither; with neither, nothing is purged. The URL must be http or https with a host, without user info or a fragment, and plain http is accepted only for a loopback host (127.0.0.0/8, `::1`, `localhost`): the site normally runs on the same machine, and the token must not cross a network in clear. Use https for any other host. A wrong or half-set configuration stops `serve` at start-up with an error that names the key; neither errors nor log lines ever contain the token.
|
||||
|
||||
### What a purge sends
|
||||
|
||||
Each committed save sends one request, with no debounce:
|
||||
|
||||
```http
|
||||
POST /api/__site__/cache/purge
|
||||
Authorization: Bearer <site_cache.purge_token>
|
||||
Content-Type: application/json
|
||||
|
||||
{"kind":"faq","slug":"shipping"}
|
||||
```
|
||||
|
||||
The kind names what was saved (a plugin picks it, for example `faq`, `menu`, `menu_entry`, `post` or `category`) and the slug which record; the site may log them. The site answers `{"purged": <n>}`, and cooler logs `cooler: site cache purged` with the kind, slug, status and count.
|
||||
|
||||
### Timing and failures
|
||||
|
||||
`cooler.PurgeSiteCacheAfterCommit` registers the request on the save's transaction through `lagoon.AfterCommit`. It is sent only after the commit, so a page view between the purge and the commit cannot re-cache the old content, and a save that fails or is rolled back sends nothing. Outside a transaction it is sent at once.
|
||||
|
||||
A purge never fails a save. The request is bounded by `cooler.SiteCachePurgeTimeout` (2 s) and detached from the admin request's cancellation; an unreachable site, an error status or a timeout is logged as one `cooler: site cache purge failed` warning and the save answers as usual.
|
||||
|
||||
### Purging from a plugin
|
||||
|
||||
The `serve` command publishes the purger at start-up (`cooler.Install`), and `cooler.SiteCacheFromApp` returns it, or builds it from the keys in commands and tests that do not run `serve`. An admin controller calls it from its after-save hooks, which run inside the save transaction, and takes that transaction from `cabana.TxFromContext`:
|
||||
|
||||
```go src=modules/cooler/example_test.go#articlesController.purge
|
||||
// purge registers one site-cache purge for the saved article, sent after the
|
||||
// save transaction commits (at once outside one). Without a purge configured
|
||||
// the purger is nil and nothing is sent.
|
||||
func (c articlesController) purge(ctx context.Context, model any) error {
|
||||
article, ok := model.(*Article)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
tx, _ := cabana.TxFromContext(ctx)
|
||||
cooler.PurgeSiteCacheAfterCommit(ctx, tx, cooler.SiteCacheFromApp(c.app()), "article", article.Slug)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
Each of `pact.FormAfterCreate`, `pact.FormAfterUpdate` and `pact.FormAfterDelete` calls it:
|
||||
|
||||
```go src=modules/cooler/example_test.go#articlesController.FormAfterUpdate
|
||||
// FormAfterUpdate runs inside the update's transaction.
|
||||
func (c articlesController) FormAfterUpdate(ctx context.Context, model any) error {
|
||||
return c.purge(ctx, model)
|
||||
}
|
||||
```
|
||||
|
||||
A nil purger (no purge configured) does nothing, so the hooks need no configuration check.
|
||||
|
||||
### The site side
|
||||
|
||||
The Nuxt layers of the core plugins (the Journal, menu and FAQ layers) each ship the same route, so a site that extends any of them has it:
|
||||
|
||||
- `POST /api/__site__/cache/purge` removes every entry of Nitro's route cache (pages, their `_payload.json` and feeds) and answers `{"purged": <n>}`.
|
||||
- Its token is `runtimeConfig.siteCache.purgeToken`, set through `NUXT_SITE_CACHE_PURGE_TOKEN` to the same value as `site_cache.purge_token`. With a token shorter than 32 characters the route answers 404.
|
||||
- It answers 404 to any request that came through a reverse proxy (`X-Forwarded-For`, `X-Real-IP` or `Forwarded`) and 401 to a missing or wrong bearer token, compared in constant time.
|
||||
- Each layer also ships the route rule `'/api/__site__/**': { cache: false }`. Without it, a site rule such as `'/**': { swr: 300 }` would wrap the purge route in Nitro's cached handler, which answers from the cache without running the route.
|
||||
|
||||
The three copies are byte-identical, so it does not matter which layer's copy Nitro picks.
|
||||
|
||||
@@ -55,6 +55,7 @@ These are the keys most applications set. Each module's reference page lists all
|
||||
| `http.cors.*`, `http.trusted_proxies` | CORS and the proxies whose forwarded client IP is trusted. | [surf](../../modules/surf/README.md) |
|
||||
| `http.rate_limit.store` | Where `throttle` counters live: `memory` (per instance, the default) or `redis` (shared by every instance). | [cooler](../../modules/cooler/README.md) |
|
||||
| `cache.driver`, `cache.prefix`, `cache.memory.max_entries` | The cache driver (`memory`, the default, or `redis`), the key prefix (default from `app.name`) and the memory driver's size bound (default 10000). | [cooler](../../modules/cooler/README.md) |
|
||||
| `site_cache.purge_url`, `site_cache.purge_token` | The host site's page-cache purge route and its bearer token (at least 32 characters). Set both or neither; http only to a loopback host. Keep the token in `SUMMER_SITE_CACHE__PURGE_TOKEN`. | [cooler](../../modules/cooler/README.md) |
|
||||
| `redis.url` | Redis URL (`redis://`, `rediss://` or `unix://`), required when a Redis cache or rate-limit store is selected. Keep it in `SUMMER_REDIS__URL` when it carries a password. | [cooler](../../modules/cooler/README.md) |
|
||||
| `http.access_log` | Logs one record per request with method, matched route pattern, status and duration. Defaults to `false`. | [surf](../../modules/surf/README.md) |
|
||||
| `storage.uploads.bucket_url` | Uploads bucket, `file://`, `mem://` or `s3://` (the last only in a binary built with the `summercms_s3` tag from `build.tags` in `summer.yaml`; credentials from the AWS environment). Required by `serve`. | [lagoon](../../modules/lagoon/README.md) |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# cooler
|
||||
|
||||
Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, and a Redis-backed rate-limit store.
|
||||
Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, a Redis-backed rate-limit store, and the site-cache purge after admin saves.
|
||||
|
||||
`import "git.golem15.com/golem15/summercms/modules/cooler"`
|
||||
|
||||
@@ -8,7 +8,7 @@ Cache drivers and shared counters for SummerCMS: memory and Redis caches with a
|
||||
|
||||
## Overview
|
||||
|
||||
cooler holds the state that several application instances share. `cooler.Cache` is a Laravel-core cache (get, set, has, forget, remember, increment, flush) with per-entry TTLs under a per-application key prefix, kept in process by an otter cache or shared through Redis. The second piece is a Redis-backed rate-limit store: with `http.rate_limit.store: redis`, the `serve` command of [surf](../surf/README.md) calls `cooler.Install`, which builds a Redis client from `redis.url`, and publishes the store so the `throttle` middleware counts in Redis instead of in the process. Two instances behind one load balancer then enforce one limit together. It is the counterpart of Laravel's `Cache` facade with its array and Redis stores, and of the Redis store behind `RateLimiter` and the `throttle` middleware.
|
||||
cooler holds the state that several application instances share. `cooler.Cache` is a Laravel-core cache (get, set, has, forget, remember, increment, flush) with per-entry TTLs under a per-application key prefix, kept in process by an otter cache or shared through Redis. The second piece is a Redis-backed rate-limit store: with `http.rate_limit.store: redis`, the `serve` command of [surf](../surf/README.md) calls `cooler.Install`, which builds a Redis client from `redis.url`, and publishes the store so the `throttle` middleware counts in Redis instead of in the process. Two instances behind one load balancer then enforce one limit together. The third piece is the site-cache purge: with `site_cache.purge_url` and `site_cache.purge_token` set, an admin save in a plugin sends one bearer-token POST to the host site's purge route after the save commits, so a cached page shows the change on its next view. It is the counterpart of Laravel's `Cache` facade with its array and Redis stores, and of the Redis store behind `RateLimiter` and the `throttle` middleware.
|
||||
|
||||
## Role in the framework
|
||||
|
||||
@@ -19,6 +19,7 @@ Use cases:
|
||||
- Cache an expensive query or an upstream answer for a few minutes, in a plugin or a job.
|
||||
- Keep `throttle` limits exact when an application runs as several instances behind a load balancer.
|
||||
- Share one Redis connection setting (`redis.url`) between the framework services that need Redis.
|
||||
- Drop a server-rendered site's page cache right after an editor saves a FAQ, a menu or a post in the admin, instead of waiting for the cache's TTL.
|
||||
|
||||
## Features
|
||||
|
||||
@@ -35,6 +36,10 @@ Use cases:
|
||||
- Keys carry a per-application prefix: `cache.prefix`, or by default `app.name` lower-cased with every run of other characters than `a-z` and `0-9` replaced by `_`, then `:` (`summer:` when `app.name` is empty). Cache keys are `<prefix>cache:<key>`, rate-limit keys `<prefix>rl:<key>`.
|
||||
- `cooler.OpenRedis` accepts `redis://`, `rediss://` (TLS) and `unix://` URLs, does not dial at construction, and skips the `CLIENT SETINFO` and maintenance-notification handshakes.
|
||||
- Only `serve` installs cooler: `route:list` and `surf.Assemble` in tests keep the in-process store and never dial Redis.
|
||||
- Site-cache purge: `cooler.SiteCachePurger` sends `POST {"kind":…,"slug":…}` with `Authorization: Bearer <token>` to `site_cache.purge_url`. `cooler.PurgeSiteCacheAfterCommit` registers that request on the save transaction through `lagoon.AfterCommit`, so it runs only after the save commits and never for a save that rolled back. One request per committed save, no debounce.
|
||||
- A purge never fails a save: each request is bounded by `cooler.SiteCachePurgeTimeout` (2 s) and detached from the request's cancellation, and a failure, an error status or a timeout is one warning (`cooler: site cache purge failed`) with the kind, slug and status. A success is one info line (`cooler: site cache purged`) with the purged count the site answered. Log lines and errors never carry the token or the full URL.
|
||||
- The purge URL must be http or https with a host and without user info or a fragment; plain http is accepted only for a loopback host (127.0.0.0/8, `::1`, `localhost`), so the token never crosses a network in clear. The token needs at least `cooler.SiteCachePurgeMinTokenLength` (32) characters, the same minimum the site's route applies. Setting only one of the two keys, or an invalid value, fails `serve` at boot with an error that names the key.
|
||||
- `cooler.Install` publishes the purger at boot unless a plugin already published one (a plugin that still reads its own deprecated keys), and `cooler.SiteCacheFromApp` gives admin hooks the same purger lazily in commands and tests that do not run `serve`. Plugins take the save transaction from `cabana.TxFromContext` themselves; cooler imports neither surf nor cabana.
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -85,6 +90,40 @@ url: redis://10.0.0.7:6379/0
|
||||
|
||||
Nothing else changes: routes keep `throttle:60,1` or `throttle:<bucket>`, and `serve` publishes the Redis store before it builds the router.
|
||||
|
||||
An admin controller purges the host site's page cache after its saves. The hook runs inside the save transaction, and the purge waits for the commit:
|
||||
|
||||
```go
|
||||
package blog
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/cabana"
|
||||
"git.golem15.com/golem15/summercms/modules/cooler"
|
||||
)
|
||||
|
||||
// Article is the blog's article model.
|
||||
type Article struct {
|
||||
Slug string
|
||||
}
|
||||
|
||||
type articlesController struct {
|
||||
app func() *backpack.App
|
||||
}
|
||||
|
||||
// FormAfterUpdate purges the site cache once the save commits.
|
||||
func (c articlesController) FormAfterUpdate(ctx context.Context, model any) error {
|
||||
tx, ok := cabana.TxFromContext(ctx)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
purger := cooler.SiteCacheFromApp(c.app())
|
||||
cooler.PurgeSiteCacheAfterCommit(ctx, tx, purger, "article", model.(*Article).Slug)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
A tool outside `serve` can use the store directly:
|
||||
|
||||
```go
|
||||
@@ -133,6 +172,17 @@ func AllowExport(ctx context.Context, app *backpack.App, fallback cooler.Attempt
|
||||
| `cooler.NewRateLimitStore` | Builds a `cooler.RateLimitStore` from a go-redis client, a key prefix and a fallback. |
|
||||
| `cooler.RateLimitStore.Attempt` | Admits and counts one request in one Lua script; falls back on a Redis error. |
|
||||
| `cooler.OpenRedis` | Builds a go-redis client from a `redis://`, `rediss://` or `unix://` URL without dialing; errors never contain the URL. |
|
||||
| `cooler.SiteCachePurger` | Sends the site-cache purge request; a nil purger does nothing. |
|
||||
| `cooler.NewSiteCachePurger` | Builds the purger from the values of `site_cache.purge_url` and `site_cache.purge_token`; both empty returns nil; errors name those keys and never the token. |
|
||||
| `cooler.NewSiteCachePurgerFor` | The same with the config keys its errors name passed in, for a plugin that reads deprecated keys of its own. |
|
||||
| `cooler.SiteCachePurger.Purge` | Sends one purge request (2 s timeout, detached from cancellation) and logs the outcome; never fails. |
|
||||
| `cooler.SiteCachePurger.Endpoint` | The purge URL's host and path, for log lines. |
|
||||
| `cooler.PurgeSiteCacheAfterCommit` | Registers one purge to run after the given transaction commits (at once outside a transaction). |
|
||||
| `cooler.SiteCacheFromApp` | Returns the published purger, or builds it from the framework keys and publishes it; nil when unset or invalid. |
|
||||
| `cooler.SiteCachePurgeURLKey` | `site_cache.purge_url`. |
|
||||
| `cooler.SiteCachePurgeTokenKey` | `site_cache.purge_token`. |
|
||||
| `cooler.SiteCachePurgeTimeout` | Bound of one purge request (2 s). |
|
||||
| `cooler.SiteCachePurgeMinTokenLength` | Shortest accepted purge token (32). |
|
||||
| `cooler.RedisURLKey` | `redis.url`. |
|
||||
| `cooler.RateLimitStoreKey` | `http.rate_limit.store`. |
|
||||
| `cooler.CacheDriverKey` | `cache.driver`. |
|
||||
@@ -153,12 +203,15 @@ func AllowExport(ctx context.Context, app *backpack.App, fallback cooler.Attempt
|
||||
| `http.rate_limit.store` | `memory` | Where `throttle` counters live: `memory` (per instance) or `redis` (shared). Any other value fails at boot. Environment: `SUMMER_HTTP__RATE_LIMIT__STORE`. |
|
||||
| `redis.url` | none | Redis URL (`redis://`, `rediss://` or `unix://`). Required when a Redis cache or rate-limit store is selected; keep it in the environment when it carries a password. Environment: `SUMMER_REDIS__URL`. |
|
||||
| `cache.prefix` | from `app.name` | Prefix of every cooler key. Environment: `SUMMER_CACHE__PREFIX`. |
|
||||
| `site_cache.purge_url` | none | The host site's purge route, for example `http://127.0.0.1:3000/api/__site__/cache/purge`. http only to a loopback host, https otherwise. Set together with the token, or neither (no purge). Environment: `SUMMER_SITE_CACHE__PURGE_URL`. |
|
||||
| `site_cache.purge_token` | none | Bearer token of the purge request, at least 32 characters, the same value as the site's `NUXT_SITE_CACHE_PURGE_TOKEN`. Keep it in the environment. Environment: `SUMMER_SITE_CACHE__PURGE_TOKEN`. |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: [backpack](../backpack/README.md), [compass](../compass/README.md).
|
||||
- SummerCMS modules: [backpack](../backpack/README.md), [compass](../compass/README.md), [lagoon](../lagoon/README.md) (`lagoon.AfterCommit` for the after-commit purge).
|
||||
- Third-party: `github.com/redis/go-redis/v9` (Redis client), `github.com/maypok86/otter/v2` (memory driver).
|
||||
- Standard library: `context`, `errors`, `fmt`, `log/slog`, `strconv`, `strings`, `sync`, `time`.
|
||||
- Third-party (purge): `gorm.io/gorm` (the save transaction handle).
|
||||
- Standard library: `bytes`, `context`, `encoding/json`, `errors`, `fmt`, `io`, `log/slog`, `net`, `net/http`, `net/url`, `strconv`, `strings`, `sync`, `time`.
|
||||
- Tests additionally use `github.com/testcontainers/testcontainers-go` (Redis container).
|
||||
|
||||
## Testing
|
||||
@@ -167,4 +220,4 @@ func AllowExport(ctx context.Context, app *backpack.App, fallback cooler.Attempt
|
||||
go test ./modules/cooler/...
|
||||
```
|
||||
|
||||
`TestRedisCacheSmoke` starts a Redis container through testcontainers-go and needs Docker; `go test -short ./modules/cooler/...` skips it, and the other tests need no external services. The cross-instance rate-limit test lives in surf (`go test ./modules/surf -run TestRateLimitAcrossInstancesSmoke`), starts a Redis container through testcontainers-go and needs Docker; `-short` skips it.
|
||||
`TestSiteCachePurgeConfig` and `TestSiteCachePurgeRequest` cover the purge configuration and the request against an `httptest` receiver without external services. `TestRedisCacheSmoke` starts a Redis container through testcontainers-go and needs Docker; `go test -short ./modules/cooler/...` skips it, and the other tests need no external services. The cross-instance rate-limit test lives in surf (`go test ./modules/surf -run TestRateLimitAcrossInstancesSmoke`), starts a Redis container through testcontainers-go and needs Docker; `-short` skips it.
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
// Package cooler holds the shared-state services of a SummerCMS application:
|
||||
// a Laravel-core Cache with a memory (otter) and a Redis driver under a
|
||||
// per-application key prefix, and a Redis-backed fixed-window rate-limit
|
||||
// store that keeps limits across application instances. One redis.url key
|
||||
// configures Redis for both. The framework installs cooler at boot (surf's
|
||||
// serve command) and FromApp serves other commands, so applications import
|
||||
// nothing.
|
||||
// per-application key prefix, a Redis-backed fixed-window rate-limit
|
||||
// store that keeps limits across application instances, and the site-cache
|
||||
// purge that asks the host site to drop its page caches after admin saves.
|
||||
// One redis.url key configures Redis for the cache and the store. The
|
||||
// framework installs cooler at boot (surf's serve command) and FromApp and
|
||||
// SiteCacheFromApp serve other commands, so applications import nothing.
|
||||
//
|
||||
// cooler imports neither surf nor cabana: surf imports cooler, and the
|
||||
// rate-limit store satisfies surf.Store structurally through Attempter.
|
||||
@@ -153,6 +154,11 @@ func (h *Handle) RateLimitStore(fallback Attempter) Attempter {
|
||||
// answers. A missing or invalid redis.url, an unknown driver name or a
|
||||
// non-positive cache.memory.max_entries is a boot error naming the key.
|
||||
//
|
||||
// Install also publishes the site-cache purger (SiteCachePurger) built from
|
||||
// site_cache.purge_url and site_cache.purge_token, unless a plugin already
|
||||
// published one; an invalid site_cache configuration is a boot error naming
|
||||
// the key, and an enabled purger is logged with its Endpoint.
|
||||
//
|
||||
// Only the serve command calls Install; route:list and tests never dial Redis.
|
||||
func Install(ctx context.Context, app *backpack.App) (*Handle, error) {
|
||||
if app == nil {
|
||||
@@ -163,6 +169,9 @@ func Install(ctx context.Context, app *backpack.App) (*Handle, error) {
|
||||
}
|
||||
cfg := app.Config
|
||||
h := &Handle{prefix: keyPrefix(cfg), log: loggerFromApp(app)}
|
||||
if err := installSiteCache(app, h.log); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
store, err := driverName(cfg, RateLimitStoreKey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -4,10 +4,17 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/cabana"
|
||||
"git.golem15.com/golem15/summercms/modules/compass"
|
||||
"git.golem15.com/golem15/summercms/modules/cooler"
|
||||
"git.golem15.com/golem15/summercms/modules/pact"
|
||||
)
|
||||
|
||||
func ExampleFromApp() {
|
||||
@@ -40,3 +47,79 @@ func ExampleFromApp() {
|
||||
fmt.Println(titles, loads)
|
||||
// Output: [Hello Summer] 1
|
||||
}
|
||||
|
||||
// Article is a blog's article model.
|
||||
type Article struct {
|
||||
Slug string
|
||||
}
|
||||
|
||||
// articlesController is a blog's admin controller (its pact.AdminController
|
||||
// methods are left out here).
|
||||
type articlesController struct {
|
||||
app func() *backpack.App
|
||||
}
|
||||
|
||||
// purge registers one site-cache purge for the saved article, sent after the
|
||||
// save transaction commits (at once outside one). Without a purge configured
|
||||
// the purger is nil and nothing is sent.
|
||||
func (c articlesController) purge(ctx context.Context, model any) error {
|
||||
article, ok := model.(*Article)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
tx, _ := cabana.TxFromContext(ctx)
|
||||
cooler.PurgeSiteCacheAfterCommit(ctx, tx, cooler.SiteCacheFromApp(c.app()), "article", article.Slug)
|
||||
return nil
|
||||
}
|
||||
|
||||
// FormAfterCreate runs inside the create's transaction.
|
||||
func (c articlesController) FormAfterCreate(ctx context.Context, model any) error {
|
||||
return c.purge(ctx, model)
|
||||
}
|
||||
|
||||
// FormAfterUpdate runs inside the update's transaction.
|
||||
func (c articlesController) FormAfterUpdate(ctx context.Context, model any) error {
|
||||
return c.purge(ctx, model)
|
||||
}
|
||||
|
||||
// FormAfterDelete runs inside the delete's transaction.
|
||||
func (c articlesController) FormAfterDelete(ctx context.Context, model any) error {
|
||||
return c.purge(ctx, model)
|
||||
}
|
||||
|
||||
var (
|
||||
_ pact.FormAfterCreate = articlesController{}
|
||||
_ pact.FormAfterUpdate = articlesController{}
|
||||
_ pact.FormAfterDelete = articlesController{}
|
||||
)
|
||||
|
||||
func ExampleSiteCacheFromApp() {
|
||||
// The host site's purge route, standing in on loopback.
|
||||
site := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
fmt.Println(r.Method, r.URL.Path, string(body))
|
||||
_, _ = w.Write([]byte(`{"purged":3}`))
|
||||
}))
|
||||
defer site.Close()
|
||||
|
||||
dir, err := os.MkdirTemp("", "cooler-example")
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
defer os.RemoveAll(dir)
|
||||
cfg, err := compass.Open(compass.Options{Dir: dir, Environ: []string{
|
||||
"SUMMER_SITE_CACHE__PURGE_URL=" + site.URL + "/api/__site__/cache/purge",
|
||||
"SUMMER_SITE_CACHE__PURGE_TOKEN=example-purge-token-0123456789abcdef",
|
||||
}})
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
app := backpack.New(cfg)
|
||||
c := articlesController{app: func() *backpack.App { return app }}
|
||||
if err := c.FormAfterUpdate(context.Background(), &Article{Slug: "hello-summer"}); err != nil {
|
||||
fmt.Println(err)
|
||||
}
|
||||
// Output: POST /api/__site__/cache/purge {"kind":"article","slug":"hello-summer"}
|
||||
}
|
||||
|
||||
260
modules/cooler/sitecache.go
Normal file
260
modules/cooler/sitecache.go
Normal file
@@ -0,0 +1,260 @@
|
||||
package cooler
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/lagoon"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
const (
|
||||
// SiteCachePurgeURLKey is the config key of the host site's cache purge
|
||||
// URL. Environment: SUMMER_SITE_CACHE__PURGE_URL.
|
||||
SiteCachePurgeURLKey = "site_cache.purge_url"
|
||||
// SiteCachePurgeTokenKey is the config key of the bearer token the purge
|
||||
// request carries. Environment: SUMMER_SITE_CACHE__PURGE_TOKEN.
|
||||
SiteCachePurgeTokenKey = "site_cache.purge_token"
|
||||
|
||||
// SiteCachePurgeTimeout bounds one purge request.
|
||||
SiteCachePurgeTimeout = 2 * time.Second
|
||||
|
||||
// SiteCachePurgeMinTokenLength is the shortest purge token accepted, the
|
||||
// same minimum the host site's purge route applies.
|
||||
SiteCachePurgeMinTokenLength = 32
|
||||
|
||||
// siteCacheAnswerLimit is how much of the purge answer is read.
|
||||
siteCacheAnswerLimit = 4 << 10
|
||||
)
|
||||
|
||||
// SiteCachePurger asks the host site to drop its page caches after an admin
|
||||
// save. A nil *SiteCachePurger is disabled and does nothing.
|
||||
type SiteCachePurger struct {
|
||||
url string
|
||||
endpoint string
|
||||
token string
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
// NewSiteCachePurger returns the purger for purgeURL and token, read from
|
||||
// site_cache.purge_url and site_cache.purge_token. Validation errors name
|
||||
// those keys; see NewSiteCachePurgerFor for the rules.
|
||||
func NewSiteCachePurger(purgeURL, token string) (*SiteCachePurger, error) {
|
||||
return NewSiteCachePurgerFor(SiteCachePurgeURLKey, SiteCachePurgeTokenKey, purgeURL, token)
|
||||
}
|
||||
|
||||
// NewSiteCachePurgerFor returns the purger for purgeURL and token, read
|
||||
// from the config keys urlKey and tokenKey, which its errors name. With
|
||||
// both values empty the purge is disabled and it returns nil, nil.
|
||||
// Otherwise both must be set, the URL must be http or https with a host and
|
||||
// without user info or a fragment, an http URL must point at a loopback
|
||||
// host (127.0.0.0/8, ::1 or localhost) so the bearer token never crosses
|
||||
// the network in clear, and the token must hold at least
|
||||
// SiteCachePurgeMinTokenLength characters. An error never contains the
|
||||
// token.
|
||||
func NewSiteCachePurgerFor(urlKey, tokenKey, purgeURL, token string) (*SiteCachePurger, error) {
|
||||
purgeURL = strings.TrimSpace(purgeURL)
|
||||
if purgeURL == "" && token == "" {
|
||||
return nil, nil
|
||||
}
|
||||
if purgeURL == "" {
|
||||
return nil, fmt.Errorf("%s is set but %s is not; set both or neither", tokenKey, urlKey)
|
||||
}
|
||||
if token == "" {
|
||||
return nil, fmt.Errorf("%s is set but %s is not; set both or neither", urlKey, tokenKey)
|
||||
}
|
||||
u, err := url.Parse(purgeURL)
|
||||
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" || u.Hostname() == "" {
|
||||
return nil, fmt.Errorf("%s must be an http or https URL with a host", urlKey)
|
||||
}
|
||||
if u.User != nil {
|
||||
return nil, fmt.Errorf("%s must not carry user info", urlKey)
|
||||
}
|
||||
if u.Fragment != "" || strings.Contains(purgeURL, "#") {
|
||||
return nil, fmt.Errorf("%s must not carry a fragment", urlKey)
|
||||
}
|
||||
if u.Scheme == "http" && !loopbackHost(u.Hostname()) {
|
||||
return nil, fmt.Errorf("%s: an http URL must point at a loopback host (127.0.0.0/8, ::1 or localhost); use https for any other host", urlKey)
|
||||
}
|
||||
if len(token) < SiteCachePurgeMinTokenLength {
|
||||
return nil, fmt.Errorf("%s must hold at least %d characters", tokenKey, SiteCachePurgeMinTokenLength)
|
||||
}
|
||||
return &SiteCachePurger{url: u.String(), endpoint: u.Host + u.Path, token: token, client: &http.Client{}}, nil
|
||||
}
|
||||
|
||||
// Endpoint returns the purge URL's host and path, without the scheme, user
|
||||
// info or query, for log lines.
|
||||
func (p *SiteCachePurger) Endpoint() string {
|
||||
if p == nil {
|
||||
return ""
|
||||
}
|
||||
return p.endpoint
|
||||
}
|
||||
|
||||
func loopbackHost(host string) bool {
|
||||
if strings.EqualFold(host, "localhost") {
|
||||
return true
|
||||
}
|
||||
ip := net.ParseIP(host)
|
||||
return ip != nil && ip.IsLoopback()
|
||||
}
|
||||
|
||||
type siteCachePurgeBody struct {
|
||||
Kind string `json:"kind"`
|
||||
Slug string `json:"slug"`
|
||||
}
|
||||
|
||||
// Purge sends the purge request for one saved entity: a POST of
|
||||
// {"kind":kind,"slug":slug} with the bearer token, bounded by
|
||||
// SiteCachePurgeTimeout and detached from ctx's cancellation. It logs the
|
||||
// outcome and returns nothing: a purge never fails a save. The token is
|
||||
// never logged.
|
||||
func (p *SiteCachePurger) Purge(ctx context.Context, kind, slug string) {
|
||||
if p == nil {
|
||||
return
|
||||
}
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), SiteCachePurgeTimeout)
|
||||
defer cancel()
|
||||
log := slog.Default()
|
||||
|
||||
raw, err := json.Marshal(siteCachePurgeBody{Kind: kind, Slug: slug})
|
||||
if err != nil {
|
||||
log.Warn("cooler: site cache purge failed", "kind", kind, "slug", slug, "error", err.Error())
|
||||
return
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, p.url, bytes.NewReader(raw))
|
||||
if err != nil {
|
||||
log.Warn("cooler: site cache purge failed", "kind", kind, "slug", slug, "error", "invalid request")
|
||||
return
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+p.token)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "application/json")
|
||||
|
||||
res, err := p.client.Do(req)
|
||||
if err != nil {
|
||||
log.Warn("cooler: site cache purge failed", "kind", kind, "slug", slug, "error", purgeTransportError(err))
|
||||
return
|
||||
}
|
||||
defer res.Body.Close()
|
||||
body, _ := io.ReadAll(io.LimitReader(res.Body, siteCacheAnswerLimit))
|
||||
if res.StatusCode < 200 || res.StatusCode > 299 {
|
||||
log.Warn("cooler: site cache purge failed", "kind", kind, "slug", slug, "status", res.StatusCode)
|
||||
return
|
||||
}
|
||||
attrs := []any{"kind", kind, "slug", slug, "status", res.StatusCode}
|
||||
var answer struct {
|
||||
Purged *int `json:"purged"`
|
||||
}
|
||||
if json.Unmarshal(body, &answer) == nil && answer.Purged != nil {
|
||||
attrs = append(attrs, "purged", *answer.Purged)
|
||||
}
|
||||
log.Info("cooler: site cache purged", attrs...)
|
||||
}
|
||||
|
||||
// purgeTransportError describes a failed request without the request URL
|
||||
// or headers.
|
||||
func purgeTransportError(err error) string {
|
||||
var uerr *url.Error
|
||||
if errors.As(err, &uerr) {
|
||||
err = uerr.Err
|
||||
}
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return "timeout"
|
||||
}
|
||||
return err.Error()
|
||||
}
|
||||
|
||||
// PurgeSiteCacheAfterCommit registers p.Purge(kind, slug) to run after the
|
||||
// transaction db belongs to commits, through lagoon.AfterCommit: a failed
|
||||
// save never purges, and a concurrent page view cannot re-cache the old
|
||||
// content between the purge and the commit. Each call sends one request
|
||||
// after its commit; there is no debounce. A nil p does nothing; outside a
|
||||
// transaction (db nil or not in one) the purge runs at once.
|
||||
func PurgeSiteCacheAfterCommit(ctx context.Context, db *gorm.DB, p *SiteCachePurger, kind, slug string) {
|
||||
if p == nil {
|
||||
return
|
||||
}
|
||||
lagoon.AfterCommit(ctx, db, func(ctx context.Context, _ *gorm.DB) {
|
||||
p.Purge(ctx, kind, slug)
|
||||
})
|
||||
}
|
||||
|
||||
// SiteCacheFromApp returns the *SiteCachePurger published on app (by
|
||||
// Install, by a plugin's alias configuration or by an earlier call), or
|
||||
// builds one from site_cache.purge_url and site_cache.purge_token and
|
||||
// publishes it. It returns nil when neither key is set, and when the
|
||||
// configuration is invalid, which it logs as one error naming the key (the
|
||||
// serve command fails at boot on the same configuration). Admin hooks call
|
||||
// it, so it serves commands and tests that never run serve.
|
||||
func SiteCacheFromApp(app *backpack.App) *SiteCachePurger {
|
||||
if app == nil {
|
||||
return nil
|
||||
}
|
||||
if p, ok := app.Lookup[*SiteCachePurger](); ok && p != nil {
|
||||
return p
|
||||
}
|
||||
p, err := siteCacheFromConfig(app)
|
||||
if err != nil {
|
||||
loggerFromApp(app).Error("cooler: site cache purge disabled", "error", err.Error())
|
||||
return nil
|
||||
}
|
||||
if p == nil {
|
||||
return nil
|
||||
}
|
||||
if err := app.Publish(p); err != nil {
|
||||
if winner, ok := app.Lookup[*SiteCachePurger](); ok && winner != nil {
|
||||
return winner
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
// siteCacheFromConfig builds the purger from the framework keys.
|
||||
func siteCacheFromConfig(app *backpack.App) (*SiteCachePurger, error) {
|
||||
if app == nil || app.Config == nil {
|
||||
return nil, nil
|
||||
}
|
||||
p, err := NewSiteCachePurger(app.Config.String(SiteCachePurgeURLKey), app.Config.String(SiteCachePurgeTokenKey))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("cooler: %w", err)
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
|
||||
// installSiteCache publishes the site-cache purger unless one is already
|
||||
// published (a plugin's alias configuration): an invalid configuration is a
|
||||
// boot error naming the key.
|
||||
func installSiteCache(app *backpack.App, log *slog.Logger) error {
|
||||
if p, ok := app.Lookup[*SiteCachePurger](); ok && p != nil {
|
||||
log.Info("cooler: site cache purge enabled", "endpoint", p.Endpoint())
|
||||
return nil
|
||||
}
|
||||
p, err := siteCacheFromConfig(app)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if p == nil {
|
||||
return nil
|
||||
}
|
||||
if err := app.Publish(p); err != nil {
|
||||
return fmt.Errorf("cooler: %w", err)
|
||||
}
|
||||
log.Info("cooler: site cache purge enabled", "endpoint", p.Endpoint())
|
||||
return nil
|
||||
}
|
||||
385
modules/cooler/sitecache_test.go
Normal file
385
modules/cooler/sitecache_test.go
Normal file
@@ -0,0 +1,385 @@
|
||||
package cooler
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/compass"
|
||||
)
|
||||
|
||||
const (
|
||||
purgeToken32 = "0123456789abcdef0123456789abcdef"
|
||||
purgeToken40 = "site-cache-unit-token-0123456789abcdefgh"
|
||||
)
|
||||
|
||||
// captureLogs routes slog.Default to a text handler on a buffer for the
|
||||
// test and returns the buffer.
|
||||
func captureLogs(t *testing.T) *syncBuffer {
|
||||
t.Helper()
|
||||
buf := &syncBuffer{}
|
||||
prev := slog.Default()
|
||||
slog.SetDefault(slog.New(slog.NewTextHandler(buf, &slog.HandlerOptions{Level: slog.LevelDebug})))
|
||||
t.Cleanup(func() { slog.SetDefault(prev) })
|
||||
return buf
|
||||
}
|
||||
|
||||
type syncBuffer struct {
|
||||
mu sync.Mutex
|
||||
buf bytes.Buffer
|
||||
}
|
||||
|
||||
func (b *syncBuffer) Write(p []byte) (int, error) {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
return b.buf.Write(p)
|
||||
}
|
||||
|
||||
func (b *syncBuffer) lines() []string {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
var out []string
|
||||
for _, line := range strings.Split(strings.TrimSpace(b.buf.String()), "\n") {
|
||||
if line != "" {
|
||||
out = append(out, line)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestSiteCachePurgeConfig(t *testing.T) {
|
||||
refused := []struct {
|
||||
name, url, token, key string
|
||||
}{
|
||||
{"only the URL", "http://127.0.0.1:3016/api/__site__/cache/purge", "", SiteCachePurgeTokenKey},
|
||||
{"only the token", "", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"a bad scheme", "ftp://127.0.0.1/purge", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"no scheme", "127.0.0.1:3016/purge", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"no host", "http:///purge", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"user info", "http://user:secret@127.0.0.1/purge", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"a fragment", "http://127.0.0.1/purge#top", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"an empty fragment", "http://127.0.0.1/purge#", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"http to a non-loopback host", "http://example.test/purge", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"http to a private address", "http://10.0.0.5:3016/purge", purgeToken40, SiteCachePurgeURLKey},
|
||||
{"a 31-character token", "http://127.0.0.1/purge", purgeToken32[:31], SiteCachePurgeTokenKey},
|
||||
}
|
||||
for _, tc := range refused {
|
||||
t.Run("refuses "+tc.name, func(t *testing.T) {
|
||||
p, err := NewSiteCachePurger(tc.url, tc.token)
|
||||
if err == nil || p != nil {
|
||||
t.Fatalf("NewSiteCachePurger(%q) = %v, %v; want an error", tc.url, p, err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.key) {
|
||||
t.Fatalf("error %q does not name %s", err, tc.key)
|
||||
}
|
||||
if tc.token != "" && strings.Contains(err.Error(), tc.token) {
|
||||
t.Fatalf("error %q carries the token", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("both empty disables the purge", func(t *testing.T) {
|
||||
p, err := NewSiteCachePurger("", "")
|
||||
if p != nil || err != nil {
|
||||
t.Fatalf("NewSiteCachePurger(\"\", \"\") = %v, %v; want nil, nil", p, err)
|
||||
}
|
||||
})
|
||||
|
||||
accepted := []struct {
|
||||
url, endpoint string
|
||||
}{
|
||||
{"https://cache.example.test/api/__site__/cache/purge", "cache.example.test/api/__site__/cache/purge"},
|
||||
{"http://127.0.0.1:3016/api/__site__/cache/purge", "127.0.0.1:3016/api/__site__/cache/purge"},
|
||||
{"http://127.9.8.7/purge", "127.9.8.7/purge"},
|
||||
{"http://[::1]:3000/purge", "[::1]:3000/purge"},
|
||||
{"http://localhost:3000/purge?x=1", "localhost:3000/purge"},
|
||||
}
|
||||
for _, tc := range accepted {
|
||||
t.Run("accepts "+tc.url, func(t *testing.T) {
|
||||
p, err := NewSiteCachePurger(tc.url, purgeToken32)
|
||||
if err != nil || p == nil {
|
||||
t.Fatalf("NewSiteCachePurger(%q) = %v, %v; want a purger", tc.url, p, err)
|
||||
}
|
||||
if p.Endpoint() != tc.endpoint {
|
||||
t.Fatalf("Endpoint() = %q, want %q", p.Endpoint(), tc.endpoint)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("errors name the keys a caller passes", func(t *testing.T) {
|
||||
const urlKey, tokenKey = "acme.site_cache.purge_url", "acme.site_cache.purge_token"
|
||||
if _, err := NewSiteCachePurgerFor(urlKey, tokenKey, "http://example.test/purge", purgeToken40); err == nil || !strings.Contains(err.Error(), urlKey) {
|
||||
t.Fatalf("http to a remote host: %v, want an error naming %s", err, urlKey)
|
||||
}
|
||||
if _, err := NewSiteCachePurgerFor(urlKey, tokenKey, "http://127.0.0.1/purge", purgeToken32[:31]); err == nil || !strings.Contains(err.Error(), tokenKey) {
|
||||
t.Fatalf("short token: %v, want an error naming %s", err, tokenKey)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("SiteCacheFromApp builds from the framework keys and publishes once", func(t *testing.T) {
|
||||
cfg := testConfig(t)
|
||||
app := backpack.New(cfg)
|
||||
if p := SiteCacheFromApp(app); p != nil {
|
||||
t.Fatalf("no keys: SiteCacheFromApp = %v, want nil", p)
|
||||
}
|
||||
if err := cfg.Set(SiteCachePurgeURLKey, "http://127.0.0.1:3000/api/__site__/cache/purge"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := cfg.Set(SiteCachePurgeTokenKey, purgeToken40); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
first := SiteCacheFromApp(app)
|
||||
if first == nil || first.Endpoint() != "127.0.0.1:3000/api/__site__/cache/purge" {
|
||||
t.Fatalf("SiteCacheFromApp = %v", first)
|
||||
}
|
||||
if again := SiteCacheFromApp(app); again != first {
|
||||
t.Fatal("a second call built another purger")
|
||||
}
|
||||
if got, ok := app.Lookup[*SiteCachePurger](); !ok || got != first {
|
||||
t.Fatal("the purger was not published")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("SiteCacheFromApp logs an invalid configuration and returns nil", func(t *testing.T) {
|
||||
logs := captureLogs(t)
|
||||
cfg := testConfig(t)
|
||||
if err := cfg.Set(SiteCachePurgeURLKey, "http://example.test/purge"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := cfg.Set(SiteCachePurgeTokenKey, purgeToken40); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if p := SiteCacheFromApp(backpack.New(cfg)); p != nil {
|
||||
t.Fatalf("SiteCacheFromApp = %v, want nil", p)
|
||||
}
|
||||
lines := logs.lines()
|
||||
if len(lines) != 1 || !strings.Contains(lines[0], "level=ERROR") || !strings.Contains(lines[0], SiteCachePurgeURLKey) {
|
||||
t.Fatalf("log = %q", lines)
|
||||
}
|
||||
assertNoToken(t, logs)
|
||||
})
|
||||
|
||||
t.Run("Install fails on an invalid configuration and keeps a published purger", func(t *testing.T) {
|
||||
cfg := testConfig(t)
|
||||
if err := cfg.Set(SiteCachePurgeURLKey, "http://127.0.0.1/purge"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := Install(context.Background(), backpack.New(cfg)); err == nil || !strings.Contains(err.Error(), SiteCachePurgeTokenKey) {
|
||||
t.Fatalf("Install with only the URL = %v, want an error naming %s", err, SiteCachePurgeTokenKey)
|
||||
}
|
||||
alias, err := NewSiteCachePurgerFor("acme.purge_url", "acme.purge_token", "http://127.0.0.1:4000/purge", purgeToken40)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
app := backpack.New(cfg)
|
||||
if err := app.Publish(alias); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h, err := Install(context.Background(), app)
|
||||
if err != nil {
|
||||
t.Fatalf("Install with a published purger: %v", err)
|
||||
}
|
||||
defer h.Close()
|
||||
if got := SiteCacheFromApp(app); got != alias {
|
||||
t.Fatal("Install replaced the published purger")
|
||||
}
|
||||
})
|
||||
|
||||
var nilPurger *SiteCachePurger
|
||||
if nilPurger.Endpoint() != "" {
|
||||
t.Fatal("a nil purger has an endpoint")
|
||||
}
|
||||
}
|
||||
|
||||
// purgeHit is one request the test receiver got.
|
||||
type purgeHit struct {
|
||||
method, path, auth, contentType string
|
||||
body map[string]string
|
||||
}
|
||||
|
||||
func purgeReceiver(t *testing.T, handle func(w http.ResponseWriter, r *http.Request)) (*httptest.Server, func() []purgeHit) {
|
||||
t.Helper()
|
||||
var mu sync.Mutex
|
||||
var hits []purgeHit
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
raw, _ := io.ReadAll(r.Body)
|
||||
var body map[string]string
|
||||
_ = json.Unmarshal(raw, &body)
|
||||
mu.Lock()
|
||||
hits = append(hits, purgeHit{r.Method, r.URL.Path, r.Header.Get("Authorization"), r.Header.Get("Content-Type"), body})
|
||||
mu.Unlock()
|
||||
handle(w, r)
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
return srv, func() []purgeHit {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return append([]purgeHit(nil), hits...)
|
||||
}
|
||||
}
|
||||
|
||||
func newTestPurger(t *testing.T, srv *httptest.Server) *SiteCachePurger {
|
||||
t.Helper()
|
||||
p, err := NewSiteCachePurger(srv.URL+"/api/__site__/cache/purge", purgeToken40)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func assertNoToken(t *testing.T, logs *syncBuffer) {
|
||||
t.Helper()
|
||||
for _, line := range logs.lines() {
|
||||
if strings.Contains(line, purgeToken40) || strings.Contains(line, "Bearer") || strings.Contains(line, "Authorization") {
|
||||
t.Fatalf("log line carries the token or the Authorization header: %s", line)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSiteCachePurgeRequest(t *testing.T) {
|
||||
t.Run("sends the contract request and logs the purged count", func(t *testing.T) {
|
||||
logs := captureLogs(t)
|
||||
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"purged":7}`))
|
||||
})
|
||||
newTestPurger(t, srv).Purge(context.Background(), "category", "cordyceps")
|
||||
got := hits()
|
||||
if len(got) != 1 {
|
||||
t.Fatalf("%d requests, want 1", len(got))
|
||||
}
|
||||
h := got[0]
|
||||
if h.method != http.MethodPost || h.path != "/api/__site__/cache/purge" {
|
||||
t.Fatalf("request %s %s", h.method, h.path)
|
||||
}
|
||||
if h.auth != "Bearer "+purgeToken40 || !strings.HasPrefix(h.contentType, "application/json") {
|
||||
t.Fatalf("Authorization %q, Content-Type %q", h.auth, h.contentType)
|
||||
}
|
||||
if len(h.body) != 2 || h.body["kind"] != "category" || h.body["slug"] != "cordyceps" {
|
||||
t.Fatalf("body = %#v", h.body)
|
||||
}
|
||||
lines := logs.lines()
|
||||
if len(lines) != 1 || !strings.Contains(lines[0], "level=INFO") || !strings.Contains(lines[0], `msg="cooler: site cache purged"`) ||
|
||||
!strings.Contains(lines[0], "kind=category") || !strings.Contains(lines[0], "slug=cordyceps") ||
|
||||
!strings.Contains(lines[0], "status=200") || !strings.Contains(lines[0], "purged=7") {
|
||||
t.Fatalf("log = %q", lines)
|
||||
}
|
||||
assertNoToken(t, logs)
|
||||
})
|
||||
|
||||
t.Run("an answer without purged logs the status only", func(t *testing.T) {
|
||||
logs := captureLogs(t)
|
||||
srv, _ := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusNoContent) })
|
||||
newTestPurger(t, srv).Purge(context.Background(), "post", "hello")
|
||||
lines := logs.lines()
|
||||
if len(lines) != 1 || !strings.Contains(lines[0], "level=INFO") || !strings.Contains(lines[0], "status=204") || strings.Contains(lines[0], "purged=") {
|
||||
t.Fatalf("log = %q", lines)
|
||||
}
|
||||
})
|
||||
|
||||
for _, status := range []int{http.StatusInternalServerError, http.StatusUnauthorized} {
|
||||
t.Run("a "+http.StatusText(status)+" answer logs one warning", func(t *testing.T) {
|
||||
logs := captureLogs(t)
|
||||
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(status) })
|
||||
newTestPurger(t, srv).Purge(context.Background(), "post", "hello")
|
||||
if len(hits()) != 1 {
|
||||
t.Fatalf("%d requests, want 1", len(hits()))
|
||||
}
|
||||
lines := logs.lines()
|
||||
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], `msg="cooler: site cache purge failed"`) ||
|
||||
!strings.Contains(lines[0], "kind=post") || !strings.Contains(lines[0], "slug=hello") {
|
||||
t.Fatalf("log = %q", lines)
|
||||
}
|
||||
if !strings.Contains(lines[0], "status="+strconv.Itoa(status)) {
|
||||
t.Fatalf("log %q lacks status=%d", lines[0], status)
|
||||
}
|
||||
assertNoToken(t, logs)
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("a slow receiver is abandoned after the timeout", func(t *testing.T) {
|
||||
logs := captureLogs(t)
|
||||
release := make(chan struct{})
|
||||
srv, _ := purgeReceiver(t, func(_ http.ResponseWriter, r *http.Request) {
|
||||
select {
|
||||
case <-release:
|
||||
case <-r.Context().Done():
|
||||
}
|
||||
})
|
||||
defer close(release)
|
||||
start := time.Now()
|
||||
newTestPurger(t, srv).Purge(context.Background(), "post", "slow")
|
||||
elapsed := time.Since(start)
|
||||
if elapsed < SiteCachePurgeTimeout-200*time.Millisecond || elapsed > SiteCachePurgeTimeout+1500*time.Millisecond {
|
||||
t.Fatalf("Purge returned after %s, want about %s", elapsed, SiteCachePurgeTimeout)
|
||||
}
|
||||
lines := logs.lines()
|
||||
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], "error=timeout") {
|
||||
t.Fatalf("log = %q", lines)
|
||||
}
|
||||
assertNoToken(t, logs)
|
||||
})
|
||||
|
||||
t.Run("an unreachable receiver logs one warning without the URL's token", func(t *testing.T) {
|
||||
logs := captureLogs(t)
|
||||
srv, _ := purgeReceiver(t, func(http.ResponseWriter, *http.Request) {})
|
||||
p := newTestPurger(t, srv)
|
||||
srv.Close()
|
||||
p.Purge(context.Background(), "category", "gone")
|
||||
lines := logs.lines()
|
||||
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], "kind=category") {
|
||||
t.Fatalf("log = %q", lines)
|
||||
}
|
||||
assertNoToken(t, logs)
|
||||
})
|
||||
|
||||
t.Run("a cancelled parent context still sends", func(t *testing.T) {
|
||||
captureLogs(t)
|
||||
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(`{"purged":1}`)) })
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
newTestPurger(t, srv).Purge(ctx, "post", "after-cancel")
|
||||
if got := hits(); len(got) != 1 || got[0].body["slug"] != "after-cancel" {
|
||||
t.Fatalf("requests = %#v, want one", got)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("a nil purger sends nothing", func(t *testing.T) {
|
||||
logs := captureLogs(t)
|
||||
var p *SiteCachePurger
|
||||
p.Purge(context.Background(), "post", "nil")
|
||||
PurgeSiteCacheAfterCommit(context.Background(), nil, nil, "post", "nil")
|
||||
if lines := logs.lines(); len(lines) != 0 {
|
||||
t.Fatalf("log = %q, want nothing", lines)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("outside a transaction the after-commit purge runs at once", func(t *testing.T) {
|
||||
captureLogs(t)
|
||||
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(`{"purged":2}`)) })
|
||||
PurgeSiteCacheAfterCommit(context.Background(), nil, newTestPurger(t, srv), "category", "now")
|
||||
if got := hits(); len(got) != 1 || got[0].body["kind"] != "category" || got[0].body["slug"] != "now" {
|
||||
t.Fatalf("requests = %#v, want one", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// testConfig returns an empty config that ignores the process environment.
|
||||
func testConfig(t *testing.T) *compass.Config {
|
||||
t.Helper()
|
||||
cfg, err := compass.Open(compass.Options{Dir: t.TempDir(), Environ: []string{}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return cfg
|
||||
}
|
||||
Reference in New Issue
Block a user