diff --git a/README.md b/README.md index ff8e887..ff0e186 100644 --- a/README.md +++ b/README.md @@ -208,6 +208,26 @@ The public Journal API is unchanged: `POST` and `PUT /_journal/api/v1/posts` sti Editor images are uploaded by the admin through the field's upload route and stored by `classes.StoreMedia`, the same storage as `POST /_journal/api/v1/media/upload`: the key is `/journal//` with the same numbered suffix when the name is taken and the same public URL. An editor upload must be a JPEG, PNG, GIF or WebP file whose first bytes sniff as the type its extension names (no SVG), at most 10240 KiB. Its name is lowercased and every character outside `a-z`, `0-9`, `.`, `_` and `-` becomes `-`, so the inserted `src` needs no escaping. The public upload route keeps its own naming, request fields, response and messages. +## Configuration + +### Site cache purge + +A host site that caches its pages (for example a Nuxt app with `swr` route rules) would keep serving a page rendered before an admin edit until its cache entry expires. With the two keys below set, an admin save of a Journal post or category asks the host site to drop its page caches, so the edit shows on the next page view. + +| Key | Environment variable | Value | +|-----|----------------------|-------| +| `golem15.journal.site_cache.purge_url` | `SUMMER_GOLEM15__JOURNAL__SITE_CACHE__PURGE_URL` | The host site's purge route, for example `http://127.0.0.1:3000/api/__journal__/cache/purge`. | +| `golem15.journal.site_cache.purge_token` | `SUMMER_GOLEM15__JOURNAL__SITE_CACHE__PURGE_TOKEN` | The bearer token the route expects, at least 32 characters. Keep it in the server's environment file, never in a committed config file. | + +With neither key set, nothing is purged and no request is sent. Boot fails, naming the key, when only one of them is set, when the URL is not http or https with a host, when it carries user info or a fragment, when an `http` URL points at a host other than loopback (`127.0.0.0/8`, `::1`, `localhost`; the token would cross the network in clear, so use `https` for any other host), or when the token is shorter than 32 characters. When the purge is enabled, Boot logs the URL's host and path once. + +The request is a `POST` to the URL with `Authorization: Bearer `, `Content-Type: application/json` and the body `{"kind":"post","slug":""}` or `{"kind":"category","slug":""}`. The receiver answers `200` with `{"purged":}`, the number of cache entries it removed. + +- **Timing.** The purge is registered with `lagoon.AfterCommit` on the admin save transaction: it runs only after the save commits, and never for a save that fails, so a concurrent page view cannot re-cache the old content. It runs before the save response, so the cache is clear when the administrator sees the save confirmed. +- **Never fails a save.** The request has a 2 s timeout and ignores the cancellation of the admin request. A refused, failing, slow or unreachable purge is logged as a warning (`journal: site cache purge failed`) and the save response is unchanged; a successful one logs `journal: site cache purged` with the kind, slug, status and purged count. The token and the Authorization header are never logged. +- **What purges.** Admin creates, updates and deletes of posts and categories, including a save that changes nothing. Tags, the posts import action and the public Journal API writes do not purge. +- **Receiver.** The host's Nuxt app built on vue-journal-layer provides the route (`POST /api/__journal__/cache/purge`, token in `NUXT_JOURNAL_CACHE_PURGE_TOKEN`). Generate one token on the server and put the same value in both environments. + ## API reference | Identifier | Description | @@ -250,6 +270,13 @@ Editor images are uploaded by the admin through the field's upload route and sto | `classes.HTMLRemovals` | Removed element and attribute names (distinct, first seen, at most ten each) with exact `ElementCount` and `AttributeCount`, removed `Comments`, and `Other` for any other change; `Empty` reports no removal. | | `classes.FormatTranslation` | Sets `content_html` in one locale's translated attributes: rendered from a non-blank `content` with `FormatHTML`, otherwise a supplied `content_html` sanitized with `SanitizeHTML`. | | `classes.CheckHTML` | Predicate: an error when the `FormatHTML` allow-list would strip or change the HTML, including any tag that repeats an attribute. No write path refuses with it any more (admin saves strip and report); it is the counterpart of the site layer's gate and is used by the policy tests. | +| `classes.SiteCachePurger` | Sends the host site cache purge request; a nil purger does nothing. | +| `classes.NewSiteCachePurger` | Builds the purger from the purge URL and token and validates them; both empty disables the purge (nil, nil). | +| `classes.SiteCachePurger.Purge` | One purge request for a kind and slug, with a 2 s timeout, logged and never returned as an error. | +| `classes.SiteCachePurger.Endpoint` | The purge URL's host and path, for log lines. | +| `classes.PurgeSiteCacheAfterCommit` | Registers a purge to run after the surrounding save transaction commits. | +| `classes.SiteCachePurgeURLKey`, `classes.SiteCachePurgeTokenKey` | The config keys `golem15.journal.site_cache.purge_url` and `purge_token`. | +| `classes.SiteCachePurgeMinTokenLength`, `classes.SiteCachePurgeTimeout` | The shortest accepted token (32) and the request timeout (2 s). | ## CLI commands @@ -281,6 +308,8 @@ go test ./... Postgres integration smokes start Docker unless `-short`. Missing Docker is a failure, not a skip, unless `-short`. +The site cache purge has unit tests in `classes` (`TestSiteCachePurgeConfig`, `TestSiteCachePurgeRequest`) and Postgres tests through the admin API (`TestCategoriesAdminPurgesSiteCache`, `TestPostsAdminPurgesSiteCache`) that point the plugin at a local receiver and check one authorized request after each commit. + The Blog Editor has its own suite and drift gate: ```sh diff --git a/classes/site_cache.go b/classes/site_cache.go index 22b5b5b..c2ee3a3 100644 --- a/classes/site_cache.go +++ b/classes/site_cache.go @@ -8,8 +8,10 @@ import ( "fmt" "io" "log/slog" + "net" "net/http" "net/url" + "strings" "time" "git.golem15.com/golem15/summercms/modules/lagoon" @@ -35,25 +37,68 @@ const ( // save of a Journal post or category. A nil *SiteCachePurger is disabled // and does nothing. type SiteCachePurger struct { - url string - token string - client *http.Client + url string + endpoint string + token string + client *http.Client } +// SiteCachePurgeMinTokenLength is the shortest purge token accepted, the +// same minimum the host site's purge route applies. +const SiteCachePurgeMinTokenLength = 32 + // NewSiteCachePurger returns the purger for purgeURL and token. With both -// empty the purge is disabled and it returns nil, nil. +// empty the purge is disabled and it returns nil, nil. Otherwise both must +// be set, the URL must be http or https with a host and without user info +// or a fragment, an http URL must point at a loopback host (127.0.0.0/8, +// ::1 or localhost) so the bearer token never crosses the network in +// clear, and the token must hold at least SiteCachePurgeMinTokenLength +// characters. The error names the offending key and never the token. func NewSiteCachePurger(purgeURL, token string) (*SiteCachePurger, error) { + purgeURL = strings.TrimSpace(purgeURL) if purgeURL == "" && token == "" { return nil, nil } - u, err := url.Parse(purgeURL) - if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" { - return nil, fmt.Errorf("%s must be an http or https URL with a host", SiteCachePurgeURLKey) + if purgeURL == "" { + return nil, fmt.Errorf("%s is set but %s is not; set both or neither", SiteCachePurgeTokenKey, SiteCachePurgeURLKey) } if token == "" { - return nil, errors.New(SiteCachePurgeTokenKey + " must be set") + return nil, fmt.Errorf("%s is set but %s is not; set both or neither", SiteCachePurgeURLKey, SiteCachePurgeTokenKey) } - return &SiteCachePurger{url: purgeURL, token: token, client: &http.Client{}}, nil + u, err := url.Parse(purgeURL) + if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" || u.Hostname() == "" { + return nil, fmt.Errorf("%s must be an http or https URL with a host", SiteCachePurgeURLKey) + } + if u.User != nil { + return nil, fmt.Errorf("%s must not carry user info", SiteCachePurgeURLKey) + } + if u.Fragment != "" || strings.Contains(purgeURL, "#") { + return nil, fmt.Errorf("%s must not carry a fragment", SiteCachePurgeURLKey) + } + if u.Scheme == "http" && !loopbackHost(u.Hostname()) { + return nil, fmt.Errorf("%s: an http URL must point at a loopback host (127.0.0.0/8, ::1 or localhost); use https for any other host", SiteCachePurgeURLKey) + } + if len(token) < SiteCachePurgeMinTokenLength { + return nil, fmt.Errorf("%s must hold at least %d characters", SiteCachePurgeTokenKey, SiteCachePurgeMinTokenLength) + } + return &SiteCachePurger{url: u.String(), endpoint: u.Host + u.Path, token: token, client: &http.Client{}}, nil +} + +// Endpoint returns the purge URL's host and path, without the scheme, user +// info or query, for log lines. +func (p *SiteCachePurger) Endpoint() string { + if p == nil { + return "" + } + return p.endpoint +} + +func loopbackHost(host string) bool { + if strings.EqualFold(host, "localhost") { + return true + } + ip := net.ParseIP(host) + return ip != nil && ip.IsLoopback() } type siteCachePurgeBody struct { diff --git a/controllers/admin_registry.go b/controllers/admin_registry.go index 4c641b9..873faad 100644 --- a/controllers/admin_registry.go +++ b/controllers/admin_registry.go @@ -30,12 +30,18 @@ func init() { var _ pact.FormAfterCreate = postsAdminController{} var _ pact.FormAfterUpdate = postsAdminController{} var _ pact.FormBeforeDelete = postsAdminController{} + var _ pact.FormAfterDelete = postsAdminController{} var _ pact.HasAdminActions = postsAdminController{} var _ cabana.FieldRelationProvider = postsAdminController{} var _ pact.AdminController = categoriesAdminController{} var _ pact.AdminPermissioned = categoriesAdminController{} var _ pact.AdminRecordSource = categoriesAdminController{} + var _ pact.FormBeforeCreate = categoriesAdminController{} + var _ pact.FormBeforeUpdate = categoriesAdminController{} + var _ pact.FormAfterCreate = categoriesAdminController{} + var _ pact.FormAfterUpdate = categoriesAdminController{} + var _ pact.FormAfterDelete = categoriesAdminController{} var _ cabana.FieldRelationProvider = categoriesAdminController{} var _ pact.AdminController = tagsAdminController{} diff --git a/controllers/categories.go b/controllers/categories.go index 6996eba..04d1c90 100644 --- a/controllers/categories.go +++ b/controllers/categories.go @@ -5,6 +5,7 @@ import ( "errors" "strings" + "git.golem15.com/golem15/sm-journal-plugin/classes" "git.golem15.com/golem15/sm-journal-plugin/models" "git.golem15.com/golem15/summercms/modules/backpack" "git.golem15.com/golem15/summercms/modules/cabana" @@ -60,14 +61,31 @@ func (categoriesAdminController) FormBeforeUpdate(ctx context.Context, model any return prepareCategoryWrite(cat) } +// FormAfterCreate purges the host site's page caches once the save +// transaction commits (G-16.2-2, D-19). +func (c categoriesAdminController) FormAfterCreate(ctx context.Context, model any) error { + return c.purgeAfterCommit(ctx, model) +} + // FormAfterUpdate purges the host site's page caches once the save // transaction commits (G-16.2-2, D-19). func (c categoriesAdminController) FormAfterUpdate(ctx context.Context, model any) error { + return c.purgeAfterCommit(ctx, model) +} + +// FormAfterDelete purges the host site's page caches once the delete +// commits. +func (c categoriesAdminController) FormAfterDelete(ctx context.Context, model any) error { + return c.purgeAfterCommit(ctx, model) +} + +func (c categoriesAdminController) purgeAfterCommit(ctx context.Context, model any) error { cat, ok := model.(*models.Category) if !ok || cat == nil { return errors.New("categories admin: form model is not a category") } - purgeSiteCache(ctx, c.app, "category", cat.Slug) + purger, tx := siteCachePurger(ctx, c.app) + classes.PurgeSiteCacheAfterCommit(ctx, tx, purger, "category", cat.Slug) return nil } @@ -95,6 +113,8 @@ var ( _ pact.AdminRecordSource = categoriesAdminController{} _ pact.FormBeforeCreate = categoriesAdminController{} _ pact.FormBeforeUpdate = categoriesAdminController{} + _ pact.FormAfterCreate = categoriesAdminController{} _ pact.FormAfterUpdate = categoriesAdminController{} + _ pact.FormAfterDelete = categoriesAdminController{} _ cabana.FieldRelationProvider = categoriesAdminController{} ) diff --git a/controllers/posts.go b/controllers/posts.go index 5e59c50..15d29c2 100644 --- a/controllers/posts.go +++ b/controllers/posts.go @@ -6,6 +6,7 @@ import ( "fmt" "strconv" + "git.golem15.com/golem15/sm-journal-plugin/classes" "git.golem15.com/golem15/sm-journal-plugin/console" "git.golem15.com/golem15/sm-journal-plugin/models" "git.golem15.com/golem15/summercms/modules/backpack" @@ -226,16 +227,41 @@ func preparePostWrite(ctx context.Context, post *models.Post) error { } // FormAfterCreate writes content_html for every non-default locale whose -// content the save wrote (D-02). +// content the save wrote (D-02), then registers the host site cache purge +// for after the commit. func (c postsAdminController) FormAfterCreate(ctx context.Context, model any) error { - return c.writeContentHTMLTranslations(ctx, model) + return c.afterPostWrite(ctx, model) } // FormAfterUpdate writes content_html for every non-default locale whose // content the save wrote (D-02). Untouched locales are not listed and keep -// their content_html. +// their content_html. It then registers the host site cache purge for after +// the commit; an untouched save purges too, which is harmless. func (c postsAdminController) FormAfterUpdate(ctx context.Context, model any) error { - return c.writeContentHTMLTranslations(ctx, model) + return c.afterPostWrite(ctx, model) +} + +// FormAfterDelete registers the host site cache purge for after the delete +// commits. +func (c postsAdminController) FormAfterDelete(ctx context.Context, model any) error { + post, ok := model.(*models.Post) + if !ok || post == nil { + return errors.New("posts admin: form model is not a post") + } + purger, tx := siteCachePurger(ctx, c.app) + classes.PurgeSiteCacheAfterCommit(ctx, tx, purger, "post", post.Slug) + return nil +} + +func (c postsAdminController) afterPostWrite(ctx context.Context, model any) error { + if err := c.writeContentHTMLTranslations(ctx, model); err != nil { + return err + } + if post, ok := model.(*models.Post); ok && post != nil { + purger, tx := siteCachePurger(ctx, c.app) + classes.PurgeSiteCacheAfterCommit(ctx, tx, purger, "post", post.Slug) + } + return nil } // writeContentHTMLTranslations stores each written non-default content diff --git a/controllers/site_cache.go b/controllers/site_cache.go index adc8fba..962458c 100644 --- a/controllers/site_cache.go +++ b/controllers/site_cache.go @@ -6,27 +6,28 @@ import ( "git.golem15.com/golem15/sm-journal-plugin/classes" "git.golem15.com/golem15/summercms/modules/backpack" "git.golem15.com/golem15/summercms/modules/cabana" + "gorm.io/gorm" ) -// purgeSiteCache registers a site cache purge for one saved entity to run -// after the save transaction commits. Without a published -// *classes.SiteCachePurger (no purge configured) or outside a save -// transaction it does nothing. -func purgeSiteCache(ctx context.Context, app func() *backpack.App, kind, slug string) { +// siteCachePurger returns the published *classes.SiteCachePurger and the +// save transaction of ctx, for classes.PurgeSiteCacheAfterCommit. Without a +// purger (no purge configured) or outside a save transaction it returns a +// nil purger, which PurgeSiteCacheAfterCommit ignores. +func siteCachePurger(ctx context.Context, app func() *backpack.App) (*classes.SiteCachePurger, *gorm.DB) { if app == nil { - return + return nil, nil } a := app() if a == nil { - return + return nil, nil } purger, ok := a.Lookup[*classes.SiteCachePurger]() if !ok || purger == nil { - return + return nil, nil } tx, ok := cabana.TxFromContext(ctx) if !ok { - return + return nil, nil } - classes.PurgeSiteCacheAfterCommit(ctx, tx, purger, kind, slug) + return purger, tx } diff --git a/plugin.go b/plugin.go index 2fe0bb0..20d4eb3 100644 --- a/plugin.go +++ b/plugin.go @@ -4,6 +4,7 @@ import ( "embed" "fmt" "io/fs" + "log/slog" "net/http" "strconv" "time" @@ -74,7 +75,11 @@ func (p *Plugin) wireSiteCache(app *backpack.App) error { if purger == nil { return nil } - return app.Publish(purger) + if err := app.Publish(purger); err != nil { + return err + } + slog.Default().Info("journal: site cache purge enabled", "endpoint", purger.Endpoint()) + return nil } func (p *Plugin) LangFS() fs.FS { return langFS } diff --git a/site_cache_admin_test.go b/site_cache_admin_test.go index a8728fb..005cd37 100644 --- a/site_cache_admin_test.go +++ b/site_cache_admin_test.go @@ -165,3 +165,38 @@ func TestCategoriesAdminPurgesSiteCache(t *testing.T) { t.Fatalf("the purge saw content %q, want the committed %q (purge before commit)", req.Seen, edited) } } + +// TestPostsAdminPurgesSiteCache proves an admin update of a post sends one +// authorized purge request with kind post after the save commits. +func TestPostsAdminPurgesSiteCache(t *testing.T) { + recv := newPurgeReceiver(t) + env := newPostsEnv(t, recv.env()...) + token := env.admin(t, controllers.PermissionAccessPosts) + + rec := env.expect(t, http.StatusCreated, http.MethodPost, adminAPI(adminPostsPath), token, map[string]any{ + "title": map[string]any{"en": "Purge post", "pl": "Wpis"}, + "slug": map[string]any{"en": "purge-post", "pl": "wpis"}, + "content": map[string]any{"en": "

Before

"}, + "excerpt": map[string]any{"en": "Excerpt", "pl": "Wyjątek"}, + }) + id := adminCreatedID(t, rec) + recv.take() + + recv.setProbe(func() string { + var post models.Post + if err := env.gdb.First(&post, id).Error; err != nil { + return "error: " + err.Error() + } + return post.ContentHTML + }) + served := servedForm(t, env, token, id) + const edited = "

After the edit

" + body := servedBody(served, "title", "slug", "excerpt") + body["content"] = map[string]any{"en": edited} + env.expect(t, http.StatusOK, http.MethodPut, adminAPI(fmt.Sprintf("%s/%d", adminPostsPath, id)), token, body) + + req := recv.expectOnePurge(t, "update", "post", "purge-post") + if req.Seen != edited { + t.Fatalf("the purge saw content_html %q, want the committed %q (purge before commit)", req.Seen, edited) + } +}