diff --git a/README.md b/README.md index 0c6190c..a8c9ea9 100644 --- a/README.md +++ b/README.md @@ -108,11 +108,37 @@ Runtime config, overridable by environment variables at start: |-----|-----|---------|---------| | `runtimeConfig.journal.apiBase` | `NUXT_JOURNAL_API_BASE` | `http://127.0.0.1:8096/_journal/api/v1` | Server-side API base for SSR, the sitemap and RSS. Use the internal address of the SummerCMS application, not its public URL. | | `runtimeConfig.public.journal.apiBase` | `NUXT_PUBLIC_JOURNAL_API_BASE` | `/_journal/api/v1` | Browser API base for client-side navigation; the reverse proxy routes it to the application. | +| `runtimeConfig.journal.cachePurgeToken` | `NUXT_JOURNAL_CACHE_PURGE_TOKEN` | empty | Server-only bearer token of the [site cache purge](#site-cache-purge) route. With fewer than 32 characters the route answers 404. | The module also copies `routes`, `perPage`, `sort`, `relatedCount` and `rss` into `runtimeConfig.public.journal`. Server-side requests forward the visitor's `x-forwarded-for` and append the address of the peer that reached Nuxt. The Journal API rate-limits anonymous reads per client address; with a trusted proxy configured for the Nuxt server's address, each visitor keeps their own bucket instead of all SSR traffic sharing one. The API base always comes from runtime config, never from request input. +## Site cache purge + +An app that caches its pages with Nitro route rules (`swr` or `cache`) would otherwise serve a page rendered before a Journal edit until the cache entry expires. The layer adds a purge route that the SummerCMS Journal plugin calls after an admin save of a post or category commits (its `golem15.journal.site_cache.purge_url` and `purge_token` settings), so the edit shows on the next page view. + +**Request.** `POST /api/__journal__/cache/purge` with `Authorization: Bearer `, `Content-Type: application/json` and the body `{"kind":"post"|"category","slug":""}`. The answer is `200` with `{"purged":}`, the number of cache entries removed, and `cache-control: no-store`. The route logs one line, `[journal] cache purge: removed route cache entries ( )`; the slug loses control characters and is cut to 200 characters first. + +**Guards.** + +- POST only. +- `404` when `runtimeConfig.journal.cachePurgeToken` holds fewer than 32 characters, so an app without a token has no purge route. +- `404` for any request that carries `X-Forwarded-For`, `X-Real-IP` or `Forwarded`: the route is for a caller on the same host, never for traffic through the reverse proxy. +- `401` for a missing or wrong bearer token. The comparison runs in constant time over sha256 digests. +- The layer's `routeRules` set `'/api/__journal__/**': { cache: false }`. Nitro wraps every handler that matches an app rule such as `'/**': { swr }` in its cached handler, which keys by path, does not skip POST and hides the request headers from the handler, so without this rule the route would answer from the cache. + +**What it removes.** Every entry of Nitro's route cache (the `nitro:routes` group of the `cache` storage): pages, their `_payload.json` and feeds. Other cache groups stay. One save can change many pages (the menu lists every category on every page, and the home, category, author and related-post lists show posts), and Nitro keys entries by a hash of the path, so a full purge is the reliable choice; it costs one fresh render per page on its next view. + +**Browser revalidation.** Nitro answers a route-cached page with `s-maxage=, stale-while-revalidate`, which gives browsers no `max-age`, so they guess a freshness lifetime from `Last-Modified`; its 304 answer carries `public, max-age=, s-maxage=`. Either way a browser could keep its copy past a purge. The layer's Nitro plugin `server/plugins/journal-revalidate.ts` rewrites every `cache-control` value that carries `s-maxage` to `max-age=0` for browsers (`privateRevalidate` in `app/utils/siteCache.ts`), so a browser asks on each view and gets a cheap 304 while the page is unchanged. Shared caches keep `s-maxage`. Static assets carry no `s-maxage` and are untouched. + +**Deployment.** + +- Generate the token on the server, at least 32 characters, and set the same value in the Nuxt app's `NUXT_JOURNAL_CACHE_PURGE_TOKEN` and in the backend's purge token setting. Keep it out of committed files. +- The backend calls the route on the Nuxt server's loopback address (for example `http://127.0.0.1:3000/api/__journal__/cache/purge`), not through the public site. +- The reverse proxy must not forward POST to Nuxt (for example nginx `limit_except GET HEAD { deny all; }`), so the route stays off the public path even before its own checks. +- The purge clears the cache storage of the Nuxt process that answers. A server with several Nuxt processes needs a shared storage driver for the `cache` mount, or every process keeps its own entries. + ## Theming Every visual value is a token in `app/assets/css/journal-tokens.css`: @@ -167,9 +193,13 @@ A release is an annotated git tag (`v0.1.0`) on `main`, pushed to git.golem15.co - `pnpm test` runs the unit tests in `tests/*.test.ts` with `node --test` (Node strips the TypeScript types; no test framework dependency). They cover the pure helpers in `app/utils/`: route normalization and path filling, the client against a fake fetcher, pagination, the HTML sanitizer (the Journal plugin's FormatHTML cases, the bypasses from its code review, and the Blog Editor corpus in `tests/fixtures/editor-corpus.json`, a byte copy of the plugin's), the embed expansion and its sanitize-first order, the post page view, sitemap entries, SEO values, and pl/en key and placeholder parity. They need no build, network or database. - `scripts/check-git-install.sh ` installs the layer from the git tag into a scratch app, checks that `pnpm-lock.yaml` pins the tag's commit, runs `nuxt build`, and runs the SSR smoke against the build. Run it from a directory that is not inside a directory named `vue-journal-layer` (Nuxt treats such a path segment as the layer). - `testing/smoke-ssr.mjs --output <.output> --fixtures --expect ` boots a fixture Journal API and the built server and checks each line of the expectations file (`STATUS PATH [contains=TEXT] [notcontains=TEXT] [type=PREFIX]`). It also checks that SSR requests forwarded `x-forwarded-for` and that categories were requested with `include_empty=1`. -- `testing/fixture-api.mjs` (`startFixtureApi`) serves the anonymous Journal routes from JSON fixtures in `testing/fixtures/acme/`; `testing/nuxt-server.mjs` (`startNuxtServer`) starts a built server. Apps can import both from `node_modules/vue-journal-layer/testing/`. +- `testing/purge-check.mjs --output <.output> --fixtures --kind post|category --slug --path ` proves the [site cache purge](#site-cache-purge) on a built server whose route rules cache pages: it caches the page, changes the entity in a temporary copy of the fixtures, requires the cached page to stay unchanged, purges with a random token, and requires the change on the next request. It also checks `max-age=0` on the page and its 304, the purged `_payload.json`, a second purge after re-caching, `401` without or with a wrong token, `404` behind a proxy and `404` on a server without a token. The last line is `purge-check: ok ( checks)`. +- `testing/fixture-api.mjs` (`startFixtureApi`) serves the anonymous Journal routes from JSON fixtures in `testing/fixtures/acme/`, and `reload()` re-reads them; `testing/nuxt-server.mjs` (`startNuxtServer`) starts a built server. Apps can import both from `node_modules/vue-journal-layer/testing/`. ```bash pnpm build:playground node testing/smoke-ssr.mjs --output .playground/.output --fixtures testing/fixtures/acme --expect testing/expect-acme.txt +node testing/purge-check.mjs --output .playground/.output --fixtures testing/fixtures/acme --kind post --slug hello-acme --path /blog/hello-acme ``` + +The playground caches every page with `routeRules: { '/**': { swr: 300 } }`, like a real host. diff --git a/app/utils/siteCache.ts b/app/utils/siteCache.ts index 84a1c97..e1b1d93 100644 --- a/app/utils/siteCache.ts +++ b/app/utils/siteCache.ts @@ -83,3 +83,18 @@ export function parsePurgeEvent(body: unknown): PurgeEvent { const slug = typeof raw.slug === 'string' ? raw.slug.replace(/[\u0000-\u001f\u007f-\u009f]/g, '').slice(0, MAX_SLUG_LENGTH) : '' return { kind, slug } } + +/** + * Makes browsers revalidate a shared-cache response: for a cache-control + * value that carries s-maxage it sets max-age=0, replacing any max-age= + * or prepending it, and keeps every other directive (shared caches keep + * s-maxage). Other values, such as a static asset's + * "public, max-age=31536000, immutable", are returned unchanged. + */ +export function privateRevalidate(cacheControl: string): string { + if (typeof cacheControl !== 'string' || !/(^|[\s,])s-maxage\s*=/i.test(cacheControl)) return cacheControl + if (/(^|[\s,])max-age\s*=\s*\d*/i.test(cacheControl)) { + return cacheControl.replace(/(^|[\s,])max-age\s*=\s*\d*/i, '$1max-age=0') + } + return `max-age=0, ${cacheControl}` +} diff --git a/server/plugins/journal-revalidate.ts b/server/plugins/journal-revalidate.ts new file mode 100644 index 0000000..6874807 --- /dev/null +++ b/server/plugins/journal-revalidate.ts @@ -0,0 +1,29 @@ +// Browser revalidation of route-cached pages. Nitro's route cache answers +// 200 with `cache-control: s-maxage=, stale-while-revalidate`, which +// gives browsers no max-age, so they apply heuristic freshness from +// Last-Modified; a conditional request it answers 304 with +// `public, max-age=, s-maxage=`, so after one revalidation a browser +// could reuse its copy for the whole TTL, past any site cache purge. +// +// This plugin passes every cache-control value a response sets through +// privateRevalidate: values with s-maxage get max-age=0, so browsers ask on +// each view (a cheap 304 while the page is unchanged) and shared caches keep +// s-maxage. The 304 path ends the response before any beforeResponse hook +// runs, so the rewrite wraps the request's res.setHeader instead. Static +// assets carry no s-maxage and are untouched. + +import { privateRevalidate } from '../../app/utils/siteCache.ts' + +export default defineNitroPlugin((nitroApp) => { + nitroApp.hooks.hook('request', (event) => { + const res = event.node.res + const setHeader = res.setHeader + res.setHeader = function (name, value) { + if (typeof name === 'string' && name.toLowerCase() === 'cache-control') { + if (typeof value === 'string') value = privateRevalidate(value) + else if (Array.isArray(value)) value = value.map((v) => privateRevalidate(String(v))) + } + return setHeader.call(this, name, value) + } as typeof res.setHeader + }) +}) diff --git a/testing/purge-check.mjs b/testing/purge-check.mjs index 9368e09..81655dc 100644 --- a/testing/purge-check.mjs +++ b/testing/purge-check.mjs @@ -6,6 +6,10 @@ // // node purge-check.mjs --output <.output dir> --fixtures --kind post|category --slug --path // +// It also checks that route-cached pages and their 304 revalidations tell +// browsers max-age=0, that a page's _payload.json (when the build serves +// one) is purged too, that a second purge after re-caching runs (the route +// is never cached itself) and that a server without a token answers 404. // The fixtures are copied to a temporary directory, so the source fixtures // never change. The last line is "purge-check: ok ( checks)" on success; // every failure prints "purge-check: FAIL ..." and exits 1. @@ -64,12 +68,50 @@ function appendMarker(dir, kind, slug, marker) { writeFileSync(file, JSON.stringify(rows, null, 2)) } +/** @param {string} base @param {string} path @param {Record} [headers] */ +async function get(base, path, headers = {}) { + const res = await fetch(base + path, { redirect: 'manual', headers, signal: AbortSignal.timeout(20000) }) + return { status: res.status, headers: res.headers, body: await res.text() } +} + +/** @param {string} base @param {{ kind: string, slug: string }} event @param {Record} headers */ +async function purge(base, event, headers) { + const res = await fetch(base + PURGE_PATH, { + method: 'POST', + headers: { 'content-type': 'application/json', ...headers }, + body: JSON.stringify(event), + signal: AbortSignal.timeout(20000), + }) + const text = await res.text() + /** @type {any} */ + let json = null + try { + json = JSON.parse(text) + } catch { + json = null + } + return { status: res.status, json, text } +} + +/** The payload path Nuxt serves for a page path. @param {string} path */ +function payloadPath(path) { + return `${path.replace(/\/+$/, '')}/_payload.json` +} + +/** @param {string | null} value */ +function revalidates(value) { + return /(^|[\s,])max-age=0(\s*,|\s*$)/.test(value ?? '') && /(^|[\s,])s-maxage=/.test(value ?? '') +} + async function main() { const args = parseArgs(process.argv.slice(2)) + const event = { kind: args.kind, slug: args.slug } const dir = mkdtempSync(join(tmpdir(), 'purge-check-')) cpSync(resolve(args.fixtures), dir, { recursive: true }) const token = randomBytes(24).toString('hex') + const auth = { authorization: `Bearer ${token}` } const marker = `purge-check-marker-${randomBytes(6).toString('hex')}` + const second = `purge-check-second-${randomBytes(6).toString('hex')}` /** @type {string[]} */ const failures = [] @@ -84,72 +126,83 @@ async function main() { const api = await startFixtureApi({ dir }) /** @type {{ url: string, close: () => Promise } | null} */ let nuxt = null - try { + // The steps stop at the first failure that makes the later ones + // meaningless; the failures are reported after the servers stop. + const steps = async () => { nuxt = await startNuxtServer({ outputDir: args.output, apiBase: `${api.url}/_journal/api/v1`, env: { NUXT_JOURNAL_CACHE_PURGE_TOKEN: token }, }) const base = nuxt.url - /** @param {string} path @param {Record} [headers] */ - const get = async (path, headers = {}) => { - const res = await fetch(base + path, { redirect: 'manual', headers, signal: AbortSignal.timeout(20000) }) - return { status: res.status, headers: res.headers, body: await res.text() } - } - /** @param {Record} headers */ - const purge = async (headers) => { - const res = await fetch(base + PURGE_PATH, { - method: 'POST', - headers: { 'content-type': 'application/json', ...headers }, - body: JSON.stringify({ kind: args.kind, slug: args.slug }), - signal: AbortSignal.timeout(20000), - }) - const text = await res.text() - /** @type {any} */ - let json = null - try { - json = JSON.parse(text) - } catch { - json = null - } - return { status: res.status, json, text } - } - const auth = { authorization: `Bearer ${token}` } + const page = args.path + const payload = payloadPath(page) - // The steps stop at the first failure that makes the later ones - // meaningless; the failures are reported after the servers stop. - const steps = async () => { - // 1. The page renders and is cached. - const first = await get(args.path) - if (!check(first.status === 200, `GET ${args.path} before the change: status ${first.status}, want 200`)) return + // 1. The page (and its payload, when the build has one) render and are cached. + const first = await get(base, page) + if (!check(first.status === 200, `GET ${page} before the change: status ${first.status}, want 200`)) return + const hasPayload = (await get(base, payload)).status === 200 // 2. Change the content behind the API; the cached page must not show it. appendMarker(dir, args.kind, args.slug, marker) api.reload() - const cached = await get(args.path) - check(cached.status === 200, `GET ${args.path} after the change: status ${cached.status}, want 200`) - if (!check(!cached.body.includes(marker), `GET ${args.path} shows the change before any purge: the page is not route-cached (missing swr route rules?)`)) return + const cached = await get(base, page) + check(cached.status === 200, `GET ${page} after the change: status ${cached.status}, want 200`) + if (!check(!cached.body.includes(marker), `GET ${page} shows the change before any purge: the page is not route-cached (missing swr route rules?)`)) return - // 3. An authorized purge removes the cached page. - const purged = await purge(auth) + // 3. An authorized purge removes the cached entries. + const purged = await purge(base, event, auth) check(purged.status === 200, `authorized purge: status ${purged.status}, want 200 (${purged.text.slice(0, 200)})`) check(typeof purged.json?.purged === 'number' && purged.json.purged >= 1, `authorized purge: answer ${purged.text.slice(0, 200)}, want purged >= 1`) - // 4. The next request shows the change. - const fresh = await get(args.path) - check(fresh.status === 200 && fresh.body.includes(marker), `GET ${args.path} after the purge: status ${fresh.status}, change shown: ${fresh.body.includes(marker)}`) - - // 5. Refused purges. - const none = await purge({}) - check(none.status === 401, `purge without a token: status ${none.status}, want 401`) - const wrong = await purge({ authorization: `Bearer ${randomBytes(24).toString('hex')}` }) - check(wrong.status === 401, `purge with a wrong token: status ${wrong.status}, want 401`) - const proxied = await purge({ ...auth, 'x-forwarded-for': '203.0.113.7' }) - check(proxied.status === 404, `proxied purge with the right token: status ${proxied.status}, want 404`) + // 4. The next request shows the change and tells browsers to revalidate. + const fresh = await get(base, page) + check(fresh.status === 200 && fresh.body.includes(marker), `GET ${page} after the purge: status ${fresh.status}, change shown: ${fresh.body.includes(marker)}`) + const cc = fresh.headers.get('cache-control') + check(revalidates(cc), `GET ${page} after the purge: cache-control "${cc}", want max-age=0 and s-maxage`) + const etag = fresh.headers.get('etag') + if (check(!!etag, `GET ${page} after the purge: no ETag`)) { + const revalidated = await get(base, page, { 'if-none-match': /** @type {string} */ (etag) }) + check(revalidated.status === 304, `conditional GET ${page}: status ${revalidated.status}, want 304`) + const cc304 = revalidated.headers.get('cache-control') + check(revalidates(cc304), `conditional GET ${page}: cache-control "${cc304}", want max-age=0 and s-maxage`) } + if (hasPayload) { + const p = await get(base, payload) + check(p.status === 200 && p.body.includes(marker), `GET ${payload} after the purge: status ${p.status}, change shown: ${p.body.includes(marker)}`) + } + + // 5. A second purge after the page was cached again runs too: the purge + // route itself is never route-cached. + appendMarker(dir, args.kind, args.slug, second) + api.reload() + const recached = await get(base, page) + check(!recached.body.includes(second), `GET ${page} after the second change: shown before the second purge (page not re-cached)`) + const again = await purge(base, event, auth) + check(again.status === 200 && typeof again.json?.purged === 'number' && again.json.purged >= 1, `second purge: status ${again.status}, answer ${again.text.slice(0, 200)}, want 200 with purged >= 1`) + const fresh2 = await get(base, page) + check(fresh2.body.includes(second), `GET ${page} after the second purge: the second change is not shown (the purge answer came from a cache)`) + + // 6. Refused purges. + const none = await purge(base, event, {}) + check(none.status === 401, `purge without a token: status ${none.status}, want 401`) + const wrong = await purge(base, event, { authorization: `Bearer ${randomBytes(24).toString('hex')}` }) + check(wrong.status === 401, `purge with a wrong token: status ${wrong.status}, want 401`) + const proxied = await purge(base, event, { ...auth, 'x-forwarded-for': '203.0.113.7' }) + check(proxied.status === 404, `proxied purge with the right token: status ${proxied.status}, want 404`) + const realIP = await purge(base, event, { ...auth, 'x-real-ip': '203.0.113.7' }) + check(realIP.status === 404, `purge with X-Real-IP and the right token: status ${realIP.status}, want 404`) + + // 7. A server without a token has no purge route. + await nuxt.close() + nuxt = await startNuxtServer({ outputDir: args.output, apiBase: `${api.url}/_journal/api/v1`, env: { NUXT_JOURNAL_CACHE_PURGE_TOKEN: '' } }) + const disabled = await purge(nuxt.url, event, auth) + check(disabled.status === 404, `purge on a server without a token: status ${disabled.status}, want 404`) + } + try { await steps() } finally { - if (nuxt) await nuxt.close() + if (nuxt) await /** @type {{ close: () => Promise }} */ (nuxt).close() await api.close() rmSync(dir, { recursive: true, force: true }) }