Compare commits

...

67 Commits

Author SHA1 Message Date
Jakub Zych
f3e8a984b0 docs(quick-261015-w7f): docs since v0.1.14 and the Upgrading to 0.2 notes
- new setup/upgrading.md: version rule, the two 0.2 boot refusals with
  their fixes, behaviour changes, and what 0.2 adds since 0.1
- architecture module table lists all 28 modules
- WinterCMS map: cache and reorder are provided; rows for field types,
  sanitizing, singletons, API tokens, CORS, Turnstile, menu item types
- configuration keys for CORS profiles, api_docs, auth.cookie, turnstile
- index, setup intro and frontend page link the new topics
2026-10-15 23:24:56 +02:00
Jakub Zych
76d788821a docs: decide to release the 19.3 gap closure as v0.2.0 2026-10-15 23:14:08 +02:00
Jakub Zych
694a565941 test(19.3): persist human verification items as UAT 2026-10-15 23:11:05 +02:00
Jakub Zych
56fe2ae372 docs(19.3): re-verification after gap closure (human checks pending) 2026-10-15 23:10:59 +02:00
Jakub Zych
c5d9a7b6b1 docs(19.3-06): state and roadmap after the gap closure 2026-10-15 23:01:46 +02:00
Jakub Zych
350d0355b2 docs(19.3-06): gap closure summary, release handed off to the phase session 2026-10-15 23:01:09 +02:00
Jakub Zych
0c8109c670 docs(19.3-06): review disposition and validation map after the gap closure 2026-10-15 22:51:13 +02:00
Jakub Zych
b2f4044111 test(19.3-06): gate requires the review-fix tests
- go, docker and layers stages require the plan-06 tests by name
  (CR-01, WR-01..WR-09)
2026-10-15 22:47:45 +02:00
Jakub Zych
8d80580083 test(19.3-06): unit tests for the review fixes
- TestCORSCredentialedOriginRules, TestCORSAnchoredPatternNoSuffixMatch (D-04)
- TestAPIDocsCSPNoHostSource (D-17)
- TestLoadCookieConfigProductionInferredSecure, TestZeroCookieConfigDefaults (D-03)
- TestSiteCachePurgeWaitAndClose, TestSiteCacheBulkCommitCoalesced (D-10)
2026-10-15 22:45:16 +02:00
Jakub Zych
4d4e37932c docs(19.3-06): pair a remote purge URL with the site's allowProxied (WR-03)
- caching.md: an https purge_url on another host goes through the site's
  proxy; set NUXT_SITE_CACHE_ALLOW_PROXIED=true there, the token is then the
  only gate
- cooler README: the purge_url row points at it
2026-10-15 22:42:36 +02:00
Jakub Zych
3bb6d61843 docs(19.3-06): progress record after Task 4 2026-10-15 22:40:30 +02:00
Jakub Zych
db6a7f5c3f fix(19.3-06): queue site-cache purges after commit and coalesce them (WR-01)
- PurgeSiteCacheAfterCommit queues the purge on a per-purger worker after
  commit: one request in flight, saves committed meanwhile collapse into
  one follow-up, so the admin request never waits for the site
- new cooler.SiteCachePurger.Wait drains the queue; Handle.Close waits for
  it at shutdown (timeout plus one second)
- Purge stays the synchronous send; exported names and signatures unchanged
2026-10-15 22:36:24 +02:00
Jakub Zych
50a584c757 fix(19.3-06): site-cache purge never follows redirects (WR-02)
- the purge client's CheckRedirect returns http.ErrUseLastResponse: a 3xx
  is logged as a failed purge with its status and the bearer token never
  reaches the Location
2026-10-15 22:32:32 +02:00
Jakub Zych
d0a580c60c fix(19.3-06): warn at boot when turnstile routes have no secret (WR-08)
- BuildRouter logs one warning with the route count and the production
  flag when routes name turnstile:open or turnstile:closed and the
  verifier has no secret; the secret is never logged
2026-10-15 22:30:57 +02:00
Jakub Zych
caa5b5d8ff fix(19.3-06): refuse protected uploads into a public_url bucket (WR-07)
- attach.Store returns attach.ErrProtectedNeedsPrivateBucket for an
  is_public false upload while storage.uploads.public_url is set, before
  any blob is written
- OpenBucket's public_url line becomes a warning naming the refusal
2026-10-15 22:30:00 +02:00
Jakub Zych
f705343904 fix(19.3-06): api-docs CSP allows only nonce-carrying scripts (WR-06)
- the two pinned jsDelivr scripts carry the per-request nonce next to
  their SRI hash; script-src holds only the nonce, no host source
2026-10-15 22:28:34 +02:00
Jakub Zych
85d8b6a85a fix(19.3-06): zero CookieConfig writes SameSite=Lax and Secure (WR-09)
- withDefaults gives the zero config Secure and every unset SameSite Lax;
  a config with any field set keeps its Secure value
2026-10-15 22:27:31 +02:00
Jakub Zych
4dae8331c5 fix(19.3-06): production refuses an inferred insecure token cookie (WR-05)
- LoadCookieConfig refuses a Secure that resolves to false in production,
  also when inferred from an http or unset app.url, naming
  auth.cookie.secure and app.url
2026-10-15 22:26:43 +02:00
Jakub Zych
e32940d90f fix(19.3-06): credentialed CORS refuses wildcard-host and unanchored origins (WR-04)
- with supports_credentials, a * origin must be the whole leftmost host
  label, patterns must be anchored with ^ and $, and no origin rule may
  match the probe origins outside any configured domain
- globally and per profile; non-credentialed configs load as before
2026-10-15 22:25:43 +02:00
Jakub Zych
072353663e fix(19.3-06): cors:<profile> headers on every response of the route (CR-01)
- Router.wrap applies the route's profile outside recovery, body limit and
  locale, whatever its position in the list or the group nesting
- bufferedResponse.commit merges Vary instead of replacing it
- TestCORSProfileHeadersOnRefusals, TestCORSProfileVaryKeptWithHandlerVary
  required by name in the gate
2026-10-15 22:24:08 +02:00
Jakub Zych
e9ad640999 docs(19.3): state after gap closure planning 2026-10-15 21:21:00 +02:00
Jakub Zych
94d9344cdd docs(19.3): revise gap closure plan 06 after checker feedback
Task 1 tests every refusal status routing.md names (403 and the
Turnstile 422 and 503 added). Task 4 ends with a committed progress
record and the plan gains a resume rule, so a context reset after
Task 4 continues at Task 5 without a partial SUMMARY.
2026-10-15 21:19:55 +02:00
Jakub Zych
629e75f152 docs(19.3): create gap closure plan 06 (CR-01, WR-01..WR-09) 2026-10-15 21:12:58 +02:00
Jakub Zych
ebdd896d82 docs(19.3): add phase verification report (gaps found) 2026-10-15 20:28:18 +02:00
Jakub Zych
6cd29eae35 docs(19.3): record code review disposition 2026-10-15 20:18:44 +02:00
Jakub Zych
1ec6aa4364 docs(19.3): add code review report 2026-10-15 20:18:33 +02:00
Jakub Zych
f80e2e8636 docs(19.3-05): state and roadmap after the v0.1.27 release 2026-10-15 20:10:23 +02:00
Jakub Zych
03a9d1f7f1 docs(19.3-05): complete unit tests, phase gate and release plan
- Task 7 reply: release; v0.1.27 tagged and pushed
- plugins bumped to v0.1.27 and pushed; layer and app commits pushed
2026-10-15 20:09:46 +02:00
Jakub Zych
9921768bbf docs(19.3-05): summary through Task 6 and the release sheet; state at the Task 7 checkpoint 2026-10-15 20:02:00 +02:00
Jakub Zych
8f5325300c docs(19.3-05): validation map filled, wave 0 complete 2026-10-15 19:56:21 +02:00
Jakub Zych
f57fc12a78 test(19.3-05): gate plugins, layers and docs stages
- plugins: vet and -short in FAQ, menu and Journal, purge helper and
  deprecated-key suites by name; their Docker admin purge suites in docker
- layers: cmp of the shared purge files, 11 sitePurge: names per layer
- docs: TestDocsTree, docs:build --check, no consuming application named in
  framework lines added since the phase base
2026-10-15 19:56:21 +02:00
Jakub Zych
78ba385793 test(19.3-05): cooler caches, rate-limit parity, Install and after-commit purge
- miniredis v2.40.0 as a test-only dependency (approved at the Task 4 check)
- memory and Redis cache suites, Flush limited to <prefix>cache:
- one parity table through surf.MemoryStore, miniredis and real Redis
- Install, FromApp and site_cache validation; Postgres after-commit purge
- gate: cooler names in the go and docker stages
2026-10-15 19:49:34 +02:00
Jakub Zych
b139651674 docs(19.3-05): record the miniredis checkpoint reply 2026-10-15 19:41:38 +02:00
Jakub Zych
df7db6cfc2 docs(19.3-05): log S3 object content-type observation in deferred items 2026-10-15 19:37:44 +02:00
Jakub Zych
38f6d65c09 test(19.3-05): build tags, S3 tag guard, DiskExtension and public_url
- manifest_tags_test.go: build.tags load, trim, dedupe and validation
  errors naming build.tags; SaveManifest keeps tags and plugin order;
  goBuildArgs puts one -tags before -o
- s3_test.go: s3:// refused without summercms_s3 naming the tag, opens
  with it (both build modes, any scheme case)
- diskext_test.go: RESEARCH probe rows and table consistency
- url_test.go: public_url base joining, File.URL and thumbnails,
  malformed values refused naming the key
- store_test.go: Postgres regression table through Store
- gate: attach names in both tag modes, Store regression in docker
2026-10-15 19:37:35 +02:00
Jakub Zych
055f51c94f test(19.3-05): surf CORS, profiles, api-docs and store lookup; cabana reorder hook
- cors_profile_test.go: credentials with * refused (global and profile),
  pattern errors, anchored wildcard origins, Vary: Origin on dynamic
  answers, api/www credentialed case, unchanged * answers, profile
  preflight routes and boot errors, turnstile factory params
- apidocs_test.go: off by default, pinned 5.33.1 page with the RESEARCH
  SRI hashes and nonce CSP, ordered compact JSON, provider count, path
  validation, escaped title
- store_lookup_test.go: MemoryStore fallback, published Store, no dial
- reorder_hook_test.go (Postgres): hook sees the new order in the tx,
  its error rolls back, no hook answers unchanged
- gate: surf names in the go stage, cabana names in the docker stage
2026-10-15 19:30:52 +02:00
Jakub Zych
619916b5d2 test(19.3-05): phase gate with bouncer cookie and Turnstile unit suites
- scripts/check-phase19.3.sh: fail-closed gate (go stage, self-test, --all
  with NOT RUN docker); a required name that fails, skips or is missing
  fails its stage
- cookie_test.go: LoadCookieConfig defaults and errors, TokenCookie
  attributes and refresh-window Max-Age, forget cookie, refresh and logout
  handlers with a real Refresh and MemoryBlacklist
- turnstile_test.go: open/closed verdict matrix, timeout, form fields,
  token sources, body restore, statuses and secret-free logs
2026-10-15 19:24:32 +02:00
Jakub Zych
6035ccc194 docs(19.3-04): record plan 04 state and roadmap progress 2026-10-15 19:17:20 +02:00
Jakub Zych
6f9f2e20a4 docs(19.3-04): complete framework site-cache purge plan
- cooler.SiteCachePurger, pact.ReorderAfterSave, FAQ/menu/Journal adoption, shared Nuxt purge route
- Task 4 reply push-all: plugins, layers and summercms master pushed; layer tags v0.2.0, v0.2.0, v0.4.0
2026-10-15 19:17:00 +02:00
Jakub Zych
af9dad634c feat(19.3-04): pact.ReorderAfterSave hook inside cabana's reorder transaction
- optional ReorderAfterSave(ctx, partition) runs after the new order is
  written; an error rolls the reorder back through lifecycleFailure
- reorderPost runs in lagoon.Transaction so lagoon.AfterCommit work a hook
  registers (a site-cache purge) runs only after the commit
- pact and cabana READMEs, admin hook table, tree reorder docs
2026-10-15 18:31:49 +02:00
Jakub Zych
0bc390f39f feat(19.3-04): cooler site-cache purge service with site_cache.purge_url and purge_token
- cooler.SiteCachePurger moved from the Journal plugin: loopback-only http,
  32-character token, 2 s detached request, no token in logs or errors
- NewSiteCachePurgerFor names caller keys (for plugin alias keys)
- PurgeSiteCacheAfterCommit through lagoon.AfterCommit, one POST per save
- SiteCacheFromApp for hooks outside serve; Install publishes the purger
  unless a plugin already did, and fails boot on an invalid config
- caching docs section, cooler README, configuration row
2026-10-15 18:21:10 +02:00
Jakub Zych
1d8e50a1ee docs(19.3-03): record plan 03 state and roadmap progress 2026-10-15 18:14:44 +02:00
Jakub Zych
17054f9f85 docs(19.3-03): complete S3 build tag, content-derived disk names and public_url plan 2026-10-15 18:14:18 +02:00
Jakub Zych
2e87efae54 feat(19.3-03): storage.uploads.public_url for direct bucket or CDN file URLs
- OpenBucket validates public_url (absolute http/https, host, no user info, query
  or fragment; the error names the key) and records it next to public_path_prefix
- a set public_url logs one start-up line: the bucket behind it must hold only
  public files, since it bypasses the is_public check
- attach.PublicURLBase; PublicURL, File.URL and thumbnail URLs use the base when
  set and keep the public_path_prefix proxy URLs otherwise
- TestPublicURLSmoke; storage Public URL base section, attachments, configuration
  and lagoon README
2026-10-15 18:13:00 +02:00
Jakub Zych
2e22e9f9a6 fix(19.3-03): name stored uploads after their sniffed content through attach.DiskExtension
- DiskExtension: a fixed table of specific sniffed types keeps a matching client
  extension or uses the canonical one; generic sniffs keep the client extension
  unless it is script-capable (then txt or bin); never asks package mime
- Store names the blob with DiskExtension(sniffed, ext); file_name, the allow-list
  and the stored content_type are unchanged
- TestStoreDiskExtensionSmoke: a PNG uploaded as x.html is stored and served as .png,
  report.docx keeps .docx, an HTML body named note.txt keeps .txt
- attachments docs on disk names, storage deploy note on nosniff and allow-lists,
  lagoon README
2026-10-15 18:09:49 +02:00
Jakub Zych
2382040b25 feat(19.3-03): summer.yaml build.tags and the summercms_s3 tag that links the S3 uploads driver
- Manifest.Tags from build.tags (^[a-z0-9_]+$, duplicates dropped), round-tripped by SaveManifest
- summer build passes -tags=<list> through goBuildArgs and prints the build tags line
- attach: s3blob imported only by s3_on.go (//go:build summercms_s3); an untagged binary
  refuses an s3:// bucket_url naming the tag, build.tags and -tags
- go mod tidy adds the AWS SDK v2 modules as indirect requirements
- storage, modules, scaffolding, configuration and scaling docs plus the lagoon README
2026-10-15 18:07:05 +02:00
Jakub Zych
b1fab9930b docs(19.3-02): record plan 02 state and roadmap progress 2026-10-15 18:02:28 +02:00
Jakub Zych
e3ba89a411 docs(19.3-02): complete cookie transport, credentialed CORS, Turnstile and api-docs plan 2026-10-15 18:01:54 +02:00
Jakub Zych
20bc1a0795 feat(19.3-02): opt-in /api-docs Swagger UI page with a pinned, SRI-checked swagger-ui-dist
- surf: APIDocsProvider (one plugin) supplies a YAML or JSON OpenAPI document,
  converted once at boot to compact JSON in source key order; zero or several
  providers or a spec that does not convert fail boot
- http.api_docs.enabled/path/title mount GET /api-docs and /api-docs.json
  (summercms.apidocs, public, refusing to replace a plugin route)
- the page loads swagger-ui-dist 5.33.1 from jsDelivr with sha384 SRI and
  crossorigin on all three assets, a per-request CSP nonce for its inline
  script, nosniff and no-referrer
- docs: API documentation page section, README rows, ExampleAPIDocsProvider
2026-10-15 17:59:54 +02:00
Jakub Zych
597c79bae7 feat(19.3-02): Cloudflare Turnstile verification as turnstile:open|closed middleware
- bouncer: Turnstile.Verify posts secret/response/remoteip to siteverify with
  a 5 s bound and PHP's open and closed modes (503 unconfigured in production,
  422 on a missing, rejected or overlong token, an error or a timeout); errors
  never carry the secret or the token
- bouncer: Turnstile.Middleware reads the token from the query, a JSON body or
  a form body, restores the body and writes refusals through a published
  TurnstileWriter or neutral JSON
- surf: factory turnstile:open|closed from turnstile.secret/turnstile.field
  (a published *bouncer.Turnstile wins), client IP with trusted proxies; any
  other param fails boot
- docs: Turnstile section, middleware table, README rows
2026-10-15 17:54:58 +02:00
Jakub Zych
5055e8bb61 feat(19.3-02): per-group CORS policies through cors:<profile> with router-registered preflights
- surf: LoadCORSProfiles reads http.cors_profiles.<name> (http.cors keys
  without paths, validated the same way); factory cors:<profile> sets the
  profile's headers; an unknown profile fails boot naming plugin and route
- BuildRouter registers OPTIONS <path> (owner surf, recovery only) for every
  path whose routes name cors:<profile>; two profiles on one path, an existing
  OPTIONS route or overlap with http.cors.paths fail boot
- docs: CORS profiles section, middleware table and README rows
2026-10-15 17:51:58 +02:00
Jakub Zych
e7659f2f4b feat(19.3-02): cookie token refresh across subdomains with hardened credentialed CORS
- bouncer: CookieConfig from auth.cookie.* (LoadCookieConfig validates domain,
  same_site none without Secure, secure false in production), TokenCookie with
  Max-Age = refresh window, ForgetTokenCookie, CookieRefreshHandler and
  CookieLogoutHandler with an app-supplied writer
- surf: http.cors refuses credentials with a * origin and bad or PHP-delimited
  patterns at boot, turns wildcard origins into anchored patterns, echoes the
  exact origin and sends Vary: Origin on every dynamic CORS answer; the
  non-credentialed * answer is unchanged
- smoke tests for the www -> api refresh and the refused wildcard config
2026-10-15 17:49:35 +02:00
Jakub Zych
866e4fd3e0 docs(19.3-01): record plan 01 state and roadmap progress 2026-10-15 17:42:42 +02:00
Jakub Zych
d86f27cc9c docs(19.3-01): complete cooler cache and Redis rate-limit store plan 2026-10-15 17:41:58 +02:00
Jakub Zych
fd7f204979 feat(19.3-01): add cooler.Cache with otter memory and Redis drivers
- Cache: Get, Set, Has, Forget, Remember (degrades on cache errors), Increment, Flush
  with per-entry TTLs under <prefix>cache:
- memory driver on otter v2.3.0 bounded by cache.memory.max_entries; Redis driver
  with SET PX, INCRBY and a Flush limited to SCAN MATCH <prefix>cache:* plus UNLINK
- Install publishes the Cache (sharing one Redis client with the rate limiter);
  FromApp serves jobs and commands without a dial
- new docs/services/caching.md, configuration rows, cooler README
2026-10-15 17:40:23 +02:00
Jakub Zych
fa4d89d0b9 feat(19.3-01): share rate limits across instances through cooler's Redis store
- new module cooler: Install, Handle, OpenRedis (no URL in errors), RateLimitStore
  (atomic fixed-window Lua script with MemoryStore semantics, per-instance fallback)
- serve installs cooler and publishes the Redis surf.Store before BuildRouter;
  BuildRouter looks up a published Store, route:list keeps the memory store
- config keys http.rate_limit.store and redis.url; go-redis v9.23.0
- cooler README, root table row, surf README and rate-limiting, configuration
  and scaling docs
2026-10-15 17:35:05 +02:00
Jakub Zych
77dd11c804 docs(19.3): decide cooler dependencies
- go-redis v9.23.0 as the Redis client, otter v2.3.0 as the memory driver
- miniredis test-only after a human check (plan 05), s3blob behind a build tag (plan 03)
2026-10-15 17:27:33 +02:00
Jakub Zych
7357525b0b docs(19.3): create phase plan 2026-10-15 14:45:51 +02:00
Jakub Zych
55dbb18295 docs(19.3): add pattern map 2026-10-15 14:12:24 +02:00
Jakub Zych
25b97338e1 docs(19.3): add research and validation strategy 2026-10-15 13:34:17 +02:00
Jakub Zych
401883ae3b docs(state): record phase 19.3 context session 2026-10-15 11:51:35 +02:00
Jakub Zych
2d352ff582 docs(19.3): capture phase context 2026-10-15 11:51:29 +02:00
Jakub Zych
fcd39918f3 docs(18.1): create gap closure plans 07-09 for review findings 2026-10-14 22:20:49 +02:00
Jakub Zych
38c9c7aa3c docs(18.1): add phase verification report and human UAT items 2026-10-14 21:30:27 +02:00
Jakub Zych
929cfd8669 docs(18.1): add code review report 2026-10-14 21:24:17 +02:00
Jakub Zych
a9dd5656d9 docs(18.1-06): record plan complete in state and roadmap 2026-10-14 21:14:10 +02:00
Jakub Zych
c1a65c9f22 docs(18.1-06): complete phase 18.1 tests, gate and release plan (push-all) 2026-10-14 21:13:47 +02:00
109 changed files with 19086 additions and 147 deletions

View File

@@ -1156,7 +1156,7 @@ Plans:
4. A host test proves checkout, activation, `invoice.paid`, `past_due` with its grace period, cancel and duplicate-event idempotency against a Stripe fake.
5. The new code has unit tests, delivered in the phase's last plan.
**Plans:** 5/6 plans executed
**Plans:** 6/6 plans executed
Plans:
**Wave 1**
@@ -1171,7 +1171,16 @@ Plans:
- [x] 18.1-05-PLAN.md — vue-paymentgateway-layer subscription client, composables and Pg components (D-14); tag `v0.2.0` with go-ahead
**Wave 4** *(blocked on 18.1-01..18.1-05)*
- [ ] 18.1-06-PLAN.md — Unit tests last across all repositories, PHP test map, `scripts/check-phase18.1.sh`, validated 18.1-VALIDATION.md, final push approval
- [x] 18.1-06-PLAN.md — Unit tests last across all repositories, PHP test map, `scripts/check-phase18.1.sh`, validated 18.1-VALIDATION.md, final push approval
**Gap closure wave 1** *(from 18.1-REVIEW.md; the user chose fix over PHP parity on 2026-10-14)*
- [ ] 18.1-07-PLAN.md — Webhook and sync state (pgsub, pgstripe): the job applies the subscription Stripe holds under a per-subscription advisory lock (CR-01, D-21 amended for ordering); a deletion before creation is retried; plan fallback by product and plan_code (WR-04); SyncSubscription writes only gateway-owned columns and refreshes the plan (WR-06)
**Gap closure wave 2** *(blocked on 18.1-07)*
- [ ] 18.1-08-PLAN.md — Checkout, admin sync and layer: no cross-currency price (CR-02), first-write-wins customer id and idempotency keys (WR-02), guarded checkout for any open subscription (WR-03), framework `pact.AdminBulkAction.Detached` and plan sync outside transactions (WR-01), buttons recover after Back (WR-07)
**Gap closure wave 3** *(blocked on 18.1-07, 18.1-08)*
- [ ] 18.1-09-PLAN.md — Unit tests last for the gap closure across four repositories, the extended gate, VALIDATION rows, layer 0.2.1, final push approval (framework v0.1.27, layer v0.2.1, optional first plugin tags)
### Phase 19: Chat and ChatChannels
@@ -1246,10 +1255,22 @@ Plans:
3. An S3 blob driver can be selected through framework config, so the application does not import it. `/api-docs` and `/api-docs.json` can be served. Turnstile verification middleware is available.
4. The touched modules' READMEs and `docs/` pages are updated. The new code has unit tests, delivered in the phase's last plan.
**Plans:** 0 plans
**Plans:** 6/6 plans executed
Plans:
- [ ] TBD (run /msd-plan-phase 19.3 to break down)
**Wave 1**
- [x] 19.3-01-PLAN.md — Decision note, then the `cooler` module: Redis-backed rate-limit store wired by serve, memory (otter) and Redis cache drivers (wave 1)
**Wave 2** *(blocked on Wave 1 completion)*
- [x] 19.3-02-PLAN.md — bouncer cookie transport and Turnstile, surf CORS hardening and `cors:<profile>`, opt-in /api-docs (wave 2)
- [x] 19.3-03-PLAN.md — summer.yaml `build.tags` and the `summercms_s3` S3 driver, disk-name extension fix, `storage.uploads.public_url` (wave 2)
- [x] 19.3-04-PLAN.md — Site-cache purge in cooler, `pact.ReorderAfterSave`, Journal/menu/FAQ adoption, shared Nuxt purge route in three layers, push checkpoint, app pointer bumps (wave 2)
**Wave 3** *(blocked on Wave 2 completion)*
- [x] 19.3-05-PLAN.md — Unit tests for all new code, `scripts/check-phase19.3.sh`, miniredis legitimacy check, release checkpoint (wave 3)
**Wave 4** *(gap closure)*
- [x] 19.3-06-PLAN.md — Gap closure: `cors:<profile>` applied outermost (CR-01), review warnings WR-01..WR-09 (queued purges, no redirects, layer `allowProxied`, credentialed CORS origin rules, production cookie Secure, api-docs CSP, protected uploads vs `public_url`, Turnstile boot warning, zero `CookieConfig`), unit tests, release checkpoint v0.1.28 (wave 4)
### Phase 19.4: OIDC provider on wristband (INSERTED)
@@ -1364,11 +1385,11 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 →
| 16.3. Backend admin personal API tokens with scopes (INSERTED) | 4/4 | Complete | 2026-10-12 |
| 17. Blog companions | 6/6 | Complete | 2026-10-13 |
| 18. Payment Gateway | 8/8 | Complete | 2026-10-14 |
| 18.1. Payment Gateway Subscriptions | 5/6 | In Progress| |
| 18.1. Payment Gateway Subscriptions | 6/6 | In Progress| |
| 19. Chat and ChatChannels | 0/TBD | Not started | - |
| 19.1. Chat satellites | 0/TBD | Not started | - |
| 19.2. Chat polish against WavePath clients | 0/TBD | Not started | - |
| 19.3. Framework transport and infrastructure for WavePath | 0/TBD | Not started | - |
| 19.3. Framework transport and infrastructure for WavePath | 6/6 | In Progress| |
| 19.4. OIDC provider on wristband | 0/TBD | Not started | - |
| 19.5. Social login in sm-user-plugin | 0/TBD | Not started | - |
| 19.6. Two-factor auth and device flow in sm-user-plugin | 0/TBD | Not started | - |

View File

@@ -1,19 +1,19 @@
---
msd_state_version: "1.0"
milestone: v1.0
current_phase: "18.1"
current_phase_name: Payment Gateway Subscriptions (INSERTED)
status: executing
stopped_at: Completed 18.1-05-PLAN.md
last_updated: "2026-10-14T16:42:31.908Z"
last_activity: 2026-10-14
last_activity_desc: Phase 18.1 execution started
state_head: 967c0f870c4f2ba1a725658485cfb6b0922a0431
current_phase: "19.3"
current_phase_name: Framework transport and infrastructure for WavePath (INSERTED)
status: Phase 19.3 plans complete; release and whole-phase docs pending in the phase 19.3 main session
stopped_at: Completed 19.3-06-PLAN.md (gap closure; release handed off to the phase 19.3 main session)
last_updated: "2026-10-15T21:01:36.545Z"
last_activity: 2026-10-15
last_activity_desc: 19.3-06 gap closure complete, release handed off
state_head: 350d0355b27bdf38e106415cf56b14d0e903c6ae
progress:
total_phases: 40
completed_phases: 31
total_plans: 182
completed_plans: 181
total_plans: 191
completed_plans: 188
gsd_state_version: "1.0"
milestone_name: milestone
---
@@ -25,14 +25,14 @@ milestone_name: milestone
See: .planning/PROJECT.md (updated 2026-10-12)
**Core value:** An existing WinterCMS-shaped app can be ported plugin by plugin to a single Go binary without its frontend noticing: the PHP version's API contract is the acceptance test.
**Current focus:** Phase 18.1 — Payment Gateway Subscriptions (INSERTED)
**Current focus:** Phase 19.3 — Framework transport and infrastructure for WavePath (INSERTED)
## Current Position
Phase: 18.1 (Payment Gateway Subscriptions (INSERTED)) — EXECUTING
Phase: 19.3 (Framework transport and infrastructure for WavePath (INSERTED)) — EXECUTING
Plan: 6 of 6
Status: Ready to execute
Last activity: 2026-10-14 — Phase 18.1 execution started
Status: Phase 19.3 plans complete; release and whole-phase docs pending in the phase 19.3 main session
Last activity: 2026-10-15 — 19.3-06 gap closure complete, release handed off
Progress: [████░░░░░░] 44%
@@ -238,6 +238,13 @@ Progress: [████░░░░░░] 44%
| Phase 18.1 P03 | 21 min | 4 tasks | 27 files |
| Phase 18.1 P04 | 22 min | 5 tasks | 36 files |
| Phase 18.1 P05 | 20 min | 3 tasks | 20 files |
| Phase 18.1 P06 | 50min | 5 tasks | 39 files |
| Phase 19.3 P01 | 14 min | 2 tasks | 20 files |
| Phase 19.3 P02 | 15min | 4 tasks | 11 files |
| Phase 19.3 P03 | 10 min | 3 tasks | 20 files |
| Phase 19.3 P04 | 60min | 5 tasks | 14 files |
| Phase 19.3 P05 | 49min | 7 tasks | 38 files |
| Phase 19.3 P06 | 40min | 6 tasks | 24 files |
## Accumulated Context
@@ -800,6 +807,29 @@ Recent decisions affecting current work:
- [Phase 18.1]: Layer release 18.1-05: human reply "hold"; vue-paymentgateway-layer main 3dea9cd unpushed, no v0.2.0 tag; plan 06 publishes
- [Phase 18.1]: Layer subscription redirects follow only absolute http(s) checkout_url/portal_url from the API answer
- [Phase 18.1]: PgPlans shows the chosen currency price or the plan first price; subscribe sends the currency of the price shown
- [Phase 18.1]: Phase 18.1 published on push-all: summercms v0.1.26 (taglist, flat reorder, localized reorder.flat fix), vue-paymentgateway-layer v0.2.0, pgsub and pgstripe require summercms v0.1.26
- [Phase 18.1]: scripts/check-phase18.1.sh is the phase gate; it never installs packages and fails loudly on a pnpm store mismatch; the layer check-git-install.sh stays open
- [Phase 19.3]: 19.3-01: cooler memory cache driver is otter v2 (bounded); stdlib map rejected as unbounded
- [Phase 19.3]: 19.3-01: unreachable Redis at start-up is a warning; missing/malformed redis.url with a Redis driver is a boot error naming the key
- [Phase 19.3]: 19.3-01: cooler key prefix = cache.prefix or slug(app.name)+':' (default summer:); cache keys <prefix>cache:, limiter keys <prefix>rl:
- [Phase 19.3]: 19.3-01: Remember degrades on cache errors; Redis Flush is SCAN MATCH <prefix>cache:* + UNLINK, never FLUSHDB
- [Phase 19.3]: 19.3-02: bouncer cookie transport reads auth.cookie.* through CookieConfigSource (compass satisfies it, bouncer imports no framework module); TokenCookie Max-Age is the guard's refresh window
- [Phase 19.3]: 19.3-02: credentialed CORS with a * origin, uncompilable or PHP-delimited origin patterns fail boot; dynamic CORS answers echo the exact origin with Vary: Origin; the non-credentialed * answer is unchanged
- [Phase 19.3]: 19.3-02: route-group CORS uses cors:<profile> from http.cors_profiles.<name>; the router registers OPTIONS preflight routes (owner surf) and refuses overlap with http.cors.paths
- [Phase 19.3]: 19.3-02: Turnstile open/closed follows PHP; with a secret, a transport error or timeout is 422 in both modes and a missing token fails without a siteverify call; a published *bouncer.Turnstile or bouncer.TurnstileWriter overrides the defaults
- [Phase 19.3]: 19.3-02: /api-docs is opt-in (http.api_docs.enabled), spec from exactly one surf.APIDocsProvider converted at boot, swagger-ui-dist pinned to 5.33.1 with sha384 SRI and a nonce CSP
- [Phase 19.3]: 19.3-03: S3 driver opt-in via summer.yaml build.tags summercms_s3 (attach.S3BuildTag); untagged binaries refuse s3:// at boot naming the tag
- [Phase 19.3]: 19.3-03: attach.DiskExtension derives the stored extension from the sniffed type (fixed table, script-capable client extensions become txt/bin); file_name keeps the client name
- [Phase 19.3]: 19.3-03: storage.uploads.public_url applies to every attach URL; start-up line and docs require a public-files-only bucket
- [Phase 19.3]: Site-cache purge is a cooler service (site_cache.purge_url, site_cache.purge_token); failures never fail a save
- [Phase 19.3]: pact.ReorderAfterSave runs inside cabana's reorder transaction; reorderPost uses lagoon.Transaction so AfterCommit fires
- [Phase 19.3]: Journal golem15.journal.site_cache.* kept as deprecated aliases for one release; framework keys win
- [Phase 19.3]: 19.3-04 Task 4 reply push-all: plugins, layers, summercms master pushed; layer tags vue-faq/menu v0.2.0, vue-journal v0.4.0; no framework tag
- [Phase 19.3]: 19.3-05: phase gate scripts/check-phase19.3.sh requires every new test by name (go test -json / TAP detectors); --all green incl. Docker
- [Phase 19.3]: 19.3-05: miniredis v2.40.0 test-only in modules/cooler (Task 4 reply: approved)
- [Phase 19.3]: Phase 19.3 released as summercms v0.1.27 (Task 7 reply: release); sm-faq-plugin, sm-menu-plugin and sm-journal-plugin require v0.1.27 on origin; app plugin submodule pointers not bumped yet
- [Phase 19.3]: Phase 19.3 gate scripts/check-phase19.3.sh is fail-closed: a skipped, missing or red required test fails the stage; miniredis is test-only
- [Phase 19.3]: 19.3-06: release of v0.1.28 (tag, plugin require bumps, layer tags v0.3.0/v0.3.0/v0.5.0, pushes) handed off to the phase 19.3 main session with the whole-phase docs update; gap closure pushed and tagged nothing
### Pending Todos
@@ -878,6 +908,6 @@ Items acknowledged and carried forward from previous milestone close:
## Session Continuity
Last session: 2026-10-14T16:42:31.184Z
Stopped at: Completed 18.1-05-PLAN.md
Last session: 2026-10-15T21:01:19.601Z
Stopped at: Completed 19.3-06-PLAN.md (gap closure; release handed off to the phase 19.3 main session)
Resume file: None

View File

@@ -0,0 +1,34 @@
# Decision: go-redis v9 and otter v2 for cooler
**Date:** 2026-10-15
**Status:** Accepted
**Context:** Phase 19.3 (`cooler` module, D-06..D-09, D-13), 19.3-RESEARCH.md Standard Stack and Package Legitimacy Audit
## Decision
The new `cooler` module (cache drivers, the Redis-backed rate-limit store, later the site-cache purge) depends on:
- **`github.com/redis/go-redis/v9` at v9.23.0** as the Redis client (locked by D-07).
- The official Redis client for Go, MIT/BSD, maintained by Redis.
- `redis.ParseURL` reads `redis://`, `rediss://` (TLS) and `unix://` URLs, so the one config key `redis.url` covers local, managed and socket setups.
- `redis.NewScript` with `Script.Run` sends EVALSHA and falls back to EVAL on NOSCRIPT, which the atomic fixed-window rate-limit script needs.
- `redis.NewClient` does not dial; the first command does. `cooler.FromApp` relies on this to give jobs and console commands a cache without a connection at construction.
- cooler sets `DisableIdentity` and disables maintenance notifications, so the client sends no `CLIENT SETINFO` or `CLIENT MAINT_NOTIFICATIONS` handshake to servers that do not know them.
- **`github.com/maypok86/otter/v2` at v2.3.0** as the in-process memory cache driver (D-07 discretion: otter or a stdlib map).
- Bounded size (`MaximumSize`), per-entry expiry (`ExpiryCalculator`), lock-scoped `Compute` for an atomic `Increment`, an injectable clock for tests. No runtime dependencies (its go.mod requires only testify, for its own tests).
- Already named for `cooler` in `.planning/research/ARCHITECTURE.md` and `.planning/research/go-ecosystem.md`.
- **A stdlib map was rejected:** a map with a mutex and a sweeper is about a hundred lines, but it is unbounded. Cache keys often derive from request data (`user:<id>`, a client IP), so the map grows without limit; bounding it means writing an LRU by hand, which is exactly what otter already does well.
Both versions were checked on proxy.golang.org during research (existence, publish time, VCS origin `github.com/redis/go-redis` and `github.com/maypok86/otter`).
## Related dependencies
- **`github.com/alicebob/miniredis/v2` v2.40.0, test-only.** An in-process Redis for the `-short` unit tests of cooler (EVAL/EVALSHA, PTTL, INCRBY, SCAN, UNLINK, `FastForward` for TTLs). The package name came from training knowledge, so it is installed only in plan 05, after a blocking human check of the package. Production code never imports it.
- **`gocloud.dev/blob/s3blob`**, from the already required `gocloud.dev` v0.46.0. Plan 03 imports it only from a file under the `summercms_s3` build tag, so binaries built without the tag carry no S3 code. The import adds the AWS SDK v2 modules to `go.mod` and `go.sum` as indirect requirements.
- **Docker-backed Redis tests** use the generic `testcontainers.Run` with the `redis:7-alpine` image from the already required `github.com/testcontainers/testcontainers-go`. No testcontainers Redis sub-module is added.
## Consequences
- `go.mod` requires `github.com/redis/go-redis/v9 v9.23.0` and `github.com/maypok86/otter/v2 v2.3.0`; `modules/cooler` is the only importer of both. `surf` imports `cooler`, never go-redis directly.
- Config keys introduced with them: `redis.url`, `cache.driver` (`memory`|`redis`), `cache.prefix`, `cache.memory.max_entries`, `http.rate_limit.store` (`memory`|`redis`). Apps and deploy docs depend on these once they ship.
- Replacing the Redis client later means re-validating the rate-limit Lua script and the key layout (`<prefix>cache:`, `<prefix>rl:`) against the new client.

View File

@@ -0,0 +1,21 @@
# Release the 19.3 gap closure as v0.2.0, not v0.1.28
**Date:** 2026-10-15
**Decision:** The next framework tag after v0.1.27 is **v0.2.0**.
## Why
The 19.3-06 review fixes include changes that can make an app that boots on v0.1.27 refuse to boot after the upgrade:
- WR-05: in production, a token cookie whose `Secure` is inferred as false (http or empty `app.url`) is refused at boot.
- WR-04: credentialed CORS refuses wildcard-host and unanchored origin patterns at boot.
Smaller behaviour changes ride along: site-cache purges run queued after commit (WR-01), protected uploads into a `public_url` bucket are refused (WR-07), Journal's old purge keys are deprecated aliases.
In a v0.x Go module the minor version is the signal for "check your config before upgrading". Consumers pin exact versions, so the bump costs nothing technically.
## Consequences
- Plugins and apps move with `go get git.golem15.com/golem15/summercms@v0.2.0`.
- The docs and the release notes carry an "Upgrading to 0.2" section naming the two boot refusals and the config that fixes each.
- From here, the minor version is bumped when an upgrade can refuse a previously valid config or changes a documented behaviour; patch versions stay safe to take without reading.

View File

@@ -0,0 +1,403 @@
---
phase: 18.1-payment-gateway-subscriptions
plan: 06
subsystem: testing
tags: [subscriptions, stripe, cabana, taglist, reorder, gate, testcontainers, vitest, node-test, release]
# Dependency graph
requires:
- phase: 18.1-01
provides: cabana taglist field and flat reorder (tag v0.1.26 was held)
- phase: 18.1-02
provides: pgsub core (models, contracts, classes.Service, baseline)
- phase: 18.1-03
provides: golem15.pgstripesubscriptions (gateway, webhook, job, plan sync) and the subscription secrets
- phase: 18.1-04
provides: pgsub admin screens and customer API ("api-research" contract)
- phase: 18.1-05
provides: vue-paymentgateway-layer subscription pieces (tag v0.2.0 was held)
provides:
- "scripts/check-phase18.1.sh: fail-closed gate over four repositories (go, plugins, admin, layers, docs, postgres, --self-test, --all, PHASE181_REQUIRE_POSTGRES)"
- "unit and Postgres coverage of every behaviour plans 01 to 05 added, and the nine-row PHP test map"
- "fix: the localized list schema keeps reorder.flat"
- "validated 18.1-VALIDATION.md (nyquist_compliant)"
- "release: framework v0.1.26, layer v0.2.0, plugin requires on v0.1.26, all four repositories pushed"
affects: [consuming applications' subscription cutover, phase 20 import, /msd-verify-work 18.1]
# Actuals (#2632): chars/4 over the added lines of the four repositories (planning docs excluded)
actuals:
tokens: 60749
tasks: 5
commits: 11
commits: 11
plan_head_before: "summercms.go dbf8aaac74d6f94c5307ff86afda32d852fb3891; sm-paymentgatewaysubscriptions-plugin b72876253a33932109003fe211c228c5c89fcb44; sm-pgstripe-plugin 592b4741dedec2e7376ba7f2a94f8032cd04c3dc; vue-paymentgateway-layer 3dea9cd8cf82d46a9317e4f120c7544384c68ca3"
plan_head_after: "summercms.go 5c2b0e1d8a1929fa8154c262b288937690e69bce; sm-paymentgatewaysubscriptions-plugin 472b75bb420675d6a0e547622e9f28383a7092bc; sm-pgstripe-plugin baf3c55aa0cba87ae80babbb9b7e0711b9f6e86d; vue-paymentgateway-layer 30046f52590d69f3e6a396ac93b37bb425aa0894"
tech-stack:
added: []
patterns:
- "Phase gate requires tests by name per repository and package; the TAP detector also requires a minimum count of tests by name prefix"
- "A test-only export_test.go wrapper exposes an unexported helper to the external test package"
- "Concurrency proofs: two goroutines in overlapping transactions, exactly one claim or one fired event"
- "Open-transaction probe: pg_stat_activity 'idle in transaction' checked from a gateway fake at each call"
- "Golden schema transcribed from the PHP dump, not derived from the migration"
key-files:
created:
- scripts/check-phase18.1.sh
- ../sm-paymentgatewaysubscriptions-plugin/models/subscription_test.go
- ../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan_test.go
- ../sm-paymentgatewaysubscriptions-plugin/models/feature_list_test.go
- ../sm-paymentgatewaysubscriptions-plugin/contracts/registry_test.go
- ../sm-paymentgatewaysubscriptions-plugin/lang_parity_test.go
- ../sm-paymentgatewaysubscriptions-plugin/apply_events_test.go
- ../sm-paymentgatewaysubscriptions-plugin/user_subscribable_test.go
- ../sm-paymentgatewaysubscriptions-plugin/baseline_test.go
- ../sm-paymentgatewaysubscriptions-plugin/service_api_test.go
- ../sm-paymentgatewaysubscriptions-plugin/admin_actions_test.go
- ../sm-paymentgatewaysubscriptions-plugin/customer_api_authz_test.go
- ../sm-pgstripe-plugin/subscriptions/decode_test.go
- ../sm-pgstripe-plugin/subscriptions/job_test.go
- ../sm-pgstripe-plugin/subscriptions/sync_plan_test.go
- ../sm-pgstripe-plugin/subscriptions/gateway_test.go
- ../sm-pgstripe-plugin/subscriptions/php_map_test.go
- ../sm-pgstripe-plugin/subscription_secrets_test.go
modified:
- modules/cabana/list_schema.go
- modules/cabana/export_test.go
- modules/cabana/taglist_field_test.go
- modules/cabana/reorder_flat_test.go
- modules/cabana/catalog_smoke_test.go
- modules/cabana/README.md
- admin/tests/form/TaglistField.test.ts
- admin/tests/views/ReorderView.test.ts
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md
- .planning/phases/18.1-payment-gateway-subscriptions/deferred-items.md
- ../sm-paymentgatewaysubscriptions-plugin/admin_smoke_test.go
- ../sm-paymentgatewaysubscriptions-plugin/README.md
- ../sm-paymentgatewaysubscriptions-plugin/go.mod
- ../sm-pgstripe-plugin/subscriptions/stripefake_test.go
- ../sm-pgstripe-plugin/README.md
- ../sm-pgstripe-plugin/subscriptions/README.md
- ../sm-pgstripe-plugin/go.mod
- ../vue-paymentgateway-layer/tests/subscriptions.test.ts
- ../vue-paymentgateway-layer/tests/subscribe.test.ts
- ../vue-paymentgateway-layer/tests/subscriptionComponents.test.ts
- ../vue-paymentgateway-layer/README.md
key-decisions:
- "Final push (D-22): human reply \"push-all\"; framework master and tag v0.1.26, pgsub and pgstripe main with summercms v0.1.26 required, layer main and tag v0.2.0 pushed; nothing force-pushed"
- "The localized list schema now carries reorder.flat (framework fix 0f09586, part of v0.1.26)"
- "The layer's check-git-install.sh stays open: no gate or release step runs it"
- "The gate never installs packages; a pnpm store mismatch fails the layer stage with the offline recovery command"
patterns-established:
- "Phase gate per phase: required names per repository and package, fail on skip, miss, todo, not ok or no tests, NOT RUN postgres loud"
requirements-completed: [SC-1, SC-2, SC-3, SC-4, SC-5, D-01, D-03, D-04, D-05, D-08, D-09, D-11, D-12, D-13, D-15, D-16, D-17, D-18, D-19, D-21, D-22]
coverage:
- id: D1
description: "Fail-closed phase gate over four repositories with a self-test of every detector"
requirement: SC-5
verification:
- kind: other
ref: "PHASE181_REQUIRE_POSTGRES=1 bash scripts/check-phase18.1.sh --all (all seven stages PASS)"
status: pass
human_judgment: false
- id: D2
description: "Framework taglist and flat reorder matrices, the localized reorder.flat fix, admin specs"
requirement: D-17
verification:
- kind: unit
ref: "modules/cabana/taglist_field_test.go#TestTaglistFieldValidate, #TestTaglistFieldHydrate; reorder_flat_test.go#TestReorderFlatNodes, #TestReorderFlatCompile"
status: pass
- kind: integration
ref: "modules/cabana/catalog_smoke_test.go#TestReorderFlatSmoke (partitioned subtest)"
status: pass
- kind: unit
ref: "admin/tests/form/TaglistField.test.ts; admin/tests/views/ReorderView.test.ts"
status: pass
human_judgment: false
- id: D3
description: "pgsub models, registries, event matrix, concurrent claim, user subscriber, golden baseline, service API, admin actions, customer API authorization, lang parity"
requirement: SC-1
verification:
- kind: unit
ref: "sm-paymentgatewaysubscriptions-plugin: TestSubscriptionChecks, TestPlanPrices, TestFeatureList, TestRegistry, TestLangParity"
status: pass
- kind: integration
ref: "sm-paymentgatewaysubscriptions-plugin: TestApplyEvents, TestClaimEventConcurrent, TestUserSubscribable, TestBaselineSchema, TestServiceAPI, TestAdminSyncActions, TestCustomerAPIAuthz"
status: pass
human_judgment: false
- id: D4
description: "pgstripe decode, job (STRIPE-006/007, duplicate, concurrent), plan sync, gateway, PHP map, subscription secrets"
requirement: SC-2
verification:
- kind: unit
ref: "sm-pgstripe-plugin/subscriptions: TestDecodeSubscription, TestDecodeInvoice, TestJobArgsMinimal"
status: pass
- kind: integration
ref: "sm-pgstripe-plugin/subscriptions: TestSubscriptionJob, TestSyncPlan, TestSubscriptionGateway, TestStripeSubscriptionReliability, TestSubscriptionAuthentication; root TestStripeSubscriptionSecrets"
status: pass
human_judgment: false
- id: D5
description: "Layer subscription client, flows and component specs (36 'subscriptions:' tests)"
requirement: D-14
verification:
- kind: unit
ref: "vue-paymentgateway-layer tests/subscriptions.test.ts, subscribe.test.ts, subscriptionComponents.test.ts"
status: pass
human_judgment: false
- id: D6
description: "Release: four repositories pushed, v0.1.26 and v0.2.0 tags on origin, plugin requires on v0.1.26"
requirement: D-22
verification:
- kind: other
ref: "git rev-parse <branch> == origin/<branch> in all four repositories; git ls-remote --tags origin v0.1.26 / v0.2.0"
status: pass
human_judgment: false
- id: D7
description: "Live Stripe test-mode checkout, the portal cancel_at question (D-21) and the admin and layer screens in a real browser"
verification: []
human_judgment: true
rationale: "Needs a real Stripe test account, a public webhook URL and a browser; listed as manual-only rows in 18.1-VALIDATION.md"
duration: 50min
completed: 2026-10-14
status: complete
---
# Phase 18.1 Plan 06: Tests, gate and release Summary
**Every behaviour of plans 01 to 05 now has a unit or Postgres test across the framework, pgsub, pgstripe and the layer. The nine PHP pgstripe tests map to named Go cases. `scripts/check-phase18.1.sh --all` is green with Postgres required. The tests exposed one framework bug (the localized list schema dropped `reorder.flat`), which is now fixed. On the human's "push-all" the phase was published as framework v0.1.26 and layer v0.2.0, and both plugins now require v0.1.26.**
## Performance
- **Duration:** about 50 min. Tasks 1 to 4 ran from 18:24:41Z to 19:09Z, then the checkpoint, then the release and close-out from 19:10Z to about 19:15Z.
- **Tasks:** 5 of 5.
- **Files:** 39 across four repositories, plus the planning docs.
## Task Commits
| Task | Repo | Commit | Message |
|------|------|--------|---------|
| 1 | summercms.go | `3cc4a5f` | test: fail-closed phase 18.1 gate with detector self-test |
| 2 | summercms.go | `0f09586` | fix: keep reorder.flat in the localized list schema |
| 2 | summercms.go | `323b6d6` | test: taglist and flat reorder matrices, partitioned flat smoke, admin specs |
| 2 | vue-paymentgateway-layer | `30046f5` | test: subscription client, flow and component coverage |
| 3 | sm-paymentgatewaysubscriptions-plugin | `1da437f` | test: pgsub unit and Postgres coverage |
| 3 | summercms.go | `6f71cd1` | test: require the pgsub tests in the gate |
| 4 | sm-pgstripe-plugin | `88f9125` | test: decode, job, plan sync, gateway, PHP map and secret coverage |
| 4 | summercms.go | `7790434` | test: require the pgstripe tests in the gate |
| 4 | summercms.go | `5c2b0e1` | docs: validated VALIDATION map and deferred items |
| 5 | sm-paymentgatewaysubscriptions-plugin | `472b75b` | chore: require summercms v0.1.26 |
| 5 | sm-pgstripe-plugin | `baf3c55` | chore: require summercms v0.1.26 |
**Plan metadata:** the docs commits that add this SUMMARY and the STATE/ROADMAP update, pushed after the tag. The tag is not moved.
## Task 5: release decision
**Reply (verbatim):** "push-all"
This is what was done, in the order of the push sheet. Nothing was force-pushed, and every repository was 0 commits behind its origin before its push.
**Framework**
- master pushed: `a377953..5c2b0e1`.
- Annotated tag `v0.1.26` created on `5c2b0e1` (tag object `03c33dd`) and pushed.
**pgsub**
- Commit `472b75b` raises the require to `git.golem15.com/golem15/summercms v0.1.26`; the replace onto `../summercms.go` is unchanged.
- `go mod tidy` resolved through the replace, so no proxy fetch was needed (default GOPROXY, no GONOSUMDB) and go.sum is unchanged.
- vet and the `-short` tests pass.
- main pushed: `aad4647..472b75b`.
**pgstripe**
- Commit `baf3c55` makes the same bump, with the same tidy, vet and `-short` results.
- main pushed: `0313820..baf3c55`.
**Layer**
- main pushed: `ac2b960..30046f5`.
- Annotated tag `v0.2.0` created on `30046f5` (tag object `f1107fa`) and pushed.
**Check:** after the pushes, each local branch equals its origin branch:
| Repository | Branch | Hash |
|------------|--------|------|
| summercms.go | master | `5c2b0e1` (before this SUMMARY's docs commits, which are pushed afterwards) |
| pgsub | main | `472b75b` |
| pgstripe | main | `baf3c55` |
| layer | main | `30046f5` |
`git ls-remote --tags` shows `v0.1.26` on the framework origin and `v0.2.0` on the layer origin.
## Tests added
**Framework (cabana, admin)**
- Docker-free tests:
- `TestTaglistFieldValidate`: 19 accepted and 17 refused cases, both modes, the locale, multiple fields.
- `TestTaglistFieldHydrate`: 12 stored shapes.
- `TestReorderFlatNodes`: flat parent 422, duplicate and zero id 422, missing, extra or swapped node 409, a 200-node flat list, and the tree contrast.
- `TestReorderFlatCompile` now also checks the localized list schema.
- Postgres test: the subtest `TestReorderFlatSmoke/partitioned`, over the new in-memory `acme.shelf` fixture.
- vitest:
- 9 TaglistField coverage specs: script markup, Backspace, blur, paste, datalist, customTags, comma string mode, array mode, aria.
- 2 flat ReorderView specs: the keyboard and the three drop zones.
**pgsub**
- Docker-free tests: `TestSubscriptionChecks`, `TestPlanPrices`, `TestFeatureList`, `TestRegistry`, `TestLangParity`. `TestLangParity` checks keys, `:placeholders` and every key the sources name; a planted placeholder mismatch made it fail.
- Postgres tests:
- `TestApplyEvents`: 272 cases (status pairs times cancel flags) against an independent copy of the PHP rule, plus deleted, invoices and refused updates.
- `TestClaimEventConcurrent`: two overlapping claims, five rounds, plus a rollback.
- `TestUserSubscribable`.
- `TestBaselineSchema`: golden names from the PHP dump, jsonb columns, RESTRICT, a re-run, rollback then migrate, the users column only when missing.
- `TestServiceAPI`: a probe gateway checks for open transactions and committed customer ids at each call; a sanity subtest proves the probe sees an open transaction.
- `TestAdminSyncActions`.
- `TestCustomerAPIAuthz`: routes × anonymous, malformed and admin tokens, the owner and another user. It covers the byte-exact none answer and ignored redirect and subscriber fields.
**pgstripe**
- Docker-free tests: `TestDecodeSubscription`, `TestDecodeInvoice`, `TestJobArgsMinimal` (a fixed key list, no PII).
- Postgres tests:
- `TestSubscriptionJob`: STRIPE-006 rollback through a listener that fails on demand, STRIPE-007 skips, duplicate and concurrent jobs, both shapes.
- `TestSyncPlan`.
- `TestSubscriptionGateway`.
- `TestStripeSubscriptionReliability` and `TestSubscriptionAuthentication` (the PHP map).
- Root `TestStripeSubscriptionSecrets`.
- `TestSDKPin` remains the STRIPE-003 case.
**Layer:** 9 more `subscriptions:` tests (36 in total):
- refusals passed through unchanged, and the `usePlans` key;
- a retry after a refusal, and the buttons' copy before a click;
- the currency prop;
- the confirmed 404 none answer end to end;
- the Cancel visibility matrix;
- the manage flow with 401, 409, 422, 429 and 502;
- the cancel busy guard.
## PHP test map (pgstripe 8b47457)
| PHP | Go |
|-----|----|
| ReliabilityTest `test_stripe_006_idempotency_marker_after_job_success` :33 | `subscriptions.TestStripeSubscriptionReliability/stripe_006_marker_after_job_success` |
| ReliabilityTest `test_stripe_006_idempotency_marker_double_processing_prevented` :99 | `.../stripe_006_double_processing_prevented` |
| DataHandlingTest `test_stripe_001_api_keys_stored_as_plaintext` :28 | `TestStripeSubscriptionSecrets/stripe_001_not_plaintext` |
| DataHandlingTest `test_stripe_001_keys_encrypted_at_rest` :81 | `.../stripe_001_encrypted_at_rest` |
| DataHandlingTest `test_stripe_001_secret_keys_round_trip_encrypted_to_db` :153 | `.../stripe_001_round_trip` |
| DataHandlingTest `test_pay_10_lazy_decrypt_handles_plaintext` :227 | `.../pay_10_import_plaintext_and_encrypted` |
| DataHandlingTest `test_stripe_003_sdk_constraint_pinned` :278 | `TestSDKPin` (unchanged) |
| AuthenticationTest `test_pay_09_..._accepts_valid_sig` :176 | `subscriptions.TestSubscriptionAuthentication/pay_09_accepts_valid` (both URLs) |
| AuthenticationTest `test_pay_09_..._rejects_invalid_sig` :121 | `.../pay_09_rejects_invalid` (both URLs) |
| AuthenticationTest `test_stripe_002_*` :42, :74 | Phase 18 scope: `TestStripeAuthentication` (payment notification) |
## Gate results
`PHASE181_REQUIRE_POSTGRES=1 bash scripts/check-phase18.1.sh --all` passed every stage, and the Task 5 verify run (clean trees plus `--all`) printed `final-push-sheet-ok`.
| Stage | Result |
|-------|--------|
| self-test | PASS |
| go | 5 required framework tests |
| plugins | pgsub 9 named (models 3, contracts 1, root 5); pgstripe 7 named (root 4, subscriptions 3); vet and `-short` green in both repositories |
| admin | vue-tsc; vitest 1401 tests in 92 files |
| layers | 105 TAP tests, 0 failures; 36 named `subscriptions:` (minimum 36) |
| docs | TestDocsTree, docs:build, the OpenAPI check, the dist check and the phase 10 hygiene check |
| postgres | framework 3; pgsub 14; pgstripe 4 in the root and 7 in subscriptions |
No suite was NOT RUN.
## VALIDATION
`18.1-VALIDATION.md` is `status: validated`, `nyquist_compliant: true` and `wave_0_complete: true`. It covers every task of plans 01 to 06 and keeps three manual-only rows:
- the portal `cancel_at` check;
- a live test-mode checkout;
- the screens in a browser.
## Decisions Made
- Release per the reply "push-all", in the push sheet order. The tag was set on the code-complete HEAD `5c2b0e1`; the later docs commits are pushed without moving it.
- The STRIPE-002 payment-notification tests stay Phase 18 scope.
- The gate copies the Phase 18 detectors instead of sourcing them, and adds a check for the number of tests by name prefix.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 1 - Bug] The localized list schema dropped `reorder.flat`**
- **Found during:** Task 2. This was the deferred item from plan 18.1-04.
- **Issue:** `ListSchema.Localize` copied only the reorder title and partition, so `GET .../schema/list` answered a flat reorder without `flat: true`.
- **Fix:** `Flat` is now copied. The README and the docs already described the flag, so no doc change was needed.
- **Verification:** the RED subtest `TestReorderFlatCompile/the_localized_list_schema_keeps_flat` failed first; its RED evidence was checked as RED_EVIDENCE_OK. It passes after the fix.
- **Committed in:** `0f09586` (fix), then `323b6d6` (test).
### Adaptations
1. **Test-only wrappers in `modules/cabana/export_test.go`** (`CheckTaglistValues`, `HydrateTaglistFields`, `CheckReorderNodes`, `IsReorderStale`). The existing tests are in package `cabana_test`, which cannot call the unexported helpers.
2. **The partitioned flat case uses a fixture of its own.** It runs on the in-memory plugin `acme.shelf`, so the catalog fixture is unchanged.
3. **One shared source scan.** `TestAdminLangParity`'s scan moved into the helper `langUsedKeys`, which `TestLangParity` also uses. The behaviour of the existing test is unchanged.
4. **A price failure switch in the package Stripe fake** (`failPrices`), so a test can fail the price step after the product step.
5. **`subscriptions/decode_test.go` is an internal test**, because the decoders are unexported.
6. **The STRIPE-006 failure is injected through a listener** that fails on demand, registered by the test. The failure then happens after Apply saved the row, which proves that the row and the marker roll back together.
7. **The gate's layer stage never installs.** If pnpm aborts on the store mismatch, the stage fails and prints the offline, frozen-lockfile recovery command.
8. **A success-URL case with a fragment was dropped.** Go and PHP both append `?session_id` after a `#fragment`. That is PHP parity, and it is left untested rather than pinned.
9. **Most coverage tasks had no failing test first.** The tdd="true" coverage tasks tested behaviour that already existed, so only the `reorder.flat` defect produced a failing test before the fix. The plan is `type: execute`, so the plan-level TDD gate does not apply.
**Total deviations:** 1 auto-fixed (Rule 1) and 9 adaptations.
**Impact on plan:** test-side plumbing only, plus one framework bug fix. No scope or dependency was added.
## Deferred items
| Item | Status |
|------|--------|
| `reorder.flat` in the localized list schema | resolved by `0f09586` |
| pnpm store-dir mismatch (watch item) | resolved: the gate ran the layer suite green from a clean shell, and the layer stage never installs |
| `vue-paymentgateway-layer/scripts/check-git-install.sh` targets vue-journal-layer | **still open**: no gate or release step runs it |
Because of the last item, an install check of the published `v0.2.0` tag needs a quick task on the layer first.
## Issues Encountered
- **The tool layer unescaped `<` in written test literals.** The affected expectations were rewritten with explicit bytes.
- **An interactive `cp` alias blocked a file restore after a mutation check.** The file was restored with `command cp -f` and confirmed clean.
## Threat Flags
None. The plan added tests, a one-line framework fix, two go.mod require bumps and the releases.
- **T-18.1-32:** the gate requires every test by name, and `--self-test` proves its detectors.
- **T-18.1-33:** every Stripe call in the tests goes to the fakes, and the secrets in tests are test-only literals.
- **T-18.1-34:** the full gate ran before the human's reply, and the pushes followed the resolvable order without a force push.
- **T-18.1-SC:** no package was installed.
## Known Stubs
None.
## User Setup Required
None.
## Next Phase Readiness
- The phase is published.
- Applications can require `git.golem15.com/golem15/summercms v0.1.26`.
- They can pin `vue-paymentgateway-layer#v0.2.0`.
- They can build pgsub and pgstripe from main.
- One limitation remains: pgstripe requires `sm-paymentgatewaysubscriptions-plugin v0.0.0` with a replace, and pgsub has no tag. Building from git without the replace needs a pgsub tag or a pseudo-version, the same as the existing sm-paymentgateway-plugin pattern.
- Next: `/msd-verify-work 18.1` with the manual-only rows: a live Stripe test-mode checkout, the portal `cancel_at` check and the browser pass.
---
*Phase: 18.1-payment-gateway-subscriptions*
*Completed: 2026-10-14*
## Self-Check: PASSED
- Files: the gate script, all 17 new test files and the validated VALIDATION map exist.
- Commits on their origin branches:
| Repository | Commits |
|------------|---------|
| summercms.go | `3cc4a5f`, `0f09586`, `323b6d6`, `6f71cd1`, `7790434`, `5c2b0e1` |
| pgsub | `1da437f`, `472b75b` |
| pgstripe | `88f9125`, `baf3c55` |
| layer | `30046f5` |
- Tags: `v0.1.26` is on the framework origin and `v0.2.0` on the layer origin.

View File

@@ -0,0 +1,281 @@
---
phase: 18.1-payment-gateway-subscriptions
plan: 07
type: execute
wave: 1
depends_on: []
files_modified:
- ../sm-paymentgatewaysubscriptions-plugin/classes/lock.go
- ../sm-paymentgatewaysubscriptions-plugin/classes/apply.go
- ../sm-paymentgatewaysubscriptions-plugin/classes/service.go
- ../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go
- ../sm-paymentgatewaysubscriptions-plugin/core_smoke_test.go
- ../sm-paymentgatewaysubscriptions-plugin/README.md
- ../sm-pgstripe-plugin/subscriptions/gateway.go
- ../sm-pgstripe-plugin/subscriptions/job.go
- ../sm-pgstripe-plugin/subscriptions/decode.go
- ../sm-pgstripe-plugin/subscriptions/decode_test.go
- ../sm-pgstripe-plugin/subscriptions/host_test.go
- ../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go
- ../sm-pgstripe-plugin/subscriptions/README.md
- .planning/phases/18.1-payment-gateway-subscriptions/COVERAGE.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-CONTEXT.md
autonomous: true
gap_closure: true
requirements: [SC-2, SC-4, D-11, D-21, CR-01, WR-04, WR-06]
estimate:
tokens: 120000
raw_tokens: 120000
tasks: 3
confidence: low
must_haves:
truths:
- "CR-01 (stale snapshot, review sequence 1): for customer.subscription.created and .updated the webhook job applies the subscription state Stripe holds at the time the job runs (Gateway.RetrieveSubscription, a Stripe call made outside any database transaction), not the event's data.object; while the job retrieves and applies, it holds a session-level Postgres advisory lock keyed by classes.SubscriptionLockKey(subscription id), so two jobs about one subscription never interleave. A stored active row therefore cannot be set back to incomplete by an older or concurrently retried event, and TestFreshStateSmoke/stale_updated_after_active proves it on Postgres."
- "CR-01 (deleted before created, review sequence 2): ApplyDeleted on a subscription id with no stored row returns an error wrapping classes.ErrSubscriptionNotStored, so the job rolls its marker back and River retries it instead of committing the marker; a created event retried afterwards stores the state Stripe reports (canceled), so a deleted subscription never comes back active and never fires activated."
- "CR-01 (spurious reactivated, review sequence 3): after Service.Cancel set cancel_at_period_end locally, an older queued customer.subscription.updated carrying cancel_at_period_end false applies Stripe's current value (true), so SubscriptionReactivated does not fire."
- "CR-01 claim-first is kept (D-11, STRIPE-006/007): a duplicate event is answered from the processed-event marker before any Stripe call, an event of another application (metadata app) is skipped with its marker kept and without a Stripe call, a failed retrieval or a failed Apply leaves no marker and returns the error, and when Stripe answers that the subscription does not exist (404 or resource_missing, ErrSubscriptionMissing) the job applies the event's own snapshot with a Warn log, as before."
- "WR-04: Apply resolves the plan by the exact price id (stripe_price_id, then stripe_prices), else by the Stripe product id of the subscribed price (stripe_product_id), else by the plan_code metadata Checkout wrote (any plan with that code, active or not), so a subscription created on a price that a later sync archived, or on a plan whose code was changed, is stored; contracts.Update carries ProductID and PlanCode, SubscriptionArgs carries product_id and plan_code, and job arguments still hold no e-mail, name or address."
- "WR-06: Service.SyncSubscription retrieves outside any transaction, then in one short transaction re-reads the row FOR UPDATE, writes only the gateway-owned columns (stripe_status, cancel_at_period_end, both period bounds, canceled_at, ended_at, trial_ends_at, stripe_customer_id) and, when the gateway implements contracts.SubscriptionRetriever, subscription_plan_id from the retrieved price (same resolution as Apply), under the same per-subscription advisory lock as the job; a plan switch, subscriber or customer id written between the caller's load and the sync is never overwritten, and no event fires (PHP reconcile parity)."
- "D-21 is amended for ordering by this gap closure (user decision 2026-10-14 on verification human item 1): out-of-order events no longer behave as on PHP; the READMEs' 'Out-of-order events' lines and 18.1-CONTEXT.md § Amendments say so. D-21's other defaults (cancel_at_period_end mapping, both payload shapes, latest subscription with has_access) are unchanged."
- "Compatibility: no route, event name, table, column, config key or customer API response shape changes; contracts.Gateway keeps its eight methods (the retriever is an optional interface); pgsub still links no stripe-go (D-06); every commit keeps go vet and go test ./... green in both plugin repositories, existing assertions that encoded the replaced behaviour are adapted in the same commit and named in the SUMMARY, and nothing is pushed or tagged (release is plan 09)."
artifacts:
- path: "../sm-paymentgatewaysubscriptions-plugin/classes/lock.go"
provides: "session-level advisory lock helper and the lock keys"
contains: "pg_advisory_lock"
- path: "../sm-pgstripe-plugin/subscriptions/gateway.go"
provides: "RetrieveSubscription and ErrSubscriptionMissing"
contains: "RetrieveSubscription"
- path: "../sm-pgstripe-plugin/subscriptions/job.go"
provides: "fresh-state application under the subscription lock, deletion retry"
contains: "SubscriptionLockKey"
- path: "../sm-paymentgatewaysubscriptions-plugin/classes/apply.go"
provides: "ErrSubscriptionNotStored and the price, product, plan-code plan resolution"
contains: "ErrSubscriptionNotStored"
- path: "../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go"
provides: "Postgres smoke of the stale-event, deletion and sync paths"
contains: "TestFreshStateSmoke"
key_links:
- from: "../sm-pgstripe-plugin/subscriptions/job.go"
to: "../sm-pgstripe-plugin/subscriptions/gateway.go"
via: "work calls NewGateway(p.app).RetrieveSubscription before the claim-first transaction"
pattern: "RetrieveSubscription"
- from: "../sm-pgstripe-plugin/subscriptions/job.go"
to: "../sm-paymentgatewaysubscriptions-plugin/classes/lock.go"
via: "classes.WithSessionLock(ctx, db, classes.SubscriptionLockKey(id), ...)"
pattern: "WithSessionLock"
- from: "../sm-paymentgatewaysubscriptions-plugin/classes/service.go"
to: "../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go"
via: "SyncSubscription type-asserts contracts.SubscriptionRetriever"
pattern: "SubscriptionRetriever"
prohibitions:
- statement: "No Stripe call is made while a database transaction is open in the job or in Service.SyncSubscription."
status: unresolved
verification: judgment
- statement: "pgsub imports no stripe-go package (D-06)."
status: unresolved
verification: automated
- statement: "Nothing is pushed or tagged by this plan."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not in user-story format; no story is invented, as in plans 01 to 06): the generic core of `Golem15.PaymentGatewaySubscriptions` and pgstripe's subscription gateway and webhook run on SummerCMS, and a consuming application's subscription flow works as it does on PHP.
This plan's slice (gap closure 1 of 3, wave 1): the webhook and sync state findings of `18.1-REVIEW.md`: CR-01 (stale event snapshots, including the deletion retry), WR-04 (subscriptions on archived prices or renamed plans never stored) and WR-06 (SyncSubscription's stale full-row save without a plan refresh). The user decided on 2026-10-14 (verification human item 1) to fix these rather than accept them as PHP parity.
## Decision amendment: D-21 superseded for event ordering
D-21 said "out-of-order events behave as PHP does". The user's 2026-10-14 decision on verification human item 1 supersedes that clause for ordering: the job now applies the state Stripe holds (re-retrieved), serialised per subscription, and a deletion that arrives before its subscription is stored is retried. This is a deliberate departure from PHP behaviour, recorded here, in both READMEs and in `18.1-CONTEXT.md` § Amendments (Task 3). D-21's other defaults stay: PHP's `cancel_at_period_end` mapping, both payload shapes, and `GET /subscription` answering the latest subscription with `has_access` and `in_grace_period`.
## Decisions taken in this plan (Claude's discretion)
- **Re-retrieve, not a watermark.** Stripe's `event.created` has one-second resolution, and Checkout emits `created` and `updated` within the same second, so a watermark cannot order exactly the race the review describes. Re-retrieving needs no schema change, so no migration is added.
- **Retrieval outside the transaction, serialised by a session-level advisory lock.** `pg_advisory_lock` runs on a connection pinned with GORM's `DB.Connection`; no transaction is open during the Stripe call. The claim-first transaction runs on the same connection after the call. The lock is released in a deferred unlock that ignores request cancellation, and closing the connection releases it too.
- **Stripe 404 falls back to the snapshot.** A signed event names a subscription of the same account and mode, so a 404 is near-impossible. If it happens, the PHP behaviour (apply the snapshot) is the floor, so no event is silently dropped. Existing tests whose Stripe fake holds no object for an event keep this path, and the host helper sets the fake's state to each event's object (Task 1) so the host flow runs the fresh path.
- **Plan fallback order: price, product, plan_code.** The product id survives repricing and code changes (`upsertProduct` updates one product per plan), and the `plan_code` metadata covers a plan whose product id was never stored.
- **Optional interface, not a ninth Gateway method.** `contracts.SubscriptionRetriever` keeps the published `contracts.Gateway` interface unchanged for other implementers (D-05 Google Play stays possible).
## Artifacts this phase produces (this plan)
- `../sm-paymentgatewaysubscriptions-plugin/classes/lock.go` (new): `WithSessionLock(ctx context.Context, db *gorm.DB, key string, fn func(conn *gorm.DB) error) error`, `SubscriptionLockKey(gatewaySubscriptionID string) string`, unexported const `lockKeyPrefix`.
- `../sm-paymentgatewaysubscriptions-plugin/classes/apply.go`: `ErrSubscriptionNotStored`; unexported `resolvePlan(db *gorm.DB, u contracts.Update) (*models.SubscriptionPlan, string, error)`, `planForProduct`, `planByCode`; `Apply` and `ApplyDeleted` use them.
- `../sm-paymentgatewaysubscriptions-plugin/classes/service.go`: `SyncSubscription` rewritten (retrieve outside the transaction, lock, gateway-owned columns, plan refresh); unexported `gatewayFields`.
- `../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go`: `Update.ProductID`, `Update.PlanCode`, interface `SubscriptionRetriever` with `RetrieveSubscription(ctx context.Context, subscriptionID string) (Update, error)`.
- `../sm-pgstripe-plugin/subscriptions/gateway.go`: `ErrSubscriptionMissing`, `(*Gateway).RetrieveSubscription`, `SyncSubscription` built on it, the assertion `var _ contracts.SubscriptionRetriever = (*Gateway)(nil)`, `MetadataPlanCode` used for the checkout metadata key.
- `../sm-pgstripe-plugin/subscriptions/decode.go`: const `MetadataPlanCode`; `subscriptionItem.Price.Product`; `SubscriptionArgs.ProductID` (`product_id`) and `SubscriptionArgs.PlanCode` (`plan_code`) filled by `args()`.
- `../sm-pgstripe-plugin/subscriptions/job.go`: `work` re-retrieves created and updated events under the subscription lock, runs deletions under it, maps `ProductID` and `PlanCode` in `update()`; unexported `applySubscription`.
- `../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go` (new): `TestFreshStateSmoke` with subtests `stale_updated_after_active`, `missing_at_stripe_applies_snapshot`, `deleted_before_created`, `repriced_plan_by_product`, `sync_keeps_plan_switch`.
- Adapted tests: `host_test.go` (`subEvent` stores the event's object in the fake), `decode_test.go` (`TestJobArgsMinimal` key list gains `plan_code` and `product_id`), pgsub `core_smoke_test.go` (`event_diff` expects `ErrSubscriptionNotStored` for an unknown deletion).
- Docs: both READMEs (features, API reference, deviations), `COVERAGE.md` (subscriptions.retrieve row), `18.1-CONTEXT.md` (D-21 amendment).
<objective>
Close CR-01, WR-04 and WR-06: the webhook job and the reconcile sync apply the state Stripe holds, serialised per subscription, never a stale snapshot over newer data; a deletion that arrives before its subscription is stored is retried; a billed subscription on an archived price or a renamed plan is stored; and a sync never overwrites columns it does not own. D-21 is superseded for ordering per the user's 2026-10-14 decision.
Purpose: money-relevant correctness of access (a paid customer must not lose access, a canceled one must not regain it) before any production cutover.
Output: commits in sm-paymentgatewaysubscriptions-plugin and sm-pgstripe-plugin (code, adapted tests, READMEs), one planning-docs commit in summercms.go (COVERAGE, CONTEXT amendment). New unit-test coverage is plan 09; this plan carries one Postgres smoke test (`TestFreshStateSmoke`).
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-VERIFICATION.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-CONTEXT.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-03-SUMMARY.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-06-SUMMARY.md
Repositories: ../sm-paymentgatewaysubscriptions-plugin (pgsub) and ../sm-pgstripe-plugin (pgstripe). Both require summercms v0.1.26 with a `replace` onto ../summercms.go; pgstripe replaces pgsub with ../sm-paymentgatewaysubscriptions-plugin, so a pgsub change is visible to pgstripe without a commit. Commit per repository (pgsub first when both change), stage files by explicit path, no co-author tags, never push. Docker-backed suites skip under -short; run them on a Docker host (`sudo -u golem` if the socket is denied although the user is in the docker group).
<interfaces>
- `classes.Service.Apply(ctx, tx, contracts.Update) (*models.Subscription, error)`: classes/apply.go:28-126; the row lookup FOR UPDATE at :52, `planForPrice` at :300-323, `loadPlan` at :326.
- `classes.Service.ApplyDeleted(ctx, tx, gateway, subscriptionID string)`: classes/apply.go:170-205; the unknown-row branch at :183-187 answers nil, nil today.
- `classes.Service.ClaimEvent(ctx, tx, eventID, eventType) (bool, error)` :255 and `EventProcessed(ctx, db, eventID) (bool, error)` :271.
- `classes.Service.SyncSubscription(ctx, sub)`: classes/service.go:335-352 (full-row Save today); `gatewayOrStripe` :249; `lagoon.Transaction` usage pattern in `ensureCustomer` :403.
- `contracts.Gateway` (8 methods) and `contracts.Update`: contracts/gateway.go:24-109.
- pgstripe `subscriptions.Gateway`: `client(ctx)` gateway.go:56, `SyncSubscription` :263-306 (decodes `remote.LastResponse.RawJSON` with `decodeSubscription`), `failed` :328, checkout metadata `plan_code` literal :153.
- pgstripe `work` job.go:119-147, `route` :152-182, `(*SubscriptionArgs).update()` :187-205; `decodeSubscription`, `subscriptionObject.args()` decode.go:103, :145; metadata consts decode.go:21-25.
- Test helpers (package subscriptions_test): `bootSubscriptionsHost` host_test.go:91, `subEvent` :393, `deliver` :316, `work` :307, `activate` :377, `subscription` :330, `recorder` :220, `plan` :204; fake `setSubscription` stripefake_test.go:190, `GET /v1/subscriptions/{id}` answers 404 `No such subscription` when unset (:196-204); `subscriptionObject(subFields)` eventbody_test.go:38.
- GORM v1.31.2 `(*gorm.DB).Connection(fc func(tx *gorm.DB) error) error` pins one pooled connection (finisher_api.go:613). Postgres `hashtextextended(text, bigint)` gives the bigint key for `pg_advisory_lock`/`pg_advisory_unlock`.
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: a stale customer.subscription.updated can no longer overwrite the state Stripe holds (retrieve outside the transaction, per-subscription advisory lock, claim-first apply)</name>
<reversibility rating="reversible">Internal job behaviour behind unchanged routes, events and tables; reverting the commit restores snapshot application.</reversibility>
<files>../sm-paymentgatewaysubscriptions-plugin/classes/lock.go, ../sm-pgstripe-plugin/subscriptions/gateway.go, ../sm-pgstripe-plugin/subscriptions/job.go, ../sm-pgstripe-plugin/subscriptions/host_test.go, ../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go, ../sm-pgstripe-plugin/subscriptions/README.md</files>
<read_first>../sm-pgstripe-plugin/subscriptions/job.go (whole file), ../sm-pgstripe-plugin/subscriptions/gateway.go (client, SyncSubscription, failed), ../sm-pgstripe-plugin/subscriptions/decode.go (decodeSubscription, args), ../sm-pgstripe-plugin/subscriptions/plugin.go (db, service, log accessors), ../sm-paymentgatewaysubscriptions-plugin/classes/apply.go (ClaimEvent, EventProcessed), ../sm-pgstripe-plugin/subscriptions/host_test.go (bootSubscriptionsHost, subEvent, deliver, work, activate), ../sm-pgstripe-plugin/subscriptions/stripefake_test.go (setSubscription, getSubscription), ../sm-pgstripe-plugin/subscriptions/eventbody_test.go (subFields, subscriptionObject), ../sm-pgstripe-plugin/subscriptions/README.md (Features: Job, Sync; Deviations), .planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md (CR-01 in full)</read_first>
<action>Implements CR-01's preferred fix (re-retrieve in the job) and supersedes D-21 for ordering per the user's 2026-10-14 decision; D-11's claim-first transaction is kept.
pgsub (commit first): create classes/lock.go with `WithSessionLock(ctx, db, key, fn)`: it pins one connection with `db.WithContext(ctx).Connection`, runs `SELECT pg_advisory_lock(hashtextextended(?, 0))` with key on it, calls fn with that connection, and in a deferred step runs `SELECT pg_advisory_unlock(hashtextextended(?, 0))` on the same connection through `context.WithoutCancel(ctx)`, logging (ids only) an unlock error at Error. A nil db or empty key is an error. Add `SubscriptionLockKey(gatewaySubscriptionID)` returning `lockKeyPrefix + "subscription:" + id` with `lockKeyPrefix = "golem15.paymentgatewaysubscriptions:"`. Package doc of the file: a session-level lock, never held inside a transaction, so a provider call may run under it.
pgstripe gateway.go: add `ErrSubscriptionMissing` (Stripe answered that the subscription does not exist). Add exported `RetrieveSubscription(ctx, subscriptionID) (contracts.Update, error)`: an empty id is an error; it builds the client with `g.client(ctx)`, calls `sc.V1Subscriptions.Retrieve(ctx, id, nil)`, maps a `*stripe.Error` with HTTP status 404 or code `resource_missing` to an error wrapping ErrSubscriptionMissing (logged through `g.failed`), requires `remote.LastResponse.RawJSON`, decodes it with `decodeSubscription` and returns `obj.args().update()` (the job's own mapping, so the subscriber and app metadata come from Stripe). Keep `SyncSubscription` unchanged in this task.
pgstripe job.go: in `work`, before the existing transaction, handle created and updated events whose Subscription has an id through a new unexported `applySubscription(ctx, svc, db, args)`: (1) `svc.EventProcessed(ctx, db, args.EventID)`; an error is returned, true logs "event already processed" at Info and returns nil without a Stripe call; (2) when the app tag differs from `metadataApp` (same condition as today) fall through to the existing transaction path, which claims and skips with the marker kept (STRIPE-007) and makes no Stripe call; (3) otherwise run `classes.WithSessionLock(ctx, db, classes.SubscriptionLockKey(s.ID), fn)` where fn calls `NewGateway(p.app).RetrieveSubscription(ctx, s.ID)`; on an error wrapping ErrSubscriptionMissing it logs at Warn ("Stripe has no such subscription; the event snapshot is applied", event id and subscription id) and uses `s.update()`; on any other error it returns the error (no marker, River retries); then it runs `lagoon.Transaction(ctx, conn, ...)` that claims the marker with `svc.ClaimEvent` (not claimed: return nil) and calls `svc.Apply(ctx, tx, u)`. The job's Error log on failure stays as it is. Deleted and invoice events keep today's path in this task.
Test adaptation (same pgstripe commit, behaviour-preserving): in host_test.go `subEvent`, store the event's object in the fake with `e.fake.setSubscription(...)` before returning the body, so Stripe holds the state each delivered event describes (real Stripe semantics) and the host flow exercises the fresh path; run the full subscriptions suite and adapt only assertions the fresh path legitimately changes, naming each in the SUMMARY.
Smoke (allowed in a non-final plan): create subscriptions/fresh_state_smoke_test.go, package subscriptions_test, `TestFreshStateSmoke` on `bootSubscriptionsHost`: subtest `stale_updated_after_active` activates a subscription (`e.activate`), sets the fake's object to active, delivers a customer.subscription.updated whose snapshot says incomplete (built with `eventBody` and `subscriptionObject` directly, not `subEvent`, so the fake keeps active), and asserts the row stays active and no subscription event fires; subtest `missing_at_stripe_applies_snapshot` delivers a created event for an id the fake does not hold and asserts the row is stored with the snapshot's status. Later tasks add subtests.
README (pgstripe subscriptions, same commit): Features "Job" describes the fresh retrieval, the lock and the snapshot fallback on a 404; API reference lists `RetrieveSubscription` and `ErrSubscriptionMissing`. pgsub README API reference lists `WithSessionLock` and `SubscriptionLockKey` in the pgsub commit.</action>
<verify>
<automated>go -C ../sm-paymentgatewaysubscriptions-plugin vet ./... &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -short ./... &amp;&amp; go -C ../sm-pgstripe-plugin vet ./... &amp;&amp; go -C ../sm-pgstripe-plugin test -count=1 -short ./...</automated>
<fails_when>non-zero exit: vet reports an issue or a Docker-free test fails in either plugin repository</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; OUT=$(go -C ../sm-pgstripe-plugin test -count=1 -run '^(TestFreshStateSmoke|TestHostSubscriptionFlow|TestSubscriptionJob|TestStripeSubscriptionReliability|TestSubscriptionWebhookSmoke)$' -v ./subscriptions 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -c '^--- PASS: Test' | grep -qx 5</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), or fewer than five top-level PASS lines (a stale snapshot regressed the row, the host flow or a STRIPE-006/007 job case broke, or a test skipped)</fails_when>
</verify>
<acceptance_criteria>
- `grep -n "pg_advisory_lock\|pg_advisory_unlock\|WithoutCancel" ../sm-paymentgatewaysubscriptions-plugin/classes/lock.go` prints at least three lines.
- `grep -n "func (g \*Gateway) RetrieveSubscription" ../sm-pgstripe-plugin/subscriptions/gateway.go` prints one line and `grep -n "ErrSubscriptionMissing" ../sm-pgstripe-plugin/subscriptions/job.go` prints at least one line.
- `grep -n "SubscriptionLockKey" ../sm-pgstripe-plugin/subscriptions/job.go` prints at least one line.
- `grep -n "func TestFreshStateSmoke" ../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go` prints one line.
- `test "$(go -C ../sm-paymentgatewaysubscriptions-plugin list -deps ./... | grep -c stripe-go)" = 0` succeeds (D-06).
- `grep -n "RetrieveSubscription" ../sm-pgstripe-plugin/subscriptions/README.md` prints at least one line.
</acceptance_criteria>
<done>A created or updated event applies the state Stripe holds under the per-subscription lock, end to end from the signed webhook through the River job to the stored row, and the stale-update smoke passes on Postgres.</done>
</task>
<task type="auto">
<name>Task 2: Deletion before creation is retried, and a billed subscription on an archived price or a renamed plan is stored (CR-01 sequence 2, WR-04)</name>
<reversibility rating="costly">`contracts.Update` gains two fields and `ApplyDeleted` answers a new sentinel error for an unknown row: additive to a published plugin API, but any caller that relied on nil, nil for an unknown deletion sees an error.</reversibility>
<files>../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go, ../sm-paymentgatewaysubscriptions-plugin/classes/apply.go, ../sm-paymentgatewaysubscriptions-plugin/core_smoke_test.go, ../sm-paymentgatewaysubscriptions-plugin/README.md, ../sm-pgstripe-plugin/subscriptions/decode.go, ../sm-pgstripe-plugin/subscriptions/decode_test.go, ../sm-pgstripe-plugin/subscriptions/job.go, ../sm-pgstripe-plugin/subscriptions/gateway.go, ../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go, ../sm-pgstripe-plugin/subscriptions/README.md</files>
<read_first>../sm-paymentgatewaysubscriptions-plugin/classes/apply.go (whole file), ../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go (Update), ../sm-paymentgatewaysubscriptions-plugin/core_smoke_test.go (event_diff subtest, the sub_never_seen deletion around :183-195), ../sm-pgstripe-plugin/subscriptions/decode.go (subscriptionItem, args, metadata consts), ../sm-pgstripe-plugin/subscriptions/decode_test.go (TestJobArgsMinimal subKeys list around :224), ../sm-pgstripe-plugin/subscriptions/job.go (route, update), ../sm-pgstripe-plugin/subscriptions/gateway.go (CreateCheckoutSession metadata), ../sm-pgstripe-plugin/subscriptions/sync_plan.go (upsertProduct: one product per plan; the lookup key and transfer_lookup_key), .planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md (CR-01 sequence 2 and its ApplyDeleted fix; WR-04 in full)</read_first>
<action>Implements CR-01's deletion fix and WR-04; D-08 (an unknown subscriber alias fails loudly) and D-11 are unchanged.
pgsub (commit first): contracts/gateway.go `Update` gains `ProductID string` (the gateway's product of the subscribed price; used to find the plan when the price id is unknown) and `PlanCode string` (the plan code the checkout recorded; the last fallback). classes/apply.go: add `ErrSubscriptionNotStored` ("the gateway subscription has no stored row yet"); `ApplyDeleted` on an unknown row keeps its Warn log and returns an error wrapping ErrSubscriptionNotStored that names the gateway subscription id (no row change, no event), and its doc says the caller retries. Replace the plan lookup in `Apply` with unexported `resolvePlan(db, u)` returning the plan and how it matched: `planForPrice` (unchanged, exact), else `planForProduct(db, u.ProductID)` (`stripe_product_id = ?`, lowest id), else `planByCode(db, u.PlanCode)` (`code = ?`, active or not); an empty id or code skips its step. A match by product or code logs at Warn with the subscription id, the unknown price id and the matched plan code; no match keeps today's behaviour (the stored plan when the row has one, else the "no plan holds price id" error naming the price, product and code). Update the doc comments of Apply and ApplyDeleted. Adapt core_smoke_test.go `event_diff`: the `sub_never_seen` deletion now expects `errors.Is(err, classes.ErrSubscriptionNotStored)`, a nil row and no event (the transaction helper must not fail the test on that expected error; use the existing `env.txErr` form). README (pgsub): API reference lists `ErrSubscriptionNotStored`, `Update.ProductID`, `Update.PlanCode`; Usage or Features states the plan resolution order.
pgstripe (second commit): decode.go adds `MetadataPlanCode = "plan_code"` to the metadata consts, `Product pgstripe.ExpandableID` (json `product`) on `subscriptionItem.Price`, and `SubscriptionArgs.ProductID` (json `product_id,omitempty`) and `SubscriptionArgs.PlanCode` (json `plan_code,omitempty`) filled by `args()` from the first item's price product and `Metadata[MetadataPlanCode]`; the doc of WebhookJobArgs lists the two new keys (ids and a code, no personal data). job.go `update()` maps both onto `contracts.Update`. gateway.go writes the checkout metadata key through `MetadataPlanCode` (same value). Move deleted events under the subscription lock: in `work`, a deleted event with an id runs `classes.WithSessionLock(ctx, db, classes.SubscriptionLockKey(s.ID), ...)` around the existing claim-first transaction on the pinned connection, so it serialises with created and updated jobs; an ErrSubscriptionNotStored from ApplyDeleted rolls the marker back and is returned (River retries; the job's Error log names it). Adapt decode_test.go `TestJobArgsMinimal`: the subscription key list gains `plan_code` and `product_id` (still no e-mail, name or address key). Extend `TestFreshStateSmoke` with `deleted_before_created` (a deleted event for a subscription with no row returns an error from Work and leaves no marker; after a created event whose fake state is canceled is applied, the retried deletion succeeds, the row is canceled, has_access is false and activated never fired) and `repriced_plan_by_product` (a plan whose stripe_prices no longer hold the subscription's price but whose stripe_product_id equals the price's product stores the subscription on that plan). README (subscriptions): job arguments list, the deletion retry, and the plan fallback.</action>
<verify>
<automated>go -C ../sm-paymentgatewaysubscriptions-plugin vet ./... &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -short ./... &amp;&amp; go -C ../sm-pgstripe-plugin vet ./... &amp;&amp; go -C ../sm-pgstripe-plugin test -count=1 -short ./...</automated>
<fails_when>non-zero exit: vet reports an issue, TestJobArgsMinimal finds an unexpected key, or another Docker-free test fails</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 ./... &amp;&amp; OUT=$(go -C ../sm-pgstripe-plugin test -count=1 -run '^(TestFreshStateSmoke|TestHostSubscriptionFlow|TestSubscriptionJob|TestStripeSubscriptionReliability)$' -v ./subscriptions 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -c '^--- PASS: Test' | grep -qx 4</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), a pgsub Postgres test fails (TestApplyEvents, TestSubscriptionsCoreSmoke and the rest of the root package), or fewer than four pgstripe top-level PASS lines</fails_when>
</verify>
<acceptance_criteria>
- `grep -n "ErrSubscriptionNotStored" ../sm-paymentgatewaysubscriptions-plugin/classes/apply.go` prints at least two lines.
- `grep -n "func resolvePlan\|func planForProduct\|func planByCode" ../sm-paymentgatewaysubscriptions-plugin/classes/apply.go` prints three lines.
- `grep -n "ProductID\|PlanCode" ../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go` prints at least two lines.
- `grep -n "MetadataPlanCode" ../sm-pgstripe-plugin/subscriptions/decode.go ../sm-pgstripe-plugin/subscriptions/gateway.go` prints at least two lines.
- `grep -n '"plan_code"\|"product_id"' ../sm-pgstripe-plugin/subscriptions/decode_test.go` prints at least one line.
- `grep -n "deleted_before_created\|repriced_plan_by_product" ../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go` prints two lines.
</acceptance_criteria>
<done>A deletion that outruns its subscription is retried instead of lost, the retried creation stores Stripe's canceled state, and a subscription on an archived price or a renamed plan is stored on the right plan.</done>
</task>
<task type="auto">
<name>Task 3: SyncSubscription writes only gateway-owned columns of a locked, fresh row and refreshes the plan (WR-06); D-21 amendment recorded in the READMEs, COVERAGE and CONTEXT</name>
<reversibility rating="costly">Adds the optional `contracts.SubscriptionRetriever` interface and changes what `Service.SyncSubscription` writes (no longer the whole row): published plugin API, additive, documented.</reversibility>
<files>../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go, ../sm-paymentgatewaysubscriptions-plugin/classes/service.go, ../sm-paymentgatewaysubscriptions-plugin/README.md, ../sm-pgstripe-plugin/subscriptions/gateway.go, ../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go, ../sm-pgstripe-plugin/subscriptions/README.md, .planning/phases/18.1-payment-gateway-subscriptions/COVERAGE.md, .planning/phases/18.1-payment-gateway-subscriptions/18.1-CONTEXT.md</files>
<read_first>../sm-paymentgatewaysubscriptions-plugin/classes/service.go (SyncSubscription :335-352, gatewayOrStripe, ensureCustomer for the short-transaction pattern), ../sm-paymentgatewaysubscriptions-plugin/classes/apply.go (resolvePlan from Task 2, loadPlan), ../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go, ../sm-paymentgatewaysubscriptions-plugin/service_api_test.go (subtest "sync subscription saves the gateway state without events" around :480, which uses the fixture gateway without a retriever), ../sm-paymentgatewaysubscriptions-plugin/fixture_plugin_test.go (fakeGateway.SyncSubscription :199), ../sm-pgstripe-plugin/subscriptions/gateway.go (SyncSubscription, RetrieveSubscription from Task 1), ../sm-pgstripe-plugin/subscriptions/host_test.go (sync_subscription subtest :874), ../sm-paymentgatewaysubscriptions-plugin/README.md (Differences from PHP, API reference), ../sm-pgstripe-plugin/subscriptions/README.md (Sync, Deviations from PHP), .planning/phases/18.1-payment-gateway-subscriptions/COVERAGE.md, .planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md (WR-06)</read_first>
<action>Implements WR-06 and records the D-21 ordering amendment.
pgsub (commit first): contracts/gateway.go adds the optional interface `SubscriptionRetriever` with `RetrieveSubscription(ctx context.Context, subscriptionID string) (Update, error)` (doc: a Gateway may implement it to report the provider's current state as an Update; the service uses it to sync and to map the plan; it never writes the database). classes/service.go rewrites `SyncSubscription(ctx, sub)` with the same signature: a nil sub or a sub without a gateway subscription id is an error; the gateway resolves as today; everything runs inside `WithSessionLock(ctx, s.db, SubscriptionLockKey(id), ...)` so a sync never interleaves with a webhook job of the same subscription. Inside the lock, outside any transaction: when the gateway implements SubscriptionRetriever, call it and keep the Update; otherwise copy the caller's row, call `gw.SyncSubscription(ctx, &copy)` and derive an Update from the copy's gateway-owned fields (no plan refresh on that path). Then one `lagoon.Transaction` on the pinned connection: re-read the row by primary key FOR UPDATE (none is an error naming the id), apply the gateway-owned fields through unexported `gatewayFields(row, u)` (stripe_status only when it is one of models.Statuses and non-empty; cancel_at_period_end as reported; current_period_start, current_period_end, canceled_at, ended_at, trial_ends_at only when non-nil, never cleared; stripe_customer_id only when non-empty), set subscription_plan_id from `resolvePlan` when the retriever path resolved a plan, save with `Select` of exactly those columns plus updated_at (never a full-row Save), load the plan with `loadPlan`, and copy the saved row into the caller's `*sub`. No event fires (PHP reconcile parity). Update the doc comment and the README: API reference lists `SubscriptionRetriever`; "Differences from PHP" replaces "Out-of-order gateway events behave as on PHP: the last applied state wins" with the new rule (the job applies the provider's current state serialised per subscription; a deletion before its subscription is stored is retried; superseding D-21 for ordering by the 2026-10-14 decision) and adds that a sync writes only the gateway-owned columns and refreshes the plan.
pgstripe (second commit): gateway.go asserts `var _ contracts.SubscriptionRetriever = (*Gateway)(nil)` and reimplements `SyncSubscription` on top of `RetrieveSubscription` (same in-memory fields as today, so the contract method keeps its behaviour). Extend `TestFreshStateSmoke` with `sync_keeps_plan_switch`: load a subscription, then switch its plan and customer id in the database (as Apply would), set the fake's object to a new status on the second plan's price, call `svc.SyncSubscription` with the stale loaded copy, and assert the row has the fake's status, the second plan, the switched customer id, and that the caller's copy now matches the row; no event fires. README (subscriptions): "Sync" describes the retriever path; "Deviations from PHP" replaces the "Out-of-order events" line with the fresh-state rule, the 404 snapshot fallback, the deletion retry and the D-21 amendment.
Planning docs (summercms.go, own commit, staged by path): COVERAGE.md row "subscriptions.retrieve (syncSubscription / reconcile)" keeps INTEGRATE and its reason names the webhook job's fresh-state retrieval (CR-01, plan 18.1-07). 18.1-CONTEXT.md § Amendments gains one bullet: "D-21 amendment (2026-10-14, gap closure 18.1-07, user decision on verification human item 1): out-of-order events no longer behave as PHP; created and updated events apply the subscription Stripe holds, serialised per subscription, and a deletion before its subscription is stored is retried. D-21's other defaults are unchanged." Do not edit plans 01 to 06.</action>
<verify>
<automated>go -C ../sm-paymentgatewaysubscriptions-plugin vet ./... &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -short ./... &amp;&amp; go -C ../sm-pgstripe-plugin vet ./... &amp;&amp; go -C ../sm-pgstripe-plugin test -count=1 -short ./...</automated>
<fails_when>non-zero exit: vet reports an issue or a Docker-free test fails in either plugin repository</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 ./... &amp;&amp; go -C ../sm-pgstripe-plugin test -count=1 ./... &amp;&amp; bash scripts/check-phase18.1.sh plugins</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), any Postgres test of either repository fails (TestServiceAPI's sync subtest, TestHostSubscriptionFlow/sync_subscription, TestFreshStateSmoke/sync_keeps_plan_switch among them), or the phase gate's plugins stage reports a required name failed, skipped or missing</fails_when>
</verify>
<acceptance_criteria>
- `grep -n "type SubscriptionRetriever interface" ../sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go` prints one line.
- `grep -n "SubscriptionRetriever\|gatewayFields\|WithSessionLock" ../sm-paymentgatewaysubscriptions-plugin/classes/service.go` prints at least three lines.
- `sed -n '/func (s \*Service) SyncSubscription/,/^}/p' ../sm-paymentgatewaysubscriptions-plugin/classes/service.go | grep -c "Select("` prints at least 1.
- `grep -n "contracts.SubscriptionRetriever" ../sm-pgstripe-plugin/subscriptions/gateway.go` prints one line.
- `grep -n "sync_keeps_plan_switch" ../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go` prints one line.
- `grep -c "D-21 amendment" .planning/phases/18.1-payment-gateway-subscriptions/18.1-CONTEXT.md` prints at least 1.
- `grep -n "18.1-07" .planning/phases/18.1-payment-gateway-subscriptions/COVERAGE.md` prints at least one line.
- `git -C ../sm-paymentgatewaysubscriptions-plugin status --porcelain -- classes contracts core_smoke_test.go README.md; git -C ../sm-pgstripe-plugin status --porcelain -- subscriptions` prints nothing (every file of this plan is committed).
</acceptance_criteria>
<done>A reconcile sync never overwrites a newer plan, subscriber or customer id, picks up a plan change made in the portal, and the D-21 ordering amendment is recorded where readers look.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| Stripe webhook to the River job | signed events arrive out of order and are retried concurrently |
| job to Stripe API | the job now reads the authoritative subscription before writing |
| reconcile caller to the subscriptions row | a sync must not overwrite state written by the webhook job |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-18.1-35 | Tampering | pgstripe `work` / pgsub `Apply`: an older event snapshot overwrites newer state (CR-01) | high | mitigate | created and updated apply `RetrieveSubscription`'s current state; `WithSessionLock` on `SubscriptionLockKey` serialises jobs of one subscription; the claim-first transaction stays (Task 1) |
| T-18.1-36 | Elevation of Privilege | `ApplyDeleted` on an unknown row commits the marker, a later created grants access (CR-01 sequence 2) | high | mitigate | `ErrSubscriptionNotStored` rolls the marker back and River retries; the retried created event stores Stripe's canceled state (Task 2) |
| T-18.1-37 | Repudiation | a billed subscription is never stored after a reprice or code change (WR-04) | medium | mitigate | plan resolution by price, then product, then plan_code; an unresolved plan still fails loudly with the ids (Task 2) |
| T-18.1-38 | Tampering | `SyncSubscription` lost update (WR-06) | medium | mitigate | row re-read FOR UPDATE, `Select` of the gateway-owned columns only, same subscription lock as the job (Task 3) |
| T-18.1-39 | Denial of Service | a leaked session advisory lock or a slow Stripe retrieve stalls jobs of one subscription | low | mitigate | the lock is keyed per subscription, released in a deferred unlock that ignores cancellation and by connection close; a failed retrieve returns the error so River retries with backoff |
| T-18.1-40 | Information Disclosure | job arguments gain `product_id` and `plan_code` | low | accept | ids and a plan code only, no personal data; `TestJobArgsMinimal`'s fixed key list is extended and still refuses e-mail, name and address keys |
| T-18.1-SC | Tampering | npm/pip/cargo installs | high | mitigate | no package is installed by this plan; GORM `Connection` and Postgres advisory locks are already available |
</threat_model>
<verification>
- Task 1: vet and -short in both plugin repositories; on a Docker host TestFreshStateSmoke, TestHostSubscriptionFlow, TestSubscriptionJob, TestStripeSubscriptionReliability and TestSubscriptionWebhookSmoke pass; pgsub links no stripe-go.
- Task 2: vet and -short in both repositories (TestJobArgsMinimal with the two new keys); the full pgsub Postgres suite and the pgstripe ordering, host, job and reliability tests pass.
- Task 3: the full Postgres suites of both repositories and the gate's plugins stage pass; both trees are clean.
</verification>
<success_criteria>
- CR-01: a stale or concurrently retried subscription event cannot regress the stored state, a deletion before creation is retried, and no spurious reactivated fires (TestFreshStateSmoke on Postgres; full coverage in plan 09).
- WR-04: a subscription on an archived price or a renamed plan is stored on its plan.
- WR-06: SyncSubscription writes only gateway-owned columns of a locked fresh row and refreshes the plan.
- D-21's ordering clause is recorded as superseded; nothing else about D-21 changes; no public route, event, table or response shape changes.
</success_criteria>
<output>
Create `.planning/phases/18.1-payment-gateway-subscriptions/18.1-07-SUMMARY.md` when done, with the commits per repository, every adapted existing assertion (file, test, old and new expectation), the smoke results (or NOT RUN with the reason) and the D-21 amendment text.
</output>

View File

@@ -0,0 +1,414 @@
---
phase: 18.1-payment-gateway-subscriptions
plan: 08
type: execute
wave: 2
depends_on: ["18.1-07"]
files_modified:
- ../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go
- ../sm-paymentgatewaysubscriptions-plugin/models/subscription.go
- ../sm-paymentgatewaysubscriptions-plugin/api/subscription.go
- ../sm-paymentgatewaysubscriptions-plugin/api/json.go
- ../sm-paymentgatewaysubscriptions-plugin/classes/lock.go
- ../sm-paymentgatewaysubscriptions-plugin/classes/service.go
- ../sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go
- ../sm-paymentgatewaysubscriptions-plugin/contracts/subscriber.go
- ../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go
- ../sm-paymentgatewaysubscriptions-plugin/customer_api_smoke_test.go
- ../sm-paymentgatewaysubscriptions-plugin/README.md
- ../sm-pgstripe-plugin/subscriptions/gateway.go
- ../sm-pgstripe-plugin/subscriptions/gateway_test.go
- ../sm-pgstripe-plugin/subscriptions/README.md
- modules/pact/capabilities.go
- modules/cabana/crud.go
- modules/pact/README.md
- modules/cabana/README.md
- docs/backend/admin-controllers.md
- ../vue-paymentgateway-layer/app/utils/subscribe.ts
- ../vue-paymentgateway-layer/app/components/Pg/SubscribeButton.vue
- ../vue-paymentgateway-layer/app/components/Pg/ManageSubscriptionButton.vue
- ../vue-paymentgateway-layer/app/components/Pg/MySubscription.vue
- ../vue-paymentgateway-layer/i18n/locales/en.json
- ../vue-paymentgateway-layer/i18n/locales/pl.json
- ../vue-paymentgateway-layer/README.md
- .planning/phases/18.1-payment-gateway-subscriptions/COVERAGE.md
autonomous: true
gap_closure: true
requirements: [SC-3, D-06, D-13, D-14, D-15, CR-02, WR-01, WR-02, WR-03, WR-07]
estimate:
tokens: 160000
raw_tokens: 160000
tasks: 4
confidence: low
must_haves:
truths:
- "CR-02 (customer API): POST /_paymentgatewaysubscriptions/api/v1/subscription/checkout answers 422 {\"error\": \"Validation failed\", \"errors\": {\"currency\": [...]}} without calling the gateway when the plan has no Stripe price of its own in the requested currency (SubscriptionPlan.CheckoutPriceID is empty): a plan whose stripe_prices map has entries uses only that map, and a plan without a synced map uses stripe_price_id only in the plan's own currency. No currency is ever billed at another currency's price."
- "CR-02 (gateway): the Stripe gateway's CreateCheckoutSession uses CheckoutPriceID and answers ErrNoPriceForCurrency without calling Stripe for a currency without its own price, so Service.Checkout callers outside the customer API get the same refusal; the zero-decimal yen is no longer in SupportsCurrency (the core shows minor units divided by 100), and SubscriptionPlan.StripePriceIDFor keeps PHP's lenient fallback for other readers."
- "WR-02: the Stripe customer create carries an idempotency key built from the subscriber alias and id, a hash of the parameters and a 10-minute window, and the built-in user subscriber stores stripe_customer_id only when none is stored (UPDATE ... WHERE stripe_customer_id IS NULL OR = ''), re-reading and adopting the stored id otherwise; Service.Checkout then uses the stored id, and the write ignores request cancellation, so a customer Stripe created is never lost or overwritten by a concurrent checkout."
- "WR-03: checkout is refused with 409 Already subscribed while the user has any subscription whose status is not canceled or incomplete_expired (active, trialing, past_due, unpaid, incomplete, paused); past_due and unpaid users get the layer's existing billing-portal button on that 409. Service.CheckoutGuarded re-checks under a per-subscriber session advisory lock (classes.SubscriberLockKey) around the re-resolved subscriber, the customer and the session create, and the session create carries an idempotency key per subscriber, plan, currency and parameters in a 10-minute window, so parallel requests from two tabs yield at most one Checkout session per plan and currency."
- "WR-01: the admin bulk action sync_to_stripe is a framework-detached bulk action (pact.AdminBulkAction.Detached, a new additive framework field): cabana loads the selected plans through the list scope without a transaction or row locks, and each plan's Stripe calls run outside any transaction with its ids saved right after them in a short write that ignores request cancellation; the toolbar sync_all_to_stripe opens no transaction and counts a failed save as a failure (no discarded transaction result); the per-plan result toast is unchanged."
- "WR-07: after a redirect to Stripe Checkout or the billing portal, a back/forward-cache restore (pageshow with persisted true) re-enables PgSubscribeButton, PgManageSubscriptionButton and PgMySubscription's manage and cancel actions (flow.restore()), and a navigation that throws re-enables the button with the generic error; each component removes its pageshow listener on unmount."
- "Compatibility: routes, event names, tables, config keys and the customer API response shapes are unchanged (the new refusals reuse the existing 409 and 422 bodies and are documented); contracts.Gateway keeps its methods; Service.Checkout keeps its behaviour (CheckoutGuarded is new); non-detached bulk actions in every plugin behave exactly as before; pgsub links no stripe-go (D-06); every commit keeps go vet and go test ./... green in summercms.go and both plugin repositories and pnpm test green in the layer; nothing is pushed or tagged (release is plan 09)."
artifacts:
- path: "../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go"
provides: "strict per-currency gateway price"
contains: "func (p SubscriptionPlan) CheckoutPriceID"
- path: "../sm-paymentgatewaysubscriptions-plugin/classes/service.go"
provides: "CheckoutGuarded, Open, ErrAlreadySubscribed, cancellation-proof id saves"
contains: "CheckoutGuarded"
- path: "../sm-pgstripe-plugin/subscriptions/gateway.go"
provides: "strict price, idempotency keys, Gateway.Now"
contains: "SetIdempotencyKey"
- path: "modules/pact/capabilities.go"
provides: "the Detached bulk action field"
contains: "Detached"
- path: "../vue-paymentgateway-layer/app/utils/subscribe.ts"
provides: "watchPageRestore and the flows' restore"
contains: "pageshow"
key_links:
- from: "../sm-paymentgatewaysubscriptions-plugin/api/subscription.go"
to: "../sm-paymentgatewaysubscriptions-plugin/classes/service.go"
via: "Checkout handler calls d.svc.CheckoutGuarded and maps ErrAlreadySubscribed to 409"
pattern: "CheckoutGuarded"
- from: "../sm-pgstripe-plugin/subscriptions/gateway.go"
to: "../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go"
via: "CreateCheckoutSession reads plan.CheckoutPriceID(currency)"
pattern: "CheckoutPriceID"
- from: "../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go"
to: "modules/cabana/crud.go"
via: "sync_to_stripe declares Detached: true; BulkAction runs it outside the transaction"
pattern: "Detached"
- from: "../vue-paymentgateway-layer/app/components/Pg/SubscribeButton.vue"
to: "../vue-paymentgateway-layer/app/utils/subscribe.ts"
via: "onMounted watchPageRestore(window, flow.restore), removed onBeforeUnmount"
pattern: "watchPageRestore"
prohibitions:
- statement: "No customer API response body, status for an existing case, route or event name changes; the new refusals reuse the existing 409 and 422 bodies."
status: unresolved
verification: judgment
- statement: "Framework code and docs name no consuming application, payment provider or subscription concept in the Detached addition."
status: unresolved
verification: judgment
- statement: "Another session's uncommitted edits in summercms.go are never staged, stashed or overwritten."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not in user-story format; no story is invented, as in plans 01 to 07): the generic core of `Golem15.PaymentGatewaySubscriptions` and pgstripe's subscription gateway and webhook run on SummerCMS, and a consuming application's subscription flow works as it does on PHP.
This plan's slice (gap closure 2 of 3, wave 2, after 18.1-07 because both change `classes/service.go`, `classes/lock.go` and the Stripe gateway): checkout, admin sync and layer findings of `18.1-REVIEW.md`: CR-02 (cross-currency charge), WR-02 (duplicate Stripe customers), WR-03 (double subscriptions), WR-01 (plan sync inside the bulk transaction) and WR-07 (buttons stuck after Back). The user decided on 2026-10-14 to fix them.
## Decisions taken in this plan (Claude's discretion)
- **WR-01 needs a framework addition.** cabana runs every bulk action inside a transaction that holds `FOR UPDATE` locks on the selected rows. Saving a plan's Stripe ids "in a short transaction of its own" from inside `Run` would wait on that lock until it times out. A Stripe call made outside the lock is therefore impossible from plugin code. The plan adds one opt-in field, `pact.AdminBulkAction.Detached` (`json:"-"`, so neither the list schema nor the OpenAPI changes). It ships in framework tag v0.1.27 at plan 09's release. This is the "framework feature is missing" case of CLAUDE.md. The alternative was rejected: keep the bulk sync inside the transaction and only make product creation self-healing. That still holds the lock and the open transaction across Stripe calls, which is exactly what the review flags. Reversibility: costly, flagged on Task 3.
- **The yen is dropped rather than special-cased.** `money.FromMinor` and the customer API's `amount_minor` assume two decimals. Supporting zero-decimal currencies would change the Money JSON contract. Refusing the yen at checkout (422) is the smaller, non-breaking change, and the README records it as a deviation from PHP's list.
- **CheckoutPriceID is new, StripePriceIDFor stays.** `StripePriceIDFor` is PHP's `getStripePriceId` (lenient fallback), and other readers may rely on it. Checkouts use the strict method. A `stripe_prices` map with at least one entry decides alone. An empty or missing map means the plan was never synced, so a hand-entered `stripe_price_id` serves only the plan's own currency.
- **WR-03 serialisation without a transaction across Stripe.** Local rows appear only after the webhook, so a database lock alone cannot stop two sessions opened before payment. Three measures together cover the race:
- a session-level advisory lock per subscriber, the helper from plan 07, which serialises the check, the customer and the session;
- a Stripe idempotency key over the subscriber, plan, currency and parameters, which returns the same session to a parallel or repeated request;
- the non-terminal status guard, which refuses once a subscription exists.
`Service.Checkout` keeps PHP's unguarded behaviour for app routes that do their own check; `CheckoutGuarded` is the customer-facing entry point.
- **Idempotency keys have a 10-minute window and a parameter hash.** Stripe replays a stored response, including a 500, for 24 hours. The window bounds a replayed failure, and the hash avoids an `idempotency_error` when an application's metadata differs between calls.
- **"Send past_due and unpaid users to the portal"** is met by the existing 409 and the layer's `PgManageSubscriptionButton`, which shows on that 409. The copy loses the word "active" so it fits those users too.
## Artifacts this phase produces (this plan)
- `../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go`: `(SubscriptionPlan) CheckoutPriceID(currency string) string`.
- `../sm-paymentgatewaysubscriptions-plugin/models/subscription.go`: `TerminalStatuses`, `ScopeOpen(db *gorm.DB) *gorm.DB`.
- `../sm-paymentgatewaysubscriptions-plugin/classes/lock.go`: `SubscriberLockKey(alias string, id uint) string`.
- `../sm-paymentgatewaysubscriptions-plugin/classes/service.go`: `ErrAlreadySubscribed`, `(*Service).Open`, `(*Service).CheckoutGuarded`; `ensureCustomer` stores through a cancellation-free context and uses the stored winner; `SyncPlan` with a nil tx saves through a cancellation-free context.
- `../sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go`: first-write-wins `SetCustomerID` with re-read.
- `../sm-paymentgatewaysubscriptions-plugin/contracts/subscriber.go`: `SetCustomerID` doc (an implementation may keep a stored id).
- `../sm-paymentgatewaysubscriptions-plugin/api/subscription.go` and `api/json.go`: the 422 gateway-price gate, the 409 on any open subscription through `Open` and `CheckoutGuarded`, the updated contract comment.
- `../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go`: `sync_to_stripe` with `Detached: true`; `syncOne(ctx, svc, plan, tally)` saves through `SyncPlan(ctx, nil, plan)`; `runSyncAll` without a transaction wrapper.
- `../sm-pgstripe-plugin/subscriptions/gateway.go`: `CreateCheckoutSession` on `CheckoutPriceID`, the yen removed from `supportedCurrencies`, field `Gateway.Now func() time.Time`, unexported `idempotencyWindow` and `idempotencyKey(kind string, parts ...string) string`, keys on the customer create and the session create.
- `modules/pact/capabilities.go`: field `AdminBulkAction.Detached bool` (`json:"-"`). `modules/cabana/crud.go`: the detached branch of `CRUDService.BulkAction`, unexported `loadScoped` with `lockScoped` kept as its locking wrapper.
- `../vue-paymentgateway-layer/app/utils/subscribe.ts`: `PageRestoreTarget`, `watchPageRestore(target, handler): () => void`, `SubscribeFlow.restore()`, `ManageFlow.restore()`, a guarded `location.assign`.
- Smoke subtests: `TestCustomerAPISmoke/currency_without_gateway_price_is_422`, `TestCustomerAPISmoke/past_due_is_409`.
- Adapted tests: `gateway_test.go` subtests "the checkout currency picks the price" (no fallback across currencies) and "supported currencies" (the yen moves to the unsupported list); the customer API smoke's `pro` plan gets `stripe_prices` for the currencies it checks out in.
- Docs: pgsub README, pgstripe subscriptions README, `modules/pact/README.md`, `modules/cabana/README.md`, `docs/backend/admin-controllers.md`, the layer README, `COVERAGE.md`.
<objective>
Close CR-02, WR-02, WR-03, WR-01 and WR-07:
- a checkout charges only the price of the currency it shows;
- concurrent or repeated checkouts create neither a second Stripe customer nor a second session or subscription;
- any open subscription blocks a new checkout;
- the admin plan sync never calls Stripe inside a transaction or loses the ids Stripe created;
- the layer's buttons recover after the browser's Back button.
Purpose: these money-relevant and user-facing defects must be fixed before any application cuts over to this plugin.
Output: commits in summercms.go (framework Detached, docs), sm-paymentgatewaysubscriptions-plugin, sm-pgstripe-plugin and vue-paymentgateway-layer, plus a planning-docs commit (COVERAGE). New unit-test coverage is plan 09. This plan carries two smoke subtests and the adaptations of existing tests whose expectations the fixes change.
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-CONTEXT.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-04-SUMMARY.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-05-SUMMARY.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-07-SUMMARY.md
Repositories: summercms.go (framework), ../sm-paymentgatewaysubscriptions-plugin (pgsub), ../sm-pgstripe-plugin (pgstripe), ../vue-paymentgateway-layer (layer). The plugins `replace` summercms with ../summercms.go and pgstripe replaces pgsub, so local builds see uncommitted siblings. Commit order inside a task: framework, then pgsub, then pgstripe, then the layer; one logical change per commit; stage by explicit path; no co-author tags; never push or tag. Several sessions work in summercms.go: before touching a framework file, `git status --porcelain -- <file>` must show no foreign edit; if it does, stop and report instead of staging, stashing or overwriting.
<interfaces>
- pgsub customer API: `Checkout` api/subscription.go:173-241 (order: body, plan_code, 409 via `d.svc.Active` :192, plan 404, free 422, gateway, currency 422 via `priced` and `SupportsCurrency` :221, URLs, `d.svc.Subscriber` :229, `d.svc.Checkout` :234, `writeGatewayError` maps to 502 or 503); `priced` :245; error bodies api/errors.go:15-30 (`errAlreadySubscribed` "Already subscribed", `msgCurrencyInvalid`); the contract comment api/json.go:1-33.
- `models.SubscriptionPlan.StripePriceIDFor` :109-119 (lenient), `PriceCentsFor` :97; `models.ScopeActive`, `ForSubscriber`, status consts models/subscription.go:14-28, :100-119.
- `classes.Service.Checkout` service.go:263-279, `ensureCustomer` :392-412, `SyncPlan` :362-388 (nil tx falls back to s.db), `Active` :174, `Latest` :105, `Subscriber` :222; `WithSessionLock` and `SubscriptionLockKey` in classes/lock.go (plan 07).
- `userSubscriber.SetCustomerID` classes/user_subscribable.go:85-98 (unconditional UPDATE today); `contracts.Subscriber` contracts/subscriber.go.
- pgstripe gateway: `GetOrCreateCustomer` gateway.go:76-109, `CreateCheckoutSession` :120-181 (`StripePriceIDFor` at :128, metadata map :144-158), `supportedCurrencies` :32, `ErrNoPriceForCurrency` :25; stripe-go v87 `(*stripe.Params).SetIdempotencyKey(string)` (params.go:310) on `CustomerCreateParams` and `CheckoutSessionCreateParams`.
- Admin sync: controllers/subscriptionplans.go `AdminBulkActions` :48, `AdminActions` :62, `syncOne` :82, `runSyncSelected` :130-153, `runSyncAll` :155-179.
- Framework: `pact.AdminBulkAction` modules/pact/capabilities.go:285-291; `CRUDService.BulkAction` modules/cabana/crud.go:349-391, `lockScoped` :939-971, `actionFailure` :396, `bulkActionOf` modules/cabana/actions.go:173; bulk docs docs/backend/admin-controllers.md:439-447; pact README table :132-135; cabana README bullet :32; bulk tests modules/cabana/phase121_bulk_test.go (13 `TestBulkAction*` functions).
- Layer: `createSubscribeFlow` and `createManageFlow` app/utils/subscribe.ts:70-149 (`state.redirecting = true` then `deps.location.assign`); the three components build their flows with `location: { assign: (url) => window.location.assign(url) }`; `already_subscribed` copy i18n/locales/en.json:147 and pl.json:147; README § Subscriptions :125-179.
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: a checkout in a currency without its own Stripe price is refused end to end (customer API 422, gateway ErrNoPriceForCurrency), never billed at another currency's price (CR-02)</name>
<reversibility rating="reversible">A stricter price choice behind unchanged routes and bodies; reverting restores the fallback.</reversibility>
<files>../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go, ../sm-paymentgatewaysubscriptions-plugin/api/subscription.go, ../sm-paymentgatewaysubscriptions-plugin/api/json.go, ../sm-paymentgatewaysubscriptions-plugin/customer_api_smoke_test.go, ../sm-paymentgatewaysubscriptions-plugin/README.md, ../sm-pgstripe-plugin/subscriptions/gateway.go, ../sm-pgstripe-plugin/subscriptions/gateway_test.go, ../sm-pgstripe-plugin/subscriptions/README.md</files>
<read_first>../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go, ../sm-paymentgatewaysubscriptions-plugin/api/subscription.go, ../sm-paymentgatewaysubscriptions-plugin/api/json.go (contract comment, planPrices), ../sm-paymentgatewaysubscriptions-plugin/api/errors.go, ../sm-paymentgatewaysubscriptions-plugin/customer_api_smoke_test.go (plan setup :158-172, checkout_checks :226, checkout :246), ../sm-paymentgatewaysubscriptions-plugin/fixture_plugin_test.go (fakeGateway.SupportsCurrency :246, CreateCheckoutSession :171), ../sm-pgstripe-plugin/subscriptions/gateway.go (CreateCheckoutSession, supportedCurrencies, SupportsCurrency), ../sm-pgstripe-plugin/subscriptions/gateway_test.go (subtests "the checkout currency picks the price" :122 and "supported currencies" :241), ../sm-pgstripe-plugin/subscriptions/sync_plan.go (which currencies a sync prices), both READMEs (Customer API table, Features: Checkout and Currencies, Deviations), .planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md (CR-02 in full)</read_first>
<action>Implements CR-02 on the D-13 customer API and the D-15 plan-sync price map; response shapes stay as D-13's confirmed contract.
pgsub (commit first): models/subscription_plan.go adds `CheckoutPriceID(currency string) string`. The currency is trimmed and lower-cased. When `StripePrices` is valid with at least one entry, it answers that map's entry for the currency or "" (no fallback). Otherwise it answers `*StripePriceID` only when the currency equals the plan's `Currency` case-insensitively and the id is non-empty, else "". The doc comment of `StripePriceIDFor` says it is PHP's lenient getStripePriceId and that checkouts use CheckoutPriceID. In api/subscription.go `Checkout`, after the existing `priced` and `SupportsCurrency` check and before the URL check, `plan.CheckoutPriceID(currency) == ""` answers the same 422 `{"currency": [msgCurrencyInvalid]}` without calling the gateway. Update the handler's doc comment and the api/json.go contract comment: a 422 also covers a currency the plan has no gateway price in, and no currency is billed at another currency's price. customer_api_smoke_test.go:
- the `pro` plan gets `stripe_prices` entries for pln and eur, so the existing `checkout` subtest (EUR) still passes;
- a new subtest `currency_without_gateway_price_is_422` creates a plan with currency_prices pln and gbp and stripe_prices pln and usd, then asserts that a gbp checkout answers 422 with the currency error and makes no gateway call, and that a pln checkout answers 200 with the gateway called in pln.
README (pgsub): Customer API table row of checkout (422 text); API reference adds `CheckoutPriceID`.
pgstripe (second commit), gateway.go:
- `CreateCheckoutSession` takes its price from `plan.CheckoutPriceID(currency)`; "" stays `ErrNoPriceForCurrency` (wrapping, naming plan and currency, no Stripe call). Its comments describe the strict rule by concept and do not name the lenient model method. Update the sentinel's doc: no price of its own in the currency, no fallback across currencies.
- Drop the zero-decimal Japanese yen code from `supportedCurrencies`. `SupportsCurrency`'s doc says the list is PHP's minus the yen, because the core's money display assumes two decimals.
Adapt gateway_test.go in the same commit:
- "the checkout currency picks the price": a currency the gold plan's map lacks now expects ErrNoPriceForCurrency and no Stripe call, instead of the default price id;
- "supported currencies": the yen moves to the refused list.
Run the subscriptions suite and adapt any other assertion that relied on the fallback, naming each in the SUMMARY. README (subscriptions):
- Features "Checkout" names CheckoutPriceID and the missing fallback;
- the "Currencies" line drops the yen;
- "Deviations from PHP" gains two lines: no cross-currency fallback (PHP getStripePriceId fell back to stripe_price_id), and the yen is not supported.</action>
<verify>
<automated>go -C ../sm-paymentgatewaysubscriptions-plugin vet ./... &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -short ./... &amp;&amp; go -C ../sm-pgstripe-plugin vet ./... &amp;&amp; go -C ../sm-pgstripe-plugin test -count=1 -short ./...</automated>
<fails_when>non-zero exit: vet reports an issue or a Docker-free test fails in either plugin repository</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; OUT=$(go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -run '^(TestCustomerAPISmoke|TestCustomerAPIAuthz|TestServiceAPI)$' -v . 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -c '^--- PASS: Test' | grep -qx 3 &amp;&amp; OUT2=$(go -C ../sm-pgstripe-plugin test -count=1 -run '^(TestSubscriptionGateway|TestHostSubscriptionFlow)$' -v ./subscriptions 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT2" | grep -c '^--- PASS: Test' | grep -qx 2</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), fewer than three pgsub top-level PASS lines (the 422 gate refused a priced currency or let an unpriced one through), or fewer than two pgstripe PASS lines (the gateway fell back across currencies or the host checkout broke)</fails_when>
</verify>
<acceptance_criteria>
- `grep -n "func (p SubscriptionPlan) CheckoutPriceID" ../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go` prints one line.
- `grep -n "CheckoutPriceID" ../sm-paymentgatewaysubscriptions-plugin/api/subscription.go ../sm-pgstripe-plugin/subscriptions/gateway.go` prints at least two lines.
- `sed -n '/func (g \*Gateway) CreateCheckoutSession/,/^}/p' ../sm-pgstripe-plugin/subscriptions/gateway.go | grep -c "StripePriceIDFor"` prints 0.
- `grep -n 'supportedCurrencies = ' ../sm-pgstripe-plugin/subscriptions/gateway.go | grep -c jpy` prints 0.
- `grep -n "currency_without_gateway_price_is_422" ../sm-paymentgatewaysubscriptions-plugin/customer_api_smoke_test.go` prints one line.
- `test "$(go -C ../sm-paymentgatewaysubscriptions-plugin list -deps ./... | grep -c stripe-go)" = 0` succeeds (D-06).
</acceptance_criteria>
<done>A request for a currency the plan was never priced in at Stripe is refused before any Stripe call, from the customer API through the service to the gateway, and a priced currency checks out on its own price.</done>
</task>
<task type="auto">
<name>Task 2: One Stripe customer and one checkout per subscriber: first-write-wins customer id with idempotency keys (WR-02), and a guarded checkout refused for any open subscription under a per-subscriber lock (WR-03)</name>
<reversibility rating="costly">New exported service API (`CheckoutGuarded`, `Open`, `ErrAlreadySubscribed`, `ScopeOpen`, `TerminalStatuses`) and a relaxed `Subscriber.SetCustomerID` contract (first write may win): additive, but published plugin API once tagged.</reversibility>
<files>../sm-paymentgatewaysubscriptions-plugin/models/subscription.go, ../sm-paymentgatewaysubscriptions-plugin/classes/lock.go, ../sm-paymentgatewaysubscriptions-plugin/classes/service.go, ../sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go, ../sm-paymentgatewaysubscriptions-plugin/contracts/subscriber.go, ../sm-paymentgatewaysubscriptions-plugin/api/subscription.go, ../sm-paymentgatewaysubscriptions-plugin/api/json.go, ../sm-paymentgatewaysubscriptions-plugin/customer_api_smoke_test.go, ../sm-paymentgatewaysubscriptions-plugin/README.md, ../sm-pgstripe-plugin/subscriptions/gateway.go, ../sm-pgstripe-plugin/subscriptions/README.md, .planning/phases/18.1-payment-gateway-subscriptions/COVERAGE.md</files>
<read_first>../sm-paymentgatewaysubscriptions-plugin/classes/service.go (Checkout, ensureCustomer, Active, Latest, Subscriber), ../sm-paymentgatewaysubscriptions-plugin/classes/lock.go (plan 07), ../sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go, ../sm-paymentgatewaysubscriptions-plugin/contracts/subscriber.go, ../sm-paymentgatewaysubscriptions-plugin/models/subscription.go (statuses, scopes), ../sm-paymentgatewaysubscriptions-plugin/api/subscription.go (Checkout after Task 1), ../sm-paymentgatewaysubscriptions-plugin/user_subscribable_test.go (whether it asserts an overwrite), ../sm-paymentgatewaysubscriptions-plugin/service_api_test.go (checkout subtests, the open-transaction probe), ../sm-pgstripe-plugin/subscriptions/gateway.go (GetOrCreateCustomer, CreateCheckoutSession), ../sm-pgstripe-plugin/subscriptions/stripefake_test.go (record: whether headers are kept), .planning/phases/18.1-payment-gateway-subscriptions/COVERAGE.md, .planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md (WR-02, WR-03 in full)</read_first>
<action>Implements WR-02 and WR-03 for the D-13 customer API; D-08/D-09 (the user subscriber writes only users.stripe_customer_id) hold.
pgsub (commit first):
- models/subscription.go adds `TerminalStatuses` (canceled, incomplete_expired) and `ScopeOpen` (stripe_status NOT IN TerminalStatuses).
- classes/lock.go adds `SubscriberLockKey(alias, id)` built from the same prefix as SubscriptionLockKey, with "subscriber:", the alias, ":" and the decimal id.
- classes/service.go adds `ErrAlreadySubscribed`, `Open(ctx, alias, id)` and `CheckoutGuarded(ctx, sub, plan, successURL, cancelURL, o)`:
- `Open` returns the latest subscription with ScopeOpen and its Plan loaded, ordered like Latest, or nil.
- `CheckoutGuarded` runs inside `WithSessionLock(ctx, s.db, SubscriberLockKey(sub.MorphAlias(), sub.Key()), ...)`. Inside the lock it calls `Open`; an open subscription returns an error wrapping ErrAlreadySubscribed that names the row id and status. It re-resolves the subscriber with `s.Subscriber` (so a customer id stored by a concurrent request is seen), then delegates to `s.Checkout`. No transaction is held across the gateway calls; the lock is session-level.
- `Checkout` itself is unchanged (PHP createCheckoutSession has no guard). Its doc points customer-facing callers to CheckoutGuarded.
- `ensureCustomer`:
- stores the id in its short transaction as today, now opened with `context.WithoutCancel(ctx)`, so a customer Stripe already created is recorded even when the request was cancelled;
- after the store, reads `sub.CustomerID()`. When it differs from the created id, it logs at Warn that a concurrent checkout stored another customer, with both ids (ids only), and continues with the stored one.
- classes/user_subscribable.go `SetCustomerID`:
- runs `UPDATE users SET stripe_customer_id = ? WHERE id = ? AND (stripe_customer_id IS NULL OR stripe_customer_id = '')`;
- with one row affected it remembers id;
- with none it re-reads stripe_customer_id of that id with a raw query (no soft-delete filter): no row is ErrSubscriberNotFound, a stored non-empty id is remembered and nil returned, an empty one is an error.
- It still writes no other column.
- contracts/subscriber.go: the `SetCustomerID` doc says an implementation may keep an id that is already stored (first write wins), and CustomerID then answers the stored id.
- api/subscription.go `Checkout` answers 409 errAlreadySubscribed in two places:
- its early check (same position) uses `d.svc.Open` instead of `d.svc.Active`;
- after the subscriber is resolved it calls `d.svc.CheckoutGuarded`, and an error wrapping `classes.ErrAlreadySubscribed` answers 409.
- Other errors keep `writeGatewayError`.
- api/json.go contract comment: 409 for any subscription that is not canceled or incomplete_expired (a past_due or unpaid user manages it in the billing portal).
- customer_api_smoke_test.go adds subtest `past_due_is_409`: a past_due row of the user answers 409 "Already subscribed" and the gateway is not called. Adapt any existing assertion that expects an overwrite of a stored customer id, naming it in the SUMMARY.
README (pgsub):
- API reference lists Open, CheckoutGuarded, ErrAlreadySubscribed, ScopeOpen, TerminalStatuses and SubscriberLockKey;
- the Customer API table's 409 row;
- Usage: customer-facing checkouts call CheckoutGuarded;
- the built-in user subscriber keeps the first stored customer id.
pgstripe (second commit), gateway.go:
- adds the field `Now func() time.Time` (nil means time.Now), an unexported `idempotencyWindow` of 10 minutes, and an unexported `idempotencyKey(kind string, parts ...string) string`. The key is "pgsub-", the kind, "-", the decimal window number (Unix seconds of Now divided by the window), "-", and the first 32 hex characters of the SHA-256 (crypto/sha256, encoding/hex) of the parts joined with a NUL byte.
- `GetOrCreateCustomer` sets the key on its CustomerCreateParams: kind "customer"; parts are the alias, the id, the e-mail, the name and the sorted metadata pairs.
- `CreateCheckoutSession` sets the key on its CheckoutSessionCreateParams: kind "checkout"; parts are the alias, the id, the customer id, the price id, the success and cancel URLs, and the sorted metadata pairs.
- Keys are never logged.
README (subscriptions): the Customer and Checkout features describe the keys and the window; API reference lists `Gateway.Now`.
Planning docs (summercms.go, own commit): COVERAGE.md rows "customers.create with subscriber metadata" and "checkout.sessions.create (mode subscription)" keep INTEGRATE; their reasons name the idempotency key (WR-02 and WR-03, plan 18.1-08).</action>
<verify>
<automated>go -C ../sm-paymentgatewaysubscriptions-plugin vet ./... &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -short ./... &amp;&amp; go -C ../sm-pgstripe-plugin vet ./... &amp;&amp; go -C ../sm-pgstripe-plugin test -count=1 -short ./...</automated>
<fails_when>non-zero exit: vet reports an issue or a Docker-free test fails in either plugin repository</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 ./... &amp;&amp; OUT=$(go -C ../sm-pgstripe-plugin test -count=1 -run '^(TestSubscriptionGateway|TestHostSubscriptionFlow|TestFreshStateSmoke)$' -v ./subscriptions 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -c '^--- PASS: Test' | grep -qx 3</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), a pgsub Postgres test fails (TestCustomerAPISmoke/past_due_is_409, TestUserSubscribable, TestServiceAPI's checkout subtests among them), or fewer than three pgstripe top-level PASS lines</fails_when>
</verify>
<acceptance_criteria>
- `grep -n "func (s \*Service) CheckoutGuarded\|func (s \*Service) Open\|ErrAlreadySubscribed =" ../sm-paymentgatewaysubscriptions-plugin/classes/service.go` prints three lines.
- `grep -n "WithoutCancel" ../sm-paymentgatewaysubscriptions-plugin/classes/service.go` prints at least one line.
- `grep -n "stripe_customer_id IS NULL" ../sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go` prints one line.
- `grep -n "CheckoutGuarded\|svc.Open" ../sm-paymentgatewaysubscriptions-plugin/api/subscription.go` prints at least two lines.
- `grep -c "SetIdempotencyKey" ../sm-pgstripe-plugin/subscriptions/gateway.go` prints at least 2.
- `grep -n "past_due_is_409" ../sm-paymentgatewaysubscriptions-plugin/customer_api_smoke_test.go` prints one line.
- `grep -n "18.1-08" .planning/phases/18.1-payment-gateway-subscriptions/COVERAGE.md` prints at least two lines.
</acceptance_criteria>
<done>A double click, a second tab or a client retry yields one Stripe customer and at most one Checkout session per plan and currency, and a user with any open subscription is sent to manage it instead of buying a second one.</done>
</task>
<task type="auto">
<name>Task 3: Plan sync never holds a transaction or row lock across Stripe calls and never loses the ids Stripe created: framework detached bulk actions and the reworked sync actions (WR-01)</name>
<reversibility rating="costly">Adds `pact.AdminBulkAction.Detached`, a framework API field that becomes part of tag v0.1.27 at plan 09's release; additive and opt-in, but removing it later would break any plugin that sets it.</reversibility>
<precondition>`git status --porcelain -- modules/pact/capabilities.go modules/cabana/crud.go modules/pact/README.md modules/cabana/README.md docs/backend/admin-controllers.md` prints nothing in summercms.go (no other session has uncommitted edits in these files).</precondition>
<files>modules/pact/capabilities.go, modules/cabana/crud.go, modules/pact/README.md, modules/cabana/README.md, docs/backend/admin-controllers.md, ../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go, ../sm-paymentgatewaysubscriptions-plugin/classes/service.go, ../sm-paymentgatewaysubscriptions-plugin/README.md</files>
<read_first>modules/pact/capabilities.go (AdminBulkAction and its doc :274-316), modules/cabana/crud.go (BulkAction :349-391, lockScoped :939-971, the transaction helper :133), modules/cabana/actions.go (bulkActionOf), modules/cabana/tx_context.go (TxFromContext), modules/cabana/phase121_bulk_test.go (what the 13 bulk tests pin), docs/backend/admin-controllers.md (:430-450), modules/pact/README.md (:30, :132-135), modules/cabana/README.md (:32), ../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go (whole file), ../sm-paymentgatewaysubscriptions-plugin/classes/service.go (SyncPlan), ../sm-paymentgatewaysubscriptions-plugin/admin_actions_test.go (what TestAdminSyncActions asserts), ../sm-paymentgatewaysubscriptions-plugin/README.md (Admin section), .planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md (WR-01 in full)</read_first>
<action>Implements WR-01 for the D-15 plan sync. The framework addition is the only way to keep Stripe calls out of cabana's bulk transaction (see "Decisions taken in this plan").
Framework (summercms.go, commit first, after the precondition check):
- modules/pact/capabilities.go: `AdminBulkAction` gains `Detached bool` tagged `json:"-"`. Its doc: Run then runs outside the framework transaction; the selected records are loaded through the list scope without row locks; the context carries no write transaction; Run opens the short transactions it needs; an action that calls an external service sets it so that no transaction or row lock is held across the call; what such an action saved before a later failure stays saved. The type's doc paragraph mentions the field.
- modules/cabana/crud.go `BulkAction`: after the existing name, id and model checks, an action with Detached set takes a new branch that opens no transaction. It loads the rows with a new unexported `loadScoped(ctx, db, cc, proto, ids, lock bool)`; `lockScoped` becomes a call of it with lock true, so the locked path is byte-identical. The branch loads on `s.DB` with lock false and applies the same rules:
- no row: affected 0 without running the action;
- fewer rows than ids: the existing partial-selection 409;
- Run is called with the locale context but no write transaction;
- an error goes through `actionFailure`;
- the message is translated with `translateKey` as on the locked path.
The non-detached path is unchanged.
- Docs in the same commit, naming no application, provider or subscription:
- docs/backend/admin-controllers.md: the bulk route list gains a bullet for `Detached: true` (loaded through the same scope without row locks, `cabana.TxFromContext` reports no transaction, Run opens its own short transactions, the use case of external calls, earlier saves stay on a later failure);
- modules/pact/README.md: the `pact.AdminBulkAction` table row mentions the field;
- modules/cabana/README.md: the bulk actions bullet mentions the detached variant.
pgsub (second commit):
- controllers/subscriptionplans.go:
- The sync_to_stripe bulk action sets `Detached: true`.
- `runSyncSelected` no longer reads the framework transaction from the context. It type-checks the records as today and syncs each plan.
- `syncOne` drops its transaction parameter and calls `svc.SyncPlan(ctx, nil, plan)`.
- `runSyncAll` iterates the active plans and calls `syncOne` directly, with no transaction wrapper around it. A failed save comes back from SyncPlan as an error and becomes a failure line, never a synced count.
- Update the doc comments of AdminBulkActions, AdminActions, syncOne and both run functions: no transaction is held across the Stripe calls, and each plan's ids are saved right after its calls.
- Remove imports that become unused.
- classes/service.go `SyncPlan`: a nil tx saves through `s.db` with `context.WithoutCancel(ctx)`, so the ids Stripe created are recorded although the admin's request was cancelled. A non-nil tx keeps today's behaviour. The doc says request paths pass nil, and the save stays the `Select` of stripe_product_id, stripe_prices, stripe_price_id and updated_at.
- README (pgsub) Admin section: both sync actions run outside any transaction, and each plan's ids are saved right after its Stripe calls.
The plugin go.mod keeps `require summercms v0.1.26` with the replace. The v0.1.27 tag and the require bump are plan 09's release.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/cabana ./modules/pact &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; go run ./cmd/summer docs:build --check &amp;&amp; bash scripts/check-admin-openapi.sh --check</automated>
<fails_when>non-zero exit: vet, a Docker-free cabana or pact test, the docs tree or docs:build checker (an identifier or link in the new docs text that does not exist), or OpenAPI drift (the new field leaked into the schema)</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; OUT=$(go test -count=1 -run '^TestBulkAction' -v ./modules/cabana 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -c '^--- PASS: TestBulkAction' | grep -qx 13 &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin vet ./... &amp;&amp; OUT2=$(go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -run '^(TestAdminSyncActions|TestAdminPlansSmoke|TestServiceAPI|TestSubscriptionsCoreSmoke)$' -v . 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT2" | grep -c '^--- PASS: Test' | grep -qx 4</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), fewer than 13 framework TestBulkAction passes (the locked path changed), or fewer than four pgsub top-level PASS lines (a sync result message, a saved id or the service sync changed)</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'Detached ' modules/pact/capabilities.go | grep -c 'json:"-"'` prints 1.
- `grep -n "Detached" modules/cabana/crud.go` prints at least one line and `grep -n "func loadScoped" modules/cabana/crud.go` prints one line.
- `grep -n "Detached" docs/backend/admin-controllers.md modules/pact/README.md modules/cabana/README.md` prints at least three lines.
- `grep -rniE "stripe|subscription|horoskopia|queststream|quotify" modules/pact/capabilities.go modules/cabana/crud.go | grep -i detached` prints nothing.
- `grep -n "Detached:" ../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go` prints one line.
- `grep -c 'TxFromContext(ctx)' ../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go` prints 0 (the bulk action no longer reads the framework transaction).
- `grep -c 'lagoon.Transaction' ../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go` prints 0 (neither sync action wraps the gateway calls in a transaction).
- `git status --porcelain -- modules/pact modules/cabana docs/backend` in summercms.go prints nothing after the framework commit (committed, nothing foreign staged).
</acceptance_criteria>
<done>Selecting plans and clicking "Sync to Stripe" or "Sync all" reports per-plan results as before. No transaction or row lock is open while Stripe is called, and an id Stripe created is saved even if the request is cut off.</done>
</task>
<task type="auto">
<name>Task 4: Subscribe and manage buttons recover after Back from Stripe (bfcache) or a failed navigation, and the 409 copy fits every open subscription (WR-07, WR-03 copy)</name>
<reversibility rating="reversible">Client-side state handling and copy in the layer; reverting restores the old behaviour.</reversibility>
<files>../vue-paymentgateway-layer/app/utils/subscribe.ts, ../vue-paymentgateway-layer/app/components/Pg/SubscribeButton.vue, ../vue-paymentgateway-layer/app/components/Pg/ManageSubscriptionButton.vue, ../vue-paymentgateway-layer/app/components/Pg/MySubscription.vue, ../vue-paymentgateway-layer/i18n/locales/en.json, ../vue-paymentgateway-layer/i18n/locales/pl.json, ../vue-paymentgateway-layer/README.md</files>
<read_first>../vue-paymentgateway-layer/app/utils/subscribe.ts (whole file), ../vue-paymentgateway-layer/app/components/Pg/SubscribeButton.vue, ../vue-paymentgateway-layer/app/components/Pg/ManageSubscriptionButton.vue, ../vue-paymentgateway-layer/app/components/Pg/MySubscription.vue (manage flow, refresh), ../vue-paymentgateway-layer/tests/subscribe.test.ts and tests/subscriptionComponents.test.ts (what the existing specs pin, including copy), ../vue-paymentgateway-layer/i18n/locales/en.json and pl.json (pg.subscriptions.error.already_subscribed), ../vue-paymentgateway-layer/README.md (§ Subscriptions), .planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md (WR-07)</read_first>
<action>Implements WR-07 in the D-14 layer and the copy half of WR-03.
app/utils/subscribe.ts:
- Add an exported interface `PageRestoreTarget` with `addEventListener` and `removeEventListener` for a `pageshow` listener. Add `watchPageRestore(target, handler)`, which returns a remover:
- it registers a pageshow listener that calls handler only when the event's `persisted` is true (a back/forward-cache restore);
- the remover unregisters it;
- an undefined or null target (server rendering) returns a no-op remover.
- `SubscribeFlow` and `ManageFlow` gain `restore(): void`, which sets busy and redirecting false and error null.
- In both flows, the `deps.location.assign(url)` call moves inside a try block. A throw resets busy and redirecting and sets error to pg.subscriptions.error.generic.
- Update the header comment: the redirect keeps the button busy until the page is left, and a back/forward-cache restore re-enables it.
Components:
- SubscribeButton.vue and ManageSubscriptionButton.vue keep a remover. `onMounted` sets it from `watchPageRestore(window, () => flow.restore())`; `onBeforeUnmount` calls it.
- MySubscription.vue does the same for its manage flow and also calls `refresh()` on a restore, so the panel shows what the portal changed.
- Update each header comment.
Copy (WR-03): en and pl `pg.subscriptions.error.already_subscribed` drop "active" ("You already have a subscription. Manage it from your billing account." and "Masz już subskrypcję. Możesz nią zarządzać na koncie rozliczeniowym."), because past_due and unpaid users get the same 409 and the manage button.
README § Subscriptions: describe `restore()`, `watchPageRestore` and the back/forward-cache behaviour, and that the 409 covers any open subscription.
Run `pnpm test` and adapt only assertions that pin the old copy, naming each in the SUMMARY. Do not change package.json (the 0.2.1 version and tag are plan 09). Commit in the layer only.</action>
<verify>
<automated>OUT=$(pnpm --dir ../vue-paymentgateway-layer test 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -qE '^# fail 0$' &amp;&amp; bash scripts/check-phase18.1.sh layers</automated>
<fails_when>non-zero exit, or the TAP summary lacks "# fail 0" (a layer spec failed), or the gate's layers stage reports a required subscriptions test missing or fewer than its minimum passing</fails_when>
</verify>
<acceptance_criteria>
- `grep -n "export function watchPageRestore\|restore()" ../vue-paymentgateway-layer/app/utils/subscribe.ts` prints at least two lines and `grep -c "pageshow" ../vue-paymentgateway-layer/app/utils/subscribe.ts` prints at least 1.
- `grep -l "onBeforeUnmount" ../vue-paymentgateway-layer/app/components/Pg/SubscribeButton.vue ../vue-paymentgateway-layer/app/components/Pg/ManageSubscriptionButton.vue ../vue-paymentgateway-layer/app/components/Pg/MySubscription.vue | wc -l` prints 3.
- `grep -c "watchPageRestore" ../vue-paymentgateway-layer/app/components/Pg/SubscribeButton.vue ../vue-paymentgateway-layer/app/components/Pg/ManageSubscriptionButton.vue ../vue-paymentgateway-layer/app/components/Pg/MySubscription.vue` reports at least 1 per file.
- `grep -c '"already_subscribed": "You already have a subscription. Manage it from your billing account."' ../vue-paymentgateway-layer/i18n/locales/en.json` prints 1 and `grep -c '"already_subscribed": "Masz już subskrypcję.' ../vue-paymentgateway-layer/i18n/locales/pl.json` prints 1.
- `git -C ../vue-paymentgateway-layer diff v0.2.0 --stat -- package.json pnpm-lock.yaml` prints nothing (no dependency or version change in this plan).
</acceptance_criteria>
<done>A user who goes to Stripe Checkout or the portal and presses Back finds working Subscribe and Manage buttons, and a user with a past_due subscription is told to manage the subscription they have.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| browser to customer API | the authenticated user chooses plan and currency; the server decides the price |
| customer API and admin to Stripe | Checkout, customer and plan-sync calls create billable objects |
| admin bulk action to the database | the framework's bulk transaction and its row locks |
| browser history to the layer | back/forward-cache restores JavaScript state |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-18.1-41 | Tampering | `api.Checkout` / `Gateway.CreateCheckoutSession`: a currency without its own price is billed at another currency's price (CR-02) | high | mitigate | `CheckoutPriceID` with no cross-currency fallback in both the API 422 gate and the gateway; the yen refused (Task 1) |
| T-18.1-42 | Tampering | `ensureCustomer` / `SetCustomerID`: concurrent checkouts create two Stripe customers and the last write wins (WR-02) | medium | mitigate | idempotency key per subscriber and parameters; first-write-wins conditional UPDATE with re-read; cancellation-free store (Task 2) |
| T-18.1-43 | Tampering | `api.Checkout`: a user with a past_due, unpaid, incomplete or paused subscription, or two parallel tabs, gets a second billing subscription (WR-03) | high | mitigate | 409 for any open status; `CheckoutGuarded` re-checks under `SubscriberLockKey`; idempotent session create (Task 2) |
| T-18.1-44 | Denial of Service | plan sync holds the bulk transaction and plan row locks across Stripe calls; a rollback loses created ids (WR-01) | medium | mitigate | `Detached` bulk action; per-plan short saves through a cancellation-free context; toolbar sync without a transaction wrapper (Task 3) |
| T-18.1-45 | Elevation of Privilege | a detached bulk action bypasses the list scope, the partial-selection rule or the permissions | medium | mitigate | the detached branch loads through the same `ListExtendQuery` scope with the same empty and 409 rules; permissions are checked before BulkAction as today; the locked path is unchanged (13 TestBulkAction passes) |
| T-18.1-46 | Denial of Service | an idempotent replay returns a stored Stripe failure or rejects changed parameters | low | mitigate | 10-minute window bounds a replayed failure; the parameter hash gives changed parameters a new key; keys are never logged |
| T-18.1-47 | Denial of Service | buttons stay disabled after a back/forward-cache restore (WR-07) | low | mitigate | `watchPageRestore` plus `restore()` in all three components; a throwing navigation resets the state (Task 4) |
| T-18.1-SC | Tampering | npm/pip/cargo installs | high | mitigate | no package is installed: crypto/sha256 and encoding/hex are stdlib, the layer adds no dependency (package.json and pnpm-lock.yaml unchanged) |
</threat_model>
<verification>
- Task 1: vet and -short in both plugin repositories; on Docker the customer API smoke and authz, the service API, the gateway and the host flow pass; pgsub links no stripe-go.
- Task 2: vet and -short in both repositories; the full pgsub Postgres suite, the gateway, host and fresh-state tests pass.
- Task 3: framework vet, Docker-free cabana and pact tests, docs and OpenAPI checks; on Docker all 13 framework bulk-action tests and the pgsub admin, plan-screen, service and core smokes pass.
- Task 4: the layer suite with "# fail 0" and the gate's layers stage.
</verification>
<success_criteria>
- CR-02: no checkout is billed at another currency's price; the refusal is 422 at the API and ErrNoPriceForCurrency at the gateway.
- WR-02 and WR-03: one customer per subscriber; no checkout while any subscription is open; parallel requests yield one session per plan and currency.
- WR-01: no Stripe call under a transaction or row lock in either sync action; ids survive a cancelled request.
- WR-07: the buttons recover after Back.
- No public route, event, table or response shape changes; the framework addition is opt-in and documented.
</success_criteria>
<output>
Create `.planning/phases/18.1-payment-gateway-subscriptions/18.1-08-SUMMARY.md` when done, with the commits per repository, every adapted existing assertion (file, test, old and new expectation), the smoke results (or NOT RUN with the reason) and a note that the framework needs tag v0.1.27 before the plugins can be built from git without the replace.
</output>

View File

@@ -0,0 +1,417 @@
---
phase: 18.1-payment-gateway-subscriptions
plan: 09
type: execute
wave: 3
depends_on: ["18.1-07", "18.1-08"]
files_modified:
- scripts/check-phase18.1.sh
- modules/cabana/bulk_detached_test.go
- modules/cabana/README.md
- ../sm-pgstripe-plugin/subscriptions/ordering_test.go
- ../sm-pgstripe-plugin/subscriptions/gateway_hardening_test.go
- ../sm-pgstripe-plugin/subscriptions/stripefake_test.go
- ../sm-pgstripe-plugin/subscriptions/decode_test.go
- ../sm-pgstripe-plugin/subscriptions/README.md
- ../sm-pgstripe-plugin/go.mod
- ../sm-paymentgatewaysubscriptions-plugin/models/checkout_price_test.go
- ../sm-paymentgatewaysubscriptions-plugin/plan_resolution_test.go
- ../sm-paymentgatewaysubscriptions-plugin/sync_subscription_test.go
- ../sm-paymentgatewaysubscriptions-plugin/checkout_guard_test.go
- ../sm-paymentgatewaysubscriptions-plugin/admin_sync_detached_test.go
- ../sm-paymentgatewaysubscriptions-plugin/README.md
- ../sm-paymentgatewaysubscriptions-plugin/go.mod
- ../vue-paymentgateway-layer/tests/subscribe.test.ts
- ../vue-paymentgateway-layer/tests/subscriptionComponents.test.ts
- ../vue-paymentgateway-layer/package.json
- ../vue-paymentgateway-layer/README.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md
- .planning/phases/18.1-payment-gateway-subscriptions/deferred-items.md
autonomous: false
gap_closure: true
requirements: [SC-5, D-22, CR-01, CR-02, WR-01, WR-02, WR-03, WR-04, WR-06, WR-07]
estimate:
tokens: 200000
raw_tokens: 200000
tasks: 5
confidence: low
must_haves:
truths:
- "SC-5 (project rule: unit tests are the last plan): every behaviour plans 07 and 08 changed has a named test in this plan across summercms.go (cabana), sm-paymentgatewaysubscriptions-plugin, sm-pgstripe-plugin and vue-paymentgateway-layer; go vet ./... and go test ./... are green in the three Go repositories and the layer's node --test run reports # fail 0."
- "CR-01 proof (TestEventOrdering, Postgres): each review sequence holds. Checkout race in either order and with two concurrent jobs: the row ends active, activated fires once and no stale snapshot regresses it. Deleted before created: no marker until the row exists, a canceled row, no activated, has_access false. A stale updated after a local cancel fires no reactivated. A failed Stripe retrieval leaves no marker and is retried. A duplicate or another application's event makes no Stripe call."
- "WR-04 and WR-06 proof: TestPlanResolution covers price, stripe_prices, product and plan_code (inactive plan included) and the unresolved error; TestApplyDeletedNotStored covers the sentinel; TestSyncSubscriptionFresh covers the retriever path (plan refreshed from the price), the legacy path (no plan change), gateway-owned columns only (a plan switch, subscriber and customer id written after the caller's load survive), no events, and a sync that waits for a held SubscriptionLockKey."
- "CR-02 proof: TestCheckoutPriceID (Docker-free) covers synced map with and without the currency, unsynced own and other currency, case and an empty map; TestGatewayStrictPrice covers no fallback, the yen refused and no Stripe call on refusal; TestCustomerAPIGuards covers the 422 at the API before any gateway call."
- "WR-02 and WR-03 proof: TestCustomerIDFirstWriteWins covers the first id kept, the stored id adopted, a missing user and that only stripe_customer_id changes. TestCheckoutGuarded covers each of the six open statuses refused without a gateway call, canceled and incomplete_expired allowed, two concurrent guarded checkouts of one subscriber never overlapping at the gateway, two subscribers not blocking each other, and the second request reusing the first's customer. TestGatewayIdempotencyKeys covers the Idempotency-Key header on the customer and session creates: equal within a window for equal parameters, different across windows (Gateway.Now) and for different metadata. TestCustomerAPIGuards covers 409 for past_due, unpaid, incomplete and paused with the existing check order."
- "WR-01 proof: TestDetachedBulkAction (framework, Postgres) shows a detached Run sees no transaction and holds no row lock (another connection's SELECT ... FOR UPDATE NOWAIT succeeds during Run), with the same empty, partial-409 and error rules and earlier writes kept, while a non-detached action is unchanged. TestAdminSyncDetached shows no gateway call of either sync action runs while the database reports the session idle in transaction, a failed save is a failure line, and ids are saved although the request context was cancelled."
- "WR-07 proof: subscribe.test.ts specs named 'subscriptions: ...' cover a persisted pageshow restoring both flows, a non-persisted pageshow changing nothing, watchPageRestore's remover and its server no-op, and a throwing navigation re-enabling the button with the generic error; the components' mount and unmount wiring is asserted, and the 409 copy test pins the new text."
- "Gate: scripts/check-phase18.1.sh requires every new test name in its stage and package (go, plugins, postgres, layers) and raises the layers minimum to the new count of passing 'subscriptions:' tests; --self-test passes; PHASE181_REQUIRE_POSTGRES=1 --all is green on a Docker host; 18.1-VALIDATION.md carries rows for every task of plans 07 to 09 and stays validated and nyquist_compliant."
- "D-22 release: after the gate is green, a final checkpoint lists every unpushed commit of the four repositories and the proposed tags. The framework tag v0.1.27 carries pact.AdminBulkAction.Detached, the layer gets v0.2.1 with package.json 0.2.1, and first plugin tags are an option. Nothing is pushed or tagged before the human's reply. After a framework tag, the plugin go.mod files require it in separate commits before their pushes, in plan 06's order: framework master and tag, require bumps, plugin pushes, layer."
artifacts:
- path: "../sm-pgstripe-plugin/subscriptions/ordering_test.go"
provides: "CR-01 regression suite over the three review sequences"
contains: "TestEventOrdering"
- path: "../sm-paymentgatewaysubscriptions-plugin/checkout_guard_test.go"
provides: "WR-02, WR-03 and API guard coverage"
contains: "TestCheckoutGuarded"
- path: "modules/cabana/bulk_detached_test.go"
provides: "framework detached bulk action coverage"
contains: "TestDetachedBulkAction"
- path: "scripts/check-phase18.1.sh"
provides: "fail-closed gate requiring the gap-closure tests"
contains: "TestEventOrdering"
- path: ".planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md"
provides: "validated map including plans 07 to 09"
contains: "18.1-09"
key_links:
- from: "scripts/check-phase18.1.sh"
to: "../sm-pgstripe-plugin/subscriptions/ordering_test.go"
via: "postgres stage required name TestEventOrdering in package ./subscriptions"
pattern: "TestEventOrdering"
- from: "scripts/check-phase18.1.sh"
to: "modules/cabana/bulk_detached_test.go"
via: "postgres stage required name TestDetachedBulkAction in ./modules/cabana"
pattern: "TestDetachedBulkAction"
- from: "../sm-paymentgatewaysubscriptions-plugin/go.mod"
to: "summercms.go tag v0.1.27"
via: "require git.golem15.com/golem15/summercms v0.1.27 after the framework tag (release task)"
pattern: "summercms v0.1.27"
prohibitions:
- statement: "No test calls the network or a real Stripe account; secrets in tests are test-only literals."
status: unresolved
verification: judgment
- statement: "The gate never reports green when a required test skipped, did not run, or Docker was absent without saying NOT RUN."
status: unresolved
verification: judgment
- statement: "Nothing is pushed or tagged without the human's reply at the final checkpoint, and nothing is force-pushed."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not in user-story format; no story is invented, as in plans 01 to 08): the generic core of `Golem15.PaymentGatewaySubscriptions` and pgstripe's subscription gateway and webhook run on SummerCMS, and a consuming application's subscription flow works as it does on PHP.
This plan's slice (gap closure 3 of 3, the last wave): SC-5 for the gap closure. It adds unit and Postgres tests for everything plans 07 and 08 changed (CR-01, CR-02, WR-01, WR-02, WR-03, WR-04, WR-06, WR-07), extends the fail-closed `scripts/check-phase18.1.sh`, fills the README Testing sections and the VALIDATION map, and ends with the human's release decision (D-22), mirroring plan 06.
## Decisions taken in this plan (Claude's discretion)
- New test files per finding group (ordering, gateway hardening, plan resolution, sync, checkout guard, detached sync, detached bulk action) rather than growing plan 06's files, so the gate names and the review findings map one to one.
- `TestDetachedBulkAction` is named so that it does not match plan 08's `^TestBulkAction` count of 13, and that count stays meaningful.
- The layer's component specs render on the server (no `onMounted`), so the mount and unmount wiring is asserted from the component sources and the behaviour through `watchPageRestore` and the flows.
- Release tags (proposed on the sheet, decided by the human):
- framework `v0.1.27`, needed because pgsub uses `pact.AdminBulkAction.Detached`;
- layer `v0.2.1` with `package.json` 0.2.1;
- first plugin tags `v0.1.0` on pgsub and pgstripe, an option the orchestrator asked for. No plugin tag exists yet. sm-paymentgateway-plugin is untagged too, so a pgsub tag alone still needs consumers' `replace` lines (IN-08, out of scope).
## Artifacts this phase produces (this plan)
- `modules/cabana/bulk_detached_test.go`: `TestDetachedBulkAction`.
- `../sm-pgstripe-plugin/subscriptions/ordering_test.go`: `TestEventOrdering` with subtests `checkout_race_updated_first`, `checkout_race_concurrent`, `deleted_before_created`, `stale_cancel_flag_no_reactivated`, `retrieve_error_retries`, `duplicate_makes_no_stripe_call`, `other_app_makes_no_stripe_call`.
- `../sm-pgstripe-plugin/subscriptions/gateway_hardening_test.go`: `TestGatewayStrictPrice`, `TestGatewayIdempotencyKeys`, `TestRetrieveSubscription`.
- `../sm-pgstripe-plugin/subscriptions/stripefake_test.go`: `fakeRequest` keeps the `Idempotency-Key` header; `setFailRetrieve(bool)` makes `GET /v1/subscriptions/{id}` answer 500.
- `../sm-pgstripe-plugin/subscriptions/decode_test.go`: `TestDecodeSubscription` cases for the price's product (string and expanded) and `plan_code`.
- `../sm-paymentgatewaysubscriptions-plugin/models/checkout_price_test.go`: `TestCheckoutPriceID`, `TestTerminalStatuses`.
- `../sm-paymentgatewaysubscriptions-plugin/plan_resolution_test.go`: `TestPlanResolution`, `TestApplyDeletedNotStored`.
- `../sm-paymentgatewaysubscriptions-plugin/sync_subscription_test.go`: `TestSyncSubscriptionFresh`.
- `../sm-paymentgatewaysubscriptions-plugin/checkout_guard_test.go`: `TestCheckoutGuarded`, `TestCustomerIDFirstWriteWins`, `TestCustomerAPIGuards`.
- `../sm-paymentgatewaysubscriptions-plugin/admin_sync_detached_test.go`: `TestAdminSyncDetached`.
- Layer specs (names start with "subscriptions:") in `tests/subscribe.test.ts` and `tests/subscriptionComponents.test.ts`; `package.json` version 0.2.1.
- `scripts/check-phase18.1.sh`: the new names in `PG_CABANA_TESTS`, `PGSUB_MODELS_TESTS`, `PG_PGSUB_TESTS`, `PG_PGSTRIPE_SUBS_TESTS`, `LAYER_REQUIRE`, and a raised `LAYER_PREFIX_MIN`.
- README Testing sections (pgsub, pgstripe subscriptions, cabana, layer); `18.1-VALIDATION.md` rows for plans 07 to 09; `deferred-items.md` entries; release commits (go.mod require bumps) and the release record.
<objective>
Prove every gap-closure fix with named tests, require them in the phase gate, record them in the VALIDATION map and the README Testing sections, and publish the result only on the human's reply.
Purpose: SC-5 and the project rule that unit tests are the last plan; the gate is what /msd-verify-work re-runs for the gap closure.
Output: test commits in four repositories, the extended gate, the updated VALIDATION map, the layer version, and the release record.
Execution budget: above the single-context smart zone by design, like plan 06. The project rule puts every gap-closure test in this last plan and the user fixed three plans. The mitigation is one task per repository group with a commit per task and repository, SUMMARY notes as you go, and resuming at the first uncommitted task after a reset.
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-06-PLAN.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-06-SUMMARY.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-07-SUMMARY.md
@.planning/phases/18.1-payment-gateway-subscriptions/18.1-08-SUMMARY.md
Repositories: summercms.go (gate, cabana test, VALIDATION), ../sm-paymentgatewaysubscriptions-plugin, ../sm-pgstripe-plugin, ../vue-paymentgateway-layer. Commit per repository and task; planning docs in their own commit; stage by explicit path; no co-author tags. Docker-backed suites skip under -short; run them on a Docker host (`sudo -u golem` if the socket is denied) and record NOT RUN only when no daemon answers. A test that exposes a production bug gets the fix in the owning repository as its own `fix(18.1-09): ...` commit before the test commit, and the SUMMARY lists it. Several sessions work in summercms.go: never stage or stash another session's files.
<interfaces>
- Gate: `scripts/check-phase18.1.sh` required lists at :60-101 (`CABANA_SHORT_TESTS`, `PGSUB_MODELS_TESTS`, `PGSUB_CONTRACTS_TESTS`, `PGSUB_ROOT_TESTS`, `PGSTRIPE_ROOT_TESTS`, `PGSTRIPE_SUBS_TESTS`, `PG_CABANA_TESTS`, `PG_PGSUB_TESTS`, `PG_PGSTRIPE_ROOT_TESTS`, `PG_PGSTRIPE_SUBS_TESTS`), layer names `LAYER_REQUIRE` :134-177, `LAYER_PREFIX_MIN=36` :180, detectors and `run_self_test` :195-500.
- pgstripe test harness (package subscriptions_test): `bootSubscriptionsHost`, `subEvent`, `deliver`, `work`, `job`, `activate`, `subscription`, `recorder`, `plan`, `count` (host_test.go); fake `setSubscription`, `calls(method, path)`, `all()`, `reset()`, `record` (stripefake_test.go); `eventBody`, `subscriptionObject`, `invoiceObject` (eventbody_test.go); `TestFreshStateSmoke` from plan 07.
- pgsub test harness (package root): `subEnv` with `plan`, `user`, `tx`, `txErr`, `count`, `capi` (harness_test.go, customer_api_smoke_test.go); the fixture `fakeGateway` (fixture_plugin_test.go); the open-transaction probe of `TestServiceAPI` (service_api_test.go, pg_stat_activity `idle in transaction`); `classes.New(classes.Options{...})` with `contracts.NewGateways()` builds a Service with a test gateway that does or does not implement `contracts.SubscriptionRetriever`.
- Framework bulk test harness: modules/cabana/phase121_bulk_test.go (the 13 `TestBulkAction*` tests and their fixture controller).
- Layer: `createSubscribeFlow`, `createManageFlow`, `watchPageRestore`, `restore()` (app/utils/subscribe.ts after plan 08); test helpers `renderSSR`, `loadComponent`, `stubComponent`, `fakeT` (testing/load.ts); TAP run `pnpm test`.
- Release: plan 06 Task 5 (push sheet, options, order, verify command) and 18.1-06-SUMMARY.md § Task 5 (what ran).
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: the CR-01 ordering suite runs end to end through the phase gate (TestEventOrdering required in the postgres stage, fail-closed)</name>
<reversibility rating="reversible">Tests and a test script only.</reversibility>
<files>../sm-pgstripe-plugin/subscriptions/ordering_test.go, ../sm-pgstripe-plugin/subscriptions/stripefake_test.go, scripts/check-phase18.1.sh</files>
<read_first>../sm-pgstripe-plugin/subscriptions/job.go and gateway.go (after plan 07), ../sm-pgstripe-plugin/subscriptions/fresh_state_smoke_test.go, ../sm-pgstripe-plugin/subscriptions/host_test.go (helpers), ../sm-pgstripe-plugin/subscriptions/job_test.go (concurrent subtest pattern), ../sm-pgstripe-plugin/subscriptions/stripefake_test.go, scripts/check-phase18.1.sh (lists :60-101, run_self_test), .planning/phases/18.1-payment-gateway-subscriptions/18.1-REVIEW.md (CR-01 sequences 1 to 3), .planning/phases/18.1-payment-gateway-subscriptions/18.1-07-SUMMARY.md</read_first>
<behavior>
- checkout_race_updated_first: the fake holds active; updated (active) is applied, then created (snapshot incomplete); the row is active and activated fired exactly once.
- checkout_race_concurrent: the fake holds active; created (snapshot incomplete) and updated (snapshot active) are worked by two goroutines at once (a retried unique violation is allowed); the row ends active, activated fired once, two markers.
- deleted_before_created: Work of the deletion returns an error wrapping classes.ErrSubscriptionNotStored and leaves no marker; created with the fake canceled stores canceled and fires no activated; the retried deletion succeeds and the row is canceled with has_access false.
- stale_cancel_flag_no_reactivated: after Service.Cancel at period end (the fake now holds cancel_at_period_end true), an older updated snapshot with cancel false fires no reactivated and the flag stays true.
- retrieve_error_retries: with the fake's retrieve failing (500), Work returns an error, no marker, the row is unchanged; after the failure is switched off the same args succeed.
- duplicate_makes_no_stripe_call / other_app_makes_no_stripe_call: no GET /v1/subscriptions call is recorded for a processed event or for an event whose app tag differs from metadata_app; the marker exists in both cases.
</behavior>
<action>Per SC-5, CR-01 and the D-21 amendment of plan 07.
- stripefake_test.go adds `setFailRetrieve(on bool)`, which makes GET /v1/subscriptions/{id} answer a 500 Stripe error while on; `reset()` clears it.
- Create subscriptions/ordering_test.go (package subscriptions_test) with `TestEventOrdering` and the seven subtests of the behaviour list. Use the host helpers. Build event bodies with `eventBody` and `subscriptionObject` where the snapshot must differ from the fake's state, and call `e.job(t).Work` directly where an error is expected. Each subtest comments the review sequence it pins.
- Gate: add `TestEventOrdering` and plan 07's `TestFreshStateSmoke` to `PG_PGSTRIPE_SUBS_TESTS`.
- Run the self-test and the postgres stage.
- Commit pgstripe (test) and summercms.go (gate) separately.</action>
<verify>
<automated>bash -n scripts/check-phase18.1.sh &amp;&amp; bash scripts/check-phase18.1.sh --self-test &amp;&amp; go -C ../sm-pgstripe-plugin vet ./... &amp;&amp; go -C ../sm-pgstripe-plugin test -count=1 -short ./...</automated>
<fails_when>non-zero exit: a syntax error, a detector that accepted a planted failing, skipping, empty or missing output, or vet or a Docker-free pgstripe test fails</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; bash scripts/check-phase18.1.sh postgres</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), or a required Postgres test (TestEventOrdering and TestFreshStateSmoke among them) failed, skipped or did not run</fails_when>
</verify>
<acceptance_criteria>
- `grep -c "t.Run(\"" ../sm-pgstripe-plugin/subscriptions/ordering_test.go` prints at least 7.
- `grep -n "TestEventOrdering\|TestFreshStateSmoke" scripts/check-phase18.1.sh` prints at least two lines.
- `grep -n "func (f \*stripeFake) setFailRetrieve" ../sm-pgstripe-plugin/subscriptions/stripefake_test.go` prints one line.
</acceptance_criteria>
<done>The three review sequences of CR-01 are pinned by a Postgres suite that the gate requires by name, so a regression to snapshot application turns the gate red.</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: pgsub coverage: checkout price, terminal statuses, plan resolution, deletion sentinel, fresh sync, guarded checkout, first-write-wins customer id, API guards and the detached admin sync</name>
<reversibility rating="reversible">Tests only (plus any bug fix they expose, committed separately).</reversibility>
<files>../sm-paymentgatewaysubscriptions-plugin/models/checkout_price_test.go, ../sm-paymentgatewaysubscriptions-plugin/plan_resolution_test.go, ../sm-paymentgatewaysubscriptions-plugin/sync_subscription_test.go, ../sm-paymentgatewaysubscriptions-plugin/checkout_guard_test.go, ../sm-paymentgatewaysubscriptions-plugin/admin_sync_detached_test.go, ../sm-paymentgatewaysubscriptions-plugin/README.md, scripts/check-phase18.1.sh</files>
<read_first>../sm-paymentgatewaysubscriptions-plugin/classes (apply.go, service.go, lock.go, user_subscribable.go after plans 07 and 08), ../sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go and subscription.go, ../sm-paymentgatewaysubscriptions-plugin/api/subscription.go, ../sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go, ../sm-paymentgatewaysubscriptions-plugin/harness_test.go, fixture_plugin_test.go, service_api_test.go (the open-transaction probe), admin_actions_test.go and admin_harness_test.go (how the admin actions are posted), customer_api_smoke_test.go, ../sm-paymentgatewaysubscriptions-plugin/README.md (Testing), .planning/phases/18.1-payment-gateway-subscriptions/18.1-08-SUMMARY.md</read_first>
<behavior>
- CheckoutPriceID: map {pln, usd} answers pln and usd ids and "" for eur even with stripe_price_id set; no map and plan currency USD answers stripe_price_id for usd and USD and "" for pln; an empty map behaves as no map; currency case and spaces are ignored.
- TerminalStatuses is exactly canceled and incomplete_expired; the other six statuses are open.
- Plan resolution: an unknown price with a matching stripe_product_id stores that plan; with neither but a plan_code of an inactive plan stores that plan; nothing matching on a new row is an error naming the price, product and code; an existing row keeps its plan.
- ApplyDeleted of an unknown id: errors.Is ErrSubscriptionNotStored, nil row, no event.
- SyncSubscription: a retriever gateway on another plan's price moves the row to that plan; a plan switch, customer id and subscriber written after the caller's load survive; only gateway-owned columns change; no event; the legacy gateway path changes status but not the plan; a held SubscriptionLockKey on another connection delays the sync until released.
- CheckoutGuarded: each open status is ErrAlreadySubscribed with zero gateway calls; canceled and incomplete_expired proceed; two concurrent guarded checkouts of one subscriber never have two gateway calls in flight (an atomic in-flight counter stays at most 1) and create the customer once; two subscribers proceed in parallel.
- SetCustomerID: a second write keeps the first id and the subscriber answers it; a missing user is ErrSubscriberNotFound; no other users column or updated_at changes.
- API guards: past_due, unpaid, incomplete and paused answer 409 Already subscribed; an open subscription plus an unknown plan still answers 409 (check order unchanged); a currency without a gateway price answers 422; no gateway call in any refusal.
- Admin sync: neither action makes a gateway call while pg_stat_activity shows the session idle in transaction; a save that fails (a test trigger raising on UPDATE of one plan) is a failure line and not synced; ids are saved when the request context is cancelled during the gateway call.
</behavior>
<action>Per SC-5, CR-01, CR-02, WR-01, WR-02, WR-03, WR-04 and WR-06 (D-08, D-09, D-13 unchanged): write the tests against the behaviour of plans 07 and 08, fix any production bug they expose in its own commit, then extend the gate.
- Docker-free: models/checkout_price_test.go holds `TestCheckoutPriceID` and `TestTerminalStatuses`.
- Postgres, root package, existing harness:
- plan_resolution_test.go: `TestPlanResolution` and `TestApplyDeletedNotStored`;
- sync_subscription_test.go: `TestSyncSubscriptionFresh`. Build Services with `classes.New` and a `contracts.NewGateways()` registry holding a test gateway with, and one without, `RetrieveSubscription`. Hold the lock with `classes.WithSessionLock` in a goroutine;
- checkout_guard_test.go: `TestCheckoutGuarded`, `TestCustomerIDFirstWriteWins` and `TestCustomerAPIGuards`, through the customer API helper;
- admin_sync_detached_test.go: `TestAdminSyncDetached`. Reuse the open-transaction probe pattern of TestServiceAPI in a probe gateway. The save failure comes from a test-created trigger dropped in cleanup.
- Each test comments the review finding it pins.
- Gate: `PGSUB_MODELS_TESTS` gains TestCheckoutPriceID and TestTerminalStatuses; `PG_PGSUB_TESTS` gains the seven Postgres names.
- README (pgsub) Testing lists every new name.
- Commit pgsub (tests and README) and summercms.go (gate) separately.</action>
<verify>
<automated>go -C ../sm-paymentgatewaysubscriptions-plugin vet ./... &amp;&amp; go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -short ./... &amp;&amp; bash scripts/check-phase18.1.sh plugins &amp;&amp; bash scripts/check-phase18.1.sh --self-test</automated>
<fails_when>non-zero exit: vet or a Docker-free pgsub test fails, or the plugins stage reports a required name failed, skipped or missing</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; OUT=$(go -C ../sm-paymentgatewaysubscriptions-plugin test -count=1 -run '^(TestPlanResolution|TestApplyDeletedNotStored|TestSyncSubscriptionFresh|TestCheckoutGuarded|TestCustomerIDFirstWriteWins|TestCustomerAPIGuards|TestAdminSyncDetached)$' -v . 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -c '^--- PASS: Test' | grep -qx 7</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), or fewer than seven pgsub top-level PASS lines (a resolution step, the sentinel, a sync column, a guard, the first write, an API refusal or a detached sync property failed)</fails_when>
</verify>
<acceptance_criteria>
- `grep -c "func Test" ../sm-paymentgatewaysubscriptions-plugin/plan_resolution_test.go ../sm-paymentgatewaysubscriptions-plugin/sync_subscription_test.go ../sm-paymentgatewaysubscriptions-plugin/checkout_guard_test.go ../sm-paymentgatewaysubscriptions-plugin/admin_sync_detached_test.go ../sm-paymentgatewaysubscriptions-plugin/models/checkout_price_test.go` reports at least 1 per file.
- `grep -n "TestCheckoutGuarded\|TestSyncSubscriptionFresh\|TestCheckoutPriceID" scripts/check-phase18.1.sh` prints at least three lines.
- `grep -n "TestAdminSyncDetached" ../sm-paymentgatewaysubscriptions-plugin/README.md` prints at least one line.
- `grep -n "idle in transaction" ../sm-paymentgatewaysubscriptions-plugin/admin_sync_detached_test.go` prints at least one line.
</acceptance_criteria>
<done>Every pgsub behaviour plans 07 and 08 changed has a test that fails when it regresses, and the gate requires them by name.</done>
</task>
<task type="auto" tdd="true">
<name>Task 3: pgstripe and framework coverage: strict gateway price, idempotency keys, subscription retrieval and decoding, and the framework detached bulk action</name>
<reversibility rating="reversible">Tests and a fake only (plus any bug fix they expose, committed separately).</reversibility>
<files>../sm-pgstripe-plugin/subscriptions/gateway_hardening_test.go, ../sm-pgstripe-plugin/subscriptions/stripefake_test.go, ../sm-pgstripe-plugin/subscriptions/decode_test.go, ../sm-pgstripe-plugin/subscriptions/README.md, modules/cabana/bulk_detached_test.go, modules/cabana/README.md, scripts/check-phase18.1.sh</files>
<read_first>../sm-pgstripe-plugin/subscriptions/gateway.go (after plans 07 and 08), ../sm-pgstripe-plugin/subscriptions/gateway_test.go (harness and settings setup), ../sm-pgstripe-plugin/subscriptions/stripefake_test.go (record, fakeRequest), ../sm-pgstripe-plugin/subscriptions/decode_test.go, ../sm-pgstripe-plugin/subscriptions/README.md (Testing), modules/cabana/crud.go (BulkAction after plan 08), modules/cabana/phase121_bulk_test.go (fixture controller and helpers), modules/cabana/README.md (Testing), .planning/phases/18.1-payment-gateway-subscriptions/18.1-08-SUMMARY.md</read_first>
<behavior>
- Strict price: a plan whose map lacks the currency gets ErrNoPriceForCurrency and the fake records no call although stripe_price_id is set; an unsynced plan checks out in its own currency on stripe_price_id and is refused in another; SupportsCurrency refuses the yen code and accepts usd, eur, pln, gbp, cad, aud and chf.
- Idempotency keys: POST /v1/customers and POST /v1/checkout/sessions carry an Idempotency-Key header starting "pgsub-customer-" and "pgsub-checkout-"; two calls with equal parameters and Gateway.Now in one window send the same key; Now plus 10 minutes sends another; different checkout metadata sends another.
- Retrieval: RetrieveSubscription maps status, both period shapes, price, product (string or expanded), plan_code and the subscriber metadata; a 404 wraps ErrSubscriptionMissing; a 500 is another error.
- Decode: TestDecodeSubscription adds the product id and plan_code mappings; job arguments still hold no e-mail, name or address.
- Framework: a Detached action's Run sees cabana.TxFromContext report false; during Run another connection's SELECT ... FOR UPDATE NOWAIT on a selected row succeeds; empty selection gives affected 0 without Run; a partial selection is 409; a cabana.ValidationError from Run is 422; a row Run wrote before returning that error stays written; a non-detached action of the same fixture still sees a transaction and holds the lock.
</behavior>
<action>Per SC-5, CR-01, CR-02, WR-01, WR-02, WR-03 and WR-04: tests first, bug fixes in their own commits, then the gate.
- stripefake_test.go keeps the request's Idempotency-Key header in `fakeRequest`.
- Create subscriptions/gateway_hardening_test.go with `TestGatewayStrictPrice`, `TestGatewayIdempotencyKeys` (set `Gateway.Now` on a gateway built with `subscriptions.NewGateway`) and `TestRetrieveSubscription`, all Postgres for the settings. Extend decode_test.go `TestDecodeSubscription`.
- Framework: create modules/cabana/bulk_detached_test.go with `TestDetachedBulkAction` (Postgres) on the phase121 fixture pattern. Add a detached action to a test-only controller, so no production fixture changes. The test names no application or provider.
- READMEs Testing:
- pgstripe subscriptions: the three new names and TestEventOrdering;
- cabana: TestDetachedBulkAction.
- Gate:
- `PG_PGSTRIPE_SUBS_TESTS` gains TestGatewayStrictPrice, TestGatewayIdempotencyKeys and TestRetrieveSubscription;
- `PG_CABANA_TESTS` gains TestDetachedBulkAction.
- Commit framework test and README, pgstripe tests and README, and the gate separately.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/cabana &amp;&amp; go -C ../sm-pgstripe-plugin vet ./... &amp;&amp; go -C ../sm-pgstripe-plugin test -count=1 -short ./... &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; go run ./cmd/summer docs:build --check &amp;&amp; bash scripts/check-phase18.1.sh go &amp;&amp; bash scripts/check-phase18.1.sh plugins</automated>
<fails_when>non-zero exit: vet, a Docker-free cabana or pgstripe test (TestDecodeSubscription among them), the docs checkers on the README changes, or the go or plugins stage of the gate</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; OUT=$(go test -count=1 -run '^TestDetachedBulkAction$' -v ./modules/cabana 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -q '^--- PASS: TestDetachedBulkAction' &amp;&amp; OUT2=$(go -C ../sm-pgstripe-plugin test -count=1 -run '^(TestGatewayStrictPrice|TestGatewayIdempotencyKeys|TestRetrieveSubscription)$' -v ./subscriptions 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT2" | grep -c '^--- PASS: Test' | grep -qx 3</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN), the framework detached test did not pass, or fewer than three pgstripe top-level PASS lines</fails_when>
</verify>
<acceptance_criteria>
- `grep -n "Idempotency-Key" ../sm-pgstripe-plugin/subscriptions/stripefake_test.go ../sm-pgstripe-plugin/subscriptions/gateway_hardening_test.go` prints at least two lines.
- `grep -n "FOR UPDATE NOWAIT" modules/cabana/bulk_detached_test.go` prints at least one line.
- `grep -rniE "stripe|subscription|horoskopia|queststream|quotify" modules/cabana/bulk_detached_test.go` prints nothing.
- `grep -n "TestDetachedBulkAction\|TestGatewayStrictPrice\|TestRetrieveSubscription" scripts/check-phase18.1.sh` prints at least three lines.
</acceptance_criteria>
<done>The Stripe half of the gap closure and the framework addition are covered by named tests the gate requires.</done>
</task>
<task type="auto" tdd="true">
<name>Task 4: Layer coverage and version 0.2.1, the complete gate run with Postgres required, the VALIDATION map and the deferred items</name>
<reversibility rating="reversible">Tests, a version string and planning documents (plus any bug fix they expose, committed separately).</reversibility>
<files>../vue-paymentgateway-layer/tests/subscribe.test.ts, ../vue-paymentgateway-layer/tests/subscriptionComponents.test.ts, ../vue-paymentgateway-layer/package.json, ../vue-paymentgateway-layer/README.md, scripts/check-phase18.1.sh, .planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md, .planning/phases/18.1-payment-gateway-subscriptions/deferred-items.md</files>
<read_first>../vue-paymentgateway-layer/app/utils/subscribe.ts and the three Pg subscription components (after plan 08), ../vue-paymentgateway-layer/tests/subscribe.test.ts and tests/subscriptionComponents.test.ts (style, names), ../vue-paymentgateway-layer/testing/load.ts, ../vue-paymentgateway-layer/package.json, ../vue-paymentgateway-layer/README.md (Testing), scripts/check-phase18.1.sh (LAYER_REQUIRE, LAYER_PREFIX_MIN), .planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md, .planning/phases/18.1-payment-gateway-subscriptions/deferred-items.md, the SUMMARYs of plans 07 and 08</read_first>
<behavior>
- "subscriptions: a persisted pageshow restores the subscribe flow after a redirect": after subscribe assigned the URL (redirecting true), a stub target dispatching pageshow with persisted true leaves busy and redirecting false, and a second subscribe sends a request.
- "subscriptions: a non-persisted pageshow changes nothing": redirecting stays true.
- "subscriptions: watchPageRestore removes its listener and is a no-op without a target".
- "subscriptions: a throwing navigation re-enables the button with the generic error".
- "subscriptions: the manage flow restores after a persisted pageshow and cancel works again".
- "subscriptions: the three components wire watchPageRestore on mount and remove it on unmount" (from the component sources, since specs render on the server).
- "subscriptions: the already-subscribed copy covers any open subscription in en and pl".
</behavior>
<action>Per SC-5, WR-07 and the WR-03 copy (D-14):
- Write the seven specs of the behaviour list. Use a stub `PageRestoreTarget` whose dispatch calls the registered listener with an event object carrying `persisted`.
- Set the layer `package.json` version to 0.2.1 in its own commit (no dependency change; pnpm-lock.yaml unchanged).
- The layer README Testing section lists the new specs. A short "Changes in 0.2.1" note describes the back/forward-cache restore and the copy.
- Gate: add the seven names to `LAYER_REQUIRE` and raise `LAYER_PREFIX_MIN` to the number of passing "subscriptions:" tests the run reports.
- Run `PHASE181_REQUIRE_POSTGRES=1 bash scripts/check-phase18.1.sh --all` on a Docker host.
- VALIDATION: add one row per task of plans 07, 08 and 09 (requirement or finding ID, behaviour, type, automated command, file exists, status green or NOT RUN with the reason) and keep `status: validated`, `nyquist_compliant: true` and `wave_0_complete: true`. The manual-only rows stay: live test-mode checkout (now also watching that created and updated arrive in either order without a regression), the portal cancel_at check, and the browser pass (now including Back from Checkout).
- deferred-items.md gains one entry "Review findings outside the gap closure": WR-05 (PHP-identical grace, kept per the user's scope) and IN-01 to IN-08, status open, with the review as the source. The check-git-install.sh item stays open.
- Commit the layer, then the gate, then the planning docs.</action>
<verify>
<automated>OUT=$(pnpm --dir ../vue-paymentgateway-layer test 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -qE '^# fail 0$' &amp;&amp; bash scripts/check-phase18.1.sh layers &amp;&amp; bash scripts/check-phase18.1.sh --all &amp;&amp; grep -q "nyquist_compliant: true" .planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md &amp;&amp; grep -q "18.1-09 T5" .planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md</automated>
<fails_when>non-zero exit: a layer spec failed ("# fail 0" absent), the layers stage counted fewer passing "subscriptions:" tests than the raised minimum or missed a required name, any gate stage is red, or the VALIDATION map lacks the plan 09 rows or the Nyquist flag</fails_when>
<automated>docker info &gt;/dev/null 2&gt;&amp;1 &amp;&amp; PHASE181_REQUIRE_POSTGRES=1 bash scripts/check-phase18.1.sh --all</automated>
<fails_when>non-zero exit: no Docker daemon (record NOT RUN and leave the affected VALIDATION rows NOT RUN), or a required Postgres test in any repository failed, skipped or did not run</fails_when>
</verify>
<acceptance_criteria>
- `cat ../vue-paymentgateway-layer/tests/subscribe.test.ts ../vue-paymentgateway-layer/tests/subscriptionComponents.test.ts | grep -c "test('subscriptions: "` is at least 7 higher than before this task (the SUMMARY records both counts).
- `grep -n "a persisted pageshow restores the subscribe flow after a redirect" ../vue-paymentgateway-layer/tests/subscribe.test.ts` prints one line.
- `grep -n '"version": "0.2.1"' ../vue-paymentgateway-layer/package.json` prints one line and `git -C ../vue-paymentgateway-layer diff v0.2.0 --stat -- pnpm-lock.yaml` prints nothing.
- `grep -n "LAYER_PREFIX_MIN=" scripts/check-phase18.1.sh` prints a value of at least 43.
- `grep -n "18.1-07 T1\|18.1-08 T1\|18.1-09 T1" .planning/phases/18.1-payment-gateway-subscriptions/18.1-VALIDATION.md` prints three lines.
- `grep -n "WR-05" .planning/phases/18.1-payment-gateway-subscriptions/deferred-items.md` prints at least one line.
</acceptance_criteria>
<done>The layer fix is pinned by specs, the whole gate is green across four repositories with Postgres required, and the VALIDATION map covers the gap closure.</done>
</task>
<task type="checkpoint:decision" gate="blocking-human">
<name>Task 5: Final push approval: unpushed commits of the four repositories, framework tag v0.1.27 and the plugin require bumps, the layer tag v0.2.1, optional first plugin tags</name>
<read_first>.planning/phases/18.1-payment-gateway-subscriptions/18.1-06-PLAN.md (Task 5) and 18.1-06-SUMMARY.md (§ Task 5: order, commands, checks), the SUMMARYs of plans 07 and 08 and this plan's Task 4 notes (gate result, NOT RUN suites), deferred-items.md (check-git-install.sh)</read_first>
<action>Mirror plan 06 Task 5 (D-22): do everything automatable, print the push sheet, then stop.
Per repository (summercms.go, ../sm-paymentgatewaysubscriptions-plugin, ../sm-pgstripe-plugin, ../vue-paymentgateway-layer):
- the tree is clean for this phase's files (never stage another session's files);
- run `git fetch origin --tags`;
- origin has no commits the checkout lacks (otherwise stop and report);
- list the commits that would be pushed. For summercms.go this includes any other session's commits on master, named as such.
The proposed tags:
- framework `v0.1.27` on the commit after this plan's last docs commit; it contains `pact.AdminBulkAction.Detached`;
- layer `v0.2.1` on the commit with package.json 0.2.1;
- optional first plugin tags `v0.1.0` on pgsub and pgstripe. No earlier tag exists; sm-paymentgateway-plugin stays untagged, so consumers still need its replace.
The order (plan 06's), which keeps consumers resolvable:
1. framework master and the v0.1.27 tag;
2. in pgsub and in pgstripe, a separate commit raising `require git.golem15.com/golem15/summercms` to v0.1.27 (replace lines unchanged; `go mod tidy`, vet and -short green);
3. with plugin tags: pgsub main and its tag first, then a separate pgstripe commit raising its pgsub require to v0.1.0, then pgstripe main and its tag; without them, pgsub main then pgstripe main;
4. the layer main and v0.2.1.
Record the gate result, the NOT RUN suites, and the open check-git-install.sh item: an install check of v0.2.1 needs that script fixed first.
No push, tag or require bump before the reply. After the reply:
- "push-all": the order without plugin tags;
- "push-all-with-plugin-tags": the order with plugin tags;
- "push-no-tags": push branches only (no tags, no require bumps);
- "hold": nothing.
Never force-push. Record the reply verbatim, every pushed hash and every tag.</action>
<decision>Publish the gap closure now?</decision>
<context>Tags are read by the Go module proxy and by applications' pinned layer dependency and cannot be moved. The plugins use pact.AdminBulkAction.Detached, so a consumer building pgsub from git without the replace needs framework v0.1.27 (D-22: no push, tag or deploy without the go-ahead).</context>
<options>
<option id="push-all">
<name>Push all four repositories; tag framework v0.1.27 and layer v0.2.1; bump the plugin summercms requires (plan 06 parity, no plugin tags)</name>
<pros>The gap closure is published and consumable exactly as the phase was.</pros>
<cons>Plugins still build from git only through replace or pseudo-versions.</cons>
</option>
<option id="push-all-with-plugin-tags">
<name>As push-all, plus first tags v0.1.0 on pgsub and pgstripe with pgstripe requiring pgsub v0.1.0</name>
<pros>Applications can pin the subscription plugins by tag.</pros>
<cons>sm-paymentgateway-plugin is untagged, so a replace is still needed for it; first plugin tags are permanent.</cons>
</option>
<option id="push-no-tags">
<name>Push the branches, keep tags for later</name>
<pros>Commits are backed up and reviewable.</pros>
<cons>Without v0.1.27 the pushed plugins need the local framework replace.</cons>
</option>
<option id="hold">
<name>Hold everything</name>
<pros>Nothing leaves the machine.</pros>
<cons>No backup; the cutover waits.</cons>
</option>
</options>
<resume-signal>Reply "push-all", "push-all-with-plugin-tags", "push-no-tags" or "hold".</resume-signal>
<verify>
<automated>for r in . ../sm-paymentgatewaysubscriptions-plugin ../sm-pgstripe-plugin ../vue-paymentgateway-layer; do ST=$(git -C "$r" status --porcelain -- . ':!.planning' ':!.msd') || exit 1; test -z "$ST" || exit 1; done; bash scripts/check-phase18.1.sh --all &amp;&amp; echo final-push-sheet-ok</automated>
<fails_when>non-zero exit or "final-push-sheet-ok" absent: a repository has uncommitted non-planning changes, or the gate is red. Changes of this phase are committed and a red gate is fixed before the sheet is presented; another session's uncommitted files stop the checkpoint with a report (never staged or stashed).</fails_when>
</verify>
<acceptance_criteria>
- The SUMMARY records the sheet, the reply verbatim and, after a push reply, for every repository `git rev-parse main` (master for summercms.go) equal to the origin branch.
- After "push-all" or "push-all-with-plugin-tags": `git ls-remote --tags origin v0.1.27` in summercms.go and `git -C ../vue-paymentgateway-layer ls-remote --tags origin v0.2.1` each print one line, and `grep -n "summercms v0.1.27" ../sm-paymentgatewaysubscriptions-plugin/go.mod ../sm-pgstripe-plugin/go.mod` prints two lines.
- After "push-all-with-plugin-tags": `git -C ../sm-paymentgatewaysubscriptions-plugin ls-remote --tags origin v0.1.0` and the same for pgstripe each print one line, and `grep -n "sm-paymentgatewaysubscriptions-plugin v0.1.0" ../sm-pgstripe-plugin/go.mod` prints one line.
- After "hold": no `git ls-remote` shows a new tag and the SUMMARY lists every unpushed commit.
</acceptance_criteria>
<done>The human decided how the gap closure is published, and the record says exactly what was pushed and tagged.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| gate script to the developer | the gate's verdict decides whether re-verification starts |
| tests to external services | tests run against fakes and testcontainers only |
| local checkouts to remotes | the final push publishes code and immovable tags |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-18.1-48 | Repudiation | check-phase18.1.sh false green on the gap-closure tests | high | mitigate | every new test is required by name per stage and package; the detectors fail on skip, zero tests, missing names and too few "subscriptions:" passes; --self-test proves them; NOT RUN postgres is fatal with PHASE181_REQUIRE_POSTGRES=1 |
| T-18.1-49 | Information Disclosure | tests calling Stripe or leaking secrets | medium | mitigate | every Stripe call goes to the package fake through BackendOverride; idempotency keys and secrets in tests are test-only values and never printed |
| T-18.1-50 | Tampering | publishing red or unreviewed code; tags that cannot be moved; a plugin require naming a tag that does not exist | medium | mitigate | the final checkpoint runs the full gate first, pushes only after the human's reply, in plan 06's order (framework tag before the require bumps), and never force-pushes |
| T-18.1-SC | Tampering | npm/pip/cargo installs | high | mitigate | no package is installed; the layer version bump changes no dependency and pnpm-lock.yaml stays unchanged |
</threat_model>
<verification>
- Task 1: self-test, Docker-free pgstripe suite; the postgres stage on a Docker host with TestEventOrdering and TestFreshStateSmoke required.
- Task 2: pgsub Docker-free suite, the plugins stage and the self-test; the seven pgsub Postgres names on a Docker host.
- Task 3: framework and pgstripe Docker-free suites, docs checks, the go and plugins stages; TestDetachedBulkAction and the three pgstripe names on a Docker host.
- Task 4: the layer suite, the layers stage, `--all`, the VALIDATION frontmatter; `PHASE181_REQUIRE_POSTGRES=1 --all` on a Docker host.
- Task 5: clean trees and a green gate before the human decides the publication.
</verification>
<success_criteria>
- SC-5 for the gap closure: every fix of plans 07 and 08 has a named test in this last plan; `scripts/check-phase18.1.sh --all` is green with Postgres required (or recorded NOT RUN).
- 18.1-VALIDATION.md covers plans 07 to 09 and stays validated and Nyquist-compliant.
- The human decided the release (D-22) and the record lists every push and tag.
</success_criteria>
<output>
Create `.planning/phases/18.1-payment-gateway-subscriptions/18.1-09-SUMMARY.md` when done, with the commits per repository, every test added, any bug fixed, the gate output summary, the VALIDATION status, the push reply and every pushed hash and tag.
</output>

View File

@@ -0,0 +1,434 @@
---
phase: 18.1-payment-gateway-subscriptions
reviewed: 2026-10-14T19:22:09Z
depth: standard
files_reviewed: 96
files_reviewed_list:
- summercms.go/admin/src/components/form/PreviewField.vue
- summercms.go/admin/src/components/form/fields/TaglistField.vue
- summercms.go/admin/src/components/form/registry.ts
- summercms.go/admin/src/components/reorder/ReorderBranch.vue
- summercms.go/admin/src/components/reorder/tree.ts
- summercms.go/admin/src/views/ReorderView.vue
- summercms.go/modules/cabana/admin_openapi.go
- summercms.go/modules/cabana/crud.go
- summercms.go/modules/cabana/field_file.go
- summercms.go/modules/cabana/field_taglist.go
- summercms.go/modules/cabana/form_schema.go
- summercms.go/modules/cabana/list_schema.go
- summercms.go/modules/cabana/registry.go
- summercms.go/modules/cabana/relation_child.go
- summercms.go/modules/cabana/relation_form.go
- summercms.go/modules/cabana/reorder.go
- summercms.go/modules/cabana/schema_types.go
- summercms.go/modules/cabana/settings.go
- summercms.go/scripts/check-phase18.1.sh
- sm-paymentgatewaysubscriptions-plugin/.gitignore
- sm-paymentgatewaysubscriptions-plugin/admin.go
- sm-paymentgatewaysubscriptions-plugin/admin_navigation.go
- sm-paymentgatewaysubscriptions-plugin/admin_permissions.go
- sm-paymentgatewaysubscriptions-plugin/api/errors.go
- sm-paymentgatewaysubscriptions-plugin/api/json.go
- sm-paymentgatewaysubscriptions-plugin/api/plans.go
- sm-paymentgatewaysubscriptions-plugin/api/subscription.go
- sm-paymentgatewaysubscriptions-plugin/api/urls.go
- sm-paymentgatewaysubscriptions-plugin/assets/js/fields.js
- sm-paymentgatewaysubscriptions-plugin/boot.go
- sm-paymentgatewaysubscriptions-plugin/classes/apply.go
- sm-paymentgatewaysubscriptions-plugin/classes/service.go
- sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go
- sm-paymentgatewaysubscriptions-plugin/config/config.yaml
- sm-paymentgatewaysubscriptions-plugin/contracts/gateway.go
- sm-paymentgatewaysubscriptions-plugin/contracts/registry.go
- sm-paymentgatewaysubscriptions-plugin/contracts/subscriber.go
- sm-paymentgatewaysubscriptions-plugin/controllers/admin_registry.go
- sm-paymentgatewaysubscriptions-plugin/controllers/base.go
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans/config_form.yaml
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans/config_list.yaml
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans/config_reorder.yaml
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptions.go
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptions/_subscription_plan.htm
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptions/_subscription_subscriber.htm
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptions/config_filter.yaml
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptions/config_form.yaml
- sm-paymentgatewaysubscriptions-plugin/controllers/subscriptions/config_list.yaml
- sm-paymentgatewaysubscriptions-plugin/events/events.go
- sm-paymentgatewaysubscriptions-plugin/field_types.go
- sm-paymentgatewaysubscriptions-plugin/go.mod
- sm-paymentgatewaysubscriptions-plugin/go.work
- sm-paymentgatewaysubscriptions-plugin/models/feature_list.go
- sm-paymentgatewaysubscriptions-plugin/models/filters.go
- sm-paymentgatewaysubscriptions-plugin/models/processed_webhook_event.go
- sm-paymentgatewaysubscriptions-plugin/models/registry.go
- sm-paymentgatewaysubscriptions-plugin/models/subscription.go
- sm-paymentgatewaysubscriptions-plugin/models/subscription/columns.yaml
- sm-paymentgatewaysubscriptions-plugin/models/subscription/fields.yaml
- sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go
- sm-paymentgatewaysubscriptions-plugin/models/subscriptionplan/columns.yaml
- sm-paymentgatewaysubscriptions-plugin/models/subscriptionplan/fields.yaml
- sm-paymentgatewaysubscriptions-plugin/plugin.go
- sm-paymentgatewaysubscriptions-plugin/routes.go
- sm-paymentgatewaysubscriptions-plugin/updates/202610150001_create_golem15_pgsub_baseline.go
- sm-paymentgatewaysubscriptions-plugin/updates/202610150002_add_stripe_customer_id_to_users.go
- sm-paymentgatewaysubscriptions-plugin/updates/registry.go
- sm-pgstripe-plugin/client.go
- sm-pgstripe-plugin/config/config.yaml
- sm-pgstripe-plugin/console/import_winter.go
- sm-pgstripe-plugin/go.mod
- sm-pgstripe-plugin/go.work
- sm-pgstripe-plugin/models/settings.go
- sm-pgstripe-plugin/models/settings/fields.yaml
- sm-pgstripe-plugin/subscriptions/config/config.yaml
- sm-pgstripe-plugin/subscriptions/decode.go
- sm-pgstripe-plugin/subscriptions/gateway.go
- sm-pgstripe-plugin/subscriptions/job.go
- sm-pgstripe-plugin/subscriptions/plugin.go
- sm-pgstripe-plugin/subscriptions/sync_plan.go
- sm-pgstripe-plugin/subscriptions/webhook.go
- sm-pgstripe-plugin/updates/202610150001_add_subscription_webhook_secrets.go
- vue-paymentgateway-layer/.playground/app/pages/subscriptions/index.vue
- vue-paymentgateway-layer/app/assets/css/pg-base.css
- vue-paymentgateway-layer/app/components/Pg/ManageSubscriptionButton.vue
- vue-paymentgateway-layer/app/components/Pg/MySubscription.vue
- vue-paymentgateway-layer/app/components/Pg/Plans.vue
- vue-paymentgateway-layer/app/components/Pg/SubscribeButton.vue
- vue-paymentgateway-layer/app/composables/usePlans.ts
- vue-paymentgateway-layer/app/composables/useSubscription.ts
- vue-paymentgateway-layer/app/composables/useSubscriptionsApi.ts
- vue-paymentgateway-layer/app/utils/subscribe.ts
- vue-paymentgateway-layer/app/utils/subscriptionsClient.ts
- vue-paymentgateway-layer/nuxt.config.ts
- vue-paymentgateway-layer/package.json
findings:
critical: 2
warning: 7
info: 8
total: 17
status: issues_found
---
# Phase 18.1: Code Review Report
**Reviewed:** 2026-10-14T19:22:09Z
**Depth:** standard
**Files Reviewed:** 96 (four repositories; every path below carries its repository prefix)
**Status:** issues_found
## Summary
The review covered the cabana taglist field and flat reorder (framework and admin SPA), the pgsub core, its admin screens and customer API, the Stripe subscription gateway with its webhook, River job and plan sync, and the Nuxt layer pieces.
The framework half is in good shape. The taglist save path normalises and checks values at every save site before any transaction. The flat reorder writes only the sort column, and a submitted parent is refused. The webhook verifies the signature on the raw body before decoding it, uses a separate secret per endpoint, and claims the processed-event marker inside the transaction that applies the event. The customer API never takes a subscriber id, a subscription id or a redirect URL from the request.
Two money-relevant defects block shipping:
1. **CR-01: stale webhook data can overwrite newer data.** The webhook job applies the subscription snapshot carried in each Stripe event, with nothing to tell an old event from a new one. Stripe delivers events out of order, and River runs and retries jobs concurrently. A paid subscription can drop back to `incomplete` and lose access. A deleted subscription can come back as active.
2. **CR-02: checkout can charge a different currency or amount than the user was shown.** The customer API checks the currency against the plan's local prices. The gateway then silently falls back to `stripe_price_id`, which plan sync sets to the USD price.
The warnings cover:
- Stripe calls made while holding a database transaction and row locks.
- Duplicate Stripe customers and duplicate subscriptions under concurrent or repeated checkout.
- Subscriptions that are dropped after a plan is repriced.
- A grace period inherited from PHP that grants about a full unpaid period.
- A lost-update risk in SyncSubscription.
- Subscribe and manage buttons in the layer that stay disabled after the browser Back button.
Byte compatibility: the webhook's five plain-text answers, their order, the event names, the table, column and constraint names and the permission codes match what the SUMMARYs claim. The documented exception is `text/plain` instead of Laravel's `text/html`. CR-01, CR-02 (the usd/eur/pln loop and its fallback) and WR-05 are defects inherited from the frozen PHP source. They are reported anyway, because the Go port adds concurrent workers and a new customer API that make them easier to reach.
## Narrative Findings (AI reviewer)
## Critical Issues
### CR-01: Webhook job applies stale event snapshots with no ordering guard, so a newer subscription state can be overwritten by an older event
**Files:**
- `sm-pgstripe-plugin/subscriptions/job.go:119-147, 155-170, 187-205`
- `sm-paymentgatewaysubscriptions-plugin/classes/apply.go:28-126` (lookup and lock at 52, unconditional status write at 96, reactivated diff at 138), `apply.go:170-205` (unknown row at 183)
- `sm-paymentgatewaysubscriptions-plugin/classes/service.go:308-330`
**Issue:** `work` maps `data.object` from the Stripe event into `contracts.Update`, and `Apply` writes that status, period and cancel flag onto the row as they are. Neither the event's `created` timestamp nor any version is decoded (the `event` struct in `decode.go:30-36` has only id, type and data), so nothing stops an older event from winning. Stripe does not guarantee delivery order, and River runs several workers and retries failed jobs with backoff. Several realistic sequences break:
1. **Checkout race on a new subscription.** Stripe Checkout emits `customer.subscription.created` (status `incomplete`) and `customer.subscription.updated` (status `active`) within milliseconds.
- Both jobs run `SELECT ... FOR UPDATE`, which finds no row and so locks nothing, and both INSERT.
- If the `updated` job wins the insert, it stores `active` and fires `activated`. The `created` job fails on the unique index on `stripe_subscription_id`, rolls back its marker and is retried.
- On retry it finds the row and writes `incomplete` over `active`. No event fires.
- The customer has paid but `HasAccess` is false until Stripe's next update, typically the next renewal a month later.
2. **Deleted before created.** If the `created` job is retrying (for example "no plan holds price id" until an admin syncs the plan) and `customer.subscription.deleted` arrives:
- `ApplyDeleted` finds no row, logs a warning and returns `nil, nil`, so the deletion marker is committed.
- The `created` retry then inserts the row as `active`.
- The canceled subscription grants access with no end, and `activated` fires.
3. **Spurious `reactivated`.** `Service.Cancel` writes `cancel_at_period_end = true` locally (`service.go:324-325`). An older queued `customer.subscription.updated` carrying `cancel_at_period_end: false` then meets `previousCancel && !sub.CancelAtPeriodEnd && active` (`apply.go:138`) and fires `SubscriptionReactivated` for a cancellation the user never withdrew.
PHP's job had the same snapshot semantics. The Go port adds concurrent workers and retry-with-backoff, which makes the race in sequence 1 likely on every Checkout rather than rare.
**Fix:** Apply authoritative state rather than the snapshot, as Stripe recommends. In the job, outside the transaction, retrieve the subscription from Stripe (the gateway already has the code in `SyncSubscription`) and feed that into `Apply`, keeping the claim-first transaction. Alternatively, decode `event.created` and refuse any update older than the last one applied. That needs a stored per-subscription marker, such as a column or a side table keyed by `stripe_subscription_id`. Also treat an unknown row in `ApplyDeleted` as retryable instead of committing the marker:
```go
// decode.go
type event struct {
ID string `json:"id"`
Type string `json:"type"`
Created int64 `json:"created"`
Data struct{ Object json.RawMessage `json:"object"` } `json:"data"`
}
// job.go: route() for created/updated
fresh, err := p.gateway().Retrieve(ctx, s.ID) // outside tx, before lagoon.Transaction
...
_, err = svc.Apply(ctx, tx, fresh.update())
// apply.go: ApplyDeleted
if len(rows) == 0 {
return nil, fmt.Errorf("golem15.paymentgatewaysubscriptions: apply deleted %s: subscription not stored yet", subscriptionID) // retried
}
```
### CR-02: Checkout can charge a Stripe price in a different currency or amount than the customer API shows and validated
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/api/subscription.go:112-119, 140-149`
- `sm-pgstripe-plugin/subscriptions/gateway.go:128`
- `sm-paymentgatewaysubscriptions-plugin/models/subscription_plan.go:109-119`
- `sm-pgstripe-plugin/subscriptions/sync_plan.go:15, 55-69`
- `sm-paymentgatewaysubscriptions-plugin/api/json.go:62-75`
**Issue:** `Checkout` accepts a currency when `priced(plan, currency)` and `SupportsCurrency(currency)` are both true. `priced` only looks at the local `currency_prices` and `price_cents`/`currency`. The gateway then picks the Stripe price with `StripePriceIDFor(currency)`, which silently falls back to `stripe_price_id` when `stripe_prices` has no entry for that currency.
`SyncPlanToGateway` creates prices only for `usd`, `eur` and `pln` and sets `stripe_price_id` to the USD price. A plan whose `currency` is any other supported code (`gbp`, `cad`, `aud`, `chf`, `jpy`; the admin form accepts any 3-letter text) therefore goes like this:
- `/plans` lists it as, for example, `29.99 GBP`.
- Checkout passes the 422 gate.
- Stripe Checkout runs on the **USD** price created from the same `price_cents`: `$29.99` instead of `£29.99`.
The same happens when `currency_prices` names a currency the plan was never synced in, or when an admin typed a `stripe_price_id` for one currency and the customer asks for another.
JPY is wrong a second way. `money.FromMinor` divides by 100 for every currency, so `/plans` shows `¥5` for `price_cents: 500`, while a JPY price would bill `¥500`.
**Fix:** Before calling the gateway, require a gateway price that belongs to the requested currency, and answer 422 otherwise. Never fall back across currencies:
```go
// api/subscription.go, after the priced/SupportsCurrency check
if !gatewayPriced(plan, currency) {
writeValidation(w, map[string][]string{"currency": {msgCurrencyInvalid}})
return
}
func gatewayPriced(plan *models.SubscriptionPlan, currency string) bool {
if plan.StripePrices.Valid {
if id, ok := plan.StripePrices.Data[strings.ToLower(currency)]; ok && id != "" {
return true
}
return false // synced plan without this currency: never fall back to stripe_price_id
}
// unsynced plan with a hand-entered price id: only its own currency
return plan.StripePriceID != nil && *plan.StripePriceID != "" && strings.EqualFold(plan.Currency, currency)
}
```
Also make `CreateCheckoutSession` refuse the fallback in the same way (`ErrNoPriceForCurrency`), because other callers of `Service.Checkout` hit the same path. Either drop `jpy` from `supportedCurrencies` or handle zero-decimal currencies in `planPrices`.
## Warnings
### WR-01: Plan sync calls Stripe while holding the bulk-action transaction and row locks; a rollback loses the created Stripe ids
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go:130-153, 155-178`
- `sm-paymentgatewaysubscriptions-plugin/classes/service.go:362-388`
**Issue:** cabana's `BulkAction` opens a transaction, locks the selected plans with `FOR UPDATE` and runs `runSyncSelected` inside it (`summercms.go/modules/cabana/crud.go:369-388`). `syncOne` then makes one product call and up to three price list, create and archive calls per plan, all while the transaction and the row locks are held. This contradicts the `contracts.Gateway` doc ("no transaction is held across a provider call").
Stripe can create a product and its prices and then the transaction can still roll back:
- the admin's request is cancelled or times out (the context is shared);
- a lock wait or statement times out;
- the commit fails.
In each case the saved `stripe_product_id`/`stripe_prices` are lost, and the next sync creates a duplicate product. `runSyncAll` does the same per plan inside `lagoon.Transaction`, discards the transaction error (`_ = lagoon.Transaction(...)`, line 173), and has already counted the plan as `synced`.
**Fix:**
1. Make the bulk action collect only the plan ids.
2. Run each plan's gateway call outside any transaction, then save the ids in a short transaction of its own, the way `ensureCustomer` already does.
3. In `runSyncAll`, check the returned error and count a failed commit as a failure:
```go
if err := lagoon.Transaction(ctx, db, func(ctx context.Context, tx *gorm.DB) error { ... }); err != nil {
tally.failures = append(tally.failures, plan.Code+": save failed")
}
```
### WR-02: Concurrent or retried checkouts create duplicate Stripe customers, and the last write wins
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/classes/service.go:392-411`
- `sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go:89`
- `sm-pgstripe-plugin/subscriptions/gateway.go:76-108`
**Issue:** `ensureCustomer` checks `sub.CustomerID()` on a subscriber loaded before the call, creates a Stripe customer without an idempotency key, and then runs `UPDATE users SET stripe_customer_id = ? WHERE id = ?` unconditionally. Two parallel checkout requests (a double click in two tabs, or a client retry after a timeout) each create a customer, and the second UPDATE overwrites the first. If the user completes the session tied to the overwritten customer:
- the subscription lives under a customer the app no longer knows;
- `POST /subscription/portal` opens the other customer's empty portal;
- the user cannot manage or cancel that subscription from the site.
**Fix:** Write the id only when none is stored, and re-read on conflict. Also pass an idempotency key derived from alias and id to `V1Customers.Create`:
```go
res := tx.Exec(`UPDATE users SET stripe_customer_id = ? WHERE id = ? AND (stripe_customer_id IS NULL OR stripe_customer_id = '')`, id, u.id)
if res.RowsAffected == 0 { /* re-read the stored id and use it; optionally delete the orphan customer */ }
```
```go
params.SetIdempotencyKey("pgsub-customer-" + s.MorphAlias() + "-" + strconv.FormatUint(uint64(s.Key()), 10))
```
### WR-03: Checkout's "Already subscribed" guard misses non-active live subscriptions and is racy, so users can be billed twice
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/api/subscription.go:87-93, 198-205`
**Issue:** Checkout refuses with 409 only when `Active()` (active or trialing) finds a row. A user whose subscription is `past_due` (still in grace and still billed by Stripe's retries), `unpaid`, `incomplete` or `paused` can start a second Checkout. A completed second Checkout gives two live Stripe subscriptions on one customer, both charging.
The check is also not atomic. Two parallel requests both pass it and both get a session URL; the layer's busy guard protects only one tab. `POST /subscription/cancel` then cancels only the newest active row, leaving the other one billing.
**Fix:**
- Refuse checkout while any subscription of the subscriber is in a non-terminal status (anything except `canceled` and `incomplete_expired`). Send `past_due`/`unpaid` users to the portal instead.
- Serialize per subscriber, either with `SELECT ... FOR UPDATE` on the users row or `pg_advisory_xact_lock(hashtext('pgsub:user:'||id))` around the check, or with a Stripe idempotency key per user and plan for a short window.
### WR-04: A subscription whose price was archived by a later sync (or whose plan code changed) is never stored, although the customer is billed
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/classes/apply.go:72-85, 300-323`
- `sm-pgstripe-plugin/subscriptions/sync_plan.go:141-151`
**Issue:** `planForPrice` resolves a plan only from the current `stripe_price_id` or the current `stripe_prices` values. Re-syncing a repriced plan archives the old price and replaces the ids in `stripe_prices`. A Checkout session created before the resync (valid for 24 h) and completed after it produces a subscription on the old price. Its `customer.subscription.created`/`updated` events then fail with "no plan holds price id", are retried for conga's default attempts and are discarded. The webhook already answered 200, so Stripe never redelivers. The customer is charged and the app has no subscription row. Changing a plan's `code` (editable in the form) has the same effect for new checkouts on the old lookup key.
**Fix:** Fall back to the `plan_code` metadata the gateway writes on every Checkout subscription (`gateway.go:153`). Carry it in `SubscriptionArgs`, add `PlanCode` to `contracts.Update`, and in `Apply` use `ActivePlanByCode` (or any plan by code) when `planForPrice` returns nil:
```go
if plan == nil && u.PlanCode != "" {
plan, err = planByCode(db, u.PlanCode)
}
```
### WR-05: The grace period is computed from the already advanced period end, so a past_due subscription keeps access for about a full unpaid period plus grace_period_days
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/models/subscription.go:137-142`
**Issue:** `InGracePeriod` grants access while `now < current_period_end + graceDays`. When Stripe renews, it moves `current_period_start`/`current_period_end` to the new period before the renewal invoice is paid. A `past_due` subscription therefore already carries the end of the period that was not paid for. With monthly billing and the default 7 days, `has_access` stays true for about 37 days after the failed payment, not 7. PHP had the same formula (inherited), but `has_access` is now exposed through the new customer API and `Service.HasAccess`.
**Fix:** Base grace on the start of the unpaid period (or on the timestamp of the move to `past_due`):
```go
return now.Before(s.CurrentPeriodStart.AddDate(0, 0, graceDays))
```
If PHP parity must be kept, document the behaviour in the README and the API contract.
### WR-06: SyncSubscription saves a stale, unlocked row in full and never refreshes the plan
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/classes/service.go:335-352`
- `sm-pgstripe-plugin/subscriptions/gateway.go:263-306`
**Issue:** The caller hands in a row loaded earlier without a lock. The gateway changes some fields in memory, and the service then runs `Save(sub)`, which writes every column. Changes that `Apply` made between the load and the save are overwritten with the stale values: `subscription_plan_id` after a plan switch, `subscriber_*`, `stripe_customer_id` and `canceled_at`. The gateway also ignores the retrieved price, so a plan change made in the portal is never reconciled by a sync.
**Fix:**
1. Re-load the row `FOR UPDATE` inside the short transaction.
2. Copy only the gateway-owned fields onto it.
3. Map the retrieved price to a plan with `planForPrice`.
Or `Select(...)` only the columns the gateway sets.
### WR-07: After a redirect the subscribe and manage buttons stay disabled for good, including after the browser Back button
**Files:**
- `vue-paymentgateway-layer/app/utils/subscribe.ts:74, 85, 115, 125, 134`
**Issue:** `state.redirecting` is set to true before `location.assign` and never reset. When the user goes to Stripe Checkout or the portal and presses Back, browsers with a back/forward cache restore the page with its JavaScript state. The button then shows "Subscribing..." and stays disabled, and `cancel()` returns null without a request. The only way out is a full reload. Leaving Checkout with Back is a common path.
**Fix:** Reset the state when the page is shown again:
```ts
if (typeof window !== 'undefined') {
window.addEventListener('pageshow', (e) => { if ((e as PageTransitionEvent).persisted) { state.redirecting = false; state.busy = false } })
}
```
Register the listener in the components' `onMounted` and remove it in `onBeforeUnmount`.
## Info
### IN-01: The per-user throttle key of the customer API never applies
**File:** `sm-paymentgatewaysubscriptions-plugin/routes.go:54-60, 77-82`
**Issue:** `throttle:paymentgatewaysubscriptions-api` is group middleware, and it runs before `pgapi.RequireJWT`. Nothing sets the bouncer principal until the jwt guard runs (`bouncer/jwt.go:63`), so `bouncer.User` is always empty when the bucket key is computed. Every request is keyed by client IP, and the documented "120 per minute per frontend user" is not what happens. The order API has the same pattern.
**Fix:** Run the guard before the throttle on the four Bearer routes. Alternatively, drop the per-user branch and correct the comment and README.
### IN-02: The `metadata_app` filter lets untagged events from other applications through
**File:** `sm-pgstripe-plugin/subscriptions/job.go:136`
**Issue:** `want != "" && got != "" && got != want` skips only events tagged with a different app. On a shared Stripe account, an application that writes no tag (including the PHP apps before cutover) gets every subscription event applied. Each fails on "no plan holds price id" and is retried until it is discarded, which floods the error logs.
**Fix:** When `metadata_app` is set, skip (and mark) untagged events too, or make that behaviour a config option.
### IN-03: The admin sync result shows the raw Stripe error text
**File:** `sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go:86-92`
**Issue:** `cause.Error()` (the Stripe message) is put into the toast. Elsewhere the gateway deliberately logs only Stripe's type, code and request id, because "the message may echo request data".
**Fix:** Map `*stripe.Error` to a localized line naming the code, or reuse the `failed()` attributes.
### IN-04: The plans table decides "free" differently from the server
**File:** `vue-paymentgateway-layer/app/components/Pg/Plans.vue:35-38, 78-81`
**Issue:** The server treats a plan as free when `price_cents == 0` (checkout answers 422, `api/subscription.go:105-108`). The layer treats it as free when every listed price is zero. A plan with `price_cents: 0` and non-zero `currency_prices` shows a Subscribe button that always fails with "unavailable". A plan with mixed zero and non-zero prices shows `0.00` with a Subscribe button.
**Fix:** Expose a `free` boolean in `PlanJSON`, or apply the same rule on both sides.
### IN-05: TaglistField leaves a typed separator in the input when the draft is empty
**File:** `summercms.go/admin/src/components/form/fields/TaglistField.vue:86-99`
**Issue:** Typing `,` (or a space in space mode) into an empty input leaves `draft` at `''`, which is unchanged, so Vue does not patch the input. The separator stays visible until the next keystroke or blur.
**Fix:** Write the remaining text back to the element (`(event.target as HTMLInputElement).value = rest`) after splitting.
### IN-06: Taglist required message is not localized; `customTags: false` without options is accepted at boot
**File:** `summercms.go/modules/cabana/field_taglist.go:300, 314, 252-263`
**Issue:** The required error is a hard-coded English string, while the other taglist errors use phrase keys. A field with `customTags: false` and no `options` compiles and then refuses every tag at save time.
**Fix:** Use the localized required message. Fail boot when `customTags: false` has no options.
### IN-07: Test-mode and live-mode Stripe customer ids share `users.stripe_customer_id`
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/updates/202610150002_add_stripe_customer_id_to_users.go:18-21`
- `sm-pgstripe-plugin/subscriptions/gateway.go:80-82`
**Issue:** A customer created while `test_mode` is on is stored in the same column and reused after switching to live. Live checkout and portal calls then fail with "No such customer" and the user gets a permanent 502. Nothing recovers from it.
**Fix:** When Stripe answers `resource_missing` for the stored customer, clear it and create a new one. Alternatively, store the customer per mode.
### IN-08: Repository hygiene of the new plugin module
**Files:**
- `sm-paymentgatewaysubscriptions-plugin/go.work:9`
- `sm-paymentgatewaysubscriptions-plugin/go.mod:5-7, 132-134`
**Issue:**
- The committed `go.work` replaces `sm-user-plugin` with `../sm-summercmsio-app/...`, which names a consuming application from a shared core plugin repository.
- `go.mod` requires `sm-paymentgateway-plugin v0.0.0`, which resolves only through the module's own `replace` directives. Consumers ignore those, so every consumer must add its own replace.
- The tagged `summercms v0.1.26` requirement is shadowed by `replace ../summercms.go` in every local build.
**Fix:** Point the workspace at a neutral path or leave `go.work` untracked. Require tagged versions of the sibling plugins once they exist.
---
_Reviewed: 2026-10-14T19:22:09Z_
_Reviewer: Claude (msd-code-reviewer)_
_Depth: standard_

View File

@@ -0,0 +1,44 @@
---
status: testing
phase: 18.1-payment-gateway-subscriptions
source: [18.1-VERIFICATION.md]
started: 2026-10-14T19:30:26.848Z
updated: 2026-10-14T19:30:26.848Z
---
## Current Test
number: 1
name: Decide on review findings CR-01 (out-of-order webhook snapshots) and CR-02 (checkout currency fallback to stripe_price_id) before the production cutover
expected: |
Either accept them as PHP parity (D-21 and the frozen getStripePriceId fallback), recorded as overrides or a backlog item, or open a decimal gap-closure phase or quick task: CR-01 by retrieving the subscription from Stripe in the job (or an event.created watermark) and retrying ApplyDeleted on an unknown row; CR-02 by refusing a currency without its own entry in stripe_prices in api/subscription.go and in CreateCheckoutSession
awaiting: user response
## Tests
### 1. Decide on review findings CR-01 (out-of-order webhook snapshots) and CR-02 (checkout currency fallback to stripe_price_id) before the production cutover
expected: Either accept them as PHP parity (D-21 and the frozen getStripePriceId fallback), recorded as overrides or a backlog item, or open a decimal gap-closure phase or quick task: CR-01 by retrieving the subscription from Stripe in the job (or an event.created watermark) and retrying ApplyDeleted on an unknown row; CR-02 by refusing a currency without its own entry in stripe_prices in api/subscription.go and in CreateCheckoutSession
result: [pending]
### 2. Live Stripe test-mode checkout end to end: test-mode keys and the subscription webhook secret in pgstripe settings, `stripe listen --forward-to <host>/webhook/stripe/subscriptions`, run checkout from the layer playground (PgPlans -> PgSubscribeButton) with the user subscriber
expected: Stripe Checkout opens on the plan's price in the shown currency; after payment the webhook answers 200 'Webhook received', the job stores the subscription as active, golem15.pgsub.subscription.activated fires once, GET /subscription answers has_access true; observe whether customer.subscription.created arrives as incomplete followed by updated active (the CR-01 race window)
result: [pending]
### 3. Billing-portal cancellation (D-21 cancel_at question): open the portal through PgManageSubscriptionButton, cancel the subscription, inspect the delivered customer.subscription.updated
expected: If Stripe sends cancel_at_period_end: true, the row gets cancel_at_period_end true and PgMySubscription shows the will-cancel notice. If it sends only cancel_at (newer API versions), the row keeps cancel_at_period_end false and the cancellation is invisible; that would need a follow-up mapping
result: [pending]
### 4. Browser pass through the admin screens and the layer components
expected: Taglist chips (Enter, comma, remove, suggestions, read-only) on the plan form; flat drag reorder of plans writes sort_order; bulk sync_to_stripe and toolbar sync_all_to_stripe toasts with per-plan failures; Subscriptions list filters and the preview subscriber partial; the layer's plans table, subscribe redirect, my-subscription cancel through PgConfirm, portal redirect; press Back from Stripe Checkout and check whether the subscribe button is stuck on 'Subscribing...' (review WR-07, unfixed in app/utils/subscribe.ts)
result: [pending]
## Summary
total: 4
passed: 0
issues: 0
pending: 4
skipped: 0
blocked: 0
## Gaps

View File

@@ -0,0 +1,223 @@
---
phase: 18.1-payment-gateway-subscriptions
verified: 2026-10-14T19:35:00Z
status: human_needed
score: 5/5 roadmap success criteria verified (plan must-haves 41/41 verified; 0 present-behavior-unverified)
covered_files:
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-01-PLAN.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-01-SUMMARY.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-02-PLAN.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-02-SUMMARY.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-03-PLAN.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-03-SUMMARY.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-04-PLAN.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-04-SUMMARY.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-05-PLAN.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-05-SUMMARY.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-06-PLAN.md
- .planning/phases/18.1-payment-gateway-subscriptions/18.1-06-SUMMARY.md
- admin/src/components/form/fields/TaglistField.vue
- admin/src/views/ReorderView.vue
- modules/cabana/crud.go
- modules/cabana/field_taglist.go
- modules/cabana/form_schema.go
- modules/cabana/list_schema.go
- modules/cabana/reorder.go
- scripts/check-phase18.1.sh
covered_digest: "v2:sha256:3f6d8db93c3ed3d1a76a29d5111f67aa4333965ceacec7e0571c0c30035b0e70"
behavior_unverified: 0
overrides_applied: 0
coincidental_reliance_items:
- truth: "SC-4: duplicate-event idempotency"
reason: incidental-ordering
harden: "TestHostSubscriptionFlow/duplicate_event proves two deliveries of one event fire once, but only for the same event id. Two different events about one subscription (created incomplete and updated active, or deleted before created) are applied in whatever order River runs them (default 4 workers per queue). D-21 accepts PHP's 'apply as they arrive'; if that stops being acceptable, declare an ordering rule (event.created watermark per subscription, or re-retrieve from Stripe in the job). See CR-01."
human_verification:
- test: "Decide on review findings CR-01 (out-of-order webhook snapshots) and CR-02 (checkout currency fallback to stripe_price_id) before the production cutover"
expected: "Either accept them as PHP parity (D-21 and the frozen getStripePriceId fallback), recorded as overrides or a backlog item, or open a decimal gap-closure phase or quick task: CR-01 by retrieving the subscription from Stripe in the job (or an event.created watermark) and retrying ApplyDeleted on an unknown row; CR-02 by refusing a currency without its own entry in stripe_prices in api/subscription.go and in CreateCheckoutSession"
why_human: "Both are faithful ports of PHP behaviour (verified against pgstripe jobs/ProcessStripeSubscriptionWebhook.php and SubscriptionPlan::getStripePriceId) and D-21 explicitly chose PHP's out-of-order semantics, so they do not fail the 'works as it does on PHP' goal. They are still money-relevant and easier to reach in Go (conga runs 4 workers per queue by default; the new customer API validates currency against local prices only). Accepting or fixing them is a product decision."
- test: "Live Stripe test-mode checkout end to end: test-mode keys and the subscription webhook secret in pgstripe settings, `stripe listen --forward-to <host>/webhook/stripe/subscriptions`, run checkout from the layer playground (PgPlans -> PgSubscribeButton) with the user subscriber"
expected: "Stripe Checkout opens on the plan's price in the shown currency; after payment the webhook answers 200 'Webhook received', the job stores the subscription as active, golem15.pgsub.subscription.activated fires once, GET /subscription answers has_access true; observe whether customer.subscription.created arrives as incomplete followed by updated active (the CR-01 race window)"
why_human: "The Stripe fake cannot prove Stripe's hosted Checkout or real delivery order"
- test: "Billing-portal cancellation (D-21 cancel_at question): open the portal through PgManageSubscriptionButton, cancel the subscription, inspect the delivered customer.subscription.updated"
expected: "If Stripe sends cancel_at_period_end: true, the row gets cancel_at_period_end true and PgMySubscription shows the will-cancel notice. If it sends only cancel_at (newer API versions), the row keeps cancel_at_period_end false and the cancellation is invisible; that would need a follow-up mapping"
why_human: "Depends on the Stripe account's API version and portal configuration"
- test: "Browser pass through the admin screens and the layer components"
expected: "Taglist chips (Enter, comma, remove, suggestions, read-only) on the plan form; flat drag reorder of plans writes sort_order; bulk sync_to_stripe and toolbar sync_all_to_stripe toasts with per-plan failures; Subscriptions list filters and the preview subscriber partial; the layer's plans table, subscribe redirect, my-subscription cancel through PgConfirm, portal redirect; press Back from Stripe Checkout and check whether the subscribe button is stuck on 'Subscribing...' (review WR-07, unfixed in app/utils/subscribe.ts)"
why_human: "vitest and SSR specs do not drive a real browser or the back/forward cache"
---
# Phase 18.1: Payment Gateway Subscriptions Verification Report
**Phase goal:** The generic core of `Golem15.PaymentGatewaySubscriptions` and pgstripe's subscription gateway and webhook run on SummerCMS. Horoskopia's subscription flow (plans, checkout, Stripe webhook, `golem15.pgsub.*` events, cancel and portal) works as it does on PHP.
**Mode:** mvp (the ROADMAP goal is not in user-story form; verified against the goal text and its five success criteria)
**Verified:** 2026-10-14T19:35:00Z
**Status:** human_needed
**Re-verification:** No, initial verification
## Goal Achievement
### Observable Truths (roadmap success criteria)
| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| 1 | SC-1: Subscription and SubscriptionPlan port on a squashed, idempotent v2.4.0 baseline that adopts existing PHP databases, with polymorphic subscribers registered at build time and a built-in `user` subscribable | VERIFIED | `sm-paymentgatewaysubscriptions-plugin/updates/202610150001_create_golem15_pgsub_baseline.go`: three tables, every statement `IF NOT EXISTS`, PHP index and constraint names, eight-value status CHECK, RESTRICT FK; dormant tables absent. `202610150002_add_stripe_customer_id_to_users.go` is `ADD COLUMN IF NOT EXISTS` with a no-op rollback. `boot.go` bootRegistries walks `party.Activated` and collects `HasSubscribables` and `HasSubscriptionGateways`, failing on duplicates. `classes/user_subscribable.go` resolves soft-deleted users (Unscoped) and writes only `stripe_customer_id`. TestBaselineSchema, TestRegistry and TestUserSubscribable passed in this run's gate. |
| 2 | SC-2: the `SubscriptionGateway` contract (including `syncPlanToGateway`) has a Stripe implementation; `POST /webhook/stripe/subscriptions` and its legacy alias verify the event, enqueue a job, process it idempotently and fire the six `golem15.pgsub.*` events | VERIFIED | `contracts/gateway.go` declares the 8-method Gateway including SyncPlanToGateway. `sm-pgstripe-plugin/subscriptions/gateway.go` and `sync_plan.go` implement it (usd/eur/pln loop, transfer_lookup_key, archive). `plugin.go` routes both `WebhookPath` and `LegacyWebhookPath` to one handler behind one 60/min bucket. `webhook.go` verifies the signature on the raw body capped at 1 MiB, cheap-checks the marker and enqueues only the five handled types. `job.go` claims the marker first in one transaction (`ClaimEvent`, ON CONFLICT DO NOTHING) and routes to Apply, ApplyDeleted and ApplyInvoice. `events/events.go` holds the six PHP names. The event diff in `classes/apply.go` matches PHP `fireStateChangeEvents` line for line. TestSubscriptionWebhookSmoke, TestSubscriptionJob, TestApplyEvents, TestClaimEventConcurrent, TestSyncPlan and TestSubscriptionGateway are in the gate. |
| 3 | SC-3: SubscriptionPlans and Subscriptions admin screens, a Go service API, a headless customer API and subscription components in `vue-paymentgateway-layer` | VERIFIED | Admin: `controllers/subscriptionplans.go` (sync_to_stripe, sync_all_to_stripe through Service.SyncPlan), `models/subscriptionplan/fields.yaml` (`features: type: taglist, mode: array`), `config_reorder.yaml` (flat), `controllers/subscriptions.go` and the subscriber partial with the unknown-alias danger callout; three permission codes verbatim; navigation order 510. Framework support (`modules/cabana/field_taglist.go`, flat reorder in `reorder.go`) is in tag v0.1.26, including fix `0f09586`. Service API: `classes/service.go` (Latest, Active, Checkout, Portal, Cancel, SyncPlan, SyncSubscription, HasAccess, InGracePeriod and more). Customer API: `routes.go` `/_paymentgatewaysubscriptions/api/v1` with public `/plans` and four `RequireJWT` routes; redirect URLs only from config. Layer: `subscriptionsClient.ts` (five paths), `useSubscriptionsApi.ts` (runtimeConfig server and public bases), PgPlans, PgSubscribeButton (createSubscribeFlow), PgMySubscription, PgManageSubscriptionButton; package version 0.2.0, tag v0.2.0 on origin. Gate: admin smokes, TestCustomerAPISmoke and TestCustomerAPIAuthz passed; the layer reported 36 `subscriptions:` TAP tests passing, 105/105 total. |
| 4 | SC-4: a host test proves checkout, activation, `invoice.paid`, `past_due` with its grace period, cancel and duplicate-event idempotency against a Stripe fake | VERIFIED | `sm-pgstripe-plugin/subscriptions/host_test.go` TestHostSubscriptionFlow has subtests checkout_activation, invoice_paid_basil, invoice_paid_pre_basil, past_due_grace (access inside the grace period, none after it), cancel_at_period_end, cancel_immediately, reactivation, duplicate_event (two deliveries, past_due fires once, one marker, third delivery answers 'Already processed'), app_tag, plan_sync, portal, sync_subscription and real_worker. All six event names are asserted. The run is in the gate's postgres stage. |
| 5 | SC-5: the new code has unit tests, delivered in the phase's last plan | VERIFIED | Plan 06 commits: summercms.go `323b6d6` and `3cc4a5f` (taglist and flat reorder matrices, gate with self-test); pgsub `1da437f` (13 files, 2,469 lines: models, registry, apply events, baseline, service API, admin actions, customer API authz, lang parity); pgstripe `88f9125` (decode, job, sync_plan, gateway, php_map, subscription secrets); layer `30046f5`. `scripts/check-phase18.1.sh --self-test` passed in this run. |
**Score:** 5/5 roadmap truths verified (0 present, behavior-unverified). All 41 plan must-have truths across plans 01 to 06 trace to the code above or to the D-ID table below.
### Requirements Coverage (phase-local IDs; REQUIREMENTS.md maps no IDs to 18.1)
| ID | Plans | Status | Evidence |
|----|-------|--------|----------|
| SC-1 | 02, 06 | SATISFIED | Truth 1 |
| SC-2 | 02, 03, 06 | SATISFIED | Truth 2 |
| SC-3 | 01, 02, 04, 05, 06 | SATISFIED | Truth 3 |
| SC-4 | 03, 06 | SATISFIED | Truth 4 |
| SC-5 | 06 | SATISFIED | Truth 5 |
| D-01 | 02, 03, 06 | SATISFIED | Frozen status, gateway, table and event strings (`events/events.go`, `models.Statuses`, baseline CHECK); lead fees, metering, placements and analytics not ported (no such files) |
| D-02 | 02, 04 | SATISFIED | Three permission codes verbatim incl. `manage_placements` with no screen (`admin_permissions.go`) |
| D-03 | 02, 06 | SATISFIED | `models/feature_list.go` list plus legacy map decode; TestFeatureList |
| D-04 | 02, 06 | SATISFIED | Idempotent baseline; TestBaselineSchema golden names |
| D-05 | 02, 06 | SATISFIED | `GatewayStripe`, `GatewayGooglePlay`; `ErrGatewayNotInstalled` (`contracts/registry.go`) |
| D-06 | 02 | SATISFIED | `go list -deps ./...` of pgsub names 0 stripe-go packages (run here) |
| D-07 | 03 | SATISFIED | `golem15.pgstripesubscriptions` is its own plugin; `go list -deps .` of the pgstripe root names 0 pgsub packages (run here) |
| D-08 | 02, 04, 06 | SATISFIED | Build-time registries in `boot.go`; unknown alias returns ErrUnknownSubscriberType; the admin partial shows a danger notice |
| D-09 | 02, 06 | SATISFIED | users column migration plus a targeted UPDATE only (`user_subscribable.go:89`) |
| D-10 | 03 | SATISFIED | Two routes, one handler, one 60/min IP bucket (`subscriptions/plugin.go`) |
| D-11 | 03, 06 | SATISFIED | Webhook never writes the marker; the job claims first; TestSubscriptionJob STRIPE-006/007 |
| D-12 | 03, 06 | SATISFIED | Five plain-text answers in PHP order (`webhook.go`); job args are PII-free (`WebhookJobArgs`); TestJobArgsMinimal |
| D-13 | 02, 04, 05, 06 | SATISFIED | Service API and customer API; TestServiceAPI, TestCustomerAPIAuthz |
| D-14 | 05 | SATISFIED | Layer client, composables, four Pg components, en/pl copy, no new dependency (`git diff v0.1.0 v0.2.0 -- package.json` changes only the version; lockfile unchanged) |
| D-15 | 03, 06 | SATISFIED | `sync_plan.go` port of d00f850; TestSyncPlan |
| D-16 | 02, 06 | SATISFIED | Dormant tables not created; JSONB baseline confirmed at the 02 T1 checkpoint |
| D-17 | 01, 04, 06 | SATISFIED | Core taglist and flat reorder in cabana; the plan form uses them; framework tests in the gate's go and postgres stages |
| D-18 | 03, 06 | SATISFIED | Additive secret columns and `SubscriptionWebhookSecret` (`client.go` diff is +19 lines, additive only); `--webhook-secret-target`; TestStripeSubscriptionSecrets |
| D-19 | 03, 06 | SATISFIED | `metadata_app` config key, empty by default (`gateway.go:148`, `job.go` filter); no billing_owner_id |
| D-20 | 03 | SATISFIED | Subpackage plugin registered in init() |
| D-21 | 03, 04, 05, 06 | SATISFIED (manual item open) | PHP cancel mapping kept, out-of-order applied as they arrive, both payload shapes decoded (TestDecodeSubscription, TestDecodeInvoice); GET /subscription returns the latest with has_access and in_grace_period. The portal `cancel_at` question is a human item. |
| D-22 | 01, 02, 04, 05, 06 | SATISFIED | All four repos pushed (`main...origin/main` clean for pgsub, pgstripe and the layer); tags v0.1.26 and v0.2.0 on origin; plugin go.mod files require v0.1.26 (`472b75b`, `baf3c55`). summercms.go is one commit ahead of origin: the planning-only review report `929cfd8`. |
No orphaned requirements: REQUIREMENTS.md has no rows for Phase 18.1.
### Required Artifacts
| Artifact | Status | Details |
|----------|--------|---------|
| `summercms.go/modules/cabana/field_taglist.go` | VERIFIED | compileTaglistKeys and checkTaglistValues wired from form_schema.go and crud.go; tests in the gate |
| `summercms.go/modules/cabana/reorder.go` | VERIFIED | flat inference and save; `list_schema.go` keeps Flat after fix 0f09586 (in v0.1.26) |
| `summercms.go/admin/src/components/form/fields/TaglistField.vue` | VERIFIED | registered in registry.ts; vitest 1401 passed |
| `sm-paymentgatewaysubscriptions-plugin/{plugin,boot}.go`, `contracts/*`, `classes/*`, `events/events.go`, `updates/*` | VERIFIED | substantive, wired through Boot and published services |
| `sm-paymentgatewaysubscriptions-plugin/controllers/*`, `routes.go`, `api/*` | VERIFIED | admin and customer API wired; smokes in the gate |
| `sm-pgstripe-plugin/subscriptions/{plugin,gateway,sync_plan,webhook,decode,job}.go` | VERIFIED | gateway published via SubscriptionGateways(); job registered via Jobs() |
| `sm-pgstripe-plugin/updates/202610150001_add_subscription_webhook_secrets.go` | VERIFIED | ADD COLUMN IF NOT EXISTS |
| `vue-paymentgateway-layer/app/{utils,composables,components/Pg}/*` (subscriptions) | VERIFIED | no v-html in the new components (the only v-html is the pre-existing BankTransfer.vue) |
| `summercms.go/scripts/check-phase18.1.sh` | VERIFIED | PHASE181_REQUIRE_POSTGRES, self-test, required names per repo |
### Key Link Verification
| From | To | Via | Status |
|------|----|-----|--------|
| pgsub `boot.go` | `contracts/registry.go` | HasSubscriptionGateways and HasSubscribables collected from party.Activated | WIRED |
| pgstripe `subscriptions/plugin.go` | pgsub registry | `SubscriptionGateways()` returns `NewGateway(p.app)` | WIRED |
| `subscriptions/webhook.go` | `client.go` | `pgstripe.SubscriptionWebhookSecret` then `VerifySignature` | WIRED |
| `subscriptions/webhook.go` | `subscriptions/job.go` | `conga.From(app).Enqueue(WebhookJobArgs)` | WIRED |
| `subscriptions/job.go` | `classes/apply.go` | ClaimEvent, Apply, ApplyDeleted, ApplyInvoice in one lagoon.Transaction | WIRED |
| `classes/apply.go` | `events/events.go` | `s.bus.Fire` inside tx | WIRED |
| `controllers/subscriptionplans.go` | `classes/service.go` | `svc.SyncPlan` | WIRED |
| `routes.go` | `api/subscription.go` | `pgapi.RequireJWT(...)` | WIRED |
| `models/subscriptionplan/fields.yaml` | cabana taglist | `type: taglist`, `mode: array` | WIRED |
| layer `SubscribeButton.vue` | `utils/subscribe.ts` | `createSubscribeFlow` | WIRED |
| layer `subscriptionsClient.ts` | pgsub `routes.go` | `/plans`, `/subscription`, `/subscription/checkout`, `/portal`, `/cancel` | WIRED |
| `scripts/check-phase18.1.sh` | host_test, baseline_test, layer tests | required names TestHostSubscriptionFlow, TestBaselineSchema, `subscriptions.test.ts` | WIRED |
### Data-Flow Trace (Level 4)
| Artifact | Data | Source | Real data | Status |
|----------|------|--------|-----------|--------|
| GET /plans (`api/plans.go`) | active plans | `Service.ActivePlans` GORM query, is_active ordered by sort_order | yes | FLOWING |
| GET /subscription | latest subscription and flags | `Service.Latest` query plus HasAccess and InGracePeriod | yes | FLOWING |
| Webhook to row | subscription state | signed event to job args to `Apply` upsert | yes | FLOWING |
| PgPlans and PgMySubscription | plans, subscription | usePlans and useSubscription through the client over useSubscriptionsApi | yes | FLOWING |
### Behavioral Spot-Checks and Probe Execution
| Behavior | Command | Result | Status |
|----------|---------|--------|--------|
| pgsub links no Stripe SDK (D-06) | `go -C ../sm-paymentgatewaysubscriptions-plugin list -deps ./... \| grep -c stripe-go` | 0 | PASS |
| pgstripe root links no pgsub (D-07) | `go -C ../sm-pgstripe-plugin list -deps . \| grep -c paymentgatewaysubscriptions` | 0 | PASS |
| Full phase gate, Postgres required | `PHASE181_REQUIRE_POSTGRES=1 bash scripts/check-phase18.1.sh --all` | exit 0; PASS self-test, go, plugins, admin (vitest 1401), layers (105/105 TAP, 36 subscriptions), docs, postgres (cabana 3, pgsub 14, pgstripe root 4, subscriptions 7 incl. TestHostSubscriptionFlow); "Phase 18.1 gate passed" | PASS |
| Tags published | `git ls-remote --tags origin v0.1.26` / layer `v0.2.0` | both present; fix 0f09586 is an ancestor of v0.1.26 | PASS |
| Layer release additive | `git -C ../vue-paymentgateway-layer diff v0.1.0 v0.2.0 --stat -- package.json pnpm-lock.yaml` | package.json version line only | PASS |
The orchestrator also reports `scripts/check-phase18.sh --all` and `go test ./...` (40 packages) green in this run, so Phase 18 did not regress.
### Code Review Findings Weighed (18.1-REVIEW.md)
**CR-01 (stale webhook snapshots).** I compared `job.go` and `classes/apply.go` with PHP `pgstripe/jobs/ProcessStripeSubscriptionWebhook.php` at the port source. PHP also:
- applies `data.object` as a snapshot with no `event.created` check (`firstOrNew` with no lock);
- answers a deleted-before-created with a logged return and a recorded marker;
- fails `subscription_plan_id` validation on an unknown price, which throws and retries.
The Go code reproduces each of these, adds a `FOR UPDATE` lock on the existing row and makes the claim-first marker transactional. Those are strict improvements. D-21 locks "out-of-order events behave as PHP does". The goal "works as it does on PHP" is therefore met, and CR-01 is not a blocker against it.
The race is still real. conga runs 4 workers per queue by default (`modules/conga/client.go:23`), so concurrent created and updated jobs for one subscription are plausible on every Checkout. It is recorded as a human decision item and in `coincidental_reliance_items`.
**CR-02 (currency fallback).** The PHP `SubscriptionPlan::getStripePriceId` falls back to `stripe_price_id` exactly as Go `StripePriceIDFor` does. The gateway's usd/eur/pln sync is the frozen d00f850 code. Horoskopia's own checkout (`plugins/golem15/horoscope/routes.php:641`) always passes `pln`, which plan sync prices, so Horoskopia's flow is unaffected.
The new customer API widens the exposure. Its `priced()` check validates local prices only, so a gbp, cad, aud, chf or jpy plan, or a `currency_prices` entry that was never synced, passes the 422 gate and is billed in USD. This is a WARNING for any other consumer and a human decision item.
**Warnings WR-01 to WR-07.** None defeats a success criterion:
- WR-05 (grace from the advanced period end) is identical to PHP `isInGracePeriod`.
- WR-02, WR-03 and WR-07 are in new Go and layer code, not PHP parity. WR-07 is confirmed unfixed: `subscribe.ts` sets `redirecting` and never resets it.
- They are listed as anti-pattern warnings below for a follow-up.
### Anti-Patterns Found
| File | Line | Pattern | Severity | Impact |
|------|------|---------|----------|--------|
| sm-pgstripe-plugin/subscriptions/job.go | 119-147 | event snapshot applied with no ordering guard (CR-01, PHP parity per D-21) | WARNING | older event can overwrite newer state; human decision |
| sm-paymentgatewaysubscriptions-plugin/api/subscription.go | 112-119, 140-149 | local-price currency check, gateway falls back to stripe_price_id (CR-02, fallback inherited) | WARNING | non-synced currency billed at the USD price via the new API |
| sm-paymentgatewaysubscriptions-plugin/controllers/subscriptionplans.go | 130-178 | Stripe calls inside the bulk transaction; `_ = lagoon.Transaction` (WR-01) | WARNING | lost Stripe ids on rollback, duplicate products |
| sm-paymentgatewaysubscriptions-plugin/classes/user_subscribable.go | 89 | unconditional customer id UPDATE, no idempotency key (WR-02) | WARNING | duplicate Stripe customers under parallel checkout |
| sm-paymentgatewaysubscriptions-plugin/api/subscription.go | 87-93 | 409 guard only on active or trialing; not atomic (WR-03) | WARNING | possible double subscription |
| vue-paymentgateway-layer/app/utils/subscribe.ts | 85 | `redirecting` never reset (WR-07) | WARNING | button stuck after Back from Stripe (bfcache) |
| sm-paymentgatewaysubscriptions-plugin/go.work | 9 | names `../sm-summercmsio-app` (IN-08) | INFO | dev workspace path only; same convention as sm-pgstripe-plugin's go.work since Phase 18 |
No unreferenced TBD, FIXME or XXX markers and no TODO, HACK or placeholder text in the phase's non-test files across the four repositories.
### Prohibitions (judgment tier, autonomous LLM verdict, non-authoritative)
| Plan | Prohibition | Verdict |
|------|-------------|---------|
| 01 | framework code and fixtures name no consuming application or payment key | holds: the v0.1.25..v0.1.26 diff of `modules/` and `docs/` adds no stripe, subscription or app names |
| 01 | existing field types and tree reorders byte-identical | holds per the gate (`TestReorderPostgres`, vitest 1401, check-admin-dist clean); flagged for human review |
| 02, 04 | pgsub imports no stripe-go | holds (`go list -deps`: 0) |
| 02, 03, 04 | no consuming-app names in code, comments, README or fixtures | holds for code, README and fixtures; the committed `go.work` names `sm-summercmsio-app` as a local path (INFO) |
| 02 | user subscriber writes only `stripe_customer_id` | holds (`UPDATE users SET stripe_customer_id = ? WHERE id = ?` only) |
| 03 | root pgstripe changes additive only | holds (`client.go` +19 lines; new migration ADD COLUMN IF NOT EXISTS) |
| 03, 06 | no test calls the network | holds per the plans' BackendOverride fake; not independently audited line by line; flagged |
| 04 | customer API takes no subscriber, subscription or redirect from the request | holds (`customer()` uses the JWT user; URLs from config; TestCustomerAPIAuthz) |
| 05 | layer adds no npm dependency; no v-html in the new components | holds |
| 06 | gate never green on a skip or missing Docker | holds (self-test passed; PHASE181_REQUIRE_POSTGRES=1 used) |
| 01, 06 | nothing pushed or tagged without the human's reply | recorded as replies in VALIDATION.md (hold, create-and-push, push-all); not independently verifiable from git; flagged |
### Human Verification Required
1. **Decision on CR-01 and CR-02.** Accept them as PHP parity (record overrides or a backlog item) or open a gap-closure phase before the Horoskopia cutover.
2. **Live Stripe test-mode checkout.** Run it end to end and confirm activation and a single `activated` event. Watch the created and updated delivery order.
3. **Billing-portal cancellation.** Check whether Stripe sends `cancel_at_period_end` or `cancel_at` (D-21).
4. **Browser pass.** Cover the admin taglist, flat reorder and sync toasts, and the layer components, including Back from Stripe Checkout (WR-07).
### Gaps Summary
No success criterion failed. Every roadmap SC and every phase-local ID (SC-1 to SC-5, D-01 to D-22) has implementation evidence in the four repositories, and the phase gate ran green in this verification.
The two critical review findings reproduce the frozen PHP behaviour. D-21 explicitly chose that behaviour for event ordering, and Horoskopia's own checkout currency (pln) is unaffected. They therefore do not defeat "works as it does on PHP". They are money-relevant, and the Go runtime (4 concurrent workers) and the new customer API make them easier to reach. The status is `human_needed` rather than `passed` for two reasons: the product decision on CR-01 and CR-02, and the three manual items (live Stripe checkout, the portal `cancel_at` question, the browser pass). If the decision is to fix them, the review gives concrete fixes for CR-01 (re-retrieve in the job or an `event.created` watermark, retry ApplyDeleted on an unknown row) and CR-02 (refuse cross-currency fallback in the API and the gateway).
---
_Verified: 2026-10-14T19:35:00Z_
_Verifier: Claude (msd-verifier)_

View File

@@ -0,0 +1,256 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
- .planning/notes/2026-10-15-cooler-go-redis-otter.md
- go.mod
- go.sum
- modules/cooler/README.md
- modules/cooler/cooler.go
- modules/cooler/redis.go
- modules/cooler/ratelimit.go
- modules/cooler/memory.go
- modules/cooler/rediscache.go
- modules/cooler/smoke_test.go
- modules/surf/serve.go
- modules/surf/router.go
- modules/surf/store_lookup_test.go
- modules/surf/README.md
- README.md
- docs/services/rate-limiting.md
- docs/services/caching.md
- docs/setup/configuration.md
- docs/architecture/performance-and-scaling.md
autonomous: true
requirements: [SC-2, SC-4, D-06, D-07, D-08, D-09, D-19]
estimate:
tokens: 160000
raw_tokens: 160000
tasks: 2
confidence: low
must_haves:
truths:
- "D-07: a decision note under .planning/notes/ names go-redis v9 (v9.23.0) as the Redis client, otter v2 (v2.3.0) as the memory cache driver with the reasons a stdlib map was rejected, miniredis v2.40.0 as a test-only dependency installed in plan 05 after a blocking human check, and gocloud.dev/blob/s3blob (AWS SDK v2 pulled in as indirect modules) for plan 03; the note's commit is an ancestor of the first commit that adds go-redis to go.mod."
- "D-06, D-19: a new module `cooler` (import git.golem15.com/golem15/summercms/modules/cooler) exists with a README in the standard structure (H1, summary, import line, **Name:** mnemonic, Overview, Role in the framework, Features, Usage, API reference, Configuration, Dependencies, Testing) and a row in the root README module table whose purpose text equals the README summary sentence."
- "D-09: with `http.rate_limit.store: redis` and `redis.url` set, `summer serve` publishes cooler's Redis-backed store as surf.Store before BuildRouter, so two application instances sharing one Redis count one fixed window together: the third request to a `throttle:2,1` route is 429 whichever instance answers it."
- "D-09: the Redis store reproduces surf.MemoryStore semantics through one atomic Lua script: the first attempt opens the window, a denied attempt does not increment, the window never extends, retryAfter is the remaining window; keys are `<prefix>rl:<key>`."
- "D-09 (user answer): when Redis does not answer, Attempt falls back to a per-instance surf.MemoryStore and logs a warning at most once per minute; an unreachable Redis at start-up is a warning, never a boot failure; a missing or malformed redis.url when a Redis driver is selected is a boot error that names the key and never contains the URL or its password."
- "D-09: `route:list` and surf.Assemble never dial Redis: BuildRouter looks up a published surf.Store and falls back to NewMemoryStore; only the serve command installs cooler."
- "D-08: cooler.Cache offers Get, Set, Has, Forget, Remember, Increment and Flush with per-entry TTL (0 = forever) and a per-app key prefix; `cache.driver: memory` (default) uses a size-bounded otter v2 cache, `cache.driver: redis` uses go-redis with keys `<prefix>cache:<key>`; Flush on Redis removes only keys under `<prefix>cache:` by SCAN and UNLINK; no tags and no locks."
- "D-09: the framework wires everything at boot: serve publishes cooler.Cache, and cooler.FromApp gives jobs and other commands the same cache lazily without a dial at construction; apps import nothing."
artifacts:
- path: ".planning/notes/2026-10-15-cooler-go-redis-otter.md"
provides: "decision note naming go-redis v9, otter v2, miniredis (test-only) and s3blob"
contains: "go-redis"
- path: "modules/cooler/ratelimit.go"
provides: "Redis fixed-window store with MemoryStore semantics and a memory fallback"
contains: "PTTL"
- path: "modules/cooler/cooler.go"
provides: "Install, Handle, FromApp, Cache interface, config keys"
contains: "func Install"
- path: "modules/cooler/memory.go"
provides: "otter v2 memory driver"
contains: "otter"
- path: "modules/surf/serve.go"
provides: "serve installs cooler and publishes the Redis store before BuildRouter"
contains: "installCooler"
- path: "docs/services/caching.md"
provides: "the caching guide (drivers, keys, prefix, FromApp)"
contains: "cache.driver"
key_links:
- from: "modules/surf/serve.go"
to: "modules/cooler/cooler.go"
via: "installCooler calls cooler.Install, then app.Publish[Store](handle.RateLimitStore(NewMemoryStore(2*time.Minute))) when http.rate_limit.store is redis"
pattern: "cooler.Install"
- from: "modules/surf/router.go"
to: "modules/surf/limiter_store.go"
via: "BuildRouter: app.Lookup[Store]() else NewMemoryStore(2*time.Minute)"
pattern: "Lookup\\[Store\\]"
- from: "modules/cooler/ratelimit.go"
to: "modules/surf/limiter_store.go"
via: "RateLimitStore.Attempt has surf.Store's exact signature (structural, cooler never imports surf)"
pattern: "func \\(s \\*RateLimitStore\\) Attempt"
prohibitions:
- statement: "cooler imports neither surf nor cabana (surf imports cooler; cabana would close an import cycle)."
status: unresolved
verification: judgment
- statement: "Framework code, READMEs and docs never name a consuming application."
status: unresolved
verification: judgment
- statement: "Cache Flush never empties the whole Redis database."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not in user-story format; MVP mode would take a story from `/msd mvp-phase 19.3`, which was not run, so no story is invented here): the framework has the transport and infrastructure pieces a multi-client, multi-instance application needs: JWT cookie refresh across subdomains, credentialed CORS, a cache module with a Redis-backed rate-limit store, an S3 blob option, an OpenAPI page and Turnstile verification.
This plan's slice (plan 1 of 5, wave 1): SC-2. The dependency decision note lands first, then the new `cooler` module: a Redis-backed `surf.Store` wired by `serve` so rate limits hold across instances (the tracer), then the Laravel-core cache API with memory (otter v2) and Redis drivers. Plans 02, 03 and 04 build on this checkout in wave 2.
## Spec-less probe fallback
No SPEC.md and no REQUIREMENTS IDs exist for this phase: the orchestrator recorded "spec-less probe fallback skipped: phase has no requirement IDs to probe (visible skip)". No probe predicates were generated. `must_haves` derive from the ROADMAP success criteria SC-1..SC-4 and the CONTEXT decisions D-01..D-19; prohibitions are recalled in-prompt and descriptor-less (flagged unverified).
## Decisions taken in this plan (Claude's discretion)
- Memory driver: otter v2 (bounded size, per-entry TTL, atomic Compute for Increment, injectable clock). A stdlib map was rejected because a key-per-client cache grows without bound. The decision note records it (D-07 discretion).
- `serve` treats an unreachable Redis at start-up as a warning, not a boot failure, consistent with the user's answer that rate limits fall back to per-instance memory when Redis is down. A missing or malformed `redis.url` with a Redis driver selected stays a boot error (configuration, not availability).
- Key prefix: `cache.prefix`, default derived from `app.name` (lower-cased, every run of characters outside `[a-z0-9]` replaced by `_`, then `:`), or `summer:` when `app.name` is empty. Cache keys are `<prefix>cache:<key>`, rate-limit keys `<prefix>rl:<key>`.
- `Remember` degrades instead of failing a page: when the read fails it runs the callback and returns its value, skips a failing write, and logs a warning at most once per minute. The other methods return their errors.
- `cooler.FromApp` gives commands other than `serve` (the dedicated job worker, console commands) the configured cache lazily; go-redis does not dial at construction.
<assumption_delta_decision>
add-alongside: the rate-limit store pluralizes from memory to memory|redis. surf.MemoryStore stays the default and the fallback; the Redis store is added next to it behind `http.rate_limit.store: redis`. No existing behaviour changes for apps that set nothing.
</assumption_delta_decision>
## Artifacts this phase produces (this plan)
- Decision note `.planning/notes/2026-10-15-cooler-go-redis-otter.md`.
- Module `modules/cooler`: `cooler.Cache` (methods `Get`, `Set`, `Has`, `Forget`, `Remember`, `Increment`, `Flush`), `cooler.Attempter`, `cooler.Handle` (methods `Close`, `Cache`, `RateLimitRedis`, `RateLimitStore`), `cooler.Install`, `cooler.FromApp`, `cooler.OpenRedis`, `cooler.NewMemoryCache`, `cooler.NewRedisCache`, `cooler.RateLimitStore` with `cooler.NewRateLimitStore` and method `Attempt`, constants `cooler.RedisURLKey` (`redis.url`), `cooler.CacheDriverKey` (`cache.driver`), `cooler.CachePrefixKey` (`cache.prefix`), `cooler.CacheMaxEntriesKey` (`cache.memory.max_entries`), `cooler.RateLimitStoreKey` (`http.rate_limit.store`), `cooler.DefaultMaxEntries` (10000), `cooler.PingTimeout` (3 s), `cooler.AttemptTimeout` (500 ms).
- Config keys: `redis.url`, `cache.driver` (`memory`|`redis`), `cache.prefix`, `cache.memory.max_entries`, `http.rate_limit.store` (`memory`|`redis`).
- `modules/surf`: unexported `installCooler(ctx, app) (*cooler.Handle, error)` in serve.go; `BuildRouter` looks up a published `surf.Store`.
- Smoke tests: `TestBuildRouterStoreSmoke`, `TestRateLimitAcrossInstancesSmoke` (Docker) in `modules/surf/store_lookup_test.go`; `TestOpenRedisRedactsSmoke`, `TestMemoryCacheSmoke`, `TestRedisCacheSmoke` (Docker) in `modules/cooler/smoke_test.go`. Full unit coverage is plan 05.
- Docs: new `docs/services/caching.md`; `docs/services/rate-limiting.md` section "Sharing limits across instances"; rows in `docs/setup/configuration.md`; the scaling note in `docs/architecture/performance-and-scaling.md`; cooler README; surf README configuration and dependency rows; root README table row.
- Dependencies: `github.com/redis/go-redis/v9` v9.23.0, `github.com/maypok86/otter/v2` v2.3.0.
<objective>
Land the dependency decision note, then the `cooler` module: a Redis-backed rate-limit store that `serve` wires before the router (rate limits hold across instances), and the Laravel-core cache API with memory and Redis drivers, documented in the same commits.
Purpose: SC-2 and D-06..D-09, D-19. Plans 02 and 04 extend surf and cooler on top of this; plan 03 needs the decision note for s3blob.
Output: one docs commit (the note), then code commits with README and docs, all green under go vet and go test.
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-CONTEXT.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-RESEARCH.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-PATTERNS.md
Concurrency: Phase 18.1 gap-closure plans may run in parallel in this checkout (they touch modules/pact, modules/cabana and docs/backend, not this plan's files). Before Task 1 run `git status --porcelain -- go.mod go.sum modules/surf modules/cooler README.md docs/services docs/setup docs/architecture`; if it lists a file this plan touches, stop and report (never stash or overwrite another session's work). Stage files by explicit path; no co-author tags; the decision note (planning doc) and code go in separate commits; a module README and the docs pages change in the same commit as the code they describe. Docker-backed smoke tests skip under `-short`; run them on a Docker host (if the socket is denied although the user is in the docker group, run through `sudo -u golem`) and record NOT RUN in the SUMMARY when no daemon answers.
<interfaces>
Existing seams (read; extend, do not fork):
- `modules/surf/limiter_store.go`: `type Store interface { Attempt(key string, max int, decay time.Duration) (allowed bool, attempts int, retryAfter time.Duration) }` (:11-13), `NewMemoryStore(sweep time.Duration) *MemoryStore`, Attempt semantics (:68-93): expired or missing entry resets to count 0 and resetAt now+decay, retryAfter = resetAt-now floored at 0, `count >= max` denies without incrementing, otherwise count++ and allow.
- `modules/surf/router.go`: `BuildRouter` (:468) builds `NewFixedWindowLimiter(NewMemoryStore(2*time.Minute), trusted)` at :476; `FixedWindowLimiter.store` (limiter.go:40) is the field an internal test reads.
- `modules/surf/serve.go`: `ServeCommand` Run opens the DB, `publishUploads` (:51-55), then `BuildRouter` (:56).
- `modules/backpack/services.go`: `Publish[T]` keyed by `reflect.TypeFor[T]()`, a duplicate is an error; `Lookup[T]() (T, bool)`.
- `modules/postcard/mailer.go` `Activate` (:85-110): nil-app guard, driver from config with a default, `app.Publish[Mailer]`, errors prefixed by the module name.
- `modules/compass/config.go`: `String`, `Int`, `Bool`, `Has`, `Lookup`, `Environment`.
- `modules/postcard/mailpit_test.go` (:35-60) and `modules/beachcomber/postgres_test.go` (:29-54): per-test `testcontainers.Run` with a `testing.Short()` skip.
- RESEARCH Pattern 1 (boot wiring), Pattern 2 (the Lua script, probed on Redis 7.2.5), Pitfall 7 (url.Parse errors echo the URL), Pitfall 13 (outage), Pitfall 14 (go-redis handshake options), Code Examples (otter driver, Redis client from redis.url).
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: decision note first, then two serve instances sharing one Redis count one rate-limit window through cooler's Redis surf.Store</name>
<reversibility rating="costly">D-07: every Redis-touching module and app config builds on go-redis v9 and on the redis.url and http.rate_limit.store keys once they ship.</reversibility>
<files>.planning/notes/2026-10-15-cooler-go-redis-otter.md, go.mod, go.sum, modules/cooler/cooler.go, modules/cooler/redis.go, modules/cooler/ratelimit.go, modules/cooler/smoke_test.go, modules/cooler/README.md, modules/surf/serve.go, modules/surf/router.go, modules/surf/store_lookup_test.go, modules/surf/README.md, README.md, docs/services/rate-limiting.md, docs/setup/configuration.md, docs/architecture/performance-and-scaling.md</files>
<read_first>.planning/notes/2026-10-13-paymentgateway-hashids-go-hashids.md (decision note structure), .planning/research/STACK.md, .planning/research/go-ecosystem.md (Cache row), .planning/research/ARCHITECTURE.md (the cooler row near line 73), modules/surf/limiter_store.go, modules/surf/limiter.go, modules/surf/router.go (BuildRouter :468-580), modules/surf/serve.go, modules/surf/routelist_command.go, modules/backpack/services.go, modules/postcard/mailer.go, modules/postcard/README.md, modules/postcard/mailpit_test.go, modules/surf/README.md, README.md (module table :94-120), docs/services/rate-limiting.md, docs/setup/configuration.md, docs/architecture/performance-and-scaling.md, 19.3-RESEARCH.md (Pattern 1, Pattern 2, Pitfalls 7, 13, 14, Code Examples)</read_first>
<action>Per D-07 the note lands before any code. Write `.planning/notes/2026-10-15-cooler-go-redis-otter.md` in the structure of the hashids note (Decision, Related dependencies, Consequences): go-redis `github.com/redis/go-redis/v9` v9.23.0 is the Redis client (locked by D-07; official client; ParseURL handles redis, rediss and unix URLs; NewScript and Run give EVALSHA with EVAL fallback); otter `github.com/maypok86/otter/v2` v2.3.0 is the memory driver (bounded MaximumSize, per-entry ExpiryCalculator, atomic Compute, injectable Clock, no runtime dependencies; a stdlib map was rejected as unbounded); miniredis `github.com/alicebob/miniredis/v2` v2.40.0 is test-only, installed in plan 05 only after a blocking human check of the package; `gocloud.dev/blob/s3blob` comes from the already-required gocloud.dev v0.46.0, is imported only by a file under the `summercms_s3` build tag (plan 03) and adds AWS SDK v2 modules to go.mod and go.sum as indirect requirements; the Docker-backed Redis tests use the generic testcontainers Run with redis:7-alpine (no new module). Commit the note alone (`docs(19.3): decide cooler dependencies`) before the first go.mod change.
Then `go get github.com/redis/go-redis/v9@v9.23.0`.
cooler (package doc names the module role; errors prefixed `cooler:`; cooler imports neither surf nor cabana):
- cooler.go: the key constants listed in this plan's Artifacts; `Attempter` with exactly surf.Store's Attempt signature; `Handle` with `Close() error`, `RateLimitRedis() bool` and `RateLimitStore(fallback Attempter) Attempter`; `Install(ctx context.Context, app *backpack.App) (*Handle, error)`: nil-app guard; `http.rate_limit.store` empty means memory, any value other than memory or redis is a boot error naming the key; when redis is selected, `redis.url` empty is a boot error naming the key (and the SUMMER_REDIS__URL variable), else `OpenRedis` builds the client and a Ping bounded by `PingTimeout` runs; a failed Ping logs one warning (the endpoint host only, never the URL) and boot continues. The prefix helper resolves `cache.prefix` as described in this plan's Decisions.
- redis.go: `OpenRedis(ctx context.Context, rawURL string) (*redis.Client, error)` parses with redis.ParseURL and on error returns a fixed message naming `redis.url` and the accepted schemes without wrapping or quoting the raw error (Pitfall 7); sets DisableIdentity true and MaintNotificationsConfig mode disabled (Pitfall 14); does not dial.
- ratelimit.go: `RateLimitStore` built by `NewRateLimitStore(client *redis.Client, prefix string, fallback Attempter) *RateLimitStore`; the Lua script of RESEARCH Pattern 2 as a package-level redis.NewScript; `Attempt` runs it with key `<prefix>rl:<key>`, ARGV max and decay in milliseconds clamped to at least 1, under a context bounded by `AttemptTimeout`; it maps the three integers to (allowed, attempts, retryAfter as milliseconds); on any Redis error it returns fallback.Attempt(key, max, decay) and logs one warning per minute at most.
surf: serve.go gains `installCooler(ctx, app) (*cooler.Handle, error)` called after publishUploads and before BuildRouter (deferred Close); when `RateLimitRedis()` it publishes `app.Publish[Store](h.RateLimitStore(NewMemoryStore(2*time.Minute)))`. router.go BuildRouter: when app is non-nil and `app.Lookup[Store]()` returns a non-nil store, use it; otherwise NewMemoryStore(2*time.Minute) as today. routelist_command.go is not changed: route:list never installs cooler.
Docs in the same commit as the code: cooler README in the standard structure (summary sentence "Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, and a Redis-backed rate-limit store.", mnemonic about a cool box at the beach keeping things fresh and at hand; at this commit list only identifiers that exist); root README table row (alphabetical, after conga) with that sentence and the mnemonic; surf README Configuration row `http.rate_limit.store` and the serve paragraph (serve installs cooler), Dependencies gains cooler; docs/services/rate-limiting.md: a section "Sharing limits across instances" (keys, the memory fallback with its warning, route:list never dials) and the sentence that each instance counts on its own now says it applies to the memory store; docs/setup/configuration.md rows `redis.url` and `http.rate_limit.store`; docs/architecture/performance-and-scaling.md: the rate-limit note points at the Redis store. No application or client is named.
Smoke tests (plan 05 owns full coverage): modules/surf/store_lookup_test.go `TestBuildRouterStoreSmoke` (no Docker: an app with http.rate_limit.store redis and redis.url redis://127.0.0.1:1 but no install builds a router whose limiter store is a *MemoryStore and returns within a second; an app with a published counting fake Store routes a throttle:2,1 request through the fake) and `TestRateLimitAcrossInstancesSmoke` (skips under -short; redis:7-alpine through testcontainers.Run; two apps share redis.url, each runs installCooler and surf.Assemble with a test plugin route GET /ping carrying throttle:2,1; requests A, B, A answer 200, 200, 429 with Retry-After); modules/cooler/smoke_test.go `TestOpenRedisRedactsSmoke` (a URL carrying user, password and a control character yields an error whose text contains neither the user nor the password; a rediss URL parses).</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/cooler/... ./modules/surf &amp;&amp; go test -count=1 -run '^(TestBuildRouterStoreSmoke|TestRateLimitAcrossInstancesSmoke)$' -v ./modules/surf 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 2 &amp;&amp; go test -count=1 -run '^TestOpenRedisRedactsSmoke$' -v ./modules/cooler 2&gt;&amp;1 | grep -q -- '^--- PASS: TestOpenRedisRedactsSmoke' &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit: vet or a short suite is red, fewer than two "--- PASS" lines for the surf smoke tests (the Docker test skipped because no daemon answered, or failed), the redaction smoke did not pass, or the docs checker reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `N=$(git log --format=%H --diff-filter=A -- .planning/notes/2026-10-15-cooler-go-redis-otter.md) && G=$(git log --format=%H -S 'github.com/redis/go-redis/v9 v9.23.0' -- go.mod) && test -n "$N" && test -n "$G" && git merge-base --is-ancestor "$N" "$G"` exits 0 (one note commit, one go.mod commit, the note first).
- `grep -n 'github.com/redis/go-redis/v9 v9.23.0' go.mod` prints one line.
- `grep -n 'NewMemoryStore(2\*time.Minute), trusted' modules/surf/router.go` prints nothing and `grep -n 'Lookup\[Store\]' modules/surf/router.go` prints one line.
- `go list -deps ./modules/cooler | grep -E 'summercms/modules/(surf|cabana)$'` prints nothing.
- `grep -n '^| \[cooler\](modules/cooler/README.md)' README.md` prints one line, and the purpose cell equals the second non-empty line of modules/cooler/README.md.
- `grep -n 'http.rate_limit.store' docs/services/rate-limiting.md docs/setup/configuration.md modules/surf/README.md` prints at least one line per file.
</acceptance_criteria>
<done>Two application instances behind one Redis enforce one shared rate limit; route:list and tests never dial Redis; a Redis outage degrades to per-instance limits with a warning.</done>
</task>
<task type="auto">
<name>Task 2: Plugins and jobs cache values through cooler.Cache: otter memory driver, Redis driver under the app prefix, Install publishes it and FromApp serves other commands</name>
<reversibility rating="costly">D-08: the cache.driver, cache.prefix and cache.memory.max_entries keys and the Cache method set become what plugins and app configs depend on.</reversibility>
<files>go.mod, go.sum, modules/cooler/cooler.go, modules/cooler/memory.go, modules/cooler/rediscache.go, modules/cooler/smoke_test.go, modules/cooler/README.md, docs/services/caching.md, docs/setup/configuration.md</files>
<read_first>modules/cooler/cooler.go and modules/cooler/redis.go (Task 1), 19.3-RESEARCH.md (Code Examples: memory driver with per-entry TTL, Anti-Patterns: the database-wide flush), docs/services/rate-limiting.md (frontmatter and tone of a services page), docs/services/mail.md (a driver-selection page), modules/cooler/README.md (Task 1)</read_first>
<action>Per D-08 and D-09. `go get github.com/maypok86/otter/v2@v2.3.0` (named in the Task 1 note).
cooler.go: the `Cache` interface with `Get(ctx, key) ([]byte, bool, error)`, `Set(ctx, key string, value []byte, ttl time.Duration) error` (ttl 0 or less is forever), `Has(ctx, key) (bool, error)`, `Forget(ctx, key) error`, `Remember(ctx, key string, ttl time.Duration, fn func(context.Context) ([]byte, error)) ([]byte, error)` (degrades as this plan's Decisions say), `Increment(ctx, key string, by int64) (int64, error)` (a missing key starts at 0 and gets no TTL; an existing key keeps its expiry, as Redis INCRBY does), `Flush(ctx) error`. Install reads `cache.driver` (empty means memory; anything but memory or redis is a boot error naming the key), builds the driver, reuses a Cache that FromApp already published, otherwise publishes `app.Publish[Cache]`; `Handle.Cache()` returns it. When both the cache and the rate limiter use Redis they share one client. `FromApp(app *backpack.App) (Cache, error)` returns the published Cache or builds the configured one without a Ping and publishes it (on a duplicate-publish race it returns the winner).
memory.go: `NewMemoryCache(prefix string, maxEntries int) (Cache, error)` on otter v2 with MaximumSize from `cache.memory.max_entries` (default DefaultMaxEntries; zero or negative is a boot error naming the key) and an ExpiryCalculator reading an absolute expiresAt stored with the value (forever as an excessively long period, per otter's docs); Increment uses Compute so it is atomic per key; values are copied on Set and Get so callers cannot mutate stored bytes.
rediscache.go: `NewRedisCache(client *redis.Client, prefix string) Cache`; keys `<prefix>cache:<key>`; Set uses SET with PX (no expiry when ttl is 0 or less); Has uses EXISTS; Forget uses UNLINK; Increment uses INCRBY; Flush walks SCAN MATCH `<prefix>cache:*` in batches of 500 and removes each batch with UNLINK (never the database-wide flush command, which would also drop rate-limit counters and other apps' keys).
Docs in the same commit: new docs/services/caching.md (frontmatter section services, order 45, title Caching; how WinterCMS's Cache facade maps to cooler.Cache, the driver keys, the prefix and key layout, TTL semantics, Remember's degrade rule, FromApp in jobs and commands, the Flush scope; neutral names such as acme and blog); docs/setup/configuration.md rows `cache.driver`, `cache.prefix`, `cache.memory.max_entries`; cooler README Features, Usage, API reference, Configuration (every key with its default and SUMMER_ variable), Dependencies (go-redis v9, otter v2), Testing.
Smoke tests in modules/cooler/smoke_test.go: `TestMemoryCacheSmoke` (no Docker: Set, Get, Has, Increment from missing and existing, Remember calling the callback once, Forget, Flush, a 1 ms TTL expiring) and `TestRedisCacheSmoke` (skips under -short; redis:7-alpine; the same round trip, and Flush leaving a `<prefix>rl:` key and a foreign key in place).</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -run '^(TestMemoryCacheSmoke|TestRedisCacheSmoke)$' -v ./modules/cooler 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 2 &amp;&amp; go test -count=1 -short ./... &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; go run ./cmd/summer docs:build --check</automated>
<fails_when>Non-zero exit: vet or the short suite is red, fewer than two "--- PASS" lines for the cache smoke tests (the Redis test skipped without Docker, or failed), or the docs checker or docs:build --check reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'github.com/maypok86/otter/v2 v2.3.0' go.mod` prints one line.
- `grep -cE 'FlushDB|FlushAll' modules/cooler/rediscache.go` prints 0.
- `head -8 docs/services/caching.md | grep -c -E '^(section: services|order: 45)$'` prints 2.
- `grep -n 'cache.driver' docs/setup/configuration.md modules/cooler/README.md docs/services/caching.md` prints at least one line per file.
- `grep -rniE 'wavepath|fonoteka|grzyby' modules/cooler docs/services/caching.md` prints nothing.
</acceptance_criteria>
<done>A plugin or job reads and writes a prefixed cache that is in-process by default and shared through Redis when configured, with Flush limited to the app's cache keys.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| application to Redis | network hop carrying credentials in redis.url and shared counters other instances trust |
| request data to cache and limiter keys | keys can derive from client IPs, user ids or request values |
| config and environment to boot | redis.url and driver keys come from YAML or SUMMER_ variables |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-19.3-01 | Information disclosure | cooler.OpenRedis, Install errors and logs | high | mitigate | parse errors are replaced by a fixed message naming the key, never the URL or the raw url.Parse text; the Ping warning logs only the host; TestOpenRedisRedactsSmoke asserts user and password are absent |
| T-19.3-02 | Denial of service | RateLimitStore.Attempt during a Redis outage | medium | mitigate | per-instance MemoryStore fallback (no site-wide 429, no unlimited traffic), warning at most once per minute, 500 ms bound per attempt |
| T-19.3-03 | Tampering | cache and limiter key space on a shared Redis | medium | mitigate | `<prefix>cache:` and `<prefix>rl:` namespaces; Flush is SCAN plus UNLINK under the cache namespace only; go-redis sends keys as arguments (no command injection) |
| T-19.3-04 | Denial of service | memory cache growth from request-derived keys | medium | mitigate | otter MaximumSize from cache.memory.max_entries (default 10000), non-positive values refused at boot |
| T-19.3-05 | Information disclosure | route:list and tests dialing Redis | low | mitigate | only serve calls installCooler; BuildRouter only looks up; TestBuildRouterStoreSmoke |
| T-19.3-SC | Tampering | Go module installs (go-redis v9.23.0, otter v2.3.0) | high | mitigate | both named in the decision note before install, versions pinned, verified on proxy.golang.org in RESEARCH (Package Legitimacy Audit: OK); miniredis ([ASSUMED] name) is not installed here and waits for plan 05's blocking human check |
</threat_model>
<verification>
- Task 1: the note precedes the go.mod change; vet and the short cooler and surf suites; the two surf smoke tests (Docker for the cross-instance one); the redaction smoke; TestDocsTree.
- Task 2: the memory and Redis cache smokes; the whole short suite; TestDocsTree and docs:build --check.
- Feedback sampling (19.3-VALIDATION.md): after each commit `go vet ./... && go test -short ./modules/cooler ./modules/surf`.
</verification>
<success_criteria>
- SC-2: cooler ships memory and Redis drivers; surf uses a Redis-backed Store published at serve time; the decision note landed first.
- route:list never dials Redis; the memory store stays the default.
- `go vet ./...` and `go test -short ./...` green at every commit; the docs checker green.
</success_criteria>
<output>
Create `.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-01-SUMMARY.md` when done, with the commits (note first), the identifiers added, the Docker smoke tests run or NOT RUN, and any deviation.
</output>

View File

@@ -0,0 +1,256 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 01
subsystem: infra
tags: [cooler, redis, go-redis, otter, cache, rate-limit, surf]
requires:
- phase: 19.3 (planning)
provides: CONTEXT D-06..D-09/D-19, RESEARCH Pattern 1/2 (boot wiring, Lua fixed-window script)
provides:
- "decision note .planning/notes/2026-10-15-cooler-go-redis-otter.md (go-redis v9.23.0, otter v2.3.0, miniredis test-only, s3blob)"
- "module cooler: Cache (memory/otter + Redis drivers), RateLimitStore (Redis surf.Store), Install, FromApp, OpenRedis"
- "serve installs cooler before BuildRouter; BuildRouter uses a published surf.Store"
- "config keys redis.url, cache.driver, cache.prefix, cache.memory.max_entries, http.rate_limit.store"
affects: [19.3-02, 19.3-03, 19.3-04, 19.3-05]
actuals:
tokens: 17150
tasks: 2
commits: 3
plan_head_before: 7357525b0b4f00560dc9f200c67827f8f16e0795
plan_head_after: fd7f204979f7ee73decf1e8ceb29bc99310c34ec
tech-stack:
added: ["github.com/redis/go-redis/v9 v9.23.0", "github.com/maypok86/otter/v2 v2.3.0"]
patterns:
- "Structural interface (cooler.Attempter == surf.Store method set) so surf imports cooler, never the reverse"
- "serve-only service install: publish on backpack before BuildRouter; route:list/tests keep defaults"
- "Secret-safe Redis URL handling: fixed error naming the key, logs only the endpoint"
key-files:
created:
- .planning/notes/2026-10-15-cooler-go-redis-otter.md
- modules/cooler/cooler.go
- modules/cooler/redis.go
- modules/cooler/ratelimit.go
- modules/cooler/memory.go
- modules/cooler/rediscache.go
- modules/cooler/smoke_test.go
- modules/cooler/example_test.go
- modules/cooler/README.md
- modules/surf/store_lookup_test.go
- docs/services/caching.md
modified:
- go.mod
- go.sum
- modules/surf/serve.go
- modules/surf/router.go
- modules/surf/README.md
- README.md
- docs/services/rate-limiting.md
- docs/setup/configuration.md
- docs/architecture/performance-and-scaling.md
key-decisions:
- "Memory cache driver is otter v2 (bounded MaximumSize); a stdlib map was rejected as unbounded"
- "An unreachable Redis at start-up is a warning; a missing/malformed redis.url with a Redis driver selected is a boot error naming the key"
- "Key prefix: cache.prefix, else slug(app.name)+':' (non [a-z0-9] runs -> '_', trimmed), else 'summer:'; cache keys <prefix>cache:, limiter keys <prefix>rl:"
- "Remember degrades: a failed read still runs the callback and skips the write; a failed write is skipped; warnings at most once per minute"
- "Cache and rate limiter share one Redis client when both select Redis; FromApp builds lazily without a ping"
- "Redis Flush escapes glob characters of the prefix before SCAN MATCH so it matches only this app's cache keys"
patterns-established:
- "cooler.FromApp for commands outside serve (dedicated job worker, console commands)"
- "Docker-gated smoke tests with testcontainers.Run(redis:7-alpine) and a testing.Short() skip"
requirements-completed: [SC-2, SC-4, D-06, D-07, D-08, D-09, D-19]
coverage:
- id: D1
description: "Decision note naming go-redis, otter, miniredis (test-only) and s3blob lands before the first go.mod change"
requirement: D-07
verification:
- kind: other
ref: "git merge-base --is-ancestor <note commit 77dd11c> <go.mod commit fa4d89d>"
status: pass
human_judgment: false
- id: D2
description: "Two serve instances sharing one Redis count one throttle:2,1 window (200, 200, 429 with Retry-After)"
requirement: D-09
verification:
- kind: integration
ref: "modules/surf/store_lookup_test.go#TestRateLimitAcrossInstancesSmoke (Docker, redis:7-alpine)"
status: pass
human_judgment: false
- id: D3
description: "BuildRouter uses a published surf.Store, otherwise MemoryStore; selecting Redis without install never dials"
requirement: D-09
verification:
- kind: unit
ref: "modules/surf/store_lookup_test.go#TestBuildRouterStoreSmoke"
status: pass
human_judgment: false
- id: D4
description: "redis.url parse errors never contain the URL, user or password; rediss:// parses with TLS"
requirement: D-09
verification:
- kind: unit
ref: "modules/cooler/smoke_test.go#TestOpenRedisRedactsSmoke"
status: pass
human_judgment: false
- id: D5
description: "cooler.Cache memory driver round trip (Set/Get/Has/Increment/Remember once/Forget/Flush/1 ms TTL, copies, max_entries refusal)"
requirement: D-08
verification:
- kind: unit
ref: "modules/cooler/smoke_test.go#TestMemoryCacheSmoke"
status: pass
human_judgment: false
- id: D6
description: "cooler.Cache Redis driver round trip; Flush leaves <prefix>rl: and foreign keys in place"
requirement: D-08
verification:
- kind: integration
ref: "modules/cooler/smoke_test.go#TestRedisCacheSmoke (Docker, redis:7-alpine)"
status: pass
human_judgment: false
- id: D7
description: "Redis outage: rate limiter falls back to per-instance memory with a rate-limited warning"
requirement: D-09
verification:
- kind: manual_procedural
ref: "ad-hoc test against redis://127.0.0.1:1 (fallback admitted 2 attempts, ~500 ms each); full test in plan 05"
status: pass
human_judgment: true
rationale: "Only an ad-hoc run proved the outage path; plan 05 owns the committed miniredis/outage tests"
- id: D8
description: "cooler README, root table row, surf README, caching/rate-limiting/configuration/scaling docs"
requirement: SC-4
verification:
- kind: other
ref: "go test ./cmd/summer -run TestDocsTree; summer docs:build --check"
status: pass
human_judgment: false
duration: 14min
completed: 2026-10-15
status: complete
---
# Phase 19.3 Plan 01: cooler cache and Redis rate-limit store Summary
**New `cooler` module: a Redis fixed-window `surf.Store` (atomic Lua script, per-instance memory fallback) that `serve` publishes before the router, plus `cooler.Cache` with an otter v2 memory driver and a go-redis v9 driver under a per-app key prefix**
## Performance
- **Duration:** 14 min
- **Started:** 2026-10-15T15:27:12Z
- **Completed:** 2026-10-15T15:41:00Z
- **Tasks:** 2 (Task 1 tracer, Task 2 expansion)
- **Files modified:** 20
## Accomplishments
- Decision note committed first (77dd11c), ahead of the go.mod change (fa4d89d): go-redis v9.23.0, otter v2.3.0, miniredis v2.40.0 test-only (plan 05, after a human check), s3blob behind a build tag (plan 03).
- `cooler.RateLimitStore`: one Lua script per attempt with `surf.MemoryStore` semantics (first attempt opens the window, denied attempts do not increment, the window never extends), keys `<prefix>rl:<key>`, 500 ms bound, fallback to a per-instance `surf.MemoryStore` with a warning at most once per minute.
- `serve` calls `installCooler` after `publishUploads` and before `BuildRouter`; `BuildRouter` takes `app.Lookup[Store]()` or falls back to `NewMemoryStore(2*time.Minute)`. `route:list` is unchanged and never dials Redis.
- `cooler.Cache` (Get, Set, Has, Forget, Remember, Increment, Flush). The memory driver uses otter, bounded by `cache.memory.max_entries` (default 10000). The Redis driver uses SET PX, EXISTS, UNLINK and INCRBY. Its Flush runs SCAN MATCH `<prefix>cache:*` in batches of 500 and removes each batch with UNLINK. It never calls FLUSHDB.
- `cooler.Install` validates `cache.driver` / `http.rate_limit.store`, shares one Redis client, pings once (warning on failure) and publishes the Cache; `cooler.FromApp` serves jobs and commands lazily.
- Docs: new `docs/services/caching.md` (order 45), the section "Sharing limits across instances" in `docs/services/rate-limiting.md`, configuration rows, the scaling note, the cooler README in the standard structure, surf README rows, and the root table row.
## Task Commits
1. **Decision note (Task 1, D-07 ordering)** - `77dd11c` (docs)
2. **Task 1: Tracer: Redis surf.Store shared by two serve instances** - `fa4d89d` (feat)
3. **Task 2: cooler.Cache with otter memory and Redis drivers** - `fd7f204` (feat)
**Plan metadata:** see the final docs(19.3-01) commit.
## Identifiers added
- cooler: `Cache` (methods `Get`, `Set`, `Has`, `Forget`, `Remember`, `Increment`, `Flush`), `Attempter`, `Handle` (`Close`, `Cache`, `RateLimitRedis`, `RateLimitStore`), `Install`, `FromApp`, `OpenRedis`, `NewMemoryCache`, `NewRedisCache`, `RateLimitStore` / `NewRateLimitStore` / `Attempt`, constants `RedisURLKey`, `CacheDriverKey`, `CachePrefixKey`, `CacheMaxEntriesKey`, `RateLimitStoreKey`, `DefaultMaxEntries`, `PingTimeout`, `AttemptTimeout`.
- surf: unexported `installCooler(ctx, app) (*cooler.Handle, error)`; `BuildRouter` store lookup.
- Tests: `TestBuildRouterStoreSmoke`, `TestRateLimitAcrossInstancesSmoke`, `TestOpenRedisRedactsSmoke`, `TestMemoryCacheSmoke`, `TestRedisCacheSmoke`, `ExampleFromApp`.
## Docker smoke tests
All run on this host (Docker daemon answered): `TestRateLimitAcrossInstancesSmoke` PASS, `TestRedisCacheSmoke` PASS (redis:7-alpine via testcontainers). The non-short `go test ./modules/cooler/... ./modules/surf/...` run is green.
## Files Created/Modified
- `.planning/notes/2026-10-15-cooler-go-redis-otter.md`: dependency decision note.
- `modules/cooler/cooler.go`: config keys, the Cache interface, Handle, Install, FromApp, the prefix helper, the shared warner and the Remember helper.
- `modules/cooler/redis.go`: OpenRedis (secret-safe parse errors, handshakes disabled, no dial).
- `modules/cooler/ratelimit.go`: the Lua fixed-window RateLimitStore with its fallback.
- `modules/cooler/memory.go`: the otter v2 memory driver.
- `modules/cooler/rediscache.go`: the Redis driver with a prefix-scoped Flush.
- `modules/cooler/smoke_test.go`, `modules/cooler/example_test.go`: smoke tests and the docs example.
- `modules/surf/serve.go`, `modules/surf/router.go`: `installCooler` and the store lookup.
- `modules/surf/store_lookup_test.go`: the surf smoke tests.
- READMEs and docs listed in key-files.
## Decisions Made
See key-decisions in the frontmatter. All are from the plan's "Claude's discretion" list, except the glob escaping in Flush and the trimmed slug, which are listed under deviations.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] The docs checker refuses Go code blocks without a `src=` reference**
- **Found during:** Task 2 (docs/services/caching.md)
- **Issue:** `TestDocsTree` failed with "go code block has no src= reference" on the caching guide's inline example.
- **Fix:** Added `modules/cooler/example_test.go` (`ExampleFromApp`, a verified Output) and pointed the guide at it with `src=modules/cooler/example_test.go#ExampleFromApp`, then ran `summer docs:sync`.
- **Files modified:** modules/cooler/example_test.go, docs/services/caching.md
- **Verification:** TestDocsTree and `docs:build --check` are green.
- **Committed in:** fd7f204
**2. [Rule 2 - Missing critical] SCAN MATCH treats the prefix as a pattern**
- **Found during:** Task 2 (rediscache.go Flush)
- **Issue:** A `cache.prefix` that contains `*`, `?`, `[` or `]` would let Flush match, and UNLINK, keys of other prefixes.
- **Fix:** `escapeGlob` escapes those characters before `SCAN MATCH <prefix>cache:*`.
- **Files modified:** modules/cooler/rediscache.go
- **Verification:** TestRedisCacheSmoke (a foreign key and the `rl:` key survive Flush).
- **Committed in:** fd7f204
**3. [Rule 1 - Acceptance gate] The rate-limiting guide did not name `http.rate_limit.store`**
- **Found during:** Task 1 acceptance criteria (one grep line per file was required).
- **Issue:** The guide showed only the YAML form (`rate_limit: store: redis`).
- **Fix:** The guide now names the dotted key and its default. The fix was amended into the unpushed Task 1 commit.
- **Committed in:** fa4d89d
**4. [Discretion refinement] Prefix slug and helper visibility**
- The slug of `app.name` also trims `_` at both ends (`Acme!` gives `acme:`, not `acme_:`). This is documented in the README and the caching guide.
- The prefix helper is unexported (`keyPrefix`) because the plan's artifact list names no exported prefix function.
**5. [Interpretation] Remember after a failed read**
- When the read fails, Remember runs the callback and returns its value without attempting the write, because the backend that just failed would very likely fail the write too. When the read succeeds with a miss and the write then fails, the failed write is skipped with a warning. Both cases match the plan's degrade rule.
---
**Total deviations:** 3 auto-fixed (1 blocking, 1 missing critical, 1 acceptance gate) plus 2 documented discretion choices.
**Impact on plan:** No scope creep. The identifiers, keys and behaviour match the plan.
## Issues Encountered
- During a Redis outage each `Attempt` waits up to `AttemptTimeout` (500 ms), because go-redis retries its dial within that bound before cooler falls back. go-redis also prints its own `connection pool: failed to dial` line for every failure, independent of cooler's once-a-minute warning. Both are within the plan (T-19.3-02 accepts a 500 ms bound). Plan 05 or a follow-up could add a short circuit-breaker and fewer dial retries if outage latency matters.
## Known Stubs
None.
## User Setup Required
None. Redis is opt-in through `http.rate_limit.store: redis` / `cache.driver: redis` and `redis.url`.
## Next Phase Readiness
- Plans 02, 03 and 04 can build on this commit. Plan 03 relies on the s3blob entry in the note. Plan 04 adds the site-cache purge to cooler.
- Plan 05 owns full unit coverage: a miniredis parity table for RateLimitStore against MemoryStore, the outage fallback, the Install/FromApp edge cases and the Flush glob escaping.
---
*Phase: 19.3-framework-transport-and-infrastructure-for-wavepath*
*Completed: 2026-10-15*
## Self-Check: PASSED

View File

@@ -0,0 +1,287 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 02
type: execute
wave: 2
depends_on: ["19.3-01"]
files_modified:
- modules/bouncer/cookie.go
- modules/bouncer/turnstile.go
- modules/bouncer/README.md
- modules/surf/cors.go
- modules/surf/router.go
- modules/surf/apidocs.go
- modules/surf/transport_smoke_test.go
- modules/surf/README.md
- docs/services/authentication.md
- docs/services/routing.md
autonomous: true
requirements: [SC-1, SC-3, SC-4, D-01, D-02, D-03, D-04, D-05, D-16, D-17, D-18, D-19]
estimate:
tokens: 190000
raw_tokens: 190000
tasks: 4
confidence: low
must_haves:
truths:
- "D-01: bouncer ships config-driven cookie builders (CookieConfig.TokenCookie, CookieConfig.ForgetTokenCookie) and optional handlers (CookieRefreshHandler, CookieLogoutHandler); the refresh handler reads the token cookie, calls the app-supplied Refresh func (bouncer.Refresh or RefreshAudienceFor, which rotate the token and blacklist the old JTI), sets the rotated cookie and hands the result to the app's writer func; sm-user-plugin is not changed."
- "D-02: cookie attributes come from one block auth.cookie.* (name, domain, secure, same_site, path); Max-Age equals the refresh window the app passes (the guard's refresh_ttl, 20160 min by default), never the access ttl; the per-guard ttl and refresh_ttl keys (golem15.user.jwt.*, admin.jwt.*) stay where they are."
- "D-03: with auth.cookie unset the cookie is token, Path=/, HttpOnly, SameSite=Lax, host-only (no Domain), Secure exactly when app.url is https; an invalid domain, same_site none without Secure, or secure false in the production environment is an error from LoadCookieConfig that names the key (bouncer imports no framework module: LoadCookieConfig takes a three-method interface that *compass.Config satisfies)."
- "D-04: http.cors with supports_credentials true and a * origin (global or any profile) fails boot naming the key; an allowed_origins_patterns entry that does not compile, or a PHP-delimited one such as #^...$#, fails boot; a * inside an allowed_origins entry becomes an anchored pattern; a dynamic origin answer (credentials, patterns or several origins) echoes the exact request origin and sends Vary: Origin on every response of a CORS path, preflight and disallowed origin included; the existing * non-credentialed behaviour stays byte-identical (TestCORSOptionsMatchesLaravel green); a www origin calling an api host with credentials works end to end."
- "D-05: a named middleware cors:<profile> reads http.cors_profiles.<profile> (the http.cors keys without paths); every route carrying it gets an OPTIONS route that answers the preflight from the profile; an unknown profile, a profile with paths, or a profiled route whose path also matches http.cors.paths fails boot."
- "D-18: bouncer.Turnstile.Verify(ctx, token, ip, failClosed) posts secret, response and remoteip as a form to Cloudflare siteverify with a 5 s timeout; open mode passes on an empty secret; closed mode answers 503 when unconfigured in the production environment and 422 on a missing token, success false, an error or a timeout; a named middleware turnstile:open|closed reads turnstile_token (or turnstile.field) from the query, a JSON body or a form body, restores the body for the handler and writes errors through an app-published bouncer.TurnstileWriter or a neutral default; the secret comes from turnstile.secret (SUMMER_TURNSTILE__SECRET)."
- "D-16, D-17: with http.api_docs.enabled true and exactly one plugin implementing surf.APIDocsProvider, GET /api-docs (http.api_docs.path) serves a Swagger UI page loading swagger-ui-dist 5.33.1 from jsDelivr with sha384 SRI and crossorigin on all three assets, and GET /api-docs.json serves the provider's YAML or JSON spec converted to compact JSON once at boot in source key order; both are public; off by default (404); zero or several providers with the flag on fail boot."
artifacts:
- path: "modules/bouncer/cookie.go"
provides: "CookieConfig, LoadCookieConfig, TokenCookie, ForgetTokenCookie, CookieRefreshHandler, CookieLogoutHandler"
contains: "func LoadCookieConfig"
- path: "modules/bouncer/turnstile.go"
provides: "Turnstile verifier, verdicts, middleware, writer type"
contains: "challenges.cloudflare.com/turnstile/v0/siteverify"
- path: "modules/surf/cors.go"
provides: "validated CORS config, shared policy, profile middleware and preflight handler"
contains: "cors_profiles"
- path: "modules/surf/apidocs.go"
provides: "APIDocsProvider, the pinned Swagger UI page and the spec route"
contains: "5.33.1"
key_links:
- from: "modules/surf/router.go"
to: "modules/surf/cors.go"
via: "BuildRouter registers the cors factory; compile registers OPTIONS routes for routes whose chain names cors:<profile>"
pattern: "\"cors\""
- from: "modules/surf/router.go"
to: "modules/bouncer/turnstile.go"
via: "BuildRouter registers the turnstile factory with a bouncer.Turnstile built from turnstile.secret and the environment, ClientIP with trusted proxies"
pattern: "\"turnstile\""
- from: "modules/bouncer/cookie.go"
to: "modules/bouncer/refresh.go"
via: "the app's Refresh func wraps bouncer.Refresh or RefreshAudienceFor; the handler sets TokenCookie(newToken, refreshTTL)"
pattern: "TokenCookie"
- from: "modules/surf/router.go"
to: "modules/surf/apidocs.go"
via: "BuildRouter calls mountAPIDocs after mountUploads"
pattern: "mountAPIDocs"
prohibitions:
- statement: "bouncer still imports no SummerCMS module."
status: unresolved
verification: judgment
- statement: "Existing non-credentialed * CORS answers (all four apps today) stay byte-identical."
status: unresolved
verification: judgment
- statement: "Neither the Turnstile secret nor a token ever appears in a log line or error."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not in user-story format; no story is invented, see plan 01): the framework has the transport and infrastructure pieces a multi-client, multi-instance application needs: JWT cookie refresh across subdomains, credentialed CORS, a cache module with a Redis-backed rate-limit store, an S3 blob option, an OpenAPI page and Turnstile verification.
This plan's slice (plan 2 of 5, wave 2): SC-1 and two thirds of SC-3. Cookie transport in bouncer with hardened credentialed CORS (the tracer: a www page refreshes its session on the api host), named CORS profiles, Turnstile verification, and the opt-in /api-docs page.
## Spec-less probe fallback
Skipped visibly: the phase has no requirement IDs to probe, so no probe predicates were generated (see plan 01).
## Decisions taken in this plan (Claude's discretion)
- Cookie builders are methods on `bouncer.CookieConfig`, not guard methods: one config serves every guard, and the refresh window is passed per call (the guard's `refresh_ttl`).
- `bouncer.LoadCookieConfig` takes `bouncer.CookieConfigSource` (`Lookup`, `String`, `Environment`), which `*compass.Config` satisfies structurally, so bouncer keeps importing no framework module (pattern-mapper flag 1). Apps call it in their plugin Boot and return its error, which fails boot on a bad cookie config.
- The handlers return `(http.Handler, error)` so missing options are a boot error, not a request-time panic. The refresh handler reads only the cookie (as PHP does); a missing cookie or a refused refresh answers 401 through the writer and leaves the cookie alone. The logout handler revokes through the app's Revoke func when a cookie is present, always sends the forget cookie, and answers 200.
- CORS profiles live under `http.cors_profiles.<name>`; the middleware name is `cors:<name>`.
- Turnstile open mode with a secret set: a transport error or timeout is a 422 failure (PHP lets the exception surface as a 500); a missing token is a 422 without a siteverify call (PHP sends it and Cloudflare answers false). A token longer than 2048 characters fails without a call. Both are recorded here as deliberate, outcome-preserving divergences.
- The default Turnstile error bodies are neutral JSON: 503 `{"error":"turnstile_unavailable"}`, 422 `{"error":"turnstile_failed"}`; an app keeps its PHP envelope by publishing a `bouncer.TurnstileWriter`.
- The api-docs spec is converted at boot; a spec that does not parse fails boot naming the plugin (PHP answers 500 per request). The page sets a per-request CSP nonce for its one inline script.
- Plan 02's config keys are documented in the surf and bouncer READMEs and the services pages; the curated key table in docs/setup/configuration.md is left to plans 01 and 03 so wave-2 plans never edit the same file.
## Artifacts this phase produces (this plan)
- `modules/bouncer/cookie.go`: `CookieConfig` (fields `Name`, `Domain`, `Path`, `Secure`, `SameSite`), `CookieConfigSource`, `LoadCookieConfig`, `DefaultCookieName` (`token`), methods `TokenCookie(token string, refreshTTL time.Duration) *http.Cookie` and `ForgetTokenCookie() *http.Cookie`, `CookieResult` (`Status`, `Token`, `Err`), `CookieRefreshOptions` (`Cookie`, `RefreshTTL`, `Refresh`, `Write`), `CookieLogoutOptions` (`Cookie`, `Revoke`, `Write`), `CookieRefreshHandler`, `CookieLogoutHandler`, `ErrNoTokenCookie`.
- `modules/bouncer/turnstile.go`: `Turnstile` (fields `Secret`, `Production`, `Endpoint`, `Client`), `NewTurnstile`, `TurnstileSiteverifyURL`, `TurnstileTimeout` (5 s), `TurnstileMaxTokenLength` (2048), `DefaultTurnstileField` (`turnstile_token`), `TurnstileVerdict` with `TurnstilePassed`, `TurnstileFailed`, `TurnstileUnavailable`, method `Verify`, method `Middleware`, `TurnstileWriter`.
- `modules/surf/cors.go`: `LoadCORSProfiles`, validation inside `LoadCORSConfig`, profile middleware and preflight handler; `modules/surf/router.go`: factories `cors` and `turnstile`, OPTIONS routes for profiled routes; `modules/surf/apidocs.go`: `APIDocsProvider`, `SwaggerUIVersion` (`5.33.1`), unexported `mountAPIDocs`.
- Config keys: `auth.cookie.name`, `auth.cookie.domain`, `auth.cookie.path`, `auth.cookie.secure`, `auth.cookie.same_site`; `http.cors_profiles.<name>.*`; `turnstile.secret`, `turnstile.field`; `http.api_docs.enabled`, `http.api_docs.path`, `http.api_docs.title`.
- Named middleware `cors:<profile>`, `turnstile:open`, `turnstile:closed`; routes `GET /api-docs` and `GET /api-docs.json` (opt-in).
- Smoke tests in `modules/surf/transport_smoke_test.go`: `TestCookieRefreshAcrossSubdomainsSmoke`, `TestCORSCredentialsWildcardRefusedSmoke`, `TestCORSProfileSmoke`, `TestTurnstileMiddlewareSmoke`, `TestAPIDocsSmoke`. Full unit coverage is plan 05.
- Docs: `docs/services/authentication.md` sections "Cookie transport" and "Turnstile"; `docs/services/routing.md` CORS hardening, "CORS profiles", "API documentation page"; bouncer and surf READMEs.
<objective>
Give the framework cookie-based token refresh across subdomains, credentialed CORS that refuses unsafe configs (globally and per route group), Cloudflare Turnstile verification as middleware, and an opt-in Swagger UI page with its JSON spec, each with README and docs in the same commit.
Purpose: SC-1, the api-docs and Turnstile parts of SC-3, and D-01..D-05, D-16..D-18 (placement per D-19).
Output: bouncer and surf changes with smoke tests and docs.
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-CONTEXT.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-RESEARCH.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-PATTERNS.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/COVERAGE.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-01-SUMMARY.md
Concurrency: plans 19.3-03 and 19.3-04 run in the same wave in this checkout and own other files (lagoon/attach, internal/build, go.mod, cooler, pact, cabana). Before Task 1 run `git status --porcelain -- modules/bouncer modules/surf docs/services/authentication.md docs/services/routing.md`; if it lists a file this plan touches that this plan did not change, stop and report. A transient vet or test failure in a package another wave-2 plan owns is not this plan's: re-run once after a short pause, then report it instead of editing that package. Stage files by explicit path; no co-author tags. PHP reference files live under /media/nvme/dev/jz/wavepath.org; read them for behaviour, never name that application in framework code or docs.
<interfaces>
Existing seams (read; extend, do not fork):
- `modules/bouncer/refresh.go`: `Refresh(secret, tokenString string, refreshTTL time.Duration, bl BlacklistStore, grace time.Duration, issuerURL string) (string, error)`, `RefreshAudienceFor(ctx, users, secret, tokenString, audience, refreshTTL, bl, grace, issuerURL)`, `VerifyRefreshableClaimsAudience` (logout of a token past its access lifetime).
- `modules/bouncer/mint.go`: `Mint(secret, sub, issuerURL string, ttl time.Duration) (token, jti string, err error)`; `modules/bouncer/blacklist.go`: `NewMemoryBlacklist`, `IsBlacklisted`.
- `modules/cabana/auth.go` :503-517: refusal of `cookie_secure: false` in the production environment (`app.Config.Environment()`).
- `modules/surf/cors.go`: `CORSConfig` and koanf tags, `LoadCORSConfig`, `pathScopedCORS`, `writeOptions` (Laravel preflight bytes), `corsAllowOrigin`, `compileLaravelGlob`; `modules/surf/cors_coverage_test.go` :99-147 `TestCORSOptionsMatchesLaravel`.
- `modules/surf/router.go`: `RegisterMiddlewareFactory`, `wrap` (factory lookup by `base:param`, throttle validation at :421 is the model for validating cors and turnstile params at boot), `compile` (:345, ServeMux registration, overlap families, `handleRoute` conflict to error), `BuildRouter` (:468), `BucketProvider` type assertion (:538) as the model for `APIDocsProvider`; `modules/surf/uploads.go` `mountUploads` and `uploadsPrefix` (opt-in route, prefix validation, conflict check, `GroupRaw`).
- `modules/surf/clientip.go`: `ClientIP(r, trusted)`, `TrustedProxies(cfg)`.
- PHP: `AuthApiController.php` `tokenCookie`/`forgetTokenCookie` (:342-381), `TurnstileVerifier.php`, `ApiDocsController.php`, php-cors `CorsService.php` :100-131 (wildcard to pattern), :204-220 (origin answer).
- RESEARCH Pattern 3 (cookies), Pattern 4 (CORS), Pattern 7 (Turnstile), Pattern 9 (api-docs, SRI hashes for 5.33.1), Pitfalls 1-5, 12.
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: a page on www refreshes its session on the api host: credentialed CORS answers the exact origin and the cookie refresh handler rotates the httpOnly token cookie and blacklists the old JTI</name>
<reversibility rating="costly">D-02: apps' config files and deploy docs reference the auth.cookie.* keys once they ship.</reversibility>
<files>modules/bouncer/cookie.go, modules/bouncer/README.md, modules/surf/cors.go, modules/surf/router.go, modules/surf/transport_smoke_test.go, modules/surf/README.md, docs/services/authentication.md, docs/services/routing.md</files>
<read_first>modules/bouncer/refresh.go, modules/bouncer/jwt.go (extractToken and cookie names), modules/bouncer/mint.go, modules/bouncer/blacklist.go, modules/bouncer/README.md, modules/cabana/auth.go (:440-520), modules/compass/config.go (Lookup, String, Environment), modules/surf/cors.go, modules/surf/cors_test.go, modules/surf/cors_coverage_test.go, modules/surf/router.go (BuildRouter), modules/surf/README.md (Configuration), docs/services/authentication.md (Refreshing and revoking), docs/services/routing.md (CORS), /media/nvme/dev/jz/wavepath.org/plugins/golem15/wavepath/controllers/api/AuthApiController.php (:280-400), /media/nvme/dev/jz/wavepath.org/vendor/fruitcake/php-cors/src/CorsService.php (:100-131, :204-220), 19.3-RESEARCH.md (Pattern 3, Pattern 4, Pitfalls 1-4)</read_first>
<action>bouncer/cookie.go per D-01..D-03: `CookieConfigSource` with `Lookup(path string) (any, bool)`, `String(path string) string`, `Environment() string`. `LoadCookieConfig(src CookieConfigSource) (CookieConfig, error)` reads auth.cookie.name (default DefaultCookieName; must be a valid cookie name), auth.cookie.path (default /; must start with /), auth.cookie.domain (default empty, host-only; a non-empty value must pass (*http.Cookie).Valid, which also catches the silent drop of Pitfall 1), auth.cookie.same_site (lax, strict or none, case-insensitive; default lax), auth.cookie.secure (unset: true exactly when app.url has the https scheme; an explicit false in the production environment is an error, as cabana refuses backend.cookie_secure false); same_site none without Secure is an error. Every error names the key and is prefixed `bouncer:`. `TokenCookie(token, refreshTTL)` sets Name, Value, Path, Domain, MaxAge refreshTTL in whole seconds, Expires now plus refreshTTL, HttpOnly true, Secure and SameSite from the config (D-02: Max-Age is the refresh window). `ForgetTokenCookie()` has the same attributes, an empty value, MaxAge -1 and Expires at Unix 0. `CookieRefreshHandler(opts CookieRefreshOptions) (http.Handler, error)` refuses a nil Refresh or Write or a non-positive RefreshTTL; per request it reads the cookie named opts.Cookie.Name (missing or empty: Write with 401 and ErrNoTokenCookie), calls Refresh(r.Context(), token) (error: Write with 401 and the error, cookie untouched), otherwise http.SetCookie with TokenCookie(newToken, RefreshTTL) and Write with 200 and the new token. `CookieLogoutHandler(opts CookieLogoutOptions) (http.Handler, error)` refuses a nil Revoke or Write; when the cookie is present it calls Revoke and passes a revoke error to Write in the result; it always sets ForgetTokenCookie and writes 200. bouncer gains no framework import.
surf/cors.go per D-04: LoadCORSConfig validates and returns errors naming http.cors keys: supports_credentials true with a * entry in allowed_origins; an allowed_origins_patterns entry that does not compile, or one wrapped in PHP delimiters (# or / at both ends), with a hint to drop the delimiters; it translates an allowed_origins entry containing * (other than the bare *) into an anchored pattern the way php-cors convertWildcardToPattern does (quote the rest, * becomes any characters). Factor pathScopedCORS's per-request logic into an unexported policy type that both the global wrapper and Task 2's profiles reuse. When the origin answer is dynamic (credentials true, any pattern, or more than one listed origin) every response on a CORS path, preflight and disallowed origin included, carries Vary: Origin (combined with the existing preflight Vary values), and an allowed origin is echoed exactly. A non-dynamic config (bare * without credentials, the four apps' setting) keeps today's bytes. BuildRouter already returns LoadCORSConfig's error.
Docs in the same commits: bouncer README (Features, Usage example of an app mounting the refresh handler with RefreshAudienceFor and its own envelope writer, API reference rows for every new identifier, Configuration for auth.cookie.*); docs/services/authentication.md section "Cookie transport" (defaults, why Max-Age is the refresh window, Domain for api and www subdomains and the trust boundary it creates, CSRF note: refresh and logout accept POST only, prefer a JSON body or custom header, SameSite Lax treats sibling subdomains as same-site); docs/services/routing.md CORS section (credentials rules, exact-origin echo and Vary, wildcard entries, pattern errors, the api and www example); surf README Configuration rows updated for the hardened keys. Neutral host names (example.com).
Smoke tests in modules/surf/transport_smoke_test.go: `TestCookieRefreshAcrossSubdomainsSmoke` (config: app.url https://api.example.com, auth.cookie.domain example.com, http.cors paths api/*, allowed_origins https://www.example.com, methods and headers *, supports_credentials true; a test plugin route POST /api/auth/refresh on CookieRefreshHandler whose Refresh wraps bouncer.Refresh with a MemoryBlacklist and a 20160-minute window; a token from bouncer.Mint in the cookie; Origin https://www.example.com: 200, Set-Cookie token with a new value, Domain=example.com, Path=/, Max-Age=1209600, HttpOnly, Secure, SameSite=Lax, Access-Control-Allow-Origin equal to the origin, Allow-Credentials true, Vary containing Origin, the old jti blacklisted; the OPTIONS preflight answers 204 with Vary containing Origin) and `TestCORSCredentialsWildcardRefusedSmoke` (credentials plus * makes BuildRouter fail with an error naming http.cors).</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/bouncer ./modules/surf &amp;&amp; go test -count=1 -short -run '^(TestCookieRefreshAcrossSubdomainsSmoke|TestCORSCredentialsWildcardRefusedSmoke|TestCORSOptionsMatchesLaravel)$' -v ./modules/surf 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 3 &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit: vet or the bouncer or surf short suite is red, fewer than three "--- PASS" lines (a smoke test or the Laravel preflight parity test failed or did not run), or the docs checker reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `go list -f '{{join .Imports "\n"}}' ./modules/bouncer | grep -c 'golem15/summercms/modules'` prints 0.
- `grep -n 'func LoadCookieConfig(src CookieConfigSource) (CookieConfig, error)' modules/bouncer/cookie.go` prints one line.
- `grep -n 'auth.cookie.same_site' modules/bouncer/README.md docs/services/authentication.md` prints at least one line per file.
- `grep -n 'supports_credentials' docs/services/routing.md` prints at least two lines.
- `grep -rniE 'wavepath' modules/bouncer modules/surf docs/services/authentication.md docs/services/routing.md` prints nothing.
</acceptance_criteria>
<done>A browser on www keeps its session alive through the api host with an httpOnly cookie, and a credentialed CORS config that would reflect any origin cannot boot.</done>
</task>
<task type="auto">
<name>Task 2: A route group carries its own credentialed CORS policy through cors:&lt;profile&gt;, with preflights answered by an OPTIONS route the router registers</name>
<reversibility rating="costly">D-05: http.cors_profiles.* keys and the cors:&lt;profile&gt; middleware name are what plugin route declarations and app configs reference.</reversibility>
<files>modules/surf/cors.go, modules/surf/router.go, modules/surf/transport_smoke_test.go, modules/surf/README.md, docs/services/routing.md</files>
<read_first>modules/surf/cors.go (Task 1 policy type), modules/surf/router.go (wrap :399-456, compile :345-387, handleRoute, overlap family registration), modules/surf/overlap.go, modules/surf/limiter.go (ValidateThrottle as the boot-validation model), docs/services/routing.md (Middleware and CORS sections), 19.3-RESEARCH.md (Pattern 4 profiles, Pitfall 5)</read_first>
<action>Per D-05. cors.go: `LoadCORSProfiles(cfg *compass.Config) (map[string]CORSConfig, error)` reads every child of http.cors_profiles with the CORSConfig keys; a profile that sets paths is an error; each profile goes through the Task 1 validation with errors naming http.cors_profiles.<name>.<key>; profile names follow the plugin-id character set (lower-case letters, digits, underscore and dash). Add an unexported middleware built from a profile's policy that, for non-OPTIONS requests, sets the same headers the global wrapper sets for an allowed origin (and Vary: Origin when dynamic), and an unexported preflight handler that answers OPTIONS from the profile with writeOptions semantics.
router.go: BuildRouter loads the profiles and registers factory `cors` (plugin surf); wrap validates the param like throttle: an unknown profile fails boot naming the plugin, the route and the profile. compile registers, for every route whose middleware list contains cors:<p>, one `OPTIONS <path>` route on the preflight handler wrapped only in recover and recordPattern (preflights carry no credentials, so no auth, throttle or body middleware); two routes on the same path with different profiles fail boot; a ServeMux conflict surfaces through handleRoute as an error; a profiled route whose path matches a glob of http.cors.paths fails boot with a message saying the global wrapper would answer its preflight first. route:list shows the OPTIONS routes with plugin surf.
Docs in the same commit: docs/services/routing.md section "CORS profiles" (config example with a www origin and credentials, the middleware name, the generated OPTIONS route, the overlap rule); surf README Configuration rows http.cors_profiles.<name>.* and the cors middleware in Features and API reference (LoadCORSProfiles).
Smoke test `TestCORSProfileSmoke` appended to modules/surf/transport_smoke_test.go: profile spa (allowed_origins https://www.example.com, methods GET and POST, headers Content-Type, credentials true) on GET /x/me with cors:spa and no global http.cors; OPTIONS /x/me with that Origin and Access-Control-Request-Method GET answers 204 with the exact origin, Allow-Credentials true and Vary containing Origin; GET with that origin carries the exact origin; GET with https://evil.example.org carries no Allow-Origin and does carry Vary: Origin; a route naming cors:missing makes BuildRouter fail naming missing.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/surf &amp;&amp; go test -count=1 -short -run '^(TestCORSProfileSmoke|TestCookieRefreshAcrossSubdomainsSmoke|TestCORSOptionsMatchesLaravel)$' -v ./modules/surf 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 3 &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit: vet or the surf short suite is red, fewer than three "--- PASS" lines (the profile smoke, the tracer smoke or the Laravel preflight parity failed or did not run), or the docs checker reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'func LoadCORSProfiles(cfg \*compass.Config) (map\[string\]CORSConfig, error)' modules/surf/cors.go` prints one line.
- `grep -n 'RegisterMiddlewareFactory("surf", "cors"' modules/surf/router.go` prints one line.
- `grep -n 'cors_profiles' modules/surf/README.md docs/services/routing.md` prints at least one line per file.
</acceptance_criteria>
<done>A plugin's route group can opt into its own credentialed CORS policy, and browsers' preflights for it get a 204 instead of a 405.</done>
</task>
<task type="auto">
<name>Task 3: A guest or registration route rejects bots through turnstile:closed or turnstile:open, verified against Cloudflare siteverify with PHP's two modes</name>
<files>modules/bouncer/turnstile.go, modules/bouncer/README.md, modules/surf/router.go, modules/surf/transport_smoke_test.go, modules/surf/README.md, docs/services/authentication.md</files>
<read_first>/media/nvme/dev/jz/wavepath.org/plugins/golem15/wavepath/classes/auth/TurnstileVerifier.php, /media/nvme/dev/jz/wavepath.org/plugins/golem15/wavepath/controllers/api/GuestCallController.php (:55-75), /media/nvme/dev/jz/wavepath.org/plugins/golem15/wavepath/controllers/api/AuthApiController.php (:60-75, :175-190), ../sm-journal-plugin/classes/site_cache.go (bounded outbound POST, transport error without the URL), modules/surf/bodylimit.go, modules/surf/router.go (wrap order: body cap outside named middleware), modules/surf/clientip.go, .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/COVERAGE.md (turnstile rows), 19.3-RESEARCH.md (Pattern 7, Pitfall 12)</read_first>
<action>Per D-18 and the COVERAGE.md turnstile rows. bouncer/turnstile.go: `Turnstile` with Secret, Production, Endpoint (default TurnstileSiteverifyURL, overridable for tests) and Client (default an *http.Client with Timeout TurnstileTimeout); `NewTurnstile(secret string, production bool) *Turnstile`. `Verify(ctx, token, ip string, failClosed bool) (TurnstileVerdict, error)`: empty secret gives Passed in open mode, and in closed mode Unavailable in production and Passed elsewhere; an empty token or one longer than TurnstileMaxTokenLength gives Failed without a call; otherwise one POST of the form fields secret, response and remoteip (remoteip omitted when ip is empty) under a context bounded by TurnstileTimeout, reading at most 4 KiB of the answer; Passed only when the JSON field success is exactly true; any transport error, timeout, non-2xx status or undecodable answer gives Failed with the error returned for logging (never containing the secret or the token; a url.Error is unwrapped like site_cache.go's purgeTransportError). `TurnstileWriter` is `func(w http.ResponseWriter, r *http.Request, status int, verdict TurnstileVerdict)`. `Middleware(failClosed bool, field string, clientIP func(*http.Request) string, write TurnstileWriter) func(http.Handler) http.Handler`: reads the token from the query parameter named field, else from the body (a JSON object's top-level string field, or an application/x-www-form-urlencoded field), reading the body fully (the router's body cap already bounds it) and restoring r.Body with the same bytes before calling the next handler; Unavailable writes 503, Failed writes 422, Passed calls next; a transport error is logged once per request at warn with the error text only. A nil write uses the neutral default bodies from this plan's Decisions.
router.go BuildRouter: build the Turnstile from turnstile.secret, turnstile.field (default DefaultTurnstileField) and Production = the environment is production; writer from app.Lookup[bouncer.TurnstileWriter]() when published; register factory `turnstile` whose param open or closed selects failClosed; wrap validates the param (anything else fails boot naming the plugin and route); the clientIP func is ClientIP with the router's trusted proxies.
Docs in the same commit: docs/services/authentication.md section "Turnstile" (the two modes and when to use each, statuses, the token field, the writer for app envelopes, turnstile.secret with SUMMER_TURNSTILE__SECRET and the mapping from a PHP TURNSTILE_SECRET variable, test secrets from Cloudflare's testing page, the single-use token note, the order-of-errors caveat versus validating in the handler, and Verify for in-handler use); bouncer README API rows; surf README Configuration rows turnstile.secret and turnstile.field and the middleware.
Smoke test `TestTurnstileMiddlewareSmoke` appended to modules/surf/transport_smoke_test.go, against an httptest siteverify fake (Endpoint override) that answers success true for token ok and false otherwise: POST /guest with turnstile:closed and a JSON body {"turnstile_token":"ok","name":"x"} reaches the handler with the full body; token bad answers 422; POST /register with turnstile:open and an empty secret passes without calling the fake; turnstile:closed with an empty secret in SUMMER_ENV production answers 503; turnstile:maybe fails BuildRouter.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/bouncer ./modules/surf &amp;&amp; go test -count=1 -short -run '^TestTurnstileMiddlewareSmoke$' -v ./modules/surf 2&gt;&amp;1 | grep -q -- '^--- PASS: TestTurnstileMiddlewareSmoke' &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit: vet or a short suite is red, the Turnstile smoke did not print "--- PASS", or the docs checker reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'https://challenges.cloudflare.com/turnstile/v0/siteverify' modules/bouncer/turnstile.go` prints one line.
- `grep -n 'TurnstileTimeout *= *5 \* time.Second' modules/bouncer/turnstile.go` prints one line.
- `grep -n 'RegisterMiddlewareFactory("surf", "turnstile"' modules/surf/router.go` prints one line.
- `grep -n 'SUMMER_TURNSTILE__SECRET' docs/services/authentication.md` prints at least one line.
</acceptance_criteria>
<done>An app guards its guest and registration endpoints with Turnstile in PHP's fail-closed and fail-open modes, keeping its own error envelope.</done>
</task>
<task type="auto">
<name>Task 4: A developer opens /api-docs and reads the app's public API in a pinned, SRI-checked Swagger UI fed by /api-docs.json</name>
<files>modules/surf/apidocs.go, modules/surf/router.go, modules/surf/transport_smoke_test.go, modules/surf/README.md, docs/services/routing.md</files>
<read_first>/media/nvme/dev/jz/wavepath.org/plugins/golem15/wavepath/controllers/ApiDocsController.php, modules/surf/uploads.go (mountUploads, uploadsPrefix), modules/surf/router.go (BucketProvider assertion, mountUploads call), modules/surf/README.md, docs/services/routing.md, 19.3-RESEARCH.md (Pattern 9: the three sha384 hashes for swagger-ui-dist 5.33.1, Package Legitimacy Audit row for swagger-ui-dist)</read_first>
<action>Per D-16 and D-17. apidocs.go: `APIDocsProvider` interface with `APIDocsSpec() ([]byte, error)`, type-asserted on plugins like BucketProvider (surf-defined, not pact). `SwaggerUIVersion` constant 5.33.1 (5.33.2 is avoided: flagged too new in RESEARCH) and unexported constants for the three jsDelivr asset URLs under https://cdn.jsdelivr.net/npm/swagger-ui-dist@5.33.1/ with their sha384 integrity values copied from RESEARCH Pattern 9. `mountAPIDocs(r *Router, app *backpack.App, plugins []party.Plugin) error`, called from BuildRouter after mountUploads: nothing unless http.api_docs.enabled is true; path from http.api_docs.path (default /api-docs; validated as an absolute URL path with the uploadsPrefix rules); JSON route at path plus .json; title http.api_docs.title (default API documentation); zero providers or more than one fails boot naming the providers; the provider's bytes are converted once: a body whose first non-space byte is { must be valid JSON, anything else goes through goccy/go-yaml YAMLToJSON; then json.Compact; a conversion error fails boot naming the plugin. Routes under plugin id summercms.apidocs through GroupRaw with no middleware (public), refusing to mount over an existing route (boot error like mountUploads). GET path renders an html/template page: the title, the css link and both scripts with integrity and crossorigin anonymous, and one inline script with a per-request random nonce that calls SwaggerUIBundle with url set to the JSON path (template-escaped), dom_id #swagger-ui, the standalone preset and layout StandaloneLayout; headers Content-Type text/html; charset=utf-8, X-Content-Type-Options nosniff, Referrer-Policy no-referrer, and Content-Security-Policy with script-src limited to the nonce and https://cdn.jsdelivr.net, object-src none, base-uri none and frame-ancestors none. GET path.json writes the cached bytes with Content-Type application/json.
Docs in the same commit: docs/services/routing.md section "API documentation page" (provider interface with an embedded YAML example under a neutral plugin, the three keys, why the version is pinned with SRI, both routes public, off by default); surf README Configuration rows http.api_docs.* and API reference rows APIDocsProvider and SwaggerUIVersion.
Smoke test `TestAPIDocsSmoke` appended to modules/surf/transport_smoke_test.go: a provider plugin returning YAML whose keys are not in alphabetical order (openapi, info, paths); with the flag off GET /api-docs answers 404; with it on GET /api-docs.json answers compact JSON whose keys keep the source order, and GET /api-docs contains swagger-ui-dist@5.33.1 and three integrity="sha384- attributes and a nonce matching the CSP header; two providers fail BuildRouter.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/surf &amp;&amp; go test -count=1 -short -run '^(TestAPIDocsSmoke|TestCORSProfileSmoke|TestTurnstileMiddlewareSmoke|TestCookieRefreshAcrossSubdomainsSmoke|TestCORSCredentialsWildcardRefusedSmoke)$' -v ./modules/surf 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 5 &amp;&amp; go test -count=1 -short ./... &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; go run ./cmd/summer docs:build --check</automated>
<fails_when>Non-zero exit: vet or a short suite is red, fewer than five "--- PASS" lines for the plan's smoke tests, or the docs checker or docs:build --check reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -c 'sha384-' modules/surf/apidocs.go` prints 3.
- `grep -n 'swagger-ui-dist@5.33.1' modules/surf/apidocs.go` prints at least one line and `grep -c 'swagger-ui-dist@5"' modules/surf/apidocs.go` prints 0 (no floating major).
- `grep -n 'goccy/go-yaml' modules/surf/apidocs.go` prints one line.
- `grep -n 'http.api_docs.enabled' modules/surf/README.md docs/services/routing.md` prints at least one line per file.
</acceptance_criteria>
<done>An app that ships a spec can turn on a public API docs page whose CDN assets cannot be swapped under it.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| browser on a sibling subdomain to the api host | credentialed cross-origin requests and the token cookie |
| request to Turnstile middleware | untrusted token and body; the siteverify call crosses to Cloudflare |
| jsDelivr CDN to the api-docs page | third-party scripts executed in the app's origin |
| config to boot | CORS, cookie, Turnstile and api-docs keys validated before serving |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-19.3-06 | Information disclosure | surf CORS with credentials and a * origin | high | mitigate | LoadCORSConfig and LoadCORSProfiles refuse the combination at boot; TestCORSCredentialsWildcardRefusedSmoke |
| T-19.3-07 | Tampering | shared caches storing a CORS answer for another origin | medium | mitigate | Vary: Origin on every dynamic CORS response, preflight and disallowed origin included; exact-origin echo |
| T-19.3-08 | Spoofing | token cookie attributes (SameSite none without Secure, secure false in production, invalid Domain dropped silently) | high | mitigate | LoadCookieConfig errors for each, with (*http.Cookie).Valid for the domain; httpOnly always on |
| T-19.3-09 | Information disclosure | cookie Domain covering every subdomain | medium | mitigate | host-only default; Domain is opt-in and the docs name the subdomain trust boundary |
| T-19.3-10 | Tampering | CSRF on the cookie refresh and logout handlers from a sibling subdomain | low | accept | refresh only rotates a cookie (the answer is unreadable cross-origin unless CORS allows that origin); logout CSRF is a forced logout; POST-only routes and the docs' JSON-or-custom-header advice |
| T-19.3-11 | Spoofing | Turnstile bypass by an unconfigured secret | high | mitigate | closed mode answers 503 in production with an empty secret; open mode is documented for registration only; with a secret both modes require success true |
| T-19.3-12 | Denial of service | slow siteverify piling up requests | medium | mitigate | TurnstileTimeout 5 s on the client and the context; at most 4 KiB of the answer read |
| T-19.3-13 | Tampering | Turnstile middleware consuming the request body | medium | mitigate | the body (already capped) is read once and restored; TestTurnstileMiddlewareSmoke asserts the handler sees it |
| T-19.3-14 | Information disclosure | Turnstile secret or token in logs and errors | high | mitigate | the secret travels only in the form body; logged errors carry the transport error text without URL, secret or token |
| T-19.3-15 | Tampering | swagger-ui assets on jsDelivr | medium | mitigate | exact version 5.33.1 with sha384 SRI and crossorigin on all three assets; nonce CSP limits scripts to the nonce and the CDN host |
| T-19.3-16 | Information disclosure | /api-docs exposing the API surface | low | accept | off by default; the spec is the app's hand-written public surface (D-16); both routes public by design (D-17) |
| T-19.3-17 | Tampering | XSS through http.api_docs.title or the spec URL | medium | mitigate | html/template contextual escaping for HTML and JS contexts |
| T-19.3-SC | Tampering | npm/pip/cargo or Go module installs | high | mitigate | this plan installs nothing: swagger-ui-dist is loaded from the CDN at a pinned version with SRI (5.33.2 flagged too new is avoided); go.mod and go.sum unchanged |
</threat_model>
<verification>
- Task 1: the bouncer and surf short suites, the two tracer smokes and the Laravel preflight parity test, TestDocsTree.
- Task 2: the surf short suite, the profile smoke with the tracer smoke and the parity test, TestDocsTree.
- Task 3: the bouncer and surf short suites, the Turnstile smoke, TestDocsTree.
- Task 4: the five smoke tests, the whole short suite, TestDocsTree and docs:build --check.
- Feedback sampling (19.3-VALIDATION.md): after each commit `go vet ./... && go test -short ./modules/bouncer ./modules/surf`.
</verification>
<success_criteria>
- SC-1: cookie refresh with configurable Domain, SameSite and Secure, JTI blacklisted, Max-Age from refresh_ttl; credentialed CORS as global config and as cors:<profile> middleware.
- SC-3: /api-docs and /api-docs.json served opt-in; Turnstile middleware available.
- Existing CORS answers for non-credentialed * configs unchanged; go vet and the short suite green at every commit.
</success_criteria>
<output>
Create `.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-02-SUMMARY.md` when done, with the commits, the identifiers and config keys added, the divergences from PHP recorded in Decisions, and any deviation.
</output>

View File

@@ -0,0 +1,251 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 02
subsystem: auth
tags: [bouncer, surf, cookies, cors, turnstile, openapi, swagger-ui, sri]
requires:
- phase: 19.3-01
provides: "surf BuildRouter store lookup and cooler wiring (this plan extends BuildRouter next to it)"
provides:
- "bouncer cookie transport: CookieConfig/LoadCookieConfig (auth.cookie.*), TokenCookie/ForgetTokenCookie, CookieRefreshHandler/CookieLogoutHandler"
- "surf http.cors hardening: credentials+* and bad/PHP-delimited patterns fail boot, wildcard origins as anchored patterns, exact echo, Vary: Origin on every dynamic answer"
- "surf named CORS profiles: http.cors_profiles.<name>, cors:<profile> middleware, router-registered OPTIONS preflight routes"
- "bouncer Turnstile verifier and middleware; surf turnstile:open|closed factory from turnstile.secret/turnstile.field"
- "surf opt-in /api-docs Swagger UI page and /api-docs.json from one APIDocsProvider plugin (swagger-ui-dist 5.33.1 with SRI)"
affects: [19.3-05, 19.4, 19.5, 19.6]
actuals:
tokens: 26655
tasks: 4
commits: 4
plan_head_before: 866e4fd3e0013d4ddcf41a823fca43d68ac7734f
plan_head_after: 20bc1a0795eef25af5221871c2a41ee75d48744d
tech-stack:
added: []
patterns:
- "Structural config interface (bouncer.CookieConfigSource) so bouncer reads *compass.Config without importing it"
- "Shared corsPolicy used by the global path-scoped wrapper and the cors:<profile> middleware"
- "Router-registered OPTIONS routes (wrapped only in recovery and recordPattern) for route-level CORS"
- "Optional plugin interface type-asserted in BuildRouter (APIDocsProvider), like BucketProvider"
- "Published-service override: a *bouncer.Turnstile or bouncer.TurnstileWriter on the app replaces the defaults"
key-files:
created:
- modules/bouncer/cookie.go
- modules/bouncer/turnstile.go
- modules/surf/apidocs.go
- modules/surf/transport_smoke_test.go
modified:
- modules/surf/cors.go
- modules/surf/router.go
- modules/surf/example_test.go
- modules/bouncer/README.md
- modules/surf/README.md
- docs/services/authentication.md
- docs/services/routing.md
key-decisions:
- "Cookie builders are methods on bouncer.CookieConfig (one config for every guard); the refresh window is passed per call, so Max-Age is the guard's refresh_ttl"
- "LoadCookieConfig takes bouncer.CookieConfigSource (Lookup, String, Environment), which *compass.Config satisfies; bouncer still imports no framework module"
- "Cookie handlers return (http.Handler, error) so missing options fail boot; refresh reads only the cookie and leaves it alone on 401; logout always clears the cookie and answers 200, passing a revoke error to the writer"
- "A CORS answer is dynamic with credentials, any pattern (wildcard origins included) or more than one origin; dynamic answers always carry Vary: Origin; the non-credentialed * answer is byte-identical to before"
- "CORS profiles live under http.cors_profiles.<name> (names [a-z0-9_-]+); the router registers OPTIONS routes owned by surf; overlap with http.cors.paths fails boot"
- "Turnstile divergences from PHP (outcome-preserving): open mode with a secret treats a transport error or timeout as 422 (PHP surfaces a 500); a missing token is 422 without a siteverify call; tokens over 2048 characters fail without a call"
- "Default Turnstile bodies are neutral JSON ({\"error\":\"turnstile_unavailable\"} 503, {\"error\":\"turnstile_failed\"} 422); a published bouncer.TurnstileWriter keeps an app's envelope; a published *bouncer.Turnstile replaces the config-built verifier"
- "api-docs spec converted at boot (a spec that does not convert fails boot naming the plugin, where PHP answers 500 per request); the page uses a per-request CSP nonce and validatorUrl null"
patterns-established:
- "Boot validation of route-level middleware params in Router.wrap (cors:<profile>, turnstile:open|closed) next to throttle and body.limit"
- "Docs Go blocks reference tested example code (ExampleAPIDocsProvider, BlogPlugin.APIDocsSpec)"
requirements-completed: [SC-1, SC-3, SC-4, D-01, D-02, D-03, D-04, D-05, D-16, D-17, D-18, D-19]
coverage:
- id: D1
description: "A www page refreshes its session on the api host: credentialed CORS echoes the exact origin with Vary: Origin, the refresh handler sets token Domain=example.com Max-Age=1209600 HttpOnly Secure SameSite=Lax and blacklists the old jti; the preflight answers 204 with Vary: Origin"
requirement: SC-1
verification:
- kind: integration
ref: "modules/surf/transport_smoke_test.go#TestCookieRefreshAcrossSubdomainsSmoke"
status: pass
human_judgment: false
- id: D2
description: "Credentialed CORS with a * origin fails boot naming http.cors; the Laravel preflight bytes for the non-credentialed * config are unchanged"
requirement: D-04
verification:
- kind: unit
ref: "modules/surf/transport_smoke_test.go#TestCORSCredentialsWildcardRefusedSmoke"
status: pass
- kind: unit
ref: "modules/surf/cors_coverage_test.go#TestCORSOptionsMatchesLaravel"
status: pass
human_judgment: false
- id: D3
description: "cors:<profile> gives a route group its own credentialed policy; the router-registered OPTIONS route answers the preflight (204, exact origin, credentials, Vary: Origin); a disallowed origin gets Vary but no Allow-Origin; cors:missing fails boot"
requirement: D-05
verification:
- kind: integration
ref: "modules/surf/transport_smoke_test.go#TestCORSProfileSmoke"
status: pass
human_judgment: false
- id: D4
description: "turnstile:closed passes a good token with the full body restored, answers 422 for a rejected token and 503 unconfigured in production; turnstile:open passes without a secret and without a siteverify call; turnstile:maybe fails boot"
requirement: D-18
verification:
- kind: integration
ref: "modules/surf/transport_smoke_test.go#TestTurnstileMiddlewareSmoke"
status: pass
human_judgment: false
- id: D5
description: "/api-docs is 404 by default; enabled, /api-docs.json serves compact JSON in source key order and /api-docs carries swagger-ui-dist@5.33.1, three sha384 integrity attributes and a nonce matching the CSP; two providers fail boot"
requirement: D-17
verification:
- kind: integration
ref: "modules/surf/transport_smoke_test.go#TestAPIDocsSmoke"
status: pass
- kind: unit
ref: "modules/surf/example_test.go#ExampleAPIDocsProvider"
status: pass
human_judgment: false
- id: D6
description: "The Swagger UI page renders and loads the spec in a real browser under its Content-Security-Policy and SRI"
requirement: D-17
verification: []
human_judgment: true
rationale: "No browser test runs here; the bundle contains Function(\"return this\") fallbacks, so whether the CSP without unsafe-eval lets Swagger UI start needs a browser check"
- id: D7
description: "A browser on a real www subdomain keeps its session through the api host with the shared Domain cookie"
requirement: SC-1
verification: []
human_judgment: true
rationale: "The smoke test drives the handler with httptest; browser cookie-jar and SameSite behaviour across real subdomains needs a manual check"
- id: D8
description: "READMEs and docs updated in the same commits; docs checker and docs:build --check green"
requirement: SC-4
verification:
- kind: other
ref: "go test -run '^TestDocsTree$' ./cmd/summer && go run ./cmd/summer docs:build --check"
status: pass
human_judgment: false
duration: 15min
completed: 2026-10-15
status: complete
---
# Phase 19.3 Plan 02: Cookie transport, credentialed CORS, Turnstile and API docs Summary
**httpOnly token-cookie refresh across subdomains in bouncer, boot-validated credentialed CORS with per-group cors:<profile> policies and router-registered preflights, Cloudflare Turnstile as turnstile:open|closed middleware, and an opt-in /api-docs Swagger UI page pinned to swagger-ui-dist 5.33.1 with SRI**
## Performance
- **Duration:** 15 min
- **Started:** 2026-10-15T15:45:56Z
- **Completed:** 2026-10-15T16:00:55Z
- **Tasks:** 4
- **Files modified:** 11
## Accomplishments
- Cookie transport (D-01..D-03): `bouncer.LoadCookieConfig` reads `auth.cookie.{name,domain,path,secure,same_site}` with PHP defaults (token, `/`, HttpOnly, Lax, host-only, Secure when `app.url` is https) and refuses an invalid domain, `none` without Secure and `secure: false` in production. `bouncer.CookieConfig.TokenCookie` sets Max-Age to the refresh window. `bouncer.CookieRefreshHandler` and `bouncer.CookieLogoutHandler` wrap the app's refresh or revoke func and writer.
- CORS hardening (D-04): `surf.LoadCORSConfig` refuses credentials with `*`, uncompilable patterns and PHP-delimited patterns, naming the key. Wildcard origins become anchored patterns. Dynamic answers echo the exact origin and always send `Vary: Origin`. `TestCORSOptionsMatchesLaravel` is still green.
- CORS profiles (D-05): `surf.LoadCORSProfiles` reads `http.cors_profiles.<name>` and the `cors:<profile>` factory applies them. The router registers an `OPTIONS` route (owner `surf`) per profiled path. Boot errors cover an unknown profile, conflicting profiles, an existing OPTIONS route and overlap with `http.cors.paths`.
- Turnstile (D-18): `bouncer.Turnstile.Verify` follows PHP's open and closed modes, with a 5 s bound, a 4 KiB answer cap and the 2048-character token limit. `bouncer.Turnstile.Middleware` reads the token from the query, a JSON body or a form body and restores the body. surf registers `turnstile:open|closed` from `turnstile.secret` (`SUMMER_TURNSTILE__SECRET`) and `turnstile.field`, using the trusted-proxy client IP.
- API docs (D-16, D-17): `surf.APIDocsProvider` and `http.api_docs.{enabled,path,title}` mount public `GET /api-docs` and `/api-docs.json`. The spec is converted once at boot. The page pins swagger-ui-dist `5.33.1`, carries sha384 SRI on all three assets (re-verified against jsDelivr during execution) and uses a nonce CSP.
## Task Commits
1. **Task 1 (tracer): cookie refresh across subdomains + CORS hardening** - `e7659f2` (feat)
2. **Task 2: cors:<profile> with router-registered preflights** - `5055e8b` (feat)
3. **Task 3: Turnstile verifier and turnstile:open|closed middleware** - `597c79b` (feat)
4. **Task 4: opt-in /api-docs page with pinned SRI assets** - `20bc1a0` (feat)
## Files Created/Modified
- `modules/bouncer/cookie.go` - CookieConfig, LoadCookieConfig, TokenCookie/ForgetTokenCookie, cookie refresh/logout handlers
- `modules/bouncer/turnstile.go` - Turnstile verifier, verdicts, middleware, TurnstileWriter
- `modules/surf/cors.go` - validation, corsPolicy, wildcard origins, Vary, LoadCORSProfiles, preflight route registration
- `modules/surf/router.go` - `cors` and `turnstile` factories, param validation in wrap, preflight routes, mountAPIDocs call, buildTurnstile
- `modules/surf/apidocs.go` - APIDocsProvider, SwaggerUIVersion, pinned assets, page and spec routes
- `modules/surf/transport_smoke_test.go` - the five smoke tests
- `modules/surf/example_test.go` - BlogPlugin.APIDocsSpec, ExampleAPIDocsProvider (docs snippet source)
- `modules/bouncer/README.md`, `modules/surf/README.md`, `docs/services/authentication.md`, `docs/services/routing.md` - Cookie transport, Turnstile, CORS rules, CORS profiles, API documentation page
## Config keys and identifiers added
- Keys: `auth.cookie.name`, `auth.cookie.domain`, `auth.cookie.path`, `auth.cookie.secure`, `auth.cookie.same_site`; `http.cors_profiles.<name>.*`; `turnstile.secret`, `turnstile.field`; `http.api_docs.enabled`, `http.api_docs.path`, `http.api_docs.title`.
- Middleware: `cors:<profile>`, `turnstile:open`, `turnstile:closed`. Routes: `GET /api-docs`, `GET /api-docs.json` (opt-in), plus `OPTIONS <path>` per profiled path.
- bouncer: `CookieConfig`, `CookieConfigSource`, `LoadCookieConfig`, `DefaultCookieName`, `CookieResult`, `CookieRefreshOptions`, `CookieLogoutOptions`, `CookieRefreshHandler`, `CookieLogoutHandler`, `ErrNoTokenCookie`, `Turnstile`, `NewTurnstile`, `TurnstileSiteverifyURL`, `TurnstileTimeout`, `TurnstileMaxTokenLength`, `DefaultTurnstileField`, `TurnstileVerdict` (`TurnstilePassed`, `TurnstileFailed`, `TurnstileUnavailable`), `TurnstileWriter`.
- surf: `LoadCORSProfiles`, `APIDocsProvider`, `SwaggerUIVersion`.
## Decisions Made
See `key-decisions` in the frontmatter. The divergences from PHP are deliberate:
- Credentials with `*` is a boot error (php-cors reflects any origin).
- In Turnstile open mode with a secret, a transport error or timeout is a 422 (PHP surfaces a 500).
- A missing Turnstile token, or one over 2048 characters, is a 422 without a siteverify call.
- A spec that does not convert fails boot (PHP answers 500 per request).
- The Swagger UI version is exact and SRI-checked (PHP loads the floating `@5` without integrity).
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] A published `*bouncer.Turnstile` overrides the config-built verifier**
- **Found during:** Task 3
- **Issue:** The smoke test must point the verifier's `Endpoint` at an httptest fake, but BuildRouter builds the verifier from config, so the test had no way to reach it.
- **Fix:** `buildTurnstile` uses a `*bouncer.Turnstile` published on the app when there is one. Otherwise it builds from `turnstile.secret`. This is documented in the surf README and the Turnstile section.
- **Files modified:** modules/surf/router.go, docs
- **Committed in:** 597c79b
**2. [Rule 2 - Missing critical] Turnstile middleware handles a body read error**
- **Found during:** Task 3
- **Issue:** The plan had the middleware read the whole body but said nothing about a read error, such as the router's body cap tripping.
- **Fix:** An `*http.MaxBytesError` answers 413 and any other read error answers 400, both through the writer with `TurnstileFailed`. The handler is never called with a broken body.
- **Committed in:** 597c79b
**3. [Rule 3 - Blocking] Docs checker requires Go blocks to come from tested code**
- **Found during:** Task 2 and Task 4
- **Issue:** A Go code block in docs/ must carry a `src=` reference to a function a Test or Example runs.
- **Fix:** The CORS profile group example became prose. For the API docs page, `BlogPlugin.APIDocsSpec` and `ExampleAPIDocsProvider` (with `// Output:`) were added to `modules/surf/example_test.go`, which is not in the plan's file list. `TestDocsDeclarations` calls the method.
- **Committed in:** 20bc1a0
**4. [Rule 2] Small hardening and accuracy additions**
- Zero-value `CookieConfig` fields default to Name `token` and Path `/` in the builders and handlers.
- Turnstile `success:false` with Cloudflare error codes (sanitized) is logged at debug; transport failures are logged at warn.
- `validatorUrl: null` in the Swagger config, so the spec URL is never sent to swagger.io.
- docs/services/routing.md's middleware table gained the turnstile row in Task 3 (routing.md was not in Task 3's file list).
- The surf README's standard-library list gained `log/slog` and `net/url`, which were missing before this plan.
- **Committed in:** e7659f2, 597c79b, 20bc1a0
---
**Total deviations:** 4 (2 Rule 3, 2 Rule 2).
**Impact on plan:** None of these change behaviour the plan specified. The two seams it adds (a published verifier, the example test) are small and documented.
## Issues Encountered
None.
## User Setup Required
None. Apps opt in through config (`auth.cookie.*`, `http.cors_profiles`, `SUMMER_TURNSTILE__SECRET`, `http.api_docs.enabled`).
## Next Phase Readiness
- Plan 05 (unit tests) has full coverage to write for cookie.go, turnstile.go, cors.go (profiles, wildcard origins, PHP-delimited patterns) and apidocs.go. The smoke tests here are the starting point.
- Human UAT: check that the Swagger UI page renders in a browser under its CSP (D6), and that the cookie survives across real subdomains (D7).
## Self-Check: PASSED
- FOUND: modules/bouncer/cookie.go, modules/bouncer/turnstile.go, modules/surf/apidocs.go, modules/surf/transport_smoke_test.go
- FOUND commits: e7659f2, 5055e8b, 597c79b, 20bc1a0
- `go vet ./...` and `go test -short ./...` green; five smoke tests plus TestCORSOptionsMatchesLaravel pass; TestDocsTree and `docs:build --check` green
- bouncer imports no SummerCMS module; no "wavepath" in modules/bouncer, modules/surf or docs/services
---
*Phase: 19.3-framework-transport-and-infrastructure-for-wavepath*
*Completed: 2026-10-15*

View File

@@ -0,0 +1,260 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 03
type: execute
wave: 2
depends_on: ["19.3-01"]
files_modified:
- go.mod
- go.sum
- internal/build/manifest.go
- internal/build/build.go
- internal/build/manifest_tags_smoke_test.go
- modules/lagoon/attach/bucket.go
- modules/lagoon/attach/s3_on.go
- modules/lagoon/attach/s3_off.go
- modules/lagoon/attach/diskext.go
- modules/lagoon/attach/store.go
- modules/lagoon/attach/thumb.go
- modules/lagoon/attach/infra_smoke_test.go
- modules/lagoon/README.md
- docs/services/storage.md
- docs/database/attachments.md
- docs/architecture/go-modules-and-workspaces.md
- docs/architecture/performance-and-scaling.md
- docs/console/scaffolding.md
- docs/setup/configuration.md
autonomous: true
requirements: [SC-3, SC-4, D-12, D-13, D-14, D-15, D-19]
estimate:
tokens: 130000
raw_tokens: 130000
tasks: 3
confidence: low
must_haves:
truths:
- "D-13: summer.yaml accepts build.tags (a list of tags matching ^[a-z0-9_]+$); summer build passes them as go build -tags=<comma list> and prints them; summer plugin:add (SaveManifest) round-trips build.tags; summer dev inherits them through build.App."
- "D-13: a binary built with the summercms_s3 tag opens storage.uploads.bucket_url s3://... through gocloud.dev/blob/s3blob, which only the tagged file s3_on.go imports, so an application never imports the driver itself; a binary built without the tag fails boot on an s3:// bucket_url with an error that names summercms_s3, build.tags in summer.yaml and the -tags flag for raw go build."
- "D-14: S3-compatible endpoints are configured only through gocloud's bucket_url query (region, endpoint, use_path_style, request_checksum_calculation, response_checksum_validation) with credentials from the standard AWS environment; no s3.* key set exists."
- "D-15: attach.Store names the stored blob with attach.DiskExtension(sniffedType, clientExt): a PNG uploaded as x.html is stored and served as .png while file_name keeps x.html; a docx (sniffed application/zip) keeps .docx; an HTML body uploaded as .txt stays .txt; a script-capable client extension under a generic sniff becomes txt or bin; the allow-list check on the client extension is unchanged."
- "D-15: the storage docs carry a deploy note: serve uploads with X-Content-Type-Options nosniff and keep a file-type allow-list in the static server."
- "D-12: with storage.uploads.public_url set (an absolute http or https base), attach.PublicURL, File.URL and thumbnail URLs are that base plus the partition path; with it unset URLs keep storage.uploads.public_path_prefix and the /storage/uploads proxy route, which also serves S3 buckets; a malformed public_url fails boot naming the key, and a set public_url logs one start-up line warning that the bucket behind it must hold only public files."
artifacts:
- path: "modules/lagoon/attach/s3_on.go"
provides: "the tagged s3blob import"
contains: "//go:build summercms_s3"
- path: "modules/lagoon/attach/s3_off.go"
provides: "s3Enabled false without the tag"
contains: "//go:build !summercms_s3"
- path: "modules/lagoon/attach/diskext.go"
provides: "DiskExtension"
contains: "func DiskExtension"
- path: "internal/build/manifest.go"
provides: "Manifest.Tags, build.tags load and save"
contains: "Tags"
key_links:
- from: "internal/build/build.go"
to: "internal/build/manifest.go"
via: "App passes m.Tags to goBuildArgs, which adds -tags=<joined>"
pattern: "goBuildArgs"
- from: "modules/lagoon/attach/bucket.go"
to: "modules/lagoon/attach/s3_off.go"
via: "OpenBucket refuses an s3 scheme when s3Enabled is false"
pattern: "s3Enabled"
- from: "modules/lagoon/attach/store.go"
to: "modules/lagoon/attach/diskext.go"
via: "newDiskName(DiskExtension(sniffed, ext))"
pattern: "DiskExtension\\("
- from: "modules/lagoon/attach/thumb.go"
to: "modules/lagoon/attach/bucket.go"
via: "PublicURL prefers PublicURLBase() over PublicPathPrefix()"
pattern: "PublicURLBase"
prohibitions:
- statement: "No new top-level s3.* configuration keys (D-14)."
status: unresolved
verification: judgment
- statement: "Uploads that pass today's allow-list still pass; only the stored name of mismatched or script-capable uploads changes."
status: unresolved
verification: judgment
- statement: "Application call sites outside the framework are not changed in this phase (deferred follow-up)."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not in user-story format; no story is invented, see plan 01): the framework has the transport and infrastructure pieces a multi-client, multi-instance application needs: JWT cookie refresh across subdomains, credentialed CORS, a cache module with a Redis-backed rate-limit store, an S3 blob option, an OpenAPI page and Turnstile verification.
This plan's slice (plan 3 of 5, wave 2): the S3 part of SC-3 and the folded stored-XSS todo. `summer.yaml` gains `build.tags`, the `summercms_s3` tag links the s3blob driver (the tracer: tagged binary opens s3://, untagged binary names the tag), then the disk-name extension fix and `storage.uploads.public_url`.
## Spec-less probe fallback
Skipped visibly: the phase has no requirement IDs to probe, so no probe predicates were generated (see plan 01).
## Decisions taken in this plan (Claude's discretion)
- The build tag is `summercms_s3` (exported as `attach.S3BuildTag` for docs and errors). summer.yaml declares it as a list under `build.tags`; each tag must match `^[a-z0-9_]+$`, duplicates are dropped, and the tags are passed to go build as one `-tags=` argument (exec args, no shell). `summer build` prints a `build tags: …` line, because app build scripts that run a second raw `go build` must repeat the tag (Pitfall 6); the boot error is the safety net.
- `DiskExtension` follows RESEARCH Pattern 6: a table of specific sniffed types with their accepted extension sets (the client's extension is kept when it belongs to the set, else the canonical one); every other sniff counts as generic and keeps the client's extension unless it is script-capable, which becomes `txt` for a text/* sniff and `bin` otherwise. It never calls `mime.ExtensionsByType` (host-dependent). Store passes the sniffed type before its extension-based fallback.
- `public_url` applies to every URL the package builds, as D-12 states; the start-up warning and the docs carry the "public files only" rule instead of a narrower per-file switch.
- The bucket-open error keeps quoting bucket_url (credentials never travel in it per D-14).
<assumption_delta_decision>
add-alongside: blob drivers pluralize from file and mem to file, mem and s3. fileblob and memblob stay linked in every binary; s3blob is added only to binaries built with the summercms_s3 tag, so binaries without the tag keep their size and dependency set.
</assumption_delta_decision>
## Artifacts this phase produces (this plan)
- `internal/build`: `Manifest.Tags`, `ValidateBuildTag`, unexported `goBuildArgs(binRel string, tags []string) []string`; `summer.yaml` key `build.tags`; the `build tags:` output line.
- `modules/lagoon/attach`: `S3BuildTag` (`summercms_s3`), files `s3_on.go` (`//go:build summercms_s3`, blank import of `gocloud.dev/blob/s3blob`) and `s3_off.go` (`//go:build !summercms_s3`), unexported `s3Enabled`; `DiskExtension(contentType, clientExt string) string`; `PublicURLBase() string`; config key `storage.uploads.public_url`.
- Dependencies: AWS SDK v2 modules as indirect requirements of `gocloud.dev/blob/s3blob` (named in plan 01's decision note).
- Smoke tests in `internal/build/manifest_tags_smoke_test.go` (`TestManifestBuildTagsSmoke`) and `modules/lagoon/attach/infra_smoke_test.go` (`TestOpenBucketS3Smoke` in both tag modes, `TestStoreDiskExtensionSmoke` on Postgres, `TestPublicURLSmoke`). Full unit coverage is plan 05.
- Docs: `docs/services/storage.md` sections "S3 and S3-compatible buckets", "Public URL base" and a deploy note; `docs/database/attachments.md` disk names; `docs/architecture/go-modules-and-workspaces.md` `build.tags`; `docs/console/scaffolding.md`; rows in `docs/setup/configuration.md`; the shared-uploads note in `docs/architecture/performance-and-scaling.md`; lagoon README.
<objective>
Let an application choose an S3 (or S3-compatible) uploads bucket from config through a summer.yaml build tag, give it a public URL base for a bucket or CDN, and close the stored-XSS hole where a disk name kept the client's extension, each documented in the same commit.
Purpose: SC-3's S3 driver, D-12..D-15 (placement per D-19), and the folded todo `2026-10-11-uploads-keep-client-extension-stored-xss`.
Output: internal/build and lagoon/attach changes with smoke tests and docs.
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-CONTEXT.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-RESEARCH.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-PATTERNS.md
@.planning/notes/2026-10-15-cooler-go-redis-otter.md
@.planning/todos/pending/2026-10-11-uploads-keep-client-extension-stored-xss.md
Concurrency: plans 19.3-02 and 19.3-04 run in the same wave in this checkout and own other files (bouncer, surf, cooler, pact, cabana). This plan is the only wave-2 plan that changes go.mod and go.sum. Before Task 1 run `git status --porcelain -- go.mod go.sum internal/build modules/lagoon docs/services/storage.md docs/database docs/architecture docs/console docs/setup`; if it lists a file this plan touches that this plan did not change, stop and report. A transient vet or test failure in a package another wave-2 plan owns is not this plan's: re-run once after a short pause, then report it. Stage files by explicit path; no co-author tags. Docker-backed smoke tests skip under `-short`; run them on a Docker host (`sudo -u golem` when the socket is denied) and record NOT RUN in the SUMMARY when no daemon answers. No real S3 or MinIO is available: the s3 smoke opens the bucket only (s3blob opens lazily); a real upload stays a manual UAT item (19.3-VALIDATION.md).
<interfaces>
Existing seams (read; extend, do not fork):
- `internal/build/manifest.go`: `Manifest{Module, Binary, Plugins}`, `LoadManifest` (koanf yaml parser into a tree; `asString`, `asSlice`, `asMap`), `SaveManifest` (writes module, binary and plugins only, lines :90-103, so a new key is dropped by `plugin:add` unless written here).
- `internal/build/build.go` :49: `exec.CommandContext(ctx, "go", "build", "-o", binRel, ".")`; `internal/dev/watch.go` :46 reuses `build.App`.
- `modules/lagoon/attach/bucket.go`: blank imports of fileblob and memblob (:13-14), `normalizeBucketURL` (unknown schemes pass through unchanged), `OpenBucket` (records public_path_prefix through `setPublicPathPrefix` under `prefixMu`), error style `attach: … (set SUMMER_…)`.
- `modules/lagoon/attach/store.go` :124-152: client extension allow-list, `contentType := baseMediaType(http.DetectContentType(head))`, the octet-stream fallback to `mime.TypeByExtension`, `newDiskName(ext)`.
- `modules/lagoon/attach/thumb.go` :83-99: `PublicURL(key)` and `(*File).URL()`; `Thumb`/`ThumbKey` write thumbnails eagerly, so their URLs work behind a bucket or CDN base too.
- `modules/lagoon/attach/store_test.go` `TestStoreSmoke` (:48): the Postgres harness and `testing.Short()` skip to reuse.
- gocloud.dev v0.46.0 `blob/s3blob` (Scheme s3, URL parameters per RESEARCH Pattern 5).
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: an app lists summercms_s3 under build.tags, summer build passes it to go build, and the binary opens an s3:// bucket; an untagged binary names the tag at boot</name>
<reversibility rating="costly">D-13: the summercms_s3 tag name and the build.tags key end up in app summer.yaml files and build scripts.</reversibility>
<files>go.mod, go.sum, internal/build/manifest.go, internal/build/build.go, internal/build/manifest_tags_smoke_test.go, modules/lagoon/attach/bucket.go, modules/lagoon/attach/s3_on.go, modules/lagoon/attach/s3_off.go, modules/lagoon/attach/infra_smoke_test.go, modules/lagoon/README.md, docs/services/storage.md, docs/architecture/go-modules-and-workspaces.md, docs/console/scaffolding.md, docs/setup/configuration.md, docs/architecture/performance-and-scaling.md</files>
<read_first>internal/build/manifest.go, internal/build/build.go, internal/build/build_test.go (TestLoadHelloManifest, TestMakeAndAddPluginOnHelloCopy), internal/build/scaffold.go (:80-160, plugin:add round trip), internal/dev/watch.go (:40-60), modules/lagoon/attach/bucket.go, modules/lagoon/attach/bucket_test.go, modules/lagoon/README.md (attach bullet and API rows), docs/services/storage.md, docs/architecture/go-modules-and-workspaces.md (The summer.yaml manifest), docs/console/scaffolding.md (:10-60), docs/setup/configuration.md, docs/architecture/performance-and-scaling.md (Scaling and operations), ../sm-grzybyfunkcjonalne-app/scripts/build.sh (the second raw go build, read only), 19.3-RESEARCH.md (Pattern 5, Pitfall 6, Pitfall 11)</read_first>
<action>Per D-13 and D-14.
internal/build: Manifest gains `Tags []string`. LoadManifest reads the optional `build` mapping and its `tags` list (a non-mapping build or a non-list tags is an error naming build.tags); every entry is trimmed and checked by `ValidateBuildTag(tag string) error` (pattern ^[a-z0-9_]+$, error names the tag); duplicates are dropped keeping the first. SaveManifest writes a `build:` block with `tags:` entries after plugins when Tags is non-empty, validating each, so `summer plugin:add` keeps them. `goBuildArgs(binRel string, tags []string) []string` returns build, then -tags=<comma-joined> when tags exist, then -o binRel and the package dot; App uses it and, when tags exist, prints `build tags: <comma list>` to out before building.
attach: move the s3 driver behind the tag: s3_on.go (constraint summercms_s3) blank-imports gocloud.dev/blob/s3blob and sets the unexported constant s3Enabled to true; s3_off.go (constraint not summercms_s3) sets it to false. bucket.go exports `S3BuildTag` = summercms_s3 and, in OpenBucket after normalizeBucketURL, refuses an s3 scheme (case-insensitive) when s3Enabled is false with an error that starts with `attach: storage.uploads.bucket_url uses s3://` and names S3BuildTag, `build.tags` in summer.yaml and the `-tags summercms_s3` flag (or GOFLAGS) for a raw go build. Run `go mod tidy` (it adds the AWS SDK v2 modules s3blob needs as indirect requirements); nothing outside the tagged file imports s3blob.
Docs in the same commit: docs/services/storage.md section "S3 and S3-compatible buckets" (the summer.yaml build.tags example, the boot error, raw go build users add -tags summercms_s3, a bucket_url for AWS and one for an S3-compatible endpoint with region, endpoint, use_path_style and the two checksum parameters, credentials from AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY or a profile, never in config, and that the existing /storage/uploads route also serves an S3 bucket); docs/architecture/go-modules-and-workspaces.md manifest example and text for build.tags; docs/console/scaffolding.md summer build and summer.yaml rows mention build tags; docs/setup/configuration.md storage.uploads.bucket_url row lists s3:// with the tag; docs/architecture/performance-and-scaling.md says shared uploads can also be an S3 bucket; lagoon README attach bullet and API row for S3BuildTag. Neutral app names (acme).
Smoke tests: internal/build/manifest_tags_smoke_test.go `TestManifestBuildTagsSmoke` (a temp summer.yaml with build.tags summercms_s3 and a duplicate loads one tag; SaveManifest then LoadManifest keeps it; goBuildArgs contains -tags=summercms_s3 before -o; a tag with a dash or a space is refused). modules/lagoon/attach/infra_smoke_test.go `TestOpenBucketS3Smoke` (t.Setenv for dummy AWS credentials, AWS_REGION and AWS_EC2_METADATA_DISABLED; bucket_url s3://smoke-bucket?region=us-east-1: with s3Enabled false the error contains summercms_s3 and build.tags; with it true the bucket opens and closes without network).</action>
<verify>
<automated>go vet ./... &amp;&amp; go vet -tags summercms_s3 ./modules/lagoon/... &amp;&amp; go build -tags summercms_s3 ./... &amp;&amp; go test -count=1 -short -run '^TestManifestBuildTagsSmoke$' -v ./internal/build 2&gt;&amp;1 | grep -q -- '^--- PASS: TestManifestBuildTagsSmoke' &amp;&amp; go test -count=1 -short -run '^TestOpenBucketS3Smoke$' -v ./modules/lagoon/attach 2&gt;&amp;1 | grep -q -- '^--- PASS: TestOpenBucketS3Smoke' &amp;&amp; go test -count=1 -short -tags summercms_s3 -run '^TestOpenBucketS3Smoke$' -v ./modules/lagoon/attach 2&gt;&amp;1 | grep -q -- '^--- PASS: TestOpenBucketS3Smoke' &amp;&amp; go test -count=1 -short ./internal/build ./modules/lagoon/... &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit: vet fails in either tag mode, the tagged build does not compile, a smoke test did not print "--- PASS" (in either tag mode), the short suites are red, or the docs checker reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -rln 'gocloud.dev/blob/s3blob' --include='*.go' modules internal cmd` prints exactly `modules/lagoon/attach/s3_on.go`.
- `head -1 modules/lagoon/attach/s3_on.go` prints `//go:build summercms_s3` and `head -1 modules/lagoon/attach/s3_off.go` prints `//go:build !summercms_s3`.
- `grep -n 'github.com/aws/aws-sdk-go-v2' go.mod` prints at least one line.
- `grep -n 'build.tags\|summercms_s3' docs/services/storage.md docs/architecture/go-modules-and-workspaces.md` prints at least one line per file.
- `grep -rnE '"s3\.[a-z_]+' --include='*.go' modules/lagoon` prints nothing (no s3.* config keys).
</acceptance_criteria>
<done>An app turns on S3 uploads with one summer.yaml line and a bucket_url, and a binary that lacks the driver says exactly which tag to add.</done>
</task>
<task type="auto">
<name>Task 2: An uploaded PNG named x.html is stored and served as .png: Store derives the disk-name extension from the sniffed type through DiskExtension, with a deploy note on nosniff and allow-lists</name>
<files>modules/lagoon/attach/diskext.go, modules/lagoon/attach/store.go, modules/lagoon/attach/infra_smoke_test.go, modules/lagoon/README.md, docs/database/attachments.md, docs/services/storage.md</files>
<read_first>modules/lagoon/attach/store.go (whole file), modules/lagoon/attach/store_test.go (TestStoreSmoke harness), .planning/todos/pending/2026-10-11-uploads-keep-client-extension-stored-xss.md, .planning/phases/12-p-ytarium-api-collections-and-albums/12-REVIEW.md (WR-01), modules/surf/uploads.go (nosniff, CSP sandbox and attachment disposition already sent by the proxy route), docs/database/attachments.md, docs/services/storage.md (Serving files), 19.3-RESEARCH.md (Pattern 6 and its DetectContentType probe table, Pitfall 8)</read_first>
<action>Per D-15. diskext.go: `DiskExtension(contentType, clientExt string) string` (clientExt is the lower-cased, already allow-listed extension without a dot; contentType is a base media type). A table maps specific sniffed types to accepted extension sets with the first entry canonical: image/png png; image/jpeg jpg, jpeg, jpe; image/gif gif; image/webp webp; image/bmp bmp; image/x-icon and image/vnd.microsoft.icon ico; application/pdf pdf; audio/mpeg mp3; audio/wave and audio/wav wav; application/ogg and audio/ogg ogg, oga, opus; video/webm webm, mkv; video/mp4 mp4, m4v; font/woff2 woff2; font/woff woff; font/ttf ttf; font/otf otf; application/vnd.ms-fontobject eot; application/x-rar-compressed rar; application/x-gzip gz, tgz. For a table type the client extension is kept when it is in the set, else the canonical extension is returned. Any other type is generic: the client extension is kept unless it is script-capable (html, htm, xhtml, xht, shtml, svg, svgz, xml, xsl, xslt, js, mjs, css, swf, php, phtml, pht, phar), which yields txt for a text/* type and bin otherwise. The helper never asks package mime for extensions (its answer depends on the host's mime tables). store.go: keep the sniffed base type in its own variable before the octet-stream fallback, and name the blob with newDiskName(DiskExtension(sniffed, ext)); file_name, the allow-list check and the stored content_type rule are unchanged.
Docs in the same commit: docs/database/attachments.md explains that the disk name's extension follows the content, with the x.html example, and that file_name keeps the client's name; docs/services/storage.md gains a deploy note: when a static server (nginx) serves the uploads directory or bucket directly, send X-Content-Type-Options nosniff and allow only the file types the app accepts, as the /storage/uploads route already does; lagoon README API row for DiskExtension and the Store description.
Smoke test `TestStoreDiskExtensionSmoke` appended to modules/lagoon/attach/infra_smoke_test.go (skips under -short; reuses the TestStoreSmoke Postgres harness and a mem:// bucket; Limits.Extensions widened to html, png, docx and txt): an 8-byte PNG signature plus IHDR uploaded as x.html gives a DiskName ending .png, a URL ending .png, FileName x.html and ContentType image/png; a zip body uploaded as report.docx keeps .docx; an HTML body uploaded as note.txt keeps .txt.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/lagoon/... &amp;&amp; go test -count=1 -run '^TestStoreDiskExtensionSmoke$' -v ./modules/lagoon/attach 2&gt;&amp;1 | grep -q -- '^--- PASS: TestStoreDiskExtensionSmoke' &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit: vet or the lagoon short suites are red, the Store regression smoke did not print "--- PASS" (skipped without Docker, or failed), or the docs checker reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'DiskExtension(' modules/lagoon/attach/store.go` prints one line.
- `grep -c 'ExtensionsByType' modules/lagoon/attach/diskext.go` prints 0.
- `grep -n 'nosniff' docs/services/storage.md` prints at least one line and `grep -n 'DiskExtension' modules/lagoon/README.md` prints at least one line.
</acceptance_criteria>
<done>A disguised HTML upload can no longer be served back as HTML from its own URL, and real documents keep their extensions.</done>
</task>
<task type="auto">
<name>Task 3: Files on a bucket or CDN get direct URLs through storage.uploads.public_url, while apps without it keep the /storage/uploads proxy URLs</name>
<reversibility rating="costly">D-12: storage.uploads.public_url is referenced by app configs and determines the URLs clients store.</reversibility>
<files>modules/lagoon/attach/bucket.go, modules/lagoon/attach/thumb.go, modules/lagoon/attach/infra_smoke_test.go, modules/lagoon/README.md, docs/services/storage.md, docs/database/attachments.md, docs/setup/configuration.md</files>
<read_first>modules/lagoon/attach/bucket.go (prefixMu, setPublicPathPrefix, OpenBucket), modules/lagoon/attach/thumb.go (:79-99), modules/lagoon/attach/url_test.go, modules/surf/uploads.go (uploadsPrefix validation as the model for URL checks), docs/services/storage.md (Bucket URLs, The WinterCMS layout, Serving files), docs/database/attachments.md (:80-90), docs/setup/configuration.md, 19.3-RESEARCH.md (Pitfall 15)</read_first>
<action>Per D-12. bucket.go: a second package value next to publicPathPrefix under the same prefixMu, set by OpenBucket from storage.uploads.public_url (trimmed; empty means unset) and read by `PublicURLBase() string`. A set value must parse as an absolute http or https URL with a host, without user info, query or fragment; otherwise OpenBucket fails with an error naming storage.uploads.public_url and SUMMER_STORAGE__UPLOADS__PUBLIC_URL. When set, OpenBucket logs one info line naming the host and stating that every file URL points there, so the bucket behind it must hold only public files (the proxy route checks is_public, a public bucket or CDN does not). thumb.go PublicURL: when PublicURLBase is non-empty the URL is that base with trailing slashes trimmed, one slash and the key; otherwise the current prefix logic, unchanged. File.URL and Thumb follow through PublicURL.
Docs in the same commit: docs/services/storage.md section "Public URL base" (bucket or CDN base example under a neutral domain, the partition path layout, the public-files-only rule, that leaving it unset keeps proxy URLs, which work for S3 too); docs/database/attachments.md URL paragraph mentions public_url; docs/setup/configuration.md row storage.uploads.public_url; lagoon README API rows PublicURLBase and the PublicURL description.
Smoke test `TestPublicURLSmoke` appended to modules/lagoon/attach/infra_smoke_test.go (no Docker): OpenBucket on mem:// with public_url https://cdn.example.com/uploads/ makes PublicURL of a partitioned key https://cdn.example.com/uploads/<key>; without it the URL starts with /storage/uploads; ftp://x and https://u:p@cdn.example.com fail naming storage.uploads.public_url. Because the base is package state, the test restores it with t.Cleanup.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/lagoon/... ./internal/build &amp;&amp; go test -count=1 -short -run '^(TestPublicURLSmoke|TestOpenBucketS3Smoke)$' -v ./modules/lagoon/attach 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 2 &amp;&amp; go test -count=1 -short ./... &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; go run ./cmd/summer docs:build --check</automated>
<fails_when>Non-zero exit: vet or a short suite is red, fewer than two "--- PASS" lines for the URL and S3 smokes, or the docs checker or docs:build --check reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'func PublicURLBase() string' modules/lagoon/attach/bucket.go` prints one line.
- `grep -n 'storage.uploads.public_url' docs/services/storage.md docs/setup/configuration.md modules/lagoon/README.md` prints at least one line per file.
- `grep -rniE 'wavepath|fonoteka|grzyby' modules/lagoon docs/services/storage.md docs/database/attachments.md` prints nothing.
</acceptance_criteria>
<done>An app on S3 or behind a CDN serves file URLs straight from that base; apps that set nothing see the same URLs as before.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| client upload to attach.Store | untrusted bytes and an untrusted file name |
| bucket or CDN to the browser | stored files served from the app's or a CDN's origin |
| summer.yaml to go build | manifest values become command arguments |
| AWS environment to s3blob | credentials outside summer config |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-19.3-18 | Tampering / Elevation of privilege | attach.Store disk name (stored XSS) | high | mitigate | DiskExtension from the sniffed type, script-capable extensions never kept under a generic sniff; TestStoreDiskExtensionSmoke; deploy note on nosniff and allow-lists |
| T-19.3-19 | Information disclosure | storage.uploads.public_url exposing non-public files | medium | mitigate | start-up info line and docs state the public-files-only rule; blob keys stay unguessable (22 random hex characters); unset keeps the is_public-checking proxy |
| T-19.3-20 | Tampering | summer.yaml build.tags reaching go build | medium | mitigate | ValidateBuildTag ^[a-z0-9_]+$ at load and save; tags passed as one exec argument, no shell |
| T-19.3-21 | Repudiation / misconfiguration | s3:// in a binary built without the tag | medium | mitigate | OpenBucket boot error naming summercms_s3, build.tags and -tags; summer build prints the tags |
| T-19.3-22 | Information disclosure | AWS credentials | medium | mitigate | credentials only from the AWS environment or profile (D-14); docs forbid keys in config; gocloud s3 URLs carry no secrets |
| T-19.3-SC | Tampering | Go module installs (AWS SDK v2 via gocloud.dev/blob/s3blob) | high | mitigate | s3blob is a sub-package of the already-required gocloud.dev v0.46.0; its AWS modules are named in plan 01's decision note before go mod tidy; linked only with the summercms_s3 tag |
</threat_model>
<verification>
- Task 1: vet in both tag modes, the tagged build, the manifest and S3 smokes (both modes), the short suites, TestDocsTree.
- Task 2: the lagoon short suites, the Postgres Store regression smoke, TestDocsTree.
- Task 3: the URL and S3 smokes, the whole short suite, TestDocsTree and docs:build --check.
- Feedback sampling (19.3-VALIDATION.md): after each commit `go vet ./... && go test -short ./modules/lagoon/... ./internal/build`; per wave also `go vet -tags summercms_s3 ./modules/lagoon/... && go test -short -tags summercms_s3 ./modules/lagoon/attach`.
</verification>
<success_criteria>
- SC-3: an S3 blob driver is selected through framework config and summer.yaml; the application imports nothing.
- D-15: disk names follow content; the regression smoke passes.
- D-12: public_url builds direct URLs; unset keeps proxy URLs.
- go vet (both tag modes) and the short suite green at every commit.
</success_criteria>
<output>
Create `.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-03-SUMMARY.md` when done, with the commits, the identifiers and keys added, the go.mod additions, the Docker smoke run or NOT RUN, and any deviation.
</output>

View File

@@ -0,0 +1,226 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 03
subsystem: storage
tags: [lagoon, attach, s3, gocloud, build-tags, summer-build, stored-xss, cdn]
requires:
- phase: 19.3-01
provides: "decision note .planning/notes/2026-10-15-cooler-go-redis-otter.md naming gocloud.dev/blob/s3blob and its AWS SDK v2 modules"
provides:
- "summer.yaml build.tags (Manifest.Tags, ValidateBuildTag, goBuildArgs, build tags output line)"
- "attach.S3BuildTag (summercms_s3): s3blob linked only by s3_on.go; an untagged binary refuses s3:// at boot naming the tag"
- "attach.DiskExtension: stored extension follows the sniffed content (stored-XSS fix)"
- "storage.uploads.public_url and attach.PublicURLBase for direct bucket or CDN URLs"
affects: [19.3-05, fonoteka-upload-call-sites-follow-up, apps-on-s3]
actuals:
tokens: 15400
tasks: 3
commits: 3
plan_head_before: b1fab9930b11090d7a5515f05a3842e15be06809
plan_head_after: 2e87efae54141a12a197f1e8de844a1996ffba32
tech-stack:
added: ["gocloud.dev/blob/s3blob (sub-package of gocloud.dev v0.46.0, summercms_s3 tag only)", "github.com/aws/aws-sdk-go-v2 v1.41.9 and 17 sibling modules (indirect)", "github.com/aws/smithy-go v1.26.0 (indirect)", "github.com/google/wire v0.7.0 (indirect)"]
patterns:
- "Feature build tag split: s3_on.go (//go:build summercms_s3) blank-imports the driver and sets a const; s3_off.go sets it false; the boot guard names the tag"
- "Manifest keys must be written back by SaveManifest so summer plugin:add round-trips them"
- "Extension derived from content through a fixed in-code table, never package mime"
key-files:
created:
- internal/build/manifest_tags_smoke_test.go
- modules/lagoon/attach/s3_on.go
- modules/lagoon/attach/s3_off.go
- modules/lagoon/attach/diskext.go
- modules/lagoon/attach/infra_smoke_test.go
modified:
- go.mod
- go.sum
- internal/build/manifest.go
- internal/build/build.go
- modules/lagoon/attach/bucket.go
- modules/lagoon/attach/store.go
- modules/lagoon/attach/store_test.go
- modules/lagoon/attach/thumb.go
- modules/lagoon/README.md
- docs/services/storage.md
- docs/database/attachments.md
- docs/architecture/go-modules-and-workspaces.md
- docs/architecture/performance-and-scaling.md
- docs/console/scaffolding.md
- docs/setup/configuration.md
key-decisions:
- "Build tag summercms_s3 (attach.S3BuildTag), declared in summer.yaml as build.tags; tags match ^[a-z0-9_]+$, duplicates dropped, passed to go build as one -tags= exec argument; summer build prints a build tags line"
- "DiskExtension: specific sniffed types keep a matching client extension or use the canonical one; generic sniffs keep the client extension unless script-capable (txt for text/*, bin otherwise); Store passes the sniffed type before its octet-stream extension fallback"
- "storage.uploads.public_url applies to every URL attach builds, private files included; the start-up info line and the docs carry the public-files-only rule"
- "public_url is validated before the bucket opens (absolute http/https, host, no user info, query or fragment); errors name the key and SUMMER_STORAGE__UPLOADS__PUBLIC_URL without echoing the value"
patterns-established:
- "Tag-mode smoke tests branch on the unexported const (s3Enabled) so one test passes in both tag modes"
requirements-completed: [SC-3, SC-4, D-12, D-13, D-14, D-15, D-19]
coverage:
- id: D1
description: "summer.yaml build.tags loads (dedup, validation), round-trips through SaveManifest, and reaches go build as -tags before -o"
requirement: D-13
verification:
- kind: unit
ref: "internal/build/manifest_tags_smoke_test.go#TestManifestBuildTagsSmoke"
status: pass
human_judgment: false
- id: D2
description: "Untagged binary refuses s3:// naming summercms_s3, build.tags and -tags; tagged binary opens s3://smoke-bucket without network"
requirement: D-13
verification:
- kind: unit
ref: "modules/lagoon/attach/infra_smoke_test.go#TestOpenBucketS3Smoke (go test -short, and go test -short -tags summercms_s3)"
status: pass
human_judgment: false
- id: D3
description: "Real upload to an S3-compatible bucket (R2/MinIO/Hetzner) through the tagged binary"
requirement: SC-3
human_judgment: true
rationale: "No real S3 or MinIO available; manual UAT item in 19.3-VALIDATION.md"
- id: D4
description: "PNG uploaded as x.html is stored and served as .png with file_name x.html; report.docx keeps .docx; HTML body as note.txt keeps .txt"
requirement: D-15
verification:
- kind: integration
ref: "modules/lagoon/attach/store_test.go#TestStoreDiskExtensionSmoke (Docker, postgres:16-alpine)"
status: pass
human_judgment: false
- id: D5
description: "public_url set gives base plus partition path for PublicURL and File.URL; unset keeps /storage/uploads; ftp://, user info, a bare path and a query fail naming the key"
requirement: D-12
verification:
- kind: unit
ref: "modules/lagoon/attach/infra_smoke_test.go#TestPublicURLSmoke"
status: pass
human_judgment: false
- id: D6
description: "Docs: S3 section, Public URL base, deploy note (nosniff, allow-list), disk-name paragraph, build.tags in manifest/scaffolding docs, configuration rows, scaling note, lagoon README"
requirement: SC-4
verification:
- kind: other
ref: "go test ./cmd/summer -run TestDocsTree; go run ./cmd/summer docs:build --check"
status: pass
human_judgment: false
duration: 10min
completed: 2026-10-15
status: complete
---
# Phase 19.3 Plan 03: S3 build tag, content-derived disk names and public_url Summary
**`summer.yaml` `build.tags` with the `summercms_s3` tag that links gocloud's s3blob only into tagged binaries (untagged ones name the tag at boot), `attach.DiskExtension` so a PNG uploaded as `x.html` is stored and served as `.png`, and `storage.uploads.public_url` for direct bucket or CDN file URLs**
## Performance
- **Duration:** 10 min
- **Started:** 2026-10-15T16:03:46Z
- **Completed:** 2026-10-15T16:13:06Z
- **Tasks:** 3 (Task 1 tracer, Tasks 2 and 3 expansion)
- **Files modified:** 20
## Accomplishments
- `internal/build`: `Manifest.Tags` read from the optional `build.tags` list (trimmed, `ValidateBuildTag` against `^[a-z0-9_]+$`, duplicates dropped, a non-mapping `build` or non-list `tags` is an error naming `build.tags`). `SaveManifest` writes the `build:` block after the plugins, so `summer plugin:add` keeps it. `goBuildArgs` puts `-tags=<comma list>` before `-o`, and `App` prints `build tags: …`. `summer dev` inherits this through `build.App`.
- `attach`: `s3_on.go` (`//go:build summercms_s3`) is the only importer of `gocloud.dev/blob/s3blob`. `OpenBucket` refuses an `s3://` URL (case-insensitive scheme) in an untagged binary. The error starts with `attach: storage.uploads.bucket_url uses s3://` and names `summercms_s3`, `build.tags` in summer.yaml, `-tags summercms_s3` and `GOFLAGS`. No `s3.*` keys exist. The S3 configuration lives in the gocloud `bucket_url` query, and credentials come from the AWS environment.
- `attach.DiskExtension(contentType, clientExt)` uses a fixed table of 22 specific sniffed types and a list of script-capable extensions. `Store` names the blob with `newDiskName(DiskExtension(sniffed, ext))`. `file_name`, the allow-list and the stored `content_type` rule are unchanged.
- `storage.uploads.public_url`: OpenBucket validates and records it (`PublicURLBase()`) next to the path prefix under `prefixMu` and logs one info line naming the host and the public-files-only rule. `PublicURL` (and therefore `File.URL` and thumbnail URLs) uses the base when it is set. Otherwise the URLs are unchanged.
- Docs: `docs/services/storage.md` gained the sections "S3 and S3-compatible buckets" and "Public URL base", a deploy note on nosniff and allow-lists, and an s3 row in the bucket URL table. Also updated: the disk-name paragraph and URL sentence in `attachments.md`, `build.tags` in `go-modules-and-workspaces.md`, the `summer build`/`summer dev`/`summer.yaml` rows in `scaffolding.md`, the `bucket_url` and `public_url` rows in `configuration.md`, the shared-uploads note in `performance-and-scaling.md`, and the lagoon README (overview, features, API rows `S3BuildTag`, `DiskExtension`, `PublicURLBase`, `PublicURL`, `OpenBucket`, `Store`, config rows, dependencies).
## Task Commits
1. **Task 1 (tracer): build.tags and the summercms_s3 S3 driver tag** - `2382040` (feat)
2. **Task 2: content-derived disk names through attach.DiskExtension** - `2e22e9f` (fix)
3. **Task 3: storage.uploads.public_url** - `2e87efa` (feat)
**Plan metadata:** the final docs(19.3-03) commit.
## Identifiers and keys added
- internal/build: `Manifest.Tags`, `ValidateBuildTag`, unexported `goBuildArgs(binRel string, tags []string) []string` and `loadBuildTags`; summer.yaml key `build.tags`; output line `build tags: <list>`.
- attach: `S3BuildTag` (`summercms_s3`), unexported `s3Enabled` and `checkS3Driver`; `DiskExtension`; `PublicURLBase`, unexported `setPublicURLBase` and `parsePublicURL`; config key `storage.uploads.public_url` (env `SUMMER_STORAGE__UPLOADS__PUBLIC_URL`).
- Tests: `TestManifestBuildTagsSmoke`, `TestOpenBucketS3Smoke` (both tag modes), `TestStoreDiskExtensionSmoke` (Postgres), `TestPublicURLSmoke`.
## go.mod additions
`go mod tidy` added indirect requirements only: `github.com/aws/aws-sdk-go-v2` v1.41.9 with `config`, `credentials`, `feature/ec2/imds`, `feature/s3/transfermanager`, `internal/configsources`, `internal/endpoints/v2`, `internal/v4a`, `aws/protocol/eventstream`, `service/s3` v1.102.2, `service/internal/{accept-encoding,checksum,presigned-url,s3shared}`, `service/{signin,sso,ssooidc,sts}`, `github.com/aws/smithy-go` v1.26.0 and `github.com/google/wire` v0.7.0. go.sum also gained hashes for `google/go-replayers` and `google/martian/v3` (test-only modules of gocloud). None of them is linked into a binary without the tag. `grep -rln gocloud.dev/blob/s3blob` over modules, internal and cmd lists only `s3_on.go`.
## Docker smoke tests
Run on this host (the Docker daemon answered): `TestStoreDiskExtensionSmoke` PASS (postgres:16-alpine). The non-short `go test ./modules/lagoon/...` run after Task 2 was green, and so were the non-short `TestPublicURL|TestFileURL|TestStatic|TestOpenBucket` runs after Task 3. No real S3 or MinIO was available: the S3 smoke only opens the bucket (s3blob connects lazily), so a real upload stays a manual UAT item (19.3-VALIDATION.md).
## Verification
- Task 1: `go vet ./...` and `go vet -tags summercms_s3 ./modules/lagoon/...` pass. `go build -tags summercms_s3 ./...` passes. Both smokes passed in both tag modes, and the short suites and TestDocsTree passed.
- Task 2: vet, the lagoon short suites, the Postgres regression smoke and TestDocsTree passed.
- Task 3: both smokes passed (2 PASS lines). The whole `go test -short ./...` run, TestDocsTree and `summer docs:build --check` ("no problems found") passed.
- Wave sampling: `go vet -tags summercms_s3 ./modules/lagoon/... && go test -short -tags summercms_s3 ./modules/lagoon/attach` is green.
## Decisions Made
See key-decisions in the frontmatter. All come from the plan's "Claude's discretion" list.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] The Store regression smoke lives in store_test.go, not infra_smoke_test.go**
- **Found during:** Task 2
- **Issue:** The Postgres harness `attachGorm` is in the external test package `attach_test` (lifecycle_test.go). `infra_smoke_test.go` has to be in the internal package `attach`, because `TestOpenBucketS3Smoke` reads the unexported `s3Enabled` and `TestPublicURLSmoke` resets the package URL state. A file cannot belong to both packages.
- **Fix:** `TestStoreDiskExtensionSmoke` is appended to `modules/lagoon/attach/store_test.go` next to `TestStoreSmoke`, whose harness it reuses. The test name and the `-run` filter in the plan's verify command are unchanged. PATTERNS.md suggested this file too.
- **Files modified:** modules/lagoon/attach/store_test.go
- **Committed in:** 2e22e9f
**2. [Rule 1 - Acceptance gate] A comment in bucket.go named the s3blob import path**
- **Found during:** Task 1 acceptance criteria (only `s3_on.go` may mention `gocloud.dev/blob/s3blob`)
- **Fix:** The `S3BuildTag` doc comment now says "s3blob, imported only by s3_on.go". The fix was made before the Task 1 commit.
- **Committed in:** 2382040
**3. [Rule 2 - Hardening] DiskExtension normalises its inputs**
- **Found during:** Task 2
- **Issue:** `DiskExtension` is exported, and the deferred app call sites will pass `filepath.Ext` output and raw content types.
- **Fix:** It strips media-type parameters, lower-cases both inputs and trims a leading dot. A client extension that fails the `[a-z0-9]{1,10}` pattern falls back to `txt` or `bin` like a script-capable one. `Store`'s inputs are already normalised, so its behaviour does not change.
- **Committed in:** 2e22e9f
### Notes
- `github.com/google/wire` v0.7.0 is an indirect requirement of s3blob that the plan 01 decision note does not name (the note names the AWS SDK v2 modules). It comes with the already-required gocloud.dev v0.46.0 and is linked only under the tag.
- The Task 3 acceptance grep for consumer names over `modules/lagoon` matches three pre-existing test files that this plan does not touch. This plan added no matching line (`git diff` count 0). The finding is logged in `deferred-items.md`, together with a pre-existing gofmt issue in `internal/build/registry.go`.
---
**Total deviations:** 3 auto-fixed (1 blocking, 1 acceptance gate, 1 hardening).
**Impact on plan:** None on scope. The identifiers, keys and behaviour match the plan.
## Issues Encountered
None.
## Known Stubs
None.
## Threat Flags
None. All new surface is in the plan's threat register: T-19.3-18 to T-19.3-22 and T-19.3-SC.
## User Setup Required
None for the framework. An app that switches to S3 adds `summercms_s3` under `build.tags`, sets an `s3://` `bucket_url` and provides AWS credentials in the environment. An app with a second raw `go build` in its build script must also add `-tags summercms_s3` there.
## Next Phase Readiness
- Plan 05 owns full unit coverage: a `DiskExtension` table test (every table type, the script-capable list, input normalisation), `ValidateBuildTag` and loader edge cases (non-mapping `build`, non-list `tags`), and `parsePublicURL` cases.
- Follow-up outside this phase: move the app upload call sites (`storeUpload`, `StorePublicFile`) to `attach.DiskExtension`. The app build script needs `-tags summercms_s3` when the app moves to S3.
---
*Phase: 19.3-framework-transport-and-infrastructure-for-wavepath*
*Completed: 2026-10-15*

View File

@@ -0,0 +1,346 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 04
type: execute
wave: 2
depends_on: ["19.3-01"]
files_modified:
- modules/cooler/sitecache.go
- modules/cooler/cooler.go
- modules/cooler/sitecache_test.go
- modules/cooler/README.md
- docs/services/caching.md
- README.md
- modules/pact/capabilities.go
- modules/pact/README.md
- modules/cabana/reorder.go
- modules/cabana/README.md
- docs/backend/admin-controllers.md
- docs/backend/lists-and-filters.md
- ../sm-faq-plugin/controllers/faqs.go
- ../sm-faq-plugin/controllers/site_cache.go
- ../sm-faq-plugin/site_cache_admin_test.go
- ../sm-faq-plugin/README.md
- ../sm-menu-plugin/controllers/admin_registry.go
- ../sm-menu-plugin/controllers/menus.go
- ../sm-menu-plugin/controllers/menuentries.go
- ../sm-menu-plugin/controllers/site_cache.go
- ../sm-menu-plugin/site_cache_admin_test.go
- ../sm-menu-plugin/README.md
- ../sm-journal-plugin/plugin.go
- ../sm-journal-plugin/classes/site_cache.go
- ../sm-journal-plugin/classes/site_cache_test.go
- ../sm-journal-plugin/controllers/site_cache.go
- ../sm-journal-plugin/README.md
- ../vue-faq-layer/server/lib/siteCachePurge.ts
- ../vue-faq-layer/server/api/__site__/cache/purge.post.ts
- ../vue-faq-layer/nuxt.config.ts
- ../vue-faq-layer/tests/sitePurge.test.ts
- ../vue-faq-layer/README.md
- ../vue-faq-layer/package.json
- ../vue-menu-layer/server/lib/siteCachePurge.ts
- ../vue-menu-layer/server/api/__site__/cache/purge.post.ts
- ../vue-menu-layer/nuxt.config.ts
- ../vue-menu-layer/tests/sitePurge.test.ts
- ../vue-menu-layer/README.md
- ../vue-menu-layer/package.json
- ../vue-journal-layer/server/lib/siteCachePurge.ts
- ../vue-journal-layer/server/api/__site__/cache/purge.post.ts
- ../vue-journal-layer/nuxt.config.ts
- ../vue-journal-layer/tests/sitePurge.test.ts
- ../vue-journal-layer/README.md
- ../vue-journal-layer/package.json
- ../sm-grzybyfunkcjonalne-app/go.mod
- ../sm-grzybyfunkcjonalne-app/go.sum
- ../sm-grzybyfunkcjonalne-app/plugins/golem15/journal
- ../sm-grzybyfunkcjonalne-app/plugins/golem15/menu
- ../sm-grzybyfunkcjonalne-app/plugins/golem15/faq
- ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app
- ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/package.json
- ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/pnpm-lock.yaml
autonomous: false
requirements: [SC-4, D-10, D-11, D-19]
estimate:
tokens: 220000
raw_tokens: 220000
tasks: 5
confidence: low
must_haves:
truths:
- "D-10: cooler holds the site-cache purge service: one target from site_cache.purge_url and site_cache.purge_token (http only to a loopback host, token of at least 32 characters, errors name the key and never the token); PurgeSiteCacheAfterCommit registers one bearer-token POST of {kind, slug} through lagoon.AfterCommit, so it runs only after the admin save commits; a failure or timeout (2 s) is logged and never fails the save; one POST per committed save, no debounce (user answer)."
- "D-10: serve validates and publishes the purger at boot (a bad site_cache config fails boot); cooler.SiteCacheFromApp gives hooks the same purger lazily in commands and tests that do not run serve; cooler imports neither surf nor cabana (plugins take the save transaction from cabana.TxFromContext themselves)."
- "D-11: the FAQ plugin purges after a FAQ create, update or delete; the menu plugin purges after a menu create, update or delete and after a menu-entry create, update, delete or reorder; the existing FAQ locale check and menu placement checks run first and still refuse bad saves."
- "D-11: cabana's reorder save calls a new optional pact.ReorderAfterSave(ctx, partition) hook inside its transaction after the new order is written; a hook error rolls the reorder back; controllers without the hook behave exactly as before."
- "D-11: the Journal plugin moves onto cooler.SiteCachePurger: framework keys win; when only golem15.journal.site_cache.purge_url and purge_token are set they still work for one release with a deprecation warning; classes.SiteCachePurger, NewSiteCachePurger and PurgeSiteCacheAfterCommit stay as aliases, so no Journal deployment or importer breaks."
- "D-11: vue-journal-layer, vue-menu-layer and vue-faq-layer each ship the same shared purge route POST /api/__site__/cache/purge (runtimeConfig siteCache.purgeToken, NUXT_SITE_CACHE_PURGE_TOKEN) with its own '/api/__site__/**': { cache: false } route rule, so a site without Journal still gets purges; the two shared files are byte-identical across the three layers (pinned hash in each layer's test); vue-journal-layer keeps /api/__journal__/cache/purge and journal.cachePurgeToken as aliases for one release."
- "D-11: commit order is framework, then plugin and layer repos (pushed and tagged only on the human's reply), then the app pointer bumps as separate commits."
artifacts:
- path: "modules/cooler/sitecache.go"
provides: "SiteCachePurger moved from the Journal plugin with framework keys, SiteCacheFromApp, PurgeSiteCacheAfterCommit"
contains: "site_cache.purge_url"
- path: "modules/pact/capabilities.go"
provides: "the optional ReorderAfterSave hook"
contains: "ReorderAfterSave"
- path: "../vue-faq-layer/server/api/__site__/cache/purge.post.ts"
provides: "the shared Nitro purge route"
contains: "siteCache"
- path: "../sm-menu-plugin/controllers/menuentries.go"
provides: "menu-entry purge hooks including reorder"
contains: "ReorderAfterSave"
key_links:
- from: "../sm-faq-plugin/controllers/faqs.go"
to: "modules/cooler/sitecache.go"
via: "FormAfterCreate/Update/Delete call cooler.PurgeSiteCacheAfterCommit with cooler.SiteCacheFromApp and cabana.TxFromContext"
pattern: "PurgeSiteCacheAfterCommit"
- from: "modules/cabana/reorder.go"
to: "modules/pact/capabilities.go"
via: "reorderPost type-asserts pact.ReorderAfterSave inside the Transaction closure with txCtx"
pattern: "pact.ReorderAfterSave"
- from: "modules/cooler/cooler.go"
to: "modules/cooler/sitecache.go"
via: "Install builds and publishes the purger from site_cache.* unless a plugin already published one"
pattern: "SiteCachePurger"
- from: "../sm-journal-plugin/plugin.go"
to: "modules/cooler/sitecache.go"
via: "wireSiteCache publishes a cooler purger from the alias keys only when the framework keys are unset"
pattern: "NewSiteCachePurgerFor"
prohibitions:
- statement: "No breaking change to the Journal, menu or FAQ plugins: existing keys, exported names, routes and admin behaviour keep working."
status: unresolved
verification: judgment
- statement: "Nothing is pushed or tagged in any repository without the human's reply at the Task 4 checkpoint."
status: unresolved
verification: judgment
- statement: "Framework code and docs never name a consuming application."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not in user-story format; no story is invented, see plan 01): the framework has the transport and infrastructure pieces a multi-client, multi-instance application needs: JWT cookie refresh across subdomains, credentialed CORS, a cache module with a Redis-backed rate-limit store, an S3 blob option, an OpenAPI page and Turnstile verification.
This plan's slice (plan 4 of 5, wave 2): the folded site-cache purge todo (D-10, D-11). The Journal-only purger becomes a framework service in `cooler`; the FAQ plugin and vue-faq-layer prove it end to end (the tracer), cabana gains a reorder hook so the menu plugin purges after saves, deletes and reorders, Journal moves onto the service with alias keys, the shared Nuxt route lands in all three layers, and, after the human's go-ahead, the app pointers move.
## Spec-less probe fallback
Skipped visibly: the phase has no requirement IDs to probe, so no probe predicates were generated (see plan 01).
## Decisions taken in this plan (Claude's discretion)
- Identifiers: `cooler.SiteCachePurger`, `cooler.NewSiteCachePurger(purgeURL, token)` (errors name the framework keys), `cooler.NewSiteCachePurgerFor(urlKey, tokenKey, purgeURL, token)` (errors name the given keys, used by the Journal alias path), `cooler.PurgeSiteCacheAfterCommit`, `cooler.SiteCacheFromApp`, constants `SiteCachePurgeURLKey`, `SiteCachePurgeTokenKey`, `SiteCachePurgeTimeout`, `SiteCachePurgeMinTokenLength`.
- Purge kinds: `faq` (slug = the FAQ slug), `menu` (slug = the menu code), `menu_entry` (slug = the entry id, or `partition:<id>` for a reorder), Journal keeps `post` and `category`. The Nuxt route purges every route-cache entry regardless of kind; the kind only reaches its log line.
- The hook is `pact.ReorderAfterSave` with `ReorderAfterSave(ctx context.Context, partition *uint64) error`, the reorder request's partition, so a partitioned reorder (menu entries per menu) can name what changed.
- The shared Nuxt helper lives in `server/lib/siteCachePurge.ts`, outside Nitro's and Nuxt's auto-import directories, so three layers can carry the same exports without duplicate auto-import warnings; the route imports it by relative path. Its exported names differ from the Journal layer's `app/utils/siteCache.ts` (kept unchanged for the alias route).
- Layer releases are additive minors: vue-faq-layer 0.2.0, vue-menu-layer 0.2.0, vue-journal-layer 0.4.0. Plugin repos get no tag (apps mount them as submodules). Plugin go.mod require lines stay on the current framework tag; the plugins build through their checked-in go.work replace until plan 05's release decision tags the framework.
- Only `sm-grzybyfunkcjonalne-app` gets pointer bumps here: it mounts all three plugins and its pointers sit at the plugins' pre-adoption heads. `sm-summercmsio-app` mounts Journal at a commit 45 commits behind main, so its bump is a separate follow-up recorded in the SUMMARY, not part of this phase.
## Artifacts this phase produces (this plan)
- `modules/cooler/sitecache.go`: the identifiers listed in Decisions; config keys `site_cache.purge_url` (SUMMER_SITE_CACHE__PURGE_URL) and `site_cache.purge_token` (SUMMER_SITE_CACHE__PURGE_TOKEN); `Install` publishes the purger.
- `modules/pact/capabilities.go`: `pact.ReorderAfterSave`; `modules/cabana/reorder.go` calls it inside the reorder transaction.
- Plugins: FAQ `FormAfterCreate`/`FormAfterUpdate` (purge appended) and `FormAfterDelete`; menu `menusAdminController` with an app accessor and `FormAfterCreate`/`FormAfterUpdate`/`FormAfterDelete`; `menuEntriesAdminController` `FormAfterDelete` and `ReorderAfterSave`; a `purgeSiteCache` helper in each plugin's controllers package; Journal alias wiring and `classes` aliases.
- Layers: `server/lib/siteCachePurge.ts` (`MIN_SITE_PURGE_TOKEN_LENGTH`, `SITE_ROUTE_CACHE_BASE`, `authorizeSitePurge`, `purgeSiteRouteCache`, `parseSitePurgeEvent`), `server/api/__site__/cache/purge.post.ts`, runtimeConfig `siteCache.purgeToken`, route rule `/api/__site__/**`, specs `tests/sitePurge.test.ts` (names start with `sitePurge:`), versions 0.2.0 / 0.2.0 / 0.4.0, tags `v0.2.0` (faq, menu) and `v0.4.0` (journal) on the human's reply.
- Tests: `modules/cooler/sitecache_test.go` (the Journal purger tests ported: `TestSiteCachePurgeConfig`, `TestSiteCachePurgeRequest`); `TestFaqAdminPurgesSiteCache` and `TestMenuAdminPurgesSiteCache` (Postgres) in the plugins. Full unit coverage is plan 05.
- Docs: `docs/services/caching.md` section "Site cache purge"; cooler README and root README summary sentence; pact and cabana READMEs; `docs/backend/admin-controllers.md` hook table row; `docs/backend/lists-and-filters.md` reorder hook note; plugin and layer READMEs.
<objective>
Turn the Journal-only site-cache purge into a framework service that every admin save can use, adopt it in the FAQ, menu and Journal plugins (with a new reorder hook in cabana) and ship the shared Nuxt purge route in all three layers, committing framework first, then plugin and layer repos, pushing only on the human's go-ahead, then moving the app pointers.
Purpose: D-10 and D-11 (the folded todo `site-cache-purge-for-menu-and-faq`), placement per D-19, docs per SC-4.
Output: commits in summercms.go, three plugin repos, three layer repos and, after the reply, the app and its vue app.
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-CONTEXT.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-RESEARCH.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-PATTERNS.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-01-SUMMARY.md
@.planning/notes/core-plugins-own-repos.md
@.planning/todos/pending/site-cache-purge-for-menu-and-faq.md
Repositories (all siblings of summercms.go under /media/nvme/dev/golem15/summercms.io/summercms): sm-faq-plugin (main 919cf06 at planning time), sm-menu-plugin (main 9c0f833), sm-journal-plugin (main 31d901c, tags up to v0.1.1), vue-faq-layer (main 321bc90, v0.1.0), vue-menu-layer (main 2deac88, v0.1.0), vue-journal-layer (main 25e9bfe, v0.3.0), sm-grzybyfunkcjonalne-app (submodules plugins/golem15/{journal,menu,faq} and vue-grzybyfunkcjonalne-app, which pins the three layers by tag in package.json). Each plugin's checked-in go.work replaces summercms with ../summercms.go, so plugin tests compile against this checkout. Layers run `node --test` through `pnpm --dir <layer> test`.
Concurrency and commit rules: plans 19.3-02 and 19.3-03 run in the same wave in summercms.go and own other files. Phase 18.1 plan 08 (pending at planning time) edits modules/pact/capabilities.go, modules/cabana/crud.go, the pact and cabana READMEs and docs/backend/admin-controllers.md; Task 2 asserts a clean tree for those paths first. Commit in each repo separately, staging by explicit path, no co-author tags; never stage files inside a submodule from the app repo; use `ssu` for submodule pointer work in the app. The framework commit of a task lands before the plugin or layer commit that uses it. Nothing is pushed or tagged before the Task 4 reply. Docker-backed tests skip under `-short`; run them on a Docker host (`sudo -u golem` when the socket is denied) and record NOT RUN in the SUMMARY when no daemon answers.
<interfaces>
Existing seams (read; move or extend, do not fork):
- `../sm-journal-plugin/classes/site_cache.go` (whole file): `SiteCachePurger`, `NewSiteCachePurger` (validation: both or neither, http(s) with host, no user info or fragment, http only to loopback, token at least `SiteCachePurgeMinTokenLength` = 32), `Endpoint`, `Purge` (2 s timeout, `context.WithoutCancel`, 4 KiB answer read, log lines without the token, `purgeTransportError`), `PurgeSiteCacheAfterCommit` (lagoon.AfterCommit); `../sm-journal-plugin/classes/site_cache_test.go` (`TestSiteCachePurgeConfig`, `TestSiteCachePurgeRequest` assert `journal:` log messages); `../sm-journal-plugin/controllers/site_cache.go` (Lookup plus cabana.TxFromContext); `../sm-journal-plugin/plugin.go` `wireSiteCache` (:60-82, runs in Boot); `../sm-journal-plugin/controllers/categories.go` :64-88 (hook pattern); `../sm-journal-plugin/site_cache_admin_test.go` (purge receiver harness).
- `modules/pact/capabilities.go` :748-765 (`FormAfterCreate`, `FormAfterUpdate`, `FormAfterDelete` doc style); `modules/cabana/reorder.go` `reorderPost` (:297-359, the Transaction closure, `txCtx`, the node loop, `lifecycleFailure`), `modules/cabana/crud.go` :1025-1040 (`FormAfterDelete` call pattern), `modules/cabana/tx_context.go` `TxFromContext`; `modules/lagoon/transaction.go` :103 `AfterCommit`.
- `../sm-faq-plugin/controllers/faqs.go` (`app func() *backpack.App` field, `FormAfterCreate`/`FormAfterUpdate` = requireLocale, asserts :73-74), `../sm-faq-plugin/faq_smoke_test.go` `bootFAQ` (cfg.Set for config keys).
- `../sm-menu-plugin/controllers/admin_registry.go` (`AdminControllers(app)`), `menus.go` (no app field yet), `menuentries.go` (`app` field, `FormAfterCreate`/`FormAfterUpdate` with checkPlacement :186-260, asserts :268-279), `controllers/menuentries/config_reorder.yaml` (partition menu_id), `../sm-menu-plugin/menu_smoke_test.go` (boot harness, reorder calls).
- `../vue-journal-layer/server/api/__journal__/cache/purge.post.ts`, `app/utils/siteCache.ts` (authorizePurge, purgeRouteCache, parsePurgeEvent), `tests/siteCache.test.ts`, `nuxt.config.ts` :27-51 (runtimeConfig and the cache:false rule with its comment); `../vue-menu-layer/nuxt.config.ts`, `../vue-faq-layer/nuxt.config.ts`.
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: saving a FAQ in the admin purges the site's Nuxt route cache through cooler's purger and the shared /api/__site__/cache/purge route in vue-faq-layer</name>
<reversibility rating="costly">D-10: site_cache.purge_url, site_cache.purge_token and the /api/__site__/cache/purge route with NUXT_SITE_CACHE_PURGE_TOKEN become deployment configuration across sites.</reversibility>
<files>modules/cooler/sitecache.go, modules/cooler/cooler.go, modules/cooler/sitecache_test.go, modules/cooler/README.md, docs/services/caching.md, README.md, ../sm-faq-plugin/controllers/faqs.go, ../sm-faq-plugin/controllers/site_cache.go, ../sm-faq-plugin/site_cache_admin_test.go, ../sm-faq-plugin/README.md, ../vue-faq-layer/server/lib/siteCachePurge.ts, ../vue-faq-layer/server/api/__site__/cache/purge.post.ts, ../vue-faq-layer/nuxt.config.ts, ../vue-faq-layer/tests/sitePurge.test.ts, ../vue-faq-layer/README.md, ../vue-faq-layer/package.json</files>
<read_first>../sm-journal-plugin/classes/site_cache.go, ../sm-journal-plugin/classes/site_cache_test.go, ../sm-journal-plugin/controllers/site_cache.go, ../sm-journal-plugin/controllers/categories.go, ../sm-journal-plugin/site_cache_admin_test.go, modules/cooler/cooler.go and modules/cooler/README.md (plan 01), docs/services/caching.md (plan 01), modules/lagoon/transaction.go (AfterCommit), ../sm-faq-plugin/controllers/faqs.go, ../sm-faq-plugin/faq_smoke_test.go, ../sm-faq-plugin/README.md, ../vue-journal-layer/server/api/__journal__/cache/purge.post.ts, ../vue-journal-layer/app/utils/siteCache.ts, ../vue-journal-layer/tests/siteCache.test.ts, ../vue-journal-layer/nuxt.config.ts, ../vue-faq-layer/nuxt.config.ts, ../vue-faq-layer/package.json, ../vue-faq-layer/README.md, 19.3-RESEARCH.md (Pattern 8, Pitfall 10)</read_first>
<action>Framework first (one summercms.go commit, README and docs included), per D-10. modules/cooler/sitecache.go: move the Journal purger nearly verbatim with these changes: key constants SiteCachePurgeURLKey = site_cache.purge_url and SiteCachePurgeTokenKey = site_cache.purge_token; NewSiteCachePurger(purgeURL, token) delegates to NewSiteCachePurgerFor(SiteCachePurgeURLKey, SiteCachePurgeTokenKey, purgeURL, token), whose validation errors name the keys it is given; log messages `cooler: site cache purged` and `cooler: site cache purge failed` (attributes kind, slug, status, purged; never the token or the full URL); PurgeSiteCacheAfterCommit(ctx, db, p, kind, slug) unchanged (nil purger or nil db does nothing); `SiteCacheFromApp(app *backpack.App) *SiteCachePurger` returns a published purger, else builds one from the framework keys and publishes it (both keys empty or an invalid config returns nil; an invalid config logs one error naming the key). No debounce: one POST per committed save. cooler.go Install: unless a *SiteCachePurger is already published (a plugin's alias), build it from the framework keys; an invalid config is a boot error; a non-nil purger is published and one info line names its Endpoint. cooler imports neither surf nor cabana. Port the Journal tests to modules/cooler/sitecache_test.go as TestSiteCachePurgeConfig and TestSiteCachePurgeRequest with the framework keys and the cooler log messages. Docs: docs/services/caching.md section "Site cache purge" (both keys with SUMMER_ variables, loopback-only http, token length, after-commit timing, failures never fail a save, the Nuxt side: POST /api/__site__/cache/purge with NUXT_SITE_CACHE_PURGE_TOKEN and its cache:false route rule, a plugin hook example using cabana.TxFromContext and SiteCacheFromApp under a neutral plugin name); cooler README Features, API reference and Configuration rows; the cooler summary sentence in its README and the root README row both gain "and the site-cache purge after admin saves".
FAQ plugin (one sm-faq-plugin commit), per D-11: controllers/site_cache.go adds `purgeSiteCache(ctx context.Context, app func() *backpack.App, kind, slug string)` taking the transaction from cabana.TxFromContext and the purger from cooler.SiteCacheFromApp, doing nothing when either is missing. faqs.go: FormAfterCreate and FormAfterUpdate run requireLocale first and purge kind faq with the FAQ's slug only when it passed; a new FormAfterDelete purges kind faq; add the pact.FormAfterDelete compile-time assert. README: a "Site cache" paragraph naming the framework keys. site_cache_admin_test.go `TestFaqAdminPurgesSiteCache` (Postgres through the existing TestMain; an httptest receiver set through cfg.Set of the two framework keys with a 32-character token): create, update and delete each send exactly one POST with Bearer token and kind faq after the commit; a create refused by the locale check sends none; a receiver answering 500 leaves the save successful.
vue-faq-layer (one commit), per D-11: server/lib/siteCachePurge.ts ports authorize, purge and parse from the Journal util under the names MIN_SITE_PURGE_TOKEN_LENGTH, SITE_ROUTE_CACHE_BASE, authorizeSitePurge, purgeSiteRouteCache and parseSitePurgeEvent (kind accepted when it matches lower-case letters and underscores up to 40 characters, else unknown; slug sanitised as in the Journal util). server/api/__site__/cache/purge.post.ts mirrors the Journal route: no-store header, runtimeConfig siteCache.purgeToken, 404 when disabled or proxied, 401 unauthorized, answer {purged}, log prefix [site-cache]; it imports the lib by relative path and names no layer. nuxt.config.ts gains runtimeConfig siteCache.purgeToken '' (comment naming NUXT_SITE_CACHE_PURGE_TOKEN and the 32-character minimum) and routeRules '/api/__site__/**': { cache: false } with the Journal layer's explanatory comment. tests/sitePurge.test.ts (names start with "sitePurge:"): authorize ok and unauthorized with a 32-character token, purge removes only nitro:routes: keys from a fake storage, and a hash pin: sha256 of server/lib/siteCachePurge.ts and of the route file equal the constants SHARED_LIB_SHA256 and SHARED_ROUTE_SHA256 (write the files, compute the hashes, then write the constants; Tasks 2 and 3 copy the files and these constants byte for byte). README section "Site cache purge"; package.json version 0.2.0.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/cooler &amp;&amp; go test -count=1 -short -run '^(TestSiteCachePurgeConfig|TestSiteCachePurgeRequest)$' -v ./modules/cooler 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 2 &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; go -C ../sm-faq-plugin vet ./... &amp;&amp; go -C ../sm-faq-plugin test -count=1 -run '^TestFaqAdminPurgesSiteCache$' -v ./... 2&gt;&amp;1 | grep -q -- '^--- PASS: TestFaqAdminPurgesSiteCache' &amp;&amp; OUT=$(pnpm --dir ../vue-faq-layer test 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -qE '^# fail 0$' &amp;&amp; printf '%s\n' "$OUT" | grep -cE '^ok [0-9]+ - sitePurge:' | grep -qE '^[3-9]$|^[1-9][0-9]$'</automated>
<fails_when>Non-zero exit: vet or the cooler suite is red, the ported purger tests did not both pass, the docs checker failed, the FAQ purge test did not print "--- PASS" (Docker missing or a purge missing or extra), a layer spec failed ("# fail" not 0), or fewer than three "sitePurge:" specs passed.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'SiteCachePurgeURLKey *= *"site_cache.purge_url"' modules/cooler/sitecache.go` prints one line.
- `go list -deps ./modules/cooler | grep -E 'summercms/modules/(surf|cabana)$'` prints nothing.
- `grep -n 'FormAfterDelete' ../sm-faq-plugin/controllers/faqs.go` prints at least two lines (method and assert).
- `grep -n "'/api/__site__/\*\*': { cache: false }" ../vue-faq-layer/nuxt.config.ts` prints one line and `grep -n '"version": "0.2.0"' ../vue-faq-layer/package.json` prints one line.
- `git -C ../sm-faq-plugin log --format=%ct -1` is not earlier than `git log --format=%ct -1 -- modules/cooler/sitecache.go` (the framework commit came first).
</acceptance_criteria>
<done>An editor saves a FAQ and the site shows it on the next page view instead of after the SWR window.</done>
</task>
<task type="auto">
<name>Task 2: Saving, deleting or reordering menus and menu entries purges the site cache: cabana runs a new pact.ReorderAfterSave hook inside the reorder transaction, the menu plugin adopts the purge and vue-menu-layer ships the shared route</name>
<reversibility rating="costly">D-11: pact.ReorderAfterSave is a public plugin interface; renaming or re-shaping it breaks every plugin that implements it.</reversibility>
<precondition>`git status --porcelain -- modules/pact modules/cabana docs/backend` in summercms.go prints nothing (no in-flight Phase 18.1 edits in the shared cabana and pact files).</precondition>
<files>modules/pact/capabilities.go, modules/pact/README.md, modules/cabana/reorder.go, modules/cabana/README.md, docs/backend/admin-controllers.md, docs/backend/lists-and-filters.md, ../sm-menu-plugin/controllers/admin_registry.go, ../sm-menu-plugin/controllers/menus.go, ../sm-menu-plugin/controllers/menuentries.go, ../sm-menu-plugin/controllers/site_cache.go, ../sm-menu-plugin/site_cache_admin_test.go, ../sm-menu-plugin/README.md, ../vue-menu-layer/server/lib/siteCachePurge.ts, ../vue-menu-layer/server/api/__site__/cache/purge.post.ts, ../vue-menu-layer/nuxt.config.ts, ../vue-menu-layer/tests/sitePurge.test.ts, ../vue-menu-layer/README.md, ../vue-menu-layer/package.json</files>
<read_first>modules/pact/capabilities.go (:735-770), modules/pact/README.md, modules/cabana/reorder.go (:240-360), modules/cabana/crud.go (:870-890 lifecycleFailure, :1025-1040), modules/cabana/README.md (reorder rows), docs/backend/admin-controllers.md (:200-230 hook table), docs/backend/lists-and-filters.md (Tree reorder), ../sm-menu-plugin/controllers/admin_registry.go, ../sm-menu-plugin/controllers/menus.go, ../sm-menu-plugin/controllers/menuentries.go, ../sm-menu-plugin/controllers/menuentries/config_reorder.yaml, ../sm-menu-plugin/menu_smoke_test.go (boot harness and reorder requests), ../sm-menu-plugin/README.md, ../sm-faq-plugin/controllers/site_cache.go (Task 1 helper), the Task 1 files of vue-faq-layer (server/lib, route, tests/sitePurge.test.ts with the two hash constants), ../vue-menu-layer/nuxt.config.ts, ../vue-menu-layer/package.json, ../vue-menu-layer/README.md</read_first>
<action>Framework first (one summercms.go commit), per D-11 and RESEARCH Pitfall 9. pact/capabilities.go: `ReorderAfterSave` interface with `ReorderAfterSave(ctx context.Context, partition *uint64) error`, documented like the FormAfter hooks: optional, runs after a reorder save wrote the new order, inside the same transaction (cabana.TxFromContext works), an error rolls the reorder back. cabana/reorder.go reorderPost: inside the Transaction closure, after the node loop and before the tree is read again, type-assert cc.Controller to pact.ReorderAfterSave and call it with txCtx and in.Partition, passing its error through lifecycleFailure. Docs: pact README API row; cabana README reorder description; docs/backend/admin-controllers.md hook table row for pact.ReorderAfterSave; docs/backend/lists-and-filters.md Tree reorder paragraph on the hook (purging a cache after a reorder as the example).
Menu plugin (one sm-menu-plugin commit): controllers/site_cache.go with the same purgeSiteCache helper as the FAQ plugin; admin_registry.go passes app to menusAdminController, which gains the app field; menus.go adds FormAfterCreate, FormAfterUpdate and FormAfterDelete purging kind menu with the menu code, plus their compile-time asserts; menuentries.go appends the purge (kind menu_entry, slug the entry id) to FormAfterCreate and FormAfterUpdate after their existing checks pass, adds FormAfterDelete, and adds ReorderAfterSave purging kind menu_entry with slug partition:<id> (or reorder without a partition), plus asserts. README: the "Site cache" paragraph. site_cache_admin_test.go `TestMenuAdminPurgesSiteCache` (Postgres through the existing TestMain, receiver through the framework keys): menu create, menu update, entry create, entry update, the reorder POST and entry delete each send exactly one purge after the commit with the expected kind; an entry refused by the placement check sends none; a 500 receiver leaves saves and the reorder successful.
vue-menu-layer (one commit): copy server/lib/siteCachePurge.ts, server/api/__site__/cache/purge.post.ts and tests/sitePurge.test.ts byte for byte from vue-faq-layer (the hash constants stay identical); nuxt.config.ts gains runtimeConfig siteCache.purgeToken and the '/api/__site__/**' cache:false rule with the same comments; README section "Site cache purge"; package.json version 0.2.0.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/cabana ./modules/pact &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; go -C ../sm-menu-plugin vet ./... &amp;&amp; go -C ../sm-menu-plugin test -count=1 -run '^TestMenuAdminPurgesSiteCache$' -v ./... 2&gt;&amp;1 | grep -q -- '^--- PASS: TestMenuAdminPurgesSiteCache' &amp;&amp; cmp ../vue-faq-layer/server/lib/siteCachePurge.ts ../vue-menu-layer/server/lib/siteCachePurge.ts &amp;&amp; cmp ../vue-faq-layer/server/api/__site__/cache/purge.post.ts ../vue-menu-layer/server/api/__site__/cache/purge.post.ts &amp;&amp; OUT=$(pnpm --dir ../vue-menu-layer test 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -qE '^# fail 0$' &amp;&amp; printf '%s\n' "$OUT" | grep -qE '^ok [0-9]+ - sitePurge:'</automated>
<fails_when>Non-zero exit: vet or the cabana and pact short suites are red, the docs checker failed, the menu purge test did not print "--- PASS" (Docker missing, or a purge missing after a save, delete or reorder), the shared layer files differ between vue-faq-layer and vue-menu-layer (cmp non-zero), or a layer spec failed.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'ReorderAfterSave(ctx context.Context, partition \*uint64) error' modules/pact/capabilities.go` prints one line and `grep -n 'pact.ReorderAfterSave' modules/cabana/reorder.go` prints one line.
- `grep -n 'ReorderAfterSave' docs/backend/admin-controllers.md modules/pact/README.md` prints at least one line per file.
- `grep -cE 'func \([a-z]* ?menusAdminController\) FormAfterDelete\(' ../sm-menu-plugin/controllers/menus.go` prints 1 and `grep -cE 'func \([a-z]* ?menuEntriesAdminController\) ReorderAfterSave\(' ../sm-menu-plugin/controllers/menuentries.go` prints 1.
- `grep -n '"version": "0.2.0"' ../vue-menu-layer/package.json` prints one line.
</acceptance_criteria>
<done>An editor reorders or edits a menu and every page shows the new navigation on its next view.</done>
</task>
<task type="auto">
<name>Task 3: Journal saves purge through the framework service, old Journal purge keys and the old Nuxt route keep working for one release, and vue-journal-layer ships the shared route</name>
<files>../sm-journal-plugin/plugin.go, ../sm-journal-plugin/classes/site_cache.go, ../sm-journal-plugin/classes/site_cache_test.go, ../sm-journal-plugin/controllers/site_cache.go, ../sm-journal-plugin/README.md, ../vue-journal-layer/server/lib/siteCachePurge.ts, ../vue-journal-layer/server/api/__site__/cache/purge.post.ts, ../vue-journal-layer/nuxt.config.ts, ../vue-journal-layer/tests/sitePurge.test.ts, ../vue-journal-layer/README.md, ../vue-journal-layer/package.json</files>
<read_first>../sm-journal-plugin/plugin.go (:50-90), ../sm-journal-plugin/classes/site_cache.go, ../sm-journal-plugin/classes/site_cache_test.go, ../sm-journal-plugin/controllers/site_cache.go, ../sm-journal-plugin/controllers/posts.go (:240-265), ../sm-journal-plugin/site_cache_admin_test.go (alias env), ../sm-journal-plugin/README.md (site cache section), modules/cooler/sitecache.go (Task 1), ../vue-journal-layer/nuxt.config.ts, ../vue-journal-layer/README.md, ../vue-journal-layer/package.json, the Task 1 shared files of vue-faq-layer</read_first>
<action>Per D-11; Journal is a core plugin, so nothing it exports or reads today may break. classes/site_cache.go becomes a thin alias layer for one release: `type SiteCachePurger = cooler.SiteCachePurger`; the two key constants keep their golem15.journal.site_cache.* values with a Deprecated comment pointing at cooler.SiteCachePurgeURLKey and cooler.SiteCachePurgeTokenKey; SiteCachePurgeTimeout and SiteCachePurgeMinTokenLength alias cooler's; NewSiteCachePurger(purgeURL, token) returns cooler.NewSiteCachePurgerFor with the Journal keys; PurgeSiteCacheAfterCommit forwards to cooler's. plugin.go wireSiteCache: when site_cache.purge_url or site_cache.purge_token is set, do nothing (the framework owns the purger) and log one warning if the Journal keys are also set (they are ignored); otherwise, when the Journal keys are set, build the purger from them through classes.NewSiteCachePurger, publish it (so cooler.Install and SiteCacheFromApp reuse it and the menu and FAQ plugins purge through it too) and log one deprecation warning naming both framework keys; neither set: nothing. controllers/site_cache.go uses cooler.SiteCacheFromApp. classes/site_cache_test.go keeps the alias-key config cases (errors name the Journal keys) and moves its log-line assertions to the cooler messages; the request cases now live in cooler (Task 1). README: the site cache section documents the framework keys and the alias keys as deprecated for one release. Commit in sm-journal-plugin.
vue-journal-layer (one commit): copy the two shared files and tests/sitePurge.test.ts byte for byte from vue-faq-layer; nuxt.config.ts gains runtimeConfig siteCache.purgeToken and the '/api/__site__/**' cache:false rule beside the existing journal ones, which stay (the old /api/__journal__/cache/purge route, app/utils/siteCache.ts and journal.cachePurgeToken are unchanged and documented as aliases for one release); README section for the shared route and the alias; package.json version 0.4.0.</action>
<verify>
<automated>go -C ../sm-journal-plugin vet ./... &amp;&amp; go -C ../sm-journal-plugin test -count=1 ./... &amp;&amp; go -C ../sm-journal-plugin test -count=1 -run '^(TestCategoriesAdminPurgesSiteCache|TestPostsAdminPurgesSiteCache|TestSiteCachePurgeConfig)$' -v ./... 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 3 &amp;&amp; cmp ../vue-faq-layer/server/lib/siteCachePurge.ts ../vue-journal-layer/server/lib/siteCachePurge.ts &amp;&amp; cmp ../vue-faq-layer/server/api/__site__/cache/purge.post.ts ../vue-journal-layer/server/api/__site__/cache/purge.post.ts &amp;&amp; OUT=$(pnpm --dir ../vue-journal-layer test 2&gt;&amp;1) &amp;&amp; printf '%s\n' "$OUT" | grep -qE '^# fail 0$' &amp;&amp; printf '%s\n' "$OUT" | grep -qE '^ok [0-9]+ - sitePurge:'</automated>
<fails_when>Non-zero exit: vet or the Journal suite is red, fewer than three "--- PASS" lines for the two alias-key admin purge tests and the alias config test (Docker missing, or the alias path broke), the shared layer files differ from vue-faq-layer, or a layer spec (old siteCache or new sitePurge) failed.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'type SiteCachePurger = cooler.SiteCachePurger' ../sm-journal-plugin/classes/site_cache.go` prints one line.
- `grep -n 'SiteCachePurgeURLKey *= *"golem15.journal.site_cache.purge_url"' ../sm-journal-plugin/classes/site_cache.go` prints one line (alias key kept).
- `test -f ../vue-journal-layer/server/api/__journal__/cache/purge.post.ts` exits 0 and `grep -n "'/api/__journal__/\*\*': { cache: false }" ../vue-journal-layer/nuxt.config.ts` prints one line (old route kept).
- `grep -n '"version": "0.4.0"' ../vue-journal-layer/package.json` prints one line.
</acceptance_criteria>
<done>Existing Journal deployments keep purging with their current keys, and new deployments configure one framework purge that serves Journal, menus and FAQs.</done>
</task>
<task type="checkpoint:decision" gate="blocking-human">
<name>Task 4: Push decision: publish the plugin and layer commits (and layer tags), with or without the framework commits, or hold</name>
<read_first>.planning/notes/core-plugins-own-repos.md, this plan's Task 1 to 3 SUMMARY notes (commit hashes per repository, Docker tests run or NOT RUN), .planning/phases/18.1-payment-gateway-subscriptions/18.1-05-SUMMARY.md (how a layer release checkpoint was recorded)</read_first>
<action>Do everything automatable, print a sheet, then stop. For summercms.go, sm-faq-plugin, sm-menu-plugin, sm-journal-plugin, vue-faq-layer, vue-menu-layer and vue-journal-layer: `git status --porcelain` empty for this plan's files, `git fetch origin --tags`, `git log --oneline HEAD..origin/<branch>` empty (otherwise stop and report; never merge or rebase on your own), and `git ls-remote --tags origin` shows none of v0.2.0 (faq and menu layers) and v0.4.0 (journal layer). Run the three layer test suites and the three plugin suites once more. Sheet: per repository the commits that would be pushed; the layer tags; that the plugins import modules/cooler, which exists only in local summercms.go commits until summercms master is pushed (apps build through replace ../summercms.go, so local builds work either way); that summercms master may also carry plans 19.3-02 and 19.3-03 commits (list them); that Task 5 (app pointer bumps) needs the plugin commits and layer tags on origin. No push and no tag before the reply. After "push-all": push summercms master (no tag), the three plugin main branches, the three layer main branches, then create and push annotated tags v0.2.0 (vue-faq-layer, vue-menu-layer) and v0.4.0 (vue-journal-layer); never force. After "push-plugins-layers": the same without summercms. After "hold": nothing, and Task 5 is skipped. Record the reply verbatim and every pushed hash and tag in the SUMMARY.</action>
<decision>Push the plugin and layer repositories (and tag the layers) now, together with summercms master, without it, or hold?</decision>
<context>Pushed commits and tags are read by every site that pins the layers by tag and by everyone who clones the plugins; a tag cannot be moved, only superseded. The framework tag itself is decided in plan 05.</context>
<options>
<option id="push-all">
<name>Push summercms master, the plugins and the layers, and tag the layers</name>
<pros>Every pushed plugin builds against origin; Task 5 can bump the app now.</pros>
<cons>summercms master carries untagged wave-2 work before the phase's unit tests (plan 05).</cons>
</option>
<option id="push-plugins-layers">
<name>Push the plugins and layers and tag the layers; keep summercms local</name>
<pros>The framework stays unpublished until plan 05's release decision; Task 5 can run.</pros>
<cons>The pushed plugins do not build for anyone without this framework checkout until summercms master is pushed.</cons>
</option>
<option id="hold">
<name>Hold everything</name>
<pros>One publication for the whole phase at plan 05.</pros>
<cons>Task 5 is skipped; the app keeps its current pointers until a later bump.</cons>
</option>
</options>
<resume-signal>Reply "push-all", "push-plugins-layers" or "hold".</resume-signal>
<verify>
<automated>go -C ../sm-faq-plugin test -count=1 ./... &amp;&amp; go -C ../sm-menu-plugin test -count=1 ./... &amp;&amp; go -C ../sm-journal-plugin test -count=1 ./... &amp;&amp; pnpm --dir ../vue-faq-layer test &gt;/dev/null &amp;&amp; pnpm --dir ../vue-menu-layer test &gt;/dev/null &amp;&amp; pnpm --dir ../vue-journal-layer test &gt;/dev/null &amp;&amp; echo push-gate-ok</automated>
<fails_when>Non-zero exit or "push-gate-ok" absent: a plugin or layer suite is red; the executor fixes it before presenting the sheet.</fails_when>
</verify>
<acceptance_criteria>
- The SUMMARY records the reply verbatim; after a push reply, `git -C ../vue-faq-layer ls-remote --tags origin refs/tags/v0.2.0`, `git -C ../vue-menu-layer ls-remote --tags origin refs/tags/v0.2.0` and `git -C ../vue-journal-layer ls-remote --tags origin refs/tags/v0.4.0` each print one line and each plugin's `git rev-parse main` equals `git rev-parse origin/main`.
- After "hold", none of those tags exists on origin and `git -C ../sm-faq-plugin log --oneline origin/main..main` lists the Task 1 commit.
</acceptance_criteria>
<done>The human decided what is published, and the record names every pushed commit and tag.</done>
</task>
<task type="auto">
<name>Task 5: The app picks up the purge: bump its Journal, menu and FAQ plugin pointers and its vue app's layer tags in separate commits (only after a push reply)</name>
<precondition>Task 4's reply was "push-all" or "push-plugins-layers" (the plugin commits and the layer tags are on origin); after "hold" this task is skipped and the SUMMARY records it.</precondition>
<files>../sm-grzybyfunkcjonalne-app/plugins/golem15/journal, ../sm-grzybyfunkcjonalne-app/plugins/golem15/menu, ../sm-grzybyfunkcjonalne-app/plugins/golem15/faq, ../sm-grzybyfunkcjonalne-app/go.mod, ../sm-grzybyfunkcjonalne-app/go.sum, ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app, ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/package.json, ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/pnpm-lock.yaml</files>
<read_first>../sm-grzybyfunkcjonalne-app/.gitmodules, ../sm-grzybyfunkcjonalne-app/go.mod (replace lines), ../sm-grzybyfunkcjonalne-app/DEPLOY.md (site cache purge keys), ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/package.json, ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/nuxt.config.ts (routeRules), .planning/notes/core-plugins-own-repos.md</read_first>
<action>Per D-11's order (plugins pushed first, pointer bumps as separate commits). In the app: check `git status --porcelain` and stop on unrelated uncommitted work. Move plugins/golem15/journal, menu and faq to the pushed main heads (with `ssu` or `git -C <submodule> fetch` and checkout of the pushed hash; never stage files inside a submodule from the app repo); run `go mod tidy` in the app (the framework replace now brings go-redis, otter and the AWS SDK modules) and `go build ./...`; commit go.mod and go.sum alone (`chore(deps): tidy for summercms cooler`) and then the three pointers (`chore: bump journal, menu and faq plugins for the framework site-cache purge`). In vue-grzybyfunkcjonalne-app (its own repo): set vue-journal-layer to #v0.4.0, vue-menu-layer and vue-faq-layer to #v0.2.0 in package.json, run `pnpm install`, `pnpm run build` and its tests if present, commit package.json and pnpm-lock.yaml there; then commit the vue app pointer in the app repo separately. The deployment keeps its golem15.journal.site_cache.* and NUXT_JOURNAL_CACHE_PURGE_TOKEN values: the alias path makes menu and FAQ saves purge through the old Journal route, so no server config changes now; record moving the deployment to site_cache.* and NUXT_SITE_CACHE_PURGE_TOKEN as a follow-up. Nothing is pushed here (plan 05's release decision covers the app and vue app pushes).</action>
<verify>
<automated>go -C ../sm-grzybyfunkcjonalne-app build ./... &amp;&amp; test "$(git -C ../sm-grzybyfunkcjonalne-app/plugins/golem15/faq rev-parse HEAD)" = "$(git -C ../sm-faq-plugin rev-parse main)" &amp;&amp; test "$(git -C ../sm-grzybyfunkcjonalne-app/plugins/golem15/menu rev-parse HEAD)" = "$(git -C ../sm-menu-plugin rev-parse main)" &amp;&amp; test "$(git -C ../sm-grzybyfunkcjonalne-app/plugins/golem15/journal rev-parse HEAD)" = "$(git -C ../sm-journal-plugin rev-parse main)" &amp;&amp; grep -q 'vue-journal-layer.git#v0.4.0' ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/package.json &amp;&amp; test -z "$(git -C ../sm-grzybyfunkcjonalne-app status --porcelain -- go.mod go.sum plugins vue-grzybyfunkcjonalne-app)"</automated>
<fails_when>Non-zero exit: the app does not build, a submodule pointer differs from the plugin's main head, the journal layer pin is not v0.4.0, or the bumps are left uncommitted.</fails_when>
</verify>
<acceptance_criteria>
- `git -C ../sm-grzybyfunkcjonalne-app log --oneline -3` shows the tidy commit and the pointer commit as separate commits (plus the vue app pointer commit).
- `grep -c '#v0.2.0' ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/package.json` prints 2.
</acceptance_criteria>
<done>The app runs the adopted plugins and the layer versions that carry the shared purge route, with its deployment config unchanged.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| Go app to the Nitro server | a bearer-token POST over loopback (or https) after admin saves |
| network to the Nitro purge route | anyone who can reach the site can call the route |
| plugin hooks to the save transaction | hook errors roll back admin writes |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-19.3-23 | Denial of service / Spoofing | the shared Nitro purge route | medium | mitigate | 404 below a 32-character token and for proxied requests (X-Forwarded-For, X-Real-IP, Forwarded), sha256 plus timingSafeEqual bearer compare, cache:false route rule in every layer; sitePurge specs |
| T-19.3-24 | Information disclosure | purge token in Go logs and errors | high | mitigate | validation errors name keys only; log lines carry kind, slug, status and endpoint host and path, never the token or query; ported TestSiteCachePurgeRequest asserts it |
| T-19.3-25 | Information disclosure | token sent in clear over the network | high | mitigate | http purge URLs must point at a loopback host, other hosts require https; user info refused |
| T-19.3-26 | Denial of service | a slow or failing purge breaking admin saves | medium | mitigate | after-commit only (lagoon.AfterCommit), detached context with a 2 s timeout, failures logged and swallowed; the plugin tests assert a 500 receiver leaves saves successful |
| T-19.3-27 | Tampering | purge before commit letting a page re-cache stale content | low | mitigate | PurgeSiteCacheAfterCommit registers on the save transaction; the FAQ and menu tests check the purge arrives after the commit |
| T-19.3-28 | Tampering | a ReorderAfterSave hook failing | low | accept | the hook runs inside the reorder transaction by design so its failure rolls the reorder back, as FormAfter hooks do; purge hooks never return errors |
| T-19.3-29 | Tampering | the three copies of the shared route drifting apart | low | mitigate | byte-identical files checked with cmp in Tasks 2 and 3 and a pinned sha256 in each layer's spec |
| T-19.3-SC | Tampering | npm/pip/cargo or Go module installs | high | mitigate | no new packages: the layers reuse node:crypto and Nitro, the plugins use framework modules only; the app's go mod tidy only records modules the framework already pinned (decision note of plan 01) |
</threat_model>
<verification>
- Task 1: cooler suite and the ported purger tests, docs checker, the FAQ Postgres purge test, the vue-faq-layer specs.
- Task 2: cabana and pact short suites, docs checker, the menu Postgres purge test (saves, deletes, reorder), byte equality of the shared files, the vue-menu-layer specs.
- Task 3: the whole Journal suite with the alias tests, byte equality, the vue-journal-layer specs (old and new).
- Task 4: every plugin and layer suite before the push decision.
- Task 5: the app builds on the bumped pointers.
- Manual (19.3-VALIDATION.md): save a menu entry in a running admin and see the page update without waiting for the SWR window.
</verification>
<success_criteria>
- D-10: cooler holds the purge service with framework keys; failures never fail a save.
- D-11: FAQ, menu (incl. reorder) and Journal purge after committed saves; three layers ship the shared route; Journal aliases keep existing deployments working.
- The cross-repo order holds and nothing is pushed without the human's reply.
</success_criteria>
<output>
Create `.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-04-SUMMARY.md` when done, with the commits per repository, the identifiers and keys added, the shared-file hashes, the Task 4 reply verbatim and every pushed hash and tag, the Docker tests run or NOT RUN, and the follow-ups (deployment move to site_cache.* and NUXT_SITE_CACHE_PURGE_TOKEN; the sm-summercmsio-app Journal pointer).
</output>

View File

@@ -0,0 +1,227 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 04
subsystem: caching
tags: [cooler, site-cache, purge, cabana, reorder, pact, faq, menu, journal, nuxt-layers, nitro]
status: complete
requires:
- phase: 19.3-01
provides: "modules/cooler (cache module) that now also holds the site-cache purge service"
provides:
- "cooler.SiteCachePurger with framework keys site_cache.purge_url and site_cache.purge_token, published by cooler.Install; cooler.SiteCacheFromApp; cooler.PurgeSiteCacheAfterCommit"
- "pact.ReorderAfterSave(ctx, partition *uint64) error, called by cabana's reorder inside its transaction"
- "FAQ, menu and menu-entry admin saves, deletes and reorders purge the host site's page cache after commit"
- "Journal purges through cooler; golem15.journal.site_cache.* and classes.SiteCachePurger/NewSiteCachePurger/PurgeSiteCacheAfterCommit kept as aliases for one release"
- "Shared Nitro route POST /api/__site__/cache/purge (runtimeConfig siteCache.purgeToken) in vue-faq-layer v0.2.0, vue-menu-layer v0.2.0 and vue-journal-layer v0.4.0"
affects: [19.3-05, sm-grzybyfunkcjonalne-app deployment, sm-summercmsio-app journal pointer]
actuals:
tokens: 56800
tasks: 5
commits: 2
plan_head_before: 1d8e50a1ee707b1f844c6fbe47bfe1597d8dc2e2
plan_head_after: af9dad634cd4ff90bef1cd1a042b18d7dca97e18
tech-stack:
added: []
patterns:
- "Framework-owned after-commit side effect: hooks take the save transaction from cabana.TxFromContext and register through lagoon.AfterCommit; failures are logged and swallowed"
- "Lazy service lookup (SiteCacheFromApp) so commands and tests that do not run serve still get the same purger"
- "Plugin-key alias path: framework keys win; legacy plugin keys publish the framework service with a deprecation warning"
- "Byte-identical shared files across layers, pinned by sha256 in each layer's spec"
key-files:
created:
- modules/cooler/sitecache.go
- modules/cooler/sitecache_test.go
- ../sm-faq-plugin/controllers/site_cache.go
- ../sm-faq-plugin/site_cache_admin_test.go
- ../sm-menu-plugin/controllers/site_cache.go
- ../sm-menu-plugin/site_cache_admin_test.go
- ../vue-faq-layer/server/lib/siteCachePurge.ts
- ../vue-faq-layer/server/api/__site__/cache/purge.post.ts
- ../vue-faq-layer/tests/sitePurge.test.ts
- ../vue-menu-layer/server/lib/siteCachePurge.ts
- ../vue-menu-layer/server/api/__site__/cache/purge.post.ts
- ../vue-menu-layer/tests/sitePurge.test.ts
- ../vue-journal-layer/server/lib/siteCachePurge.ts
- ../vue-journal-layer/server/api/__site__/cache/purge.post.ts
- ../vue-journal-layer/tests/sitePurge.test.ts
modified:
- modules/cooler/cooler.go
- modules/cooler/example_test.go
- modules/cooler/README.md
- modules/pact/capabilities.go
- modules/pact/README.md
- modules/cabana/reorder.go
- modules/cabana/README.md
- README.md
- docs/services/caching.md
- docs/backend/admin-controllers.md
- docs/backend/lists-and-filters.md
- docs/setup/configuration.md
- ../sm-faq-plugin/controllers/faqs.go
- ../sm-menu-plugin/controllers/admin_registry.go
- ../sm-menu-plugin/controllers/menus.go
- ../sm-menu-plugin/controllers/menuentries.go
- ../sm-journal-plugin/plugin.go
- ../sm-journal-plugin/classes/site_cache.go
- ../sm-journal-plugin/controllers/site_cache.go
- ../sm-grzybyfunkcjonalne-app/go.mod
- ../sm-grzybyfunkcjonalne-app/go.sum
- ../sm-grzybyfunkcjonalne-app/DEPLOY.md
- ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/package.json
- ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/pnpm-lock.yaml
key-decisions:
- "Site-cache purge is a cooler service with framework keys site_cache.purge_url and site_cache.purge_token (http only to loopback, token at least 32 characters, 2 s timeout, after commit only, failures never fail a save)"
- "pact.ReorderAfterSave(ctx, partition *uint64) runs inside cabana's reorder transaction; cabana's reorderPost now uses lagoon.Transaction so AfterCommit callbacks registered by the hook fire"
- "Purge kinds: faq (slug), menu (code), menu_entry (id, or partition:<id> on reorder); Journal keeps post and category"
- "Journal alias keys golem15.journal.site_cache.* publish the cooler purger with a deprecation warning when the framework keys are unset; framework keys win"
- "Layer releases are additive minors: vue-faq-layer v0.2.0, vue-menu-layer v0.2.0, vue-journal-layer v0.4.0; plugins get no tag"
- "Task 4 reply push-all: summercms master, three plugin mains and three layer mains pushed, layer tags created; no framework tag (plan 05 owns it)"
requirements-completed: [SC-4, D-10, D-11, D-19]
duration: 60min
completed: 2026-10-15
---
# Phase 19.3 Plan 04: Framework site-cache purge Summary
**The site-cache purge moved from the Journal plugin into cooler (keys `site_cache.purge_url` and `site_cache.purge_token`). Cabana gained a `pact.ReorderAfterSave` hook. FAQ, menu and Journal admin saves, deletes and reorders now purge the page cache after commit, through a shared `POST /api/__site__/cache/purge` route that ships in all three Nuxt layers. The plugins and layers are pushed and tagged, and the app's pointers are bumped locally.**
## Performance
- **Duration:** about 60 min (2026-10-15T16:16:40Z to 17:16Z, across two executor runs split at the Task 4 checkpoint)
- **Tasks:** 5 of 5
- **Files modified:** 14 framework, 4 FAQ, 6 menu, 6 Journal, 6 per layer, 7 app (incl. submodule pointers), 2 vue app
## Accomplishments
- `cooler.SiteCachePurger` is now a framework service. It is validated and published at boot, so a bad `site_cache` config fails boot. `SiteCacheFromApp` gives commands and tests the same purger lazily. `PurgeSiteCacheAfterCommit` sends one bearer POST of `{kind, slug}` per committed save.
- `pact.ReorderAfterSave` is called inside cabana's reorder transaction after the new order is written. A hook error rolls the reorder back. Controllers without the hook behave as before.
- FAQ create, update and delete purge, and so do menu create, update and delete and menu-entry create, update, delete and reorder. The existing locale and placement checks still run first.
- Journal moved onto cooler. Its old keys and exported names still work as aliases, with a deprecation warning.
- The shared Nitro route and helper are byte-identical in all three layers, pinned by sha256 in each layer's spec. Each layer has its own `'/api/__site__/**': { cache: false }` rule. vue-journal-layer keeps `/api/__journal__/cache/purge` and `journal.cachePurgeToken` as aliases.
- The plugins, layers and summercms master are pushed, and the layer tags are on origin. sm-grzybyfunkcjonalne-app now runs the adopted plugins and layer versions. Its deployment config is unchanged because the Journal alias path keeps it working.
## Task Commits
| Task | Name | Repository: commit |
|------|------|--------------------|
| 1 (tracer) | FAQ save purges the site cache through cooler and the shared Nuxt route | summercms.go 0bc390f, sm-faq-plugin c60ae6e, vue-faq-layer e26f22e |
| 2 | pact.ReorderAfterSave hook; menu plugin purges; vue-menu-layer route | summercms.go af9dad6, sm-menu-plugin 559d96b, vue-menu-layer 3bc7729 |
| 3 | Journal moves onto cooler with alias keys; vue-journal-layer route | sm-journal-plugin d7943fb, vue-journal-layer 3e0b377 |
| 4 | Push decision (checkpoint:decision, blocking-human) | pushes and tags below, no commits |
| 5 | App picks up the purge | sm-grzybyfunkcjonalne-app f9cafe2 (go mod tidy), d604509 (journal/menu/faq pointers), 85939c3 (vue app pointer), 129c0df (DEPLOY.md, deviation); vue-grzybyfunkcjonalne-app 0ee97d0 (layer tags) |
Framework commits measured from the ledger: `git rev-list --count 1d8e50a..HEAD` = 2.
## Task 4: push decision
**Reply (verbatim):** push-all
Before pushing, I re-ran `git fetch origin --tags` in all seven repositories. None was behind origin. `git ls-remote --tags origin` showed no v0.2.0 on vue-faq-layer or vue-menu-layer and no v0.4.0 on vue-journal-layer, and none of these tags existed locally. The previous run had already printed push-gate-ok (all plugin and layer suites green, Docker tests run).
Pushed (no force, no framework tag):
| Repository | Branch | Range pushed |
|------------|--------|--------------|
| summercms.go | master | a9dd565..af9dad6 (also carries plans 19.3-01, -02 and -03 and the phase planning commits) |
| sm-faq-plugin | main | 919cf06..c60ae6e |
| sm-menu-plugin | main | 9c0f833..559d96b |
| sm-journal-plugin | main | 31d901c..d7943fb |
| vue-faq-layer | main | 321bc90..e26f22e |
| vue-menu-layer | main | 2deac88..3bc7729 |
| vue-journal-layer | main | 25e9bfe..3e0b377 |
Annotated tags created and pushed:
| Repository | Tag | Tag object | Commit |
|------------|-----|------------|--------|
| vue-faq-layer | v0.2.0 | 8acbde2 | e26f22e |
| vue-menu-layer | v0.2.0 | a6942aa | 3bc7729 |
| vue-journal-layer | v0.4.0 | 9213af5 | 3e0b377 |
Checked after the push: each plugin's `main` equals `origin/main`, summercms `master` equals `origin/master`, and `ls-remote` prints each tag. The latest framework tag on origin is still v0.1.9 in the tag listing; no new framework tag was made.
**Not pushed:** sm-grzybyfunkcjonalne-app (4 commits) and vue-grzybyfunkcjonalne-app (1 commit) stay local. Plan 05's release decision covers those pushes. Nothing was deployed.
## Identifiers and keys added
- **cooler:** `SiteCachePurger`, `NewSiteCachePurger`, `NewSiteCachePurgerFor`, `SiteCachePurger.Purge`, `SiteCachePurger.Endpoint`, `PurgeSiteCacheAfterCommit`, `SiteCacheFromApp`, `SiteCachePurgeURLKey`, `SiteCachePurgeTokenKey`, `SiteCachePurgeTimeout`, `SiteCachePurgeMinTokenLength`
- **pact:** `ReorderAfterSave`
- **Config keys:** `site_cache.purge_url` (SUMMER_SITE_CACHE__PURGE_URL), `site_cache.purge_token` (SUMMER_SITE_CACHE__PURGE_TOKEN)
- **Layers:** runtimeConfig `siteCache.purgeToken` (NUXT_SITE_CACHE_PURGE_TOKEN); route `POST /api/__site__/cache/purge`; `server/lib/siteCachePurge.ts` exports `MIN_SITE_PURGE_TOKEN_LENGTH`, `SITE_ROUTE_CACHE_BASE`, `authorizeSitePurge`, `purgeSiteRouteCache`, `parseSitePurgeEvent`
- **Purge kinds:** `faq`, `menu`, `menu_entry` (slug `partition:<id>` on reorder), Journal `post` and `category`
**Shared-file sha256 pins (identical in all three layers):**
- `server/lib/siteCachePurge.ts`: `7c34121ae06cf011b30a1dc5d787a94657b71b8768bed3a200be0f41566001b8`
- `server/api/__site__/cache/purge.post.ts`: `c459012280cdf59551e57f0440fcaeb24566a5ba6e3dccf31133e67a683f1c06`
## Tests
- Docker-backed tests **ran** (not skipped): `TestFaqAdminPurgesSiteCache`, `TestMenuAdminPurgesSiteCache` and the Journal admin purge test on Postgres. All suites were green.
- Extra tests: `TestSiteCacheFrameworkKeysWin` (Journal) and `TestSiteCachePurgeConfig` subtests in cooler.
- Task 5: the app passed `go build ./...` and `go test -short ./...`. In the vue app, `pnpm run build` succeeded, `pnpm test` passed 27/27 and `smoke:ssr` passed 38 checks. The built Nitro config carries `"/api/__site__/**": { "cache": false }` and the `/api/__site__/cache/purge` handler. The Task 5 automated verify printed TASK5-VERIFY-OK, and `grep -c '#v0.2.0'` returned 2.
## Decisions Made
See key-decisions in the frontmatter. Task 4 reply: push-all.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] cabana reorder runs in lagoon.Transaction**
- **Found during:** Task 2
- **Issue:** `reorderPost` in `modules/cabana/reorder.go` used a plain GORM Transaction, so `lagoon.AfterCommit` skipped and a purge registered from `ReorderAfterSave` would never be sent.
- **Fix:** It now runs in `lagoon.Transaction`. The existing reorder tests pass.
- **Commit:** summercms.go af9dad6
**2. [Docs checker] cooler example and configuration row**
- **Found during:** Task 1
- **Issue:** The docs checker needed real snippets for caching.md.
- **Fix:** `modules/cooler/example_test.go` gained `ExampleSiteCacheFromApp` and a neutral `articlesController`, and caching.md pulls them via `src=`. A `docs/setup/configuration.md` row was added for the new keys, which was not in files_modified.
- **Commit:** summercms.go 0bc390f
**3. [Commit order] af9dad6 amended after the menu plugin commit**
- **Found during:** Task 2
- **Issue:** The acceptance grep wants exactly one `pact.ReorderAfterSave` line in reorder.go, but one comment repeated the name.
- **Fix:** The comment was reworded and af9dad6 amended. This happened before any push.
**4. [Rule 2 - Docs correctness] sm-grzybyfunkcjonalne-app DEPLOY.md**
- **Found during:** Task 5
- **Issue:** After the pointer bump, DEPLOY.md still said menu and FAQ edits do not purge the cache. It also named the old `journal: site cache purged` log line, which is now `cooler: site cache purged`.
- **Fix:** Three sentences were updated: which saves purge, the cooler log lines, and the boot-time deprecation warning for the journal keys, which keep working. `scripts/check-deploy.sh` passes.
- **Files modified:** ../sm-grzybyfunkcjonalne-app/DEPLOY.md
- **Commit:** sm-grzybyfunkcjonalne-app 129c0df (separate docs commit after the pointer bumps)
**5. [Rule 3 - Blocking] pnpm install without a TTY**
- **Found during:** Task 5
- **Issue:** `pnpm install` aborted with ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY.
- **Fix:** It was re-run with `--config.confirmModulesPurge=false`, which rebuilds the local node_modules only. The lockfile diff touches only the three layer entries.
## Issues Encountered
None beyond the deviations. Pre-existing untracked files were left alone: sm-grzybyfunkcjonalne-app `.gsd/` and its nested `sm-grzybyfunkcjonalne-app` entry, and sm-journal-plugin `go.work.sum`.
## Deferred Items
Logged in `deferred-items.md`:
- Pre-existing gofmt findings (missing trailing newlines) in `modules/cabana/account_tokens.go`, `account_tokens_test.go`, `token_postgres_smoke_test.go`, `token_routes_test.go`, `token_scopes_test.go` and sm-journal-plugin `classes/blog_editor_test.go`.
## Follow-ups
- Move the grzybyfunkcjonalne deployment to `site_cache.purge_url`/`site_cache.purge_token` and `NUXT_SITE_CACHE_PURGE_TOKEN` (route `/api/__site__/cache/purge`) before the Journal alias keys are removed. Until then the binary logs a deprecation warning at boot.
- Bump the Journal pointer in sm-summercmsio-app, which is 45 commits behind main.
- Push sm-grzybyfunkcjonalne-app (f9cafe2, d604509, 85939c3, 129c0df) and vue-grzybyfunkcjonalne-app (0ee97d0) under plan 05's release decision.
- Manual check from 19.3-VALIDATION.md: save a menu entry in a running admin and see the page update without waiting for the SWR window.
## Next Phase Readiness
Plan 05 (unit tests and the framework release decision) can start. summercms master on origin already carries the cooler purge service that the pushed plugins import.
## Self-Check: PASSED

View File

@@ -0,0 +1,324 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 05
type: execute
wave: 3
depends_on: ["19.3-01", "19.3-02", "19.3-03", "19.3-04"]
files_modified:
- scripts/check-phase19.3.sh
- modules/bouncer/cookie_test.go
- modules/bouncer/turnstile_test.go
- modules/surf/cors_profile_test.go
- modules/surf/apidocs_test.go
- modules/surf/store_lookup_test.go
- modules/cabana/reorder_hook_test.go
- internal/build/manifest_tags_test.go
- modules/lagoon/attach/s3_test.go
- modules/lagoon/attach/diskext_test.go
- modules/lagoon/attach/store_test.go
- modules/lagoon/attach/url_test.go
- go.mod
- go.sum
- modules/cooler/memory_test.go
- modules/cooler/redis_test.go
- modules/cooler/ratelimit_test.go
- modules/cooler/docker_test.go
- modules/cooler/install_test.go
- modules/cooler/sitecache_test.go
- ../sm-journal-plugin/site_cache_keys_test.go
- ../sm-faq-plugin/controllers/site_cache_test.go
- ../sm-menu-plugin/controllers/site_cache_test.go
- ../vue-faq-layer/tests/sitePurge.test.ts
- ../vue-menu-layer/tests/sitePurge.test.ts
- ../vue-journal-layer/tests/sitePurge.test.ts
- ../sm-faq-plugin/go.mod
- ../sm-faq-plugin/go.sum
- ../sm-menu-plugin/go.mod
- ../sm-menu-plugin/go.sum
- ../sm-journal-plugin/go.mod
- ../sm-journal-plugin/go.sum
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md
autonomous: false
requirements: [SC-1, SC-2, SC-3, SC-4, D-01, D-02, D-03, D-04, D-05, D-07, D-08, D-09, D-10, D-11, D-12, D-13, D-15, D-16, D-17, D-18]
estimate:
tokens: 210000
raw_tokens: 210000
tasks: 7
confidence: low
must_haves:
truths:
- "SC-4: every piece of new code in the phase has unit tests delivered in this, the phase's last plan, covering the Wave 0 list of 19.3-VALIDATION.md: bouncer cookies and Turnstile, surf CORS hardening and profiles, api-docs and the store lookup, cabana's reorder hook, internal/build tags, attach S3 (both tag modes), DiskExtension, the Store regression and public_url, cooler memory, Redis, rate-limit parity, Install, FromApp and the site-cache purge, the plugin purge helpers and the layer purge route."
- "D-01..D-03 (tests): TokenCookie carries name, path, domain, Secure, SameSite Lax and HttpOnly with Max-Age equal to the refresh window (refresh_ttl*60, not ttl); ForgetTokenCookie clears with matching attributes; the refresh handler rotates the cookie and blacklists the old jti; a missing cookie answers 401; an invalid domain, same_site none without Secure and secure false in production fail LoadCookieConfig."
- "D-04, D-05 (tests): credentials with * fail boot globally and per profile; bad and PHP-delimited patterns fail; wildcard origin entries match subdomains only as anchored patterns; Vary: Origin is on every dynamic answer including preflight and a disallowed origin; the api and www subdomain case works with credentials; non-credentialed * answers are unchanged; profile routes get a working OPTIONS preflight and invalid profile setups fail boot."
- "D-07..D-09 (tests): the Redis rate-limit store and surf.MemoryStore produce the same (allowed, attempts, retryAfter) sequence on one table of attempts (miniredis with FastForward, and real Redis in Docker); concurrent attempts never admit more than max; a Redis error falls back to memory with one warning; the memory cache is bounded and its Increment is atomic; Redis Flush leaves keys outside <prefix>cache: alone; BuildRouter uses a published Store and never dials Redis on its own."
- "D-10, D-11 (tests): a purge runs only after commit (a rolled-back save sends none), a failing or slow receiver never fails the save, SiteCacheFromApp builds lazily and Install refuses a bad config; framework keys win over the Journal alias keys and the alias path logs a deprecation; the plugin helpers do nothing without a transaction or purger; each layer's purge route specs cover disabled, proxied, unauthorized, ok, kind parsing and slug sanitising, with the shared files' hashes pinned."
- "D-12, D-13, D-15..D-18 (tests): build.tags load, validate and round-trip through SaveManifest and reach go build args; s3:// opens with the tag and names the tag without it; DiskExtension's table keeps consistent client extensions and rewrites script-capable ones; public_url builds direct URLs and refuses malformed bases; api-docs is off by default, pins 5.33.1 with three SRI hashes and serves ordered compact JSON; Turnstile's open and closed matrix, timeout and body restore hold."
- "scripts/check-phase19.3.sh fails closed: a required test that fails, skips or does not run fails its stage; --all runs every stage and says NOT RUN for Docker stages when no daemon answers."
- "miniredis is installed only after a blocking human check of the package; nothing is pushed or tagged without the human's reply at the release checkpoint."
artifacts:
- path: "scripts/check-phase19.3.sh"
provides: "the phase gate with go, docker, plugins, layers and docs stages"
contains: "check-phase19.3"
- path: "modules/cooler/ratelimit_test.go"
provides: "the parity table shared by MemoryStore and the Redis store"
contains: "TestRateLimitParity"
- path: "modules/bouncer/cookie_test.go"
provides: "cookie builder, loader and handler tests"
contains: "TestTokenCookieMaxAgeIsRefreshWindow"
- path: "modules/lagoon/attach/diskext_test.go"
provides: "DiskExtension table tests"
contains: "TestDiskExtensionTable"
key_links:
- from: "scripts/check-phase19.3.sh"
to: "modules/cooler/ratelimit_test.go"
via: "the go and docker stages require TestRateLimitParity by name"
pattern: "TestRateLimitParity"
- from: "modules/cooler/ratelimit_test.go"
to: "modules/surf/limiter_store.go"
via: "the external test package cooler_test runs the same table against surf.NewMemoryStore"
pattern: "surf.NewMemoryStore"
prohibitions:
- statement: "Tests never reach the network beyond localhost, httptest servers, miniredis and testcontainers."
status: unresolved
verification: judgment
- statement: "No production code changes in this plan except fixes for defects the new tests expose, each recorded as a deviation."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not in user-story format; no story is invented, see plan 01): the framework has the transport and infrastructure pieces a multi-client, multi-instance application needs: JWT cookie refresh across subdomains, credentialed CORS, a cache module with a Redis-backed rate-limit store, an S3 blob option, an OpenAPI page and Turnstile verification.
This plan's slice (plan 5 of 5, wave 3): SC-4's unit tests, delivered last as CLAUDE.md requires, behind a fail-closed phase gate script, then the phase's release decision.
## Spec-less probe fallback
Skipped visibly: the phase has no requirement IDs to probe, so no probe predicates were generated (see plan 01).
## Decisions taken in this plan (Claude's discretion)
- The gate follows scripts/check-phase18.1.sh: named required tests per stage, `go test -v` output checked for a `--- PASS` line per name, a skip or a missing name fails the stage, `--all` reports Docker stages as NOT RUN without a daemon (an env var makes that fail instead).
- Smoke tests from plans 01 to 04 stay; this plan adds the unit files next to them and extends the plan-01 store lookup file and the plan-04 purger test file instead of duplicating them.
- Rate-limit parity runs one table of (key, max, decay, advance) steps through a small adapter: for MemoryStore the test waits on real time with short decays; for miniredis it calls FastForward; for Docker Redis it waits. The table includes window expiry, denial without increment and retryAfter never growing.
- The release proposal is the highest `v0.1.*` tag plus one (v0.1.27 at planning time), pushed only on the human's reply, after which the three plugins' go.mod require that tag.
## Artifacts this phase produces (this plan)
- `scripts/check-phase19.3.sh` with stages `go`, `docker`, `plugins`, `layers`, `docs` and `--all`.
- Test files: `modules/bouncer/cookie_test.go`, `modules/bouncer/turnstile_test.go`, `modules/surf/cors_profile_test.go`, `modules/surf/apidocs_test.go`, additions in `modules/surf/store_lookup_test.go`, `modules/cabana/reorder_hook_test.go`, `internal/build/manifest_tags_test.go`, `modules/lagoon/attach/s3_test.go`, `modules/lagoon/attach/diskext_test.go`, additions in `modules/lagoon/attach/store_test.go` and `url_test.go`, `modules/cooler/memory_test.go`, `redis_test.go`, `ratelimit_test.go`, `docker_test.go`, `install_test.go`, additions in `sitecache_test.go`, `../sm-journal-plugin/site_cache_keys_test.go`, `../sm-faq-plugin/controllers/site_cache_test.go`, `../sm-menu-plugin/controllers/site_cache_test.go`, extended `tests/sitePurge.test.ts` in the three layers.
- Test dependency `github.com/alicebob/miniredis/v2` v2.40.0 (after the human check).
- 19.3-VALIDATION.md per-task map filled and `wave_0_complete: true`.
- Release: proposed framework tag, plugin go.mod bumps to it, pushes, all on the human's reply.
<objective>
Deliver the unit tests for all new code of the phase, a fail-closed phase gate that runs them by name, and the release decision.
Purpose: SC-4 ("the new code has unit tests, delivered in the phase's last plan") and the test rows of 19.3-VALIDATION.md for every decision.
Output: test files in summercms.go, the three plugins and the three layers; the gate script; the release record.
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-RESEARCH.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-01-SUMMARY.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-02-SUMMARY.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-03-SUMMARY.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-04-SUMMARY.md
Rules: tests only; a production fix is allowed only for a defect a new test exposes, committed separately as `fix(19.3): …` and recorded as a deviation. Commit per repository, staging by explicit path, no co-author tags. Before each task run `git status --porcelain` on the paths it touches and stop on foreign uncommitted edits (Phase 18.1 work may be in flight in cabana). Docker suites skip under `-short`; run them on a Docker host (`sudo -u golem` when the socket is denied) and record NOT RUN when no daemon answers. Nothing is pushed or tagged before the Task 7 reply.
<interfaces>
Code under test (identifiers from the plan SUMMARYs; read them, do not re-derive):
- bouncer (plan 02): `CookieConfig`, `CookieConfigSource`, `LoadCookieConfig`, `TokenCookie`, `ForgetTokenCookie`, `CookieRefreshHandler`, `CookieLogoutHandler`, `CookieResult`, `ErrNoTokenCookie`, `Turnstile`, `NewTurnstile`, `Verify`, `Middleware`, `TurnstileVerdict`, `TurnstileWriter`, `TurnstileMaxTokenLength`; existing `Mint`, `Refresh`, `NewMemoryBlacklist`.
- surf (plans 01, 02): `LoadCORSConfig`, `LoadCORSProfiles`, factories `cors` and `turnstile`, OPTIONS routes, `APIDocsProvider`, `SwaggerUIVersion`, `BuildRouter` store lookup, `installCooler`; existing `TestCORSOptionsMatchesLaravel` and the plan-02 smoke file.
- cabana and pact (plan 04): `pact.ReorderAfterSave`, `reorderPost`; the reorder Postgres harness in `modules/cabana/reorder_flat_test.go` and the tree reorder tests.
- internal/build (plan 03): `Manifest.Tags`, `ValidateBuildTag`, `goBuildArgs`, `SaveManifest`.
- attach (plan 03): `S3BuildTag`, `s3Enabled`, `DiskExtension`, `PublicURLBase`, `PublicURL`, `Store`, the infra smoke file.
- cooler (plans 01, 04): `Install`, `Handle`, `FromApp`, `OpenRedis`, `NewMemoryCache`, `NewRedisCache`, `NewRateLimitStore`, `SiteCachePurger`, `NewSiteCachePurger`, `NewSiteCachePurgerFor`, `PurgeSiteCacheAfterCommit`, `SiteCacheFromApp`, config key constants.
- Plugins (plan 04): each controllers package's `purgeSiteCache` helper, Journal `wireSiteCache` and `classes` aliases; layers: `server/lib/siteCachePurge.ts` exports and the pinned hash constants.
- Gate model: `scripts/check-phase18.1.sh` (stages, required-name lists, NOT RUN handling).
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: scripts/check-phase19.3.sh runs the bouncer cookie and Turnstile unit suites by name and fails when one is missing, skipped or red</name>
<files>scripts/check-phase19.3.sh, modules/bouncer/cookie_test.go, modules/bouncer/turnstile_test.go</files>
<read_first>scripts/check-phase18.1.sh (structure, required-name lists, run helper, NOT RUN handling), modules/bouncer/cookie.go, modules/bouncer/turnstile.go, modules/bouncer/refresh_test.go (fixtures and helpers), modules/surf/transport_smoke_test.go (plan 02 smokes), 19.3-VALIDATION.md (SC-1 and Turnstile rows)</read_first>
<action>Gate: create scripts/check-phase19.3.sh (bash, set -euo pipefail, NO_COLOR) modelled on check-phase18.1.sh with a run helper that executes `go test -count=1 -v -run '^(names)$'` for a package and fails unless every required name prints a `--- PASS` line (a SKIP or absent name fails); stage `go` (go vet ./... and go vet -tags summercms_s3 ./modules/lagoon/..., go test -short ./..., then the required Docker-free names per package); placeholders for stages docker, plugins, layers and docs that later tasks fill; `--all` runs every stage; a Docker stage without a daemon prints NOT RUN and fails only when PHASE193_REQUIRE_DOCKER=1. Environment overrides for the sibling paths like check-phase18.1.sh. Header comment lists the stages and failure rules and names no application.
Tests (per D-01..D-03, D-18): modules/bouncer/cookie_test.go: TestLoadCookieConfigDefaults (empty source: token, /, host-only, Lax, Secure true for https app.url and false for http), TestLoadCookieConfigErrors (table: invalid domain, same_site none with Secure false, secure false in production, unknown same_site, path without leading slash; each error names its key), TestTokenCookieAttributes, TestTokenCookieMaxAgeIsRefreshWindow (20160 minutes gives Max-Age 1209600 while the access ttl is 60 minutes), TestForgetTokenCookie (same attributes, empty value, MaxAge -1), TestCookieRefreshHandlerRotatesAndBlacklists (real bouncer.Refresh with a MemoryBlacklist: new cookie value, old jti blacklisted, writer got 200 and the token), TestCookieRefreshHandlerMissingCookie (401, ErrNoTokenCookie, no Set-Cookie), TestCookieRefreshHandlerRefreshError (401, cookie untouched), TestCookieLogoutHandler (revoke called with the cookie token, forget cookie set, 200; no cookie still forgets), TestCookieHandlerOptionsValidated. modules/bouncer/turnstile_test.go against an httptest siteverify (Endpoint override): TestTurnstileVerifyMatrix (table over open and closed, empty secret in development and production, empty token, a 2049-character token, success true, success false, non-2xx, invalid JSON; asserts the verdict and whether the fake was called), TestTurnstileVerifyTimeout (a fake that sleeps past a 50 ms context deadline gives Failed with a timeout error), TestTurnstileFormFields (secret, response and remoteip received; remoteip absent for an empty ip), TestTurnstileMiddlewareTokenSources (query, JSON body, urlencoded form), TestTurnstileMiddlewareRestoresBody, TestTurnstileMiddlewareStatuses (503, 422, pass; nil writer gives the neutral default bodies), TestTurnstileNeverLogsSecret (a captured slog handler sees neither the secret nor the token after a transport error). Add all names to the go stage.</action>
<verify>
<automated>bash -n scripts/check-phase19.3.sh &amp;&amp; bash scripts/check-phase19.3.sh go &amp;&amp; go test -count=1 -short -run '^(TestTokenCookieMaxAgeIsRefreshWindow|TestCookieRefreshHandlerRotatesAndBlacklists|TestTurnstileVerifyMatrix|TestTurnstileMiddlewareRestoresBody)$' -v ./modules/bouncer 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 4</automated>
<fails_when>Non-zero exit: the script has a syntax error, its go stage fails (a required name missing, skipped or failing, or vet red), or fewer than four of the sampled bouncer tests print "--- PASS".</fails_when>
</verify>
<acceptance_criteria>
- `grep -c 'TestTurnstileVerifyMatrix\|TestCookieRefreshHandlerRotatesAndBlacklists' scripts/check-phase19.3.sh` prints at least 2.
- Renaming one required test locally (not committed) makes `bash scripts/check-phase19.3.sh go` exit non-zero; record the check in the SUMMARY.
- `bash scripts/check-phase19.3.sh nosuchstage` exits non-zero with a usage line listing go, docker, plugins, layers, docs and --all.
</acceptance_criteria>
<done>The phase has a gate that proves the cookie and Turnstile behaviour by name and cannot pass on a silently skipped test.</done>
</task>
<task type="auto">
<name>Task 2: Unit tests for surf (CORS hardening and profiles, api-docs, store lookup) and cabana's reorder hook, wired into the gate</name>
<files>modules/surf/cors_profile_test.go, modules/surf/apidocs_test.go, modules/surf/store_lookup_test.go, modules/cabana/reorder_hook_test.go, scripts/check-phase19.3.sh</files>
<read_first>modules/surf/cors.go, modules/surf/router.go, modules/surf/apidocs.go, modules/surf/cors_test.go, modules/surf/cors_coverage_test.go, modules/surf/transport_smoke_test.go, modules/surf/store_lookup_test.go, modules/cabana/reorder.go, modules/cabana/reorder_flat_test.go (Postgres harness), modules/pact/capabilities.go (ReorderAfterSave), 19.3-RESEARCH.md (Pattern 4, Pattern 9)</read_first>
<action>Per D-04, D-05, D-09, D-11, D-16, D-17. cors_profile_test.go: TestCORSCredentialsWildcardRefused (global and profile, error names the key), TestCORSPatternErrors (uncompilable and #-delimited patterns), TestCORSWildcardOriginEntry (https://*.example.com allows https://www.example.com and refuses https://example.com.evil.org), TestCORSVaryOnDynamicAnswers (allowed, disallowed and preflight responses carry Vary: Origin when credentials or patterns are set), TestCORSSubdomainCredentialed (api host, www origin, credentials: exact origin and Allow-Credentials on GET and preflight), TestCORSStarUnchanged (a * config without credentials produces the same header set as before for GET and OPTIONS; keep TestCORSOptionsMatchesLaravel green), TestCORSProfilePreflightRoute, TestCORSProfileUnknown, TestCORSProfileWithPathsRefused, TestCORSProfileOverlapGlobal, TestCORSProfileConflictingProfilesSamePath, TestTurnstileFactoryParams (open and closed accepted, anything else fails boot). apidocs_test.go: TestAPIDocsOffByDefault, TestAPIDocsServesPinnedPage (5.33.1, three integrity attributes equal to the RESEARCH hashes, crossorigin, CSP nonce equals the script nonce, nosniff), TestAPIDocsSpecJSONOrder (YAML keys keep their order, output is compact), TestAPIDocsJSONSpecInput, TestAPIDocsProviderCount (zero and two providers fail), TestAPIDocsPathValidation, TestAPIDocsTitleEscaped (a title with markup is escaped). store_lookup_test.go additions: TestBuildRouterStoreFallback, TestBuildRouterUsesPublishedStore, TestBuildRouterNeverDials (redis config without install builds instantly with a MemoryStore). cabana reorder_hook_test.go (Postgres harness, skips under -short): TestReorderHookRunsInTx (the hook sees the new sort values through cabana.TxFromContext and receives the partition), TestReorderHookErrorRollsBack (sort values unchanged, error answer), TestReorderWithoutHookUnchanged. Add every name to the gate: the Docker-free ones to the go stage, the cabana ones to the docker stage.</action>
<verify>
<automated>go vet ./... &amp;&amp; bash scripts/check-phase19.3.sh go &amp;&amp; go test -count=1 -run '^(TestReorderHookRunsInTx|TestReorderHookErrorRollsBack|TestReorderWithoutHookUnchanged)$' -v ./modules/cabana 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 3</automated>
<fails_when>Non-zero exit: vet fails, the gate's go stage fails (a required surf name missing, skipped or red), or fewer than three cabana hook tests print "--- PASS" (Docker missing or a hook defect).</fails_when>
</verify>
<acceptance_criteria>
- `grep -c 'TestCORSSubdomainCredentialed\|TestAPIDocsServesPinnedPage\|TestBuildRouterNeverDials\|TestReorderHookRunsInTx' scripts/check-phase19.3.sh` prints at least 4.
- `go test -count=1 -short -run '^TestCORSOptionsMatchesLaravel$' ./modules/surf` exits 0.
</acceptance_criteria>
<done>CORS, api-docs, the store lookup and the reorder hook are pinned by tests the gate requires.</done>
</task>
<task type="auto">
<name>Task 3: Unit tests for summer.yaml build tags, the S3 tag guard in both build modes, DiskExtension, the Store regression table and public_url, wired into the gate</name>
<files>internal/build/manifest_tags_test.go, modules/lagoon/attach/s3_test.go, modules/lagoon/attach/diskext_test.go, modules/lagoon/attach/store_test.go, modules/lagoon/attach/url_test.go, scripts/check-phase19.3.sh</files>
<read_first>internal/build/manifest.go, internal/build/build.go, internal/build/build_test.go, internal/build/manifest_tags_smoke_test.go, modules/lagoon/attach/bucket.go, modules/lagoon/attach/s3_on.go, modules/lagoon/attach/s3_off.go, modules/lagoon/attach/diskext.go, modules/lagoon/attach/store.go, modules/lagoon/attach/store_test.go, modules/lagoon/attach/url_test.go, modules/lagoon/attach/infra_smoke_test.go, 19.3-RESEARCH.md (Pattern 5, Pattern 6 probe table)</read_first>
<action>Per D-12, D-13, D-15. internal/build/manifest_tags_test.go: TestManifestBuildTags (load, trim, dedupe), TestManifestBuildTagsInvalid (dash, space, upper case, non-list build.tags, non-mapping build), TestSaveManifestKeepsBuildTags (plugin:add-style round trip keeps tags and plugin order; no build block when empty), TestGoBuildArgs (no tags: no -tags argument; tags: one -tags=a,b argument before -o). attach/s3_test.go (no build constraint; branches on s3Enabled): TestOpenBucketS3 (without the tag the error names S3BuildTag, build.tags and -tags; with the tag the bucket opens with dummy AWS env and closes; an uppercase S3 scheme behaves the same). attach/diskext_test.go: TestDiskExtensionTable (rows from the RESEARCH probe: png body as html gives png, jpeg body keeps jpeg and maps png to jpg, mkv under video/webm kept, docx under application/zip kept, ics under text/plain kept, html under text/plain gives txt, svg under text/xml gives txt, js under application/octet-stream gives bin, pdf as txt gives pdf). store_test.go addition TestStoreDiskExtension (Postgres, skips under -short): the regression table through Store with URL and file_name assertions. url_test.go additions: TestPublicURLBase (trailing slashes, key joining, thumbnails through PublicURL), TestPublicURLValidation (ftp, user info, query, fragment, relative value each fail naming storage.uploads.public_url; unset keeps the prefix). Add names to the gate: Docker-free ones to the go stage (also run the attach names with -tags summercms_s3 in that stage), TestStoreDiskExtension to the docker stage.</action>
<verify>
<automated>go vet ./... &amp;&amp; go vet -tags summercms_s3 ./modules/lagoon/... &amp;&amp; bash scripts/check-phase19.3.sh go &amp;&amp; go test -count=1 -short -tags summercms_s3 -run '^TestOpenBucketS3$' -v ./modules/lagoon/attach 2&gt;&amp;1 | grep -q -- '^--- PASS: TestOpenBucketS3$' &amp;&amp; go test -count=1 -run '^TestStoreDiskExtension$' -v ./modules/lagoon/attach 2&gt;&amp;1 | grep -q -- '^--- PASS: TestStoreDiskExtension$'</automated>
<fails_when>Non-zero exit: vet fails in either tag mode, the gate's go stage fails, the tagged S3 test or the Postgres Store regression did not print "--- PASS".</fails_when>
</verify>
<acceptance_criteria>
- `grep -c 'TestDiskExtensionTable\|TestSaveManifestKeepsBuildTags\|TestOpenBucketS3\|TestPublicURLValidation\|TestStoreDiskExtension' scripts/check-phase19.3.sh` prints at least 5.
- `grep -n 'summercms_s3' scripts/check-phase19.3.sh` prints at least one line (the tagged run is part of the gate).
</acceptance_criteria>
<done>The S3 opt-in, the build tags, the disk-name fix and public_url are pinned in both build modes.</done>
</task>
<task type="checkpoint:human-verify" gate="blocking-human">
<name>Task 4: Package legitimacy check before installing github.com/alicebob/miniredis/v2 v2.40.0 (test-only)</name>
<read_first>.planning/notes/2026-10-15-cooler-go-redis-otter.md (miniredis paragraph), 19.3-RESEARCH.md (Package Legitimacy Audit, Supporting stack, assumption A3)</read_first>
<acceptance_criteria>
- The SUMMARY records the human's reply verbatim ("approved" or "use-docker-only") before any go.mod change of Task 5.
- Until the reply, `grep -c 'alicebob/miniredis' go.mod` prints 0.
</acceptance_criteria>
<what-built>Nothing installed yet. RESEARCH verified the module and version on proxy.golang.org (v2.40.0 published 2026-10-10, VCS origin github.com/alicebob/miniredis, one dependency gopher-lua) but marked the package choice [ASSUMED] (name from training). Plan 01's decision note names it as test-only. Never auto-approved.</what-built>
<how-to-verify>
1. Open https://pkg.go.dev/github.com/alicebob/miniredis/v2 and https://github.com/alicebob/miniredis: confirm the repository, the maintainer, the MIT licence, recent activity and that v2.40.0 is a tagged release.
2. Run `GOFLAGS=-mod=mod go list -m -json github.com/alicebob/miniredis/v2@v2.40.0` in a scratch module and confirm Origin.URL is https://github.com/alicebob/miniredis.
3. Confirm it is acceptable as a test-only dependency (it lands in go.mod and go.sum; no production package imports it).
</how-to-verify>
<resume-signal>Reply "approved" to install it in Task 5, or "use-docker-only" to drop miniredis (Task 5 then runs the Redis tests on testcontainers only, all in the docker stage).</resume-signal>
</task>
<task type="auto">
<name>Task 5: Unit tests for cooler: memory and Redis caches, rate-limit parity with MemoryStore, Install and FromApp, and the after-commit site-cache purge, wired into the gate</name>
<precondition>Task 4's reply is recorded in the SUMMARY; miniredis is installed only after "approved".</precondition>
<files>go.mod, go.sum, modules/cooler/memory_test.go, modules/cooler/redis_test.go, modules/cooler/ratelimit_test.go, modules/cooler/docker_test.go, modules/cooler/install_test.go, modules/cooler/sitecache_test.go, scripts/check-phase19.3.sh</files>
<read_first>modules/cooler/*.go and modules/cooler/smoke_test.go, modules/surf/limiter_store.go, modules/surf/limiter_test.go (MemoryStore expectations), modules/lagoon/transaction.go (AfterCommit), modules/beachcomber/postgres_test.go (TestMain with flag.Parse and testing.Short), modules/postcard/mailpit_test.go, 19.3-RESEARCH.md (Pattern 2 probe output, Pitfalls 13 and 14, Docker-gated test example)</read_first>
<action>Per D-07..D-10. After "approved": `go get github.com/alicebob/miniredis/v2@v2.40.0` (test files only import it). memory_test.go: TestMemoryCache (every method, ttl 0 forever, copy-on-get), TestMemoryCacheTTL (expiry through a short TTL), TestMemoryCacheBounded (max_entries respected), TestMemoryIncrementConcurrent (100 goroutines, final value exact, TTL kept), TestRememberDegrades (a failing driver still returns the callback value and logs once). redis_test.go (miniredis, Docker-free): TestRedisCache (methods, keys under <prefix>cache:, PX TTL with FastForward), TestRedisFlushOnlyPrefix (rl keys and foreign keys survive). ratelimit_test.go (package cooler_test): TestRateLimitParity (one table through surf.NewMemoryStore and NewRateLimitStore on miniredis with FastForward: first attempt opens, denial does not increment, window never extends, retryAfter never grows, expiry opens a new window), TestRateLimitConcurrentNeverExceedsMax (50 goroutines, max 10, exactly 10 admitted), TestRateLimitFallbackOnError (stopped miniredis: fallback store answers, one warning per minute), TestRateLimitKeyPrefix. install_test.go: TestInstallConfigErrors (unknown driver and store values, missing redis.url, malformed URL without the password in the error), TestInstallPingWarning (unreachable Redis: no error, one warning), TestInstallReusesFromApp, TestFromAppLazy (no dial at construction), TestInstallSiteCacheValidation (bad site_cache config is a boot error; a published alias purger is reused). sitecache_test.go additions: TestSiteCacheAfterCommitOnly (Postgres: rollback sends nothing, commit sends one), TestSiteCacheFailureNeverFails (500 and a slow receiver past SiteCachePurgeTimeout leave the transaction committed), TestSiteCacheFromApp (builds once, publishes, nil when unset). docker_test.go (skips under -short; redis:7-alpine): TestRateLimitParityDocker (the same table on real Redis), TestRedisCacheDocker. If the reply was "use-docker-only", move the miniredis cases into docker_test.go on testcontainers and record the change. Add the names to the go and docker stages of the gate.</action>
<verify>
<automated>go vet ./... &amp;&amp; bash scripts/check-phase19.3.sh go &amp;&amp; go test -count=1 -run '^(TestRateLimitParityDocker|TestRedisCacheDocker|TestSiteCacheAfterCommitOnly|TestSiteCacheFailureNeverFails)$' -v ./modules/cooler 2&gt;&amp;1 | grep -c -- '^--- PASS: Test' | grep -qx 4</automated>
<fails_when>Non-zero exit: vet fails, the gate's go stage fails (a cooler name missing, skipped or red), or fewer than four Docker-backed cooler tests print "--- PASS".</fails_when>
</verify>
<acceptance_criteria>
- After "approved": `grep -n 'github.com/alicebob/miniredis/v2 v2.40.0' go.mod` prints one line and `grep -rl 'alicebob/miniredis' --include='*.go' modules | grep -vc '_test.go$'` prints 0.
- `grep -c 'TestRateLimitParity\|TestRateLimitConcurrentNeverExceedsMax\|TestSiteCacheAfterCommitOnly' scripts/check-phase19.3.sh` prints at least 3.
</acceptance_criteria>
<done>The cache, the shared rate limits and the purge service are proven equivalent to the semantics they replace, fast in the -short loop and on real Redis.</done>
</task>
<task type="auto">
<name>Task 6: Unit tests for the plugin purge wiring and the layers' purge route, the plugins and layers stages of the gate, and the validation map</name>
<files>../sm-journal-plugin/site_cache_keys_test.go, ../sm-faq-plugin/controllers/site_cache_test.go, ../sm-menu-plugin/controllers/site_cache_test.go, ../vue-faq-layer/tests/sitePurge.test.ts, ../vue-menu-layer/tests/sitePurge.test.ts, ../vue-journal-layer/tests/sitePurge.test.ts, scripts/check-phase19.3.sh, .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md</files>
<read_first>../sm-journal-plugin/plugin.go (wireSiteCache), ../sm-journal-plugin/classes/site_cache.go, ../sm-faq-plugin/controllers/site_cache.go, ../sm-menu-plugin/controllers/site_cache.go, the three layers' server/lib/siteCachePurge.ts and tests/sitePurge.test.ts (plan 04), ../vue-journal-layer/tests/siteCache.test.ts (matrix style), scripts/check-phase18.1.sh (plugins and layers stages), 19.3-VALIDATION.md</read_first>
<action>Per D-10, D-11. sm-journal-plugin/site_cache_keys_test.go (Docker-free, boots the plugin on a compass config without a database): TestSiteCacheFrameworkKeysWin (framework keys set: wireSiteCache publishes nothing; alias keys also set: one warning that they are ignored), TestSiteCacheAliasKeysDeprecated (only alias keys: a purger is published whose validation errors name the Journal keys, one deprecation warning naming both framework keys), TestSiteCacheNoKeys. sm-faq-plugin and sm-menu-plugin controllers/site_cache_test.go: TestPurgeSiteCacheWithoutTx and TestPurgeSiteCacheWithoutPurger (no panic, no request). Layers: extend tests/sitePurge.test.ts identically in all three (the file stays byte-identical; its hash constants cover only the lib and route files): sitePurge: disabled below 32 characters, proxied before the token check, unauthorized for a wrong or malformed bearer, ok in constant time over sha256 digests, purge removes only nitro:routes: keys and returns the count, parseSitePurgeEvent accepts faq, menu, menu_entry, post and category, maps other kinds to unknown, strips control characters and cuts slugs at 200. Commit each repository separately. Gate: stage plugins runs go vet ./... and go test -short ./... in the three plugins with the required names, and their Docker tests (TestFaqAdminPurgesSiteCache, TestMenuAdminPurgesSiteCache, TestCategoriesAdminPurgesSiteCache, TestPostsAdminPurgesSiteCache) in the docker stage; stage layers runs pnpm test in the three layers, requires "# fail 0", the sitePurge: names and cmp equality of the shared files across the layers; stage docs runs TestDocsTree, summer docs:build --check and a case-insensitive grep that no new framework file names a consuming application. Fill 19.3-VALIDATION.md's per-task map with the final commands and statuses and set wave_0_complete true (leave nyquist_compliant to /msd-validate-phase).</action>
<verify>
<automated>bash scripts/check-phase19.3.sh plugins &amp;&amp; bash scripts/check-phase19.3.sh layers &amp;&amp; bash scripts/check-phase19.3.sh docs &amp;&amp; grep -q '^wave_0_complete: true$' .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md</automated>
<fails_when>Non-zero exit: a plugin, layer or docs stage fails (a required name missing, skipped or red, a layer "# fail" other than 0, the shared layer files differing), or VALIDATION.md still says wave_0_complete false.</fails_when>
</verify>
<acceptance_criteria>
- `cmp ../vue-faq-layer/tests/sitePurge.test.ts ../vue-menu-layer/tests/sitePurge.test.ts` and `cmp ../vue-faq-layer/tests/sitePurge.test.ts ../vue-journal-layer/tests/sitePurge.test.ts` exit 0.
- `grep -c 'TestSiteCacheFrameworkKeysWin\|TestPurgeSiteCacheWithoutTx\|sitePurge:' scripts/check-phase19.3.sh` prints at least 3.
</acceptance_criteria>
<done>The purge adoption across plugins and layers is pinned by tests and the gate covers every repository the phase touched.</done>
</task>
<task type="checkpoint:decision" gate="blocking-human">
<name>Task 7: Release gate: the whole phase gate green, then a human decides the framework tag, the plugin require bumps and the pushes</name>
<read_first>.planning/phases/18.1-payment-gateway-subscriptions/18.1-01-SUMMARY.md (release gate record), .planning/notes/2026-10-07-embedded-admin-release-gate.md, 19.3-04-SUMMARY.md (what plan 04 pushed or held), this plan's Task 1 to 6 SUMMARY notes</read_first>
<action>Do everything automatable, print a release sheet, then stop. In summercms.go: `git status --porcelain -- modules internal cmd scripts docs go.mod go.sum` prints nothing, `git fetch origin --tags`, `git log --oneline HEAD..origin/master` prints nothing (otherwise stop and report). Propose the highest `git tag -l 'v0.1.*' --sort=-v:refname` plus one and confirm with `git ls-remote --tags origin` that it is free. Run `bash scripts/check-phase19.3.sh --all` (verify). Sheet: the proposed tag; the public surfaces it publishes (cooler and its keys redis.url, cache.*, http.rate_limit.store, site_cache.*; auth.cookie.* and the bouncer cookie and Turnstile API; http.cors hardening, http.cors_profiles and cors:<profile>; turnstile.*; http.api_docs.*; summer.yaml build.tags and the summercms_s3 tag; storage.uploads.public_url; pact.ReorderAfterSave); the framework commits to push, including any earlier unpushed master commits (list them); the plugin and layer commits still unpushed after plan 04 and the app commits from plan 04 Task 5; that apps building against ../summercms.go need `go mod tidy` once for go-redis, otter and the AWS SDK modules; the Docker stages run or NOT RUN. No push and no tag before the reply. After "release": annotated tag on master, push master and the tag; in sm-faq-plugin, sm-menu-plugin and sm-journal-plugin set the summercms require to the new tag with `GOWORK=off go get git.golem15.com/golem15/summercms@<tag>` and `GOWORK=off go mod tidy`, run their suites, commit go.mod and go.sum, push main; push the layer test commits; push the app and vue app commits of plan 04 Task 5 (vue app first, then the app). After "tag-only": tag and push summercms only. After "hold": nothing. Never force. Record the reply verbatim and every pushed hash and tag.</action>
<decision>Tag and push the framework release, bump and push the plugins, and push the layer and app commits now?</decision>
<context>A pushed tag is read by plugin go.mod files and the Go module proxy and publishes every new key and interface of this phase; it cannot be moved, only superseded. Holding costs nothing locally: apps and plugins build through replace and go.work.</context>
<options>
<option id="release">
<name>Tag and push everything</name>
<pros>Plugins require a framework version that contains cooler; every repository on origin builds on its own.</pros>
<cons>A defect found in UAT needs a follow-up tag.</cons>
</option>
<option id="tag-only">
<name>Tag and push the framework only</name>
<pros>The framework surface is published; plugin and app pushes wait for UAT.</pros>
<cons>Plugins on origin keep requiring the old tag until bumped.</cons>
</option>
<option id="hold">
<name>Hold until UAT (/msd-verify-work)</name>
<pros>One publication after manual checks (subdomain cookies, a real S3 bucket, a live purge).</pros>
<cons>Nothing on origin carries the phase yet.</cons>
</option>
</options>
<resume-signal>Reply "release", "tag-only" or "hold".</resume-signal>
<verify>
<automated>bash scripts/check-phase19.3.sh --all &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; (cd examples/hello &amp;&amp; go test ./...) &amp;&amp; echo release-gate-ok</automated>
<fails_when>Non-zero exit or "release-gate-ok" absent: a gate stage, the docs checker or the examples/hello workspace tests are red; the executor fixes the defect (as a recorded deviation) before presenting the sheet and never presents a red gate.</fails_when>
</verify>
<acceptance_criteria>
- The SUMMARY records the proposed tag, the reply verbatim and, after "release" or "tag-only", `git ls-remote --tags origin "refs/tags/$RELEASE_TAG"` non-empty with `git rev-parse master` equal to `git rev-parse origin/master`.
- After "release", `grep -n "git.golem15.com/golem15/summercms $RELEASE_TAG" ../sm-faq-plugin/go.mod ../sm-menu-plugin/go.mod ../sm-journal-plugin/go.mod` prints one line per file.
- After "hold", `git ls-remote --tags origin "refs/tags/$RELEASE_TAG"` prints nothing.
</acceptance_criteria>
<done>The phase is either published with plugins pinned to it, or held with a record of exactly what waits.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| Go module proxy to the build | a new test-only module enters go.sum |
| test harnesses to the network | httptest, miniredis and testcontainers only |
| local checkouts to origin | tags and pushes published to every consumer |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-19.3-30 | Repudiation | the phase gate passing on skipped or unmatched tests | medium | mitigate | named required tests per stage; a missing, skipped or failing name fails the stage; Task 1 proves it by renaming a test locally |
| T-19.3-31 | Tampering | publishing an unverified release | medium | mitigate | Task 7 blocking checkpoint after the full gate; no force pushes; tag proposed and checked free on origin |
| T-19.3-32 | Information disclosure | tests asserting secrets stay out of logs | low | mitigate | TestTurnstileNeverLogsSecret, the redaction test for redis.url, the purge token log assertions |
| T-19.3-SC | Tampering | Go module install github.com/alicebob/miniredis/v2 v2.40.0 ([ASSUMED] name) | high | mitigate | blocking human legitimacy check (Task 4) before go get; test-only import verified by the Task 5 acceptance criterion; "use-docker-only" fallback avoids the module entirely |
</threat_model>
<verification>
- Task 1: the gate script parses and its go stage passes with the bouncer names; a renamed test makes it fail.
- Tasks 2, 3, 5: the go stage with each task's names plus the Docker-backed names of that task.
- Task 6: the plugins, layers and docs stages; VALIDATION.md updated.
- Task 7: `scripts/check-phase19.3.sh --all`, TestDocsTree and the examples/hello workspace tests before the human decision.
</verification>
<success_criteria>
- SC-4: unit tests for all new code, delivered in the phase's last plan, behind a fail-closed gate.
- Every decision D-01..D-19 with code has at least one named test in the gate.
- miniredis installed only after the human check; the release published only on the human's reply.
</success_criteria>
<output>
Create `.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-SUMMARY.md` when done, with the test files and names per repository, the gate stages run or NOT RUN, the Task 4 and Task 7 replies verbatim, every pushed hash and tag, and any production fix made as a deviation.
</output>

View File

@@ -0,0 +1,313 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 05
subsystem: testing
tags: [gate, bouncer, turnstile, cors, api-docs, cooler, redis, miniredis, rate-limit, s3, build-tags, site-cache, nuxt-layers, release]
status: complete
requires:
- phase: 19.3-01
provides: "modules/cooler (caches, Redis rate-limit store, Install, FromApp), surf store lookup"
- phase: 19.3-02
provides: "bouncer token cookies and Turnstile, surf CORS hardening, CORS profiles, api-docs"
- phase: 19.3-03
provides: "summer.yaml build.tags, attach S3 build tag, DiskExtension, public_url"
- phase: 19.3-04
provides: "cooler site-cache purger, pact.ReorderAfterSave, plugin purge helpers, layer purge route"
provides:
- "scripts/check-phase19.3.sh: fail-closed gate with self-test, go, docker, plugins, layers and docs stages and --all"
- "SC-4 unit tests for every piece of new code of the phase, in summercms.go, three plugins and three layers"
- "github.com/alicebob/miniredis/v2 v2.40.0 as a test-only dependency (human-approved)"
- "19.3-VALIDATION.md per-task map filled, wave_0_complete true"
- "summercms v0.1.27 tagged and pushed; sm-faq-plugin, sm-menu-plugin and sm-journal-plugin require v0.1.27 on origin"
affects: [/msd-verify-work 19.3, /msd-validate-phase 19.3, sm-grzybyfunkcjonalne-app plugin submodule pointers]
actuals:
tokens: 56050
tasks: 7
commits: 9
plan_head_before: 6035ccc194a74a9124ece17ce3c8301689cbc8ac
plan_head_after: 9921768bbf0814671a89eafd45f949afcc113a11
tech-stack:
added:
- "github.com/alicebob/miniredis/v2 v2.40.0 (test-only; brings github.com/yuin/gopher-lua v1.1.1 indirect)"
patterns:
- "One behaviour table run through several stores (parity adapter: real-time sleep for MemoryStore and Docker Redis, FastForward for miniredis)"
- "Gate stages require tests by exact top-level name through go test -json and TAP detectors; a skip, a missing name or zero tests fails"
- "Route handler specs in node --test: Nitro auto-imports stubbed on globalThis, the real route file imported"
key-files:
created:
- scripts/check-phase19.3.sh
- modules/bouncer/cookie_test.go
- modules/bouncer/turnstile_test.go
- modules/surf/cors_profile_test.go
- modules/surf/apidocs_test.go
- modules/cabana/reorder_hook_test.go
- internal/build/manifest_tags_test.go
- modules/lagoon/attach/s3_test.go
- modules/lagoon/attach/diskext_test.go
- modules/cooler/memory_test.go
- modules/cooler/redis_test.go
- modules/cooler/ratelimit_test.go
- modules/cooler/install_test.go
- modules/cooler/docker_test.go
- ../sm-journal-plugin/site_cache_keys_test.go
- ../sm-faq-plugin/controllers/site_cache_test.go
- ../sm-menu-plugin/controllers/site_cache_test.go
modified:
- modules/surf/store_lookup_test.go
- modules/lagoon/attach/store_test.go
- modules/lagoon/attach/url_test.go
- modules/cooler/sitecache_test.go
- modules/cooler/README.md
- go.mod
- go.sum
- ../sm-journal-plugin/site_cache_admin_test.go
- ../sm-journal-plugin/go.mod
- ../sm-journal-plugin/go.sum
- ../sm-faq-plugin/go.mod
- ../sm-faq-plugin/go.sum
- ../sm-menu-plugin/go.mod
- ../sm-menu-plugin/go.sum
- ../vue-faq-layer/tests/sitePurge.test.ts
- ../vue-menu-layer/tests/sitePurge.test.ts
- ../vue-journal-layer/tests/sitePurge.test.ts
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md
key-decisions:
- "Phase gate reuses check-phase18.1.sh's go test -json detector: a skip, a missing name, no tests, zero tests or a build failure fails the stage"
- "miniredis is test-only: only _test.go files in modules/cooler import it (Task 4 reply: approved)"
- "Journal's plan-04 Docker test renamed to TestCategoriesAdminFrameworkKeysWin so the unit test owns TestSiteCacheFrameworkKeysWin"
- "Phase 19.3 released as summercms v0.1.27 (Task 7 reply: release); the three plugins require v0.1.27, the app's plugin submodule pointers are not bumped yet"
requirements-completed: [SC-1, SC-2, SC-3, SC-4, D-01, D-02, D-03, D-04, D-05, D-07, D-08, D-09, D-10, D-11, D-12, D-13, D-15, D-16, D-17, D-18]
coverage:
- deliverable: "Fail-closed phase gate scripts/check-phase19.3.sh"
human_judgment: false
verification:
- kind: command
ref: "bash scripts/check-phase19.3.sh --all"
status: pass
- deliverable: "SC-4 unit tests across framework, plugins and layers"
human_judgment: false
verification:
- kind: command
ref: "bash scripts/check-phase19.3.sh --all (go, docker, plugins, layers stages)"
status: pass
- kind: command
ref: "go test -count=1 ./... (41 packages, Docker suites included)"
status: pass
- deliverable: "Release v0.1.27 with plugins bumped and all commits pushed"
human_judgment: false
verification:
- kind: command
ref: "git ls-remote --tags origin refs/tags/v0.1.27 (non-empty); master == origin/master in every repo"
status: pass
- kind: command
ref: "GOWORK=off go test -count=1 ./... in sm-faq-plugin, sm-menu-plugin, sm-journal-plugin after the bump"
status: pass
duration: 49min (wall time 17:19Z to 18:08Z, including the Task 4 and Task 7 checkpoint waits)
completed: 2026-10-15
---
# Phase 19.3 Plan 05: unit tests, phase gate and release Summary
**Fail-closed gate `scripts/check-phase19.3.sh` requires every new 19.3 test by name across the framework, three plugins and three Nuxt layers. It passed on all six stages, Docker included. The phase is released as summercms v0.1.27, and the FAQ, menu and Journal plugins now require that tag on origin.**
## Checkpoint replies
### Task 4 (miniredis legitimacy check)
The reply below was recorded verbatim (commit b139651) before any go.mod change in Task 5:
> approved
The orchestrator sent this instruction with it: install github.com/alicebob/miniredis/v2@v2.40.0 as a test-only dependency, so only `_test.go` files in modules/cooler may import it. Checked after Task 5: `grep -n 'github.com/alicebob/miniredis/v2 v2.40.0' go.mod` prints line 10, and `grep -rl 'alicebob/miniredis' --include='*.go' modules | grep -vc '_test.go$'` prints 0.
### Task 7 (release decision)
The reply below is recorded verbatim:
> release
The orchestrator scoped it to exactly the "release" option:
1. Tag v0.1.27 and push master and the tag.
2. Bump the three plugins, test them and push them.
3. Push the layer test commits, with no new layer tags.
4. Push the plan 04 app commits, vue app first.
It also set these rules: never force-push, run fetch and a divergence check before each push, deploy nothing, and leave the app's plugin submodule pointer bump out of scope.
## Release record
- **Tag:** `v0.1.27` is an annotated tag (object d0e39437) on summercms.go master 9921768. `git ls-remote --tags origin refs/tags/v0.1.27` prints `d0e39437b5b56ba77b82267693de189bf5d1329d` and peels to `9921768bbf0814671a89eafd45f949afcc113a11`.
- **Preflight before each push:**
- `git fetch origin --tags` ran in every repository, and nothing was behind origin.
- v0.1.27 was free on origin before tagging, and `go vet ./... && go build ./...` passed on 9921768.
- The only change since the green `--all` gate run is planning docs.
- **summercms.go:** pushed master `af9dad6..9921768`. This covers plan 04's 6f9f2e2 and 6035ccc and this plan's 619916b through 9921768. Then the tag was pushed. `git rev-parse master` equals `git rev-parse origin/master`. The final SUMMARY and state commit lands after the tag and is pushed plainly to master.
- **Plugins:**
- Each one ran `GOWORK=off go get git.golem15.com/golem15/summercms@v0.1.27` and `GOWORK=off go mod tidy`.
- The tidy added go-redis v9.23.0, otter v2.3.0, google/wire and the AWS SDK v2 modules as indirect requires.
- `GOWORK=off go vet ./...` and `GOWORK=off go test -count=1 ./...` passed in all three, Docker suites included. `go test -short ./...` also passed through the workspace.
- Pushed main:
- sm-faq-plugin `c60ae6e..fa90e83` (e117bb4, then fa90e83 "chore(19.3-05): require summercms v0.1.27").
- sm-menu-plugin `559d96b..fe27b9a` (6c08f1a, fe27b9a).
- sm-journal-plugin `d7943fb..a99e9a1` (ebfb710, a99e9a1).
- `grep -n "git.golem15.com/golem15/summercms v0.1.27"` prints one line per go.mod: faq line 7, menu line 6, journal line 8.
- **Layers (test commits only, no new tags):**
- vue-faq-layer `e26f22e..96b6bc0`
- vue-menu-layer `3bc7729..dd0ee35`
- vue-journal-layer `3e0b377..5936f66`
- **App, vue app first:**
- vue-grzybyfunkcjonalne-app `bc143fa..0ee97d0`.
- Then sm-grzybyfunkcjonalne-app `4bab6f1..129c0df` (f9cafe2, d604509, 85939c3, 129c0df).
- **No force-push, no deploy.**
## Performance
- **Started:** 2026-10-15T17:19:19Z, across three executor runs with checkpoints at Task 4 and Task 7.
- **Completed:** 2026-10-15T18:08Z
- **Tasks:** 7 of 7
- **Framework commits since the plan ledger:** 9 (6035ccc..9921768), measured when this SUMMARY was written. The completion commit follows.
- **Commits in other repositories:** 9 in all. Six are test commits (Task 6) and three are plugin go.mod bumps (Task 7).
## Accomplishments
- The gate has stages `--self-test`, `go`, `docker`, `plugins`, `layers` and `docs`, plus `--all`.
- A stage fails when a required test is missing, skipped or red.
- When no Docker daemon answers, `--all` reports docker as NOT RUN. It fails instead when PHASE193_REQUIRE_DOCKER=1.
- The cooler suites cover:
- the memory and Redis caches;
- the rate-limit parity table, run through surf.MemoryStore, miniredis and real Redis;
- concurrency, the fallback on a Redis error and the key prefixes;
- Install and FromApp;
- the site-cache purge on Postgres: rollback, commit, nested transactions, a 500 answer, a slow receiver and an unreachable one.
- In the plugins: tests for the FAQ and menu `purgeSiteCache` helpers, and for the Journal's deprecated keys (framework keys win; alias keys are deprecated or invalid; no keys).
- In the layers: the shared purge spec grows from 6 to 11 tests and is byte-identical in all three layers. The new tests cover:
- route-handler specs for 404, 401 and 200;
- the purge count and the log line;
- timing-safe comparison across bearer lengths;
- parse edge cases.
- Release: v0.1.27 is published. The three plugins on origin build against it without a sibling checkout.
## Task Commits
| Task | Repo | Commit | Message |
|------|------|--------|---------|
| 1 (tracer) | summercms.go | 619916b | test(19.3-05): phase gate with bouncer cookie and Turnstile unit suites |
| 2 | summercms.go | 055f51c | test(19.3-05): surf CORS, profiles, api-docs and store lookup; cabana reorder hook |
| 3 | summercms.go | 38f6d65 | test(19.3-05): build tags, S3 tag guard, DiskExtension and public_url |
| (docs) | summercms.go | df7db6c | docs(19.3-05): log S3 object content-type observation in deferred items |
| 4 | summercms.go | b139651 | docs(19.3-05): record the miniredis checkpoint reply |
| 5 | summercms.go | 78ba385 | test(19.3-05): cooler caches, rate-limit parity, Install and after-commit purge |
| 6 | sm-journal-plugin | ebfb710 | test(19.3-05): Journal site-cache keys: framework keys win, alias keys deprecated |
| 6 | sm-faq-plugin | e117bb4 | test(19.3-05): purgeSiteCache does nothing without a transaction or purger |
| 6 | sm-menu-plugin | 6c08f1a | test(19.3-05): purgeSiteCache does nothing without a transaction or purger |
| 6 | vue-faq-layer | 96b6bc0 | test(19.3-05): site purge route handler specs and edge cases |
| 6 | vue-menu-layer | dd0ee35 | test(19.3-05): site purge route handler specs and edge cases |
| 6 | vue-journal-layer | 5936f66 | test(19.3-05): site purge route handler specs and edge cases |
| 6 | summercms.go | f57fc12 | test(19.3-05): gate plugins, layers and docs stages |
| 6 | summercms.go | 8f53253 | docs(19.3-05): validation map filled, wave 0 complete |
| (docs) | summercms.go | 9921768 | docs(19.3-05): summary through Task 6 and the release sheet; state at the Task 7 checkpoint |
| 7 | summercms.go | tag v0.1.27 (d0e3943 on 9921768) | v0.1.27: cooler, bouncer cookies and Turnstile, CORS hardening and profiles, api-docs, build tags and opt-in S3, public_url, pact.ReorderAfterSave (Phase 19.3) |
| 7 | sm-faq-plugin | fa90e83 | chore(19.3-05): require summercms v0.1.27 |
| 7 | sm-menu-plugin | fe27b9a | chore(19.3-05): require summercms v0.1.27 |
| 7 | sm-journal-plugin | a99e9a1 | chore(19.3-05): require summercms v0.1.27 |
The Task 7 pushes also carried plan 04's app commits: vue-grzybyfunkcjonalne-app 0ee97d0, then sm-grzybyfunkcjonalne-app f9cafe2, d604509, 85939c3 and 129c0df.
## Test names per repository (all required by the gate)
- **summercms.go, go stage (-short):**
- bouncer: 17 names (TestLoadCookieConfigDefaults … TestTurnstileNeverLogsSecret).
- surf: 23 names (TestCORSCredentialsWildcardRefused … TestBuildRouterNeverDials, including TestCORSOptionsMatchesLaravel).
- internal/build: 4 names.
- attach: 4 names, run twice (untagged and with `-tags summercms_s3`).
- cooler: TestMemoryCache, TestMemoryCacheTTL, TestMemoryCacheBounded, TestMemoryIncrementConcurrent, TestRememberDegrades, TestRedisCache, TestRedisFlushOnlyPrefix, TestRateLimitParity, TestRateLimitConcurrentNeverExceedsMax, TestRateLimitFallbackOnError, TestRateLimitKeyPrefix, TestInstallConfigErrors, TestInstallPingWarning, TestInstallReusesFromApp, TestFromAppLazy, TestInstallSiteCacheValidation, TestSiteCachePurgeConfig, TestSiteCachePurgeRequest, TestSiteCacheFromApp.
- **summercms.go, docker stage:**
- cabana: TestReorderHookRunsInTx, TestReorderHookErrorRollsBack, TestReorderWithoutHookUnchanged.
- attach: TestStoreDiskExtension.
- cooler: TestRateLimitParityDocker, TestRedisCacheDocker, TestSiteCacheAfterCommitOnly, TestSiteCacheFailureNeverFails.
- **Plugins, plugins stage:**
- FAQ and menu `./controllers`: TestPurgeSiteCacheWithoutTx, TestPurgeSiteCacheWithoutPurger.
- Journal: TestSiteCacheFrameworkKeysWin, TestSiteCacheAliasKeysDeprecated, TestSiteCacheNoKeys; `./classes`: TestSiteCachePurgeConfig, TestSiteCachePurgeRequest.
- **Plugins, docker stage:** TestFaqAdminPurgesSiteCache, TestMenuAdminPurgesSiteCache, TestCategoriesAdminPurgesSiteCache, TestPostsAdminPurgesSiteCache, TestCategoriesAdminFrameworkKeysWin.
- **Layers, layers stage:** 11 `sitePurge:` names per layer, plus a `cmp` of server/lib/siteCachePurge.ts, server/api/__site__/cache/purge.post.ts and tests/sitePurge.test.ts across the three layers.
## Gate runs
All runs were on 2026-10-15 on a Docker host:
- `bash scripts/check-phase19.3.sh --all` passed self-test, go, plugins, layers, docs and docker, ending with "Phase 19.3 gate passed". The docker stage actually ran; it was not NOT RUN.
- `go test -count=1 -run '^TestDocsTree$' ./cmd/summer` and `(cd examples/hello && go test ./...)` both passed, giving release-gate-ok.
- A full `go test -count=1 ./...` with all Docker suites exited 0: 41 packages ok and no FAIL lines.
- Before tagging, `go vet ./... && go build ./...` passed on 9921768.
- After the bump, each plugin passed `GOWORK=off go vet ./...` and `GOWORK=off go test -count=1 ./...`, Docker included.
- Fail-closed checks (every planted break was reverted and never committed):
- Task 1: renaming TestForgetTokenCookie made the go stage exit 1, and `nosuchstage` exits 2 with a usage line.
- Task 6: a planted consuming-application name in modules/cooler/redis.go made `docs` exit 1, and a one-line drift in vue-menu-layer's spec made `layers` exit 1.
## Decisions Made
- The gate reuses check-phase18.1.sh's `go test -json` detector. A skip, a missing name, no tests, zero tests or a build failure all fail the stage.
- Each cooler Docker test starts its own container instead of sharing one through a TestMain: Postgres for the purge tests, redis:7-alpine for the rest. Nothing starts before flag parsing, and each test skips itself under `-short`.
- The docs stage scans the lines the phase added (`git diff <first 19.3 commit>^`, with `.planning` excluded) for consuming-application names. The names are split in the script so the scan does not match itself, and PHASE193_BASE overrides the base.
- TestStoreDiskExtension, row "js under octet-stream": the test expects content_type text/javascript and the disk name `.bin`. The content type comes from the extension because the sniff is generic (the plan 03 rule); the disk name follows D-15.
- Release (Task 7, reply "release"): v0.1.27 is an annotated tag on master. The plugin bump commits use the subject `chore(19.3-05): require summercms v0.1.27`. The app's plugin submodule pointers stay where they are (out of scope; see Follow-ups).
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] Task 3 verify grep could never match**
- **Found during:** Task 3
- **Issue:** `grep -q -- '^--- PASS: TestOpenBucketS3$'` cannot match, because `go test -v` appends the duration to the line.
- **Fix:** checked with `grep -qE '^--- PASS: <name>( |$)'` instead (T3-VERIFY-OK). The gate itself uses the JSON detector.
- **Commit:** 38f6d65
**2. [Rule 3 - Blocking] Duplicate test name in the Journal test binary**
- **Found during:** Task 6
- **Issue:** plan 04 had added a Docker test named `TestSiteCacheFrameworkKeysWin` in package journal_test, and this plan gives the same name to the new unit test in package journal. `-run '^TestSiteCacheFrameworkKeysWin$'` matched both. Under `-short` the Docker one skipped, and the gate's skip detector failed the stage.
- **Fix:** renamed the Docker test to `TestCategoriesAdminFrameworkKeysWin` (it saves a category with both key sets) and required it in the docker stage. The unit test keeps the plan's name.
- **Files modified:** ../sm-journal-plugin/site_cache_admin_test.go
- **Commit:** ebfb710
**3. [Discretion] Branch the plugin helper unit tests cannot reach**
- `TestPurgeSiteCacheWithoutPurger` covers a nil app, a nil app result and an app without keys.
- No unit test can reach the branch "inside a transaction, no purger". cabana has no exported constructor for a context carrying a transaction, and this plan allows no production changes.
- The plugins' Docker admin suites reach that branch. The gap is logged in deferred-items.md.
No production code changed in this plan.
## Issues Encountered
- On each failing call against a stopped Redis, go-redis logs `failed to dial after 5 attempts` to stderr. With Redis down, `Cache.Remember` waits about 1.7 s for those retries unless the caller's context has a deadline. Logged in deferred-items.md as a follow-up.
- The earlier follow-up, a committed test for the rate-limit store's fallback during a Redis outage, is now TestRateLimitFallbackOnError. Each attempt answers from the fallback within about AttemptTimeout and logs one warning naming the endpoint.
- Earlier finding (df7db6c), unchanged: S3 object metadata carries the sniffed content type. An HTML body uploaded as note.txt is stored with Content-Type text/html.
## Follow-ups
- **Not done, outside the release reply's scope:** sm-grzybyfunkcjonalne-app still points its plugin submodules at the plan 04 commits: plugins/golem15/faq c60ae6e, journal d7943fb and menu 559d96b. Bumping them to fa90e83, a99e9a1 and fe27b9a (the v0.1.27 go.mod commits) is a separate app commit. It needs its own go-ahead, and then a push.
- Apps that build against ../summercms.go need `go mod tidy` once for go-redis, otter and the AWS SDK modules. sm-grzybyfunkcjonalne-app already has it (f9cafe2).
- Nothing was deployed. Manual UAT for /msd-verify-work 19.3 is still open: subdomain cookies, a real S3 bucket and a live purge.
- Left untouched in the working trees, as instructed: sm-journal-plugin/go.work.sum, sm-grzybyfunkcjonalne-app/.gsd/, and an untracked `sm-grzybyfunkcjonalne-app/sm-grzybyfunkcjonalne-app` entry inside the app checkout. It is not from this plan.
## Known Stubs
None.
## Next Phase Readiness
Plan 05 is the last plan of phase 19.3, so the phase is complete. The next steps are /msd-verify-work 19.3 for manual UAT and the app's plugin pointer bump listed above.
## Self-Check: PASSED
- All 17 created test and gate files exist.
- All 20 listed commits are reachable from origin in their repositories: summercms.go, three plugins, three layers, the vue app and the app.
- `refs/tags/v0.1.27` on origin is d0e3943, which peels to 9921768.

View File

@@ -0,0 +1,665 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 06
type: execute
wave: 4
depends_on: ["19.3-05"]
gap_closure: true
files_modified:
- modules/surf/router.go
- modules/surf/cors.go
- modules/surf/apidocs.go
- modules/surf/cors_profile_test.go
- modules/surf/apidocs_test.go
- modules/surf/README.md
- docs/services/routing.md
- modules/bouncer/cookie.go
- modules/bouncer/cookie_test.go
- modules/bouncer/README.md
- docs/services/authentication.md
- modules/lagoon/attach/store.go
- modules/lagoon/attach/bucket.go
- modules/lagoon/attach/url_test.go
- modules/lagoon/README.md
- docs/services/storage.md
- modules/cooler/sitecache.go
- modules/cooler/cooler.go
- modules/cooler/sitecache_test.go
- modules/cooler/README.md
- docs/services/caching.md
- scripts/check-phase19.3.sh
- ../sm-faq-plugin/site_cache_admin_test.go
- ../sm-menu-plugin/site_cache_admin_test.go
- ../sm-journal-plugin/site_cache_admin_test.go
- ../sm-journal-plugin/classes/site_cache_test.go
- ../sm-faq-plugin/go.mod
- ../sm-faq-plugin/go.sum
- ../sm-menu-plugin/go.mod
- ../sm-menu-plugin/go.sum
- ../sm-journal-plugin/go.mod
- ../sm-journal-plugin/go.sum
- ../vue-faq-layer/server/lib/siteCachePurge.ts
- ../vue-faq-layer/server/api/__site__/cache/purge.post.ts
- ../vue-faq-layer/tests/sitePurge.test.ts
- ../vue-faq-layer/nuxt.config.ts
- ../vue-faq-layer/README.md
- ../vue-faq-layer/package.json
- ../vue-menu-layer/server/lib/siteCachePurge.ts
- ../vue-menu-layer/server/api/__site__/cache/purge.post.ts
- ../vue-menu-layer/tests/sitePurge.test.ts
- ../vue-menu-layer/nuxt.config.ts
- ../vue-menu-layer/README.md
- ../vue-menu-layer/package.json
- ../vue-journal-layer/server/lib/siteCachePurge.ts
- ../vue-journal-layer/server/api/__site__/cache/purge.post.ts
- ../vue-journal-layer/tests/sitePurge.test.ts
- ../vue-journal-layer/nuxt.config.ts
- ../vue-journal-layer/README.md
- ../vue-journal-layer/package.json
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-REVIEW-DISPOSITION.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-PROGRESS.md
autonomous: false
requirements: [SC-1, SC-3, SC-4, D-03, D-04, D-05, D-10, D-11, D-12, D-17, D-18]
estimate:
tokens: 240000
raw_tokens: 240000
tasks: 7
confidence: low
must_haves:
truths:
- "SC-1 / D-05 (CR-01, the verification gap): every response of a route that names cors:<profile>, on the route or on an enclosing group, carries the profile's Access-Control-Allow-Origin (the exact allowed origin), Access-Control-Allow-Credentials and Vary: Origin whatever the entry's position in the list: a 401 or 403 from a guard listed before it or set on the group, a 422 or 503 from turnstile:closed, a 429 from throttle, a 413 from a middleware reading an over-limit body, and a recovered panic 500 in JSON and raw groups; a disallowed origin gets Vary: Origin and no Allow-Origin; a handler's own Vary is merged, not replaced."
- "CR-01 gate and docs: TestCORSProfileHeadersOnRefusals and TestCORSProfileVaryKeptWithHandlerVary are required by name in SURF_TESTS of scripts/check-phase19.3.sh; docs/services/routing.md and modules/surf/README.md state that the profile applies to every response of the route whatever its position in the list or the group nesting."
- "D-04 (WR-04): with supports_credentials true, an allowed_origins entry whose wildcard is not a whole leftmost host label (https://*, *://*, https://*example.com, a * in the scheme), an allowed_origins_patterns entry without ^ and $, and any origin pattern that matches a probe origin outside the configured domains (https://*.com, ^https?://.*$) fail boot naming the key and index, globally and per profile; credentialed https://*.example.com and anchored patterns keep loading; non-credentialed configs load exactly as before and TestCORSOptionsMatchesLaravel stays green."
- "D-03 (WR-05, WR-09): in the production environment LoadCookieConfig refuses a token cookie whose Secure resolves to false whether auth.cookie.secure was set or inferred from an http or unset app.url, with an error naming auth.cookie.secure and app.url; the zero CookieConfig writes token, Path=/, HttpOnly, SameSite=Lax and Secure, and any config with SameSite unset writes SameSite=Lax."
- "D-17, D-12, D-18 (WR-06, WR-07, WR-08): the /api-docs page sends script-src with the per-request nonce as its only source and every script tag carries that nonce (SRI unchanged); attach.Store returns attach.ErrProtectedNeedsPrivateBucket for an is_public false upload while storage.uploads.public_url is set, before any blob is written, and OpenBucket warns at boot; BuildRouter logs one warning naming the route count when routes name turnstile:open or turnstile:closed and the secret is empty."
- "D-10, D-11 (WR-02, WR-01): the purge client never follows a redirect (a 3xx is logged as a failed purge and the token never reaches the Location); PurgeSiteCacheAfterCommit queues the purge after commit on a per-purger worker, so the admin request does not wait for the site, at most one purge request is in flight per purger and saves committed meanwhile collapse into one follow-up request (a bulk delete of N rows sends at most two); cooler.SiteCachePurger.Wait drains the queue and cooler's Handle.Close waits for it at shutdown; every exported cooler purge name and signature, the Journal aliases and the FAQ, menu and Journal controllers are unchanged."
- "D-11 (WR-03): the shared purge route in vue-faq-layer, vue-menu-layer and vue-journal-layer accepts a proxied request when runtimeConfig siteCache.allowProxied (NUXT_SITE_CACHE_ALLOW_PROXIED) is true, the bearer token then being the only gate, and refuses it as before when the flag is unset; the three shared files stay byte-identical with updated hash pins; docs/services/caching.md pairs an https purge_url on another host with that flag."
- "Release: scripts/check-phase19.3.sh --all passes with Docker and requires every new test by name; 19.3-REVIEW-DISPOSITION.md records CR-01 and WR-01..WR-09 as fixed; nothing is tagged or pushed in any repository before the human's reply at the Task 7 checkpoint."
artifacts:
- path: "modules/surf/router.go"
provides: "Router.wrap applying the route's CORS profile outside recovery, body limit and locale; the boot warning for turnstile routes without a secret"
contains: "profileMiddleware"
- path: "modules/surf/cors.go"
provides: "credentialed origin rules with probe origins"
contains: "cors-probe.invalid"
- path: "modules/surf/apidocs.go"
provides: "nonce on every script tag of the api-docs page"
contains: 'nonce="{{.Nonce}}"'
- path: "modules/bouncer/cookie.go"
provides: "production refusal of an inferred insecure cookie; zero-config defaults"
contains: "resolves to false"
- path: "modules/lagoon/attach/store.go"
provides: "refusal of protected uploads into a public_url bucket"
contains: "ErrProtectedNeedsPrivateBucket"
- path: "modules/cooler/sitecache.go"
provides: "redirect-free purge client, queued and coalesced after-commit purges, Wait"
contains: "ErrUseLastResponse"
- path: "../vue-faq-layer/server/lib/siteCachePurge.ts"
provides: "allowProxied opt-in in the shared authorization rules"
contains: "allowProxied"
- path: "scripts/check-phase19.3.sh"
provides: "gate requiring the gap-closure tests by name"
contains: "TestCORSProfileHeadersOnRefusals"
key_links:
- from: "modules/surf/router.go"
to: "modules/surf/cors.go"
via: "wrap skips the cors:<profile> entry in its middleware loop and wraps the recovered handler in r.corsProfiles[profile].profileMiddleware just inside recordPattern"
pattern: "corsProfiles\\[profile\\]"
- from: "scripts/check-phase19.3.sh"
to: "modules/surf/cors_profile_test.go"
via: "SURF_TESTS requires TestCORSProfileHeadersOnRefusals by name in the go stage"
pattern: "TestCORSProfileHeadersOnRefusals"
- from: "modules/cooler/sitecache.go"
to: "modules/lagoon/transaction.go"
via: "PurgeSiteCacheAfterCommit registers an AfterCommit callback that only queues the purge"
pattern: "enqueue"
- from: "modules/cooler/cooler.go"
to: "modules/cooler/sitecache.go"
via: "Handle.Close waits for the purger Install published before closing Redis"
pattern: "Wait\\("
- from: "../vue-faq-layer/server/api/__site__/cache/purge.post.ts"
to: "../vue-faq-layer/server/lib/siteCachePurge.ts"
via: "the route passes runtimeConfig siteCache.allowProxied to authorizeSitePurge"
pattern: "allowProxied"
prohibitions:
- statement: "No breaking change to the Journal, menu and FAQ plugins: cooler's exported purge names and signatures, the Journal aliases and every plugin controller stay as they are; only plugin test files change before the release bump."
status: unresolved
verification: judgment
- statement: "Existing non-credentialed * CORS answers stay byte-identical (TestCORSOptionsMatchesLaravel green)."
status: unresolved
verification: judgment
- statement: "Framework code and docs never name a consuming application."
status: unresolved
verification: judgment
- statement: "Nothing is pushed or tagged in any repository before the human's reply at the Task 7 checkpoint."
status: unresolved
verification: judgment
- statement: "Info findings IN-01..IN-11 are not addressed in this plan."
status: unresolved
verification: judgment
---
## Phase Goal
ROADMAP Goal (not a user story; standard goal-backward planning, as in plans 01 to 05): the framework has the transport and infrastructure pieces a multi-client, multi-instance application needs: JWT cookie refresh across subdomains, credentialed CORS, a cache module with a Redis-backed rate-limit store, an S3 blob option, an OpenAPI page and Turnstile verification.
This plan's slice (plan 6, gap closure, wave 4): close the one verification gap (CR-01: `cors:<profile>` headers missing from refusals, 413 and 500, which hides the 401 the SPA needs for its cookie refresh) and fix the nine review warnings WR-01..WR-09 of 19.3-REVIEW.md, then release v0.1.28 on the human's reply.
## Gap being closed (19.3-VERIFICATION.md `gaps[0].missing`)
| Missing item | Task |
|--------------|------|
| `Router.wrap` skips the `cors:<profile>` entry and applies `r.corsProfiles[profile].profileMiddleware` after recovery, just inside `recordPattern` | Task 1 |
| Test: guard listed before `cors:spa` → 401 with ACAO, ACAC, `Vary: Origin`; same for a body-limit 413 and a recovered 500; in SURF_TESTS | Task 1 (plus a group-level guard, a 403, a 429 and the Turnstile 422 and 503) |
| docs/services/routing.md and modules/surf/README.md: the profile applies to every response regardless of position | Task 1 |
Review warnings: WR-04, WR-05, WR-09 in Task 2; WR-06, WR-07, WR-08 in Task 3; WR-02, WR-01 in Task 4; WR-03 in Task 5. Unit-test breadth and gate wiring in Task 6; release in Task 7. IN-01..IN-11 are out of scope (user decision).
## Decisions taken in this plan (user decisions of this gap run, then Claude's discretion)
- One plan, not split (user). Each finding lands as its own commit with its README and docs lines, carrying one focused regression test; the unit-test tables and the gate wiring for Tasks 2 to 5 come in Task 6, the plan's last code task (project rule: unit tests last). Task 1 wires its own test into the gate because the verification gap names that step.
- CR-01: besides moving the profile outermost, `bufferedResponse.commit` merges `Vary` instead of replacing it. Found while planning: once the profile sets headers on the outer writer, a handler that sets its own `Vary` would otherwise erase `Vary: Origin` on commit (the global `http.cors` wrapper has the same latent problem and benefits).
- WR-04: three rules apply only with credentials: a wildcard is a whole leftmost host label, a pattern is anchored with `^` and `$`, and no origin pattern may match the probe origins `https://cors-probe.invalid`, `http://cors-probe.invalid`, `https://cors-probe.com` and `null`. Refusing (not auto-anchoring) keeps the matching semantics of existing non-credentialed configs. No app in the sibling repos uses credentialed CORS today.
- WR-05: the review's primary fix (a boot error), not only a warning. WR-09: the zero `CookieConfig` gets Secure and SameSite=Lax; a partially filled config keeps its Secure value and gets Lax when SameSite is unset.
- WR-06: the nonce goes on all three script tags and `script-src` keeps only the nonce. `default-src`/`connect-src`/`style-src` are not added, so Swagger UI "Try it out" against other servers keeps working; human UAT item 4 (render under CSP) remains.
- WR-07: refuse (`attach.ErrProtectedNeedsPrivateBucket`) rather than only warn; `public_url` is new in v0.1.27 and no deployment sets it. cabana's upload handler answers the new error through its existing default branch (logged, opaque error); cabana is not changed.
- WR-08: a warning at boot, not a boot error (turnstile:closed already answers 503 in production without a secret).
- WR-01: in-flight coalescing on the purger (one request in flight, later saves collapse into one follow-up), no timer debounce. This supersedes plan 04's "one request per committed save, no debounce" by the user's decision in this gap run. `Purge` stays the synchronous send; `PurgeSiteCacheAfterCommit` queues. New exported `(*SiteCachePurger).Wait(ctx)` for tests, commands and shutdown. The plugin admin tests wait on it before asserting, which is why the plugins require the new framework tag at release.
- WR-03: the review's second option (the route accepts proxied requests on an explicit opt-in, `siteCache.allowProxied`). The first option, restricting `site_cache.purge_url` to loopback, would refuse at boot an https purge URL that a Journal deployment may already carry through the alias keys, which breaks a core plugin. The default stays "refuse proxied". Layer versions are additive minors: vue-faq-layer 0.3.0, vue-menu-layer 0.3.0, vue-journal-layer 0.5.0, tagged only at Task 7.
- Release: highest `v0.1.*` plus one (v0.1.28 at planning time). After the tag, the three plugins require it, as in plan 05. The app's layer pins and plugin pointers are a follow-up outside this plan.
<assumption_delta_decision>
Detector signal: `chosen` on "configurable" (SC-1 cookie attributes). Primary noun: the resolved token-cookie attributes (`bouncer.CookieConfig`). Decision: no-change. The attributes have been configurable since plan 02; this plan only tightens the derived Secure value in production (WR-05) and fills the zero-config defaults (WR-09).
</assumption_delta_decision>
## Context budget and resume point
The plan stays one plan (user decision), so its estimate exceeds one executor context. It runs in at most two contexts, split after Task 4. Tasks 1-4 hold the six framework fixes and the three plugin repositories; Tasks 5-7 hold the layers, the unit-test tables, the full gate and the release.
- **Record.** The last step of Task 4 writes `.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-PROGRESS.md` and commits it as a planning-docs commit. The record is deliberately not a SUMMARY. execute-phase treats a plan that has a `-SUMMARY.md` as finished, so a partial SUMMARY would skip Tasks 5-7 and the release checkpoint.
- **Resume rule.** Before Task 1, an executor of this plan checks for the record. When the record exists and `19.3-06-SUMMARY.md` does not, it does three things. It confirms every commit the record lists (`git -C <repo> cat-file -e <hash>`). It does not redo the tasks the record marks done. It starts at the task named on the record's `next:` line. A later task is skipped only when its commits exist (subject with `(19.3-06)` and the finding ID) and its acceptance criteria hold. A partly done task resumes at its first missing commit.
- **After a reset.** If the executor's context ends after Task 4, execute-phase's resume gate finds plan-06 commits without a SUMMARY and asks how to recover. The fitting answer is a continuation executor: its completed tasks are the record's table and its resume point is the `next:` task. Re-executing from scratch would duplicate the plugin commits in the sibling repositories.
- **Close-out.** The SUMMARY carries over the record's commits, verify results and notes. The record stays as written.
## Artifacts this phase produces (this plan)
- surf: `Router.wrap` applies `cors:<profile>` outermost (inside `recordPattern`); `bufferedResponse.commit` merges `Vary`; credentialed origin rules in `validateCORSConfig` (probe origins); api-docs page with a nonce on every script and a nonce-only `script-src`; boot warning `surf: turnstile.secret is empty` with a route count.
- bouncer: `LoadCookieConfig` refuses an inferred insecure cookie in production; `CookieConfig` zero-value defaults (SameSite=Lax, Secure).
- lagoon/attach: `attach.ErrProtectedNeedsPrivateBucket`; `attach.Store` refuses protected uploads while `storage.uploads.public_url` is set; the OpenBucket boot line becomes a warning.
- cooler: purge client without redirects; queued, coalesced after-commit purges; `cooler.SiteCachePurger.Wait`; `Handle.Close` drains the purger.
- Layers: `siteCache.allowProxied` (`NUXT_SITE_CACHE_ALLOW_PROXIED`) in the shared purge lib, route and nuxt.config, byte-identical in vue-faq-layer 0.3.0, vue-menu-layer 0.3.0 and vue-journal-layer 0.5.0, with two new `sitePurge:` specs and updated hash pins.
- Plugin tests: FAQ, menu and Journal admin purge tests wait for the queued purge.
- Tests: TestCORSProfileHeadersOnRefusals, TestCORSProfileVaryKeptWithHandlerVary, TestCORSCredentialedOriginRules, TestCORSAnchoredPatternNoSuffixMatch, TestAPIDocsCSPNoHostSource, TestTurnstileEmptySecretWarnsAtBoot, TestLoadCookieConfigProductionInferredSecure, TestZeroCookieConfigDefaults, TestStoreProtectedNeedsPrivateBucket, TestSiteCachePurgeNoRedirect, TestSiteCachePurgeAsyncCoalesced, TestSiteCachePurgeWaitAndClose, TestSiteCacheBulkCommitCoalesced (Docker); regression rows in TestCORSCredentialsWildcardRefused, TestLoadCookieConfigErrors, TestTokenCookieAttributes, TestAPIDocsServesPinnedPage.
- Gate: every new name required in scripts/check-phase19.3.sh (go, docker and layers stages).
- Docs: routing.md (CORS profiles, credentials rules, api-docs CSP), authentication.md (cookie Secure rule, zero config, Turnstile warning), storage.md (public_url refuses protected uploads), caching.md (queued purges, redirects, remote purge URL with allowProxied); surf, bouncer, lagoon and cooler READMEs; layer READMEs.
- Planning: 19.3-REVIEW-DISPOSITION.md (CR-01, WR-01..WR-09 fixed), 19.3-VALIDATION.md rows for plan 06.
- Release (on the human's reply): tag v0.1.28, plugin go.mod bumps, layer tags v0.3.0, v0.3.0 and v0.5.0.
<objective>
Close the CR-01 verification gap so a credentialed SPA can read every answer of a `cors:<profile>` route, fix review warnings WR-01..WR-09 with docs and tests, and release v0.1.28 on the human's reply.
Purpose: SC-1/D-05 (the failed truth), the hardening behind D-03, D-04, D-10, D-11, D-12, D-17 and D-18, and SC-4 (docs and unit tests for the changed code).
Output: commits in summercms.go, the three plugins (tests only) and the three layers; the updated gate; the disposition record; the release record.
</objective>
<execution_context>
@~/.claude/msd-core/workflows/execute-plan.md
@~/.claude/msd-core/templates/summary.md
</execution_context>
<context>
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VERIFICATION.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-REVIEW.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-CONTEXT.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-02-SUMMARY.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-04-SUMMARY.md
@.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-SUMMARY.md
Rules: one commit per finding where practical, README and docs in the same commit as the code they describe, planning docs in separate commits; stage by explicit path, never `git add -A`; no co-author tags. Before each task run `git status --porcelain` on the paths it touches and stop on foreign uncommitted edits (other sessions work in this checkout; `.planning/state.json` and `.msd/` are theirs, leave them alone; sm-journal-plugin's untracked go.work.sum stays untouched). Framework commits land before the plugin and layer commits that depend on them. The plugins build against this checkout through their checked-in go.work. Docker suites skip under `-short`; run them on a Docker host (`sudo -u golem` when the socket is denied) and record NOT RUN when no daemon answers. Framework code and docs never name a consuming application (the gate's docs stage scans added lines). Nothing is pushed or tagged before the Task 7 reply. State updates go through `msd_run query state.*` handlers. Before Task 1, apply the resume rule in "Context budget and resume point" (19.3-06-PROGRESS.md).
<interfaces>
Seams to change (read them; do not re-derive):
- surf (`modules/surf/router.go`): `route` (`middleware`, `raw`, `preflight`), `Router.wrap` (:404-473: the loop composing `rt.middleware` with index 0 outermost, then `bodyLimit`, `r.locale`, `recoverJSON`/`recoverBare`, then `recordPattern`), `BuildRouter` (`cors` factory :521-527, `turnstile` factory :528-540, the final `for _, rt := range r.routes { r.wrap(rt) }` loop, `addCORSPreflights`), `buildTurnstile` (:648), `recoverJSON`/`recoverBare`/`bufferedResponse.commit` (:735-805); `modules/surf/accesslog.go` `recordPattern`, `accessLogger(app)`; `modules/surf/cors.go` `validateCORSConfig`, `wildcardOriginPattern`, `newCORSPolicy`, `corsPolicy.apply`, `addVary`, `profileMiddleware`, `routeCORSProfile`, `LoadCORSProfiles`; `modules/surf/apidocs.go` asset constants, `apiDocsPage`, `apiDocsPageHandler`.
- surf tests (`modules/surf/cors_profile_test.go`, internal package): `routesPluginFunc`, `transportApp`, `transportBodyLimits`, `spaProfileYAML` (profiles `spa`, credentialed, origin https://www.example.com, and `open`), `writeHTTPConfig`, `corsRequest`, `serveCORS`, `varyHas`, `okHandler`.
- bouncer (`modules/bouncer/cookie.go`): `CookieConfig`, `LoadCookieConfig`, `cookieSecure`, `withDefaults`, `TokenCookie`, `ForgetTokenCookie`, `CookieRefreshHandler`, `CookieLogoutHandler`; `Turnstile` fields `Secret`, `Production`.
- attach: `Store` (store.go :111), error vars (store.go :25-40), `PublicURLBase`, `setPublicURLBase` (internal), `OpenBucket` (bucket.go :113-155); `TestStoreSmokeRefusals` uses `&gorm.DB{}` and memblob for refusals that happen before any write.
- cooler (`modules/cooler/sitecache.go`): `SiteCachePurger` (`url`, `endpoint`, `token`, `client`), `NewSiteCachePurgerFor`, `Purge`, `purgeTransportError`, `PurgeSiteCacheAfterCommit`, `SiteCacheFromApp`, `installSiteCache`; `cooler.go` `Handle`, `Handle.Close`, `Install`; `modules/surf/serve.go` defers `coolerHandle.Close()`.
- Plugins: the receivers in `../sm-faq-plugin/site_cache_admin_test.go`, `../sm-menu-plugin/site_cache_admin_test.go` (`take`, `expectOnePurge`, `expectNone`) and `../sm-journal-plugin/site_cache_admin_test.go` (`newPostsEnv`, `env`, `take`, `expectOnePurge`); `../sm-journal-plugin/classes/site_cache_test.go` :209-215. Heads at planning time: sm-faq-plugin fa90e83, sm-menu-plugin fe27b9a, sm-journal-plugin a99e9a1.
- Layers: `server/lib/siteCachePurge.ts` (`SitePurgeAuthInput`, `authorizeSitePurge`), `server/api/__site__/cache/purge.post.ts` (`PROXY_HEADERS`), `tests/sitePurge.test.ts` (`SHARED_LIB_SHA256`, `SHARED_ROUTE_SHA256`, 11 `sitePurge:` specs), `nuxt.config.ts` runtimeConfig `siteCache`. Heads: vue-faq-layer 96b6bc0 (v0.2.0), vue-menu-layer dd0ee35 (v0.2.0), vue-journal-layer 5936f66 (v0.4.0).
- Gate (`scripts/check-phase19.3.sh`): lists `BOUNCER_TESTS`, `SURF_TESTS`, `ATTACH_TESTS` (run untagged and with `-tags summercms_s3`), `COOLER_TESTS`, `DOCKER_COOLER_TESTS`, `LAYER_REQUIRE`, `LAYER_SHARED`; `go_named` requires a `pass` event per name and fails on skip.
</interfaces>
</context>
<tasks>
<task type="tracer">
<name>Task 1: Tracer: a cross-origin SPA reads every refusal of a cors:&lt;profile&gt; route; Router.wrap applies the profile outermost, so a 401 or 403 from a guard listed before it or set on the group, a Turnstile 422 or 503, a 429, a 413 and a recovered 500 all carry Allow-Origin, Allow-Credentials and Vary: Origin (CR-01)</name>
<reversibility rating="reversible">Internal composition order in wrap; the cors:&lt;profile&gt; name and the http.cors_profiles keys are unchanged.</reversibility>
<precondition>`git status --porcelain -- modules/surf docs/services/routing.md scripts/check-phase19.3.sh` in summercms.go prints nothing.</precondition>
<files>modules/surf/router.go, modules/surf/cors.go, modules/surf/cors_profile_test.go, modules/surf/README.md, docs/services/routing.md, scripts/check-phase19.3.sh</files>
<read_first>modules/surf/router.go (wrap :404-473, BuildRouter cors factory :521-527, recoverJSON, recoverBare and bufferedResponse.commit :735-805), modules/surf/cors.go (corsPolicy.apply, addVary, profileMiddleware, routeCORSProfile, addCORSPreflights), modules/surf/accesslog.go (recordPattern), modules/surf/bodylimit.go, modules/surf/cors_profile_test.go (the helpers in the interfaces block, TestCORSProfilePreflightRoute), modules/surf/transport_smoke_test.go (TestCORSProfileSmoke, turnstileRoutes, TestTurnstileMiddlewareSmoke: the published verifier and the production app), modules/bouncer/turnstile.go (Verify: an empty token is refused before any siteverify call), modules/surf/cors_coverage_test.go (TestCORSOptionsMatchesLaravel), docs/services/routing.md (middleware table :101, CORS profiles :252-270), modules/surf/README.md (:30-31), scripts/check-phase19.3.sh (SURF_TESTS), 19.3-REVIEW.md (CR-01), 19.3-VERIFICATION.md (gap missing list)</read_first>
<action>Per D-05 (CR-01, the verification gap). router.go wrap: before the middleware loop, resolve the route's profile with routeCORSProfile(rt) (return its error as is) and keep today's unknown-profile boot error with its message. Inside the loop, an entry whose base is cors (with a param) is validated and then skipped, so it is never composed at its list position. Group entries arrive through rt.middleware, so group nesting needs nothing extra. After the recovery wrapper (recoverJSON or recoverBare) and before recordPattern, wrap h in r.corsProfiles[profile].profileMiddleware when the route names a profile. Preflight routes keep their current path. Keep the cors factory registration in BuildRouter (the name must keep resolving and stay reserved for surf) and change its comment to say wrap applies the profile itself. bufferedResponse.commit: copy every buffered header key as today except Vary. Merge each comma-separated Vary value into the destination through addVary, so a Vary: Origin the profile set on the outer writer survives a handler that sets its own Vary. Preflights never pass through commit, so the bytes TestCORSOptionsMatchesLaravel pins must not change. Change the profileMiddleware doc comment: it runs outermost, outside the body limit, locale and recovery.
Test in cors_profile_test.go, reusing the helpers: TestCORSProfileHeadersOnRefusals with spaProfileYAML, no http.cors, and a test plugin that registers three named middlewares: deny, which answers 401 without calling next, forbid, which answers 403 without calling next, and readall, which reads the whole body and answers 413 on *http.MaxBytesError. Subtests, each sending Origin https://www.example.com:
- route [deny, cors:spa]: 401;
- subtest guard_403, route [forbid, cors:spa]: 403;
- a group Use(deny) whose route names cors:spa: 401;
- [throttle:1,1, cors:spa]: the second request gets 429;
- [body.limit:16, readall, cors:spa] with a 64-byte body: 413;
- a handler that panics behind cors:spa: 500 (JSON);
- a GroupRaw with cors:spa and a panicking handler: 500;
- subtest turnstile_422: an app with a published bouncer.NewTurnstile("test-secret", false), as TestTurnstileMiddlewareSmoke publishes one, and a POST route [turnstile:closed, cors:spa]; a request without a token answers 422. The empty token is refused before any siteverify call, so no fake server is needed;
- subtest turnstile_503: transportApp in the production environment with no turnstile.secret, POST route [turnstile:closed, cors:spa]: 503.
The two Turnstile subtests assemble their own app, because BuildRouter builds one verifier per router. Every status the docs sentence below names (401, 403, 422, 503, 429, 413, 500) has a subtest here. Each asserts Access-Control-Allow-Origin equal to the origin, Access-Control-Allow-Credentials true and Vary containing Origin. One more case: [deny, cors:spa] with Origin https://evil.example.org gives 401 with no Allow-Origin and with Vary containing Origin. TestCORSProfileVaryKeptWithHandlerVary: a handler behind cors:spa sets Vary: Accept-Language, and the response's Vary names both Origin and Accept-Language. Add both names to SURF_TESTS in scripts/check-phase19.3.sh.
Docs, in the same commit:
- docs/services/routing.md "CORS profiles": replace the sentence that tells readers to list the profile beside the guard. The new text says the route (or its group) lists cors:spa among its middleware names, and surf applies the profile outside every other layer of the route, whatever its position in the list or the group nesting. Every response of the route therefore carries the profile's headers, including refusals from guards (401, 403), throttle (429) and Turnstile (422, 503), the body limit (413) and recovered panics (500); otherwise the browser would hide them from the page. Shorten the same statement into the cors:&lt;profile&gt; row of the middleware table.
- modules/surf/README.md: the Features bullet on named CORS profiles gets the same statement, including the words "whatever its position".
Commit: `fix(19.3-06): cors:<profile> headers on every response of the route (CR-01)`.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/surf &amp;&amp; go test -count=1 -short -run '^(TestCORSProfileHeadersOnRefusals|TestCORSProfileVaryKeptWithHandlerVary|TestCORSOptionsMatchesLaravel|TestCORSProfilePreflightRoute|TestCORSProfileSmoke)$' -v ./modules/surf 2&gt;&amp;1 | grep -cE '^--- PASS: Test[A-Za-z]+ \(' | grep -qx 5 &amp;&amp; go test -count=1 -short -run '^TestCORSProfileHeadersOnRefusals$/^(guard_403|turnstile_422|turnstile_503)$' -v ./modules/surf 2&gt;&amp;1 | grep -cE '^\s+--- PASS: TestCORSProfileHeadersOnRefusals/(guard_403|turnstile_422|turnstile_503) \(' | grep -qx 3 &amp;&amp; bash scripts/check-phase19.3.sh go &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit: vet or the surf short suite is red, fewer than five top-level "--- PASS" lines (a refusal still lacks the profile headers, a handler's Vary erased Origin, the Laravel preflight bytes changed or the profile smoke broke), fewer than three passing guard_403, turnstile_422 and turnstile_503 subtests (a 403 or Turnstile refusal the docs name is untested or lacks the headers), the gate's go stage refuses (a SURF_TESTS name missing, skipped or red), or the docs checker reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -c 'TestCORSProfileHeadersOnRefusals\|TestCORSProfileVaryKeptWithHandlerVary' scripts/check-phase19.3.sh` prints 2.
- `grep -cE '"(guard_403|turnstile_422|turnstile_503)"' modules/surf/cors_profile_test.go` prints at least 3, and the verify step shows each of the three subtests passing (every status the routing.md sentence names has a subtest).
- `grep -c 'profileMiddleware' modules/surf/router.go` prints at least 2 (the factory and the wrap call).
- `grep -n 'whatever its position' docs/services/routing.md modules/surf/README.md` prints at least one line per file.
- `grep -c 'next to its guard' docs/services/routing.md` prints 0.
- `git log --oneline -1 --grep='(CR-01)' -- modules/surf/router.go` prints one line (the fix commit touches router.go).
</acceptance_criteria>
<done>A browser on the SPA origin can read every answer of a profiled route, so the 401 that triggers the cookie refresh reaches the page, and the gate requires the proof by name.</done>
</task>
<task type="auto">
<name>Task 2: Credentialed sessions cannot be opened by another spelling: credentialed CORS refuses wildcard-host and unanchored origins (WR-04), production refuses an inferred insecure token cookie (WR-05), and the zero CookieConfig writes SameSite=Lax and Secure (WR-09)</name>
<reversibility rating="reversible">Boot validation and builder defaults; no key, name or signature changes.</reversibility>
<files>modules/surf/cors.go, modules/surf/cors_profile_test.go, modules/surf/README.md, docs/services/routing.md, modules/bouncer/cookie.go, modules/bouncer/cookie_test.go, modules/bouncer/README.md, docs/services/authentication.md</files>
<read_first>modules/surf/cors.go (validateCORSConfig, phpDelimited, wildcardOriginPattern, newCORSPolicy), modules/surf/cors_profile_test.go (TestCORSCredentialsWildcardRefused :36-71, TestCORSPatternErrors, TestCORSWildcardOriginEntry, TestCORSSubdomainCredentialed), modules/surf/cors_coverage_test.go (pattern literals), docs/services/routing.md (CORS credentials rules), modules/surf/README.md (http.cors rows), modules/bouncer/cookie.go (LoadCookieConfig :47-110, cookieSecure, withDefaults :262-272, TokenCookie, ForgetTokenCookie, both handlers), modules/bouncer/cookie_test.go (TestLoadCookieConfigErrors :95, TestTokenCookieAttributes :126-153), modules/bouncer/README.md (:34, :159-187), docs/services/authentication.md (Cookie transport :60-90), 19.3-REVIEW.md (WR-04, WR-05, WR-09)</read_first>
<action>Three commits, one per finding, each with its README and docs lines.
WR-04 (per D-04), surf/cors.go validateCORSConfig. These rules apply only when SupportsCredentials is true; non-credentialed configs keep loading exactly as today.
- (a) An allowed_origins entry containing * (the bare * is already refused) is accepted only when its scheme is literal and every * is a whole leftmost host label followed by a dot, as in https://*.example.com.
- (b) Every allowed_origins_patterns entry must start with ^ and end with $.
- (c) No origin pattern may match any of the probe origins https://cors-probe.invalid, http://cors-probe.invalid, https://cors-probe.com and null (an unexported list). This covers the compiled wildcard entries too, and it catches https://*, *://*, https://*.com and ^https?://.*$.
Each error names the key with its index (for example http.cors.allowed_origins[1] or http.cors_profiles.spa.allowed_origins_patterns[0]) and says what is accepted. Profiles get the same checks through LoadCORSProfiles. Regression: append rows to TestCORSCredentialsWildcardRefused. https://* and the unanchored https://.*\.example\.com are refused with credentials, and both still load without credentials. Docs:
- routing.md credentials rules: list the three rules, including that patterns must be anchored with ^ and $;
- surf README: the http.cors allowed_origins and allowed_origins_patterns rows mention them.
Commit: `fix(19.3-06): credentialed CORS refuses wildcard-host and unanchored origins (WR-04)`.
WR-05 (per D-03), bouncer/cookie.go LoadCookieConfig: in the production environment, a Secure that resolves to false is an error, whether auth.cookie.secure was set explicitly or inferred from an http or unset app.url. Keep the explicit-false message. For the inferred case, return an error prefixed bouncer: that contains "auth.cookie.secure resolves to false in the production environment", names app.url, and tells the reader to set app.url to an https URL or auth.cookie.secure: true. Non-production environments keep inferring false for an http app.url. Update the LoadCookieConfig doc comment. Regression: append rows to TestLoadCookieConfigErrors for production with app.url http and production with app.url unset. Docs: the auth.cookie.secure row in authentication.md and the error sentence below that table; the bouncer README Configuration row. Commit: `fix(19.3-06): production refuses an inferred insecure token cookie (WR-05)`.
WR-09 (per D-03), withDefaults:
- a CookieConfig equal to the zero value also gets Secure true; decide this before Name and Path are filled;
- any config whose SameSite is the zero value gets http.SameSiteLaxMode;
- a config with any field set keeps its Secure value.
Confirm that TokenCookie, ForgetTokenCookie and both handlers go through withDefaults. Update the CookieConfig field docs and the withDefaults comment: SameSite is Lax when unset; Secure has no default except for the zero config, which is Secure; LoadCookieConfig derives Secure from app.url. Regression:
- extend the zero-config assertions in TestTokenCookieAttributes to SameSite Lax and Secure;
- a partial config ({Name: sid}) gets SameSite Lax and stays not Secure.
Docs:
- bouncer README: the CookieConfig row and the Cookie transport bullet;
- authentication.md: one sentence after the key table on a hand-built config.
Commit: `fix(19.3-06): zero CookieConfig writes SameSite=Lax and Secure (WR-09)`.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short ./modules/surf ./modules/bouncer &amp;&amp; go test -count=1 -short -run '^(TestCORSCredentialsWildcardRefused|TestCORSWildcardOriginEntry|TestCORSSubdomainCredentialed|TestCORSOptionsMatchesLaravel|TestLoadCookieConfigErrors|TestLoadCookieConfigDefaults|TestTokenCookieAttributes)$' -v ./modules/surf ./modules/bouncer 2&gt;&amp;1 | grep -cE '^--- PASS: Test[A-Za-z]+ \(' | grep -qx 7 &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit, or fewer than seven top-level "--- PASS" lines: a credentialed wildcard-host or unanchored entry still loads, an accepted credentialed config (https://*.example.com, the api and www case) is refused, the Laravel preflight bytes changed, production accepts an inferred insecure cookie, the zero config lacks SameSite=Lax or Secure, or the docs checker fails.</fails_when>
</verify>
<acceptance_criteria>
- `grep -cE 'cors-probe\.(invalid|com)' modules/surf/cors.go` prints at least 2.
- `grep -ni 'anchored' docs/services/routing.md` prints at least one line.
- `grep -c 'resolves to false' modules/bouncer/cookie.go` prints at least 1.
- `git log --oneline --grep='(WR-04)$' --grep='(WR-05)$' --grep='(WR-09)$'` prints 3 lines (one commit per finding).
</acceptance_criteria>
<done>No credentialed CORS config can trust arbitrary origins, and no production token cookie is written without Secure, whatever the config spelling.</done>
</task>
<task type="auto">
<name>Task 3: Exposure and silent misconfiguration closed: the api-docs page runs only nonce-carrying scripts (WR-06), protected uploads are refused while public_url serves the bucket publicly (WR-07), and turnstile routes without a secret are named in a boot warning (WR-08)</name>
<reversibility rating="reversible">An added error value, a CSP header value and a log line; all can be relaxed later without a migration.</reversibility>
<files>modules/surf/apidocs.go, modules/surf/apidocs_test.go, modules/surf/router.go, modules/surf/cors_profile_test.go, modules/surf/README.md, docs/services/routing.md, docs/services/authentication.md, modules/lagoon/attach/store.go, modules/lagoon/attach/bucket.go, modules/lagoon/attach/url_test.go, modules/lagoon/README.md, docs/services/storage.md</files>
<read_first>modules/surf/apidocs.go (asset constants :32-45, apiDocsPage :53-80, apiDocsPageHandler :188-210), modules/surf/apidocs_test.go (TestAPIDocsServesPinnedPage), docs/services/routing.md (API documentation page :272-300), modules/surf/README.md (:39 and the turnstile.secret row :165), modules/surf/router.go (BuildRouter final wrap loop, buildTurnstile), modules/surf/accesslog.go (accessLogger), modules/bouncer/turnstile.go (Turnstile fields), modules/surf/cors_profile_test.go (TestTurnstileFactoryParams), docs/services/authentication.md (Turnstile keys :180-190), modules/lagoon/attach/store.go (error vars :25-40, Store :111-185), modules/lagoon/attach/bucket.go (OpenBucket :113-155), modules/lagoon/attach/url_test.go (TestPublicURLBase and its cleanup), modules/lagoon/attach/store_test.go (TestStoreSmokeRefusals), modules/cabana/field_file.go (:810-845, how an unknown Store error is answered), modules/lagoon/README.md (:41, :194-195, :235), docs/services/storage.md (:84-96), 19.3-REVIEW.md (WR-06, WR-07, WR-08)</read_first>
<action>Three commits, one per finding, each with its docs.
WR-06 (per D-17), apidocs.go:
- The two external script tags carry nonce="{{.Nonce}}" next to their integrity and crossorigin attributes.
- The Content-Security-Policy keeps the per-request nonce as the only script-src source: script-src 'nonce-n'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'. There is no host source.
- The CSS link and its SRI stay. Remove the CDN host constant if nothing else uses it. The version and the integrity values do not change.
Regression: TestAPIDocsServesPinnedPage asserts that all three script tags carry the header's nonce and that the script-src directive has no https: source. Docs: the API documentation page paragraph in routing.md says scripts run only with the page's per-request nonce; the two jsDelivr scripts carry it as well as their SRI hash, so no other jsDelivr script can run. Update the matching surf README Features bullet. Commit: `fix(19.3-06): api-docs CSP allows only nonce-carrying scripts (WR-06)`.
WR-07 (per D-12), attach:
- A new exported error, ErrProtectedNeedsPrivateBucket. Its message names storage.uploads.public_url and says that a protected (is_public false) upload needs a private bucket: unset public_url or make the relation public.
- Store returns it when Upload.Public is false and PublicURLBase() is not empty, before it reads the body or writes a blob. Public uploads, and any upload without public_url, behave as today.
- OpenBucket's line about a set public_url moves from Info to Warn and says protected uploads are refused.
- cabana answers an unknown Store error through its default branch (logged, opaque error). Leave cabana unchanged and record in the SUMMARY what the admin sees.
Regression in url_test.go (package attach, so it can set the base): TestStoreProtectedNeedsPrivateBucket, using memblob and &gorm.DB{} as TestStoreSmokeRefusals does. With the base set, a Public false upload returns the error and leaves the bucket empty. Reset the base in t.Cleanup and do not mark the test parallel. Docs:
- storage.md: the public_url paragraph says protected uploads are refused while public_url is set;
- lagoon README: the attach bullet, an API reference row for attach.ErrProtectedNeedsPrivateBucket, and the storage.uploads.public_url configuration row.
Commit: `fix(19.3-06): refuse protected uploads into a public_url bucket (WR-07)`.
WR-08 (per D-18), router.go BuildRouter:
- After the loop that wraps every route, count the routes whose middleware list names turnstile:open or turnstile:closed.
- When the count is positive and the verifier from buildTurnstile has an empty Secret, log one warning through accessLogger(app). The message starts "surf: turnstile.secret is empty" and says that turnstile:open passes every request, while turnstile:closed passes outside production and answers 503 in production. The attributes are routes (the count) and production (the verifier's Production flag).
- Never log the secret.
- No warning when no route names turnstile, or when the secret is set.
Regression in cors_profile_test.go: TestTurnstileEmptySecretWarnsAtBoot. Publish a *slog.Logger with a capturing handler on the app. Two turnstile routes and no secret give one warning with routes 2. A published verifier with a secret gives none, and so does an app without turnstile routes. Docs: the turnstile.secret rows in the surf README and in authentication.md mention the start-up warning. Commit: `fix(19.3-06): warn at boot when turnstile routes have no secret (WR-08)`.</action>
<verify>
<automated>go vet ./... &amp;&amp; go vet -tags summercms_s3 ./modules/lagoon/... &amp;&amp; go test -count=1 -short ./modules/surf ./modules/lagoon/... &amp;&amp; go test -count=1 -short -run '^(TestAPIDocsServesPinnedPage|TestAPIDocsSmoke|TestTurnstileEmptySecretWarnsAtBoot|TestTurnstileFactoryParams|TestStoreProtectedNeedsPrivateBucket|TestPublicURLBase)$' -v ./modules/surf ./modules/lagoon/attach 2&gt;&amp;1 | grep -cE '^--- PASS: Test[A-Za-z]+ \(' | grep -qx 6 &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer</automated>
<fails_when>Non-zero exit (vet in either tag mode, a short suite, the docs checker), or fewer than six top-level "--- PASS" lines: a script tag lacks the nonce or script-src still names a host, a protected upload into a public_url bucket is stored or leaves a blob, or the turnstile warning is missing, repeated or logged with a secret set.</fails_when>
</verify>
<acceptance_criteria>
- `grep -c 'nonce="{{.Nonce}}"' modules/surf/apidocs.go` prints 3.
- `grep -n 'ErrProtectedNeedsPrivateBucket' modules/lagoon/attach/store.go modules/lagoon/README.md docs/services/storage.md` prints at least one line per file.
- `grep -c 'turnstile.secret is empty' modules/surf/router.go` prints 1.
- `git log --oneline --grep='(WR-06)$' --grep='(WR-07)$' --grep='(WR-08)$'` prints 3 lines (one commit per finding).
</acceptance_criteria>
<done>Only nonce-carrying scripts run on /api-docs, protected files can no longer land in a publicly served bucket, and an operator learns at start-up when bot protection is off.</done>
</task>
<task type="auto">
<name>Task 4: An admin save no longer waits for the site, and a bulk delete sends at most two purges: the purge client never follows redirects (WR-02), and PurgeSiteCacheAfterCommit queues onto a coalescing per-purger worker that Wait and shutdown drain (WR-01)</name>
<reversibility rating="costly">cooler.SiteCachePurger.Wait joins cooler's published API and the plugin tests call it; reshaping it later needs a coordinated plugin release.</reversibility>
<precondition>`git status --porcelain -- modules/cooler docs/services/caching.md` in summercms.go prints nothing, and `git -C ../sm-faq-plugin status --porcelain`, `git -C ../sm-menu-plugin status --porcelain` and `git -C ../sm-journal-plugin status --porcelain -uno` print nothing.</precondition>
<files>modules/cooler/sitecache.go, modules/cooler/cooler.go, modules/cooler/sitecache_test.go, modules/cooler/README.md, docs/services/caching.md, ../sm-faq-plugin/site_cache_admin_test.go, ../sm-menu-plugin/site_cache_admin_test.go, ../sm-journal-plugin/site_cache_admin_test.go, ../sm-journal-plugin/classes/site_cache_test.go, .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-PROGRESS.md</files>
<read_first>modules/cooler/sitecache.go (whole), modules/cooler/cooler.go (Handle, Handle.Close :110-117, Install :163-215), modules/surf/serve.go (:40-120, coolerHandle.Close at shutdown), modules/lagoon/transaction.go (AfterCommit, runAfterCommit), modules/cooler/sitecache_test.go (purgeReceiver, newTestPurger, captureLogs, the outside-a-transaction subtest :374-381, TestSiteCacheAfterCommitOnly :445, TestSiteCacheFailureNeverFails :519), modules/cooler/README.md (:39, :175-180), docs/services/caching.md (Site cache purge :100-140), ../sm-faq-plugin/site_cache_admin_test.go, ../sm-menu-plugin/site_cache_admin_test.go, ../sm-journal-plugin/site_cache_admin_test.go, ../sm-journal-plugin/classes/site_cache_test.go (:205-216), ../sm-faq-plugin/controllers/site_cache.go and ../sm-journal-plugin/classes/site_cache.go (call sites that stay unchanged), 19.3-REVIEW.md (WR-01, WR-02)</read_first>
<action>Framework first, two commits.
WR-02 (per D-10): NewSiteCachePurgerFor builds the client with a CheckRedirect that returns http.ErrUseLastResponse. A 3xx answer is never followed, so the bearer token never goes to the Location and the POST never turns into a GET. Purge logs the 3xx through the existing "cooler: site cache purge failed" warning with its status. Regression in sitecache_test.go: TestSiteCachePurgeNoRedirect. Receiver A answers 307 with a Location pointing at receiver B; B receives nothing; one failed warning carries status 307; no log line contains the token. Docs: "Timing and failures" in caching.md says redirects are not followed and are logged as failures with their status. Commit: `fix(19.3-06): site-cache purge never follows redirects (WR-02)`.
WR-01 (per D-10 and D-11). The user's decision in this gap run supersedes plan 04's "one request per committed save, no debounce".
Keep every exported name and signature: SiteCachePurger, NewSiteCachePurger, NewSiteCachePurgerFor, Purge, Endpoint, PurgeSiteCacheAfterCommit, SiteCacheFromApp and the constants. Purge stays the synchronous single send.
Queue:
- SiteCachePurger gets unexported queue state: a mutex, a sending flag, one pending kind and slug, a coalesced count, and an idle channel that is closed when a busy period ends.
- An unexported enqueue(kind, slug). When nothing is in flight, it marks sending and starts one goroutine that sends. While a send is in flight, it replaces the pending body with the latest one and counts it.
- The goroutine loops. It sends through the same code as Purge (a detached background context bounded by SiteCachePurgeTimeout). Then it takes the pending body, if any, and sends again. Otherwise it clears sending and closes idle.
- The success and failure log lines keep their messages and attributes. They add coalesced only when it is above zero, so the existing cooler and Journal log assertions keep passing.
After commit and shutdown:
- PurgeSiteCacheAfterCommit registers an after-commit callback that calls enqueue. The admin request returns as soon as the commit's callbacks have queued. A nil purger still does nothing; outside a transaction the purge queues at once.
- New exported method Wait(ctx context.Context) error. It returns nil at once for a nil or idle purger. Otherwise it blocks until the purger is idle, looping when a new busy period starts, or until ctx ends, then returning ctx.Err().
- cooler.go: installSiteCache returns the purger it published, and Install keeps it on the Handle in an unexported field. Handle.Close first waits for it with a context bounded by SiteCachePurgeTimeout plus one second and ignores the deadline error. Then it closes the Redis client as today. Close no longer returns before the drain when there is no Redis client.
- Update the doc comments:
- PurgeSiteCacheAfterCommit: queued after commit, fire and forget, at most one request in flight per purger, later saves collapse into one follow-up;
- Purge: the blocking send the queue uses;
- Wait.
Existing tests:
- The tests that count requests after PurgeSiteCacheAfterCommit call Wait before counting: the outside-a-transaction subtest, TestSiteCacheAfterCommitOnly and TestSiteCacheFailureNeverFails.
- Every test that queues a purge waits before it returns, because captureLogs restores slog.Default afterwards.
Regression: TestSiteCachePurgeAsyncCoalesced (Docker-free). A receiver blocks until released. Twenty PurgeSiteCacheAfterCommit calls with a nil db each return within 50 ms. After the release and Wait, the receiver has exactly two requests, the second carrying the last slug, and the purged log line of the second carries coalesced 18.
Docs, in the same commit:
- caching.md, rewrite "What a purge sends" and "Timing and failures":
- the purge is queued after commit and never blocks the save;
- at most one request is in flight per process;
- saves committed meanwhile collapse into one follow-up request, because each purge clears the whole route cache;
- Wait serves tests and commands;
- serve waits for a queued purge on shutdown, up to the timeout plus a second.
- cooler README: the Features bullet; the API rows for PurgeSiteCacheAfterCommit and Purge; a new cooler.SiteCachePurger.Wait row.
Commit: `fix(19.3-06): queue site-cache purges after commit and coalesce them (WR-01)`.
Plugins, test-only, one commit per repository after the framework commit; plugin controllers and the Journal aliases stay untouched:
- In site_cache_admin_test.go of sm-faq-plugin, sm-menu-plugin and sm-journal-plugin, every read of the purge receiver first drains the purger used by the app under test. That covers take, expectOnePurge, expectNone and the direct take calls. Drain through cooler.SiteCacheFromApp(the harness's app).Wait with a 5-second context, so each save's purge is asserted after it was sent. The probe and Seen assertions stay.
- In sm-journal-plugin classes/site_cache_test.go, the outside-a-transaction subtest waits before reading hits, and its name says the purge is queued at once.
Each commit: `test(19.3-06): wait for the queued site-cache purge before asserting (WR-01)`, staged by explicit path.
Progress record, the last step of Task 4, after the three plugin commits and this task's verify (see "Context budget and resume point"). Write .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-PROGRESS.md with:
- a first line saying it is the mid-plan record of 19.3-06, superseded by 19.3-06-SUMMARY.md once that file exists;
- the line `next: Task 5` on its own;
- a Completed Tasks table for Tasks 1-4 in the executor's checkpoint shape: task, name, commit (repository, short hash and subject), key files, one row per commit;
- the HEAD of summercms.go and of the six sibling repositories (the layers still at their planning-time heads);
- each task's verify result, with the docker stage marked as run or NOT RUN;
- the notes the SUMMARY needs from Tasks 1-4: deviations, decisions taken while executing, and what the admin shows for a refused protected upload (Task 3).
Commit it alone as `docs(19.3-06): progress record after Task 4`, staged by its path. Do not write 19.3-06-SUMMARY.md here.</action>
<verify>
<automated>go vet ./... &amp;&amp; go test -count=1 -short -race ./modules/cooler &amp;&amp; go test -count=1 -short -run '^(TestSiteCachePurgeNoRedirect|TestSiteCachePurgeAsyncCoalesced|TestSiteCachePurgeRequest|TestSiteCachePurgeConfig|TestSiteCacheFromApp)$' -v ./modules/cooler 2&gt;&amp;1 | grep -cE '^--- PASS: Test[A-Za-z]+ \(' | grep -qx 5 &amp;&amp; bash scripts/check-phase19.3.sh plugins &amp;&amp; bash scripts/check-phase19.3.sh docker</automated>
<fails_when>Non-zero exit: vet, a data race report, a cooler unit test, fewer than five top-level "--- PASS" lines (a redirect was followed, a call blocked or more than two requests went out), the plugins stage, or the docker stage. The docker stage fails when a framework or plugin purge test is red or skipped, for example a plugin test that read the receiver before the queued purge was sent, or when no Docker daemon answers.</fails_when>
</verify>
<acceptance_criteria>
- `grep -c 'ErrUseLastResponse' modules/cooler/sitecache.go` prints 1.
- `grep -n 'func (p \*SiteCachePurger) Wait(ctx context.Context) error' modules/cooler/sitecache.go` prints one line and `grep -n 'cooler.SiteCachePurger.Wait' modules/cooler/README.md` prints one line.
- `git -C ../sm-faq-plugin diff fa90e83 -- controllers`, `git -C ../sm-menu-plugin diff fe27b9a -- controllers` and `git -C ../sm-journal-plugin diff a99e9a1 -- controllers classes/site_cache.go plugin.go` each print nothing (call sites and aliases unchanged).
- `grep -c 'Wait(' ../sm-faq-plugin/site_cache_admin_test.go ../sm-menu-plugin/site_cache_admin_test.go ../sm-journal-plugin/site_cache_admin_test.go` prints at least 1 for each file.
- `git -C ../sm-faq-plugin log --format=%ct -1` is not earlier than `git log --format=%ct -1 --grep='(WR-01)' -- modules/cooler/sitecache.go` (framework commit first).
- `grep -cx 'next: Task 5' .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-PROGRESS.md` prints 1, and `for id in CR-01 WR-04 WR-05 WR-09 WR-06 WR-07 WR-08 WR-02 WR-01; do grep -q "($id)" .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-PROGRESS.md || echo "missing $id"; done` prints nothing.
- `git log --oneline -1 -- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-PROGRESS.md` prints one line, and `test -e .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-SUMMARY.md` exits non-zero (no SUMMARY before the plan's end).
</acceptance_criteria>
<done>An editor's save, or a 50-row bulk delete, answers without waiting for the site, the site still gets its purge after the last commit, and the bearer token never follows a redirect. The committed progress record lets a fresh executor start at Task 5.</done>
</task>
<task type="auto">
<name>Task 5: A site behind its TLS proxy accepts purges when the operator opts in: the shared route honours runtimeConfig siteCache.allowProxied (NUXT_SITE_CACHE_ALLOW_PROXIED), byte-identical in the three layers, and the caching guide pairs it with an https purge_url (WR-03)</name>
<reversibility rating="costly">siteCache.allowProxied and NUXT_SITE_CACHE_ALLOW_PROXIED become deployment configuration of every site on the new layer versions.</reversibility>
<precondition>`git -C ../vue-faq-layer status --porcelain`, `git -C ../vue-menu-layer status --porcelain` and `git -C ../vue-journal-layer status --porcelain` print nothing; Task 4's caching.md commit is in place.</precondition>
<files>../vue-faq-layer/server/lib/siteCachePurge.ts, ../vue-faq-layer/server/api/__site__/cache/purge.post.ts, ../vue-faq-layer/tests/sitePurge.test.ts, ../vue-faq-layer/nuxt.config.ts, ../vue-faq-layer/README.md, ../vue-faq-layer/package.json, ../vue-menu-layer/server/lib/siteCachePurge.ts, ../vue-menu-layer/server/api/__site__/cache/purge.post.ts, ../vue-menu-layer/tests/sitePurge.test.ts, ../vue-menu-layer/nuxt.config.ts, ../vue-menu-layer/README.md, ../vue-menu-layer/package.json, ../vue-journal-layer/server/lib/siteCachePurge.ts, ../vue-journal-layer/server/api/__site__/cache/purge.post.ts, ../vue-journal-layer/tests/sitePurge.test.ts, ../vue-journal-layer/nuxt.config.ts, ../vue-journal-layer/README.md, ../vue-journal-layer/package.json, docs/services/caching.md, modules/cooler/README.md</files>
<read_first>../vue-faq-layer/server/lib/siteCachePurge.ts, ../vue-faq-layer/server/api/__site__/cache/purge.post.ts, ../vue-faq-layer/tests/sitePurge.test.ts (SHARED_LIB_SHA256, SHARED_ROUTE_SHA256, the route handler specs), ../vue-faq-layer/nuxt.config.ts (runtimeConfig siteCache :26-32), ../vue-faq-layer/README.md (Site cache purge), ../vue-menu-layer/nuxt.config.ts, ../vue-journal-layer/nuxt.config.ts (the journal alias entries stay), the three package.json, docs/services/caching.md (Site cache purge after Task 4), modules/cooler/sitecache.go (the https rule in NewSiteCachePurgerFor), 19.3-REVIEW.md (WR-03), 19.3-04-SUMMARY.md (shared files and hash pins)</read_first>
<action>Per D-10 and D-11. The model is the review's second option, an explicit opt-in. Restricting site_cache.purge_url to loopback would refuse at boot an https URL that Journal deployments may carry through the alias keys, which would break a core plugin. The default stays "refuse proxied".
vue-faq-layer first, one commit:
- server/lib/siteCachePurge.ts: SitePurgeAuthInput gains an optional allowProxied boolean. When it is true, authorizeSitePurge skips the proxied verdict, so the bearer token is the only gate (at least 32 characters, compared in constant time). The doc comments say so.
- server/api/__site__/cache/purge.post.ts passes allowProxied from runtimeConfig.siteCache.allowProxied, accepting true or the string "true". Its header comment documents the flag.
- nuxt.config.ts: runtimeConfig siteCache gains allowProxied: false. Its comment names NUXT_SITE_CACHE_ALLOW_PROXIED and says when to set it: the backend reaches the site through a TLS-terminating proxy, which adds X-Forwarded-For, X-Real-IP or Forwarded.
- tests/sitePurge.test.ts, two new specs; the existing specs stay unchanged:
- "sitePurge: allowProxied lets a proxied request reach the token check": with allowProxied true, a proxied request is ok with the right bearer and unauthorized with a wrong one, and is still disabled without a token;
- "sitePurge: route accepts a proxied request with allowProxied and the right bearer": in the file's route-handler spec style, it answers 200 with the count, and the same request answers 404 with the flag false.
- Recompute SHARED_LIB_SHA256 and SHARED_ROUTE_SHA256 once the lib and route are final.
- README: the Site cache purge section documents the flag and the environment variable.
- package.json: version 0.3.0.
Then copy the three shared files byte for byte into vue-menu-layer and vue-journal-layer. Add the same nuxt.config.ts entry and README paragraph to each. Versions: 0.3.0 for menu, 0.5.0 for journal. One commit per repository. vue-journal-layer keeps the deprecated /api/__journal__/cache/purge alias and app/utils/siteCache.ts unchanged. No tags here; Task 7 creates them.
Framework, one docs commit:
- caching.md "Site cache purge" gains a paragraph that pairs an https purge_url on another host with NUXT_SITE_CACHE_ALLOW_PROXIED=true on the site. It explains that the proxy adds forwarding headers, which the route refuses by default. It also says that the token is then the only gate, so it must stay long and secret.
- cooler README: the site_cache.purge_url configuration row points at that paragraph.
Commits:
- each layer: `feat(19.3-06): purge route accepts proxied requests on siteCache.allowProxied (WR-03)`;
- summercms.go: `docs(19.3-06): pair a remote purge URL with the site's allowProxied (WR-03)`.</action>
<verify>
<automated>bash scripts/check-phase19.3.sh layers &amp;&amp; ( for l in ../vue-faq-layer ../vue-menu-layer ../vue-journal-layer; do pnpm -C "$l" test 2&gt;&amp;1 | grep -cE '^\s*ok [0-9]+ - sitePurge: (allowProxied lets a proxied request reach the token check|route accepts a proxied request with allowProxied and the right bearer)$' | grep -qx 2 || exit 1; done ) &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; go run ./cmd/summer docs:build --check</automated>
<fails_when>Non-zero exit. Possible causes: the shared files differ between the layers; a layer spec fails, for example a stale hash pin; one of the two new sitePurge specs does not print an "ok" line in a layer; or the docs checker or docs:build --check reports a problem.</fails_when>
</verify>
<acceptance_criteria>
- `grep -n 'allowProxied' ../vue-faq-layer/nuxt.config.ts ../vue-menu-layer/nuxt.config.ts ../vue-journal-layer/nuxt.config.ts` prints at least one line per file.
- `grep -n '"version": "0.3.0"' ../vue-faq-layer/package.json ../vue-menu-layer/package.json` prints one line per file and `grep -n '"version": "0.5.0"' ../vue-journal-layer/package.json` prints one line.
- `grep -n 'NUXT_SITE_CACHE_ALLOW_PROXIED' docs/services/caching.md` prints at least one line.
- `test -f ../vue-journal-layer/server/api/__journal__/cache/purge.post.ts` exits 0 (alias kept).
- `git -C ../vue-faq-layer tag --points-at HEAD`, `git -C ../vue-menu-layer tag --points-at HEAD` and `git -C ../vue-journal-layer tag --points-at HEAD` print nothing (no tag before Task 7).
</acceptance_criteria>
<done>The two halves of the purge feature agree: a remote https purge URL works once the site opts in, and a site that does not opt in refuses proxied requests as before.</done>
</task>
<task type="auto">
<name>Task 6: Unit tests for every fix, required by name in scripts/check-phase19.3.sh, the whole gate green with Docker, and the review disposition recorded</name>
<files>modules/surf/cors_profile_test.go, modules/surf/apidocs_test.go, modules/bouncer/cookie_test.go, modules/cooler/sitecache_test.go, scripts/check-phase19.3.sh, .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-REVIEW-DISPOSITION.md, .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md</files>
<read_first>scripts/check-phase19.3.sh (the lists, go_named, LAYER_REQUIRE), 19.3-06-PROGRESS.md (Tasks 1-4), `git log --oneline --grep='(19.3-06)'` (the Task 1-5 commits), modules/surf/cors.go, modules/surf/apidocs.go, modules/bouncer/cookie.go, modules/cooler/sitecache.go, modules/cooler/cooler.go (Handle.Close), modules/cooler/install_test.go (Install with a test config), modules/cooler/sitecache_test.go (the Postgres harness of TestSiteCacheAfterCommitOnly), 19.3-REVIEW-DISPOSITION.md, 19.3-VALIDATION.md (Per-Task Verification Map)</read_first>
<action>Unit tests, Docker-free unless noted, committed as `test(19.3-06): unit tests for the review fixes`:
- surf cors_profile_test.go:
- TestCORSCredentialedOriginRules: a table over http.cors and http.cors_profiles.spa with credentials (per D-04).
- Accepted: https://www.example.com, https://*.example.com, https://*.api.example.com:8443, ^https://[a-z]+\.example\.org$.
- Refused: https://*, *://*, https://*example.com, https://*.com, http*://www.example.com, the unanchored https://.*\.example\.com, ^https://.*\.example\.com without $, ^https?://.*$ and ^.*$.
- Each refusal names the key with its index, and every refused entry still loads without credentials.
- TestCORSAnchoredPatternNoSuffixMatch: a credentialed profile with ^https://[a-z]+\.example\.com$ answers https://www.example.com with the exact origin, and https://www.example.com.attacker.net without Allow-Origin but with Vary: Origin.
- surf apidocs_test.go, TestAPIDocsCSPNoHostSource (per D-17): parse the Content-Security-Policy into directives.
- script-src holds exactly one source, 'nonce-n', and object-src 'none', base-uri 'none' and frame-ancestors 'none' are present.
- Every script tag in the body carries nonce n, and two requests get different nonces.
- The three integrity values are unchanged.
- bouncer cookie_test.go (per D-03):
- TestLoadCookieConfigProductionInferredSecure, a table: production with app.url http, unset or malformed gives an error naming auth.cookie.secure and app.url; production with https gives Secure; production with an http app.url and auth.cookie.secure true gives Secure; development with http gives Secure false and no error.
- TestZeroCookieConfigDefaults: the zero config's TokenCookie and ForgetTokenCookie carry token, /, HttpOnly, SameSite Lax and Secure; {Name: sid} gets SameSite Lax and stays not Secure; a CookieRefreshHandler and a CookieLogoutHandler built from the zero config write Set-Cookie lines containing SameSite=Lax and Secure.
- cooler sitecache_test.go (per D-10):
- TestSiteCachePurgeWaitAndClose:
- Wait on a nil purger and on an idle one returns nil at once.
- With a blocking receiver, Wait under an expired context returns context.DeadlineExceeded, and the queued purge is still sent after the release.
- Install with the site_cache keys (memory drivers, no Redis), then one queued purge, then Handle.Close: Close returns after the receiver got the request, and returns within SiteCachePurgeTimeout plus a second when the receiver hangs.
- TestSiteCacheBulkCommitCoalesced (Postgres harness, skips under -short), with a blocking receiver, in two scenarios:
- 25 purges registered inside one lagoon.Transaction;
- 10 separate transactions in a loop.
Each Transaction returns before the receiver is released, in under 500 ms. After the release and Wait, the receiver got exactly two requests per scenario, the second carrying the last slug.
Gate, one commit `test(19.3-06): gate requires the review-fix tests`:
- append to the lists:
- BOUNCER_TESTS: TestLoadCookieConfigProductionInferredSecure, TestZeroCookieConfigDefaults;
- SURF_TESTS: TestCORSCredentialedOriginRules, TestCORSAnchoredPatternNoSuffixMatch, TestAPIDocsCSPNoHostSource, TestTurnstileEmptySecretWarnsAtBoot;
- ATTACH_TESTS: TestStoreProtectedNeedsPrivateBucket (it then runs in both tag modes);
- COOLER_TESTS: TestSiteCachePurgeNoRedirect, TestSiteCachePurgeAsyncCoalesced, TestSiteCachePurgeWaitAndClose;
- DOCKER_COOLER_TESTS: TestSiteCacheBulkCommitCoalesced;
- LAYER_REQUIRE: the two sitePurge names from Task 5.
- Add one line to the header comment naming the plan-06 fixes (CR-01, WR-01..WR-09).
- Run `PHASE193_REQUIRE_DOCKER=1 bash scripts/check-phase19.3.sh --all` on a Docker host.
- Prove fail-closed: rename one new required test locally, check that the go stage exits non-zero, revert, and record the result in the SUMMARY.
Planning docs, a separate commit `docs(19.3-06): review disposition and validation map after the gap closure`:
- 19.3-REVIEW-DISPOSITION.md: set CR-01 and WR-01..WR-09 to fixed, both in the frontmatter findings and in the table. The Source cell names the fix commit(s), and for WR-01 and WR-03 also the plugin and layer commits. The open count becomes 11; IN-01..IN-11 stay open.
- 19.3-VALIDATION.md: the Per-Task Verification Map gains rows for plan 06 (CR-01 and each WR, with test names and gate stage).</action>
<verify>
<automated>PHASE193_REQUIRE_DOCKER=1 bash scripts/check-phase19.3.sh --all &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; (cd examples/hello &amp;&amp; go test ./...)</automated>
<fails_when>Non-zero exit, or "Phase 19.3 gate passed" absent. Possible causes: a stage refuses (a required name missing, skipped or red; the layer copies differ; the docs checker or the consuming-application scan fails); no Docker daemon answers (PHASE193_REQUIRE_DOCKER=1 turns NOT RUN into a failure); or the examples/hello workspace tests are red.</fails_when>
</verify>
<acceptance_criteria>
- `grep -c 'TestCORSCredentialedOriginRules\|TestLoadCookieConfigProductionInferredSecure\|TestZeroCookieConfigDefaults\|TestStoreProtectedNeedsPrivateBucket\|TestSiteCachePurgeAsyncCoalesced\|TestSiteCacheBulkCommitCoalesced\|TestAPIDocsCSPNoHostSource\|TestTurnstileEmptySecretWarnsAtBoot' scripts/check-phase19.3.sh` prints at least 8.
- `grep -c 'allowProxied' scripts/check-phase19.3.sh` prints at least 2.
- `grep -cE '^\| (CR-01|WR-0[1-9]) \| [a-z]+ \| fixed \|' .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-REVIEW-DISPOSITION.md` prints 10 and `grep -c '^open: 11$' .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-REVIEW-DISPOSITION.md` prints 1.
- The SUMMARY records that a locally renamed required test made `bash scripts/check-phase19.3.sh go` exit non-zero (reverted, never committed).
</acceptance_criteria>
<done>Every fix is pinned by a named unit test the gate cannot pass without, the whole phase gate is green with Docker, and the review record shows CR-01 and WR-01..WR-09 fixed.</done>
</task>
<task type="checkpoint:decision" gate="blocking-human">
<name>Task 7: Release gate: with the whole gate green, a human decides on tagging v0.1.28, bumping the plugin requires, tagging the layers and the pushes</name>
<read_first>.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-PLAN.md (Task 7) and 19.3-05-SUMMARY.md (release record), .planning/notes/core-plugins-own-repos.md, .planning/notes/2026-10-07-embedded-admin-release-gate.md, 19.3-06-PROGRESS.md (Tasks 1-4) and this plan's Task 5-6 notes</read_first>
<action>Do everything that can be automated, print a release sheet, then stop.
Checks before the sheet:
- In summercms.go, `git status --porcelain -- modules internal cmd scripts docs go.mod go.sum` prints nothing.
- `git fetch origin --tags`, then `git log --oneline HEAD..origin/master` prints nothing. Otherwise stop and report.
- Propose the highest `git tag -l 'v0.1.*' --sort=-v:refname` plus one (v0.1.28 at planning time) and confirm with `git ls-remote --tags origin` that it is free.
- Run the same fetch and divergence checks in sm-faq-plugin, sm-menu-plugin, sm-journal-plugin, vue-faq-layer, vue-menu-layer and vue-journal-layer, and confirm that v0.3.0 (faq and menu layers) and v0.5.0 (journal layer) are free on origin.
- Run the full gate (verify).
The sheet lists:
- the proposed tag;
- what it publishes:
- cors:&lt;profile&gt; applied outermost and the Vary merge;
- the credentialed CORS origin rules;
- the production cookie Secure rule and the zero-config defaults;
- the nonce-only api-docs CSP;
- attach.ErrProtectedNeedsPrivateBucket;
- the Turnstile boot warning;
- purges without redirects, queued and coalesced, and cooler.SiteCachePurger.Wait;
- the framework commits to push, including the planning commits since v0.1.27;
- the plugin test commits and the layer commits;
- that the plugin tests now call SiteCachePurger.Wait, so each plugin's go.mod must require the new tag before it builds on origin without go.work;
- that sites pick up siteCache.allowProxied only after their layer pins move (a follow-up, not part of this plan);
- whether the Docker stage ran.
No push and no tag before the reply. Never force-push, deploy nothing, and leave the app pointer and layer-pin bumps out of scope.
After "release":
- annotated tag on master; push master, then the tag;
- in each plugin: `GOWORK=off go get git.golem15.com/golem15/summercms@<tag>`, `GOWORK=off go mod tidy`, `GOWORK=off go vet ./...` and `GOWORK=off go test -count=1 ./...`; commit go.mod and go.sum as `chore(19.3-06): require summercms <tag>`; push main;
- push the three layer mains, then create and push annotated tags v0.3.0 (vue-faq-layer, vue-menu-layer) and v0.5.0 (vue-journal-layer).
After "tag-only": tag and push summercms only. After "hold": nothing.
Record the reply verbatim, with every pushed hash and tag.</action>
<decision>Tag and push the framework release v0.1.28, bump and push the plugins, push and tag the layers now?</decision>
<context>A pushed tag is read by the plugins' go.mod files, the Go module proxy and every site that pins a layer by tag. It cannot be moved, only superseded. Holding costs nothing locally, because apps and plugins build through replace and go.work, but the pushed plugins' tests then cannot build without a sibling checkout until the bump.</context>
<options>
<option id="release">
<name>Tag and push everything</name>
<pros>The CR-01 fix reaches every consumer; plugins and layers on origin build and test on their own.</pros>
<cons>A defect found in UAT needs another tag.</cons>
</option>
<option id="tag-only">
<name>Tag and push the framework only</name>
<pros>The fixed framework is published; plugin and layer publication waits for UAT.</pros>
<cons>The plugin test commits and the layer flag stay local; the plugins on origin keep requiring v0.1.27.</cons>
</option>
<option id="hold">
<name>Hold until UAT (/msd-verify-work 19.3)</name>
<pros>One publication after the manual checks (subdomain cookies, real S3, live purge, Swagger UI under its CSP).</pros>
<cons>Origin keeps the CR-01 defect in v0.1.27.</cons>
</option>
</options>
<resume-signal>Reply "release", "tag-only" or "hold".</resume-signal>
<verify>
<automated>bash scripts/check-phase19.3.sh --all &amp;&amp; go test -count=1 -run '^TestDocsTree$' ./cmd/summer &amp;&amp; (cd examples/hello &amp;&amp; go test ./...) &amp;&amp; echo release-gate-ok</automated>
<fails_when>Non-zero exit or "release-gate-ok" absent: a gate stage, the docs checker or the examples/hello workspace tests are red. The executor fixes the defect, records it as a deviation and never presents a red gate.</fails_when>
</verify>
<acceptance_criteria>
- The SUMMARY records the proposed tag and the reply verbatim. After "release" or "tag-only", `git ls-remote --tags origin "refs/tags/$RELEASE_TAG"` is non-empty and `git rev-parse master` equals `git rev-parse origin/master`.
- After "release", `grep -n "git.golem15.com/golem15/summercms $RELEASE_TAG" ../sm-faq-plugin/go.mod ../sm-menu-plugin/go.mod ../sm-journal-plugin/go.mod` prints one line per file. `git -C ../vue-faq-layer ls-remote --tags origin refs/tags/v0.3.0`, `git -C ../vue-menu-layer ls-remote --tags origin refs/tags/v0.3.0` and `git -C ../vue-journal-layer ls-remote --tags origin refs/tags/v0.5.0` each print one line.
- After "hold", `git ls-remote --tags origin "refs/tags/$RELEASE_TAG"` prints nothing.
</acceptance_criteria>
<done>The gap closure is either published with the plugins and layers pinned to it, or held, with a record of exactly what waits.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| browser on the SPA origin to a profiled API route | credentialed cross-origin requests whose answers the page must be able to read |
| config to boot | CORS origins, cookie Secure, public_url and the Turnstile secret, validated or reported before serving |
| Go app to the site's purge route | bearer-token POST, loopback or https through a proxy |
| CDN scripts to the api-docs page | third-party code executed in the app's origin |
| local checkouts to origin | tags and pushes that every consumer reads |
## STRIDE Threat Register
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|-----------|----------|-----------|----------|-------------|-----------------|
| T-19.3-33 | Tampering | profiled-route refusals without `Vary: Origin` or CORS headers (a shared cache serving one origin's answer to another; the SPA blind to its 401) | medium | mitigate | the profile is applied outermost in `Router.wrap`; `bufferedResponse.commit` merges `Vary`; TestCORSProfileHeadersOnRefusals and TestCORSProfileVaryKeptWithHandlerVary in the gate |
| T-19.3-34 | Information disclosure | credentialed CORS trusting arbitrary origins through `https://*`, `*://*` or unanchored patterns | high | mitigate | `validateCORSConfig` wildcard-label rule, anchoring rule and probe origins, globally and per profile; TestCORSCredentialedOriginRules, TestCORSAnchoredPatternNoSuffixMatch |
| T-19.3-35 | Information disclosure | purge bearer token forwarded on a redirect (https to http on the same host) | high | mitigate | `CheckRedirect` returns `http.ErrUseLastResponse`; TestSiteCachePurgeNoRedirect asserts the Location receives nothing |
| T-19.3-36 | Denial of service | synchronous per-row purges holding admin requests past server write timeouts | medium | mitigate | after-commit queue with one request in flight and coalescing; `Wait` and `Handle.Close` drain within the timeout plus a second; TestSiteCachePurgeAsyncCoalesced, TestSiteCacheBulkCommitCoalesced |
| T-19.3-37 | Spoofing | purge route with `allowProxied` reachable from the network | medium | mitigate | opt-in flag, default false; the token stays at least 32 characters and is compared in constant time over sha256 digests; caching.md documents that the token is then the only gate; sitePurge specs cover both flag states |
| T-19.3-38 | Information disclosure | auth token cookie without Secure in production through an http or unset `app.url` | high | mitigate | `LoadCookieConfig` boot error naming `auth.cookie.secure` and `app.url`; TestLoadCookieConfigProductionInferredSecure |
| T-19.3-39 | Information disclosure | hand-built zero `CookieConfig` writing a cookie without SameSite or Secure | medium | mitigate | `withDefaults` gives the zero config Secure and every unset SameSite Lax; TestZeroCookieConfigDefaults |
| T-19.3-40 | Elevation of privilege | api-docs CSP allowing any jsDelivr script (DOM XSS loading attacker code) | medium | mitigate | `script-src` holds only the per-request nonce; every script tag carries it; SRI kept; TestAPIDocsCSPNoHostSource |
| T-19.3-41 | Information disclosure | protected (`is_public` false) files in a bucket served publicly through `public_url` | high | mitigate | `attach.Store` returns `ErrProtectedNeedsPrivateBucket` before writing; OpenBucket warns at boot; TestStoreProtectedNeedsPrivateBucket |
| T-19.3-42 | Spoofing | `turnstile:*` routes unprotected by a missing secret, unnoticed | medium | mitigate | one boot warning with the route count and the production flag, never the secret; TestTurnstileEmptySecretWarnsAtBoot |
| T-19.3-43 | Tampering | the three layer copies of the purge route drifting apart | low | mitigate | byte-identical files checked with `cmp` in the layers stage, sha256 pins updated in each spec |
| T-19.3-44 | Tampering | publishing an unverified release, or a moved tag | medium | mitigate | Task 7 blocking checkpoint after the full gate; tags checked free on origin; no force-push |
| T-19.3-SC | Tampering | npm/pip/cargo or Go module installs | high | mitigate | this plan installs nothing: no new Go module, no npm package; the plugin `go get` at release only moves the framework require to the tag this plan creates |
</threat_model>
<verification>
- Task 1: the surf short suite, the five CORS tests, the guard_403, turnstile_422 and turnstile_503 subtests (every status routing.md names is tested), the gate's go stage with the new SURF_TESTS names, TestDocsTree.
- Task 2: the surf and bouncer short suites and seven regression and parity tests, TestDocsTree.
- Task 3: vet in both tag modes, the surf and lagoon short suites, six named tests, TestDocsTree.
- Task 4: the cooler suite under -race, five named purge tests, the gate's plugins and docker stages; the committed progress record naming Task 5 as next.
- Task 5: the gate's layers stage (cmp, then the existing specs), both new specs in every layer, TestDocsTree, docs:build --check.
- Task 6: `PHASE193_REQUIRE_DOCKER=1 bash scripts/check-phase19.3.sh --all`, TestDocsTree, the examples/hello workspace tests.
- Task 7: the full gate again before the human decision.
- Manual (19.3-VERIFICATION.md human items, unchanged by this plan): cookie refresh across real subdomains (now readable through a `cors:<profile>` route as well), a real S3 bucket, a live Nuxt purge (now queued), and Swagger UI under its now stricter CSP.
</verification>
<success_criteria>
- SC-1 / D-05 gap closed: every response of a `cors:<profile>` route carries the profile's headers, proven by a test the gate requires, with docs updated.
- WR-01..WR-09 fixed, each in its own commit with docs and a regression test, and recorded as fixed in 19.3-REVIEW-DISPOSITION.md.
- No breaking change to the core plugins; the non-credentialed `*` CORS answers are unchanged.
- The whole phase gate is green with Docker; the release is published only on the human's reply.
</success_criteria>
<output>
Create `.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-SUMMARY.md` when done, never before Task 7; the Tasks 1-4 part comes from 19.3-06-PROGRESS.md, which stays as written. Include:
- the commits per repository and per finding;
- the identifiers added (`attach.ErrProtectedNeedsPrivateBucket`, `cooler.SiteCachePurger.Wait`, runtimeConfig `siteCache.allowProxied`);
- what the admin shows for a refused protected upload;
- the new test names per gate stage, and the gate stages run or NOT RUN;
- the fail-closed check of Task 6;
- the Task 7 reply verbatim, with every pushed hash and tag;
- deviations;
- follow-ups: the app's plugin pointers and layer pins to the new versions; moving remote purge deployments to `NUXT_SITE_CACHE_ALLOW_PROXIED`; IN-01..IN-11 still open.
After the SUMMARY, update STATE through `msd_run query state.*` handlers and post a short g15office comment that phase 19.3's gap closure advanced (`.planning/G15OFFICE.md`).
</output>

View File

@@ -0,0 +1,69 @@
# 19.3-06 progress record after Task 4
Mid-plan record of plan 19.3-06 (gap closure CR-01, WR-01..WR-09), superseded by 19.3-06-SUMMARY.md once that file exists. It is not a SUMMARY: Tasks 5-7 are still to run.
next: Task 5
Recorded: 2026-10-15. Plan start: 2026-10-15T20:20:38Z. Ledger base (summercms.go HEAD before the plan): e9ad640.
## Completed Tasks
| Task | Name | Commit (repository, hash, subject) | Key files |
|------|------|------------------------------------|-----------|
| 1 | Tracer: cors:&lt;profile&gt; headers on every refusal (CR-01) | summercms.go 0723536 `fix(19.3-06): cors:<profile> headers on every response of the route (CR-01)` | modules/surf/router.go, modules/surf/cors.go, modules/surf/cors_profile_test.go, modules/surf/README.md, docs/services/routing.md, docs/architecture/request-lifecycle.md, scripts/check-phase19.3.sh |
| 2 | Credentialed CORS origin rules (WR-04) | summercms.go e32940d `fix(19.3-06): credentialed CORS refuses wildcard-host and unanchored origins (WR-04)` | modules/surf/cors.go, modules/surf/cors_profile_test.go, modules/surf/README.md, docs/services/routing.md |
| 2 | Production cookie Secure rule (WR-05) | summercms.go 4dae833 `fix(19.3-06): production refuses an inferred insecure token cookie (WR-05)` | modules/bouncer/cookie.go, modules/bouncer/cookie_test.go, modules/bouncer/README.md, docs/services/authentication.md |
| 2 | Zero CookieConfig defaults (WR-09) | summercms.go 85d8b6a `fix(19.3-06): zero CookieConfig writes SameSite=Lax and Secure (WR-09)` | modules/bouncer/cookie.go, modules/bouncer/cookie_test.go, modules/bouncer/README.md, docs/services/authentication.md |
| 3 | Nonce-only api-docs CSP (WR-06) | summercms.go f705343 `fix(19.3-06): api-docs CSP allows only nonce-carrying scripts (WR-06)` | modules/surf/apidocs.go, modules/surf/apidocs_test.go, modules/surf/README.md, docs/services/routing.md |
| 3 | Protected uploads refused with public_url (WR-07) | summercms.go caa5b5d `fix(19.3-06): refuse protected uploads into a public_url bucket (WR-07)` | modules/lagoon/attach/store.go, modules/lagoon/attach/bucket.go, modules/lagoon/attach/url_test.go, modules/lagoon/README.md, docs/services/storage.md |
| 3 | Turnstile boot warning (WR-08) | summercms.go d0a580c `fix(19.3-06): warn at boot when turnstile routes have no secret (WR-08)` | modules/surf/router.go, modules/surf/cors_profile_test.go, modules/surf/README.md, docs/services/authentication.md |
| 4 | Purge without redirects (WR-02) | summercms.go 50a584c `fix(19.3-06): site-cache purge never follows redirects (WR-02)` | modules/cooler/sitecache.go, modules/cooler/sitecache_test.go, modules/cooler/README.md, docs/services/caching.md |
| 4 | Queued, coalesced purges and Wait (WR-01) | summercms.go db6a7f5 `fix(19.3-06): queue site-cache purges after commit and coalesce them (WR-01)` | modules/cooler/sitecache.go, modules/cooler/cooler.go, modules/cooler/sitecache_test.go, modules/cooler/example_test.go, modules/cooler/README.md, docs/services/caching.md |
| 4 | FAQ plugin test waits (WR-01) | sm-faq-plugin e52b286 `test(19.3-06): wait for the queued site-cache purge before asserting (WR-01)` | site_cache_admin_test.go |
| 4 | Menu plugin test waits (WR-01) | sm-menu-plugin 4b42978 `test(19.3-06): wait for the queued site-cache purge before asserting (WR-01)` | site_cache_admin_test.go |
| 4 | Journal plugin test waits (WR-01) | sm-journal-plugin 4f633f0 `test(19.3-06): wait for the queued site-cache purge before asserting (WR-01)` | site_cache_admin_test.go, classes/site_cache_test.go |
## Heads after Task 4
| Repository | HEAD |
|------------|------|
| summercms.go | db6a7f5 (this record's commit follows it) |
| sm-faq-plugin | e52b286 (planning head fa90e83) |
| sm-menu-plugin | 4b42978 (planning head fe27b9a) |
| sm-journal-plugin | 4f633f0 (planning head a99e9a1) |
| vue-faq-layer | 96b6bc0 (planning-time head, v0.2.0) |
| vue-menu-layer | dd0ee35 (planning-time head, v0.2.0) |
| vue-journal-layer | 5936f66 (planning-time head, v0.4.0) |
Nothing pushed or tagged in any repository.
## Verify results
- Task 1: PASS. `go vet ./...`, surf short suite, the five named CORS tests (5 top-level PASS), guard_403, turnstile_422 and turnstile_503 (3 subtest PASS), `check-phase19.3.sh go` passed, TestDocsTree ok. Both new tests failed against the old router.go (checked by restoring it locally, then the new file back). Tracer gate: automated verify re-run green, expansion continued.
- Task 2: PASS. Surf and bouncer short suites, 7 top-level PASS, TestDocsTree ok.
- Task 3: PASS. Vet in both tag modes, surf and lagoon short suites, 6 top-level PASS, TestDocsTree ok.
- Task 4: PASS. `go test -short -race ./modules/cooler` ok, 5 top-level PASS, `check-phase19.3.sh plugins` passed, `check-phase19.3.sh docker` RUN and passed (Docker answered for user jin; framework cooler, cabana, attach suites and the FAQ, menu and Journal admin purge suites).
## Notes for the SUMMARY
Deviations:
- [Rule 2 - docs] Task 1 also updated docs/architecture/request-lifecycle.md ("The wrapping order" list gains the CORS profile step), since the list would otherwise be wrong after the change. Same commit as CR-01.
- [Rule 1 - test] Task 2 (WR-05): TestLoadCookieConfigDefaults had production rows with an http and an empty app.url expecting Secure false without error; they now run in development (the production case is an error by design and is covered in TestLoadCookieConfigErrors).
- [Rule 1 - test] Task 4 (WR-01): ExampleSiteCacheFromApp printed the purge from the receiver synchronously; it now calls `cooler.SiteCacheFromApp(app).Wait` after the save, and the `purge` doc comment of the example controller says the purge is queued. docs/services/caching.md's src= copy was refreshed with `summer docs:sync`.
- [Rule 1 - test] Task 4: "a slow receiver is abandoned and the save stays committed" now asserts the save returns within 500 ms (it no longer waits for the 2 s purge) and waits for the queue before reading the log.
Decisions taken while executing:
- CR-01: `Router.wrap` resolves the profile through `routeCORSProfile` (two profiles stay a boot error), skips the `cors:<profile>` entry in its loop after the unknown-profile check, and wraps the recovered handler in `profileMiddleware` just inside `recordPattern`. `bufferedResponse.commit` merges each comma-separated `Vary` value through `addVary`.
- WR-04: anchoring is checked on the parsed expression (`regexp/syntax`): every alternative must begin with `^` and end with `$`, so `^a|b$` is refused as unanchored. Wildcard entries must be `<literal scheme>://*.<rest>` with one `*`. Probe origins: https://cors-probe.invalid, http://cors-probe.invalid, https://cors-probe.com, null.
- WR-05: the explicit-false message stays; the inferred case says "auth.cookie.secure resolves to false in the production environment" and names app.url.
- WR-09: the zero-config Secure is decided before Name and Path are filled; any unset SameSite becomes Lax.
- WR-07: the check sits in `attach.Store` right after the argument checks, before the extension check and before the body is read.
- WR-08: the warning goes through `accessLogger(app)` (the published `*slog.Logger`, else `slog.Default`) with attributes `routes` and `production`.
- WR-01: `installSiteCache` returns the purger in effect, also one a plugin published before Install (the Journal alias keys), so `Handle.Close` drains whichever purger the app uses. Queued sends use a background context bounded by `SiteCachePurgeTimeout`. Log lines add `coalesced` only when above zero. `Purge` tolerates a nil client field (falls back to a no-redirect client).
- Plugin tests: `purgeReceiver` gains a `drain` hook set by `drainFrom(t, app)`; `take` (and so `expectOnePurge` and `expectNone`) waits on `cooler.SiteCacheFromApp(app).Wait` with a 5 s context first. In TestCategoriesAdminFrameworkKeysWin both receivers drain the app's purger.
What the admin shows for a refused protected upload (Task 3, WR-07): cabana's `writeFileError` has no branch for `attach.ErrProtectedNeedsPrivateBucket`, so it reaches the default path: `logFileFailure` logs the error (its message names storage.uploads.public_url) and the client gets the generic opaque 500 error body. cabana is unchanged, as planned.
Acceptance notes:
- The `git log --oneline --grep='(WR-0N)$'` checks of Tasks 2 and 3 print 4 and 5 lines rather than 3: commits of earlier phases (02) also end with `(WR-04)` and `(WR-07)`. The three 19.3-06 commits per task are present.
- `grep -n 'cooler.SiteCachePurger.Wait' modules/cooler/README.md` prints two lines (the Features bullet and the API row); Task 5's cooler README edit rewords the bullet so only the API row names it.

View File

@@ -0,0 +1,256 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
plan: 06
subsystem: infra
tags: [gap-closure, cors, cors-profiles, vary, bouncer, cookies, csp, api-docs, attach, public_url, turnstile, cooler, site-cache, purge, nuxt-layers, gate]
status: complete
requires:
- phase: 19.3-02
provides: "bouncer token cookies and Turnstile, surf CORS hardening, CORS profiles, api-docs"
- phase: 19.3-03
provides: "attach public_url and the S3 build tag"
- phase: 19.3-04
provides: "cooler site-cache purger, plugin purge helpers, the shared layer purge route"
- phase: 19.3-05
provides: "scripts/check-phase19.3.sh fail-closed gate, v0.1.27"
provides:
- "cors:<profile> applied outermost by Router.wrap: every response of a profiled route (guard 401/403, Turnstile 422/503, 429, 413, recovered 500) carries the profile's CORS headers; Vary merged (CR-01)"
- "credentialed CORS refuses wildcard-host and unanchored origins at boot, globally and per profile (WR-04)"
- "production refuses an inferred insecure token cookie; the zero CookieConfig writes SameSite=Lax and Secure (WR-05, WR-09)"
- "nonce-only script-src on /api-docs (WR-06)"
- "attach.ErrProtectedNeedsPrivateBucket: protected uploads refused while storage.uploads.public_url is set, plus a boot warning (WR-07)"
- "boot warning for turnstile routes without a secret (WR-08)"
- "site-cache purge client never follows redirects (WR-02); PurgeSiteCacheAfterCommit queues onto a coalescing per-purger worker; cooler.SiteCachePurger.Wait; Handle.Close drains (WR-01)"
- "siteCache.allowProxied (NUXT_SITE_CACHE_ALLOW_PROXIED) in vue-faq-layer 0.3.0, vue-menu-layer 0.3.0, vue-journal-layer 0.5.0 (local commits, untagged) (WR-03)"
- "11 new unit tests and 2 layer specs required by name in scripts/check-phase19.3.sh"
- "19.3-REVIEW-DISPOSITION.md: CR-01 and WR-01..WR-09 fixed, open 11 (IN-01..IN-11)"
affects: [phase 19.3 main session (docs update, v0.1.28 release), /msd-verify-work 19.3, sm-faq-plugin, sm-menu-plugin, sm-journal-plugin, vue-faq-layer, vue-menu-layer, vue-journal-layer, app layer pins]
actuals:
tokens: 49100
tasks: 6
commits: 14
plan_head_before: e9ad640999fee13dc5be8cd0cd96205e2f2d7cad
plan_head_after: 0c8109c6702b75beb62ba627b19d66223120c029
tech-stack:
added: []
patterns:
- "Route-level middleware that must decorate every response (CORS profile) is wrapped outside recovery, body limit and locale, not placed by its position in the middleware list"
- "Credentialed origin rules checked on the parsed regexp (regexp/syntax) plus probe origins outside the configured domains"
- "After-commit side effects only enqueue; a per-purger worker keeps one request in flight and coalesces the rest; Wait and Close drain"
key-files:
created: []
modified:
- modules/surf/router.go
- modules/surf/cors.go
- modules/surf/apidocs.go
- modules/surf/cors_profile_test.go
- modules/surf/apidocs_test.go
- modules/bouncer/cookie.go
- modules/bouncer/cookie_test.go
- modules/lagoon/attach/store.go
- modules/lagoon/attach/bucket.go
- modules/lagoon/attach/url_test.go
- modules/cooler/sitecache.go
- modules/cooler/cooler.go
- modules/cooler/sitecache_test.go
- modules/cooler/example_test.go
- scripts/check-phase19.3.sh
- modules/surf/README.md
- modules/bouncer/README.md
- modules/lagoon/README.md
- modules/cooler/README.md
- docs/services/routing.md
- docs/services/authentication.md
- docs/services/storage.md
- docs/services/caching.md
- docs/architecture/request-lifecycle.md
- ../sm-faq-plugin/site_cache_admin_test.go
- ../sm-menu-plugin/site_cache_admin_test.go
- ../sm-journal-plugin/site_cache_admin_test.go
- ../sm-journal-plugin/classes/site_cache_test.go
- ../vue-{faq,menu,journal}-layer/server/lib/siteCachePurge.ts
- ../vue-{faq,menu,journal}-layer/server/api/__site__/cache/purge.post.ts
- ../vue-{faq,menu,journal}-layer/tests/sitePurge.test.ts
- ../vue-{faq,menu,journal}-layer/nuxt.config.ts
- ../vue-{faq,menu,journal}-layer/README.md
- ../vue-{faq,menu,journal}-layer/package.json
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-REVIEW-DISPOSITION.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md
key-decisions:
- "Release of v0.1.28, the plugin require bumps and the layer tags handed off to the phase 19.3 main session (human reply at Task 7); this plan pushed and tagged nothing"
- "CR-01: Router.wrap resolves the profile via routeCORSProfile, skips the cors:<profile> entry in its loop and wraps the recovered handler in profileMiddleware just inside recordPattern; bufferedResponse.commit merges Vary through addVary"
- "WR-03: explicit opt-in (siteCache.allowProxied) instead of restricting purge_url to loopback, which would refuse https alias URLs Journal deployments may carry"
- "WR-01: installSiteCache returns the purger in effect (including one a plugin published before Install), so Handle.Close drains whichever purger the app uses"
patterns-established:
- "Outermost route decoration: a CORS profile is applied around recovery, so refusals and panics carry its headers"
- "Queued after-commit purges with coalescing and an explicit Wait for tests and shutdown"
requirements-completed: [SC-1, SC-3, SC-4, D-03, D-04, D-05, D-10, D-11, D-12, D-17, D-18]
duration: 40min
completed: 2026-10-15
---
# Phase 19.3 Plan 06: Review gap closure (CR-01, WR-01..WR-09) Summary
**cors:&lt;profile&gt; now decorates every response of a profiled route including guard, Turnstile, throttle, body-limit and panic refusals; credentialed CORS, cookie Secure, api-docs CSP, protected uploads and Turnstile secrets are checked at boot; site-cache purges are redirect-free, queued and coalesced, and the layers accept proxied purges on an explicit allowProxied opt-in. Release handed off to the phase 19.3 main session.**
## Performance
- **Duration:** about 40 min of execution (two executor contexts, split after Task 4)
- **Started:** 2026-10-15T20:20:38Z
- **Last task commit:** 2026-10-15T20:51:13Z
- **Completed:** 2026-10-15T21:00Z (SUMMARY)
- **Tasks:** 6 of 7 executed; Task 7 (release checkpoint) handed off
- **Files modified:** 24 in summercms.go (excluding .planning), 4 test files in three plugins, 6 files in each of three layers
## Accomplishments
- **CR-01 (verification gap closed):** a cross-origin SPA reads every refusal of a `cors:<profile>` route: 401/403 from a guard listed before the profile or set on the group, Turnstile 422/503, 429, 413 and a recovered 500 in JSON and raw groups all carry Allow-Origin, Allow-Credentials and `Vary: Origin`; a disallowed origin gets `Vary: Origin` only; a handler's own `Vary` is merged.
- **WR-04:** with `supports_credentials`, wildcard entries other than a whole leftmost host label, unanchored patterns and patterns matching probe origins outside the configured domains fail boot naming key and index, globally and per profile. Non-credentialed configs unchanged (`TestCORSOptionsMatchesLaravel` green).
- **WR-05 / WR-09:** production refuses a token cookie whose Secure resolves to false, set or inferred from an http/unset `app.url`; zero `CookieConfig` writes `token`, `Path=/`, HttpOnly, SameSite=Lax, Secure.
- **WR-06:** `/api-docs` sends `script-src 'nonce-…'` as the only source; every script tag carries the nonce; SRI unchanged.
- **WR-07:** `attach.Store` returns `attach.ErrProtectedNeedsPrivateBucket` before any blob is written when `is_public` is false and `storage.uploads.public_url` is set; `OpenBucket` warns at boot.
- **WR-08:** `BuildRouter` logs one warning naming the route count when turnstile routes exist and the secret is empty.
- **WR-02 / WR-01:** the purge client never follows a redirect; `PurgeSiteCacheAfterCommit` only enqueues after commit; one request in flight per purger, saves committed meanwhile collapse into one follow-up (bulk delete of N rows sends at most two); `cooler.SiteCachePurger.Wait`; `Handle.Close` drains. Exported cooler purge names, Journal aliases and the FAQ, menu and Journal controllers unchanged.
- **WR-03:** the shared layer purge route honours `siteCache.allowProxied` / `NUXT_SITE_CACHE_ALLOW_PROXIED`; the three shared files are byte-identical (sha256 `0ea9f0f9…193b260e`) with updated hash pins; `docs/services/caching.md` pairs an https `purge_url` on another host with the flag.
- **Gate:** 11 Go tests and 2 layer specs added to the required lists; the whole gate passes with Docker.
## Task Commits
| Task | Name | Commit (repository, hash) |
|------|------|---------------------------|
| 1 | Tracer: cors:&lt;profile&gt; headers on every response of the route (CR-01) | summercms.go `0723536` (fix) |
| 2 | Credentialed CORS refuses wildcard-host and unanchored origins (WR-04) | summercms.go `e32940d` (fix) |
| 2 | Production refuses an inferred insecure token cookie (WR-05) | summercms.go `4dae833` (fix) |
| 2 | Zero CookieConfig writes SameSite=Lax and Secure (WR-09) | summercms.go `85d8b6a` (fix) |
| 3 | api-docs CSP allows only nonce-carrying scripts (WR-06) | summercms.go `f705343` (fix) |
| 3 | Refuse protected uploads into a public_url bucket (WR-07) | summercms.go `caa5b5d` (fix) |
| 3 | Warn at boot when turnstile routes have no secret (WR-08) | summercms.go `d0a580c` (fix) |
| 4 | Site-cache purge never follows redirects (WR-02) | summercms.go `50a584c` (fix) |
| 4 | Queue site-cache purges after commit and coalesce them (WR-01) | summercms.go `db6a7f5` (fix) |
| 4 | Plugin tests wait for the queued purge (WR-01) | sm-faq-plugin `e52b286`, sm-menu-plugin `4b42978`, sm-journal-plugin `4f633f0` (test) |
| 4 | Progress record after Task 4 | summercms.go `3bb6d61` (docs) |
| 5 | Purge route accepts proxied requests on siteCache.allowProxied (WR-03) | vue-faq-layer `a229f6a` (0.3.0), vue-menu-layer `c4639c2` (0.3.0), vue-journal-layer `f5d7aac` (0.5.0) (feat) |
| 5 | Pair a remote purge URL with the site's allowProxied (WR-03) | summercms.go `4d4e379` (docs) |
| 6 | Unit tests for the review fixes | summercms.go `8d80580` (test) |
| 6 | Gate requires the review-fix tests | summercms.go `b2f4044` (test) |
| 6 | Review disposition and validation map after the gap closure | summercms.go `0c8109c` (docs) |
| 7 | Release gate | handed off, no commit (see below) |
summercms.go commits measured from the ledger: `git rev-list --count e9ad640..0c8109c` = 14 (12 code/docs plus 2 planning-docs commits).
## Task 7: handed off
The release checkpoint was reached with every pre-check green (see Verification). The human's reply, verbatim:
> "ok, differently. there is main agent that was doing 19.3 phase and he was waiting for gap closure to update /docs and make a release then. in other words, you can finalize your gap closure, write SUMMARY.md and leave him the rest - he will update docs for whole phase 19.3, create a tag and do a release."
Outcome: none of the three options (release / tag-only / hold) was executed by this plan. Proposed tag at the checkpoint: **v0.1.28**. Nothing was pushed or tagged in any repository; no plugin `go.mod` was changed.
## Handoff to the phase 19.3 main session
State at handoff:
- summercms.go `master` at `0c8109c` plus this plan's SUMMARY and tracking commits, 20+ commits ahead of `origin/master` (includes planning commits since v0.1.27). Highest tag v0.1.27; v0.1.28 free on origin.
- sm-faq-plugin, sm-menu-plugin, sm-journal-plugin: each 1 commit ahead of `origin/main` (the WR-01 test commits above).
- vue-faq-layer, vue-menu-layer, vue-journal-layer: each 1 commit ahead of `origin/main`; v0.3.0 / v0.3.0 / v0.5.0 free on origin.
What remains, in order:
1. Whole-phase docs update for 19.3 (`docs/`); this plan's docs changes are already committed (routing, authentication, storage, caching, request-lifecycle, the four module READMEs).
2. Annotated tag `v0.1.28` on summercms.go `master`; push `master`, then the tag.
3. In each plugin (sm-faq-plugin, sm-menu-plugin, sm-journal-plugin): `GOWORK=off go get git.golem15.com/golem15/summercms@v0.1.28`, `GOWORK=off go mod tidy`, `GOWORK=off go vet ./...`, `GOWORK=off go test -count=1 ./...`; commit go.mod and go.sum as `chore: require summercms v0.1.28` (plan wording: `chore(19.3-06): require summercms v0.1.28`); push main. Required: the plugin tests now call `cooler.SiteCachePurger.Wait`, so the plugins do not build on origin without go.work until this bump.
4. Push the three layer mains, then create and push annotated tags v0.3.0 (vue-faq-layer), v0.3.0 (vue-menu-layer), v0.5.0 (vue-journal-layer).
5. Follow-up, out of scope here: app layer pins (sites get `siteCache.allowProxied` only after their pins move) and plugin submodule pointer bumps in apps.
6. g15office phase comment (skipped here; to be posted with the release).
## Files Created/Modified
- `modules/surf/router.go`, `modules/surf/cors.go`: profile applied outermost, Vary merge, credentialed origin rules with probe origins, Turnstile boot warning.
- `modules/surf/apidocs.go`: nonce on every script tag, nonce-only script-src.
- `modules/bouncer/cookie.go`: production Secure rule, zero-config defaults.
- `modules/lagoon/attach/store.go`, `bucket.go`: `ErrProtectedNeedsPrivateBucket`, boot warning.
- `modules/cooler/sitecache.go`, `cooler.go`: no-redirect client, queue and coalescing worker, `Wait`, drain on `Handle.Close`.
- `scripts/check-phase19.3.sh`: CR-01 tests in SURF_TESTS, plan-06 tests in BOUNCER/SURF/ATTACH/COOLER/DOCKER_COOLER lists and LAYER_REQUIRE.
- Module READMEs and `docs/services/{routing,authentication,storage,caching}.md`, `docs/architecture/request-lifecycle.md`.
- Plugin `site_cache_admin_test.go` (three plugins) and `sm-journal-plugin/classes/site_cache_test.go`: wait for the queued purge.
- Layers: shared `siteCachePurge.ts` and `purge.post.ts`, specs with new hash pins, `nuxt.config.ts`, README, version bump.
## Decisions Made
- CR-01: `Router.wrap` resolves the profile through `routeCORSProfile` (two profiles stay a boot error), skips the `cors:<profile>` entry after the unknown-profile check and wraps the recovered handler in `profileMiddleware` just inside `recordPattern`; `bufferedResponse.commit` merges each comma-separated `Vary` value through `addVary`.
- WR-04: anchoring checked on the parsed expression (`regexp/syntax`): every alternative must begin with `^` and end with `$`, so `^a|b$` is refused. Wildcards must be `<literal scheme>://*.<rest>` with one `*`. Probe origins: https://cors-probe.invalid, http://cors-probe.invalid, https://cors-probe.com, null.
- WR-05: the explicit-false message stays; the inferred case says "auth.cookie.secure resolves to false in the production environment" and names app.url.
- WR-09: zero-config Secure is decided before Name and Path are filled; any unset SameSite becomes Lax.
- WR-07: the check sits in `attach.Store` right after the argument checks, before the extension check and before the body is read. In the admin, cabana's `writeFileError` has no branch for the new error, so it reaches the default path: `logFileFailure` logs the error (its message names `storage.uploads.public_url`) and the client gets the generic opaque 500. cabana unchanged, as planned.
- WR-08: the warning goes through `accessLogger(app)` with attributes `routes` and `production`.
- WR-01: `installSiteCache` returns the purger in effect, also one a plugin published before Install (Journal alias keys). Queued sends use a background context bounded by `SiteCachePurgeTimeout`. Log lines add `coalesced` only when above zero. `Purge` tolerates a nil client field.
- WR-03: explicit opt-in, default stays "refuse proxied"; accepts `true` or the string `"true"` from runtimeConfig.
- Plugin tests: `purgeReceiver` gains a `drain` hook set by `drainFrom(t, app)`; `take` (so `expectOnePurge` and `expectNone`) waits on `cooler.SiteCacheFromApp(app).Wait` with a 5 s context first. In TestCategoriesAdminFrameworkKeysWin both receivers drain the app's purger.
- Task 7: release handed off to the phase 19.3 main session.
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 2 - Docs] request-lifecycle wrapping order**
- **Found during:** Task 1
- **Issue:** `docs/architecture/request-lifecycle.md` "The wrapping order" list would be wrong after CR-01.
- **Fix:** the list gains the CORS profile step.
- **Commit:** `0723536`
**2. [Rule 1 - Test] Production cookie rows moved to development**
- **Found during:** Task 2 (WR-05)
- **Issue:** `TestLoadCookieConfigDefaults` had production rows with http and empty `app.url` expecting Secure false without error, now an error by design.
- **Fix:** those rows run in development; the production case is covered by `TestLoadCookieConfigErrors` and `TestLoadCookieConfigProductionInferredSecure`.
- **Commit:** `4dae833`
**3. [Rule 1 - Test] ExampleSiteCacheFromApp waits for the queued purge**
- **Found during:** Task 4 (WR-01)
- **Issue:** the example printed the purge synchronously from the receiver.
- **Fix:** it calls `cooler.SiteCacheFromApp(app).Wait` after the save; the example controller's `purge` comment says the purge is queued; `docs/services/caching.md` src= copy refreshed with `summer docs:sync`.
- **Commit:** `db6a7f5`
**4. [Rule 1 - Test] Slow-receiver subtest asserts the save is not blocked**
- **Found during:** Task 4
- **Issue:** "a slow receiver is abandoned and the save stays committed" waited for the 2 s purge.
- **Fix:** asserts the save returns within 500 ms and waits for the queue before reading the log.
- **Commit:** `db6a7f5`
**5. Task 7 not executed as specified**
- The checkpoint answer was none of release / tag-only / hold; the release moved to the phase 19.3 main session (see Handoff).
**Total deviations:** 4 auto-fixed (1 docs, 3 test), 1 scope handoff. No CLAUDE.md-driven adjustments.
## Verification
- Tasks 1-4: per-task verify PASS (details in 19.3-06-PROGRESS.md): vet in both tag modes, surf/bouncer/lagoon/cooler short suites, `go test -short -race ./modules/cooler`, `check-phase19.3.sh go`, `plugins` and `docker` (RUN). Both CR-01 tests failed against the old router.go. Tracer gate: automated verify re-run green before expansion.
- Task 5: `check-phase19.3.sh layers` passed, both new sitePurge specs `ok` in each layer, shared files byte-identical, TestDocsTree and `summer docs:build --check` ok.
- Task 6: `PHASE193_REQUIRE_DOCKER=1 bash scripts/check-phase19.3.sh --all` passed; TestDocsTree and `examples/hello` tests passed.
- **Fail-closed proof (Task 6):** renaming `TestZeroCookieConfigDefaults` locally made `bash scripts/check-phase19.3.sh go` exit 1 ("refuse: required tests did not pass: TestZeroCookieConfigDefaults"); reverted, never committed.
- Task 7 pre-checks: `bash scripts/check-phase19.3.sh --all` passed all stages including docker (RUN); TestDocsTree and examples/hello passed; v0.1.28 free on origin; v0.3.0 / v0.3.0 / v0.5.0 free on the layer origins.
Acceptance quirks:
- `git log --oneline --grep='(WR-04)$'` and `(WR-07)$` print more than three lines: phase-02 commits end with the same suffixes. The three 19.3-06 commits per task are present.
- The cooler README Features bullet was reworded in `4d4e379` so `cooler.SiteCachePurger.Wait` appears only in the API row.
## Issues Encountered
None blocking. Info findings IN-01..IN-11 left open, as planned.
## User Setup Required
Sites whose backend reaches the purge route through a TLS-terminating proxy set `NUXT_SITE_CACHE_ALLOW_PROXIED=true` after moving their layer pin (follow-up, see Handoff).
## Next Phase Readiness
Gap closure complete in code, docs and tests. The phase 19.3 main session finishes the whole-phase docs update and the release (Handoff, steps 1-6), then `/msd-verify-work 19.3`.
## Self-Check: PASSED
All 14 summercms.go commits, the 3 plugin commits and the 3 layer commits verified with `git cat-file -e`; plan_head_before/after measured from the ledger base e9ad640.

View File

@@ -0,0 +1,176 @@
# Phase 19.3: Framework transport and infrastructure for WavePath - Context
**Gathered:** 2026-10-15
**Status:** Ready for planning
<domain>
## Phase Boundary
The framework gains the transport and infrastructure pieces a multi-client, multi-instance application needs:
- JWT cookie transport for refresh across subdomains.
- Credentialed CORS.
- A cache module with memory and Redis drivers, plus a Redis-backed `surf.Store` so rate limits hold across instances.
- An S3 blob option selected through config.
- An opt-in `/api-docs` page and `/api-docs.json`.
- Cloudflare Turnstile verification.
Two todos are folded in: a framework-level site-cache purge service, adopted by the Journal, menu and FAQ plugins, and hardening of upload disk-name extensions.
Framework code never names WavePath or any other application (READMEs, docs and examples use neutral names). Repos touched: `summercms.go`. For the purge adoption only: `sm-journal-plugin`, `sm-menu-plugin`, `sm-faq-plugin` and the matching vue layers (`vue-journal-layer`, `vue-menu-layer`, `vue-faq-layer`).
Not in this phase: OIDC (19.4), social login (19.5), 2FA and device flow (19.6), and plugin-registrable token scopes (dropped, see the division note).
</domain>
<decisions>
## Implementation Decisions
### Cookie transport (bouncer)
- **D-01:** `bouncer` ships config-driven cookie builders (`TokenCookie` and `ForgetTokenCookie` or similar) and an optional ready-made cookie refresh/logout handler. The handler reads the token cookie, calls the existing `Refresh*` path (which already rotates the token and blacklists the old JTI) and sets the rotated cookie. The app supplies a writer func so it keeps its own response envelope. The shared `sm-user-plugin` is NOT changed to set cookies; apps opt in.
- **D-02:** Cookie attributes live in one framework block, `auth.cookie.*`: `name`, `domain`, `secure`, `same_site`, `path`. Cookie Max-Age comes from the guard's `refresh_ttl`, not its `ttl`: the cookie must outlive the access token, or refresh has no cookie to read (PHP comment in `AuthApiController::tokenCookie`). The existing per-guard `ttl`/`refresh_ttl` keys (`golem15.user.jwt.*`, `admin.jwt.*`) stay where they are. — **Reversibility:** costly — apps' config files and deploy docs reference these keys once they ship.
- **D-03:** Defaults when unset match PHP: name `token`, path `/`, httpOnly, `SameSite=Lax`, no Domain (host-only), Secure when the app URL is https, Max-Age = refresh window (20160 min default).
### Credentialed CORS (surf)
- **D-04:** Harden the existing path-scoped `http.cors` (`surf.CORSConfig.SupportsCredentials`):
- Refuse, or at least loudly reject at boot, credentials combined with a `*` origin.
- Always echo the exact origin, and set `Vary: Origin` on preflight too.
- Test the api ↔ www subdomain case.
- **D-05:** Also add a named CORS middleware (for example `cors:<profile>`) so a route group can carry its own CORS policy on top of the global path-scoped one.
### Cache and rate-limit store (new module `cooler`)
- **D-06:** New module **`cooler`** for the cache. The name comes from the research: `.planning/research/ARCHITECTURE.md` lists `cooler` for "in-process cache, rate-limit buckets", and `IDEA_LIB_NAMES.md` has `summer-cooler`. It gets a beach-name README and a row in the root table.
- **D-07:** Redis client is **go-redis v9** (`github.com/redis/go-redis/v9`). A decision note under `.planning/notes/` names it before code lands. The memory driver is otter v2 (research/go-ecosystem.md) or a stdlib map; the researcher picks one and the note records it. — **Reversibility:** costly — every Redis-touching module and app config builds on this client.
- **D-08:** The cache API is a Laravel-core subset: Get, Set, Has, Forget, Remember, Increment and Flush, with TTL and a per-app key prefix. No tags. An atomic lock only if a 19.x port is shown to need it.
- **D-09:** Selection uses one shared Redis block, `redis.url`, plus `cache.driver: memory|redis` and `http.rate_limit.store: memory|redis`. The framework wires everything at boot, so apps import nothing. `cooler` implements `surf.Store` for Redis (an atomic fixed-window script with the same semantics as `surf.MemoryStore`), so `surf` takes no Redis dependency.
### Site-cache purge (folded todo, in `cooler`)
- **D-10:** A framework purge service in `cooler`:
- One configured target: `site_cache.purge_url` and `site_cache.purge_token`.
- A loopback POST with a bearer token, sent after a committed admin save.
- Failures are logged and never fail the save.
- **D-11:** Plugin adoption is in this phase:
- Journal moves onto the service. Its current `golem15.journal.site_cache.*` keys stay as aliases for one release.
- The menu plugin purges after a menu or menu-entry save, reorder or delete. The FAQ plugin purges after a FAQ save or delete.
- The Nuxt purge route moves out of `vue-journal-layer` to a shared place, so a site without Journal still gets purges.
- Plugin repos are committed and pushed first; pointer bumps in apps are separate commits.
- Core plugins must not break: the alias keys keep existing Journal deployments working.
### S3 and uploads (lagoon/attach)
- **D-12:** New key `storage.uploads.public_url`. When set (a bucket or CDN base), `File` URLs are that base plus the partition path. When unset, files go through the existing `/storage/uploads` proxy route, which works for S3 too.
- **D-13:** The S3 driver (`gocloud.dev/blob/s3blob`) is opt-in through a **build tag**. `summer build` sets the tag from `summer.yaml`, so an app still never imports the driver itself. A `bucket_url: s3://…` in a binary built without the tag must fail at boot with an error that names the tag.
- **D-14:** S3-compatible endpoints (MinIO, R2, Hetzner) are configured through gocloud's native `bucket_url` query (`region`, `endpoint`, `use_path_style`). Credentials come from the standard AWS env vars. No parallel `s3.*` key set.
- **D-15:** Folded stored-XSS fix: `attach.Store` takes the disk-name extension from the sniffed MIME type, through one shared helper. The client name stays only in `file_name`. A regression test checks that a PNG uploaded as `x.html` gets a `.png` disk name and URL. The deploy docs gain a note on nosniff and a file-type allow-list. App call sites (fonoteka `storeUpload`/`StorePublicFile`) move to the helper in a follow-up, not in this phase.
### API docs (surf)
- **D-16:** The spec is supplied by an app or plugin: an embedded YAML or JSON file, converted to JSON once and cached. This matches PHP, which serves a hand-written YAML covering only the public surface. Nothing is generated from swag here.
- **D-17:** Routes are opt-in through config (`http.api_docs.enabled` plus the path), off by default. `/api-docs` serves a Swagger UI page from pinned jsDelivr `swagger-ui-dist@5` with SRI, like PHP. `/api-docs.json` serves the spec. Both are public and unauthenticated.
### Turnstile (bouncer)
- **D-18:** `bouncer` gets a Turnstile verifier, `Verify(ctx, token, ip, failClosed)`, with a 5 s timeout, plus a named middleware factory such as `turnstile:open|closed`. Semantics follow PHP `TurnstileVerifier`:
- Fail-open: an empty secret passes (dev no-op, the registration path).
- Fail-closed: 503 when unconfigured in production, 422 on a missing token, a siteverify false, an exception or a timeout (the guest entry path).
- Error bodies come from a writer func, so the app keeps the PHP shape. The secret is read from config, which maps the `TURNSTILE_*` env.
### Module placement
- **D-19:** `cooler` holds the cache, the Redis `surf.Store` adapter and the site-cache purge. `surf` holds `/api-docs` and the named CORS middleware. `bouncer` holds the cookie helpers and handler, and Turnstile. `lagoon/attach` holds the S3 tag, `public_url` and the extension fix.
### Claude's Discretion
- The exact Go identifiers, the handler signatures and the name of the named-CORS-middleware profile config.
- otter vs a stdlib map for the memory cache driver (recorded in the decision note).
- Whether the token cookie builder lives on a guard or as free functions.
- The exact build-tag name and how `summer.yaml` declares it.
### Folded Todos
- **Purge the Nuxt site cache on menu and FAQ admin saves** (`.planning/todos/pending/site-cache-purge-for-menu-and-faq.md`, g15office `zugdxorb`). Today the purge is Journal-specific, so menu and FAQ edits show up only after the 300 s SWR TTL. Folded as D-10 and D-11 with full plugin adoption.
- **Uploads keep the client file extension (stored XSS)** (`.planning/todos/pending/2026-10-11-uploads-keep-client-extension-stored-xss.md`, g15office `bemfi2zs`). Folded as D-15. Only the framework side is in scope; fonoteka call sites are a follow-up.
</decisions>
<canonical_refs>
## Canonical References
**Downstream agents MUST read these before planning or implementing.**
### Scope and division
- `.planning/ROADMAP.md` § Phase 19.3: goal and success criteria.
- `.planning/notes/wavepath-framework-asks-division.md`: which WavePath asks land in 19.3 and which were dropped (token scopes).
- `/media/nvme/dev/jz/wavepath.org/.planning/notes/summercms-framework-asks-for-wavepath.md`: the hand-off list, sections A3, A6 and A8.
### PHP reference behaviour
- `/media/nvme/dev/jz/wavepath.org/plugins/golem15/wavepath/controllers/api/AuthApiController.php`: `tokenCookie`, `forgetTokenCookie`, `refresh`, `logout` (cookie attributes, Max-Age = refresh window, rotation).
- `/media/nvme/dev/jz/wavepath.org/plugins/golem15/wavepath/classes/auth/TurnstileVerifier.php`: fail-open vs fail-closed semantics, 5 s timeout.
- `/media/nvme/dev/jz/wavepath.org/plugins/golem15/wavepath/controllers/ApiDocsController.php`: the `/api-docs` UI and `/api-docs.json` spec serving, pinned CDN.
- `/media/nvme/dev/jz/wavepath.org/config/session.php`: SESSION_DOMAIN and same_site.
### Research and dependency rules
- `.planning/research/ARCHITECTURE.md` (line ~73): the `cooler` module role.
- `.planning/research/go-ecosystem.md` (Cache row): otter and go-redis.
- `.planning/research/STACK.md`: the decided dependency set. go-redis needs a decision note.
- `../IDEA_LIB_NAMES.md`: the original `summer-cooler` name.
### Folded todos
- `.planning/todos/pending/site-cache-purge-for-menu-and-faq.md`
- `.planning/todos/pending/2026-10-11-uploads-keep-client-extension-stored-xss.md`
- `.planning/phases/12-p-ytarium-api-collections-and-albums/12-REVIEW.md` (WR-01): origin of the XSS finding.
### Repo conventions
- `CLAUDE.md`: doc rules (README and docs in the same commit), beach-module rules, sibling-repo commit order.
- `.planning/notes/core-plugins-own-repos.md`: plugin repo and submodule flow for the purge adoption.
</canonical_refs>
<code_context>
## Existing Code Insights
### Reusable Assets
- `modules/bouncer/refresh.go`: `Refresh`, `RefreshAudience` and `RefreshAudienceFor` already rotate the token and blacklist the old JTI with a grace window. The cookie handler wraps these.
- `modules/bouncer/jwt.go`: `NewJWTGuard` and `NewBackendJWTGuard` already read tokens from configured cookie names after the bearer token.
- `modules/bouncer/blacklist.go`: the `BlacklistStore` interface. A Redis implementation in `cooler` is a natural extra, but it is not required.
- `modules/surf/cors.go`: `CORSConfig` (Laravel `config/cors.php` keys, including `supports_credentials` and origin patterns) and `pathScopedCORS`.
- `modules/surf/limiter_store.go`: the `surf.Store` interface (`Attempt(key, max, decay)`) and `MemoryStore`. The Redis store must match its semantics.
- `modules/lagoon/attach/bucket.go`: `OpenBucket` and `normalizeBucketURL` on `gocloud.dev/blob` (file and mem blank-imported today). S3 is the `s3blob` driver behind a build tag.
- `modules/lagoon/attach/store.go`: `Store` already sniffs the content type, which is where the disk-name extension fix goes.
- `../sm-journal-plugin/classes/site_cache.go`: the existing Journal purge to generalise.
- `internal/build`: generates `main.go` and `plugins.gen.go` from `summer.yaml`, which is where build tags get set.
### Established Patterns
- Opt-in routes gated by config (`storage.uploads.serve` mounts the uploads route in `surf.BuildRouter`). `/api-docs` follows the same pattern.
- Named middleware and middleware factories (`RegisterMiddlewareFactory`, `name:param`). Turnstile and named CORS use them.
- Per-guard JWT config keys (`golem15.user.jwt.ttl/refresh_ttl`, `admin.jwt.*`).
- New dependencies need a decision note first. YAML goes through `goccy/go-yaml`.
### Integration Points
- Boot wiring: the framework reads `redis.url`, `cache.driver` and `http.rate_limit.store` and installs the `cooler` Redis store into surf's limiter before routes are built.
- `summer build` and `summer.yaml`: the S3 build tag.
- Plugins (Journal, menu, FAQ): call the purge service after committed admin saves.
</code_context>
<specifics>
## Specific Ideas
- The cookie must outlive the access token (Max-Age = `refresh_ttl`). The PHP comment explains the re-login loop that happens otherwise.
- Swagger UI is pinned to `swagger-ui-dist@5` on jsDelivr, as in PHP.
- Turnstile keeps the two PHP modes exactly: registration fails open, guest entry fails closed.
</specifics>
<deferred>
## Deferred Ideas
- Switching fonoteka.go's `storeUpload`/`StorePublicFile` to the new extension helper: a follow-up quick task in fonoteka.go.
- Cache tags and atomic locks: add them only when a ported plugin needs them.
- Generating `/api-docs` specs from swag annotations for frontend routes: later, if wanted.
- A Redis-backed `bouncer.BlacklistStore`: optional, not required by the criteria.
### Reviewed Todos (not folded)
- **Benchmark Płytarium on WinterCMS vs SummerCMS** (`2026-10-01-benchmark-...`): matched on keywords only; unrelated to 19.3.
- **Port the FAQ plugin as sm-faq-plugin**: already its own repo; only the purge wiring touches it here.
</deferred>
---
*Phase: 19.3-framework-transport-and-infrastructure-for-wavepath*
*Context gathered: 2026-10-15*

View File

@@ -0,0 +1,123 @@
# Phase 19.3: Framework transport and infrastructure for WavePath - Discussion Log
> **Audit trail only.** Do not use as input to planning, research, or execution agents.
> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.
**Date:** 2026-10-15
**Phase:** 19.3-framework-transport-and-infrastructure-for-wavepath
**Areas discussed:** Cookie refresh ownership, Cache + Redis store, S3 option & public URLs, api-docs & Turnstile, Module placement
---
## Todos
| Option | Description | Selected |
|--------|-------------|----------|
| Site cache purge (menu/FAQ) | Framework-level Nuxt purge service | ✓ |
| Uploads stored-XSS extension | Disk-name extension from sniffed MIME in lagoon/attach | ✓ |
| None | Keep todos separate | |
## Cookie refresh ownership
| Option | Description | Selected |
|--------|-------------|----------|
| Helpers + optional handler | Cookie builders plus a ready-made refresh/logout handler with a writer func | ✓ |
| Helpers only | Apps write their own handler | |
| Full handler in sm-user-plugin | Change the shared plugin's /refresh | |
| Option | Description | Selected |
|--------|-------------|----------|
| Framework `auth.cookie.*` | One block; Max-Age from the guard's refresh_ttl | ✓ |
| Per guard | `golem15.user.jwt.cookie.*`, `admin.jwt.cookie.*` | |
| Option | Description | Selected |
|--------|-------------|----------|
| Match PHP | token, /, httpOnly, Lax, host-only, Secure on https | ✓ |
| Stricter | Strict + always Secure | |
| Option | Description | Selected |
|--------|-------------|----------|
| Harden + test existing | Boot check, exact-origin echo, Vary, subdomain tests | |
| Add named middleware too | Hardening plus a per-group named CORS middleware | ✓ |
## Cache + Redis store
| Option | Description | Selected |
|--------|-------------|----------|
| go-redis v9 | Official, most widely used, already in research | ✓ |
| rueidis | Faster, smaller ecosystem | |
| Option | Description | Selected |
|--------|-------------|----------|
| Laravel-core subset | Get/Set/Has/Forget/Remember/Increment/Flush, no tags | ✓ |
| Include tags | Laravel cache tags | |
| Minimal KV | Get/Set/Delete only | |
| Option | Description | Selected |
|--------|-------------|----------|
| One redis block, auto-wired | redis.url + cache.driver + http.rate_limit.store | ✓ |
| App wires it | App builds client and sets the store | |
| Option | Description | Selected |
|--------|-------------|----------|
| Framework service only | Plugin adoption as follow-up | |
| Service + plugin adoption | Journal (alias keys), menu, FAQ and Nuxt route in this phase | ✓ |
## S3 option & public URLs
| Option | Description | Selected |
|--------|-------------|----------|
| Configurable base URL | `storage.uploads.public_url`, else proxy | ✓ |
| Always proxy | Stream everything through Go | |
| Presigned URLs | Direct and presigned | |
| Option | Description | Selected |
|--------|-------------|----------|
| Always linked | Blank import, +10–15 MB | |
| Build tag | Opt-in tag set by summer build from summer.yaml | ✓ |
| Option | Description | Selected |
|--------|-------------|----------|
| Via bucket_url query | gocloud native params, AWS env creds | ✓ |
| Explicit config keys | `storage.uploads.s3.*` | |
| Option | Description | Selected |
|--------|-------------|----------|
| Sniffed-MIME extension in attach | Framework helper + test; fonoteka sites follow-up | ✓ |
| Also fix fonoteka call sites | Include app changes in this phase | |
## api-docs & Turnstile
| Option | Description | Selected |
|--------|-------------|----------|
| App/plugin-supplied file | Embedded YAML/JSON, as PHP | ✓ |
| Generated from swag | Reuse the cabana pipeline | |
| Option | Description | Selected |
|--------|-------------|----------|
| Opt-in via config | `http.api_docs.enabled`, Swagger UI from pinned CDN | ✓ |
| Plugin capability | pact interface | |
| Embed Swagger UI | ~1.5 MB in binary | |
| Option | Description | Selected |
|--------|-------------|----------|
| Verifier + middleware factory | PHP fail-open/closed semantics | ✓ |
| Verifier only | Plugins call it in handlers | |
## Module placement
**User's choice:** First asked for a suggestion ("api-docs can go to surf, turnstile i guess too, or do we have a better module for that? what do you suggest for cache itself, do we have some earlier research around it maybe?"). Claude pointed to `cooler` in research/ARCHITECTURE.md and IDEA_LIB_NAMES.md and proposed: cooler (cache, Redis Store, purge), surf (api-docs), bouncer (Turnstile).
| Option | Description | Selected |
|--------|-------------|----------|
| Yes, as suggested | cooler / surf / bouncer | ✓ |
| Turnstile in surf instead | | |
| Purge in its own module | | |
## Claude's Discretion
- Go identifiers, handler signatures, the named CORS profile config, otter vs stdlib map, the build-tag name.
## Deferred Ideas
- fonoteka call sites onto the extension helper; cache tags and locks; swag-generated api-docs; a Redis BlacklistStore.

View File

@@ -0,0 +1,292 @@
# Phase 19.3: Framework transport and infrastructure for WavePath - Pattern Map
**Mapped:** 2026-10-15
**Files analyzed:** 27 (new or modified, across summercms.go and 6 sibling repos)
**Analogs found:** 25 / 27
All analog paths below are git-tracked source (framework paths are relative to `summercms.go/`; sibling repo paths are relative to `summercms/`).
## File Classification
| New/Modified File | Role | Data Flow | Closest Analog | Match Quality |
|---|---|---|---|---|
| `modules/cooler/README.md` | doc | - | `modules/postcard/README.md` | exact |
| `modules/cooler/cooler.go` (Cache, Install, FromApp) | service | request-response | `modules/postcard/mailer.go` (`Activate`, `driverFromApp`, `app.Publish[Mailer]`) | role-match |
| `modules/cooler/memory.go` (otter driver) | service | CRUD (in-proc) | `modules/surf/limiter_store.go` `MemoryStore` | role-match |
| `modules/cooler/redis.go` | service/config | request-response | `modules/postcard/drivers.go` (driver from config) | partial |
| `modules/cooler/ratelimit.go` (Redis `surf.Store`) | store | atomic counter | `modules/surf/limiter_store.go` | exact (semantics) |
| `modules/cooler/sitecache.go` | service | event-driven (after-commit HTTP) | `../sm-journal-plugin/classes/site_cache.go` | exact (move) |
| `modules/cooler/*_test.go` (miniredis + Docker Redis) | test | - | `modules/postcard/mailpit_test.go`, `modules/beachcomber/postgres_test.go` (TestMain) | exact |
| `modules/surf/serve.go` (cooler.Install + publish Store) | wiring | boot | same file, `publishUploads` (lines 51-56, 125-140) | exact |
| `modules/surf/router.go` (`BuildRouter` store lookup, `cors`/`turnstile` factories, OPTIONS routes) | router | request-response | same file lines 467-500 (`throttle` factory) | exact |
| `modules/surf/cors.go` (hardening + profiles) | middleware | request-response | same file | exact |
| `modules/surf/apidocs.go` | route/handler | request-response | `modules/surf/uploads.go` (`mountUploads`, opt-in route), `BucketProvider` type-assert in router.go 121-126 | role-match |
| `modules/bouncer/cookie.go` | utility + handler | request-response | `modules/bouncer/refresh.go`; secure check `modules/cabana/auth.go` 503-517 | role-match |
| `modules/bouncer/turnstile.go` | service + middleware | request-response (outbound HTTP) | `../sm-journal-plugin/classes/site_cache.go` (bounded outbound POST, error redaction) | partial |
| `modules/lagoon/attach/bucket.go` (s3 guard, `public_url`) | config | file-I/O | same file lines 37-80 | exact |
| `modules/lagoon/attach/s3_on.go` / `s3_off.go` | config (build tag) | - | blank imports in `bucket.go` lines 13-14 | partial |
| `modules/lagoon/attach/diskext.go` + `store.go` edit | utility | transform | `store.go` lines 130-160 | exact |
| `modules/lagoon/attach/file.go` (URL with public_url) | model | transform | `bucket.go` `PublicPathPrefix` lines 17-35 | exact |
| `internal/build/manifest.go` (`build.tags`) | config | file-I/O | same file `LoadManifest`/`SaveManifest` lines 40-103 | exact |
| `internal/build/build.go` (`-tags`) | utility | batch | same file line 49 | exact |
| `modules/pact/capabilities.go` (new `ReorderAfterSave` hook) | interface | event-driven | same file lines 748-765 (`FormAfter*`) | exact |
| `modules/cabana/reorder.go` (call hook in tx) | controller | CRUD | `modules/cabana/crud.go` `deleteRecord` 1025-1040 | exact |
| `../sm-journal-plugin/plugin.go` + `classes/site_cache.go` + `controllers/site_cache.go` | plugin wiring | event-driven | themselves (alias keys) | exact |
| `../sm-menu-plugin/controllers/menus.go`, `menuentries.go` | controller hooks | event-driven | `../sm-journal-plugin/controllers/categories.go` 64-88 | exact |
| `../sm-faq-plugin/controllers/faqs.go` | controller hooks | event-driven | same as above | exact |
| `../vue-{journal,menu,faq}-layer/server/api/__site__/cache/purge.post.ts` | route (Nitro) | request-response | `../vue-journal-layer/server/api/__journal__/cache/purge.post.ts` + `app/utils/siteCache.ts` | exact |
| `../vue-{journal,menu,faq}-layer/nuxt.config.ts` (runtimeConfig + routeRules) | config | - | `../vue-journal-layer/nuxt.config.ts` lines 27-51 | exact |
| `.planning/notes/<date>-cooler-redis-otter.md` (decision note) | doc | - | existing `.planning/notes/*` | role-match |
| `README.md` root table row, `docs/` pages | doc | - | `README.md` line 110 (postcard row) | exact |
## Pattern Assignments
### `modules/cooler/README.md` (doc)
**Analog:** `modules/postcard/README.md` lines 1-30. Structure to copy:
```markdown
# postcard
Transactional mail from plugin-owned Markdown templates ... SMTP driver.
`import "git.golem15.com/golem15/summercms/modules/postcard"`
**Name:** What you send home in summer; it delivers the application's mail.
## Overview
## Role in the framework
## Features
## Usage
```
Then API reference, Configuration (`redis.url`, `cache.driver`, `cache.prefix`, `cache.memory.max_entries`, `http.rate_limit.store`, `site_cache.purge_url`, `site_cache.purge_token`), Dependencies (go-redis v9, otter v2), Testing. Root `README.md` row format (line 110):
```markdown
| [postcard](modules/postcard/README.md) | <summary sentence> | <mnemonic> |
```
### `modules/cooler/cooler.go` (service, publish at boot)
**Analog:** `modules/postcard/mailer.go` lines 85-110
```go
func Activate[P interface{ ID() string }](app *backpack.App, _ []P) error {
if app == nil {
return fmt.Errorf("postcard: app is nil")
}
driver, opts, err := driverFromApp(app)
if err != nil {
return err
}
m := &mailer{ ... }
if err := app.Publish[Mailer](m); err != nil {
return fmt.Errorf("postcard: %w", err)
}
return nil
}
```
Copy: nil-app guard, `driverFromApp`-style config read (default `memory`), publish under the interface type (`app.Publish[cooler.Cache]`), errors prefixed `cooler:`. Never put `redis.url` in an error (RESEARCH Pitfall 7).
### `modules/cooler/ratelimit.go` (Redis `surf.Store`)
**Analog:** `modules/surf/limiter_store.go` lines 11-13 (interface) and 68-93 (semantics to match)
```go
type Store interface {
Attempt(key string, max int, decay time.Duration) (allowed bool, attempts int, retryAfter time.Duration)
}
...
if !ok {
e = &counterEntry{count: 0, resetAt: now.Add(decay)}
s.entries[key] = e
}
retryAfter := e.resetAt.Sub(now)
if retryAfter < 0 { retryAfter = 0 }
if e.count >= max { return false, e.count, retryAfter }
e.count++
return true, e.count, retryAfter
```
Implement structurally (do NOT import surf). Lua script from RESEARCH Pattern 2; fallback to a `MemoryStore` passed in by surf on Redis error. Keys `<prefix>rl:`.
### `modules/cooler/memory.go`
**Analog:** `MemoryStore` (same file above) for lifecycle (constructor + `Close`/stop). Use otter v2 per RESEARCH "Code Examples"; keys `<prefix>cache:`.
### `modules/cooler/sitecache.go` (move)
**Analog:** `../sm-journal-plugin/classes/site_cache.go` (whole file, ~185 lines). Copy verbatim, changing: key constants to `site_cache.purge_url` / `site_cache.purge_token`; log prefix `journal:` to `cooler:`; drop Journal-specific doc wording. Key pieces:
```go
const (
SiteCachePurgeTimeout = 2 * time.Second
siteCacheAnswerLimit = 4 << 10
)
const SiteCachePurgeMinTokenLength = 32
func NewSiteCachePurger(purgeURL, token string) (*SiteCachePurger, error) // nil,nil when both empty; loopback-only http
func (p *SiteCachePurger) Purge(ctx context.Context, kind, slug string) {
if p == nil { return }
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), SiteCachePurgeTimeout)
...
req.Header.Set("Authorization", "Bearer "+p.token)
...
log.Warn("journal: site cache purge failed", "kind", kind, "slug", slug, "error", purgeTransportError(err))
}
func PurgeSiteCacheAfterCommit(ctx context.Context, db *gorm.DB, p *SiteCachePurger, kind, slug string) {
if p == nil { return }
lagoon.AfterCommit(ctx, db, func(ctx context.Context, _ *gorm.DB) { p.Purge(ctx, kind, slug) })
}
```
Add a context-only helper (cabana tx lookup) modelled on `../sm-journal-plugin/controllers/site_cache.go`:
```go
purger, ok := a.Lookup[*classes.SiteCachePurger]()
if !ok || purger == nil { return nil, nil }
tx, ok := cabana.TxFromContext(ctx)
```
Note: cooler importing cabana would risk a cycle (cabana is heavy); prefer the helper taking `*gorm.DB` and let plugins call `cabana.TxFromContext` themselves, as Journal does. Add burst coalescing (RESEARCH Pattern 8).
### `modules/cooler/*_test.go`
**Docker analog:** `modules/postcard/mailpit_test.go` lines 35-60 (per-test `testcontainers.Run` with `testing.Short()` skip):
```go
if testing.Short() { t.Skip("requires testcontainers mailpit") }
ctr, err := testcontainers.Run(ctx, mailpitImage,
testcontainers.WithExposedPorts("1025/tcp", "8025/tcp"),
testcontainers.WithWaitStrategy(wait.ForListeningPort("1025/tcp"), ...),
)
t.Cleanup(func() { _ = testcontainers.TerminateContainer(ctr) })
```
Use `redis:7-alpine`, port `6379/tcp`. If a TestMain is used instead, copy `modules/beachcomber/postgres_test.go` lines 29-54 (`testShort()` with `flag.Parse()`). Fast loop: miniredis.
### `modules/surf/serve.go` (boot wiring)
**Analog:** same file lines 51-59:
```go
bucket, err := publishUploads(ctx, app)
if err != nil { return err }
defer bucket.Close()
r, err := BuildRouter(app, plugins)
```
Insert `cooler.Install` + `app.Publish[Store](...)` between `publishUploads` and `BuildRouter` (RESEARCH Pattern 1). `route:list` must not get this.
### `modules/surf/router.go`
**Analog:** `BuildRouter` lines 467-500:
```go
lim := NewFixedWindowLimiter(NewMemoryStore(2*time.Minute), trusted)
r.limiter = lim
if err := r.RegisterMiddlewareFactory("surf", "throttle", func(param string) pact.Middleware {
return lim.Middleware(param)
}); err != nil { return nil, err }
```
Replace store with `app.Lookup[Store]()` fallback; register `cors` and `turnstile` factories the same way. Optional providers: type-assert like `BucketProvider` (lines 121-126) for `APIDocsProvider`. `Router.compile` must add `OPTIONS` routes for chains containing `cors:<p>` (Pitfall 5).
### `modules/surf/cors.go`
**Analog:** itself. Edit points: `pathScopedCORS` pattern compile loop silently `continue`s on bad regex (make `LoadCORSConfig` return error); `Vary` only when `value != "*"` (set on every dynamic answer incl. preflight via `writeOptions`); `corsAllowOrigin` returns `"*"` under `allowAny` (boot error with credentials). Profile config reuses `CORSConfig` with `cfg.LoadSection("http.cors_profiles.<name>", &out)` as in `LoadCORSConfig`. Keep `TestCORSOptionsMatchesLaravel` (`cors_coverage_test.go` 99-147) green.
### `modules/surf/apidocs.go`
**Analog:** `modules/surf/uploads.go` (`mountUploads` via `GroupRaw`, ~148-170, gated by `storage.uploads.serve`) for opt-in mounting; config gate `http.api_docs.enabled`. YAML via `goccy/go-yaml` `YAMLToJSON` then `json.Compact` at boot; HTML via `html/template`.
### `modules/bouncer/cookie.go`
**Analogs:** `modules/bouncer/refresh.go` lines 12-50 (function signatures the handler wraps: `Refresh(secret, tokenString, refreshTTL, bl, grace, issuerURL)`, `RefreshAudienceFor(ctx, users, ...)`); production-secure refusal from `modules/cabana/auth.go` 503-517:
```go
if strings.EqualFold(strings.TrimSpace(app.Config.Environment()), "production") {
return false, errors.New("cabana: backend.cookie_secure: false is not allowed in the production environment")
}
```
Note bouncer imports nothing framework-side: `LoadCookieConfig` should take plain values (or a minimal interface) rather than `*compass.Config`/`*backpack.App`, or confirm compass is acceptable. Shape from RESEARCH Pattern 3.
### `modules/bouncer/turnstile.go`
**Analog:** outbound-POST hygiene from `site_cache.go` (`http.NewRequestWithContext`, `io.LimitReader`, `purgeTransportError` without URL). Factory registered in surf (`router.go`), not bouncer.
### `modules/lagoon/attach/bucket.go`, `s3_on.go`, `s3_off.go`
**Analog:** `bucket.go` lines 1-80. Blank-import style (lines 13-14) moves to the tagged file:
```go
_ "gocloud.dev/blob/fileblob"
_ "gocloud.dev/blob/memblob"
```
Insert the `s3Enabled` guard in `OpenBucket` after `normalizeBucketURL` (default branch passes `s3://` unchanged). `public_url` stored next to `publicPathPrefix` with the same `prefixMu`/getter pattern (lines 17-35). Error messages keep the `attach:` prefix and `SUMMER_...` env hint style.
### `modules/lagoon/attach/diskext.go` + `store.go`
**Analog:** `store.go` lines 141-151:
```go
contentType := baseMediaType(http.DetectContentType(head))
if contentType == "application/octet-stream" {
if byExt := baseMediaType(mime.TypeByExtension("." + ext)); byExt != "" { contentType = byExt }
}
...
diskName, err := newDiskName(ext)
```
Change to `newDiskName(DiskExtension(contentType, ext))`. Regression test in `store_test.go` with widened `Limits.Extensions`.
### `internal/build/manifest.go` / `build.go`
**Analog:** `manifest.go` 40-103: parse via `asMap/asSlice/asString` from `tree`, validate, append; `SaveManifest` writes with `fmt.Fprintf(&b, ...)` lines. Add `build.tags` in both (round-trip test). `build.go` line 49:
```go
cmd := exec.CommandContext(ctx, "go", "build", "-o", binRel, ".")
```
insert `"-tags=" + strings.Join(tags, ",")` when non-empty.
### `modules/pact/capabilities.go` + `modules/cabana/reorder.go`
**Analog:** pact lines 748-765:
```go
// FormAfterDelete optionally runs after delete, inside the same transaction.
type FormAfterDelete interface {
FormAfterDelete(ctx context.Context, model any) error
}
```
Call site pattern from `crud.go` 1035-1039:
```go
if hook, ok := cc.Controller.(pact.FormAfterDelete); ok && hook != nil {
if err := lifecycleFailure(cc, hook.FormAfterDelete(ctx, model)); err != nil { return err }
}
```
Place in `reorderPost` inside the `Transaction` closure after the node loop (reorder.go ~343), using `txCtx`. Check for in-flight cabana edits (phase 18.1).
### Plugin adoption: `../sm-journal-plugin`
- `plugin.go` `wireSiteCache` (lines 60-82): read framework keys first, else alias keys `classes.SiteCachePurgeURLKey`/`TokenKey` with `slog.Warn` deprecation; publish the cooler purger instead of `*classes.SiteCachePurger` (or have Install publish it and Journal only fill the alias case). Keep `classes` exported names as thin aliases/wrappers for one release (no breaking change).
- `controllers/site_cache.go`: `Lookup[*cooler.SiteCachePurger]`.
### `../sm-menu-plugin/controllers/menus.go`, `menuentries.go`, `../sm-faq-plugin/controllers/faqs.go`
**Analog:** `../sm-journal-plugin/controllers/categories.go` lines 64-88:
```go
func (c categoriesAdminController) FormAfterCreate(ctx context.Context, model any) error {
return c.purgeAfterCommit(ctx, model)
}
func (c categoriesAdminController) purgeAfterCommit(ctx context.Context, model any) error {
cat, ok := model.(*models.Category)
...
purger, tx := siteCachePurger(ctx, c.app)
classes.PurgeSiteCacheAfterCommit(ctx, tx, purger, "category", cat.Slug)
return nil
}
```
Caveat: faqs.go (lines 55-60) and menuentries.go (line 190+) already implement `FormAfterCreate/Update` for validation; append the purge call after the existing check rather than adding a second method. Add `FormAfterDelete` and the new reorder hook (menuentries). Controllers need an `app` accessor like Journal's `c.app` (menuentries has `registry()`/`pool()` helpers at 76-100 to model it on). Add compile-time asserts like faqs.go 73-74.
### `../vue-*-layer/server/api/__site__/cache/purge.post.ts`
**Analog:** `../vue-journal-layer/server/api/__journal__/cache/purge.post.ts` (whole file) and `app/utils/siteCache.ts` (`authorizePurge`, `parsePurgeEvent`, `purgeRouteCache`). Copy into each layer with `config.siteCache.purgeToken`; menu/faq layers need their own copy of the util (they have only `server/plugins/*-client-address.ts` today). Journal layer keeps the old route as alias.
### `../vue-*-layer/nuxt.config.ts`
**Analog:** `../vue-journal-layer/nuxt.config.ts` lines 27-51:
```ts
runtimeConfig: { journal: { apiBase: ..., cachePurgeToken: '' } },
routeRules: { '/api/__journal__/**': { cache: false } },
```
Add `siteCache: { purgeToken: '' }` and `'/api/__site__/**': { cache: false }` to all three.
## Shared Patterns
### Publish/lookup services on the app
**Source:** `modules/postcard/mailer.go` 100; `../sm-journal-plugin/controllers/site_cache.go` 24
`app.Publish[Iface](impl)` at boot; consumers `a.Lookup[T]()` with `ok && v != nil` check; nil service means disabled.
### Named middleware factories
**Source:** `modules/surf/router.go` 118-123, 478-482. Apply to `cors:<profile>`, `turnstile:open|closed`.
### Production-only refusals
**Source:** `modules/cabana/auth.go` 503-517 (`app.Config.Environment() == "production"`). Apply to cookie `secure: false`, Turnstile unconfigured closed mode.
### Secret-safe errors and logs
**Source:** `site_cache.go` `purgeTransportError` and "error names the key, never the token". Apply to redis.url, Turnstile secret, purge token.
### Docker-gated tests
**Source:** `modules/beachcomber/postgres_test.go` 29-54, `modules/postcard/mailpit_test.go` 35-60.
## No Analog Found
| File | Role | Data Flow | Reason |
|---|---|---|---|
| `modules/lagoon/attach/s3_on.go`/`s3_off.go` | build-tag split | - | No `//go:build` feature tags in the repo yet; use RESEARCH Pattern 5 |
| `modules/surf/apidocs.go` HTML shell | template | - | No CDN+SRI page exists; use RESEARCH Pattern 9 hashes (swagger-ui-dist 5.33.1) |
## Metadata
**Analog search scope:** `modules/{surf,bouncer,lagoon/attach,cabana,pact,postcard,beachcomber}`, `internal/build`, `../sm-journal-plugin`, `../sm-menu-plugin`, `../sm-faq-plugin`, `../vue-journal-layer`, `../vue-menu-layer`, `../vue-faq-layer`
**Files scanned:** ~30
**Pattern extraction date:** 2026-10-15

View File

@@ -0,0 +1,683 @@
# Phase 19.3: Framework transport and infrastructure for WavePath - Research
**Researched:** 2026-10-15
**Domain:** Go HTTP transport (cookies, CORS), Redis cache and distributed rate limiting, gocloud blob/S3, OpenAPI page, Cloudflare Turnstile, cross-repo site-cache purge
**Confidence:** HIGH for the in-repo seams and dependency versions; MEDIUM for the S3-compatible-provider and Nuxt-layer details
<user_constraints>
## User Constraints (from CONTEXT.md)
### Locked Decisions
#### Cookie transport (bouncer)
- **D-01:** `bouncer` ships config-driven cookie builders (`TokenCookie` and `ForgetTokenCookie` or similar) and an optional ready-made cookie refresh/logout handler. The handler reads the token cookie, calls the existing `Refresh*` path (which already rotates the token and blacklists the old JTI) and sets the rotated cookie. The app supplies a writer func so it keeps its own response envelope. The shared `sm-user-plugin` is NOT changed to set cookies; apps opt in.
- **D-02:** Cookie attributes live in one framework block, `auth.cookie.*`: `name`, `domain`, `secure`, `same_site`, `path`. Cookie Max-Age comes from the guard's `refresh_ttl`, not its `ttl`: the cookie must outlive the access token, or refresh has no cookie to read (PHP comment in `AuthApiController::tokenCookie`). The existing per-guard `ttl`/`refresh_ttl` keys (`golem15.user.jwt.*`, `admin.jwt.*`) stay where they are. — **Reversibility:** costly — apps' config files and deploy docs reference these keys once they ship.
- **D-03:** Defaults when unset match PHP: name `token`, path `/`, httpOnly, `SameSite=Lax`, no Domain (host-only), Secure when the app URL is https, Max-Age = refresh window (20160 min default).
#### Credentialed CORS (surf)
- **D-04:** Harden the existing path-scoped `http.cors` (`surf.CORSConfig.SupportsCredentials`):
- Refuse, or at least loudly reject at boot, credentials combined with a `*` origin.
- Always echo the exact origin, and set `Vary: Origin` on preflight too.
- Test the api ↔ www subdomain case.
- **D-05:** Also add a named CORS middleware (for example `cors:<profile>`) so a route group can carry its own CORS policy on top of the global path-scoped one.
#### Cache and rate-limit store (new module `cooler`)
- **D-06:** New module **`cooler`** for the cache. The name comes from the research: `.planning/research/ARCHITECTURE.md` lists `cooler` for "in-process cache, rate-limit buckets", and `IDEA_LIB_NAMES.md` has `summer-cooler`. It gets a beach-name README and a row in the root table.
- **D-07:** Redis client is **go-redis v9** (`github.com/redis/go-redis/v9`). A decision note under `.planning/notes/` names it before code lands. The memory driver is otter v2 (research/go-ecosystem.md) or a stdlib map; the researcher picks one and the note records it. — **Reversibility:** costly — every Redis-touching module and app config builds on this client.
- **D-08:** The cache API is a Laravel-core subset: Get, Set, Has, Forget, Remember, Increment and Flush, with TTL and a per-app key prefix. No tags. An atomic lock only if a 19.x port is shown to need it.
- **D-09:** Selection uses one shared Redis block, `redis.url`, plus `cache.driver: memory|redis` and `http.rate_limit.store: memory|redis`. The framework wires everything at boot, so apps import nothing. `cooler` implements `surf.Store` for Redis (an atomic fixed-window script with the same semantics as `surf.MemoryStore`), so `surf` takes no Redis dependency.
#### Site-cache purge (folded todo, in `cooler`)
- **D-10:** A framework purge service in `cooler`:
- One configured target: `site_cache.purge_url` and `site_cache.purge_token`.
- A loopback POST with a bearer token, sent after a committed admin save.
- Failures are logged and never fail the save.
- **D-11:** Plugin adoption is in this phase:
- Journal moves onto the service. Its current `golem15.journal.site_cache.*` keys stay as aliases for one release.
- The menu plugin purges after a menu or menu-entry save, reorder or delete. The FAQ plugin purges after a FAQ save or delete.
- The Nuxt purge route moves out of `vue-journal-layer` to a shared place, so a site without Journal still gets purges.
- Plugin repos are committed and pushed first; pointer bumps in apps are separate commits.
- Core plugins must not break: the alias keys keep existing Journal deployments working.
#### S3 and uploads (lagoon/attach)
- **D-12:** New key `storage.uploads.public_url`. When set (a bucket or CDN base), `File` URLs are that base plus the partition path. When unset, files go through the existing `/storage/uploads` proxy route, which works for S3 too.
- **D-13:** The S3 driver (`gocloud.dev/blob/s3blob`) is opt-in through a **build tag**. `summer build` sets the tag from `summer.yaml`, so an app still never imports the driver itself. A `bucket_url: s3://…` in a binary built without the tag must fail at boot with an error that names the tag.
- **D-14:** S3-compatible endpoints (MinIO, R2, Hetzner) are configured through gocloud's native `bucket_url` query (`region`, `endpoint`, `use_path_style`). Credentials come from the standard AWS env vars. No parallel `s3.*` key set.
- **D-15:** Folded stored-XSS fix: `attach.Store` takes the disk-name extension from the sniffed MIME type, through one shared helper. The client name stays only in `file_name`. A regression test checks that a PNG uploaded as `x.html` gets a `.png` disk name and URL. The deploy docs gain a note on nosniff and a file-type allow-list. App call sites (fonoteka `storeUpload`/`StorePublicFile`) move to the helper in a follow-up, not in this phase.
#### API docs (surf)
- **D-16:** The spec is supplied by an app or plugin: an embedded YAML or JSON file, converted to JSON once and cached. This matches PHP, which serves a hand-written YAML covering only the public surface. Nothing is generated from swag here.
- **D-17:** Routes are opt-in through config (`http.api_docs.enabled` plus the path), off by default. `/api-docs` serves a Swagger UI page from pinned jsDelivr `swagger-ui-dist@5` with SRI, like PHP. `/api-docs.json` serves the spec. Both are public and unauthenticated.
#### Turnstile (bouncer)
- **D-18:** `bouncer` gets a Turnstile verifier, `Verify(ctx, token, ip, failClosed)`, with a 5 s timeout, plus a named middleware factory such as `turnstile:open|closed`. Semantics follow PHP `TurnstileVerifier`:
- Fail-open: an empty secret passes (dev no-op, the registration path).
- Fail-closed: 503 when unconfigured in production, 422 on a missing token, a siteverify false, an exception or a timeout (the guest entry path).
- Error bodies come from a writer func, so the app keeps the PHP shape. The secret is read from config, which maps the `TURNSTILE_*` env.
#### Module placement
- **D-19:** `cooler` holds the cache, the Redis `surf.Store` adapter and the site-cache purge. `surf` holds `/api-docs` and the named CORS middleware. `bouncer` holds the cookie helpers and handler, and Turnstile. `lagoon/attach` holds the S3 tag, `public_url` and the extension fix.
### Claude's Discretion
- The exact Go identifiers, the handler signatures and the name of the named-CORS-middleware profile config.
- otter vs a stdlib map for the memory cache driver (recorded in the decision note).
- Whether the token cookie builder lives on a guard or as free functions.
- The exact build-tag name and how `summer.yaml` declares it.
### Deferred Ideas (OUT OF SCOPE)
- Switching fonoteka.go's `storeUpload`/`StorePublicFile` to the new extension helper: a follow-up quick task in fonoteka.go.
- Cache tags and atomic locks: add them only when a ported plugin needs them.
- Generating `/api-docs` specs from swag annotations for frontend routes: later, if wanted.
- A Redis-backed `bouncer.BlacklistStore`: optional, not required by the criteria.
Reviewed todos not folded: the WinterCMS-vs-SummerCMS benchmark, and porting the FAQ plugin (already its own repo).
</user_constraints>
<phase_requirements>
## Phase Requirements
No requirement IDs are mapped (ROADMAP: `**Requirements**: TBD`). The success criteria and decisions stand in for them:
| ID | Description | Research Support |
|----|-------------|------------------|
| SC-1 | bouncer refresh issues a configurable httpOnly `token` cookie and blacklists the old JTI; TTL/refresh window are config keys; credentialed CORS middleware | §Cookie transport, §CORS hardening, §Named CORS profiles; Pitfalls 1-4 |
| SC-2 | cooler with memory and Redis drivers; Redis-backed `surf.Store`; decision note first | §Standard Stack, §Redis fixed-window script (probed against Redis 7.2.5), §Boot wiring without import cycles |
| SC-3 | S3 via config (build tag), `/api-docs` + `/api-docs.json`, Turnstile middleware | §S3 build tag, §API docs, §Turnstile |
| SC-4 | READMEs and docs/ updated; unit tests in the last plan | §Docs checker constraints, §Validation Architecture |
| D-10/D-11 | Site-cache purge service and plugin/layer adoption | §Site-cache purge, Pitfall 9 (no reorder hook in cabana), Pitfall 10 (Nitro route cache) |
| D-15 | Disk-name extension from the sniffed type | §Extension helper (DetectContentType probe), Pitfall 8 |
</phase_requirements>
## Project Constraints (from CLAUDE.md)
- One Go module `git.golem15.com/golem15/summercms`; a new module (`cooler`) gets a beach name, a README in the standard structure (H1, one-sentence summary, import line, `**Name:**` mnemonic, Overview, Features, Usage, API reference, optional Configuration/CLI, Dependencies, Testing) and a row in the root `README.md` table whose purpose text reuses the summary sentence.
- **A new dependency needs a research doc or a decision note naming it** (`.planning/research/STACK.md` holds the decided set). go-redis, otter and (test-only) miniredis each need to be named in a note before code lands. `gocloud.dev/blob/s3blob` is a sub-package of the already-required `gocloud.dev v0.46.0`, but it pulls the AWS SDK v2 into `go.sum`; name it in the same note.
- YAML is parsed with `goccy/go-yaml`, never `gopkg.in/yaml.v3`.
- Standard library first.
- A change to a module's exported API, config keys, CLI commands or dependencies updates that module's README and the affected `docs/` pages **in the same commit**. Every identifier in a README or docs page must exist in the package (the docs checker enforces it); config keys are reviewed by hand.
- Framework code never names a consuming application (READMEs, docs, examples use neutral names such as `blog` or `acme`). The docs checker's forbidden pattern covers only two names (see §Docs checker); "WavePath" is not covered, so review by hand.
- Keep `go vet ./...` and `go test ./...` green at every commit. Docker tests skip under `-short`; a `TestMain` that starts containers calls `flag.Parse()` and checks `testing.Short()`.
- Lean planning: few, large plans. Present the plan count with one-line scopes and wait for confirmation before writing PLAN.md files.
- **Unit tests are the last plan of the phase.** Earlier plans may carry smoke tests.
- STATE.md updates go through `msd_run query state.*` handlers; g15office gets a comment when the phase advances.
- No co-author tags; one logical change per commit; planning docs and code in separate commits; stage files by explicit path; check for other sessions' in-flight edits before touching shared files such as cabana (phase 18.1 is executing in parallel).
- Sibling plugin repos: commit and push the plugin first, then the pointer bump in the app as a separate commit. Never stage files inside a submodule from the app repo. Don't push, tag or deploy without the user's go-ahead.
- User global rule: core plugins (user, blog/journal, pages, payment) must not get breaking changes unless asked. This is why D-11 keeps the Journal alias keys.
## Summary
Every piece of this phase lands on an existing, well-defined seam, so most of the research is about wiring rather than new technology. `surf.Store` is a three-value interface with no context and no error (`Attempt(key string, max int, decay time.Duration) (allowed bool, attempts int, retryAfter time.Duration)`), so a Redis implementation in `cooler` satisfies it structurally without importing `surf`. That lets `surf` import `cooler` (not the reverse) and keeps the dependency graph acyclic. The fixed-window Lua script below was run against the local Redis 7.2.5 and reproduces `MemoryStore` exactly: the first attempt opens the window, a denied attempt does not increment, and the window never extends.
The biggest traps are not in the libraries. First, `surf.BuildRouter` hard-codes `NewMemoryStore` and is also called by `route:list`, so the Redis store must be published before `serve` builds the router, and `route:list` must never dial Redis. Second, `cabana`'s reorder POST runs no controller hook today, so "menu purges after reorder" (D-11) needs a new optional `pact` hook. Third, route-level `cors:<profile>` middleware never sees a preflight, because the stdlib ServeMux answers `OPTIONS` on a `GET`-only pattern with 405 before any middleware runs; the router has to register `OPTIONS` routes for profiled routes. Fourth, every app's `scripts/build.sh` runs `summer build` and then a second, raw `go build` for linux/amd64, which drops any tag `summer build` set. The D-13 boot error is what catches that.
PHP parity notes that change the plan: the PHP `/api-docs` page does **not** use SRI and loads the floating `swagger-ui-dist@5`, so "with SRI, like PHP" requires pinning an exact version (5.33.1 is recommended; the hashes are below). PHP's php-cors v1.4.0 reflects **any** origin when `*` is combined with credentials, which is exactly what D-04 forbids, so Go diverges from PHP deliberately. Turnstile's PHP "fail-open" mode only skips verification when the secret is empty. With a secret set, both modes reject a `success:false`.
**Primary recommendation:** Build `cooler` first (decision note, then the cache with an otter v2 memory driver, the go-redis v9.23.0 driver, the Redis `surf.Store` and the `serve`-time wiring via `app.Lookup[surf.Store]`). Then the transport pieces in `bouncer` and `surf`, the attach S3/extension work, and the purge adoption across repos, with unit tests as the last plan.
## Architectural Responsibility Map
| Capability | Primary Tier | Secondary Tier | Rationale |
|------------|-------------|----------------|-----------|
| Token cookie build/clear, refresh/logout handler | API / Backend (`bouncer`) | Browser (cookie jar) | Cookie attributes are server-decided; the handler wraps `bouncer.Refresh*` |
| Credentialed CORS (global + profiles) | API / Backend (`surf`) | Browser (enforces) | Outermost HTTP wrapper; preflights answered before routing |
| Cache (memory/Redis) | API / Backend (`cooler`) | Database/Storage (Redis) | Shared state for multi-instance apps |
| Distributed rate limit | API / Backend (`surf` limiter) | Database/Storage (Redis via `cooler`) | `surf` keeps the policy; `cooler` owns the atomic counter |
| Site-cache purge | API / Backend (`cooler` service, plugin hooks) | Frontend Server (Nuxt/Nitro purge route) | After-commit POST from Go to the SSR server on loopback |
| S3 uploads, `public_url` | Database/Storage (`lagoon/attach`, bucket) | CDN / Static | Bucket choice is boot config; URL base may be a CDN |
| Disk-name extension (XSS) | API / Backend (`attach.Store`) | CDN / Static (nginx type mapping) | The extension decides the Content-Type a static server sends |
| `/api-docs` page and JSON | API / Backend (`surf`) | CDN (jsDelivr assets with SRI) | Server renders the shell; the browser loads pinned assets |
| Turnstile verification | API / Backend (`bouncer` verifier, `surf` factory) | External (Cloudflare siteverify) | Secret never leaves the server |
## Standard Stack
### Core
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| `github.com/redis/go-redis/v9` | **v9.23.0** (2026-10-05) [VERIFIED: proxy.golang.org @latest] | Redis client for `cooler` | Locked by D-07; official Redis client; `go 1.26.0` in its go.mod, fine for the project's Go 1.27 [VERIFIED: proxy.golang.org .mod] |
| `github.com/maypok86/otter/v2` | **v2.3.0** (2025-12-22) [VERIFIED: proxy.golang.org @latest] | Memory cache driver | Named in `.planning/research/ARCHITECTURE.md:73` (`cooler` row: `otter`) and `go-ecosystem.md:29`; recommendation reasoning below |
| `gocloud.dev/blob/s3blob` | part of `gocloud.dev v0.46.0` (already in go.mod) [VERIFIED: go.mod] | S3 driver behind a build tag | Locked by D-13/D-14; `Scheme = "s3"` registered in `init()` [VERIFIED: gocloud.dev@v0.46.0/blob/s3blob/s3blob.go:87-110] |
| `github.com/goccy/go-yaml` | v1.19.2 (already in go.mod) | YAML→JSON for the api-docs spec | Project rule; `yaml.YAMLToJSON` uses `UseOrderedMap()`, so key order survives [VERIFIED: goccy/go-yaml@v1.19.2/yaml.go:240-251] |
| stdlib `net/http` | Go 1.27 | Cookies (`http.Cookie`, `(*http.Cookie).Valid`), CORS, Turnstile client | Standard library first |
### Supporting (test only)
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| `github.com/alicebob/miniredis/v2` | v2.40.0 (2026-10-10) [VERIFIED: proxy.golang.org]; name from training [ASSUMED provenance] | In-process Redis for `-short` unit tests | Supports EVAL/EVALSHA/SCRIPT LOAD, PTTL, SET, INCR/INCRBY, SCAN, UNLINK, and `FastForward` for TTLs [VERIFIED: miniredis@v2.40.0/README.md:47-95,221-225,253]; one dep (`gopher-lua`) |
| `github.com/testcontainers/testcontainers-go` | v0.44.0 (already required) | Docker-gated real-Redis test | Use the generic `testcontainers.Run(ctx, "redis:7-alpine", …)` as `modules/postcard/mailpit_test.go:47-58` does. No `modules/redis` sub-module is needed |
### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| otter v2 | stdlib `map` + mutex + sweeper | ~100 lines, no dependency, but **unbounded**: a key-per-client cache (`Remember("user:"+id)`) grows forever; adding LRU bounds is a hand-rolled cache. otter gives a size bound (`MaximumSize`), per-entry TTL (`ExpiryCalculator`), atomic `Compute` for Increment and an injectable `Clock` for tests, with no runtime dependencies (go.mod requires only testify, for its own tests) [VERIFIED: otter@v2.3.0 go.mod, options.go:71-154, cache.go:157-224] |
| miniredis for unit tests | testcontainers Redis only | Real Redis is authoritative but needs Docker (skipped under `-short`); use both: miniredis for the fast loop, one Docker test that runs the same script on real Redis |
| Build tag for S3 (locked) | Generated blank import in `plugins.gen.go` | A generated import survives the apps' second raw `go build`; a tag does not (Pitfall 6). D-13 locks the tag, so keep the boot-time error as the safety net |
**Installation (framework module):**
```bash
go get github.com/redis/go-redis/v9@v9.23.0 github.com/maypok86/otter/v2@v2.3.0
go get github.com/alicebob/miniredis/v2@v2.40.0 # test-only, after its decision note
go mod tidy # also pulls aws-sdk-go-v2 into go.sum once the tagged s3blob import exists
```
**Memory-driver recommendation for the decision note (D-07 discretion): otter v2.** Reasons: bounded memory (DoS-safe when keys derive from request data), per-entry TTL including "forever" (otter's docs: "To indicate no expiration, an entry may be given an excessively long period" [VERIFIED: otter@v2.3.0/expiry_calculator.go:23-37]), lock-scoped `Compute` for an atomic Increment, a testable `Clock`, and the framework's own architecture research already names it.
## Package Legitimacy Audit
The `package-legitimacy` seam supports npm/pypi/crates only; Go modules were checked against proxy.golang.org (existence, publish time, VCS origin).
| Package | Registry | Age | Downloads | Source Repo | Verdict | Disposition |
|---------|----------|-----|-----------|-------------|---------|-------------|
| github.com/redis/go-redis/v9 v9.23.0 | proxy.golang.org | v9.23.0 10 days; module years | n/a (Go) | github.com/redis/go-redis (proxy Origin) | OK (manual) | Approved; named in CONTEXT D-07 and project research |
| github.com/maypok86/otter/v2 v2.3.0 | proxy.golang.org | ~10 months | n/a | github.com/maypok86/otter | OK (manual) | Approved; named in project research |
| github.com/alicebob/miniredis/v2 v2.40.0 | proxy.golang.org | 5 days (module years old) | n/a | github.com/alicebob/miniredis | OK (manual), name `[ASSUMED]` | Test-only; needs a decision note, and the planner adds `checkpoint:human-verify` before `go get` |
| swagger-ui-dist (CDN asset, not installed) | npm | 5.33.2: 3 days | 13.5M/wk | github.com/swagger-api/swagger-ui | **SUS** (`too-new`, for 5.33.2) | Pin **5.33.1** (published 2026-10-01) instead; jsDelivr bytes match the npm tarball (checked) |
**Packages removed due to [SLOP] verdict:** none. (The seam reports `SLOP` for `swagger-ui-dist@5.33.1` only because it treats `name@version` as a package name; the package-level check is `SUS: too-new`, which applies to the 3-day-old latest.)
**Packages flagged as suspicious [SUS]:** swagger-ui-dist latest (5.33.2). Avoided by pinning 5.33.1.
## Architecture Patterns
### System Architecture Diagram
```
┌──────────── summer serve (surf.ServeCommand) ───────────────┐
config (compass) ──► │ lagoon.Open → attach.OpenBucket(s3:// needs tag) → cooler. │
redis.url, cache.*, │ Install(app): parse redis.url, PING (timeout) → publish │
http.rate_limit.store │ cooler.Cache, *cooler.SiteCache, and (store=redis) │
│ surf.Store ← cooler.NewRateLimitStore(client, memFallback)│
│ BuildRouter: store := app.Lookup[surf.Store] else Memory │
└─────────────────────────────┬───────────────────────────────┘
▼
HTTP request ─► accessLog ─► stripLocale ─► pathScopedCORS (global http.cors; answers
OPTIONS on CORS paths, Vary: Origin) ─► ServeMux ─► per-route chain:
recordPattern → recover → locale → bodyLimit → [named mw: cors:<profile>,
turnstile:open|closed, throttle:… → surf.Store.Attempt ──► Redis EVALSHA
(fallback: per-instance MemoryStore on error)] → handler
│
cookie refresh: POST /…/auth/refresh ─► bouncer.CookieRefreshHandler: read
auth.cookie.name → app Refresh func (bouncer.Refresh*: rotate + blacklist JTI)
→ Set-Cookie (Max-Age = refresh_ttl) → app writer (PHP envelope)
│
admin save (cabana tx) ─► FormAfter{Create,Update,Delete} / new reorder hook ─►
cooler.PurgeAfterCommit ─► lagoon.AfterCommit ─► POST loopback
/api/__site__/cache/purge (Bearer) ─► Nitro: remove nitro:routes:* keys
│
GET /api-docs ─► HTML shell (jsDelivr swagger-ui-dist@5.33.1 + SRI) ─► GET /api-docs.json
─► spec bytes (provider YAML→JSON once, cached)
```
### Recommended Project Structure
```
modules/cooler/
├── README.md # beach name, summary, Configuration (redis.url, cache.*, site_cache.*)
├── cooler.go # Cache interface, Install(ctx, app), FromApp(app), key prefixing
├── memory.go # otter v2 driver ([]byte values, per-entry TTL)
├── redis.go # go-redis client from redis.url (no URL in errors)
├── ratelimit.go # Redis fixed-window Attempt (Lua) + memory fallback
├── sitecache.go # SiteCachePurger, PurgeAfterCommit (moved from the Journal plugin)
└── *_test.go # miniredis (-short) + Docker real Redis
modules/bouncer/cookie.go # CookieConfig, LoadCookieConfig, TokenCookie/ForgetTokenCookie, handlers
modules/bouncer/turnstile.go # Turnstile verifier (Verify) + Middleware(mode, ip, writer)
modules/surf/cors.go # hardening + profile middleware + OPTIONS registration
modules/surf/apidocs.go # APIDocsProvider, mount behind http.api_docs.enabled
modules/lagoon/attach/s3_on.go / s3_off.go # //go:build summercms_s3 / !summercms_s3
modules/lagoon/attach/diskext.go # DiskExtension(contentType, clientExt)
internal/build/manifest.go # summer.yaml build.tags (load + SaveManifest round-trip)
```
### Pattern 1: Boot wiring without an import cycle
**What:** `cooler` implements `surf.Store` structurally; `surf` imports `cooler`; `serve` publishes the store under the interface type before `BuildRouter`.
**Evidence:** `surf.Store` [VERIFIED: modules/surf/limiter_store.go:11-13]:
```go
type Store interface {
Attempt(key string, max int, decay time.Duration) (allowed bool, attempts int, retryAfter time.Duration)
}
```
`BuildRouter` today [VERIFIED: modules/surf/router.go:475-477]:
```go
// longest bucket decay is 1 minute; sweep at 2x
lim := NewFixedWindowLimiter(NewMemoryStore(2*time.Minute), trusted)
r.limiter = lim
```
`app.Publish[T]` keys the registry by `reflect.TypeFor[T]()` [VERIFIED: modules/backpack/services.go:22-48], so the store must be published as `app.Publish[surf.Store](…)`, which only `surf` (or `main`) can write. Import graph today: `lagoon` depends on compass, festival, backpack, bonfire, lagoon/attach, pact, towel, phrasebook, postcard, signpost and party (not surf), and `bouncer` depends on nothing in the framework [VERIFIED: `go list -deps`], so `surf → cooler → {lagoon, compass, backpack, go-redis}` is acyclic.
**Example:**
```go
// modules/surf/serve.go, inside ServeCommand Run, after publishUploads:
c, err := cooler.Install(ctx, app) // parses redis.url, pings when a redis driver is selected, publishes cooler.Cache
if err != nil { return err }
defer c.Close()
if c.RateLimitRedis() { // http.rate_limit.store == "redis"
if err := app.Publish[Store](c.RateLimitStore(NewMemoryStore(2 * time.Minute))); err != nil { return err }
}
r, err := BuildRouter(app, plugins)
// modules/surf/router.go, BuildRouter:
var store Store
if app != nil {
if s, ok := app.Lookup[Store](); ok && s != nil { store = s }
}
if store == nil { store = NewMemoryStore(2 * time.Minute) }
lim := NewFixedWindowLimiter(store, trusted)
```
`route:list` calls only `BuildRouter`, so it keeps the memory store and never touches Redis. Note `conga`'s dedicated worker command does not go through `serve`; a job that needs the cache should call `cooler.FromApp(app)`, which constructs lazily (go-redis `NewClient` does not dial until the first command).
### Pattern 2: Redis fixed-window script with MemoryStore semantics
**Semantics to match** [VERIFIED: modules/surf/limiter_store.go:68-93]: an expired or missing entry resets to `count: 0, resetAt: now.Add(decay)`; `retryAfter = resetAt - now` (floored at 0); `if e.count >= max { return false, e.count, retryAfter }`; otherwise `e.count++` and allow. Expiry is lazy and the window never extends.
```lua
-- KEYS[1] = <prefix>rl:<bucket key>; ARGV[1] = max; ARGV[2] = decay in ms (>= 1)
local max = tonumber(ARGV[1])
local decay = tonumber(ARGV[2])
local count = tonumber(redis.call('GET', KEYS[1]) or '0')
local ttl = redis.call('PTTL', KEYS[1])
if ttl < 0 then -- -2 missing, -1 no expiry: open a fresh window
count = 0
redis.call('SET', KEYS[1], 0, 'PX', decay)
ttl = decay
end
if count >= max then
return {0, count, ttl} -- denied: count unchanged
end
count = redis.call('INCR', KEYS[1]) -- INCR keeps the TTL
return {1, count, ttl}
```
Probe on the local Redis 7.2.5 with max=3, decay=2000 ms [VERIFIED: redis-cli --eval, this session]:
```
1 1 2000
1 2 1957
1 3 1952
0 3 1945 <- denied, count stays 3
TTL=1939 GET=3
1 1 2000 <- after 2.1 s: new window
```
Go side: `var attemptScript = redis.NewScript(src)`; `res, err := attemptScript.Run(ctx, client, []string{key}, max, ms).Int64Slice()` (EVALSHA with EVAL fallback) [VERIFIED: go-redis@v9.23.0/script.go:34,193 and command.go:887]. Use `context.WithTimeout(context.Background(), ~500ms)` because `Attempt` has no ctx. Clamp `decay.Milliseconds()` to at least 1, or `SET PX 0` errors. Redis uses its own clock, so instance clock skew does not matter.
### Pattern 3: Cookie transport in bouncer (D-01..D-03)
PHP reference [VERIFIED: wavepath.org …/AuthApiController.php:342-381]: `cookie(self::COOKIE_NAME, $token, $minutes = config('jwt.refresh_ttl', 20160), '/', domain, secure, true /*httpOnly*/, false, 'lax')`; forget = the same attributes, empty value and far-past expiry. Domain is `WAVEPATH_COOKIE_DOMAIN` or null; Secure is true in production or on a secure request [VERIFIED: …/CookieAttributes.php]. The `token` cookie is **unencrypted** (`config/cookie.php` `unencryptedCookies` lists `'token'`) [VERIFIED: wavepath.org config/cookie.php:17-22], so Go can read cookies the PHP app issued. User-plugin defaults: `ttl: 60`, `refresh_ttl: 20160`, `blacklist_grace: 0` [VERIFIED: fonoteka.go/plugins/golem15/user/config/config.yaml].
```go
// bouncer/cookie.go (identifiers are Claude's discretion)
type CookieConfig struct {
Name string // auth.cookie.name, default "token"
Domain string // auth.cookie.domain, default "" (host-only)
Path string // auth.cookie.path, default "/"
Secure bool // auth.cookie.secure, default: app.url scheme == https
SameSite http.SameSite // auth.cookie.same_site: lax|strict|none, default lax
}
func (c CookieConfig) TokenCookie(token string, refreshTTL time.Duration) *http.Cookie {
return &http.Cookie{Name: c.Name, Value: token, Path: c.Path, Domain: c.Domain,
MaxAge: int(refreshTTL / time.Second), Expires: time.Now().Add(refreshTTL),
HttpOnly: true, Secure: c.Secure, SameSite: c.SameSite}
}
func (c CookieConfig) ForgetTokenCookie() *http.Cookie { /* same attrs, Value "", MaxAge -1, Expires Unix(0) */ }
// Handler: app supplies Refresh (choose Refresh / RefreshAudienceFor) and the envelope writer.
type CookieRefreshOptions struct {
Cookie CookieConfig
RefreshTTL time.Duration
Refresh func(ctx context.Context, token string) (string, error)
Write func(w http.ResponseWriter, r *http.Request, status int, err error)
}
```
Validate at boot with `(*http.Cookie).Valid()`, which rejects an invalid Domain [VERIFIED: GOROOT net/http/cookie.go:349-353]. Reject `same_site: none` without Secure, and refuse `secure: false` in production, mirroring `adminCookieSecure` [VERIFIED: modules/cabana/auth.go:503-517]. Production is `app.Config.Environment() == "production"` (`SUMMER_ENV`) [VERIFIED: modules/compass/config.go:75-83].
### Pattern 4: CORS hardening (D-04) and profiles (D-05)
Current keys [VERIFIED: modules/surf/cors.go:13-22]: `paths`, `allowed_methods`, `allowed_origins`, `allowed_origins_patterns`, `allowed_headers`, `exposed_headers`, `max_age`, `supports_credentials`. Today `corsAllowOrigin` returns `"*"` when any origin is allowed, even with credentials [VERIFIED: cors.go:134-137]; `Vary: Origin` is only set when an origin is allowed and the value is not `*` [cors.go:71-75]; invalid `allowed_origins_patterns` are silently skipped [cors.go:44-51]; and wildcard entries in `allowed_origins` such as `https://*.example.com` are treated as literals. php-cors v1.4.0 converts them to patterns [VERIFIED: wavepath vendor/fruitcake/php-cors/src/CorsService.php:100-131].
PHP behaviour [VERIFIED: CorsService.php:204-220]: `*` without credentials sends `*`; a single listed origin sends that origin unconditionally; otherwise it echoes an allowed origin and **always** adds `Vary: Origin`; `*` **with** credentials reflects every origin. Recommended Go rules:
1. Boot error: `supports_credentials: true` with `*` in `allowed_origins` (global and every profile).
2. Boot error: an `allowed_origins_patterns` entry that does not compile. Also error on a PHP-delimited pattern such as `#^https://…$#`, since Go would compile it as literal `#` characters.
3. Translate `*` inside an `allowed_origins` entry into an anchored pattern (php-cors `convertWildcardToPattern`).
4. When the answer is dynamic (credentials, patterns or more than one origin), add `Vary: Origin` to **every** response on a CORS path, preflight and disallowed origin included. Keep the current `*` non-credentialed behaviour byte-identical: all four apps run `allowed_origins: ["*"]` without credentials, and `TestCORSOptionsMatchesLaravel` pins the preflight headers [VERIFIED: modules/surf/cors_coverage_test.go:99-147].
Profiles: config `http.cors_profiles.<name>` with the `CORSConfig` keys minus `paths`. Factory `cors` registered in `BuildRouter` like `throttle`. Because `ServeMux` answers `OPTIONS` on a `GET`/`POST` pattern with 405 before route middleware, `Router.compile` must register `OPTIONS <path>` for every route whose chain contains `cors:<p>`, with a handler that answers the preflight from the profile. Boot error when such a route's path also matches the global `http.cors.paths` (the global wrapper would answer the preflight first).
### Pattern 5: S3 build tag (D-13/D-14)
```go
// modules/lagoon/attach/s3_on.go
//go:build summercms_s3
package attach
import _ "gocloud.dev/blob/s3blob"
const s3Enabled = true
// modules/lagoon/attach/s3_off.go
//go:build !summercms_s3
package attach
const s3Enabled = false
// in OpenBucket, before blob.OpenBucket:
if strings.EqualFold(u.Scheme, "s3") && !s3Enabled {
return nil, fmt.Errorf("attach: storage.uploads.bucket_url uses s3:// but this binary was built without the summercms_s3 build tag; add it under build.tags in summer.yaml and rebuild (raw go build needs -tags summercms_s3)")
}
```
`normalizeBucketURL` already passes unknown schemes through unchanged (`default: return raw, nil`) [VERIFIED: modules/lagoon/attach/bucket.go:42-57]. `summer build` runs `exec.CommandContext(ctx, "go", "build", "-o", binRel, ".")` [VERIFIED: internal/build/build.go:49]; add `-tags=<joined>` from the manifest. Recommended `summer.yaml` shape (discretion):
```yaml
build:
tags:
- summercms_s3
```
Validate each tag against `^[a-z0-9_]+$`. `SaveManifest` (used by `plugin:add`) rewrites only `module`, `binary` and `plugins` [VERIFIED: internal/build/manifest.go:90-103], so it **must** round-trip `build.tags` or `summer plugin:add` silently drops S3. `dev` watch reuses `build.App` [VERIFIED: internal/dev/watch.go:45-46], so it inherits the tags.
S3 URL query (gocloud v0.46.0) [VERIFIED: gocloud.dev@v0.46.0/blob/s3blob/s3blob.go:118-136, aws/aws.go:70-180]: s3blob-specific `ssetype`, `kmskeyid`, `accelerate`, `use_path_style` (legacy `s3ForcePathStyle`) and `disable_https`; AWS config `region`, `endpoint`, `hostname_immutable`, `profile`, `role`, `anonymous`, `dualstack`, `fips`, `rate_limiter_capacity`, `request_checksum_calculation`, `response_checksum_validation` (`when_supported|when_required`). Unknown params are an error. Example for an S3-compatible endpoint:
```
s3://uploads?region=auto&endpoint=https://<account>.r2.cloudflarestorage.com&use_path_style=true&request_checksum_calculation=when_required&response_checksum_validation=when_required
```
### Pattern 6: Disk-name extension helper (D-15)
`Store` derives the disk name from the client extension: `diskName, err := newDiskName(ext)` [VERIFIED: modules/lagoon/attach/store.go:151] after sniffing `contentType := baseMediaType(http.DetectContentType(head))` [store.go:141]. The defaults already reject `x.html` (`DefaultFileExtensions` has no html, svg or xml [store.go:54-59]), so the regression test must widen `Limits.Extensions` (for example `[]string{"html", "png"}`). DetectContentType probe (Go 1.27) [VERIFIED: go run, this session]: png→`image/png`, docx/zip→`application/zip`, mkv→`video/webm`, mp3(ID3)→`audio/mpeg`, txt/ics/svg/`hello <script>`→`text/plain; charset=utf-8`, `<html>`→`text/html; charset=utf-8`, `<?xml`→`text/xml`, pdf→`application/pdf`, woff2→`font/woff2`, rar→`application/x-rar-compressed`, avif and short mp4 and OLE .doc→`application/octet-stream`. `mime.ExtensionsByType("image/jpeg")` returned `[.jfif .jpe .jpeg .jpg .pjp .pjpeg]` on this host: host-dependent, so do not use it.
Recommended rule:
```go
// DiskExtension picks the stored extension. clientExt is already lower-cased and allow-listed.
// 1. A specific sniffed type (table: image/png→png, image/jpeg→{jpg,jpeg}, image/gif, image/webp,
// application/pdf, audio/mpeg→mp3, video/webm→{webm,mkv}, font/woff2, application/x-rar-compressed→rar, …):
// keep clientExt when it belongs to that type's set, else the type's canonical extension.
// 2. A generic sniff (application/octet-stream, application/zip, text/plain, text/html, text/xml):
// keep clientExt unless it is script-capable (html, htm, xhtml, xht, shtml, svg, svgz, xml, xsl,
// xslt, js, mjs, css, swf, php, phtml): then "txt" for text/*, "bin" otherwise.
func DiskExtension(contentType, clientExt string) string
```
The allow-list check stays on the client extension, so accepted uploads do not change; only the stored name and URL change for mismatched uploads, as the todo expects.
### Pattern 7: Turnstile (D-18)
PHP [VERIFIED: wavepath.org …/TurnstileVerifier.php]: siteverify `https://challenges.cloudflare.com/turnstile/v0/siteverify`, form fields `secret`, `response`, `remoteip`. **Open mode:** an empty secret returns true; otherwise POST (no explicit timeout, exceptions not caught) and return `json('success') ?? false`. **Closed mode:** an empty secret returns `app.env !== 'production'`; an empty token returns false; POST with `timeout(5)` and `success === true`; any exception returns false. Controllers check `isConfiguredForGuests()` first and answer 503 `guest_unavailable`, then 422 `captcha_failed`; registration answers 422 `TURNSTILE_FAILED` [VERIFIED: GuestCallController.php:62-70, AuthApiController.php:68-72,182-189]. The token comes from the request field `turnstile_token` (`$request->get`). Cloudflare [CITED: developers.cloudflare.com/turnstile/get-started/server-side-validation/]: form or JSON accepted; tokens are at most 2048 chars, valid for 300 s and single-use (`timeout-or-duplicate`); optional `idempotency_key`. Test secrets [CITED: developers.cloudflare.com/turnstile/troubleshooting/testing/]: `1x0000000000000000000000000000000AA` passes, `2x0000000000000000000000000000000AA` fails, `3x0000000000000000000000000000000AA` returns timeout-or-duplicate, each with the dummy token `XXXX.DUMMY.TOKEN.XXXX`.
Design: `bouncer.Turnstile{Secret, Production bool, Endpoint (test override), Client *http.Client{Timeout: 5s}}` with `Verify(ctx, token, ip string, failClosed bool) (Verdict, error)`, so the middleware can tell "unconfigured" (503) from "failed" (422). Register the `turnstile` factory in `surf.BuildRouter` (it has `ClientIP` and trusted proxies; `bouncer` cannot import `surf`). The writer comes from `app.Lookup[bouncer.TurnstileWriter]()` with a neutral default JSON body. Config: `turnstile.secret` (env `SUMMER_TURNSTILE__SECRET`), optional `turnstile.field` (default `turnstile_token`). The middleware reads the field from a JSON or form body and **restores `r.Body`** (the body cap is already applied outside named middleware [VERIFIED: modules/surf/router.go:442-445]).
### Pattern 8: Site-cache purge (D-10/D-11)
Move `SiteCachePurger` from `sm-journal-plugin/classes/site_cache.go` into `cooler` nearly verbatim: URL/token validation (http only to loopback; token ≥ `SiteCachePurgeMinTokenLength = 32`), `SiteCachePurgeTimeout = 2 * time.Second`, `context.WithoutCancel`, a 4 KiB answer read, no token in logs, and `PurgeSiteCacheAfterCommit` via `lagoon.AfterCommit` [VERIFIED: sm-journal-plugin/classes/site_cache.go:21-48,114-184]. Framework keys `site_cache.purge_url` / `site_cache.purge_token`. Journal alias: in its `Register`, when the framework keys are unset and `golem15.journal.site_cache.purge_url`/`purge_token` are set [site_cache.go:24,27], build the cooler purger from the alias keys and log a deprecation warning. The framework never names the plugin's keys. Hooks: Journal already purges in `FormAfterCreate/Update/Delete` [VERIFIED: sm-journal-plugin/controllers/categories.go:64-88]; FAQ and menu add them. **Reorder has no hook** (Pitfall 9). Coalesce bursts (bulk delete of N rows means N after-commit purges) with a per-purger trailing debounce or an in-flight flag. (Superseded: the user chose no debounce, one POST per committed save; see Open Questions Q4.)
Nuxt side: Nitro dedupes scanned handlers by route+method+env and keeps the first [VERIFIED: vue-journal-layer node_modules nitropack@2.13.4 dist/core/index.mjs:894-901]. So the shared place can be **the same route file in each of vue-journal-layer, vue-menu-layer and vue-faq-layer** (`server/api/__site__/cache/purge.post.ts`, runtimeConfig `siteCache.purgeToken`, env `NUXT_SITE_CACHE_PURGE_TOKEN`), with no new repo needed. The Journal layer keeps `/api/__journal__/cache/purge` and `journal.cachePurgeToken` as aliases for one release. Each layer needs `routeRules: { '/api/__site__/**': { cache: false } }` (Pitfall 10). Sites consume layers as pnpm git deps pinned by tag (`"vue-journal-layer": "git+https://…/vue-journal-layer.git#v0.3.0"` [VERIFIED: sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/package.json:20-22]), so each layer needs a new tag plus a site bump.
### Pattern 9: API docs (D-16/D-17)
PHP serves `GET api-docs` (inline HTML) and `GET api-docs.json` (YAML parsed once, process-cached, `JSON_UNESCAPED_SLASHES`, 500 "API specification is unavailable." without leaking a path) [VERIFIED: wavepath.org …/ApiDocsController.php; routes.php:1086-1087]. PHP loads `https://cdn.jsdelivr.net/npm/swagger-ui-dist@5` **without** `integrity`. Go: a surf-defined optional interface (`surf.APIDocsProvider { APIDocsSpec() ([]byte, error) }`, type-asserted like `BucketProvider` [router.go:121-126]); boot error when `http.api_docs.enabled` is true and zero or more than one provider exists. Convert with `yaml.YAMLToJSON` then `json.Compact`, once at boot. Mount with `GroupRaw` like `mountUploads` [VERIFIED: modules/surf/uploads.go:148-170]. Config keys: `http.api_docs.enabled`, `http.api_docs.path` (default `/api-docs`; JSON at `<path>.json`), optional `http.api_docs.title`.
Pinned assets, swagger-ui-dist **5.33.1** [VERIFIED: npm pack tarball and jsDelivr bytes, sha384, this session]:
```
swagger-ui.css sha384-Ov4/wv3j2bmct8cDc5X4ngJZohVPzEmc6uDPH8WeljUxO5vtoykvMEfbu9Vh6RaW
swagger-ui-bundle.js sha384-ZPehFMQommnnuaZ4rpxgkgTT2DKFVp4hZC/7pLit+9Lek9T1YGSo23eHFbvNkXkw
swagger-ui-standalone-preset.js sha384-My2aDM4r2Mbm3ybHcubKm9O9U8FEjvF/O5nGvE9YK5dzqOTbWEKa79RPJ1krdMaF
```
(5.33.2 bundle is `sha384-gNnUp++d7WqR9OKX4OMoUmV2w0JxEs1uu6NPXgcynGClIX5DOu6ol4Lp+lINyZcJ`; its css and preset hashes equal 5.33.1's.) Use `<link … integrity="…" crossorigin="anonymous">` and `<script … integrity="…" crossorigin="anonymous">`, and pass the spec URL into `SwaggerUIBundle({url: …})` from config with `html/template` escaping.
### Anti-Patterns to Avoid
- **`FLUSHDB` for `Cache.Flush`** (Laravel's Redis store does this): it wipes rate-limit counters and every other app on the same DB. Use `SCAN MATCH <prefix>cache:*` plus `UNLINK` in batches.
- **Publishing the Redis store from `BuildRouter`**: `route:list` would dial Redis.
- **Shared key space for cache and limiter**: use `<prefix>cache:` and `<prefix>rl:`.
- **Route-level CORS without OPTIONS routes**: preflights get a 405.
- **`mime.ExtensionsByType` for disk names**: host-dependent output.
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Bounded TTL cache | map + LRU + sweeper | otter v2 | Eviction, per-entry expiry and lock-scoped compute are subtle |
| Redis protocol, pooling, scripts | raw RESP client | go-redis v9 `NewScript`/`Run` | EVALSHA/NOSCRIPT fallback, pooling, TLS (`rediss://`) |
| S3 signing and endpoints | AWS SigV4 client | gocloud `s3blob` (tagged) | Checksums and path style are already URL params |
| Cookie serialization | string building | `http.SetCookie` + `(*http.Cookie).Valid()` | Domain validation and attribute escaping |
| YAML→JSON | custom walker | `goccy/go-yaml` `YAMLToJSON` | Keeps key order (`UseOrderedMap`) |
| Atomic check-and-increment | GET then INCR from Go | single Lua script | Two round trips race across instances |
| Site-cache purge HTTP | a new client per plugin | the existing Journal purger moved to `cooler` | Already hardened (loopback-only http, token length, no token in logs) |
**Key insight:** in this phase every library choice is small; the risk is in seams (boot order, ServeMux preflights, manifest round-trips, a second `go build`).
## Common Pitfalls
### Pitfall 1: Go strips the leading dot of a cookie Domain and silently drops an invalid one
**What goes wrong:** `Domain: ".example.com"` is written as `Domain=example.com`; an invalid domain is logged and dropped, which makes a host-only cookie, so refresh breaks across subdomains without any error [VERIFIED: GOROOT net/http/cookie.go:277-292].
**How to avoid:** validate `auth.cookie.domain` at boot with `(*http.Cookie).Valid()`. Expect byte differences from PHP's `domain=.x` in parity fixtures (browsers treat both the same).
### Pitfall 2: Cookie Max-Age from `ttl` instead of `refresh_ttl`
**What goes wrong:** the cookie dies with the access token and the SPA loops on re-login (PHP comment, AuthApiController.php:344-349). **How to avoid:** the builder takes `refreshTTL`, and a test asserts `MaxAge == refresh_ttl*60`.
### Pitfall 3: CORS `*` + credentials (php-cors reflects any origin)
**How to avoid:** boot error (D-04). **Warning sign:** a config that "works" with credentials and `*` in PHP.
### Pitfall 4: Missing `Vary: Origin` on dynamic CORS answers
**What goes wrong:** a shared cache stores a response without `Access-Control-Allow-Origin` (or with another origin) and serves it to a different origin. **How to avoid:** always `Vary: Origin` on CORS paths when the origin answer is dynamic, including preflight (D-04).
### Pitfall 5: `cors:<profile>` never sees preflights
**What goes wrong:** ServeMux returns 405 for `OPTIONS` on `GET /x` before middleware [ASSUMED: Go 1.22+ ServeMux method-mismatch behaviour; confirm with a router test]. **How to avoid:** register `OPTIONS` routes for profiled routes, plus a boot check against overlap with global `http.cors.paths`.
### Pitfall 6: App build scripts drop the S3 tag
**What goes wrong:** `scripts/build.sh` runs `summer build`, then `CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go -C "$ROOT" build -trimpath -o bin/grzybyfunkcjonalne .` [VERIFIED: sm-grzybyfunkcjonalne-app/scripts/build.sh:52], with no tags. **How to avoid:** the D-13 boot error names the tag; the docs tell raw `go build` users to add `-tags summercms_s3` (or `GOFLAGS=-tags=summercms_s3`). Optionally `summer build` can print the tag line.
### Pitfall 7: Redis URL credentials leak through errors
**What goes wrong:** `url.Parse` errors echo the whole URL. Probe: `parse "redis://user:s3cretpw@host:6379/0\x7f": net/url: invalid control character in URL` [VERIFIED: go run, this session]. **How to avoid:** wrap as `cooler: redis.url is invalid` without `%w` of the raw error, or redact; never log the URL.
### Pitfall 8: Extension "fix" that breaks downloads
**What goes wrong:** blindly mapping the sniffed type renames `.docx`/`.xlsx` to `.zip` and `.ics` to `.txt`. **How to avoid:** apply rule 2 of Pattern 6 (keep a consistent or non-script client extension for generic sniffs).
### Pitfall 9: cabana reorder runs no controller hook
**What goes wrong:** `reorderPost` updates sort/parent columns with `UpdateColumns` inside a transaction and calls no `pact` hook [VERIFIED: modules/cabana/reorder.go:297-358], so menu-entry reorders cannot purge today. `deleteRecord` does run `pact.FormAfterDelete` (single and bulk) [VERIFIED: modules/cabana/crud.go:1025-1040, callers crud.go:274,328]. **How to avoid:** add an optional `pact` interface (for example `ReorderAfterSave(ctx context.Context) error`) called inside the reorder transaction (so `cabana.TxFromContext` works). That is a framework change in cabana/pact with README/docs updates; check for in-flight cabana edits first.
### Pitfall 10: The Nitro route cache swallows the purge route
**What goes wrong:** with a site rule `'/**': { swr }`, Nitro wraps every matching handler in its cached handler, so the purge answers from cache [VERIFIED: vue-journal-layer/nuxt.config.ts:45-50 comment and rule `'/api/__journal__/**': { cache: false }`]. **How to avoid:** every layer that ships the shared route also ships its `cache: false` rule.
### Pitfall 11: `SaveManifest` drops new summer.yaml keys
See Pattern 5. Add a round-trip test.
### Pitfall 12: The Turnstile middleware consumes the body
**How to avoid:** buffer (already capped), parse, restore `r.Body`. Also the PHP guest path validates `name` before Turnstile; a middleware runs before the handler, so the error order changes. Offer the verifier for in-handler use where parity on error order matters.
### Pitfall 13: Redis outage behaviour of the rate limiter
`Attempt` cannot return an error. **Recommend:** on a Redis error, fall back to a per-instance `MemoryStore` passed in by `surf`, and log once per interval. The alternatives are deny-all (an outage becomes a site-wide 429) or allow-all (the limit disappears).
### Pitfall 14: go-redis default handshakes against miniredis and older servers
go-redis v9.23.0 defaults `maintnotifications` to `ModeAuto` ("Client tries to send command, disables feature on error") and sends `CLIENT SETINFO` unless `DisableIdentity` is set [VERIFIED: go-redis@v9.23.0/maintnotifications/config.go:18-20,71,138; options.go:406-416,441-446]. **Recommend:** set `MaintNotificationsConfig: &maintnotifications.Config{Mode: maintnotifications.ModeDisabled}` and `DisableIdentity: true` in `cooler` for quiet logs.
### Pitfall 15: `public_url` bypasses the `is_public` check
**What goes wrong:** the proxy route serves only `is_public` rows (`StaticHandlerPublic`); a public-read bucket or CDN serves any key, private files included (keys are 22 random hex characters). **How to avoid:** document that `public_url` requires a bucket holding only public files, or accept unguessable keys knowingly. Optionally use `public_url` only when `IsPublic` is set.
## Code Examples
### Memory driver with per-entry TTL (otter v2)
```go
// Source: otter@v2.3.0 options.go / expiry_calculator.go / cache.go (verified API names)
type entry struct{ val []byte; ttl time.Duration } // ttl<=0 means forever
const forever = 50 * 365 * 24 * time.Hour
c, err := otter.New(&otter.Options[string, entry]{
MaximumSize: cfgMaxEntries, // cache.memory.max_entries, e.g. 10000
ExpiryCalculator: otter.ExpiryWritingFunc(func(e otter.Entry[string, entry]) time.Duration {
if e.Value.ttl <= 0 { return forever }
return e.Value.ttl
}),
})
// Increment: atomic per key; it keeps the existing TTL via entry.ttl, as Redis INCRBY does.
v, _ := c.Compute(key, func(old entry, found bool) (entry, otter.ComputeOp) {
n := int64(0)
if found { n, _ = strconv.ParseInt(string(old.val), 10, 64) }
return entry{val: []byte(strconv.FormatInt(n+by, 10)), ttl: old.ttl}, otter.WriteOp
})
```
(Check `ExpiryWritingFunc` update semantics during implementation: it recomputes on update, so carry `ttl` in the value as shown. Note `ttl` here is the original duration, so an Increment restarts the window; if exact Redis semantics matter, store an absolute `expiresAt` and return `time.Until(expiresAt)`.)
### Redis client from `redis.url`
```go
// Source: go-redis@v9.23.0 options.go:875-905 (ParseURL supports redis://, rediss://, unix://; unknown query params error)
opt, err := redis.ParseURL(raw)
if err != nil { return nil, errors.New("cooler: redis.url is not a valid redis:// rediss:// or unix:// URL") } // never echo raw
opt.DisableIdentity = true
opt.MaintNotificationsConfig = &maintnotifications.Config{Mode: maintnotifications.ModeDisabled}
client := redis.NewClient(opt)
pctx, cancel := context.WithTimeout(ctx, 3*time.Second); defer cancel()
if err := client.Ping(pctx).Err(); err != nil { return nil, fmt.Errorf("cooler: redis ping failed: %v", redactErr(err)) }
```
### Docker-gated real-Redis test (mirror the postcard pattern)
```go
func TestMain(m *testing.M) {
flag.Parse()
if testing.Short() { os.Exit(m.Run()) }
ctx := context.Background()
ctr, err := testcontainers.Run(ctx, "redis:7-alpine",
testcontainers.WithExposedPorts("6379/tcp"),
testcontainers.WithWaitStrategy(wait.ForListeningPort("6379/tcp")))
// … set redisURL, run, testcontainers.TerminateContainer(ctr)
}
```
## State of the Art
| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| Floating `swagger-ui-dist@5` CDN without integrity (PHP) | exact version + SRI | — | A floating version cannot carry SRI |
| AWS SDK default request checksums everywhere | `request_checksum_calculation=when_required` for S3-compatible stores | SDK v2 2025 defaults [ASSUMED date] | Some MinIO/R2/Hetzner setups reject the default CRC checksums [ASSUMED]; the URL params exist [VERIFIED] |
| Per-instance rate limits (`surf.MemoryStore`) | Redis fixed window shared by instances | this phase | `docs/services/rate-limiting.md` currently says "each application instance counts on its own" and must be updated |
**Deprecated/outdated:** `gopkg.in/yaml.v3` (archived, project rule); the go-redis `DisableIndentity` misspelling (deprecated, use `DisableIdentity`) [VERIFIED: options.go:406-416].
## Docs checker constraints
- `summer docs:build --check` / `go test ./cmd/summer -run TestDocsTree` checks identifiers, links, anchors, `src=` snippets and command names. Identifiers are indexed from the non-test Go files of every directory under `modules/`, keyed by the **last path element**. Two packages with the same last element are a problem [VERIFIED: internal/docsite/check_identifiers.go:20-60], so do not name a sub-package after an existing package (no `modules/cooler/redis` if anything else is `redis`; prefer files in `cooler`).
- Every module directory with Go files needs a `README.md`; READMEs are published in the `api` section automatically [VERIFIED: internal/docsite/load.go:318-344].
- The forbidden-name check is `regexp.MustCompile(`(?i)fonoteka|p[lł][yý]tarium`)` [VERIFIED: internal/docsite/check_forbidden.go:14]. WavePath is not covered, so review by hand (or extend the pattern in this phase).
- `go/parser` indexes build-tagged files regardless of tags, so identifiers in `s3_on.go` count.
- Pages to touch: `docs/services/rate-limiting.md` (Redis store), `docs/services/storage.md` (S3, `public_url`, extension, nosniff/allow-list deploy note), `docs/services/authentication.md` (cookie transport, Turnstile), `docs/services/routing.md` (CORS hardening, profiles, OPTIONS, api-docs), `docs/database/attachments.md` (disk name), a new `docs/services/caching.md`, and the `summer build` pages (`docs/console/*`) for `build.tags`.
## Assumptions Log
| # | Claim | Section | Risk if Wrong |
|---|-------|---------|---------------|
| A1 | ServeMux answers OPTIONS on a GET-only pattern with 405 before middleware | Pitfall 5 | Profile preflight design is wrong; a router test settles it in minutes |
| A2 | S3-compatible stores (R2/MinIO/Hetzner) may reject the SDK's default checksums; `when_required` fixes it | Pattern 5, State of the Art | Uploads to a provider fail; documented URL params mitigate |
| A3 | miniredis is a suitable test double (name from training; existence and features verified) | Standard Stack | Needs the decision note and user confirmation as a new test dependency |
| A4 | Nitro's first-wins dedupe follows layer priority (project first) | Pattern 8 | Harmless if the route files are identical; diverging copies would make the winner layer-order dependent |
| A5 | Laravel's cache prefix default is `slug(APP_NAME,'_').'_cache_'`; recommend `cache.prefix` defaulting from `app.name` | cooler config | Cosmetic; only matters for key layout docs |
| A6 | Symfony serializes PHP cookies as lower-case attributes (`path=/; domain=…; secure; httponly; samesite=lax`) | Pitfall 1 | Parity fixtures that compare Set-Cookie bytes need normalization |
| A7 | Fallback-to-memory is the preferred Redis-outage behaviour for rate limits | Pitfall 13 | Product choice; confirmed by the user (Open Questions Q3) |
## Open Questions (RESOLVED)
All five questions were answered by the user during planning (2026-10-15). The answers below are authoritative and the plans follow them.
1. **Shared Nuxt purge route: same file in three layers, or a new tiny layer repo?** (RESOLVED)
- What we know: Nitro dedupes duplicate routes (first wins), so three identical copies work without a new repo.
- What's unclear: whether the user prefers a new `vue-*-layer` repo (needs creating, and every site adds it to `extends`).
- Recommendation: three identical copies plus a test in each layer that compares the shared util file's hash.
- **RESOLVED:** the same route file is copied into `vue-journal-layer`, `vue-menu-layer` and `vue-faq-layer`, each with its own `'/api/__site__/**': { cache: false }` route rule. No new layer repo. (Plan 04.)
2. **Turnstile env name.** PHP reads `TURNSTILE_SECRET`; compass maps only `SUMMER_*`. Recommendation: config key `turnstile.secret` (env `SUMMER_TURNSTILE__SECRET`), with the deploy docs mapping the old name. (RESOLVED)
- **RESOLVED:** config key `turnstile.secret`, set through `SUMMER_TURNSTILE__SECRET`; the docs record the mapping from the PHP `TURNSTILE_SECRET` variable. (Plan 02.)
3. **Redis outage policy for rate limits** (A7). Recommendation: memory fallback plus a warning. (RESOLVED)
- **RESOLVED:** when Redis is down, the rate limiter falls back to a per-instance `surf.MemoryStore` and logs a warning. (Plan 01.)
4. **Purge coalescing.** Debounce window (for example 250 ms trailing) vs one purge per commit. Recommendation: trailing debounce in `cooler`. (RESOLVED)
- **RESOLVED:** no debounce. One purge POST per committed save; the recommendation above is superseded. (Plan 04.)
5. **Plan count.** Suggested lean split for the user to confirm: (1) decision notes + `cooler` cache/Redis store + serve wiring + caching/rate-limit docs; (2) bouncer cookies + Turnstile + surf CORS hardening/profiles + api-docs; (3) attach S3 tag + `summer.yaml` `build.tags` + `public_url` + extension helper; (4) site-cache purge in `cooler` + cabana reorder hook + Journal/menu/FAQ plugins + three vue layers + app pointer bumps; (5) unit tests. Plan 4 depends on plan 1; plans 2 and 3 are independent. (RESOLVED)
- **RESOLVED:** the 5-plan layout above is confirmed.
## Environment Availability
| Dependency | Required By | Available | Version | Fallback |
|------------|------------|-----------|---------|----------|
| Go | everything | ✓ | go1.27.0 | — |
| Docker daemon | testcontainers Redis/Postgres tests | ✓ (`docker ps` ok) | client 29.7.2 | `-short` with miniredis |
| Redis server (local) | manual probes | ✓ | 7.2.5 | testcontainers `redis:7-alpine` / `redis:8-alpine` (images present locally) |
| redis-cli | probes | ✓ | — | — |
| node / pnpm / npm | vue layer tests (`node --test`) | ✓ | — | — |
| S3/MinIO | real S3 smoke | ✗ (no MinIO image found) | — | Open the bucket in a unit test only (s3blob opens lazily); real upload stays a human UAT item |
| Cloudflare siteverify | Turnstile live check | network-dependent | — | `httptest` server via the verifier's endpoint override |
**Missing dependencies with no fallback:** none.
**Missing dependencies with fallback:** MinIO/S3 (unit-level open test + human UAT).
## Validation Architecture
### Test Framework
| Property | Value |
|----------|-------|
| Framework | Go `testing` (Go 1.27); `node --test` in the vue layers |
| Config file | none (go.mod); layers: `package.json` `"test": "node --test --test-reporter=tap tests/*.test.ts"` |
| Quick run command | `go test -short ./modules/cooler ./modules/surf ./modules/bouncer ./modules/lagoon/... ./internal/build` |
| Tagged run | `go vet -tags summercms_s3 ./modules/lagoon/... && go test -short -tags summercms_s3 ./modules/lagoon/attach` |
| Full suite command | `go vet ./... && go test ./... && go test ./cmd/summer -run TestDocsTree && (cd examples/hello && go test ./...)` |
### Phase Requirements → Test Map
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| SC-1 | TokenCookie attrs (name/path/domain/secure/samesite, MaxAge=refresh_ttl); forget cookie; refresh handler rotates and blacklists the old JTI; invalid domain or `none` without Secure fails boot | unit | `go test -short ./modules/bouncer -run 'TestTokenCookie|TestCookieRefresh'` | ❌ Wave 0 |
| SC-1 | CORS: credentials+`*` boot error; echo exact origin; `Vary: Origin` on preflight and disallowed origin; api↔www subdomain; `*` non-credentialed unchanged; profile OPTIONS route | unit | `go test -short ./modules/surf -run 'TestCORS'` | partial (cors_test.go, cors_coverage_test.go exist) |
| SC-2 | Memory driver Get/Set/Has/Forget/Remember/Increment/Flush with TTL and prefix | unit | `go test -short ./modules/cooler -run TestMemory` | ❌ Wave 0 |
| SC-2 | Redis driver + Flush only under prefix | unit (miniredis) + docker | `go test ./modules/cooler -run TestRedis` | ❌ Wave 0 |
| SC-2 | Redis `Attempt` matches `MemoryStore` (table-driven against both stores; FastForward for expiry; concurrent attempts never exceed max) | unit + docker | `go test ./modules/cooler -run TestRateLimitParity` | ❌ Wave 0 |
| SC-2 | BuildRouter uses a published `surf.Store`; route:list does not dial | unit | `go test -short ./modules/surf -run TestBuildRouterStore` | ❌ Wave 0 |
| SC-3 | `s3://` without the tag: boot error names `summercms_s3`; with the tag it opens | unit (both tag modes) | `go test -short ./modules/lagoon/attach -run TestOpenBucketS3` (+ `-tags summercms_s3`) | ❌ Wave 0 |
| SC-3 | `summer.yaml` `build.tags` → `go build -tags`; SaveManifest round-trip | unit | `go test -short ./internal/build -run 'TestManifestBuildTags|TestSaveManifest'` | ❌ Wave 0 |
| SC-3 | `public_url` URL building | unit | `go test -short ./modules/lagoon/attach -run TestPublicURL` | partial (url_test.go) |
| D-15 | PNG uploaded as `x.html` gets a `.png` disk name and URL; docx keeps docx; html-as-txt stays txt | unit | `go test -short ./modules/lagoon/attach -run 'TestStoreDiskExtension'` | ❌ Wave 0 |
| SC-3 | `/api-docs` HTML carries the pinned version and SRI; `/api-docs.json` compact JSON in key order; off by default; boot errors for zero or two providers | unit | `go test -short ./modules/surf -run TestAPIDocs` | ❌ Wave 0 |
| SC-3 | Turnstile open/closed matrix (empty secret dev/prod, missing token, success false, timeout via slow httptest, body restored) | unit | `go test -short ./modules/bouncer -run TestTurnstile` | ❌ Wave 0 |
| D-10/11 | Purger validation, after-commit only, failure never fails the save, alias keys | unit | `go test -short ./modules/cooler -run TestSiteCache`; in plugin repos `go test ./...` (inside an app checkout) | ❌ Wave 0 (journal tests exist to port) |
| D-11 | cabana reorder hook runs inside the tx | unit | `go test -short ./modules/cabana -run TestReorderHook` | ❌ Wave 0 |
| D-11 | Nuxt shared purge route authorize/purge util | unit (node) | `pnpm -C ../vue-menu-layer test` (and faq, journal) | ❌ Wave 0 (journal util tests exist) |
| SC-4 | Docs identifiers/links | docs | `go test ./cmd/summer -run TestDocsTree` | ✅ |
### Sampling Rate
- **Per task commit:** the quick run command for the touched packages, plus `go vet ./...`.
- **Per wave merge:** the full suite command (Docker available) plus the tagged run.
- **Phase gate:** full suite + tagged run + docs checker green before `/msd-verify-work`; plugin repos' `go test ./...` from an app checkout; layer `pnpm test`.
### Wave 0 Gaps
- [ ] `modules/cooler/*_test.go`: memory, redis (miniredis), Docker TestMain, rate-limit parity table shared with `MemoryStore`
- [ ] `modules/bouncer/cookie_test.go`, `turnstile_test.go`
- [ ] `modules/surf/cors_profile_test.go`, `apidocs_test.go`, store-lookup test
- [ ] `modules/lagoon/attach/s3_test.go` (both tag modes), `diskext_test.go`, Store regression
- [ ] `internal/build` manifest tags + SaveManifest round-trip
- [ ] `modules/cabana` reorder hook test
- [ ] Framework dependency install: `go get github.com/alicebob/miniredis/v2@v2.40.0` (after its decision note)
## Security Domain
### Applicable ASVS Categories
| ASVS Category | Applies | Standard Control |
|---------------|---------|-----------------|
| V2 Authentication | yes (Turnstile anti-automation) | `bouncer` verifier against Cloudflare siteverify; secret server-side only |
| V3 Session Management | yes (token cookie) | httpOnly, Secure, SameSite=Lax, Max-Age=refresh window, rotation + JTI blacklist (`bouncer.Refresh*`, Postgres blacklist) |
| V4 Access Control | yes (CORS, purge route, api-docs public by design) | Exact-origin allow-list; purge bearer token ≥32 chars, loopback-only http, proxied requests refused |
| V5 Input Validation | yes | Upload sniffing + extension helper; config validation at boot (cookie domain, CORS patterns, tags regex, purge URL) |
| V6 Cryptography | no new crypto | Constant-time token compare stays in the Nuxt util (sha256 + timingSafeEqual) |
| V12 Files and Resources | yes | Disk extension from content, nosniff + CSP sandbox on `/storage/uploads`, `public_url` exposure documented |
| V14 Configuration | yes | Secrets via `SUMMER_*` env; Redis URL never logged; SRI on CDN assets |
### Known Threat Patterns for this stack
| Pattern | STRIDE | Standard Mitigation |
|---------|--------|---------------------|
| Credentialed CORS reflecting any origin | Information disclosure | Boot error for `*` + credentials; exact-origin echo; `Vary: Origin` |
| Cache poisoning across origins | Tampering | `Vary: Origin` on every dynamic CORS response |
| CSRF on cookie-authenticated endpoints from sibling subdomains (SameSite=Lax treats same-site as safe) | Spoofing/Tampering | CORS is not CSRF protection: require JSON or a custom header on state-changing cookie routes; optionally check `Origin` in the cookie handler against the allowed list |
| Cookie scoped too widely (`Domain=.example.com` reaches every subdomain) | Information disclosure | Domain explicit and opt-in (default host-only); document the subdomain trust boundary |
| Stored XSS via a disk-name extension | Tampering/Elevation | `DiskExtension` from the sniffed type; never keep script-capable extensions; nginx nosniff + allow-list |
| Private files via `public_url` | Information disclosure | Bucket only for public files, or proxy route; document |
| Turnstile bypass by an empty secret in production | Spoofing | Closed mode: 503 in production when unconfigured; open mode only on registration paths |
| Turnstile token replay | Spoofing | Single-use enforced by Cloudflare (`timeout-or-duplicate`); never cache verdicts per token |
| Siteverify hang causing request pile-up | DoS | 5 s client timeout; context-aware request |
| Redis credential leak in logs | Information disclosure | Never format `redis.url` or raw `url.Parse` errors (Pitfall 7) |
| Rate limit bypass during a Redis outage | DoS/Abuse | Memory fallback per instance plus a warning |
| Redis key injection or collision | Tampering | Key prefixing `<prefix>rl:` / `<prefix>cache:`; keys are opaque strings to Redis (no command injection through go-redis args) |
| CDN asset tampering on `/api-docs` | Tampering | Exact version + SRI sha384 + `crossorigin="anonymous"` |
| Purge route abuse | DoS | Bearer ≥32 chars, constant-time compare, refuses proxied requests, nginx `limit_except GET HEAD` |
## Sources
### Primary (HIGH confidence)
- In-repo source read this session: `modules/surf/{limiter_store.go,limiter.go,cors.go,router.go,serve.go,uploads.go}`, `modules/lagoon/attach/{bucket.go,store.go,thumb.go}`, `modules/bouncer/{refresh.go,jwt.go,blacklist.go}`, `modules/cabana/{auth.go,reorder.go,crud.go}`, `modules/compass/{config.go,env.go}`, `modules/backpack/services.go`, `internal/build/{build.go,manifest.go}`, `internal/docsite/{check_identifiers.go,check_forbidden.go}`
- Sibling repos: `sm-journal-plugin/classes/site_cache.go`, `controllers/{categories.go,site_cache.go}`; `vue-journal-layer/{server/api/__journal__/cache/purge.post.ts,app/utils/siteCache.ts,nuxt.config.ts}`; `sm-grzybyfunkcjonalne-app/{scripts/build.sh,summer.yaml,.gitmodules,DEPLOY.md}`; fonoteka.go user plugin `api_controller.go` and `config.yaml`
- PHP reference: wavepath.org `AuthApiController.php`, `CookieAttributes.php`, `TurnstileVerifier.php`, `GuestCallController.php`, `ApiDocsController.php`, `config/cors.php`, `config/cookie.php`, `vendor/fruitcake/php-cors` v1.4.0
- Module sources from proxy.golang.org: go-redis v9.23.0, otter v2.3.0, miniredis v2.40.0; gocloud.dev v0.46.0 from the module cache; nitropack 2.13.4 dist
- Probes: Redis 7.2.5 Lua script run; Go `http.DetectContentType` and `url.Parse` runs; swagger-ui-dist tarball vs jsDelivr sha384
### Secondary (MEDIUM confidence)
- developers.cloudflare.com Turnstile server-side validation and testing pages (fetched this session)
### Tertiary (LOW confidence)
- S3-compatible provider checksum incompatibilities; Laravel cache prefix default; Symfony cookie serialization (marked [ASSUMED])
## Metadata
**Confidence breakdown:**
- Standard stack: HIGH. Versions come from proxy.golang.org, and API identifiers were read from the module sources.
- Architecture: HIGH. Boot order, import graph and the reorder-hook gap were read from source; the Lua script was probed on real Redis.
- Pitfalls: HIGH for the in-repo ones; MEDIUM for S3-compatible providers and the ServeMux 405 behaviour (A1, a one-test check).
**Research date:** 2026-10-15
**Valid until:** 2026-11-14 (go-redis and swagger-ui-dist release often; re-check pins before tagging)

View File

@@ -0,0 +1,123 @@
---
phase: 19.3
review: 19.3-REVIEW.md
titles: json
findings:
- id: CR-01
severity: critical
disposition: fixed
title: "`cors:<profile>` headers are missing from every response produced outside the profile middleware (401, 403, 413, 422, 429, 500)"
- id: WR-01
severity: warning
disposition: fixed
title: "The site-cache purge runs synchronously in the admin request, and bulk delete multiplies it"
- id: WR-02
severity: warning
disposition: fixed
title: "The purge client follows redirects and forwards the bearer token, including https-to-http on the same host"
- id: WR-03
severity: warning
disposition: fixed
title: "The shipped purge route refuses every proxied request, so the https/remote mode the Go side advertises cannot work"
- id: WR-04
severity: warning
disposition: fixed
title: "Credentialed CORS validation only refuses a bare `*`; a wildcard entry or an unanchored pattern still trusts arbitrary origins with credentials"
- id: WR-05
severity: warning
disposition: fixed
title: "In production, the cookie's Secure flag is inferred false without an error when `app.url` is http or unset"
- id: WR-06
severity: warning
disposition: fixed
title: "The api-docs CSP allowlists all of jsDelivr for scripts, which cancels the nonce"
- id: WR-07
severity: warning
disposition: fixed
title: "`storage.uploads.public_url` puts protected (`is_public=false`) files in a publicly served bucket"
- id: WR-08
severity: warning
disposition: fixed
title: "`turnstile:open` without a secret switches off bot protection in production, with nothing logged"
- id: WR-09
severity: warning
disposition: fixed
title: "The zero `CookieConfig` does not give the documented default attributes (no SameSite, no Secure)"
- id: IN-01
severity: info
disposition: open
title: "The cache prefix has no enforced separator, so one application's Flush can reach another's keys"
- id: IN-02
severity: info
disposition: open
title: "The purge token is not trimmed although the URL is"
- id: IN-03
severity: info
disposition: open
title: "The purge and the Journal alias wiring log through `slog.Default()` instead of the app logger"
- id: IN-04
severity: info
disposition: open
title: "`SiteCacheFromApp` logs an Error on every call when the configuration is invalid"
- id: IN-05
severity: info
disposition: open
title: "The Turnstile middleware buffers the whole body even for content types it never parses"
- id: IN-06
severity: info
disposition: open
title: "Each preflight route copies the constraints of the first route on its path only"
- id: IN-07
severity: info
disposition: open
title: "The `DiskExtension` script-capable denylist is incomplete"
- id: IN-08
severity: info
disposition: open
title: "The cookie refresh and logout handlers do not check `Origin`"
- id: IN-09
severity: info
disposition: open
title: "Timing-sensitive cooler unit tests may flake on loaded CI"
- id: IN-10
severity: info
disposition: open
title: "The `purgeSiteCache` helper is copied across the FAQ and menu plugins, with a third variant in the Journal"
- id: IN-11
severity: info
disposition: open
title: "A published `*bouncer.Turnstile` silently overrides `turnstile.secret`"
open: 11
total: 21
recorded: 2026-10-15T18:18:43.865Z
---
# Phase 19.3: Code Review Disposition
| Finding | Severity | Disposition | Source |
|---------|----------|-------------|--------|
| CR-01 | critical | fixed | summercms.go 0723536 (19.3-06) |
| WR-01 | warning | fixed | summercms.go db6a7f5; sm-faq-plugin e52b286, sm-menu-plugin 4b42978, sm-journal-plugin 4f633f0 (19.3-06) |
| WR-02 | warning | fixed | summercms.go 50a584c (19.3-06) |
| WR-03 | warning | fixed | vue-faq-layer a229f6a, vue-menu-layer c4639c2, vue-journal-layer f5d7aac; summercms.go 4d4e379 (19.3-06) |
| WR-04 | warning | fixed | summercms.go e32940d (19.3-06) |
| WR-05 | warning | fixed | summercms.go 4dae833 (19.3-06) |
| WR-06 | warning | fixed | summercms.go f705343 (19.3-06) |
| WR-07 | warning | fixed | summercms.go caa5b5d (19.3-06) |
| WR-08 | warning | fixed | summercms.go d0a580c (19.3-06) |
| WR-09 | warning | fixed | summercms.go 85d8b6a (19.3-06) |
| IN-01 | info | open | - |
| IN-02 | info | open | - |
| IN-03 | info | open | - |
| IN-04 | info | open | - |
| IN-05 | info | open | - |
| IN-06 | info | open | - |
| IN-07 | info | open | - |
| IN-08 | info | open | - |
| IN-09 | info | open | - |
| IN-10 | info | open | - |
| IN-11 | info | open | - |
Dispositions: `open` (recorded, not yet triaged), `fixed`, `skipped`, `deferred`.
Set `deferred` by hand and put the reason in the Source cell; both are preserved. A `|` in the reason is kept as prose and escaped on the next run.
Re-running the gate keeps every row it can. A row the current review no longer reports is kept and its Source cell flagged, so a finding does not leave this record silently. ONE exception: when a finding id is REUSED by a different finding, the earlier decision cannot keep a row — the id is taken — and it is dropped. A RECORDED decision (anything but `open`) is named on the console when that happens; a row still at `open` is replaced silently, because `open` records no decision to lose.

View File

@@ -0,0 +1,254 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
reviewed: 2026-10-15T18:16:47Z
depth: standard
files_reviewed: 35
files_reviewed_list:
- go.mod
- internal/build/build.go
- internal/build/manifest.go
- modules/bouncer/cookie.go
- modules/bouncer/turnstile.go
- modules/cabana/reorder.go
- modules/cooler/cooler.go
- modules/cooler/memory.go
- modules/cooler/ratelimit.go
- modules/cooler/redis.go
- modules/cooler/rediscache.go
- modules/cooler/sitecache.go
- modules/lagoon/attach/bucket.go
- modules/lagoon/attach/diskext.go
- modules/lagoon/attach/s3_off.go
- modules/lagoon/attach/s3_on.go
- modules/lagoon/attach/store.go
- modules/lagoon/attach/thumb.go
- modules/pact/capabilities.go
- modules/surf/apidocs.go
- modules/surf/cors.go
- modules/surf/router.go
- modules/surf/serve.go
- scripts/check-phase19.3.sh
- modules/cooler/memory_test.go
- modules/cooler/install_test.go
- modules/cooler/ratelimit_test.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-faq-plugin/controllers/site_cache.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-faq-plugin/controllers/faqs.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-menu-plugin/controllers/site_cache.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-menu-plugin/controllers/menus.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-menu-plugin/controllers/menuentries.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-menu-plugin/controllers/admin_registry.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-journal-plugin/classes/site_cache.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-journal-plugin/controllers/site_cache.go
- /media/nvme/dev/golem15/summercms.io/summercms/sm-journal-plugin/plugin.go
- /media/nvme/dev/golem15/summercms.io/summercms/vue-faq-layer/server/lib/siteCachePurge.ts
- /media/nvme/dev/golem15/summercms.io/summercms/vue-faq-layer/server/api/__site__/cache/purge.post.ts
findings:
critical: 1
warning: 9
info: 11
total: 21
status: issues_found
---
# Phase 19.3: Code Review Report
**Reviewed:** 2026-10-15T18:16:47Z
**Depth:** standard
**Files Reviewed:** 35 (framework sources, the plugins' purge wiring, the shared Nuxt purge route; tests got a lighter pass)
**Status:** issues_found
## Narrative Findings (AI reviewer)
## Summary
The phase is careful in most of the places the brief asked about. The Redis Lua script is atomic and matches `surf.MemoryStore`: a missing key or one without a TTL opens the window, a denied attempt does not increment, and INCR keeps the TTL. Keys are namespaced `<prefix>rl:` / `<prefix>cache:`, and Flush escapes SCAN glob characters. The S3 driver is linked only under `summercms_s3`, and an untagged binary refuses `s3://` at boot. Build tags are validated by regex and passed as a single exec argument. The api-docs template escapes the title and spec URL, and the assets carry SRI. The purge token is compared in constant time over sha256 digests. Every cabana write path that reaches the plugins' hooks (`crud.go:134`, `reorder.go:319`) now runs in `lagoon.Transaction`, so after-commit semantics hold.
The defects are at the edges between modules:
- The `cors:<profile>` middleware runs in declaration order. Responses from the guard, throttle, Turnstile, body limit or panic recovery therefore carry no CORS headers, and the browser cannot read them (CR-01).
- The site-cache purge runs synchronously inside the admin request after commit, and bulk deletes repeat it per row.
- The purge client follows redirects with the bearer token attached.
- The shipped Nuxt route returns 404 to every proxied request, so the https mode that the Go side advertises cannot be used.
- Several hardening gaps remain in credentialed CORS validation, cookie Secure inference, the api-docs CSP and Turnstile misconfiguration visibility.
Already logged in `deferred-items.md` and not counted again: S3 object metadata keeps the sniffed `text/html` / `text/javascript` content type; Redis outage latency in `Cache.Remember` plus go-redis stderr logging; the unreachable "in transaction but no purger" plugin branch; the gofmt and consumer-name items.
## Critical Issues
### CR-01: `cors:<profile>` headers are missing from every response produced outside the profile middleware (401, 403, 413, 422, 429, 500)
**File:** `modules/surf/router.go:416-466`, `modules/surf/cors.go:255-262`
**Issue:** `wrap` composes route middleware so that `rt.middleware[0]` is outermost (`for i := len-1; i >= 0; i-- { h = mw(h) }`). It then puts `bodyLimit`, `r.locale` and `recoverJSON`/`recoverBare` outside all of them. `profileMiddleware` only sets `Access-Control-Allow-Origin` when the request reaches it. With the ordering the docs suggest (`docs/services/routing.md:265`: "lists `cors:spa` among its middleware names, next to its guard"), a route such as `[]string{"auth", "throttle:60,1", "turnstile:open", "cors:spa"}` answers:
- a guard refusal (401, expired token),
- a throttle denial (429),
- a Turnstile refusal (422/503),
- a body-limit trip (413, always outside), and
- a recovered panic (500, always outside)
all without ACAO, ACAC or `Vary: Origin`. The browser turns each of these into an opaque network error. A credentialed SPA, which is the use case profiles exist for, can no longer see the 401 that should trigger its cookie refresh, or the 422 that should re-show the Turnstile widget. The global `pathScopedCORS` wrapper does not have this problem because it runs before the mux.
**Fix:** Apply the profile outside every other route layer, whatever position it has in the list. In `wrap`, take the `cors:` entry out of the loop and wrap it last, after `recoverJSON` and before `recordPattern`:
```go
profile, _ := routeCORSProfile(rt) // already validated in addCORSPreflights
// ... loop skips names with base "cors" ...
if rt.raw { h = recoverBare(h) } else { h = recoverJSON(h) }
if profile != "" {
h = r.corsProfiles[profile].profileMiddleware(h)
}
return recordPattern(rt.method+" "+rt.path, h), nil
```
Add a test that sends a cross-origin request to a `cors:spa` route behind a refusing guard and asserts that the 401 carries `Access-Control-Allow-Origin`.
## Warnings
### WR-01: The site-cache purge runs synchronously in the admin request, and bulk delete multiplies it
**File:** `modules/cooler/sitecache.go:123-195`, `modules/lagoon/transaction.go:86,194-205`, `modules/cabana/crud.go:1025-1040`
**Issue:** `lagoon.Transaction` runs after-commit callbacks inline (`runAfterCommit` after `gdb.Transaction` returns). `PurgeSiteCacheAfterCommit` registers `p.Purge`, which is a blocking HTTP POST with a 2 s bound. Every admin save therefore waits for the Nuxt site. When the site is down or slow, each save takes up to 2 s longer. `BulkDelete` calls `deleteRecord` per row, and the FAQ, menu and menu-entry hooks each register a purge, so deleting N rows sends N sequential full-cache purges after commit. With N=50 and a hung site that is about 100 s, which is past typical server write timeouts. The client then sees a failed request for a delete that has committed. Each purge also wipes the whole `nitro:routes` cache (`siteCachePurge.ts:62-71`), so after the first purge the remaining N-1 do nothing.
**Fix:** Coalesce per transaction and send asynchronously. One option is a purger method that marks "purge pending" in the after-commit buffer once per transaction (dedupe on the `*SiteCachePurger`). Another is to make `Purge` enqueue onto a single background worker with a small debounce, for example:
```go
func (p *SiteCachePurger) Purge(ctx context.Context, kind, slug string) {
if p == nil { return }
select { case p.queue <- siteCachePurgeBody{kind, slug}: default: /* one already pending */ }
}
```
Document that the purge is fire-and-forget.
### WR-02: The purge client follows redirects and forwards the bearer token, including https-to-http on the same host
**File:** `modules/cooler/sitecache.go:93,148`
**Issue:** `&http.Client{}` uses the default redirect policy. Go forwards `Authorization` on a redirect when the destination hostname equals the original or is a subdomain of it, and it ignores the scheme and port. A purge URL `https://site.example.com/api/__site__/cache/purge` that answers 301/302/307/308 to `http://site.example.com/...` (a common proxy misconfiguration, or a trailing-slash redirect behind a TLS terminator) therefore sends the token in clear. `NewSiteCachePurgerFor` (lines 87-89) exists to prevent exactly that. A 301/302 also turns the POST into a GET, so the purge silently does nothing.
**Fix:**
```go
client: &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }},
```
A 3xx then logs as `site cache purge failed status=30x`.
### WR-03: The shipped purge route refuses every proxied request, so the https/remote mode the Go side advertises cannot work
**File:** `/media/nvme/dev/golem15/summercms.io/summercms/vue-faq-layer/server/api/__site__/cache/purge.post.ts:109-121`, `modules/cooler/sitecache.go:60-63,87-89`
**Issue:** On the Go side, `site_cache.purge_url` accepts any https host ("use https for any other host"). The route that all three layers ship returns 404 whenever `X-Forwarded-For`, `X-Real-IP` or `Forwarded` is present. A remote https endpoint is reached through a TLS-terminating proxy almost by definition, and nginx, Caddy and Traefik add those headers by default. The configuration therefore passes boot validation and fails every save with `status=404`. Operators get a warning per save and no explanation. The two halves of one feature disagree about the threat model.
**Fix:** Pick one model. Either restrict `purge_url` to loopback hosts in `NewSiteCachePurgerFor` (drop the https/remote allowance and its docs), or let the route accept proxied requests when an explicit runtime flag is set (for example `siteCache.allowProxied`), relying on the bearer token alone, and document the pairing in `docs/services/caching.md`.
### WR-04: Credentialed CORS validation only refuses a bare `*`; a wildcard entry or an unanchored pattern still trusts arbitrary origins with credentials
**File:** `modules/surf/cors.go:45-58,79-88,120-137`
**Issue:** `validateCORSConfig` rejects `supports_credentials: true` only when `allowed_origins` contains exactly `"*"`. These still pass:
- `allowed_origins: ["https://*"]` or `["*://*"]`, which `wildcardOriginPattern` turns into `^https://.*$` and which trusts every https origin with credentials;
- `allowed_origins_patterns: ["https://.*\\.example\\.com"]` (delimiters stripped, as the error message tells PHP users to do, but without `^`/`$`), which `MatchString` treats as a substring match, so `https://x.example.com.attacker.net` is reflected with `Access-Control-Allow-Credentials: true`.
The phase made "credentials with `*` is a boot error" a deliberate hardening step (02-SUMMARY), and these are the same exposure under a different spelling.
**Fix:** When `SupportsCredentials` is true:
- require every `allowed_origins_patterns` entry to start with `^` and end with `$` (or anchor them yourself with `^(?:` + p + `)$` in `newCORSPolicy`);
- refuse wildcard origins whose host part is only `*` (for example, require at least one literal label after the `*`, as in `https://*.example.com`).
Add these cases to `TestCORSCredentialsWildcardRefused`.
### WR-05: In production, the cookie's Secure flag is inferred false without an error when `app.url` is http or unset
**File:** `modules/bouncer/cookie.go:100-107,116-121`
**Issue:** The production guard only refuses an *explicit* `auth.cookie.secure: false`. When the key is unset, `cookieSecure` derives Secure from `app.url`. If `app.url` is missing, mistyped or `http://` (common when TLS terminates at a proxy and `APP_URL` was copied from a dev `.env`), production writes the httpOnly auth token cookie without `Secure`, and nothing errors or warns. `SameSite=None` is caught; Lax/Strict are not.
**Fix:** In the production environment, treat an inferred `secure=false` like an explicit one:
```go
if !secure && strings.EqualFold(strings.TrimSpace(src.Environment()), "production") {
return cfg, errors.New("bouncer: auth.cookie.secure resolves to false in production (app.url is not https); set app.url to https or auth.cookie.secure: true")
}
```
At the least, log a warning naming `app.url`.
### WR-06: The api-docs CSP allowlists all of jsDelivr for scripts, which cancels the nonce
**File:** `modules/surf/apidocs.go:207`
**Issue:** `script-src 'nonce-…' https://cdn.jsdelivr.net` lets any script from jsDelivr execute, and that includes every npm package and every GitHub repo (`/gh/<user>/<repo>`). Any HTML injection on the page (Swagger UI renders `description` markdown from the spec, and Swagger UI has a history of DOM XSS) can load attacker code from jsDelivr without a nonce. The SRI attributes protect only the three tags that carry them. The policy also has no `default-src`, `style-src` or `connect-src`.
**Fix:** Add the nonce to the three external `<script>` tags as well (`<script nonce="{{.Nonce}}" src=… integrity=…>`) and drop the host source: `script-src 'nonce-X'; style-src https://cdn.jsdelivr.net/npm/swagger-ui-dist@5.33.1/ 'unsafe-inline'; default-src 'self'; img-src 'self' data:; object-src 'none'; base-uri 'none'; frame-ancestors 'none'`. Alternatively, pin the host source to the exact path `https://cdn.jsdelivr.net/npm/swagger-ui-dist@5.33.1/`.
### WR-07: `storage.uploads.public_url` puts protected (`is_public=false`) files in a publicly served bucket
**File:** `modules/lagoon/attach/bucket.go:131-155`, `modules/lagoon/attach/store.go:155-172`
**Issue:** With `public_url` set, the bucket is served directly by the CDN/S3 host, and that host "serves any key without the is_public check" (the info log at line 151). `Store` still writes every upload to that one bucket, including those from protected relations (`Relation.Public == false`, which cabana serves through `serveProtectedFile`). The protection then rests only on the 88-bit random key staying secret. Common public-bucket setups that allow `s3:ListBucket`, CDN access logs, and any leak of the disk name expose protected files. The only safeguard is an Info-level log line.
**Fix:** When `public_url` is set, either refuse to store `Public: false` uploads (`ErrProtectedNeedsPrivateBucket`), or write protected files under a key prefix that the operator is told to deny publicly (for example `protected/…`) and check it at boot. At minimum, raise the log line to Warn and state in `docs/services/storage.md` that protected relations must not be used with `public_url`.
### WR-08: `turnstile:open` without a secret switches off bot protection in production, with nothing logged
**File:** `modules/surf/router.go:648-661`, `modules/bouncer/turnstile.go:105-111`
**Issue:** Open mode passes every request when `turnstile.secret` is empty, in every environment. Closed mode passes outside an environment named exactly `production` (so also `staging` and `prod`). This matches PHP, but surf builds the verifier once at boot and never reports that routes declaring `turnstile:*` are unprotected. A missing `SUMMER_TURNSTILE__SECRET` on deploy silently opens registration to bots.
**Fix:** After `wrap` has validated the routes, if any route names `turnstile:open|closed` and `buildTurnstile` returned an empty secret, log one Warn at boot ("turnstile.secret is empty: N routes accept requests without a Turnstile check"). Consider making it a boot error in production for `turnstile:closed`.
### WR-09: The zero `CookieConfig` does not give the documented default attributes (no SameSite, no Secure)
**File:** `modules/bouncer/cookie.go:24-37,262-272`
**Issue:** The field docs say SameSite "is http.SameSiteLaxMode unless auth.cookie.same_site says otherwise", and `withDefaults` says "a zero CookieConfig writes the default token cookie on '/'". `withDefaults` fills only Name and Path. A handler built with `CookieRefreshOptions{Cookie: bouncer.CookieConfig{}}` (an easy mistake in an app that skips `LoadCookieConfig`) writes the auth token cookie with no `SameSite` attribute, which Safari treats as None, and without `Secure`.
**Fix:** In `withDefaults`, set `if c.SameSite == 0 { c.SameSite = http.SameSiteLaxMode }`, and either make `CookieRefreshHandler`/`CookieLogoutHandler` reject a config not produced by `LoadCookieConfig` or document that a zero config is insecure.
## Info
### IN-01: The cache prefix has no enforced separator, so one application's Flush can reach another's keys
**File:** `modules/cooler/cooler.go:335-347`, `modules/cooler/rediscache.go:199-220`
**Issue:** `cache.prefix` is used verbatim. Prefix `acme` gives `acmecache:*`, and that SCAN MATCH also matches an application whose prefix is `acmecache:` (`acmecache:cache:x`). Similarly, `acme:` flushes keys of an application prefixed `acme:cache:`. The default prefix is the slug of `app.name`, so staging and production of one app that share a Redis database also share cache and rate-limit counters, and one Flush clears both.
**Fix:** Append `:` when the configured prefix does not end in one, and mention `cache.prefix` per environment in the caching guide.
### IN-02: The purge token is not trimmed although the URL is
**File:** `modules/cooler/sitecache.go:67-93`
**Issue:** A token read from an env file with a trailing newline or space passes the length check. Go then refuses to send the header (`invalid header field value`), or the Nuxt comparison fails. The result is a warning on every save with no hint at the cause.
**Fix:** `token = strings.TrimSpace(token)` (and the same in the Journal's alias check at `plugin.go:79`).
### IN-03: The purge and the Journal alias wiring log through `slog.Default()` instead of the app logger
**File:** `modules/cooler/sitecache.go:132`, `/media/nvme/dev/golem15/summercms.io/summercms/sm-journal-plugin/plugin.go:75`
**Issue:** Install and the cache drivers use `loggerFromApp`. `Purge` and `wireSiteCache` do not, so their lines bypass any app-published logger (format, level, destination).
**Fix:** Keep the logger on `SiteCachePurger` (set it in `installSiteCache`/`SiteCacheFromApp`) and use it in `Purge`.
### IN-04: `SiteCacheFromApp` logs an Error on every call when the configuration is invalid
**File:** `modules/cooler/sitecache.go:211-214`
**Issue:** Outside serve (job workers, console commands), each admin-hook call rebuilds, fails and logs again.
**Fix:** Cache the failure, for example by publishing a sentinel, or log through a `warner`.
### IN-05: The Turnstile middleware buffers the whole body even for content types it never parses
**File:** `modules/bouncer/turnstile.go:252-286`
**Issue:** The body is read into memory before the media type is checked. On a route with an upload-sized `body.limit`, a multipart request with no token is buffered in full before it is refused, which is pre-auth memory pressure. Multipart forms are also never searched for the token, so an HTML form with a file input always fails.
**Fix:** Parse `Content-Type` first, read the body only for JSON or urlencoded types, and either support `multipart/form-data` through `r.ParseMultipartForm` or document that it is unsupported.
### IN-06: Each preflight route copies the constraints of the first route on its path only
**File:** `modules/surf/cors.go:320-345`
**Issue:** When several routes share a path pattern but declare different constraints, the OPTIONS route uses the constraints of the first one in `r.routes`, so a valid preflight for another route can get 404. The overlap check against `http.cors.paths` matches the route *pattern* text (`/api/{slug}`) against globs, so a global glob such as `api/posts` that matches real requests to a profiled `/api/{slug}` route is not detected.
**Fix:** Merge constraints (accept when any route's constraints match), and test the global globs against a sample path built from each pattern.
### IN-07: The `DiskExtension` script-capable denylist is incomplete
**File:** `modules/lagoon/attach/diskext.go:39-42`
**Issue:** The list omits extensions that common server setups execute or render: `php3`-`php8`, `phps`, `asp`, `aspx`, `jsp`, `cgi`, `pl`, `py`, `mht`, `mhtml`. The extension allowlist runs first (`store.go:128`), so this matters only when an application widens `Limits.Extensions`, but the denylist is meant to be the backstop for that case.
**Fix:** Add those extensions, or match `^php\d*$|^pht` style families.
### IN-08: The cookie refresh and logout handlers do not check `Origin`
**File:** `modules/bouncer/cookie.go:213-260`
**Issue:** With `same_site: none` (allowed when Secure), a cross-site form POST can rotate or revoke the user's token (logout CSRF). This is low impact but has an easy fix.
**Fix:** Optionally verify `Origin`/`Sec-Fetch-Site` against the CORS profile's allowed origins when `SameSite` is None.
### IN-09: Timing-sensitive cooler unit tests may flake on loaded CI
**File:** `modules/cooler/memory_test.go:142-170`, `modules/cooler/install_test.go:224`
**Issue:** The tests assert that a 40 ms entry is present right after Set, a 60 ms TTL around `Increment`, and that `FromApp` finishes in under 100 ms. These can fail under CPU contention, and the phase gate fails on any test failure.
**Fix:** Use TTLs of at least 1 s with an injected clock, or widen the margins (for example, assert presence with a 1 s TTL and absence after the TTL plus a poll loop).
### IN-10: The `purgeSiteCache` helper is copied across the FAQ and menu plugins, with a third variant in the Journal
**File:** `/media/nvme/dev/golem15/summercms.io/summercms/sm-faq-plugin/controllers/site_cache.go:15-32`, `/media/nvme/dev/golem15/summercms.io/summercms/sm-menu-plugin/controllers/site_cache.go:15-32`, `/media/nvme/dev/golem15/summercms.io/summercms/sm-journal-plugin/controllers/site_cache.go:19-36`
**Issue:** These are the same logic (app nil check, `cabana.TxFromContext`, `SiteCacheFromApp`, `PurgeSiteCacheAfterCommit`) in three places. A fix for WR-01 would need three plugin releases.
**Fix:** Add a framework helper such as `cabana.PurgeSiteCacheOnSave(ctx, app, kind, slug)` (cabana already imports lagoon, and cooler does not import cabana), and have the plugins call it.
### IN-11: A published `*bouncer.Turnstile` silently overrides `turnstile.secret`
**File:** `modules/surf/router.go:655-657`
**Issue:** This seam was added for tests (02-SUMMARY deviation 1). Any plugin can publish a verifier with an empty secret and `Production: false` and switch off Turnstile application-wide, and nothing logs it.
**Fix:** Log at boot when a published verifier replaces a configured secret.
---
_Reviewed: 2026-10-15T18:16:47Z_
_Reviewer: Claude (msd-code-reviewer)_
_Depth: standard_

View File

@@ -0,0 +1,44 @@
---
status: testing
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
source: [19.3-VERIFICATION.md]
started: 2026-10-15T21:11:00.207Z
updated: 2026-10-15T21:11:00.207Z
---
## Current Test
number: 1
name: Cookie refresh across real api/www subdomains in a browser. Deploy to staging with auth.cookie.domain set to the parent domain and cors:<profile> (or http.cors) credentialed for the www origin. Log in on www, wait past the access TTL.
expected: |
The SPA reads the 401, the refresh call sends the cookie, the rotated cookie replaces the old one, the replayed old token answers 401.
awaiting: user response
## Tests
### 1. Cookie refresh across real api/www subdomains in a browser. Deploy to staging with auth.cookie.domain set to the parent domain and cors:<profile> (or http.cors) credentialed for the www origin. Log in on www, wait past the access TTL.
expected: The SPA reads the 401, the refresh call sends the cookie, the rotated cookie replaces the old one, the replayed old token answers 401.
result: [pending]
### 2. S3-compatible upload. Build with build.tags [summercms_s3], set storage bucket_url s3://...?endpoint=..., upload a public file, fetch it via storage.uploads.public_url and via /storage/uploads; try a protected (is_public false) upload with public_url set.
expected: Both URLs serve the public file; the protected upload is refused with attach.ErrProtectedNeedsPrivateBucket (admin answers the opaque 500 and logs the reason).
result: [pending]
### 3. Live Nuxt purge. Save a menu entry and a FAQ entry in admin against a running Nuxt site using the 0.3.0/0.5.0 layers; for a proxied deployment set NUXT_SITE_CACHE_ALLOW_PROXIED=true.
expected: The page updates without waiting for the 300 s SWR TTL; the app log shows 'cooler: site cache purged' after the save returned.
result: [pending]
### 4. Swagger UI under its new nonce-only CSP. Enable http.api_docs, open /api-docs in a browser.
expected: The UI renders the spec with no CSP violations in the console (script-src is now only 'nonce-...').
result: [pending]
## Summary
total: 4
passed: 0
issues: 0
pending: 4
skipped: 0
blocked: 0
## Gaps

View File

@@ -0,0 +1,110 @@
---
phase: "19.3"
slug: "framework-transport-and-infrastructure-for-wavepath"
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
status: draft
nyquist_compliant: false
wave_0_complete: true
created: "2026-10-15"
---
# Phase 19.3 — Validation Strategy
> Per-phase validation contract for feedback sampling during execution.
---
## Test Infrastructure
| Property | Value |
|----------|-------|
| **Framework** | Go `testing` (miniredis for fast Redis tests, testcontainers Redis behind `-short`); `node --test` in the vue layers |
| **Config file** | none (go.mod); layers: `package.json` `test` script |
| **Quick run command** | `go test -short ./modules/cooler ./modules/surf ./modules/bouncer ./modules/lagoon/... ./internal/build` |
| **Tagged run** | `go vet -tags summercms_s3 ./modules/lagoon/... && go test -short -tags summercms_s3 ./modules/lagoon/attach` |
| **Full suite command** | `go vet ./... && go test ./... && go test ./cmd/summer -run TestDocsTree && (cd examples/hello && go test ./...)` |
| **Estimated runtime** | ~30 s quick, ~180 s full (Docker) |
---
## Sampling Rate
- **After every task commit:** quick run command for the touched packages, plus `go vet ./...`
- **After every plan wave:** full suite command plus the tagged run
- **Before `/msd-verify-work`:** full suite + tagged run + docs checker green; plugin repos' `go test ./...` from an app checkout; layer `pnpm test`
- **Max feedback latency:** 60 seconds (quick run)
---
## Per-Task Verification Map
Filled by plan 05; the plan 06 rows (gap closure, run of 2026-10-15 with `PHASE193_REQUIRE_DOCKER=1 --all` on a Docker host) follow the SC-4 row. Every row is a stage of `scripts/check-phase19.3.sh`, which runs the named tests and fails when one is missing, skipped or red (go test -json and TAP detectors). Statuses from the run of 2026-10-15 on a Docker host.
| Req | Behavior | Test Type | Automated Command | File Exists | Status |
|-----|----------|-----------|-------------------|-------------|--------|
| SC-1, D-01..D-03 | TokenCookie attributes, Max-Age = refresh window, forget cookie, refresh handler rotates and blacklists the jti, missing cookie 401, LoadCookieConfig errors | unit | `bash scripts/check-phase19.3.sh go` (BOUNCER_TESTS: `TestTokenCookieMaxAgeIsRefreshWindow`, `TestCookieRefreshHandlerRotatesAndBlacklists`, ...) | ✅ `modules/bouncer/cookie_test.go` | ✅ green |
| SC-1, D-04, D-05 | CORS credentials+`*` boot error, patterns, wildcard origin entries, `Vary: Origin`, api↔www credentialed, `*` unchanged, profiles and their OPTIONS routes | unit | `bash scripts/check-phase19.3.sh go` (SURF_TESTS: `TestCORSSubdomainCredentialed`, `TestCORSProfilePreflightRoute`, `TestCORSOptionsMatchesLaravel`, ...) | ✅ `modules/surf/cors_profile_test.go` | ✅ green |
| SC-2, D-07 | Memory driver methods, TTL, bound, atomic Increment; Remember degrades | unit | `bash scripts/check-phase19.3.sh go` (COOLER_TESTS: `TestMemoryCache`, `TestMemoryCacheBounded`, `TestMemoryIncrementConcurrent`, `TestRememberDegrades`) | ✅ `modules/cooler/memory_test.go` | ✅ green |
| SC-2, D-07 | Redis driver (miniredis, `FastForward`), Flush only under `<prefix>cache:` | unit + docker | `bash scripts/check-phase19.3.sh go` (`TestRedisCache`, `TestRedisFlushOnlyPrefix`); `... docker` (`TestRedisCacheDocker`) | ✅ `redis_test.go`, `docker_test.go` | ✅ green |
| SC-2, D-08 | Redis `Attempt` parity with `surf.MemoryStore`, never more than max under concurrency, fallback on error, key prefix | unit + docker | `bash scripts/check-phase19.3.sh go` (`TestRateLimitParity`, `TestRateLimitConcurrentNeverExceedsMax`, `TestRateLimitFallbackOnError`, `TestRateLimitKeyPrefix`); `... docker` (`TestRateLimitParityDocker`) | ✅ `modules/cooler/ratelimit_test.go` | ✅ green |
| SC-2, D-09 | Install config errors, ping warning, FromApp lazy and reused | unit | `bash scripts/check-phase19.3.sh go` (`TestInstallConfigErrors`, `TestInstallPingWarning`, `TestInstallReusesFromApp`, `TestFromAppLazy`) | ✅ `modules/cooler/install_test.go` | ✅ green |
| SC-2, D-09 | BuildRouter uses a published `surf.Store`, falls back, never dials | unit | `bash scripts/check-phase19.3.sh go` (`TestBuildRouterStoreFallback`, `TestBuildRouterUsesPublishedStore`, `TestBuildRouterNeverDials`) | ✅ `modules/surf/store_lookup_test.go` | ✅ green |
| SC-3, D-12 | `s3://` without the tag names `summercms_s3`; with it opens | unit (both tag modes) | `bash scripts/check-phase19.3.sh go` (ATTACH_TESTS, untagged and `-tags summercms_s3`: `TestOpenBucketS3`) | ✅ `modules/lagoon/attach/s3_test.go` | ✅ green |
| SC-3, D-13 | `summer.yaml` `build.tags` load, validate, round-trip, reach `go build` | unit | `bash scripts/check-phase19.3.sh go` (BUILD_TESTS: `TestManifestBuildTags`, `TestSaveManifestKeepsBuildTags`, `TestGoBuildArgs`, ...) | ✅ `internal/build/manifest_tags_test.go` | ✅ green |
| SC-3 | `public_url` URL building and validation | unit | `bash scripts/check-phase19.3.sh go` (`TestPublicURLBase`, `TestPublicURLValidation`) | ✅ `modules/lagoon/attach/url_test.go` | ✅ green |
| D-15 | DiskExtension table; Store regression through Postgres | unit + docker | `bash scripts/check-phase19.3.sh go` (`TestDiskExtensionTable`); `... docker` (`TestStoreDiskExtension`) | ✅ `diskext_test.go`, `store_test.go` | ✅ green |
| SC-3, D-16, D-17 | `/api-docs` off by default, pinned 5.33.1 with SRI, ordered compact spec JSON | unit | `bash scripts/check-phase19.3.sh go` (`TestAPIDocsServesPinnedPage`, `TestAPIDocsSpecJSONOrder`, ...) | ✅ `modules/surf/apidocs_test.go` | ✅ green |
| SC-3, D-18 | Turnstile open/closed matrix, timeout, form fields, body restored, secret never logged | unit | `bash scripts/check-phase19.3.sh go` (`TestTurnstileVerifyMatrix`, `TestTurnstileMiddlewareRestoresBody`, ...; `TestTurnstileFactoryParams` in surf) | ✅ `modules/bouncer/turnstile_test.go` | ✅ green |
| D-10, D-11 | Purger after commit only, failure and slow receiver never fail the save, SiteCacheFromApp, Install validation | unit + docker | `bash scripts/check-phase19.3.sh go` (`TestSiteCachePurgeConfig`, `TestSiteCachePurgeRequest`, `TestSiteCacheFromApp`, `TestInstallSiteCacheValidation`); `... docker` (`TestSiteCacheAfterCommitOnly`, `TestSiteCacheFailureNeverFails`) | ✅ `modules/cooler/sitecache_test.go` | ✅ green |
| D-11 | cabana reorder hook runs inside the transaction, rolls back on error | docker | `bash scripts/check-phase19.3.sh docker` (`TestReorderHookRunsInTx`, `TestReorderHookErrorRollsBack`, `TestReorderWithoutHookUnchanged`) | ✅ `modules/cabana/reorder_hook_test.go` | ✅ green |
| D-10, D-11 | Plugin purge helpers; Journal framework keys win, alias keys deprecated | unit + docker | `bash scripts/check-phase19.3.sh plugins` (`TestPurgeSiteCacheWithoutTx`, `TestPurgeSiteCacheWithoutPurger`, `TestSiteCacheFrameworkKeysWin`, `TestSiteCacheAliasKeysDeprecated`, `TestSiteCacheNoKeys`); `... docker` (`TestFaqAdminPurgesSiteCache`, `TestMenuAdminPurgesSiteCache`, `TestCategoriesAdminPurgesSiteCache`, `TestPostsAdminPurgesSiteCache`, `TestCategoriesAdminFrameworkKeysWin`) | ✅ three plugins | ✅ green |
| D-11 | Nuxt shared purge route (lib and route handler), shared files byte-identical | unit (node) | `bash scripts/check-phase19.3.sh layers` (11 `sitePurge:` names per layer, `cmp` of the three shared files) | ✅ `tests/sitePurge.test.ts` in three layers | ✅ green |
| SC-4 | Docs identifiers and links; no consuming application named in framework lines of the phase | docs | `bash scripts/check-phase19.3.sh docs` | ✅ | ✅ green |
| SC-1, D-05 (CR-01, plan 06) | `cors:<profile>` headers on every response of the route: guard 401 and 403 (route and group), throttle 429, Turnstile 422 and 503, body limit 413, recovered 500 (JSON and raw); disallowed origin gets `Vary: Origin` only; handler `Vary` merged | unit | `bash scripts/check-phase19.3.sh go` (SURF_TESTS: `TestCORSProfileHeadersOnRefusals`, `TestCORSProfileVaryKeptWithHandlerVary`) | ✅ `modules/surf/cors_profile_test.go` | ✅ green |
| D-04 (WR-04, plan 06) | Credentialed CORS: wildcard must be the leftmost host label, patterns anchored, no rule matches the probe origins; global and per profile; non-credentialed configs unchanged | unit | `bash scripts/check-phase19.3.sh go` (SURF_TESTS: `TestCORSCredentialedOriginRules`, `TestCORSAnchoredPatternNoSuffixMatch`, rows in `TestCORSCredentialsWildcardRefused`) | ✅ `modules/surf/cors_profile_test.go` | ✅ green |
| D-03 (WR-05, plan 06) | Production refuses a Secure that resolves to false, set or inferred from app.url | unit | `bash scripts/check-phase19.3.sh go` (BOUNCER_TESTS: `TestLoadCookieConfigProductionInferredSecure`, rows in `TestLoadCookieConfigErrors`) | ✅ `modules/bouncer/cookie_test.go` | ✅ green |
| D-03 (WR-09, plan 06) | Zero `CookieConfig` writes SameSite=Lax and Secure; unset SameSite is Lax | unit | `bash scripts/check-phase19.3.sh go` (BOUNCER_TESTS: `TestZeroCookieConfigDefaults`, `TestTokenCookieAttributes`) | ✅ `modules/bouncer/cookie_test.go` | ✅ green |
| D-17 (WR-06, plan 06) | api-docs `script-src` holds only the per-request nonce; every script tag carries it; SRI unchanged | unit | `bash scripts/check-phase19.3.sh go` (SURF_TESTS: `TestAPIDocsCSPNoHostSource`, `TestAPIDocsServesPinnedPage`) | ✅ `modules/surf/apidocs_test.go` | ✅ green |
| D-12 (WR-07, plan 06) | `attach.Store` refuses a protected upload while `public_url` is set, before any blob is written | unit (both tag modes) | `bash scripts/check-phase19.3.sh go` (ATTACH_TESTS: `TestStoreProtectedNeedsPrivateBucket`) | ✅ `modules/lagoon/attach/url_test.go` | ✅ green |
| D-18 (WR-08, plan 06) | One boot warning with the route count when turnstile routes have no secret; none with a secret or without such routes | unit | `bash scripts/check-phase19.3.sh go` (SURF_TESTS: `TestTurnstileEmptySecretWarnsAtBoot`) | ✅ `modules/surf/cors_profile_test.go` | ✅ green |
| D-10 (WR-02, plan 06) | The purge client never follows a redirect; a 3xx is a failed purge, the token never reaches the Location | unit | `bash scripts/check-phase19.3.sh go` (COOLER_TESTS: `TestSiteCachePurgeNoRedirect`) | ✅ `modules/cooler/sitecache_test.go` | ✅ green |
| D-10, D-11 (WR-01, plan 06) | After-commit purges queued, one in flight per purger, later saves coalesced; Wait and Handle.Close drain; a bulk commit sends at most two; plugin admin tests wait for the queue | unit + docker | `bash scripts/check-phase19.3.sh go` (COOLER_TESTS: `TestSiteCachePurgeAsyncCoalesced`, `TestSiteCachePurgeWaitAndClose`); `... docker` (`TestSiteCacheBulkCommitCoalesced`, the FAQ, menu and Journal admin purge suites) | ✅ `modules/cooler/sitecache_test.go`, three plugins | ✅ green |
| D-11 (WR-03, plan 06) | Shared purge route accepts proxied requests on `siteCache.allowProxied`, refuses them without it; files byte-identical | unit (node) | `bash scripts/check-phase19.3.sh layers` (LAYER_REQUIRE: `sitePurge: allowProxied lets a proxied request reach the token check`, `sitePurge: route accepts a proxied request with allowProxied and the right bearer`) | ✅ `tests/sitePurge.test.ts` in three layers | ✅ green |
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
---
## Wave 0 Requirements
- [x] `modules/cooler/*_test.go`: memory, redis (miniredis), Docker suites (per-test containers instead of a TestMain), rate-limit parity table shared with `MemoryStore`
- [x] `modules/bouncer/cookie_test.go`, `turnstile_test.go`
- [x] `modules/surf/cors_profile_test.go`, `apidocs_test.go`, store-lookup test
- [x] `modules/lagoon/attach/s3_test.go` (both tag modes), `diskext_test.go`, Store regression
- [x] `internal/build` manifest tags + SaveManifest round-trip
- [x] `modules/cabana` reorder hook test
- [x] Dependency install: `github.com/alicebob/miniredis/v2@v2.40.0` (test-only, after the Task 4 human check: "approved")
---
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|----------|-------------|------------|-------------------|
| Cookie refresh across real api/www subdomains in a browser | SC-1 | Needs real DNS + TLS | Deploy to staging, log in on www, wait past access TTL, confirm refresh via cookie |
| S3-compatible provider (R2/MinIO/Hetzner) upload | SC-3 | Needs real bucket credentials | Build with `summercms_s3`, set `bucket_url`, upload a file, fetch via `public_url` and proxy |
| Nuxt purge after menu/FAQ save | D-11 | Needs running Nuxt + app | Save a menu entry in admin, confirm the page updates without waiting for SWR TTL |
---
## Validation Sign-Off
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
- [ ] Wave 0 covers all MISSING references
- [ ] No watch-mode flags
- [ ] Feedback latency < 60s
- [ ] `nyquist_compliant: true` set in frontmatter
**Approval:** pending

View File

@@ -0,0 +1,268 @@
---
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
verified: 2026-10-15T21:08:52Z
status: human_needed
score: 4/4 roadmap success criteria verified; plan must-haves 44/44 verified (36 from plans 01-05, 8 from gap-closure plan 06); 0 present-behavior-unverified
covered_files:
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-01-PLAN.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-01-SUMMARY.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-02-PLAN.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-02-SUMMARY.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-03-PLAN.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-03-SUMMARY.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-04-PLAN.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-04-SUMMARY.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-PLAN.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-SUMMARY.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-PLAN.md
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-06-SUMMARY.md
- go.mod
- go.sum
- internal/build/build.go
- internal/build/manifest.go
- modules/bouncer/cookie.go
- modules/bouncer/turnstile.go
- modules/cabana/reorder.go
- modules/cooler/cooler.go
- modules/cooler/memory.go
- modules/cooler/ratelimit.go
- modules/cooler/redis.go
- modules/cooler/rediscache.go
- modules/cooler/sitecache.go
- modules/lagoon/attach/bucket.go
- modules/lagoon/attach/diskext.go
- modules/lagoon/attach/s3_off.go
- modules/lagoon/attach/s3_on.go
- modules/lagoon/attach/store.go
- modules/lagoon/attach/thumb.go
- modules/pact/capabilities.go
- modules/surf/apidocs.go
- modules/surf/cors.go
- modules/surf/router.go
- modules/surf/serve.go
- scripts/check-phase19.3.sh
covered_digest: "v2:sha256:469d289a206fd43b90c3401d9971e57193333d0c2efefa1c88a2356520b1a092"
behavior_unverified: 0
overrides_applied: 0
re_verification:
previous_status: gaps_found
previous_verified: 2026-10-15T20:45:00Z
previous_score: 3/4 roadmap success criteria (SC-1 partial); plan must-haves 35/36 (D-05 failed)
gaps_closed:
- "SC-1 / D-05 (CR-01): cors:<profile> headers now reach every response of a profiled route. Router.wrap skips the cors entry in its middleware loop and wraps the recovered handler in r.corsProfiles[profile].profileMiddleware just inside recordPattern (modules/surf/router.go:483-485, fix commit 0723536). Verifier probe on HEAD c5d9a7b: [deny, cors:spa] 401, group Use(deny) + route cors:spa 401, nested group 401, handler panic 500 (JSON), middleware panic 500, raw-group panic 500, oversize body 413, throttle 429 all carry ACAO=https://www.example.com, ACAC=true, Vary: Origin; disallowed origin gets Vary: Origin only. The same probe against the pre-fix tree (0723536^) fails 7 of 11 cases."
gaps_remaining: []
regressions: []
advisory:
- finding: "WR-04 residual: credentialed allowed_origins https://*.co.uk loads (probe origins catch https://*.com and https://*, not multi-label public suffixes such as co.uk or com.pl)"
category: security
reason: "Raised in this re-verification by a scratch probe of surf.LoadCORSConfig (https://*.co.uk -> no error; https://*.com, https://*, https://*example.com -> error). Plan 06's must-have names probe origins and the https://*.com example, and that contract holds; refusing public-suffix wildcards would need a public-suffix list (a new dependency) or an extra probe set. Operator must write such an entry deliberately. Resolve by adding probes such as https://cors-probe.co.uk / .com.pl, or accept and note in docs/services/routing.md."
evidence_status: "scratch-probe output only; new scope relative to the carried-forward gap, not blocking"
human_verification:
- test: "Cookie refresh across real api/www subdomains in a browser. Deploy to staging with auth.cookie.domain set to the parent domain and cors:<profile> (or http.cors) credentialed for the www origin. Log in on www, wait past the access TTL."
expected: "The SPA reads the 401, the refresh call sends the cookie, the rotated cookie replaces the old one, the replayed old token answers 401."
why_human: "Needs real DNS, TLS and a browser's cookie and CORS enforcement; httptest cannot reproduce SameSite/Domain handling."
- test: "S3-compatible upload. Build with build.tags [summercms_s3], set storage bucket_url s3://...?endpoint=..., upload a public file, fetch it via storage.uploads.public_url and via /storage/uploads; try a protected (is_public false) upload with public_url set."
expected: "Both URLs serve the public file; the protected upload is refused with attach.ErrProtectedNeedsPrivateBucket (admin answers the opaque 500 and logs the reason)."
why_human: "Needs bucket credentials. Note deferred-items.md: the blob's stored Content-Type is the sniffed type."
- test: "Live Nuxt purge. Save a menu entry and a FAQ entry in admin against a running Nuxt site using the 0.3.0/0.5.0 layers; for a proxied deployment set NUXT_SITE_CACHE_ALLOW_PROXIED=true."
expected: "The page updates without waiting for the 300 s SWR TTL; the app log shows 'cooler: site cache purged' after the save returned."
why_human: "Needs a running Nuxt site and the app; purges are now asynchronous."
- test: "Swagger UI under its new nonce-only CSP. Enable http.api_docs, open /api-docs in a browser."
expected: "The UI renders the spec with no CSP violations in the console (script-src is now only 'nonce-...')."
why_human: "Browser rendering; plan 06 (WR-06) tightened script-src, and whether the pinned swagger-ui-bundle 5.33.1 needs anything beyond nonce-tagged scripts is only observable in a browser. SRI values were checked automatically in the initial verification."
---
# Phase 19.3: Framework transport and infrastructure for WavePath Verification Report
**Phase Goal:** The framework has the transport and infrastructure pieces a multi-client, multi-instance application needs: JWT cookie refresh across subdomains, credentialed CORS, a cache module with a Redis-backed rate-limit store, an S3 blob option, an OpenAPI page and Turnstile verification.
**Mode:** mvp (ROADMAP marks the phase `mode: mvp`, but the goal is not written as a user story; as in the initial report, this report uses standard goal-backward verification with a derived User Flow Coverage table. Run `/msd mvp-phase 19.3` to restate the goal, or drop `mode: mvp`, to make the mode consistent.)
**Verified:** 2026-10-15T21:08:52Z (HEAD `c5d9a7b`, local master, not yet tagged or pushed: expected, the release follows verification)
**Status:** human_needed
**Re-verification:** Yes, after gap-closure plan 19.3-06
## Re-verification Summary
| Item | Previous (2026-10-15T20:45Z) | Now | Evidence |
|------|------------------------------|-----|----------|
| SC-1 / D-05, CR-01: `cors:<profile>` headers on refusals, 413, 500 | ✗ FAILED (partial) | ✓ VERIFIED | Verifier probe (below) on HEAD: 11/11 PASS. Same probe on the pre-fix tree `0723536^`: 7/11 FAIL. `TestCORSProfileHeadersOnRefusals` and `TestCORSProfileVaryKeptWithHandlerVary` are required by name in `SURF_TESTS` and passed in the gate |
| WR-01..WR-09 (warnings) | open | fixed, verified in code and by named tests | see Plan 06 must-haves |
| Previously verified must-haves (35) | ✓ | ✓ no regressions | full gate with Docker green; cross-instance smokes and `go test -short ./...` green |
### Resolved gap (history)
The initial verification recorded one gap. `Router.wrap` composed `cors:<profile>` at its list position, inside the body limit and panic recovery. A guard listed before the profile, a guard on an enclosing group and a recovered 500 therefore answered without `Access-Control-Allow-Origin`, and the browser hid the 401 that starts the SPA's cookie refresh. Plan 06 (commit `0723536`) resolves the profile through `routeCORSProfile`, skips the `cors:` entry in the middleware loop after the unknown-profile check, and wraps the recovered handler in `r.corsProfiles[profile].profileMiddleware` just inside `recordPattern` (`modules/surf/router.go:417-485`). `bufferedResponse.commit` now merges `Vary` through `addVary` instead of replacing it (`router.go:837-852`), so a handler's own `Vary` does not drop `Vary: Origin`. The docs say the profile applies whatever its position: `docs/services/routing.md:101,271`, `modules/surf/README.md:31` and `docs/architecture/request-lifecycle.md:40`.
## User Flow Coverage (derived; the goal is not a user story)
Flow: a SPA on `www.example.com` talks to an API on `api.example.com` run as several instances.
| Step | Expected | Evidence | Status |
|------|----------|----------|--------|
| Log in, get a cookie | httpOnly `token` cookie, Domain/SameSite/Secure from `auth.cookie.*`, Max-Age = refresh window; production refuses an insecure cookie | `bouncer.LoadCookieConfig`, `CookieConfig.TokenCookie`; `TestTokenCookieMaxAgeIsRefreshWindow`, `TestLoadCookieConfigProductionInferredSecure`, `TestZeroCookieConfigDefaults` passed | VERIFIED |
| Access token expires, API answers 401 | The SPA can read the 401 cross-origin | Global `http.cors`: verified in the initial run. `cors:<profile>`: 401 carries ACAO+ACAC+Vary whatever the order or group nesting (probe, this run) | VERIFIED (real browser: human item 1) |
| Refresh through the cookie | New cookie, old jti blacklisted, replay refused | `TestCookieRefreshHandlerRotatesAndBlacklists` (real `bouncer.Refresh`), `TestCookieRefreshAcrossSubdomainsSmoke` passed in this run | VERIFIED |
| Requests land on different instances | Throttle counts one window across instances | `TestRateLimitAcrossInstancesSmoke` (two apps, one real Redis container) passed in this run; the 429 also carries the profile's CORS headers (probe) | VERIFIED |
| Upload a file to S3 | `bucket_url: s3://` in a tagged binary, no app import | `s3_on.go`/`s3_off.go`; `TestOpenBucketS3` in both tag modes; protected uploads refused when `public_url` is set (`TestStoreProtectedNeedsPrivateBucket`) | VERIFIED (real bucket: human item 2) |
| Read the API docs | `/api-docs` and `/api-docs.json` | `mountAPIDocs`; nonce on all three script tags (`apidocs.go:65-67`), `script-src 'nonce-…'` only (`apidocs.go:208`); `TestAPIDocsCSPNoHostSource` passed | VERIFIED (rendering under CSP: human item 4) |
| Register behind Turnstile | `turnstile:open|closed` middleware; refusals readable cross-origin | `bouncer.Turnstile`, surf factory; Turnstile 422/503 carry the profile's headers (`TestCORSProfileHeadersOnRefusals/turnstile_*`); boot warning without a secret (`TestTurnstileEmptySecretWarnsAtBoot`) | VERIFIED |
## Goal Achievement
### Observable Truths (roadmap success criteria)
| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| SC-1 | `bouncer` refresh issues a new httpOnly `token` cookie with configurable `Domain`/`SameSite`/`Secure`, blacklists the old JTI; 60-min TTL and 14-day refresh window are config keys; credentialed CORS for a configured SPA origin is available as middleware | ✓ VERIFIED | Cookie half unchanged from the initial run and still green. CORS half: global `http.cors` (initial run) and now `cors:<profile>` decorate every response of the route; verifier probe 11/11 on HEAD, 7/11 FAIL on the pre-fix tree; credentialed wildcard/unanchored origins refused at boot (`TestCORSCredentialedOriginRules`, `TestCORSAnchoredPatternNoSuffixMatch`) |
| SC-2 | Cache module with memory and Redis drivers; Redis-backed `Store` so rate limits hold across instances; Redis client decision note before code | ✓ VERIFIED | Unchanged since the initial run; cooler suites and `TestRateLimitParityDocker`, `TestRedisCacheDocker` passed in the gate; `TestRateLimitAcrossInstancesSmoke` passed separately |
| SC-3 | S3 blob driver selectable through framework config without the app importing it; `/api-docs` + `/api-docs.json` servable; Turnstile middleware available | ✓ VERIFIED | Unchanged artifacts plus plan 06 hardening (WR-06, WR-07, WR-08), each pinned by a required test that passed in the gate (attach tests ran untagged and with `-tags summercms_s3`) |
| SC-4 | Touched modules' READMEs and `docs/` updated; new code has unit tests in the last plan | ✓ VERIFIED | Plan 06 updated the surf, bouncer, lagoon and cooler READMEs and `docs/services/{routing,authentication,storage,caching}.md`, `docs/architecture/request-lifecycle.md`; `TestDocsTree` and `docs:build --check` passed in the gate's docs stage. Plan 06 commits its tests in `8d80580` (Task 6) after the fix commits, so the "unit tests last" rule holds for the gap-closure plan |
**Score:** 4/4 roadmap truths verified. Plan must-haves 44/44 verified (0 present-but-behavior-unverified).
### Plan must-haves
| Plan | Truths | Status | Notes |
|------|--------|--------|-------|
| 01 (cooler, Redis store) | 8 | 8 ✓ | Regression check: cooler required tests and Docker parity passed in this gate run |
| 02 (cookies, CORS, Turnstile, api-docs) | 7 | 7 ✓ | D-05 previously failed, now VERIFIED (CR-01 closed) |
| 03 (S3 tag, DiskExtension, public_url) | 6 | 6 ✓ | Regression check: build and attach tests (both tag modes) and `TestStoreDiskExtension` (Postgres) passed |
| 04 (site-cache purge, plugins, layers) | 7 | 7 ✓ | Regression check: plugin Docker admin suites passed with the queued purge; three layers' shared files still byte-identical (gate `cmp`) |
| 05 (tests, gate) | 8 | 8 ✓ | Gate still fail-closed on required names (lists include the plan 06 tests) |
| 06 (gap closure) | 8 | 8 ✓ | See below |
**Plan 06 truths:**
| # | Truth (abridged) | Status | Evidence |
|---|------------------|--------|----------|
| 1 | CR-01: every response of a `cors:<profile>` route carries ACAO (exact origin), ACAC, `Vary: Origin` for 401/403 (route-before or group), 422/503 Turnstile, 429, 413, panic 500 (JSON and raw); disallowed origin gets `Vary: Origin` only; handler `Vary` merged | ✓ VERIFIED | Verifier probe 11/11 (real HTTP server via `httptest.NewServer`); `TestCORSProfileHeadersOnRefusals` (11 subtests incl. Turnstile 422/503, 403) and `TestCORSProfileVaryKeptWithHandlerVary` passed in the gate |
| 2 | CR-01 gate and docs | ✓ VERIFIED | `scripts/check-phase19.3.sh:80` lists both tests in `SURF_TESTS`; docs lines cited above |
| 3 | D-04 / WR-04: credentialed wildcard-host, unanchored and probe-matching origins fail boot, globally and per profile; non-credentialed configs unchanged | ✓ VERIFIED | `cors.go` `wildcardHostLabel`, `anchoredOriginPattern` (parsed with `regexp/syntax`), `corsProbeOrigins`; `TestCORSCredentialedOriginRules`, `TestCORSAnchoredPatternNoSuffixMatch`, `TestCORSOptionsMatchesLaravel`, `TestCORSStarUnchanged` passed. Verifier probe: `https://*.com`, `https://*`, `https://*example.com` refused, `https://*.example.com` loads. Residual: see Advisory |
| 4 | D-03 / WR-05, WR-09: production refuses Secure resolving to false (set or inferred); zero `CookieConfig` writes Lax + Secure | ✓ VERIFIED | `cookie.go:111` error names `auth.cookie.secure` and `app.url`; `TestLoadCookieConfigProductionInferredSecure`, `TestZeroCookieConfigDefaults`, `TestLoadCookieConfigErrors` passed |
| 5 | D-17, D-12, D-18 / WR-06, WR-07, WR-08 | ✓ VERIFIED | `apidocs.go:65-67,208`; `attach/store.go:42,136` returns `ErrProtectedNeedsPrivateBucket` after the argument checks, before the body is read; boot warning in `bucket.go`; Turnstile secret warning in `BuildRouter`; `TestAPIDocsCSPNoHostSource`, `TestStoreProtectedNeedsPrivateBucket` (both tag modes), `TestTurnstileEmptySecretWarnsAtBoot` passed |
| 6 | D-10, D-11 / WR-02, WR-01: no redirects; queued, coalesced after-commit purges; `Wait`; `Handle.Close` drains; exported names unchanged | ✓ VERIFIED | `sitecache.go:109,115` `CheckRedirect: noRedirect` → `http.ErrUseLastResponse`; `enqueue`/`work`/`Wait` (`sitecache.go:212-275`) keep one request in flight and one pending follow-up; `cooler.go:119` waits in `Close`. `TestSiteCachePurgeNoRedirect` (redirect target receives 0 requests, 307 logged, token not logged), `TestSiteCachePurgeAsyncCoalesced`, `TestSiteCachePurgeWaitAndClose`, Docker `TestSiteCacheBulkCommitCoalesced`, `TestSiteCacheAfterCommitOnly` passed. Plugin commits change only test files (`git show --stat`) |
| 7 | D-11 / WR-03: layer purge route honours `siteCache.allowProxied`; shared files byte-identical with updated pins; caching docs pair remote https `purge_url` with the flag | ✓ VERIFIED | `purge.post.ts` passes `allowProxied` (`=== true` or `'true'`) to `authorizeSitePurge`; sha256 of the shared files is identical in vue-faq-layer, vue-menu-layer, vue-journal-layer; layer TAP 24/37/170 pass, including "allowProxied lets a proxied request reach the token check" |
| 8 | Release precondition: gate `--all` green with Docker and names every new test; disposition records CR-01, WR-01..WR-09 fixed; nothing tagged or pushed before the human reply | ✓ VERIFIED | Gate exit 0 (below). 19.3-REVIEW-DISPOSITION.md: 10 fixed, 11 info open. `git tag --contains 0723536` empty; summercms.go 22 commits ahead of `origin/master`, no `v0.1.28` on origin; plugins and layers each 1 commit ahead of origin with no tag on HEAD |
### Required Artifacts (plan 06)
| Artifact | Expected | Status | Details |
|----------|----------|--------|---------|
| `modules/surf/router.go` | profile outermost; Turnstile boot warning | ✓ VERIFIED | `profileMiddleware` applied at `router.go:483-485` |
| `modules/surf/cors.go` | credentialed origin rules with probe origins | ✓ VERIFIED | `cors-probe.invalid` at `cors.go:92-93` |
| `modules/surf/apidocs.go` | nonce on every script tag | ✓ VERIFIED | 3 tags |
| `modules/bouncer/cookie.go` | production refusal, zero-config defaults | ✓ VERIFIED | "resolves to false" at `cookie.go:111` |
| `modules/lagoon/attach/store.go` | protected-upload refusal | ✓ VERIFIED | |
| `modules/cooler/sitecache.go` | no-redirect client, queue, Wait | ✓ VERIFIED | |
| `../vue-faq-layer/server/lib/siteCachePurge.ts` (and two siblings) | allowProxied opt-in | ✓ VERIFIED | byte-identical |
| `scripts/check-phase19.3.sh` | requires gap-closure tests | ✓ VERIFIED | |
Artifacts from plans 01-05: unchanged status (✓ VERIFIED; `cors.go` upgraded from ⚠️ PARTIAL).
### Key Link Verification (plan 06)
| From | To | Via | Status |
|------|----|-----|--------|
| `surf/router.go` wrap | `surf/cors.go` | `routeCORSProfile(rt)` then `r.corsProfiles[profile].profileMiddleware(h)` after recovery, inside `recordPattern` | WIRED (probe confirms runtime effect) |
| `scripts/check-phase19.3.sh` | `surf/cors_profile_test.go` | `SURF_TESTS` names `TestCORSProfileHeadersOnRefusals` | WIRED (ran in the go stage: 29 surf tests passed) |
| `cooler/sitecache.go` | `lagoon.AfterCommit` | `PurgeSiteCacheAfterCommit` callback only calls `p.enqueue` | WIRED |
| `cooler/cooler.go` | `cooler/sitecache.go` | `Handle.Close` → `h.purger.Wait(ctx)` | WIRED |
| layer `purge.post.ts` | layer `siteCachePurge.ts` | `authorizeSitePurge({..., allowProxied})` | WIRED |
Key links from plans 01-05: unchanged, WIRED.
### Data-Flow Trace (Level 4)
Not applicable: no rendered dynamic data. The runtime flows (cookie → Refresh → blacklist; request → throttle → shared Redis counter; save → after-commit → queued purge POST; refusal → profile CORS headers) are each exercised by a passing test or the verifier probe.
### Behavioral Spot-Checks
| Behavior | Command | Result | Status |
|----------|---------|--------|--------|
| Phase gate, all stages, Docker required | `PHASE193_REQUIRE_DOCKER=1 bash scripts/check-phase19.3.sh --all` | exit 0; PASS self-test, go, plugins, layers, docs, docker ("Phase 19.3 gate passed") | ✓ PASS |
| CR-01 on HEAD (`cors:<profile>`) | scratch module outside the repo (`replace` to the repo), `surf.Assemble` + `httptest.NewServer`, profile `spa` credentialed for `https://www.example.com` | `[deny,cors:spa]` 401, `[cors:spa,deny]` 401, group `Use(deny)` + route `cors:spa` 401, group `Use(cors:spa)` > inner group `Use(deny)` 401, handler panic 500, `[boom,cors:spa]` middleware panic 500, raw-group panic 500, `body.limit:16` 64-byte POST 413, throttle 200 then 429: all ACAO=`https://www.example.com`, ACAC=`true`, Vary=`Origin`. Disallowed origin 401: no ACAO/ACAC, Vary=`Origin`. ALL PASS | ✓ PASS |
| Same probe on the pre-fix tree (`git archive 0723536^`) | identical probe, `replace` to the extracted tree | 7/11 FAIL (route-before 401, group 401, all three 500s, 429, disallowed-origin Vary) | discriminating, as expected |
| Credentialed wildcard validation | scratch module calling `surf.LoadCORSConfig` | `https://*.example.com` ok; `https://*.com`, `https://*`, `https://*example.com` refused; `https://*.co.uk` ok (Advisory) | ✓ PASS (contract) |
| Cross-instance rate limit and cookie refresh smokes | `go test -count=1 -run '^(TestRateLimitAcrossInstancesSmoke\|TestCookieRefreshAcrossSubdomainsSmoke\|TestCookieRefreshHandlerRotatesAndBlacklists)$' ./modules/surf ./modules/bouncer` | all PASS (real Redis container) | ✓ PASS |
| Regression, whole module (once) | `go test -short -count=1 ./...` | all packages ok | ✓ PASS |
| examples/hello workspace module | `cd examples/hello && go test -count=1 ./...` | ok | ✓ PASS |
### Probe Execution
No `scripts/*/tests/probe-*.sh` exist and no plan declares one. The phase gate was run by the verifier in its own process (above).
### Requirements Coverage
REQUIREMENTS.md maps no IDs to phase 19.3 (no orphans). Phase-local IDs from PLAN frontmatter (plans 01-06):
| ID | Source Plans | Status | Evidence |
|----|--------------|--------|----------|
| SC-1 | 02, 05, 06 | ✓ SATISFIED | cookie tests; CR-01 closed |
| SC-2 | 01, 05 | ✓ SATISFIED | cooler, Redis store, note ordering |
| SC-3 | 02, 03, 05, 06 | ✓ SATISFIED | S3 tag, api-docs (nonce-only CSP), Turnstile |
| SC-4 | 01-06 | ✓ SATISFIED | READMEs, docs, tests last in each plan |
| D-01, D-02 | 02, 05 | ✓ SATISFIED | cookie tests |
| D-03 | 02, 05, 06 | ✓ SATISFIED | plus WR-05/WR-09 hardening |
| D-04 | 02, 05, 06 | ✓ SATISFIED | WR-04 fixed; public-suffix residual is Advisory |
| D-05 | 02, 05, 06 | ✓ SATISFIED | CR-01 closed |
| D-06, D-07, D-08, D-09 | 01, 05 | ✓ SATISFIED | |
| D-10, D-11 | 04, 05, 06 | ✓ SATISFIED | WR-01/02/03 fixed; live purge is human item 3 |
| D-12 | 03, 05, 06 | ✓ SATISFIED | WR-07 fixed |
| D-13, D-14, D-15 | 03, 05 | ✓ SATISFIED | real bucket is human item 2 |
| D-16, D-17 | 02, 05, 06 | ✓ SATISFIED | WR-06 fixed; rendering under CSP is human item 4 |
| D-18 | 02, 05, 06 | ✓ SATISFIED | WR-08 fixed |
| D-19 | 01-04 | ✓ SATISFIED | placement matches (cooler / surf / bouncer / lagoon/attach) |
### Prohibitions (all judgment-tier; non-authoritative verifier verdicts)
Plans 01-05: verdicts from the initial run still hold (cooler imports, bouncer imports, Flush scope, byte-identical `*` CORS, Turnstile secret never logged, no new `s3.*` keys, Journal/menu/FAQ compatibility, test network scope).
| Plan | Prohibition | Verdict | Evidence |
|------|-------------|---------|----------|
| 06 | No breaking change to Journal, menu, FAQ: exported cooler purge names/signatures, Journal aliases, plugin controllers unchanged; only plugin test files change | holds | plugin commits `e52b286`, `4b42978`, `4f633f0` touch only `*_test.go`; `PurgeSiteCacheAfterCommit` and `SiteCachePurger.Purge` keep their signatures (`Wait` is additive); Journal alias tests passed. Note: the plugin tests now call `SiteCachePurger.Wait`, so the plugins need the v0.1.28 require bump before they build on origin without go.work (recorded in the SUMMARY handoff) |
| 06 | Non-credentialed `*` CORS byte-identical | holds | `TestCORSOptionsMatchesLaravel`, `TestCORSStarUnchanged` passed |
| 06 | Framework code and docs never name a consuming application | holds | gate docs stage: "no consuming application named in framework lines since 2d352ff^" |
| 06 | Nothing pushed or tagged before the human's reply | holds | no tag contains `0723536`; no `v0.1.28` on origin; plugins and layers 1 commit ahead with untagged HEADs |
| 06 | IN-01..IN-11 not addressed | holds | disposition shows all 11 `open` |
### Anti-Patterns Found
No `TBD`/`FIXME`/`XXX`/`TODO`/`HACK` markers in files changed by plan 06 (the only match is the gate's TAP `SKIP|TODO` directive parser at `scripts/check-phase19.3.sh:261`). No stubs: every plan 06 change is exercised by a named test that ran.
| File | Finding | Severity | Impact |
|------|---------|----------|--------|
| `modules/surf/cors.go` | Credentialed `https://*.co.uk`-style wildcards (multi-label public suffix) still load | 📋 Advisory | Needs a deliberate misconfiguration; outside plan 06's stated contract. See frontmatter `advisory` |
| 19.3-REVIEW.md IN-01..IN-11 | Info findings, open by design | ℹ️ Info | recorded in 19.3-REVIEW-DISPOSITION.md |
| deferred-items.md | S3 object Content-Type from sniff; Redis `Remember` outage latency; pre-existing gofmt/naming in older files | ℹ️ Info | follow-ups, unchanged |
| 19.3-VALIDATION.md | still `status: draft`, `nyquist_compliant: false` | ℹ️ Info | `/msd-validate-phase 19.3` can settle it |
### Advisory (New Scope, Unevidenced)
| # | Finding | Category | Why Advisory |
|---|---------|----------|--------------|
| 1 | Credentialed wildcard on a multi-label public suffix (`https://*.co.uk`) passes boot validation | security | new scope relative to the carried-forward gap; only scratch-probe evidence; plan 06's contract (probe origins, `https://*.com`) holds |
### Human Verification Required
1. **Cookie refresh across real subdomains.** Deploy to staging with `auth.cookie.domain` set to the parent domain and `cors:<profile>` (or `http.cors`) credentialed for the www origin. Log in on www and wait past the access TTL.
- Expected: the SPA reads the 401, the refresh call sends the cookie, the rotated cookie replaces it.
- Why human: needs real DNS and TLS, and a browser's cookie and CORS enforcement.
2. **Real S3/R2/MinIO upload.** Build with `build.tags: [summercms_s3]`, set `bucket_url: s3://...?endpoint=...`, upload, fetch via `public_url` and via `/storage/uploads`; try a protected upload with `public_url` set.
- Expected: both URLs serve the public file; the protected upload is refused.
- Why human: needs bucket credentials.
3. **Live Nuxt purge.** Save a menu entry and a FAQ entry in admin against a site on the new layer versions (set `NUXT_SITE_CACHE_ALLOW_PROXIED=true` behind a proxy).
- Expected: the page updates without waiting for the 300 s SWR TTL; the purge is logged after the save returned.
- Why human: needs a running Nuxt site and the app.
4. **Swagger UI under the nonce-only CSP.** Enable `http.api_docs`, open `/api-docs`.
- Expected: the UI renders the spec with no CSP violations in the console.
- Why human: browser rendering; WR-06 tightened `script-src` since the initial verification.
The initial report's item 5 (triage WR-01..WR-09) is settled: all ten CR/WR findings are `fixed` in 19.3-REVIEW-DISPOSITION.md.
### Gaps Summary
No gaps. The one gap from the initial verification (SC-1 / D-05, CR-01) is closed in code, docs and gate. The verifier reproduced the earlier failure scenarios in a scratch module outside the repo: every case now carries the profile's headers on HEAD, and the same probe still fails on the pre-fix tree, so it actually tests the fix. The full gate passes with Docker required, and the earlier must-haves show no regression. The status is `human_needed` because four checks need real infrastructure or a browser. The release steps in the 19.3-06 SUMMARY handoff remain for the phase session: whole-phase docs, the v0.1.28 tag, the plugin require bumps and the layer tags. Until the plugins move to v0.1.28, their tests do not build on origin without go.work.
---
_Verified: 2026-10-15T21:08:52Z_
_Verifier: Claude (msd-verifier)_

View File

@@ -0,0 +1,38 @@
# API Coverage — Cloudflare Turnstile siteverify, Redis (go-redis v9), S3 (gocloud s3blob)
> Full coverage by default. Opt-outs are explicit, reasoned decisions.
Scope: the three external services phase 19.3 integrates. Turnstile siteverify is called by `bouncer.Turnstile` (plan 19.3-02, D-18) with the PHP `TurnstileVerifier` semantics as the contract. Redis is reached through go-redis v9.23.0 from `cooler` (plan 19.3-01, D-07..D-09). S3 and S3-compatible stores are reached through `gocloud.dev/blob/s3blob` behind the `summercms_s3` build tag (plan 19.3-03, D-13, D-14). Capability names carry the service as a prefix so they stay unique in one matrix.
| capability | decision | reason |
|---|---|---|
| turnstile: siteverify POST as form fields secret and response | INTEGRATE | same transport as PHP (Http::asForm) |
| turnstile: remoteip field from the trusted client IP | INTEGRATE | surf.ClientIP with http.trusted_proxies, as PHP passes $request->ip() |
| turnstile: 5 s request timeout, context-aware | INTEGRATE | D-18; applied to both modes |
| turnstile: read success from the answer (success === true) | INTEGRATE | |
| turnstile: error-codes read for the log line (never the token or secret) | INTEGRATE | diagnostics only; the verdict stays success-based |
| turnstile: fail-open mode (empty secret passes) | INTEGRATE | D-18 registration path |
| turnstile: fail-closed mode (503 unconfigured in production, 422 on missing token, false, error or timeout) | INTEGRATE | D-18 guest entry path |
| turnstile: reject a token longer than 2048 characters without a call | INTEGRATE | Cloudflare's documented token limit; saves a siteverify call |
| turnstile: idempotency_key for retried verification | OPT-OUT | the verifier makes exactly one attempt per request and never retries, so there is nothing to deduplicate |
| turnstile: hostname check of the answer | OPT-OUT | not in the PHP contract; the site key is bound to its hostnames in the Cloudflare dashboard |
| turnstile: action check of the answer | OPT-OUT | not in the PHP contract; the widgets of the ported clients set no action |
| turnstile: cdata and challenge_ts fields | OPT-OUT | not in the PHP contract; tokens are single-use and expire after 300 s on Cloudflare's side |
| turnstile: JSON request body instead of form | OPT-OUT | form encoding is what PHP sends; Cloudflare accepts both |
| redis: connection from redis.url (redis://, rediss:// TLS, unix://, ACL user and password in the URL) | INTEGRATE | go-redis ParseURL; the URL never appears in an error or log |
| redis: client handshake without CLIENT SETINFO and without maintenance notifications | INTEGRATE | DisableIdentity and maintnotifications ModeDisabled (RESEARCH Pitfall 14) |
| redis: PING at serve start-up with a 3 s timeout | INTEGRATE | an unreachable server is a warning, never a boot failure (user answer: memory fallback) |
| redis: GET, SET with PX, EXISTS, UNLINK, INCRBY for the cache | INTEGRATE | D-08 Get, Set, Has, Forget, Increment |
| redis: SCAN MATCH under the cache prefix plus UNLINK for Flush | INTEGRATE | Flush never empties the whole database (rate-limit counters and other apps share it) |
| redis: EVALSHA with EVAL fallback for the fixed-window rate-limit script | INTEGRATE | go-redis NewScript and Run; probed on Redis 7.2.5 |
| redis: Sentinel and Cluster clients | OPT-OUT | not needed yet: one redis.url names one server; a later phase adds them if a deployment needs them |
| redis: pub/sub, streams and locks | OPT-OUT | explicitly out of scope: D-08 excludes tags and locks until a ported plugin needs them |
| redis: Redis-backed bouncer.BlacklistStore | OPT-OUT | deferred in CONTEXT (optional, not required by the success criteria) |
| s3: open s3://bucket through gocloud URL query (region, endpoint, use_path_style, disable_https) | INTEGRATE | D-14; no parallel s3.* key set |
| s3: request_checksum_calculation and response_checksum_validation query parameters | INTEGRATE | passed through gocloud for S3-compatible stores (R2, MinIO, Hetzner); documented |
| s3: credentials from the standard AWS environment and shared config (profile, role) | INTEGRATE | D-14; never in summer config |
| s3: server-side encryption (ssetype, kmskeyid) and accelerate, dualstack, fips | INTEGRATE | native gocloud URL parameters; no framework code, documented only |
| s3: object write, read, delete and list through blob.Bucket | INTEGRATE | the existing attach paths (Store, StaticHandler, DeleteKeys) run unchanged on the s3 driver |
| s3: public object URLs through storage.uploads.public_url | INTEGRATE | D-12 |
| s3: presigned GET URLs | OPT-OUT | not needed: public files use public_url or the /storage/uploads proxy route (D-12) |
| s3: bucket creation and lifecycle or CORS configuration | OPT-OUT | operator-managed outside the application |

View File

@@ -0,0 +1,20 @@
# Phase 19.3 deferred items
Out-of-scope findings logged by executors. Not fixed in the plan that found them.
## From 19.3-03
- `internal/build/registry.go` is not gofmt-clean (const block alignment, `registryFileName` / `generatedMark`). Pre-existing, untouched by 19.3-03; `go vet` is unaffected.
- The 19.3-03 Task 3 acceptance grep `grep -rniE 'wavepath|fonoteka|grzyby' modules/lagoon ...` matches pre-existing test files outside the plan's scope: `modules/lagoon/hidden_marshal_test.go` (comments naming the consumer app), `modules/lagoon/migrations_test.go` (`golem15.fonoteka` as a sample plugin ID) and `modules/lagoon/attach/lifecycle_test.go` (`Golem15\Fonoteka\Models\Album` morph type). CLAUDE.md says framework code never names a consuming application; a follow-up quick task could switch these to neutral names (`acme`). 19.3-03 added no such names.
## From 19.3-04
- Pre-existing gofmt findings (missing trailing newlines), untouched by 19.3-04: `modules/cabana/account_tokens.go`, `modules/cabana/account_tokens_test.go`, `modules/cabana/token_postgres_smoke_test.go`, `modules/cabana/token_routes_test.go`, `modules/cabana/token_scopes_test.go`, and in the Journal plugin `classes/blog_editor_test.go`. `go vet` and the suites are unaffected.
- Deployment follow-up: move the grzybyfunkcjonalne deployment from `golem15.journal.site_cache.*` and `NUXT_JOURNAL_CACHE_PURGE_TOKEN` to `site_cache.purge_url`/`site_cache.purge_token` and `NUXT_SITE_CACHE_PURGE_TOKEN` (route `/api/__site__/cache/purge`) before the Journal alias keys are removed.
- `sm-summercmsio-app` mounts the Journal plugin 45 commits behind main; bump its pointer in a separate follow-up.
## From 19.3-05
- Security observation (not a test failure; plan 03 kept the stored `content_type` rule by design): `attach.Store` writes the blob with the sniffed content type as object metadata, so an HTML body uploaded as `note.txt` is stored as `<id>.txt` with `Content-Type: text/html`, and a generic sniff takes the type from the extension (`app.js` with binary bytes is stored as `.bin` with `text/javascript`). The surf uploads route neutralises this (nosniff, sandbox CSP, attachment disposition), but with `storage.uploads.public_url` pointing at an S3 bucket the bucket serves the object's stored Content-Type directly, so such a file renders as HTML on the bucket or CDN host. `docs/services/storage.md`'s deploy note says a CDN "picks the Content-Type from the file extension", which is not true for S3 object metadata. A follow-up could write a non-rendering content type (or `Content-Disposition: attachment`) for non-inline types to the blob metadata, and correct the deploy note. Pinned by `TestStoreDiskExtension` rows "html named txt" and "js under octet-stream".
- Redis outage latency (follow-up from 19.3-01 now pinned by tests): `TestRateLimitFallbackOnError` asserts a rate-limit attempt against a stopped Redis answers from the fallback within `AttemptTimeout` (about 500 ms per throttled request during an outage). `Cache.Remember` on the Redis driver has no internal bound: against a stopped Redis it waits for go-redis's dial retries (about 1.7 s, five dial attempts) unless the caller's context has a deadline, and go-redis logs `failed to dial after 5 attempts` to stderr on every failing call, outside slog and the once-per-minute warner. A follow-up could bound Remember's read and write (as Attempt does), lower `MaxRetries`/dial timeout in `OpenRedis`, and route go-redis's logger through slog.
- Coverage note: the plugins' `purgeSiteCache` branch "inside a save transaction but no purger configured" cannot be reached from a unit test, because cabana has no exported way to put a transaction in a context (`withTx` is internal). The plugins' Docker admin suites run every save through it without purge keys; a test-only helper in cabana (like `cabana.WithFieldWrites`) would let a unit test reach it.

View File

@@ -99,6 +99,7 @@ The `migrate`, `migrate:status`, `migrate:rollback`, `serve` and admin commands
| [cabana](modules/cabana/README.md) | Schema-driven admin backend that compiles WinterCMS-style YAML list, form, filter and relation definitions at boot and serves them as a JSON admin API next to the embedded admin SPA. | The beach hut where the staff run the beach; it is the admin backend. |
| [compass](modules/compass/README.md) | Layered YAML configuration with per-environment directories, `SUMMER_` environment overrides, embedded plugin defaults and dot-path access. | Points you the right way; configuration tells the application how to behave in each environment. |
| [conga](modules/conga/README.md) | Background jobs on River over the shared Postgres pool: transactional dispatch, a `summer_jobs` progress record, in-process or dedicated workers, and a wall-clock scheduler. | A line of dancers moving one after another; jobs queue up and are worked through in order. |
| [cooler](modules/cooler/README.md) | Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, a Redis-backed rate-limit store, and the site-cache purge after admin saves. | The cool box you bring to the beach; it keeps things fresh and at hand for everyone in the group. |
| [festival](modules/festival/README.md) | Typed, synchronous event bus with listener priorities, payload collection and stop-when-handled dispatch. | A summer festival announces events and the crowd reacts; it is the event bus. |
| [fetchguard](modules/fetchguard/README.md) | Guarded outbound HTTP client and fetcher that blocks private and reserved addresses, enforces host, size and timeout limits, and offers an explicit trusted mode for operator-configured endpoints. | A lifeguard for outbound fetches; it watches every request the application sends out and stops the dangerous ones. |
| [flare](modules/flare/README.md) | Web Push delivery with VAPID (RFC 8292) and aes128gcm payload encryption (RFC 8291) behind a small Pusher interface. | A signal flare reaches people far away; it sends Web Push notifications to browsers. |

View File

@@ -69,7 +69,7 @@ There is no `go.sum` in the application root: builds go through the workspace, a
## The summer.yaml manifest
The manifest in the application root names the application module, the binary `summer build` writes to `bin/`, and the plugins in activation order:
The manifest in the application root names the application module, the binary `summer build` writes to `bin/`, the plugins in activation order and, optionally, Go build tags:
```yaml
module: git.golem15.com/acme/acme
@@ -79,10 +79,15 @@ plugins:
module: git.golem15.com/acme/acme/plugins/user
- id: acme.blog
module: git.golem15.com/acme/acme/plugins/blog
build:
tags:
- summercms_s3
```
`summer build` turns this list into `plugins.gen.go`. The order is the manifest order, adjusted so that each plugin comes after the plugins it requires. Plugins with no dependency between them keep the order you wrote.
`build.tags` lists Go build tags that `summer build` (and `summer dev`) passes to `go build` as `-tags=<tag>,<tag>`, printing them as `build tags: …`. Each tag must consist of lower-case letters, digits and underscores; a duplicate is ignored. Framework features that pull in large dependencies sit behind a tag, such as `summercms_s3` for S3 uploads buckets (see [Storage](../services/storage.md#s3-and-s3-compatible-buckets)). `summer plugin:add` keeps the list when it rewrites the manifest. A script that runs its own `go build` must pass the same tags.
## Local plugins in a workspace
A plugin you develop inside the application lives in `plugins/<name>` as its own module. `summer make:plugin acme.blog` creates it there, and `summer plugin:add plugins/blog` registers it:

View File

@@ -24,12 +24,12 @@ Each framework module is one Go package under `modules/`, documented by its READ
| Concern | Modules |
|---------|---------|
| Plugins and the container | [party](../../modules/party/README.md), [pact](../../modules/pact/README.md), [backpack](../../modules/backpack/README.md), [festival](../../modules/festival/README.md), [compass](../../modules/compass/README.md) |
| Plugins and the container | [party](../../modules/party/README.md), [pact](../../modules/pact/README.md), [backpack](../../modules/backpack/README.md), [festival](../../modules/festival/README.md), [compass](../../modules/compass/README.md), [signpost](../../modules/signpost/README.md) |
| HTTP | [surf](../../modules/surf/README.md), [towel](../../modules/towel/README.md), [wire](../../modules/wire/README.md), [bouncer](../../modules/bouncer/README.md), [wristband](../../modules/wristband/README.md), [fetchguard](../../modules/fetchguard/README.md) |
| Data | [lagoon](../../modules/lagoon/README.md), [beachcomber](../../modules/beachcomber/README.md) |
| Admin | [cabana](../../modules/cabana/README.md), [boardwalk](../../modules/boardwalk/README.md) |
| Services | [phrasebook](../../modules/phrasebook/README.md), [postcard](../../modules/postcard/README.md), [conga](../../modules/conga/README.md), [lighthouse](../../modules/lighthouse/README.md), [flare](../../modules/flare/README.md), [sunscreen](../../modules/sunscreen/README.md) |
| Console and tooling | [bonfire](../../modules/bonfire/README.md), [tide](../../modules/tide/README.md) |
| Admin | [cabana](../../modules/cabana/README.md), [boardwalk](../../modules/boardwalk/README.md), [seaglass](../../modules/seaglass/README.md) |
| Services | [cooler](../../modules/cooler/README.md), [phrasebook](../../modules/phrasebook/README.md), [postcard](../../modules/postcard/README.md), [conga](../../modules/conga/README.md), [lighthouse](../../modules/lighthouse/README.md), [flare](../../modules/flare/README.md), [sunscreen](../../modules/sunscreen/README.md) |
| Console and tooling | [bonfire](../../modules/bonfire/README.md), [tide](../../modules/tide/README.md), [sunrise](../../modules/sunrise/README.md) |
| Utilities | [sieve](../../modules/sieve/README.md) |
## What runs where

View File

@@ -61,14 +61,14 @@ Some costs come over with the port unchanged, and a few framework behaviours nee
- Garbage collection pauses are sub-millisecond and do not matter at this scale.
> [!WARNING]
> Rate limits are counted per process. The throttle middleware that `serve` builds keeps its counters in memory (`surf.MemoryStore`, behind the `surf.Store` interface), and `serve` offers no way to supply a different store yet. With N replicas the effective limit is N times the configured one. Until a shared store exists, divide the per-route limits by the replica count, or enforce the limit at the load balancer. [Rate limiting](../services/rate-limiting.md#named-buckets) describes the buckets.
> Rate limits are counted per process by default. The throttle middleware keeps its counters in memory (`surf.MemoryStore`, behind the `surf.Store` interface), so with N replicas the effective limit is N times the configured one. Set `http.rate_limit.store: redis` and `redis.url`, and `serve` counts in Redis through [cooler](../../modules/cooler/README.md)'s `cooler.RateLimitStore`, shared by every replica; see [Sharing limits across instances](../services/rate-limiting.md#sharing-limits-across-instances).
## Scaling and operations
The application is one stateless binary. It starts in milliseconds, ships in a small container image and needs no OPcache warm-up after a deploy.
- Vertical scaling is more cores. The Go runtime uses them all, so there is no worker count to tune.
- Horizontal scaling is N replicas behind a load balancer that share Postgres. Uploads must be shared too: point `storage.uploads.bucket_url` at a `file://` directory on storage that every replica mounts (see [Storage](../services/storage.md#bucket-urls)). Apply the blacklist and rate-limit notes above before you add the second replica.
- Horizontal scaling is N replicas behind a load balancer that share Postgres. Uploads must be shared too: point `storage.uploads.bucket_url` at a `file://` directory on storage that every replica mounts, or at an S3 or S3-compatible bucket (`s3://`, in a binary built with the `summercms_s3` tag; see [Storage](../services/storage.md#s3-and-s3-compatible-buckets)). Apply the blacklist and rate-limit notes above before you add the second replica.
- Centrifugo scales on its own. [lighthouse](../../modules/lighthouse/README.md) only publishes to it and issues connection tokens, so adding application replicas does not change the realtime server; see [The Centrifugo driver](../services/realtime.md#the-centrifugo-driver).
A rollout runs the migrations once, then starts each replica behind the reverse proxy:

View File

@@ -37,12 +37,13 @@ surf wraps every non-raw route in the same layers. From the outside in:
1. Optional locale-prefix rewrite, when a `surf.LocaleResolver` is published: an enabled locale as the first path segment is stripped so later layers and routing see the unprefixed path.
2. CORS, only for the paths configured under `http.cors.paths`, including preflight requests.
3. Panic recovery. A panic becomes an opaque JSON 500 from [wire](../../modules/wire/README.md), and because the response is buffered until the handler returns, the client never receives half a body.
4. The request locale, taken from the published `surf.LocaleResolver` when one exists, or from the `Accept-Language` header otherwise, and stored in the context.
5. The body limit: `http.body_limits.default_bytes`, or the route's own `body.limit:<bytes>`.
6. The route's middleware, in the order you listed them: group middleware first, then the route's own.
7. The path constraints. A request whose parameter fails `pact.Router.Where` or `pact.Router.WhereIn` gets a 404 before your handler runs.
8. Your handler.
3. The route's CORS profile, when it or its group names `cors:<profile>`. It sits here whatever its position in the middleware list, so refusals and recovered panics carry its headers too.
4. Panic recovery. A panic becomes an opaque JSON 500 from [wire](../../modules/wire/README.md), and because the response is buffered until the handler returns, the client never receives half a body.
5. The request locale, taken from the published `surf.LocaleResolver` when one exists, or from the `Accept-Language` header otherwise, and stored in the context.
6. The body limit: `http.body_limits.default_bytes`, or the route's own `body.limit:<bytes>`.
7. The route's middleware, in the order you listed them: group middleware first, then the route's own (`cors:<profile>` excepted, see step 3).
8. The path constraints. A request whose parameter fails `pact.Router.Where` or `pact.Router.WhereIn` gets a 404 before your handler runs.
9. Your handler.
Raw groups, declared with `pact.Router.GroupRaw`, are for webhooks and file streams. They skip the default body limit, refuse house middleware, and a panic in them returns a bare 500.

View File

@@ -215,6 +215,7 @@ Behaviour overrides such as `formBeforeCreate` or `listExtendQuery` become optio
| `pact.FormBeforeCreate`, `pact.FormAfterCreate` | Around a create, inside its transaction. |
| `pact.FormBeforeUpdate`, `pact.FormAfterUpdate` | Around an update, inside its transaction. |
| `pact.FormBeforeDelete`, `pact.FormAfterDelete` | Around a delete, inside its transaction. |
| `pact.ReorderAfterSave` | After a reorder save wrote the new order, inside its transaction, with the request's partition (nil without one). An error rolls the reorder back. See [Tree reorder](lists-and-filters.md#tree-reorder). |
| `pact.DropdownOptionsProvider` | Supplies the options of a `dropdown` field that names a method. |
A hook or scope that has to read the database during a write should use the transaction the write runs in, `cabana.TxFromContext(ctx)`, rather than the application pool: it then sees the write's own snapshot and takes no second connection while the transaction holds row locks. The handle is valid only until the hook returns, and the list route, which runs no transaction, reports none.

View File

@@ -223,6 +223,8 @@ A save posts `{partition, nodes: [{id, parent_id, sort_order}]}` to `POST .../re
- A duplicate id, a parent outside the set, a cycle and a node deeper than `maxDepth` are 422 `validation_failed` on `nodes`; a partitioned page without `partition` is a 422 on `partition`.
- Otherwise the parent and sort columns of every node are written, without running the model's save hooks, and the answer is the new tree.
The model's save hooks do not run on a reorder, and neither do the controller's `pact.FormAfterUpdate` hooks. A controller that has to react to a new order implements `pact.ReorderAfterSave`: `ReorderAfterSave(ctx, partition)` runs after every node was written, inside the same transaction (`cabana.TxFromContext` returns it), with the posted partition, or nil when the reorder has none. A returned error rolls the whole reorder back. The transaction is a `lagoon.Transaction`, so work registered with `lagoon.AfterCommit` runs only once the new order is committed; purging a site's page cache after a menu reorder is the typical case (`cooler.PurgeSiteCacheAfterCommit`, see [Site cache purge](../services/caching.md#site-cache-purge)).
Both routes need the controller's permissions; the save is a write route (CSRF header, and a personal token needs a scope that allows POST).
In the admin, a list whose schema carries `reorder` shows a "Reorder records" button before "New record". It opens the reorder page, and when the list is filtered on the partition column it opens that partition. The page shows the tree with a drag handle and move up, move down, outdent and indent buttons on every row. A row can also be dragged before, after or into another row (never into its own subtree), and the arrow keys on a focused handle move it the same way as the buttons. There is no Save button: 400 ms after the last move the page posts the whole set, shows "Order saved", and sends a pending save at once when the administrator leaves. Only one save is in flight at a time: moves made while it runs are sent after it settles, with the tree as it is then, so the stored order never ends older than the screen. A refused save puts the last saved order back, unless a later save is waiting (it carries the whole tree); a 409 reloads the tree. A save that settles after the page switched partition does not touch the new partition's tree. A partitioned page has a selector for the partition.

View File

@@ -12,8 +12,8 @@ The `summer` tool creates and builds applications and generates the files a plug
| Command | Purpose |
|---------|---------|
| `summer build` | Reads `summer.yaml`, generates `plugins.gen.go` and `main.go`, and builds the binary into `bin/`. Run it from the application directory or any directory below it. |
| `summer dev` | Builds the application, starts the binary, and rebuilds and restarts it whenever a Go or YAML source, `go.mod`, `go.work`, `.env` or `summer.yaml` changes. |
| `summer build` | Reads `summer.yaml`, generates `plugins.gen.go` and `main.go`, and builds the binary into `bin/`, passing the manifest's `build.tags` to `go build` as `-tags`. Run it from the application directory or any directory below it. |
| `summer dev` | Builds the application as `summer build` does (build tags included), starts the binary, and rebuilds and restarts it whenever a Go or YAML source, `go.mod`, `go.work`, `.env` or `summer.yaml` changes. |
```sh
summer build
@@ -44,7 +44,7 @@ The application gets these files:
|------|----------|
| `go.mod` | Module `git.golem15.com/golem15/sm-<name>-app`, requiring the framework through a relative `replace`. |
| `go.work` | The workspace with the application and its plugin modules, as relative paths. It is committed. |
| `summer.yaml` | The manifest: module path, binary name and plugin list. |
| `summer.yaml` | The manifest: module path, binary name, plugin list and optional `build.tags` (for example `summercms_s3` for an S3 uploads bucket). |
| `config/` | `app.yaml`, `database.yaml`, `http.yaml` and `storage.yaml`, enough for `serve` and `route:list`. |
| `.env`, `.env.example` | The database DSN, a freshly generated application key and a random admin token secret (`SUMMER_ADMIN__JWT__SECRET`). `.env` is readable by its owner only and never committed; `.env.example` lists the variables without the key and the secret. |
| `compose.yaml` | PostgreSQL 16 on `127.0.0.1:54320`, with credentials matching `.env`. |

View File

@@ -82,7 +82,7 @@ fmt.Println(url)
// /storage/uploads/5f1/d0c/2e9/thumb_7_200_200_0_0_crop.jpg
```
URLs start with `storage.uploads.public_path_prefix` (`/storage/uploads` by default). `attach.File.URL` returns the URL of the original, the path WinterCMS's `File::getPath()` returns, and `attach.PublicURL` the URL of any blob key; [Storage](../services/storage.md#the-wintercms-layout) shows the configuration that reproduces WinterCMS's URLs exactly.
URLs start with `storage.uploads.public_path_prefix` (`/storage/uploads` by default), or with `storage.uploads.public_url` when a public bucket or CDN serves the files (see [Public URL base](../services/storage.md#public-url-base)). `attach.File.URL` returns the URL of the original, the path WinterCMS's `File::getPath()` returns, and `attach.PublicURL` the URL of any blob key; [Storage](../services/storage.md#the-wintercms-layout) shows the configuration that reproduces WinterCMS's URLs exactly.
Originals in JPEG, PNG, GIF and WebP can be thumbnailed. The thumbnailer cannot write WebP, so the thumbnail of a `.webp` original holds JPEG bytes under the original's `.webp` name, and it is stored with the `image/jpeg` content type. Before decoding, the thumbnailer reads the image size from the file header, so an image larger than 4096 by 4096 pixels is never decoded. As WinterCMS's `File::makeThumb` does, an original that is missing, does not decode or is too large gets WinterCMS's 200 by 200 broken-image picture (`attach.BrokenImagePNG`) stored as its thumbnail, and the reason is logged at warn level: one unusable upload never makes the pages that list it fail. `attach.StaticHandler` serves originals and thumbnails under that prefix; `attach.StaticHandlerPublic` does the same and answers 404 for a row whose `is_public` flag is false. Mount the gated handler when a bucket holds any private file.
@@ -93,7 +93,9 @@ Originals in JPEG, PNG, GIF and WebP can be thumbnailed. The thumbnailer cannot
`attach.Store` turns an upload into a `system_files` row. It takes an `attach.Upload` (the client's file name, the body and the public flag) and an `attach.Limits`, writes the bytes under the WinterCMS partition key and inserts an unattached row whose `sort_order` equals its id, as WinterCMS's Sortable trait does. The caller attaches the row later by setting `Field`, `AttachmentType` and `AttachmentID`, usually when the record's form is saved. When the row insert fails, the blob is deleted again.
The client file name never reaches a blob key. The disk name is 22 random hexadecimal characters plus the lower-cased extension, and the extension must be one to ten letters or digits from the allowed list: `Limits.Extensions`, or, when that is empty, `attach.DefaultImageExtensions` in image mode (jpg, jpeg, png, gif and webp) and `attach.DefaultFileExtensions` otherwise. The file list is WinterCMS's default list without the types that can carry script: svg, js, map, css, less, scss, swf and xml. A refused extension returns `attach.ErrFileType`.
The client file name never reaches a blob key. The disk name is 22 random hexadecimal characters plus an extension, and the client's lower-cased extension must be one to ten letters or digits from the allowed list: `Limits.Extensions`, or, when that is empty, `attach.DefaultImageExtensions` in image mode (jpg, jpeg, png, gif and webp) and `attach.DefaultFileExtensions` otherwise. The file list is WinterCMS's default list without the types that can carry script: svg, js, map, css, less, scss, swf and xml. A refused extension returns `attach.ErrFileType`.
The disk name's extension follows the content, not the client's name, because a web server or CDN in front of the bucket picks the Content-Type from the extension. `attach.DiskExtension` derives it from the sniffed type and the client's extension. For a specific type such as PNG, JPEG, PDF, MP3 or WebM, the client's extension is kept when it names that type, else the type's usual extension is used: a PNG uploaded as `x.html` (with `html` allowed) is stored as `<hex>.png`, and its URL ends in `.png`. For a generic sniff (plain text, HTML, XML, a zip container or unknown bytes) the client's extension is kept, so `report.docx` stays `.docx` and an HTML body uploaded as `note.txt` stays `.txt`, unless the extension is one a browser renders or runs (html, svg, xml, js, css, php and the like), which becomes `.txt` for text and `.bin` otherwise. `file_name` always keeps the client's name, so downloads still carry it.
The content type is sniffed from the first bytes of the body, never taken from the client. `Limits.MIMETypes` narrows it further: an entry with a slash is a MIME pattern such as `image/png` or `image/*`, an entry without one is an extension, and a file that matches no entry returns `attach.ErrMIMEType`. `Limits.MaxBytes` is enforced while the body streams into the bucket; a body one byte over the limit returns `attach.ErrTooLarge` and leaves no blob and no row behind.

View File

@@ -12,10 +12,10 @@ Start with [Installation](setup/installation.md) to set up the toolchain and the
## Where to start
- [Setup](setup/introduction.md): what SummerCMS is, installation, configuration, the move from WinterCMS and a [plugin porting walkthrough](setup/porting-a-plugin.md).
- [Setup](setup/introduction.md): what SummerCMS is, installation, configuration, the move from WinterCMS, a [plugin porting walkthrough](setup/porting-a-plugin.md) and [Upgrading](setup/upgrading.md) to a newer release, with what to check before 0.2.
- [Architecture](architecture/introduction.md): the single binary, Go modules, the application lifecycle, the request lifecycle and [performance and scaling](architecture/performance-and-scaling.md) compared with PHP-FPM.
- [Plugins](plugins/registration.md): registering a plugin, scheduling, extending other plugins and testing.
- [Backend](backend/admin-controllers.md): admin controllers, forms, lists, relations, users, settings and the admin SPA.
- [Plugins](plugins/registration.md): registering a plugin, scheduling, extending other plugins, [menu item types](plugins/menu-item-types.md) and testing.
- [Backend](backend/admin-controllers.md): admin controllers, forms, lists, relations, [plugin field types](backend/field-types.md), users and [API tokens](backend/users-and-permissions.md#api-tokens), settings and the admin SPA.
- [Database](database/models.md): models, migrations, queries, relations, casts, attachments and transactions.
- [Services](services/configuration.md): configuration, events, routing, authentication, mail, jobs, realtime, search and the other services, plus what SummerCMS does not provide for the frontend.
- [Services](services/configuration.md): configuration, events, routing, authentication, [caching](services/caching.md), mail, jobs, realtime, search and the other services, plus what SummerCMS does not provide for the frontend.
- [Console](console/introduction.md): the `summer` tool, the application binary's commands and writing your own.

View File

@@ -1,6 +1,6 @@
---
title: Authentication
description: Mint and verify JWTs with bouncer, turn guards into route middleware, revoke tokens through a jti blacklist and hash passwords with bcrypt.
description: JWTs with bouncer, cookie transport, guards as route middleware, a jti blacklist, Cloudflare Turnstile checks and bcrypt passwords.
section: services
order: 50
---
@@ -58,6 +58,38 @@ Revoked token IDs are kept in a `bouncer.BlacklistStore`:
Logging out is blacklisting the token's `jti`. Setting a user's `TokensValidAfter` to now revokes all their tokens at once, for example after a password change.
## Cookie transport
A browser frontend can keep the token in an httpOnly cookie instead of JavaScript-readable storage. The cookie attributes come from one block, `auth.cookie.*`, which `bouncer.LoadCookieConfig` reads into a `bouncer.CookieConfig` at boot:
```yaml
cookie:
domain: example.com
same_site: lax
```
This fragment belongs in `config/auth.yaml`. With the block unset the cookie is named `token`, has `Path=/`, `HttpOnly` and `SameSite=Lax`, no `Domain` (host-only) and `Secure` exactly when `app.url` is https. The keys:
| Key | Default | Controls |
|-----|---------|----------|
| `auth.cookie.name` | `token` | Cookie name. |
| `auth.cookie.domain` | empty (host-only) | A parent domain shares the cookie with its subdomains. |
| `auth.cookie.path` | `/` | Cookie path. |
| `auth.cookie.secure` | true when `app.url` is https | In the production environment a value that resolves to false is refused, whether `false` is set or inferred from an http or unset `app.url`. |
| `auth.cookie.same_site` | `lax` | `lax`, `strict` or `none`; `none` requires Secure. |
A `bouncer.CookieConfig` built by hand instead of through `bouncer.LoadCookieConfig` gets `SameSite=Lax` when `SameSite` is unset, and the zero config writes `token` on `/` with `HttpOnly`, `SameSite=Lax` and `Secure`; a config with any field set keeps its own `Secure` value.
An invalid domain, `same_site: none` without Secure, or a cookie that is not Secure in production makes `bouncer.LoadCookieConfig` return an error naming the key. In production that covers `secure: false` and an unset `auth.cookie.secure` with an http or missing `app.url`; that error names `auth.cookie.secure` and `app.url`, and you fix it by setting `app.url` to the public https URL or `auth.cookie.secure: true`. Return the error from your plugin's boot so the application does not start with a cookie browsers would drop or send in clear.
`bouncer.CookieConfig.TokenCookie` builds the cookie for a token and a lifetime. Pass the guard's refresh window (`refresh_ttl`, 20160 minutes by default), not its access `ttl`: the access token inside expires after `ttl`, the frontend's 401 handling then calls the refresh endpoint, and that endpoint needs the cookie to still be there. A cookie that died with the token would force a new sign-in every hour. `bouncer.CookieConfig.ForgetTokenCookie` builds the matching cookie that deletes it; it carries the same name, path and domain, or the browser would keep the original.
`bouncer.CookieRefreshHandler` is a ready-made refresh endpoint. It reads the token from the cookie (only from the cookie), calls your refresh func, which wraps `bouncer.Refresh` or `bouncer.RefreshAudienceFor` and so rotates the token and blacklists the old `jti`, sets the rotated cookie and passes a `bouncer.CookieResult` to your writer, which answers in the application's own envelope. A missing cookie (`bouncer.ErrNoTokenCookie`) or a refused refresh reaches the writer with status 401 and leaves the cookie alone. `bouncer.CookieLogoutHandler` revokes the cookie's token through your revoke func, always clears the cookie and answers 200. The [bouncer README](../../modules/bouncer/README.md) has a complete example.
**Subdomains.** For a frontend on `www.example.com` calling an API on `api.example.com`, set `auth.cookie.domain: example.com`, list `https://www.example.com` in the API's CORS origins with `supports_credentials: true` (see [Routing](routing.md#cors)), and send requests with credentials. A `Domain` cookie reaches every subdomain of `example.com`, so every host under it is inside the trust boundary: an application on another subdomain receives the token too. Keep the default host-only cookie when frontend and API share one host.
**CSRF.** CORS controls who may read a response, not who may send a request. `SameSite=Lax` treats sibling subdomains as the same site, so a page on any `*.example.com` host can make the browser send the cookie. Mount the refresh and logout handlers on POST routes only, and for routes that act on the cookie prefer a JSON body or a custom header, which a cross-site form cannot send without a CORS preflight. The refresh answer is unreadable to an origin CORS does not allow, and a forced logout is the worst a cross-site POST to the logout route can do.
## Guards and middleware
A guard implements `bouncer.Guard`: it turns a request into a principal or an error. `bouncer.NewJWTGuard` is the frontend guard. It reads the bearer token, then any cookie names you give it, verifies the token, checks the blacklist and the user's cutoff, and loads the user through your `bouncer.UserProvider`. On failure it answers 401 with `{"error":true,"message":...}`. `bouncer.NewBackendJWTGuard` is the same for the admin audience.
@@ -136,6 +168,30 @@ fmt.Println(rec.Code, rec.Body.String())
// 401 {"error":"Authentication required"}
```
## Turnstile
[Cloudflare Turnstile](https://developers.cloudflare.com/turnstile/) keeps bots off guest and registration endpoints. The frontend widget produces a token, the request carries it, and the server checks it with Cloudflare's siteverify endpoint. surf registers the check as named middleware with two modes:
| Middleware | Without a secret | With a secret |
|------------|------------------|---------------|
| `turnstile:open` | passes (a development no-op) | the token must verify |
| `turnstile:closed` | 503 in the production environment, passes elsewhere | the token must verify |
Use `turnstile:closed` where an unprotected endpoint would be a hole, such as guest entry: a production deployment that forgot the secret answers 503 instead of letting every bot in. Use `turnstile:open` where the endpoint has other protection and must keep working without Turnstile, such as registration. With a secret set both modes behave the same: a missing token, a token Cloudflare rejects, a token longer than 2048 characters, a failed siteverify call or one slower than 5 seconds answers 422.
The middleware reads the token from the `turnstile_token` field (`turnstile.field` changes the name): the query string first, then a JSON object body or a form body. It reads the body and restores it, so the handler sees the same bytes. A multipart body is not searched; put the token in the query string there.
| Key | Default | Controls |
|-----|---------|----------|
| `turnstile.secret` | empty | The site's secret key. Set it through `SUMMER_TURNSTILE__SECRET`; a PHP application's `TURNSTILE_SECRET` value moves to that variable. When it is empty and routes name `turnstile:open` or `turnstile:closed`, the start-up logs one warning, `surf: turnstile.secret is empty`, with the number of such routes, since they are then unprotected. |
| `turnstile.field` | `turnstile_token` | The request field carrying the token. |
Refusals are written with neutral bodies, `{"error":"turnstile_unavailable"}` (503) and `{"error":"turnstile_failed"}` (422). To keep an existing response envelope, publish a `bouncer.TurnstileWriter` on the app in your plugin's `Register`; it receives the status and the `bouncer.TurnstileVerdict`. A published `*bouncer.Turnstile` replaces the verifier built from config, for example to give it its own `http.Client`.
Cloudflare's [testing page](https://developers.cloudflare.com/turnstile/troubleshooting/testing/) lists secrets that always pass (`1x0000000000000000000000000000000AA`) or always fail (`2x0000000000000000000000000000000AA`), for staging environments and manual checks. A token is valid for 300 seconds and only once: verify it before any write and never retry a verification with the same token.
The middleware runs before the handler, so a request with both a bad token and invalid fields gets the Turnstile answer, not the validation errors. When a ported endpoint must keep the order it had, such as validating a name first, call `bouncer.Turnstile.Verify` from the handler instead of using the middleware: it takes the context, token, client IP and mode and returns the verdict, plus an error for logging that never contains the secret or the token.
## Passwords
`bouncer.HashPassword` hashes with bcrypt at the cost you give it, `bouncer.CheckPassword` compares in constant time, and `bouncer.NeedsRehash` reports a hash made below your configured cost. WinterCMS stores bcrypt hashes too, so existing passwords keep working:

178
docs/services/caching.md Normal file
View File

@@ -0,0 +1,178 @@
---
title: Caching
description: Cache values with TTLs through cooler.Cache, in process or shared through Redis, under a per-application key prefix.
section: services
order: 45
---
# Caching
WinterCMS and Laravel cache through the `Cache` facade: `Cache::get`, `Cache::put`, `Cache::remember` and friends, backed by the store `config/cache.php` selects. SummerCMS has the same core through [cooler](../../modules/cooler/README.md): one `cooler.Cache` per application, with an in-process driver by default and a Redis driver shared by every instance.
## The Cache API
| Laravel | cooler |
|---------|--------|
| `Cache::get($key)` | `cooler.Cache.Get` returns the bytes and whether the key was found. |
| `Cache::put($key, $value, $ttl)` | `cooler.Cache.Set` stores bytes for a TTL. |
| `Cache::forever($key, $value)` | `cooler.Cache.Set` with a TTL of 0. |
| `Cache::has($key)` | `cooler.Cache.Has`. |
| `Cache::forget($key)` | `cooler.Cache.Forget`. |
| `Cache::remember($key, $ttl, $callback)` | `cooler.Cache.Remember`. |
| `Cache::increment($key, $by)` | `cooler.Cache.Increment`. |
| `Cache::flush()` | `cooler.Cache.Flush`, limited to this application's cache keys. |
Values are bytes, so encode structured values yourself, for example with `encoding/json`. There are no tags and no locks.
A plugin looks the cache up on the application with `app.Lookup[cooler.Cache]()`, or asks `cooler.FromApp` for it. Here the second `cooler.Cache.Remember` call is answered from the cache:
```go src=modules/cooler/example_test.go#ExampleFromApp
// serve publishes the Cache at start-up; a job or console command asks
// FromApp, which builds the configured driver (memory here) once.
app := backpack.New(nil)
cache, err := cooler.FromApp(app)
if err != nil {
fmt.Println(err)
return
}
ctx := context.Background()
loads := 0
popular := func(context.Context) ([]byte, error) {
loads++
return json.Marshal([]string{"Hello", "Summer"})
}
var titles []string
for range 2 {
raw, err := cache.Remember(ctx, "blog:popular", 10*time.Minute, popular)
if err != nil {
fmt.Println(err)
return
}
if err := json.Unmarshal(raw, &titles); err != nil {
fmt.Println(err)
return
}
}
fmt.Println(titles, loads)
// Output: [Hello Summer] 1
```
## Drivers
`cache.driver` selects the driver:
- `memory` (the default) keeps values in the process, in an otter cache bounded by `cache.memory.max_entries` (default 10000; `cooler.DefaultMaxEntries`). Beyond that, the least valuable entries are evicted, so keys derived from request data cannot grow memory without limit. Each instance has its own memory cache.
- `redis` keeps values in Redis at `redis.url`, shared by every instance. When the rate limiter also uses Redis (`http.rate_limit.store: redis`, see [Rate limiting](rate-limiting.md#sharing-limits-across-instances)), both use one connection pool.
```yaml
# config/cache.yaml
driver: redis
```
```yaml
# config/redis.yaml (or SUMMER_REDIS__URL, which keeps a password out of YAML)
url: redis://10.0.0.7:6379/0
```
A missing or malformed `redis.url`, an unknown driver name or a `cache.memory.max_entries` below 1 fails the start-up with an error that names the key; an error never shows the Redis URL. Redis being down at start-up is a warning, not a failed start.
## Keys and the prefix
Every key is stored as `<prefix>cache:<key>`. The prefix is `cache.prefix` when set; by default it is `app.name` lower-cased, with every run of characters other than letters and digits replaced by `_`, followed by `:`, so `app.name: Acme Blog` gives `acme_blog:` and the key `blog:popular` is stored as `acme_blog:cache:blog:popular`. With `app.name` empty the prefix is `summer:`. Rate-limit counters live under `<prefix>rl:`, so several applications, and the cache and the limiter of one application, can share a Redis without touching each other's keys.
## TTLs
A TTL of 0 or less keeps a value until it is forgotten, flushed or, in the memory driver, evicted. A positive TTL expires the value after that long; Redis stores it with millisecond precision. `cooler.Cache.Increment` starts a missing key at 0 without a TTL and keeps the expiry of an existing key, as Redis `INCRBY` does.
## Remember degrades instead of failing
`cooler.Cache.Remember` reads the key and, on a miss, calls the callback, stores its value for the TTL and returns it. When the cache itself fails, for example while Redis is down, it does not fail the page: a failed read still calls the callback and returns its value, a failed write is skipped, and both are logged as a warning at most once per minute. Only an error from the callback is returned. The other methods return cache errors to the caller.
## Jobs and console commands
The `serve` command installs cooler at start-up (`cooler.Install`) and publishes the cache on the application, so plugins and the in-process job worker find it with `app.Lookup[cooler.Cache]()`. Other commands, such as a dedicated `queue:work` process or a console command, do not run `serve`; they call `cooler.FromApp`, which returns the published cache or builds the configured one and publishes it. It does not connect at construction: the Redis driver dials on the first call, so a command that never touches the cache never needs Redis.
## Flush
`cooler.Cache.Flush` removes this application's cache keys and nothing else. The Redis driver walks `SCAN MATCH <prefix>cache:*` in batches of 500 and removes each batch with `UNLINK`; it never empties the whole Redis database, which would also drop rate-limit counters and other applications' keys. The memory driver empties the process cache.
## Site cache purge
A server-rendered site in front of the API usually caches its pages for a few minutes (a Nuxt `routeRules` `swr` window, for example). Without help, an edit in the admin shows on the site only when that window ends. cooler closes the gap: after an admin save commits, it queues a purge request to the site, and the site drops its route cache, so the next page view renders the change.
Configure the target once for the application:
| Key | Environment | Meaning |
|-----|-------------|---------|
| `site_cache.purge_url` | `SUMMER_SITE_CACHE__PURGE_URL` | The site's purge route, for example `http://127.0.0.1:3000/api/__site__/cache/purge`. |
| `site_cache.purge_token` | `SUMMER_SITE_CACHE__PURGE_TOKEN` | The bearer token, at least 32 characters (`cooler.SiteCachePurgeMinTokenLength`). |
Set both or neither; with neither, nothing is purged. The URL must be http or https with a host, without user info or a fragment, and plain http is accepted only for a loopback host (127.0.0.0/8, `::1`, `localhost`): the site normally runs on the same machine, and the token must not cross a network in clear. Use https for any other host. A wrong or half-set configuration stops `serve` at start-up with an error that names the key; neither errors nor log lines ever contain the token.
### What a purge sends
Each committed save queues a purge; the request looks like this:
```http
POST /api/__site__/cache/purge
Authorization: Bearer <site_cache.purge_token>
Content-Type: application/json
{"kind":"faq","slug":"shipping"}
```
The kind names what was saved (a plugin picks it, for example `faq`, `menu`, `menu_entry`, `post` or `category`) and the slug which record; the site may log them. The site answers `{"purged": <n>}`, and cooler logs `cooler: site cache purged` with the kind, slug, status and count.
At most one request is in flight per process (per purger). Saves committed while it is sent collapse into one follow-up request carrying the last kind and slug, because each purge clears the site's whole route cache: a bulk delete of fifty rows sends at most two requests. The log line of such a follow-up adds `coalesced` with the number of saves it stands in for besides its own.
### Timing and failures
`cooler.PurgeSiteCacheAfterCommit` registers the purge on the save's transaction through `lagoon.AfterCommit`. It is queued only after the commit, so a page view between the purge and the commit cannot re-cache the old content, and a save that fails or is rolled back sends nothing. Outside a transaction it is queued at once. Queuing never blocks: the admin request answers without waiting for the site, and a background worker sends the request.
A purge never fails a save. Each request is bounded by `cooler.SiteCachePurgeTimeout` (2 s) and detached from the admin request's cancellation; an unreachable site, an error status or a timeout is logged as one `cooler: site cache purge failed` warning and the save answers as usual. Redirects are not followed: a 3xx answer is logged as a failed purge with its status, so the bearer token never goes to another URL and the POST never turns into a GET. Point `site_cache.purge_url` at the final URL.
Because the purge is fire and forget, a test or a console command that wants to see its effect calls `cooler.SiteCachePurger.Wait`, which returns once nothing is queued or in flight (or when its context ends). `serve` waits for a queued purge on shutdown, up to the timeout plus one second, through `cooler.Handle.Close`.
### Purging from a plugin
The `serve` command publishes the purger at start-up (`cooler.Install`), and `cooler.SiteCacheFromApp` returns it, or builds it from the keys in commands and tests that do not run `serve`. An admin controller calls it from its after-save hooks, which run inside the save transaction, and takes that transaction from `cabana.TxFromContext`:
```go src=modules/cooler/example_test.go#articlesController.purge
// purge queues one site-cache purge for the saved article once the save
// transaction commits (at once outside one); the save never waits for the
// site. Without a purge configured the purger is nil and nothing is sent.
func (c articlesController) purge(ctx context.Context, model any) error {
article, ok := model.(*Article)
if !ok {
return nil
}
tx, _ := cabana.TxFromContext(ctx)
cooler.PurgeSiteCacheAfterCommit(ctx, tx, cooler.SiteCacheFromApp(c.app()), "article", article.Slug)
return nil
}
```
Each of `pact.FormAfterCreate`, `pact.FormAfterUpdate` and `pact.FormAfterDelete` calls it:
```go src=modules/cooler/example_test.go#articlesController.FormAfterUpdate
// FormAfterUpdate runs inside the update's transaction.
func (c articlesController) FormAfterUpdate(ctx context.Context, model any) error {
return c.purge(ctx, model)
}
```
A nil purger (no purge configured) does nothing, so the hooks need no configuration check. A tree or sortable list's reorder save runs `pact.ReorderAfterSave` the same way; see [Tree reorder](../backend/lists-and-filters.md#tree-reorder).
### The site side
The Nuxt layers of the core plugins (the Journal, menu and FAQ layers) each ship the same route, so a site that extends any of them has it:
- `POST /api/__site__/cache/purge` removes every entry of Nitro's route cache (pages, their `_payload.json` and feeds) and answers `{"purged": <n>}`.
- Its token is `runtimeConfig.siteCache.purgeToken`, set through `NUXT_SITE_CACHE_PURGE_TOKEN` to the same value as `site_cache.purge_token`. With a token shorter than 32 characters the route answers 404.
- It answers 404 to any request that came through a reverse proxy (`X-Forwarded-For`, `X-Real-IP` or `Forwarded`), unless the site opts in with `NUXT_SITE_CACHE_ALLOW_PROXIED` (see [A purge URL on another host](#a-purge-url-on-another-host)), and 401 to a missing or wrong bearer token, compared in constant time.
- Each layer also ships the route rule `'/api/__site__/**': { cache: false }`. Without it, a site rule such as `'/**': { swr: 300 }` would wrap the purge route in Nitro's cached handler, which answers from the cache without running the route.
The three copies are byte-identical, so it does not matter which layer's copy Nitro picks.
### A purge URL on another host
When the site does not run on the backend's machine, `site_cache.purge_url` is an https URL on the site's host, for example `https://www.example.com/api/__site__/cache/purge`. The request then reaches Nuxt through the site's TLS-terminating proxy (nginx, Caddy, Traefik), which adds `X-Forwarded-For`, `X-Real-IP` or `Forwarded`, and the route refuses such requests by default with 404. Pair the https purge URL with `NUXT_SITE_CACHE_ALLOW_PROXIED=true` on the site (`runtimeConfig.siteCache.allowProxied`): the route then accepts proxied requests, and the bearer token is the only gate. Keep the token long and secret (the same value in `site_cache.purge_token` and `NUXT_SITE_CACHE_PURGE_TOKEN`), and keep the default, loopback and no flag, whenever the site runs next to the backend.

View File

@@ -27,9 +27,12 @@ A WinterCMS plugin that shipped components and AJAX handlers is ported as routes
Build the frontend with any framework that can call a JSON API, as its own project with its own build and deployment:
- **Data:** call the plugins' JSON routes. [Routing](routing.md) shows how routes, auth groups and JSON responses are declared, and [Queries and pagination](../database/queries-and-pagination.md) the list envelope.
- **Signing in:** the user plugin issues JWTs; send them as a bearer token or in the cookie the guard reads. See [Authentication](authentication.md).
- **Signing in:** the user plugin issues JWTs; send them as a bearer token or in the cookie the guard reads. A browser frontend can keep the token in an httpOnly cookie that the API sets, with `bouncer.CookieRefreshHandler` and `bouncer.CookieLogoutHandler` as its refresh and logout endpoints. See [Authentication](authentication.md) and [Cookie transport](authentication.md#cookie-transport).
- **Live updates:** instead of polling an AJAX handler, subscribe to realtime channels. The frontend connects to Centrifugo with a token from the token route and receives model broadcasts and explicit events. See [Realtime](realtime.md).
- **Cross-origin calls:** when the frontend runs on another origin, allow it in `http.cors`, as [Routing](routing.md) describes.
- **Cross-origin calls:** when the frontend runs on another origin, allow it in `http.cors`, as [Routing](routing.md#cors) describes. A credentialed single-page app that needs its own policy on some routes names a profile with the `cors:<profile>` middleware; see [CORS profiles](routing.md#cors-profiles).
- **Bot protection:** guard public forms such as sign-up or guest entry with Cloudflare Turnstile: the frontend widget's token goes with the request, and the `turnstile:open` or `turnstile:closed` middleware checks it. See [Turnstile](authentication.md#turnstile).
- **Page cache:** a server-rendered frontend that caches its pages can have them purged after every admin save, so an edit shows on the next page view. See [Site cache purge](caching.md#site-cache-purge).
- **API documentation:** publish a browsable Swagger UI page of the public API for frontend developers with `http.api_docs.enabled`. See [API documentation page](routing.md#api-documentation-page).
- **Push notifications:** see [Web Push](push.md).
The admin is the one frontend SummerCMS ships. It is a single-page app built the same way, against the admin API; see [Admin SPA](../backend/admin-spa.md).

View File

@@ -33,7 +33,32 @@ Each `surf.Bucket` has the maximum number of requests, the window length (`Decay
A request over the limit gets a 429 response with the body `{"message":"Too Many Attempts."}` and a `Retry-After` header. Every throttled response carries `X-RateLimit-Limit` and `X-RateLimit-Remaining`, and a rejected one also `X-RateLimit-Reset`.
The limiter is a fixed window: the counter resets when the window ends, as Laravel's cache limiter does. Counters live in the process (`surf.MemoryStore`, behind the `surf.Store` interface), so each application instance counts on its own. Behind a load balancer with several instances, the effective limit is the configured limit times the number of instances.
The limiter is a fixed window: the counter resets when the window ends, as Laravel's cache limiter does. By default counters live in the process (`surf.MemoryStore`, behind the `surf.Store` interface). With this memory store each application instance counts on its own, so behind a load balancer with several instances the effective limit is the configured limit times the number of instances. [Sharing limits across instances](#sharing-limits-across-instances) moves the counters to Redis.
## Sharing limits across instances
Laravel shares its limiter counters through the cache store, usually Redis. SummerCMS does the same through [cooler](../../modules/cooler/README.md): set `http.rate_limit.store` to `redis` (the default is `memory`) and point `redis.url` at a Redis every instance reaches.
```yaml
# config/http.yaml
rate_limit:
store: redis
```
```yaml
# config/redis.yaml
url: redis://10.0.0.7:6379/0
```
Keep a URL that carries a password in the environment instead: `SUMMER_REDIS__URL=redis://:secret@10.0.0.7:6379/0`. `rediss://` connects over TLS and `unix://` through a socket.
When `serve` starts, it calls `cooler.Install`, which builds the Redis client and pings it, and publishes a `cooler.RateLimitStore` as the `surf.Store` before the router is built. Routes do not change: `throttle:60,1` and `throttle:<bucket>` count in Redis, with the same fixed-window rules as the memory store. Each attempt is one atomic Lua script, so two instances answering requests from the same client count one window together: with `throttle:2,1`, the third request in the minute gets a 429 whichever instance answers it.
Keys are `<prefix>rl:<key>`. The prefix is `cache.prefix`, or by default `app.name` lower-cased with other characters than letters and digits replaced by `_`, followed by `:`, so several applications can share one Redis.
When Redis does not answer, each attempt waits at most `cooler.AttemptTimeout` (500 ms) and then counts in a per-instance `surf.MemoryStore`, as if the memory store were selected, until Redis answers again. A warning is logged at most once a minute. Redis being down at start-up is a warning too, not a failed start; a missing or malformed `redis.url` fails the start with an error that names the key and never shows the URL.
Only `serve` connects to Redis. `route:list` builds the router with the memory store, and so does `surf.Assemble` in tests, so neither needs a running Redis.
## Client IP and trusted proxies

View File

@@ -98,6 +98,8 @@ The framework registers these names:
|------|------|
| `throttle:<bucket>` or `throttle:<max>,<minutes>` | Rate limiting; see [Rate limiting](rate-limiting.md). |
| `body.limit:<bytes>` | Replaces the default request body limit for the route. |
| `cors:<profile>` | CORS headers from `http.cors_profiles.<profile>` on every response of the route, refusals and recovered panics included, whatever its position in the list; see [CORS profiles](#cors-profiles). |
| `turnstile:open`, `turnstile:closed` | Cloudflare Turnstile verification; see [Turnstile](authentication.md#turnstile). |
| `locale.from-principal` | Switches the request locale to the signed-in user's preferred locale. |
| `backend` | The admin guard, when the admin is enabled. |
@@ -224,4 +226,77 @@ cors:
This fragment belongs in `config/http.yaml`. List the frontend's exact origin; `*` is for public, credential-free APIs only.
The rules surf applies to the origin answer:
- `supports_credentials: true` together with `*` in `allowed_origins` fails the start-up. A credentialed answer that trusted any origin would let every site read responses made with the user's cookies; Laravel's php-cors reflects the origin in that case, surf refuses it.
- With `supports_credentials: true`, three more rules close other spellings of the same exposure, and each refusal names the key with its index, such as `http.cors.allowed_origins[1]`:
- an `allowed_origins` entry with a `*` is accepted only when the scheme is literal and the `*` is the whole leftmost host label, as in `https://*.example.com`; `https://*`, `*://*`, `https://*example.com` and `http*://www.example.com` fail the start-up;
- every `allowed_origins_patterns` entry must be anchored with `^` and `$`, as in `^https://[a-z]+\.example\.com$`; an unanchored pattern matches inside any origin, so `https://.*\.example\.com` would trust `https://x.example.com.attacker.net`;
- no origin rule may match an origin outside the configured domains: surf tries each rule against probe origins such as `https://cors-probe.invalid` and `null`, which refuses `https://*.com` and `^https?://.*$`.
Without credentials these entries load as before.
- An allowed origin is echoed exactly in `Access-Control-Allow-Origin`; `*` is sent only for `allowed_origins: ["*"]` without credentials, and that answer is unchanged from earlier releases.
- When the answer depends on the request's `Origin` (with `supports_credentials`, with any pattern, or with more than one listed origin), every response on a CORS path carries `Vary: Origin`, the preflight and a disallowed origin's response included, so a shared cache never serves one origin's answer to another.
- An `allowed_origins` entry with a `*` inside, such as `https://*.example.com`, matches like Laravel's `Str::is`: the `*` stands for any characters and the rest is literal.
- `allowed_origins_patterns` entries are Go regular expressions. One that does not compile fails the start-up, and so does one still wrapped in PHP delimiters, such as `#^https://.*\.example\.com$#`: drop the delimiters and any flags when porting it.
A frontend on `https://www.example.com` that calls an API on `api.example.com` with the session cookie (see [Cookie transport](authentication.md#cookie-transport)) needs the www origin listed and credentials on:
```yaml
cors:
paths: ["api/*"]
allowed_origins: ["https://www.example.com"]
allowed_methods: ["*"]
allowed_headers: ["*"]
supports_credentials: true
```
The browser sends the request with `credentials: "include"`; the answer carries `Access-Control-Allow-Origin: https://www.example.com`, `Access-Control-Allow-Credentials: true` and `Vary: Origin`.
surf answers every `OPTIONS` request on a CORS path itself, with 204, before any route runs, so a plugin never registers an `OPTIONS` route. A preflight gets the same headers Laravel's `HandleCors` sends: with `allowed_methods: ["*"]` the `Access-Control-Allow-Methods` value is the requested method, upper-cased, and with `allowed_headers: ["*"]` the `Access-Control-Allow-Headers` value is the requested header list.
## CORS profiles
`http.cors` applies one policy to a set of paths. A route group that needs its own policy, for example a credentialed one for a single-page app while the rest of the API stays public, names a profile with the `cors:<profile>` middleware. Profiles live under `http.cors_profiles.<name>` and take the `http.cors` keys except `paths`:
```yaml
cors_profiles:
spa:
allowed_origins: ["https://www.example.com"]
allowed_methods: ["GET", "POST"]
allowed_headers: ["Content-Type"]
supports_credentials: true
```
This fragment belongs in `config/http.yaml`. The route, or its group, then lists `cors:spa` among its middleware names, as it lists `throttle:60,1`. surf applies the profile outside every other layer of the route, whatever its position in the list or the group nesting. Every response of the route therefore carries the profile's headers, including refusals from guards (401, 403), throttle (429) and Turnstile (422, 503), the body limit (413) and recovered panics (500); otherwise the browser would hide them from the page, and a single-page app could not see the 401 that starts its cookie refresh.
A profile is checked like `http.cors`: credentials with a `*` origin, a pattern that does not compile and a PHP-delimited pattern fail the start-up, with errors naming `http.cors_profiles.<name>.<key>`. Profile names use lower-case letters, digits, underscore and dash, and a profile that sets `paths` fails too.
ServeMux answers an `OPTIONS` request on a path registered only for `GET` or `POST` with 405 before any route middleware runs, so the middleware alone would never see a browser's preflight. surf therefore registers one `OPTIONS <path>` route for every path whose routes name `cors:<profile>`, owned by `surf` in `route:list`. It answers the preflight from the profile, with the same headers as the global preflight, and runs no other middleware: a preflight carries no credentials, so no auth, throttle or body limit applies. The start-up fails when a route names an unknown profile, when two routes on one path name different profiles, when a plugin already declared an `OPTIONS` route on that path, and when the path also matches `http.cors.paths`, because the global wrapper answers every preflight on its paths before routing.
## API documentation page
An application can publish a Swagger UI page for its public API. The page is off by default; with `http.api_docs.enabled: true` surf mounts two public routes, without any middleware:
- `GET /api-docs`: the Swagger UI page.
- `GET /api-docs.json`: the OpenAPI document as compact JSON.
The document comes from the one plugin that implements `surf.APIDocsProvider`. It returns the document as YAML or JSON, usually a file it embeds:
```go src=modules/surf/example_test.go#BlogPlugin.APIDocsSpec
// APIDocsSpec supplies the document behind the opt-in /api-docs page
// (surf.APIDocsProvider).
func (p *BlogPlugin) APIDocsSpec() ([]byte, error) {
return []byte(openAPISpec), nil
}
```
surf converts the document once at start-up: a document starting with `{` must be valid JSON, anything else is read as YAML, and the result keeps the source's key order. A document that does not convert fails the start-up with the plugin's name, and so do zero or several providers while the page is enabled. Nothing is generated from the code: the document describes exactly the surface you choose to publish.
| Key | Default | Controls |
|-----|---------|----------|
| `http.api_docs.enabled` | `false` | Mounts the page and the JSON route. |
| `http.api_docs.path` | `/api-docs` | The page path; the JSON route is the same path plus `.json`. It must be an absolute URL path, and neither route may replace a plugin route. |
| `http.api_docs.title` | `API documentation` | The page title. |
The page loads `swagger-ui-dist` from jsDelivr at the exact version `surf.SwaggerUIVersion`, and every script and stylesheet carries a Subresource Integrity hash with `crossorigin="anonymous"`, so the browser refuses a file that changed on the CDN. A floating version such as `@5` cannot carry a hash, which is why the version is pinned and moves only with a framework release. The page sends a `Content-Security-Policy` whose only script source is a per-request nonce: scripts run only with the page's nonce. The two jsDelivr scripts carry it as well as their SRI hash and the inline start-up script carries it, so no other jsDelivr script can run, even if HTML were injected into the page. The page also sends `X-Content-Type-Options: nosniff` and `Referrer-Policy: no-referrer`.

View File

@@ -1,6 +1,6 @@
---
title: Storage
description: Configure the uploads bucket that serve opens, choose file or memory bucket URLs, serve stored files, and size upload routes.
description: Configure the uploads bucket that serve opens, choose file, memory or S3 bucket URLs, serve stored files, and size upload routes.
section: services
order: 90
---
@@ -14,6 +14,7 @@ WinterCMS stores uploads on a Laravel filesystem disk. SummerCMS stores them in
|-----|--------|
| `file:///var/lib/acme/uploads` | In a directory on the server. |
| `mem://` | In memory, for tests. Everything is lost when the process exits. |
| `s3://acme-uploads?region=eu-central-1` | In an S3 or S3-compatible bucket. The binary must be built with the `summercms_s3` tag; see [S3 and S3-compatible buckets](#s3-and-s3-compatible-buckets). |
The keys go in `config/storage.yaml`:
@@ -39,6 +40,61 @@ An original then has the URL `/storage/app/uploads/public/<partition>/<disk_name
Application code gets the bucket with `app.Lookup[*blob.Bucket]()` and reads and writes it through the `gocloud.dev/blob` API. Model attachments, thumbnails and deleting files after commit are covered in [Attachments](../database/attachments.md).
## S3 and S3-compatible buckets
The S3 driver (`gocloud.dev/blob/s3blob`) is not linked into every binary, since it brings in the AWS SDK. A binary links it when it is built with the Go build tag `summercms_s3` (`attach.S3BuildTag`). The application never imports the driver itself: list the tag under `build.tags` in `summer.yaml` and `summer build` passes it to `go build`:
```yaml
module: git.golem15.com/acme/acme
binary: acme
plugins:
- id: acme.blog
module: git.golem15.com/acme/acme/plugins/blog
build:
tags:
- summercms_s3
```
`summer build` prints `build tags: summercms_s3` before it builds, and `summer dev` builds the same way. A build script that runs its own `go build` after `summer build`, for example to cross-compile a release, must repeat the tag: `go build -tags summercms_s3 -o bin/acme .`, or `GOFLAGS=-tags=summercms_s3` in its environment. A binary built without the tag refuses an `s3://` bucket URL at start-up:
```text
attach: storage.uploads.bucket_url uses s3:// but this binary was built without the summercms_s3 build tag; list summercms_s3 under build.tags in summer.yaml and rebuild with summer build (a raw go build needs -tags summercms_s3, or GOFLAGS=-tags=summercms_s3)
```
The bucket is configured only through `bucket_url` and its query parameters, as gocloud.dev defines them; there are no separate `s3.*` keys. For AWS:
```yaml
uploads:
bucket_url: s3://acme-uploads?region=eu-central-1
```
For an S3-compatible service (MinIO, Cloudflare R2, Hetzner Object Storage and others), add the service's `endpoint`, path-style addressing and, for services that do not support the newer AWS checksums, the two checksum parameters:
```yaml
uploads:
bucket_url: s3://acme-uploads?region=auto&endpoint=https://storage.example.com&use_path_style=true&request_checksum_calculation=when_required&response_checksum_validation=when_required
```
Credentials come from the standard AWS environment: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`, a profile (`AWS_PROFILE`, or `profile=` in the URL) or an instance role. Never put keys in `bucket_url` or in a config file. The bucket opens at start-up without a request to the service, so wrong credentials or a missing bucket show up on the first upload or read.
The [Serving files](#serving-files) route reads through the bucket, so it serves an S3 bucket exactly as it serves a directory.
## Public URL base
By default every file and thumbnail URL starts with `public_path_prefix`, a path on the application's own host. When a public bucket or a CDN serves the files directly, set `storage.uploads.public_url` (environment `SUMMER_STORAGE__UPLOADS__PUBLIC_URL`) to its base URL:
```yaml
uploads:
bucket_url: s3://acme-uploads?region=eu-central-1
public_url: https://cdn.example.com/uploads
```
`attach.PublicURL`, `attach.File.URL` and thumbnail URLs then are that base, one slash and the blob key, which keeps the partition layout: `https://cdn.example.com/uploads/5f1/d0c/2e9/5f1d0c2e9a7b4c3d8e6f.jpg`. `attach.PublicURLBase` returns the base. The value must be an absolute `http` or `https` URL with a host and without user info, query or fragment; anything else stops the start-up with an error naming the key. Thumbnails are written to the bucket when they are first requested through `attach.File.Thumb`, so their URLs work behind the base as well.
The `/storage/uploads` route checks `is_public` before it serves a file; a public bucket or a CDN serves any key it holds. A bucket behind `public_url` therefore holds public files only: while `public_url` is set, `attach.Store` refuses a protected (`is_public` false) upload with `attach.ErrProtectedNeedsPrivateBucket` before it writes anything. Keep protected relations on an application whose bucket is private, with `public_url` unset, or make the relation public. `serve` logs one warning at start-up naming the host and this rule.
Leave `public_url` unset to keep the URLs under `public_path_prefix`. With `storage.uploads.serve` on, those URLs go through the [Serving files](#serving-files) route, which works for an S3 bucket too, and private files stay protected.
## Serving files
The framework mounts no file route unless you ask for one. Set `storage.uploads.serve` to `true` (environment `SUMMER_STORAGE__UPLOADS__SERVE=true`, default `false`) and the router registers `GET <public_path_prefix>/{path...}` on `attach.StaticHandlerPublic`:
@@ -63,6 +119,8 @@ The route looks up the bucket and the database on each request: `route:list` lis
Without the key, the application decides where files are served: mount `attach.StaticHandlerPublic` itself, or put a web server or CDN in front of the bucket directory. Either way, serve uploads from a separate origin when you can; the [Attachments](../database/attachments.md) page explains why. The built-in route is meant for development and for deployments that cannot split the origin yet.
> **Deploy note.** When a static server such as nginx, or a CDN, serves the uploads directory or bucket directly, it picks the Content-Type from the file extension and sends none of the route's headers. `attach.Store` names each file after its sniffed content (`attach.DiskExtension`), so a PNG uploaded as `x.html` is stored as `.png`, but configure the server as the route behaves anyway: send `X-Content-Type-Options: nosniff` on every upload, and serve only the extensions the application accepts (for example the image and document types in `attach.DefaultFileExtensions`), answering 404 for anything else.
## Upload size
Every non-raw route has a request body limit of `http.body_limits.default_bytes`. A route that accepts uploads raises its own limit with the `body.limit:<bytes>` middleware; see [Routing](routing.md). `http.body_limits.upload_bytes` is required and validated at start-up, but the framework applies it to no route; a plugin that wants its upload routes to follow it reads it in `Register` and puts the value in the route's `body.limit`.

View File

@@ -26,6 +26,9 @@ Each row names the WinterCMS concept, the SummerCMS identifiers that replace it,
| Laravel collections: `map`, `filter`, `reduce`, `keyBy`, `groupBy`, `unique`, `partition`, `first` and `pluck` | Generic functions on plain slices and maps: `sieve.Map`, `sieve.TryMap`, `sieve.Filter`, `sieve.Reduce`, `sieve.KeyBy`, `sieve.GroupBy`, `sieve.Uniq`, `sieve.Partition` and `sieve.Find`. Pluck is `sieve.Map` with a field accessor, unique by key is `sieve.UniqBy`, `chunk` is the standard `slices.Chunk`, and sets are built with `sieve.SetOf` | [sieve](../../modules/sieve/README.md) |
| `fields.yaml` and `columns.yaml` | The same YAML, embedded in the plugin through `pact.AdminAssets` and compiled at boot into a `cabana.CompiledController` | [Forms](../backend/forms.md), [Lists and filters](../backend/lists-and-filters.md), [cabana](../../modules/cabana/README.md) |
| Backend controllers with the Form, List and Relation behaviours | A `pact.AdminController` returned from `pact.HasAdminControllers`; the generic admin API replaces the behaviours, and hooks such as `pact.FormBeforeCreate` and `pact.ListExtendQuery` replace behaviour overrides | [Admin controllers](../backend/admin-controllers.md), [cabana](../../modules/cabana/README.md) |
| The Reorder behaviour and `config_reorder.yaml` | A `config_reorder.yaml` next to `config_list.yaml` and the `reorder` toolbar button: a drag-and-drop tree, or a flat list for records without a parent column; `pact.ReorderAfterSave` runs after a reorder save | [Lists and filters](../backend/lists-and-filters.md#tree-reorder), [cabana](../../modules/cabana/README.md) |
| Form widgets and `registerFormWidgets` | A plugin field type: `pact.HasFormFieldTypes` returning `pact.FormFieldType` values, usable as `type:` in any controller's `fields.yaml`; a one-off element with a server action is a `type: widget` field | [Plugin field types](../backend/field-types.md), [Partials and widgets](../backend/partials-and-widgets.md#form-widgets) |
| HTML cleaning of rich editor content | Every `richeditor` value is sanitized on save by `seaglass.Sanitize`, an allowlist policy; plugin code calls `seaglass.SanitizeReport` to learn what was removed | [Rich text](../backend/forms.md#rich-text), [seaglass](../../modules/seaglass/README.md) |
| `routes.php` | `pact.HasRoutes`, declaring routes on a `pact.Router` with groups, middleware names and `pact.Router.Where` constraints | [Routing](../services/routing.md), [surf](../../modules/surf/README.md) |
| Route middleware | Named middleware of type `pact.Middleware`, registered through `pact.HasMiddleware` | [Routing](../services/routing.md), [surf](../../modules/surf/README.md) |
| `config/*.php`, `.env` and `Config::get` | `compass.Config` with per-environment directories and `SUMMER_` overrides; plugin defaults through `pact.HasConfig` | [Configuration](../services/configuration.md), [compass](../../modules/compass/README.md) |
@@ -37,6 +40,12 @@ Each row names the WinterCMS concept, the SummerCMS identifiers that replace it,
| `registerSchedule` and the scheduler | `pact.HasSchedule` returning `pact.ScheduledCommand` entries with a `pact.Cadence` | [Task scheduling](../plugins/scheduling.md), [pact](../../modules/pact/README.md) |
| Mail templates in `views/mail` | `pact.HasMailTemplates`, sent through `postcard.Mailer` | [Mail](../services/mail.md), [postcard](../../modules/postcard/README.md) |
| Settings models and `registerSettings` | `pact.HasSettings` returning `pact.SettingsItem` entries; an entry with `'url' => Backend::url(...)` sets the `Controller` field to the admin controller ID | [Settings](../backend/settings.md), [cabana](../../modules/cabana/README.md) |
| A backend controller that only ever edits one record (its index redirects to the update screen) | A singleton controller: `pact.AdminSingleton` returns the one record, and a settings entry's `Controller` links to it | [Admin controllers](../backend/admin-controllers.md#singleton-controllers), [Settings](../backend/settings.md) |
| Personal access tokens for API clients | Administrator API tokens with scopes and expiry, sent as `Authorization: Bearer g15_…` and managed in My account or with `admin:token:create` and the other `admin:token:*` commands | [Users and permissions](../backend/users-and-permissions.md#api-tokens), [cabana](../../modules/cabana/README.md) |
| The Laravel `Cache` facade and `Cache::remember` | `cooler.Cache` with `cooler.Cache.Get`, `cooler.Cache.Set`, `cooler.Cache.Remember` and `cooler.Cache.Forget`, kept in memory or in Redis (`cache.driver`) | [Caching](../services/caching.md), [cooler](../../modules/cooler/README.md) |
| The Laravel CORS config (`config/cors.php`) | The same keys under `http.cors`, plus named profiles under `http.cors_profiles` applied with the `cors:<profile>` middleware | [Routing](../services/routing.md#cors), [surf](../../modules/surf/README.md) |
| A captcha on public forms | Cloudflare Turnstile as the `turnstile:open` and `turnstile:closed` middleware, with `turnstile.secret` | [Authentication](../services/authentication.md#turnstile), [bouncer](../../modules/bouncer/README.md) |
| The Static Pages menu item events `pages.menuitem.listTypes`, `pages.menuitem.getTypeInfo` and `pages.menuitem.resolveItem` | Typed menu item types: `pact.HasMenuItemTypes` returning `pact.MenuItemType` values, collected in the `signpost.Registry` that menu and sitemap plugins read | [Menu item types](../plugins/menu-item-types.md), [signpost](../../modules/signpost/README.md) |
| Laravel broadcasting | `lighthouse.Publisher` drivers and models that implement `lighthouse.Broadcastable` | [Realtime](../services/realtime.md), [lighthouse](../../modules/lighthouse/README.md) |
| Laravel Scout search | Models that implement `beachcomber.Searchable`, synced after commit | [Search](../services/search.md), [beachcomber](../../modules/beachcomber/README.md) |
| The Laravel HTTP client | `fetchguard.Fetch` with a `fetchguard.Policy` that blocks private addresses and limits size and time | [Outbound HTTP](../services/outbound-http.md), [fetchguard](../../modules/fetchguard/README.md) |
@@ -53,10 +62,8 @@ SummerCMS does not port the WinterCMS frontend or the PHP helpers that Go alread
| The AJAX framework and Snowboard | Not provided. The frontend calls the JSON API and subscribes to realtime channels; see [Frontend and AJAX](../services/frontend-and-ajax.md). |
| The media manager | Not provided. Store uploads as model attachments; see [Attachments](../database/attachments.md). |
| Import and export in backend lists | Not provided. |
| Record sorting (the Reorder behaviour) | Not provided. |
| The fluent collection object (the Collection class and method chaining) | Not provided. Call the [sieve](../../modules/sieve/README.md) functions and the standard `slices` and `maps` packages on plain slices and maps. |
| Behaviours and dynamic class extension | Not provided. Use Go interfaces and composition. |
| Cache | Not provided. Use the Go standard library or a service another plugin publishes. |
| Session | Not provided. The API is stateless and authenticates with tokens; see [Frontend and AJAX](../services/frontend-and-ajax.md). |
## Plugin.php in Go

View File

@@ -52,10 +52,19 @@ These are the keys most applications set. Each module's reference page lists all
| `app.timezone` | Timezone of scheduled commands. Defaults to UTC. | [conga](../../modules/conga/README.md) |
| `app.locale`, `app.fallback_locale` | Default and fallback translation locales. | [phrasebook](../../modules/phrasebook/README.md) |
| `http.body_limits.default_bytes`, `http.body_limits.upload_bytes` | Request body limits. Required by `serve` and `route:list`. | [surf](../../modules/surf/README.md) |
| `http.cors.*`, `http.trusted_proxies` | CORS and the proxies whose forwarded client IP is trusted. | [surf](../../modules/surf/README.md) |
| `http.cors.*`, `http.trusted_proxies` | CORS and the proxies whose forwarded client IP is trusted. With `supports_credentials: true`, a wildcard host other than the whole leftmost label and an origin pattern not anchored with `^` and `$` fail the start-up. | [surf](../../modules/surf/README.md) |
| `http.cors_profiles.*` | Named CORS profiles with the `http.cors` keys except `paths`, applied to a route or group with the `cors:<profile>` middleware; checked like `http.cors`. | [surf](../../modules/surf/README.md) |
| `http.api_docs.enabled`, `http.api_docs.path`, `http.api_docs.title` | The opt-in Swagger UI page for the public API (off by default; path `/api-docs`, title `API documentation`). | [surf](../../modules/surf/README.md) |
| `http.rate_limit.store` | Where `throttle` counters live: `memory` (per instance, the default) or `redis` (shared by every instance). | [cooler](../../modules/cooler/README.md) |
| `cache.driver`, `cache.prefix`, `cache.memory.max_entries` | The cache driver (`memory`, the default, or `redis`), the key prefix (default from `app.name`) and the memory driver's size bound (default 10000). | [cooler](../../modules/cooler/README.md) |
| `site_cache.purge_url`, `site_cache.purge_token` | The host site's page-cache purge route and its bearer token (at least 32 characters). Set both or neither; http only to a loopback host. Keep the token in `SUMMER_SITE_CACHE__PURGE_TOKEN`. | [cooler](../../modules/cooler/README.md) |
| `redis.url` | Redis URL (`redis://`, `rediss://` or `unix://`), required when a Redis cache or rate-limit store is selected. Keep it in `SUMMER_REDIS__URL` when it carries a password. | [cooler](../../modules/cooler/README.md) |
| `http.access_log` | Logs one record per request with method, matched route pattern, status and duration. Defaults to `false`. | [surf](../../modules/surf/README.md) |
| `storage.uploads.bucket_url` | Uploads bucket, `file://` or `mem://`. Required by `serve`. | [lagoon](../../modules/lagoon/README.md) |
| `storage.uploads.bucket_url` | Uploads bucket, `file://`, `mem://` or `s3://` (the last only in a binary built with the `summercms_s3` tag from `build.tags` in `summer.yaml`; credentials from the AWS environment). Required by `serve`. | [lagoon](../../modules/lagoon/README.md) |
| `storage.uploads.public_url` | Absolute `http` or `https` base of a public bucket or CDN; file and thumbnail URLs become that base plus the partition path. Unset keeps `storage.uploads.public_path_prefix`. Only for a bucket holding public files alone. | [lagoon](../../modules/lagoon/README.md) |
| `storage.uploads.serve` | Mounts `attach.StaticHandlerPublic` under `storage.uploads.public_path_prefix` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Defaults to `false`. | [surf](../../modules/surf/README.md) |
| `auth.cookie.name`, `auth.cookie.domain`, `auth.cookie.path`, `auth.cookie.secure`, `auth.cookie.same_site` | The httpOnly token cookie for a browser frontend (defaults `token`, host-only, `/`, Secure when `app.url` is https, `lax`). In production a cookie that is not Secure fails the start-up. | [bouncer](../../modules/bouncer/README.md) |
| `turnstile.secret`, `turnstile.field` | The Cloudflare Turnstile secret behind the `turnstile:open` and `turnstile:closed` middleware (keep it in `SUMMER_TURNSTILE__SECRET`) and the request field carrying the token (default `turnstile_token`). | [surf](../../modules/surf/README.md) |
| `admin.jwt.secret` | Secret for admin tokens. Required as soon as a plugin registers an admin controller. | [cabana](../../modules/cabana/README.md) |
| `backend.uri` | Path the admin is served under. Defaults to `/backend`. | [cabana](../../modules/cabana/README.md) |
| `mail.driver`, `mail.from`, `mail.smtp.*` | Mail delivery: `memory`, `log` or `smtp`. | [postcard](../../modules/postcard/README.md) |
@@ -64,6 +73,8 @@ These are the keys most applications set. Each module's reference page lists all
| `search.driver`, `search.typesense.*` | The search engine and its Typesense settings. | [beachcomber](../../modules/beachcomber/README.md) |
| `push.enabled`, `push.public_key`, `push.private_key` | Web Push with VAPID keys. | [flare](../../modules/flare/README.md) |
Some keys are checked more strictly from 0.2 on, so a value that started an older release can stop the start-up; [Upgrading](upgrading.md#upgrading-to-02) lists them and their fixes.
> [!NOTE]
> The documentation checks every identifier, link and command name it shows against the code, but not configuration keys. The keys on this page are reviewed by hand; the module reference pages are the authority.

View File

@@ -30,9 +30,12 @@ The data layer supports PostgreSQL only, and SummerCMS targets Go 1.27.
## How these docs are organised
- **Setup** covers installation, configuration and the move from WinterCMS.
- **Setup** covers installation, configuration, the move from WinterCMS and [Upgrading](upgrading.md) to a newer framework release.
- **Architecture** explains the single binary, Go modules, the application lifecycle and how a request reaches your code.
- **Plugins** covers registering a plugin, scheduling, extending other plugins and testing.
- **Plugins** covers registering a plugin, scheduling, extending other plugins, menu item types and testing.
- **Backend** covers the admin: controllers, forms and field types, lists and filters, relations, users and API tokens, settings and the admin SPA.
- **Database** covers models, migrations, queries and pagination, relations, casts and validation, attachments and transactions.
- **Services** covers configuration, events, routing, authentication, caching, rate limiting, storage, logging, mail, jobs, realtime, search, push and the other runtime services.
- **Console** lists the `summer` tool's commands and the commands of every application binary, and shows how to write your own.
- **API reference** has one page per framework module.

73
docs/setup/upgrading.md Normal file
View File

@@ -0,0 +1,73 @@
---
title: Upgrading
description: Move an application to a newer framework release, what the version numbers promise, and the config to check before you take 0.2.
section: setup
order: 60
---
# Upgrading
An application created by `summer new` builds against a framework checkout next to it; the update script moves that checkout to the newest release tag, reinstalls the `summer` tool and rebuilds the application (see [Updating an application](installation.md#updating-an-application)). An application or plugin that requires a tagged framework version in its `go.mod` moves with the go command instead:
```sh
go get git.golem15.com/golem15/summercms@v0.2.0
go mod tidy
```
Then rebuild, run `migrate` with the new binary and restart it.
## Version numbers
SummerCMS is in the 0.x series, and the version tells you whether to read before you upgrade:
- The minor version (0.1 to 0.2) goes up when an upgrade can refuse a configuration that the previous release accepted, or changes a documented behaviour. Read the section for that version below before you deploy it.
- A patch version (0.2.0 to 0.2.1) is safe to take without reading: it fixes bugs and adds features, and an application that started on the previous patch starts on it too.
Applications pin exact versions, so a new release never reaches an application until it moves its `go.mod` or its checkout.
## Upgrading to 0.2
0.2 tightens two configuration checks so that settings which leaked credentials stop the start-up instead. Most applications are not affected: the first concerns applications that load an `auth.cookie` block, the second applications that send credentialed CORS answers. Start the new binary once in each environment before you switch traffic to it; a refused configuration fails `serve` at start-up with an error that names the key.
### Boot refusals
**A token cookie that is not Secure in production.** In the production environment, `bouncer.LoadCookieConfig` refuses a token cookie whose `Secure` attribute resolves to false. That covers `auth.cookie.secure: false` and, since 0.2, an unset `auth.cookie.secure` with an http or empty `app.url`, from which the cookie inherited `Secure=false` before. The error names `auth.cookie.secure` and `app.url`.
It affects applications whose plugins load an `auth.cookie` block for a browser frontend (the cookie transport). To fix it, set `app.url` to the application's public https URL, or set `auth.cookie.secure: true` explicitly. See [Cookie transport](../services/authentication.md#cookie-transport).
**Credentialed CORS with a wildcard host or an unanchored pattern.** With `supports_credentials: true`, globally under `http.cors` or in any profile under `http.cors_profiles.<name>`, these entries now fail the start-up, and the error names the key with its index, such as `http.cors.allowed_origins[1]` or `http.cors_profiles.spa.allowed_origins_patterns[0]`:
- an `allowed_origins` entry with a `*` that is not the whole leftmost host label after a literal scheme, such as `https://*`, `*://*`, `https://*example.com` or `http*://www.example.com` (`https://*.example.com` stays valid);
- an `allowed_origins_patterns` entry not anchored with `^` and `$`, in every alternative of the expression;
- any origin rule that matches an origin outside the configured domains, such as `https://*.com` or `^https?://.*$`.
It affects applications that send credentialed CORS answers, to a browser frontend on another origin that calls the API with cookies. To fix it, list the frontend's exact origins, or write anchored patterns such as `^https://[a-z]+\.example\.com$`. Without credentials these entries load as before. See [CORS](../services/routing.md#cors) and [CORS profiles](../services/routing.md#cors-profiles).
### Behaviour changes
These need no configuration change, but a test or an operator may notice them:
- **Site-cache purges are queued.** After an admin save commits, the purge request to the site is queued and sent by a background worker, with at most one request in flight per purger; saves committed meanwhile collapse into one follow-up request. The save no longer waits for the site. A test that asserts a purge calls `cooler.SiteCachePurger.Wait` first, and `serve` drains the queue on shutdown. See [Site cache purge](../services/caching.md#site-cache-purge).
- **Protected uploads need a private bucket.** While `storage.uploads.public_url` is set, `attach.Store` refuses a protected (`is_public` false) upload with `attach.ErrProtectedNeedsPrivateBucket` before it writes anything, and `serve` logs one warning at start-up that the bucket must hold public files only. Keep protected relations on a bucket without `public_url`, or make the relation public. See [Public URL base](../services/storage.md#public-url-base).
- **The Journal plugin's purge keys are deprecated.** `golem15.journal.site_cache.purge_url` and `golem15.journal.site_cache.purge_token` still work as aliases of the framework's `site_cache.purge_url` and `site_cache.purge_token`, which win when both are set; the plugin logs a deprecation warning. Move the values to the `site_cache` keys.
- **Cookie defaults.** A `bouncer.CookieConfig` built by hand with every field zero writes `SameSite=Lax` and `Secure`; a config with any field set keeps its own `Secure` value.
- **The API documentation page** at `/api-docs` runs only scripts that carry the page's per-request nonce. See [API documentation page](../services/routing.md#api-documentation-page).
- **Turnstile without a secret.** When `turnstile.secret` is empty and routes name `turnstile:open` or `turnstile:closed`, the start-up logs one warning with the number of such routes. See [Turnstile](../services/authentication.md#turnstile).
- **CORS profile headers on every answer.** A route that names `cors:<profile>` carries the profile's headers on every response, including refusals from guards, throttle, Turnstile and the body limit, and recovered panics.
## What 0.2 adds since 0.1
The 0.1 series grew through many patch releases. These are the larger features that arrived after its first public releases, with the page that documents each:
- Caching with memory and Redis drivers, and a Redis store that shares `throttle` counters between instances: [Caching](../services/caching.md), [Rate limiting](../services/rate-limiting.md).
- The site-cache purge after admin saves, including reorder saves through `pact.ReorderAfterSave`: [Site cache purge](../services/caching.md#site-cache-purge).
- Token cookies with ready-made refresh and logout handlers, and Turnstile bot protection: [Cookie transport](../services/authentication.md#cookie-transport), [Turnstile](../services/authentication.md#turnstile).
- Credentialed CORS, per-route CORS profiles (`cors:<profile>`) and the opt-in API documentation page: [CORS](../services/routing.md#cors), [CORS profiles](../services/routing.md#cors-profiles), [API documentation page](../services/routing.md#api-documentation-page).
- Go build tags in `summer.yaml` (`build.tags`), S3 uploads buckets behind the `summercms_s3` tag and a public URL base for a bucket or CDN: [The summer.yaml manifest](../architecture/go-modules-and-workspaces.md#the-summeryaml-manifest), [S3 and S3-compatible buckets](../services/storage.md#s3-and-s3-compatible-buckets), [Public URL base](../services/storage.md#public-url-base).
- Serving uploads from the application and a per-request access log: [Serving files](../services/storage.md#serving-files), [Request log](../services/logging.md#request-log).
- Plugin-provided admin field types: [Plugin field types](../backend/field-types.md).
- Rich text, repeater, money, password and taglist fields, and dependent dropdowns: [Rich text](../backend/forms.md#rich-text), [Repeater](../backend/forms.md#repeater), [Money](../backend/forms.md#money), [Form-only fields](../backend/forms.md#form-only-fields), [Taglist](../backend/forms.md#taglist), [Dependent dropdowns](../backend/forms.md#dependent-dropdowns).
- Record reordering as a tree or a flat list: [Tree reorder](../backend/lists-and-filters.md#tree-reorder), [Flat reorder](../backend/lists-and-filters.md#flat-reorder).
- Singleton controllers, record actions and write-only settings secrets: [Singleton controllers](../backend/admin-controllers.md#singleton-controllers), [Record actions](../backend/admin-controllers.md#record-actions), [Write-only secrets](../backend/settings.md#write-only-secrets).
- Personal API tokens for administrators and the `admin:token:*` commands: [API tokens](../backend/users-and-permissions.md#api-tokens), [Admin accounts](../console/setup-and-maintenance.md#admin-accounts).
- Menu item types that content plugins register for menus and sitemaps: [Menu item types](../plugins/menu-item-types.md).
- New framework modules: [cooler](../../modules/cooler/README.md) (caches, shared counters and the site-cache purge), [seaglass](../../modules/seaglass/README.md) (the rich text sanitizer), [sieve](../../modules/sieve/README.md) (collection helpers) and [signpost](../../modules/signpost/README.md) (the menu item type registry).

25
go.mod
View File

@@ -7,6 +7,7 @@ ignore ./admin/node_modules
require (
github.com/alecthomas/chroma/v2 v2.27.0
github.com/alicebob/miniredis/v2 v2.40.0
github.com/disintegration/imaging v1.6.2
github.com/fsnotify/fsnotify v1.10.1
github.com/go-gormigrate/gormigrate/v2 v2.1.7
@@ -19,8 +20,10 @@ require (
github.com/knadh/koanf/providers/env/v2 v2.0.1
github.com/knadh/koanf/providers/file v1.2.1
github.com/knadh/koanf/v2 v2.3.6
github.com/maypok86/otter/v2 v2.3.0
github.com/microcosm-cc/bluemonday v1.0.27
github.com/nicksnyder/go-i18n/v2 v2.6.1
github.com/redis/go-redis/v9 v9.23.0
github.com/riverqueue/river v0.47.0
github.com/riverqueue/river/riverdriver/riverdatabasesql v0.47.0
github.com/riverqueue/river/rivertype v0.47.0
@@ -43,6 +46,25 @@ require (
dario.cat/mergo v1.0.2 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/aws/aws-sdk-go-v2 v1.41.9 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 // indirect
github.com/aws/aws-sdk-go-v2/config v1.32.20 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.19.19 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.25 // indirect
github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.2.3 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.25 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.25 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.26 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.25 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25 // indirect
github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.1.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.19 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.2 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.42.3 // indirect
github.com/aws/smithy-go v1.26.0 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
@@ -65,6 +87,7 @@ require (
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/google/wire v0.7.0 // indirect
github.com/googleapis/gax-go/v2 v2.19.0 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
@@ -106,6 +129,7 @@ require (
github.com/tidwall/sjson v1.2.5 // indirect
github.com/tklauser/go-sysconf v0.4.0 // indirect
github.com/tklauser/numcpus v0.12.0 // indirect
github.com/yuin/gopher-lua v1.1.1 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 // indirect
@@ -114,6 +138,7 @@ require (
go.opentelemetry.io/otel/sdk v1.44.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.44.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.uber.org/atomic v1.12.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect

24
go.sum
View File

@@ -36,6 +36,8 @@ github.com/alecthomas/chroma/v2 v2.27.0 h1:FodwmyOBgJULFYmDqibcp9pvfDLWdtPRh9v/r
github.com/alecthomas/chroma/v2 v2.27.0/go.mod h1:NjJ3ciIgrqBNeIkWZ4e46nseoLDslxU1LmfCoL+wcY8=
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alicebob/miniredis/v2 v2.40.0 h1:6IgTQbvm2JiMYZ8yur/acNJzVjtSOWCelxXbk/LUBXc=
github.com/alicebob/miniredis/v2 v2.40.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/aws/aws-sdk-go-v2 v1.41.9 h1:/rYeyO2+HrMztAmxAq9++XJtFMqSIpSsNA0yDGALYq4=
github.com/aws/aws-sdk-go-v2 v1.41.9/go.mod h1:+HsoOEX80qAVUitj1A2DhCNTjmb3edVyuDypb6LNEeo=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 h1:h5+3VT69KUBK24grGuuA5saDJTj2IIjLb9au668Fo5I=
@@ -76,6 +78,10 @@ github.com/aws/smithy-go v1.26.0 h1:9ouqbi+NyKP7fV3Te7UElCwdAb6Y8uk7LGwPE5tVe/s=
github.com/aws/smithy-go v1.26.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
@@ -149,6 +155,12 @@ github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-replayers/grpcreplay v1.3.0 h1:1Keyy0m1sIpqstQmgz307zhiJ1pV4uIlFds5weTmxbo=
github.com/google/go-replayers/grpcreplay v1.3.0/go.mod h1:v6NgKtkijC0d3e3RW8il6Sy5sqRVUwoQa4mHOGEy8DI=
github.com/google/go-replayers/httpreplay v1.2.0 h1:VM1wEyyjaoU53BwrOnaf9VhAyQQEEioJvFYxYcLRKzk=
github.com/google/go-replayers/httpreplay v1.2.0/go.mod h1:WahEFFZZ7a1P4VM1qEeHy+tME4bwyqPcwWbNlUI1Mcg=
github.com/google/martian/v3 v3.3.3 h1:DIhPTQrbPkgs2yJYdXU/eNACCG5DVQjySNRNlflZ9Fc=
github.com/google/martian/v3 v3.3.3/go.mod h1:iEPrYcgCF7jA9OtScMFQyAlZZ4YXTKEtJ1E6RWzmBA0=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
@@ -181,6 +193,8 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw=
github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU=
github.com/knadh/koanf/maps v0.1.2 h1:RBfmAW5CnZT+PJ1CVc1QSJKf4Xu9kxfQgYVQSu8hpbo=
github.com/knadh/koanf/maps v0.1.2/go.mod h1:npD/QZY3V6ghQDdcQzl1W4ICNVTkohC8E73eI2xW4yI=
github.com/knadh/koanf/parsers/yaml v1.1.1 h1:u70vV5IyaM0HvONh8HoqBC97oTgO33KcpZbTLiKVinU=
@@ -203,6 +217,8 @@ github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8S
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o4kU=
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/maypok86/otter/v2 v2.3.0 h1:8H8AVVFUSzJwIegKwv1uF5aGitTY+AIrtktg7OcLs8w=
github.com/maypok86/otter/v2 v2.3.0/go.mod h1:XgIdlpmL6jYz882/CAx1E4C1ukfgDKSaw4mWq59+7l8=
github.com/mdelapenya/tlscert v0.2.0 h1:7H81W6Z/4weDvZBNOfQte5GpIMo0lGYEeWbkGp5LJHI=
github.com/mdelapenya/tlscert v0.2.0/go.mod h1:O4njj3ELLnJjGdkN7M/vIVCpZ+Cf0L6muqOG4tLSl8o=
github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
@@ -240,6 +256,8 @@ github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/redis/go-redis/v9 v9.23.0 h1:/TTVdJa8BRNsybwwGGnmiYktHNlUXtbOQxlWpHBPL3s=
github.com/redis/go-redis/v9 v9.23.0/go.mod h1:EWP2UUk+XDaI1s5nQB5mablZN6vccXlmBQTyfnF87D0=
github.com/riverqueue/river v0.47.0 h1:j8HOEyiOE8gRRhVS2wllKams372TH1WWiH6xCakXHqc=
github.com/riverqueue/river v0.47.0/go.mod h1:Wgmwx475ZBd8lQnNrJgyG2DWH7BfyNiSAtv0rC9bJBQ=
github.com/riverqueue/river/riverdriver v0.47.0 h1:qU8VkjdMl9plqeRg57SxsDUM/i/eECaSYejZ7HynC60=
@@ -295,8 +313,12 @@ github.com/wneessen/go-mail v0.8.1 h1:tVcncj02/QySVFw3zr/kXOzZcuFQqBNT6K+Rbgm/pc
github.com/wneessen/go-mail v0.8.1/go.mod h1:dWZ61zadzCIyvB4y1/YzC5O7MrbbzBfPkARmbosdf8w=
github.com/yuin/goldmark v1.8.6 h1:d0VcaP1sx9GkFVkoW+KtggpGi2KZ965i14b0+bDQST4=
github.com/yuin/goldmark v1.8.6/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/detectors/gcp v1.42.0 h1:kpt2PEJuOuqYkPcktfJqWWDjTEd/FNgrxcniL7kQrXQ=
@@ -317,6 +339,8 @@ go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRk
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.uber.org/atomic v1.12.0 h1:BvcXdFKuviU4fTL/f+SxdQ5qJX/Jix8pAkgdUcb3XOE=
go.uber.org/atomic v1.12.0/go.mod h1:I6c4cg+6HCxRjfjSsYtApoFILnpc0CGUdGkXVqbYVNk=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=

View File

@@ -46,7 +46,10 @@ func App(ctx context.Context, appDir string, out io.Writer) error {
return fmt.Errorf("build: create bin dir: %w", err)
}
cmd := exec.CommandContext(ctx, "go", "build", "-o", binRel, ".")
if len(m.Tags) > 0 {
fmt.Fprintf(out, "build tags: %s\n", strings.Join(m.Tags, ","))
}
cmd := exec.CommandContext(ctx, "go", goBuildArgs(binRel, m.Tags)...)
cmd.Dir = appDir
cmd.Stdout = out
cmd.Stderr = out
@@ -59,6 +62,17 @@ func App(ctx context.Context, appDir string, out io.Writer) error {
return nil
}
// goBuildArgs returns the go command arguments for summer build: build, the
// -tags flag when the manifest lists build tags, then -o binRel and the
// application package. The tags travel as one exec argument, never a shell.
func goBuildArgs(binRel string, tags []string) []string {
args := []string{"build"}
if len(tags) > 0 {
args = append(args, "-tags="+strings.Join(tags, ","))
}
return append(args, "-o", binRel, ".")
}
func generate(appDir string, m Manifest) error {
mainSrc, err := generateMain(m)
if err != nil {

View File

@@ -13,13 +13,18 @@ import (
const manifestName = "summer.yaml"
var pluginIDRe = regexp.MustCompile(`^[a-z][a-z0-9]*\.[a-z][a-z0-9]*$`)
var (
pluginIDRe = regexp.MustCompile(`^[a-z][a-z0-9]*\.[a-z][a-z0-9]*$`)
buildTagRe = regexp.MustCompile(`^[a-z0-9_]+$`)
)
// Manifest is the ordered app plugin list used to generate imports.
// Manifest is the ordered app plugin list used to generate imports, plus
// the Go build tags summer build passes to go build (summer.yaml build.tags).
type Manifest struct {
Module string
Binary string
Plugins []Plugin
Tags []string
}
// Plugin is one compiled plugin module in manifest order.
@@ -76,10 +81,67 @@ func LoadManifest(path string) (Manifest, error) {
seenMod[p.Module] = struct{}{}
m.Plugins = append(m.Plugins, p)
}
tags, err := loadBuildTags(tree["build"])
if err != nil {
return Manifest{}, err
}
m.Tags = tags
return m, nil
}
// SaveManifest writes summer.yaml with plugins in slice order.
// loadBuildTags reads the optional build mapping and its tags list. Entries
// are trimmed and validated; duplicates are dropped keeping the first.
func loadBuildTags(raw any) ([]string, error) {
if raw == nil {
return nil, nil
}
section := asMap(raw)
if section == nil {
return nil, fmt.Errorf("build: build must be a mapping (build.tags)")
}
rawTags, ok := section["tags"]
if !ok || rawTags == nil {
return nil, nil
}
list, ok := rawTags.([]any)
if !ok {
return nil, fmt.Errorf("build: build.tags must be a list of tags")
}
var tags []string
seen := make(map[string]struct{}, len(list))
for i, item := range list {
s, ok := item.(string)
if !ok {
return nil, fmt.Errorf("build: build.tags[%d] must be a string", i)
}
tag := strings.TrimSpace(s)
if err := ValidateBuildTag(tag); err != nil {
return nil, fmt.Errorf("build: build.tags[%d]: %w", i, err)
}
if _, dup := seen[tag]; dup {
continue
}
seen[tag] = struct{}{}
tags = append(tags, tag)
}
return tags, nil
}
// ValidateBuildTag requires a Go build tag of lowercase letters, digits and
// underscores, so a summer.yaml value can never smuggle a flag into go build.
func ValidateBuildTag(tag string) error {
if tag == "" {
return fmt.Errorf("build tag is empty")
}
if !buildTagRe.MatchString(tag) {
return fmt.Errorf("build tag %q must match ^[a-z0-9_]+$", tag)
}
return nil
}
// SaveManifest writes summer.yaml with plugins in slice order and, when
// present, the build.tags list.
func SaveManifest(path string, m Manifest) error {
if err := validateModulePath(m.Module); err != nil {
return fmt.Errorf("build: module: %w", err)
@@ -101,6 +163,16 @@ func SaveManifest(path string, m Manifest) error {
fmt.Fprintf(&b, " - id: %s\n", p.ID)
fmt.Fprintf(&b, " module: %s\n", p.Module)
}
if len(m.Tags) > 0 {
b.WriteString("build:\n")
b.WriteString(" tags:\n")
for i, tag := range m.Tags {
if err := ValidateBuildTag(tag); err != nil {
return fmt.Errorf("build: build.tags[%d]: %w", i, err)
}
fmt.Fprintf(&b, " - %s\n", tag)
}
}
if err := os.WriteFile(path, []byte(b.String()), 0o644); err != nil {
return fmt.Errorf("build: write %s: %w", path, err)
}

View File

@@ -0,0 +1,70 @@
package build
import (
"os"
"path/filepath"
"slices"
"strings"
"testing"
)
func TestManifestBuildTagsSmoke(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, manifestName)
body := "module: git.golem15.com/acme/acme\n" +
"binary: acme\n" +
"plugins:\n" +
" - id: acme.blog\n" +
" module: git.golem15.com/acme/acme/plugins/blog\n" +
"build:\n" +
" tags:\n" +
" - summercms_s3\n" +
" - \" summercms_s3 \"\n"
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
m, err := LoadManifest(path)
if err != nil {
t.Fatal(err)
}
if !slices.Equal(m.Tags, []string{"summercms_s3"}) {
t.Fatalf("Tags = %q, want [summercms_s3]", m.Tags)
}
if err := SaveManifest(path, m); err != nil {
t.Fatal(err)
}
again, err := LoadManifest(path)
if err != nil {
t.Fatal(err)
}
if !slices.Equal(again.Tags, m.Tags) || len(again.Plugins) != 1 {
t.Fatalf("round trip = %+v, want tags %q and one plugin", again, m.Tags)
}
args := goBuildArgs("bin/acme", again.Tags)
tagAt := slices.Index(args, "-tags=summercms_s3")
outAt := slices.Index(args, "-o")
if tagAt < 0 || outAt < 0 || tagAt > outAt {
t.Fatalf("goBuildArgs = %q, want -tags=summercms_s3 before -o", args)
}
if plain := goBuildArgs("bin/acme", nil); !slices.Equal(plain, []string{"build", "-o", "bin/acme", "."}) {
t.Fatalf("goBuildArgs without tags = %q", plain)
}
for _, bad := range []string{"summercms-s3", "summercms s3"} {
if err := ValidateBuildTag(bad); err == nil {
t.Fatalf("ValidateBuildTag(%q) accepted", bad)
}
raw := "module: git.golem15.com/acme/acme\nbinary: acme\nbuild:\n tags:\n - \"" + bad + "\"\n"
if err := os.WriteFile(path, []byte(raw), 0o644); err != nil {
t.Fatal(err)
}
if _, err := LoadManifest(path); err == nil || !strings.Contains(err.Error(), "build.tags") {
t.Fatalf("LoadManifest with tag %q: err = %v, want a build.tags error", bad, err)
}
}
if err := SaveManifest(path, Manifest{Module: "git.golem15.com/acme/acme", Binary: "acme", Tags: []string{"a b"}}); err == nil {
t.Fatal("SaveManifest accepted an invalid tag")
}
}

View File

@@ -0,0 +1,189 @@
package build
import (
"os"
"path/filepath"
"slices"
"strings"
"testing"
)
// Unit tests (plan 05 of phase 19.3) for summer.yaml build.tags (D-12).
const tagsManifestHead = "module: git.golem15.com/acme/acme\n" +
"binary: acme\n" +
"plugins:\n" +
" - id: acme.blog\n" +
" module: git.golem15.com/acme/acme/plugins/blog\n"
func writeManifest(t *testing.T, body string) string {
t.Helper()
path := filepath.Join(t.TempDir(), manifestName)
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
return path
}
func TestManifestBuildTags(t *testing.T) {
for _, tc := range []struct {
name string
body string
want []string
}{
{"no build block", tagsManifestHead, nil},
{"empty build block", tagsManifestHead + "build: {}\n", nil},
{"null tags", tagsManifestHead + "build:\n tags:\n", nil},
{"empty list", tagsManifestHead + "build:\n tags: []\n", nil},
{"one tag", tagsManifestHead + "build:\n tags: [summercms_s3]\n", []string{"summercms_s3"}},
{"trimmed and deduplicated in order", tagsManifestHead + "build:\n tags:\n - \" netgo \"\n - summercms_s3\n - netgo\n - \"summercms_s3\\t\"\n - osusergo\n", []string{"netgo", "summercms_s3", "osusergo"}},
{"digits and underscores", tagsManifestHead + "build:\n tags: [go1_x, \"123\"]\n", []string{"go1_x", "123"}},
} {
t.Run(tc.name, func(t *testing.T) {
m, err := LoadManifest(writeManifest(t, tc.body))
if err != nil {
t.Fatal(err)
}
if !slices.Equal(m.Tags, tc.want) {
t.Fatalf("Tags = %q, want %q", m.Tags, tc.want)
}
if len(m.Plugins) != 1 || m.Plugins[0].ID != "acme.blog" {
t.Fatalf("plugins = %+v", m.Plugins)
}
})
}
for _, tag := range []string{"summercms_s3", "a", "0", "x_1_y"} {
if err := ValidateBuildTag(tag); err != nil {
t.Errorf("ValidateBuildTag(%q) = %v", tag, err)
}
}
}
func TestManifestBuildTagsInvalid(t *testing.T) {
for _, tc := range []struct {
name string
yaml string
want string
}{
{"dash", "build:\n tags: [summercms-s3]\n", "build.tags[0]"},
{"space", "build:\n tags: [netgo, \"summercms s3\"]\n", "build.tags[1]"},
{"upper case", "build:\n tags: [SummerCMS_S3]\n", "build.tags[0]"},
{"flag injection", "build:\n tags: [\"x -ldflags=-X\"]\n", "build.tags[0]"},
{"comma", "build:\n tags: [\"a,b\"]\n", "build.tags[0]"},
{"empty string", "build:\n tags: [\" \"]\n", "build.tags[0]"},
{"non-string entry", "build:\n tags: [42]\n", "build.tags[0]"},
{"non-list tags", "build:\n tags: summercms_s3\n", "build.tags"},
{"mapping tags", "build:\n tags:\n s3: true\n", "build.tags"},
{"non-mapping build", "build: [summercms_s3]\n", "build.tags"},
{"scalar build", "build: summercms_s3\n", "build.tags"},
} {
t.Run(tc.name, func(t *testing.T) {
_, err := LoadManifest(writeManifest(t, tagsManifestHead+tc.yaml))
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("err = %v, want one naming %s", err, tc.want)
}
})
}
for _, bad := range []string{"", "A", "a-b", "a b", "a.b", "-tags", "a,b", "ä"} {
if err := ValidateBuildTag(bad); err == nil {
t.Errorf("ValidateBuildTag(%q) accepted", bad)
}
}
}
func TestSaveManifestKeepsBuildTags(t *testing.T) {
path := writeManifest(t, tagsManifestHead+
" - id: acme.shop\n"+
" module: git.golem15.com/acme/acme/plugins/shop\n"+
"build:\n tags:\n - summercms_s3\n - netgo\n")
m, err := LoadManifest(path)
if err != nil {
t.Fatal(err)
}
// What plugin:add does: append a plugin and save.
m.Plugins = append(m.Plugins, Plugin{ID: "acme.faq", Module: "git.golem15.com/acme/acme/plugins/faq"})
if err := SaveManifest(path, m); err != nil {
t.Fatal(err)
}
again, err := LoadManifest(path)
if err != nil {
t.Fatal(err)
}
var ids []string
for _, p := range again.Plugins {
ids = append(ids, p.ID)
}
if !slices.Equal(ids, []string{"acme.blog", "acme.shop", "acme.faq"}) {
t.Fatalf("plugin order = %q", ids)
}
if !slices.Equal(again.Tags, []string{"summercms_s3", "netgo"}) {
t.Fatalf("Tags = %q after the round trip", again.Tags)
}
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(string(raw), "build:\n tags:\n - summercms_s3\n - netgo\n") {
t.Fatalf("build block is not written after the plugins:\n%s", raw)
}
// No tags: no build block at all.
m.Tags = nil
if err := SaveManifest(path, m); err != nil {
t.Fatal(err)
}
raw, _ = os.ReadFile(path)
if strings.Contains(string(raw), "build:") {
t.Fatalf("empty tags wrote a build block:\n%s", raw)
}
m.Tags = []string{}
if err := SaveManifest(path, m); err != nil {
t.Fatal(err)
}
raw, _ = os.ReadFile(path)
if strings.Contains(string(raw), "build:") {
t.Fatalf("an empty tag list wrote a build block:\n%s", raw)
}
// SaveManifest refuses an invalid tag and leaves the file alone.
before, _ := os.ReadFile(path)
m.Tags = []string{"ok", "not-ok"}
if err := SaveManifest(path, m); err == nil || !strings.Contains(err.Error(), "build.tags[1]") {
t.Fatalf("SaveManifest with an invalid tag: %v", err)
}
after, _ := os.ReadFile(path)
if string(after) != string(before) {
t.Fatal("SaveManifest wrote a file with an invalid tag")
}
}
func TestGoBuildArgs(t *testing.T) {
if got := goBuildArgs("bin/acme", nil); !slices.Equal(got, []string{"build", "-o", "bin/acme", "."}) {
t.Fatalf("no tags = %q", got)
}
if got := goBuildArgs("bin/acme", []string{}); slices.ContainsFunc(got, func(a string) bool { return strings.HasPrefix(a, "-tags") }) {
t.Fatalf("empty tags = %q", got)
}
got := goBuildArgs("bin/acme", []string{"a", "b"})
if !slices.Equal(got, []string{"build", "-tags=a,b", "-o", "bin/acme", "."}) {
t.Fatalf("tags = %q", got)
}
n := 0
for _, a := range got {
if strings.HasPrefix(a, "-tags") {
n++
}
}
if n != 1 {
t.Fatalf("%d -tags arguments", n)
}
// The tags of a loaded manifest reach the arguments unchanged.
m, err := LoadManifest(writeManifest(t, tagsManifestHead+"build:\n tags: [summercms_s3, netgo]\n"))
if err != nil {
t.Fatal(err)
}
if got := goBuildArgs("bin/"+m.Binary, m.Tags); !slices.Equal(got, []string{"build", "-tags=summercms_s3,netgo", "-o", "bin/acme", "."}) {
t.Fatalf("manifest tags = %q", got)
}
}

View File

@@ -31,6 +31,8 @@ Use cases:
- Context helpers: `bouncer.WithUser` and `bouncer.User` for the principal, `bouncer.WithCredential` and `bouncer.Credential` for the credential behind it (for example an API token record).
- Scope limits for personal API tokens: `bouncer.ScopeLimit` on `bouncer.Principal.Scope` (nil means unrestricted, as for a JWT session). A token guard sets it after filtering the token's scopes by the request method; `Full` means the token acts as the owner on every method (`*`) or on GET/HEAD (`admin:read`).
- Blacklist stores behind `bouncer.BlacklistStore`: `bouncer.MemoryBlacklist` for tests and `bouncer.PostgresBlacklist` for production, which works on a caller-supplied table with `jti`, `expires_at` and `valid_until` columns and rejects unsafe table names.
- Cookie transport: `bouncer.LoadCookieConfig` reads the `auth.cookie.*` block into a `bouncer.CookieConfig` and refuses an invalid domain, `same_site: none` without Secure and, in the production environment, a Secure that resolves to false (set to `false`, or inferred from an http or unset `app.url`); `bouncer.CookieConfig.TokenCookie` builds the httpOnly token cookie whose Max-Age is the refresh window, and `bouncer.CookieConfig.ForgetTokenCookie` the matching cookie that deletes it. A hand-built config with `SameSite` unset writes `SameSite=Lax`, and the zero `bouncer.CookieConfig` writes `token` on `/` with `HttpOnly`, `SameSite=Lax` and `Secure`. `bouncer.CookieRefreshHandler` and `bouncer.CookieLogoutHandler` are ready-made handlers for a cookie-based refresh and logout; the application supplies the refresh or revoke call and a writer for its own response envelope.
- Cloudflare Turnstile: `bouncer.Turnstile.Verify` checks a token against siteverify with a 5 second timeout, in PHP's open mode (an empty secret passes) or closed mode (an empty secret is `bouncer.TurnstileUnavailable` in production); `bouncer.Turnstile.Middleware` reads the token from the query, a JSON body or a form body, restores the body and writes refusals through a `bouncer.TurnstileWriter`. surf registers it as `turnstile:open` and `turnstile:closed`.
- Passwords: `bouncer.HashPassword`, `bouncer.CheckPassword` and `bouncer.NeedsRehash` (bcrypt, cost chosen by the caller).
## Usage
@@ -79,6 +81,47 @@ func Login(secret string) (string, error) {
}
```
An application that keeps the token in an httpOnly cookie, for a frontend on `www.example.com` calling the API on `api.example.com`, loads the cookie config at boot and mounts the refresh handler on a POST route:
```go
package acme
import (
"context"
"encoding/json"
"net/http"
"time"
"git.golem15.com/golem15/summercms/modules/bouncer"
)
// RefreshRoute rotates the cookie's token. cfg is the app config
// (app.Config); users, secret and bl are the guard's; refreshTTL is its
// refresh_ttl.
func RefreshRoute(cfg bouncer.CookieConfigSource, users bouncer.UserProvider, secret string, bl bouncer.BlacklistStore, refreshTTL time.Duration) (http.Handler, error) {
cookie, err := bouncer.LoadCookieConfig(cfg) // a bad auth.cookie block fails boot
if err != nil {
return nil, err
}
return bouncer.CookieRefreshHandler(bouncer.CookieRefreshOptions{
Cookie: cookie,
RefreshTTL: refreshTTL,
Refresh: func(ctx context.Context, token string) (string, error) {
return bouncer.RefreshAudienceFor(ctx, users, secret, token, bouncer.AudienceUser, refreshTTL, bl, 0, "https://api.example.com/api/auth/refresh")
},
Write: func(w http.ResponseWriter, r *http.Request, res bouncer.CookieResult) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(res.Status)
if res.Err != nil {
_ = json.NewEncoder(w).Encode(map[string]any{"success": false, "error": "UNAUTHENTICATED"})
return
}
_ = json.NewEncoder(w).Encode(map[string]any{"success": true})
},
})
}
```
## API reference
| Identifier | Description |
@@ -113,15 +156,51 @@ func Login(secret string) (string, error) {
| `bouncer.BlacklistStore` | Revoked-jti store with a grace window and sweeping. |
| `bouncer.NewMemoryBlacklist` | In-process blacklist for tests. |
| `bouncer.NewPostgresBlacklist` | Blacklist over a `*sql.DB` and a table name. |
| `bouncer.CookieConfig` | Token cookie attributes: `Name`, `Domain`, `Path`, `Secure`, `SameSite`. An unset `SameSite` is Lax; the zero config is also `Secure`, while a config with any field set keeps its `Secure` value. |
| `bouncer.CookieConfigSource` | The config methods `bouncer.LoadCookieConfig` reads (`Lookup`, `String`, `Environment`); `*compass.Config` satisfies it. |
| `bouncer.LoadCookieConfig` | Reads and validates `auth.cookie.*`; every error names the key. |
| `bouncer.DefaultCookieName` | `token`, the cookie name when `auth.cookie.name` is unset. |
| `bouncer.CookieConfig.TokenCookie` | The httpOnly cookie carrying a token, Max-Age the refresh window passed in. |
| `bouncer.CookieConfig.ForgetTokenCookie` | The same cookie with an empty value and a past expiry, which deletes it. |
| `bouncer.CookieRefreshHandler` | Reads the token cookie, calls the application's refresh func, sets the rotated cookie and hands a `bouncer.CookieResult` to the writer. |
| `bouncer.CookieRefreshOptions` | Cookie config, refresh window, refresh func and writer of `bouncer.CookieRefreshHandler`. |
| `bouncer.CookieLogoutHandler` | Revokes the cookie's token through the application's func, clears the cookie and writes 200. |
| `bouncer.CookieLogoutOptions` | Cookie config, revoke func and writer of `bouncer.CookieLogoutHandler`. |
| `bouncer.CookieResult` | What the cookie handlers pass to the writer: status, rotated token and error. |
| `bouncer.ErrNoTokenCookie` | The refresh request carried no token cookie (401). |
| `bouncer.Turnstile` | Cloudflare Turnstile verifier: `Secret`, `Production`, `Endpoint`, `Client`. |
| `bouncer.NewTurnstile` | A verifier with the default endpoint and a client with `bouncer.TurnstileTimeout`. |
| `bouncer.Turnstile.Verify` | Checks a token for a client IP in open or closed mode; returns a `bouncer.TurnstileVerdict` and an error for logging. |
| `bouncer.Turnstile.Middleware` | Middleware that reads the token field, verifies it and answers 503 or 422 through a `bouncer.TurnstileWriter`. |
| `bouncer.TurnstileVerdict` | `bouncer.TurnstilePassed`, `bouncer.TurnstileFailed` or `bouncer.TurnstileUnavailable`. |
| `bouncer.TurnstileWriter` | Writes a Turnstile refusal in the application's envelope; publish one on the app to replace the neutral default. |
| `bouncer.TurnstileSiteverifyURL` | Cloudflare's siteverify endpoint. |
| `bouncer.TurnstileTimeout` | 5 seconds, the bound on one siteverify call. |
| `bouncer.TurnstileMaxTokenLength` | 2048, Cloudflare's token limit; longer tokens fail without a call. |
| `bouncer.DefaultTurnstileField` | `turnstile_token`, the default request field. |
| `bouncer.HashPassword` | Returns a bcrypt hash at the given cost. |
| `bouncer.CheckPassword` | Reports whether a password matches a hash. |
| `bouncer.NeedsRehash` | Reports whether a hash was made below the configured cost. |
## Configuration
`bouncer.LoadCookieConfig` reads these keys; nothing else in bouncer reads config (surf builds the Turnstile verifier from `turnstile.secret` and `turnstile.field`, see the [surf README](../surf/README.md#configuration)). The per-guard token lifetimes (`ttl`, `refresh_ttl`) stay with the guard's own config, such as the user plugin's `jwt` block or `admin.jwt.*`.
| Key | Default | Controls |
|-----|---------|----------|
| `auth.cookie.name` | `token` | Cookie name; must be a valid cookie name. |
| `auth.cookie.domain` | empty (host-only) | Cookie domain. Set a parent domain such as `example.com` to share the cookie between `api.example.com` and `www.example.com`; every subdomain then receives it. An invalid domain fails boot. |
| `auth.cookie.path` | `/` | Cookie path; must start with `/`. |
| `auth.cookie.secure` | true when `app.url` is https | Sends the cookie over https only. In the production environment a value that resolves to false is refused, whether `false` is set or inferred from an http or unset `app.url`. |
| `auth.cookie.same_site` | `lax` | `lax`, `strict` or `none`; `none` requires Secure. |
Environment variables follow the compass mapping, for example `SUMMER_AUTH__COOKIE__DOMAIN`.
## Dependencies
- SummerCMS modules: none.
- Third-party: `github.com/golang-jwt/jwt/v5`, `golang.org/x/crypto/bcrypt`.
- Standard library: `context`, `crypto/rand`, `database/sql`, `encoding/hex`, `encoding/json`, `errors`, `fmt`, `math`, `net/http`, `reflect`, `regexp`, `strconv`, `strings`, `sync`, `time`.
- Standard library: `bytes`, `context`, `crypto/rand`, `database/sql`, `encoding/hex`, `encoding/json`, `errors`, `fmt`, `io`, `log/slog`, `math`, `mime`, `net`, `net/http`, `net/url`, `reflect`, `regexp`, `strconv`, `strings`, `sync`, `time`.
- Tests additionally use `github.com/testcontainers/testcontainers-go` with its `modules/postgres` package, and `github.com/jackc/pgx/v5/stdlib`.
## Testing
@@ -130,4 +209,4 @@ func Login(secret string) (string, error) {
go test ./modules/bouncer/...
```
The Postgres blacklist concurrency test starts a PostgreSQL container through testcontainers-go and needs Docker. Run `go test -short ./modules/bouncer/...` to skip it; the remaining tests need no external services.
The Postgres blacklist concurrency test starts a PostgreSQL container through testcontainers-go and needs Docker. Run `go test -short ./modules/bouncer/...` to skip it; the remaining tests need no external services (Turnstile tests use an `httptest` siteverify fake through `Endpoint`).

295
modules/bouncer/cookie.go Normal file
View File

@@ -0,0 +1,295 @@
package bouncer
import (
"context"
"errors"
"fmt"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
// DefaultCookieName is the token cookie name when auth.cookie.name is unset.
const DefaultCookieName = "token"
// ErrNoTokenCookie is reported by CookieRefreshHandler when the request has
// no token cookie or an empty one.
var ErrNoTokenCookie = errors.New(msgTokenNotProvided)
// CookieConfig holds the attributes of the httpOnly token cookie. Every guard
// shares one config; the lifetime is passed per call (the guard's
// refresh_ttl), so the cookie outlives the access token it carries.
type CookieConfig struct {
// Name is the cookie name (auth.cookie.name, default "token").
Name string
// Domain is empty for a host-only cookie (the default); a parent domain
// such as example.com shares the cookie with every subdomain.
Domain string
// Path is the cookie path (auth.cookie.path, default "/").
Path string
// Secure sends the cookie over https only. It has no default except for
// the zero CookieConfig, which writes a Secure cookie; LoadCookieConfig
// derives it from app.url unless auth.cookie.secure is set.
Secure bool
// SameSite is http.SameSiteLaxMode when unset (also on a hand-built
// config) and otherwise what auth.cookie.same_site says.
SameSite http.SameSite
}
// CookieConfigSource is the part of the application config LoadCookieConfig
// reads. *compass.Config satisfies it, so bouncer imports no framework module.
type CookieConfigSource interface {
Lookup(path string) (any, bool)
String(path string) string
Environment() string
}
// LoadCookieConfig reads the auth.cookie block:
//
// - auth.cookie.name: default DefaultCookieName; must be a valid cookie name.
// - auth.cookie.path: default "/"; must start with "/".
// - auth.cookie.domain: default empty (host-only); a non-empty value must
// be a valid cookie domain, so it is never dropped silently when the
// cookie is written.
// - auth.cookie.same_site: lax (default), strict or none, case-insensitive.
// - auth.cookie.secure: unset means true exactly when app.url is https. In
// the production environment a Secure that resolves to false is refused,
// whether set explicitly or inferred from an http or unset app.url.
//
// same_site none without Secure is refused, as browsers drop such cookies.
// Every error names the key.
func LoadCookieConfig(src CookieConfigSource) (CookieConfig, error) {
cfg := CookieConfig{Name: DefaultCookieName, Path: "/", SameSite: http.SameSiteLaxMode}
if src == nil {
return cfg, nil
}
if raw, ok := src.Lookup("auth.cookie.name"); ok && raw != nil {
name := strings.TrimSpace(fmt.Sprint(raw))
if name == "" || (&http.Cookie{Name: name, Value: "x"}).Valid() != nil {
return cfg, fmt.Errorf("bouncer: auth.cookie.name %q is not a valid cookie name", name)
}
cfg.Name = name
}
if raw, ok := src.Lookup("auth.cookie.path"); ok && raw != nil {
p := strings.TrimSpace(fmt.Sprint(raw))
if !strings.HasPrefix(p, "/") || (&http.Cookie{Name: "x", Value: "x", Path: p}).Valid() != nil {
return cfg, fmt.Errorf("bouncer: auth.cookie.path %q must be an absolute URL path starting with /", p)
}
cfg.Path = p
}
if raw, ok := src.Lookup("auth.cookie.domain"); ok && raw != nil {
d := strings.TrimSpace(fmt.Sprint(raw))
if d != "" {
if (&http.Cookie{Name: "x", Value: "x", Domain: d}).Valid() != nil {
return cfg, fmt.Errorf("bouncer: auth.cookie.domain %q is not a valid cookie domain", d)
}
cfg.Domain = d
}
}
if raw, ok := src.Lookup("auth.cookie.same_site"); ok && raw != nil {
switch v := strings.ToLower(strings.TrimSpace(fmt.Sprint(raw))); v {
case "", "lax":
cfg.SameSite = http.SameSiteLaxMode
case "strict":
cfg.SameSite = http.SameSiteStrictMode
case "none":
cfg.SameSite = http.SameSiteNoneMode
default:
return cfg, fmt.Errorf("bouncer: auth.cookie.same_site %q must be lax, strict or none", v)
}
}
secure, explicit, err := cookieSecure(src)
if err != nil {
return cfg, err
}
if !secure && strings.EqualFold(strings.TrimSpace(src.Environment()), "production") {
if explicit {
return cfg, errors.New("bouncer: auth.cookie.secure: false is not allowed in the production environment")
}
return cfg, errors.New("bouncer: auth.cookie.secure resolves to false in the production environment: it is unset and app.url is not an https URL; set app.url to an https URL or auth.cookie.secure: true")
}
cfg.Secure = secure
if cfg.SameSite == http.SameSiteNoneMode && !cfg.Secure {
return cfg, errors.New("bouncer: auth.cookie.same_site none requires auth.cookie.secure true (browsers drop SameSite=None cookies without Secure)")
}
return cfg, nil
}
// cookieSecure reads auth.cookie.secure; unset falls back to the app.url
// scheme. explicit reports whether the key was set.
func cookieSecure(src CookieConfigSource) (secure, explicit bool, err error) {
raw, ok := src.Lookup("auth.cookie.secure")
if !ok || raw == nil {
u, perr := url.Parse(strings.TrimSpace(src.String("app.url")))
return perr == nil && strings.EqualFold(u.Scheme, "https"), false, nil
}
switch v := raw.(type) {
case bool:
return v, true, nil
case string:
b, perr := strconv.ParseBool(strings.TrimSpace(v))
if perr != nil {
return false, true, fmt.Errorf("bouncer: auth.cookie.secure %q must be true or false", v)
}
return b, true, nil
default:
return false, true, fmt.Errorf("bouncer: auth.cookie.secure must be true or false, got %T", raw)
}
}
// TokenCookie returns the httpOnly cookie carrying token. Max-Age is
// refreshTTL in whole seconds: the refresh window, never the access ttl, so
// the refresh endpoint still has a cookie to read after the token expired.
func (c CookieConfig) TokenCookie(token string, refreshTTL time.Duration) *http.Cookie {
c = c.withDefaults()
return &http.Cookie{
Name: c.Name,
Value: token,
Path: c.Path,
Domain: c.Domain,
MaxAge: int(refreshTTL / time.Second),
Expires: time.Now().Add(refreshTTL),
HttpOnly: true,
Secure: c.Secure,
SameSite: c.SameSite,
}
}
// ForgetTokenCookie returns a cookie with the same name, path, domain and
// flags as TokenCookie, an empty value and an expiry in the past, so the
// browser deletes the token cookie.
func (c CookieConfig) ForgetTokenCookie() *http.Cookie {
c = c.withDefaults()
return &http.Cookie{
Name: c.Name,
Value: "",
Path: c.Path,
Domain: c.Domain,
MaxAge: -1,
Expires: time.Unix(0, 0).UTC(),
HttpOnly: true,
Secure: c.Secure,
SameSite: c.SameSite,
}
}
// CookieResult is what the cookie handlers pass to the application's writer.
// Status is the HTTP status to answer with; Token is the rotated token on a
// successful refresh; Err is the refusal (refresh) or the revoke failure
// (logout, which still answers 200 and clears the cookie).
type CookieResult struct {
Status int
Token string
Err error
}
// CookieRefreshOptions configures CookieRefreshHandler.
type CookieRefreshOptions struct {
// Cookie names the cookie to read and the attributes of the new one.
Cookie CookieConfig
// RefreshTTL is the guard's refresh window (refresh_ttl); it becomes the
// new cookie's Max-Age.
RefreshTTL time.Duration
// Refresh rotates the token, typically by calling Refresh or
// RefreshAudienceFor, which blacklist the old jti.
Refresh func(ctx context.Context, token string) (string, error)
// Write answers the request in the application's response envelope.
Write func(w http.ResponseWriter, r *http.Request, res CookieResult)
}
// CookieLogoutOptions configures CookieLogoutHandler.
type CookieLogoutOptions struct {
// Cookie names the cookie to read and clear.
Cookie CookieConfig
// Revoke blacklists the token, for example through
// VerifyRefreshableClaimsAudience and BlacklistStore.Add.
Revoke func(ctx context.Context, token string) error
// Write answers the request in the application's response envelope.
Write func(w http.ResponseWriter, r *http.Request, res CookieResult)
}
// CookieRefreshHandler returns a handler that reads the token cookie, calls
// opts.Refresh and, on success, sets the rotated cookie and writes 200 with
// the new token. A missing or empty cookie writes 401 with ErrNoTokenCookie,
// and a refused refresh writes 401 with the refusal; both leave the cookie
// alone. A nil Refresh or Write, or a non-positive RefreshTTL, is an error.
// Mount it on a POST route.
func CookieRefreshHandler(opts CookieRefreshOptions) (http.Handler, error) {
if opts.Refresh == nil {
return nil, errors.New("bouncer: CookieRefreshOptions.Refresh is nil")
}
if opts.Write == nil {
return nil, errors.New("bouncer: CookieRefreshOptions.Write is nil")
}
if opts.RefreshTTL <= 0 {
return nil, errors.New("bouncer: CookieRefreshOptions.RefreshTTL must be positive")
}
cookie := opts.Cookie.withDefaults()
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := readTokenCookie(r, cookie.Name)
if token == "" {
opts.Write(w, r, CookieResult{Status: http.StatusUnauthorized, Err: ErrNoTokenCookie})
return
}
next, err := opts.Refresh(r.Context(), token)
if err != nil {
opts.Write(w, r, CookieResult{Status: http.StatusUnauthorized, Err: err})
return
}
http.SetCookie(w, cookie.TokenCookie(next, opts.RefreshTTL))
opts.Write(w, r, CookieResult{Status: http.StatusOK, Token: next})
}), nil
}
// CookieLogoutHandler returns a handler that revokes the token in the cookie
// (when there is one) through opts.Revoke, always clears the cookie and
// writes 200; a revoke failure is passed to Write in CookieResult.Err. A nil
// Revoke or Write is an error. Mount it on a POST route.
func CookieLogoutHandler(opts CookieLogoutOptions) (http.Handler, error) {
if opts.Revoke == nil {
return nil, errors.New("bouncer: CookieLogoutOptions.Revoke is nil")
}
if opts.Write == nil {
return nil, errors.New("bouncer: CookieLogoutOptions.Write is nil")
}
cookie := opts.Cookie.withDefaults()
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var revokeErr error
if token := readTokenCookie(r, cookie.Name); token != "" {
revokeErr = opts.Revoke(r.Context(), token)
}
http.SetCookie(w, cookie.ForgetTokenCookie())
opts.Write(w, r, CookieResult{Status: http.StatusOK, Err: revokeErr})
}), nil
}
// withDefaults fills an empty Name and Path and an unset SameSite (Lax). The
// zero CookieConfig also gets Secure, so it writes the default token cookie
// on "/" with HttpOnly, SameSite=Lax and Secure; a config with any field set
// keeps its Secure value (LoadCookieConfig derives it from app.url).
// TokenCookie, ForgetTokenCookie and both handlers go through it.
func (c CookieConfig) withDefaults() CookieConfig {
if c == (CookieConfig{}) {
c.Secure = true
}
if c.SameSite == 0 {
c.SameSite = http.SameSiteLaxMode
}
if strings.TrimSpace(c.Name) == "" {
c.Name = DefaultCookieName
}
if c.Path == "" {
c.Path = "/"
}
return c
}
func readTokenCookie(r *http.Request, name string) string {
c, err := r.Cookie(name)
if err != nil {
return ""
}
return strings.TrimSpace(c.Value)
}

View File

@@ -0,0 +1,530 @@
package bouncer
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
// cookieSource is a CookieConfigSource over a flat map of dotted keys.
type cookieSource struct {
values map[string]any
env string
}
func (s cookieSource) Lookup(path string) (any, bool) {
v, ok := s.values[path]
return v, ok
}
func (s cookieSource) String(path string) string {
if v, ok := s.values[path].(string); ok {
return v
}
return ""
}
func (s cookieSource) Environment() string { return s.env }
func TestLoadCookieConfigDefaults(t *testing.T) {
cfg, err := LoadCookieConfig(nil)
if err != nil {
t.Fatal(err)
}
if cfg.Name != DefaultCookieName || cfg.Path != "/" || cfg.Domain != "" || cfg.SameSite != http.SameSiteLaxMode || cfg.Secure {
t.Fatalf("nil source = %+v", cfg)
}
// An http or unset app.url infers Secure false outside production only;
// production refuses it (TestLoadCookieConfigErrors).
for _, tc := range []struct {
url string
env string
secure bool
}{
{"https://app.example.com", "production", true},
{"HTTPS://app.example.com", "production", true},
{"http://localhost:8000", "development", false},
{"", "development", false},
} {
cfg, err := LoadCookieConfig(cookieSource{values: map[string]any{"app.url": tc.url}, env: tc.env})
if err != nil {
t.Fatalf("app.url %q: %v", tc.url, err)
}
if cfg.Name != "token" || cfg.Path != "/" || cfg.Domain != "" || cfg.SameSite != http.SameSiteLaxMode {
t.Fatalf("app.url %q: defaults = %+v", tc.url, cfg)
}
if cfg.Secure != tc.secure {
t.Fatalf("app.url %q: Secure = %t, want %t", tc.url, cfg.Secure, tc.secure)
}
}
cfg, err = LoadCookieConfig(cookieSource{values: map[string]any{
"auth.cookie.name": "sid",
"auth.cookie.path": "/api",
"auth.cookie.domain": "example.com",
"auth.cookie.same_site": "Strict",
"auth.cookie.secure": "true",
}, env: "production"})
if err != nil {
t.Fatal(err)
}
if cfg.Name != "sid" || cfg.Path != "/api" || cfg.Domain != "example.com" || cfg.SameSite != http.SameSiteStrictMode || !cfg.Secure {
t.Fatalf("explicit = %+v", cfg)
}
cfg, err = LoadCookieConfig(cookieSource{values: map[string]any{
"auth.cookie.same_site": "none",
"auth.cookie.secure": true,
}})
if err != nil || cfg.SameSite != http.SameSiteNoneMode || !cfg.Secure {
t.Fatalf("none with secure = %+v, %v", cfg, err)
}
// secure false outside production is allowed.
cfg, err = LoadCookieConfig(cookieSource{values: map[string]any{
"app.url": "https://app.example.com",
"auth.cookie.secure": false,
}, env: "development"})
if err != nil || cfg.Secure {
t.Fatalf("secure false in development = %+v, %v", cfg, err)
}
}
func TestLoadCookieConfigErrors(t *testing.T) {
for _, tc := range []struct {
name string
values map[string]any
env string
key string
}{
{"invalid domain", map[string]any{"auth.cookie.domain": "exa mple.com"}, "", "auth.cookie.domain"},
{"same_site none without secure", map[string]any{"auth.cookie.same_site": "none", "auth.cookie.secure": false}, "development", "auth.cookie.same_site"},
{"same_site none on http app", map[string]any{"auth.cookie.same_site": "none", "app.url": "http://localhost"}, "", "auth.cookie.same_site"},
{"secure false in production", map[string]any{"auth.cookie.secure": false}, "production", "auth.cookie.secure"},
{"secure string false in production", map[string]any{"auth.cookie.secure": "false"}, "Production", "auth.cookie.secure"},
{"secure inferred from http app.url in production", map[string]any{"app.url": "http://api.example.com"}, "production", "app.url"},
{"secure inferred from unset app.url in production", map[string]any{}, "production", "auth.cookie.secure resolves to false"},
{"secure not a bool", map[string]any{"auth.cookie.secure": "maybe"}, "", "auth.cookie.secure"},
{"secure wrong type", map[string]any{"auth.cookie.secure": 1}, "", "auth.cookie.secure"},
{"unknown same_site", map[string]any{"auth.cookie.same_site": "relaxed"}, "", "auth.cookie.same_site"},
{"path without leading slash", map[string]any{"auth.cookie.path": "api"}, "", "auth.cookie.path"},
{"invalid name", map[string]any{"auth.cookie.name": "to ken"}, "", "auth.cookie.name"},
{"empty name", map[string]any{"auth.cookie.name": " "}, "", "auth.cookie.name"},
} {
t.Run(tc.name, func(t *testing.T) {
_, err := LoadCookieConfig(cookieSource{values: tc.values, env: tc.env})
if err == nil {
t.Fatal("want an error")
}
if !strings.Contains(err.Error(), tc.key) {
t.Fatalf("error %q does not name %s", err, tc.key)
}
})
}
}
func TestTokenCookieAttributes(t *testing.T) {
cfg := CookieConfig{Name: "sid", Domain: "example.com", Path: "/api", Secure: true, SameSite: http.SameSiteLaxMode}
c := cfg.TokenCookie("tok", time.Hour)
if c.Name != "sid" || c.Value != "tok" || c.Domain != "example.com" || c.Path != "/api" {
t.Fatalf("cookie = %+v", c)
}
if !c.HttpOnly || !c.Secure || c.SameSite != http.SameSiteLaxMode {
t.Fatalf("flags = %+v", c)
}
if c.Valid() != nil {
t.Fatalf("cookie invalid: %v", c.Valid())
}
header := c.String()
for _, part := range []string{"sid=tok", "Path=/api", "Domain=example.com", "Max-Age=3600", "HttpOnly", "Secure", "SameSite=Lax"} {
if !strings.Contains(header, part) {
t.Fatalf("Set-Cookie %q lacks %q", header, part)
}
}
// A zero config writes the default cookie on "/", host-only, with
// SameSite=Lax and Secure.
z := CookieConfig{}.TokenCookie("v", time.Minute)
if z.Name != DefaultCookieName || z.Path != "/" || z.Domain != "" || !z.HttpOnly {
t.Fatalf("zero config cookie = %+v", z)
}
if z.SameSite != http.SameSiteLaxMode || !z.Secure {
t.Fatalf("zero config cookie SameSite %v Secure %t, want Lax and Secure", z.SameSite, z.Secure)
}
// A partial config gets SameSite=Lax and keeps its Secure value.
p := CookieConfig{Name: "sid"}.TokenCookie("v", time.Minute)
if p.SameSite != http.SameSiteLaxMode || p.Secure {
t.Fatalf("partial config cookie SameSite %v Secure %t, want Lax and not Secure", p.SameSite, p.Secure)
}
if strings.Contains(z.String(), "Domain=") {
t.Fatalf("host-only cookie carries a domain: %s", z.String())
}
}
func TestTokenCookieMaxAgeIsRefreshWindow(t *testing.T) {
const (
accessTTL = 60 * time.Minute
refreshTTL = 20160 * time.Minute
)
c := CookieConfig{}.TokenCookie("tok", refreshTTL)
if c.MaxAge != 1209600 {
t.Fatalf("MaxAge = %d, want 1209600 (refresh_ttl*60)", c.MaxAge)
}
if c.MaxAge == int(accessTTL/time.Second) {
t.Fatal("MaxAge equals the access ttl")
}
if d := time.Until(c.Expires); d < refreshTTL-time.Minute || d > refreshTTL+time.Minute {
t.Fatalf("Expires in %s, want about %s", d, refreshTTL)
}
}
func TestForgetTokenCookie(t *testing.T) {
cfg := CookieConfig{Name: "sid", Domain: "example.com", Path: "/api", Secure: true, SameSite: http.SameSiteNoneMode}
set := cfg.TokenCookie("tok", time.Hour)
f := cfg.ForgetTokenCookie()
if f.Name != set.Name || f.Domain != set.Domain || f.Path != set.Path || f.Secure != set.Secure || f.SameSite != set.SameSite || f.HttpOnly != set.HttpOnly {
t.Fatalf("forget %+v does not match set %+v", f, set)
}
if f.Value != "" || f.MaxAge != -1 || !f.Expires.Before(time.Now()) {
t.Fatalf("forget cookie = %+v", f)
}
if !strings.Contains(f.String(), "Max-Age=0") {
t.Fatalf("forget header %q does not expire the cookie", f.String())
}
}
// recordWriter captures what a cookie handler passes to the app's writer.
type recordWriter struct {
calls int
res CookieResult
}
func (rw *recordWriter) write(w http.ResponseWriter, _ *http.Request, res CookieResult) {
rw.calls++
rw.res = res
w.WriteHeader(res.Status)
_, _ = w.Write([]byte(res.Token))
}
func refreshRequest(name, value string) *http.Request {
r := httptest.NewRequest(http.MethodPost, "https://api.example.com/auth/refresh", nil)
if name != "" {
r.AddCookie(&http.Cookie{Name: name, Value: value})
}
return r
}
func TestCookieRefreshHandlerRotatesAndBlacklists(t *testing.T) {
const (
issuer = "https://api.example.com/auth/login"
refreshTTL = 20160 * time.Minute
)
old, oldJTI, err := Mint(secret, "42", issuer, time.Hour)
if err != nil {
t.Fatal(err)
}
bl := NewMemoryBlacklist()
cfg := CookieConfig{Name: "token", Domain: "example.com", Path: "/", Secure: true, SameSite: http.SameSiteLaxMode}
rw := &recordWriter{}
h, err := CookieRefreshHandler(CookieRefreshOptions{
Cookie: cfg,
RefreshTTL: refreshTTL,
Refresh: func(_ context.Context, token string) (string, error) {
return Refresh(secret, token, refreshTTL, bl, 0, "https://api.example.com/auth/refresh")
},
Write: rw.write,
})
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, refreshRequest("token", old))
if rec.Code != http.StatusOK || rw.calls != 1 || rw.res.Status != http.StatusOK || rw.res.Err != nil {
t.Fatalf("status %d, writer %d calls, res %+v", rec.Code, rw.calls, rw.res)
}
if rw.res.Token == "" || rw.res.Token == old || rec.Body.String() != rw.res.Token {
t.Fatalf("writer token %q", rw.res.Token)
}
cookies := rec.Result().Cookies()
if len(cookies) != 1 {
t.Fatalf("Set-Cookie count = %d", len(cookies))
}
c := cookies[0]
if c.Name != "token" || c.Value != rw.res.Token || c.Domain != "example.com" || !c.HttpOnly || !c.Secure || c.MaxAge != 1209600 {
t.Fatalf("rotated cookie = %+v", c)
}
claims := decodeClaims(t, c.Value, secret)
if claims["jti"] == oldJTI || claims["sub"] != "42" {
t.Fatalf("new claims = %#v", claims)
}
blocked, err := bl.IsBlacklisted(t.Context(), oldJTI)
if err != nil || !blocked {
t.Fatalf("old jti blacklisted=%t err=%v", blocked, err)
}
// The old token no longer refreshes.
rw2 := &recordWriter{}
h2, _ := CookieRefreshHandler(CookieRefreshOptions{
Cookie: cfg, RefreshTTL: refreshTTL,
Refresh: func(_ context.Context, token string) (string, error) {
return Refresh(secret, token, refreshTTL, bl, 0, "https://api.example.com/auth/refresh")
},
Write: rw2.write,
})
rec2 := httptest.NewRecorder()
h2.ServeHTTP(rec2, refreshRequest("token", old))
if rec2.Code != http.StatusUnauthorized || len(rec2.Result().Cookies()) != 0 {
t.Fatalf("replayed refresh: %d, cookies %v", rec2.Code, rec2.Result().Cookies())
}
}
func TestCookieRefreshHandlerMissingCookie(t *testing.T) {
for _, tc := range []struct{ name, cookie, value string }{
{"no cookie", "", ""},
{"other cookie", "session", "abc"},
{"empty cookie", "token", ""},
{"blank cookie", "token", " "},
} {
t.Run(tc.name, func(t *testing.T) {
called := false
rw := &recordWriter{}
h, err := CookieRefreshHandler(CookieRefreshOptions{
RefreshTTL: time.Hour,
Refresh: func(context.Context, string) (string, error) {
called = true
return "x", nil
},
Write: rw.write,
})
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, refreshRequest(tc.cookie, tc.value))
if called {
t.Fatal("Refresh called without a token cookie")
}
if rec.Code != http.StatusUnauthorized || !errors.Is(rw.res.Err, ErrNoTokenCookie) {
t.Fatalf("status %d err %v", rec.Code, rw.res.Err)
}
if rec.Header().Get("Set-Cookie") != "" {
t.Fatalf("Set-Cookie sent: %q", rec.Header().Get("Set-Cookie"))
}
})
}
}
func TestCookieRefreshHandlerRefreshError(t *testing.T) {
refusal := errors.New("token expired past refresh window")
rw := &recordWriter{}
h, err := CookieRefreshHandler(CookieRefreshOptions{
Cookie: CookieConfig{Name: "sid"},
RefreshTTL: time.Hour,
Refresh: func(_ context.Context, token string) (string, error) {
if token != "stale" {
t.Fatalf("Refresh got %q", token)
}
return "", refusal
},
Write: rw.write,
})
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, refreshRequest("sid", "stale"))
if rec.Code != http.StatusUnauthorized || !errors.Is(rw.res.Err, refusal) || rw.res.Token != "" {
t.Fatalf("status %d res %+v", rec.Code, rw.res)
}
if rec.Header().Get("Set-Cookie") != "" {
t.Fatalf("cookie touched on a refused refresh: %q", rec.Header().Get("Set-Cookie"))
}
}
func TestCookieLogoutHandler(t *testing.T) {
cfg := CookieConfig{Name: "token", Domain: "example.com", Path: "/", Secure: true, SameSite: http.SameSiteLaxMode}
revokeErr := errors.New("blacklist down")
for _, tc := range []struct {
name string
cookie string
revokeErr error
wantCall bool
}{
{"revokes the cookie token", "tok-1", nil, true},
{"revoke failure still forgets", "tok-2", revokeErr, true},
{"no cookie still forgets", "", nil, false},
} {
t.Run(tc.name, func(t *testing.T) {
var got []string
rw := &recordWriter{}
h, err := CookieLogoutHandler(CookieLogoutOptions{
Cookie: cfg,
Revoke: func(_ context.Context, token string) error {
got = append(got, token)
return tc.revokeErr
},
Write: rw.write,
})
if err != nil {
t.Fatal(err)
}
name := ""
if tc.cookie != "" {
name = "token"
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, refreshRequest(name, tc.cookie))
if tc.wantCall && (len(got) != 1 || got[0] != tc.cookie) {
t.Fatalf("Revoke calls = %v", got)
}
if !tc.wantCall && len(got) != 0 {
t.Fatalf("Revoke called without a cookie: %v", got)
}
if rec.Code != http.StatusOK || rw.res.Status != http.StatusOK || !errors.Is(rw.res.Err, tc.revokeErr) {
t.Fatalf("status %d res %+v", rec.Code, rw.res)
}
cookies := rec.Result().Cookies()
if len(cookies) != 1 {
t.Fatalf("Set-Cookie count = %d", len(cookies))
}
f := cookies[0]
if f.Name != "token" || f.Value != "" || f.MaxAge != -1 || f.Domain != "example.com" || f.Path != "/" || !f.Secure || !f.HttpOnly {
t.Fatalf("forget cookie = %+v", f)
}
})
}
}
func TestCookieHandlerOptionsValidated(t *testing.T) {
refresh := func(context.Context, string) (string, error) { return "", nil }
revoke := func(context.Context, string) error { return nil }
write := func(http.ResponseWriter, *http.Request, CookieResult) {}
for _, tc := range []struct {
name string
opts CookieRefreshOptions
want string
}{
{"nil refresh", CookieRefreshOptions{RefreshTTL: time.Hour, Write: write}, "Refresh"},
{"nil write", CookieRefreshOptions{RefreshTTL: time.Hour, Refresh: refresh}, "Write"},
{"zero ttl", CookieRefreshOptions{Refresh: refresh, Write: write}, "RefreshTTL"},
{"negative ttl", CookieRefreshOptions{RefreshTTL: -time.Second, Refresh: refresh, Write: write}, "RefreshTTL"},
} {
h, err := CookieRefreshHandler(tc.opts)
if err == nil || h != nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("refresh %s: handler %v err %v", tc.name, h, err)
}
}
if h, err := CookieRefreshHandler(CookieRefreshOptions{RefreshTTL: time.Hour, Refresh: refresh, Write: write}); err != nil || h == nil {
t.Fatalf("valid refresh options: %v", err)
}
for _, tc := range []struct {
name string
opts CookieLogoutOptions
want string
}{
{"nil revoke", CookieLogoutOptions{Write: write}, "Revoke"},
{"nil write", CookieLogoutOptions{Revoke: revoke}, "Write"},
} {
h, err := CookieLogoutHandler(tc.opts)
if err == nil || h != nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("logout %s: handler %v err %v", tc.name, h, err)
}
}
if h, err := CookieLogoutHandler(CookieLogoutOptions{Revoke: revoke, Write: write}); err != nil || h == nil {
t.Fatalf("valid logout options: %v", err)
}
}
func TestLoadCookieConfigProductionInferredSecure(t *testing.T) {
for _, tc := range []struct {
name string
env string
values map[string]any
wantErr bool
secure bool
}{
{"production, http app.url", "production", map[string]any{"app.url": "http://api.example.com"}, true, false},
{"production, unset app.url", "production", map[string]any{}, true, false},
{"production, malformed app.url", "production", map[string]any{"app.url": "://not a url"}, true, false},
{"production, https app.url", "production", map[string]any{"app.url": "https://api.example.com"}, false, true},
{"production, http app.url and secure true", "production", map[string]any{"app.url": "http://api.example.com", "auth.cookie.secure": true}, false, true},
{"development, http app.url", "development", map[string]any{"app.url": "http://localhost:8000"}, false, false},
} {
t.Run(tc.name, func(t *testing.T) {
cfg, err := LoadCookieConfig(cookieSource{values: tc.values, env: tc.env})
if tc.wantErr {
if err == nil {
t.Fatalf("config = %+v, want an error", cfg)
}
for _, key := range []string{"auth.cookie.secure", "app.url"} {
if !strings.Contains(err.Error(), key) {
t.Errorf("error %q does not name %s", err, key)
}
}
return
}
if err != nil {
t.Fatal(err)
}
if cfg.Secure != tc.secure {
t.Fatalf("Secure = %t, want %t", cfg.Secure, tc.secure)
}
})
}
}
func TestZeroCookieConfigDefaults(t *testing.T) {
var zero CookieConfig
for name, c := range map[string]*http.Cookie{
"TokenCookie": zero.TokenCookie("tok", time.Hour),
"ForgetTokenCookie": zero.ForgetTokenCookie(),
} {
if c.Name != DefaultCookieName || c.Path != "/" || c.Domain != "" || !c.HttpOnly || c.SameSite != http.SameSiteLaxMode || !c.Secure {
t.Errorf("%s of the zero config = %+v, want token on / with HttpOnly, SameSite=Lax and Secure", name, c)
}
}
partial := CookieConfig{Name: "sid"}.TokenCookie("tok", time.Hour)
if partial.SameSite != http.SameSiteLaxMode || partial.Secure {
t.Errorf("{Name: sid} cookie SameSite %v Secure %t, want Lax and not Secure", partial.SameSite, partial.Secure)
}
write := func(w http.ResponseWriter, _ *http.Request, res CookieResult) { w.WriteHeader(res.Status) }
refresh, err := CookieRefreshHandler(CookieRefreshOptions{
RefreshTTL: time.Hour,
Refresh: func(context.Context, string) (string, error) { return "rotated", nil },
Write: write,
})
if err != nil {
t.Fatal(err)
}
logout, err := CookieLogoutHandler(CookieLogoutOptions{
Revoke: func(context.Context, string) error { return nil },
Write: write,
})
if err != nil {
t.Fatal(err)
}
for name, h := range map[string]http.Handler{"refresh": refresh, "logout": logout} {
req := httptest.NewRequest(http.MethodPost, "/auth/"+name, nil)
req.AddCookie(&http.Cookie{Name: DefaultCookieName, Value: "old"})
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
set := rec.Header().Get("Set-Cookie")
if set == "" {
t.Fatalf("%s: no Set-Cookie (status %d)", name, rec.Code)
}
for _, want := range []string{DefaultCookieName + "=", "Path=/", "HttpOnly", "SameSite=Lax", "Secure"} {
if !strings.Contains(set, want) {
t.Errorf("%s: Set-Cookie %q lacks %s", name, set, want)
}
}
}
}

View File

@@ -0,0 +1,304 @@
package bouncer
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"mime"
"net"
"net/http"
"net/url"
"strings"
"time"
)
const (
// TurnstileSiteverifyURL is Cloudflare's server-side validation endpoint.
TurnstileSiteverifyURL = "https://challenges.cloudflare.com/turnstile/v0/siteverify"
// TurnstileTimeout bounds one siteverify call.
TurnstileTimeout = 5 * time.Second
// TurnstileMaxTokenLength is Cloudflare's documented token limit; a
// longer token fails without a call.
TurnstileMaxTokenLength = 2048
// DefaultTurnstileField is the request field carrying the token.
DefaultTurnstileField = "turnstile_token"
)
// turnstileAnswerLimit caps how much of the siteverify answer is read.
const turnstileAnswerLimit = 4 << 10
// errTurnstileRefused marks a siteverify answer with success false: a normal
// rejection, logged below warn level.
var errTurnstileRefused = errors.New("bouncer: turnstile siteverify refused the token")
// TurnstileVerdict is the outcome of a Turnstile check.
type TurnstileVerdict int
const (
// TurnstilePassed lets the request through.
TurnstilePassed TurnstileVerdict = iota
// TurnstileFailed is a missing or rejected token, or a siteverify call
// that failed or timed out (422).
TurnstileFailed
// TurnstileUnavailable is fail-closed mode without a secret in the
// production environment (503).
TurnstileUnavailable
)
// String names the verdict for logs.
func (v TurnstileVerdict) String() string {
switch v {
case TurnstilePassed:
return "passed"
case TurnstileFailed:
return "failed"
case TurnstileUnavailable:
return "unavailable"
default:
return fmt.Sprintf("TurnstileVerdict(%d)", int(v))
}
}
// TurnstileWriter writes a Turnstile refusal in the application's response
// envelope: status is 503 for TurnstileUnavailable and 422 for
// TurnstileFailed. An application publishes one on the backpack app to keep
// its own error shape.
type TurnstileWriter func(w http.ResponseWriter, r *http.Request, status int, verdict TurnstileVerdict)
// Turnstile verifies Cloudflare Turnstile tokens against siteverify.
type Turnstile struct {
// Secret is the site's secret key; empty means unconfigured.
Secret string
// Production makes an unconfigured fail-closed check unavailable
// instead of passing.
Production bool
// Endpoint defaults to TurnstileSiteverifyURL; tests point it at a fake.
Endpoint string
// Client defaults to an *http.Client with Timeout TurnstileTimeout.
Client *http.Client
}
// NewTurnstile returns a verifier for secret with the default endpoint and
// client.
func NewTurnstile(secret string, production bool) *Turnstile {
return &Turnstile{
Secret: secret,
Production: production,
Endpoint: TurnstileSiteverifyURL,
Client: &http.Client{Timeout: TurnstileTimeout},
}
}
// Verify checks token for the client at ip. With failClosed false (open
// mode, for registration) an empty secret passes; with failClosed true
// (closed mode, for guest entry) an empty secret is TurnstileUnavailable in
// production and passes elsewhere. With a secret set both modes require a
// siteverify answer with success true: an empty token, a token longer than
// TurnstileMaxTokenLength, success false, a transport error, a timeout, a
// non-2xx status or an undecodable answer is TurnstileFailed. The returned
// error explains a failure for logging; it never contains the secret or the
// token.
func (t *Turnstile) Verify(ctx context.Context, token, ip string, failClosed bool) (TurnstileVerdict, error) {
if t == nil || strings.TrimSpace(t.Secret) == "" {
if failClosed && (t == nil || t.Production) {
return TurnstileUnavailable, nil
}
return TurnstilePassed, nil
}
if token == "" || len(token) > TurnstileMaxTokenLength {
return TurnstileFailed, nil
}
form := url.Values{"secret": {t.Secret}, "response": {token}}
if ip != "" {
form.Set("remoteip", ip)
}
if ctx == nil {
ctx = context.Background()
}
ctx, cancel := context.WithTimeout(ctx, TurnstileTimeout)
defer cancel()
endpoint := t.Endpoint
if endpoint == "" {
endpoint = TurnstileSiteverifyURL
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, strings.NewReader(form.Encode()))
if err != nil {
return TurnstileFailed, errors.New("bouncer: turnstile siteverify request could not be built")
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
client := t.Client
if client == nil {
client = &http.Client{Timeout: TurnstileTimeout}
}
res, err := client.Do(req)
if err != nil {
return TurnstileFailed, fmt.Errorf("bouncer: turnstile siteverify failed: %s", turnstileTransportError(err))
}
defer res.Body.Close()
body, err := io.ReadAll(io.LimitReader(res.Body, turnstileAnswerLimit))
if err != nil {
return TurnstileFailed, fmt.Errorf("bouncer: turnstile siteverify answer could not be read: %s", turnstileTransportError(err))
}
if res.StatusCode < 200 || res.StatusCode > 299 {
return TurnstileFailed, fmt.Errorf("bouncer: turnstile siteverify answered %d", res.StatusCode)
}
var answer struct {
Success *bool `json:"success"`
ErrorCodes []string `json:"error-codes"`
}
if err := json.Unmarshal(body, &answer); err != nil || answer.Success == nil {
return TurnstileFailed, errors.New("bouncer: turnstile siteverify answer is not a siteverify JSON object")
}
if !*answer.Success {
return TurnstileFailed, fmt.Errorf("%w (error-codes: %s)", errTurnstileRefused, turnstileCodes(answer.ErrorCodes))
}
return TurnstilePassed, nil
}
// turnstileTransportError describes a failed call without the URL.
func turnstileTransportError(err error) string {
var uerr *url.Error
if errors.As(err, &uerr) {
err = uerr.Err
}
if errors.Is(err, context.DeadlineExceeded) {
return "timeout"
}
return err.Error()
}
// turnstileCodes joins Cloudflare's error codes for a log line, keeping only
// the characters the documented codes use.
func turnstileCodes(codes []string) string {
out := make([]string, 0, len(codes))
for _, c := range codes {
c = strings.Map(func(r rune) rune {
if (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') || r == '-' || r == '_' {
return r
}
return -1
}, c)
if c != "" {
out = append(out, c)
}
if len(out) == 8 {
break
}
}
if len(out) == 0 {
return "none"
}
return strings.Join(out, ",")
}
// Middleware returns middleware that verifies the token in the request field
// named field (DefaultTurnstileField when empty): the query parameter first,
// else the body, a JSON object's top-level string field or an
// application/x-www-form-urlencoded field. The body is read fully (bound it
// with the router's body limit) and restored with the same bytes before the
// handler runs. TurnstileUnavailable writes 503 and TurnstileFailed 422
// through write, or through neutral JSON bodies ({"error":"turnstile_unavailable"},
// {"error":"turnstile_failed"}) when write is nil; TurnstilePassed calls the
// handler. clientIP supplies the remoteip (nil uses the connection address).
// A failed siteverify call is logged once at warn with the error text only.
func (t *Turnstile) Middleware(failClosed bool, field string, clientIP func(*http.Request) string, write TurnstileWriter) func(http.Handler) http.Handler {
if field == "" {
field = DefaultTurnstileField
}
if write == nil {
write = defaultTurnstileWriter
}
if clientIP == nil {
clientIP = remoteAddrIP
}
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token, err := turnstileToken(r, field)
if err != nil {
status := http.StatusBadRequest
var tooLarge *http.MaxBytesError
if errors.As(err, &tooLarge) {
status = http.StatusRequestEntityTooLarge
}
write(w, r, status, TurnstileFailed)
return
}
verdict, verr := t.Verify(r.Context(), token, clientIP(r), failClosed)
if verr != nil {
if errors.Is(verr, errTurnstileRefused) {
slog.DebugContext(r.Context(), "bouncer: turnstile token refused", "error", verr.Error())
} else {
slog.WarnContext(r.Context(), "bouncer: turnstile verification failed", "error", verr.Error())
}
}
switch verdict {
case TurnstilePassed:
next.ServeHTTP(w, r)
case TurnstileUnavailable:
write(w, r, http.StatusServiceUnavailable, verdict)
default:
write(w, r, http.StatusUnprocessableEntity, TurnstileFailed)
}
})
}
}
// turnstileToken reads field from the query, else from the body, and
// restores r.Body.
func turnstileToken(r *http.Request, field string) (string, error) {
if v := r.URL.Query().Get(field); v != "" {
return v, nil
}
if r.Body == nil || r.Body == http.NoBody {
return "", nil
}
body, err := io.ReadAll(r.Body)
_ = r.Body.Close()
r.Body = io.NopCloser(bytes.NewReader(body))
if err != nil {
return "", err
}
mediaType, _, _ := mime.ParseMediaType(r.Header.Get("Content-Type"))
switch {
case mediaType == "application/x-www-form-urlencoded":
values, err := url.ParseQuery(string(body))
if err != nil {
return "", nil
}
return values.Get(field), nil
case mediaType == "application/json" || strings.HasSuffix(mediaType, "+json") ||
(mediaType == "" && bytes.HasPrefix(bytes.TrimSpace(body), []byte("{"))):
var obj map[string]json.RawMessage
if json.Unmarshal(body, &obj) != nil {
return "", nil
}
var v string
if raw, ok := obj[field]; !ok || json.Unmarshal(raw, &v) != nil {
return "", nil
}
return v, nil
}
return "", nil
}
func defaultTurnstileWriter(w http.ResponseWriter, _ *http.Request, status int, verdict TurnstileVerdict) {
code := "turnstile_failed"
if verdict == TurnstileUnavailable {
code = "turnstile_unavailable"
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
_, _ = io.WriteString(w, `{"error":"`+code+`"}`)
}
func remoteAddrIP(r *http.Request) string {
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return r.RemoteAddr
}
return host
}

View File

@@ -0,0 +1,391 @@
package bouncer
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
)
const turnstileTestSecret = "0x4AAAAAAA-test-secret-value"
// siteverify is an httptest fake of Cloudflare's siteverify endpoint. It
// records every form it receives and answers with the configured status and
// body.
type siteverify struct {
srv *httptest.Server
calls atomic.Int32
mu sync.Mutex
forms []url.Values
status int
body string
delay time.Duration
}
func newSiteverify(t *testing.T, status int, body string) *siteverify {
t.Helper()
f := &siteverify{status: status, body: body}
f.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
f.calls.Add(1)
_ = r.ParseForm()
f.mu.Lock()
f.forms = append(f.forms, r.PostForm)
f.mu.Unlock()
if f.delay > 0 {
select {
case <-time.After(f.delay):
case <-r.Context().Done():
return
}
}
w.WriteHeader(f.status)
_, _ = io.WriteString(w, f.body)
}))
t.Cleanup(f.srv.Close)
return f
}
func (f *siteverify) verifier(secret string, production bool) *Turnstile {
return &Turnstile{Secret: secret, Production: production, Endpoint: f.srv.URL, Client: f.srv.Client()}
}
func (f *siteverify) lastForm(t *testing.T) url.Values {
t.Helper()
f.mu.Lock()
defer f.mu.Unlock()
if len(f.forms) == 0 {
t.Fatal("siteverify was not called")
}
return f.forms[len(f.forms)-1]
}
func TestTurnstileVerifyMatrix(t *testing.T) {
const ok = `{"success":true}`
for _, tc := range []struct {
name string
secret string
production bool
failClosed bool
token string
status int
body string
want TurnstileVerdict
wantCall bool
wantErr bool
}{
{"open, no secret, development", "", false, false, "tok", 200, ok, TurnstilePassed, false, false},
{"open, no secret, production", "", true, false, "tok", 200, ok, TurnstilePassed, false, false},
{"closed, no secret, development", "", false, true, "tok", 200, ok, TurnstilePassed, false, false},
{"closed, no secret, production", "", true, true, "tok", 200, ok, TurnstileUnavailable, false, false},
{"closed, blank secret, production", " ", true, true, "tok", 200, ok, TurnstileUnavailable, false, false},
{"open, empty token", turnstileTestSecret, true, false, "", 200, ok, TurnstileFailed, false, false},
{"closed, empty token", turnstileTestSecret, true, true, "", 200, ok, TurnstileFailed, false, false},
{"open, token over the limit", turnstileTestSecret, false, false, strings.Repeat("a", TurnstileMaxTokenLength+1), 200, ok, TurnstileFailed, false, false},
{"closed, token at the limit", turnstileTestSecret, false, true, strings.Repeat("a", TurnstileMaxTokenLength), 200, ok, TurnstilePassed, true, false},
{"open, success true", turnstileTestSecret, false, false, "tok", 200, ok, TurnstilePassed, true, false},
{"closed, success true", turnstileTestSecret, true, true, "tok", 200, ok, TurnstilePassed, true, false},
{"open, success false", turnstileTestSecret, false, false, "tok", 200, `{"success":false,"error-codes":["invalid-input-response"]}`, TurnstileFailed, true, true},
{"closed, success false", turnstileTestSecret, true, true, "tok", 200, `{"success":false}`, TurnstileFailed, true, true},
{"open, non-2xx", turnstileTestSecret, false, false, "tok", 500, ok, TurnstileFailed, true, true},
{"closed, non-2xx", turnstileTestSecret, true, true, "tok", 403, ok, TurnstileFailed, true, true},
{"open, invalid JSON", turnstileTestSecret, false, false, "tok", 200, `<html>`, TurnstileFailed, true, true},
{"closed, JSON without success", turnstileTestSecret, true, true, "tok", 200, `{"hostname":"x"}`, TurnstileFailed, true, true},
} {
t.Run(tc.name, func(t *testing.T) {
f := newSiteverify(t, tc.status, tc.body)
got, err := f.verifier(tc.secret, tc.production).Verify(t.Context(), tc.token, "203.0.113.7", tc.failClosed)
if got != tc.want {
t.Fatalf("verdict = %s, want %s (err %v)", got, tc.want, err)
}
if called := f.calls.Load() > 0; called != tc.wantCall {
t.Fatalf("siteverify called = %t, want %t", called, tc.wantCall)
}
if (err != nil) != tc.wantErr {
t.Fatalf("err = %v, want error %t", err, tc.wantErr)
}
})
}
// A nil verifier behaves like an unconfigured one in production.
var nilT *Turnstile
if v, _ := nilT.Verify(t.Context(), "tok", "", true); v != TurnstileUnavailable {
t.Fatalf("nil closed = %s", v)
}
if v, _ := nilT.Verify(t.Context(), "tok", "", false); v != TurnstilePassed {
t.Fatalf("nil open = %s", v)
}
if TurnstilePassed.String() != "passed" || TurnstileFailed.String() != "failed" || TurnstileUnavailable.String() != "unavailable" {
t.Fatal("verdict names changed")
}
}
func TestTurnstileVerifyTimeout(t *testing.T) {
f := newSiteverify(t, 200, `{"success":true}`)
f.delay = 2 * time.Second
ctx, cancel := context.WithTimeout(t.Context(), 50*time.Millisecond)
defer cancel()
start := time.Now()
got, err := f.verifier(turnstileTestSecret, true).Verify(ctx, "tok", "", false)
if got != TurnstileFailed {
t.Fatalf("verdict = %s", got)
}
if err == nil || !strings.Contains(err.Error(), "timeout") {
t.Fatalf("err = %v, want a timeout", err)
}
if elapsed := time.Since(start); elapsed > time.Second {
t.Fatalf("Verify took %s past a 50 ms deadline", elapsed)
}
}
func TestTurnstileFormFields(t *testing.T) {
f := newSiteverify(t, 200, `{"success":true}`)
v := f.verifier(turnstileTestSecret, false)
if got, err := v.Verify(t.Context(), "the-token", "198.51.100.4", false); got != TurnstilePassed || err != nil {
t.Fatalf("verdict %s err %v", got, err)
}
form := f.lastForm(t)
if form.Get("secret") != turnstileTestSecret || form.Get("response") != "the-token" || form.Get("remoteip") != "198.51.100.4" {
t.Fatalf("form = %v", form)
}
if _, err := v.Verify(t.Context(), "the-token", "", false); err != nil {
t.Fatal(err)
}
form = f.lastForm(t)
if _, ok := form["remoteip"]; ok {
t.Fatalf("remoteip sent for an empty ip: %v", form)
}
if form.Get("response") != "the-token" {
t.Fatalf("form = %v", form)
}
}
func passHandler(called *bool) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
*called = true
w.WriteHeader(http.StatusNoContent)
})
}
func TestTurnstileMiddlewareTokenSources(t *testing.T) {
for _, tc := range []struct {
name string
build func() *http.Request
}{
{"query", func() *http.Request {
return httptest.NewRequest(http.MethodPost, "/register?turnstile_token=src-token", strings.NewReader(`{"turnstile_token":"other"}`))
}},
{"json body", func() *http.Request {
r := httptest.NewRequest(http.MethodPost, "/register", strings.NewReader(`{"email":"a@b.c","turnstile_token":"src-token"}`))
r.Header.Set("Content-Type", "application/json; charset=utf-8")
return r
}},
{"json body without content type", func() *http.Request {
return httptest.NewRequest(http.MethodPost, "/register", strings.NewReader(` {"turnstile_token":"src-token"}`))
}},
{"vendor json body", func() *http.Request {
r := httptest.NewRequest(http.MethodPost, "/register", strings.NewReader(`{"turnstile_token":"src-token"}`))
r.Header.Set("Content-Type", "application/vnd.api+json")
return r
}},
{"urlencoded form", func() *http.Request {
r := httptest.NewRequest(http.MethodPost, "/register", strings.NewReader(`email=a%40b.c&turnstile_token=src-token`))
r.Header.Set("Content-Type", "application/x-www-form-urlencoded")
return r
}},
} {
t.Run(tc.name, func(t *testing.T) {
f := newSiteverify(t, 200, `{"success":true}`)
called := false
h := f.verifier(turnstileTestSecret, true).Middleware(true, "", nil, nil)(passHandler(&called))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, tc.build())
if rec.Code != http.StatusNoContent || !called {
t.Fatalf("status %d called %t body %s", rec.Code, called, rec.Body)
}
if got := f.lastForm(t).Get("response"); got != "src-token" {
t.Fatalf("token sent = %q", got)
}
})
}
// A custom field name is honoured; the default field is then ignored.
f := newSiteverify(t, 200, `{"success":true}`)
called := false
h := f.verifier(turnstileTestSecret, true).Middleware(true, "cf_token", nil, nil)(passHandler(&called))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest(http.MethodPost, "/x?turnstile_token=wrong&cf_token=right", nil))
if !called || f.lastForm(t).Get("response") != "right" {
t.Fatalf("custom field: called %t form %v", called, f.lastForm(t))
}
}
func TestTurnstileMiddlewareRestoresBody(t *testing.T) {
for _, tc := range []struct {
name, contentType, body string
}{
{"json", "application/json", `{"turnstile_token":"tok","name":"Ann"}`},
{"form", "application/x-www-form-urlencoded", `turnstile_token=tok&name=Ann`},
{"other media type", "text/plain", `turnstile_token=tok`},
} {
t.Run(tc.name, func(t *testing.T) {
f := newSiteverify(t, 200, `{"success":true}`)
var seen []byte
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
seen, _ = io.ReadAll(r.Body)
w.WriteHeader(http.StatusOK)
})
// Open mode without a secret passes even when no token is found,
// so the restored body is checked for every media type.
v := f.verifier(turnstileTestSecret, false)
if tc.contentType == "text/plain" {
v = f.verifier("", false)
}
r := httptest.NewRequest(http.MethodPost, "/submit", strings.NewReader(tc.body))
r.Header.Set("Content-Type", tc.contentType)
rec := httptest.NewRecorder()
v.Middleware(false, "", nil, nil)(next).ServeHTTP(rec, r)
if rec.Code != http.StatusOK {
t.Fatalf("status %d", rec.Code)
}
if string(seen) != tc.body {
t.Fatalf("handler read %q, want %q", seen, tc.body)
}
})
}
}
// turnstileCall records what a TurnstileWriter received.
type turnstileCall struct {
status int
verdict TurnstileVerdict
}
func TestTurnstileMiddlewareStatuses(t *testing.T) {
f := newSiteverify(t, 200, `{"success":true}`)
for _, tc := range []struct {
name string
v *Turnstile
failClosed bool
target string
want int
verdict TurnstileVerdict
code string
}{
{"unavailable", f.verifier("", true), true, "/x?turnstile_token=t", 503, TurnstileUnavailable, "turnstile_unavailable"},
{"failed without token", f.verifier(turnstileTestSecret, true), true, "/x", 422, TurnstileFailed, "turnstile_failed"},
{"passed", f.verifier(turnstileTestSecret, true), false, "/x?turnstile_token=t", 204, TurnstilePassed, ""},
} {
t.Run(tc.name+" default writer", func(t *testing.T) {
called := false
rec := httptest.NewRecorder()
tc.v.Middleware(tc.failClosed, "", nil, nil)(passHandler(&called)).ServeHTTP(rec, httptest.NewRequest(http.MethodPost, tc.target, nil))
if rec.Code != tc.want || called != (tc.want == 204) {
t.Fatalf("status %d called %t", rec.Code, called)
}
if tc.code != "" {
if rec.Header().Get("Content-Type") != "application/json" {
t.Fatalf("content type %q", rec.Header().Get("Content-Type"))
}
var body map[string]string
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil || body["error"] != tc.code || len(body) != 1 {
t.Fatalf("body %s", rec.Body)
}
}
})
t.Run(tc.name+" app writer", func(t *testing.T) {
var calls []turnstileCall
write := func(w http.ResponseWriter, _ *http.Request, status int, verdict TurnstileVerdict) {
calls = append(calls, turnstileCall{status, verdict})
w.WriteHeader(status)
}
called := false
rec := httptest.NewRecorder()
tc.v.Middleware(tc.failClosed, "", nil, write)(passHandler(&called)).ServeHTTP(rec, httptest.NewRequest(http.MethodPost, tc.target, nil))
if tc.want == 204 {
if len(calls) != 0 || !called {
t.Fatalf("writer calls %v called %t", calls, called)
}
return
}
if len(calls) != 1 || calls[0].status != tc.want || calls[0].verdict != tc.verdict || called {
t.Fatalf("writer calls %v called %t", calls, called)
}
})
}
// A body over the router's limit answers 413 without calling siteverify.
before := f.calls.Load()
called := false
h := f.verifier(turnstileTestSecret, true).Middleware(true, "", nil, nil)(passHandler(&called))
rec := httptest.NewRecorder()
r := httptest.NewRequest(http.MethodPost, "/x", strings.NewReader(strings.Repeat("a", 64)))
r.Header.Set("Content-Type", "application/json")
r.Body = http.MaxBytesReader(rec, r.Body, 8)
h.ServeHTTP(rec, r)
if rec.Code != http.StatusRequestEntityTooLarge || called || f.calls.Load() != before {
t.Fatalf("over-limit body: status %d called %t", rec.Code, called)
}
// The client IP func supplies remoteip.
h = f.verifier(turnstileTestSecret, true).Middleware(true, "", func(*http.Request) string { return "192.0.2.9" }, nil)(passHandler(&called))
h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest(http.MethodPost, "/x?turnstile_token=t", nil))
if got := f.lastForm(t).Get("remoteip"); got != "192.0.2.9" {
t.Fatalf("remoteip = %q", got)
}
}
func TestTurnstileNeverLogsSecret(t *testing.T) {
var logs bytes.Buffer
prev := slog.Default()
slog.SetDefault(slog.New(slog.NewTextHandler(&logs, &slog.HandlerOptions{Level: slog.LevelDebug})))
defer slog.SetDefault(prev)
const token = "client-token-must-not-be-logged"
// A transport error: the endpoint refuses connections.
dead := httptest.NewServer(http.NotFoundHandler())
endpoint := dead.URL + "/siteverify?secret=" + turnstileTestSecret
dead.Close()
v := &Turnstile{Secret: turnstileTestSecret, Endpoint: endpoint, Client: &http.Client{Timeout: time.Second}}
called := false
rec := httptest.NewRecorder()
v.Middleware(false, "", nil, nil)(passHandler(&called)).ServeHTTP(rec, httptest.NewRequest(http.MethodPost, "/x?turnstile_token="+token, nil))
if rec.Code != http.StatusUnprocessableEntity || called {
t.Fatalf("transport error: status %d called %t", rec.Code, called)
}
// A refusal (success false) is logged at debug with the codes only.
f := newSiteverify(t, 200, `{"success":false,"error-codes":["invalid-input-response","`+turnstileTestSecret+`"]}`)
rec = httptest.NewRecorder()
f.verifier(turnstileTestSecret, false).Middleware(false, "", nil, nil)(passHandler(&called)).ServeHTTP(rec, httptest.NewRequest(http.MethodPost, "/x?turnstile_token="+token, nil))
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("refusal: status %d", rec.Code)
}
out := logs.String()
if !strings.Contains(out, "turnstile verification failed") || !strings.Contains(out, "turnstile token refused") {
t.Fatalf("expected both log lines, got:\n%s", out)
}
if strings.Contains(out, turnstileTestSecret) || strings.Contains(out, token) {
t.Fatalf("log leaks the secret or the token:\n%s", out)
}
// The returned error never carries them either.
_, err := v.Verify(t.Context(), token, "", false)
if err == nil || strings.Contains(err.Error(), turnstileTestSecret) || strings.Contains(err.Error(), token) || strings.Contains(err.Error(), "siteverify?") {
t.Fatalf("err = %v", err)
}
if errors.Is(err, errTurnstileRefused) {
t.Fatal("a transport error is not a refusal")
}
}

View File

@@ -40,7 +40,7 @@ Use cases:
- Markdown and multilingual fields: `type: markdown` edits source on a host text column; `cabana.RenderMarkdown` turns that source into HTML with the pinned goldmark engine and no unsafe HTML, and leftover script or iframe tags, event handlers, or javascript, vbscript or data URLs are rejected; the admin form preview renders through it via POST `/markdown/preview`. `type: mltext`, `type: mltextarea` and `type: mlmarkdown` take a JSON object of locale code to string; `mltextarea` is the multi-line variant and takes `size` like `textarea`. The form schema's `cabana.FormMeta.EnabledLocales` lists every enabled code when a `cabana.TranslationWriter` is published, default locale first and then the other enabled codes in the writer's order, so every ML control opens on the locale stored in the host columns; create seeds an empty string per code, and Show/save replace the host scalar with that map via `hydrateMLRecord` and `TranslationWriter.TranslatedExact` (missing non-default codes stay empty; D-11 fallback is not applied). A GET host string is merged onto the seed so sibling locales are not dropped. The default locale fills the host column; other locales reach a plugin-published `cabana.TranslationWriter` after the host row has a primary key, still inside the controller's permissioned save transaction. Relation-child create, update and show on `cabana.RelationService` use the same lift, apply and `hydrateMLRecord` path as controller save, still with no standalone translate-write route. An unknown field type, including an unknown `ml*` type, fails boot.
- Rich text: `type: richeditor` edits HTML in a host text column (the admin SPA shows a WYSIWYG editor). It takes the common field keys plus `size`; any other key fails boot, and settings and relation forms refuse the type (supported in controller forms only). On create and update a submitted value must be a string or null (anything else is a 422 on the field) and is sanitized through [seaglass](../seaglass/README.md) before it is filled: HTML the policy keeps whole is stored byte for byte, otherwise the clean HTML is stored and what was removed is reported in `meta.notices` under the field (`backend::lang.richeditor.removed_elements`, `removed_attributes`, `removed_comments` with the count and names, or `markup_changed`).
- Repeater: `type: repeater` edits a JSON list in one column (`lagoon.Jsonable`) with a nested `form.fields` of `text`, `textarea`, `number`, `checkbox`, `switch`, `dropdown` (static, method or `dependsOn` a field of the same row) and `richeditor` fields, plus `prompt`, `minItems`, `maxItems` and `titleFrom` (`cabana.RepeaterForm` in the schema; the span is always full). Other nested types, a nested repeater, nested `tab`, `context` or `preset`, and WinterCMS's `groups` and `style` fail boot; settings and relation forms refuse the type. A save decodes the rows before the projection: null is an empty list, anything but a list of objects is a 422 on the field, each row keeps only the nested names with scalars coerced per type, a required nested value missing in a row is a 422 under `field.index.nested`, the row bounds are 422s on the field (`repeater.max_items`, `repeater.min_items`), nested richeditor values are sanitized with notices under the row path, and nested dependent dropdowns are rechecked. Record responses carry the list of rows.
- Tree reorder (D-09): a `config_reorder.yaml` (`title`, `nameFrom`, `modelClass`, `parentColumn` default `parent_id`, `sortColumn` default `sort_order`, `maxDepth`, and an optional `partition` with `column`, `relation`, `nameFrom` and `label`) gives the controller a reorder page and the built-in `reorder` toolbar button; unknown keys, columns that are not record columns, a partition relation that is not a belongsTo relation field over the column, and a `reorder` button without the file fail boot. The list schema carries `reorder` (`cabana.ListReorder`). GET `.../reorder` answers the scoped tree (`cabana.AdminReorderTree`); POST `.../reorder` locks the scoped set (ListExtendQuery plus the partition) in one transaction, answers 409 `reorder_stale` when the posted ids differ from it in any way, 422 for a duplicate, a foreign parent, a cycle or a depth above `maxDepth`, and otherwise writes both tree columns without model hooks.
- Tree reorder (D-09): a `config_reorder.yaml` (`title`, `nameFrom`, `modelClass`, `parentColumn` default `parent_id`, `sortColumn` default `sort_order`, `maxDepth`, and an optional `partition` with `column`, `relation`, `nameFrom` and `label`) gives the controller a reorder page and the built-in `reorder` toolbar button; unknown keys, columns that are not record columns, a partition relation that is not a belongsTo relation field over the column, and a `reorder` button without the file fail boot. The list schema carries `reorder` (`cabana.ListReorder`). GET `.../reorder` answers the scoped tree (`cabana.AdminReorderTree`); POST `.../reorder` locks the scoped set (ListExtendQuery plus the partition) in one transaction, answers 409 `reorder_stale` when the posted ids differ from it in any way, 422 for a duplicate, a foreign parent, a cycle or a depth above `maxDepth`, and otherwise writes both tree columns without model hooks, then calls the controller's optional `pact.ReorderAfterSave` with the request's partition inside the same `lagoon.Transaction` (an error rolls the reorder back; `lagoon.AfterCommit` work it registers runs after the commit).
- Flat reorder (D-17): a `config_reorder.yaml` without `parentColumn` on a record that has no `parent_id` column (WinterCMS Sortable) compiles as a flat reorder: the list schema's `reorder` and the reorder routes' data carry `flat: true` (omitted on a tree, so tree payloads are unchanged), every node's `parent_id` is null, a save writes only the sort column, a node with a parent is a 422 on `nodes` (`backend::lang.reorder.flat_parent`), and the 409 stale-set and duplicate-id rules are unchanged. `maxDepth` above 0 on a flat reorder fails boot; an explicit `parentColumn` still means a tree.
- Singleton controllers (D-11): a controller implementing `pact.AdminSingleton` needs only `config_form.yaml`; GET `.../singleton` answers the id of the record `SingletonRecord` returns (created when absent; a first request that loses the creation race on a unique index runs once more and finds the record) after checking the form scope, the form schema carries `singleton`, and the list, create, bulk and delete routes answer 404. A settings link entry may open it.
- Dependent dropdowns: `dependsOn: <field>` on a `type: dropdown` field names a sibling with a scalar value (the same form, or the same repeater row); the dropdown takes no `options` and its controller implements `pact.DependentOptionsProvider`, else boot fails. GET `.../fields/{field}/dependent-options?values[<sibling>]=<value>` answers `cabana.AdminDependentOptionsResult` (`applies` and the localized `options` with their `depth`); `{field}` is `repeater.field` for a dropdown in a repeater row; the form schema carries `dependsOn`. Every save rechecks the value with the submitted sibling value (per row in a repeater): a value that does not apply is stored null, one outside the options is a 422 `form.dependent_invalid` on the field. A menu or sitemap controller returns `signpost.ReferenceOptions` here, the one admin binding of the [signpost](../signpost/README.md) menu item type registry. Settings and relation forms refuse `dependsOn` (supported in controller forms only).

View File

@@ -13,6 +13,7 @@ import (
"strconv"
"strings"
"git.golem15.com/golem15/summercms/modules/lagoon"
"git.golem15.com/golem15/summercms/modules/pact"
"git.golem15.com/golem15/summercms/modules/towel"
"gorm.io/gorm"
@@ -293,7 +294,10 @@ func (s *service) reorderGet(w http.ResponseWriter, r *http.Request) {
// requires the submitted ids to equal it exactly (409 reorder_stale
// otherwise), refuses a duplicate id, a parent outside the set, a cycle and a
// depth above maxDepth (422), then writes the parent and sort columns of
// every node and answers the new tree.
// every node, runs the controller's optional ReorderAfterSave hook (pact) with
// the partition (its error rolls the reorder back) and answers the new tree.
// The transaction is a lagoon.Transaction, so a hook can register work with
// lagoon.AfterCommit that runs only once the new order is committed.
func (s *service) reorderPost(w http.ResponseWriter, r *http.Request) {
s.protect(w, r, func(cc *CompiledController) {
if cc.reorder == nil || cc.singleton {
@@ -312,8 +316,8 @@ func (s *service) reorderPost(w http.ResponseWriter, r *http.Request) {
}
ctx := towel.WithLocale(r.Context(), schemaLocale(r.Context(), s.translator()))
var tree AdminReorderTree
err = db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
txCtx := withTx(ctx, tx)
err = lagoon.Transaction(ctx, db, func(lctx context.Context, tx *gorm.DB) error {
txCtx := withTx(lctx, tx)
if cc.reorder.partition != nil && in.Partition == nil {
return &ValidationError{Details: map[string]any{"partition": []string{"The partition field is required."}}}
}
@@ -347,6 +351,11 @@ func (s *service) reorderPost(w http.ResponseWriter, r *http.Request) {
return lifecycleFailure(cc, err)
}
}
if hook, ok := cc.Controller.(pact.ReorderAfterSave); ok && hook != nil {
if err := lifecycleFailure(cc, hook.ReorderAfterSave(txCtx, in.Partition)); err != nil {
return err
}
}
tree, err = s.reorderTree(txCtx, tx, cc, in.Partition, false)
return err
})

View File

@@ -0,0 +1,239 @@
package cabana_test
import (
"context"
"errors"
"fmt"
"io/fs"
"net/http"
"strings"
"sync"
"testing"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/cabana"
"git.golem15.com/golem15/summercms/modules/lagoon"
"git.golem15.com/golem15/summercms/modules/pact"
"git.golem15.com/golem15/summercms/modules/party"
"gorm.io/gorm"
)
// Unit tests (plan 05 of phase 19.3) for pact.ReorderAfterSave (D-11): the
// hook runs inside the reorder transaction after the new order is written,
// receives the partition, and its error rolls the reorder back. They reuse
// the partitioned flat acme.shelf fixture of catalog_smoke_test.go.
// hookShelfPlugin is acme.shelf with a pins controller that implements
// pact.ReorderAfterSave through hook.
type hookShelfPlugin struct {
hook func(ctx context.Context, partition *uint64) error
}
func (hookShelfPlugin) ID() string { return "acme.shelf" }
func (hookShelfPlugin) Requires() []string { return nil }
func (hookShelfPlugin) Register(*backpack.App) error { return nil }
func (hookShelfPlugin) Boot(*backpack.App) error { return nil }
func (hookShelfPlugin) AdminFS() fs.FS { return shelfFS }
func (p hookShelfPlugin) AdminControllers() []pact.AdminController {
return []pact.AdminController{hookedPins{hook: p.hook}}
}
func (hookShelfPlugin) Permissions() []pact.Permission {
return []pact.Permission{{Code: "acme.shelf.access", Roles: []string{"developer"}}}
}
// hookedPins is shelfPins with the reorder hook.
type hookedPins struct {
shelfPins
hook func(ctx context.Context, partition *uint64) error
}
func (h hookedPins) ReorderAfterSave(ctx context.Context, partition *uint64) error {
return h.hook(ctx, partition)
}
var _ pact.ReorderAfterSave = hookedPins{}
// shelfHookEnv is a logged-in admin over plugin with fresh acme.shelf
// tables, two shelves and three pins (A=1 and B=2 on top, X=7 on low).
type shelfHookEnv struct {
*conformEnv
top, low shelfShelf
a, b, x uint
}
func newShelfHookEnv(t *testing.T, plugin party.Plugin) *shelfHookEnv {
t.Helper()
gdb := adminGorm(t)
models := []any{&shelfPin{}, &shelfShelf{}}
if err := gdb.Migrator().DropTable(models...); err != nil {
t.Fatal(err)
}
if err := gdb.AutoMigrate(models...); err != nil {
t.Fatal(err)
}
login := fmt.Sprintf("shelf-hook-%d", time.Now().UnixNano())
insertAdmin(t, gdb, login, login+"@example.test", adminTestPassword, true, false)
h, err := paymentsAssemble(t, gdb, []party.Plugin{plugin})
if err != nil {
t.Fatal(err)
}
env := &shelfHookEnv{conformEnv: &conformEnv{h: h, db: gdb, login: login}, top: shelfShelf{Title: "Top"}, low: shelfShelf{Title: "Low"}}
if rec := env.send(t, http.MethodPost, "/auth/login", map[string]string{"login": login, "password": adminTestPassword}, false); rec.Code != http.StatusOK {
t.Fatalf("login status=%d body=%s", rec.Code, rec.Body.String())
}
for _, s := range []*shelfShelf{&env.top, &env.low} {
if err := gdb.Create(s).Error; err != nil {
t.Fatal(err)
}
}
pins := []shelfPin{{Name: "A", ShelfID: &env.top.ID, SortOrder: 1}, {Name: "B", ShelfID: &env.top.ID, SortOrder: 2}, {Name: "X", ShelfID: &env.low.ID, SortOrder: 7}}
for i := range pins {
if err := gdb.Create(&pins[i]).Error; err != nil {
t.Fatal(err)
}
}
env.a, env.b, env.x = pins[0].ID, pins[1].ID, pins[2].ID
return env
}
// swap posts B=1, A=2 for the top shelf and returns the status and body.
func (e *shelfHookEnv) swap(t *testing.T) (int, string) {
t.Helper()
rec := e.send(t, http.MethodPost, "/acme/shelf/pins/reorder", map[string]any{
"partition": e.top.ID,
"nodes": []map[string]any{
{"id": e.b, "parent_id": nil, "sort_order": 1},
{"id": e.a, "parent_id": nil, "sort_order": 2},
},
}, true)
return rec.Code, rec.Body.String()
}
// sorts reads the sort values of every pin through db.
func sorts(t *testing.T, db *gorm.DB) map[uint]int {
t.Helper()
var all []shelfPin
if err := db.Order("id").Find(&all).Error; err != nil {
t.Fatal(err)
}
out := map[uint]int{}
for _, p := range all {
out[p.ID] = p.SortOrder
}
return out
}
func TestReorderHookRunsInTx(t *testing.T) {
var (
mu sync.Mutex
calls int
gotPart *uint64
inTx map[uint]int
outside map[uint]int
hasTx bool
afterCommit map[uint]int
)
var env *shelfHookEnv
env = newShelfHookEnv(t, hookShelfPlugin{hook: func(ctx context.Context, partition *uint64) error {
mu.Lock()
defer mu.Unlock()
calls++
gotPart = partition
tx, ok := cabana.TxFromContext(ctx)
hasTx = ok
if !ok {
return errors.New("no transaction in the hook context")
}
inTx = sorts(t, tx.WithContext(ctx))
// Another connection still sees the committed order.
outside = sorts(t, env.db)
lagoon.AfterCommit(ctx, tx, func(ctx context.Context, db *gorm.DB) {
mu.Lock()
defer mu.Unlock()
afterCommit = sorts(t, db.WithContext(ctx))
})
return nil
}})
code, body := env.swap(t)
if code != http.StatusOK {
t.Fatalf("POST status=%d body=%s", code, body)
}
mu.Lock()
defer mu.Unlock()
if calls != 1 || !hasTx {
t.Fatalf("hook calls=%d tx=%t", calls, hasTx)
}
if gotPart == nil || *gotPart != uint64(env.top.ID) {
t.Fatalf("partition = %v, want %d", gotPart, env.top.ID)
}
if inTx[env.b] != 1 || inTx[env.a] != 2 || inTx[env.x] != 7 {
t.Fatalf("hook saw %v through the transaction, want the new order", inTx)
}
if outside[env.a] != 1 || outside[env.b] != 2 {
t.Fatalf("outside the transaction %v, want the old order before commit", outside)
}
if afterCommit == nil || afterCommit[env.b] != 1 || afterCommit[env.a] != 2 {
t.Fatalf("AfterCommit work saw %v, want the committed new order", afterCommit)
}
if got := sorts(t, env.db); got[env.b] != 1 || got[env.a] != 2 || got[env.x] != 7 {
t.Fatalf("stored = %v", got)
}
}
func TestReorderHookErrorRollsBack(t *testing.T) {
for _, tc := range []struct {
name string
err error
status int
}{
{"plain error", errors.New("purge registry unavailable"), http.StatusInternalServerError},
{"validation error", &cabana.ValidationError{Details: map[string]any{"nodes": []string{"Refused."}}}, http.StatusUnprocessableEntity},
{"forbidden", &cabana.ForbiddenError{Message: "Locked."}, http.StatusForbidden},
} {
t.Run(tc.name, func(t *testing.T) {
var afterCommitRan bool
env := newShelfHookEnv(t, hookShelfPlugin{hook: func(ctx context.Context, _ *uint64) error {
tx, _ := cabana.TxFromContext(ctx)
lagoon.AfterCommit(ctx, tx, func(context.Context, *gorm.DB) { afterCommitRan = true })
return tc.err
}})
before := sorts(t, env.db)
code, body := env.swap(t)
if code != tc.status {
t.Fatalf("status=%d want %d body=%s", code, tc.status, body)
}
if after := sorts(t, env.db); fmt.Sprint(after) != fmt.Sprint(before) {
t.Fatalf("sort values changed: before %v after %v", before, after)
}
if afterCommitRan {
t.Fatal("AfterCommit work ran after a rolled-back reorder")
}
if tc.status == http.StatusInternalServerError && (strings.Contains(body, "purge registry") || !strings.Contains(body, `"error"`)) {
t.Fatalf("500 body = %s", body)
}
})
}
}
func TestReorderWithoutHookUnchanged(t *testing.T) {
plain := newShelfHookEnv(t, shelfPlugin{})
code, plainBody := plain.swap(t)
if code != http.StatusOK {
t.Fatalf("POST status=%d body=%s", code, plainBody)
}
if got := sorts(t, plain.db); got[plain.b] != 1 || got[plain.a] != 2 || got[plain.x] != 7 {
t.Fatalf("stored = %v", got)
}
// A hook that does nothing answers the same tree.
hooked := newShelfHookEnv(t, hookShelfPlugin{hook: func(context.Context, *uint64) error { return nil }})
code, hookedBody := hooked.swap(t)
if code != http.StatusOK {
t.Fatalf("hooked POST status=%d body=%s", code, hookedBody)
}
if plainBody != hookedBody {
t.Fatalf("answers differ:\nwithout hook %s\nwith hook %s", plainBody, hookedBody)
}
}

227
modules/cooler/README.md Normal file
View File

@@ -0,0 +1,227 @@
# cooler
Cache drivers and shared counters for SummerCMS: memory and Redis caches with a key prefix and TTLs, a Redis-backed rate-limit store, and the site-cache purge after admin saves.
`import "git.golem15.com/golem15/summercms/modules/cooler"`
**Name:** The cool box you bring to the beach; it keeps things fresh and at hand for everyone in the group.
## Overview
cooler holds the state that several application instances share. `cooler.Cache` is a Laravel-core cache (get, set, has, forget, remember, increment, flush) with per-entry TTLs under a per-application key prefix, kept in process by an otter cache or shared through Redis. The second piece is a Redis-backed rate-limit store: with `http.rate_limit.store: redis`, the `serve` command of [surf](../surf/README.md) calls `cooler.Install`, which builds a Redis client from `redis.url`, and publishes the store so the `throttle` middleware counts in Redis instead of in the process. Two instances behind one load balancer then enforce one limit together. The third piece is the site-cache purge: with `site_cache.purge_url` and `site_cache.purge_token` set, an admin save in a plugin sends one bearer-token POST to the host site's purge route after the save commits, so a cached page shows the change on its next view. It is the counterpart of Laravel's `Cache` facade with its array and Redis stores, and of the Redis store behind `RateLimiter` and the `throttle` middleware.
## Role in the framework
A service module on `backpack` and `compass`, below `surf`: surf imports cooler, never the reverse. `cooler.RateLimitStore` has the method set of `surf.Store` (through `cooler.Attempter`), so surf takes no Redis dependency and cooler imports neither surf nor cabana. The framework wires cooler at boot; applications import nothing.
Use cases:
- Cache an expensive query or an upstream answer for a few minutes, in a plugin or a job.
- Keep `throttle` limits exact when an application runs as several instances behind a load balancer.
- Share one Redis connection setting (`redis.url`) between the framework services that need Redis.
- Drop a server-rendered site's page cache right after an editor saves a FAQ, a menu or a post in the admin, instead of waiting for the cache's TTL.
## Features
- `cooler.Cache` methods: `cooler.Cache.Get`, `cooler.Cache.Set`, `cooler.Cache.Has`, `cooler.Cache.Forget`, `cooler.Cache.Remember`, `cooler.Cache.Increment` and `cooler.Cache.Flush`, with a TTL per entry (0 or less: no expiry). No tags and no locks.
- Memory driver (`cache.driver: memory`, the default): `cooler.NewMemoryCache` on otter v2, bounded by `cache.memory.max_entries` (default `cooler.DefaultMaxEntries`, 10000) so request-derived keys cannot grow memory without limit. Values are copied in and out, and `cooler.Cache.Increment` is atomic per key.
- Redis driver (`cache.driver: redis`): `cooler.NewRedisCache` with keys `<prefix>cache:<key>`, `SET … PX` for TTLs, `EXISTS`, `UNLINK` and `INCRBY`. `cooler.Cache.Flush` walks `SCAN MATCH <prefix>cache:*` in batches of 500 and removes each batch with `UNLINK`; it never empties the whole database, so rate-limit counters and other applications' keys stay.
- `cooler.Cache.Increment` starts a missing key at 0 without a TTL and keeps an existing key's expiry, as Redis `INCRBY` does.
- `cooler.Cache.Remember` degrades instead of failing the page: a failed read still runs the callback and returns its value, a failed write is skipped, and both are logged at most once per minute. The other methods return their errors.
- `cooler.Install` reads `cache.driver` and `http.rate_limit.store` (each `memory`, the default, or `redis`), publishes the Cache on the app (reusing one `cooler.FromApp` published earlier) and, when either selects Redis, builds one client from `redis.url`, shared by both, and pings it within `cooler.PingTimeout` (3 s).
- `cooler.FromApp` gives commands that do not run `serve` (a dedicated job worker, console commands) the published Cache, or builds the configured one and publishes it, without a ping: go-redis dials on the first command.
- Start-up never fails because Redis is down: an unanswered ping is one warning that names the endpoint (host and port, never the URL), the limiter counts per instance and `cooler.Cache.Remember` degrades until Redis answers. A missing or malformed `redis.url`, an unknown driver name or a `cache.memory.max_entries` below 1 fails start-up with an error that names the key and never contains the URL or its password.
- `cooler.RateLimitStore` runs one atomic Lua script per attempt with the semantics of `surf.MemoryStore`: the first attempt opens the window, a denied attempt does not increment, the window never extends, and the retry time is what is left of the window. Redis keeps the clock, so instance clock skew does not matter.
- Each attempt is bounded by `cooler.AttemptTimeout` (500 ms). When Redis fails, the attempt is counted in the fallback store surf passes in (a per-instance `surf.MemoryStore`) and a warning is logged at most once per minute.
- Keys carry a per-application prefix: `cache.prefix`, or by default `app.name` lower-cased with every run of other characters than `a-z` and `0-9` replaced by `_`, then `:` (`summer:` when `app.name` is empty). Cache keys are `<prefix>cache:<key>`, rate-limit keys `<prefix>rl:<key>`.
- `cooler.OpenRedis` accepts `redis://`, `rediss://` (TLS) and `unix://` URLs, does not dial at construction, and skips the `CLIENT SETINFO` and maintenance-notification handshakes.
- Only `serve` installs cooler: `route:list` and `surf.Assemble` in tests keep the in-process store and never dial Redis.
- Site-cache purge: `cooler.SiteCachePurger` sends `POST {"kind":…,"slug":…}` with `Authorization: Bearer <token>` to `site_cache.purge_url`. `cooler.PurgeSiteCacheAfterCommit` queues that request on the save transaction through `lagoon.AfterCommit`, so it is queued only after the save commits and never for a save that rolled back. The save never waits for the site: a per-purger worker sends the request, with at most one in flight, and saves committed meanwhile collapse into one follow-up request (each purge clears the site's whole route cache), so a bulk delete sends at most two. The purger's `Wait` method drains the queue for tests and commands (see the API reference), and `cooler.Handle.Close` drains it at shutdown.
- A purge never fails a save: each request is bounded by `cooler.SiteCachePurgeTimeout` (2 s) and detached from the request's cancellation, and a failure, an error status or a timeout is one warning (`cooler: site cache purge failed`) with the kind, slug and status. A success is one info line (`cooler: site cache purged`) with the purged count the site answered. Log lines and errors never carry the token or the full URL.
- The purge URL must be http or https with a host and without user info or a fragment; plain http is accepted only for a loopback host (127.0.0.0/8, `::1`, `localhost`), so the token never crosses a network in clear. The token needs at least `cooler.SiteCachePurgeMinTokenLength` (32) characters, the same minimum the site's route applies. Setting only one of the two keys, or an invalid value, fails `serve` at boot with an error that names the key.
- `cooler.Install` publishes the purger at boot unless a plugin already published one (a plugin that still reads its own deprecated keys), and `cooler.SiteCacheFromApp` gives admin hooks the same purger lazily in commands and tests that do not run `serve`. Plugins take the save transaction from `cabana.TxFromContext` themselves; cooler imports neither surf nor cabana.
## Usage
A plugin or job reads and writes the cache the framework published:
```go
package blog
import (
"context"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/cooler"
)
// Sidebar returns the rendered sidebar, cached for five minutes.
func Sidebar(ctx context.Context, app *backpack.App, render func(context.Context) ([]byte, error)) ([]byte, error) {
cache, err := cooler.FromApp(app)
if err != nil {
return nil, err
}
return cache.Remember(ctx, "blog:sidebar", 5*time.Minute, render)
}
```
`cooler.FromApp` returns the Cache `serve` published, so in a request handler `app.Lookup[cooler.Cache]()` works as well. In a separate `queue:work` process or a console command, `cooler.FromApp` builds the configured Cache the first time.
Select Redis for the cache, the rate limiter or both:
```yaml
# config/cache.yaml
driver: redis
```
Select the shared rate-limit store in the application's config and keep the URL, which may carry a password, in the environment (`SUMMER_REDIS__URL`):
```yaml
# config/http.yaml
rate_limit:
store: redis
```
```yaml
# config/redis.yaml (or SUMMER_REDIS__URL=redis://:secret@10.0.0.7:6379/0)
url: redis://10.0.0.7:6379/0
```
Nothing else changes: routes keep `throttle:60,1` or `throttle:<bucket>`, and `serve` publishes the Redis store before it builds the router.
An admin controller purges the host site's page cache after its saves. The hook runs inside the save transaction, and the purge waits for the commit:
```go
package blog
import (
"context"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/cabana"
"git.golem15.com/golem15/summercms/modules/cooler"
)
// Article is the blog's article model.
type Article struct {
Slug string
}
type articlesController struct {
app func() *backpack.App
}
// FormAfterUpdate purges the site cache once the save commits.
func (c articlesController) FormAfterUpdate(ctx context.Context, model any) error {
tx, ok := cabana.TxFromContext(ctx)
if !ok {
return nil
}
purger := cooler.SiteCacheFromApp(c.app())
cooler.PurgeSiteCacheAfterCommit(ctx, tx, purger, "article", model.(*Article).Slug)
return nil
}
```
A tool outside `serve` can use the store directly:
```go
package blog
import (
"context"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/cooler"
)
// AllowExport admits five exports a minute per user across all instances.
func AllowExport(ctx context.Context, app *backpack.App, fallback cooler.Attempter, userKey string) (bool, error) {
client, err := cooler.OpenRedis(ctx, app.Config.String(cooler.RedisURLKey))
if err != nil {
return false, err
}
defer client.Close()
store := cooler.NewRateLimitStore(client, "acme:", fallback)
allowed, _, _ := store.Attempt("export|"+userKey, 5, time.Minute)
return allowed, nil
}
```
## API reference
| Identifier | Description |
|------------|-------------|
| `cooler.Cache` | The cache: `Get`, `Set`, `Has`, `Forget`, `Remember`, `Increment` and `Flush` with per-entry TTLs under the app's key prefix. |
| `cooler.Cache.Remember` | Returns the cached value or computes, stores and returns it; degrades on cache errors. |
| `cooler.Cache.Increment` | Adds to an integer value; a missing key starts at 0 without a TTL. |
| `cooler.Cache.Flush` | Removes this application's cache keys only. |
| `cooler.NewMemoryCache` | In-process driver on otter v2, bounded to a number of entries. |
| `cooler.NewRedisCache` | Redis driver on a go-redis client, keys `<prefix>cache:<key>`. |
| `cooler.FromApp` | Returns the published Cache, or builds the configured one without a ping and publishes it. |
| `cooler.Install` | Reads cooler's config, publishes the Cache, opens and pings Redis when a Redis driver is selected, and returns a `cooler.Handle`. Called by surf's `serve`. |
| `cooler.Handle` | What `cooler.Install` set up. |
| `cooler.Handle.Cache` | The Cache `cooler.Install` published or found. |
| `cooler.Handle.Close` | Waits for the site-cache purges queued on the purger `cooler.Install` published (up to `cooler.SiteCachePurgeTimeout` plus one second), then closes the Redis client `cooler.Install` opened, if any. |
| `cooler.Handle.RateLimitRedis` | Reports whether `http.rate_limit.store` selects the Redis store. |
| `cooler.Handle.RateLimitStore` | Returns the Redis rate-limit store with the given fallback, or the fallback itself when Redis is not selected. |
| `cooler.Attempter` | One atomic fixed-window admission; the method set of `surf.Store`. |
| `cooler.RateLimitStore` | Redis fixed-window store shared by all instances, with a fallback while Redis fails. |
| `cooler.NewRateLimitStore` | Builds a `cooler.RateLimitStore` from a go-redis client, a key prefix and a fallback. |
| `cooler.RateLimitStore.Attempt` | Admits and counts one request in one Lua script; falls back on a Redis error. |
| `cooler.OpenRedis` | Builds a go-redis client from a `redis://`, `rediss://` or `unix://` URL without dialing; errors never contain the URL. |
| `cooler.SiteCachePurger` | Sends the site-cache purge request; a nil purger does nothing. |
| `cooler.NewSiteCachePurger` | Builds the purger from the values of `site_cache.purge_url` and `site_cache.purge_token`; both empty returns nil; errors name those keys and never the token. |
| `cooler.NewSiteCachePurgerFor` | The same with the config keys its errors name passed in, for a plugin that reads deprecated keys of its own. |
| `cooler.SiteCachePurger.Purge` | Sends one purge request and blocks until it is done (2 s timeout, detached from cancellation, redirects not followed); the send the after-commit queue uses. Logs the outcome; never fails. |
| `cooler.SiteCachePurger.Wait` | Blocks until no queued purge is pending or in flight, or until the context ends (returning its error); nil at once for a nil or idle purger. |
| `cooler.SiteCachePurger.Endpoint` | The purge URL's host and path, for log lines. |
| `cooler.PurgeSiteCacheAfterCommit` | Queues one purge after the given transaction commits (at once outside a transaction), fire and forget: one request in flight per purger, later saves collapse into one follow-up. |
| `cooler.SiteCacheFromApp` | Returns the published purger, or builds it from the framework keys and publishes it; nil when unset or invalid. |
| `cooler.SiteCachePurgeURLKey` | `site_cache.purge_url`. |
| `cooler.SiteCachePurgeTokenKey` | `site_cache.purge_token`. |
| `cooler.SiteCachePurgeTimeout` | Bound of one purge request (2 s). |
| `cooler.SiteCachePurgeMinTokenLength` | Shortest accepted purge token (32). |
| `cooler.RedisURLKey` | `redis.url`. |
| `cooler.RateLimitStoreKey` | `http.rate_limit.store`. |
| `cooler.CacheDriverKey` | `cache.driver`. |
| `cooler.CachePrefixKey` | `cache.prefix`. |
| `cooler.CacheMaxEntriesKey` | `cache.memory.max_entries`. |
| `cooler.DefaultMaxEntries` | Default size bound of the memory driver (10000). |
| `cooler.PingTimeout` | Bound of the start-up ping (3 s). |
| `cooler.AttemptTimeout` | Bound of one rate-limit attempt on Redis (500 ms). |
## Configuration
`cooler.Install` reads these keys from the app's [compass](../compass/README.md) config:
| Key | Default | Controls |
|-----|---------|----------|
| `cache.driver` | `memory` | Cache driver: `memory` (per instance, otter) or `redis` (shared). Any other value fails at boot. Environment: `SUMMER_CACHE__DRIVER`. |
| `cache.memory.max_entries` | `10000` | Size bound of the memory driver; below 1 fails at boot. Set it in YAML (environment values are strings). |
| `http.rate_limit.store` | `memory` | Where `throttle` counters live: `memory` (per instance) or `redis` (shared). Any other value fails at boot. Environment: `SUMMER_HTTP__RATE_LIMIT__STORE`. |
| `redis.url` | none | Redis URL (`redis://`, `rediss://` or `unix://`). Required when a Redis cache or rate-limit store is selected; keep it in the environment when it carries a password. Environment: `SUMMER_REDIS__URL`. |
| `cache.prefix` | from `app.name` | Prefix of every cooler key. Environment: `SUMMER_CACHE__PREFIX`. |
| `site_cache.purge_url` | none | The host site's purge route, for example `http://127.0.0.1:3000/api/__site__/cache/purge`. http only to a loopback host, https otherwise; an https URL on another host goes through the site's proxy, so the site sets `NUXT_SITE_CACHE_ALLOW_PROXIED=true` (see [A purge URL on another host](../../docs/services/caching.md#a-purge-url-on-another-host)). Set together with the token, or neither (no purge). Environment: `SUMMER_SITE_CACHE__PURGE_URL`. |
| `site_cache.purge_token` | none | Bearer token of the purge request, at least 32 characters, the same value as the site's `NUXT_SITE_CACHE_PURGE_TOKEN`. Keep it in the environment. Environment: `SUMMER_SITE_CACHE__PURGE_TOKEN`. |
## Dependencies
- SummerCMS modules: [backpack](../backpack/README.md), [compass](../compass/README.md), [lagoon](../lagoon/README.md) (`lagoon.AfterCommit` for the after-commit purge).
- Third-party: `github.com/redis/go-redis/v9` (Redis client), `github.com/maypok86/otter/v2` (memory driver).
- Third-party (purge): `gorm.io/gorm` (the save transaction handle).
- Standard library: `bytes`, `context`, `encoding/json`, `errors`, `fmt`, `io`, `log/slog`, `net`, `net/http`, `net/url`, `strconv`, `strings`, `sync`, `time`.
- Tests additionally use `github.com/alicebob/miniredis/v2` (an in-process Redis for the `-short` suites; only `_test.go` files import it) and `github.com/testcontainers/testcontainers-go` (Redis and Postgres containers).
## Testing
```sh
go test -short ./modules/cooler/... # no Docker: memory driver, miniredis, httptest receivers
go test ./modules/cooler/... # also the Redis and Postgres containers (needs Docker)
```
The `-short` suites need no external services. `TestMemoryCache`, `TestMemoryCacheBounded` and `TestMemoryIncrementConcurrent` cover the memory driver; `TestRedisCache` and `TestRedisFlushOnlyPrefix` run the Redis driver on miniredis (TTLs through `FastForward`). `TestRateLimitParity` runs one table of attempts through `surf.NewMemoryStore` and the Redis store and expects the same answers; `TestRateLimitFallbackOnError` stops Redis and checks the fallback store answers with one warning. `TestInstallConfigErrors`, `TestInstallPingWarning` and `TestFromAppLazy` cover start-up; `TestSiteCachePurgeConfig` and `TestSiteCachePurgeRequest` the purge configuration and request against an `httptest` receiver.
The Docker suites skip under `-short`: `TestRateLimitParityDocker` and `TestRedisCacheDocker` run the same checks on a `redis:7-alpine` container, `TestSiteCacheAfterCommitOnly` and `TestSiteCacheFailureNeverFails` run the after-commit purge in Postgres transactions, and `TestRedisCacheSmoke` is the original Redis smoke test. The cross-instance rate-limit test lives in surf (`go test ./modules/surf -run TestRateLimitAcrossInstancesSmoke`).

461
modules/cooler/cooler.go Normal file
View File

@@ -0,0 +1,461 @@
// Package cooler holds the shared-state services of a SummerCMS application:
// a Laravel-core Cache with a memory (otter) and a Redis driver under a
// per-application key prefix, a Redis-backed fixed-window rate-limit
// store that keeps limits across application instances, and the site-cache
// purge that asks the host site to drop its page caches after admin saves.
// One redis.url key configures Redis for the cache and the store. The
// framework installs cooler at boot (surf's serve command) and FromApp and
// SiteCacheFromApp serve other commands, so applications import nothing.
//
// cooler imports neither surf nor cabana: surf imports cooler, and the
// rate-limit store satisfies surf.Store structurally through Attempter.
package cooler
import (
"context"
"errors"
"fmt"
"log/slog"
"strings"
"sync"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/compass"
"github.com/redis/go-redis/v9"
)
// Config keys and defaults cooler reads.
const (
// RedisURLKey is the shared Redis URL (redis://, rediss:// or unix://),
// required when a Redis driver is selected. Environment: SUMMER_REDIS__URL.
RedisURLKey = "redis.url"
// CacheDriverKey selects the cache driver: memory (default) or redis.
CacheDriverKey = "cache.driver"
// CachePrefixKey is the per-application key prefix. The default derives
// from app.name.
CachePrefixKey = "cache.prefix"
// CacheMaxEntriesKey bounds the memory driver's entry count.
CacheMaxEntriesKey = "cache.memory.max_entries"
// RateLimitStoreKey selects where rate-limit counters live: memory
// (default, per instance) or redis (shared).
RateLimitStoreKey = "http.rate_limit.store"
// DefaultMaxEntries is the memory driver's size bound when
// cache.memory.max_entries is unset.
DefaultMaxEntries = 10000
// PingTimeout bounds the start-up Ping that Install sends to Redis.
PingTimeout = 3 * time.Second
// AttemptTimeout bounds one rate-limit attempt on Redis before the
// fallback store answers.
AttemptTimeout = 500 * time.Millisecond
)
// Driver names accepted by cache.driver and http.rate_limit.store.
const (
driverMemory = "memory"
driverRedis = "redis"
)
// defaultPrefix is the key prefix when neither cache.prefix nor app.name is set.
const defaultPrefix = "summer:"
// Attempter is one atomic fixed-window admission. It has exactly the method
// set of surf.Store, so any surf.Store is an Attempter and a RateLimitStore
// is a surf.Store.
type Attempter interface {
Attempt(key string, max int, decay time.Duration) (allowed bool, attempts int, retryAfter time.Duration)
}
// Cache is a Laravel-core cache: byte values under string keys, each with
// its own TTL, stored under the application's key prefix. A ttl of 0 or less
// keeps the value until it is forgotten, flushed or evicted. There are no tags
// and no locks.
type Cache interface {
// Get returns the value at key and whether it was found.
Get(ctx context.Context, key string) ([]byte, bool, error)
// Set stores value at key for ttl (0 or less: no expiry).
Set(ctx context.Context, key string, value []byte, ttl time.Duration) error
// Has reports whether key holds a live value.
Has(ctx context.Context, key string) (bool, error)
// Forget removes key.
Forget(ctx context.Context, key string) error
// Remember returns the value at key, or calls fn, stores its value for
// ttl and returns it. It degrades instead of failing: when the read fails
// it still calls fn and returns its value, a failing write is skipped,
// and both are logged as a warning at most once per minute. Only an error
// from fn is returned.
Remember(ctx context.Context, key string, ttl time.Duration, fn func(context.Context) ([]byte, error)) ([]byte, error)
// Increment adds by to the integer at key and returns the new value. A
// missing key starts at 0 and gets no TTL; an existing key keeps its
// expiry, as Redis INCRBY does.
Increment(ctx context.Context, key string, by int64) (int64, error)
// Flush removes every key of this application's cache, and nothing else.
Flush(ctx context.Context) error
}
// Handle is what Install set up. Close drains the site-cache purger and
// releases the Redis client Install opened, if any.
type Handle struct {
client *redis.Client
prefix string
rateRedis bool
cache Cache
log *slog.Logger
purger *SiteCachePurger
}
// Close first waits for the site-cache purges queued on the purger Install
// published, up to SiteCachePurgeTimeout plus one second, then closes the
// Redis client Install opened. It is safe on a nil Handle and when no Redis
// driver was selected. A Redis-backed Cache that Install built on that
// client stops working; serve closes the handle on shutdown.
func (h *Handle) Close() error {
if h == nil {
return nil
}
if h.purger != nil {
ctx, cancel := context.WithTimeout(context.Background(), SiteCachePurgeTimeout+time.Second)
_ = h.purger.Wait(ctx) // the deadline only bounds shutdown
cancel()
}
if h.client == nil {
return nil
}
err := h.client.Close()
h.client = nil
return err
}
// Cache returns the Cache Install published (or the one FromApp had
// published before it).
func (h *Handle) Cache() Cache {
if h == nil {
return nil
}
return h.cache
}
// RateLimitRedis reports whether http.rate_limit.store selects the Redis store.
func (h *Handle) RateLimitRedis() bool {
return h != nil && h.rateRedis && h.client != nil
}
// RateLimitStore returns the Redis rate-limit store, falling back to fallback
// while Redis does not answer. When the Redis store is not selected it
// returns fallback itself.
func (h *Handle) RateLimitStore(fallback Attempter) Attempter {
if !h.RateLimitRedis() {
return fallback
}
s := NewRateLimitStore(h.client, h.prefix, fallback)
s.warn.setLogger(h.log)
return s
}
// Install reads cooler's configuration, opens what it selects and publishes
// the Cache on app (reusing one FromApp already published).
//
// cache.driver selects the cache (memory, the default, or redis) and
// http.rate_limit.store the rate-limit store (memory or redis). When either
// selects Redis, Install builds one client from redis.url, shared by both,
// and pings it within PingTimeout. An unreachable Redis is logged as a
// warning (the endpoint only, never the URL) and start-up continues: the rate
// limiter falls back to per-instance memory and Remember degrades until Redis
// answers. A missing or invalid redis.url, an unknown driver name or a
// non-positive cache.memory.max_entries is a boot error naming the key.
//
// Install also publishes the site-cache purger (SiteCachePurger) built from
// site_cache.purge_url and site_cache.purge_token, unless a plugin already
// published one; an invalid site_cache configuration is a boot error naming
// the key, and an enabled purger is logged with its Endpoint.
//
// Only the serve command calls Install; route:list and tests never dial Redis.
func Install(ctx context.Context, app *backpack.App) (*Handle, error) {
if app == nil {
return nil, fmt.Errorf("cooler: app is nil")
}
if ctx == nil {
ctx = context.Background()
}
cfg := app.Config
h := &Handle{prefix: keyPrefix(cfg), log: loggerFromApp(app)}
purger, err := installSiteCache(app, h.log)
if err != nil {
return nil, err
}
h.purger = purger
store, err := driverName(cfg, RateLimitStoreKey)
if err != nil {
return nil, err
}
cacheDriver, err := driverName(cfg, CacheDriverKey)
if err != nil {
return nil, err
}
h.rateRedis = store == driverRedis
existing, published := app.Lookup[Cache]()
published = published && existing != nil
needRedis := h.rateRedis || (!published && cacheDriver == driverRedis)
if needRedis {
selectedBy := RateLimitStoreKey
if !h.rateRedis {
selectedBy = CacheDriverKey
}
client, err := openFromConfig(ctx, cfg, selectedBy)
if err != nil {
return nil, err
}
h.client = client
}
if published {
h.cache = existing
} else {
c, err := buildCache(cfg, cacheDriver, h.prefix, h.client)
if err != nil {
_ = h.Close()
return nil, err
}
setLogger(c, h.log)
if err := app.Publish[Cache](c); err != nil {
_ = h.Close()
return nil, fmt.Errorf("cooler: %w", err)
}
h.cache = c
}
if h.client != nil {
ping(ctx, h.client, h.log)
}
return h, nil
}
// FromApp returns the Cache published on app, or builds the configured one
// and publishes it. It never pings: go-redis dials on the first command, so a
// job worker, a console command or a plugin's Boot can call it without a
// running Redis. When another caller published first, FromApp returns that
// Cache. A Redis client FromApp opens lives as long as the process.
func FromApp(app *backpack.App) (Cache, error) {
if app == nil {
return nil, fmt.Errorf("cooler: app is nil")
}
if c, ok := app.Lookup[Cache](); ok && c != nil {
return c, nil
}
cfg := app.Config
driver, err := driverName(cfg, CacheDriverKey)
if err != nil {
return nil, err
}
prefix := keyPrefix(cfg)
var client *redis.Client
if driver == driverRedis {
client, err = openFromConfig(context.Background(), cfg, CacheDriverKey)
if err != nil {
return nil, err
}
}
c, err := buildCache(cfg, driver, prefix, client)
if err != nil {
if client != nil {
_ = client.Close()
}
return nil, err
}
setLogger(c, loggerFromApp(app))
if err := app.Publish[Cache](c); err != nil {
if client != nil {
_ = client.Close()
}
if winner, ok := app.Lookup[Cache](); ok && winner != nil {
return winner, nil
}
return nil, fmt.Errorf("cooler: %w", err)
}
return c, nil
}
// buildCache builds the selected driver. client is used by the Redis driver.
func buildCache(cfg *compass.Config, driver, prefix string, client *redis.Client) (Cache, error) {
if driver == driverRedis {
if client == nil {
return nil, errNilClient
}
return NewRedisCache(client, prefix), nil
}
n, err := maxEntries(cfg)
if err != nil {
return nil, err
}
return NewMemoryCache(prefix, n)
}
// maxEntries reads cache.memory.max_entries: unset means DefaultMaxEntries,
// anything below 1 (or not a number) is an error naming the key.
func maxEntries(cfg *compass.Config) (int, error) {
if cfg == nil || !cfg.Has(CacheMaxEntriesKey) {
return DefaultMaxEntries, nil
}
n := cfg.Int(CacheMaxEntriesKey)
if n < 1 {
return 0, fmt.Errorf("cooler: %s must be a whole number of at least 1", CacheMaxEntriesKey)
}
return n, nil
}
// openFromConfig builds the Redis client from redis.url, naming the key that
// selected Redis when the URL is missing.
func openFromConfig(ctx context.Context, cfg *compass.Config, selectedBy string) (*redis.Client, error) {
raw := ""
if cfg != nil {
raw = strings.TrimSpace(cfg.String(RedisURLKey))
}
if raw == "" {
return nil, fmt.Errorf("cooler: %s is required when %s is %s (set it in config/redis.yaml or SUMMER_REDIS__URL)", RedisURLKey, selectedBy, driverRedis)
}
return OpenRedis(ctx, raw)
}
// ping checks Redis once within PingTimeout and logs a warning on failure.
func ping(ctx context.Context, client *redis.Client, log *slog.Logger) {
pctx, cancel := context.WithTimeout(ctx, PingTimeout)
defer cancel()
if err := client.Ping(pctx).Err(); err != nil {
log.Warn("cooler: redis did not answer at start-up; rate limits count per instance and the cache degrades until it does",
"endpoint", redisEndpoint(client), "error", err)
}
}
// driverName reads a memory|redis selector. Empty means memory.
func driverName(cfg *compass.Config, key string) (string, error) {
if cfg == nil {
return driverMemory, nil
}
v := strings.ToLower(strings.TrimSpace(cfg.String(key)))
switch v {
case "", driverMemory:
return driverMemory, nil
case driverRedis:
return driverRedis, nil
}
return "", fmt.Errorf("cooler: %s must be %s or %s, got %q", key, driverMemory, driverRedis, v)
}
// keyPrefix returns the key prefix every cooler key starts with: cache.prefix
// when set, otherwise app.name lower-cased with every run of characters
// outside [a-z0-9] replaced by "_" (trimmed at both ends) and a trailing ":",
// or "summer:" when app.name is empty too. Cache keys are <prefix>cache:<key>,
// rate-limit keys <prefix>rl:<key>.
func keyPrefix(cfg *compass.Config) string {
if cfg == nil {
return defaultPrefix
}
if p := strings.TrimSpace(cfg.String(CachePrefixKey)); p != "" {
return p
}
name := slug(cfg.String("app.name"))
if name == "" {
return defaultPrefix
}
return name + ":"
}
func slug(s string) string {
var b strings.Builder
under := false
for _, r := range strings.ToLower(s) {
if (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') {
b.WriteRune(r)
under = false
continue
}
if !under {
b.WriteByte('_')
under = true
}
}
return strings.Trim(b.String(), "_")
}
func loggerFromApp(app *backpack.App) *slog.Logger {
if app != nil {
if log, ok := app.Lookup[*slog.Logger](); ok && log != nil {
return log
}
}
return slog.Default()
}
// errNilClient is returned when a Redis driver is built without a client.
var errNilClient = errors.New("cooler: redis client is nil")
// warnEvery bounds how often a degraded call is logged.
const warnEvery = time.Minute
// warner logs a message at most once per warnEvery.
type warner struct {
mu sync.Mutex
last time.Time
log *slog.Logger
}
func newWarner(log *slog.Logger) *warner { return &warner{log: log} }
func (w *warner) setLogger(log *slog.Logger) {
if w == nil || log == nil {
return
}
w.mu.Lock()
w.log = log
w.mu.Unlock()
}
func (w *warner) warn(msg string, args ...any) {
if w == nil {
return
}
w.mu.Lock()
now := time.Now()
if !w.last.IsZero() && now.Sub(w.last) < warnEvery {
w.mu.Unlock()
return
}
w.last = now
log := w.log
w.mu.Unlock()
if log == nil {
log = slog.Default()
}
log.Warn(msg, args...)
}
// setLogger hands the app's logger to a driver built by this package.
func setLogger(c Cache, log *slog.Logger) {
if s, ok := c.(interface{ setLogger(*slog.Logger) }); ok {
s.setLogger(log)
}
}
// remember is Cache.Remember for every driver: read, else compute and store,
// degrading on cache errors instead of failing the caller.
func remember(ctx context.Context, c Cache, w *warner, key string, ttl time.Duration, fn func(context.Context) ([]byte, error)) ([]byte, error) {
if fn == nil {
return nil, fmt.Errorf("cooler: Remember %q: callback is nil", key)
}
v, ok, err := c.Get(ctx, key)
if err == nil && ok {
return v, nil
}
if err != nil {
w.warn("cooler: cache read failed; computing the value without the cache", "error", err)
}
v, ferr := fn(ctx)
if ferr != nil {
return nil, ferr
}
if err == nil {
if serr := c.Set(ctx, key, v, ttl); serr != nil {
w.warn("cooler: cache write failed; the value was not stored", "error", serr)
}
}
return v, nil
}

View File

@@ -0,0 +1,133 @@
package cooler_test
import (
"context"
"fmt"
"strconv"
"testing"
"time"
"git.golem15.com/golem15/summercms/modules/cooler"
"github.com/redis/go-redis/v9"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/wait"
)
// realRedis starts redis:7-alpine for the test and returns a client from
// OpenRedis. It skips under -short.
func realRedis(t *testing.T) *redis.Client {
t.Helper()
if testing.Short() {
t.Skip("requires testcontainers redis")
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
t.Cleanup(cancel)
ctr, err := testcontainers.Run(ctx, "redis:7-alpine",
testcontainers.WithExposedPorts("6379/tcp"),
testcontainers.WithWaitStrategy(wait.ForListeningPort("6379/tcp")),
)
if err != nil {
t.Fatalf("redis: testcontainers: %v", err)
}
t.Cleanup(func() { _ = testcontainers.TerminateContainer(ctr) })
host, err := ctr.Host(ctx)
if err != nil {
t.Fatal(err)
}
port, err := ctr.MappedPort(ctx, "6379/tcp")
if err != nil {
t.Fatal(err)
}
client, err := cooler.OpenRedis(ctx, fmt.Sprintf("redis://%s:%s/0", host, port.Port()))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = client.Close() })
if err := client.Ping(ctx).Err(); err != nil {
t.Fatalf("redis ping: %v", err)
}
return client
}
func TestRateLimitParityDocker(t *testing.T) {
client := realRedis(t)
store := cooler.NewRateLimitStore(client, "acme:", nil)
runParity(t, parityStore{name: "redis", store: store, advance: sleepFor})
if got := countAdmitted(cooler.NewRateLimitStore(client, "burst:", nil), 50, 10); got != 10 {
t.Fatalf("real Redis admitted %d of 50, want 10", got)
}
ctx := t.Context()
if v, err := client.Get(ctx, "burst:rl:burst").Result(); err != nil || v != "10" {
t.Fatalf("burst counter = %q %v, want 10", v, err)
}
if ttl, err := client.PTTL(ctx, "burst:rl:burst").Result(); err != nil || ttl <= 0 || ttl > time.Minute {
t.Fatalf("burst window TTL = %s %v", ttl, err)
}
}
func TestRedisCacheDocker(t *testing.T) {
client := realRedis(t)
ctx := t.Context()
c := cooler.NewRedisCache(client, "acme:")
if err := c.Set(ctx, "greeting", []byte("hello"), 150*time.Millisecond); err != nil {
t.Fatal(err)
}
if v, ok, err := c.Get(ctx, "greeting"); err != nil || !ok || string(v) != "hello" {
t.Fatalf("Get = %q %v %v", v, ok, err)
}
if ttl, _ := client.PTTL(ctx, "acme:cache:greeting").Result(); ttl <= 0 || ttl > 150*time.Millisecond {
t.Fatalf("PTTL = %s, want (0, 150ms]", ttl)
}
time.Sleep(250 * time.Millisecond)
if ok, _ := c.Has(ctx, "greeting"); ok {
t.Fatal("the key outlived its TTL")
}
if err := c.Set(ctx, "views", []byte("1"), time.Minute); err != nil {
t.Fatal(err)
}
if n, err := c.Increment(ctx, "views", 2); err != nil || n != 3 {
t.Fatalf("Increment = %d %v", n, err)
}
if ttl, _ := client.PTTL(ctx, "acme:cache:views").Result(); ttl <= 50*time.Second {
t.Fatalf("Increment dropped the TTL: %s", ttl)
}
calls := 0
for range 2 {
v, err := c.Remember(ctx, "remembered", time.Minute, func(context.Context) ([]byte, error) { calls++; return []byte("computed"), nil })
if err != nil || string(v) != "computed" {
t.Fatalf("Remember = %q %v", v, err)
}
}
if calls != 1 {
t.Fatalf("Remember called the callback %d times", calls)
}
// Flush walks SCAN in batches: more keys than one batch, and the keys
// outside <prefix>cache: survive.
pipe := client.Pipeline()
for i := range 1200 {
pipe.Set(ctx, "acme:cache:bulk"+strconv.Itoa(i), "v", 0)
}
survivors := []string{"acme:rl:login|192.0.2.1", "other:cache:greeting", "acme:cachex"}
for _, k := range survivors {
pipe.Set(ctx, k, "keep", 0)
}
if _, err := pipe.Exec(ctx); err != nil {
t.Fatal(err)
}
if err := c.Flush(ctx); err != nil {
t.Fatal(err)
}
left, err := client.Keys(ctx, "acme:cache:*").Result()
if err != nil || len(left) != 0 {
t.Fatalf("Flush left %d cache keys (%v)", len(left), err)
}
for _, k := range survivors {
if n, err := client.Exists(ctx, k).Result(); err != nil || n != 1 {
t.Fatalf("Flush removed %q", k)
}
}
}

View File

@@ -0,0 +1,129 @@
package cooler_test
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/cabana"
"git.golem15.com/golem15/summercms/modules/compass"
"git.golem15.com/golem15/summercms/modules/cooler"
"git.golem15.com/golem15/summercms/modules/pact"
)
func ExampleFromApp() {
// serve publishes the Cache at start-up; a job or console command asks
// FromApp, which builds the configured driver (memory here) once.
app := backpack.New(nil)
cache, err := cooler.FromApp(app)
if err != nil {
fmt.Println(err)
return
}
ctx := context.Background()
loads := 0
popular := func(context.Context) ([]byte, error) {
loads++
return json.Marshal([]string{"Hello", "Summer"})
}
var titles []string
for range 2 {
raw, err := cache.Remember(ctx, "blog:popular", 10*time.Minute, popular)
if err != nil {
fmt.Println(err)
return
}
if err := json.Unmarshal(raw, &titles); err != nil {
fmt.Println(err)
return
}
}
fmt.Println(titles, loads)
// Output: [Hello Summer] 1
}
// Article is a blog's article model.
type Article struct {
Slug string
}
// articlesController is a blog's admin controller (its pact.AdminController
// methods are left out here).
type articlesController struct {
app func() *backpack.App
}
// purge queues one site-cache purge for the saved article once the save
// transaction commits (at once outside one); the save never waits for the
// site. Without a purge configured the purger is nil and nothing is sent.
func (c articlesController) purge(ctx context.Context, model any) error {
article, ok := model.(*Article)
if !ok {
return nil
}
tx, _ := cabana.TxFromContext(ctx)
cooler.PurgeSiteCacheAfterCommit(ctx, tx, cooler.SiteCacheFromApp(c.app()), "article", article.Slug)
return nil
}
// FormAfterCreate runs inside the create's transaction.
func (c articlesController) FormAfterCreate(ctx context.Context, model any) error {
return c.purge(ctx, model)
}
// FormAfterUpdate runs inside the update's transaction.
func (c articlesController) FormAfterUpdate(ctx context.Context, model any) error {
return c.purge(ctx, model)
}
// FormAfterDelete runs inside the delete's transaction.
func (c articlesController) FormAfterDelete(ctx context.Context, model any) error {
return c.purge(ctx, model)
}
var (
_ pact.FormAfterCreate = articlesController{}
_ pact.FormAfterUpdate = articlesController{}
_ pact.FormAfterDelete = articlesController{}
)
func ExampleSiteCacheFromApp() {
// The host site's purge route, standing in on loopback.
site := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
fmt.Println(r.Method, r.URL.Path, string(body))
_, _ = w.Write([]byte(`{"purged":3}`))
}))
defer site.Close()
dir, err := os.MkdirTemp("", "cooler-example")
if err != nil {
fmt.Println(err)
return
}
defer os.RemoveAll(dir)
cfg, err := compass.Open(compass.Options{Dir: dir, Environ: []string{
"SUMMER_SITE_CACHE__PURGE_URL=" + site.URL + "/api/__site__/cache/purge",
"SUMMER_SITE_CACHE__PURGE_TOKEN=example-purge-token-0123456789abcdef",
}})
if err != nil {
fmt.Println(err)
return
}
app := backpack.New(cfg)
c := articlesController{app: func() *backpack.App { return app }}
if err := c.FormAfterUpdate(context.Background(), &Article{Slug: "hello-summer"}); err != nil {
fmt.Println(err)
}
// The purge is queued; Wait drains the queue (serve does it at shutdown).
if err := cooler.SiteCacheFromApp(app).Wait(context.Background()); err != nil {
fmt.Println(err)
}
// Output: POST /api/__site__/cache/purge {"kind":"article","slug":"hello-summer"}
}

View File

@@ -0,0 +1,304 @@
package cooler_test
import (
"context"
"log/slog"
"strings"
"sync"
"testing"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/cooler"
"git.golem15.com/golem15/summercms/modules/surf"
"github.com/alicebob/miniredis/v2"
)
// appWithLogger returns an app on cfg whose published logger writes to the
// returned buffer.
func appWithLogger(t *testing.T, cfg map[string]any) (*backpack.App, *lockedBuffer) {
t.Helper()
c := emptyConfig(t)
for k, v := range cfg {
mustSet(t, c, k, v)
}
app := backpack.New(c)
buf := &lockedBuffer{}
if err := app.Publish(slog.New(slog.NewTextHandler(buf, nil))); err != nil {
t.Fatal(err)
}
return app, buf
}
func TestInstallConfigErrors(t *testing.T) {
const secretURL = "redis://alice:s3cretpw@cache.example.test:6379/0\x7f"
cases := []struct {
name string
cfg map[string]any
names []string // substrings the error must contain
}{
{"unknown cache driver", map[string]any{cooler.CacheDriverKey: "memcached"}, []string{cooler.CacheDriverKey, "memcached"}},
{"unknown rate-limit store", map[string]any{cooler.RateLimitStoreKey: "database"}, []string{cooler.RateLimitStoreKey, "database"}},
{"redis store without redis.url", map[string]any{cooler.RateLimitStoreKey: "redis"}, []string{cooler.RedisURLKey, cooler.RateLimitStoreKey}},
{"redis cache without redis.url", map[string]any{cooler.CacheDriverKey: "redis"}, []string{cooler.RedisURLKey, cooler.CacheDriverKey}},
{"a malformed redis.url", map[string]any{cooler.RateLimitStoreKey: "redis", cooler.RedisURLKey: secretURL}, []string{cooler.RedisURLKey}},
{"an unsupported scheme", map[string]any{cooler.CacheDriverKey: "redis", cooler.RedisURLKey: "http://cache.example.test"}, []string{cooler.RedisURLKey}},
{"max_entries 0", map[string]any{cooler.CacheMaxEntriesKey: 0}, []string{cooler.CacheMaxEntriesKey}},
{"max_entries not a number", map[string]any{cooler.CacheMaxEntriesKey: "lots"}, []string{cooler.CacheMaxEntriesKey}},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
app, _ := appWithLogger(t, tc.cfg)
h, err := cooler.Install(context.Background(), app)
if err == nil {
_ = h.Close()
t.Fatal("Install succeeded, want a boot error")
}
for _, want := range tc.names {
if !strings.Contains(err.Error(), want) {
t.Fatalf("error %q does not name %q", err, want)
}
}
if strings.Contains(err.Error(), "alice") || strings.Contains(err.Error(), "s3cretpw") {
t.Fatalf("error %q leaks the URL's credentials", err)
}
if c, ok := app.Lookup[cooler.Cache](); ok && c != nil {
t.Fatal("a failed Install published a Cache")
}
})
}
if _, err := cooler.Install(context.Background(), nil); err == nil {
t.Fatal("Install(nil app) must fail")
}
if _, err := cooler.FromApp(nil); err == nil {
t.Fatal("FromApp(nil app) must fail")
}
app, _ := appWithLogger(t, map[string]any{cooler.CacheDriverKey: "file"})
if _, err := cooler.FromApp(app); err == nil || !strings.Contains(err.Error(), cooler.CacheDriverKey) {
t.Fatalf("FromApp with an unknown driver = %v", err)
}
}
func TestInstallPingWarning(t *testing.T) {
addr := deadAddr(t)
app, logs := appWithLogger(t, map[string]any{
cooler.RateLimitStoreKey: "redis",
cooler.RedisURLKey: "redis://alice:s3cretpw@" + addr + "/0",
})
start := time.Now()
h, err := cooler.Install(context.Background(), app)
if err != nil {
t.Fatalf("an unreachable Redis must not fail Install: %v", err)
}
t.Cleanup(func() { _ = h.Close() })
if elapsed := time.Since(start); elapsed > cooler.PingTimeout+time.Second {
t.Fatalf("Install took %s, want at most about %s", elapsed, cooler.PingTimeout)
}
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], "endpoint="+addr) {
t.Fatalf("log = %q, want one warning naming the endpoint", lines)
}
if strings.Contains(lines[0], "alice") || strings.Contains(lines[0], "s3cretpw") || strings.Contains(lines[0], "redis://") {
t.Fatalf("warning leaks the URL: %s", lines[0])
}
if !h.RateLimitRedis() {
t.Fatal("the Redis store must stay selected so it recovers when Redis answers")
}
// The cache stays the memory default.
if v, err := h.Cache().Remember(context.Background(), "k", time.Minute, func(context.Context) ([]byte, error) { return []byte("v"), nil }); err != nil || string(v) != "v" {
t.Fatalf("Remember = %q %v", v, err)
}
if err := h.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
if err := h.Close(); err != nil {
t.Fatalf("a second Close: %v", err)
}
if h.RateLimitRedis() {
t.Fatal("RateLimitRedis after Close")
}
var nilHandle *cooler.Handle
if nilHandle.Close() != nil || nilHandle.Cache() != nil || nilHandle.RateLimitRedis() {
t.Fatal("a nil Handle must be inert")
}
}
func TestInstallReusesFromApp(t *testing.T) {
t.Run("memory: FromApp's Cache survives Install", func(t *testing.T) {
app, _ := appWithLogger(t, nil)
first, err := cooler.FromApp(app)
if err != nil {
t.Fatal(err)
}
h, err := cooler.Install(context.Background(), app)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = h.Close() })
if h.Cache() != first {
t.Fatal("Install replaced the Cache FromApp published")
}
if again, _ := cooler.FromApp(app); again != first {
t.Fatal("FromApp built a second Cache")
}
// With the memory store, RateLimitStore hands back the fallback itself.
fallback := surf.NewMemoryStore(0)
if h.RateLimitRedis() || h.RateLimitStore(fallback) != cooler.Attempter(fallback) {
t.Fatal("the memory store must return the fallback unchanged")
}
})
t.Run("redis: Install publishes one Redis cache that FromApp returns", func(t *testing.T) {
mr := miniredis.RunT(t)
app, _ := appWithLogger(t, map[string]any{
cooler.CacheDriverKey: "redis",
cooler.CachePrefixKey: "acme:",
cooler.RedisURLKey: "redis://" + mr.Addr() + "/0",
})
h, err := cooler.Install(context.Background(), app)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = h.Close() })
c, err := cooler.FromApp(app)
if err != nil || c != h.Cache() {
t.Fatalf("FromApp = %v %v, want Install's Cache", c, err)
}
if err := c.Set(context.Background(), "k", []byte("v"), 0); err != nil {
t.Fatal(err)
}
if !mr.Exists("acme:cache:k") {
t.Fatalf("keys = %q, want acme:cache:k", mr.Keys())
}
})
t.Run("concurrent FromApp calls share one Cache", func(t *testing.T) {
app, _ := appWithLogger(t, nil)
var wg sync.WaitGroup
got := make([]cooler.Cache, 20)
for i := range got {
wg.Add(1)
go func() {
defer wg.Done()
got[i], _ = cooler.FromApp(app)
}()
}
wg.Wait()
for i, c := range got {
if c == nil || c != got[0] {
t.Fatalf("call %d returned %v, want the shared Cache", i, c)
}
}
})
}
func TestFromAppLazy(t *testing.T) {
mr := miniredis.RunT(t)
app, _ := appWithLogger(t, map[string]any{
cooler.CacheDriverKey: "redis",
cooler.RedisURLKey: "redis://" + mr.Addr() + "/0",
})
c, err := cooler.FromApp(app)
if err != nil {
t.Fatal(err)
}
if n := mr.TotalConnectionCount(); n != 0 {
t.Fatalf("FromApp opened %d connections, want none before the first command", n)
}
if err := c.Set(context.Background(), "k", []byte("v"), 0); err != nil {
t.Fatal(err)
}
if n := mr.TotalConnectionCount(); n == 0 {
t.Fatal("the first command did not dial")
}
// A Redis that does not exist still builds at once.
dead, _ := appWithLogger(t, map[string]any{
cooler.CacheDriverKey: "redis",
cooler.RedisURLKey: "redis://" + deadAddr(t) + "/0",
})
start := time.Now()
if _, err := cooler.FromApp(dead); err != nil {
t.Fatalf("FromApp with an unreachable Redis: %v", err)
}
if elapsed := time.Since(start); elapsed > 100*time.Millisecond {
t.Fatalf("FromApp took %s; it must not dial", elapsed)
}
}
func TestInstallSiteCacheValidation(t *testing.T) {
const token = "install-site-cache-token-0123456789abcd"
t.Run("a bad site_cache configuration is a boot error", func(t *testing.T) {
for _, cfg := range []map[string]any{
{cooler.SiteCachePurgeURLKey: "http://127.0.0.1:3000/purge"},
{cooler.SiteCachePurgeTokenKey: token},
{cooler.SiteCachePurgeURLKey: "http://example.test/purge", cooler.SiteCachePurgeTokenKey: token},
{cooler.SiteCachePurgeURLKey: "http://127.0.0.1/purge", cooler.SiteCachePurgeTokenKey: "short"},
} {
app, _ := appWithLogger(t, cfg)
h, err := cooler.Install(context.Background(), app)
if err == nil {
_ = h.Close()
t.Fatalf("Install(%v) succeeded, want a boot error", cfg)
}
if !strings.Contains(err.Error(), "site_cache.") || strings.Contains(err.Error(), token) {
t.Fatalf("error %q must name a site_cache key and never the token", err)
}
if p, ok := app.Lookup[*cooler.SiteCachePurger](); ok && p != nil {
t.Fatal("a failed Install published a purger")
}
}
})
t.Run("a valid configuration is published and logged by endpoint", func(t *testing.T) {
app, logs := appWithLogger(t, map[string]any{
cooler.SiteCachePurgeURLKey: "http://127.0.0.1:3000/api/__site__/cache/purge",
cooler.SiteCachePurgeTokenKey: token,
})
h, err := cooler.Install(context.Background(), app)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = h.Close() })
p := cooler.SiteCacheFromApp(app)
if p == nil || p.Endpoint() != "127.0.0.1:3000/api/__site__/cache/purge" {
t.Fatalf("published purger = %v", p)
}
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "site cache purge enabled") || strings.Contains(lines[0], token) {
t.Fatalf("log = %q", lines)
}
})
t.Run("a published alias purger is reused, even with bad framework keys", func(t *testing.T) {
app, _ := appWithLogger(t, map[string]any{cooler.SiteCachePurgeURLKey: "ftp://nowhere"})
alias, err := cooler.NewSiteCachePurgerFor("acme.purge_url", "acme.purge_token", "http://127.0.0.1:4000/purge", token)
if err != nil {
t.Fatal(err)
}
if err := app.Publish(alias); err != nil {
t.Fatal(err)
}
h, err := cooler.Install(context.Background(), app)
if err != nil {
t.Fatalf("Install with a published alias purger: %v", err)
}
t.Cleanup(func() { _ = h.Close() })
if got := cooler.SiteCacheFromApp(app); got != alias {
t.Fatal("Install replaced the alias purger")
}
})
t.Run("no keys: no purger", func(t *testing.T) {
app, _ := appWithLogger(t, nil)
h, err := cooler.Install(context.Background(), app)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = h.Close() })
if p := cooler.SiteCacheFromApp(app); p != nil {
t.Fatalf("SiteCacheFromApp = %v, want nil", p)
}
})
}

154
modules/cooler/memory.go Normal file
View File

@@ -0,0 +1,154 @@
package cooler
import (
"context"
"fmt"
"log/slog"
"strconv"
"time"
"github.com/maypok86/otter/v2"
)
// forever is how long an entry without a TTL lives in the memory driver:
// otter's documented way to say "no expiration" is an excessively long period.
const forever = 50 * 365 * 24 * time.Hour
// memEntry is a stored value with its absolute expiry; a zero expiresAt
// never expires.
type memEntry struct {
val []byte
expiresAt time.Time
}
func (e memEntry) expired(now time.Time) bool {
return !e.expiresAt.IsZero() && !now.Before(e.expiresAt)
}
// memoryCache is the in-process Cache on otter v2: size-bounded, with a
// per-entry expiry computed from the absolute expiresAt stored with the value.
type memoryCache struct {
c *otter.Cache[string, memEntry]
prefix string
warn *warner
}
// NewMemoryCache returns an in-process Cache holding at most maxEntries
// entries (cache.memory.max_entries, default DefaultMaxEntries); the least
// valuable entries are evicted beyond that. Keys are <prefix>cache:<key>.
// A maxEntries below 1 is an error naming cache.memory.max_entries.
func NewMemoryCache(prefix string, maxEntries int) (Cache, error) {
if maxEntries < 1 {
return nil, fmt.Errorf("cooler: %s must be at least 1, got %d", CacheMaxEntriesKey, maxEntries)
}
c, err := otter.New(&otter.Options[string, memEntry]{
MaximumSize: maxEntries,
ExpiryCalculator: otter.ExpiryWritingFunc(func(e otter.Entry[string, memEntry]) time.Duration {
if e.Value.expiresAt.IsZero() {
return forever
}
if d := time.Until(e.Value.expiresAt); d > 0 {
return d
}
return time.Nanosecond
}),
})
if err != nil {
return nil, fmt.Errorf("cooler: memory cache: %w", err)
}
return &memoryCache{c: c, prefix: prefix, warn: newWarner(slog.Default())}, nil
}
func (m *memoryCache) key(k string) string { return m.prefix + "cache:" + k }
func (m *memoryCache) setLogger(log *slog.Logger) { m.warn.setLogger(log) }
// lookup returns the live entry for a full key, dropping an expired one.
func (m *memoryCache) lookup(full string) (memEntry, bool) {
e, ok := m.c.GetIfPresent(full)
if !ok {
return memEntry{}, false
}
if e.expired(time.Now()) {
m.c.Invalidate(full)
return memEntry{}, false
}
return e, true
}
func (m *memoryCache) Get(_ context.Context, key string) ([]byte, bool, error) {
e, ok := m.lookup(m.key(key))
if !ok {
return nil, false, nil
}
return clone(e.val), true, nil
}
func (m *memoryCache) Set(_ context.Context, key string, value []byte, ttl time.Duration) error {
e := memEntry{val: clone(value)}
if ttl > 0 {
e.expiresAt = time.Now().Add(ttl)
}
m.c.Set(m.key(key), e)
return nil
}
func (m *memoryCache) Has(_ context.Context, key string) (bool, error) {
_, ok := m.lookup(m.key(key))
return ok, nil
}
func (m *memoryCache) Forget(_ context.Context, key string) error {
m.c.Invalidate(m.key(key))
return nil
}
func (m *memoryCache) Remember(ctx context.Context, key string, ttl time.Duration, fn func(context.Context) ([]byte, error)) ([]byte, error) {
return remember(ctx, m, m.warn, key, ttl, fn)
}
// Increment adds by to the integer at key atomically (the update runs under
// otter's per-key lock). A missing or expired key starts at 0 and gets no
// TTL; an existing key keeps its expiry, as Redis INCRBY does.
func (m *memoryCache) Increment(_ context.Context, key string, by int64) (int64, error) {
var (
n int64
valErr error
)
now := time.Now()
m.c.Compute(m.key(key), func(old memEntry, found bool) (memEntry, otter.ComputeOp) {
cur := int64(0)
next := memEntry{}
if found && !old.expired(now) {
v, err := strconv.ParseInt(string(old.val), 10, 64)
if err != nil {
valErr = fmt.Errorf("cooler: cache value at %q is not an integer", key)
return old, otter.CancelOp
}
cur = v
next.expiresAt = old.expiresAt
}
n = cur + by
next.val = []byte(strconv.FormatInt(n, 10))
return next, otter.WriteOp
})
if valErr != nil {
return 0, valErr
}
return n, nil
}
// Flush removes every entry of this cache.
func (m *memoryCache) Flush(context.Context) error {
m.c.InvalidateAll()
return nil
}
func clone(b []byte) []byte {
if b == nil {
return nil
}
out := make([]byte, len(b))
copy(out, b)
return out
}

View File

@@ -0,0 +1,336 @@
package cooler
import (
"context"
"errors"
"log/slog"
"strconv"
"strings"
"sync"
"testing"
"time"
)
func newMemory(t *testing.T, prefix string, maxEntries int) *memoryCache {
t.Helper()
c, err := NewMemoryCache(prefix, maxEntries)
if err != nil {
t.Fatal(err)
}
m, ok := c.(*memoryCache)
if !ok {
t.Fatalf("NewMemoryCache returned %T", c)
}
return m
}
func TestMemoryCache(t *testing.T) {
ctx := t.Context()
m := newMemory(t, "acme:", DefaultMaxEntries)
if v, ok, err := m.Get(ctx, "missing"); v != nil || ok || err != nil {
t.Fatalf("Get(missing) = %q %v %v", v, ok, err)
}
if ok, err := m.Has(ctx, "missing"); ok || err != nil {
t.Fatalf("Has(missing) = %v %v", ok, err)
}
if err := m.Forget(ctx, "missing"); err != nil {
t.Fatalf("Forget(missing) = %v", err)
}
// Keys live under <prefix>cache:<key>.
if err := m.Set(ctx, "greeting", []byte("hello"), time.Minute); err != nil {
t.Fatal(err)
}
if _, ok := m.c.GetIfPresent("acme:cache:greeting"); !ok {
t.Fatal("Set did not store acme:cache:greeting")
}
if v, ok, err := m.Get(ctx, "greeting"); err != nil || !ok || string(v) != "hello" {
t.Fatalf("Get = %q %v %v", v, ok, err)
}
// ttl 0 and a negative ttl keep the value without expiry.
for _, ttl := range []time.Duration{0, -time.Second} {
key := "forever" + ttl.String()
if err := m.Set(ctx, key, []byte("x"), ttl); err != nil {
t.Fatal(err)
}
e, ok := m.c.GetIfPresent(m.key(key))
if !ok || !e.expiresAt.IsZero() {
t.Fatalf("ttl %s: entry %+v found=%v, want no expiry", ttl, e, ok)
}
}
// A nil value is stored and read back as found.
if err := m.Set(ctx, "nil", nil, 0); err != nil {
t.Fatal(err)
}
if _, ok, err := m.Get(ctx, "nil"); !ok || err != nil {
t.Fatalf("Get(nil value) found=%v err=%v", ok, err)
}
// Copy-on-set and copy-on-get.
in := []byte("abc")
_ = m.Set(ctx, "copy", in, 0)
in[0] = 'X'
out, _, _ := m.Get(ctx, "copy")
out[1] = 'Y'
if again, _, _ := m.Get(ctx, "copy"); string(again) != "abc" {
t.Fatalf("stored value mutated: %q", again)
}
// Increment: missing starts at 0, existing integer grows, a non-integer fails.
if n, err := m.Increment(ctx, "hits", 5); err != nil || n != 5 {
t.Fatalf("Increment(missing) = %d %v", n, err)
}
if n, err := m.Increment(ctx, "hits", -2); err != nil || n != 3 {
t.Fatalf("Increment(-2) = %d %v", n, err)
}
if n, err := m.Increment(ctx, "greeting", 1); err == nil || n != 0 || !strings.Contains(err.Error(), "greeting") {
t.Fatalf("Increment(non-integer) = %d %v, want an error naming the key", n, err)
}
if v, _, _ := m.Get(ctx, "greeting"); string(v) != "hello" {
t.Fatalf("a failed Increment changed the value: %q", v)
}
// Remember stores the computed value; an error from fn is returned.
calls := 0
fn := func(context.Context) ([]byte, error) { calls++; return []byte("computed"), nil }
for range 3 {
if v, err := m.Remember(ctx, "remembered", time.Minute, fn); err != nil || string(v) != "computed" {
t.Fatalf("Remember = %q %v", v, err)
}
}
if calls != 1 {
t.Fatalf("callback ran %d times, want 1", calls)
}
boom := errors.New("boom")
if v, err := m.Remember(ctx, "failing", time.Minute, func(context.Context) ([]byte, error) { return nil, boom }); v != nil || !errors.Is(err, boom) {
t.Fatalf("Remember(failing fn) = %q %v", v, err)
}
if ok, _ := m.Has(ctx, "failing"); ok {
t.Fatal("a failing callback stored a value")
}
if _, err := m.Remember(ctx, "nilfn", time.Minute, nil); err == nil {
t.Fatal("Remember with a nil callback must fail")
}
// Forget and Flush.
if err := m.Forget(ctx, "greeting"); err != nil {
t.Fatal(err)
}
if ok, _ := m.Has(ctx, "greeting"); ok {
t.Fatal("Forget left the key")
}
if err := m.Flush(ctx); err != nil {
t.Fatal(err)
}
for _, k := range []string{"hits", "remembered", "copy", "forever0s"} {
if ok, _ := m.Has(ctx, k); ok {
t.Fatalf("Flush left %q", k)
}
}
if _, err := NewMemoryCache("acme:", -1); err == nil || !strings.Contains(err.Error(), CacheMaxEntriesKey) {
t.Fatalf("NewMemoryCache(-1) = %v, want an error naming %s", err, CacheMaxEntriesKey)
}
}
func TestMemoryCacheTTL(t *testing.T) {
ctx := t.Context()
m := newMemory(t, "acme:", DefaultMaxEntries)
if err := m.Set(ctx, "short", []byte("x"), 40*time.Millisecond); err != nil {
t.Fatal(err)
}
if ok, _ := m.Has(ctx, "short"); !ok {
t.Fatal("a fresh 40 ms entry is missing")
}
time.Sleep(80 * time.Millisecond)
if ok, _ := m.Has(ctx, "short"); ok {
t.Fatal("a 40 ms entry outlived 80 ms")
}
if _, ok, _ := m.Get(ctx, "short"); ok {
t.Fatal("Get returned an expired entry")
}
// Increment on an existing key keeps its expiry, as INCRBY does.
if err := m.Set(ctx, "counter", []byte("5"), 60*time.Millisecond); err != nil {
t.Fatal(err)
}
before, _ := m.c.GetIfPresent(m.key("counter"))
if n, err := m.Increment(ctx, "counter", 1); err != nil || n != 6 {
t.Fatalf("Increment = %d %v", n, err)
}
after, _ := m.c.GetIfPresent(m.key("counter"))
if !after.expiresAt.Equal(before.expiresAt) {
t.Fatalf("Increment moved the expiry from %v to %v", before.expiresAt, after.expiresAt)
}
time.Sleep(100 * time.Millisecond)
if ok, _ := m.Has(ctx, "counter"); ok {
t.Fatal("an incremented entry outlived its TTL")
}
// An expired key restarts at 0 without a TTL.
if n, err := m.Increment(ctx, "counter", 2); err != nil || n != 2 {
t.Fatalf("Increment after expiry = %d %v, want 2", n, err)
}
if e, _ := m.c.GetIfPresent(m.key("counter")); !e.expiresAt.IsZero() {
t.Fatalf("a restarted counter has expiry %v, want none", e.expiresAt)
}
}
func TestMemoryCacheBounded(t *testing.T) {
ctx := t.Context()
const limit = 50
m := newMemory(t, "acme:", limit)
for i := range 5000 {
if err := m.Set(ctx, "k"+strconv.Itoa(i), []byte("v"), 0); err != nil {
t.Fatal(err)
}
}
m.c.CleanUp()
if n := m.c.EstimatedSize(); n > limit {
t.Fatalf("cache holds %d entries, bound is %d", n, limit)
}
live := 0
for i := range 5000 {
if ok, _ := m.Has(ctx, "k"+strconv.Itoa(i)); ok {
live++
}
}
if live == 0 || live > limit {
t.Fatalf("%d live keys, want between 1 and %d", live, limit)
}
}
func TestMemoryIncrementConcurrent(t *testing.T) {
ctx := t.Context()
m := newMemory(t, "acme:", DefaultMaxEntries)
if err := m.Set(ctx, "views", []byte("0"), time.Hour); err != nil {
t.Fatal(err)
}
before, _ := m.c.GetIfPresent(m.key("views"))
const workers, each = 100, 10
var wg sync.WaitGroup
start := make(chan struct{})
errs := make(chan error, workers*each)
for range workers {
wg.Add(1)
go func() {
defer wg.Done()
<-start
for range each {
if _, err := m.Increment(ctx, "views", 1); err != nil {
errs <- err
}
}
}()
}
close(start)
wg.Wait()
close(errs)
for err := range errs {
t.Fatal(err)
}
v, ok, err := m.Get(ctx, "views")
if err != nil || !ok || string(v) != strconv.Itoa(workers*each) {
t.Fatalf("views = %q %v %v, want %d", v, ok, err, workers*each)
}
after, _ := m.c.GetIfPresent(m.key("views"))
if !after.expiresAt.Equal(before.expiresAt) {
t.Fatalf("concurrent increments moved the expiry from %v to %v", before.expiresAt, after.expiresAt)
}
}
// failingCache is a Cache whose reads and writes fail as configured.
type failingCache struct {
Cache
getErr, setErr error
sets int
}
func (f *failingCache) Get(context.Context, string) ([]byte, bool, error) {
return nil, false, f.getErr
}
func (f *failingCache) Set(context.Context, string, []byte, time.Duration) error {
f.sets++
return f.setErr
}
func TestRememberDegrades(t *testing.T) {
ctx := t.Context()
value := func(context.Context) ([]byte, error) { return []byte("fresh"), nil }
t.Run("a failing read computes, skips the write and warns once", func(t *testing.T) {
buf := &syncBuffer{}
w := newWarner(slog.New(slog.NewTextHandler(buf, nil)))
f := &failingCache{getErr: errors.New("connection refused")}
for range 3 {
v, err := remember(ctx, f, w, "k", time.Minute, value)
if err != nil || string(v) != "fresh" {
t.Fatalf("remember = %q %v", v, err)
}
}
if f.sets != 0 {
t.Fatalf("a failed read still wrote %d times", f.sets)
}
lines := buf.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], "cache read failed") {
t.Fatalf("log = %q, want one read warning", lines)
}
})
t.Run("a failing write returns the value and warns once", func(t *testing.T) {
buf := &syncBuffer{}
w := newWarner(slog.New(slog.NewTextHandler(buf, nil)))
f := &failingCache{setErr: errors.New("read-only replica")}
for range 3 {
v, err := remember(ctx, f, w, "k", time.Minute, value)
if err != nil || string(v) != "fresh" {
t.Fatalf("remember = %q %v", v, err)
}
}
if f.sets != 3 {
t.Fatalf("%d writes, want 3", f.sets)
}
lines := buf.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "cache write failed") {
t.Fatalf("log = %q, want one write warning", lines)
}
})
t.Run("an error from the callback is returned", func(t *testing.T) {
boom := errors.New("boom")
f := &failingCache{getErr: errors.New("down")}
w := newWarner(slog.New(slog.NewTextHandler(&syncBuffer{}, nil)))
if v, err := remember(ctx, f, w, "k", time.Minute, func(context.Context) ([]byte, error) { return nil, boom }); v != nil || !errors.Is(err, boom) {
t.Fatalf("remember = %q %v", v, err)
}
})
t.Run("a Redis cache whose server is gone still returns the value", func(t *testing.T) {
buf := &syncBuffer{}
client, err := OpenRedis(ctx, "redis://"+closedAddr(t)+"/0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = client.Close() })
c := NewRedisCache(client, "acme:")
setLogger(c, slog.New(slog.NewTextHandler(buf, nil)))
for range 2 {
// The caller's deadline bounds go-redis's dial retries.
cctx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
v, err := c.Remember(cctx, "k", time.Minute, value)
cancel()
if err != nil || string(v) != "fresh" {
t.Fatalf("Remember = %q %v", v, err)
}
}
if lines := buf.lines(); len(lines) != 1 || !strings.Contains(lines[0], "cache read failed") {
t.Fatalf("log = %q, want one read warning", lines)
}
})
}

View File

@@ -0,0 +1,94 @@
package cooler
import (
"context"
"fmt"
"log/slog"
"time"
"github.com/redis/go-redis/v9"
)
// attemptScript is one atomic fixed-window admission with the semantics of
// surf.MemoryStore: a missing or expired key opens a window of decay
// milliseconds, a count at max denies without incrementing, otherwise the
// count is incremented. INCR keeps the TTL, so the window never extends.
// It returns {allowed (0|1), count, remaining window in ms}.
//
// KEYS[1] = <prefix>rl:<key>; ARGV[1] = max; ARGV[2] = decay in ms (>= 1).
var attemptScript = redis.NewScript(`
local max = tonumber(ARGV[1])
local decay = tonumber(ARGV[2])
local count = tonumber(redis.call('GET', KEYS[1]) or '0')
local ttl = redis.call('PTTL', KEYS[1])
if ttl < 0 then
count = 0
redis.call('SET', KEYS[1], 0, 'PX', decay)
ttl = decay
end
if count >= max then
return {0, count, ttl}
end
count = redis.call('INCR', KEYS[1])
return {1, count, ttl}
`)
// RateLimitStore is a fixed-window rate-limit store on Redis, shared by every
// application instance that points at the same Redis. Its Attempt has the
// exact signature of surf.Store, so surf uses it without cooler importing
// surf. When Redis does not answer, Attempt counts in the fallback store
// (per instance) and logs a warning at most once per minute.
type RateLimitStore struct {
client *redis.Client
prefix string
fallback Attempter
warn *warner
}
// NewRateLimitStore returns a store that keeps its counters under
// <prefix>rl:<key> on client. fallback answers while Redis fails; surf passes
// a surf.MemoryStore. A nil fallback admits every request during an outage.
func NewRateLimitStore(client *redis.Client, prefix string, fallback Attempter) *RateLimitStore {
return &RateLimitStore{client: client, prefix: prefix, fallback: fallback, warn: newWarner(slog.Default())}
}
// Attempt admits and counts one request when key is below max, in one
// atomic Redis script bounded by AttemptTimeout. The first attempt opens the
// window, a denied attempt does not increment, and retryAfter is the time
// left in the window.
func (s *RateLimitStore) Attempt(key string, max int, decay time.Duration) (allowed bool, attempts int, retryAfter time.Duration) {
ms := decay.Milliseconds()
if ms < 1 {
ms = 1
}
ctx, cancel := context.WithTimeout(context.Background(), AttemptTimeout)
defer cancel()
res, err := s.run(ctx, s.prefix+"rl:"+key, max, ms)
if err != nil {
s.warn.warn("cooler: redis rate-limit store failed; counting per instance until it answers",
"endpoint", redisEndpoint(s.client), "error", err)
if s.fallback == nil {
return true, 0, 0
}
return s.fallback.Attempt(key, max, decay)
}
ttl := time.Duration(res[2]) * time.Millisecond
if ttl < 0 {
ttl = 0
}
return res[0] == 1, int(res[1]), ttl
}
func (s *RateLimitStore) run(ctx context.Context, key string, max int, ms int64) ([]int64, error) {
if s == nil || s.client == nil {
return nil, fmt.Errorf("cooler: rate-limit store has no Redis client")
}
res, err := attemptScript.Run(ctx, s.client, []string{key}, max, ms).Int64Slice()
if err != nil {
return nil, err
}
if len(res) != 3 {
return nil, fmt.Errorf("cooler: rate-limit script returned %d values, want 3", len(res))
}
return res, nil
}

View File

@@ -0,0 +1,358 @@
package cooler_test
import (
"bytes"
"context"
"fmt"
"log/slog"
"net"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/compass"
"git.golem15.com/golem15/summercms/modules/cooler"
"git.golem15.com/golem15/summercms/modules/surf"
"github.com/alicebob/miniredis/v2"
)
// Both stores must satisfy surf.Store; the compiler checks the Redis one here.
var _ surf.Store = (*cooler.RateLimitStore)(nil)
// parityStore is one rate-limit store under test with the way its clock moves.
type parityStore struct {
name string
store surf.Store
advance func(time.Duration)
}
// parityStep is one Attempt, after the store's clock moved by advance.
type parityStep struct {
advance time.Duration
key string
max int
allowed bool
attempts int
// newWindow: retryAfter is close to the full decay (a window opened).
// Otherwise retryAfter never grows within the window and stays at or
// below decay minus the time advanced since the window opened.
newWindow bool
}
// parityDecay is long enough for real-time stores to cross it in a test and
// short enough to keep the table fast.
const parityDecay = 300 * time.Millisecond
// parityTable is the shared fixed-window behaviour of surf.MemoryStore and
// the Redis store: the first attempt opens the window, a denial does not
// increment, the window never extends, retryAfter never grows and expiry
// opens a new window.
var parityTable = []parityStep{
{key: "login|192.0.2.1", max: 3, allowed: true, attempts: 1, newWindow: true},
{key: "login|192.0.2.1", max: 3, allowed: true, attempts: 2},
{key: "login|192.0.2.1", max: 3, allowed: true, attempts: 3},
{key: "login|192.0.2.1", max: 3, allowed: false, attempts: 3},
{key: "login|192.0.2.1", max: 3, allowed: false, attempts: 3},
// Another key has its own window.
{key: "login|192.0.2.2", max: 3, allowed: true, attempts: 1, newWindow: true},
// A max of 0 denies without counting.
{key: "closed", max: 0, allowed: false, attempts: 0, newWindow: true},
// Inside the window: still denied, the window did not move.
{advance: 120 * time.Millisecond, key: "login|192.0.2.1", max: 3, allowed: false, attempts: 3},
// Past the window: a new one opens at 1.
{advance: 250 * time.Millisecond, key: "login|192.0.2.1", max: 3, allowed: true, attempts: 1, newWindow: true},
{key: "login|192.0.2.1", max: 3, allowed: true, attempts: 2},
}
// runParity drives the table through s and checks every answer.
func runParity(t *testing.T, s parityStore) {
t.Helper()
opened := map[string]time.Duration{} // advanced since the key's window opened
last := map[string]time.Duration{} // last retryAfter per key
for i, step := range parityTable {
if step.advance > 0 {
s.advance(step.advance)
for k := range opened {
opened[k] += step.advance
}
}
allowed, attempts, retry := s.store.Attempt(step.key, step.max, parityDecay)
if allowed != step.allowed || attempts != step.attempts {
t.Fatalf("%s step %d (%s max %d): allowed=%v attempts=%d, want %v %d",
s.name, i, step.key, step.max, allowed, attempts, step.allowed, step.attempts)
}
if retry <= 0 || retry > parityDecay {
t.Fatalf("%s step %d: retryAfter %s outside (0, %s]", s.name, i, retry, parityDecay)
}
if step.newWindow {
if retry < parityDecay-100*time.Millisecond {
t.Fatalf("%s step %d: a new window answered retryAfter %s, want about %s", s.name, i, retry, parityDecay)
}
opened[step.key] = 0
} else {
if prev, ok := last[step.key]; ok && retry > prev {
t.Fatalf("%s step %d: retryAfter grew from %s to %s", s.name, i, prev, retry)
}
if limit := parityDecay - opened[step.key]; retry > limit {
t.Fatalf("%s step %d: retryAfter %s, want at most %s (the window extended)", s.name, i, retry, limit)
}
}
last[step.key] = retry
}
}
func sleepFor(d time.Duration) { time.Sleep(d) }
// miniStore returns a Redis rate-limit store on a fresh miniredis.
func miniStore(t *testing.T, prefix string, fallback cooler.Attempter) (*miniredis.Miniredis, *cooler.RateLimitStore) {
t.Helper()
mr := miniredis.RunT(t)
client, err := cooler.OpenRedis(t.Context(), "redis://"+mr.Addr()+"/0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = client.Close() })
return mr, cooler.NewRateLimitStore(client, prefix, fallback)
}
func TestRateLimitParity(t *testing.T) {
t.Run("surf.MemoryStore", func(t *testing.T) {
mem := surf.NewMemoryStore(0)
runParity(t, parityStore{name: "memory", store: mem, advance: sleepFor})
})
t.Run("Redis store on miniredis", func(t *testing.T) {
mr, store := miniStore(t, "acme:", nil)
runParity(t, parityStore{name: "miniredis", store: store, advance: mr.FastForward})
})
}
// countAdmitted runs workers concurrent attempts on one key and returns how
// many were admitted.
func countAdmitted(s surf.Store, workers, max int) int {
var admitted atomic.Int64
var wg sync.WaitGroup
start := make(chan struct{})
for range workers {
wg.Add(1)
go func() {
defer wg.Done()
<-start
if ok, _, _ := s.Attempt("burst", max, time.Minute); ok {
admitted.Add(1)
}
}()
}
close(start)
wg.Wait()
return int(admitted.Load())
}
func TestRateLimitConcurrentNeverExceedsMax(t *testing.T) {
const workers, max = 50, 10
if got := countAdmitted(surf.NewMemoryStore(0), workers, max); got != max {
t.Fatalf("MemoryStore admitted %d of %d, want %d", got, workers, max)
}
mr, store := miniStore(t, "acme:", nil)
if got := countAdmitted(store, workers, max); got != max {
t.Fatalf("Redis store admitted %d of %d, want %d", got, workers, max)
}
if got, err := mr.Get("acme:rl:burst"); err != nil || got != fmt.Sprint(max) {
t.Fatalf("Redis counter = %q %v, want %d (denials must not increment)", got, err, max)
}
}
// captureDefault routes slog.Default to a buffer for the test. Stores built
// after the call log there.
func captureDefault(t *testing.T) *lockedBuffer {
t.Helper()
buf := &lockedBuffer{}
prev := slog.Default()
slog.SetDefault(slog.New(slog.NewTextHandler(buf, nil)))
t.Cleanup(func() { slog.SetDefault(prev) })
return buf
}
type lockedBuffer struct {
mu sync.Mutex
buf bytes.Buffer
}
func (b *lockedBuffer) Write(p []byte) (int, error) {
b.mu.Lock()
defer b.mu.Unlock()
return b.buf.Write(p)
}
func (b *lockedBuffer) lines() []string {
b.mu.Lock()
defer b.mu.Unlock()
var out []string
for _, l := range strings.Split(strings.TrimSpace(b.buf.String()), "\n") {
if l != "" {
out = append(out, l)
}
}
return out
}
func TestRateLimitFallbackOnError(t *testing.T) {
t.Run("a stopped Redis counts in the fallback and warns once", func(t *testing.T) {
logs := captureDefault(t)
fallback := surf.NewMemoryStore(0)
mr, store := miniStore(t, "acme:", fallback)
if ok, n, _ := store.Attempt("login", 2, time.Minute); !ok || n != 1 {
t.Fatalf("with Redis up: %v %d", ok, n)
}
addr := mr.Addr()
mr.Close()
want := []struct {
allowed bool
attempts int
}{{true, 1}, {true, 2}, {false, 2}}
for i, w := range want {
start := time.Now()
ok, n, retry := store.Attempt("login", 2, time.Minute)
if elapsed := time.Since(start); elapsed > cooler.AttemptTimeout+300*time.Millisecond {
t.Fatalf("attempt %d took %s, want at most about %s", i, elapsed, cooler.AttemptTimeout)
}
if ok != w.allowed || n != w.attempts || retry <= 0 {
t.Fatalf("fallback attempt %d = %v %d %s, want %v %d", i, ok, n, retry, w.allowed, w.attempts)
}
}
// The counts live in the fallback store itself.
if ok, n, _ := fallback.Attempt("login", 2, time.Minute); ok || n != 2 {
t.Fatalf("fallback store = %v %d, want denied at 2", ok, n)
}
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") ||
!strings.Contains(lines[0], "rate-limit store failed") || !strings.Contains(lines[0], "endpoint="+addr) {
t.Fatalf("log = %q, want one warning naming the endpoint", lines)
}
})
t.Run("without a fallback an outage admits", func(t *testing.T) {
captureDefault(t)
mr, store := miniStore(t, "acme:", nil)
mr.Close()
for range 3 {
if ok, n, retry := store.Attempt("login", 1, time.Minute); !ok || n != 0 || retry != 0 {
t.Fatalf("Attempt = %v %d %s, want true 0 0", ok, n, retry)
}
}
})
t.Run("a store without a client uses the fallback", func(t *testing.T) {
captureDefault(t)
store := cooler.NewRateLimitStore(nil, "acme:", surf.NewMemoryStore(0))
if ok, n, _ := store.Attempt("k", 1, time.Minute); !ok || n != 1 {
t.Fatalf("first = %v %d", ok, n)
}
if ok, n, _ := store.Attempt("k", 1, time.Minute); ok || n != 1 {
t.Fatalf("second = %v %d", ok, n)
}
})
t.Run("a script error answers from the fallback", func(t *testing.T) {
captureDefault(t)
mr, store := miniStore(t, "acme:", surf.NewMemoryStore(0))
mr.SetError("LOADING Redis is loading the dataset in memory")
if ok, n, _ := store.Attempt("k", 1, time.Minute); !ok || n != 1 {
t.Fatalf("Attempt = %v %d", ok, n)
}
mr.SetError("")
if ok, n, _ := store.Attempt("k", 1, time.Minute); !ok || n != 1 {
t.Fatalf("after recovery Redis counts again from 1: %v %d", ok, n)
}
})
}
func TestRateLimitKeyPrefix(t *testing.T) {
mr, store := miniStore(t, "acme:", nil)
store.Attempt("login|192.0.2.1", 5, 90*time.Second)
if got, err := mr.Get("acme:rl:login|192.0.2.1"); err != nil || got != "1" {
t.Fatalf("acme:rl:login|192.0.2.1 = %q %v", got, err)
}
if ttl := mr.TTL("acme:rl:login|192.0.2.1"); ttl != 90*time.Second {
t.Fatalf("window TTL = %s, want 90s", ttl)
}
// A sub-millisecond decay is clamped to 1 ms rather than failing.
if ok, _, _ := store.Attempt("tiny", 1, time.Microsecond); !ok {
t.Fatal("a sub-ms decay must still admit")
}
if ttl := mr.TTL("acme:rl:tiny"); ttl != time.Millisecond {
t.Fatalf("sub-ms window TTL = %s, want 1ms", ttl)
}
if keys := mr.Keys(); len(keys) != 2 {
t.Fatalf("keys = %q, want only the two rl keys", keys)
}
// Through Install: the prefix derives from app.name and the store from
// http.rate_limit.store.
cfg := emptyConfig(t)
mustSet(t, cfg, "app.name", "Acme Shop")
mustSet(t, cfg, cooler.RateLimitStoreKey, "redis")
mustSet(t, cfg, cooler.RedisURLKey, "redis://"+mr.Addr()+"/0")
app := backpack.New(cfg)
h, err := cooler.Install(context.Background(), app)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = h.Close() })
if !h.RateLimitRedis() {
t.Fatal("RateLimitRedis = false with http.rate_limit.store redis")
}
fallback := surf.NewMemoryStore(0)
s := h.RateLimitStore(fallback)
if s == cooler.Attempter(fallback) {
t.Fatal("RateLimitStore returned the fallback with the Redis store selected")
}
s.Attempt("login", 1, time.Minute)
if !mr.Exists("acme_shop:rl:login") {
t.Fatalf("keys = %q, want acme_shop:rl:login", mr.Keys())
}
// cache.prefix wins over app.name.
mustSet(t, cfg, cooler.CachePrefixKey, "shop1:")
h2, err := cooler.Install(context.Background(), backpack.New(cfg))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = h2.Close() })
h2.RateLimitStore(fallback).Attempt("login", 1, time.Minute)
if !mr.Exists("shop1:rl:login") {
t.Fatalf("keys = %q, want shop1:rl:login", mr.Keys())
}
}
// emptyConfig returns a config that ignores the process environment.
func emptyConfig(t *testing.T) *compass.Config {
t.Helper()
cfg, err := compass.Open(compass.Options{Dir: t.TempDir(), Environ: []string{}})
if err != nil {
t.Fatal(err)
}
return cfg
}
func mustSet(t *testing.T, cfg *compass.Config, key string, value any) {
t.Helper()
if err := cfg.Set(key, value); err != nil {
t.Fatal(err)
}
}
// deadAddr returns a localhost address nothing listens on.
func deadAddr(t *testing.T) string {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
addr := ln.Addr().String()
_ = ln.Close()
return addr
}

46
modules/cooler/redis.go Normal file
View File

@@ -0,0 +1,46 @@
package cooler
import (
"context"
"errors"
"strings"
"github.com/redis/go-redis/v9"
"github.com/redis/go-redis/v9/maintnotifications"
)
// errInvalidRedisURL is the only error OpenRedis returns for a URL that does
// not parse. It never wraps or quotes the parser's error: url.Parse echoes the
// whole URL, password included.
var errInvalidRedisURL = errors.New("cooler: " + RedisURLKey + " is not a valid redis://, rediss:// or unix:// URL")
// OpenRedis builds a go-redis client from a redis:// (plain), rediss:// (TLS)
// or unix:// URL, such as the value of redis.url. It does not dial: go-redis
// connects on the first command. A URL that does not parse yields an error that
// names redis.url and never contains the URL or its credentials.
//
// The client skips the CLIENT SETINFO and maintenance-notification handshakes,
// which older servers and test doubles do not know.
func OpenRedis(ctx context.Context, rawURL string) (*redis.Client, error) {
_ = ctx // reserved: go-redis does not dial at construction
rawURL = strings.TrimSpace(rawURL)
if rawURL == "" {
return nil, errors.New("cooler: " + RedisURLKey + " is empty")
}
opt, err := redis.ParseURL(rawURL)
if err != nil {
return nil, errInvalidRedisURL
}
opt.DisableIdentity = true
opt.MaintNotificationsConfig = &maintnotifications.Config{Mode: maintnotifications.ModeDisabled}
return redis.NewClient(opt), nil
}
// redisEndpoint is the part of a client's address that may be logged: the
// host and port (or socket path), never the URL, user or password.
func redisEndpoint(c *redis.Client) string {
if c == nil {
return ""
}
return c.Options().Addr
}

View File

@@ -0,0 +1,200 @@
package cooler
import (
"context"
"errors"
"net"
"strconv"
"testing"
"time"
"github.com/alicebob/miniredis/v2"
"github.com/redis/go-redis/v9"
)
// closedAddr returns a localhost address nothing listens on.
func closedAddr(t *testing.T) string {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
addr := ln.Addr().String()
_ = ln.Close()
return addr
}
// miniClient starts a miniredis and returns it with a client from OpenRedis.
func miniClient(t *testing.T) (*miniredis.Miniredis, *redis.Client) {
t.Helper()
mr := miniredis.RunT(t)
client, err := OpenRedis(t.Context(), "redis://"+mr.Addr()+"/0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = client.Close() })
return mr, client
}
func TestRedisCache(t *testing.T) {
ctx := t.Context()
mr, client := miniClient(t)
c := NewRedisCache(client, "acme:")
if v, ok, err := c.Get(ctx, "missing"); v != nil || ok || err != nil {
t.Fatalf("Get(missing) = %q %v %v", v, ok, err)
}
if ok, err := c.Has(ctx, "missing"); ok || err != nil {
t.Fatalf("Has(missing) = %v %v", ok, err)
}
// Keys live under <prefix>cache:<key>; a TTL is stored with PX.
if err := c.Set(ctx, "greeting", []byte("hello"), 10*time.Second); err != nil {
t.Fatal(err)
}
if got, err := mr.Get("acme:cache:greeting"); err != nil || got != "hello" {
t.Fatalf("raw acme:cache:greeting = %q %v", got, err)
}
if ttl := mr.TTL("acme:cache:greeting"); ttl != 10*time.Second {
t.Fatalf("TTL = %s, want 10s", ttl)
}
if v, ok, err := c.Get(ctx, "greeting"); err != nil || !ok || string(v) != "hello" {
t.Fatalf("Get = %q %v %v", v, ok, err)
}
mr.FastForward(9 * time.Second)
if ok, _ := c.Has(ctx, "greeting"); !ok {
t.Fatal("the key expired before its TTL")
}
mr.FastForward(2 * time.Second)
if ok, _ := c.Has(ctx, "greeting"); ok {
t.Fatal("the key outlived its TTL")
}
// A sub-millisecond TTL is rounded up to 1 ms, not stored forever.
if err := c.Set(ctx, "tiny", []byte("x"), time.Microsecond); err != nil {
t.Fatal(err)
}
if ttl := mr.TTL("acme:cache:tiny"); ttl != time.Millisecond {
t.Fatalf("sub-ms TTL stored as %s, want 1ms", ttl)
}
// ttl 0 and a negative ttl store without expiry; a nil value stores empty.
for _, ttl := range []time.Duration{0, -time.Second} {
if err := c.Set(ctx, "forever", nil, ttl); err != nil {
t.Fatal(err)
}
if got := mr.TTL("acme:cache:forever"); got != 0 {
t.Fatalf("ttl %s stored TTL %s, want none", ttl, got)
}
if v, ok, err := c.Get(ctx, "forever"); err != nil || !ok || len(v) != 0 {
t.Fatalf("Get(forever) = %q %v %v", v, ok, err)
}
}
// Increment: INCRBY from missing has no TTL, an existing key keeps its TTL.
if n, err := c.Increment(ctx, "hits", 4); err != nil || n != 4 {
t.Fatalf("Increment(missing) = %d %v", n, err)
}
if ttl := mr.TTL("acme:cache:hits"); ttl != 0 {
t.Fatalf("Increment(missing) set TTL %s", ttl)
}
if err := c.Set(ctx, "views", []byte("10"), time.Minute); err != nil {
t.Fatal(err)
}
if n, err := c.Increment(ctx, "views", 5); err != nil || n != 15 {
t.Fatalf("Increment(existing) = %d %v", n, err)
}
if ttl := mr.TTL("acme:cache:views"); ttl != time.Minute {
t.Fatalf("Increment changed the TTL to %s", ttl)
}
_ = c.Set(ctx, "word", []byte("abc"), 0)
if _, err := c.Increment(ctx, "word", 1); err == nil {
t.Fatal("Increment on a non-integer must fail")
}
// Remember stores with the TTL and calls the callback once.
calls := 0
fn := func(context.Context) ([]byte, error) { calls++; return []byte("computed"), nil }
for range 3 {
if v, err := c.Remember(ctx, "remembered", 30*time.Second, fn); err != nil || string(v) != "computed" {
t.Fatalf("Remember = %q %v", v, err)
}
}
if calls != 1 || mr.TTL("acme:cache:remembered") != 30*time.Second {
t.Fatalf("Remember: %d calls, TTL %s", calls, mr.TTL("acme:cache:remembered"))
}
// Forget removes only that key.
if err := c.Forget(ctx, "views"); err != nil {
t.Fatal(err)
}
if mr.Exists("acme:cache:views") || !mr.Exists("acme:cache:hits") {
t.Fatal("Forget removed the wrong keys")
}
// A cache without a client answers errNilClient on every method.
var none Cache = &redisCache{}
if _, _, err := none.Get(ctx, "k"); !errors.Is(err, errNilClient) {
t.Fatalf("Get without client = %v", err)
}
if err := none.Set(ctx, "k", nil, 0); !errors.Is(err, errNilClient) {
t.Fatalf("Set without client = %v", err)
}
if _, err := none.Has(ctx, "k"); !errors.Is(err, errNilClient) {
t.Fatalf("Has without client = %v", err)
}
if err := none.Forget(ctx, "k"); !errors.Is(err, errNilClient) {
t.Fatalf("Forget without client = %v", err)
}
if _, err := none.Increment(ctx, "k", 1); !errors.Is(err, errNilClient) {
t.Fatalf("Increment without client = %v", err)
}
if err := none.Flush(ctx); !errors.Is(err, errNilClient) {
t.Fatalf("Flush without client = %v", err)
}
}
func TestRedisFlushOnlyPrefix(t *testing.T) {
ctx := t.Context()
mr, client := miniClient(t)
// A prefix with glob characters is matched literally.
const prefix = "a*[b]?:"
c := NewRedisCache(client, prefix)
for i := range 20 {
if err := c.Set(ctx, "k"+strconv.Itoa(i), []byte("v"), 0); err != nil {
t.Fatal(err)
}
}
survivors := map[string]string{
prefix + "rl:login|192.0.2.1": "3", // this app's rate-limit counter
"other:cache:greeting": "theirs", // another app's cache
"aXbY:cache:k1": "glob", // matches the prefix as a pattern
"a*[b]?:cachex": "near", // shares the prefix, not the cache: part
"plain": "x",
}
for k, v := range survivors {
if err := mr.Set(k, v); err != nil {
t.Fatal(err)
}
}
if err := c.Flush(ctx); err != nil {
t.Fatal(err)
}
for i := range 20 {
if mr.Exists(prefix + "cache:k" + strconv.Itoa(i)) {
t.Fatalf("Flush left %scache:k%d", prefix, i)
}
}
for k, v := range survivors {
if got, err := mr.Get(k); err != nil || got != v {
t.Fatalf("Flush removed or changed %q: %q %v", k, got, err)
}
}
if err := c.Flush(ctx); err != nil {
t.Fatalf("Flush on an empty cache: %v", err)
}
if got := escapeGlob(`a*b?[c]\`); got != `a\*b\?\[c\]\\` {
t.Fatalf("escapeGlob = %q", got)
}
}

View File

@@ -0,0 +1,140 @@
package cooler
import (
"context"
"errors"
"log/slog"
"strings"
"time"
"github.com/redis/go-redis/v9"
)
// flushBatch is how many keys one SCAN step asks for during Flush.
const flushBatch = 500
// redisCache is the Cache on Redis, shared by every instance using the same
// Redis and prefix. Keys are <prefix>cache:<key>.
type redisCache struct {
client *redis.Client
prefix string
warn *warner
}
// NewRedisCache returns a Cache on client with keys <prefix>cache:<key>.
// It does not dial; the first call does.
func NewRedisCache(client *redis.Client, prefix string) Cache {
return &redisCache{client: client, prefix: prefix, warn: newWarner(slog.Default())}
}
func (r *redisCache) key(k string) string { return r.prefix + "cache:" + k }
func (r *redisCache) setLogger(log *slog.Logger) { r.warn.setLogger(log) }
func (r *redisCache) ready() error {
if r == nil || r.client == nil {
return errNilClient
}
return nil
}
func (r *redisCache) Get(ctx context.Context, key string) ([]byte, bool, error) {
if err := r.ready(); err != nil {
return nil, false, err
}
b, err := r.client.Get(ctx, r.key(key)).Bytes()
if errors.Is(err, redis.Nil) {
return nil, false, nil
}
if err != nil {
return nil, false, err
}
return b, true, nil
}
// Set stores value with SET … PX; a ttl of 0 or less stores it without expiry.
func (r *redisCache) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
if err := r.ready(); err != nil {
return err
}
if value == nil {
value = []byte{}
}
if ttl <= 0 {
return r.client.Set(ctx, r.key(key), value, 0).Err()
}
ms := ttl.Milliseconds()
if ms < 1 {
ms = 1
}
return r.client.Do(ctx, "SET", r.key(key), value, "PX", ms).Err()
}
func (r *redisCache) Has(ctx context.Context, key string) (bool, error) {
if err := r.ready(); err != nil {
return false, err
}
n, err := r.client.Exists(ctx, r.key(key)).Result()
return n > 0, err
}
func (r *redisCache) Forget(ctx context.Context, key string) error {
if err := r.ready(); err != nil {
return err
}
return r.client.Unlink(ctx, r.key(key)).Err()
}
func (r *redisCache) Remember(ctx context.Context, key string, ttl time.Duration, fn func(context.Context) ([]byte, error)) ([]byte, error) {
return remember(ctx, r, r.warn, key, ttl, fn)
}
// Increment runs INCRBY: a missing key starts at 0 without a TTL, an existing
// key keeps its expiry.
func (r *redisCache) Increment(ctx context.Context, key string, by int64) (int64, error) {
if err := r.ready(); err != nil {
return 0, err
}
return r.client.IncrBy(ctx, r.key(key), by).Result()
}
// Flush removes only this application's cache keys: it walks SCAN MATCH
// <prefix>cache:* in batches and removes each batch with UNLINK. It never
// empties the database, which would also drop rate-limit counters and other
// applications' keys.
func (r *redisCache) Flush(ctx context.Context) error {
if err := r.ready(); err != nil {
return err
}
match := escapeGlob(r.prefix+"cache:") + "*"
var cursor uint64
for {
keys, next, err := r.client.Scan(ctx, cursor, match, flushBatch).Result()
if err != nil {
return err
}
if len(keys) > 0 {
if err := r.client.Unlink(ctx, keys...).Err(); err != nil {
return err
}
}
if next == 0 {
return nil
}
cursor = next
}
}
// escapeGlob escapes the characters SCAN MATCH treats as a pattern, so a
// prefix is matched literally.
func escapeGlob(s string) string {
var b strings.Builder
for _, r := range s {
switch r {
case '*', '?', '[', ']', '\\':
b.WriteByte('\\')
}
b.WriteRune(r)
}
return b.String()
}

375
modules/cooler/sitecache.go Normal file
View File

@@ -0,0 +1,375 @@
package cooler
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"net/url"
"strings"
"sync"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/lagoon"
"gorm.io/gorm"
)
const (
// SiteCachePurgeURLKey is the config key of the host site's cache purge
// URL. Environment: SUMMER_SITE_CACHE__PURGE_URL.
SiteCachePurgeURLKey = "site_cache.purge_url"
// SiteCachePurgeTokenKey is the config key of the bearer token the purge
// request carries. Environment: SUMMER_SITE_CACHE__PURGE_TOKEN.
SiteCachePurgeTokenKey = "site_cache.purge_token"
// SiteCachePurgeTimeout bounds one purge request.
SiteCachePurgeTimeout = 2 * time.Second
// SiteCachePurgeMinTokenLength is the shortest purge token accepted, the
// same minimum the host site's purge route applies.
SiteCachePurgeMinTokenLength = 32
// siteCacheAnswerLimit is how much of the purge answer is read.
siteCacheAnswerLimit = 4 << 10
)
// SiteCachePurger asks the host site to drop its page caches after an admin
// save. A nil *SiteCachePurger is disabled and does nothing.
//
// Purges queued by PurgeSiteCacheAfterCommit go through one worker per
// purger: at most one request is in flight, and the saves committed while
// it is sent collapse into one follow-up request carrying the last kind and
// slug (each purge clears the site's whole route cache).
type SiteCachePurger struct {
url string
endpoint string
token string
client *http.Client
mu sync.Mutex
// sending is true while the worker goroutine runs.
sending bool
// pending is the follow-up body, the latest one queued while sending.
pending *siteCachePurgeBody
// coalesced counts the queued bodies pending replaced.
coalesced int
// idle is closed when the current busy period ends; nil while idle.
idle chan struct{}
}
// NewSiteCachePurger returns the purger for purgeURL and token, read from
// site_cache.purge_url and site_cache.purge_token. Validation errors name
// those keys; see NewSiteCachePurgerFor for the rules.
func NewSiteCachePurger(purgeURL, token string) (*SiteCachePurger, error) {
return NewSiteCachePurgerFor(SiteCachePurgeURLKey, SiteCachePurgeTokenKey, purgeURL, token)
}
// NewSiteCachePurgerFor returns the purger for purgeURL and token, read
// from the config keys urlKey and tokenKey, which its errors name. With
// both values empty the purge is disabled and it returns nil, nil.
// Otherwise both must be set, the URL must be http or https with a host and
// without user info or a fragment, an http URL must point at a loopback
// host (127.0.0.0/8, ::1 or localhost) so the bearer token never crosses
// the network in clear, and the token must hold at least
// SiteCachePurgeMinTokenLength characters. An error never contains the
// token.
func NewSiteCachePurgerFor(urlKey, tokenKey, purgeURL, token string) (*SiteCachePurger, error) {
purgeURL = strings.TrimSpace(purgeURL)
if purgeURL == "" && token == "" {
return nil, nil
}
if purgeURL == "" {
return nil, fmt.Errorf("%s is set but %s is not; set both or neither", tokenKey, urlKey)
}
if token == "" {
return nil, fmt.Errorf("%s is set but %s is not; set both or neither", urlKey, tokenKey)
}
u, err := url.Parse(purgeURL)
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" || u.Hostname() == "" {
return nil, fmt.Errorf("%s must be an http or https URL with a host", urlKey)
}
if u.User != nil {
return nil, fmt.Errorf("%s must not carry user info", urlKey)
}
if u.Fragment != "" || strings.Contains(purgeURL, "#") {
return nil, fmt.Errorf("%s must not carry a fragment", urlKey)
}
if u.Scheme == "http" && !loopbackHost(u.Hostname()) {
return nil, fmt.Errorf("%s: an http URL must point at a loopback host (127.0.0.0/8, ::1 or localhost); use https for any other host", urlKey)
}
if len(token) < SiteCachePurgeMinTokenLength {
return nil, fmt.Errorf("%s must hold at least %d characters", tokenKey, SiteCachePurgeMinTokenLength)
}
return &SiteCachePurger{url: u.String(), endpoint: u.Host + u.Path, token: token, client: &http.Client{CheckRedirect: noRedirect}}, nil
}
// noRedirect keeps the purge client on the configured URL: a 3xx answer is
// returned as is (and logged as a failed purge with its status), so the
// bearer token never follows a Location and the POST never becomes a GET.
func noRedirect(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }
// Endpoint returns the purge URL's host and path, without the scheme, user
// info or query, for log lines.
func (p *SiteCachePurger) Endpoint() string {
if p == nil {
return ""
}
return p.endpoint
}
func loopbackHost(host string) bool {
if strings.EqualFold(host, "localhost") {
return true
}
ip := net.ParseIP(host)
return ip != nil && ip.IsLoopback()
}
type siteCachePurgeBody struct {
Kind string `json:"kind"`
Slug string `json:"slug"`
}
// Purge sends the purge request for one saved entity: a POST of
// {"kind":kind,"slug":slug} with the bearer token, bounded by
// SiteCachePurgeTimeout and detached from ctx's cancellation. It blocks
// until the request is done; it is the send the PurgeSiteCacheAfterCommit
// queue uses. It logs the outcome and returns nothing: a purge never fails
// a save. A redirect is never followed; a 3xx answer is logged as a failed
// purge with its status. The token is never logged.
func (p *SiteCachePurger) Purge(ctx context.Context, kind, slug string) {
if p == nil {
return
}
p.send(ctx, kind, slug, 0)
}
// send is Purge; coalesced, when above zero, is logged with the outcome:
// the number of queued purges this request stands in for besides its own.
func (p *SiteCachePurger) send(ctx context.Context, kind, slug string, coalesced int) {
if ctx == nil {
ctx = context.Background()
}
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), SiteCachePurgeTimeout)
defer cancel()
log := slog.Default()
failed := func(attrs ...any) {
attrs = append([]any{"kind", kind, "slug", slug}, attrs...)
if coalesced > 0 {
attrs = append(attrs, "coalesced", coalesced)
}
log.Warn("cooler: site cache purge failed", attrs...)
}
raw, err := json.Marshal(siteCachePurgeBody{Kind: kind, Slug: slug})
if err != nil {
failed("error", err.Error())
return
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, p.url, bytes.NewReader(raw))
if err != nil {
failed("error", "invalid request")
return
}
req.Header.Set("Authorization", "Bearer "+p.token)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
client := p.client
if client == nil {
client = &http.Client{CheckRedirect: noRedirect}
}
res, err := client.Do(req)
if err != nil {
failed("error", purgeTransportError(err))
return
}
defer res.Body.Close()
body, _ := io.ReadAll(io.LimitReader(res.Body, siteCacheAnswerLimit))
if res.StatusCode < 200 || res.StatusCode > 299 {
failed("status", res.StatusCode)
return
}
attrs := []any{"kind", kind, "slug", slug, "status", res.StatusCode}
var answer struct {
Purged *int `json:"purged"`
}
if json.Unmarshal(body, &answer) == nil && answer.Purged != nil {
attrs = append(attrs, "purged", *answer.Purged)
}
if coalesced > 0 {
attrs = append(attrs, "coalesced", coalesced)
}
log.Info("cooler: site cache purged", attrs...)
}
// enqueue queues one purge. With nothing in flight it starts the worker,
// which sends at once; while a request is in flight the body becomes the
// pending follow-up, replacing (and counting) an earlier pending one.
func (p *SiteCachePurger) enqueue(kind, slug string) {
body := siteCachePurgeBody{Kind: kind, Slug: slug}
p.mu.Lock()
if p.sending {
if p.pending != nil {
p.coalesced++
}
p.pending = &body
p.mu.Unlock()
return
}
p.sending = true
p.idle = make(chan struct{})
p.mu.Unlock()
go p.work(body)
}
// work sends body, then each pending follow-up, until none is left; then it
// ends the busy period.
func (p *SiteCachePurger) work(body siteCachePurgeBody) {
coalesced := 0
for {
p.send(context.Background(), body.Kind, body.Slug, coalesced)
p.mu.Lock()
if p.pending == nil {
p.sending = false
close(p.idle)
p.idle = nil
p.mu.Unlock()
return
}
body, coalesced = *p.pending, p.coalesced
p.pending, p.coalesced = nil, 0
p.mu.Unlock()
}
}
// Wait blocks until no queued purge is pending or in flight, or until ctx
// ends, then returning ctx.Err(). It returns nil at once for a nil or idle
// purger. Tests and commands call it before they read what a purge did;
// cooler's Handle.Close calls it at shutdown.
func (p *SiteCachePurger) Wait(ctx context.Context) error {
if p == nil {
return nil
}
if ctx == nil {
ctx = context.Background()
}
for {
p.mu.Lock()
idle := p.idle
p.mu.Unlock()
if idle == nil {
return nil
}
select {
case <-idle:
// A new busy period may have started; look again.
case <-ctx.Done():
return ctx.Err()
}
}
}
// purgeTransportError describes a failed request without the request URL
// or headers.
func purgeTransportError(err error) string {
var uerr *url.Error
if errors.As(err, &uerr) {
err = uerr.Err
}
if errors.Is(err, context.DeadlineExceeded) {
return "timeout"
}
return err.Error()
}
// PurgeSiteCacheAfterCommit queues a purge of kind and slug after the
// transaction db belongs to commits, through lagoon.AfterCommit: a failed
// save never purges, and a concurrent page view cannot re-cache the old
// content between the purge and the commit. The after-commit callback only
// queues, so the save never waits for the site (fire and forget). At most
// one request is in flight per purger, and saves committed meanwhile
// collapse into one follow-up request with the last kind and slug, since
// each purge clears the site's whole route cache. A nil p does nothing;
// outside a transaction (db nil or not in one) the purge is queued at once.
// SiteCachePurger.Wait waits for the queue to drain.
func PurgeSiteCacheAfterCommit(ctx context.Context, db *gorm.DB, p *SiteCachePurger, kind, slug string) {
if p == nil {
return
}
lagoon.AfterCommit(ctx, db, func(context.Context, *gorm.DB) {
p.enqueue(kind, slug)
})
}
// SiteCacheFromApp returns the *SiteCachePurger published on app (by
// Install, by a plugin's alias configuration or by an earlier call), or
// builds one from site_cache.purge_url and site_cache.purge_token and
// publishes it. It returns nil when neither key is set, and when the
// configuration is invalid, which it logs as one error naming the key (the
// serve command fails at boot on the same configuration). Admin hooks call
// it, so it serves commands and tests that never run serve.
func SiteCacheFromApp(app *backpack.App) *SiteCachePurger {
if app == nil {
return nil
}
if p, ok := app.Lookup[*SiteCachePurger](); ok && p != nil {
return p
}
p, err := siteCacheFromConfig(app)
if err != nil {
loggerFromApp(app).Error("cooler: site cache purge disabled", "error", err.Error())
return nil
}
if p == nil {
return nil
}
if err := app.Publish(p); err != nil {
if winner, ok := app.Lookup[*SiteCachePurger](); ok && winner != nil {
return winner
}
return nil
}
return p
}
// siteCacheFromConfig builds the purger from the framework keys.
func siteCacheFromConfig(app *backpack.App) (*SiteCachePurger, error) {
if app == nil || app.Config == nil {
return nil, nil
}
p, err := NewSiteCachePurger(app.Config.String(SiteCachePurgeURLKey), app.Config.String(SiteCachePurgeTokenKey))
if err != nil {
return nil, fmt.Errorf("cooler: %w", err)
}
return p, nil
}
// installSiteCache publishes the site-cache purger unless one is already
// published (a plugin's alias configuration) and returns the purger in
// effect, nil when none is configured: an invalid configuration is a boot
// error naming the key.
func installSiteCache(app *backpack.App, log *slog.Logger) (*SiteCachePurger, error) {
if p, ok := app.Lookup[*SiteCachePurger](); ok && p != nil {
log.Info("cooler: site cache purge enabled", "endpoint", p.Endpoint())
return p, nil
}
p, err := siteCacheFromConfig(app)
if err != nil {
return nil, err
}
if p == nil {
return nil, nil
}
if err := app.Publish(p); err != nil {
return nil, fmt.Errorf("cooler: %w", err)
}
log.Info("cooler: site cache purge enabled", "endpoint", p.Endpoint())
return p, nil
}

View File

@@ -0,0 +1,878 @@
package cooler
import (
"bytes"
"context"
"database/sql"
"encoding/json"
"errors"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"sync"
"testing"
"time"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/compass"
"git.golem15.com/golem15/summercms/modules/lagoon"
_ "github.com/jackc/pgx/v5/stdlib"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/modules/postgres"
"gorm.io/gorm"
)
const (
purgeToken32 = "0123456789abcdef0123456789abcdef"
purgeToken40 = "site-cache-unit-token-0123456789abcdefgh"
)
// captureLogs routes slog.Default to a text handler on a buffer for the
// test and returns the buffer.
func captureLogs(t *testing.T) *syncBuffer {
t.Helper()
buf := &syncBuffer{}
prev := slog.Default()
slog.SetDefault(slog.New(slog.NewTextHandler(buf, &slog.HandlerOptions{Level: slog.LevelDebug})))
t.Cleanup(func() { slog.SetDefault(prev) })
return buf
}
type syncBuffer struct {
mu sync.Mutex
buf bytes.Buffer
}
func (b *syncBuffer) Write(p []byte) (int, error) {
b.mu.Lock()
defer b.mu.Unlock()
return b.buf.Write(p)
}
func (b *syncBuffer) lines() []string {
b.mu.Lock()
defer b.mu.Unlock()
var out []string
for _, line := range strings.Split(strings.TrimSpace(b.buf.String()), "\n") {
if line != "" {
out = append(out, line)
}
}
return out
}
func TestSiteCachePurgeConfig(t *testing.T) {
refused := []struct {
name, url, token, key string
}{
{"only the URL", "http://127.0.0.1:3016/api/__site__/cache/purge", "", SiteCachePurgeTokenKey},
{"only the token", "", purgeToken40, SiteCachePurgeURLKey},
{"a bad scheme", "ftp://127.0.0.1/purge", purgeToken40, SiteCachePurgeURLKey},
{"no scheme", "127.0.0.1:3016/purge", purgeToken40, SiteCachePurgeURLKey},
{"no host", "http:///purge", purgeToken40, SiteCachePurgeURLKey},
{"user info", "http://user:secret@127.0.0.1/purge", purgeToken40, SiteCachePurgeURLKey},
{"a fragment", "http://127.0.0.1/purge#top", purgeToken40, SiteCachePurgeURLKey},
{"an empty fragment", "http://127.0.0.1/purge#", purgeToken40, SiteCachePurgeURLKey},
{"http to a non-loopback host", "http://example.test/purge", purgeToken40, SiteCachePurgeURLKey},
{"http to a private address", "http://10.0.0.5:3016/purge", purgeToken40, SiteCachePurgeURLKey},
{"a 31-character token", "http://127.0.0.1/purge", purgeToken32[:31], SiteCachePurgeTokenKey},
}
for _, tc := range refused {
t.Run("refuses "+tc.name, func(t *testing.T) {
p, err := NewSiteCachePurger(tc.url, tc.token)
if err == nil || p != nil {
t.Fatalf("NewSiteCachePurger(%q) = %v, %v; want an error", tc.url, p, err)
}
if !strings.Contains(err.Error(), tc.key) {
t.Fatalf("error %q does not name %s", err, tc.key)
}
if tc.token != "" && strings.Contains(err.Error(), tc.token) {
t.Fatalf("error %q carries the token", err)
}
})
}
t.Run("both empty disables the purge", func(t *testing.T) {
p, err := NewSiteCachePurger("", "")
if p != nil || err != nil {
t.Fatalf("NewSiteCachePurger(\"\", \"\") = %v, %v; want nil, nil", p, err)
}
})
accepted := []struct {
url, endpoint string
}{
{"https://cache.example.test/api/__site__/cache/purge", "cache.example.test/api/__site__/cache/purge"},
{"http://127.0.0.1:3016/api/__site__/cache/purge", "127.0.0.1:3016/api/__site__/cache/purge"},
{"http://127.9.8.7/purge", "127.9.8.7/purge"},
{"http://[::1]:3000/purge", "[::1]:3000/purge"},
{"http://localhost:3000/purge?x=1", "localhost:3000/purge"},
}
for _, tc := range accepted {
t.Run("accepts "+tc.url, func(t *testing.T) {
p, err := NewSiteCachePurger(tc.url, purgeToken32)
if err != nil || p == nil {
t.Fatalf("NewSiteCachePurger(%q) = %v, %v; want a purger", tc.url, p, err)
}
if p.Endpoint() != tc.endpoint {
t.Fatalf("Endpoint() = %q, want %q", p.Endpoint(), tc.endpoint)
}
})
}
t.Run("errors name the keys a caller passes", func(t *testing.T) {
const urlKey, tokenKey = "acme.site_cache.purge_url", "acme.site_cache.purge_token"
if _, err := NewSiteCachePurgerFor(urlKey, tokenKey, "http://example.test/purge", purgeToken40); err == nil || !strings.Contains(err.Error(), urlKey) {
t.Fatalf("http to a remote host: %v, want an error naming %s", err, urlKey)
}
if _, err := NewSiteCachePurgerFor(urlKey, tokenKey, "http://127.0.0.1/purge", purgeToken32[:31]); err == nil || !strings.Contains(err.Error(), tokenKey) {
t.Fatalf("short token: %v, want an error naming %s", err, tokenKey)
}
})
t.Run("SiteCacheFromApp builds from the framework keys and publishes once", func(t *testing.T) {
cfg := testConfig(t)
app := backpack.New(cfg)
if p := SiteCacheFromApp(app); p != nil {
t.Fatalf("no keys: SiteCacheFromApp = %v, want nil", p)
}
if err := cfg.Set(SiteCachePurgeURLKey, "http://127.0.0.1:3000/api/__site__/cache/purge"); err != nil {
t.Fatal(err)
}
if err := cfg.Set(SiteCachePurgeTokenKey, purgeToken40); err != nil {
t.Fatal(err)
}
first := SiteCacheFromApp(app)
if first == nil || first.Endpoint() != "127.0.0.1:3000/api/__site__/cache/purge" {
t.Fatalf("SiteCacheFromApp = %v", first)
}
if again := SiteCacheFromApp(app); again != first {
t.Fatal("a second call built another purger")
}
if got, ok := app.Lookup[*SiteCachePurger](); !ok || got != first {
t.Fatal("the purger was not published")
}
})
t.Run("SiteCacheFromApp logs an invalid configuration and returns nil", func(t *testing.T) {
logs := captureLogs(t)
cfg := testConfig(t)
if err := cfg.Set(SiteCachePurgeURLKey, "http://example.test/purge"); err != nil {
t.Fatal(err)
}
if err := cfg.Set(SiteCachePurgeTokenKey, purgeToken40); err != nil {
t.Fatal(err)
}
if p := SiteCacheFromApp(backpack.New(cfg)); p != nil {
t.Fatalf("SiteCacheFromApp = %v, want nil", p)
}
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=ERROR") || !strings.Contains(lines[0], SiteCachePurgeURLKey) {
t.Fatalf("log = %q", lines)
}
assertNoToken(t, logs)
})
t.Run("Install fails on an invalid configuration and keeps a published purger", func(t *testing.T) {
cfg := testConfig(t)
if err := cfg.Set(SiteCachePurgeURLKey, "http://127.0.0.1/purge"); err != nil {
t.Fatal(err)
}
if _, err := Install(context.Background(), backpack.New(cfg)); err == nil || !strings.Contains(err.Error(), SiteCachePurgeTokenKey) {
t.Fatalf("Install with only the URL = %v, want an error naming %s", err, SiteCachePurgeTokenKey)
}
alias, err := NewSiteCachePurgerFor("acme.purge_url", "acme.purge_token", "http://127.0.0.1:4000/purge", purgeToken40)
if err != nil {
t.Fatal(err)
}
app := backpack.New(cfg)
if err := app.Publish(alias); err != nil {
t.Fatal(err)
}
h, err := Install(context.Background(), app)
if err != nil {
t.Fatalf("Install with a published purger: %v", err)
}
defer h.Close()
if got := SiteCacheFromApp(app); got != alias {
t.Fatal("Install replaced the published purger")
}
})
var nilPurger *SiteCachePurger
if nilPurger.Endpoint() != "" {
t.Fatal("a nil purger has an endpoint")
}
}
// purgeHit is one request the test receiver got.
type purgeHit struct {
method, path, auth, contentType string
body map[string]string
}
func purgeReceiver(t *testing.T, handle func(w http.ResponseWriter, r *http.Request)) (*httptest.Server, func() []purgeHit) {
t.Helper()
var mu sync.Mutex
var hits []purgeHit
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
raw, _ := io.ReadAll(r.Body)
var body map[string]string
_ = json.Unmarshal(raw, &body)
mu.Lock()
hits = append(hits, purgeHit{r.Method, r.URL.Path, r.Header.Get("Authorization"), r.Header.Get("Content-Type"), body})
mu.Unlock()
handle(w, r)
}))
t.Cleanup(srv.Close)
return srv, func() []purgeHit {
mu.Lock()
defer mu.Unlock()
return append([]purgeHit(nil), hits...)
}
}
func newTestPurger(t *testing.T, srv *httptest.Server) *SiteCachePurger {
t.Helper()
p, err := NewSiteCachePurger(srv.URL+"/api/__site__/cache/purge", purgeToken40)
if err != nil {
t.Fatal(err)
}
return p
}
// waitPurges drains the purges queued on p, failing the test after 5 s.
func waitPurges(t *testing.T, p *SiteCachePurger) {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := p.Wait(ctx); err != nil {
t.Fatalf("Wait: %v", err)
}
}
func assertNoToken(t *testing.T, logs *syncBuffer) {
t.Helper()
for _, line := range logs.lines() {
if strings.Contains(line, purgeToken40) || strings.Contains(line, "Bearer") || strings.Contains(line, "Authorization") {
t.Fatalf("log line carries the token or the Authorization header: %s", line)
}
}
}
func TestSiteCachePurgeRequest(t *testing.T) {
t.Run("sends the contract request and logs the purged count", func(t *testing.T) {
logs := captureLogs(t)
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"purged":7}`))
})
newTestPurger(t, srv).Purge(context.Background(), "category", "cordyceps")
got := hits()
if len(got) != 1 {
t.Fatalf("%d requests, want 1", len(got))
}
h := got[0]
if h.method != http.MethodPost || h.path != "/api/__site__/cache/purge" {
t.Fatalf("request %s %s", h.method, h.path)
}
if h.auth != "Bearer "+purgeToken40 || !strings.HasPrefix(h.contentType, "application/json") {
t.Fatalf("Authorization %q, Content-Type %q", h.auth, h.contentType)
}
if len(h.body) != 2 || h.body["kind"] != "category" || h.body["slug"] != "cordyceps" {
t.Fatalf("body = %#v", h.body)
}
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=INFO") || !strings.Contains(lines[0], `msg="cooler: site cache purged"`) ||
!strings.Contains(lines[0], "kind=category") || !strings.Contains(lines[0], "slug=cordyceps") ||
!strings.Contains(lines[0], "status=200") || !strings.Contains(lines[0], "purged=7") {
t.Fatalf("log = %q", lines)
}
assertNoToken(t, logs)
})
t.Run("an answer without purged logs the status only", func(t *testing.T) {
logs := captureLogs(t)
srv, _ := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusNoContent) })
newTestPurger(t, srv).Purge(context.Background(), "post", "hello")
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=INFO") || !strings.Contains(lines[0], "status=204") || strings.Contains(lines[0], "purged=") {
t.Fatalf("log = %q", lines)
}
})
for _, status := range []int{http.StatusInternalServerError, http.StatusUnauthorized} {
t.Run("a "+http.StatusText(status)+" answer logs one warning", func(t *testing.T) {
logs := captureLogs(t)
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(status) })
newTestPurger(t, srv).Purge(context.Background(), "post", "hello")
if len(hits()) != 1 {
t.Fatalf("%d requests, want 1", len(hits()))
}
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], `msg="cooler: site cache purge failed"`) ||
!strings.Contains(lines[0], "kind=post") || !strings.Contains(lines[0], "slug=hello") {
t.Fatalf("log = %q", lines)
}
if !strings.Contains(lines[0], "status="+strconv.Itoa(status)) {
t.Fatalf("log %q lacks status=%d", lines[0], status)
}
assertNoToken(t, logs)
})
}
t.Run("a slow receiver is abandoned after the timeout", func(t *testing.T) {
logs := captureLogs(t)
release := make(chan struct{})
srv, _ := purgeReceiver(t, func(_ http.ResponseWriter, r *http.Request) {
select {
case <-release:
case <-r.Context().Done():
}
})
defer close(release)
start := time.Now()
newTestPurger(t, srv).Purge(context.Background(), "post", "slow")
elapsed := time.Since(start)
if elapsed < SiteCachePurgeTimeout-200*time.Millisecond || elapsed > SiteCachePurgeTimeout+1500*time.Millisecond {
t.Fatalf("Purge returned after %s, want about %s", elapsed, SiteCachePurgeTimeout)
}
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], "error=timeout") {
t.Fatalf("log = %q", lines)
}
assertNoToken(t, logs)
})
t.Run("an unreachable receiver logs one warning without the URL's token", func(t *testing.T) {
logs := captureLogs(t)
srv, _ := purgeReceiver(t, func(http.ResponseWriter, *http.Request) {})
p := newTestPurger(t, srv)
srv.Close()
p.Purge(context.Background(), "category", "gone")
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], "kind=category") {
t.Fatalf("log = %q", lines)
}
assertNoToken(t, logs)
})
t.Run("a cancelled parent context still sends", func(t *testing.T) {
captureLogs(t)
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(`{"purged":1}`)) })
ctx, cancel := context.WithCancel(context.Background())
cancel()
newTestPurger(t, srv).Purge(ctx, "post", "after-cancel")
if got := hits(); len(got) != 1 || got[0].body["slug"] != "after-cancel" {
t.Fatalf("requests = %#v, want one", got)
}
})
t.Run("a nil purger sends nothing", func(t *testing.T) {
logs := captureLogs(t)
var p *SiteCachePurger
p.Purge(context.Background(), "post", "nil")
PurgeSiteCacheAfterCommit(context.Background(), nil, nil, "post", "nil")
if lines := logs.lines(); len(lines) != 0 {
t.Fatalf("log = %q, want nothing", lines)
}
})
t.Run("outside a transaction the after-commit purge is queued at once", func(t *testing.T) {
captureLogs(t)
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(`{"purged":2}`)) })
p := newTestPurger(t, srv)
PurgeSiteCacheAfterCommit(context.Background(), nil, p, "category", "now")
waitPurges(t, p)
if got := hits(); len(got) != 1 || got[0].body["kind"] != "category" || got[0].body["slug"] != "now" {
t.Fatalf("requests = %#v, want one", got)
}
})
}
// testConfig returns an empty config that ignores the process environment.
func testConfig(t *testing.T) *compass.Config {
t.Helper()
cfg, err := compass.Open(compass.Options{Dir: t.TempDir(), Environ: []string{}})
if err != nil {
t.Fatal(err)
}
return cfg
}
// purgeDB starts a Postgres container for the test and returns a GORM handle
// with a table to write in transactions. It skips under -short.
func purgeDB(t *testing.T) *gorm.DB {
t.Helper()
if testing.Short() {
t.Skip("requires testcontainers postgres")
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
t.Cleanup(cancel)
ctr, err := postgres.Run(ctx,
"postgres:16-alpine",
postgres.WithDatabase("cooler"),
postgres.WithUsername("cooler"),
postgres.WithPassword("cooler"),
postgres.BasicWaitStrategies(),
)
if err != nil {
t.Fatalf("postgres: %v", err)
}
t.Cleanup(func() { _ = testcontainers.TerminateContainer(ctr) })
dsn, err := ctr.ConnectionString(ctx, "sslmode=disable")
if err != nil {
t.Fatal(err)
}
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = db.Close() })
if err := db.PingContext(ctx); err != nil {
t.Fatal(err)
}
gdb, err := lagoon.Use(ctx, db)
if err != nil {
t.Fatalf("lagoon.Use: %v", err)
}
if err := gdb.Exec(`CREATE TABLE purge_posts (id serial PRIMARY KEY, slug text NOT NULL)`).Error; err != nil {
t.Fatal(err)
}
return gdb
}
func countPosts(t *testing.T, gdb *gorm.DB, slug string) int64 {
t.Helper()
var n int64
if err := gdb.Raw(`SELECT count(*) FROM purge_posts WHERE slug = ?`, slug).Scan(&n).Error; err != nil {
t.Fatal(err)
}
return n
}
func TestSiteCacheAfterCommitOnly(t *testing.T) {
gdb := purgeDB(t)
captureLogs(t)
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(`{"purged":1}`)) })
p := newTestPurger(t, srv)
ctx := context.Background()
t.Run("a rolled-back save sends nothing", func(t *testing.T) {
boom := errors.New("validation failed")
err := lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
if err := tx.Exec(`INSERT INTO purge_posts (slug) VALUES ('rolled-back')`).Error; err != nil {
return err
}
PurgeSiteCacheAfterCommit(ctx, tx, p, "post", "rolled-back")
return boom
})
if !errors.Is(err, boom) {
t.Fatalf("Transaction = %v, want the callback's error", err)
}
if n := countPosts(t, gdb, "rolled-back"); n != 0 {
t.Fatalf("%d rows after rollback", n)
}
waitPurges(t, p)
if got := hits(); len(got) != 0 {
t.Fatalf("a rolled-back save sent %d purges", len(got))
}
})
t.Run("a committed save sends one purge after the commit", func(t *testing.T) {
var during int
err := lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
if err := tx.Exec(`INSERT INTO purge_posts (slug) VALUES ('committed')`).Error; err != nil {
return err
}
PurgeSiteCacheAfterCommit(ctx, tx, p, "post", "committed")
during = len(hits())
return nil
})
if err != nil {
t.Fatal(err)
}
if during != 0 {
t.Fatalf("%d purges were sent before the commit", during)
}
waitPurges(t, p)
got := hits()
if len(got) != 1 || got[0].body["kind"] != "post" || got[0].body["slug"] != "committed" {
t.Fatalf("purges = %#v, want one for post committed", got)
}
})
t.Run("a nested transaction purges once, after the outer commit", func(t *testing.T) {
before := len(hits())
var afterInner int
err := lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
if err := lagoon.Transaction(ctx, tx, func(ctx context.Context, inner *gorm.DB) error {
PurgeSiteCacheAfterCommit(ctx, inner, p, "category", "nested")
return inner.Exec(`INSERT INTO purge_posts (slug) VALUES ('nested')`).Error
}); err != nil {
return err
}
afterInner = len(hits())
return nil
})
if err != nil {
t.Fatal(err)
}
if afterInner != before {
t.Fatal("the purge ran when the inner transaction finished")
}
waitPurges(t, p)
if got := hits(); len(got) != before+1 || got[len(got)-1].body["slug"] != "nested" {
t.Fatalf("purges = %#v, want one more for nested", got)
}
})
}
func TestSiteCacheFailureNeverFails(t *testing.T) {
gdb := purgeDB(t)
ctx := context.Background()
t.Run("a 500 answer leaves the save committed", func(t *testing.T) {
logs := captureLogs(t)
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusInternalServerError) })
p := newTestPurger(t, srv)
err := lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
PurgeSiteCacheAfterCommit(ctx, tx, p, "post", "server-error")
return tx.Exec(`INSERT INTO purge_posts (slug) VALUES ('server-error')`).Error
})
if err != nil {
t.Fatalf("a failing purge failed the save: %v", err)
}
waitPurges(t, p)
if n := countPosts(t, gdb, "server-error"); n != 1 || len(hits()) != 1 {
t.Fatalf("rows %d, purges %d; want 1 and 1", n, len(hits()))
}
if lines := logs.lines(); len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], "status=500") {
t.Fatalf("log = %q", lines)
}
assertNoToken(t, logs)
})
t.Run("a slow receiver is abandoned and the save stays committed", func(t *testing.T) {
logs := captureLogs(t)
release := make(chan struct{})
srv, _ := purgeReceiver(t, func(_ http.ResponseWriter, r *http.Request) {
select {
case <-release:
case <-r.Context().Done():
}
})
defer close(release)
p := newTestPurger(t, srv)
start := time.Now()
err := lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
PurgeSiteCacheAfterCommit(ctx, tx, p, "post", "slow")
return tx.Exec(`INSERT INTO purge_posts (slug) VALUES ('slow')`).Error
})
if err != nil {
t.Fatalf("a slow purge failed the save: %v", err)
}
if elapsed := time.Since(start); elapsed > 500*time.Millisecond {
t.Fatalf("the save waited %s for a queued purge", elapsed)
}
if n := countPosts(t, gdb, "slow"); n != 1 {
t.Fatalf("%d rows, want the save committed", n)
}
waitPurges(t, p)
if lines := logs.lines(); len(lines) != 1 || !strings.Contains(lines[0], "error=timeout") {
t.Fatalf("log = %q", lines)
}
})
t.Run("an unreachable receiver leaves the save committed", func(t *testing.T) {
captureLogs(t)
srv, _ := purgeReceiver(t, func(http.ResponseWriter, *http.Request) {})
p := newTestPurger(t, srv)
srv.Close()
err := lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
PurgeSiteCacheAfterCommit(ctx, tx, p, "post", "unreachable")
return tx.Exec(`INSERT INTO purge_posts (slug) VALUES ('unreachable')`).Error
})
if err != nil || countPosts(t, gdb, "unreachable") != 1 {
t.Fatalf("Transaction = %v, rows %d", err, countPosts(t, gdb, "unreachable"))
}
waitPurges(t, p)
})
}
func TestSiteCacheFromApp(t *testing.T) {
if p := SiteCacheFromApp(nil); p != nil {
t.Fatal("SiteCacheFromApp(nil) must be nil")
}
if p := SiteCacheFromApp(backpack.New(nil)); p != nil {
t.Fatal("an app without config has no purger")
}
cfg := testConfig(t)
app := backpack.New(cfg)
if p := SiteCacheFromApp(app); p != nil {
t.Fatalf("unset keys: %v, want nil", p)
}
if _, ok := app.Lookup[*SiteCachePurger](); ok {
t.Fatal("unset keys published a purger")
}
if err := cfg.Set(SiteCachePurgeURLKey, "https://www.example.test/api/__site__/cache/purge"); err != nil {
t.Fatal(err)
}
if err := cfg.Set(SiteCachePurgeTokenKey, purgeToken40); err != nil {
t.Fatal(err)
}
// Concurrent first calls build and publish one purger.
var wg sync.WaitGroup
got := make([]*SiteCachePurger, 20)
for i := range got {
wg.Add(1)
go func() {
defer wg.Done()
got[i] = SiteCacheFromApp(app)
}()
}
wg.Wait()
for i, p := range got {
if p == nil || p != got[0] {
t.Fatalf("call %d = %v, want the one published purger", i, p)
}
}
if pub, ok := app.Lookup[*SiteCachePurger](); !ok || pub != got[0] {
t.Fatal("the purger was not published")
}
if got[0].Endpoint() != "www.example.test/api/__site__/cache/purge" {
t.Fatalf("Endpoint = %q", got[0].Endpoint())
}
// A later config change does not rebuild the published purger.
if err := cfg.Set(SiteCachePurgeURLKey, "https://other.example.test/purge"); err != nil {
t.Fatal(err)
}
if again := SiteCacheFromApp(app); again != got[0] {
t.Fatal("SiteCacheFromApp rebuilt a published purger")
}
}
func TestSiteCachePurgeNoRedirect(t *testing.T) {
logs := captureLogs(t)
target, targetHits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(`{"purged":1}`)) })
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Location", target.URL+"/api/__site__/cache/purge")
w.WriteHeader(http.StatusTemporaryRedirect)
})
newTestPurger(t, srv).Purge(context.Background(), "post", "moved")
if len(hits()) != 1 {
t.Fatalf("%d requests to the purge URL, want 1", len(hits()))
}
if got := targetHits(); len(got) != 0 {
t.Fatalf("the redirect target received %d requests: %#v", len(got), got)
}
lines := logs.lines()
if len(lines) != 1 || !strings.Contains(lines[0], "level=WARN") || !strings.Contains(lines[0], `msg="cooler: site cache purge failed"`) || !strings.Contains(lines[0], "status=307") {
t.Fatalf("log = %q, want one failed purge with status=307", lines)
}
assertNoToken(t, logs)
}
func TestSiteCachePurgeAsyncCoalesced(t *testing.T) {
logs := captureLogs(t)
release := make(chan struct{})
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) {
<-release
_, _ = w.Write([]byte(`{"purged":3}`))
})
p := newTestPurger(t, srv)
released := false
defer func() {
if !released {
close(release)
}
waitPurges(t, p)
}()
for i := range 20 {
start := time.Now()
PurgeSiteCacheAfterCommit(context.Background(), nil, p, "post", "slug-"+strconv.Itoa(i))
if elapsed := time.Since(start); elapsed > 50*time.Millisecond {
t.Fatalf("call %d blocked for %s", i, elapsed)
}
}
close(release)
released = true
waitPurges(t, p)
got := hits()
if len(got) != 2 {
t.Fatalf("%d requests, want 2 (one in flight, one coalesced follow-up)", len(got))
}
if got[0].body["slug"] != "slug-0" || got[1].body["slug"] != "slug-19" {
t.Fatalf("slugs %q, %q; want slug-0 then the last, slug-19", got[0].body["slug"], got[1].body["slug"])
}
lines := logs.lines()
if len(lines) != 2 || strings.Contains(lines[0], "coalesced=") || !strings.Contains(lines[1], "slug=slug-19") || !strings.Contains(lines[1], "coalesced=18") {
t.Fatalf("log = %q, want the second purged line with coalesced=18", lines)
}
assertNoToken(t, logs)
}
// blockingReceiver answers once release is closed (or the request ends).
func blockingReceiver(t *testing.T) (*httptest.Server, func() []purgeHit, func()) {
t.Helper()
release := make(chan struct{})
var once sync.Once
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, r *http.Request) {
select {
case <-release:
_, _ = w.Write([]byte(`{"purged":1}`))
case <-r.Context().Done():
}
})
free := func() { once.Do(func() { close(release) }) }
t.Cleanup(free)
return srv, hits, free
}
func TestSiteCachePurgeWaitAndClose(t *testing.T) {
t.Run("nil and idle purgers return at once", func(t *testing.T) {
var nilPurger *SiteCachePurger
if err := nilPurger.Wait(context.Background()); err != nil {
t.Fatalf("nil Wait = %v", err)
}
srv, _ := purgeReceiver(t, func(http.ResponseWriter, *http.Request) {})
ctx, cancel := context.WithCancel(context.Background())
cancel()
if err := newTestPurger(t, srv).Wait(ctx); err != nil {
t.Fatalf("idle Wait under a cancelled context = %v, want nil", err)
}
})
t.Run("Wait gives up with the context and the purge is still sent", func(t *testing.T) {
captureLogs(t)
srv, hits, free := blockingReceiver(t)
p := newTestPurger(t, srv)
PurgeSiteCacheAfterCommit(context.Background(), nil, p, "post", "pending")
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Millisecond)
defer cancel()
if err := p.Wait(ctx); !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("Wait = %v, want context.DeadlineExceeded", err)
}
free()
waitPurges(t, p)
if got := hits(); len(got) != 1 || got[0].body["slug"] != "pending" {
t.Fatalf("requests = %#v, want the queued purge", got)
}
})
install := func(t *testing.T, srv *httptest.Server) (*Handle, *SiteCachePurger) {
t.Helper()
cfg := testConfig(t)
if err := cfg.Set(SiteCachePurgeURLKey, srv.URL+"/api/__site__/cache/purge"); err != nil {
t.Fatal(err)
}
if err := cfg.Set(SiteCachePurgeTokenKey, purgeToken40); err != nil {
t.Fatal(err)
}
app := backpack.New(cfg)
h, err := Install(context.Background(), app)
if err != nil {
t.Fatal(err)
}
p := SiteCacheFromApp(app)
if p == nil {
t.Fatal("Install published no purger")
}
return h, p
}
t.Run("Handle.Close waits for a queued purge", func(t *testing.T) {
captureLogs(t)
srv, hits := purgeReceiver(t, func(w http.ResponseWriter, _ *http.Request) {
time.Sleep(200 * time.Millisecond)
_, _ = w.Write([]byte(`{"purged":1}`))
})
h, p := install(t, srv)
PurgeSiteCacheAfterCommit(context.Background(), nil, p, "post", "at-shutdown")
if err := h.Close(); err != nil {
t.Fatal(err)
}
if got := hits(); len(got) != 1 || got[0].body["slug"] != "at-shutdown" {
t.Fatalf("requests after Close = %#v, want the queued purge", got)
}
})
t.Run("Handle.Close is bounded when the site hangs", func(t *testing.T) {
captureLogs(t)
srv, _, _ := blockingReceiver(t)
h, p := install(t, srv)
PurgeSiteCacheAfterCommit(context.Background(), nil, p, "post", "hung")
start := time.Now()
if err := h.Close(); err != nil {
t.Fatal(err)
}
if elapsed := time.Since(start); elapsed > SiteCachePurgeTimeout+time.Second+500*time.Millisecond {
t.Fatalf("Close took %s, want at most %s plus a second", elapsed, SiteCachePurgeTimeout)
}
waitPurges(t, p)
})
}
func TestSiteCacheBulkCommitCoalesced(t *testing.T) {
gdb := purgeDB(t)
captureLogs(t)
ctx := context.Background()
run := func(t *testing.T, save func(p *SiteCachePurger) error, last string) {
t.Helper()
srv, hits, free := blockingReceiver(t)
p := newTestPurger(t, srv)
start := time.Now()
if err := save(p); err != nil {
t.Fatal(err)
}
if elapsed := time.Since(start); elapsed > 500*time.Millisecond {
t.Fatalf("the saves took %s while the site was blocked, want under 500ms", elapsed)
}
free()
waitPurges(t, p)
got := hits()
if len(got) != 2 {
t.Fatalf("%d requests, want 2 (one in flight, one coalesced follow-up)", len(got))
}
if got[1].body["slug"] != last {
t.Fatalf("follow-up slug = %q, want the last, %q", got[1].body["slug"], last)
}
}
t.Run("25 purges in one transaction", func(t *testing.T) {
run(t, func(p *SiteCachePurger) error {
return lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
for i := range 25 {
slug := "bulk-" + strconv.Itoa(i)
if err := tx.Exec(`INSERT INTO purge_posts (slug) VALUES (?)`, slug).Error; err != nil {
return err
}
PurgeSiteCacheAfterCommit(ctx, tx, p, "post", slug)
}
return nil
})
}, "bulk-24")
})
t.Run("10 separate transactions", func(t *testing.T) {
run(t, func(p *SiteCachePurger) error {
for i := range 10 {
slug := "each-" + strconv.Itoa(i)
if err := lagoon.Transaction(ctx, gdb, func(ctx context.Context, tx *gorm.DB) error {
PurgeSiteCacheAfterCommit(ctx, tx, p, "post", slug)
return tx.Exec(`INSERT INTO purge_posts (slug) VALUES (?)`, slug).Error
}); err != nil {
return err
}
}
return nil
}, "each-9")
})
}

View File

@@ -0,0 +1,160 @@
package cooler
import (
"context"
"fmt"
"strings"
"testing"
"time"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/wait"
)
// Smoke tests only; plan 05 of phase 19.3 owns full coverage.
func TestOpenRedisRedactsSmoke(t *testing.T) {
ctx := t.Context()
_, err := OpenRedis(ctx, "redis://alice:s3cretpw@cache.example.com:6379/0\x7f")
if err == nil {
t.Fatal("a URL with a control character must not parse")
}
msg := err.Error()
if strings.Contains(msg, "alice") || strings.Contains(msg, "s3cretpw") {
t.Fatalf("error leaks credentials: %q", msg)
}
if !strings.Contains(msg, RedisURLKey) {
t.Fatalf("error should name %s: %q", RedisURLKey, msg)
}
c, err := OpenRedis(ctx, "rediss://user:pw@cache.example.com:6380/1")
if err != nil {
t.Fatalf("rediss URL: %v", err)
}
defer c.Close()
opt := c.Options()
if opt.TLSConfig == nil || opt.Addr != "cache.example.com:6380" || opt.DB != 1 {
t.Fatalf("rediss options = addr %q db %d tls %v", opt.Addr, opt.DB, opt.TLSConfig != nil)
}
if !opt.DisableIdentity {
t.Fatal("OpenRedis must disable the CLIENT SETINFO handshake")
}
}
// cacheRoundTrip runs the shared smoke sequence against one driver.
func cacheRoundTrip(t *testing.T, c Cache) {
t.Helper()
ctx := t.Context()
if err := c.Set(ctx, "greeting", []byte("hello"), 0); err != nil {
t.Fatal(err)
}
if v, ok, err := c.Get(ctx, "greeting"); err != nil || !ok || string(v) != "hello" {
t.Fatalf("Get = %q %v %v", v, ok, err)
}
if ok, err := c.Has(ctx, "greeting"); err != nil || !ok {
t.Fatalf("Has = %v %v", ok, err)
}
if n, err := c.Increment(ctx, "hits", 2); err != nil || n != 2 {
t.Fatalf("Increment from missing = %d %v", n, err)
}
if n, err := c.Increment(ctx, "hits", 3); err != nil || n != 5 {
t.Fatalf("Increment existing = %d %v", n, err)
}
calls := 0
fn := func(context.Context) ([]byte, error) { calls++; return []byte("computed"), nil }
for range 2 {
v, err := c.Remember(ctx, "remembered", time.Minute, fn)
if err != nil || string(v) != "computed" {
t.Fatalf("Remember = %q %v", v, err)
}
}
if calls != 1 {
t.Fatalf("Remember called the callback %d times, want 1", calls)
}
if err := c.Forget(ctx, "greeting"); err != nil {
t.Fatal(err)
}
if ok, _ := c.Has(ctx, "greeting"); ok {
t.Fatal("Forget left the key")
}
if err := c.Set(ctx, "short", []byte("x"), time.Millisecond); err != nil {
t.Fatal(err)
}
time.Sleep(20 * time.Millisecond)
if _, ok, _ := c.Get(ctx, "short"); ok {
t.Fatal("a 1 ms TTL did not expire")
}
if err := c.Flush(ctx); err != nil {
t.Fatal(err)
}
for _, k := range []string{"hits", "remembered"} {
if ok, _ := c.Has(ctx, k); ok {
t.Fatalf("Flush left %q", k)
}
}
}
func TestMemoryCacheSmoke(t *testing.T) {
c, err := NewMemoryCache("acme:", DefaultMaxEntries)
if err != nil {
t.Fatal(err)
}
cacheRoundTrip(t, c)
// Stored bytes are copies.
ctx := t.Context()
in := []byte("abc")
_ = c.Set(ctx, "copy", in, 0)
in[0] = 'X'
out, _, _ := c.Get(ctx, "copy")
out[1] = 'Y'
if again, _, _ := c.Get(ctx, "copy"); string(again) != "abc" {
t.Fatalf("stored value mutated: %q", again)
}
if _, err := NewMemoryCache("acme:", 0); err == nil || !strings.Contains(err.Error(), CacheMaxEntriesKey) {
t.Fatalf("max entries 0: %v", err)
}
}
func TestRedisCacheSmoke(t *testing.T) {
if testing.Short() {
t.Skip("requires testcontainers redis")
}
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Minute)
defer cancel()
ctr, err := testcontainers.Run(ctx, "redis:7-alpine",
testcontainers.WithExposedPorts("6379/tcp"),
testcontainers.WithWaitStrategy(wait.ForListeningPort("6379/tcp")),
)
if err != nil {
t.Fatalf("redis: testcontainers: %v", err)
}
t.Cleanup(func() { _ = testcontainers.TerminateContainer(ctr) })
host, err := ctr.Host(ctx)
if err != nil {
t.Fatal(err)
}
port, err := ctr.MappedPort(ctx, "6379/tcp")
if err != nil {
t.Fatal(err)
}
client, err := OpenRedis(ctx, fmt.Sprintf("redis://%s:%s/0", host, port.Port()))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = client.Close() })
// A rate-limit counter of this app and another app's cache key survive Flush.
if err := client.Set(ctx, "acme:rl:login|192.0.2.1", 3, 0).Err(); err != nil {
t.Fatal(err)
}
if err := client.Set(ctx, "other:cache:greeting", "theirs", 0).Err(); err != nil {
t.Fatal(err)
}
cacheRoundTrip(t, NewRedisCache(client, "acme:"))
for _, k := range []string{"acme:rl:login|192.0.2.1", "other:cache:greeting"} {
if n, err := client.Exists(ctx, k).Result(); err != nil || n != 1 {
t.Fatalf("Flush removed %q (exists=%d, err=%v)", k, n, err)
}
}
}

View File

@@ -10,7 +10,7 @@ Postgres data layer: the shared GORM connection, per-plugin migrations, model he
## Overview
`lagoon` is the WinterCMS models and migrations counterpart: it opens the one `*sql.DB` pool (pgx stdlib driver) that GORM and the rest of the application share, runs each plugin's gormigrate set in its own history table, and ports the Eloquent model conventions that WinterCMS plugins rely on, such as `$fillable`, `$hidden`, `$jsonable`, encrypted casts and Laravel-style validation rules. Its `attach` subpackage ports WinterCMS's `system_files` attachments, storing originals and lazily generated thumbnails in a gocloud.dev blob bucket. The application binary gets the migrate and key commands from `lagoon.RuntimeCommands`.
`lagoon` is the WinterCMS models and migrations counterpart: it opens the one `*sql.DB` pool (pgx stdlib driver) that GORM and the rest of the application share, runs each plugin's gormigrate set in its own history table, and ports the Eloquent model conventions that WinterCMS plugins rely on, such as `$fillable`, `$hidden`, `$jsonable`, encrypted casts and Laravel-style validation rules. Its `attach` subpackage ports WinterCMS's `system_files` attachments, storing originals and lazily generated thumbnails in a gocloud.dev blob bucket (a directory, memory, or S3 in binaries built with the `summercms_s3` tag). The application binary gets the migrate and key commands from `lagoon.RuntimeCommands`.
## Role in the framework
@@ -38,7 +38,7 @@ Use cases:
- Lifecycle and relations: hook interfaces matching GORM's native method names (`lagoon.HasBeforeCreate`, `lagoon.HasBeforeSave`, `lagoon.HasBeforeDelete`, `lagoon.HasAfterDelete`) plus `lagoon.HasBeforeValidate`; `lagoon.WithSoftDeleteCascade` runs a cascade inside the parent delete; `lagoon.RegisterJoinTable` wires pivot models with business columns.
- Deferred binding: WinterCMS's `deferred_bindings` table holds the uploads and related-record changes of a form whose record is not saved yet. Every operation takes a `lagoon.DeferredKey` (the form's session key, the owning backend admin's id and the master record's morph type from `lagoon.MorphType`) and never reads or changes another admin's rows, since each row stores `backend_user_id`. `lagoon.DeferredBind` and `lagoon.DeferredUnbind` port WinterCMS's duplicate and cancel rules: a repeated bind writes nothing, and an unbind of a slave with a pending bind deletes that bind and returns it so the caller can remove what it created. `lagoon.DeferredBindings` reads and locks a session's bindings for the save that commits them, `lagoon.DeferredForget` deletes them once applied, and `lagoon.DeferredSlaves` is the subquery a list uses to include pending rows. A child created under deferral carries the `lagoon.DeferredEnvelope` (`{"created":true,"pivot":{...},"upload_id":"..."}`) in `pivot_data`. `lagoon.PurgeDeferred` removes expired bindings: it deletes an unattached `system_files` row a bind points at, and its blobs only after the commit, deletes a child only when its binding carries the created envelope, keeps records that were only linked, and locks each batch with `FOR UPDATE SKIP LOCKED`.
- Imports from Laravel: `lagoon.DecryptLaravelPayload` decrypts Laravel `encrypted` payloads with the old application key, for one-off data imports.
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), public URLs (`attach.PublicURL` for any key, `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since WebP cannot be encoded; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming and, in image mode, the `attach.IsAllowedImage` content guard), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving with an optional `is_public` gate (`attach.StaticHandlerPublic`), and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
- Attachments (`attach`): the `attach.File` model for `system_files` rows, WinterCMS-compatible partitioned storage keys (`attach.BlobKey`, `attach.PartitionDirectory`), an S3 bucket option behind the `attach.S3BuildTag` build tag (an untagged binary refuses an `s3://` bucket URL at boot with an error naming the tag), public URLs (`attach.PublicURL` for any key, under `storage.uploads.public_url` when a public bucket or CDN serves the files (`attach.PublicURLBase`), `attach.File.URL` for an original, matching WinterCMS's `File::getPath()` under the WinterCMS layout), on-demand thumbnails through `attach.File.Thumb` for JPEG, PNG, GIF and WebP originals (a WebP original's thumbnail is JPEG bytes under its `.webp` name, since WebP cannot be encoded; a missing, undecodable or oversized original gets WinterCMS's broken-image picture, `attach.BrokenImagePNG`, as its thumbnail, as `File::makeThumb` does), storing uploads through `attach.Store` (a server-generated disk name whose extension follows the sniffed content through `attach.DiskExtension`, so a PNG named `x.html` is stored as `.png`, an extension allow-list with `attach.DefaultImageExtensions` and `attach.DefaultFileExtensions` as defaults, a MIME filter, a size limit enforced while streaming, in image mode the `attach.IsAllowedImage` content guard, and the refusal of protected uploads while `storage.uploads.public_url` serves the bucket publicly, `attach.ErrProtectedNeedsPrivateBucket`), attachment relation declarations (`attach.Relation`, `attach.HasRelations`), static serving with an optional `is_public` gate (`attach.StaticHandlerPublic`), and a two-phase delete that removes blobs only after the database transaction commits (`attach.DeleteForOwner`, `attach.DeleteKeys`).
## Usage
@@ -191,23 +191,27 @@ func (p *Plugin) Migrations() []*gormigrate.Migration {
| `lagoon.RegisterJoinTable` | Registers a custom pivot model for a many-to-many field. |
| `lagoon.DecryptLaravelPayload` | Decrypts a Laravel AES-256-CBC payload for data imports. |
| `attach.File` | The `system_files` row model; `attach.File.URL` is the public URL of the original. |
| `attach.PublicURL` | Public URL of a blob key under `storage.uploads.public_path_prefix`. |
| `attach.PublicURL` | Public URL of a blob key: under `storage.uploads.public_url` when set, else under `storage.uploads.public_path_prefix`. |
| `attach.PublicURLBase` | `storage.uploads.public_url` as recorded by `attach.OpenBucket`, empty when unset. |
| `attach.BrokenImagePNG` | WinterCMS's broken-image picture, stored as the thumbnail of an unusable original. |
| `attach.Owner` | Implemented by models that own attachments; returns the stored morph type name. |
| `attach.OpenBucket` | Opens the uploads bucket from config. |
| `attach.OpenBucket` | Opens the uploads bucket from config; refuses `s3://` in a binary built without `attach.S3BuildTag`. |
| `attach.S3BuildTag` | The build tag, `summercms_s3`, that links the S3 driver; list it under `build.tags` in `summer.yaml`. |
| `attach.Publish` | Stores the bucket on the `backpack.App`. |
| `attach.StaticHandler` | Serves stored files and thumbnails under a URL prefix. |
| `attach.StaticHandlerPublic` | `attach.StaticHandler` plus a 404 for rows that are not public. |
| `attach.DeleteForOwner` | Deletes an owner's attachment rows in a transaction and reports their blob keys. |
| `attach.DeleteKeys` | Deletes blobs, including thumbnails, after the transaction commits. |
| `attach.Migrations` | Creates the `system_files` table. |
| `attach.Store` | Stores an upload as an unattached `system_files` row with `sort_order` equal to its id, after the type, size and image checks. |
| `attach.Store` | Stores an upload as an unattached `system_files` row with `sort_order` equal to its id, after the type, size and image checks; the disk name's extension comes from `attach.DiskExtension`, `file_name` keeps the client's name. |
| `attach.DiskExtension` | The stored extension for a sniffed content type and the client's extension: the client's when it matches a specific type, else that type's canonical one; under a generic sniff the client's unless it is script-capable (then `txt` or `bin`). |
| `attach.Upload` | The client file name, body and public flag of one upload. |
| `attach.Limits` | Size limit, allowed extensions, allowed MIME types and image mode for `attach.Store`. |
| `attach.ErrTooLarge` | Returned by `attach.Store` for a body over `MaxBytes`. |
| `attach.ErrFileType` | Returned by `attach.Store` for a missing, malformed or disallowed extension. |
| `attach.ErrMIMEType` | Returned by `attach.Store` when the content type matches no `MIMETypes` entry. |
| `attach.ErrNotImage` | Returned by `attach.Store` in image mode for content that is not an allowed image. |
| `attach.ErrProtectedNeedsPrivateBucket` | Returned by `attach.Store` for a protected (`Public` false) upload while `storage.uploads.public_url` is set, before any blob is written: that bucket is served publicly. |
| `attach.DefaultImageExtensions` | Image-mode extensions when none are given: jpg, jpeg, png, gif, webp. |
| `attach.DefaultFileExtensions` | File-mode extensions when none are given: WinterCMS's default list without the script-capable types. |
| `attach.AllowedImageMIMEs` | The sniffed content types the image guard accepts. |
@@ -227,8 +231,9 @@ Keys are read from the compass config; the `SUMMER_` environment overlay maps a
| `database.dsn` | none (required) | Postgres connection string used by `lagoon.OpenFromApp` and every database command. |
| `app.key` | none (required) | Base64 encoding of 32 random bytes; the source of the `lagoon.Encrypted` column key. Generate one with `key:generate`. |
| `app.previous_keys` | empty | List of earlier base64 keys still accepted when decrypting, for key rotation. |
| `storage.uploads.bucket_url` | none (required by `attach.OpenBucket`) | Uploads bucket URL, `file://` or `mem://`. |
| `storage.uploads.bucket_url` | none (required by `attach.OpenBucket`) | Uploads bucket URL, `file://`, `mem://` or `s3://` (with `attach.S3BuildTag`; query parameters such as `region`, `endpoint` and `use_path_style` per gocloud.dev, credentials from the AWS environment). |
| `storage.uploads.public_path_prefix` | `/storage/uploads` | URL prefix used when building public file and thumbnail URLs. |
| `storage.uploads.public_url` | empty | Absolute `http` or `https` base of a public bucket or CDN; when set, file and thumbnail URLs are this base plus the partition path. A malformed value fails `attach.OpenBucket`; a set value logs a warning that the bucket must hold only public files, and `attach.Store` refuses protected uploads (`attach.ErrProtectedNeedsPrivateBucket`). |
| `database.deferred_bindings.purge_days` | `5` | Age in days after which `deferred:purge` removes a deferred binding, WinterCMS's `cleanUp(5)`; the `--days` flag overrides it. |
| `database.deferred_bindings.purge_at` | `03:00` | Daily `HH:MM` (in `app.timezone`) at which the scheduler runs `deferred:purge`; an empty value disables the framework entry. |
@@ -258,7 +263,7 @@ storage:
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [compass](../compass/README.md), [pact](../pact/README.md), [party](../party/README.md), [phrasebook](../phrasebook/README.md) (validation messages).
- Third-party: `gorm.io/gorm`, `gorm.io/driver/postgres`, `github.com/jackc/pgx/v5` (stdlib driver), `github.com/go-gormigrate/gormigrate/v2`, `github.com/go-playground/validator/v10`, `github.com/riverqueue/river` (its `rivermigrate` and `riverdriver/riverdatabasesql` packages, for the River schema in `lagoon.QueueMigrations`).
- Third-party, `attach` only: `gocloud.dev/blob` (file and memory drivers), `github.com/disintegration/imaging` (thumbnails), `golang.org/x/image/webp` (WebP decoding).
- Third-party, `attach` only: `gocloud.dev/blob` (file and memory drivers; the S3 driver `gocloud.dev/blob/s3blob` and the AWS SDK v2 only under the `summercms_s3` build tag), `github.com/disintegration/imaging` (thumbnails), `golang.org/x/image/webp` (WebP decoding).
- Standard library: `database/sql`, `crypto/aes`, `crypto/cipher`, `crypto/hkdf`, `log/slog`, `image`, among others.
## Testing

View File

@@ -3,6 +3,7 @@ package attach
import (
"context"
"fmt"
"log/slog"
"net/url"
"strings"
"sync"
@@ -16,9 +17,15 @@ import (
const defaultPublicPathPrefix = "/storage/uploads"
// S3BuildTag is the Go build tag that links the gocloud S3 driver
// (s3blob, imported only by s3_on.go). List it under build.tags in summer.yaml so
// summer build passes it to go build; a raw go build needs -tags summercms_s3.
const S3BuildTag = "summercms_s3"
var (
prefixMu sync.RWMutex
publicPathPrefix = defaultPublicPathPrefix
publicURLBase string
)
func setPublicPathPrefix(prefix string) {
@@ -27,6 +34,38 @@ func setPublicPathPrefix(prefix string) {
prefixMu.Unlock()
}
func setPublicURLBase(base string) {
prefixMu.Lock()
publicURLBase = base
prefixMu.Unlock()
}
// PublicURLBase is storage.uploads.public_url as recorded by OpenBucket: an
// absolute http or https base (a public bucket or a CDN) that file and
// thumbnail URLs start with instead of PublicPathPrefix. Empty when unset.
func PublicURLBase() string {
prefixMu.RLock()
defer prefixMu.RUnlock()
return publicURLBase
}
// parsePublicURL validates storage.uploads.public_url: empty means unset,
// anything else must be an absolute http or https URL with a host and
// without user info, query or fragment.
func parsePublicURL(raw string) (*url.URL, error) {
value := strings.TrimSpace(raw)
if value == "" {
return nil, nil
}
u, err := url.Parse(value)
bad := err != nil || u.Host == "" || u.User != nil || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" ||
(!strings.EqualFold(u.Scheme, "http") && !strings.EqualFold(u.Scheme, "https"))
if bad {
return nil, fmt.Errorf("attach: storage.uploads.public_url must be an absolute http or https URL such as https://cdn.example.com/uploads, without user info, query or fragment (set SUMMER_STORAGE__UPLOADS__PUBLIC_URL)")
}
return u, nil
}
// PublicPathPrefix is the URL prefix prepended to partition+filename.
func PublicPathPrefix() string {
prefixMu.RLock()
@@ -57,8 +96,27 @@ func normalizeBucketURL(raw string) (string, error) {
}
}
// OpenBucket opens storage.uploads.bucket_url (file:// or mem://) and
// records storage.uploads.public_path_prefix. An empty bucket_url fails boot.
// checkS3Driver refuses an s3:// bucket URL when the binary was built
// without S3BuildTag, naming the tag and where to set it.
func checkS3Driver(bucketURL string) error {
if s3Enabled {
return nil
}
scheme, _, ok := strings.Cut(bucketURL, "://")
if !ok || !strings.EqualFold(scheme, "s3") {
return nil
}
return fmt.Errorf("attach: storage.uploads.bucket_url uses s3:// but this binary was built without the %s build tag; "+
"list %s under build.tags in summer.yaml and rebuild with summer build (a raw go build needs -tags %s, or GOFLAGS=-tags=%s)",
S3BuildTag, S3BuildTag, S3BuildTag, S3BuildTag)
}
// OpenBucket opens storage.uploads.bucket_url (file://, mem://, or s3:// in a
// binary built with S3BuildTag) and records
// storage.uploads.public_path_prefix and storage.uploads.public_url. An
// empty bucket_url or a malformed public_url fails boot; a set public_url
// logs one warning saying the bucket behind it must hold only public files
// and that Store refuses protected uploads (ErrProtectedNeedsPrivateBucket).
func OpenBucket(ctx context.Context, cfg *compass.Config) (*blob.Bucket, error) {
if cfg == nil {
return nil, fmt.Errorf("attach: config is nil")
@@ -71,6 +129,13 @@ func OpenBucket(ctx context.Context, cfg *compass.Config) (*blob.Bucket, error)
if err != nil {
return nil, err
}
if err := checkS3Driver(bucketURL); err != nil {
return nil, err
}
publicURL, err := parsePublicURL(cfg.String("storage.uploads.public_url"))
if err != nil {
return nil, err
}
bucket, err := blob.OpenBucket(ctx, bucketURL)
if err != nil {
return nil, fmt.Errorf("attach: open bucket %q: %w", raw, err)
@@ -80,6 +145,15 @@ func OpenBucket(ctx context.Context, cfg *compass.Config) (*blob.Bucket, error)
prefix = defaultPublicPathPrefix
}
setPublicPathPrefix(prefix)
if publicURL == nil {
setPublicURLBase("")
return bucket, nil
}
setPublicURLBase(publicURL.String())
slog.Default().WarnContext(ctx, "attach: storage.uploads.public_url is set; every file and thumbnail URL points at this host, "+
"which serves any key without the is_public check, so the bucket behind it must hold only public files: "+
"protected (is_public false) uploads are refused",
slog.String("host", publicURL.Host))
return bucket, nil
}

View File

@@ -0,0 +1,78 @@
package attach
import (
"slices"
"strings"
)
// sniffedExtensions maps a specific sniffed content type to the extensions
// that may name it on disk; the first entry is the canonical one. The table
// is fixed in code on purpose: package mime's answers depend on the host's
// mime tables.
var sniffedExtensions = map[string][]string{
"image/png": {"png"},
"image/jpeg": {"jpg", "jpeg", "jpe"},
"image/gif": {"gif"},
"image/webp": {"webp"},
"image/bmp": {"bmp"},
"image/x-icon": {"ico"},
"image/vnd.microsoft.icon": {"ico"},
"application/pdf": {"pdf"},
"audio/mpeg": {"mp3"},
"audio/wave": {"wav"},
"audio/wav": {"wav"},
"application/ogg": {"ogg", "oga", "opus"},
"audio/ogg": {"ogg", "oga", "opus"},
"video/webm": {"webm", "mkv"},
"video/mp4": {"mp4", "m4v"},
"font/woff2": {"woff2"},
"font/woff": {"woff"},
"font/ttf": {"ttf"},
"font/otf": {"otf"},
"application/vnd.ms-fontobject": {"eot"},
"application/x-rar-compressed": {"rar"},
"application/x-gzip": {"gz", "tgz"},
}
// scriptCapableExtensions are extensions a browser or web server may run or
// render as a document. They are never kept on disk under a generic sniff.
var scriptCapableExtensions = []string{
"html", "htm", "xhtml", "xht", "shtml", "svg", "svgz", "xml", "xsl", "xslt",
"js", "mjs", "css", "swf", "php", "phtml", "pht", "phar",
}
// DiskExtension returns the extension, without the dot, under which an
// upload is stored, so a web server that picks the Content-Type from the
// extension serves what the bytes are rather than what the client named
// them. contentType is the sniffed base media type (http.DetectContentType
// without parameters) and clientExt the client's allow-listed extension.
//
// For a specific sniffed type (PNG, JPEG, GIF, WebP, PDF, MP3, WebM, fonts
// and so on) the client extension is kept when it belongs to that type,
// else the type's canonical extension is used: a PNG named x.html is stored
// as .png. Any other type is generic (application/octet-stream,
// application/zip, text/plain, text/html, text/xml): the client extension
// is kept, so .docx, .xlsx and .ics survive, unless it is script-capable
// (html, svg, xml, js, css, php and the like), which becomes txt for a
// text/* type and bin otherwise.
func DiskExtension(contentType, clientExt string) string {
ct := strings.ToLower(strings.TrimSpace(contentType))
if base, _, ok := strings.Cut(ct, ";"); ok {
ct = strings.TrimSpace(base)
}
ext := strings.ToLower(strings.TrimPrefix(strings.TrimSpace(clientExt), "."))
if exts, ok := sniffedExtensions[ct]; ok {
if slices.Contains(exts, ext) {
return ext
}
return exts[0]
}
if extPattern.MatchString(ext) && !slices.Contains(scriptCapableExtensions, ext) {
return ext
}
if strings.HasPrefix(ct, "text/") {
return "txt"
}
return "bin"
}

View File

@@ -0,0 +1,111 @@
package attach
import (
"net/http"
"slices"
"testing"
)
// TestDiskExtensionTable (plan 05 of phase 19.3, D-15) pins DiskExtension on
// the RESEARCH probe rows and on the consistency of its tables: a client
// extension that belongs to the sniffed type is kept, any other becomes the
// type's canonical one, and a script-capable extension under a generic type
// becomes txt (text/*) or bin.
func TestDiskExtensionTable(t *testing.T) {
for _, tc := range []struct {
contentType, clientExt, want string
}{
// RESEARCH probe rows.
{"image/png", "html", "png"},
{"image/jpeg", "jpeg", "jpeg"},
{"image/jpeg", "jpg", "jpg"},
{"image/jpeg", "png", "jpg"},
{"video/webm", "mkv", "mkv"},
{"video/webm", "webm", "webm"},
{"application/zip", "docx", "docx"},
{"application/zip", "xlsx", "xlsx"},
{"text/plain; charset=utf-8", "ics", "ics"},
{"text/plain; charset=utf-8", "html", "txt"},
{"text/xml; charset=utf-8", "svg", "txt"},
{"application/octet-stream", "js", "bin"},
{"application/pdf", "txt", "pdf"},
// HTML bytes under their own name never stay html.
{"text/html; charset=utf-8", "html", "txt"},
{"text/html; charset=utf-8", "txt", "txt"},
// Normalisation: case, a leading dot, parameters and spaces.
{"IMAGE/PNG", ".PNG", "png"},
{" image/jpeg ; q=1", "JPE", "jpe"},
{"application/octet-stream", ".Docx", "docx"},
// Extensions outside [a-z0-9]{1,10} fall back like a script-capable one.
{"application/octet-stream", "", "bin"},
{"text/plain", "", "txt"},
{"application/octet-stream", "tar.gz", "bin"},
{"text/plain", "averyverylongext", "txt"},
{"application/octet-stream", "ex e", "bin"},
// Other specific types.
{"image/gif", "png", "gif"},
{"image/webp", "webp", "webp"},
{"application/ogg", "opus", "opus"},
{"audio/mpeg", "wav", "mp3"},
{"font/woff2", "woff2", "woff2"},
{"application/x-gzip", "tgz", "tgz"},
} {
if got := DiskExtension(tc.contentType, tc.clientExt); got != tc.want {
t.Errorf("DiskExtension(%q, %q) = %q, want %q", tc.contentType, tc.clientExt, got, tc.want)
}
}
// Every specific type keeps each of its own extensions, maps a
// script-capable one to its canonical extension, and lists no
// script-capable extension itself.
for ct, exts := range sniffedExtensions {
if len(exts) == 0 {
t.Fatalf("%s has no extensions", ct)
}
for _, ext := range exts {
if !extPattern.MatchString(ext) {
t.Errorf("%s: extension %q does not match the disk name pattern", ct, ext)
}
if slices.Contains(scriptCapableExtensions, ext) {
t.Errorf("%s: script-capable extension %q is kept", ct, ext)
}
if got := DiskExtension(ct, ext); got != ext {
t.Errorf("DiskExtension(%q, %q) = %q", ct, ext, got)
}
}
for _, bad := range scriptCapableExtensions {
if got := DiskExtension(ct, bad); got != exts[0] {
t.Errorf("DiskExtension(%q, %q) = %q, want %q", ct, bad, got, exts[0])
}
}
}
// Under every generic type a script-capable extension is rewritten.
for _, ct := range []string{"text/plain; charset=utf-8", "text/html; charset=utf-8", "text/xml; charset=utf-8", "text/css"} {
for _, bad := range scriptCapableExtensions {
if got := DiskExtension(ct, bad); got != "txt" {
t.Errorf("DiskExtension(%q, %q) = %q, want txt", ct, bad, got)
}
}
}
for _, ct := range []string{"application/octet-stream", "application/zip", "application/x-msdownload", ""} {
for _, bad := range scriptCapableExtensions {
if got := DiskExtension(ct, bad); got != "bin" {
t.Errorf("DiskExtension(%q, %q) = %q, want bin", ct, bad, got)
}
}
}
// The table agrees with what the standard library actually sniffs.
for body, want := range map[string]string{
"\x89PNG\r\n\x1a\n": "png",
"%PDF-1.7\n": "pdf",
"<html><body>x</body></html>": "txt",
"<?xml version=\"1.0\"?><svg xmlns=\"http://www.w3.org/2000/svg\"/>": "txt",
} {
ct := http.DetectContentType([]byte(body))
if got := DiskExtension(ct, "html"); got != want {
t.Errorf("body %q sniffed %q: DiskExtension = %q, want %q", body, ct, got, want)
}
}
}

View File

@@ -0,0 +1,106 @@
package attach
import (
"os"
"path/filepath"
"strings"
"testing"
"git.golem15.com/golem15/summercms/modules/compass"
)
// openTestConfig writes storage.yaml into a temp config dir and opens it.
func openTestConfig(t *testing.T, storageYAML string) *compass.Config {
t.Helper()
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "storage.yaml"), []byte(storageYAML), 0o644); err != nil {
t.Fatal(err)
}
cfg, err := compass.Open(compass.Options{
Dir: dir,
Env: "development",
Environ: []string{"SUMMER_ENV=development"},
})
if err != nil {
t.Fatal(err)
}
return cfg
}
// TestOpenBucketS3Smoke runs in both tag modes: without summercms_s3 an
// s3:// bucket_url fails boot naming the tag, with it the bucket opens
// (s3blob connects lazily, so no network is needed).
func TestOpenBucketS3Smoke(t *testing.T) {
t.Setenv("AWS_ACCESS_KEY_ID", "smoke-access-key")
t.Setenv("AWS_SECRET_ACCESS_KEY", "smoke-secret-key")
t.Setenv("AWS_REGION", "us-east-1")
t.Setenv("AWS_EC2_METADATA_DISABLED", "true")
t.Cleanup(func() { setPublicPathPrefix(defaultPublicPathPrefix) })
cfg := openTestConfig(t, "uploads:\n bucket_url: \"s3://smoke-bucket?region=us-east-1\"\n")
bucket, err := OpenBucket(t.Context(), cfg)
if !s3Enabled {
if err == nil {
_ = bucket.Close()
t.Fatal("OpenBucket opened s3:// without the summercms_s3 tag")
}
for _, want := range []string{S3BuildTag, "build.tags", "-tags " + S3BuildTag} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("error %q does not name %q", err, want)
}
}
return
}
if err != nil {
t.Fatalf("OpenBucket with the summercms_s3 tag: %v", err)
}
if err := bucket.Close(); err != nil {
t.Fatalf("close: %v", err)
}
}
// TestPublicURLSmoke checks storage.uploads.public_url (D-12): set, URLs
// are the base plus the partition path; unset, they keep the proxy prefix;
// malformed, OpenBucket fails naming the key.
func TestPublicURLSmoke(t *testing.T) {
t.Cleanup(func() {
setPublicPathPrefix(defaultPublicPathPrefix)
setPublicURLBase("")
})
key := BlobKey("651a2b3c4d5e61234567.png")
cfg := openTestConfig(t, "uploads:\n bucket_url: \"mem://\"\n public_url: \"https://cdn.example.com/uploads/\"\n")
bucket, err := OpenBucket(t.Context(), cfg)
if err != nil {
t.Fatal(err)
}
_ = bucket.Close()
if got, want := PublicURL(key), "https://cdn.example.com/uploads/"+key; got != want {
t.Fatalf("PublicURL with public_url = %q, want %q", got, want)
}
if got, want := (&File{DiskName: "651a2b3c4d5e61234567.png"}).URL(), "https://cdn.example.com/uploads/651/a2b/3c4/651a2b3c4d5e61234567.png"; got != want {
t.Fatalf("File.URL with public_url = %q, want %q", got, want)
}
cfg = openTestConfig(t, "uploads:\n bucket_url: \"mem://\"\n")
bucket, err = OpenBucket(t.Context(), cfg)
if err != nil {
t.Fatal(err)
}
_ = bucket.Close()
if PublicURLBase() != "" || !strings.HasPrefix(PublicURL(key), "/storage/uploads/") {
t.Fatalf("without public_url: base %q, URL %q", PublicURLBase(), PublicURL(key))
}
for _, bad := range []string{"ftp://x", "https://u:p@cdn.example.com", "/storage/uploads", "https://cdn.example.com/?v=1"} {
cfg = openTestConfig(t, "uploads:\n bucket_url: \"mem://\"\n public_url: \""+bad+"\"\n")
bucket, err = OpenBucket(t.Context(), cfg)
if err == nil {
_ = bucket.Close()
t.Fatalf("public_url %q accepted", bad)
}
if !strings.Contains(err.Error(), "storage.uploads.public_url") || strings.Contains(err.Error(), "u:p") {
t.Fatalf("public_url %q: error %q", bad, err)
}
}
}

View File

@@ -0,0 +1,7 @@
//go:build !summercms_s3
package attach
// Without the summercms_s3 tag the S3 driver is not linked; OpenBucket refuses
// an s3:// bucket_url with an error naming the tag.
const s3Enabled = false

View File

@@ -0,0 +1,9 @@
//go:build summercms_s3
package attach
// Binaries built with the summercms_s3 tag (summer.yaml build.tags) link the
// gocloud S3 driver, so storage.uploads.bucket_url may use s3://.
import _ "gocloud.dev/blob/s3blob"
const s3Enabled = true

View File

@@ -0,0 +1,68 @@
package attach
import (
"strings"
"testing"
)
// TestOpenBucketS3 (plan 05 of phase 19.3, D-13) has no build constraint: it
// runs in both modes and branches on s3Enabled. Without the summercms_s3 tag
// an s3:// bucket_url (any case of the scheme) fails boot naming the tag and
// where to set it; with the tag the bucket opens on dummy AWS credentials
// (s3blob connects lazily, so nothing leaves the host) and closes.
func TestOpenBucketS3(t *testing.T) {
t.Setenv("AWS_ACCESS_KEY_ID", "unit-access-key")
t.Setenv("AWS_SECRET_ACCESS_KEY", "unit-secret-key")
t.Setenv("AWS_REGION", "eu-central-1")
t.Setenv("AWS_EC2_METADATA_DISABLED", "true")
t.Cleanup(func() {
setPublicPathPrefix(defaultPublicPathPrefix)
setPublicURLBase("")
})
if S3BuildTag != "summercms_s3" {
t.Fatalf("S3BuildTag = %q", S3BuildTag)
}
for _, raw := range []string{
"s3://unit-bucket?region=eu-central-1",
"S3://unit-bucket?region=eu-central-1",
"s3://unit-bucket?region=auto&endpoint=https://storage.example.com&use_path_style=true",
} {
t.Run(raw, func(t *testing.T) {
cfg := openTestConfig(t, "uploads:\n bucket_url: \""+raw+"\"\n")
bucket, err := OpenBucket(t.Context(), cfg)
if !s3Enabled {
if err == nil {
_ = bucket.Close()
t.Fatal("s3:// opened without the summercms_s3 tag")
}
for _, want := range []string{"storage.uploads.bucket_url", S3BuildTag, "build.tags", "summer.yaml", "-tags " + S3BuildTag, "GOFLAGS=-tags=" + S3BuildTag} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("error %q does not name %q", err, want)
}
}
if strings.Contains(err.Error(), "unit-bucket") || strings.Contains(err.Error(), "unit-secret") {
t.Fatalf("error %q repeats the bucket URL", err)
}
return
}
if err != nil {
t.Fatalf("OpenBucket with the tag: %v", err)
}
if err := bucket.Close(); err != nil {
t.Fatalf("close: %v", err)
}
})
}
// Other schemes never reach the tag check.
for _, raw := range []string{"mem://", "file:///tmp/x", "fileblob:///tmp/x", "s3x://b", "gs://bucket", "s3:no-slashes"} {
if err := checkS3Driver(raw); err != nil {
t.Fatalf("checkS3Driver(%q) = %v", raw, err)
}
}
err := checkS3Driver("s3://b")
if s3Enabled != (err == nil) {
t.Fatalf("checkS3Driver(s3://b) = %v with s3Enabled %t", err, s3Enabled)
}
}

View File

@@ -35,6 +35,11 @@ var (
// ErrNotImage is returned by Store in image mode when the bytes are not
// an image IsAllowedImage accepts.
ErrNotImage = errors.New("attach: file is not an allowed image")
// ErrProtectedNeedsPrivateBucket is returned by Store for a protected
// (Upload.Public false) upload while storage.uploads.public_url is set:
// the host behind public_url serves every key of the bucket without the
// is_public check, so a protected file needs a private bucket.
ErrProtectedNeedsPrivateBucket = errors.New("attach: storage.uploads.public_url is set, so the bucket is served publicly and a protected (is_public false) upload needs a private bucket: unset storage.uploads.public_url or make the relation public")
)
// DefaultImageExtensions is the extension list of an image upload when
@@ -88,6 +93,10 @@ type Limits struct {
// Store saves an upload as an unattached system_files row.
//
// A protected upload (Upload.Public false) while storage.uploads.public_url
// is set returns ErrProtectedNeedsPrivateBucket before the body is read or
// any blob is written.
//
// It accepts the client extension, lower-cased, only when it matches
// [a-z0-9]{1,10} and is allowed by Limits (else ErrFileType). It reads up to
// 1 MiB ahead to sniff the content type from the bytes; in image mode those
@@ -95,7 +104,9 @@ type Limits struct {
// checked against the sniffed type, or the extension's registered type when
// the sniff only says application/octet-stream (else ErrMIMEType). The body
// is then streamed into bucket at BlobKey of a server-generated disk name (22
// random lowercase hex characters, a dot and the extension); a body longer
// random lowercase hex characters, a dot and the extension DiskExtension
// derives from the sniffed type and the client extension, so a PNG named
// x.html is stored as .png; file_name keeps the client's name); a body longer
// than Limits.MaxBytes aborts the write, deletes the key and returns
// ErrTooLarge. Finally it inserts the row with empty attachment columns,
// is_public from Upload.Public, the byte size and the content type, and sets
@@ -121,6 +132,9 @@ func Store(ctx context.Context, db *gorm.DB, bucket *blob.Bucket, in Upload, lim
if lim.MaxBytes < 0 {
return nil, fmt.Errorf("attach: negative size limit %d", lim.MaxBytes)
}
if !in.Public && PublicURLBase() != "" {
return nil, ErrProtectedNeedsPrivateBucket
}
name := clientBaseName(in.FileName)
ext := strings.ToLower(strings.TrimPrefix(path.Ext(name), "."))
if !extPattern.MatchString(ext) || !slices.Contains(allowedExtensions(lim), ext) {
@@ -138,7 +152,8 @@ func Store(ctx context.Context, db *gorm.DB, bucket *blob.Bucket, in Upload, lim
if lim.Image && !IsAllowedImage(head) {
return nil, ErrNotImage
}
contentType := baseMediaType(http.DetectContentType(head))
sniffed := baseMediaType(http.DetectContentType(head))
contentType := sniffed
if contentType == "application/octet-stream" {
if byExt := baseMediaType(mime.TypeByExtension("." + ext)); byExt != "" {
contentType = byExt
@@ -148,7 +163,7 @@ func Store(ctx context.Context, db *gorm.DB, bucket *blob.Bucket, in Upload, lim
return nil, fmt.Errorf("%w: %s", ErrMIMEType, contentType)
}
diskName, err := newDiskName(ext)
diskName, err := newDiskName(DiskExtension(sniffed, ext))
if err != nil {
return nil, err
}

View File

@@ -1,10 +1,12 @@
package attach_test
import (
"archive/zip"
"bytes"
"context"
"errors"
"image"
"image/jpeg"
"image/png"
"io"
"regexp"
@@ -284,3 +286,141 @@ func TestStore(t *testing.T) {
}
})
}
// TestStoreDiskExtensionSmoke is the stored-XSS regression (D-15): the disk
// name's extension follows the sniffed content, file_name keeps the client's
// name, and documents whose sniff is generic keep their extension.
func TestStoreDiskExtensionSmoke(t *testing.T) {
if testing.Short() {
t.Skip("requires testcontainers postgres")
}
ctx := t.Context()
gdb := attachGorm(t)
if err := lagoon.Migrate(gdb, nil); err != nil {
t.Fatal(err)
}
bucket := memblob.OpenBucket(nil)
t.Cleanup(func() { _ = bucket.Close() })
lim := attach.Limits{Extensions: []string{"html", "png", "docx", "txt"}}
// An 8-byte PNG signature plus an IHDR chunk sniffs as image/png.
pngBody := append([]byte("\x89PNG\r\n\x1a\n"), []byte("\x00\x00\x00\x0dIHDR\x00\x00\x00\x01\x00\x00\x00\x01\x08\x02\x00\x00\x00\x90wS\xde")...)
f, err := attach.Store(ctx, gdb, bucket, attach.Upload{FileName: "x.html", Body: bytes.NewReader(pngBody), Public: true}, lim)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(f.DiskName, ".png") || !strings.HasSuffix(f.URL(), ".png") {
t.Fatalf("PNG named x.html: disk name %q, URL %q, want .png", f.DiskName, f.URL())
}
if f.FileName != "x.html" || f.ContentType != "image/png" {
t.Fatalf("PNG named x.html: file_name %q, content_type %q", f.FileName, f.ContentType)
}
var zipBody bytes.Buffer
zw := zip.NewWriter(&zipBody)
w, err := zw.Create("word/document.xml")
if err != nil {
t.Fatal(err)
}
if _, err := w.Write([]byte("<w:document/>")); err != nil {
t.Fatal(err)
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
doc, err := attach.Store(ctx, gdb, bucket, attach.Upload{FileName: "report.docx", Body: bytes.NewReader(zipBody.Bytes())}, lim)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(doc.DiskName, ".docx") || doc.FileName != "report.docx" {
t.Fatalf("docx: disk name %q, file_name %q", doc.DiskName, doc.FileName)
}
note, err := attach.Store(ctx, gdb, bucket, attach.Upload{FileName: "note.txt", Body: strings.NewReader("<html><body><script>alert(1)</script></body></html>")}, lim)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(note.DiskName, ".txt") || note.FileName != "note.txt" {
t.Fatalf("HTML body named note.txt: disk name %q, file_name %q", note.DiskName, note.FileName)
}
}
// TestStoreDiskExtension (plan 05 of phase 19.3, D-15) runs the stored-XSS
// regression table through Store on Postgres: the disk name and the URL end
// in the extension DiskExtension picks for the sniffed bytes, file_name keeps
// the client's name and content_type the sniffed type.
func TestStoreDiskExtension(t *testing.T) {
if testing.Short() {
t.Skip("requires testcontainers postgres")
}
ctx := t.Context()
gdb := attachGorm(t)
if err := lagoon.Migrate(gdb, nil); err != nil {
t.Fatal(err)
}
bucket := memblob.OpenBucket(nil)
t.Cleanup(func() { _ = bucket.Close() })
lim := attach.Limits{Extensions: []string{"html", "png", "jpg", "jpeg", "docx", "txt", "ics", "svg", "js", "pdf", "mkv"}}
var jpegBody bytes.Buffer
if err := jpeg.Encode(&jpegBody, image.NewRGBA(image.Rect(0, 0, 4, 4)), nil); err != nil {
t.Fatal(err)
}
var docxBody bytes.Buffer
zw := zip.NewWriter(&docxBody)
w, err := zw.Create("word/document.xml")
if err != nil {
t.Fatal(err)
}
if _, err := w.Write([]byte("<w:document/>")); err != nil {
t.Fatal(err)
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
webm := []byte("\x1a\x45\xdf\xa3\x9f\x42\x86\x81\x01\x42\xf7\x81\x01\x42\xf2\x81\x04\x42\xf3\x81\x08\x42\x82\x84webm\x42\x87\x81\x04\x42\x85\x81\x02")
for _, tc := range []struct {
name, fileName string
body []byte
ext, ct string
}{
{"png named html", "x.html", smokePNG(t), "png", "image/png"},
{"jpeg named png", "photo.png", jpegBody.Bytes(), "jpg", "image/jpeg"},
{"jpeg named jpeg", "photo.jpeg", jpegBody.Bytes(), "jpeg", "image/jpeg"},
{"docx under zip", "report.docx", docxBody.Bytes(), "docx", "application/zip"},
{"ics under text", "event.ics", []byte("BEGIN:VCALENDAR\r\nVERSION:2.0\r\nEND:VCALENDAR\r\n"), "ics", "text/plain"},
{"html named txt", "note.txt", []byte("<html><body><script>alert(1)</script></body></html>"), "txt", "text/html"},
{"html named html", "page.html", []byte("<!DOCTYPE html><html><script>alert(1)</script></html>"), "txt", "text/html"},
{"svg under xml", "logo.svg", []byte(`<?xml version="1.0"?><svg xmlns="http://www.w3.org/2000/svg"><script>alert(1)</script></svg>`), "txt", "text/xml"},
{"js under text", "app.js", []byte("console.log(document.cookie)\n"), "txt", "text/plain"},
// A generic sniff keeps the content_type rule (from the extension).
{"js under octet-stream", "app.js", []byte{0x00, 0x01, 0x02, 'a', 'l', 'e', 'r', 't'}, "bin", "text/javascript"},
{"pdf named txt", "doc.txt", []byte("%PDF-1.4\n%\xe2\xe3\xcf\xd3\n1 0 obj\n<<>>\nendobj\n"), "pdf", "application/pdf"},
{"mkv under webm", "clip.mkv", webm, "mkv", "video/webm"},
} {
t.Run(tc.name, func(t *testing.T) {
f, err := attach.Store(ctx, gdb, bucket, attach.Upload{FileName: tc.fileName, Body: bytes.NewReader(tc.body), Public: true}, lim)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(f.DiskName, "."+tc.ext) || !strings.HasSuffix(f.URL(), "/"+f.DiskName) || !strings.HasSuffix(f.URL(), "."+tc.ext) {
t.Fatalf("disk name %q URL %q, want .%s", f.DiskName, f.URL(), tc.ext)
}
if f.FileName != tc.fileName || !strings.HasPrefix(f.ContentType, tc.ct) {
t.Fatalf("file_name %q content_type %q, want %q %q", f.FileName, f.ContentType, tc.fileName, tc.ct)
}
var stored attach.File
if err := gdb.First(&stored, f.ID).Error; err != nil {
t.Fatal(err)
}
if stored.DiskName != f.DiskName || stored.FileName != tc.fileName {
t.Fatalf("stored row %+v", stored)
}
got, err := bucket.ReadAll(ctx, attach.BlobKey(f.DiskName))
if err != nil || !bytes.Equal(got, tc.body) {
t.Fatalf("blob under %s: %v", attach.BlobKey(f.DiskName), err)
}
})
}
}

View File

@@ -76,13 +76,18 @@ func fileExt(diskName string) string {
return strings.ToLower(ext)
}
// PublicURL returns the public URL of a blob key: storage.uploads.
// public_path_prefix and the key joined by exactly one slash. With the
// WinterCMS layout (bucket rooted at storage/app/uploads/public, prefix
// /storage/app/uploads/public) it is Winter's File::getPath() path.
// PublicURL returns the public URL of a blob key. With
// storage.uploads.public_url set (PublicURLBase) it is that base and the key
// joined by exactly one slash. Otherwise it is
// storage.uploads.public_path_prefix and the key joined the same way; with
// the WinterCMS layout (bucket rooted at storage/app/uploads/public, prefix
// /storage/app/uploads/public) that is Winter's File::getPath() path.
func PublicURL(key string) string {
prefix := strings.TrimRight(PublicPathPrefix(), "/")
key = strings.TrimLeft(key, "/")
if base := PublicURLBase(); base != "" {
return strings.TrimRight(base, "/") + "/" + key
}
prefix := strings.TrimRight(PublicPathPrefix(), "/")
if prefix == "" {
return "/" + key
}

View File

@@ -3,6 +3,7 @@ package attach
import (
"bytes"
"encoding/binary"
"errors"
"fmt"
"hash/crc32"
"image"
@@ -19,6 +20,7 @@ import (
"git.golem15.com/golem15/summercms/modules/compass"
"gocloud.dev/blob"
"gocloud.dev/blob/memblob"
"gorm.io/gorm"
)
// winterLayoutBucket opens a mem:// bucket with the WinterCMS public prefix
@@ -362,3 +364,130 @@ func TestBucketAndURLEdges(t *testing.T) {
}
}
}
// TestPublicURLBase (plan 05 of phase 19.3, D-12) builds direct URLs from
// storage.uploads.public_url: trailing slashes on the base and leading
// slashes on the key collapse to one, File.URL and thumbnails use the base,
// and clearing it restores the proxy prefix.
func TestPublicURLBase(t *testing.T) {
t.Cleanup(func() {
setPublicPathPrefix(defaultPublicPathPrefix)
setPublicURLBase("")
})
key := BlobKey("651a2b3c4d5e61234567.png")
for _, tc := range []struct{ base, key, want string }{
{"https://cdn.example.com", key, "https://cdn.example.com/" + key},
{"https://cdn.example.com/", key, "https://cdn.example.com/" + key},
{"https://cdn.example.com/uploads//", "/" + key, "https://cdn.example.com/uploads/" + key},
{"http://bucket.s3.example.com/a/b", "//" + key, "http://bucket.s3.example.com/a/b/" + key},
} {
setPublicURLBase(tc.base)
if got := PublicURL(tc.key); got != tc.want {
t.Errorf("base %q key %q: PublicURL = %q, want %q", tc.base, tc.key, got, tc.want)
}
}
bucket := memblob.OpenBucket(nil)
t.Cleanup(func() { _ = bucket.Close() })
setPublicURLBase("https://cdn.example.com/uploads/")
if PublicURLBase() != "https://cdn.example.com/uploads/" {
t.Fatalf("PublicURLBase = %q", PublicURLBase())
}
f := &File{ID: 31, DiskName: "651a2b3c4d5e61234567.png"}
if got, want := f.URL(), "https://cdn.example.com/uploads/651/a2b/3c4/651a2b3c4d5e61234567.png"; got != want {
t.Fatalf("File.URL = %q, want %q", got, want)
}
if err := bucket.WriteAll(t.Context(), BlobKey(f.DiskName), testJPEG(t), nil); err != nil {
t.Fatal(err)
}
thumb, err := f.Thumb(t.Context(), bucket, 40, 30, "crop")
if err != nil {
t.Fatal(err)
}
if want := "https://cdn.example.com/uploads/651/a2b/3c4/thumb_31_40_30_0_0_crop.png"; thumb != want {
t.Fatalf("thumb = %q, want %q", thumb, want)
}
setPublicURLBase("")
if got := f.URL(); got != "/storage/uploads/651/a2b/3c4/651a2b3c4d5e61234567.png" {
t.Fatalf("without a base File.URL = %q", got)
}
}
// TestPublicURLValidation refuses every malformed storage.uploads.public_url
// at OpenBucket, naming the key and never echoing credentials; an unset or
// blank value keeps the proxy prefix.
func TestPublicURLValidation(t *testing.T) {
t.Cleanup(func() {
setPublicPathPrefix(defaultPublicPathPrefix)
setPublicURLBase("")
})
for _, bad := range []string{
"ftp://cdn.example.com/uploads",
"file:///srv/uploads",
"https://user:secret@cdn.example.com",
"https://token@cdn.example.com",
"https://cdn.example.com/uploads?v=1",
"https://cdn.example.com/uploads?",
"https://cdn.example.com/uploads#top",
"/storage/uploads",
"cdn.example.com/uploads",
"https:///uploads",
"https://cdn.example.com:bad/",
} {
setPublicURLBase("https://stale.example.com")
cfg := openTestConfig(t, "uploads:\n bucket_url: \"mem://\"\n public_url: \""+bad+"\"\n")
bucket, err := OpenBucket(t.Context(), cfg)
if err == nil {
_ = bucket.Close()
t.Fatalf("public_url %q accepted", bad)
}
if !strings.Contains(err.Error(), "storage.uploads.public_url") || strings.Contains(err.Error(), "secret") || strings.Contains(err.Error(), "token@") {
t.Fatalf("public_url %q: error %q", bad, err)
}
if _, perr := parsePublicURL(bad); perr == nil {
t.Fatalf("parsePublicURL(%q) accepted", bad)
}
}
for _, unset := range []string{"", " "} {
setPublicURLBase("https://stale.example.com")
cfg := openTestConfig(t, "uploads:\n bucket_url: \"mem://\"\n public_url: \""+unset+"\"\n")
bucket, err := OpenBucket(t.Context(), cfg)
if err != nil {
t.Fatalf("public_url %q: %v", unset, err)
}
_ = bucket.Close()
if PublicURLBase() != "" || PublicURL("a/b.png") != "/storage/uploads/a/b.png" {
t.Fatalf("public_url %q: base %q URL %q", unset, PublicURLBase(), PublicURL("a/b.png"))
}
}
for _, ok := range []string{"https://cdn.example.com", "HTTP://cdn.example.com:8080/u/", " https://cdn.example.com/uploads "} {
u, err := parsePublicURL(ok)
if err != nil || u == nil {
t.Fatalf("parsePublicURL(%q) = %v, %v", ok, u, err)
}
}
}
func TestStoreProtectedNeedsPrivateBucket(t *testing.T) {
// Not parallel: the public URL base is package state.
t.Cleanup(func() { setPublicURLBase("") })
bucket := memblob.OpenBucket(nil)
t.Cleanup(func() { _ = bucket.Close() })
db := &gorm.DB{}
setPublicURLBase("https://cdn.example.com/uploads/")
_, err := Store(t.Context(), db, bucket, Upload{FileName: "notes.txt", Body: bytes.NewReader([]byte("secret"))}, Limits{})
if !errors.Is(err, ErrProtectedNeedsPrivateBucket) {
t.Fatalf("protected upload with public_url: error = %v, want ErrProtectedNeedsPrivateBucket", err)
}
for _, want := range []string{"storage.uploads.public_url", "private bucket"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q lacks %q", err, want)
}
}
iter := bucket.List(nil)
if obj, err := iter.Next(t.Context()); err == nil {
t.Fatalf("a blob was written: %s", obj.Key)
}
}

View File

@@ -29,7 +29,7 @@ Use cases:
- Admin controller contracts: `pact.AdminController`, `pact.HasAdminControllers`, `pact.AdminAssets` (embedded Winter-shaped admin YAML), `pact.AdminPermissioned` and `pact.AdminRecordSource`.
- Admin extension contracts, so a plugin extends the compiled admin SPA without a Node build: `pact.AdminClientAssets` (per-controller JS and CSS from the plugin's embedded `assets/` tree, Winter's `addJs`/`addCss`), `pact.HasAdminActions` with `pact.AdminAction`, `pact.AdminActionInput` and `pact.AdminActionResult` (named toolbar and widget actions whose routes, CSRF check, permissions and record scoping the framework owns), `pact.HasAdminBulkActions` with `pact.AdminBulkAction`, `pact.AdminBulkActionInput` and `pact.AdminBulkActionResult` (named actions on the rows selected in a list, which receive records the framework loaded through the list scope, never ids), `pact.HasAdminRecordActions` with `pact.AdminRecordAction`, `pact.AdminRecordActionInput` and `pact.AdminRecordActionResult` (named actions on one record, each with an `Applies` rule for the record's state), and `pact.AdminPartialData` (the curated view model a partial template renders).
- Plugin field types: `pact.HasFormFieldTypes` returns `pact.FormFieldType` entries, admin form field types any controller's `fields.yaml` uses by a short alias (`type: notes`). A type names its custom element, its scripts and styles, its own option keys and its Go funcs: `Decode` turns a submitted `pact.FormFieldValue` into a `pact.FormFieldDecoded` (the value to store plus optional `pact.FormFieldNotice` messages for the administrator, such as what a sanitizer removed) or refuses it with a `pact.FormFieldError`; `Load` serves the stored value to the element, `ListValue` formats a list column, and `Upload` stores a file from a `pact.FormFieldUpload` and answers a `pact.FormFieldUploadResult`. The admin framework owns the routes, permissions, record scope, locale storage of a translatable type and the localization of notices.
- Optional admin hooks a controller or model can implement: list and form query scoping (`pact.ListExtendQuery`, `pact.FormExtendQuery`), list row states (`pact.ListRowStates` with the fixed `pact.RowState` set `pact.RowStateDeleted`, `pact.RowStateNegative` and `pact.RowStateDisabled`), create, update and delete hooks (`pact.FormBeforeCreate`, `pact.FormAfterUpdate`, `pact.FormBeforeDelete` and their siblings), form-only fields that reach those hooks without being model columns (`pact.FormVirtualFields`), validation rules per operation for admin saves (`pact.FormRules`), relation hooks (`pact.RelationExtendManageQuery`, `pact.RelationExtendOptionsQuery`, `pact.RelationBeforeLink`), relation child hooks around creating, updating and deleting a related record (`pact.RelationBeforeCreate`, `pact.RelationAfterCreate`, `pact.RelationBeforeUpdate`, `pact.RelationAfterUpdate`, `pact.RelationBeforeDelete`, `pact.RelationAfterDelete`), filter scopes (`pact.FilterScope`, `pact.FilterOptions`) and dropdown options (`pact.DropdownOptionsProvider`).
- Optional admin hooks a controller or model can implement: list and form query scoping (`pact.ListExtendQuery`, `pact.FormExtendQuery`), list row states (`pact.ListRowStates` with the fixed `pact.RowState` set `pact.RowStateDeleted`, `pact.RowStateNegative` and `pact.RowStateDisabled`), create, update and delete hooks (`pact.FormBeforeCreate`, `pact.FormAfterUpdate`, `pact.FormBeforeDelete` and their siblings), a hook after a reorder save (`pact.ReorderAfterSave`), form-only fields that reach those hooks without being model columns (`pact.FormVirtualFields`), validation rules per operation for admin saves (`pact.FormRules`), relation hooks (`pact.RelationExtendManageQuery`, `pact.RelationExtendOptionsQuery`, `pact.RelationBeforeLink`), relation child hooks around creating, updating and deleting a related record (`pact.RelationBeforeCreate`, `pact.RelationAfterCreate`, `pact.RelationBeforeUpdate`, `pact.RelationAfterUpdate`, `pact.RelationBeforeDelete`, `pact.RelationAfterDelete`), filter scopes (`pact.FilterScope`, `pact.FilterOptions`) and dropdown options (`pact.DropdownOptionsProvider`).
- Dependent dropdown options: `pact.DependentOptionsProvider` answers a `pact.DependentOptionSet` (whether the sibling's value takes a choice at all, and the choices) for a dropdown with `dependsOn`, from the submitted values of its siblings and with a context, so the choices can come from the database; the admin framework calls it for the options route and again on every save. `pact.Option.Depth` indents a choice taken from a tree.
- Menu item types: `pact.HasMenuItemTypes` returns `pact.MenuItemType` values (key, label, default URL pattern, `PatternFrom` for a list type, the `References` an item can point at as a `pact.MenuItemReference` tree, and `Resolve`), the typed successor of WinterCMS's `pages.menuitem.*` events. `Resolve` receives a `pact.MenuItemResolve` with the pattern already layered and answers a `pact.ResolvedMenuItem` with its alternates as an ordered `[]pact.MenuItemAlternate`. [signpost](../signpost/README.md) collects them at boot.
- Singleton admin controllers: `pact.AdminSingleton` (`SingletonRecord`, first or create) makes a controller edit exactly one record, opened directly instead of from a list.
@@ -137,6 +137,7 @@ func (p *Plugin) Schedule() []pact.ScheduledCommand {
| `pact.AdminRecordActionInput` | What a record action receives: `RecordID` and `Record`, the record loaded and row-locked through the form scope. |
| `pact.AdminRecordActionResult` | What a record action returns: an optional message for the toast. |
| `pact.HasAdminRecordActions` | Registers a controller's record actions for the `recordActions` list of `config_form.yaml`. |
| `pact.ReorderAfterSave` | Optional controller hook `ReorderAfterSave(ctx, partition)`: runs after a reorder save wrote the new order, inside the same transaction (`cabana.TxFromContext` works); an error rolls the reorder back. |
| `pact.AdminPartialData` | Supplies the view model a controller partial template renders; never the GORM model. |
| `pact.HasFormFieldTypes` | Implemented by a plugin (not a controller) that registers admin form field types. |
| `pact.FormFieldType` | One field type: the alias `Name`, the custom-element tag `Element`, `Translatable`, the `Scripts` and `Styles` under `assets/`, the `Options` keys, and the funcs `CheckOptions` (optional), `Decode` (required), `Load`, `ListValue` (`func(ctx context.Context, value any) string`) and `Upload` (optional). |

View File

@@ -765,6 +765,15 @@ type FormAfterDelete interface {
FormAfterDelete(ctx context.Context, model any) error
}
// ReorderAfterSave optionally runs after a reorder save (POST .../reorder of
// a tree or sortable list) wrote the new order, inside the same transaction,
// so cabana.TxFromContext returns it and lagoon.AfterCommit work waits for
// the commit. partition is the reorder request's partition (nil when the
// list has none). A returned error rolls the reorder back.
type ReorderAfterSave interface {
ReorderAfterSave(ctx context.Context, partition *uint64) error
}
// FormVirtualFields optionally names fields of the controller's form that are
// not model columns for the purpose of the form, such as a password and its
// confirmation or a "send an invitation" checkbox. The admin framework never

View File

@@ -12,7 +12,7 @@ surf turns the routes that plugins declare through `pact.HasRoutes` into one `ht
## Role in the framework
The HTTP entry point, at the top of the stack: it depends on `backpack`, `bonfire`, `bouncer`, `cabana`, `compass`, `conga`, `lagoon`, `pact`, `party`, `towel` and `wire`, and provides the serve command. Plugins contribute routes through `pact`; `surf` collects them into one standard library ServeMux.
The HTTP entry point, at the top of the stack: it depends on `backpack`, `bonfire`, `bouncer`, `cabana`, `compass`, `conga`, `cooler`, `lagoon`, `pact`, `party`, `towel` and `wire`, and provides the serve command. Plugins contribute routes through `pact`; `surf` collects them into one standard library ServeMux.
Use cases:
@@ -27,14 +27,16 @@ Use cases:
- Overlapping constrained routes: two routes that ServeMux would refuse side by side, such as `/shelves/items/{id}` and `/shelves/{shelfId}/follow`, boot as one overlap family when their constraints keep them apart. Requests are tried against the family's routes in registration order, as Laravel does, and each route keeps its own path values and middleware.
- Named middleware from plugins (`pact.HasMiddleware`), parameterized middleware used as `name:param` (`pact.HasMiddlewareFactories`) and house middleware for the JSON envelope and error handling (`pact.HasHouseMiddleware`). Duplicate or unknown names fail boot.
- Raw groups (`surf.Router.GroupRaw`) for routes that must not be wrapped in house middleware, such as webhooks or file streams: house middleware is refused there, the default body limit is skipped and a panic returns a bare 500.
- Built-in middleware names: `throttle:<bucket>` or `throttle:<max>,<minutes>`, `body.limit:<bytes>`, `locale.from-principal`, plus `backend` (the admin guard) when the admin is enabled.
- Fixed-window rate limiting (`surf.FixedWindowLimiter`): named buckets from plugins that implement `surf.BucketProvider`, or inline limits keyed by the signed-in user, or by client IP for guests. Rejected requests get a 429 with `Retry-After` and `X-RateLimit-*` headers. The in-process `surf.MemoryStore` sits behind the `surf.Store` interface.
- Built-in middleware names: `throttle:<bucket>` or `throttle:<max>,<minutes>`, `body.limit:<bytes>`, `cors:<profile>`, `turnstile:open` and `turnstile:closed` (Cloudflare Turnstile through `bouncer.Turnstile`, built from `turnstile.secret`, with the client IP from `surf.ClientIP` and refusals written by a published `bouncer.TurnstileWriter` when there is one), `locale.from-principal`, plus `backend` (the admin guard) when the admin is enabled.
- Named CORS profiles: `cors:<profile>` applies `http.cors_profiles.<profile>` (read by `surf.LoadCORSProfiles`, validated like `http.cors`) to a route or route group. The profile wraps every other layer of the route, whatever its position in the list or the group nesting, so every response carries its headers: guard refusals (401, 403), throttle (429), Turnstile (422, 503), the body limit (413) and recovered panics (500) included. A handler's own `Vary` is merged with the profile's `Vary: Origin`, not replacing it. surf registers an `OPTIONS` route for each such path that answers the preflight from the profile, since ServeMux would answer it with 405 before the middleware runs. An unknown profile, two profiles on one path and a profiled path that `http.cors.paths` also matches fail boot.
- Fixed-window rate limiting (`surf.FixedWindowLimiter`): named buckets from plugins that implement `surf.BucketProvider`, or inline limits keyed by the signed-in user, or by client IP for guests. Rejected requests get a 429 with `Retry-After` and `X-RateLimit-*` headers. The in-process `surf.MemoryStore` sits behind the `surf.Store` interface; `surf.BuildRouter` uses a `surf.Store` published on the app instead when there is one, which is how `serve` shares counters across instances through [cooler](../cooler/README.md)'s Redis store (`http.rate_limit.store: redis`).
- Client IP resolution for limiter keys (`surf.ClientIP`) that only trusts `X-Forwarded-For` hops when the direct peer is inside a configured trusted proxy range (`surf.TrustedProxies`).
- Every non-raw route runs inside JSON panic recovery (an opaque 500 via [wire](../wire/README.md)), gets the request locale from a published `surf.LocaleResolver` when one exists or from the `Accept-Language` header otherwise (see [towel](../towel/README.md)), and a request body cap. Responses are buffered until the handler returns, so a panic never leaves a half-written body.
- Path-scoped CORS configured with the same keys as Laravel's `config/cors.php` (`surf.CORSConfig`), including preflight handling. Every `OPTIONS` request on a CORS path is answered with 204 before routing, with the headers Laravel's `HandleCors` sends: `Cache-Control: no-cache, private` always and, on a preflight (an `Origin` and an `Access-Control-Request-Method`), the requested method (upper-cased) and headers echoed in `Access-Control-Allow-Methods` and `Access-Control-Allow-Headers` when `*` allows any, `Vary` on the request headers and PHP's default `Content-Type: text/html; charset=UTF-8`.
- Path-scoped CORS configured with the same keys as Laravel's `config/cors.php` (`surf.CORSConfig`), including preflight handling. `surf.LoadCORSConfig` refuses credentials with a `*` origin and origin patterns that do not compile; an allowed origin is echoed exactly, with `Vary: Origin` on every response whose answer depends on the request origin. Every `OPTIONS` request on a CORS path is answered with 204 before routing, with the headers Laravel's `HandleCors` sends: `Cache-Control: no-cache, private` always and, on a preflight (an `Origin` and an `Access-Control-Request-Method`), the requested method (upper-cased) and headers echoed in `Access-Control-Allow-Methods` and `Access-Control-Allow-Headers` when `*` allows any, `Vary` on the request headers and PHP's default `Content-Type: text/html; charset=UTF-8`.
- Optional `surf.LocaleResolver`: a compiled plugin publishes the interface through backpack; surf looks it up without importing the plugin. When present, it may strip an enabled locale prefix before routing and writes a validated code with `towel.WithLocale`. When absent, hosts keep the raw `Accept-Language` header and `surf.LocaleFromPrincipal` overlay.
- `surf.LocaleFromPrincipal` switches the request locale to the signed-in user's preferred locale.
- Opt-in uploads route: with `storage.uploads.serve` true, `surf.BuildRouter` mounts `GET <storage.uploads.public_path_prefix>/{path...}` (owned by `summercms.surf`) on `attach.StaticHandlerPublic`, so a file is served only when its `system_files` row is public. Every response carries `X-Content-Type-Options: nosniff` and a sandboxing `Content-Security-Policy`, and a file whose type is not a raster image (JPEG, PNG, GIF, WebP, AVIF) is sent with `Content-Disposition: attachment`, so a stored HTML or SVG file never renders on the app origin. The bucket and the database are looked up per request, so `route:list` lists the route without opening either; until `serve` publishes them it answers 404. The route owns the whole prefix: `surf.BuildRouter` fails when a plugin route sits at or under it, and when the prefix is not an absolute URL path (no leading slash, a scheme or host, a query, an empty, `.` or `..` segment, or `{`/`}`).
- Opt-in API documentation page: with `http.api_docs.enabled` true, `surf.BuildRouter` mounts `GET /api-docs` (a Swagger UI page) and `GET /api-docs.json` (owned by `summercms.apidocs`, public, no middleware). The spec comes from the one plugin implementing `surf.APIDocsProvider`, as YAML or JSON, converted once at boot to compact JSON in source key order; zero or several providers, or a spec that does not convert, fail boot. The page loads swagger-ui-dist `surf.SwaggerUIVersion` from jsDelivr with Subresource Integrity hashes on all three assets and a `Content-Security-Policy` whose only script source is a per-request nonce; every script tag, the two jsDelivr scripts included, carries it, so no other jsDelivr script can run.
- Opt-in request log: with `http.access_log` true, every request writes one Info record `http request` after it is served, with `method`, `pattern` (the matched route, such as `GET /posts/{id}`), `status` and `duration_ms`. A request whose client went away adds `client_gone=true`; its 5xx is logged as 499 (client closed request) only when the client had already gone when the status was written and the handler did not panic, so a 5xx written before the cancellation and a recovered panic keep their real status. An unmatched request (a 404 or 405 from the mux, or a CORS preflight answered before routing) has an empty `pattern`. The URL path, the query string, headers and the client address are never logged, for an unmatched request too: a token route requested with another method answers 405 before any route handler runs. Records go to the app's published `*slog.Logger`, else `slog.Default`, so the [sunscreen](../sunscreen/README.md) redaction still applies.
- A read-only route table (`surf.Router.Routes`), the admin mount status (`surf.Router.AdminPrefix`) and the `serve` and `route:list` commands.
@@ -134,6 +136,9 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur
| `surf.ClientIP` | Resolves the client IP, honouring trusted proxies. |
| `surf.TrustedProxies` | Parses `http.trusted_proxies` into CIDR prefixes. |
| `surf.CORSConfig` | CORS settings; `surf.LoadCORSConfig` reads them from config. |
| `surf.LoadCORSProfiles` | Reads and validates the `http.cors_profiles.<name>` blocks the `cors:<profile>` middleware uses. |
| `surf.APIDocsProvider` | Implemented by the one plugin that supplies the OpenAPI document (`APIDocsSpec`, YAML or JSON) for the opt-in api-docs page. |
| `surf.SwaggerUIVersion` | The exact swagger-ui-dist release the api-docs page loads, pinned so its assets carry SRI hashes. |
| `surf.LocaleResolver` | Optional backpack-published request locale; `Resolve` returns a validated code and `Rewrite` strips an enabled prefix before routing. |
| `surf.LocaleFromPrincipal` | Middleware that applies the signed-in user's preferred locale. |
| `surf.ServeCommand` | The `serve` console command. |
@@ -149,13 +154,20 @@ The generated application `main` wires surf in with `surf.ServeCommand` and `sur
| `http.body_limits.upload_bytes` | none, required | Upload body cap in bytes. Must be a whole number of at least 1; it is validated at boot. |
| `http.trusted_proxies` | empty | List of CIDRs whose `X-Forwarded-For` header is trusted when resolving the client IP. Malformed entries are skipped. |
| `http.cors.paths` | empty | Path globs (`api/*`) that get CORS headers. With no `http.cors` section no CORS headers are sent. |
| `http.cors.allowed_origins` | empty | Allowed origins; `*` allows any. |
| `http.cors.allowed_origins_patterns` | empty | Regular expressions matched against the origin. |
| `http.cors.allowed_origins` | empty | Allowed origins, echoed exactly; `*` allows any (without credentials only). An entry with a `*` inside, such as `https://*.example.com`, matches any characters there, as Laravel's `Str::is` does. With credentials, such a `*` must be the whole leftmost host label after a literal scheme (`https://*.example.com`; `https://*` and `*://*` fail boot), and no entry may match an origin outside the configured domains. |
| `http.cors.allowed_origins_patterns` | empty | Go regular expressions matched against the origin. A pattern that does not compile, or one still wrapped in PHP delimiters (`#^...$#`), fails boot. With credentials, a pattern must be anchored with `^` and `$` and must not match an origin outside the configured domains (`^https?://.*$` fails boot). |
| `http.cors.allowed_methods` | empty | Methods sent in `Access-Control-Allow-Methods`; `*` allows any. |
| `http.cors.allowed_headers` | empty | Headers sent in `Access-Control-Allow-Headers`; `*` allows any. |
| `http.cors.exposed_headers` | empty | Headers sent in `Access-Control-Expose-Headers`. |
| `http.cors.max_age` | `0` | Preflight cache time in seconds. |
| `http.cors.supports_credentials` | `false` | Sends `Access-Control-Allow-Credentials: true`. |
| `http.cors.supports_credentials` | `false` | Sends `Access-Control-Allow-Credentials: true`. Combined with `*` in `allowed_origins` it fails boot. With credentials, a pattern or several origins, every response on a CORS path carries `Vary: Origin`, preflights and disallowed origins included. |
| `http.cors_profiles.<name>.*` | none | A CORS profile for `cors:<name>`: the `http.cors` keys except `paths` (`allowed_origins`, `allowed_origins_patterns`, `allowed_methods`, `allowed_headers`, `exposed_headers`, `max_age`, `supports_credentials`), validated the same way. Names use lower-case letters, digits, underscore and dash. |
| `turnstile.secret` | empty | Cloudflare Turnstile secret for `turnstile:open` and `turnstile:closed`. Without it `turnstile:open` passes and `turnstile:closed` answers 503 in the production environment (passes elsewhere), and when routes name either, `surf.BuildRouter` logs one start-up warning, `surf: turnstile.secret is empty`, with the route count. A `*bouncer.Turnstile` published on the app replaces the verifier built from this key. Environment: `SUMMER_TURNSTILE__SECRET`. |
| `turnstile.field` | `turnstile_token` | Request field (query, JSON body or form body) carrying the Turnstile token. |
| `http.rate_limit.store` | `memory` | Where `throttle` counters live: `memory` (per instance) or `redis` (shared by every instance through [cooler](../cooler/README.md), which then needs `redis.url`). Only `serve` installs the Redis store; `route:list` keeps the memory store and never dials Redis. Any other value fails at boot. Environment: `SUMMER_HTTP__RATE_LIMIT__STORE`. |
| `http.api_docs.enabled` | `false` | Mounts `GET <http.api_docs.path>` (Swagger UI page) and `GET <http.api_docs.path>.json` (the spec of the one `surf.APIDocsProvider` plugin). Both are public. Environment: `SUMMER_HTTP__API_DOCS__ENABLED`. |
| `http.api_docs.path` | `/api-docs` | Page path; the JSON route adds `.json`. Must be an absolute URL path; a plugin route on either path fails boot. |
| `http.api_docs.title` | `API documentation` | Page title. |
| `http.access_log` | `false` | Logs one `http request` record per request (method, matched pattern, status, duration in milliseconds; never the path or query). Environment: `SUMMER_HTTP__ACCESS_LOG`. |
| `storage.uploads.serve` | `false` | Mounts the uploads route `GET <storage.uploads.public_path_prefix>/{path...}` (prefix default `/storage/uploads`; it must then be an absolute URL path, or start-up fails) on `attach.StaticHandlerPublic` with `X-Content-Type-Options: nosniff`, a sandboxing CSP and `Content-Disposition: attachment` for anything but a raster image. Prefer a separate origin for uploads in production. Environment: `SUMMER_STORAGE__UPLOADS__SERVE`. |
@@ -173,20 +185,21 @@ http:
supports_credentials: true
```
The `serve` command also opens the database through [lagoon](../lagoon/README.md) and the uploads bucket through `attach.OpenBucket`, so their settings must be present as well. It starts the background job worker of [conga](../conga/README.md) in the same process unless `queue.work_in_serve` is `false` (default `true`); set it to `false` when a separate `queue:work` process runs the jobs. The other `queue.*` keys are documented in the conga README.
The `serve` command also opens the database through [lagoon](../lagoon/README.md) and the uploads bucket through `attach.OpenBucket`, so their settings must be present as well. It then installs [cooler](../cooler/README.md) (`cooler.Install`) before it builds the router: with `http.rate_limit.store: redis` it publishes cooler's Redis-backed store as the `surf.Store`, falling back to a per-instance `surf.MemoryStore` while Redis does not answer. A Redis that is down at start-up is a warning, not a boot failure; a missing or malformed `redis.url` is. It starts the background job worker of [conga](../conga/README.md) in the same process unless `queue.work_in_serve` is `false` (default `true`); set it to `false` when a separate `queue:work` process runs the jobs. The other `queue.*` keys are documented in the conga README.
## CLI commands
| Command | Flags | Description |
|---------|-------|-------------|
| `serve` | `--addr` (default `:8080`) | Opens the database and uploads bucket, assembles the router, binds the address, starts the in-process job worker (see `queue.work_in_serve`) and serves HTTP until SIGINT or SIGTERM, then shuts the server and the worker down gracefully within 10 seconds. After binding it prints the listen address with a browser URL (an unspecified host shows as 127.0.0.1) and an admin status line: the admin URL, with a warning naming `summer admin:create` when no administrator exists yet, or `admin: off` with the steps to add an admin controller. |
| `serve` | `--addr` (default `:8080`) | Opens the database and uploads bucket, installs cooler (the shared rate-limit store when `http.rate_limit.store` is `redis`), assembles the router, binds the address, starts the in-process job worker (see `queue.work_in_serve`) and serves HTTP until SIGINT or SIGTERM, then shuts the server and the worker down gracefully within 10 seconds. After binding it prints the listen address with a browser URL (an unspecified host shows as 127.0.0.1) and an admin status line: the admin URL, with a warning naming `summer admin:create` when no administrator exists yet, or `admin: off` with the steps to add an admin controller. |
| `route:list` | none | Builds the router the same way `serve` does, without opening the database or listening, and prints a table of method, pattern, plugin, middleware and raw flag for every route, followed by the admin status line (`admin: mounted at <prefix>` or `admin: off`). |
## Dependencies
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [cabana](../cabana/README.md), [compass](../compass/README.md), [conga](../conga/README.md) (the in-process job worker of `serve`), [lagoon](../lagoon/README.md) (including `lagoon/attach`), [pact](../pact/README.md), [party](../party/README.md), [towel](../towel/README.md), [wire](../wire/README.md).
- Third-party: `gocloud.dev/blob` (uploads bucket opened by `serve`).
- Standard library: `bytes`, `context`, `fmt`, `math`, `net`, `net/http`, `net/netip`, `os`, `os/signal`, `regexp`, `strconv`, `strings`, `sync`, `syscall`, `time`.
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [cabana](../cabana/README.md), [compass](../compass/README.md), [conga](../conga/README.md) (the in-process job worker of `serve`), [cooler](../cooler/README.md) (the shared rate-limit store `serve` installs), [lagoon](../lagoon/README.md) (including `lagoon/attach`), [pact](../pact/README.md), [party](../party/README.md), [towel](../towel/README.md), [wire](../wire/README.md).
- Third-party: `gocloud.dev/blob` (uploads bucket opened by `serve`), `github.com/goccy/go-yaml` (api-docs spec conversion).
- Standard library: `bytes`, `context`, `crypto/rand`, `encoding/base64`, `encoding/json`, `errors`, `fmt`, `html/template`, `log/slog`, `math`, `mime`, `net`, `net/http`, `net/netip`, `net/url`, `os`, `os/signal`, `regexp`, `sort`, `strconv`, `strings`, `sync`, `syscall`, `time`.
- Tests additionally use `github.com/testcontainers/testcontainers-go` (Redis container).
## Testing
@@ -194,4 +207,4 @@ The `serve` command also opens the database through [lagoon](../lagoon/README.md
go test ./modules/surf/...
```
The tests use `net/http/httptest` and in-memory stores and need no external services.
The tests use `net/http/httptest` and in-memory stores. `TestRateLimitAcrossInstancesSmoke` starts a Redis container through testcontainers-go and needs Docker; `go test -short ./modules/surf/...` skips it.

211
modules/surf/apidocs.go Normal file
View File

@@ -0,0 +1,211 @@
package surf
import (
"bytes"
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"html/template"
"net/http"
"sort"
"strings"
"git.golem15.com/golem15/summercms/modules/backpack"
"git.golem15.com/golem15/summercms/modules/pact"
"git.golem15.com/golem15/summercms/modules/party"
"github.com/goccy/go-yaml"
)
// APIDocsProvider is implemented by the one plugin that supplies the
// application's OpenAPI document for the opt-in /api-docs page. APIDocsSpec
// returns the document as YAML or JSON, typically an embedded file; surf
// converts it to compact JSON once at boot.
type APIDocsProvider interface {
APIDocsSpec() ([]byte, error)
}
// SwaggerUIVersion is the swagger-ui-dist release the /api-docs page loads
// from jsDelivr. The version is exact so the assets can carry Subresource
// Integrity hashes; a floating version could change under the page.
const SwaggerUIVersion = "5.33.1"
// The pinned swagger-ui-dist assets and their integrity values (checked
// against the npm tarball of the release).
const (
swaggerUICSS = "https://cdn.jsdelivr.net/npm/swagger-ui-dist@5.33.1/swagger-ui.css"
swaggerUICSSIntegrity = "sha384-Ov4/wv3j2bmct8cDc5X4ngJZohVPzEmc6uDPH8WeljUxO5vtoykvMEfbu9Vh6RaW"
swaggerUIBundle = "https://cdn.jsdelivr.net/npm/swagger-ui-dist@5.33.1/swagger-ui-bundle.js"
swaggerUIBundleIntegrity = "sha384-ZPehFMQommnnuaZ4rpxgkgTT2DKFVp4hZC/7pLit+9Lek9T1YGSo23eHFbvNkXkw"
swaggerUIPreset = "https://cdn.jsdelivr.net/npm/swagger-ui-dist@5.33.1/swagger-ui-standalone-preset.js"
swaggerUIPresetIntegrity = "sha384-My2aDM4r2Mbm3ybHcubKm9O9U8FEjvF/O5nGvE9YK5dzqOTbWEKa79RPJ1krdMaF"
)
const (
// apiDocsPluginID owns the api-docs routes in the route table.
apiDocsPluginID = "summercms.apidocs"
defaultAPIDocsPath = "/api-docs"
defaultAPIDocsName = "API documentation"
)
// apiDocsPage is the Swagger UI shell. The asset URLs and integrity values
// are constants written into the template text; the title, the nonce and the
// spec URL go through html/template escaping.
var apiDocsPage = template.Must(template.New("apidocs").Parse(`<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{{.Title}}</title>
<link rel="stylesheet" href="` + swaggerUICSS + `" integrity="` + swaggerUICSSIntegrity + `" crossorigin="anonymous">
</head>
<body>
<div id="swagger-ui"></div>
<script src="` + swaggerUIBundle + `" integrity="` + swaggerUIBundleIntegrity + `" crossorigin="anonymous" nonce="{{.Nonce}}"></script>
<script src="` + swaggerUIPreset + `" integrity="` + swaggerUIPresetIntegrity + `" crossorigin="anonymous" nonce="{{.Nonce}}"></script>
<script nonce="{{.Nonce}}">
window.addEventListener('load', function () {
window.ui = SwaggerUIBundle({
url: {{.SpecURL}},
dom_id: '#swagger-ui',
deepLinking: true,
validatorUrl: null,
presets: [SwaggerUIBundle.presets.apis, SwaggerUIStandalonePreset],
layout: 'StandaloneLayout'
});
});
</script>
</body>
</html>
`))
// mountAPIDocs registers GET <http.api_docs.path> (the Swagger UI page) and
// GET <http.api_docs.path>.json (the spec) when http.api_docs.enabled is
// true. Exactly one plugin must implement APIDocsProvider; its spec is
// converted to compact JSON here, once, and a spec that does not convert
// fails boot naming the plugin. Both routes are public (no middleware) and
// may not replace a route a plugin declared.
func mountAPIDocs(r *Router, app *backpack.App, plugins []party.Plugin) error {
if r == nil || app == nil || app.Config == nil || !app.Config.Bool("http.api_docs.enabled") {
return nil
}
path, err := apiDocsPath(app.Config.String("http.api_docs.path"))
if err != nil {
return err
}
jsonPath := path + ".json"
title := strings.TrimSpace(app.Config.String("http.api_docs.title"))
if title == "" {
title = defaultAPIDocsName
}
var ids []string
var provider APIDocsProvider
for _, p := range plugins {
if dp, ok := p.(APIDocsProvider); ok {
ids = append(ids, p.ID())
provider = dp
}
}
switch len(ids) {
case 0:
return errors.New("surf: http.api_docs.enabled is true but no plugin implements surf.APIDocsProvider")
case 1:
default:
sort.Strings(ids)
return fmt.Errorf("surf: http.api_docs.enabled is true and several plugins implement surf.APIDocsProvider (%s); exactly one may", strings.Join(ids, ", "))
}
raw, err := provider.APIDocsSpec()
if err != nil {
return fmt.Errorf("surf: api docs spec of plugin %q: %w", ids[0], err)
}
spec, err := apiDocsJSON(raw)
if err != nil {
return fmt.Errorf("surf: api docs spec of plugin %q: %w", ids[0], err)
}
for _, p := range []string{path, jsonPath} {
if owner, ok := r.seen[http.MethodGet+" "+p]; ok {
return fmt.Errorf("surf: route GET %s (plugin %q) collides with the api docs page http.api_docs.path mounts", p, owner)
}
}
r.BindPlugin(apiDocsPluginID)
r.GroupRaw("", nil, func(g pact.Router) {
g.Get(path, apiDocsPageHandler(title, jsonPath))
g.Get(jsonPath, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Header().Set("X-Content-Type-Options", "nosniff")
_, _ = w.Write(spec)
})
})
return nil
}
// apiDocsPath validates http.api_docs.path with the uploads prefix rules: an
// absolute URL path without scheme, host, query, wildcard or dot segments.
func apiDocsPath(raw string) (string, error) {
if strings.TrimSpace(raw) == "" {
return defaultAPIDocsPath, nil
}
path, err := uploadsPrefix(raw)
if err != nil {
return "", fmt.Errorf("surf: http.api_docs.path must be an absolute URL path such as /api-docs (no scheme, host, query or wildcard), got %q", raw)
}
return path, nil
}
// apiDocsJSON converts a YAML or JSON OpenAPI document to compact JSON with
// the source key order. A body starting with { must be valid JSON; anything
// else is read as YAML (key order kept). The result must be a JSON object.
func apiDocsJSON(raw []byte) ([]byte, error) {
src := bytes.TrimSpace(raw)
if len(src) == 0 {
return nil, errors.New("the spec is empty")
}
var converted []byte
if src[0] == '{' {
if !json.Valid(src) {
return nil, errors.New("the spec starts with { but is not valid JSON")
}
converted = src
} else {
out, err := yaml.YAMLToJSON(src)
if err != nil {
return nil, fmt.Errorf("the spec is not valid YAML: %w", err)
}
converted = out
}
var buf bytes.Buffer
if err := json.Compact(&buf, converted); err != nil {
return nil, fmt.Errorf("the spec does not convert to JSON: %w", err)
}
if buf.Len() == 0 || buf.Bytes()[0] != '{' {
return nil, errors.New("the spec is not an object")
}
return buf.Bytes(), nil
}
// apiDocsPageHandler renders the Swagger UI page with a fresh CSP nonce on
// every script tag, the two pinned assets and the inline start-up script.
func apiDocsPageHandler(title, specURL string) http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
nonceBytes := make([]byte, 18)
if _, err := rand.Read(nonceBytes); err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
return
}
nonce := base64.RawURLEncoding.EncodeToString(nonceBytes)
var page bytes.Buffer
if err := apiDocsPage.Execute(&page, struct{ Title, Nonce, SpecURL string }{title, nonce, specURL}); err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
return
}
h := w.Header()
h.Set("Content-Type", "text/html; charset=utf-8")
h.Set("X-Content-Type-Options", "nosniff")
h.Set("Referrer-Policy", "no-referrer")
// The nonce is the only script source: the two jsDelivr scripts
// carry it next to their SRI hash, so no other jsDelivr script runs.
h.Set("Content-Security-Policy", "script-src 'nonce-"+nonce+"'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'")
_, _ = w.Write(page.Bytes())
}
}

Some files were not shown because too many files have changed in this diff Show More