docs(19.3-06): gap closure summary, release handed off to the phase session
This commit is contained in:
Jakub Zych
@@ -0,0 +1,256 @@
|
||||
---
|
||||
phase: 19.3-framework-transport-and-infrastructure-for-wavepath
|
||||
plan: 06
|
||||
subsystem: infra
|
||||
tags: [gap-closure, cors, cors-profiles, vary, bouncer, cookies, csp, api-docs, attach, public_url, turnstile, cooler, site-cache, purge, nuxt-layers, gate]
|
||||
status: complete
|
||||
|
||||
requires:
|
||||
- phase: 19.3-02
|
||||
provides: "bouncer token cookies and Turnstile, surf CORS hardening, CORS profiles, api-docs"
|
||||
- phase: 19.3-03
|
||||
provides: "attach public_url and the S3 build tag"
|
||||
- phase: 19.3-04
|
||||
provides: "cooler site-cache purger, plugin purge helpers, the shared layer purge route"
|
||||
- phase: 19.3-05
|
||||
provides: "scripts/check-phase19.3.sh fail-closed gate, v0.1.27"
|
||||
provides:
|
||||
- "cors:<profile> applied outermost by Router.wrap: every response of a profiled route (guard 401/403, Turnstile 422/503, 429, 413, recovered 500) carries the profile's CORS headers; Vary merged (CR-01)"
|
||||
- "credentialed CORS refuses wildcard-host and unanchored origins at boot, globally and per profile (WR-04)"
|
||||
- "production refuses an inferred insecure token cookie; the zero CookieConfig writes SameSite=Lax and Secure (WR-05, WR-09)"
|
||||
- "nonce-only script-src on /api-docs (WR-06)"
|
||||
- "attach.ErrProtectedNeedsPrivateBucket: protected uploads refused while storage.uploads.public_url is set, plus a boot warning (WR-07)"
|
||||
- "boot warning for turnstile routes without a secret (WR-08)"
|
||||
- "site-cache purge client never follows redirects (WR-02); PurgeSiteCacheAfterCommit queues onto a coalescing per-purger worker; cooler.SiteCachePurger.Wait; Handle.Close drains (WR-01)"
|
||||
- "siteCache.allowProxied (NUXT_SITE_CACHE_ALLOW_PROXIED) in vue-faq-layer 0.3.0, vue-menu-layer 0.3.0, vue-journal-layer 0.5.0 (local commits, untagged) (WR-03)"
|
||||
- "11 new unit tests and 2 layer specs required by name in scripts/check-phase19.3.sh"
|
||||
- "19.3-REVIEW-DISPOSITION.md: CR-01 and WR-01..WR-09 fixed, open 11 (IN-01..IN-11)"
|
||||
affects: [phase 19.3 main session (docs update, v0.1.28 release), /msd-verify-work 19.3, sm-faq-plugin, sm-menu-plugin, sm-journal-plugin, vue-faq-layer, vue-menu-layer, vue-journal-layer, app layer pins]
|
||||
|
||||
actuals:
|
||||
tokens: 49100
|
||||
tasks: 6
|
||||
commits: 14
|
||||
plan_head_before: e9ad640999fee13dc5be8cd0cd96205e2f2d7cad
|
||||
plan_head_after: 0c8109c6702b75beb62ba627b19d66223120c029
|
||||
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "Route-level middleware that must decorate every response (CORS profile) is wrapped outside recovery, body limit and locale, not placed by its position in the middleware list"
|
||||
- "Credentialed origin rules checked on the parsed regexp (regexp/syntax) plus probe origins outside the configured domains"
|
||||
- "After-commit side effects only enqueue; a per-purger worker keeps one request in flight and coalesces the rest; Wait and Close drain"
|
||||
|
||||
key-files:
|
||||
created: []
|
||||
modified:
|
||||
- modules/surf/router.go
|
||||
- modules/surf/cors.go
|
||||
- modules/surf/apidocs.go
|
||||
- modules/surf/cors_profile_test.go
|
||||
- modules/surf/apidocs_test.go
|
||||
- modules/bouncer/cookie.go
|
||||
- modules/bouncer/cookie_test.go
|
||||
- modules/lagoon/attach/store.go
|
||||
- modules/lagoon/attach/bucket.go
|
||||
- modules/lagoon/attach/url_test.go
|
||||
- modules/cooler/sitecache.go
|
||||
- modules/cooler/cooler.go
|
||||
- modules/cooler/sitecache_test.go
|
||||
- modules/cooler/example_test.go
|
||||
- scripts/check-phase19.3.sh
|
||||
- modules/surf/README.md
|
||||
- modules/bouncer/README.md
|
||||
- modules/lagoon/README.md
|
||||
- modules/cooler/README.md
|
||||
- docs/services/routing.md
|
||||
- docs/services/authentication.md
|
||||
- docs/services/storage.md
|
||||
- docs/services/caching.md
|
||||
- docs/architecture/request-lifecycle.md
|
||||
- ../sm-faq-plugin/site_cache_admin_test.go
|
||||
- ../sm-menu-plugin/site_cache_admin_test.go
|
||||
- ../sm-journal-plugin/site_cache_admin_test.go
|
||||
- ../sm-journal-plugin/classes/site_cache_test.go
|
||||
- ../vue-{faq,menu,journal}-layer/server/lib/siteCachePurge.ts
|
||||
- ../vue-{faq,menu,journal}-layer/server/api/__site__/cache/purge.post.ts
|
||||
- ../vue-{faq,menu,journal}-layer/tests/sitePurge.test.ts
|
||||
- ../vue-{faq,menu,journal}-layer/nuxt.config.ts
|
||||
- ../vue-{faq,menu,journal}-layer/README.md
|
||||
- ../vue-{faq,menu,journal}-layer/package.json
|
||||
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-REVIEW-DISPOSITION.md
|
||||
- .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md
|
||||
|
||||
key-decisions:
|
||||
- "Release of v0.1.28, the plugin require bumps and the layer tags handed off to the phase 19.3 main session (human reply at Task 7); this plan pushed and tagged nothing"
|
||||
- "CR-01: Router.wrap resolves the profile via routeCORSProfile, skips the cors:<profile> entry in its loop and wraps the recovered handler in profileMiddleware just inside recordPattern; bufferedResponse.commit merges Vary through addVary"
|
||||
- "WR-03: explicit opt-in (siteCache.allowProxied) instead of restricting purge_url to loopback, which would refuse https alias URLs Journal deployments may carry"
|
||||
- "WR-01: installSiteCache returns the purger in effect (including one a plugin published before Install), so Handle.Close drains whichever purger the app uses"
|
||||
|
||||
patterns-established:
|
||||
- "Outermost route decoration: a CORS profile is applied around recovery, so refusals and panics carry its headers"
|
||||
- "Queued after-commit purges with coalescing and an explicit Wait for tests and shutdown"
|
||||
|
||||
requirements-completed: [SC-1, SC-3, SC-4, D-03, D-04, D-05, D-10, D-11, D-12, D-17, D-18]
|
||||
|
||||
duration: 40min
|
||||
completed: 2026-10-15
|
||||
---
|
||||
|
||||
# Phase 19.3 Plan 06: Review gap closure (CR-01, WR-01..WR-09) Summary
|
||||
|
||||
**cors:<profile> now decorates every response of a profiled route including guard, Turnstile, throttle, body-limit and panic refusals; credentialed CORS, cookie Secure, api-docs CSP, protected uploads and Turnstile secrets are checked at boot; site-cache purges are redirect-free, queued and coalesced, and the layers accept proxied purges on an explicit allowProxied opt-in. Release handed off to the phase 19.3 main session.**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** about 40 min of execution (two executor contexts, split after Task 4)
|
||||
- **Started:** 2026-10-15T20:20:38Z
|
||||
- **Last task commit:** 2026-10-15T20:51:13Z
|
||||
- **Completed:** 2026-10-15T21:00Z (SUMMARY)
|
||||
- **Tasks:** 6 of 7 executed; Task 7 (release checkpoint) handed off
|
||||
- **Files modified:** 24 in summercms.go (excluding .planning), 4 test files in three plugins, 6 files in each of three layers
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- **CR-01 (verification gap closed):** a cross-origin SPA reads every refusal of a `cors:<profile>` route: 401/403 from a guard listed before the profile or set on the group, Turnstile 422/503, 429, 413 and a recovered 500 in JSON and raw groups all carry Allow-Origin, Allow-Credentials and `Vary: Origin`; a disallowed origin gets `Vary: Origin` only; a handler's own `Vary` is merged.
|
||||
- **WR-04:** with `supports_credentials`, wildcard entries other than a whole leftmost host label, unanchored patterns and patterns matching probe origins outside the configured domains fail boot naming key and index, globally and per profile. Non-credentialed configs unchanged (`TestCORSOptionsMatchesLaravel` green).
|
||||
- **WR-05 / WR-09:** production refuses a token cookie whose Secure resolves to false, set or inferred from an http/unset `app.url`; zero `CookieConfig` writes `token`, `Path=/`, HttpOnly, SameSite=Lax, Secure.
|
||||
- **WR-06:** `/api-docs` sends `script-src 'nonce-…'` as the only source; every script tag carries the nonce; SRI unchanged.
|
||||
- **WR-07:** `attach.Store` returns `attach.ErrProtectedNeedsPrivateBucket` before any blob is written when `is_public` is false and `storage.uploads.public_url` is set; `OpenBucket` warns at boot.
|
||||
- **WR-08:** `BuildRouter` logs one warning naming the route count when turnstile routes exist and the secret is empty.
|
||||
- **WR-02 / WR-01:** the purge client never follows a redirect; `PurgeSiteCacheAfterCommit` only enqueues after commit; one request in flight per purger, saves committed meanwhile collapse into one follow-up (bulk delete of N rows sends at most two); `cooler.SiteCachePurger.Wait`; `Handle.Close` drains. Exported cooler purge names, Journal aliases and the FAQ, menu and Journal controllers unchanged.
|
||||
- **WR-03:** the shared layer purge route honours `siteCache.allowProxied` / `NUXT_SITE_CACHE_ALLOW_PROXIED`; the three shared files are byte-identical (sha256 `0ea9f0f9…193b260e`) with updated hash pins; `docs/services/caching.md` pairs an https `purge_url` on another host with the flag.
|
||||
- **Gate:** 11 Go tests and 2 layer specs added to the required lists; the whole gate passes with Docker.
|
||||
|
||||
## Task Commits
|
||||
|
||||
| Task | Name | Commit (repository, hash) |
|
||||
|------|------|---------------------------|
|
||||
| 1 | Tracer: cors:<profile> headers on every response of the route (CR-01) | summercms.go `0723536` (fix) |
|
||||
| 2 | Credentialed CORS refuses wildcard-host and unanchored origins (WR-04) | summercms.go `e32940d` (fix) |
|
||||
| 2 | Production refuses an inferred insecure token cookie (WR-05) | summercms.go `4dae833` (fix) |
|
||||
| 2 | Zero CookieConfig writes SameSite=Lax and Secure (WR-09) | summercms.go `85d8b6a` (fix) |
|
||||
| 3 | api-docs CSP allows only nonce-carrying scripts (WR-06) | summercms.go `f705343` (fix) |
|
||||
| 3 | Refuse protected uploads into a public_url bucket (WR-07) | summercms.go `caa5b5d` (fix) |
|
||||
| 3 | Warn at boot when turnstile routes have no secret (WR-08) | summercms.go `d0a580c` (fix) |
|
||||
| 4 | Site-cache purge never follows redirects (WR-02) | summercms.go `50a584c` (fix) |
|
||||
| 4 | Queue site-cache purges after commit and coalesce them (WR-01) | summercms.go `db6a7f5` (fix) |
|
||||
| 4 | Plugin tests wait for the queued purge (WR-01) | sm-faq-plugin `e52b286`, sm-menu-plugin `4b42978`, sm-journal-plugin `4f633f0` (test) |
|
||||
| 4 | Progress record after Task 4 | summercms.go `3bb6d61` (docs) |
|
||||
| 5 | Purge route accepts proxied requests on siteCache.allowProxied (WR-03) | vue-faq-layer `a229f6a` (0.3.0), vue-menu-layer `c4639c2` (0.3.0), vue-journal-layer `f5d7aac` (0.5.0) (feat) |
|
||||
| 5 | Pair a remote purge URL with the site's allowProxied (WR-03) | summercms.go `4d4e379` (docs) |
|
||||
| 6 | Unit tests for the review fixes | summercms.go `8d80580` (test) |
|
||||
| 6 | Gate requires the review-fix tests | summercms.go `b2f4044` (test) |
|
||||
| 6 | Review disposition and validation map after the gap closure | summercms.go `0c8109c` (docs) |
|
||||
| 7 | Release gate | handed off, no commit (see below) |
|
||||
|
||||
summercms.go commits measured from the ledger: `git rev-list --count e9ad640..0c8109c` = 14 (12 code/docs plus 2 planning-docs commits).
|
||||
|
||||
## Task 7: handed off
|
||||
|
||||
The release checkpoint was reached with every pre-check green (see Verification). The human's reply, verbatim:
|
||||
|
||||
> "ok, differently. there is main agent that was doing 19.3 phase and he was waiting for gap closure to update /docs and make a release then. in other words, you can finalize your gap closure, write SUMMARY.md and leave him the rest - he will update docs for whole phase 19.3, create a tag and do a release."
|
||||
|
||||
Outcome: none of the three options (release / tag-only / hold) was executed by this plan. Proposed tag at the checkpoint: **v0.1.28**. Nothing was pushed or tagged in any repository; no plugin `go.mod` was changed.
|
||||
|
||||
## Handoff to the phase 19.3 main session
|
||||
|
||||
State at handoff:
|
||||
- summercms.go `master` at `0c8109c` plus this plan's SUMMARY and tracking commits, 20+ commits ahead of `origin/master` (includes planning commits since v0.1.27). Highest tag v0.1.27; v0.1.28 free on origin.
|
||||
- sm-faq-plugin, sm-menu-plugin, sm-journal-plugin: each 1 commit ahead of `origin/main` (the WR-01 test commits above).
|
||||
- vue-faq-layer, vue-menu-layer, vue-journal-layer: each 1 commit ahead of `origin/main`; v0.3.0 / v0.3.0 / v0.5.0 free on origin.
|
||||
|
||||
What remains, in order:
|
||||
1. Whole-phase docs update for 19.3 (`docs/`); this plan's docs changes are already committed (routing, authentication, storage, caching, request-lifecycle, the four module READMEs).
|
||||
2. Annotated tag `v0.1.28` on summercms.go `master`; push `master`, then the tag.
|
||||
3. In each plugin (sm-faq-plugin, sm-menu-plugin, sm-journal-plugin): `GOWORK=off go get git.golem15.com/golem15/summercms@v0.1.28`, `GOWORK=off go mod tidy`, `GOWORK=off go vet ./...`, `GOWORK=off go test -count=1 ./...`; commit go.mod and go.sum as `chore: require summercms v0.1.28` (plan wording: `chore(19.3-06): require summercms v0.1.28`); push main. Required: the plugin tests now call `cooler.SiteCachePurger.Wait`, so the plugins do not build on origin without go.work until this bump.
|
||||
4. Push the three layer mains, then create and push annotated tags v0.3.0 (vue-faq-layer), v0.3.0 (vue-menu-layer), v0.5.0 (vue-journal-layer).
|
||||
5. Follow-up, out of scope here: app layer pins (sites get `siteCache.allowProxied` only after their pins move) and plugin submodule pointer bumps in apps.
|
||||
6. g15office phase comment (skipped here; to be posted with the release).
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `modules/surf/router.go`, `modules/surf/cors.go`: profile applied outermost, Vary merge, credentialed origin rules with probe origins, Turnstile boot warning.
|
||||
- `modules/surf/apidocs.go`: nonce on every script tag, nonce-only script-src.
|
||||
- `modules/bouncer/cookie.go`: production Secure rule, zero-config defaults.
|
||||
- `modules/lagoon/attach/store.go`, `bucket.go`: `ErrProtectedNeedsPrivateBucket`, boot warning.
|
||||
- `modules/cooler/sitecache.go`, `cooler.go`: no-redirect client, queue and coalescing worker, `Wait`, drain on `Handle.Close`.
|
||||
- `scripts/check-phase19.3.sh`: CR-01 tests in SURF_TESTS, plan-06 tests in BOUNCER/SURF/ATTACH/COOLER/DOCKER_COOLER lists and LAYER_REQUIRE.
|
||||
- Module READMEs and `docs/services/{routing,authentication,storage,caching}.md`, `docs/architecture/request-lifecycle.md`.
|
||||
- Plugin `site_cache_admin_test.go` (three plugins) and `sm-journal-plugin/classes/site_cache_test.go`: wait for the queued purge.
|
||||
- Layers: shared `siteCachePurge.ts` and `purge.post.ts`, specs with new hash pins, `nuxt.config.ts`, README, version bump.
|
||||
|
||||
## Decisions Made
|
||||
|
||||
- CR-01: `Router.wrap` resolves the profile through `routeCORSProfile` (two profiles stay a boot error), skips the `cors:<profile>` entry after the unknown-profile check and wraps the recovered handler in `profileMiddleware` just inside `recordPattern`; `bufferedResponse.commit` merges each comma-separated `Vary` value through `addVary`.
|
||||
- WR-04: anchoring checked on the parsed expression (`regexp/syntax`): every alternative must begin with `^` and end with `$`, so `^a|b$` is refused. Wildcards must be `<literal scheme>://*.<rest>` with one `*`. Probe origins: https://cors-probe.invalid, http://cors-probe.invalid, https://cors-probe.com, null.
|
||||
- WR-05: the explicit-false message stays; the inferred case says "auth.cookie.secure resolves to false in the production environment" and names app.url.
|
||||
- WR-09: zero-config Secure is decided before Name and Path are filled; any unset SameSite becomes Lax.
|
||||
- WR-07: the check sits in `attach.Store` right after the argument checks, before the extension check and before the body is read. In the admin, cabana's `writeFileError` has no branch for the new error, so it reaches the default path: `logFileFailure` logs the error (its message names `storage.uploads.public_url`) and the client gets the generic opaque 500. cabana unchanged, as planned.
|
||||
- WR-08: the warning goes through `accessLogger(app)` with attributes `routes` and `production`.
|
||||
- WR-01: `installSiteCache` returns the purger in effect, also one a plugin published before Install (Journal alias keys). Queued sends use a background context bounded by `SiteCachePurgeTimeout`. Log lines add `coalesced` only when above zero. `Purge` tolerates a nil client field.
|
||||
- WR-03: explicit opt-in, default stays "refuse proxied"; accepts `true` or the string `"true"` from runtimeConfig.
|
||||
- Plugin tests: `purgeReceiver` gains a `drain` hook set by `drainFrom(t, app)`; `take` (so `expectOnePurge` and `expectNone`) waits on `cooler.SiteCacheFromApp(app).Wait` with a 5 s context first. In TestCategoriesAdminFrameworkKeysWin both receivers drain the app's purger.
|
||||
- Task 7: release handed off to the phase 19.3 main session.
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 2 - Docs] request-lifecycle wrapping order**
|
||||
- **Found during:** Task 1
|
||||
- **Issue:** `docs/architecture/request-lifecycle.md` "The wrapping order" list would be wrong after CR-01.
|
||||
- **Fix:** the list gains the CORS profile step.
|
||||
- **Commit:** `0723536`
|
||||
|
||||
**2. [Rule 1 - Test] Production cookie rows moved to development**
|
||||
- **Found during:** Task 2 (WR-05)
|
||||
- **Issue:** `TestLoadCookieConfigDefaults` had production rows with http and empty `app.url` expecting Secure false without error, now an error by design.
|
||||
- **Fix:** those rows run in development; the production case is covered by `TestLoadCookieConfigErrors` and `TestLoadCookieConfigProductionInferredSecure`.
|
||||
- **Commit:** `4dae833`
|
||||
|
||||
**3. [Rule 1 - Test] ExampleSiteCacheFromApp waits for the queued purge**
|
||||
- **Found during:** Task 4 (WR-01)
|
||||
- **Issue:** the example printed the purge synchronously from the receiver.
|
||||
- **Fix:** it calls `cooler.SiteCacheFromApp(app).Wait` after the save; the example controller's `purge` comment says the purge is queued; `docs/services/caching.md` src= copy refreshed with `summer docs:sync`.
|
||||
- **Commit:** `db6a7f5`
|
||||
|
||||
**4. [Rule 1 - Test] Slow-receiver subtest asserts the save is not blocked**
|
||||
- **Found during:** Task 4
|
||||
- **Issue:** "a slow receiver is abandoned and the save stays committed" waited for the 2 s purge.
|
||||
- **Fix:** asserts the save returns within 500 ms and waits for the queue before reading the log.
|
||||
- **Commit:** `db6a7f5`
|
||||
|
||||
**5. Task 7 not executed as specified**
|
||||
- The checkpoint answer was none of release / tag-only / hold; the release moved to the phase 19.3 main session (see Handoff).
|
||||
|
||||
**Total deviations:** 4 auto-fixed (1 docs, 3 test), 1 scope handoff. No CLAUDE.md-driven adjustments.
|
||||
|
||||
## Verification
|
||||
|
||||
- Tasks 1-4: per-task verify PASS (details in 19.3-06-PROGRESS.md): vet in both tag modes, surf/bouncer/lagoon/cooler short suites, `go test -short -race ./modules/cooler`, `check-phase19.3.sh go`, `plugins` and `docker` (RUN). Both CR-01 tests failed against the old router.go. Tracer gate: automated verify re-run green before expansion.
|
||||
- Task 5: `check-phase19.3.sh layers` passed, both new sitePurge specs `ok` in each layer, shared files byte-identical, TestDocsTree and `summer docs:build --check` ok.
|
||||
- Task 6: `PHASE193_REQUIRE_DOCKER=1 bash scripts/check-phase19.3.sh --all` passed; TestDocsTree and `examples/hello` tests passed.
|
||||
- **Fail-closed proof (Task 6):** renaming `TestZeroCookieConfigDefaults` locally made `bash scripts/check-phase19.3.sh go` exit 1 ("refuse: required tests did not pass: TestZeroCookieConfigDefaults"); reverted, never committed.
|
||||
- Task 7 pre-checks: `bash scripts/check-phase19.3.sh --all` passed all stages including docker (RUN); TestDocsTree and examples/hello passed; v0.1.28 free on origin; v0.3.0 / v0.3.0 / v0.5.0 free on the layer origins.
|
||||
|
||||
Acceptance quirks:
|
||||
- `git log --oneline --grep='(WR-04)$'` and `(WR-07)$` print more than three lines: phase-02 commits end with the same suffixes. The three 19.3-06 commits per task are present.
|
||||
- The cooler README Features bullet was reworded in `4d4e379` so `cooler.SiteCachePurger.Wait` appears only in the API row.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
None blocking. Info findings IN-01..IN-11 left open, as planned.
|
||||
|
||||
## User Setup Required
|
||||
|
||||
Sites whose backend reaches the purge route through a TLS-terminating proxy set `NUXT_SITE_CACHE_ALLOW_PROXIED=true` after moving their layer pin (follow-up, see Handoff).
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
Gap closure complete in code, docs and tests. The phase 19.3 main session finishes the whole-phase docs update and the release (Handoff, steps 1-6), then `/msd-verify-work 19.3`.
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
All 14 summercms.go commits, the 3 plugin commits and the 3 layer commits verified with `git cat-file -e`; plan_head_before/after measured from the ledger base e9ad640.
|
||||
Reference in New Issue
Block a user