diff --git a/.planning/STATE.md b/.planning/STATE.md index 145407b..f9c8073 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -4,16 +4,16 @@ milestone: v1.0 current_phase: "19.3" current_phase_name: Framework transport and infrastructure for WavePath (INSERTED) status: executing -stopped_at: Completed 19.3-04-PLAN.md -last_updated: "2026-10-15T17:17:13.871Z" +stopped_at: 19.3-05 Task 7 release checkpoint (tasks 1-6 done, gate green; awaiting release / tag-only / hold) +last_updated: "2026-10-15T18:01:50.208Z" last_activity: 2026-10-15 last_activity_desc: Phase 19.3 execution started -state_head: 6f9f2e20a42996474165747c090e87ca29a7bfa1 +state_head: 8f5325300c94fe7fb18e60b47102bc2443a0a17c progress: total_phases: 40 completed_phases: 31 total_plans: 190 - completed_plans: 186 + completed_plans: 187 gsd_state_version: "1.0" milestone_name: milestone --- @@ -823,6 +823,8 @@ Recent decisions affecting current work: - [Phase 19.3]: pact.ReorderAfterSave runs inside cabana's reorder transaction; reorderPost uses lagoon.Transaction so AfterCommit fires - [Phase 19.3]: Journal golem15.journal.site_cache.* kept as deprecated aliases for one release; framework keys win - [Phase 19.3]: 19.3-04 Task 4 reply push-all: plugins, layers, summercms master pushed; layer tags vue-faq/menu v0.2.0, vue-journal v0.4.0; no framework tag +- [Phase 19.3]: 19.3-05: phase gate scripts/check-phase19.3.sh requires every new test by name (go test -json / TAP detectors); --all green incl. Docker +- [Phase 19.3]: 19.3-05: miniredis v2.40.0 test-only in modules/cooler (Task 4 reply: approved) ### Pending Todos @@ -901,6 +903,6 @@ Items acknowledged and carried forward from previous milestone close: ## Session Continuity -Last session: 2026-10-15T17:17:13.150Z -Stopped at: Completed 19.3-04-PLAN.md -Resume file: None +Last session: 2026-10-15T18:01:46.843Z +Stopped at: 19.3-05 Task 7 release checkpoint (tasks 1-6 done, gate green; awaiting release / tag-only / hold) +Resume file: .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-SUMMARY.md diff --git a/.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-SUMMARY.md b/.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-SUMMARY.md index 591ebdc..409c0dc 100644 --- a/.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-SUMMARY.md +++ b/.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-05-SUMMARY.md @@ -1,23 +1,218 @@ --- phase: 19.3-framework-transport-and-infrastructure-for-wavepath plan: 05 +subsystem: testing +tags: [gate, bouncer, turnstile, cors, api-docs, cooler, redis, miniredis, rate-limit, s3, build-tags, site-cache, nuxt-layers, release] status: in-progress + +requires: + - phase: 19.3-01 + provides: "modules/cooler (caches, Redis rate-limit store, Install, FromApp), surf store lookup" + - phase: 19.3-02 + provides: "bouncer token cookies and Turnstile, surf CORS hardening, CORS profiles, api-docs" + - phase: 19.3-03 + provides: "summer.yaml build.tags, attach S3 build tag, DiskExtension, public_url" + - phase: 19.3-04 + provides: "cooler site-cache purger, pact.ReorderAfterSave, plugin purge helpers, layer purge route" +provides: + - "scripts/check-phase19.3.sh: fail-closed gate with self-test, go, docker, plugins, layers and docs stages and --all" + - "SC-4 unit tests for every piece of new code of the phase, in summercms.go, three plugins and three layers" + - "github.com/alicebob/miniredis/v2 v2.40.0 as a test-only dependency (human-approved)" + - "19.3-VALIDATION.md per-task map filled, wave_0_complete true" +affects: [19.3 release (Task 7), /msd-verify-work 19.3, /msd-validate-phase 19.3] + +actuals: + tokens: 53300 + tasks: 6 + commits: 8 +plan_head_before: 6035ccc194a74a9124ece17ce3c8301689cbc8ac +plan_head_after: 8f5325300c94fe7fb18e60b47102bc2443a0a17c + +tech-stack: + added: + - "github.com/alicebob/miniredis/v2 v2.40.0 (test-only; brings github.com/yuin/gopher-lua v1.1.1 indirect)" + patterns: + - "One behaviour table run through several stores (parity adapter: real-time sleep for MemoryStore and Docker Redis, FastForward for miniredis)" + - "Gate stages require tests by exact top-level name through go test -json and TAP detectors; a skip, a missing name or zero tests fails" + - "Route handler specs in node --test: Nitro auto-imports stubbed on globalThis, the real route file imported" + +key-files: + created: + - scripts/check-phase19.3.sh + - modules/bouncer/cookie_test.go + - modules/bouncer/turnstile_test.go + - modules/surf/cors_profile_test.go + - modules/surf/apidocs_test.go + - modules/cabana/reorder_hook_test.go + - internal/build/manifest_tags_test.go + - modules/lagoon/attach/s3_test.go + - modules/lagoon/attach/diskext_test.go + - modules/cooler/memory_test.go + - modules/cooler/redis_test.go + - modules/cooler/ratelimit_test.go + - modules/cooler/install_test.go + - modules/cooler/docker_test.go + - ../sm-journal-plugin/site_cache_keys_test.go + - ../sm-faq-plugin/controllers/site_cache_test.go + - ../sm-menu-plugin/controllers/site_cache_test.go + modified: + - modules/surf/store_lookup_test.go + - modules/lagoon/attach/store_test.go + - modules/lagoon/attach/url_test.go + - modules/cooler/sitecache_test.go + - modules/cooler/README.md + - go.mod + - go.sum + - ../sm-journal-plugin/site_cache_admin_test.go + - ../vue-faq-layer/tests/sitePurge.test.ts + - ../vue-menu-layer/tests/sitePurge.test.ts + - ../vue-journal-layer/tests/sitePurge.test.ts + - .planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/19.3-VALIDATION.md + +key-decisions: + - "Phase gate reuses check-phase18.1.sh's go test -json detector: a skip, a missing name, no tests, zero tests or a build failure fails the stage" + - "miniredis is test-only: only _test.go files in modules/cooler import it (Task 4 reply: approved)" + - "Journal's plan-04 Docker test renamed to TestCategoriesAdminFrameworkKeysWin so the unit test owns TestSiteCacheFrameworkKeysWin" + - "Release proposal: framework tag v0.1.27 (highest v0.1.26 plus one, free on origin); nothing pushed or tagged before the Task 7 reply" + +requirements-completed: [] + +duration: 41min (plan wall time so far, both executor runs) +completed: pending (Task 7 release decision) --- -# Phase 19.3 Plan 05: unit tests, phase gate and release decision (in progress) +# Phase 19.3 Plan 05: unit tests, phase gate and release decision Summary + +**Fail-closed gate `scripts/check-phase19.3.sh` requiring every new 19.3 test by name across the framework, three plugins and three Nuxt layers. Green on all six stages, Docker included. Release v0.1.27 waits on the human's decision.** ## Checkpoint replies ### Task 4 (miniredis legitimacy check) -Reply, verbatim, recorded before any go.mod change of Task 5: +Reply, verbatim, recorded (commit b139651) before any go.mod change of Task 5: > approved -Orchestrator instruction with it: install github.com/alicebob/miniredis/v2@v2.40.0 as test-only (only `_test.go` files in modules/cooler may import it). +Orchestrator instruction with it: install github.com/alicebob/miniredis/v2@v2.40.0 as test-only (only `_test.go` files in modules/cooler may import it). Verified after Task 5: `grep -n 'github.com/alicebob/miniredis/v2 v2.40.0' go.mod` prints line 10; `grep -rl 'alicebob/miniredis' --include='*.go' modules | grep -vc '_test.go$'` prints 0. ### Task 7 (release decision) -Pending. +Pending. Proposed tag: **v0.1.27**. The reply, the tag and every pushed hash go here. - +## Performance + +- **Started:** 2026-10-15T17:19:19Z (first executor run; resumed after the Task 4 checkpoint) +- **Tasks:** 6 of 7 complete (Task 7 is the blocking release checkpoint) +- **Framework commits since the plan ledger:** 8 (6035ccc..8f53253) + +## Accomplishments + +- The gate has stages `--self-test`, `go`, `docker`, `plugins`, `layers` and `docs`, plus `--all`. A stage fails when a required test is missing, skipped or red. `--all` reports NOT RUN docker when no Docker daemon answers, and fails instead when PHASE193_REQUIRE_DOCKER=1. +- The cooler suites cover the memory and Redis caches and the rate-limit parity table, run through surf.MemoryStore, miniredis and real Redis. They also cover concurrency, the fallback on a Redis error, the key prefixes, Install, FromApp, and the site-cache purge on Postgres (rollback, commit, nested transactions, a 500 answer, a slow receiver and an unreachable one). +- In the plugins: the FAQ and menu `purgeSiteCache` helper tests, and the Journal's deprecated-key tests (framework keys win; alias keys deprecated or invalid; no keys). +- In the layers: the shared purge spec grows from 6 to 11 tests, byte-identical in all three layers. New ones include route-handler specs for 404, 401 and 200, the purge count and the log line, timing-safe comparison across bearer lengths, and parse edge cases. + +## Task Commits + +| Task | Repo | Commit | Message | +|------|------|--------|---------| +| 1 (tracer) | summercms.go | 619916b | test(19.3-05): phase gate with bouncer cookie and Turnstile unit suites | +| 2 | summercms.go | 055f51c | test(19.3-05): surf CORS, profiles, api-docs and store lookup; cabana reorder hook | +| 3 | summercms.go | 38f6d65 | test(19.3-05): build tags, S3 tag guard, DiskExtension and public_url | +| (docs) | summercms.go | df7db6c | docs(19.3-05): log S3 object content-type observation in deferred items | +| 4 | summercms.go | b139651 | docs(19.3-05): record the miniredis checkpoint reply | +| 5 | summercms.go | 78ba385 | test(19.3-05): cooler caches, rate-limit parity, Install and after-commit purge | +| 6 | sm-journal-plugin | ebfb710 | test(19.3-05): Journal site-cache keys: framework keys win, alias keys deprecated | +| 6 | sm-faq-plugin | e117bb4 | test(19.3-05): purgeSiteCache does nothing without a transaction or purger | +| 6 | sm-menu-plugin | 6c08f1a | test(19.3-05): purgeSiteCache does nothing without a transaction or purger | +| 6 | vue-faq-layer | 96b6bc0 | test(19.3-05): site purge route handler specs and edge cases | +| 6 | vue-menu-layer | dd0ee35 | test(19.3-05): site purge route handler specs and edge cases | +| 6 | vue-journal-layer | 5936f66 | test(19.3-05): site purge route handler specs and edge cases | +| 6 | summercms.go | f57fc12 | test(19.3-05): gate plugins, layers and docs stages | +| 6 | summercms.go | 8f53253 | docs(19.3-05): validation map filled, wave 0 complete | + +## Test names per repository (all required by the gate) + +- **summercms.go, go stage (-short):** + - bouncer: 17 names (TestLoadCookieConfigDefaults … TestTurnstileNeverLogsSecret). + - surf: 23 names (TestCORSCredentialsWildcardRefused … TestBuildRouterNeverDials, including TestCORSOptionsMatchesLaravel). + - internal/build: 4 names. + - attach: 4 names, run twice (untagged and `-tags summercms_s3`). + - cooler: TestMemoryCache, TestMemoryCacheTTL, TestMemoryCacheBounded, TestMemoryIncrementConcurrent, TestRememberDegrades, TestRedisCache, TestRedisFlushOnlyPrefix, TestRateLimitParity, TestRateLimitConcurrentNeverExceedsMax, TestRateLimitFallbackOnError, TestRateLimitKeyPrefix, TestInstallConfigErrors, TestInstallPingWarning, TestInstallReusesFromApp, TestFromAppLazy, TestInstallSiteCacheValidation, TestSiteCachePurgeConfig, TestSiteCachePurgeRequest, TestSiteCacheFromApp. +- **summercms.go, docker stage:** + - cabana: TestReorderHookRunsInTx, TestReorderHookErrorRollsBack, TestReorderWithoutHookUnchanged. + - attach: TestStoreDiskExtension. + - cooler: TestRateLimitParityDocker, TestRedisCacheDocker, TestSiteCacheAfterCommitOnly, TestSiteCacheFailureNeverFails. +- **Plugins, plugins stage:** + - FAQ and menu `./controllers`: TestPurgeSiteCacheWithoutTx, TestPurgeSiteCacheWithoutPurger. + - Journal: TestSiteCacheFrameworkKeysWin, TestSiteCacheAliasKeysDeprecated, TestSiteCacheNoKeys; `./classes` TestSiteCachePurgeConfig, TestSiteCachePurgeRequest. +- **Plugins, docker stage:** TestFaqAdminPurgesSiteCache, TestMenuAdminPurgesSiteCache, TestCategoriesAdminPurgesSiteCache, TestPostsAdminPurgesSiteCache, TestCategoriesAdminFrameworkKeysWin. +- **Layers, layers stage:** 11 `sitePurge:` names per layer, plus `cmp` of server/lib/siteCachePurge.ts, server/api/__site__/cache/purge.post.ts and tests/sitePurge.test.ts across the three layers. + +## Gate runs + +Run on 2026-10-15 on a Docker host: + +- `bash scripts/check-phase19.3.sh --all`: PASS self-test, go, plugins, layers, docs, docker. "Phase 19.3 gate passed". Docker ran; it was not NOT RUN. +- `go test -count=1 -run '^TestDocsTree$' ./cmd/summer`: ok. `(cd examples/hello && go test ./...)`: ok. Result: release-gate-ok. +- Full `go test -count=1 ./...` (all Docker suites): exit 0, 41 packages ok, no FAIL lines. +- Fail-closed checks: + - Task 1: renaming TestForgetTokenCookie made the go stage exit 1 (reverted, not committed), and `nosuchstage` exits 2 with usage. + - Task 6: a planted consuming-application name in modules/cooler/redis.go made `docs` exit 1, and a one-line drift in vue-menu-layer's spec made `layers` exit 1. Both were reverted and never committed. + +## Decisions Made + +- The gate reuses check-phase18.1.sh's `go test -json` detector. A skip, a missing name, no tests, zero tests or a build failure all fail the stage. +- The cooler Docker suites start one container per test (Postgres for the purge, redis:7-alpine for the rest) instead of a TestMain. Nothing starts before flag parsing, and `-short` skips them inside the test. +- The docs stage checks the lines the phase added (`git diff ^`, `.planning` excluded) for consuming-application names. The names are split in the script so it does not match itself; PHASE193_BASE overrides the base. +- Test expectation for TestStoreDiskExtension: the "js under octet-stream" row expects content_type text/javascript (from the extension when the sniff is generic, the plan 03 rule) and the disk name `.bin` (D-15). + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] Task 3 verify grep could never match** +- **Found during:** Task 3 +- **Issue:** `grep -q -- '^--- PASS: TestOpenBucketS3$'` cannot match because `go test -v` appends the duration. +- **Fix:** checked with `grep -qE '^--- PASS: ( |$)'` instead (T3-VERIFY-OK). The gate itself uses the JSON detector. +- **Commit:** 38f6d65 + +**2. [Rule 3 - Blocking] Duplicate test name in the Journal test binary** +- **Found during:** Task 6 +- **Issue:** plan 04 had added a Docker test `TestSiteCacheFrameworkKeysWin` in package journal_test, and the plan gives the same name to the new unit test in package journal. `-run '^TestSiteCacheFrameworkKeysWin$'` matched both, so under `-short` one skipped and the gate's skip detector failed the stage. +- **Fix:** renamed the Docker test to `TestCategoriesAdminFrameworkKeysWin` (it saves a category with both key sets) and required it in the docker stage. The unit test keeps the plan's name. +- **Files modified:** ../sm-journal-plugin/site_cache_admin_test.go +- **Commit:** ebfb710 + +**3. [Discretion] Branch the plugin helper unit tests cannot reach** +- `TestPurgeSiteCacheWithoutPurger` covers a nil app, a nil app result and an app without keys. A unit test cannot reach the branch "inside a transaction, no purger": cabana has no exported constructor for a context carrying a transaction, and this plan allows no production changes. The plugins' Docker admin suites reach it. Logged in deferred-items.md. + +No production code changed in this plan. + +## Issues Encountered + +- go-redis logs `failed to dial after 5 attempts` to stderr on each failing call against a stopped Redis. On a dead Redis, `Cache.Remember` waits for those retries (about 1.7 s) unless the caller's context has a deadline. Logged in deferred-items.md as a follow-up. The earlier follow-up, a committed test for the Redis-outage fallback of the rate-limit store, is now TestRateLimitFallbackOnError: each attempt answers from the fallback within about AttemptTimeout, with one warning naming the endpoint. +- Earlier finding (df7db6c), unchanged: S3 object metadata carries the sniffed content type, so an HTML body uploaded as note.txt is stored with Content-Type text/html. + +## Known Stubs + +None. + +## Release sheet (Task 7, awaiting the human's reply) + +- **Proposed tag:** v0.1.27. The current latest framework tag is v0.1.26 (5c2b0e1, 2026-10-14). `git ls-remote --tags origin refs/tags/v0.1.27` prints nothing, so the tag is free. +- **Preflight:** framework paths clean. `git fetch origin --tags` done. `HEAD..origin/master` is empty. origin/master is af9dad6. +- **Public surfaces the tag publishes:** + - cooler and its keys `redis.url`, `cache.*` and `http.rate_limit.store`, plus `site_cache.purge_url` and `site_cache.purge_token`. + - `auth.cookie.*` and the bouncer cookie and Turnstile API. + - `http.cors` hardening, `http.cors_profiles` and `cors:`. + - `turnstile.*` and `http.api_docs.*`. + - summer.yaml `build.tags` and the `summercms_s3` tag. + - `storage.uploads.public_url` and `pact.ReorderAfterSave`. +- **Framework commits to push:** origin/master..HEAD, which is plan 04's 6f9f2e2 and 6035ccc plus this plan's commits from 619916b on (with the final SUMMARY commit). v0.1.26..HEAD holds the whole phase. +- **Plugin and layer commits not pushed yet** (all test-only, from this plan): sm-faq-plugin e117bb4, sm-menu-plugin 6c08f1a, sm-journal-plugin ebfb710, vue-faq-layer 96b6bc0, vue-menu-layer dd0ee35, vue-journal-layer 5936f66. The layer tags v0.2.0 and v0.4.0 from plan 04 are already on origin. +- **App commits from plan 04 Task 5 not pushed yet:** vue-grzybyfunkcjonalne-app 0ee97d0 (push first), then sm-grzybyfunkcjonalne-app f9cafe2, d604509, 85939c3 and 129c0df. +- **Plugin go.mod:** the three plugins require summercms v0.1.23 (with a local replace). After "release" they move to v0.1.27 through `GOWORK=off go get` and `go mod tidy`. +- Apps building against ../summercms.go need `go mod tidy` once for go-redis, otter and the AWS SDK modules (sm-grzybyfunkcjonalne-app already has f9cafe2). + + diff --git a/.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/deferred-items.md b/.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/deferred-items.md index 316e9c9..f09207e 100644 --- a/.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/deferred-items.md +++ b/.planning/phases/19.3-framework-transport-and-infrastructure-for-wavepath/deferred-items.md @@ -16,3 +16,5 @@ Out-of-scope findings logged by executors. Not fixed in the plan that found them ## From 19.3-05 - Security observation (not a test failure; plan 03 kept the stored `content_type` rule by design): `attach.Store` writes the blob with the sniffed content type as object metadata, so an HTML body uploaded as `note.txt` is stored as `.txt` with `Content-Type: text/html`, and a generic sniff takes the type from the extension (`app.js` with binary bytes is stored as `.bin` with `text/javascript`). The surf uploads route neutralises this (nosniff, sandbox CSP, attachment disposition), but with `storage.uploads.public_url` pointing at an S3 bucket the bucket serves the object's stored Content-Type directly, so such a file renders as HTML on the bucket or CDN host. `docs/services/storage.md`'s deploy note says a CDN "picks the Content-Type from the file extension", which is not true for S3 object metadata. A follow-up could write a non-rendering content type (or `Content-Disposition: attachment`) for non-inline types to the blob metadata, and correct the deploy note. Pinned by `TestStoreDiskExtension` rows "html named txt" and "js under octet-stream". +- Redis outage latency (follow-up from 19.3-01 now pinned by tests): `TestRateLimitFallbackOnError` asserts a rate-limit attempt against a stopped Redis answers from the fallback within `AttemptTimeout` (about 500 ms per throttled request during an outage). `Cache.Remember` on the Redis driver has no internal bound: against a stopped Redis it waits for go-redis's dial retries (about 1.7 s, five dial attempts) unless the caller's context has a deadline, and go-redis logs `failed to dial after 5 attempts` to stderr on every failing call, outside slog and the once-per-minute warner. A follow-up could bound Remember's read and write (as Attempt does), lower `MaxRetries`/dial timeout in `OpenRedis`, and route go-redis's logger through slog. +- Coverage note: the plugins' `purgeSiteCache` branch "inside a save transaction but no purger configured" cannot be reached from a unit test, because cabana has no exported way to put a transaction in a context (`withTx` is internal). The plugins' Docker admin suites run every save through it without purge keys; a test-only helper in cabana (like `cabana.WithFieldWrites`) would let a unit test reach it.