From f61776d0a6936587bf666bbb3eb4a96d08031ff2 Mon Sep 17 00:00:00 2001 From: Jakub Zych Date: Sat, 19 Sep 2026 21:01:07 +0200 Subject: [PATCH] docs(06-03): complete raw groups, wire, CORS, and body-limit plan - Operator-confirmed 128MiB body limits (134217728); D-18/T-06-13 closed - HTTP-06, HTTP-08, HTTP-09 marked complete --- .planning/REQUIREMENTS.md | 12 +- .planning/ROADMAP.md | 4 +- .planning/STATE.md | 19 +- .../06-03-SUMMARY.md | 216 ++++++++++++++++++ 4 files changed, 236 insertions(+), 15 deletions(-) create mode 100644 .planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-03-SUMMARY.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 7f0364c..3c1f53b 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -55,10 +55,10 @@ Requirements for v1 (the Płytarium port). Each maps to roadmap phases. "User" b - [x] **HTTP-03**: Three mutually exclusive auth groups share the same handlers with different route subsets: JWT under /_fonoteka/api/v1, personal scoped token under /api/v1/fonoteka, and public groups (onboarding, public/{token}, public-wishlist/{token}, invitation inspection) - [x] **HTTP-04**: A rate limiter supports named buckets keyed by a resolver (token id, IP, route param), stacking two limiters on one route, and ports Płytarium's five named buckets and inline throttles 1:1 - [x] **HTTP-05**: An auth guard registry lets plugins add guards (JWT, personal token, OAuth bearer) that all resolve to the same current-user accessor -- [ ] **HTTP-06**: Response conventions are preserved: empty arrays serialize as [], timestamps as +00:00, tri-state booleans keep null, conditional keys are omitted not nulled, and no blanket envelope or error middleware wraps OAuth routes +- [x] **HTTP-06**: Response conventions are preserved: empty arrays serialize as [], timestamps as +00:00, tri-state booleans keep null, conditional keys are omitted not nulled, and no blanket envelope or error middleware wraps OAuth routes - [ ] **HTTP-07**: A guarded outbound fetch helper enforces host allow-lists, byte caps and timeouts for user-supplied URLs (manual cover URL, Discogs cover) -- [ ] **HTTP-08**: OpenAPI is generated from swaggo/swag annotations on handlers and openapi-typescript produces the admin SPA's types -- [ ] **HTTP-09**: CORS and JSON body size limits match the PHP deployment +- [x] **HTTP-08**: OpenAPI is generated from swaggo/swag annotations on handlers and openapi-typescript produces the admin SPA's types +- [x] **HTTP-09**: CORS and JSON body size limits match the PHP deployment ### Authentication and users (AUTH) @@ -188,10 +188,10 @@ Which phases cover which requirements. Updated during roadmap creation. | HTTP-03 | Phase 6 | Complete | | HTTP-04 | Phase 6 | Complete | | HTTP-05 | Phase 6 | Complete | -| HTTP-06 | Phase 6 | Pending | +| HTTP-06 | Phase 6 | Complete | | HTTP-07 | Phase 6 | Pending | -| HTTP-08 | Phase 6 | Pending | -| HTTP-09 | Phase 6 | Pending | +| HTTP-08 | Phase 6 | Complete | +| HTTP-09 | Phase 6 | Complete | | AUTH-01 | Phase 7 | Pending | | AUTH-02 | Phase 7 | Pending | | AUTH-03 | Phase 7 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 97ac58d..847ec58 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -237,7 +237,7 @@ Plans: **Wave 3** *(blocked on 06-02)* -- [ ] 06-03-PLAN.md — Contract surface: raw-group enforcement + route table + route:list, wire response helpers, swag/openapi-typescript pipeline, path-scoped CORS, body limits (blocking human-verify checkpoint for production body-size numbers), oauth group declared raw +- [x] 06-03-PLAN.md — Contract surface: raw-group enforcement + route table + route:list, wire response helpers, swag/openapi-typescript pipeline, path-scoped CORS, body limits (blocking human-verify checkpoint for production body-size numbers), oauth group declared raw **Wave 4** *(blocked on 06-01..06-04)* @@ -410,7 +410,7 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → | 3. First vertical slice — genres end to end | 4/4 | Complete | 2026-09-17 | | 4. CLI scaffolding, i18n and mail | 4/4 | Complete | 2026-09-18 | | 5. Data layer full fidelity | 6/6 | Complete | 2026-09-18 | -| 6. HTTP routing, auth groups and rate limiting | 3/5 | In Progress| | +| 6. HTTP routing, auth groups and rate limiting | 4/5 | In Progress| | | 7. User plugin and authentication | 0/TBD | Not started | - | | 8. OAuth2.1 authorization server | 0/TBD | Not started | - | | 9. Backend admin authentication and schema pipeline | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 3c1d9a7..912d10a 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,14 +3,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: executing -stopped_at: Phase 6 context gathered -last_updated: "2026-09-19T17:46:28.583Z" +stopped_at: Completed 06-03-PLAN.md +last_updated: "2026-09-19T19:00:29.025Z" last_activity: 2026-09-19 progress: total_phases: 15 completed_phases: 5 total_plans: 28 - completed_plans: 26 + completed_plans: 27 percent: 33 --- @@ -26,11 +26,11 @@ See: .planning/PROJECT.md (updated 2026-09-16) ## Current Position Phase: 06 (http-routing-auth-groups-and-rate-limiting) — EXECUTING -Plan: 3 of 5 +Plan: 4 of 5 Status: Ready to execute Last activity: 2026-09-19 -Progress: [█████████░] 93% +Progress: [██████████] 96% ## Performance Metrics @@ -72,6 +72,7 @@ Progress: [█████████░] 93% | Phase 05 P06 | 23 min | 3 tasks | 12 files | | Phase 06 P01 | 25 min | 3 tasks | 26 files | | Phase 06 P02 | 14 min | 3 tasks | 16 files | +| Phase 06 P03 | 20 min | 3 tasks | 24 files | ## Accumulated Context @@ -162,6 +163,10 @@ Recent decisions affecting current work: - [Phase 06]: fonoteka-* buckets live on the app plugin via surf.BucketProvider, not hardcoded in surf — summercms.go must stay Płytarium-agnostic - [Phase 06]: Empty PHP group builders plus test-only boot-probe routes prove middleware strings resolve without 501 shells — D-15: no 501 shells; wrap() only sees routes - [Phase 06]: php_parity.sh pins APP_DEBUG=false; three existing HTML exception fixtures need re-recording — T-06-09 production-shaped error bodies +- [Phase 06]: HasHouseMiddleware is the only plugin-facing house-tag path; Assemble/BuildRouter is the sole RegisterHouseMiddleware caller (D-16) +- [Phase 06]: Production body limits are 134217728/134217728 (128MiB), operator-confirmed 2026-09-19 from nginx client_max_body_size=128M and php.ini post_max_size=128M/upload_max_filesize=128M (D-18, T-06-13) +- [Phase 06]: CORS path globs compile as Laravel nested * because Go path.Match would miss /api/v1/fonoteka/genres (Pitfall 10) +- [Phase 06]: swag v1 Swagger 2 is converted by a local swagger2openapi helper to OpenAPI 3 for openapi-typescript 7; Phase 10 wires types into the admin SPA ### Pending Todos @@ -183,6 +188,6 @@ Items acknowledged and carried forward from previous milestone close: ## Session Continuity -Last session: 2026-09-19T17:45:48.542Z -Stopped at: Phase 6 context gathered +Last session: 2026-09-19T19:00:29.002Z +Stopped at: Completed 06-03-PLAN.md Resume file: None diff --git a/.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-03-SUMMARY.md b/.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-03-SUMMARY.md new file mode 100644 index 0000000..6edfc91 --- /dev/null +++ b/.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-03-SUMMARY.md @@ -0,0 +1,216 @@ +--- +phase: 06-http-routing-auth-groups-and-rate-limiting +plan: 03 +subsystem: auth +tags: [surf, wire, cors, body-limit, raw-group, openapi, route-table, house-middleware] + +requires: + - phase: 06-http-routing-auth-groups-and-rate-limiting + provides: pact.Router verbs and name:param factories, inv_token + jwt.auth groups, throttle factory and five fonoteka buckets +provides: + - Router.GroupRaw with sticky raw inheritance and registration-time house-envelope refusal + - pact.HasHouseMiddleware collected only from Assemble/BuildRouter's plugin loop + - Router.Routes() / RouteInfo and surf.RouteListCommand (route:list) + - wire.WriteJSON, WriteOpaque500, Time (+00:00), TriBool, Slice + - path-scoped CORS matching config/cors.php (JWT group: no headers; token group: *) + - per-route http.MaxBytesReader body limits with operator-confirmed 128MiB production numbers + - raw oauth group declared with zero handlers; inv.must-change-password house-tagged + - committed OpenAPI document from ListGenres swag annotations, validated by openapi-typescript +affects: [06-05-tests, phase-08-oauth, phase-10-admin-spa] + +tech-stack: + added: [] + patterns: + - HasHouseMiddleware is the only plugin-facing house-tag path; plugins never call Register* + - Assemble = BuildRouter + compile so route:list inspects without serving + - Laravel-style CORS globs (api/* matches nested segments), not Go path.Match + - swag v1 Swagger 2 converted by a local swagger2openapi helper to OpenAPI 3 for openapi-typescript 7 + +key-files: + created: + - surf/routetable.go + - surf/routetable_test.go + - surf/routelist_command.go + - surf/cors.go + - surf/cors_test.go + - surf/bodylimit.go + - surf/bodylimit_test.go + - wire/response.go + - wire/response_test.go + - scripts/check-openapi.sh + - scripts/swagger2openapi.go + - docs/openapi.json + - plugins/golem15/fonoteka/routes_cors_test.go + modified: + - pact/capabilities.go + - surf/router.go + - surf/router_test.go + - surf/middleware_test.go + - internal/build/build.go + - internal/build/build_test.go + - plugins/golem15/fonoteka/plugin.go + - plugins/golem15/fonoteka/routes.go + - plugins/golem15/fonoteka/controllers/genre_controller.go + - config/http.yaml + - parity/genre_security_test.go + +key-decisions: + - "HasHouseMiddleware is the only way a plugin declares house-tagged names; Assemble/BuildRouter is the sole RegisterHouseMiddleware caller (D-16)" + - "Assemble split into BuildRouter + compile so route:list can read Routes() without serving" + - "CORS path globs compile as Laravel * (nested segments), because Go path.Match would miss /api/v1/fonoteka/genres" + - "Production body limits are 134217728/134217728 (128MiB binary), operator-confirmed 2026-09-19 from nginx 128M + php.ini 128M/128M (D-18, T-06-13)" + - "swag v1 emits Swagger 2; a local swagger2openapi converter produces OpenAPI 3 for openapi-typescript 7.13.0" + +patterns-established: + - "Raw groups refuse house-tagged middleware at wrap()/Assemble time and recover with a bare 500" + - "House vs ordinary middleware is a plugin capability split (Middlewares vs HouseMiddlewares), not a Register* call site" + - "Non-raw routes wrap http.MaxBytesReader from http.body_limits.default_bytes; body.limit:N overrides innermost; raw routes stay uncapped at this layer" + - "genre_controller writeJSON/writeOpaque500 are one-line delegations to wire" + +requirements-completed: [HTTP-06, HTTP-08, HTTP-09] + +duration: 20 min +completed: 2026-09-19 +--- + +# Phase 6 Plan 03: Raw groups, wire, CORS, and body-limit Summary + +**Registration-time raw-group house-middleware refusal, route table + route:list, wire JSON helpers, path-scoped CORS, and operator-confirmed 128MiB body limits (134217728) with a committed OpenAPI document from ListGenres** + +## Performance + +- **Duration:** 20 min active execution (human-verify pause between Task 3 code and close-out not counted) +- **Started:** 2026-09-19T17:55:32Z +- **Completed:** 2026-09-19T18:57:39Z +- **Tasks:** 3 (Task 3 includes the production body-limit checkpoint) +- **Files modified:** 24 + +## Accomplishments + +- `GroupRaw` plus sticky raw inheritance: a raw group cannot spawn a non-raw child. House-envelope names are declared only through `pact.HasHouseMiddleware`; `BuildRouter` registers them and `wrap()` fails boot with `surf: raw group cannot use house-envelope middleware %q (plugin %q)`. Raw panics write a bare 500; non-raw keep the house JSON body. +- `Router.Routes()` returns a defensive `[]RouteInfo` (method, pattern, plugin, middleware, raw). `surf.RouteListCommand` renders it; generated `main` registers `route:list` next to `serve`. Mutual-exclusivity of jwt.auth vs inv_token/inv.scope is asserted over the assembled table (closes T-06-02 / T-06-10). +- `wire.WriteJSON` matches genre_controller byte-for-byte (`SetEscapeHTML(false)`, trailing-newline trim). `wire.Time` marshals Carbon `+00:00` never `Z`; `TriBool` nulls when invalid; `Slice` never returns nil. `ListGenres` delegates and carries swag annotations; `docs/openapi.json` is committed and validated by openapi-typescript 7.13.0. +- Path-scoped CORS copies `config/cors.php` (`api/*`, `_user/api/*`, `_journal/api/*`, `_feedback/api/*`, `oauth/mcp/*`). JWT `/_fonoteka/api/v1/genres` emits no `Access-Control-Allow-Origin`; personal-token `/api/v1/fonoteka/genres` emits `*`. +- Body limits: non-raw routes wrap `http.MaxBytesReader` from `http.body_limits.default_bytes`; `body.limit:N` overrides innermost; raw routes are uncapped at this layer. Production values are operator-confirmed 2026-09-19: nginx `client_max_body_size=128M`, php.ini `post_max_size=128M` and `upload_max_filesize=128M` → **134217728 / 134217728** (128 × 1024 × 1024). D-18 / T-06-13 closed. No `INTERIM` remains in production yaml. +- `golem15.fonoteka` implements `HouseMiddlewares()` with `inv.must-change-password`. The oauth group is `GroupRaw("/", Use(), empty)` with zero handlers; Phase 8 attaches `throttle:fonoteka-oauth-token` / `throttle:fonoteka-oauth-register` per-route. + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Raw group enforcement, house-middleware capability, route table, route:list** - `fa7e6d1` (feat, summercms.go) +2. **Task 2: wire helpers (framework)** - `9ecf2d1` (feat, summercms.go) +3. **Task 2: OpenAPI from ListGenres and wire delegation (app)** - `db51df6` (feat, fonoteka.go) +4. **Task 3 code: path-scoped CORS and per-route body limits (framework)** - `30539b9` (feat, summercms.go) +5. **Task 3 code: raw oauth group, house-tagged password middleware, INTERIM yaml** - `cdc324a` (feat, fonoteka.go) +6. **Task 3 checkpoint: production 128MiB body limits** - `73671a8` (feat, fonoteka.go) + +**Plan metadata:** (this commit) + +## Files Created/Modified + +- `pact/capabilities.go` — `GroupRaw` on Router; `HasHouseMiddleware` +- `surf/router.go` — raw flag, house-tagged registration, BuildRouter, per-route recoverBare/recoverJSON, body-limit wrap, path-scoped CORS +- `surf/routetable.go` — `RouteInfo` and `Routes()` defensive copy +- `surf/routelist_command.go` — `route:list` table command +- `surf/cors.go` — `CORSConfig`, Laravel glob compile, path-scoped middleware +- `surf/bodylimit.go` — `http.MaxBytesReader` default + `body.limit:N` override +- `wire/response.go` — WriteJSON, WriteOpaque500, Time, TriBool, Slice +- `internal/build/build.go` — generated main appends `surf.RouteListCommand` +- `plugins/golem15/fonoteka/plugin.go` — `HouseMiddlewares()`; `inv.must-change-password` moved out of `Middlewares()` +- `plugins/golem15/fonoteka/routes.go` — empty `GroupRaw` oauth landing spot +- `plugins/golem15/fonoteka/controllers/genre_controller.go` — wire delegation + swag comments +- `plugins/golem15/fonoteka/routes_cors_test.go` — CORS, house-capability, raw-refusal, route-table exclusivity on real plugins +- `config/http.yaml` — cors.php paths; body_limits 134217728/134217728 operator-confirmed 2026-09-19 +- `scripts/check-openapi.sh` / `scripts/swagger2openapi.go` / `docs/openapi.json` — swag → OpenAPI 3 → openapi-typescript + +## Decisions Made + +- House-tagged middleware is a plugin capability (`HasHouseMiddleware`), never a `Register*` call from plugin code (D-16, T-06-11). +- `Assemble` is `BuildRouter` then `compile()`, so `route:list` inspects the table without serving. +- CORS `*` globs match nested path segments the way Laravel `api/*` does; Go `path.Match` would leave `/api/v1/fonoteka/genres` without CORS and fail Pitfall 10. +- Body-size numbers are not a guess: operator read production nginx + php.ini on 2026-09-19; both keys are 134217728 (D-18, T-06-13). +- swag v1.16.6 stays (STACK.md rejects swag v2 RC); a local converter bridges Swagger 2 → OpenAPI 3 for openapi-typescript 7.13.0. Phase 10 owns wiring types into the admin SPA. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] Laravel glob instead of Go path.Match for CORS paths** +- **Found during:** Task 3 (path-scoped CORS) +- **Issue:** The plan specified `path.Match("api/*", trimmedPath)`. Go's `path.Match` does not let `*` cross `/`, so `api/*` would not match `api/v1/fonoteka/genres` and the personal-token group would emit no CORS headers — failing Pitfall 10 and HTTP-09. +- **Fix:** `compileLaravelGlob` treats `*` as `.*` (Laravel `fnmatch` / `api/*` semantics) against the leading-slash-stripped path. +- **Files modified:** `surf/cors.go`, `surf/cors_test.go` +- **Verification:** `go test ./surf -run TestCORS`; fonoteka `TestCORSPathScopedOnAssembledRouter` (JWT empty ACAO, token `*`) +- **Committed in:** `30539b9` (Task 3) + +**2. [Rule 3 - Blocking] Local swagger2openapi converter so openapi-typescript 7 can validate swag v1 output** +- **Found during:** Task 2 (check-openapi.sh) +- **Issue:** swag v1.16.6 emits Swagger 2.0 (`docs/swagger.json`). openapi-typescript 7.13.0 requires OpenAPI 3.x. STACK.md forbids swag v2 (RC only). +- **Fix:** `scripts/swagger2openapi.go` converts the swag JSON to OpenAPI 3; the script writes `docs/openapi.json` and deletes the intermediate swagger.json. No new dependency. +- **Files modified:** `scripts/swagger2openapi.go`, `scripts/check-openapi.sh` +- **Verification:** `bash scripts/check-openapi.sh` exits 0; `docs/openapi.json` contains `/_fonoteka/api/v1/genres` +- **Committed in:** `db51df6` (Task 2) + +--- + +**Total deviations:** 2 auto-fixed (2 Rule 3 blocking) +**Impact on plan:** Both required for HTTP-08/HTTP-09 to be true. No scope creep; no extra libraries. + +## Issues Encountered + +None beyond the two auto-fixes. The human-verify checkpoint is planned flow, not an issue: operator reported all three production values as 128 MB on 2026-09-19. + +## Checkpoint Resolution (D-18 / T-06-13) + +Operator-confirmed 2026-09-19 from the production host: + +| Source | Value | +|--------|-------| +| nginx `client_max_body_size` | 128M | +| php.ini `post_max_size` | 128M | +| php.ini `upload_max_filesize` | 128M | + +Converted with binary megabytes (nginx `m` / PHP `M` = 1024×1024): `128 * 1024 * 1024 = 134217728`. + +`fonoteka.go/config/http.yaml`: + +- `http.body_limits.default_bytes: 134217728` +- `http.body_limits.upload_bytes: 134217728` + +INTERIM comments replaced with the confirmation comment. Body-limit tests assert against config-loaded / fixture YAML values and were not changed. + +Re-run: + +- `go test ./surf -count=1 -run BodyLimit` (summercms.go) — pass +- `go test ./plugins/golem15/fonoteka/... -count=1 -short` (fonoteka.go) — pass + +## Auth Gates + +Task 3 `checkpoint:human-verify` for production body-size numbers. Operator confirmed; numbers applied; plan closed. Not a deviation. + +## Known Stubs + +None that block this plan. The oauth `GroupRaw` has zero handlers by design (Phase 8 owns the two POST routes). Empty jwt_locale / onboarding / public_invitation / public_share builders were already declared in 06-02. + +## User Setup Required + +None remaining. The production-host body-size read is done (2026-09-19). + +## Next Phase Readiness + +- Phase 8 can mount `/oauth/mcp/token` and register on the existing raw group without house envelope/error middleware. +- Phase 10 can consume `docs/openapi.json` (already valid TypeScript via openapi-typescript). +- 06-05 owns full coverage, the remaining route-table assertions, and `06-SECURITY-REVIEW.md`. +- No blockers. + +## Self-Check: PASSED + +- FOUND: surf/routetable.go, surf/routelist_command.go, surf/cors.go, surf/bodylimit.go, wire/response.go, pact/capabilities.go +- FOUND: fonoteka.go/config/http.yaml default_bytes=134217728 upload_bytes=134217728, zero INTERIM +- FOUND: fa7e6d1, 9ecf2d1, 30539b9 (summercms.go); db51df6, cdc324a, 73671a8 (fonoteka.go) +- `go test ./surf -count=1 -run BodyLimit` pass; `go test ./plugins/golem15/fonoteka/... -short` pass + +--- +*Phase: 06-http-routing-auth-groups-and-rate-limiting* +*Completed: 2026-09-19*