diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 799b145..7f0364c 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -53,7 +53,7 @@ Requirements for v1 (the Płytarium port). Each maps to roadmap phases. "User" b - [x] **HTTP-01**: Plugins register route groups on net/http ServeMux with typed params and regex constraints; unknown and malformed ids both return 404 on ownership-scoped resources - [x] **HTTP-02**: Plugins register named middleware that other plugins reference by name; the pipeline order is recover, CORS, locale, auth group, must-change-password, org context, rate limit, handler - [x] **HTTP-03**: Three mutually exclusive auth groups share the same handlers with different route subsets: JWT under /_fonoteka/api/v1, personal scoped token under /api/v1/fonoteka, and public groups (onboarding, public/{token}, public-wishlist/{token}, invitation inspection) -- [ ] **HTTP-04**: A rate limiter supports named buckets keyed by a resolver (token id, IP, route param), stacking two limiters on one route, and ports Płytarium's five named buckets and inline throttles 1:1 +- [x] **HTTP-04**: A rate limiter supports named buckets keyed by a resolver (token id, IP, route param), stacking two limiters on one route, and ports Płytarium's five named buckets and inline throttles 1:1 - [x] **HTTP-05**: An auth guard registry lets plugins add guards (JWT, personal token, OAuth bearer) that all resolve to the same current-user accessor - [ ] **HTTP-06**: Response conventions are preserved: empty arrays serialize as [], timestamps as +00:00, tri-state booleans keep null, conditional keys are omitted not nulled, and no blanket envelope or error middleware wraps OAuth routes - [ ] **HTTP-07**: A guarded outbound fetch helper enforces host allow-lists, byte caps and timeouts for user-supplied URLs (manual cover URL, Discogs cover) @@ -186,7 +186,7 @@ Which phases cover which requirements. Updated during roadmap creation. | HTTP-01 | Phase 3 | Complete | | HTTP-02 | Phase 3 | Complete | | HTTP-03 | Phase 6 | Complete | -| HTTP-04 | Phase 6 | Pending | +| HTTP-04 | Phase 6 | Complete | | HTTP-05 | Phase 6 | Complete | | HTTP-06 | Phase 6 | Pending | | HTTP-07 | Phase 6 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 1e519d8..97ac58d 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -233,7 +233,7 @@ Plans: **Wave 2** *(blocked on 06-01)* -- [ ] 06-02-PLAN.md — Rate limiting: fixed-window Store/Limiter, trusted-proxy client IP, five fonoteka buckets, PublicShareHeaders, remaining route groups declared +- [x] 06-02-PLAN.md — Rate limiting: fixed-window Store/Limiter, trusted-proxy client IP, five fonoteka buckets, PublicShareHeaders, remaining route groups declared **Wave 3** *(blocked on 06-02)* @@ -410,7 +410,7 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → | 3. First vertical slice — genres end to end | 4/4 | Complete | 2026-09-17 | | 4. CLI scaffolding, i18n and mail | 4/4 | Complete | 2026-09-18 | | 5. Data layer full fidelity | 6/6 | Complete | 2026-09-18 | -| 6. HTTP routing, auth groups and rate limiting | 2/5 | In Progress| | +| 6. HTTP routing, auth groups and rate limiting | 3/5 | In Progress| | | 7. User plugin and authentication | 0/TBD | Not started | - | | 8. OAuth2.1 authorization server | 0/TBD | Not started | - | | 9. Backend admin authentication and schema pipeline | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 4505edb..3c1d9a7 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -4,13 +4,13 @@ milestone: v1.0 milestone_name: milestone status: executing stopped_at: Phase 6 context gathered -last_updated: "2026-09-19T17:23:58.639Z" +last_updated: "2026-09-19T17:46:28.583Z" last_activity: 2026-09-19 progress: total_phases: 15 completed_phases: 5 total_plans: 28 - completed_plans: 24 + completed_plans: 26 percent: 33 --- @@ -26,11 +26,11 @@ See: .planning/PROJECT.md (updated 2026-09-16) ## Current Position Phase: 06 (http-routing-auth-groups-and-rate-limiting) — EXECUTING -Plan: 2 of 5 +Plan: 3 of 5 Status: Ready to execute Last activity: 2026-09-19 -Progress: [█████████░] 86% +Progress: [█████████░] 93% ## Performance Metrics @@ -71,6 +71,7 @@ Progress: [█████████░] 86% | Phase 05 P05 | 18 min | 3 tasks | 18 files | | Phase 05 P06 | 23 min | 3 tasks | 12 files | | Phase 06 P01 | 25 min | 3 tasks | 26 files | +| Phase 06 P02 | 14 min | 3 tasks | 16 files | ## Accumulated Context @@ -156,6 +157,11 @@ Recent decisions affecting current work: - [Phase 06]: NewJWTGuard reuses bearerToken/Verify/write401 so Registry.Middleware(jwt) is byte-identical to bouncer.Middleware (D-10) - [Phase 06]: oauth is not registered this plan; only jwt and inv_token (D-09) - [Phase 06]: Personal-token genres fixture body matches isolated seedGenres; CORS * deferred to D-18 +- [Phase 06]: Concrete limiter is FixedWindowLimiter; surf.Limiter interface remains the unused Phase 3 seam — D-03 D-05 naming collision with pre-existing Limiter interface +- [Phase 06]: Trusted-proxy list is a NewFixedWindowLimiter constructor argument, never a setter — Named-bucket Key closures and inline N,M must share one trusted list +- [Phase 06]: fonoteka-* buckets live on the app plugin via surf.BucketProvider, not hardcoded in surf — summercms.go must stay Płytarium-agnostic +- [Phase 06]: Empty PHP group builders plus test-only boot-probe routes prove middleware strings resolve without 501 shells — D-15: no 501 shells; wrap() only sees routes +- [Phase 06]: php_parity.sh pins APP_DEBUG=false; three existing HTML exception fixtures need re-recording — T-06-09 production-shaped error bodies ### Pending Todos @@ -177,6 +183,6 @@ Items acknowledged and carried forward from previous milestone close: ## Session Continuity -Last session: 2026-09-19T17:19:44.685Z +Last session: 2026-09-19T17:45:48.542Z Stopped at: Phase 6 context gathered Resume file: None diff --git a/.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-02-SUMMARY.md b/.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-02-SUMMARY.md new file mode 100644 index 0000000..890e134 --- /dev/null +++ b/.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-02-SUMMARY.md @@ -0,0 +1,171 @@ +--- +phase: 06-http-routing-auth-groups-and-rate-limiting +plan: 02 +subsystem: auth +tags: [surf, limiter, throttle, clientip, public-share, rate-limit, fixed-window] + +requires: + - phase: 06-http-routing-auth-groups-and-rate-limiting + provides: pact.Router verbs and name:param factories, bouncer.Registry, inv_token + inv.scope, GET genres on both auth groups +provides: + - surf.FixedWindowLimiter, Store/MemoryStore, Bucket, BucketProvider, and the built-in throttle middleware factory + - surf.ClientIP / TrustedProxies trusted-proxy resolver (D-04) + - five fonoteka-* buckets registered by golem15.fonoteka with PHP names, limits, and keys + - PublicShareHeaders 429 rewrite and X-Robots-Tag/Cache-Control on the public-share group + - empty jwt_locale, onboarding, public_invitation, public_share group builders (D-15) +affects: [06-03-route-groups, 06-05-tests] + +tech-stack: + added: [] + patterns: + - parameterized throttle:name and throttle:N,M via RegisterMiddlewareFactory + - BucketProvider type-asserted in surf.Assemble (not a pact interface; avoids surf/pact cycle) + - Laravel tooManyAttempts-before-hit fixed window; success responses carry X-RateLimit-Limit/Remaining + +key-files: + created: + - surf/limiter.go + - surf/limiter_store.go + - surf/clientip.go + - surf/limiter_test.go + - surf/clientip_test.go + - plugins/golem15/fonoteka/middleware/public_share_headers.go + - plugins/golem15/fonoteka/middleware/public_share_headers_test.go + - plugins/golem15/fonoteka/routes_bucket_test.go + modified: + - surf/router.go + - pact/capabilities.go + - plugins/golem15/fonoteka/plugin.go + - plugins/golem15/fonoteka/routes.go + - config/http.yaml + - parity/php_parity.sh + - .planning/ROADMAP.md + - .planning/REQUIREMENTS.md + +key-decisions: + - "Concrete limiter is FixedWindowLimiter; surf.Limiter interface remains the unused Phase 3 seam (D-03, D-05)" + - "Trusted-proxy list is a NewFixedWindowLimiter constructor argument, never a setter" + - "fonoteka-* buckets live on the app plugin via surf.BucketProvider, not hardcoded in surf" + - "Empty PHP group builders plus test-only boot-probe routes prove middleware strings resolve without 501 shells (D-15)" + - "php_parity.sh pins APP_DEBUG=false; three existing HTML exception fixtures need re-recording" + +patterns-established: + - "Assemble registers the surf throttle factory, then HasMiddleware, HasMiddlewareFactories, BucketProvider, then Routes" + - "PublicShareHeaders buffers the inner response so 429 bodies can be rewritten after the limiter writes headers" + - "Inline throttle:N,M keys by u: when bouncer.User is set, else Host|ClientIP" + +requirements-completed: [HTTP-04] + +duration: 14 min +completed: 2026-09-19 +--- + +# Phase 6 Plan 02: Fixed-window limiter, five buckets, and PublicShareHeaders Summary + +**Laravel ThrottleRequests wire parity in surf.FixedWindowLimiter (success + 429 headers, first-hit-wins fixed window), five fonoteka-* buckets registered by the app plugin, PublicShareHeaders 429 rewrite, and APP_DEBUG=false on the PHP recorder** + +## Performance + +- **Duration:** 14 min +- **Started:** 2026-09-19T17:29:07Z +- **Completed:** 2026-09-19T17:42:58Z +- **Tasks:** 3 +- **Files modified:** 16 + +## Accomplishments + +- `surf.Store` / `MemoryStore` and `surf.FixedWindowLimiter` port Laravel's tooManyAttempts-before-hit, first-hit-wins window, stacked `throttle:` middleware, and header placement (`X-RateLimit-Limit`/`Remaining` on 200; `Retry-After`/`X-RateLimit-Reset` on 429 with `{"message":"Too Many Attempts."}`) +- `surf.ClientIP` honors `X-Forwarded-For` only when `RemoteAddr` is inside `http.trusted_proxies`; empty list is RemoteAddr only +- `golem15.fonoteka` implements `surf.BucketProvider` with the five PHP buckets (60/30/30/60/120) and attaches `throttle:fonoteka-api-token` to the personal-token genres group; `PublicShareHeaders` rewrites 429 to `{"error":"Too many requests"}` and sets `X-Robots-Tag` / `Cache-Control` +- Remaining Phase-6-scope groups (`jwt_locale`, `onboarding`, `public_invitation`, `public_share`) are declared as empty builders; `TestAllRouteGroupsBoot` wraps their real middleware stacks; `php_parity.sh` exports `APP_DEBUG=false`; ROADMAP/REQUIREMENTS say five buckets + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Fixed-window Store, FixedWindowLimiter, trusted-proxy ClientIP, and router wiring** - `68c6ab8` (feat, summercms.go) +2. **Task 2: Register the five buckets, attach throttle:fonoteka-api-token, port PublicShareHeaders, declare remaining route groups** - `0acb3f6` (feat, fonoteka.go) +3. **Task 3 code: APP_DEBUG=false and boot-smoke coverage for all groups** - `fbf143e` (feat, fonoteka.go) +4. **Task 3 docs: HTTP-04 named bucket count to five** - `ab20ad0` (docs, summercms.go) + +**Plan metadata:** (this commit) + +## Files Created/Modified + +- `surf/limiter_store.go` — Store interface and mutex-guarded MemoryStore (Hit / TooManyAttempts / AvailableIn) +- `surf/limiter.go` — Bucket, BucketProvider, FixedWindowLimiter, named and inline throttle:N,M +- `surf/clientip.go` — ClientIP and TrustedProxies +- `surf/router.go` — limiter field, throttle factory, BucketProvider loop, noOpLimit removed; Limiter interface kept +- `pact/capabilities.go` — comment that BucketProvider is type-asserted in Assemble +- `plugins/golem15/fonoteka/plugin.go` — Buckets() and public.share-headers +- `plugins/golem15/fonoteka/routes.go` — token-group throttle plus four empty group builders +- `plugins/golem15/fonoteka/middleware/public_share_headers.go` — 429 rewrite and noindex/no-store +- `config/http.yaml` — `trusted_proxies: []` +- `parity/php_parity.sh` — `APP_DEBUG=false` +- `.planning/ROADMAP.md` / `.planning/REQUIREMENTS.md` — five named buckets (D-01) + +## Decisions Made + +- Named the concrete type `FixedWindowLimiter` so it coexists with the retained `surf.Limiter` interface; rate limiting is exclusively `throttle:...` middleware (D-03, D-05) +- Trusted proxies are captured at `NewFixedWindowLimiter` construction so named-bucket keys and inline N,M share one list +- Named `fonoteka-*` buckets are registered by the app plugin via `surf.BucketProvider`; surf stays Płytarium-agnostic +- Empty group builders in `routes.go` plus test-only `/boot-probe/...` routes satisfy D-15 (no 501 shells) while still failing Assemble on unknown throttle/middleware names +- Production-shaped error bodies: `APP_DEBUG=false` from this point; three already-recorded HTML exception fixtures are flagged for re-record (not 429s) + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] -short boot-smoke cannot register inv_token without *gorm.DB** +- **Found during:** Task 3 (`TestAllRouteGroupsBoot -short`) +- **Issue:** `inv_token` is registered only when Boot sees a published `*gorm.DB`. The plan's verify command uses `-short`, which skips testcontainers. +- **Fix:** If the registry has no `inv_token` middleware after Activate, register a stub Guard so wrap() can resolve the personal-token stack. Real TokenGuard remains the production path. +- **Files modified:** `plugins/golem15/fonoteka/routes_bucket_test.go` +- **Verification:** `go test ./plugins/golem15/fonoteka/... -run TestAllRouteGroupsBoot -short` passes +- **Committed in:** `fbf143e` (Task 3) + +**2. [Rule 2 - Missing Critical] Empty groups never reach wrap(), so throttle names would not fail boot** +- **Found during:** Task 3 (boot-smoke design) +- **Issue:** `r.Group(..., Use(...), func(g) {})` with zero Get/Post registers no routes; Assemble would succeed even if `public.share-headers` or a bucket name were missing. +- **Fix:** `bootProbePlugin` mounts throwaway `GET /boot-probe/.../ok` routes using each group's real middleware list (not PHP paths, not 501 shells). Assemble of user + fonoteka + probe fails on unknown names. +- **Files modified:** `plugins/golem15/fonoteka/routes_bucket_test.go` +- **Verification:** TestAllRouteGroupsBoot passes; missing-bucket Assemble already covered in surf tests +- **Committed in:** `fbf143e` (Task 3) + +--- + +**Total deviations:** 2 auto-fixed (1 Rule 2, 1 Rule 3) +**Impact on plan:** Both keep the -short boot-smoke honest without adding production 501 routes. No limiter-algorithm change. + +## Issues Encountered + +- `go test ./... -short` in fonoteka.go fails `TestParityCorpus` coverage (`ported 2 passing 0`) because `parityDB` skips under `testing.Short()` while the coverage subtest still requires passing==ported. Pre-existing; not caused by this plan. Plan verification (`./plugins/golem15/fonoteka/... -short` and `./surf/... -short`) is green. Extra `X-RateLimit-*` headers are outside tide's compare allow-list, so they will not fail a full (non-short) replay. + +## Fixture audit (T-06-09) + +`CACHE_DRIVER=array` still means live PHP 429s cannot be recorded from this harness. Three existing non-429 fixtures contain Winter HTML exception pages (`exception-name-block` / stack trace) and were recorded under debug: + +- `parity/fixtures/routes/POST___fonoteka_api_v1_invitations_{token}_accept_jwt.yaml` +- `parity/fixtures/routes/POST___fonoteka_api_v1_tokens_jwt.yaml` +- `parity/fixtures/routes/POST___fonoteka_api_v1_oauth_consent_jwt.yaml` + +Re-record these against the now-pinned `APP_DEBUG=false` isolated PHP when those routes are ported. Do not treat them as 429 contract fixtures. + +## User Setup Required + +None - no external service configuration required. + +## Next Phase Readiness + +Ready for 06-03 (route table, raw OAuth group, path-scoped CORS). Throttle factory, buckets, ClientIP, and public-share 429 shape are load-bearing. oauth group remains unregistered (D-09). + +## Self-Check: PASSED + +- FOUND: surf/limiter.go, surf/limiter_store.go, surf/clientip.go, plugins/golem15/fonoteka/middleware/public_share_headers.go, plugins/golem15/fonoteka/routes_bucket_test.go +- FOUND: 68c6ab8, 0acb3f6, fbf143e, ab20ad0 +- FOUND: zero `noopLimiter|noOpLimit` in surf/; zero "seven named" in ROADMAP.md/REQUIREMENTS.md; APP_DEBUG=false in php_parity.sh +- `go vet ./...` green in both repos; `go test ./surf/... -short` and `go test ./plugins/golem15/fonoteka/... -short` green + +--- +*Phase: 06-http-routing-auth-groups-and-rate-limiting* +*Completed: 2026-09-19*