Files
summercms/.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-VERIFICATION.md

191 lines
20 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
phase: 06-http-routing-auth-groups-and-rate-limiting
verified: 2026-09-19T19:45:00Z
status: gaps_found
score: 11/12 must-haves verified
overrides_applied: 0
mvp_mode_note: "ROADMAP mode is mvp but the phase goal is not a User Story (gsd-sdk user-story.validate valid=false). Verification used the technical roadmap contract, not a fabricated user-flow table."
gaps:
- truth: "All five named rate-limit buckets are enforced with the documented keys and limits, including a route stacking two limiters."
status: failed
reason: "The only live personal-token route lists inv_token, inv.scope:read, then throttle:fonoteka-api-token. wrap() applies names last-to-first, so InvScope runs before throttle and returns 401 without calling next. Unauthenticated GET /api/v1/fonoteka/genres never consumes the 60/min bucket. PHP attaches throttle on the group and inv.scope on the route (throttle first). Independently confirms 06-REVIEW.md CR-01; no test asserts 429 on the deny path."
artifacts:
- path: fonoteka.go/plugins/golem15/fonoteka/routes.go
issue: "Use(\"inv_token\", \"inv.scope:read\", \"throttle:fonoteka-api-token\") inverts PHP throttle-then-scope; InvScope short-circuits before the bucket"
- path: summercms.go/surf/router.go
issue: "wrap() last-to-first is the correct onion; the call-site order is wrong"
- path: fonoteka.go/plugins/golem15/fonoteka/middleware/token_scope.go
issue: "missing/invalid token writes 401 and returns; next (throttle) never runs"
missing:
- "Reorder the personal-token group to inv_token, throttle:fonoteka-api-token, inv.scope:read (inv_token must stay before throttle so tok:<id> is set)"
- "Add a test that 61 unauthenticated requests to assembled GET /api/v1/fonoteka/genres return 429 {\"message\":\"Too Many Attempts.\"}"
- "Use the same order on any future inv.scope + throttle route"
deferred:
- truth: "public/onboarding groups reachable without auth"
addressed_in: "Phase 13"
evidence: "Phase 13 goal ports onboarding/public/invitation routes and their public rate-limit buckets. Phase 6 declared empty group builders per D-15 (zero routes, not 501 shells)."
- truth: "unknown and malformed ids on ownership-scoped resources both return 404"
addressed_in: "Phase 12"
evidence: "Phase 12 ports Collections and Albums ownership-scoped endpoints. The router primitive already 404s (surf/params.go constrain, TestTypedIDRouteReturns404)."
- truth: "user-supplied cover URL fetch call sites (manual cover URL, Discogs cover)"
addressed_in: "Phase 12 / Phase 14"
evidence: "Phase 12 cover handling; Phase 14 Discogs client. 06-04 D-13 ships fetchguard only — no ManualCoverUrlFetcher or CoverImporter call site this phase."
---
# Phase 6: HTTP routing, auth groups and rate limiting Verification Report
**Phase Goal:** The three mutually exclusive auth groups (JWT, personal token, public/onboarding) share handlers with correct route subsets, named rate-limit buckets are ported 1:1, and OAuth/RFC routes are structurally exempted from any house envelope or error middleware. Security-load-bearing — auth guard registry, rate limiting and the SSRF-guarded outbound fetch helper all live here; apply the security-review agent.
**Verified:** 2026-09-19T19:45:00Z
**Status:** gaps_found
**Re-verification:** No — initial verification
**MVP mode:** ROADMAP marks this phase `mode: mvp`, but `gsd-sdk query user-story.validate` returns `valid=false` (goal is a technical contract, not `As a …, I want to …, so that ….`). User Flow Coverage is omitted; verification follows the five roadmap success criteria.
`06-REVIEW.md` (issues_found, 1 critical / 7 warning) is advisory. CR-01 is listed as a gap only because it independently falsifies HTTP-04 / SC2 in the live route table.
## Goal Achievement
### Observable Truths
| # | Truth | Status | Evidence |
| --- | --- | --- | --- |
| 1 | Same handler serves JWT `GET /_fonoteka/api/v1/genres` and personal-token `GET /api/v1/fonoteka/genres`; public/onboarding without auth; unknown/malformed ids 404 on ownership-scoped resources | ✓ VERIFIED (deferred subclauses) | `routes.go` binds one `handler := controllers.ListGenres(p.app)` to both groups. `TestGenresSharedHandler` (Postgres) returns equal 200 bodies. Public/onboarding groups exist as empty builders (D-15) — deferred to Phase 13. Router `constrain` 404s malformed/unknown ids (`TestTypedIDRouteReturns404`) — ownership-scoped resources deferred to Phase 12. |
| 2 | All five named rate-limit buckets are enforced with the documented keys and limits, including a route stacking two limiters | ✗ FAILED | Buckets exist with PHP names/limits/keys (`plugin.go` `Buckets()`: api-token 60 `tok:<id>` else IP, oauth-token 30 `oauthtok:<ip>`, oauth-register 30 `oauthreg:<ip>`, public-token 60 `pubtok:<token>`, public-ip 120 IP). Stacking is proven on a fixture (`TestFixedWindowLimiterStackedBuckets`). The live token genres route does **not** enforce `fonoteka-api-token` on 401/403: wrap last-to-first + `Use("inv_token", "inv.scope:read", "throttle:fonoteka-api-token")` puts InvScope outside throttle. InvScope returns on missing User. Zero tests fire 429 on that deny path. |
| 3 | Response conventions: empty arrays `[]`, timestamps `+00:00`, tri-state booleans keep `null`, conditional keys omitted not nulled; OAuth-group route carries no house envelope, verified by route-registration inspection | ✓ VERIFIED | `wire.Slice` / `Time` / `TriBool` / `WriteJSON` tested (`wire/response_test.go`). ListGenres preallocates `make([]GenreAggregate, 0)` and delegates to `wire.WriteJSON`. Raw panic is bare 500 (`TestRawGroupPanicBare500`); house panic is opaque JSON. `GroupRaw("/", surf.Use(), …)` declared; `TestRawGroupHouseMiddlewareRefusedAtBuild` and `TestRawGroupRefusesHouseMiddlewareOnRealPlugins` fail boot if house MW is attached. Isolation test requires Raw on oauth patterns if present. |
| 4 | Guarded outbound fetch helper rejects a non-allow-listed host and enforces a byte cap and timeout on a user-supplied cover URL fetch | ✓ VERIFIED (call sites deferred) | `fetchguard.Fetch`: allow-list before dial (`TestFetchAllowHostsRejectsUnknownHostBeforeDial`, dotted-suffix bypass test), dial-time `Control` + `isReservedOrPrivate` (`TestFetchPrivateIPBlockedInBothModes`), streaming `LimitReader` cap (`TestFetchTooLargeIsStreaming`), https-only, no redirects, 10MiB/10s defaults, typed reasons. No production caller this phase (D-13) — Phase 12/14. |
| 5 | OpenAPI from swag annotations; `openapi-typescript` produces valid TS; CORS and JSON body-size limits match the PHP deployment | ✓ VERIFIED | `genre_controller.go` swag annotations → committed `fonoteka.go/docs/openapi.json` (OpenAPI 3.0.3, path `/_fonoteka/api/v1/genres`). `npx openapi-typescript@7.13.0 docs/openapi.json -o /dev/null` exit 0. CORS: `_fonoteka` no ACAO, `/api/v1/fonoteka/genres` `Access-Control-Allow-Origin: *` (`TestCORSPathScopedOnAssembledRouter`). Body limits operator-confirmed 134217728 / 134217728, no INTERIM (`TestProductionBodyLimitsOperatorConfirmed`). |
| 6 | Auth guard registry: jwt + inv_token resolve to one `bouncer.User`; 401/403 token bodies match PHP; oauth guard not registered | ✓ VERIFIED | `bouncer.Registry` + `NewJWTGuard` + `TokenGuard`. `TestGenresSharedHandler` missing/unknown token → 401 `{"error":"Invalid token"}`; write-only token → 403 `{"error":"Missing required scope: read"}`. Duplicate/unknown/neither-interface fail boot. Grep of `Register(` finds only `"jwt"` and `"inv_token"`. |
| 7 | Parameterized (`name:param`) middleware is a surf factory, not a fixed name table | ✓ VERIFIED | `RegisterMiddlewareFactory` for `throttle`, `body.limit`, plugin `inv.scope`. `strings.Cut` in `wrap()`. |
| 8 | Fixed-window limiter matches Laravel tooManyAttempts-before-hit; success headers and 429 headers; PublicShareHeaders 429 body; ClientIP trusted-proxy rules | ✓ VERIFIED | `TestFixedWindowLimiterMemoryStoreWindow`, `FirstHitWins`, `SuccessHeaders`, `TooManyAttemptsHeaders`, `TestPublicShareHeadersRewrites429`, `TestClientIPRejectsSpoofedXFF`. In-process `MemoryStore`; type is `FixedWindowLimiter` (pre-existing `Limiter` interface left in place). |
| 9 | Raw group refuses house-envelope middleware at registration; `HasHouseMiddleware` is the only house registration path | ✓ VERIFIED | `inv.must-change-password` only in `HouseMiddlewares()` (plugin.go:73-76), not `Middlewares()`. `TestHouseMiddlewareCapabilityOnRealPlugins` boots; `TestRawGroupRefusesHouseMiddlewareOnRealPlugins` fails boot when a raw group names it. |
| 10 | Full assembled route table: zero jwt.auth on `/api/v1/fonoteka`, zero inv_token / inv.scope on `/_fonoteka/api/v1` | ✓ VERIFIED | `TestFullRouteTableAuthGroupMutualExclusivity` walks `BuildRouter` `Routes()` for real `golem15.user` + `golem15.fonoteka`. `route:list` calls `Router.Routes()` (`surf/routelist_command.go`). |
| 11 | Every T-06-01 through T-06-18 and T-06-SC is mapped in `06-SECURITY-REVIEW.md` | ✓ VERIFIED | Exactly 19 `\| T-06-` table rows. Mitigate rows name real tests; accept rows restate plan rationales. |
| 12 | `go vet` / tests green; both genres routes `status: ported` and parity corpus passing | ✓ VERIFIED | `go vet` and `go test -race -short` green in summercms.go and fonoteka plugin modules. `TestParityCorpus` ok. Manifest: both genres ids `status: ported`. |
**Score:** 11/12 truths verified (3 additional clauses deferred to later phases; not counted as failures)
### Deferred Items
| # | Item | Addressed In | Evidence |
|---|------|-------------|----------|
| 1 | public/onboarding groups reachable without auth | Phase 13 | Phase 13 ports onboarding/public/invitation routes. Empty group builders in `routes.go:24-29`. |
| 2 | unknown/malformed ids 404 on ownership-scoped resources | Phase 12 | Phase 12 Collections/Albums. Router primitive already in `surf/params.go` / `TestTypedIDRouteReturns404`. |
| 3 | fetchguard call sites (manual cover URL, Discogs cover) | Phase 12 / 14 | 06-04 D-13: helper and tests only. |
### Required Artifacts
gsd-sdk `verify.artifacts` reported missing files because PLAN paths keep the `summercms.go/` / `fonoteka.go/` prefix while CWD is already the framework repo. Files were verified at the stripped paths.
| Artifact | Expected | Status | Details |
| -------- | ----------- | ------ | ------- |
| `bouncer/registry.go` | Named Guard registry | ✓ VERIFIED | Register + Middleware; duplicate/unknown/neither fail with plugin+name |
| `bouncer/guard.go` | Guard, CredentialGuard, UnauthorizedWriter | ✓ VERIFIED | Interfaces as planned |
| `fonoteka.go/.../token_guard.go` | inv_token hash lookup, expiry, revocation, last-used | ✓ VERIFIED | SHA-256 lookup, `IsUsable()`, stamps last_used once. Wired from plugin Boot |
| `fonoteka.go/.../token_scope.go` | InvScope 401/403 PHP bodies | ✓ VERIFIED | Wired via `MiddlewareFactories()` `inv.scope` |
| `surf/limiter.go` | FixedWindowLimiter, Bucket, throttle factory | ✓ VERIFIED | Wired in `BuildRouter` `RegisterMiddlewareFactory("surf", "throttle", …)` |
| `surf/limiter_store.go` | Store + MemoryStore | ✓ VERIFIED | Hit / TooManyAttempts / AvailableIn; sweep tested |
| `surf/clientip.go` | ClientIP + TrustedProxies | ✓ VERIFIED | Used by bucket Key closures |
| `fonoteka.go/.../public_share_headers.go` | 429 rewrite + robots/cache headers | ✓ VERIFIED | Registered as `public.share-headers` |
| `surf/routetable.go` | RouteInfo + Routes() | ✓ VERIFIED | Used by isolation test and route:list |
| `pact/capabilities.go` | GroupRaw, HasHouseMiddleware | ✓ VERIFIED | Implemented by fonoteka Plugin |
| `wire/response.go` | WriteJSON, Time, TriBool, Slice | ✓ VERIFIED | ListGenres delegates WriteJSON |
| `surf/cors.go` | Path-scoped CORS | ✓ VERIFIED | Applied in `compile()` via `pathScopedCORS` |
| `fonoteka.go/docs/openapi.json` | Generated OpenAPI 3 | ✓ VERIFIED | 3.0.3; genres GET; openapi-typescript 0 |
| `fetchguard/fetch.go` | Fetch entry point | ✓ VERIFIED (intentionally unwired to app) | D-13: helper only |
| `fetchguard/policy.go` | Policy, Reason, Defaults | ✓ VERIFIED | Closed Reason set |
| `fetchguard/ip.go` | PHP CIDR table including CGNAT + metadata | ✓ VERIFIED | 100.64.0.0/10, 169.254.0.0/16 |
| `06-SECURITY-REVIEW.md` | Threat-to-test map | ✓ VERIFIED | 19 rows |
### Key Link Verification
| From | To | Via | Status | Details |
| ---- | --- | --- | ------ | ------- |
| `fonoteka/.../routes.go` | `genre_controller.go` | same `ListGenres(p.app)` value on both groups | WIRED | lines 10-16, two `g.Get("/genres", handler)` |
| `token_scope.go` | `bouncer/context.go` | `bouncer.User` / `Credential` | WIRED | lines 16-21 |
| `token_guard.go` | `models/api_token.go` | `token_hash = ?`, `IsUsable()` | WIRED | lines 46-47 |
| `fonoteka/plugin.go` | `surf/limiter.go` | `Buckets() map[string]surf.Bucket` | WIRED | lines 102-148; `_ surf.BucketProvider` |
| `surf/router.go` | `surf/limiter.go` | `RegisterMiddlewareFactory(..., "throttle", ...)` | WIRED | BuildRouter lines 419-424 |
| `routes.go` | `surf/router.go` | `GroupRaw(` | WIRED | line 33 |
| `plugin.go` | `pact/capabilities.go` | `HouseMiddlewares()` | WIRED | lines 73-76 |
| `routelist_command.go` | `routetable.go` | `.Routes()` | WIRED | line 24 |
| `genre_controller.go` | `wire/response.go` | `wire.WriteJSON` | WIRED | writeJSON wrapper line 149 |
| `fetch.go` | `ip.go` | DialContext Control → `isReservedOrPrivate` | WIRED | fetch.go:67-69, 145-163 |
| `routes_isolation_test.go` | `routetable.go` | `rt.Routes()` | WIRED | isolation test line 33 |
### Data-Flow Trace (Level 4)
| Artifact | Data Variable | Source | Produces Real Data | Status |
| -------- | ------------- | ------ | ------------------ | ------ |
| ListGenres | `GenreList.Data` | GORM query on `golem15_fonoteka_genres` + tenant album counts | Yes — `TestGenresSharedHandler` non-empty equal bodies | ✓ FLOWING |
| FixedWindowLimiter | Store counters | `MemoryStore.Hit` / `TooManyAttempts` | Yes — limiter tests increment and 429 | ✓ FLOWING |
| CORS | `http.cors.paths` | `fonoteka.go/config/http.yaml` via `LoadCORSConfig` | Yes — assembled-router CORS test | ✓ FLOWING |
| Body limits | `default_bytes` | config 134217728 | Yes — operator-confirmed test | ✓ FLOWING |
| fetchguard | response body | guarded HTTPS GET | Yes in tests (httptest TLS); no app caller | ✓ FLOWING (tests) / deferred prod |
### Behavioral Spot-Checks
| Behavior | Command | Result | Status |
| -------- | ------- | ------ | ------ |
| Framework phase packages vet+test | `go vet ./bouncer/... ./surf/... ./wire/... ./fetchguard/...` and `go test … -race -short` | exit 0 | ✓ PASS |
| Full summercms.go `-race -short` | `go test ./... -race -short` | all ok | ✓ PASS |
| Shared handler (Postgres) | `go test ./plugins/golem15/fonoteka -run TestGenresSharedHandler` | ok 4.7s | ✓ PASS |
| Isolation + CORS + buckets boot | `-run TestFullRouteTableAuthGroupMutualExclusivity\|TestCORSPathScopedOnAssembledRouter\|TestAllRouteGroupsBoot` | ok | ✓ PASS |
| Token guard (Postgres) | `go test ./plugins/golem15/fonoteka/classes/auth/` | ok | ✓ PASS |
| Body limits config | `go test . -run TestProductionBodyLimitsOperatorConfirmed` | ok | ✓ PASS |
| Parity corpus | `go test ./parity/... -run TestParityCorpus` | ok 5.3s | ✓ PASS |
| openapi-typescript | `npx openapi-typescript@7.13.0 docs/openapi.json -o /dev/null` | ✨ 7.13.0, 38.3ms, exit 0 | ✓ PASS |
| Unauthenticated token-route 429 | code trace of wrap + InvScope; no test exists | throttle not reached on 401 | ✗ FAIL |
### Probe Execution
No `scripts/*/tests/probe-*.sh` in this phase. `fonoteka.go/scripts/check-openapi.sh` is the OpenAPI pipeline (swag → swagger2openapi → openapi-typescript); the TypeScript step was run directly against the committed document (PASS above).
| Probe | Command | Result | Status |
| ----- | ------- | ------ | ------ |
| _(none declared)_ | — | — | SKIP |
### Requirements Coverage
Phase plans declare HTTP-03, HTTP-04, HTTP-05, HTTP-06, HTTP-07, HTTP-08, HTTP-09. REQUIREMENTS.md maps those seven IDs to Phase 6. HTTP-01/HTTP-02 are not in this phase's plan `requirements:` (HTTP-01 404 primitive exists from routing; HTTP-02 pipeline order is pre-existing). No orphaned Phase 6 IDs.
| Requirement | Source Plan | Description | Status | Evidence |
| ----------- | ---------- | ----------- | ------ | -------- |
| HTTP-03 | 06-01, 06-05 | Three mutually exclusive auth groups share handlers | ✓ SATISFIED (public routes deferred) | Shared ListGenres; isolation test; empty public groups D-15 |
| HTTP-04 | 06-02, 06-05 | Named buckets, stacking, 1:1 PHP port | ✗ BLOCKED | Buckets+stacking exist; live token route skips throttle on 401 |
| HTTP-05 | 06-01 | Guard registry → one current-user accessor | ✓ SATISFIED | Registry + jwt + inv_token; oauth not registered |
| HTTP-06 | 06-03 | Response conventions; no house envelope on OAuth | ✓ SATISFIED | wire helpers; GroupRaw + refusal tests |
| HTTP-07 | 06-04 | Guarded outbound fetch | ✓ SATISFIED | fetchguard tests; call sites later |
| HTTP-08 | 06-03 | swag → OpenAPI → openapi-typescript | ✓ SATISFIED | committed openapi.json; npx exit 0 |
| HTTP-09 | 06-03 | CORS and JSON body size match PHP | ✓ SATISFIED | path-scoped CORS tests; 134217728 confirmed. (MaxBytesError→413 is latent; no POST body handler this phase — see anti-patterns) |
### Anti-Patterns Found
| File | Line | Pattern | Severity | Impact |
| ---- | ---- | ------- | -------- | ------ |
| `fonoteka.go/.../routes.go` | 14-16 | throttle after InvScope on live route | 🛑 Blocker | Rate-limit bypass for unauthenticated personal-token traffic (SC2 / HTTP-04 / CR-01) |
| `surf/limiter.go` | 91-93 | fail-open if Key/store/resolve nil | ⚠️ Warning | All five production buckets set Key; latent if a plugin registers a nil Key (06-REVIEW WR-02) |
| `surf/bodylimit.go` | 10-18 | MaxBytesReader without mapping `*http.MaxBytesError` to 413 | ⚠️ Warning | Current handlers are GET; next POST will 500 unless the handler converts (WR-03). `upload_bytes` loaded and unread |
| `surf/clientip.go` | 75-78 | TrustedProxies skips bare IPs / malformed CIDRs | ⚠️ Warning | Config list is empty today; a copied `127.0.0.1` would silently no-op (WR-04) |
| `fetchguard/ip.go` | 5-45 | PHP-identical table; no NAT64/6to4 unwrap | ℹ️ Info | Matches PHP 1:1 (plan requirement). CONTEXT allowed stricter; not looser. No caller this phase (WR-05) |
| `fonoteka.go/.../token_guard.go` | 51 | last_used_ip from RemoteAddr, not ClientIP | ℹ️ Info | Parity/audit quality; not an SC (WR-01) |
| `fonoteka.go/.../routes.go` | 18-33 | Empty group builders | ℹ️ Info | Intentional D-15; not stubs — zero routes registered |
No `TBD` / `FIXME` / `XXX` debt markers in phase packages.
**Confirmation-bias pass:** (1) HTTP-04 is only partially met — buckets register, deny-path does not consume them. (2) `TestAllRouteGroupsBoot` proves Assemble, not 429. (3) No test covers unauthenticated 429 on `/api/v1/fonoteka/genres`.
### Human Verification Required
None remaining. Plan 06-03's body-size `<human-check>` was closed 2026-09-19 (128M / 128M / 128M → 134217728); config and `TestProductionBodyLimitsOperatorConfirmed` record that.
### Gaps Summary
The phase delivers the guard registry, dual-group shared genres handler, five named buckets, raw-group enforcement, wire helpers, path-scoped CORS, operator-confirmed body limits, OpenAPI pipeline, and the SSRF fetch helper. The phase **goal** is not fully achieved because HTTP-04's 1:1 rate-limit port fails on the one live personal-token route: `inv.scope` sits outside `throttle` in the onion, so missing/invalid bearers never hit `fonoteka-api-token`. That is a security control bypass (unlimited 401 spray plus a SHA-256 + SQL lookup per request), not a later-phase item.
Fix is a middleware-order change plus a deny-path 429 test. Public/onboarding handlers, ownership-scoped 404 resources, and fetchguard call sites are explicitly later-phase work and are listed under deferred, not gaps.
---
_Verified: 2026-09-19T19:45:00Z_
_Verifier: Claude (gsd-verifier)_