summercms/.planning/phases/11-jobs-realtime-and-search-infrastructure/11-03-SUMMARY.md

phase, plan, subsystem, tags, requires, provides, affects, actuals, plan_head_before, plan_head_after, tech-stack, key-files, key-decisions, patterns-established, requirements-completed, coverage, duration, completed, status

phase	plan	subsystem	tags	requires	provides	affects	actuals	plan_head_before	plan_head_after	tech-stack	key-files	key-decisions	patterns-established	requirements-completed	coverage	duration	completed	status
11-jobs-realtime-and-search-infrastructure	03	realtime	centrifugo jwt realtime river gorm-callbacks broadcast authorization	phase provides 11-jobs-realtime-and-search-infrastructure 11-01 conga (Manager.Register/Enqueue, conga.Job, StartWorker), lagoon.OnDatabase and lagoon.Transaction phase provides 07-user-plugin-and-authentication jwt.auth guard and the frontend principal (bouncer.User) phase provides 06-http-routing-auth-groups-and-rate-limiting surf Group/GroupRaw, named buckets (surf.BucketProvider), ClientIP	modules/lighthouse: transport-neutral realtime Service (From, driver registry, null/log/memory drivers, Route/Surface/Mount) lighthouse authorizer Registry, channel rules (ParseChannel, ChannelID with PHP (int) semantics, FormatChannels) and client-id ctx Model broadcasts: Broadcastable contract, Bind[T], GORM callbacks enqueueing a summer.broadcast River job in the write tx inside a savepoint, WithoutBroadcasting[T], Service.Emit modules/lighthouse/centrifugo: net/http API client, five-generator HS256 TokenIssuer, TokenHandler, ProxyHandler, driver registered as centrifugo fonoteka.go: GET /api/realtime/token (jwt.auth, throttle:ws-api), POST /api/realtime/subscribe (raw, throttle:ws-api), ws-api bucket, collection and wishlist authorizers, Album binding, config/realtime.yaml	11-06 parity goldens 11-07 unit tests 12 albums API (store/update/bulk must suppress + Emit) 13 notifications 15 cutover env mapping	tokens tasks commits 31479 3 4	f1077382f5	eab2b007f5	added patterns Drivers register from init (lighthouse.RegisterDriver, database/sql style) and are chosen by realtime.driver Drivers declare routes by surface; the app mounts them once with lighthouse.Mount and owns guard, group and bucket Model broadcasts are River jobs enqueued on the write's *sql.Tx from GORM callbacks, inside a savepoint Job args that carry byte-ordered JSON store it as a JSON string, because River's JSONB column reorders keys	created modified modules/lighthouse/lighthouse.go modules/lighthouse/drivers.go modules/lighthouse/route.go modules/lighthouse/users.go modules/lighthouse/channel.go modules/lighthouse/channel_test.go modules/lighthouse/registry.go modules/lighthouse/broadcast.go modules/lighthouse/suppress.go modules/lighthouse/job.go modules/lighthouse/README.md modules/lighthouse/centrifugo/config.go modules/lighthouse/centrifugo/client.go modules/lighthouse/centrifugo/token.go modules/lighthouse/centrifugo/handlers.go modules/lighthouse/centrifugo/driver.go ../fonoteka.go/config/realtime.yaml ../fonoteka.go/plugins/golem15/fonoteka/realtime.go ../fonoteka.go/plugins/golem15/fonoteka/realtime_smoke_test.go ../fonoteka.go/plugins/golem15/fonoteka/classes/ws/collection_authorizer.go ../fonoteka.go/plugins/golem15/fonoteka/classes/ws/wishlist_authorizer.go README.md .planning/PROJECT.md .planning/REQUIREMENTS.md ../fonoteka.go/README.md ../fonoteka.go/plugins/golem15/fonoteka/plugin.go ../fonoteka.go/plugins/golem15/fonoteka/routes.go ../fonoteka.go/plugins/golem15/fonoteka/plugin_boot_test.go ../fonoteka.go/plugins/golem15/fonoteka/routes_bucket_test.go ../fonoteka.go/plugins/golem15/fonoteka/go.mod ../fonoteka.go/plugins/golem15/fonoteka/go.sum	Broadcast job args store the payload as a JSON string (BroadcastArgs MarshalJSON/UnmarshalJSON): River keeps args in JSONB, which reorders object keys and would break payload key order No broadcast job is enqueued (and Emit is a no-op) when the driver publishes nothing: the null driver, or a driver whose Enabled() is false (Centrifugo without an API key) Broadcast callbacks are installed per *gorm.DB with Register-or-Replace, so a handle shared by several apps broadcasts through the most recently built service The subscribe proxy denies a missing channel and any user value that is not a JSON string or number (PHP would TypeError or cast true to 1); recorded as deliberate hardening Every test harness in fonoteka.go boots the centrifugo driver (bootConfig sets realtime.driver), so assembled route tables always carry the realtime routes	lighthouse.Mount(r, driver, Surfaces{UserAuth, ServerToServer, Public, Middleware}) is the one call an app makes for realtime routes Bind[T] keeps payload code out of the models package (models stay a leaf) Bulk writes: WithoutBroadcasting[T] + lagoon.Transaction + Service.Emit	RT-01 RT-02 RT-03	id description requirement verification human_judgment D1 GET /api/realtime/token behind jwt.auth then throttle:ws-api returns {"token"} whose HS256 claims are exactly sub/exp(now+3600)/info{name}; info.name null for a nameless user; 503 WebSocket not configured with an empty secret; jwt.auth 401 without a bearer; concurrent requests race-clean RT-01 kind ref status integration ../fonoteka.go/plugins/golem15/fonoteka/realtime_smoke_test.go#TestRealtimeTokenRoute pass false id description requirement verification human_judgment D2 Centrifugo client POSTs /publish and /broadcast with Authorization: apikey, the {channel,data:{event,payload,timestamp +00:00}} body, [] for an empty payload, and sends nothing without an API key RT-01 kind ref status integration ../fonoteka.go/plugins/golem15/fonoteka/realtime_smoke_test.go#TestRealtimeCentrifugoClientRequests pass false id description requirement verification human_judgment D3 Subscribe proxy: constant-time secret (missing, wrong, empty-configured all deny), empty/0 user, missing channel, presence:presence:, four segments, unknown or case-mismatched namespace, owner/editor allow with exact {"result":{"info":[]}}, wishlist id under collection denies, presence:collection denies, presence allow/override merge with authorizer override winning, user as string or number, generic HTTP 200 deny with reasons only in logs and no secrets logged, editor removal flips to deny, concurrent subscribes race-clean, raw route with throttle:ws-api RT-02 kind ref status integration ../fonoteka.go/plugins/golem15/fonoteka/realtime_smoke_test.go#TestRealtimeSubscribeProxy pass kind ref status integration ../fonoteka.go/plugins/golem15/fonoteka/realtime_smoke_test.go#TestRealtimeSubscribeProxyWithoutSecretDenies pass false id description requirement verification human_judgment D4 Channel rules: ChannelID matches PHP 8.5 (int) casts for 44 inputs (php -r table), ParseChannel presence/segment rules, FormatChannels lowercase + namespace, ClientID ctx, Registry validation, duplicate error and sorted Namespaces RT-02 kind ref status unit modules/lighthouse/channel_test.go#TestChannelIDMatchesPHP pass kind ref status unit modules/lighthouse/channel_test.go#TestParseChannel pass kind ref status unit modules/lighthouse/channel_test.go#TestRegistry pass false id description requirement verification human_judgment D5 Album create in lagoon.Transaction publishes once to acme:collection:<id> (namespace applied) with created.fonoteka.album and payload {id, collection_id, action, actor{user_id,name}, timestamp, album}; rollback publishes nothing; id-only delete publishes deleted payload without album; wishlist album publishes nothing RT-03 kind ref status integration ../fonoteka.go/plugins/golem15/fonoteka/realtime_smoke_test.go#TestAlbumBroadcastSmoke pass false id description requirement verification human_judgment D6 Three album creates under WithoutBroadcasting[models.Album] plus one Emit publish exactly one collection.bulk_updated {"reason":"bulk_create","count":3}; a write through a stale outer ctx still broadcasts RT-03 kind ref status integration ../fonoteka.go/plugins/golem15/fonoteka/realtime_smoke_test.go#TestAlbumBroadcastSmoke pass false id description requirement verification human_judgment D7 Centrifugo answering 500 leaves the album committed, logs realtime: broadcast failed without the API key, and the one-attempt job is not retried RT-03 kind ref status integration ../fonoteka.go/plugins/golem15/fonoteka/realtime_smoke_test.go#TestAlbumBroadcastSmoke pass false id description requirement verification human_judgment rationale D8 Suppressing Album never suppresses another bound type (the plan assigns the acme-type assertion to plan 11-07) RT-03 true Not asserted in this plan; plan 11-07 adds the acme-type unit test id description verification human_judgment rationale D9 Delivery order across separate broadcast jobs is not guaranteed, as with PHP's queued BroadcastEventJob true Backstop truth by design (plan marks it verification: backstop); no test asserts ordering id description verification human_judgment rationale D10 Live end to end: Centrifugo v6 with the production secret layout, fonoteka serve, the Nuxt app sees an album change true Manual smoke listed in the plan's verification, collected at /gsd-verify-work	12h 45m	2026-09-30	complete

Phase 11 Plan 03: lighthouse realtime and the Centrifugo driver Summary

The transport-neutral lighthouse package and its centrifugo driver. Płytarium's Nuxt app gets PHP-shaped connection tokens at GET /api/realtime/token. Every Centrifugo subscribe is re-authorized through a namespace registry with the ported collection and wishlist authorizers. Album writes publish to their collection channel only after commit, through a one-attempt River job enqueued in the write transaction.

Performance

• Duration: 12h 45m wall-clock (2026-09-29T21:51:18Z to 2026-09-30T10:36:24Z). The clock includes long idle gaps between tool calls; the active work was much shorter.
• Started: 2026-09-29T21:51:18Z
• Completed: 2026-09-30T10:36:24Z
• Tasks: 3
• Files modified: 31 (19 in summercms.go, 12 in fonoteka.go)

Accomplishments

• lighthouse core (D-11, D-13).
  • lighthouse.From picks the driver by realtime.driver: null (the default), log, memory, or a registered driver. An unknown name fails boot and lists the registered drivers.
  • RegisterDriver is an init-time registry. A duplicate name panics.
  • Mount puts user and public routes in Group and server-to-server routes in GroupRaw, each with the surface middleware followed by the shared middleware. It refuses a UserAuth route when no guard is given (T-11-19).
• Centrifugo driver (D-12).
  • The net/http client sends Authorization: apikey, times out after 5 s and treats any 2xx as success. Without an API key it sends no request and returns ErrNotConfigured.
  • TokenIssuer has the five generators. They are HS256 with ordered claims, ForIdentifier encodes an empty info as [], and all refuse an empty secret.
  • TokenHandler returns the PHP 401 and 503 bodies.
  • ProxyHandler checks the secret in constant time and always answers HTTP 200. A deny gets a generic body, with the reasons only in logs. An allow gets info ([] when empty) and, on presence channels, allow/override in PHP array_merge order. The body is capped at 64 KiB.
• Channel rules (RT-02).
  • ParseChannel ports parseChannel exactly.
  • ChannelID and PHPInt reproduce the PHP 8.5 (int) cast. A table test pins 44 inputs printed by php -r, including saturation and the rule that INF becomes 0.
  • FormatChannels lowercases ASCII and applies the namespace, as BroadcastEventJob does.
• Broadcasts (D-06..D-09, RT-03).
  • Models broadcast through the Broadcastable contract or Bind[T].
  • The GORM callbacks are installed through lagoon.OnDatabase and run in the production boot order. Each enqueues a summer.broadcast job on the write's *sql.Tx inside a savepoint. A failure is logged and never aborts the write.
  • Deletes are snapshotted from a fresh read before the row goes.
  • WithoutBroadcasting[T] suppresses one type for one ctx. Service.Emit enqueues one summary event.
  • The job namespaces the channels, then publishes to one channel or broadcasts to several. A failure is a Warn log, and the job is never retried.
• fonoteka.go wiring (D-14, D-16).
  • config/realtime.yaml holds the PHP defaults and the README maps the env names.
  • The ws-api bucket allows 120/min, keyed by user id, else by client IP.
  • A single lighthouse.Mount call puts jwt.auth before throttle:ws-api (user decision 5).
  • The collection and wishlist authorizers are ported.
  • The Album binding applies the PHP channel and payload overrides.
  • PROJECT.md records that websockets is no longer an app plugin. REQUIREMENTS.md notes that RT-01 uses a hand-rolled client.

Task Commits

summercms.go:

1. Task 1: token route through the neutral package - cada7a4 (feat)
2. Task 2: subscribe proxy and registry - 79fd705 (feat)
3. Task 3: transactional album broadcasts - 211c413 (feat); planning docs eab2b00 (docs)

fonoteka.go:

1. Task 1 - c59fd76 (feat: realtime wiring, ws-api bucket, Mount, config, token smoke test)
2. Task 2 - cb78a76 (feat: collection and wishlist authorizers, proxy smoke test)
3. Task 3 - a223a72 (feat: Album binding, broadcast smoke test)

Files Created/Modified

• modules/lighthouse/lighthouse.go: Service, From, config reading, job registration, callback installation, DurationSetting
• modules/lighthouse/drivers.go: Publisher, Driver, DriverFactory, RegisterDriver, the null/log drivers, MemoryDriver, Publication
• modules/lighthouse/route.go: Route, Surface, Surfaces, Mount
• modules/lighthouse/users.go: User, UserLookup, Actor, SystemActor, Service.Actor
• modules/lighthouse/channel.go, registry.go: the channel rules, PHPInt, client-id ctx, Registry, Result, Allowed, Denied, Authorizer
• modules/lighthouse/broadcast.go, suppress.go, job.go: the broadcast contract, Bind, the callbacks, WithoutBroadcasting, Emit, BroadcastArgs and the worker
• modules/lighthouse/channel_test.go: the PHP (int) table, parse/format, registry
• modules/lighthouse/centrifugo/*.go: Config/LoadConfig, Client, TokenIssuer, TokenHandler, ProxyHandler, Driver
• modules/lighthouse/README.md and the root README.md row
• fonoteka.go: realtime.go, classes/ws/*, plugin.go (field, Boot call, ws-api), routes.go (Mount), config/realtime.yaml, README.md (Configuration), realtime_smoke_test.go, test harness tweaks, go.mod/go.sum tidy

Decisions Made

See key-decisions in the frontmatter.

TDD Gate Compliance

Tasks 2 and 3 are tdd="true". For each, the tests were written first against stubs with the final signatures. The RED runs were recorded with go test -json converted to TAP, and gsd-tools check tdd-red-evidence returned RED_EVIDENCE_OK (target_test_failed) for:

• Task 2: TestRealtimeSubscribeProxy, where the stub proxy answered 418 and the allow body was expected; and TestChannelIDMatchesPHP with the other lighthouse unit tests, where the stub ChannelID returned -1.
• Task 3: TestAlbumBroadcastSmoke, where the stubs installed no callbacks and Emit enqueued nothing. The rollback and wishlist subtests passed at RED, as expected, because they assert that nothing is published.

Gate violation, flagged: there are no separate test(11-03) RED commits. The project CLAUDE.md requires go vet and go test ./... to be green at every commit, so the tests and implementation landed together in the feat commits. Plans 11-01 and 11-02 made the same call.

Deviations from Plan

Auto-fixed Issues

1. [Rule 1 - Bug] JSONB reordered broadcast payload keys

• Found during: Task 3 (TestAlbumBroadcastSmoke GREEN run)
• Issue: River stores job args in a JSONB column, and JSONB sorts object keys. So {"reason":"bulk_create","count":3} arrived at Centrifugo as {"count":3,"reason":"bulk_create"}, which breaks the PHP byte order that the 11-06 goldens will compare.
• Fix: BroadcastArgs keeps Payload json.RawMessage in Go, but its MarshalJSON/UnmarshalJSON carry the payload as a JSON string, whose content JSONB leaves alone.
• Files modified: modules/lighthouse/job.go
• Verification: the bulk subtest asserts the exact bytes {"reason":"bulk_create","count":3}.
• Committed in: 211c413

2. [Rule 2 - Correctness/operability] No broadcast jobs when the driver publishes nothing

• Found during: Task 3
• Issue: Every album write would enqueue a River job in two cases where the job would publish nothing: an install without a Centrifugo API key, and every fonoteka test harness (the centrifugo driver without a key). PHP queues the job and then logs a failed publish.
• Fix: The callbacks and Emit skip the enqueue for the null driver, and for a driver whose optional Enabled() reports false. centrifugo.Driver.Enabled reports whether an API key is set. Nothing observable is published in either case. The only difference is that no "broadcast failed" warning is logged.
• Files modified: modules/lighthouse/broadcast.go, suppress.go, centrifugo/driver.go
• Committed in: 211c413

3. [Rule 3 - Test harness] fonoteka harness boots the centrifugo driver

• Found during: Task 1
• Issue: bootConfig did not set realtime.driver, so every assembled route table used the null driver and carried no realtime routes. TestAllRouteGroupsBoot and TestFullRouteTableAuthGroupMutualExclusivity would then prove nothing about the Mount call.
• Fix: bootConfig sets realtime.driver: centrifugo, following D-13 ("Płytarium and its tests always run the Centrifugo driver"). TestAllRouteGroupsBoot now also expects the ws-api bucket. Both test files are outside the plan's files list.
• Files modified: ../fonoteka.go/plugins/golem15/fonoteka/plugin_boot_test.go, routes_bucket_test.go
• Committed in: c59fd76

4. [Hardening, documented] Proxy input handling beyond PHP

• A missing or empty channel is denied. Centrifugo always sends one; PHP would return a 500 TypeError.
• A user that is neither a JSON string nor a number (for example true) counts as empty and is denied. PHP would cast true to 1.
• A malformed JSON body is denied with the generic body.
• Registering a namespace twice is a boot error. PHP lets the last registration win. This was already specified in the plan.
• Committed in: 79fd705

5. [Additions] Extra exported surface, all documented in the README and checked with go doc

• Service.Logger, Namespace, Queue and Timeout
• DurationSetting
• DefaultDriver, DefaultQueue, DefaultTimeout and DefaultTTL
• PHPInt and WithClientID
• NewRegistry
• the Callback* name constants
• centrifugo: LoadConfig, NewDriver, NewTokenIssuer, DriverName, Driver.Enabled, Driver.Client, Driver.Issuer, Driver.Config, Config.TrustedProxies (the client IP in secret-failure logs) and the Default* constants

6. [Additions] Extra tests

• TestRealtimeCentrifugoClientRequests: the httptest check of the client bytes, which the plan placed in the Task 1 smoke test.
• TestRealtimeSubscribeProxyWithoutSecretDenies.
• lighthouse unit tests: TestParseChannel, TestFormatChannels, TestClientID and TestRegistry.

---

Total deviations: 3 auto-fixed (1 bug, 1 correctness/operability, 1 test harness), plus 3 documented notes. Impact on plan: Fix 1 is required for payload parity with PHP. Fix 2 keeps unconfigured installs and tests off River. No scope creep.

Issues Encountered

• The shell aliases rm to interactive mode, so one command stalled on a prompt. It was rerun with /bin/rm -f. Plan 11-01 hit the same thing.
• The token signing error path answers 500 {"error":"Internal server error"}. It is unreachable in practice, because the handler checks Configured() first.

Flagged assumptions (carried from the plan)

• Batch updates through Model(&T{}).Where(...) have a zero primary key and are not broadcast (Pitfall 8). A soft delete counts as a delete, and a restore is not broadcast.
• The automatic Album created/updated payload is built inside the write transaction. SaveAlbum syncs the artist pivot after tx.Save, so the payload can carry stale artists. Phase 12's store/update/bulk controllers must use WithoutBroadcasting[models.Album] plus Emit, as PHP does. classes.SerializeAlbum is still the minimal Phase 5 shape until Phase 12.
• Several apps that share one *gorm.DB broadcast through the service built last, because the callbacks are replaced. Production has one app per handle.

User Setup Required

None for development. For a deployment, set the SUMMER_REALTIME__CENTRIFUGO__* variables listed in the fonoteka.go README (the PHP values carry over unchanged). Point Centrifugo's subscribe proxy at POST /api/realtime/subscribe with the same X-Centrifugo-Secret.

Next Phase Readiness

• Plan 11-06 (tide goldens) can diff the deleted and collection.bulk_updated bytes. It can drive the Go side with the memory driver or the fake-Centrifugo pattern from realtime_smoke_test.go.
• Plan 11-07 should add:
  • the acme-type suppression test (suppressing one type never suppresses another)
  • unit tests for Mount validation errors, the log driver, Presence/Unsubscribe, all five token generators with an injected clock, the override merge with extra keys, the method-based Broadcastable path (alias, TTL, filter, payloader), and Emit without a transaction
• Phase 12 must route album store/update/bulk through WithoutBroadcasting plus Emit (see above), and must migrate gdb.Transaction call sites to lagoon.Transaction.

---

Phase: 11-jobs-realtime-and-search-infrastructure Completed: 2026-09-30

Self-Check: PASSED

• All 21 key created files exist on disk.
• summercms.go commits cada7a4, 79fd705, 211c413 and eab2b00 (plus the SUMMARY commit 499177a) exist, and so do fonoteka.go commits c59fd76, cb78a76 and a223a72. The fonoteka.go working tree is clean.
• After Task 3, go vet ./... && go test ./... passed in summercms.go. The fonoteka.go vet and test command for all three modules passed, and so did the -race -v run of TestAlbumBroadcastSmoke, TestRealtimeSubscribeProxy and TestRealtimeTokenRoute (no SKIP, no DATA RACE).
• Every acceptance-criteria grep and go doc check of all three tasks passed.