124 lines
5.5 KiB
Markdown
124 lines
5.5 KiB
Markdown
# Phase 11: Jobs, realtime and search infrastructure - Discussion Log
|
|
|
|
> **Audit trail only.** Do not use as input to planning, research, or execution agents.
|
|
> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.
|
|
|
|
**Date:** 2026-09-28
|
|
**Phase:** 11-jobs-realtime-and-search-infrastructure
|
|
**Areas discussed:** Job manager & job records, Broadcast delivery path, Realtime package split, Worker & scheduler model (incl. search sync)
|
|
|
|
---
|
|
|
|
## Job manager & job records
|
|
|
|
The first question set was interrupted; the user ran `/gsd-explore` on whether to port Apparatus as a plugin or fold it into the framework. Outcome: `.planning/notes/apparatus-dissolved-into-framework.md`.
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| (a) keep `golem15_apparatus_jobs` | Straight copy at cutover; framework carries a plugin prefix | |
|
|
| (b) `summer_jobs` | Framework-owned name, same columns and int ids, rows copied at cutover | ✓ |
|
|
| (c) configurable table name | Per-app name | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Mirror PHP exactly | Same integers; skip = COMPLETE + {skipped:true}; ERROR only on River discard | ✓ |
|
|
| Add SKIPPED status | New integer 5 | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Both | is_canceled + STOPPED and River JobCancel; long jobs still poll | ✓ |
|
|
| Poll is_canceled only | Literal PHP port | |
|
|
| River cancel + ctx only | Go-idiomatic | |
|
|
|
|
**User's choice:** Dissolve Apparatus into the framework; `summer_jobs`; mirror PHP outcomes; cancel via both.
|
|
|
|
---
|
|
|
|
## Broadcast delivery path
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| River job in the write tx | Published only after commit; `broadcasts` queue | ✓ |
|
|
| Direct publish after commit | Goroutine after-commit callback | |
|
|
| Synchronous publish | Inline in request | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| One attempt, best-effort | tries=1, 5 s timeout, warn on failure | ✓ |
|
|
| Small retry | River backoff, e.g. 3 attempts | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Per model type, ctx-scoped | `WithoutBroadcasting[T](ctx, fn)` | ✓ |
|
|
| All broadcasts in ctx | One switch | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Capture from PHP Centrifugo | tide subscribes during recorded flows; goldens | ✓ |
|
|
| Goldens derived from PHP code | Hand-written expected payloads | |
|
|
|
|
---
|
|
|
|
## Realtime package split
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Framework package + thin plugin | Generic realtime package in summercms.go | |
|
|
| All in fonoteka.go plugin | Wholesale app port | |
|
|
| Framework-bundled plugin | First-party plugin in summercms.go | |
|
|
|
|
**User's choice:** Free text: concerned about hard dependency on Centrifugo; wants transports pluggable so a project can replace it. Claude proposed a transport-neutral realtime package with drivers (centrifugo, memory, log, null), a Centrifugo sub-package, and Web Push behind its own interface. User: "yep that's perfect."
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Hand-rolled net/http | Stdlib-first, exact request bytes | ✓ |
|
|
| gocent/v3 | Official client | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Health check only | Defer push/VAPID | |
|
|
| Port all of it | Web Push + both VAPID commands + health check | ✓ |
|
|
| None | | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Only what's called | Connection + subscription tokens | |
|
|
| Full parity with PHP class | All generator variants | ✓ |
|
|
|
|
Route mounting: Claude first suggested app-mounted handlers (wristband precedent). User asked: "won't that be a problem if we implement multiple drivers in future?" Claude refined it: drivers declare `Routes()` by surface (UserAuth / ServerToServer / Public) and the app maps surfaces to its groups and buckets through `realtime.Mount`. **User's choice:** Lock it.
|
|
|
|
---
|
|
|
|
## Worker & scheduler model
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| In serve by default + queue:work | `queue.work_in_serve` toggle | ✓ |
|
|
| Only queue:work | Separate process always | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| River periodic jobs | HasSchedule → periodic jobs, leader-elected; `schedule:run` foreground + `--once` | ✓ |
|
|
| Laravel-style schedule:run | System cron every minute, own locking | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| Driver interface, Typesense driver | Searchable interface + engine driver; hand-rolled Typesense | ✓ |
|
|
| Typesense client only | Direct port, no abstraction | |
|
|
|
|
| Option | Description | Selected |
|
|
|--------|-------------|----------|
|
|
| After commit, inline, non-fatal | Like Scout queue=false | ✓ |
|
|
| River job in the write tx | Retryable, lags the write | |
|
|
|
|
---
|
|
|
|
## Claude's Discretion
|
|
|
|
- Package names and layout; ctx-to-GORM-hook plumbing for suppression; actor capture; River client config and timed-test harness; whether the Phase 8 expiry sweep moves to a periodic job; tide Centrifugo capture mechanics.
|
|
|
|
## Deferred Ideas
|
|
|
|
- The admin extension point ("Phase 10.1") is not on the roadmap. It stays deferred, and the references that call it a phase need rewording.
|
|
- More realtime/search drivers; a Jobs admin screen; backend admin API tokens; guarded HTTP client and redacting slog handler (pending todos).
|