diff --git a/.planning/phases/11-jobs-realtime-and-search-infrastructure/11-01-SUMMARY.md b/.planning/phases/11-jobs-realtime-and-search-infrastructure/11-01-SUMMARY.md new file mode 100644 index 0000000..bc2bcf1 --- /dev/null +++ b/.planning/phases/11-jobs-realtime-and-search-infrastructure/11-01-SUMMARY.md @@ -0,0 +1,306 @@ +--- +phase: 11-jobs-realtime-and-search-infrastructure +plan: 01 +subsystem: infra +tags: [river, postgres, jobs, gorm, listen-notify, cli, gormigrate] + +requires: + - phase: 03-first-vertical-slice + provides: lagoon shared *sql.DB pool and the reserved River listener seam + - phase: 08-oauth2-1-authorization-server + provides: bonfire repeatable flags (Input.Flags) used by queue:work --queue +provides: + - River v0.47.0 on the shared *sql.DB with riverdatabasesql.NewWithPgxListener (one client type, 1-connection LISTEN pool) + - lagoon.QueueMigrations under summercms.conga (River schema v7 + summer_jobs with river_job_id) + - conga package: Manager (Dispatch, Enqueue, full PHP JobManager surface), conga.Job typed adapter, StartWorker/StartServeWorker, queue:work, queue:clear + - serve runs the job worker in-process unless queue.work_in_serve is false + - lagoon.OnDatabase (Boot-order seam) and lagoon.Transaction / AfterCommit / lagoon:after_commit GORM callback + - fonoteka GORM hooks now register under serve (Boot gap closed) +affects: [11-02 scheduler, 11-03 broadcasts, 11-05 search sync, 11-07 unit tests, 13 CSV import, 14 domain jobs, 15 cutover] + +actuals: + tokens: 30550 + tasks: 3 + commits: 4 +plan_head_before: 718a35cabaf5bf61f2a497a2a011b3ab9b6435fc +plan_head_after: 6c1f94e +# fonoteka.go (separate repository) received 4 more commits: 3d4895d, 0b9a2a5, e0b1b0d, 36fc473 + +tech-stack: + added: [github.com/riverqueue/river v0.47.0, riverdriver/riverdatabasesql v0.47.0, rivertype v0.47.0 (transitive riverdriver, riverpgxv5, rivershared, lib/pq, tidwall gjson/sjson; testify bumped to v1.12.1)] + patterns: + - "Jobs are declared River-free with conga.Job[T](fn, opts...) and registered by the worker from pact.HasJobs" + - "Transactional dispatch: summer_jobs row + River InsertTx on the caller's *sql.Tx (tx.Statement.ConnPool)" + - "Row writes are raw UpdateColumns on Table(lagoon.JobsTable) so updated_at only moves on dispatch and StartJob" + - "Anything that needs *gorm.DB at Boot goes through lagoon.OnDatabase" + - "Side effects after a write use lagoon.Transaction + lagoon.AfterCommit" + +key-files: + created: + - modules/lagoon/queue_migrations.go + - modules/lagoon/ondatabase.go + - modules/lagoon/transaction.go + - modules/conga/conga.go + - modules/conga/record.go + - modules/conga/job.go + - modules/conga/client.go + - modules/conga/worker.go + - modules/conga/commands.go + - modules/conga/README.md + - ../fonoteka.go/config/queue.yaml + modified: + - modules/lagoon/migrations.go + - modules/lagoon/connection.go + - modules/surf/serve.go + - internal/build/build.go + - internal/build/stubs/artifacts.tmpl + - cmd/summer/main.go + - cmd/summer/runtime.go + - ../fonoteka.go/plugins/golem15/fonoteka/plugin.go + - ../fonoteka.go/parity/schema_diff_test.go + - ../fonoteka.go/parity/migrate_test.go + +key-decisions: + - "Job registration closes only while a worker client runs; the insert-only client has no Workers bundle, so inserts do not validate kinds" + - "A worker's lifetime is controlled by Worker.Stop, not the ctx passed to StartWorker (Start runs on context.WithoutCancel), so SIGTERM stops serve and queue:work gracefully instead of hard-cancelling jobs" + - "An app with no registered jobs gets an internal idle worker (kind summercms_conga_idle) because River refuses to start a client without workers" + - "lagoon.OnDatabase treats a published *gorm.DB as ready and derives the pool from it; existing harnesses publish only *gorm.DB" + - "River v7 leaves exactly river_job, river_leader, river_migration, river_notification and river_queue (read from pg_tables after migrating)" + +patterns-established: + - "conga.Job: typed job functions wrapped for River without importing River in plugins" + - "lagoon.OnDatabase for Boot-time GORM callback registration" + - "lagoon.Transaction/AfterCommit for commit-safe side effects" + +requirements-completed: [JOBS-01, CLI-06] + +coverage: + - id: D1 + description: "River LISTEN pickup under 1s with a 30s poll interval from a separate insert-only client; poll-only control misses in 2s" + requirement: JOBS-01 + verification: + - kind: integration + ref: "modules/conga/listen_test.go#TestListenPickupLatency" + status: pass + human_judgment: false + - id: D2 + description: "Dispatch writes summer_jobs (status 1, principal, metadata \"\") and the River job in one transaction; rollback leaves neither" + requirement: JOBS-01 + verification: + - kind: integration + ref: "modules/conga/listen_test.go#TestDispatchTransactional" + status: pass + human_judgment: false + - id: D3 + description: "PHP JobManager operations (StartJob, UpdateJobState, UpdateMetadata, FailJob, StopJob, CancelJob, CheckIfCanceled, GetMetadata, skip via CompleteJob) with PHP semantics" + requirement: JOBS-01 + verification: + - kind: integration + ref: "modules/conga/listen_test.go#TestJobManagerOperations" + status: pass + human_judgment: false + - id: D4 + description: "Retries keep the row IN_PROGRESS; the final failed attempt or a panic records ERROR with metadata error; cancel stops queued and running jobs with the row at STOPPED" + requirement: JOBS-01 + verification: + - kind: integration + ref: "modules/conga/listen_test.go#TestAttemptOutcomes" + status: pass + - kind: integration + ref: "modules/conga/listen_test.go#TestCancelJob" + status: pass + human_judgment: false + - id: D5 + description: "queue:work with --queue filters and unknown-queue error, queue:clear removes only pending jobs, serve worker honours queue.work_in_serve" + requirement: CLI-06 + verification: + - kind: integration + ref: "modules/conga/listen_test.go#TestQueueWorkCommand" + status: pass + - kind: integration + ref: "modules/conga/listen_test.go#TestQueueClear" + status: pass + - kind: integration + ref: "modules/conga/listen_test.go#TestStartServeWorker" + status: pass + - kind: unit + ref: "internal/build/build_test.go#TestGenerateMainRegistersCongaRuntimeCommands" + status: pass + human_judgment: false + - id: D6 + description: "lagoon.OnDatabase and lagoon.Transaction/AfterCommit seams; fonoteka hooks register when the database is published after Boot" + verification: + - kind: integration + ref: "modules/lagoon/ondatabase_test.go#TestOnDatabaseAfterActivate" + status: pass + - kind: integration + ref: "modules/lagoon/transaction_test.go#TestTransactionAfterCommit" + status: pass + - kind: integration + ref: "../fonoteka.go/plugins/golem15/fonoteka/plugin_boot_test.go#TestHooksRegisterWhenDatabasePublishedAfterBoot" + status: pass + human_judgment: false + - id: D7 + description: "fonoteka.go schema diff and migrate tests accept the framework queue tables" + verification: + - kind: integration + ref: "../fonoteka.go/parity/schema_diff_test.go#TestSchemaMatchesPHPSnapshot" + status: pass + - kind: integration + ref: "../fonoteka.go/parity/migrate_test.go#TestMigrateSeedsCanonicalGenres" + status: pass + human_judgment: false + +duration: 37min +completed: 2026-09-29 +status: complete +--- + +# Phase 11 Plan 01: conga job framework on River Summary + +**River v0.47.0 on the shared pool via `riverdatabasesql.NewWithPgxListener` (3-5 ms LISTEN pickup against a 30 s poll), transactional `conga.Manager.Dispatch` into a PHP-shaped `summer_jobs` record, workers in `serve` and `queue:work`, `queue:clear`, plus the `lagoon.OnDatabase` and after-commit seams.** + +## Performance + +- **Duration:** 37 min +- **Started:** 2026-09-29T12:48:56Z +- **Completed:** 2026-09-29T13:26:12Z +- **Tasks:** 3 +- **Files modified:** 50 (38 in summercms.go, 12 in fonoteka.go) + +## Accomplishments + +- `lagoon.Migrate` now runs the `summercms.conga` set: River's schema pinned at version 7 and `summer_jobs` with the exact apparatus columns plus the internal `river_job_id BIGINT`. +- `conga` package: `Dispatch` writes the row with status 1 (IN_PROGRESS, as PHP does) and enqueues on the same `*sql.Tx`. The full JobManager surface keeps PHP semantics: raw column updates, CancelJob vs StopJob split, and the final-attempt ERROR rule with panic recovery. +- One River client type. Workers use `NewWithPgxListener` with a `MaxConns 1 / MinConns 0` listener pool; the timed test proves LISTEN pickup (about 5 ms) and a poll-only control proves the test is not measuring poll latency. +- `serve` starts the worker unless `queue.work_in_serve: false`. `queue:work --queue ...` and `queue:clear [queue]` exist in the app binary and as `summer` delegates. `make:job` scaffolds a `conga.Job`. +- `lagoon.OnDatabase` closes the Boot-order gap: fonoteka's slug, artist-resolver and credential-encryption callbacks now register under `serve`. `lagoon.Transaction` / `lagoon.AfterCommit` and the `lagoon:after_commit` callback are ready for plan 11-05. + +## Task Commits + +summercms.go: +1. **Task 1: transactional dispatch picked up through LISTEN** - `0bc5c77` (feat); toolchain-line restore `05ba88a` (fix) +2. **Task 2: outcomes, cancellation, serve/queue:work/queue:clear** - `b319e7c` (feat) +3. **Task 3: OnDatabase and after-commit transactions** - `6c1f94e` (feat) + +fonoteka.go: +1. **Task 1** - `3d4895d` (chore: tidy + allow-lists); toolchain-line restore `0b9a2a5` (fix) +2. **Task 2** - `e0b1b0d` (feat: regenerated main.go, config/queue.yaml) +3. **Task 3** - `36fc473` (fix: Boot registers hooks through lagoon.OnDatabase) + +## Files Created/Modified + +- `modules/lagoon/queue_migrations.go`: `QueueMigrations`, `QueueHistoryID`, `JobsTable`, `RiverSchemaVersion` +- `modules/lagoon/ondatabase.go`: `OnDatabase` plus the per-app hook queue drained by `Publish` +- `modules/lagoon/transaction.go`: `Transaction`, `AfterCommit`, `AfterCommitCallback` (`lagoon:after_commit`) +- `modules/lagoon/connection.go`: `Publish` drains the hooks; `gormFromSQL` installs the after-commit callback +- `modules/conga/*.go`: Manager, Record/Status, Job adapter, client config, worker, commands +- `modules/surf/serve.go`: in-process worker start/stop +- `internal/build/build.go`, `internal/build/stubs/artifacts.tmpl`, `internal/build/artifact.go`: generated main registers `conga.RuntimeCommands`; the `make:job` stub is a `conga.Job` +- `cmd/summer/main.go`, `cmd/summer/runtime.go`: `queue:work` (forwards repeatable `--queue`) and `queue:clear` delegates +- READMEs: new `modules/conga/README.md`, root modules table row, `modules/lagoon/README.md`, `modules/surf/README.md` +- fonoteka.go: `config/queue.yaml`, regenerated `main.go`, `plugin.go` Boot fix, parity allow-lists, go.mod/go.sum tidy + +## Decisions Made + +See `key-decisions` in the frontmatter. In short: + +- Registration closes only while a worker runs. +- Worker lifetime is tied to `Stop`, not the start ctx. +- An app without jobs gets an idle worker. +- `OnDatabase` treats a published `*gorm.DB` as ready. +- The River v7 table set was read from `pg_tables`. + +## TDD Gate Compliance + +Tasks 2 and 3 are `tdd="true"`. For each, the RED run was observed and recorded before the implementation. The tests compiled against stubs that returned "not implemented", so they failed on their assertions. `gsd-tools check tdd-red-evidence` returned `RED_EVIDENCE_OK` for: + +- `TestJobManagerOperations`, `TestAttemptOutcomes`, `TestCancelJob`, `TestQueueClear`, `TestQueueWorkCommand` and `TestStartServeWorker` (Task 2) +- `TestOnDatabaseAfterActivate` and `TestTransactionAfterCommit` (Task 3) + +`TestHooksRegisterWhenDatabasePublishedAfterBoot` failed against the HEAD `plugin.go` and passed with the fix. + +**Gate violation, flagged:** there are no separate `test(11-01)` RED commits. Tests and implementation landed together in `feat`/`fix` commits because the project CLAUDE.md requires `go vet` and `go test ./...` to be green at every commit. The CLAUDE.md rule was treated as the higher-priority constraint. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] River cannot start a client with no workers** +- **Found during:** Task 2 (TestStartServeWorker, TestQueueWorkCommand) +- **Issue:** `river.NewClient` fails with "at least one Worker must be added" for an app without jobs, so `serve` and `queue:work` would fail. The plan's flagged assumption (c) says they must start and idle. +- **Fix:** When no job is registered, `StartWorker` adds an internal `summercms_conga_idle` worker that nothing inserts. +- **Files modified:** modules/conga/worker.go +- **Committed in:** b319e7c + +**2. [Rule 1 - Bug] `go mod tidy` dropped the `toolchain go1.27.0` line** +- **Found during:** Task 2 (reading `ensureToolchain` in internal/build) +- **Issue:** The scaffolder keeps `toolchain go1.27.0` in every module. The Task 1 tidy removed it from `examples/hello` (app and three plugins) and from `fonoteka.go/go.mod`. The same tidy also picked up pre-existing indirect-requirement drift in the example plugin modules. +- **Fix:** Restored the toolchain line in all five files and kept the tidy requirement updates. +- **Files modified:** examples/hello/go.mod, examples/hello/plugins/{base,greeter,optional}/go.mod, ../fonoteka.go/go.mod +- **Committed in:** 05ba88a (summercms.go), 0b9a2a5 (fonoteka.go) + +**3. [Rule 1 - Bug] OnDatabase must accept a published `*gorm.DB` alone** +- **Found during:** Task 3 +- **Issue:** Existing fonoteka test harnesses publish only `*gorm.DB` before `party.Activate`. With the plan's literal "both handles published" check, `RegisterHooks` would have been queued forever in those tests, and the slug and encryption hooks would have stopped firing. +- **Fix:** A published `*gorm.DB` counts as ready, and the pool comes from `gdb.DB()` when no `*sql.DB` is published. This is covered by the `gorm_only_published_runs_now` subtest. +- **Files modified:** modules/lagoon/ondatabase.go +- **Committed in:** 6c1f94e + +**4. [Rule 1 - Bug] SIGTERM would hard-cancel running jobs** +- **Found during:** Task 1/2 (River `Start` docs: cancelling the start ctx is a hard stop) +- **Issue:** `serve` and `queue:work` pass a signal ctx, so a SIGTERM would have cancelled every running job's ctx before `Stop` ran. +- **Fix:** `StartWorker` starts River with `context.WithoutCancel(ctx)`. `Worker.Stop` does the graceful stop and falls back to `StopAndCancel` when its own ctx expires. +- **Files modified:** modules/conga/worker.go +- **Committed in:** 0bc5c77 + +**5. [Plan wording] Registration closes while a worker runs, not "once a client has been built"** +- The insert-only client is built lazily on the first Dispatch and carries no Workers bundle, so it does not depend on the job registry. Closing registration only while a worker exists lets `StartWorker` register plugin jobs even after an earlier Dispatch, and lets a worker restart after `Stop`. + +**6. [Additions] Small extra exported surface** +- `conga.Worker.Queues()` is used by the `queue:work` and `serve` output. +- `lagoon.AfterCommitCallback` is the name constant of the `lagoon:after_commit` callback. +- The unexported `WorkerOptions.retryPolicy` test knob keeps retry tests fast. +- All of these are documented in the module READMEs. + +**7. [Process] Commits on `master`** +- `gsd-tools git.base-branch --is-protected master` reports true. The project config sets `git.branching_strategy: "none"`, every earlier plan committed directly to master, and the orchestrator ran this as a sequential executor on the main working tree. So the plan was committed on master in both repositories, with no branch created. + +--- + +**Total deviations:** 4 auto-fixed (1 blocking, 3 bugs), plus 3 documented plan-interpretation or process notes. +**Impact on plan:** All fixes were needed for correct boot, shutdown and green builds. No scope creep. + +## Issues Encountered + +- The shell aliases `rm` and `cp` to interactive mode, which stalled two background commands. They were rerun with `/bin/rm -f` and `/bin/cp -f`. +- `GOWORK=off` builds of `examples/hello` fail on the toolchain line. This was already true before the plan. Workspace-mode builds, the project's norm, are green. +- `fonoteka.go/parity/parity_test.go`, `cmd/summer/main.go` and `cmd/summer/runtime.go` had gofmt drift at HEAD. It is out of scope and was left untouched. + +## Flagged assumptions (planner, edge probe) + +- (a) Two Dispatch calls get distinct ids (SERIAL), and River row locking keeps one job on one worker at a time. This was not load-tested; plan 11-07 can add a concurrency case. +- (b) Dispatch is not idempotent. Each call creates a new job, as in PHP. Confirmed by design. +- (c) Confirmed by test: `queue:work` and `serve` start and idle with no registered jobs, and `queue:clear` on an empty queue prints `Cleared 0 jobs`. + +## User Setup Required + +None. `config/queue.yaml` notes that a PgBouncer in front of Postgres must use session pooling (or be bypassed) for the worker's LISTEN connection. + +## Next Phase Readiness + +- Plan 11-02 (scheduler): the worker client is the place to attach `PeriodicJobs`. +- Plan 11-03 (broadcasts): it can use `conga.Manager.Enqueue` on the write's transaction and `lagoon.OnDatabase` for callbacks. +- Plan 11-05 (search sync): `lagoon.Transaction`/`AfterCommit` are ready. Existing `gdb.Transaction` call sites (such as `album_write_service.go`) still need migrating in Phase 12. +- Plan 11-07: it should add branch coverage for `client.go` settings parsing, `selectQueues`, `Worker.Stop`'s hard-stop fallback and the `Enqueue` non-transaction path. + +## Self-Check: PASSED + +- All 15 key created files exist on disk. +- summercms.go commits 0bc5c77, 05ba88a, b319e7c and 6c1f94e exist, and so do fonoteka.go commits 3d4895d, 0b9a2a5, e0b1b0d and 36fc473. +- The RED stub files were removed. +- `go vet ./... && go test ./...` passed in summercms.go, and the vet and test commands for all three modules passed in fonoteka.go, after the final task. + +--- +*Phase: 11-jobs-realtime-and-search-infrastructure* +*Completed: 2026-09-29*