golem15/summercms
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(11-02): run plugin schedules as River periodic jobs through bonfire.Call

Browse Source 
- pact.HasSchedule with ScheduledCommand and Daily/DailyAt/Every cadences (no River import)
- bonfire.Call, Catalog and ErrUnknownCommand for in-process command runs
- conga Daily/Every wall-clock schedules in app.timezone, periodic jobs on every worker,
  scheduled queue (MaxAttempts 1, unique by args within the cadence period)
- scheduled worker runs only entries matching the compiled table; unregistered
  commands are skipped with a Warn log
- generated app main publishes bonfire.NewCatalog(commands); hello main regenerated
This commit is contained in:
Jakub Zych
2026-09-29 19:47:51 +02:00
parent 77b8ff177a
commit d9f939a1ea
15 changed files with 890 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
modules/conga/README.md
View File

@@ -10,6 +10,8 @@ conga is the SummerCMS counterpart of the WinterCMS queue plus the apparatus job
Every River client runs on the one `*sql.DB` pool that `lagoon` opens. A worker started by `conga.StartWorker` uses `riverdatabasesql.NewWithPgxListener`: all queries go through the shared pool, and only Postgres `LISTEN` goes through a dedicated pgx pool with a single connection, so a job committed by any process wakes the worker immediately instead of waiting for the poll interval.
The scheduler is the Go form of WinterCMS `registerSchedule`. Plugins declare recurring console commands through `pact.HasSchedule`; every worker turns them into River periodic jobs on wall-clock `conga.Daily` and `conga.Every` schedules in the `app.timezone` location. Each due run is a job on the `scheduled` queue that calls the command in-process through the app's `bonfire.Catalog`.
## Features
- River-free job declarations: `conga.Job` turns `func(ctx context.Context, args T) error` into a `pact.Job`; `conga.OnQueue`, `conga.MaxAttempts` and `conga.Timeout` set per-job defaults. A `pact.Job` not built by `conga.Job` is rejected with `conga.ErrNotCongaJob`.
@@ -20,6 +22,8 @@ Every River client runs on the one `*sql.DB` pool that `lagoon` opens. A worker
- Cancellation in two parts: `conga.Manager.CancelJob` is the outside cancel (sets `is_canceled` and `conga.StatusStopped`, then cancels the River job, so a queued job never starts and a running job's context is cancelled); `conga.Manager.StopJob` is what a job calls on its own row after `conga.Manager.CheckIfCanceled` reports true (status only, the WinterCMS `cancelJob`).
- Outcome rules in the worker: an error on an attempt before the last leaves the row in progress so River can retry; the final failed attempt, or a recovered panic on it, sets `conga.StatusError` with the error text under the metadata key `error`; an error on a row that was stopped or cancelled cancels the River job instead. A job that returns nil without completing its row leaves it as it is. Skipped work is recorded as complete with `{"skipped": true}` metadata.
- Workers: `conga.StartWorker` registers every plugin job and starts one River client; `conga.WorkerOptions.Queues` limits it to some queues, and an unknown queue is `conga.ErrUnknownQueue` listing the known ones. `conga.Worker.Stop` stops it gracefully and cancels running jobs when its context ends. `conga.StartServeWorker` is the variant the `serve` command uses: it starts nothing when `queue.work_in_serve` is false. An app without jobs still gets a worker that starts and idles.
- Scheduled commands: every worker carries one River periodic job per `pact.HasSchedule` entry, in plugin activation order then declaration order, with the id `<plugin id>[<index>]:<command>`. Only the elected leader enqueues. Each run is a `conga.ScheduledCommandArgs` job on `conga.QueueScheduled` with one attempt (an interrupted run is not retried; the next period runs normally) and unique by args within its cadence period, so a leader failover cannot double-enqueue a period. The worker runs a job only when its entry exists in the compiled schedule and its command and arguments match that entry exactly, so a forged `river_job` row cannot run an arbitrary command. An unregistered command, or an app with no published `bonfire.Catalog`, is logged at Warn (`schedule: command not registered; skipping`) and skipped without failing the worker or other entries. Command output is logged line by line at Info with a `command` attribute; failures are logged with the duration.
- Wall-clock schedules: `conga.Daily` fires at the next `hh:mm` in its location and `conga.Every` at the next multiple of its interval since local midnight, so a restart never delays a daily run by up to a day the way `river.PeriodicInterval(24h)` would. On a DST day `conga.Daily` keeps the wall-clock time. A schedule entry with an empty command, a zero cadence, a daily time out of range, or an interval under one second or not dividing 24h fails the worker start with an error naming the plugin id and entry index.
- Commands: `conga.RuntimeCommands` adds `queue:work` and `queue:clear` to the application binary.
## Usage
@@ -94,6 +98,19 @@ func importPosts(ctx context.Context, m *conga.Manager, files []string) error {
}
```
A plugin schedules one of its registered commands; the worker runs it:
```go
var _ pact.HasSchedule = (*Plugin)(nil)
func (p *Plugin) Schedule() []pact.ScheduledCommand {
return []pact.ScheduledCommand{
{Command: "blog:prune-drafts", Cadence: pact.Daily()},
{Command: "blog:sync-feed", Args: []string{"--quiet"}, Cadence: pact.Every(15 * time.Minute)},
}
}
```
A worker runs in the same process or in a separate one:
```go
@@ -130,11 +147,15 @@ defer w.Stop(context.Background())
| `conga.Job` | Wraps a typed job function as a `pact.Job` that conga can run on River. |
| `conga.JobOption` | Per-job option: `conga.OnQueue`, `conga.MaxAttempts`, `conga.Timeout`. |
| `conga.JobID` | Returns the record row id of the job running in a context. |
| `conga.StartWorker` | Registers plugin jobs and starts a River worker client. |
| `conga.StartWorker` | Registers plugin jobs and starts a River worker client carrying the plugins' periodic schedule jobs. |
| `conga.StartServeWorker` | The worker of the `serve` command; nil when `queue.work_in_serve` is false. |
| `conga.WorkerOptions` | Selects the queues a worker runs. |
| `conga.RuntimeCommands` | Returns the `queue:work` and `queue:clear` commands. |
| `conga.Worker` | A running worker; `conga.Worker.Stop` stops it and `conga.Worker.Queues` lists its queues. |
| `conga.Daily` | `river.PeriodicSchedule` firing at `Hour:Minute` every day in `Loc` (UTC when nil). |
| `conga.Every` | `river.PeriodicSchedule` firing at every multiple of `Interval` since local midnight in `Loc` (UTC when nil). |
| `conga.ScheduledCommandArgs` | Args of one scheduled run: compiled `Entry` id, `Command` and `Args`; kind `summer.scheduled_command`. |
| `conga.QueueScheduled` | The `scheduled` queue that scheduled runs are inserted on. |
| `conga.ErrNoDatabase` | The app has not published the shared database handles. |
| `conga.ErrNotCongaJob` | A registered `pact.Job` was not built by `conga.Job`. |
| `conga.ErrRegistrationClosed` | Registration was attempted while a worker runs. |
@@ -149,7 +170,8 @@ Keys are read from the compass config (`config/queue.yaml`, or `SUMMER_QUEUE__..
| `queue.work_in_serve` | `true` | Whether the `serve` command runs the job worker in its own process. Set `false` when a separate `queue:work` process runs the jobs. |
| `queue.max_attempts` | `3` | Attempts per job before its record becomes `conga.StatusError`, unless the job or dispatch sets its own. |
| `queue.job_timeout` | `300` | Per-attempt deadline, in seconds or as a duration string such as `5m`, unless the job sets `conga.Timeout`. |
| `queue.queues.<name>` | `default: 4` | Concurrent workers per queue. The worker runs these queues plus every queue a registered job names plus `default`. |
| `queue.queues.<name>` | `default: 4`, `scheduled: 1` | Concurrent workers per queue. The worker runs these queues plus every queue a registered job names plus `default` and `scheduled`. |
| `app.timezone` | `UTC` | IANA location of `pact.Daily`, `pact.DailyAt` and `pact.Every` schedules. An unknown name fails the worker start. |
| `database.dsn` | none (required) | Also opens the worker's single-connection `LISTEN` pool. With PgBouncer, that connection must use session pooling or go straight to Postgres; transaction pooling cannot hold a `LISTEN`. |
```yaml
@@ -172,7 +194,7 @@ queues:
## Dependencies
- SummerCMS modules: [backpack](../backpack/README.md), [bouncer](../bouncer/README.md) (the dispatching principal), [lagoon](../lagoon/README.md) (the shared pool, `lagoon.JobsTable` and the migrations that create River's schema and `summer_jobs`), [pact](../pact/README.md), [party](../party/README.md).
- SummerCMS modules: [backpack](../backpack/README.md), [bonfire](../bonfire/README.md) (commands and the `bonfire.Catalog` scheduled runs call), [bouncer](../bouncer/README.md) (the dispatching principal), [lagoon](../lagoon/README.md) (the shared pool, `lagoon.JobsTable` and the migrations that create River's schema and `summer_jobs`), [pact](../pact/README.md), [party](../party/README.md).
- Third-party: `github.com/riverqueue/river` v0.47.0 with its `riverdriver/riverdatabasesql` and `rivertype` modules. River is the Postgres job queue the project stack names: it gives transactional inserts on the shared `*sql.DB`, retries, stuck-job rescue, leader election for periodic work and `LISTEN`/`NOTIFY` wake-ups. `github.com/jackc/pgx/v5/pgxpool` opens the listener pool; `gorm.io/gorm` writes the record rows.
## Testing
@@ -181,4 +203,4 @@ queues:
go test ./modules/conga/...
```
The tests start a `postgres:16-alpine` container through testcontainers-go and migrate a fresh database per test with `lagoon.Migrate`, so they need a running Docker daemon. `TestListenPickupLatency` sets a 30-second poll interval and checks that a job committed by a separate client is picked up in under one second, while a poll-only worker does not pick it up within two. `go test -short ./modules/conga/...` skips the database tests.
The tests start a `postgres:16-alpine` container through testcontainers-go and migrate a fresh database per test with `lagoon.Migrate`, so they need a running Docker daemon. `TestListenPickupLatency` sets a 30-second poll interval and checks that a job committed by a separate client is picked up in under one second, while a poll-only worker does not pick it up within two. `TestScheduleRunsCommand` checks that an `Every(1s)` entry runs its command through a periodic job and that an unregistered command is skipped with a Warn log. `go test -short ./modules/conga/...` skips the database tests.