Files
summercms/.planning/phases/11-jobs-realtime-and-search-infrastructure/11-02-SUMMARY.md
2026-09-29 22:54:28 +02:00

17 KiB

phase, plan, subsystem, tags, requires, provides, affects, actuals, plan_head_before, plan_head_after, tech-stack, key-files, key-decisions, patterns-established, requirements-completed, coverage, duration, completed, status
phase plan subsystem tags requires provides affects actuals plan_head_before plan_head_after tech-stack key-files key-decisions patterns-established requirements-completed coverage duration completed status
11-jobs-realtime-and-search-infrastructure 02 infra
river
scheduler
periodic-jobs
cli
cron
bonfire
phase provides
11-jobs-realtime-and-search-infrastructure 11-01 conga worker (StartWorker, Manager, conga.Job, queue settings) on River v0.47.0
pact.HasSchedule, ScheduledCommand, Cadence with Daily/DailyAt/Every (no River import)
bonfire.Call, Catalog, NewCatalog, ErrUnknownCommand for in-process command runs
conga.Daily / conga.Every wall-clock river.PeriodicSchedule in app.timezone
Every worker carries plugin schedules as River PeriodicJobs; runs go to the scheduled queue (MaxAttempts 1, unique by args within the cadence period)
schedule:run (scheduler-only worker) and schedule:run --once (Laravel minute-match for system cron), plus the summer delegate
Generated app main publishes bonfire.NewCatalog(commands)
fonoteka declares fonoteka:prune-notifications daily in schedule.go
11-03 broadcasts
11-07 unit tests
14 domain jobs (prune-notifications command)
15 cutover (cron/process layout)
tokens tasks commits
13984 2 2
77b8ff177a 2237a640d2
added patterns
Plugins declare recurring commands with pact.HasSchedule; conga compiles them into River periodic jobs with ids <plugin id>[<index>]:<command>
Scheduled runs call commands in-process through the app's *bonfire.Catalog, published by the generated main
A scheduled job runs only when it matches a compiled schedule entry exactly (T-11-09)
created modified
modules/bonfire/call.go
modules/bonfire/call_test.go
modules/conga/schedule.go
modules/conga/scheduler.go
modules/conga/schedule_test.go
../fonoteka.go/plugins/golem15/fonoteka/schedule.go
modules/pact/capabilities.go
modules/pact/README.md
modules/bonfire/README.md
modules/conga/conga.go
modules/conga/worker.go
modules/conga/commands.go
modules/conga/listen_test.go
modules/conga/README.md
internal/build/build.go
internal/build/build_test.go
cmd/summer/main.go
cmd/summer/runtime.go
cmd/summer/main_test.go
examples/hello/main.go
../fonoteka.go/main.go
Periodic job ids are <plugin id>[<index>]:<command>, not <plugin id>#<index>:<command>: River's PeriodicJob.ID regex rejects '#'
Scheduled runs are unique by args and period (UniqueOpts{ByArgs: true, ByPeriod}); ByPeriod alone would collapse different entries of the shared summer.scheduled_command kind within one period
Every(d) requires d >= 1s (River's ByPeriod floor) and 24h % d == 0; DailyAt rejects hours outside 0-23 and minutes outside 0-59
schedule:run --once opens the database only when an entry is due, and reuses a published *gorm.DB when there is one; plugin commands look the DB up from the app
Every(d) in --once is due every run for d <= 1 minute, otherwise when wall-clock minutes since midnight are a multiple of d
bonfire.Catalog: the final command list is an app service, so non-CLI code can run commands with Call
Forged-row defence: jobs whose args name code to execute are checked against a compiled table before running
CLI-04
id description requirement verification human_judgment
D1 pact.HasSchedule with Daily/DailyAt/Every cadences and no River import; conga.Daily/Every compute wall-clock next runs (adjacency, DST, midnight rollover) and invalid cadences are rejected CLI-04
kind ref status
unit modules/conga/schedule_test.go#TestScheduleNext pass
kind ref status
unit modules/conga/schedule_test.go#TestScheduleEntries pass
false
id description requirement verification human_judgment
D2 bonfire.Call and Catalog run a named command in-process with args; unknown names wrap ErrUnknownCommand CLI-04
kind ref status
unit modules/bonfire/call_test.go#TestCall pass
false
id description requirement verification human_judgment
D3 A plugin's Every(1s) entry runs through a River periodic job and bonfire.Call inside a worker; an unregistered command is skipped with a Warn log CLI-04
kind ref status
integration modules/conga/schedule_test.go#TestScheduleRunsCommand pass
false
id description requirement verification human_judgment
D4 Generated app main publishes bonfire.NewCatalog(commands) after plugin commands and before NewRoot; hello and fonoteka mains regenerated CLI-04
kind ref status
unit internal/build/build_test.go#TestGenerateMainPublishesCommandCatalog pass
false
id description requirement verification human_judgment
D5 schedule:run --once minute-match semantics (daily, Every(5m), app.timezone), unknown command warns and others run, first error returned after all ran; schedule:run foreground runs a scheduler-only worker CLI-04
kind ref status
unit modules/conga/schedule_test.go#TestScheduleRunOnce pass
kind ref status
integration modules/conga/schedule_test.go#TestScheduleRunForeground pass
kind ref status
unit cmd/summer/main_test.go#TestToolCommandNames pass
false
id description requirement verification human_judgment
D6 T-11-09: a scheduled job whose command/args differ from its compiled entry, or whose entry is unknown, is skipped without running anything, including a forged river_job row CLI-04
kind ref status
integration modules/conga/schedule_test.go#TestScheduledEntryMismatchSkipped pass
false
id description requirement verification human_judgment
D7 T-11-17: two inserts of one entry inside its period leave one river_job row; a different entry in the same period gets its own row CLI-04
kind ref status
integration modules/conga/schedule_test.go#TestScheduleUniqueByPeriod pass
false
id description requirement verification human_judgment
D8 fonoteka declares fonoteka:prune-notifications daily; `fonoteka schedule:run --once` prints the no-commands line and exits 0; --help shows --once CLI-04
kind ref status
other cd ../fonoteka.go && SUMMER_GOLEM15__USER__JWT__SECRET=test-only-cli-secret go run . schedule:run --once pass
false
id description verification human_judgment rationale
D9 Concurrency backstop: with several worker processes River leader election means one process enqueues each period
true Multi-process leader election is River's guarantee and is not exercised by a test here (plan marks it verification: backstop)
385min 2026-09-29 complete

Phase 11 Plan 02: scheduler on River periodic jobs Summary

pact.HasSchedule with Daily/DailyAt/Every cadences becomes River periodic jobs on wall-clock schedules in app.timezone. Each run is a one-attempt job on the scheduled queue that calls the command in-process through a bonfire.Catalog the generated main publishes. schedule:run runs a scheduler process, and schedule:run --once serves system cron.

Performance

  • Duration: 385 min wall-clock. The clock ran from 14:09Z to 20:34Z; the active work was much shorter, and most of the span was idle time between tool calls.
  • Started: 2026-09-29T14:09:06Z
  • Completed: 2026-09-29T20:34:36Z
  • Tasks: 2
  • Files modified: 21 (19 in summercms.go, 2 in fonoteka.go)

Accomplishments

  • pact has the schedule capability: HasSchedule, ScheduledCommand, the opaque Cadence (IsZero, Interval, At) and the Daily, DailyAt and Every constructors. pact still imports no River code.
  • bonfire.Call and bonfire.Catalog run any registered command in-process with empty stdin. The generated app main publishes bonfire.NewCatalog(commands) before building the root. The hello and fonoteka mains were regenerated.
  • conga.Daily and conga.Every are river.PeriodicSchedule implementations:
    • Daily fires at the next wall-clock hh:mm. On DST days the runs are 23h or 25h apart but stay at 12:00 local.
    • Every fires at the next multiple of its interval since local midnight and rolls over to the next midnight.
    • Both fire strictly after an exact boundary.
  • Every worker carries one periodic job per schedule entry, with id <plugin id>[<index>]:<command>. Runs go to the scheduled queue (queue.queues.scheduled, default 1) with MaxAttempts 1 and uniqueness by args within the cadence period.
  • The worker runs a job only when it matches a compiled entry exactly (T-11-09). A missing catalog or an unregistered command is logged at Warn and skipped (user decision 5). Command output is logged line by line with a command attribute.
  • schedule:run runs a scheduler-only worker until SIGINT or SIGTERM. schedule:run --once runs the entries due this minute without River, prints Running scheduled command: … or No scheduled commands are ready to run., and returns the first command error. The summer delegate forwards --once.
  • fonoteka declares fonoteka:prune-notifications daily in schedule.go. plugin.go is untouched, so plan 11-03 can edit it.

Task Commits

summercms.go:

  1. Task 1: an interval command runs on schedule inside a worker through bonfire.Call - d9f939a (feat)
  2. Task 2: schedule:run as a dedicated process and from system cron - 2237a64 (feat)

fonoteka.go:

  1. Task 1 - 37133f9 (feat: regenerated main.go publishes the catalog)
  2. Task 2 - 94e95de (feat: schedule.go with the daily prune entry)

Files Created/Modified

  • modules/pact/capabilities.go, modules/pact/README.md: the schedule capability and cadences
  • modules/bonfire/call.go, call_test.go, README.md: Call, Catalog, NewCatalog and ErrUnknownCommand
  • modules/conga/schedule.go: Daily, Every, scheduleFor and appLocation
  • modules/conga/scheduler.go: QueueScheduled, ScheduledCommandArgs, entry compilation, periodic jobs, the built-in job, the T-11-09 check and the log writer
  • modules/conga/worker.go, conga.go: StartWorker sets PeriodicJobs, registers the built-in job and adds the scheduled queue
  • modules/conga/commands.go: schedule:run [--once]
  • modules/conga/schedule_test.go: seven tests. listen_test.go: the serve worker now also lists scheduled
  • internal/build/build.go, build_test.go: the catalog publish in the generated main
  • cmd/summer/main.go, runtime.go, main_test.go: the schedule:run delegate
  • examples/hello/main.go and ../fonoteka.go/main.go (regenerated), and ../fonoteka.go/plugins/golem15/fonoteka/schedule.go

Decisions Made

See key-decisions in the frontmatter.

TDD Gate Compliance

Task 2 is tdd="true". The tests were written first against a stub schedule:run that returned "not implemented".

  • RED: TestScheduleRunOnce failed on its assertions in all 6 subtests. gsd-tools check tdd-red-evidence returned RED_EVIDENCE_OK (target_test_failed). The command was go test ./modules/conga -run '^(TestScheduleRunOnce|TestScheduledEntryMismatchSkipped|TestScheduleUniqueByPeriod)$' -count=1 -json, converted to TAP.
  • TestScheduledEntryMismatchSkipped and TestScheduleUniqueByPeriod passed at RED. Task 1's action required the exact-match check (T-11-09) and the ByPeriod insert options, so both mitigations already existed. To prove the tests are real, I ran a mutation check and restored the code afterwards:
    • Running the job's own command instead of the compiled entry failed TestScheduledEntryMismatchSkipped ("no skip warning").
    • Dropping ByArgs failed TestScheduleUniqueByPeriod ("river_job rows = 1, want 2").
  • Gate violation, flagged: there is no separate test(11-02) RED commit. The project CLAUDE.md requires go vet and go test ./... to be green at every commit, so the tests and implementation landed together in 2237a64. Plan 11-01 made the same call.

Deviations from Plan

Auto-fixed Issues

1. [Rule 1 - Bug] Periodic job id separator

  • Found during: Task 1 (reading River's PeriodicJob.validate)
  • Issue: River requires PeriodicJob.ID to match \A[\w][\w\-\[\]<>\/.·:+]+\z, which has no #. The plan's <plugin id>#<index>:<command> would have failed every worker start that had a schedule.
  • Fix: Ids use <plugin id>[<index>]:<command>, for example golem15.fonoteka[0]:fonoteka:prune-notifications. Ordering, uniqueness and readability are unchanged.
  • Files modified: modules/conga/scheduler.go
  • Verification: TestScheduleEntries checks the ids. TestScheduleRunsCommand starts a worker with them.
  • Committed in: d9f939a

2. [Rule 1 - Bug] ByPeriod alone deduplicates across entries

  • Found during: Task 1 (River UniqueOpts docs: without ByArgs, uniqueness applies to all jobs of the kind)
  • Issue: Every entry shares the kind summer.scheduled_command, so ByPeriod alone would drop the second entry due in the same period.
  • Fix: UniqueOpts{ByArgs: true, ByPeriod: period}. The args hold the entry id, so the dedupe is per entry.
  • Files modified: modules/conga/scheduler.go
  • Verification: TestScheduleUniqueByPeriod checks both parts: the same entry twice gives 1 row, and a second entry gives 2 rows. The mutation check confirms the second part catches the bug.
  • Committed in: d9f939a

3. [Rule 2 - Missing validation] Cadence bounds

  • Found during: Task 1
  • Issue: River rejects ByPeriod under 1s. DailyAt(25, 0) would silently normalize to 01:00 the next day.
  • Fix: scheduleFor also rejects intervals under 1s and daily times outside 00:00-23:59, with the plugin id and entry index in the error.
  • Files modified: modules/conga/schedule.go
  • Verification: TestScheduleNext covers the invalid cadences.
  • Committed in: d9f939a

4. [Rule 3 - Blocking] schedule:run --once needs the database for real commands

  • Found during: Task 2 (fonoteka commands call app.Lookup[*gorm.DB]())
  • Issue: The plan says "no River" for --once but does not say how commands reach the database.
  • Fix: When an entry is due, --once opens and publishes the database the way queue:work does, or reuses an already published *gorm.DB. With nothing due it touches no database.
  • Files modified: modules/conga/commands.go
  • Verification: The TestScheduleRunOnce subtests. fonoteka schedule:run --once exits 0.
  • Committed in: 2237a64

5. [Behaviour change, documented] Serve and queue:work workers list the scheduled queue

  • The plan requires the scheduled queue on every known-queue set, so TestStartServeWorker now expects [default scheduled]. The unknown-queue error lists it too. TestQueueWorkCommand still passes because the list starts with default.

6. [Additions] Extra tests and small internals

  • Added TestScheduleEntries (the empty, invalid and ordering edges) and TestScheduleRunForeground (the foreground schedule:run).
  • The unexported scheduleNow clock serves the --once tests.
  • gofmt reordered the pre-existing out-of-order imports of cmd/summer/main.go and runtime.go. Both files were already being edited.

Total deviations: 4 auto-fixed (2 bugs, 1 missing validation, 1 blocking), plus 2 documented notes. Impact on plan: Fixes 1 and 2 were needed for schedules to start at all and to run every entry. No scope creep.

Issues Encountered

  • internal/build/registry.go still has gofmt drift from before this plan. It is out of scope and was left untouched.

Known Stubs

None. The fonoteka entry names fonoteka:prune-notifications, which Phase 14 registers. Until then the scheduler warns and skips it by design (user decision 5, research Open Question 5).

User Setup Required

None. For system cron, run * * * * * cd /app && ./bin/<app> schedule:run --once. Alternatively run a schedule:run process, or rely on the worker inside serve/queue:work, which already processes the scheduled queue.

Next Phase Readiness

  • Plan 11-03 can register the broadcast job through conga.Manager and edit fonoteka plugin.go. This plan left both alone.
  • Plan 11-07 should add coverage for:
    • logWriter partial-line flushing
    • the missing-catalog skip path
    • dueAt with Every(d) over 1 minute that does not divide an hour (for example 90s)
    • the summer schedule:run --once argv forwarding
  • In Phase 14, registering fonoteka:prune-notifications makes the daily entry run with no scheduler change.

Self-Check: PASSED

  • All six created files exist on disk.
  • summercms.go commits d9f939a and 2237a64 exist, and so do fonoteka.go commits 37133f9 and 94e95de.
  • After Task 2, go vet ./... && go test ./... passed in summercms.go, and the fonoteka.go vet and test command for all three modules passed.
  • All acceptance-criteria greps and go doc checks for both tasks passed.

Phase: 11-jobs-realtime-and-search-infrastructure Completed: 2026-09-29