diff --git a/.planning/STATE.md b/.planning/STATE.md index ee21e867..99ca04e3 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -1072,6 +1072,7 @@ Recent decisions affecting current work: | 261106-e5v | purge token rename leftovers: grzybyfunkcjonalne check-deploy refuses NUXT_SITE_* keys, journal/menu/faq READMEs name NUXT_CACHE_PURGE_TOKEN (todo 2026-10-20-rename-site-cache-purge-env-in-consumers.md) | 2026-11-06 | 4e2cf0c, f61c90a, 128a33d, a53511b | [261106-e5v-purge-token-rename-leftovers](./quick/261106-e5v-purge-token-rename-leftovers/) | | 261106-jhk | wristband MCP authorize drops OIDC identity scopes (openid, profile, email) so the ChatGPT connector can connect | 2026-11-06 | aba678b | [261106-jhk-wristband-mcp-authorize-ignores-oidc-ide](./quick/261106-jhk-wristband-mcp-authorize-ignores-oidc-ide/) | | 261106-kt1 | pier MCP 401 challenge names only resource_metadata, so clients request the advertised scopes (write included) | 2026-11-06 | 609caa4 | [261106-kt1-pier-mcp-401-challenge-drops-the-fixed-r](./quick/261106-kt1-pier-mcp-401-challenge-drops-the-fixed-r/) | +| 261106-kt2 | deferred:purge, migrate* and queue:clear reuse the app's published database, so the in-server scheduler no longer fails with duplicate provider for *sql.DB | 2026-11-06 | 99abe88 | [261106-kt2-lagoon-deferred-purge-fails-in-the-sched](./quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/) | ## Deferred Items diff --git a/.planning/quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/261106-kt2-PLAN.md b/.planning/quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/261106-kt2-PLAN.md new file mode 100644 index 00000000..64ea8fe6 --- /dev/null +++ b/.planning/quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/261106-kt2-PLAN.md @@ -0,0 +1,222 @@ +--- +phase: quick-261106-kt2 +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - modules/lagoon/commands.go + - modules/lagoon/purge_test.go + - modules/conga/schedule_test.go + - modules/conga/commands.go + - modules/conga/commands_test.go +autonomous: true +requirements: + - QUICK-261106-kt2 + +estimate: + tokens: 45000 + raw_tokens: 45000 + tasks: 2 + confidence: low + +must_haves: + truths: + - "When serve's in-process worker (or schedule:run) runs the framework entry summercms.lagoon[0]:deferred:purge, the command runs on the app's published database and succeeds; it no longer fails with `backpack: duplicate provider for *sql.DB`" + - "`summer deferred:purge` from the CLI (no database published) still opens, publishes and closes its own pool and purges exactly as before (TestPurgeDeferredCommand unchanged and green)" + - "A command run in-process never closes the shared pool it borrowed: after deferred:purge the published *sql.DB still answers PingContext and app.Lookup[*gorm.DB]() returns the same handle" + - "migrate, migrate:rollback and migrate:status share lagoon's withDB helper and therefore also work in-process with a published database" + - "queue:clear run in-process on an app with a published database (a plugin HasSchedule entry) clears the queue instead of failing with the duplicate-provider error" + artifacts: + - path: modules/lagoon/commands.go + provides: "withDB reuses the published *gorm.DB when present; otherwise the unchanged open -> Publish -> Close CLI path" + - path: modules/lagoon/purge_test.go + provides: "TestPurgeDeferredCommandPublishedDB: deferred:purge through bonfire.Call on an app whose pool is already published, real purge counts, pool still open afterwards" + - path: modules/conga/schedule_test.go + provides: "TestScheduledDeferredPurgeInProcess: callScheduled and schedule:run --once run deferred:purge from lagoon.RuntimeCommands on an app with a published pool" + - path: modules/conga/commands.go + provides: "queue:clear uses withAppDB" + - path: modules/conga/commands_test.go + provides: "TestQueueClear also clears through bonfire.Call on the published app" + key_links: + - from: modules/conga/scheduler.go runScheduled / callScheduled + to: modules/lagoon/commands.go deferred:purge Run -> withDB + via: "bonfire.Catalog.Call in-process; withDB must take app.Lookup[*gorm.DB]() instead of lagoon.Publish a second time" + - from: modules/lagoon/commands.go withDB + to: modules/backpack/services.go Registry.Publish + via: "the duplicate-provider error at services.go:36 is only reachable on the CLI-open branch, which now runs only when nothing is published" +--- + + +Fix the nightly framework schedule entry `deferred:purge`, which fails every night +in production since 2026-10-04 with +`level=ERROR msg="schedule: command failed" component=conga command=deferred:purge error="backpack: duplicate provider for *sql.DB"` +(framework v0.6.0; same code in v0.6.2). Root cause, verified by reading the code: +lagoon's unexported `withDB` (`modules/lagoon/commands.go:486`) always calls +`OpenFromApp` and then `lagoon.Publish`, which publishes `*sql.DB` on the app +(`modules/lagoon/connection.go:169`). From the CLI nothing is published yet, so it +works. Under serve the pool is already published (`modules/surf/serve.go:70-77`), and +serve's in-process worker runs the entry through `runScheduled` -> `callScheduled` -> +`bonfire.Catalog.Call` (`modules/conga/scheduler.go:148-182`), so the second Publish hits +`backpack: duplicate provider for *sql.DB` (`modules/backpack/services.go:36`). The same +happens under `schedule:run` (its `withDB` publishes first) and `schedule:run --once` +(`withAppDB` publishes first). Plugin commands such as `chat:prune-pending` succeed because +they take published services via `app.Lookup` and never publish a pool. + +The fix follows the two guarded helpers that already exist: cabana's `withAdminDB` +(`modules/cabana/commands.go:218`) and conga's `withAppDB` (`modules/conga/commands.go:340`): +reuse the published handle when there is one, open/publish/close only for a CLI run. + +Purpose: expired deferred bindings (abandoned uploads and records created under deferral) +get purged again by the scheduler; the CLI command keeps working unchanged. +Output: lagoon `withDB` fixed (deferred:purge plus the three migrate commands), conga +`queue:clear` switched to `withAppDB`, Docker-backed regression tests on both the lagoon +command and the conga scheduler path. + + + +@~/.claude/msd-core/workflows/execute-plan.md +@~/.claude/msd-core/templates/summary.md + + + +@.planning/STATE.md +@CLAUDE.md + +Verified facts (planner read these; do not re-investigate): +- Framework-registered `pact.ScheduledCommand` entries: only `lagoon.FrameworkSchedule` + (`modules/lagoon/schedule.go:57`, `deferred:purge` daily at `database.deferred_bindings.purge_at`, + default "03:00" in `app.timezone`, default UTC per `appLocation` in `modules/conga/schedule.go:91`). + No other module returns a `ScheduledCommand`. +- Sweep of command paths that open and publish a pool (`lagoon.OpenFromApp` + `lagoon.Publish` + outside tests): lagoon `withDB` (migrate, migrate:rollback, migrate:status, deferred:purge: + UNGUARDED, the bug); conga `withDB` (queue:work, schedule:run foreground, queue:clear: + unguarded); cabana `withAdminDB` (guarded); surf serve (the pool owner, intentionally opens). + `lagoon.Publish` is the only place that publishes `*sql.DB`. +- `lagoon.OpenFromApp` also calls `loadEncryptionKeysFromApp` -> `PublishEncryptionKeys`, which is + idempotent (`modules/lagoon/encrypted.go:130`). On the reuse branch the keys were already loaded + by whoever opened the pool (serve); deferred:purge does not touch Encrypted columns. +- `purgeBucket` (`modules/lagoon/commands.go:468`) already prefers the published `*blob.Bucket`. + With no bucket and no `storage.uploads.bucket_url` it returns nil, and `PurgeDeferred` refuses only + when an expired binding points at a file. +- Under `bonfire.Call` `confirmProduction` never asks (`modules/lagoon/commands.go:32-50`). +- Test helpers that exist: lagoon `newPurgeEnv`, `purgeConfig`, `purgePlugin`, `dedicatedDB` + (`modules/lagoon/purge_test.go`, `postgres_test.go`; Docker, skip under -short via `lagoonDB`); + conga `migratedDB`, `testApp` (publishes db + GORM like serve), `commandApp`, `testAppKey` + (`modules/conga/postgres_test.go:113,154`, `listen_test.go:274,295`), `onceRun`, `mustTime`, + `captureHandler` with `find(level, msg, key, value)` (`modules/conga/schedule_test.go:38-70,259`). +- Docker tests: if the Docker socket is denied although the user is in the docker group, run + them through `sudo -u golem` (CLAUDE.md). +- Concurrency: quick task 261106-kt1 is editing `modules/pier` and `docs/services/mcp.md`; do not + touch those. Stage files explicitly by path; leave `.planning/STATE.md`, `.planning/state.json`, + `.planning/G15OFFICE.md` and `examples/hello/plugins.gen.go` (other sessions' edits) alone. +- No README/docs change is expected: `withDB` and `withAppDB` are unexported, and the commands' + flags, output, config keys and CLI behaviour are unchanged. A grep of `modules/lagoon/README.md`, + `modules/conga/README.md`, `docs/plugins/scheduling.md` and `docs/console/` found no statement that + these commands always open their own connection. + +@modules/lagoon/commands.go +@modules/conga/commands.go +@modules/conga/scheduler.go + + + + + + Task 1: Tracer - scheduler -> catalog -> deferred:purge runs on the app's published pool (test first, then fix lagoon withDB) + modules/lagoon/purge_test.go, modules/conga/schedule_test.go, modules/lagoon/commands.go + + - modules/lagoon/commands.go (lines 304-366 deferred:purge Run, 464-496 purgeBucket and withDB) + - modules/lagoon/purge_test.go (lines 49-73 newPurgeEnv, 386-400 purgeConfig, 444-503 purgePlugin and TestPurgeDeferredCommand) + - modules/conga/scheduler.go (lines 145-200 runScheduled, callScheduled, scheduledCatalog) + - modules/conga/schedule_test.go (lines 38-70 captureHandler, 259-310 onceRun and onceApp) + - modules/conga/postgres_test.go (lines 111-176 migratedDB and testApp) + - modules/cabana/commands.go (lines 218-234 withAdminDB, the pattern to mirror) + + + - lagoon TestPurgeDeferredCommandPublishedDB: an app whose pool was opened with OpenFromApp and published with Publish (as serve does) runs deferred:purge --days 2 through bonfire.Call and gets nil error and "purged 2 deferred bindings older than 2 days (1 files, 1 children, 0 skipped)"; afterwards the published *sql.DB still pings and app.Lookup[*gorm.DB]() is the same handle; migrate:status through bonfire.Call on the same app also returns nil. + - conga TestScheduledDeferredPurgeInProcess: with testApp's published pool and a published bonfire.NewCatalog(lagoon.RuntimeCommands(app, nil)), callScheduled(ctx, app, log, "deferred:purge", nil, &buf) returns skipped false and nil error, buf contains "purged 0 deferred bindings older than 5 days", the capture handler has an Info "schedule: command finished" record with command=deferred:purge, the pool still pings; schedule:run --once at 2026-03-10 03:00:00 UTC returns nil and prints "Ran 1 due entries (0 failed)". + - Before the fix both tests fail with an error containing "backpack: duplicate provider for *sql.DB" (the production error). + - Existing TestPurgeDeferredCommand (CLI path through bonfire.NewRoot, nothing published) stays green unchanged. + + + RED. In modules/lagoon/purge_test.go, directly after TestPurgeDeferredCommand, add TestPurgeDeferredCommandPublishedDB with a doc comment naming the bug (the in-process scheduler runs deferred:purge on an app whose pool serve already published). Build the environment the way TestPurgeDeferredCommand does: newPurgeEnv(t, "lagoon_purge_published"), one unattached file via e.file(t, ""), one created child via e.child, and two bindings backdated 3 days via e.bind (DeferredFileType for the file, "acme_purge_children" with DeferredEnvelope{Created: true} for the child). Create the app with backpack.New(purgeConfig(t, ...)) carrying database.dsn = e.dsn and app.key = base64 of 32 bytes of 7 (same as the existing test; app.key is required so that, before the fix, the failure is the duplicate-provider error and not the app.key error). Open the pool exactly as serve does: OpenFromApp(t.Context(), app), register t.Cleanup closing that sqlDB, then Publish(app, sqlDB, gdb); then attach.Publish(app, e.bucket). Run the command the way bonfire.Catalog.Call does for the scheduler: bonfire.Call(t.Context(), RuntimeCommands(app, plugins), "deferred:purge", []string{"--days", "2"}, &out) with plugins = a purgePlugin listing &purgeChild{}. Assert nil error, the output line above, that e.fileExists/e.blobExists/e.bindingIDs show the file, its blob and both bindings gone, that sqlDB.PingContext(t.Context()) still succeeds, and that app.Lookup[*gorm.DB]() returns the same gdb pointer. Then call bonfire.Call on the same app for "migrate:status" and assert nil error (the migrate commands share the helper). + + In modules/conga/schedule_test.go add TestScheduledDeferredPurgeInProcess (Docker-backed through migratedDB, so it skips under -short). Use db, dsn := migratedDB(t) and app, _ := testApp(t, db, dsn, map with "app.key": testAppKey) — testApp publishes db and GORM on the app like serve. Publish slog.New over a fresh captureHandler on the app (as onceApp does), then publish bonfire.NewCatalog(lagoon.RuntimeCommands(app, nil)), the catalog the application binary publishes. Call callScheduled(t.Context(), app, loggerFromApp(app), "deferred:purge", nil, &buf) — the exact call runScheduled makes in serve's worker — and assert the behaviour above (skipped false, nil error, the "purged 0 deferred bindings older than 5 days" text in buf, the Info record via captureHandler.find, db.PingContext ok). Then run onceRun(t, app, nil, mustTime(t, time.UTC, "2026-03-10 03:00:00")) — the framework entry is due at the default 03:00 UTC — and assert nil error and "Ran 1 due entries (0 failed)" in the output. Add the lagoon import if schedule_test.go does not have it yet. + + Run both new tests (commands in verify) BEFORE touching production code and confirm each fails with an error containing "backpack: duplicate provider for *sql.DB". Copy the failing lines into the SUMMARY as the RED evidence. Do not commit the failing tests on their own: CLAUDE.md requires go test ./... green at every commit, so tests and fix land in one commit. + + GREEN. In modules/lagoon/commands.go change withDB: when app is non-nil and app.Lookup[*gorm.DB]() returns ok with a non-nil handle, call fn with that handle and return — no OpenFromApp, no Publish, and no Close, because the pool belongs to whoever published it (serve, schedule:run, queue:work, a test). Otherwise keep the current branch byte-for-byte: OpenFromApp, defer sqlDB.Close(), Publish, fn — so `summer deferred:purge` and the migrate commands from the CLI behave exactly as today. Mirror cabana's withAdminDB. Rewrite the doc comment of withDB in the repository's style: it runs fn on the app's published database when there is one (in-process runs: the scheduler in serve or schedule:run, bonfire.Catalog.Call), and otherwise opens and publishes a pool for the CLI run and closes it when fn returns. Do not change deferred:purge's Run body, purgeBucket, flags, output or Long text. This one change fixes deferred:purge and, through the shared helper, migrate, migrate:rollback and migrate:status. + + Re-run: both new tests plus the existing TestPurgeDeferredCommand pass. Commit with message "fix(lagoon): run deferred:purge and the migrate commands on the app's published database (quick-261106-kt2)" and a body of two or three lines stating the production symptom and that the CLI path is unchanged. Stage exactly modules/lagoon/commands.go, modules/lagoon/purge_test.go and modules/conga/schedule_test.go by path. No Co-Authored-By trailer (user rule). + + + cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./modules/lagoon/... ./modules/conga/... && L=$(go test ./modules/lagoon -run 'TestPurgeDeferredCommand' -count=1 -v 2>&1) && echo "$L" | grep -q -- '--- PASS: TestPurgeDeferredCommand ' && echo "$L" | grep -q -- '--- PASS: TestPurgeDeferredCommandPublishedDB' && C=$(go test ./modules/conga -run 'TestScheduledDeferredPurgeInProcess' -count=1 -v 2>&1) && echo "$C" | grep -q -- '--- PASS: TestScheduledDeferredPurgeInProcess' && grep -n 'Lookup\[\*gorm.DB\]' modules/lagoon/commands.go + + RED evidence (duplicate provider for *sql.DB from both new tests before the fix) is recorded; after the fix TestPurgeDeferredCommand, TestPurgeDeferredCommandPublishedDB and TestScheduledDeferredPurgeInProcess pass against testcontainers Postgres; lagoon withDB reuses a published *gorm.DB and never closes it; the CLI branch is unchanged; one commit with the three files. + + + + Task 2: Same fix for conga queue:clear, sweep record and full checks + modules/conga/commands_test.go, modules/conga/commands.go + + - modules/conga/commands.go (lines 94-127 queue:clear, 338-345 withAppDB, 399-411 withDB) + - modules/conga/commands_test.go (lines 20-85 TestQueueClear) + + + - TestQueueClear additionally clears through bonfire.Call on the published app: while the blocking job still holds the single default worker slot, two more pending ping jobs are enqueued, and queue:clear via bonfire.Call(ctx, RuntimeCommands(app, nil), "queue:clear", nil, &out) returns nil with "Cleared 2 jobs" in the output; the existing final assertions (block completes, zero ping rows left, zero pings ran) still hold. + - Before the fix that new step fails with "backpack: duplicate provider for *sql.DB". + - The existing CLI-style runClear (commandApp, nothing published) is unchanged and still passes. + + + RED. In modules/conga/commands_test.go extend TestQueueClear: after the existing runClear assertions and BEFORE close(release) (the blocking job keeps the one default worker slot busy, so new pings stay pending), enqueue two more pingArgs jobs with m.Enqueue as the test already does (use a different Tag, for example "pending-in-process"), then run queue:clear through bonfire.Call(ctx, RuntimeCommands(app, nil), "queue:clear", nil, &out) on the test's own published `app` — the in-process path a plugin pact.HasSchedule entry naming queue:clear would take. Assert nil error and "Cleared 2 jobs" in the output. Leave the existing runClear (commandApp, the CLI path) and the final assertions as they are. Run it and confirm the new step fails with "backpack: duplicate provider for *sql.DB"; record it in the SUMMARY. + + GREEN. In modules/conga/commands.go change only queue:clear's Run to call withAppDB instead of withDB. Leave queue:work and schedule:run (foreground) on withDB: both are long-running processes started from the CLI that own their pool, and running either as an in-process scheduled entry is not meaningful. Extend withAppDB's doc comment to say it serves schedule:run --once and queue:clear, which may run in-process on an app that already published its pool. Do not change flags, output, the confirmation question or the Long text. + + Sweep record for the SUMMARY (no code change beyond this task): framework ScheduledCommand registrations are only lagoon.FrameworkSchedule's deferred:purge; remaining unguarded OpenFromApp + Publish paths are conga withDB for queue:work and schedule:run (CLI pool owners, intentionally) and surf serve (the pool owner); cabana withAdminDB was already guarded. Confirm with grep -rn 'lagoon.OpenFromApp' modules --include=*.go excluding _test.go. + + Full checks (commands in verify): go vet ./..., go test -short ./... (CLAUDE.md keeps it green at every commit), then the Docker-backed go test ./modules/lagoon/... ./modules/conga/... -count=1. If the Docker socket is denied, rerun the Docker step through sudo -u golem per CLAUDE.md. + + Commit with message "fix(conga): queue:clear reuses the app's published database (quick-261106-kt2)", staging exactly modules/conga/commands.go and modules/conga/commands_test.go by path. No Co-Authored-By trailer. Do not push or tag. + + + cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./... && go test -short ./... && Q=$(go test ./modules/conga -run 'TestQueueClear$' -count=1 -v 2>&1) && echo "$Q" | grep -q -- '--- PASS: TestQueueClear ' && grep -B1 'return clearQueue(ctx, app, queue, out)' modules/conga/commands.go | grep -q withAppDB && go test ./modules/lagoon/... ./modules/conga/... -count=1 + + queue:clear uses withAppDB; TestQueueClear passes both its CLI run and the new in-process run; go vet ./..., go test -short ./... and the Docker-backed lagoon and conga suites are green; sweep result recorded; one commit with the two conga files; nothing under modules/pier or docs/services/mcp.md touched. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| river_job rows -> scheduled command runner | A scheduled job row names an entry and command; runScheduled runs only an exact match of the compiled schedule table (T-11-09, unchanged) | +| in-process command -> app's shared database pool | A command run through bonfire.Catalog.Call borrows the pool serve owns and must not close or replace it | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-261106-kt2-01 | Denial of Service | lagoon withDB / conga withAppDB reuse branch closing the borrowed pool would take serve's database away | high | mitigate | Reuse branch calls fn only, with no defer Close and no Publish; TestPurgeDeferredCommandPublishedDB and TestScheduledDeferredPurgeInProcess assert the published *sql.DB still pings and the same *gorm.DB stays published after the command | +| T-261106-kt2-02 | Information Disclosure | deferred:purge failing nightly keeps abandoned pending uploads (files and records) past the configured retention (default 5 days) | medium | mitigate | The fix itself plus the scheduler-path regression test that runs the framework entry through callScheduled and schedule:run --once | +| T-261106-kt2-03 | Elevation of Privilege | scheduler executing an arbitrary command from a forged river_job row | low | accept | Unchanged: runScheduled matches entry, command and args against the compiled table (T-11-09, covered by TestScheduledEntryMismatchSkipped); this plan does not touch scheduler.go | +| T-261106-kt2-04 | Tampering | CLI branch regressions (migrate, deferred:purge from the terminal) | low | mitigate | CLI branch kept byte-for-byte; existing TestPurgeDeferredCommand (bonfire.NewRoot, nothing published) and TestQueueClear's runClear (commandApp) keep covering it | +| T-261106-kt2-SC | Tampering | npm/pip/cargo installs | low | accept | No package installs or dependency changes in this plan | + + + +- `go vet ./...` and `go test -short ./...` green. +- Docker: `go test ./modules/lagoon/... ./modules/conga/... -count=1` green, including TestPurgeDeferredCommand, TestPurgeDeferredCommandPublishedDB, TestScheduledDeferredPurgeInProcess and TestQueueClear. +- SUMMARY holds the RED output (duplicate provider for *sql.DB) for the three new test steps and the sweep result. +- `git show --stat HEAD~1 HEAD` lists exactly the three Task 1 files and the two Task 2 files; nothing under modules/pier, docs/ or .planning/ in the code commits. + + + +The framework schedule entry deferred:purge succeeds when serve's worker, schedule:run or schedule:run --once runs it on an app whose database is already published, without closing that pool; `summer deferred:purge` and the migrate commands from the CLI behave as before; queue:clear works in-process too; all pinned by Docker-backed tests that failed with the production error before the fix. + + + +Create `.planning/quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/261106-kt2-SUMMARY.md` when done, with the RED evidence, the sweep result, the two commit hashes, and a note that no README/docs change was needed (unexported helpers, command behaviour from the CLI unchanged). Note for the user: the fix reaches production only with the next framework patch tag and an app bump; do not tag or push here. + diff --git a/.planning/quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/261106-kt2-SUMMARY.md b/.planning/quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/261106-kt2-SUMMARY.md new file mode 100644 index 00000000..9cbb9bfe --- /dev/null +++ b/.planning/quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/261106-kt2-SUMMARY.md @@ -0,0 +1,106 @@ +--- +phase: quick-261106-kt2 +plan: 01 +subsystem: lagoon, conga +tags: [scheduler, deferred-bindings, database-pool, bugfix] +status: complete +requires: [] +provides: + - "lagoon withDB reuses the published *gorm.DB (deferred:purge, migrate, migrate:rollback, migrate:status work in-process)" + - "conga queue:clear runs through withAppDB" +affects: [conga scheduler, serve in-process worker, schedule:run] +tech-stack: + added: [] + patterns: ["reuse the published handle, open/publish/close only for a CLI run (same as cabana withAdminDB, conga withAppDB)"] +key-files: + created: [] + modified: + - modules/lagoon/commands.go + - modules/lagoon/purge_test.go + - modules/conga/schedule_test.go + - modules/conga/commands.go + - modules/conga/commands_test.go +decisions: + - "lagoon withDB borrows the published *gorm.DB without Publish or Close; the pool belongs to whoever published it" + - "queue:work and foreground schedule:run stay on conga withDB: long-running CLI processes that own their pool" +metrics: + duration: ~15min + completed: 2026-11-06 +actuals: + tokens: 2070 + tasks: 2 + commits: 3 +plan_head_before: 609caa4f3f1f24f123696ab1d8fd8b6bfcef113a +plan_head_after: 99abe88b6a33275817ae3f90d33b2f8a50717a2f +--- + +# Quick 261106-kt2: deferred:purge fails in the scheduler, Summary + +The nightly framework entry `deferred:purge` now runs on the app's published database pool under serve's in-process worker, `schedule:run` and `schedule:run --once`. It no longer fails with `backpack: duplicate provider for *sql.DB`. `queue:clear` got the same fix. + +## What changed + +- `modules/lagoon/commands.go` `withDB`: when the app already has a published `*gorm.DB`, `fn` runs on that handle. There is no `OpenFromApp`, no `Publish` and no `Close`. Otherwise the CLI branch (open, defer Close, Publish, fn) is unchanged. This fixes `deferred:purge`, `migrate`, `migrate:rollback` and `migrate:status`. It mirrors cabana's `withAdminDB`. +- `modules/conga/commands.go`: `queue:clear` calls `withAppDB` instead of `withDB`, and the `withAppDB` doc comment now names schedule:run --once and queue:clear. + +## Commits + +| Task | Commit | Message | +|------|--------|---------| +| 1 | 16a2008 | fix(lagoon): run deferred:purge and the migrate commands on the app's published database (quick-261106-kt2) | +| 2 | 99abe88 | fix(conga): queue:clear reuses the app's published database (quick-261106-kt2) | + +`actuals.commits` reads 3 because the ledger range `609caa4..HEAD` also holds `8ebfedf`, a docs commit from the concurrent quick task 261106-kt1. This task made 2 commits. + +## RED evidence (before the fix) + +``` +=== RUN TestPurgeDeferredCommandPublishedDB + purge_test.go:532: deferred:purge: backpack: duplicate provider for *sql.DB +--- FAIL: TestPurgeDeferredCommandPublishedDB (0.24s) + + schedule_test.go:275: callScheduled: skipped=false err=conga: scheduled command deferred:purge: backpack: duplicate provider for *sql.DB +--- FAIL: TestScheduledDeferredPurgeInProcess (0.28s) + + commands_test.go:86: in-process queue:clear: backpack: duplicate provider for *sql.DB +--- FAIL: TestQueueClear (10.38s) +``` + +TestPurgeDeferredCommand (the CLI path) passed both before and after the fix. + +## GREEN / verification + +- `go vet ./...`: ok +- `go test -short ./...`: ok +- Docker: `go test ./modules/lagoon/... ./modules/conga/... -count=1`: all ok (lagoon 22.2s, lagoon/attach 40.8s, lagoon/attach/webp, conga 31.5s). Includes TestPurgeDeferredCommand, TestPurgeDeferredCommandPublishedDB, TestScheduledDeferredPurgeInProcess and TestQueueClear. +- The new tests check that the published `*sql.DB` still answers PingContext after the command and that `app.Lookup[*gorm.DB]()` returns the same handle (T-261106-kt2-01). + +## Sweep result + +`grep -rn 'OpenFromApp(' modules --include=*.go` (non-test): +- `modules/lagoon/commands.go` withDB: now guarded (this task) +- `modules/cabana/commands.go` withAdminDB: already guarded +- `modules/conga/commands.go` withDB: used only by queue:work and foreground schedule:run. Both are long-running CLI processes that own their pool, so this is intentional. queue:clear moved to withAppDB. +- `modules/surf/serve.go`: serve owns the pool, so it opens one on purpose. + +The only framework `ScheduledCommand` registration is `lagoon.FrameworkSchedule` (`deferred:purge`). + +## Deviations from Plan + +**1. [Rule 3 - Blocking] TestQueueClear's app needed `app.key`** +- **Found during:** Task 2 RED +- **Issue:** testApp in TestQueueClear had no `app.key`, so the in-process queue:clear first failed with `lagoon: app.key is empty or invalid` rather than the production error. +- **Fix:** added `"app.key": testAppKey` to that test's testApp config. The RED run then showed `backpack: duplicate provider for *sql.DB`. The CLI-style runClear and the final assertions are unchanged. +- **Files modified:** modules/conga/commands_test.go +- **Commit:** 99abe88 + +## Notes + +- No README or docs change was needed: `withDB` and `withAppDB` are unexported, and the commands' flags, output, config keys and CLI behaviour are unchanged. +- The fix reaches production only with the next framework patch tag and an app bump. Nothing was tagged or pushed. +- No files under modules/pier or docs/services/mcp.md were touched. + +## Self-Check: PASSED + +- FOUND: modules/lagoon/commands.go, modules/lagoon/purge_test.go, modules/conga/schedule_test.go, modules/conga/commands.go, modules/conga/commands_test.go +- FOUND: 16a2008, 99abe88