25 KiB
25 KiB
phase, verified, status, score, covered_files, covered_digest, covered_files_note, mvp_mode_note, behavior_unverified, overrides_applied, re_verification, advisory, behavior_unverified_items, human_verification
| phase | verified | status | score | covered_files | covered_digest | covered_files_note | mvp_mode_note | behavior_unverified | overrides_applied | re_verification | advisory | behavior_unverified_items | human_verification | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 11-jobs-realtime-and-search-infrastructure | 2026-09-30T20:26:54Z | human_needed | 5/5 roadmap success criteria verified; plan truths 67/70 verified, 3 backstop (insufficient_spec, routed to human) |
|
v2:sha256:8668b94496d5c813e84363932641ab9a9f277a34db12d3268aa94cae599fafc7 | fonoteka.go files are outside the project root and cannot be fingerprinted. They were checked at fonoteka.go HEAD 1c88199 with a clean working tree and are listed in the report body. | ROADMAP marks Phase 11 mode: mvp, but the goal is not a User Story and no 11-*-PLAN.md carries one. Following the Phase 1/3/5/8/9/10 precedent, the five ROADMAP success criteria are the contract, User Flow Coverage is derived from them, and plan must_haves are supporting evidence. | 0 | 0 |
|
|
Phase 11: Jobs, realtime and search infrastructure — Verification Report
Phase Goal: River jobs run on the correct dual-driver split, Centrifugo publishing and channel authorization match the existing server, and Typesense sync stays a re-gated pre-filter — all brought up before the API phases that depend on them.
Verified: 2026-09-30T20:26:54Z (summercms.go HEAD c6f6bdf, fonoteka.go HEAD 1c88199; fonoteka.go clean, summercms.go has only unrelated pre-existing planning/untracked workspace changes)
Status: human_needed
Re-verification: Yes — CR-01 gap closure
MVP note: ROADMAP marks this phase mode: mvp, but the goal is not a User Story and no plan carries one. Following the precedent of Phases 1, 3, 5, 8, 9 and 10, the five ROADMAP success criteria are the contract.
User Flow Coverage
Derived story: As a host application developer, I want to dispatch River jobs, schedule commands, publish realtime model events through the existing Centrifugo and keep Album search documents in Typesense, so that the API phases can rely on jobs, realtime and search without the PHP backend.
| Step | Expected | Evidence | Status |
|---|---|---|---|
| Dispatch a job and see its outcome | summer_jobs row + River job in one tx; complete/fail/skip queryable | conga.Dispatch (InsertTx on the gorm sql.Tx), Manager.Get; TestDispatchTransactional, TestOutcome pass | ✓ |
| Worker picks jobs up promptly | LISTEN wake-up, not poll latency | TestListenPickupLatency re-run: 3ms pickup with a 30s poll; poll-only control not picked up in 2s | ✓ |
Run queue:work / schedule:run |
Foreground worker; scheduler runs registered commands | conga.RuntimeCommands wired into generated main and fonoteka main.go; TestQueueWork, TestScheduleRunForeground pass | ✓ |
| Nuxt gets a token and subscribes | Same HS256 claims; proxy re-authorizes | TokenIssuer matches PHP JwtTokenGenerator claim-for-claim; ProxyHandler looks up registry per request; parity routes replay green | ✓ (live Centrifugo: human) |
| Album edit reaches Typesense correctly | Committed document, collection-scoped, gated | Cabana and SaveAlbum use lagoon.Transaction; the assembled-router regression compares the one imported document's artist_ids with committed ordered pivots |
✓ |
Goal Achievement
Roadmap Success Criteria
| # | Success criterion | Status | Evidence |
|---|---|---|---|
| 1 | River on shared *sql.DB (riverdatabasesql, transactional enqueue) + LISTEN on a small separate pgx pool (NewWithPgxListener, one client), timed pickup test; outcomes queryable via job manager | ✓ VERIFIED | modules/conga/worker.go:135-146 builds riverdatabasesql.NewWithPgxListener(sqlDB, listener) with the published *sql.DB; listenerPool sets MaxConns 1/MinConns 0. One worker client per Manager (second start errors); inserts reuse it when running. conga.go:158-198 InsertTx on tx.Statement.ConnPool.(*sql.Tx). runAttempt sets ERROR only on final attempt/panic; skip = CompleteJob {"skipped":true}; Manager.Get exposes the row. Verifier re-ran TestListenPickupLatency: listen 3.0ms, poll-only control not picked up in 2s. |
| 2 | summer queue:work runs the worker; summer schedule:run runs recurring commands |
✓ VERIFIED (WR-02 warning) | conga/commands.go queue:work, schedule:run (daemon and --once), queue:clear; appended by internal/build/build.go:115 and in ../fonoteka.go/main.go:39; bonfire.NewCatalog published (build.go:124). Scheduled worker calls commands through the catalog. Tests: TestQueueWork, TestScheduleRunsCommand, TestScheduleRunForeground, TestScheduleRunOnce, TestScheduleUniqueByPeriod pass in the gate. Caveat WR-02: a scheduled command that opens its own DB via the withDB pattern fails inside a running worker (backpack: duplicate provider for *sql.DB). |
| 3 | Token + subscription JWTs with the same secret/claims/channel names at GET /api/realtime/token; publishes to existing Centrifugo | ✓ VERIFIED (live server: human) | centrifugo/token.go ports all five PHP generators with identical claims (sub string id, exp, info.name only; anonymous 300s; identifier info []). TokenHandler 401/503/200 bodies match PHP routes.php. Client posts /publish and /broadcast with Authorization: apikey, 5s timeout. fonoteka mounts with jwt.auth + throttle:ws-api (routes.go:86). Album channels collection:<id> only for kind=collection. TestTokenClaims, TestTokenHandler, TestClientRequests, parity token/subscribe routes and TestBroadcastGoldens (deleted + bulk vs PHP goldens) pass; created/updated goldens pending Phase 12 by design. |
| 4 | Namespace authorizer registry re-validates every subscribe; broadcastable model with bulk suppression emits exactly one summary event | ✓ VERIFIED | ProxyHandler calls svc.Registry().Get(namespace) and Authorize per request, no cache; constant-time secret compare; generic deny body. fonoteka registers collection and wishlist authorizers. WithoutBroadcasting[T] + Service.Emit; TestBulkEmitsOnce asserts exactly one publication (waitPublications fails on extras) and none on rollback. Broadcast callbacks now register .Before("gorm:commit_or_rollback_transaction") (deferred item 1 fixed). TestProxy, TestWsAuthorizer, TestSuppression, TestBroadcastTx pass. |
| 5 | Typesense sync scoped by collection_id behind a settings kill-switch, degrades gracefully without DB/config | ✓ VERIFIED | Album.ToSearchableArray refuses collection_id 0; settingsGate reads search_use_typesense per sync and treats read errors as off; syncOne returns early with no engine/api_key or no published DB. CR-01 is closed: Cabana writes and SaveAlbum use lagoon.Transaction, unmanaged GORM transactions are warned and skipped, and TestAlbumsAdminSearchUsesCommittedArtists, TestSaveAlbumDefersAfterCommitUntilArtistsSync, TestTransactionAfterCommit, and TestSyncAfterCommit prove committed-row/pivot timing and rollback silence. |
Plan must-have truths (supporting evidence)
70 truths carried forward from the initial verification. 67 are verified by code reading plus the named tests the gate runs (all pass, none skipped except the two declared Phase 12 goldens); 3 are verification: backstop (11-02 multi-process leader election, 11-03 delivery order, 11-07 real clients) and route to human verification as insufficient_spec. The sole failed truth is now closed by Plan 11-08:
Score: 5/5 roadmap success criteria verified; 67/70 plan truths verified (3 backstop checks routed to human).
| Plan | Truth | Status | Evidence |
|---|---|---|---|
| 11-05 | D-20: sync after commit; rolled-back write sends nothing | ✓ VERIFIED | Cabana CRUD create/update/delete/bulk-delete and relation Link/Unlink, plus fonoteka SaveAlbum, now use lagoon.Transaction. Foreign plain GORM transactions are refused. Focused behavioral tests prove one post-commit import with committed artist order and zero calls on rollback. |
| 11-01 | Outside a Lagoon-managed or implicit single-statement transaction, callbacks run immediately | ✓ VERIFIED (clarified) | Truly non-transactional handles still run immediately. A foreign gorm.TxCommitter is now explicitly detected, warned, and skipped because Lagoon cannot observe its commit. |
Required Artifacts
| Artifact | Status | Details |
|---|---|---|
modules/lagoon/queue_migrations.go, ondatabase.go, transaction.go |
✓ VERIFIED | Migrations wired from migrations.go; OnDatabase drained by Publish; Transaction buffers callbacks, validates exact parent ownership, and refuses unmanaged transactions |
modules/conga/* (conga, worker, client, commands, schedule, scheduler, job, record) |
✓ VERIFIED | Substantive, wired into serve (surf/serve.go:57), generated main and fonoteka main |
modules/pact/capabilities.go, modules/bonfire/call.go |
✓ VERIFIED | HasSchedule/Daily/DailyAt/Every; Catalog/Call used by scheduler |
modules/lighthouse/*, modules/lighthouse/centrifugo/* |
✓ VERIFIED | Registry, Mount, Broadcastable, suppression, token issuer, proxy, HTTP client, health command |
modules/flare/* |
✓ VERIFIED | RFC 8291 encryption, VAPID, commands; registered by fonoteka Commands() |
modules/beachcomber/*, beachcomber/typesense/* |
✓ VERIFIED | Wired through OnDatabase and fonoteka wireSearch; after-commit and rollback semantics are covered by TestSyncAfterCommit |
modules/tide/centrifugo.go, centrifugo_golden.go, cmd/summer/parity.go |
✓ VERIFIED | Loopback recorder, goldens, parity:broadcasts |
../fonoteka.go/plugins/golem15/fonoteka/{realtime,search,schedule,routes}.go, classes/ws/*, models/album_search.go |
✓ VERIFIED | Authorizers, Album binding, settings gate, daily prune entry, route mount |
../fonoteka.go/config/{queue,realtime,search,push}.yaml |
✓ VERIFIED | PHP defaults; push disabled |
../fonoteka.go/parity/fixtures/broadcasts/{deleted,bulk,created,updated}.yaml |
✓ VERIFIED | created/updated pending Phase 12 and never counted as passing |
scripts/check-phase11.sh |
✓ VERIFIED | Re-run by verifier: phase11 all passed (3m27s) |
Key Link Verification
| From | To | Via | Status |
|---|---|---|---|
| lagoon/migrations.go | queue_migrations.go | QueueMigrations( | ✓ WIRED |
| conga/conga.go | River | InsertTx on *sql.Tx | ✓ WIRED |
| surf/serve.go | conga/worker.go | conga.StartServeWorker | ✓ WIRED |
| internal/build/build.go | conga/commands.go, bonfire | conga.RuntimeCommands, bonfire.NewCatalog | ✓ WIRED |
| conga/worker.go | scheduler.go | cfg.PeriodicJobs | ✓ WIRED |
| fonoteka routes.go | lighthouse/route.go | lighthouse.Mount | ✓ WIRED |
| lighthouse/broadcast.go | conga | Enqueue on write tx in savepoint | ✓ WIRED |
| centrifugo/handlers.go | registry.go | Registry().Get per subscribe | ✓ WIRED |
| beachcomber/sync.go | lagoon/transaction.go | lagoon.AfterCommit | ✓ WIRED; managed transactions buffer until commit and unmanaged transactions are refused |
| cabana CRUD/relation writes | lagoon/transaction.go | lagoon.Transaction | ✓ WIRED; three CRUD and two relation write entry points use the managed transaction |
| fonoteka SaveAlbum | lagoon/transaction.go | lagoon.Transaction | ✓ WIRED; Album save and ordered artist pivot sync share the managed transaction |
| fonoteka plugin.go | search.go | wireSearch | ✓ WIRED |
Data-Flow Trace (Level 4)
| Artifact | Data | Source | Real data | Status |
|---|---|---|---|---|
| Album search document | artist_ids | reload inside syncOne after managed commit |
Reads the committed ordered pivots; assembled-router test compares the imported ids to a committed query and requires exactly one document | ✓ FLOWING |
Album updated broadcast payload |
album.artists | albumPayload Preload through the managed write transaction |
Cabana write timing is now commit-safe; delivery/payload compatibility remains covered by the existing broadcast tests and Phase 12 goldens | ✓ FLOWING |
Token info.name |
user name | SetUserLookup DB read |
Real query | ✓ FLOWING |
Behavioral Spot-Checks
| Behavior | Command | Result | Status |
|---|---|---|---|
| LISTEN pickup not poll latency | go test ./modules/conga/ -run '^TestListenPickupLatency$' -v -count=1 |
listen 3.0ms (30s poll); poll-only not picked up in 2s | ✓ PASS |
| Committed artist pivots reach Typesense once | TestAlbumsAdminSearchUsesCommittedArtists (recorded by check-phase11.sh --all) |
PASS — committed order equals the one imported document's artist_ids |
✓ PASS |
| Managed/unmanaged after-commit invariants | TestTransactionAfterCommit, TestTransactionEdges, TestSyncAfterCommit (recorded gate evidence) |
PASS — commit ordering, rollback silence, exact parent ownership, clean handles, unmanaged refusal | ✓ PASS |
| Full phase gate | bash scripts/check-phase11.sh --all |
Recorded final run: self-test, hygiene, go, postgres, named, evidence passed; phase11 all passed |
✓ PASS |
| Removal coverage | PHASE11_RC=RC-15 bash scripts/check-phase11.sh --removal |
Recorded final run: removing exact parent ownership makes the named Lagoon test fail as required; file restored | ✓ PASS |
| Pre-existing hello failure | go test ./examples/hello -run TestTypedItemRoute at HEAD and in a 718a35c worktree |
Same failure (surf: config http.body_limits.default_bytes is required) at both |
ℹ pre-existing, not a Phase 11 regression |
Probe Execution
Step 7c: no scripts/*/tests/probe-*.sh probes declared or present; the phase gate scripts/check-phase11.sh --all was run instead (above).
Requirements Coverage
| Requirement | Source Plan | Status | Evidence |
|---|---|---|---|
| JOBS-01 | 11-01, 11-07 | ✓ SATISFIED | SC-1 evidence. Note: the requirement text names riverpgxv5 for LISTEN; the implementation uses a pgxpool through NewWithPgxListener, which is what ROADMAP SC-1 specifies |
| CLI-04 | 11-02, 11-07 | ✓ SATISFIED (WR-02 warning) | SC-2 evidence |
| CLI-06 | 11-01, 11-07 | ✓ SATISFIED | queue:work in-process (serve) and foreground |
| RT-01 | 11-03, 11-04, 11-06, 11-07 | ✓ SATISFIED (live: human) | SC-3 evidence; hand-rolled client per the D-12/D-16 note |
| RT-02 | 11-03, 11-06, 11-07 | ✓ SATISFIED | SC-4 evidence; channel names are opaque collection:<id> as in PHP |
| RT-03 | 11-03, 11-06, 11-07 | ✓ SATISFIED | Bulk suppression + single Emit; rollback publishes nothing (broadcast jobs ride the write tx) |
| SRCH-01 | 11-05, 11-07, 11-08 | ✓ SATISFIED | Scoping, kill-switch and graceful degradation hold; CR-01 closure makes Cabana/SaveAlbum sync commit-safe and proves committed ordered pivots plus rollback silence |
No orphaned requirements: REQUIREMENTS.md maps exactly these seven IDs to Phase 11, and every one is claimed by a plan. The status table still says Gaps Found for JOBS-01/CLI-04/CLI-06/RT-01/RT-02/RT-03; that planning metadata is stale relative to this re-verification and should be refreshed by the orchestrator when it records the phase result.
Prohibitions
All prohibitions are verification: test and each has a named test run by the gate; high threats additionally have RC removal checks in 11-SECURITY-REVIEW.md. None flagged. The SRCH-01 prohibitions (no index without positive collection_id; no request while the switch is off or the key is empty; engine failure never fails the write) hold, including inside plain transactions.
Anti-Patterns and Review Findings
No TBD/FIXME/XXX markers were found in the Plan 11-08 implementation/test files. The final 11-REVIEW.md reviews all 13 gap-closure files and reports zero critical, warning, or info findings. Its resolved findings confirm exact parent-transaction ownership, fail-closed expected-skip enforcement, exact named-test coverage, and RC-15 removal coverage. No re-verification regression or new-scope blocker remains.
Advisory (New Scope, Unevidenced)
None.
Decision Coverage
All 20 trackable 11-CONTEXT.md decisions are honored by shipped artifacts (check.decision-coverage-verify: 20/20, non-blocking gate).
Test Quality Audit
No disabled requirement-linked tests or circular expected-value generators were found in the CR-01 closure tests. The strongest assertions are behavioral: the assembled admin route must commit ordered pivots and send exactly one matching import; rollback paths must make zero callbacks/engine calls; removal checks RC-14/RC-15 must turn the named Lagoon test red. The final phase gate rejects missing, skipped, zero-match, or unexpectedly passing named tests.
Human Verification Required
- Live Centrifugo v6 + unchanged Nuxt — change an album, confirm the event arrives with a Go-issued token.
- Live Typesense 26.0 upsert — enable the switch, edit an album's artists, query the collection, and confirm
artist_idsmatches the committed pivot order. - Real-browser Web Push through the flare VAPID driver.
- Multi-process scheduler leader election (11-02 backstop).
- Broadcast delivery order accepted as unordered (11-03 backstop).
Gaps Summary
No automated implementation gaps remain. Plan 11-08 closes CR-01 at every required level: substantive transaction ownership/refusal logic exists, all real Album write paths are wired to it, committed data flows to the Typesense import, and behavioral/removal tests fail when the protections are removed. The clean code review and final check-phase11.sh --all evidence show no regression.
Five pre-existing human checks remain: live Centrifugo/Nuxt integration, live Typesense, browser Web Push, multi-process River leader election, and the declared unordered broadcast-delivery backstop. Under the verifier decision tree these make the final status human_needed; they are not code gaps and do not reopen CR-01.
Verified: 2026-09-30T20:26:54Z Verifier: the agent (gsd-verifier)