Files
summercms/.planning/phases/11-jobs-realtime-and-search-infrastructure/11-VALIDATION.md
Jakub Zych a34c6ece18 docs(11-07): security review with removal checks and the validated test map
- 11-SECURITY-REVIEW.md: T-11-01..T-11-30 and T-11-SC with each plan's
  severity and disposition, mitigation, test and result; RC-01..RC-13
  removal checks for every high mitigated threat; the three defects fixed
  in 11-07
- 11-VALIDATION.md: task ids, plans and waves per row, commands run,
  status validated, nyquist_compliant and wave_0_complete true
2026-09-30 14:49:36 +02:00

96 lines
9.8 KiB
Markdown

---
phase: "11"
slug: "jobs-realtime-and-search-infrastructure"
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
status: validated
nyquist_compliant: true
wave_0_complete: true
created: "2026-09-29"
validated: "2026-09-30"
gate: "scripts/check-phase11.sh --all"
---
# Phase 11 — Validation Strategy
> Per-phase validation contract for feedback sampling during execution. Seeded from `11-RESEARCH.md` § Validation Architecture. Requirements: JOBS-01, CLI-04, CLI-06, RT-01, RT-02, RT-03, SRCH-01.
---
## Test Infrastructure
| Property | Value |
|----------|-------|
| **Framework** | Go `testing` + testify (assertions) + testcontainers-go v0.44.0 postgres module |
| **Config file** | none; package `TestMain` starts `postgres:16-alpine` with ICU pl-PL and is skipped under `-short` (pattern: `modules/lagoon/postgres_test.go`) |
| **Quick run command** | `go test -short ./modules/<touched package>/...` plus `go vet ./...` |
| **Full suite command** | `go vet ./... && go test ./...` in summercms.go, then `cd ../fonoteka.go && go vet ./... && go test ./...` |
| **Estimated runtime** | ~20 seconds for a `-short` package run; several minutes for the full testcontainers suite in both repos |
---
## Sampling Rate
- **After every task commit:** the quick `-short` command for the touched package, plus `go vet ./...`
- **After every plan wave:** the full suite in both repos (testcontainers)
- **Before `/gsd-verify-work`:** full suite green in summercms.go and fonoteka.go
- **Max feedback latency:** 60 seconds for the quick command
---
## Per-Task Verification Map
Task IDs are `<plan>-T<task>`. Every row's command was run on 2026-09-30 and passed; `scripts/check-phase11.sh --named` runs all of them by exact name and refuses a skip, a missing pass or "no tests to run". Framework commands run from `summercms.go`, fonoteka.go commands from `../fonoteka.go`.
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
| 11-01-T1, 11-07-T1 | 11-01, 11-07 | 1, 5 | JOBS-01 | T-11-13 | LISTEN pickup under 1 s with a 30 s poll; poll-only control misses | integration | `go test ./modules/conga -run '^TestListenPickupLatency$' -count=3` | ✅ `modules/conga/listen_test.go` | ✅ green |
| 11-01-T1, 11-07-T1 | 11-01, 11-07 | 1, 5 | JOBS-01 | T-11-12 | Row and River job share the caller's transaction; rollback leaves neither | integration | `go test ./modules/conga -run '^TestDispatchTransactional$' -count=1` | ✅ `modules/conga/listen_test.go` | ✅ green |
| 11-01-T2, 11-07-T1 | 11-01, 11-07 | 1, 5 | JOBS-01 | T-11-15, T-11-16 | Only the final attempt records ERROR; panics recovered; skip is COMPLETE with metadata; cancel leaves STOPPED | integration | `go test ./modules/conga -run '^(TestOutcome.*\|TestCancel.*\|TestStopJobFromWorker\|TestManagerPHPSemantics)$' -count=1` | ✅ `modules/conga/worker_test.go`, `manager_test.go` | ✅ green |
| 11-01-T2, 11-07-T1 | 11-01, 11-07 | 1, 5 | JOBS-01 | T-11-14 | Only available, scheduled and retryable jobs of one queue are cleared | integration | `go test ./modules/conga -run '^(TestQueueClear\|TestClearQueueStatesAndBatches)$' -count=1` | ✅ `modules/conga/commands_test.go` | ✅ green |
| 11-01-T2, 11-07-T1 | 11-01, 11-07 | 1, 5 | CLI-06 | — | queue:work filters queues and names the known ones; serve honours queue.work_in_serve | integration | `go test ./modules/conga -run '^TestQueueWork$' -count=1` | ✅ `modules/conga/commands_test.go` | ✅ green |
| 11-02-T1, 11-02-T2, 11-07-T1 | 11-02, 11-07 | 2, 5 | CLI-04 | T-11-09, T-11-17, T-11-18 | Forged or mismatched scheduled jobs are skipped; runs unique per period; invalid cadences refused | unit + integration | `go test ./modules/conga -run '^TestSchedule.*$' -count=1` ; `go test ./modules/bonfire -run '^(TestCall\|TestCallEdges)$' -count=1` ; `go test ./plugins/golem15/fonoteka -run '^TestFonotekaScheduleSkipsUnregisteredPrune$' -count=1` | ✅ `modules/conga/schedule_test.go`, `modules/bonfire/call_test.go`, fonoteka `schedule_test.go` | ✅ green |
| 11-03-T1, 11-07-T2 | 11-03, 11-07 | 2, 5 | RT-01 | T-11-04 | 503 when the secret is empty (after the user check); 401 without a principal or user; exact claim set | unit | `go test ./modules/lighthouse/centrifugo -run '^TestToken.*$' -count=1 -race` | ✅ `modules/lighthouse/centrifugo/token_test.go` | ✅ green |
| 11-03-T1, 11-04-T2, 11-07-T2 | 11-03, 11-04, 11-07 | 2, 4, 5 | RT-01 | T-11-06, T-11-22 | API key never logged or in errors; no request without a key; push only to allow-listed https hosts | unit | `go test ./modules/lighthouse/centrifugo -run '^(TestClient.*\|TestHealthCommand)$' -count=1` ; `go test ./modules/flare -count=1 -race` | ✅ `client_test.go`, `commands_test.go`, `modules/flare/*_test.go` | ✅ green |
| 11-03-T2, 11-07-T2 | 11-03, 11-07 | 2, 5 | RT-02 | T-11-01, T-11-03 | Missing, wrong or unconfigured proxy secret denies; deny reason logged, not returned | unit | `go test ./modules/lighthouse/centrifugo -run '^TestProxy.*$' -count=1 -race` | ✅ `modules/lighthouse/centrifugo/proxy_test.go` | ✅ green |
| 11-03-T2, 11-07-T2 | 11-03, 11-07 | 2, 5 | RT-02 | T-11-02 | Non-member denied on every subscribe; wishlist id under the collection namespace denied | integration | `go test ./plugins/golem15/fonoteka -run '^(TestWsAuthorizer\|TestRealtimeSubscribeProxy)$' -count=1 -race` | ✅ fonoteka `ws_authorizer_test.go` | ✅ green |
| 11-03-T3, 11-07-T2 | 11-03, 11-07 | 2, 5 | RT-03 | T-11-05, T-11-20 | Broadcast enqueued in the write transaction (single statements included); suppression per type; one bulk event | integration | `go test ./modules/lighthouse -run '^(TestBroadcastTx\|TestSuppression\|TestBulkEmitsOnce\|TestBroadcastEdges\|TestBroadcastSwallowedReadFailure)$' -count=1 -race` ; `go test ./plugins/golem15/fonoteka -run '^TestAlbumBroadcastBinding$' -count=1` | ✅ `modules/lighthouse/broadcast_test.go`, fonoteka `album_realtime_test.go` | ✅ green |
| 11-01-T3, 11-07-T1 | 11-01, 11-07 | 1, 5 | RT-03 | — | GORM hooks registered through OnDatabase in the production boot order; after-commit handle has a clean statement | integration | `go test ./modules/lagoon -run '^(TestOnDatabaseAfterActivate\|TestTransactionAfterCommit\|TestQueueMigrationsUpDown)$' -count=1` | ✅ `modules/lagoon/*_test.go` | ✅ green |
| 11-06-T1, 11-06-T2 | 11-06 | 3 | RT-03 | T-11-28, T-11-30 | deleted and bulk publications equal the PHP goldens; created and updated skip until Phase 12 asserts them | parity | `go test ./parity -run '^TestBroadcastGoldens$' -count=1 -v` | ✅ fonoteka `parity/broadcast_goldens_test.go` | ✅ green |
| 11-05-T1, 11-05-T2, 11-07-T2 | 11-05, 11-07 | 3, 5 | SRCH-01 | T-11-07, T-11-25, T-11-26, T-11-27 | Documents always carry a positive `collection_id`; nothing sent while the kill-switch is off or the key is empty; failures never touch the write | integration | `go test ./modules/beachcomber/... -run '^TestSync.*$' -count=1 -v` ; `go test ./modules/beachcomber/typesense -run '^TestEngineWire$' -count=1` ; `go test ./plugins/golem15/fonoteka -run '^(TestAlbumSearchable\|TestAlbumSearchSmoke)$' -count=1` | ✅ `modules/beachcomber/sync_test.go`, `typesense/engine_test.go`, fonoteka `album_search_test.go` | ✅ green |
| 11-07-T3 | 11-07 | 5 | all seven | T-11-21, T-11-SC | The phase gate fails closed and refuses excluded client libraries and an unpinned River | gate | `scripts/check-phase11.sh --self-test` ; `scripts/check-phase11.sh --all` | ✅ `scripts/check-phase11.sh` | ✅ green |
*Status: ✅ green · ❌ red · ⚠️ flaky*
---
## Wave 0 Requirements
- [x] `modules/conga/postgres_test.go` — TestMain with testcontainers, exposing both `*sql.DB` and the DSN (11-01); lighthouse and beachcomber got the same harness in 11-07
- [x] A fake Centrifugo `httptest` helper (records method, path, headers, body): `tide.CentrifugoRecorder` (11-06) and the test-local fakes in `modules/lighthouse/centrifugo/client_test.go` and fonoteka `realtime_smoke_test.go`
- [x] A fake Typesense `httptest` helper: `modules/beachcomber/typesense/engine_test.go` and fonoteka `search_smoke_test.go`
- [x] fonoteka.go `parity/schema_diff_test.go` + `parity/migrate_test.go` allow-list updates for the River and `summer_jobs` tables (11-01)
- [x] `go get github.com/riverqueue/river@v0.47.0 github.com/riverqueue/river/riverdriver/riverdatabasesql@v0.47.0 github.com/riverqueue/river/rivertype@v0.47.0` (11-01; the hygiene stage pins v0.47.0)
---
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|----------|-------------|------------|-------------------|
| Nuxt client connects to the real Centrifugo with a Go-issued token and receives an album event | RT-01, RT-03 | Needs the running Nuxt app and the Centrifugo server | Start Centrifugo v6 with the production secret layout, run `summer serve`, log in via the Nuxt app, change an album, and watch the event arrive |
| Real Typesense receives the upsert and the Nuxt search pre-filter still works | SRCH-01 | Needs a Typesense server | Start `typesense/typesense:26.0`, enable the kill-switch, save an album, query the collection |
---
## Validation Sign-Off
- [x] All tasks have `<automated>` verify or Wave 0 dependencies
- [x] Sampling continuity: no 3 consecutive tasks without automated verify
- [x] Wave 0 covers all MISSING references
- [x] No watch-mode flags
- [x] Feedback latency < 60s (the `-short` package runs; the testcontainers suites take minutes and run per wave and in the gate)
- [x] `nyquist_compliant: true` set in frontmatter
**Approval:** validated 2026-09-30 by plan 11-07 (`scripts/check-phase11.sh --all` prints "phase11 all passed"). The two manual-only rows above are collected at /gsd-verify-work.