docs(phase-11): add validation strategy

This commit is contained in:
Jakub Zych
2026-09-29 12:32:56 +02:00
parent c82950c2a6
commit 9dabc6c5a1

View File

@@ -0,0 +1,92 @@
---
phase: "11"
slug: "jobs-realtime-and-search-infrastructure"
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
status: draft
nyquist_compliant: false
wave_0_complete: false
created: "2026-09-29"
---
# Phase 11 — Validation Strategy
> Per-phase validation contract for feedback sampling during execution. Seeded from `11-RESEARCH.md` § Validation Architecture. Requirements: JOBS-01, CLI-04, CLI-06, RT-01, RT-02, RT-03, SRCH-01.
---
## Test Infrastructure
| Property | Value |
|----------|-------|
| **Framework** | Go `testing` + testify (assertions) + testcontainers-go v0.44.0 postgres module |
| **Config file** | none; package `TestMain` starts `postgres:16-alpine` with ICU pl-PL and is skipped under `-short` (pattern: `modules/lagoon/postgres_test.go`) |
| **Quick run command** | `go test -short ./modules/<touched package>/...` plus `go vet ./...` |
| **Full suite command** | `go vet ./... && go test ./...` in summercms.go, then `cd ../fonoteka.go && go vet ./... && go test ./...` |
| **Estimated runtime** | ~20 seconds for a `-short` package run; several minutes for the full testcontainers suite in both repos |
---
## Sampling Rate
- **After every task commit:** the quick `-short` command for the touched package, plus `go vet ./...`
- **After every plan wave:** the full suite in both repos (testcontainers)
- **Before `/gsd-verify-work`:** full suite green in summercms.go and fonoteka.go
- **Max feedback latency:** 60 seconds for the quick command
---
## Per-Task Verification Map
Task IDs are filled in once the plans exist; rows are keyed by behaviour until then.
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
| TBD | TBD | TBD | JOBS-01 | — | N/A | integration | `go test ./modules/conga -run TestListenPickupLatency -count=1` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | JOBS-01 | — | N/A | integration | `go test ./modules/conga -run TestDispatchTransactional` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | JOBS-01 | — | N/A | integration | `go test ./modules/conga -run 'TestOutcome|TestCancel'` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | JOBS-01 | — | N/A | integration | `go test ./modules/conga -run TestQueueClear` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | CLI-06 | — | N/A | integration | `go test ./modules/conga -run TestQueueWork` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | CLI-04 | — | N/A | unit + integration | `go test ./modules/conga -run TestSchedule` ; `go test ./modules/bonfire -run TestCall` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | RT-01 | T-11-xx | 503 when the secret is empty; 401 without a valid JWT | unit | `go test ./modules/lighthouse/centrifugo -run TestToken` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | RT-01 | — | API key never logged; no request when the key is empty | unit | `go test ./modules/lighthouse/centrifugo -run TestClient` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | RT-02 | T-11-01 | Missing/wrong proxy secret denies; deny reason logged, not returned | unit | `go test ./modules/lighthouse/centrifugo -run TestProxy` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | RT-02 | T-11-xx | Non-member denied on every subscribe | integration | `cd ../fonoteka.go && go test ./plugins/golem15/fonoteka -run TestWsAuthorizer` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | RT-03 | — | N/A | integration | `go test ./modules/lighthouse -run 'TestBroadcastTx|TestSuppression|TestBulkEmitsOnce'` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | RT-03 | — | N/A | integration | `go test ./modules/lagoon -run TestOnDatabaseAfterActivate` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | RT-03 | — | N/A | parity | `cd ../fonoteka.go && go test ./parity -run TestBroadcastGoldens` | ❌ W0 | ⬜ pending |
| TBD | TBD | TBD | SRCH-01 | — | Documents always carry a positive `collection_id` | integration | `go test ./modules/beachcomber/... -run TestSync` ; `cd ../fonoteka.go && go test ./plugins/golem15/fonoteka -run TestAlbumSearchable` | ❌ W0 | ⬜ pending |
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
---
## Wave 0 Requirements
- [ ] `modules/conga/postgres_test.go` — TestMain with testcontainers, exposing both `*sql.DB` and the DSN (the listener pool needs the DSN)
- [ ] A fake Centrifugo `httptest` helper (records method, path, headers, body), shared by lighthouse tests and tide
- [ ] A fake Typesense `httptest` helper
- [ ] fonoteka.go `parity/schema_diff_test.go` + `parity/migrate_test.go` allow-list updates for the River and `summer_jobs` tables
- [ ] `go get github.com/riverqueue/river@v0.47.0 github.com/riverqueue/river/riverdriver/riverdatabasesql@v0.47.0 github.com/riverqueue/river/rivertype@v0.47.0`
---
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|----------|-------------|------------|-------------------|
| Nuxt client connects to the real Centrifugo with a Go-issued token and receives an album event | RT-01, RT-03 | Needs the running Nuxt app and the Centrifugo server | Start Centrifugo v6 with the production secret layout, run `summer serve`, log in via the Nuxt app, change an album, and watch the event arrive |
| Real Typesense receives the upsert and the Nuxt search pre-filter still works | SRCH-01 | Needs a Typesense server | Start `typesense/typesense:26.0`, enable the kill-switch, save an album, query the collection |
---
## Validation Sign-Off
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
- [ ] Wave 0 covers all MISSING references
- [ ] No watch-mode flags
- [ ] Feedback latency < 60s
- [ ] `nyquist_compliant: true` set in frontmatter
**Approval:** pending