docs(phase-11): add validation strategy
This commit is contained in:
Jakub Zych
@@ -0,0 +1,92 @@
|
||||
---
|
||||
phase: "11"
|
||||
slug: "jobs-realtime-and-search-infrastructure"
|
||||
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
|
||||
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
wave_0_complete: false
|
||||
created: "2026-09-29"
|
||||
---
|
||||
|
||||
# Phase 11 — Validation Strategy
|
||||
|
||||
> Per-phase validation contract for feedback sampling during execution. Seeded from `11-RESEARCH.md` § Validation Architecture. Requirements: JOBS-01, CLI-04, CLI-06, RT-01, RT-02, RT-03, SRCH-01.
|
||||
|
||||
---
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Framework** | Go `testing` + testify (assertions) + testcontainers-go v0.44.0 postgres module |
|
||||
| **Config file** | none; package `TestMain` starts `postgres:16-alpine` with ICU pl-PL and is skipped under `-short` (pattern: `modules/lagoon/postgres_test.go`) |
|
||||
| **Quick run command** | `go test -short ./modules/<touched package>/...` plus `go vet ./...` |
|
||||
| **Full suite command** | `go vet ./... && go test ./...` in summercms.go, then `cd ../fonoteka.go && go vet ./... && go test ./...` |
|
||||
| **Estimated runtime** | ~20 seconds for a `-short` package run; several minutes for the full testcontainers suite in both repos |
|
||||
|
||||
---
|
||||
|
||||
## Sampling Rate
|
||||
|
||||
- **After every task commit:** the quick `-short` command for the touched package, plus `go vet ./...`
|
||||
- **After every plan wave:** the full suite in both repos (testcontainers)
|
||||
- **Before `/gsd-verify-work`:** full suite green in summercms.go and fonoteka.go
|
||||
- **Max feedback latency:** 60 seconds for the quick command
|
||||
|
||||
---
|
||||
|
||||
## Per-Task Verification Map
|
||||
|
||||
Task IDs are filled in once the plans exist; rows are keyed by behaviour until then.
|
||||
|
||||
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|
||||
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
|
||||
| TBD | TBD | TBD | JOBS-01 | — | N/A | integration | `go test ./modules/conga -run TestListenPickupLatency -count=1` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | JOBS-01 | — | N/A | integration | `go test ./modules/conga -run TestDispatchTransactional` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | JOBS-01 | — | N/A | integration | `go test ./modules/conga -run 'TestOutcome|TestCancel'` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | JOBS-01 | — | N/A | integration | `go test ./modules/conga -run TestQueueClear` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | CLI-06 | — | N/A | integration | `go test ./modules/conga -run TestQueueWork` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | CLI-04 | — | N/A | unit + integration | `go test ./modules/conga -run TestSchedule` ; `go test ./modules/bonfire -run TestCall` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | RT-01 | T-11-xx | 503 when the secret is empty; 401 without a valid JWT | unit | `go test ./modules/lighthouse/centrifugo -run TestToken` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | RT-01 | — | API key never logged; no request when the key is empty | unit | `go test ./modules/lighthouse/centrifugo -run TestClient` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | RT-02 | T-11-01 | Missing/wrong proxy secret denies; deny reason logged, not returned | unit | `go test ./modules/lighthouse/centrifugo -run TestProxy` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | RT-02 | T-11-xx | Non-member denied on every subscribe | integration | `cd ../fonoteka.go && go test ./plugins/golem15/fonoteka -run TestWsAuthorizer` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | RT-03 | — | N/A | integration | `go test ./modules/lighthouse -run 'TestBroadcastTx|TestSuppression|TestBulkEmitsOnce'` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | RT-03 | — | N/A | integration | `go test ./modules/lagoon -run TestOnDatabaseAfterActivate` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | RT-03 | — | N/A | parity | `cd ../fonoteka.go && go test ./parity -run TestBroadcastGoldens` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | SRCH-01 | — | Documents always carry a positive `collection_id` | integration | `go test ./modules/beachcomber/... -run TestSync` ; `cd ../fonoteka.go && go test ./plugins/golem15/fonoteka -run TestAlbumSearchable` | ❌ W0 | ⬜ pending |
|
||||
|
||||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||||
|
||||
---
|
||||
|
||||
## Wave 0 Requirements
|
||||
|
||||
- [ ] `modules/conga/postgres_test.go` — TestMain with testcontainers, exposing both `*sql.DB` and the DSN (the listener pool needs the DSN)
|
||||
- [ ] A fake Centrifugo `httptest` helper (records method, path, headers, body), shared by lighthouse tests and tide
|
||||
- [ ] A fake Typesense `httptest` helper
|
||||
- [ ] fonoteka.go `parity/schema_diff_test.go` + `parity/migrate_test.go` allow-list updates for the River and `summer_jobs` tables
|
||||
- [ ] `go get github.com/riverqueue/river@v0.47.0 github.com/riverqueue/river/riverdriver/riverdatabasesql@v0.47.0 github.com/riverqueue/river/rivertype@v0.47.0`
|
||||
|
||||
---
|
||||
|
||||
## Manual-Only Verifications
|
||||
|
||||
| Behavior | Requirement | Why Manual | Test Instructions |
|
||||
|----------|-------------|------------|-------------------|
|
||||
| Nuxt client connects to the real Centrifugo with a Go-issued token and receives an album event | RT-01, RT-03 | Needs the running Nuxt app and the Centrifugo server | Start Centrifugo v6 with the production secret layout, run `summer serve`, log in via the Nuxt app, change an album, and watch the event arrive |
|
||||
| Real Typesense receives the upsert and the Nuxt search pre-filter still works | SRCH-01 | Needs a Typesense server | Start `typesense/typesense:26.0`, enable the kill-switch, save an album, query the collection |
|
||||
|
||||
---
|
||||
|
||||
## Validation Sign-Off
|
||||
|
||||
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
|
||||
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
|
||||
- [ ] Wave 0 covers all MISSING references
|
||||
- [ ] No watch-mode flags
|
||||
- [ ] Feedback latency < 60s
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter
|
||||
|
||||
**Approval:** pending
|
||||
Reference in New Issue
Block a user