Files
summercms/.planning/phases/03-first-vertical-slice-genres-end-to-end/03-VALIDATION.md
Jakub Zych 13887ee0b1 docs(03-04): record Phase 3 validation and security evidence
Map T-03-01 through T-03-SC to passing tests, record the check-phase3.sh
gate, and mark nyquist_compliant after that gate exited 0.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 20:39:53 +02:00

9.4 KiB

phase, slug, status, nyquist_compliant, wave_0_complete, created, verified
phase slug status nyquist_compliant wave_0_complete created verified
03 first-vertical-slice-genres-end-to-end complete true true 2026-09-17 2026-09-17

Phase 3 — Validation Strategy

Test Infrastructure

Property Value
Framework Go 1.27 testing, httptest; existing ../fonoteka.go/parity testcontainers Postgres suite; framework lagoon tests now share the same STACK-named testcontainers modules
Config Root go.mod plus app ../fonoteka.go/go.mod; existing go.work only covers the framework and hello example
Quick run go vet ./... && go test ./... in each of summercms.go and ../fonoteka.go
Full suite bash scripts/check-phase3.sh — root and app vet/test/race, focused real genres parity, genre security/tenant tests, corpus audit, Phase 2 Go/CLI harness regression (TestParitySynthetic + CLI record/replay smoke)
Repeatable gate scripts/check-phase3.sh; Phase 2 PHP self-replay remains scripts/check-phase2.sh --fresh-php
Measured runtime Successful check-phase3.sh run: ~21s after cache warmup (2026-09-17). Focused TestParityCorpus$ 5.115s; focused genre security 5.003s; TestParitySynthetic 4.951s; CLI smoke recorded + replay matched.

Sampling Rate

  • After every implementation task commit: root go vet ./... && go test ./...; after app files exist, run the same commands in ../fonoteka.go.
  • After each plan wave: run the focused Postgres route/genre test and the ported parity subtest in the app, plus the quick checks.
  • Before Phase 3 verification: bash scripts/check-phase3.sh. Docker unavailable is a failed gate; testing.Short remains a fast local option only.

Per-Task Verification Map

The user confirmed four plans on 2026-09-17. Every task has an automated <verify> command and an observable acceptance criterion. Rows below are filled from plan summaries plus the 03-04 gate.

Task ID Wave Requirement Threat Automated evidence Status
03-01-01 1 DATA-01, DATA-02 T-03-01, T-03-SC Root/app vet/test; Postgres 16 ICU migration smoke, 15 seeds, separate history Pass — 03-01 SUMMARY (d0d8450 / 55f110e)
03-01-02 1 HTTP-01, HTTP-02, QA-04 T-03-02, T-03-03 Root/app vet/test; persisted-user JWT request reaches real genre row Pass — 03-01 SUMMARY (4ee4c4a / d7915a8); 401/423 never reach handler
03-02-01 2 QA-04 T-03-04 Root/app vet/test; focused TestGenre nonzero/foreign/editor counts Pass — 03-02 SUMMARY (fc7d581)
03-02-02 2 QA-04 T-03-05 Root/app vet/test; focused TestGenre non_empty, 422, 15-name order Pass — 03-02 SUMMARY (56be82f / f5adabc)
03-03-01 3 DATA-02, HTTP-01 T-03-01, T-03-07 Root/app vet/test; rollback isolation and hello typed route Pass — 03-03 SUMMARY (8e3bf26 / 91e3df4)
03-03-02 3 QA-04 T-03-06 Root/app vet/test; TestParityCorpus one real pass and 153 pending Pass — 03-03 SUMMARY (0cc1a4d); fixture unmodified
03-04-01 4 DATA-01, DATA-02, HTTP-01, HTTP-02 T-03-01 to T-03-03 Root vet/test/race and app vet/test; adversarial auth/routing tests Pass — 92255a46ed039f605231ad71eed243436c4a1fbc; re-run green in 03-04 gate
03-04-02 4 QA-04, HTTP-02 T-03-04 to T-03-07 bash scripts/check-phase3.sh; app test/race, Go/CLI harness regression, security review Pass — 2026-09-17, phase3 check passed; app c6615683cbd5bb10cc141bee69f1938c46450ffe; script c2dfa7b62f3ba993dbfc50313f437c367770b3cb; gate follow-up fda61f0c1519e0756e966a64b868a8f92085fcb7
Capability Requirement Threat Test and evidence
Shared pgx/GORM connection and plugin migrations DATA-01, DATA-02 T-03-01 schema drift TestWrongICULocaleFailsOpen, TestTwoPluginMigrationSetsIsolated, TestNoAutoMigrate, app TestMigrateSeedsCanonicalGenres, TestRollbackLastIsolatesFonoteka. ICU pl-PL required before GORM. Two plugin histories survive independent up/down/rollback. No AutoMigrate.
Named route and middleware pipeline HTTP-01, HTTP-02 T-03-02 auth bypass TestAssembleMissingMiddlewareFailsBoot, TestUnauthenticatedNamedGuardDoesNotReachHandler, TestPipelineOrderRecoverCORSLocaleAuthPasswordOrgRateHandler, TestCORSPreflightBypassesNamedAuth, TestTypedIDRouteReturns404. Missing names fail boot with plugin+name; unauthenticated GET never sets handler Cache-Control.
JWT guard HTTP-02, QA-04 T-03-03 token forgery TestVerifyRejectsBadTokens, TestVerifyRejectsAlgNoneEmptySecretAndAbsentExp, TestVerifyAndMiddlewareOmitTokenAndSecret, app TestGenreSecurityBoundaries, TestEmptyJWTSecretFailsBoot. HS256 pinned; empty secret fails Boot; 401 bodies omit token/secret.
Active collection and genre aggregate QA-04 T-03-04 cross-tenant data leak TestGenreCountsScopedToActiveCollection, TestGenreSecurityBoundaries/alice-counts-ignore-foreign, /bob-counts-ignore-alice. Foreign albums stay at count 0. Invalid stored context falls back to lowest-ID accessible collection.
Polish order QA-04 T-03-05 sort drift TestOrderClauseAllowList; integration 15-name order under ICU pl-PL; TestGenreQueryFailsOnWrongLocale. No COLLATE. Invalid non_empty returns PHP 422 envelope.
Recorded parity QA-04 T-03-06 false green Unmodified fixtures/routes/get_genres_jwt.yaml; TestParityCorpus 154 recorded, 1 passing, 153 pending, 0 failing, 0 unrecorded; TestParityContract/honest-counts and ported-mutated-response.

Wave 0 Requirements

  • Existing ../fonoteka.go/parity/TestMain starts a Postgres container and the recorded fixture exists. Plan 01 added real app boot and a focused route smoke test before Plan 03 moved newTarget.
  • Test helper for applying both plugin migration sets and seeding Alice's active collection lives in the app test package (genres_seed_test.go), not tide.
  • No additional test framework or watch mode was required. testcontainers-go v0.44.0 is the STACK-named dependency used by framework lagoon isolation tests.

Manual-Only Verifications

Behavior Requirement Why manual Test instructions Result
Database provisioning QA-04 Production Postgres lies outside the test suite Use a fresh database created with TEMPLATE template0, LOCALE_PROVIDER icu, ICU_LOCALE 'pl-PL', and UTF8 encoding; run the startup locale check before migrations. Documented in ../fonoteka.go/README.md. Testcontainers databases are created the same way in lagoon/postgres_test.go and app TestMain. Wrong-locale Open fails.

Phase 3 gate evidence (03-04-02)

Commands (no credentials):

go vet ./... && go test ./... && go test -race ./...          # summercms.go
(cd ../fonoteka.go && go vet ./... && go test ./... && go test -race ./...)
(cd ../fonoteka.go && go test ./parity -count=1 -run 'TestParityCorpus$')
(cd ../fonoteka.go && go test ./parity -count=1 -run 'TestGenreSecurityBoundaries|TestGenreCountsScopedToActiveCollection|TestGenreQueryFailsOnWrongLocale')
(cd ../fonoteka.go && go run ./parity/check_corpus.go \
  --manifest ../fonoteka.go/parity/manifest.yaml \
  --routes /media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/routes.php \
  --require-recorded --require-clients --check-secrets)
(cd ../fonoteka.go && go test ./parity -run TestParitySynthetic -count=1)
bash scripts/check-phase3.sh

Observed (2026-09-17, bash scripts/check-phase3.sh exit 0):

  • Docker present; missing Docker is an explicit refuse, not a skip.
  • Root vet/test/race: all packages ok (cached after first run).
  • App vet/test/race: ok git.golem15.com/golem15/fonoteka/parity.
  • Focused genres parity (TestParityCorpus$): ok in 5.115s — one real ported pass.
  • Focused genre security/tenant: ok in 5.003s.
  • Corpus audit: recorded 154/154.
  • TestParitySynthetic: ok in 4.951s.
  • CLI smoke: parity:record wrote a loopback fixture; parity:replay printed replay matched.
  • Script printed phase3 check passed.

Corpus honesty (app TestParityContract / TestParityCorpus): 154 recorded, 1 passing, 153 pending, 0 failing, 0 unrecorded. A deliberate album_count mutation fails ReplayFlow. The recorded PHP fixture is unchanged (Bearer {{jwt:alice}}, "album_count":0).

PHP --fresh-php (Phase 2 sign-off, not this gate): as of 2026-09-17 scripts/check-phase2.sh --fresh-php reports 150/154 on four wishlist album_count routes (expected 1, live 2). Phase 3 did not edit PHP, tide, or those fixtures. The Phase 3 script therefore re-runs the Phase 2 Go/CLI harness pieces and leaves PHP self-replay at check-phase2.sh.

Prior implementation commits (03-01 through 03-03, plus 03-04 Task 1) ran go vet ./... and go test ./... green in their summaries.

Security review: 03-SECURITY-REVIEW.md maps T-03-01 through T-03-07 and T-03-SC to passing tests; threats_open: 0; no open high-severity JWT or cross-tenant issue.

Validation Sign-Off

  • Plan IDs and per-task commands filled after plan-count checkpoint.
  • Every task has automated feedback; no three consecutive tasks without it.
  • Security threat model appears in each plan and the final plan tests its high severity mitigations.
  • Root and app quick checks, race checks, Postgres integration, and recorded parity are green.
  • Mark nyquist_compliant: true only after implementation evidence exists.

Approval: Phase 3 gate evidence recorded 2026-09-17. nyquist_compliant: true.