golem15/summercms
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(02-05): record measured Phase 2 gate evidence

Browse Source 
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Jakub Zych
2026-09-17 14:15:13 +02:00
parent 066c3d3ab7
commit 84a5f007e3
1 changed files with 67 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

102
.planning/phases/02-api-parity-harness-bootstrap/02-VALIDATION.md
View File

@@ -1,10 +1,11 @@
---
phase: 02
slug: api-parity-harness-bootstrap
status: draft
nyquist_compliant: false
wave_0_complete: false
status: complete
nyquist_compliant: true
wave_0_complete: true
created: 2026-09-16
verified: 2026-09-17
---
# Phase 2 — Validation Strategy
@@ -15,55 +16,86 @@ created: 2026-09-16
|---|---|
| Framework | Go 1.27 `testing`, `httptest`; Testcontainers Postgres for app integration |
| Config file | Root `go.mod`; app `../fonoteka.go/go.mod`; no separate test config |
| Quick run | `go vet ./... && go test ./...` in root, then the same from `../fonoteka.go` once code exists |
| Full suite | Quick run plus `go test -race ./...` in both modules, CLI synthetic smoke, PHP self-replay on isolated local DB, route coverage audit |
| Estimated runtime | Measure during execution; Testcontainers and PHP checks are slower than package tests |
| Quick run | `go vet ./... && go test ./...` in root, then the same from `../fonoteka.go` |
| Full suite | `bash scripts/check-phase2.sh --fresh-php` (root and app vet/test/race, TestParitySynthetic Postgres, corpus audit, CLI smoke, disposable MariaDB PHP self-replay) |
| Repeatable gate | `scripts/check-phase2.sh --fresh-php`; Phase 1 gate preserved at `scripts/check-phase1.sh` |
| Measured runtime | Successful `--fresh-php` run: **116s** (2026-09-17). Root vet/test cached; app `go test -race ./parity` 6.912s; `TestParitySynthetic` 4.199s; PHP self-replay (seed + 154 routes + three client flows) after MariaDB pull-less start. First attempt failed MariaDB readiness (urlsafe password/`-p` flag clash) and was fixed before this run. |
## Sampling Rate
- After every implementation task commit: root `go vet ./... && go test ./...`; once app files exist, also run the app module checks.
- After each plan wave: run the above and the slice's CLI smoke against a local `httptest.Server` or isolated PHP server as applicable.
- Before `$gsd-verify-work`: run both modules' vet, test and race suites; run testcontainers Postgres integration; run PHP self-replay and inspect the 154-route coverage report.
- Keep quick checks in seconds after build cache warmup. Measure actual runtime; no arbitrary latency promise is set.
- Before `$gsd-verify-work`: `bash scripts/check-phase2.sh --fresh-php`.
- Keep quick checks in seconds after build cache warmup. Measured actual runtime above.
## Per-Task Verification Map
| Task ID | Wave | Requirement | Threat Ref | Test type | Automated evidence | Status |
|---|---:|---|---|---|---|---|
| 02-01-01 | 1 | QA-01, QA-02, QA-03 | T-02-01, T-02-SC | CLI integration | Root vet/test and `TestParityRoundTrip`/`TestParityCommands`; red state observed only before green commit | Pending |
| 02-01-02 | 1 | QA-01, QA-02, QA-03 | T-02-01 | unit + CLI | Root vet/test; malformed YAML, bounded bodies and byte/JSON diffs | Pending |
| 02-02-01 | 2 | QA-01, QA-03 | T-02-01, T-02-02 | proxy integration | `TestProxy`, `TestParityCommands`; named session, fixed upstream, safe flush | Pending |
| 02-02-02 | 2 | QA-01, QA-02, QA-03 | T-02-02, T-02-03 | unit + proxy | `TestCapture`, `TestScrub`, `TestNormalize`, `TestDiff`, `TestHeaders`, `TestFlow` | Pending |
| 02-02-03 | 2 | QA-01, QA-02, QA-03 | T-02-03, T-02-04 | CLI + manifest | `TestManifest`, `TestCoverage`; 16-route resume across two batches | Pending |
| 02-03-01 | 3 | QA-01, QA-03 | T-02-02, T-02-04, T-02-05 | PHP capture + audit | `check_corpus.go --allow-incomplete` against source; first PHP fixture and seed self-replay; both modules vet/test | Pending |
| 02-03-02 | 3 | QA-01, QA-03 | T-02-02, T-02-04, T-02-05 | batched PHP capture | Per-batch incomplete audit and PHP replay, then strict `--require-recorded` reports 154/154; both modules vet/test | Pending |
| 02-03-03 | 3 | QA-01, QA-03 | T-02-02, T-02-05 | real client capture | `capture_clients.mjs --check-deps` and `--capture`; corpus requires client flows and secret scan | Pending |
| 02-04-01 | 4 | QA-02, QA-03 | T-02-06, T-02-SC | Postgres integration | `TestParitySynthetic` starts testcontainers Postgres; both modules vet/test | Pending |
| 02-04-02 | 4 | QA-02, QA-03 | T-02-04, T-02-06 | app integration | `TestParityCorpus` shows 154 recorded/pending and zero false Go passes | Pending |
| 02-05-01 | 5 | QA-02, QA-03 | T-02-03 | contract unit | `TestFlowContract`, `TestDiffContract`, `TestManifestContract` | Pending |
| 02-05-02 | 5 | QA-01, QA-02, QA-03 | T-02-01, T-02-02, T-02-04, T-02-06 | security + app integration | `TestProxySecurity`, `TestParityCommandContract`, `TestParityContract` | Pending |
| 02-05-03 | 5 | QA-01, QA-02, QA-03 | T-02-01 to T-02-06 | phase gate | `bash scripts/check-phase2.sh --fresh-php` runs both modules' vet/test/race, Postgres, corpus audit and disposable PHP self-replay | Pending |
| 02-01-01 | 1 | QA-01, QA-02, QA-03 | T-02-01, T-02-SC | CLI integration | Root vet/test and `TestParityRoundTrip`/`TestParityCommands`; red state observed only before green commit | Pass — 02-01 SUMMARY; re-run green in 02-05 gate |
| 02-01-02 | 1 | QA-01, QA-02, QA-03 | T-02-01 | unit + CLI | Root vet/test; malformed YAML, bounded bodies and byte/JSON diffs | Pass — 02-01 SUMMARY; re-run green in 02-05 gate |
| 02-02-01 | 2 | QA-01, QA-03 | T-02-01, T-02-02 | proxy integration | `TestProxy`, `TestParityCommands`; named session, fixed upstream, safe flush | Pass — 02-02 SUMMARY; `TestProxySecurity` in 02-05 |
| 02-02-02 | 2 | QA-01, QA-02, QA-03 | T-02-02, T-02-03 | unit + proxy | `TestCapture`, `TestScrub`, `TestNormalize`, `TestDiff`, `TestHeaders`, `TestFlow` | Pass — 02-02 SUMMARY; `TestDiffContract`/`TestFlowContract` in 02-05 |
| 02-02-03 | 2 | QA-01, QA-02, QA-03 | T-02-03, T-02-04 | CLI + manifest | `TestManifest`, `TestCoverage`; 16-route resume across two batches | Pass — 02-02 SUMMARY; `TestManifestContract` in 02-05 |
| 02-03-01 | 3 | QA-01, QA-03 | T-02-02, T-02-04, T-02-05 | PHP capture + audit | `check_corpus.go --allow-incomplete` against source; first PHP fixture and seed self-replay; both modules vet/test | Pass — 02-03 SUMMARY |
| 02-03-02 | 3 | QA-01, QA-03 | T-02-02, T-02-04, T-02-05 | batched PHP capture | Per-batch incomplete audit and PHP replay, then strict `--require-recorded` reports 154/154; both modules vet/test | Pass — 02-03 SUMMARY |
| 02-03-03 | 3 | QA-01, QA-03 | T-02-02, T-02-05 | real client capture | `capture_clients.mjs --check-deps` and `--capture`; corpus requires client flows and secret scan | Pass — 02-03 SUMMARY; `--require-clients --check-secrets` green in 02-05 gate |
| 02-04-01 | 4 | QA-02, QA-03 | T-02-06, T-02-SC | Postgres integration | `TestParitySynthetic` starts testcontainers Postgres; both modules vet/test | Pass — 02-04 SUMMARY; 02-05 gate: `go test ./parity -run TestParitySynthetic -count=1` **4.199s** |
| 02-04-02 | 4 | QA-02, QA-03 | T-02-04, T-02-06 | app integration | `TestParityCorpus` shows 154 recorded/pending and zero false Go passes | Pass — 02-04 SUMMARY; `TestParityContract` in 02-05 |
| 02-05-01 | 5 | QA-02, QA-03 | T-02-03 | contract unit | `TestFlowContract`, `TestDiffContract`, `TestManifestContract` | Pass — `go test ./tide -run 'TestFlowContract\|TestDiffContract\|TestManifestContract' -count=1` |
| 02-05-02 | 5 | QA-01, QA-02, QA-03 | T-02-01, T-02-02, T-02-04, T-02-06 | security + app integration | `TestProxySecurity`, `TestParityCommandContract`, `TestParityContract` | Pass — framework focused tests green; app `go test ./parity -run 'TestParitySynthetic\|TestParityCorpus\|TestParityContract' -count=1` **4.888s** (Postgres started) |
| 02-05-03 | 5 | QA-01, QA-02, QA-03 | T-02-01 to T-02-06 | phase gate | `bash scripts/check-phase2.sh --fresh-php` | Pass — 2026-09-17, 116s, `phase2 check passed` |
## Wave 0 Requirements
- [ ] First implementation slice adds `tide` package tests and `cmd/summer` command smoke test so record/replay is executable from its first commit.
- [ ] App integration slice creates `../fonoteka.go/parity/parity_test.go`, a Postgres-backed synthetic handler, and a pending-route fixture status check before any real Go API port exists.
- [ ] Manifest validation rejects duplicate/missing route ids and reports exactly 154 PHP route definitions as the source snapshot.
- [x] First implementation slice adds `tide` package tests and `cmd/summer` command smoke test so record/replay is executable from its first commit.
- [x] App integration slice creates `../fonoteka.go/parity/parity_test.go`, a Postgres-backed synthetic handler, and a pending-route fixture status check before any real Go API port exists.
- [x] Manifest validation rejects duplicate/missing route ids and reports exactly 154 PHP route definitions as the source snapshot.
## Manual-Only Verifications
| Behavior | Requirement | Why manual | Test instructions |
|---|---|---|---|
| Fresh PHP backend capture and self-replay | QA-01, QA-02 | The local PHP checkout, disposable DB and real credentials are required | Run `bash scripts/check-phase2.sh --fresh-php`; it must prove a unique empty `fonoteka_parity_*` DB, run documented artisan bootstrap, and replay all fixtures against its own `127.0.0.1:8423` child. Save output and confirm zero failures. |
| Nuxt and MCP real sessions | QA-01 | Actual browser and MCP clients are required | Run `capture_clients.mjs --capture` with the isolated PHP/proxy setup; inspect committed `nuxt/` and `mcp/` flows and the secret scan. |
| Behavior | Requirement | Why manual | Test instructions | Result |
|---|---|---|---|---|
| Fresh PHP backend capture and self-replay | QA-01, QA-02 | The local PHP checkout, disposable DB and real credentials are required | Run `bash scripts/check-phase2.sh --fresh-php` | **Pass (automated in the gate).** Disposable MariaDB `fonoteka_parity_1789647136_3363199` on `127.0.0.1:32769`. Artisan preflight: database name matched, **tables=0** before `winter:up`. `fonoteka:oauth-client` printed `client_id` and redacted `client_secret`. PHP child on `127.0.0.1:8423`. Seed matched. Manifest self-replay: **recorded 154/154 passing 154 failing 0 unrecorded 0**. Fresh schema + seed, then `nuxt-browse`, `mcp-tools`, `mcp-oauth` all matched. `PHP_PARITY_TARGET=http://example.com` exits 1: `refuse: caller-supplied PHP_PARITY_TARGET is not permitted`. |
| Nuxt and MCP real sessions | QA-01 | Actual browser and MCP clients are required | Run `capture_clients.mjs --capture` with the isolated PHP/proxy setup | **Pass (recorded in 02-03; verified in 02-05).** Corpus `--require-clients --check-secrets` printed `recorded 154/154`. Gate replayed committed `nuxt-browse`, `mcp-tools`, and `mcp-oauth` (with `/tmp/summercms-parity/pkce.vars` merged) against the fresh PHP child. |
## Phase 2 gate evidence (02-05-03)
Commands (no credentials):
```bash
go vet ./... && go test ./... && go test -race ./... # summercms.go
(cd ../fonoteka.go && go vet ./... && go test ./... && go test -race ./...)
(cd ../fonoteka.go && go test ./parity -run TestParitySynthetic -count=1)
go run ../fonoteka.go/parity/check_corpus.go \
--manifest ../fonoteka.go/parity/manifest.yaml \
--routes /media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/routes.php \
--require-recorded --require-clients --check-secrets
bash scripts/check-phase2.sh --fresh-php
PHP_PARITY_TARGET=http://example.com bash scripts/check-phase2.sh --fresh-php
```
Observed:
- Root vet/test/race: all packages `ok` (tide race 1.377s on the first 02-05 gate attempt).
- App vet/test/race: `ok git.golem15.com/golem15/fonoteka/parity` (race 6.912s, testcontainers Postgres).
- `TestParitySynthetic`: `ok` in 4.199s; Docker present (missing Docker is an explicit `TestMain` failure, not a skip).
- Corpus: `recorded 154/154` with `--require-recorded --require-clients --check-secrets`.
- CLI smoke: `parity:record` wrote a loopback fixture; `parity:replay` printed `replay matched`.
- Fresh DB: unique empty `fonoteka_parity_*` confirmed via `php artisan tinker` before migrations.
- Admin bootstrap: `php artisan winter:up` with process-local `ADMIN_*`; OAuth bootstrap: `php artisan fonoteka:oauth-client "Parity MCP" ...` succeeded with secret redacted from logs.
- PHP origin pinned to `http://127.0.0.1:8423`; trap removes the MariaDB container and PHP child.
- PHP self-replay: seed match; **154/154 passing**; Nuxt and both MCP flows match.
Prior implementation commits (02-01 through 02-04, plus 02-05 tasks 1–2) ran `go vet ./...` and `go test ./...` green in their summaries, satisfying QA-03's per-commit requirement.
## Validation Sign-Off
- [ ] Each finalized plan task has an automated verify or an explicit manual gate.
- [ ] No three consecutive tasks lack automated feedback.
- [ ] Testcontainers Postgres test executes; it is not silently skipped in the phase completion run.
- [ ] Root and app module `go vet`, `go test`, and `go test -race` pass.
- [ ] PHP self-replay and 154-route coverage evidence are recorded.
- [ ] Set `nyquist_compliant: true` only after all mapped checks exist and pass.
- [x] Each finalized plan task has an automated verify or an explicit manual gate.
- [x] No three consecutive tasks lack automated feedback.
- [x] Testcontainers Postgres test executes; it is not silently skipped in the phase completion run.
- [x] Root and app module `go vet`, `go test`, and `go test -race` pass.
- [x] PHP self-replay and 154-route coverage evidence are recorded.
- [x] Set `nyquist_compliant: true` only after all mapped checks exist and pass.
**Approval:** Pending execution evidence.
**Approval:** Phase 2 gate evidence recorded 2026-09-17. `nyquist_compliant: true`.