Files
summercms/.planning/phases/02-api-parity-harness-bootstrap/02-VERIFICATION.md
2026-09-17 14:23:11 +02:00

196 lines
19 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
phase: 02-api-parity-harness-bootstrap
verified: 2026-09-17T12:21:50Z
status: passed
score: 4/4 must-haves verified
overrides_applied: 0
human_verification: []
decision_coverage:
honored: 16
total: 16
not_honored: []
---
# Phase 2: API parity harness bootstrap Verification Report
**Phase Goal:** A fixture recorder captures request/response pairs from the running PHP backend — including real Nuxt and MCP flows — and a replay-and-diff harness with a normalizer for nondeterministic fields can run against any HTTP backend. The recorder and replayer are `summer parity:*` commands on the Phase 1 command kernel (bonfire); recording itself only needs a running PHP backend, but the tooling waits for Phase 1.
**Verified:** 2026-09-17T12:21:50Z
**Status:** passed
**Re-verification:** No — initial verification
Must-haves are the four ROADMAP success criteria (they override plan-level truths). Plan `must_haves` were checked as supporting evidence. `gsd-sdk query user-story.validate` is not required: the ROADMAP goal is an infrastructure outcome, not a user story.
## Goal Achievement
### Observable Truths
| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| 1 | The fixture recorder captures a request/response pair from the live PHP backend for at least one route and stores it as a replayable fixture | ✓ VERIFIED | 154 committed one-step route fixtures under `../fonoteka.go/parity/fixtures/routes/` plus seed `fixtures/seed/bootstrap.yaml`. First JWT genres case `get_genres_jwt.yaml` is a live PHP `GET /_fonoteka/api/v1/genres` body (Winter catalog rows + placeholders). `go run …/check_corpus.go --require-recorded --require-clients --check-secrets` printed `recorded 154/154`. VALIDATION.md: `--fresh-php` self-replay **154/154 passing, 0 failing, 0 unrecorded**. Also real client flows `nuxt-browse.yaml` (170KB, 3k+ lines), `mcp-tools.yaml` (me + genres read + POST create), `mcp-oauth.yaml` (well-known, register, PKCE authorize/consent/token). |
| 2 | The replay-and-diff harness runs a recorded fixture against an arbitrary `httptest.Server` backend and reports a byte-level diff | ✓ VERIFIED | `tide.ReplayFlow` + `compareBodies` (JSON path diffs via `UseNumber` + exact-byte offset for non-JSON). CLI: `parity:replay --fixtures --target`. Tests: `TestParityRoundTrip` / `TestParityCommands` record then replay against httptest; changed JSON fails at `$.data`; changed bytes report offset. This session: `go test ./cmd/summer -run 'TestParityCommandContract\|TestParityCommands'` PASS. App: `TestParitySynthetic` replays SQL-backed POST/GET through `httptest.NewServer` (live this session, 13.297s). |
| 3 | The harness's normalizer and assertions explicitly catch the parity classes named in research (nil vs `[]`, date format, tri-state booleans, envelope/conditional keys) on a synthetic test case, not just status codes | ✓ VERIFIED | `tide/diff_contract_test.go` `TestDiffContract`: baseline + key-order pass; mutations fail with a named JSON path for null vs array, empty object vs array, Carbon `Z` vs `+00:00`, non-date text, present-null vs absent date, tri-state true/false/null, missing `meta` envelope, extra `links` key, conditional `reservation`, money string vs number, integer id vs string/fraction, exact slug. `normalize.go` asserts Carbon `+00:00` then masks `*_at`; integer IDs masked after type check; slugs exact. This session: `go test ./tide -run TestDiffContract` PASS. |
| 4 | `go vet` and `go test ./...` are green, and the harness's own integration tests run against testcontainers Postgres | ✓ VERIFIED | This session: root `go vet ./... && go test ./...` exit 0 (all packages ok, including `tide` and `cmd/summer`). App `go vet ./... && go test -short ./...` exit 0. Live `go test ./parity -run TestParitySynthetic -count=1` **ok 13.297s** with Docker (TestMain fails closed if Docker is missing; `-short` skips the container). VALIDATION.md: full `--fresh-php` gate 116s including race + 154-route PHP self-replay. Plan 05 is the dedicated unit/contract-test plan. |
**Score:** 4/4 truths verified
Supporting plan truths (D-01…D-16, CLI on bonfire, 154-route corpus, pending≠passing) all hold in code; they are not extra score rows.
### Required Artifacts
`gsd-sdk query verify.artifacts`: 21/24 passed. The three failures are **false negatives**: the checker treats directories as files. Manual: `fixtures/routes/` has **154** YAML files; `fixtures/nuxt/` and `fixtures/mcp/` exist with committed flows.
| Artifact | Expected | Status | Details |
|----------|----------|--------|---------|
| `tide/flow.go` | Versioned flow/step contracts | ✓ VERIFIED | 219 lines; `Flow`/`Step`/`Request`/`Response`/`Capture`/`Normalize` |
| `tide/fixture.go` | Validated YAML load/atomic save | ✓ VERIFIED | 243 lines; goccy unknown-field reject; temp+sync+rename |
| `tide/record.go` | HTTP recording engine | ✓ VERIFIED | 147 lines; `RecordFlow`, bounded bodies, capture/scrub |
| `tide/replay.go` | Backend-independent replay | ✓ VERIFIED | 104 lines; `ReplayFlow` against injected target; continue-after-diff |
| `cmd/summer/parity.go` | Bonfire `parity:*` commands | ✓ VERIFIED | 307 lines; `parity:proxy`/`record`/`replay` |
| `tide/proxy.go` | Loopback capture proxy | ✓ VERIFIED | 380 lines; `httputil.ReverseProxy` Rewrite/SetURL; loopback-only |
| `tide/variables.go` | Capture, expand, scrub | ✓ VERIFIED | 645 lines; `{{name}}`, 0600 store, credential reject |
| `tide/normalize.go` | Shape-aware masking | ✓ VERIFIED | 149 lines; Carbon `+00:00`, integer ids, slug exact |
| `tide/manifest.go` | Generic 154-route runner | ✓ VERIFIED | 561 lines; `--next-batch` cap 15; pending/ported |
| `tide/report.go` | Coverage table | ✓ VERIFIED | `CoverageHeaders`/`SummaryLine`; used by CLI `out.Table` |
| `../fonoteka.go/parity/manifest.yaml` | Exact 154-route map | ✓ VERIFIED | 154 `- id:` rows, all `status: pending` |
| `../fonoteka.go/parity/fixtures/seed/bootstrap.yaml` | API-created seed | ✓ VERIFIED | Recorded bootstrap flow, 8468 bytes |
| `../fonoteka.go/parity/fixtures/routes/` | One fixture per route | ✓ VERIFIED | 154 YAML files (checker missed directory) |
| `../fonoteka.go/parity/fixtures/nuxt/` | Real Nuxt session | ✓ VERIFIED | `nuxt-browse.yaml` 170062 bytes |
| `../fonoteka.go/parity/fixtures/mcp/` | Real MCP sessions | ✓ VERIFIED | `mcp-tools.yaml` + `mcp-oauth.yaml` |
| `../fonoteka.go/parity/parity_test.go` | Corpus runner + Phase 3 seam | ✓ VERIFIED | `newTarget` / `seedHooks`; 154 pending subtests |
| `../fonoteka.go/parity/synthetic_test.go` | Postgres + httptest handler | ✓ VERIFIED | testcontainers `postgres:16-alpine`; POST/GET `/synthetic/items` |
| `../fonoteka.go/parity/testdata/synthetic-seed.yaml` | SQL-backed write flow | ✓ VERIFIED | Exists; replayed by `TestParitySynthetic` |
| `../fonoteka.go/parity/testdata/synthetic-read.yaml` | Read + named seed hook | ✓ VERIFIED | `synthetic-item` hook |
| `tide/diff_contract_test.go` | QA-02 negative tests | ✓ VERIFIED | Table of parity-class mutations |
| `tide/proxy_security_test.go` | Loopback/scrub regressions | ✓ VERIFIED | Non-loopback, caps, secret scan |
| `../fonoteka.go/parity/parity_contract_test.go` | Pending≠passing contracts | ✓ VERIFIED | 154 pending, zero Go passes |
| `scripts/check-phase2.sh` | Repeatable phase gate | ✓ VERIFIED | 374 lines; refuses `PHP_PARITY_TARGET`; MariaDB + PHP child |
| `.planning/phases/02-api-parity-harness-bootstrap/02-VALIDATION.md` | Measured evidence | ✓ VERIFIED | `nyquist_compliant: true`; 116s `--fresh-php` |
**Artifacts:** 24/24 verified (3 via manual directory check)
### Key Link Verification
`gsd-sdk query verify.key-links` reported 0/18 verified. Same false negative as Phase 1: the checker greps target *filenames* inside source; Go imports package paths. Manual wiring:
| From | To | Via | Status | Details |
|------|----|-----|--------|---------|
| `cmd/summer/main.go` | `cmd/summer/parity.go` | `toolCommands` | ✓ WIRED | `parityProxyCommand()`, `parityRecordCommand()`, `parityReplayCommand()` in `toolCommands()` |
| `cmd/summer/parity.go` | `tide/record.go` | `RecordFlow` | ✓ WIRED | `tide.RecordFlow(ctx, spec, cfg)` in `runParityRecord` |
| `cmd/summer/parity.go` | `tide/replay.go` | `ReplayFlow` | ✓ WIRED | `tide.ReplayFlow(ctx, flow, cfg)` in `runParityReplay` |
| `cmd/summer/parity.go` | `tide/proxy.go` | `parity:proxy` | ✓ WIRED | `tide.NewProxy` + `ListenAndServe`; `tide.LoadRules` |
| `tide/proxy.go` | `tide/rules.go` | per-path rules | ✓ WIRED | `p.cfg.Rules.Match` / header filter / capture copy |
| `tide/proxy.go` | `tide/variables.go` | session store | ✓ WIRED | `OpenStore`, `CaptureStep`, `ScrubStep` before flush |
| `tide/manifest.go` | `tide/record.go` | route-case record | ✓ WIRED | `RecordFlow(...)` inside `RecordManifest` |
| `tide/replay.go` | `tide/variables.go` | expand/capture | ✓ WIRED | `expandRequest` then `CaptureStep` then `expandResponse` |
| `tide/replay.go` | `tide/normalize.go` | compare-time mask | ✓ WIRED | `compareStep` → `compareBodies` → `normalizeJSON` |
| `../fonoteka.go/parity/manifest.yaml` | `fixtures/routes/` | every case fixture | ✓ WIRED | `ValidateManifest(..., ModeRequireRecorded)` + corpus `recorded 154/154` |
| `fixtures/seed/bootstrap.yaml` | route cases | captured ids | ✓ WIRED | Seed captures feed `{{jwt:…}}` / `{{id:…}}` in route YAML |
| `capture_clients.mjs` | `127.0.0.1:8422` | unchanged clients | ✓ WIRED | Spawns `parity:proxy`; `NUXT_DEV_BACKEND_ORIGIN` and `FONOTEKA_API_URL` = proxy |
| `parity_test.go` | `manifest.yaml` | pending subtests | ✓ WIRED | `tide.LoadManifest` + `t.Run` per route id |
| `parity_test.go` | `synthetic_test.go` | `newTarget` / hooks | ✓ WIRED | `newTarget` → `newSyntheticHandler`; `seedHooks["synthetic-item"]` |
| `synthetic_test.go` | `tide/replay.go` | httptest replay | ✓ WIRED | `tide.ReplayFlow(..., Target: srv.URL)` |
| `scripts/check-phase2.sh` | `parity_test.go` | `go test ./parity` | ✓ WIRED | `go test ./...` plus explicit `TestParitySynthetic` |
| `scripts/check-phase2.sh` | `check_corpus.go` | corpus audit | ✓ WIRED | `go run ./parity/check_corpus.go --require-recorded --require-clients --check-secrets` |
| `tide/diff_contract_test.go` | `tide/diff.go` | public Replay API | ✓ WIRED | Mutations go through `RecordFlow`/`ReplayFlow`, not private helpers |
**Wiring:** 18/18 connections verified (manual)
### Data-Flow Trace (Level 4)
| Artifact | Data | Source | Produces real data | Status |
|----------|------|--------|--------------------|--------|
| `parity:record` | fixture YAML body | Live HTTP response bytes | Yes — CLI tests record httptest JSON; PHP corpus bodies are Winter JSON (genres catalog, `+00:00` timestamps) | ✓ FLOWING |
| `parity:proxy` | multi-step Nuxt/MCP flows | Unchanged clients via `:8422` → PHP `:8423` | Yes — `nuxt-browse` includes onboarding, embed.js, collections/albums; `mcp-oauth` includes well-known + PKCE | ✓ FLOWING |
| `parity:replay` | path diffs | `compareBodies` / `diffJSON` / `diffBytes` | Yes — CLI mismatch prints expected/actual; contract tests assert path names | ✓ FLOWING |
| `TestParitySynthetic` | row name | POST `/synthetic/items` → Postgres → GET | Yes — live this session; mutated name fails at `$[0].name` | ✓ FLOWING |
| `check_corpus.go` | 154/154 | manifest vs snapshot vs `routes.php` | Yes — this session printed `recorded 154/154` | ✓ FLOWING |
## Requirements Coverage
PLAN frontmatter IDs: QA-01, QA-02, QA-03 (every plan). REQUIREMENTS.md maps the same three to Phase 2. No Phase 2 orphans. QA-04 is Phase 3; QA-05 is Phase 15.
| Requirement | Source Plan | Description | Status | Evidence |
|-------------|-------------|-------------|--------|----------|
| QA-01 | 01, 02, 03, 05 | Recorder captures every PHP route plus real Nuxt and MCP flows | ✓ SATISFIED | 154 route fixtures, seed, `nuxt-browse`, `mcp-tools`, `mcp-oauth`; corpus `--require-recorded --require-clients --check-secrets`; VALIDATION.md 154/154 PHP self-replay |
| QA-02 | 01, 02, 04, 05 | Replay-and-diff with normalizer and named parity-class assertions | ✓ SATISFIED | `TestDiffContract` / `TestNormalize` / `TestDiff`; synthetic httptest replay. Go backend for PHP routes is Phase 3 — ROADMAP SC2 is httptest, which is what shipped |
| QA-03 | 01–05 | `go vet` / `go test ./...` green; phase ends with unit-test plan; integration uses testcontainers Postgres | ✓ SATISFIED | Plan 05 is the dedicated test plan; this session root vet/test green; `TestParitySynthetic` starts Postgres (13.297s). Per-commit green claimed in summaries; not re-audited commit-by-commit |
**Coverage:** 3/3 requirements satisfied
### Decision Coverage
All trackable CONTEXT.md decisions are honored by shipped artifacts. `gsd-sdk query check.decision-coverage-verify`: **16/16 honored**, `not_honored: []`. D-01 proxy+manifest, D-02 loopback PHP, D-03 API seed + artisan-only admin/OAuth, D-04 `summer parity:*` on bonfire, D-05/D-06 YAML flows, D-07/D-11 placeholders, D-08 `fonoteka.go/parity/`, D-09/D-10 seed + named hooks, D-12 httptest + CLI target, D-13–D-16 diff/headers/normalize/coverage.
## Behavioral Verification
| Check | Result | Detail |
|-------|--------|--------|
| Root `go vet ./... && go test ./...` | ✓ | All packages ok (this session) |
| App `go vet ./... && go test -short ./...` | ✓ | `ok git.golem15.com/golem15/fonoteka/parity` |
| `TestDiffContract` / `TestFlowContract` / `TestManifestContract` | ✓ | `ok tide 0.032s` |
| `TestParityCommands` / `TestParityCommandContract` | ✓ | Discovers `parity:record|proxy|replay`; mismatch exits nonzero |
| `go run ./cmd/summer parity:record --help` | ✓ | Flags `--spec --target --output --manifest --fixtures` |
| `parity:replay --help` / `parity:proxy --help` | ✓ | Defaults listen `127.0.0.1:8422`, upstream `http://127.0.0.1:8423` |
| `check_corpus.go --require-recorded --require-clients --check-secrets` | ✓ | `recorded 154/154` |
| `TestParitySynthetic` (testcontainers) | ✓ | Live this session, 13.297s |
| `TestParityCorpus` `-short` | ✓ | 154 pending subtests; `ported-mismatch` skipped without Docker; coverage pending=154 passing=0 |
| Full `scripts/check-phase2.sh --fresh-php` | ℹ not re-run | VALIDATION.md 2026-09-17: 116s, 154/154 PHP self-replay, client flows match, caller `PHP_PARITY_TARGET` refused. Spot-checks above confirm the same corpus and harness still hold |
## Anti-Patterns Found
No `TODO` / `FIXME` / `XXX` / `TBD` / `HACK` in phase Go/sh/mjs. No stub `not implemented` handlers on the record/replay path. No disabled requirement tests in the framework module.
| File | Line | Pattern | Severity | Impact |
|------|------|---------|----------|--------|
| `tide/variables.go` `replaceIsolated` | ~530 | Short captured numeric IDs are substituted anywhere a token boundary matches | ⚠️ Warning | Catalog/pagination/IP octets that happen to equal a captured id become `{{id:album}}` / `{{id:token}}` (e.g. Rock genre id → `{{id:token}}`, `127.0.0.{{id:album}}`, CSS `1.4` in embed.js). PHP self-replay still passed because seed IDs are stable. A later Postgres port can false-fail or false-pass if sequences differ. Compare-time `normalize.go` masking is the intended ID handling; record-time global replace of 1–2 digit ids is overly broad. |
| `../fonoteka.go/parity/synthetic_test.go` | 103 | `t.Skip` under `-short` | ℹ️ Info | Fast loop skips Postgres; TestMain still fails closed without Docker when not `-short`. Phase gate runs without `-short`. |
| `TestParityCorpus/ported-mismatch` | — | Skipped under `-short` | ℹ️ Info | Needs the synthetic handler + DB; full/non-short run covers it (`TestParityContract` too). |
**Anti-patterns:** 3 found (0 blockers, 1 warning, 2 info)
## Test Quality Audit
| Test File | Linked Req | Active | Skipped | Circular | Assertion Level | Verdict |
|-----------|-----------|--------|---------|----------|----------------|---------|
| `tide/diff_contract_test.go` | QA-02 | yes | no | no | Value (path + expected/actual) | ✓ |
| `tide/flow_contract_test.go` | QA-02, D-16 | yes | no | no | Behavioral (continuation, capture) | ✓ |
| `tide/normalize_test.go` / `diff_test.go` | QA-02 | yes | no | no | Value | ✓ |
| `cmd/summer/parity_test.go` | QA-01, QA-02 | yes | no | no | Behavioral (CLI record/replay) | ✓ |
| `tide/proxy_security_test.go` | QA-01, D-07 | yes | no | no | Behavioral (loopback, scrub) | ✓ |
| `../fonoteka.go/parity/parity_test.go` | QA-01, QA-03 | yes | `-short` on DB-backed mismatch | no | Value (154 pending, fixture load) | ✓ |
| `../fonoteka.go/parity/synthetic_test.go` | QA-02, QA-03 | yes | `-short` only | no | Behavioral (SQL write/read + path diff) | ✓ |
| `../fonoteka.go/parity/parity_contract_test.go` | QA-02, QA-03 | yes | no (uses `parityDB`) | no | Value (pending≠pass) | ✓ |
| `../fonoteka.go/parity/check_corpus.go` | QA-01 | n/a (cmd) | no | no | Value (154 ids, secret scan) | ✓ |
**Disabled tests on requirements:** 0 blockers (`-short` skips are the documented fast loop, not the sign-off path).
**Circular patterns detected:** 0. Fixtures are captured from live PHP / unchanged Nuxt+MCP, not generated from the Go SUT. Synthetic expected bodies are independently authored testdata.
**Insufficient assertions:** 0 for this phase. `TestParityCorpus` pending rows only load fixtures (honest: no Go handler yet); value-level PHP match is the `--fresh-php` gate, not `go test`.
**Provenance:** VALID — expected bodies come from the PHP backend (and real clients), which is the external oracle QA-01 names.
## Human Verification Required
N/A — Infrastructure/tooling phase with no user-facing product UI. Acceptance is CLI + corpus + tests. VALIDATION.md already automated the PHP capture/self-replay and client-flow checks that were listed as manual during planning.
## Gaps Summary
**No blocking gaps.** All four ROADMAP success criteria hold in the codebase and were exercised this session (root vet/test, app short tests, live TestParitySynthetic, corpus audit, contract tests). The 154-route PHP MariaDB gate was not re-run; VALIDATION.md plus the still-present corpus and harness are consistent with that claim.
The integer-placeholder over-scrub (warning above) does not fail a Phase 2 success criterion. Phase 3's first green genres diff should treat captured-id collisions as a fixture-hygiene check if Postgres IDs diverge from the recording DB.
## Recommended Fix Plans
None — status is `passed`.
## Verification Metadata
**Verification approach:** Goal-backward from ROADMAP success criteria
**Must-haves source:** ROADMAP.md success criteria (override PLAN.md frontmatter)
**Automated checks:** 18 passed, 0 failed (gsd-sdk filename-grep false negatives excluded)
**Human checks required:** 0
**Total verification time:** ~18 min
---
*Verified: 2026-09-17T12:21:50Z*
*Verifier: the agent (subagent)*