docs(02): research parity harness and validation
This commit is contained in:
@@ -0,0 +1,22 @@
|
|||||||
|
# Phase 2 Pattern Map
|
||||||
|
|
||||||
|
**Mapped:** 2026-09-16
|
||||||
|
|
||||||
|
| Planned file or area | Closest existing analog | Pattern to reuse |
|
||||||
|
|---|---|---|
|
||||||
|
| `cmd/summer` parity commands | `cmd/summer/main.go`, `bonfire/command.go` | Add three `bonfire.Command` values to `toolCommands()`, with `Run(ctx, in, out) error`, string flags, injected output, and colon names. Keep process exit handling in `main()`. |
|
||||||
|
| CLI command tests | `cmd/summer/main_test.go`, `bonfire/output_test.go` | Assert command discovery and behavior through `bonfire.NewRoot` with injected writers; do not invoke a TTY or read real credentials. |
|
||||||
|
| `tide` framework library | No existing HTTP library in framework; `internal/build` is the closest separation example | Keep reusable types and behavior independent of Fonoteka route names. CLI calls package APIs; app corpus imports package by module path. Use context cancellation and explicit dependencies. |
|
||||||
|
| YAML fixture IO | `internal/build/manifest.go` | Strict decode, validate before use, deterministic output and atomic file write. Preserve raw JSON body text; reject unsafe relative paths for binary sidecars. |
|
||||||
|
| Proxy HTTP handling | Go `net/http/httputil.ReverseProxy` official API | Fixed upstream, `Rewrite`/`SetURL`, loopback listener, bounded body capture, restore body for forwarding. Avoid `Director`, which official docs deprecate. |
|
||||||
|
| Synthetic round-trip tests | `internal/build/build_test.go`, `examples/hello/hello_test.go` | Use temp directories and observable outputs. HTTP tests use `httptest.Server`; compare request/response bytes and CLI exit status. |
|
||||||
|
| App manifest and corpus | `../fonoteka.go` has no analog yet; PHP `plugins/golem15/fonoteka/routes.php` is source of truth | Create `parity/manifest.yaml` and `parity/fixtures/{seed,nuxt,mcp,routes}`. Route ids must encode HTTP method, full path pattern and auth group to distinguish repeated paths. Count 154 definitions. |
|
||||||
|
| App Postgres integration | No app code yet; Phase 1 `scripts/check-phase1.sh` demonstrates cross-module verification | `parity/parity_test.go` starts Testcontainers Postgres, runs a synthetic SQL-backed handler through `httptest.Server`, executes seed and route flows as subtests, and marks unported real routes pending. Later phases replace synthetic handler with app handler. |
|
||||||
|
| Phase-wide check script | `scripts/check-phase1.sh` | Run vet/test/race separately in root and sibling app module; root `./...` does not traverse sibling modules. Reserve full coverage expansion for final plan. |
|
||||||
|
|
||||||
|
## Integration constraints
|
||||||
|
|
||||||
|
- `bonfire.NewRoot` accepts `[]bonfire.Command`; no plugin registry or new CLI framework is needed.
|
||||||
|
- Framework `go.mod` currently has Cobra and Koanf but no YAML fixture or Testcontainers dependency. Add goccy/go-yaml to framework when needed; keep Testcontainers in app test module.
|
||||||
|
- The PHP source and existing Nuxt/MCP clients are read-only references for this phase. They are run against a fresh parity DB, not edited.
|
||||||
|
- `../fonoteka.go` is a sibling repository. The executor needs write access there for app corpus and test files.
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
# Phase 2: API parity harness bootstrap — Research
|
||||||
|
|
||||||
|
**Researched:** 2026-09-16
|
||||||
|
**Status:** Ready for planning after the two context checkpoints below
|
||||||
|
|
||||||
|
## What exists
|
||||||
|
|
||||||
|
- Phase 1 is implemented. `bonfire.Command` is a value with `Name`, `Flags`, `Args`, and `Run(ctx, Input, Output) error`; `cmd/summer/main.go` registers tool commands through `bonfire.NewRoot`. `parity:proxy`, `parity:record`, and `parity:replay` can join `toolCommands()` without a second CLI framework. `bonfire.Flag` currently represents string flags only, so the commands should parse string values rather than assume native bool/int flags.
|
||||||
|
- The sibling app repository is `../fonoteka.go` (not `/media/nvme/dev/golem15/fonoteka.go`). It contains only `go.mod`, README, and CLAUDE.md. Phase 2 must create its `parity/` tree. It cannot claim a real Go Fonoteka handler exists yet.
|
||||||
|
- The PHP source `/media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/routes.php` contains **154** `Route::get/post/put/patch/delete/...` declarations by a line anchored count. The groups include JWT with and without password gate, public share, throttled onboarding, scoped personal tokens, and unprefixed OAuth. Core user register/login routes used by seed flow live outside this file and should be listed separately from the 154-route coverage denominator.
|
||||||
|
- The PHP README runs `php artisan serve --port=8422`; Nuxt's `NUXT_DEV_BACKEND_ORIGIN` and MCP's `FONOTEKA_API_URL` can point at a local recording proxy. The PHP originals and clients do not need code changes.
|
||||||
|
|
||||||
|
## Recommended implementation slices
|
||||||
|
|
||||||
|
1. **One real round trip first.** A generic `tide` library owns a versioned YAML flow schema and a `Record`/`Replay` API. A `summer parity:record` call against a local `httptest.Server` writes one route fixture; `summer parity:replay --target` reads that same fixture and reports a mismatch with expected and actual bytes. This is the first complete user path, not a schema-only plan.
|
||||||
|
2. **Capture both sources.** Add `parity:proxy` with `httputil.ReverseProxy` for live Nuxt/MCP sessions and manifest-driven `parity:record` for exhaustive route cases. Use `Rewrite`/`SetURL` for upstream routing; wrap the transport or response body to observe bytes while forwarding unchanged. `ModifyResponse` sees a response before it is copied to the client, but a recorder must replace the consumed body for forwarding. Cap buffered request and response sizes and record truncation as an error rather than silently committing a partial fixture. Group proxy requests into named sessions through an explicit local recording session header or CLI session id; ordered arrival alone is insufficient across concurrent browser requests. Preserve cookies and redirects needed by PKCE, while retaining only declared headers in fixtures.
|
||||||
|
3. **One fixture model.** Flow name/description, ordered steps, request method/path/raw query/kept headers/body, expected response status/kept headers/body, `capture`, `normalize`, optional `seed_hook`, and route id/status belong in a strict versioned schema. JSON body text stays a YAML literal block scalar for review and raw diagnostics; for binary responses, use a relative sidecar file plus digest and content type. Reject absolute or parent-traversing sidecar paths. Route manifest and fixture schema are app-specific data over the framework-owned types.
|
||||||
|
4. **Replay state explicitly.** A variable store resolves `{{jwt:alice}}`, `{{token:mcp-read}}`, ids, share tokens, OAuth codes, and PKCE verifier references before each request; capture rules populate variables from recorded and replayed responses. Scrub credentials at record time, before writing YAML or error logs. An unrecognized credential-shaped value in a kept auth/cookie field should fail the recording; a missing variable should fail the flow, not send an unresolved placeholder. Seed runs first, then ordered route and client flows. Pending routes are recorded but do not fail Go CI until marked `ported`; PHP self-replay checks every recorded flow.
|
||||||
|
5. **Diff without weakening contracts.** Decode JSON into maps/slices with `json.Decoder.UseNumber`, compare key presence, array/object/null/bool/string/number types and values recursively, and report JSON-path-like locations. `UseNumber` retains numeric lexemes and avoids float64 conversion; decide whether numerically equivalent spellings are acceptable, but never equate a number with a string. Ignore JSON object key order. Keep raw byte comparison for non-JSON bodies and printable raw-body diagnostics. The term “byte-level diff” in the roadmap should mean exact body bytes for non-JSON and an exact expected/actual rendering at mismatched JSON paths; JSON key order is intentionally ignored by D-13.
|
||||||
|
6. **Normalize narrowly.** Match explicit JSON paths (`*_at`, known ids, captured values) with small in-house path rules; run a shape assertion before masking. For dates require an offset `+00:00` style where the PHP contract does, including key present with `null` where recorded. Assert ids are JSON integers before masking. Preserve nil versus `[]`, tri-state bool, fixed-decimal string versus number, envelope keys, and conditional key presence. A rule can be disabled per step; a broad “ignore timestamps/ids” filter would hide these failures.
|
||||||
|
7. **Report the whole run.** Continue after a failed comparison, but stop dependent steps within a flow after a failed capture. Print status/header/path/body differences and a route coverage table (recorded/passing/failing/unrecorded) against 154 manifest route ids. Exclude seed and client-only flows from the denominator. CLI exits nonzero on a ported failure or an unrecorded required route; app tests create one subtest per flow so `-run` can select it.
|
||||||
|
|
||||||
|
## Validation Architecture
|
||||||
|
|
||||||
|
- Fast checks at every implementation commit: `go vet ./... && go test ./...` in the framework root, plus the same in `../fonoteka.go` once that module has code. `go.work` does not make root `./...` traverse sibling modules; check each explicitly, as Phase 1's `scripts/check-phase1.sh` does for the hello modules.
|
||||||
|
- Framework synthetic tests should cover a recorded `httptest.Server` round trip, HTTP error/redirect/cookie preservation, YAML round trip, capture and placeholder resolution, secret scrub, path traversal rejection, JSON token-type diff, each named parity class, header allow-list, flow continuation, and CLI nonzero exit. This is meaningful behavioral coverage rather than snapshotting implementation.
|
||||||
|
- The app repository should have a testcontainers Postgres integration test using a small synthetic `http.Handler` backed by the container, with a real write/read flow and a temporary SQL seed hook. It proves the harness and database state protocol before Phase 3 provides the first ported route. The app test must not report the 154 PHP routes as passing against an absent Go app. Testcontainers' Postgres module exposes `postgres.Run`, `ConnectionString`, cleanup and snapshot options; one container/database per suite and ordered flows are consistent with D-09 and the cost constraint.
|
||||||
|
- An operator-only PHP self-check should run on an isolated local database with `summer parity:replay --target <php-origin>` and must pass before committed fixtures are accepted. The Phase 2 completion evidence includes one actual PHP capture, the complete 154-route manifest and scripted route-case coverage, and named Nuxt/MCP recorded sessions. External-service-dependent routes may need deterministic error cases with declared expected statuses, but every route must have at least one recorded case under QA-01.
|
||||||
|
- Reserve the final plan for comprehensive unit and integration tests, per `CLAUDE.md`. Earlier slices still need smoke tests and both modules' vet/test commands to stay green at each commit.
|
||||||
|
|
||||||
|
## Risks and decisions to settle
|
||||||
|
|
||||||
|
1. **Port collision in locked decisions.** D-01 puts the proxy on `:8422`; D-02 and the PHP README put `php artisan serve` on `:8422`. Both cannot bind the same host address. The simplest local setup is PHP on `127.0.0.1:8423`, proxy on `127.0.0.1:8422`, with Nuxt/MCP pointed at the proxy. That changes D-02's literal PHP port and needs user confirmation or a CONTEXT.md correction before executable plans are written. Another viable arrangement binds PHP to `127.0.0.2:8422` and proxy to `127.0.0.1:8422`, preserving the number but adding host alias complexity.
|
||||||
|
2. **Scope is large.** `QA-01` asks for all 154 routes plus real client flows, while the roadmap's first success criterion says at least one captured route. The plan must satisfy QA-01 and separately show the one-route vertical slice early. It must not turn the 154-route corpus into a future-phase placeholder.
|
||||||
|
3. **No Go app handler yet.** D-12's in-process app replay cannot honestly run against the real port before Phase 3. Use a synthetic handler with testcontainers in Phase 2 and preserve an app handler wiring seam for Phase 3. `pending|ported` gates Go regressions; PHP self-replay validates the full corpus now.
|
||||||
|
4. **Secrets and proxy exposure.** Bind the proxy to loopback, require a fixed upstream URL rather than an arbitrary request-chosen destination, redact auth/cookie/query/body credential sources before disk/log output, reject unsanitized fixtures, and keep fixture writes atomic. Raw fixture bodies can still include private user data, so the parity dataset must contain only deterministic test identities.
|
||||||
|
5. **Replay changes state.** Mutating route cases can invalidate later cases. Give the manifest deterministic order and explicit seed or reset strategy. Reserve unique test data per case or use database snapshots for destructive cases; testcontainers Postgres snapshots are available but PHP self-replay may need a fresh database instead.
|
||||||
|
6. **Cross-repo execution.** Planning docs live here, while app corpus and test live in `../fonoteka.go`; execution must have write access to that sibling checkout. Existing framework `go.work` need not absorb the app module if it is intentionally separate.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
- Local: `bonfire/command.go`, `cmd/summer/main.go`, `../fonoteka.go/go.mod`, PHP `routes.php`, PHP and Nuxt/MCP READMEs, `.planning/research/PITFALLS.md`, Phase 2 CONTEXT.md.
|
||||||
|
- [Go `httputil.ReverseProxy`](https://pkg.go.dev/net/http/httputil) — `Rewrite`, `SetURL`, `ModifyResponse`, and forwarding semantics.
|
||||||
|
- [Go `encoding/json`](https://pkg.go.dev/encoding/json) — `Decoder.UseNumber` and `json.Number`.
|
||||||
|
- [Go `httptest`](https://pkg.go.dev/net/http/httptest) — in-process HTTP servers for synthetic and app tests.
|
||||||
|
- [goccy/go-yaml](https://github.com/goccy/go-yaml) — tagged field encode/decode and custom marshaling.
|
||||||
|
- [Testcontainers Postgres module](https://golang.testcontainers.org/modules/postgres/) — `postgres.Run`, connection string, cleanup and snapshots.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Phase: 02-api-parity-harness-bootstrap*
|
||||||
|
*Ready for planning: after port and plan-count checkpoints*
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
---
|
||||||
|
phase: 02
|
||||||
|
slug: api-parity-harness-bootstrap
|
||||||
|
status: draft
|
||||||
|
nyquist_compliant: false
|
||||||
|
wave_0_complete: false
|
||||||
|
created: 2026-09-16
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 2 — Validation Strategy
|
||||||
|
|
||||||
|
## Test Infrastructure
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|---|---|
|
||||||
|
| Framework | Go 1.27 `testing`, `httptest`; Testcontainers Postgres for app integration |
|
||||||
|
| Config file | Root `go.mod`; app `../fonoteka.go/go.mod`; no separate test config |
|
||||||
|
| Quick run | `go vet ./... && go test ./...` in root, then the same from `../fonoteka.go` once code exists |
|
||||||
|
| Full suite | Quick run plus `go test -race ./...` in both modules, CLI synthetic smoke, PHP self-replay on isolated local DB, route coverage audit |
|
||||||
|
| Estimated runtime | Measure during execution; Testcontainers and PHP checks are slower than package tests |
|
||||||
|
|
||||||
|
## Sampling Rate
|
||||||
|
|
||||||
|
- After every implementation task commit: root `go vet ./... && go test ./...`; once app files exist, also run the app module checks.
|
||||||
|
- After each plan wave: run the above and the slice's CLI smoke against a local `httptest.Server` or isolated PHP server as applicable.
|
||||||
|
- Before `$gsd-verify-work`: run both modules' vet, test and race suites; run testcontainers Postgres integration; run PHP self-replay and inspect the 154-route coverage report.
|
||||||
|
- Keep quick checks in seconds after build cache warmup. Measure actual runtime; no arbitrary latency promise is set.
|
||||||
|
|
||||||
|
## Per-Task Verification Map
|
||||||
|
|
||||||
|
Plan/task IDs are assigned after the required plan-count checkpoint. The planner must attach each row to an exact task and automated command.
|
||||||
|
|
||||||
|
| Slice | Requirement | Threat Ref | Secure behavior | Test type | Automated command or assertion | Initial status |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| One-fixture record and replay through `summer` | QA-01, QA-02 | T-02-01 | Loopback target and bounded body handling | integration | `go test ./tide ./cmd/summer` with `httptest.Server`, fixture written then diffed | Pending |
|
||||||
|
| Live PHP proxy and scripted route recording | QA-01 | T-02-01, T-02-02 | Fixed upstream, scrub before write, no credential log | integration + operator smoke | `go test ./tide ./cmd/summer`; CLI capture from isolated PHP server | Pending |
|
||||||
|
| Structural JSON/byte diff and rule assertions | QA-02 | T-02-03 | Normalization rejects wrong token type or malformed date | unit | `go test ./tide -run 'Diff|Normalize|Capture|Scrub'` | Pending |
|
||||||
|
| 154-route manifest and client flows | QA-01 | T-02-02 | Only deterministic test identities committed | manifest audit + PHP replay | manifest validator reports `154/154 recorded`; PHP self-replay zero failures | Pending |
|
||||||
|
| Go replay seam and Postgres state | QA-02, QA-03 | T-02-04 | No false green for pending routes; seed hook allow-list | integration | `go test ./parity` in app module with testcontainers Postgres | Pending |
|
||||||
|
| Phase-wide test plan | QA-03 | T-02-01 to T-02-04 | Regression tests for all listed secure behaviors | unit + race + CLI | `go vet ./... && go test ./... && go test -race ./...` in both modules | Pending |
|
||||||
|
|
||||||
|
## Wave 0 Requirements
|
||||||
|
|
||||||
|
- [ ] First implementation slice adds `tide` package tests and `cmd/summer` command smoke test so record/replay is executable from its first commit.
|
||||||
|
- [ ] App integration slice creates `../fonoteka.go/parity/parity_test.go`, a Postgres-backed synthetic handler, and a pending-route fixture status check before any real Go API port exists.
|
||||||
|
- [ ] Manifest validation rejects duplicate/missing route ids and reports exactly 154 PHP route definitions as the source snapshot.
|
||||||
|
|
||||||
|
## Manual-Only Verifications
|
||||||
|
|
||||||
|
| Behavior | Requirement | Why manual | Test instructions |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Fresh PHP backend capture and self-replay | QA-01, QA-02 | Requires isolated PHP backend and database with local credentials | Start PHP on the agreed port with fresh parity DB, run seed/record, route manifest, Nuxt and MCP flows through proxy, then replay corpus against PHP; save command output and confirm zero failures. |
|
||||||
|
| Nuxt and MCP real sessions | QA-01 | Browser consent and external MCP interaction cannot be fully represented by synthetic handler | Point `NUXT_DEV_BACKEND_ORIGIN` and `FONOTEKA_API_URL` at proxy, perform documented flows, confirm committed fixtures under `nuxt/` and `mcp/` contain placeholders and no credentials. |
|
||||||
|
|
||||||
|
## Validation Sign-Off
|
||||||
|
|
||||||
|
- [ ] Each finalized plan task has an automated verify or an explicit manual gate.
|
||||||
|
- [ ] No three consecutive tasks lack automated feedback.
|
||||||
|
- [ ] Testcontainers Postgres test executes; it is not silently skipped in the phase completion run.
|
||||||
|
- [ ] Root and app module `go vet`, `go test`, and `go test -race` pass.
|
||||||
|
- [ ] PHP self-replay and 154-route coverage evidence are recorded.
|
||||||
|
- [ ] Set `nyquist_compliant: true` only after all mapped checks exist and pass.
|
||||||
|
|
||||||
|
**Approval:** Pending execution evidence.
|
||||||
Reference in New Issue
Block a user