# tide HTTP parity toolkit that records request and response fixtures from a reference backend, replays them against a new one and reports normalized differences. `import "git.golem15.com/golem15/summercms/modules/tide"` ## Overview `tide` is the acceptance-test engine for porting an existing WinterCMS or other PHP backend to SummerCMS: the reference backend's real responses define the contract, and the Go port must reproduce them. It records YAML fixtures (flows of request and response steps) either by driving a spec against a target or by sitting as a loopback reverse proxy in front of the reference backend while a real client uses it, then replays those fixtures against the port and diffs the responses after masking values that legitimately differ, such as IDs and timestamps. It also checks realtime side effects: a fake Centrifugo recorder captures the publications a backend sends while a flow runs, and broadcast golden files hold them normalised for comparison. Upstream sidecars record the calls a backend makes to outside vendors, through a loopback HTTPS recording proxy, and replay them offline through an asserting fake. The `summer` CLI's parity:proxy, parity:record, parity:replay, parity:broadcasts and parity:upstream commands are thin wrappers around this package. It has no WinterCMS counterpart. ## Features - Flow fixtures: `tide.Flow` is a versioned, ordered list of `tide.Step` values, loaded strictly (unknown fields rejected) with `tide.LoadFlow` and `tide.ParseFlow`, and written atomically with `tide.SaveFlow` or `tide.SaveFlowExclusive`. Response bodies can live in sidecar files, confined to the fixture directory and checked against an optional SHA-256 digest. - Recording: `tide.RecordFlow` executes a spec flow against a target URL and fills in the responses, with a bounded body size (`tide.DefaultMaxBody`, 8 MiB). - Multipart uploads: a request can carry `parts` (`tide.Part`: a text field with `value`, or a file field with `file`, `filename`, `content_type` and `sha256`) instead of a `body`. File bytes stay in files beside the fixture (for example `files/cover.png`, read from `tide.RecordConfig` or `tide.ReplayConfig` `BaseDir`) and are never inlined into the YAML. Parts are encoded in declaration order with one fixed boundary, `tide.MultipartBoundary`, so the reference backend and the port receive byte-identical bodies; the encoded Content-Type replaces a recorded multipart Content-Type. Variables expand in part values, never in file bytes. `tide.LoadFlow` fails when a part file is missing or its SHA-256 differs, naming the part, and a request with both `body` and `parts` is invalid. - Recording proxy: `tide.NewProxy` builds a reverse proxy that only binds to and forwards to loopback addresses, groups traffic into named sessions (from the `tide.SessionHeader` request header or a default session) and writes one fixture per complete session on `tide.Proxy.Flush`. - Capture rules: `tide.Rules` (loaded with `tide.LoadRules`) decide which request and response headers are kept per route and which values are captured into variables, from response JSON paths, headers, redirect query strings or form fields. - Variables: `tide.Store` holds captured values such as tokens and IDs in a mode-0600 file, `tide.Store.Expand` substitutes `{{name}}` placeholders before a request is sent, and `tide.ScrubStep` puts placeholders back into fixtures. Scrubbing fails when a step still holds an unclassified token- or password-shaped value, so credentials do not leak into committed fixtures. - Replay and diff: `tide.ReplayFlow` re-sends each step, compares status, a fixed set of contract headers and the body, and returns `tide.Result` with per-step `tide.Diff` entries. JSON bodies are compared structurally after masking `id`, `*_id` and `*_ids` values and `*_at` timestamps; other bodies are compared byte for byte. A `Content-Disposition` header is compared after masking its `YYYY-MM-DD` dates on both sides, so a download named after the day it was made, such as `attachment; filename=export-2026-09-17.csv`, replays on a later day; each masked token must be a real calendar date, and a different name, an invalid date or a date on one side only is still a difference. Uploaded-file URLs under `url` and `thumb_url` keys are compared by shape: under the uploads prefix (`tide.ReplayConfig` `UploadPrefix`, default `tide.DefaultUploadPrefix`, `/storage/app/uploads/public`) an original must be `/xxx/yyy/zzz/` with the partition taken from the disk name, and a thumbnail `/xxx/yyy/zzz/thumb______.`. The partition, disk name and file id are masked; the prefix, size, offsets, mode and extension stay, so a thumbnail of another size or an upload URL under another prefix is still a difference. - Fake Centrifugo: `tide.NewCentrifugoRecorder` returns an `http.Handler` that records every POST to a path ending in `/publish` or `/broadcast` as a `tide.Publication` (method, path, whether `Authorization: apikey ` carried the configured key, JSON body) and answers `{"result":{}}`. Paths ending in `/presence` answer `{"result":{"presence":{}}}`, `/unsubscribe` and `/info` answer `{"result":{}}`, anything else is 404. Bodies are capped at `tide.MaxPublicationBody` (1 MiB). The API key is only compared, never stored. `tide.CentrifugoRecorder.ListenAndServe` binds loopback addresses only, like the recording proxy. - Broadcast goldens: `tide.RecordBroadcasts` runs a flow against a loopback reference backend whose Centrifugo API URL points at a recorder on `tide.DefaultCentrifugoListen` (`127.0.0.1:8424`). With `tide.BroadcastConfig` `Step` set, earlier steps run as setup and only that step's publications are kept. The result is a `tide.BroadcastGolden`, written with `tide.WriteBroadcastGolden` (which refuses token-shaped bodies) and read strictly with `tide.LoadBroadcastGolden`. A golden with `pending` set is recorded but not yet asserted. - Broadcast normalisation: `tide.NormalizePublications` masks only `$.data.timestamp` and `$.data.payload.timestamp` (ISO 8601 with an offset) as `"{{timestamp}}"`, `$.data.payload.actor` (an object of exactly `user_id` and `name`) as `"{{actor}}"`, and values equal to an `id:*` variable of a `tide.Store`: numbers or strings under `id`, `*_id` or `*_ids` keys, and the numeric last segment of a channel name such as `room:12`. Carbon `+00:00` values of `*_at` keys anywhere under `$.data.payload.album` become `"{{datetime}}"`. A notification publication's own row fields are masked too: a Carbon `+00:00` `$.data.payload.created_at` becomes `"{{datetime}}"`, and a positive integer `$.data.payload.id` that no `id:*` variable names becomes a bare `{{id}}` (a captured one keeps `{{id:name}}`); a date of another shape is left as it is, so a format change shows as a difference. A masked number is written as a bare `{{id:name}}`, so a number that becomes a string still differs. A value matching two id variables is an error. `tide.DiffPublications` compares the count, method, path, authorization flag and body (structurally, key order ignored) and reports paths such as `$[0].body.data.payload.id`. - Upstream sidecars: `tide.UpstreamSidecar` (version 1) holds the ordered `tide.UpstreamExchange` values a backend sent to outside services during one fixture, stored as `.upstream.yaml` (`tide.UpstreamPath`). `tide.LoadUpstream` reads one strictly (a missing file wraps `fs.ErrNotExist`). `tide.WriteUpstream` masks every variable value as `{{name}}`, replaces a JSON string longer than 1024 characters that decodes as base64 (also behind a `data:` URL prefix) with `{{sha256:}}`, and refuses an Authorization or X-Api-Key value that no variable masks. A multipart request keeps ordered `tide.UpstreamPart` values: plain fields by value, files by SHA-256. Bodies are capped at `tide.MaxUpstreamBody` (32 MiB). - Upstream fake: `tide.NewUpstreamFake` returns an `http.RoundTripper` that answers each request from the next recorded exchange without dialing and asserts it: method, scheme, host, path, query (order-insensitive), the `tide.UpstreamCompareHeaders` with placeholders expanded from a `tide.Store`, and the body (JSON semantically, multipart part by part, hashed base64 by digest). Mismatch messages never print credential header values. `tide.UpstreamFake.Verify` joins every mismatch, extra request and unconsumed exchange. Hand the fake to code under test with `fetchguard.WithTransport`. - Upstream recording proxy: `tide.NewUpstreamProxy` builds a loopback-only CONNECT proxy (`tide.DefaultUpstreamProxyListen`, `127.0.0.1:8425`) that terminates TLS with per-host certificates signed by a local parity CA (`tide.EnsureParityCA`: ECDSA P-256, certificate mode 0644, key mode 0600, standard library crypto only). In script mode it answers from a `tide.UpstreamScript` (the first unused entry whose method, host and path match; no match answers 599 and fails the recording); in forward mode it sends each request once to the vendor through a `fetchguard.Client` in `fetchguard.PublicOnlyMode`. `tide.UpstreamProxy.Flush` writes the sidecar with `tide.WriteUpstream` and refuses when any request failed. The CA directory and the vars file must be outside the sidecar's directory. - Manifests: `tide.Manifest` lists routes with auth groups, a pending or ported status, cases and fixture paths; `tide.RecordManifest` records missing cases in batches of at most `tide.MaxBatch`, and `tide.ReplayManifest` replays every recorded case into a `tide.Coverage` table. ## Usage Record a flow once against the reference backend, then replay it against the port: ```go spec, err := tide.LoadFlow("testdata/parity/posts.spec.yaml") if err != nil { return err } flow, err := tide.RecordFlow(ctx, spec, tide.RecordConfig{Target: "http://127.0.0.1:8000"}) if err != nil { return err } if err := tide.SaveFlow("testdata/parity/posts.yaml", flow); err != nil { return err } res, err := tide.ReplayFlow(ctx, flow, tide.ReplayConfig{ Target: "http://127.0.0.1:8080", BaseDir: "testdata/parity", }) if err != nil { return err } for _, step := range res.Steps { for _, d := range step.Diffs { fmt.Printf("%s %s: want %s, got %s\n", step.ID, d.Path, d.Expected, d.Actual) } } ``` Record the publications a reference backend sends for one flow step, then compare a new backend's publications with them: ```go golden, err := tide.RecordBroadcasts(ctx, spec, tide.BroadcastConfig{ Target: "http://127.0.0.1:8000", APIKey: "test-only-key", Store: store, Step: "delete", IDs: []string{"id:room", "id:item"}, }) if err != nil { return err } if err := tide.WriteBroadcastGolden("testdata/broadcasts/deleted.yaml", golden); err != nil { return err } // Later, with the new backend publishing to rec (a tide.CentrifugoRecorder): norm, err := tide.NormalizePublications(rec.Publications(), idsOfTheNewBackend) if err != nil { return err } for _, d := range tide.DiffPublications(golden.Publications, norm) { fmt.Printf("%s: want %s, got %s\n", d.Path, d.Expected, d.Actual) } ``` ## API reference | Identifier | Description | |------------|-------------| | `tide.Flow` | Versioned, ordered list of request and response steps; the fixture format. | | `tide.Step` | One request and response pair, with capture and normalizer overrides. | | `tide.LoadFlow` | Reads and validates a flow file. | | `tide.SaveFlow` | Writes a validated flow atomically. | | `tide.RecordFlow` | Executes a spec flow against a target and returns the recorded flow. | | `tide.Request` | The outbound call of a step: method, path, query, headers and a `Body` or multipart `Parts`. | | `tide.Part` | One multipart field: a text value, or a file beside the fixture pinned by its SHA-256. | | `tide.MultipartBoundary` | The fixed boundary multipart request bodies are encoded with. | | `tide.RecordConfig` | Target, client, body cap, vars store, capture rules and part-file directory for `tide.RecordFlow`. | | `tide.ReplayConfig` | Target, client, body cap, vars store, fixture directory and upload URL prefix for `tide.ReplayFlow`. | | `tide.DefaultUploadPrefix` | The default uploads URL prefix, `/storage/app/uploads/public`. | | `tide.ReplayFlow` | Replays a recorded flow against a target and diffs every step. | | `tide.Result` | Replay outcome: overall status and per-step `tide.StepResult` values. | | `tide.Diff` | One structural JSON or byte-level mismatch. | | `tide.MismatchError` | Error that carries a failing `tide.Result`. | | `tide.NewProxy` | Builds the loopback recording reverse proxy from a `tide.ProxyConfig`. | | `tide.Proxy` | The recording proxy: `tide.Proxy.Handler`, `tide.Proxy.ListenAndServe`, `tide.Proxy.Flush`. | | `tide.Rules` | Header keep lists and capture rules for proxy sessions. | | `tide.Store` | Named capture variables, optionally persisted to a private file. | | `tide.OpenStore` | Opens a file-backed store, or a memory-only store for an empty path. | | `tide.Manifest` | Route list with auth groups, status, cases and fixture paths. | | `tide.ValidateManifest` | Checks a manifest in `tide.ModeAllowIncomplete` or `tide.ModeRequireRecorded` mode. | | `tide.RecordManifest` | Records a manifest's seed flow and missing route cases in batches. | | `tide.ReplayManifest` | Replays every recorded route case and builds a coverage table. | | `tide.CentrifugoRecorder` | Fake Centrifugo HTTP API: `tide.CentrifugoRecorder.Publications`, `tide.CentrifugoRecorder.Reset`, `tide.CentrifugoRecorder.ListenAndServe`. | | `tide.NewCentrifugoRecorder` | Builds a recorder from `tide.CentrifugoRecorderOptions` (the expected API key). | | `tide.Publication` | One recorded publish or broadcast request: method, path, authorization flag, JSON body. | | `tide.RecordBroadcasts` | Runs a flow (or one step of it) against a loopback backend and returns its normalised publications as a golden. | | `tide.BroadcastConfig` | Target, recorder address, API key, vars store, step, id variables and settle time for `tide.RecordBroadcasts`. | | `tide.BroadcastGolden` | Versioned broadcast golden: name, flow, optional pending reason, publications. | | `tide.LoadBroadcastGolden` | Reads a golden strictly and checks every body parses. | | `tide.WriteBroadcastGolden` | Writes a golden atomically, refusing token-shaped bodies. | | `tide.NormalizePublications` | Masks timestamps, the actor, album dates and captured ids in publication bodies. | | `tide.DiffPublications` | Structural diff of two publication lists. | | `tide.DefaultCentrifugoListen` | Default recorder address, `127.0.0.1:8424`. | | `tide.UpstreamSidecar` | Versioned list of recorded vendor exchanges for one fixture. | | `tide.UpstreamExchange` | One recorded vendor request and its response. | | `tide.UpstreamRequest` | Method, absolute URL, compared headers and a body or multipart parts. | | `tide.UpstreamPart` | One recorded multipart part: name and value, or filename, content type and SHA-256. | | `tide.UpstreamResponse` | Status, headers and body the fake replays. | | `tide.UpstreamPath` | Maps `x.yaml` to its sidecar path `x.upstream.yaml`. | | `tide.LoadUpstream` | Reads a version-1 sidecar strictly. | | `tide.WriteUpstream` | Masks variables, hashes long base64 strings, refuses unmasked credentials and writes the sidecar (mode 0644). | | `tide.UpstreamCompareHeaders` | The request headers the fake asserts. | | `tide.MaxUpstreamBody` | Body cap for the fake and the recording proxy, 32 MiB. | | `tide.UpstreamFake` | Asserting `http.RoundTripper` over a sidecar: `tide.UpstreamFake.RoundTrip`, `tide.UpstreamFake.Verify`. | | `tide.NewUpstreamFake` | Builds the fake from a sidecar and a variables store. | | `tide.UpstreamProxyConfig` | Listen address, CA directory, output path, mode, script path and vars path for the recording proxy. | | `tide.UpstreamProxy` | The recording proxy: `tide.UpstreamProxy.ListenAndServe`, `tide.UpstreamProxy.Flush`. | | `tide.NewUpstreamProxy` | Validates the config, loads the script and vars, and creates or reuses the parity CA. | | `tide.UpstreamScript` | Hand-authored vendor responses for script mode. | | `tide.UpstreamScriptResponse` | One scripted answer: method, host, path and `tide.UpstreamResponse`; each answers once. | | `tide.EnsureParityCA` | Creates or reuses the parity CA in a directory and returns the certificate path. | | `tide.DefaultUpstreamProxyListen` | Default recording proxy address, `127.0.0.1:8425`. | | `tide.Coverage` | Recorded, passing, failing and unrecorded counts, with table rows and a summary line. | ## CLI commands `summer parity:broadcasts` wraps `tide.RecordBroadcasts` and `tide.WriteBroadcastGolden`: ```sh summer parity:broadcasts \ --flow testdata/broadcasts/flows/item-lifecycle.yaml \ --step delete --name deleted --ids id:room,id:item \ --target http://127.0.0.1:8000 \ --vars /tmp/parity/vars.yaml \ --api-key test-only-key \ --out testdata/broadcasts/deleted.yaml ``` `--listen` defaults to `127.0.0.1:8424`, `--api-key` to `$PARITY_CENTRIFUGO_API_KEY` and `--settle` to `500ms`. `--ids` defaults to the `id:*` variables the flow mentions. `--pending` stores a reason the golden is not asserted yet. The target and the listen address must be loopback, and the vars file must be outside the golden's directory. The command writes: ```yaml version: 1 name: deleted flow: "items/lifecycle#delete" publications: - method: POST path: /api/publish authorization: true body: |- {"channel":"room:{{id:room}}","data":{"event":"deleted","payload":{"id":{{id:item}},"actor":"{{actor}}","timestamp":"{{timestamp}}"},"timestamp":"{{timestamp}}"}} ``` `summer parity:upstream` wraps `tide.NewUpstreamProxy` and `tide.UpstreamProxy.Flush`: ```sh summer parity:upstream \ --ca-dir /tmp/parity/ca \ --vars /tmp/parity/vars.yaml \ --script testdata/parity/vendor-script.yaml \ --out testdata/parity/routes/POST_items_jwt__ok.upstream.yaml ``` `--listen` defaults to `127.0.0.1:8425` and `--mode` to `script`. The command prints the CA certificate path; run the reference backend with `HTTPS_PROXY` pointing at the proxy and that certificate as its CA file, then interrupt the command to write the sidecar. Security rules: the proxy listens on loopback only; the CA key stays mode 0600 in `--ca-dir`, outside the fixtures tree; the vars file is outside the sidecar's directory and every value in it is masked; an unmasked Authorization or X-Api-Key value refuses the write; `--mode forward` reaches the real vendor and is for hand recording only, never CI. ## Dependencies - SummerCMS modules: [fetchguard](../fetchguard/README.md) (the recording proxy's forward mode). - Third-party: `github.com/goccy/go-yaml` (fixture, rules, manifest and sidecar parsing). - Standard library: `net/http`, `net/http/httputil`, `encoding/json`, `crypto/sha256`, `crypto/subtle`, `crypto/tls`, `crypto/x509`, `crypto/ecdsa`, among others. ## Testing ```sh go test ./modules/tide/... ``` The tests run recording, the proxies, replay, the upstream fake and the fake Centrifugo recorder against local `net/http/httptest` servers and use the sample spec in `modules/tide/testdata/`; they need no external services.