summercms/modules/tide/README.md

# tide

HTTP parity toolkit that records request and response fixtures from a reference backend, replays them against a new one and reports normalized differences.

`import "git.golem15.com/golem15/summercms/modules/tide"`

## Overview

`tide` is the acceptance-test engine for porting an existing WinterCMS or other PHP backend to SummerCMS: the reference backend's real responses define the contract, and the Go port must reproduce them. It records YAML fixtures (flows of request and response steps) either by driving a spec against a target or by sitting as a loopback reverse proxy in front of the reference backend while a real client uses it, then replays those fixtures against the port and diffs the responses after masking values that legitimately differ, such as IDs and timestamps. It also checks realtime side effects: a fake Centrifugo recorder captures the publications a backend sends while a flow runs, and broadcast golden files hold them normalised for comparison. The `summer` CLI's parity:proxy, parity:record, parity:replay and parity:broadcasts commands are thin wrappers around this package. It has no WinterCMS counterpart.

## Features

- Flow fixtures: `tide.Flow` is a versioned, ordered list of `tide.Step` values, loaded strictly (unknown fields rejected) with `tide.LoadFlow` and `tide.ParseFlow`, and written atomically with `tide.SaveFlow` or `tide.SaveFlowExclusive`. Response bodies can live in sidecar files, confined to the fixture directory and checked against an optional SHA-256 digest.
- Recording: `tide.RecordFlow` executes a spec flow against a target URL and fills in the responses, with a bounded body size (`tide.DefaultMaxBody`, 8 MiB).
- Multipart uploads: a request can carry `parts` (`tide.Part`: a text field with `value`, or a file field with `file`, `filename`, `content_type` and `sha256`) instead of a `body`. File bytes stay in files beside the fixture (for example `files/cover.png`, read from `tide.RecordConfig` or `tide.ReplayConfig` `BaseDir`) and are never inlined into the YAML. Parts are encoded in declaration order with one fixed boundary, `tide.MultipartBoundary`, so the reference backend and the port receive byte-identical bodies; the encoded Content-Type replaces a recorded multipart Content-Type. Variables expand in part values, never in file bytes. `tide.LoadFlow` fails when a part file is missing or its SHA-256 differs, naming the part, and a request with both `body` and `parts` is invalid.
- Recording proxy: `tide.NewProxy` builds a reverse proxy that only binds to and forwards to loopback addresses, groups traffic into named sessions (from the `tide.SessionHeader` request header or a default session) and writes one fixture per complete session on `tide.Proxy.Flush`.
- Capture rules: `tide.Rules` (loaded with `tide.LoadRules`) decide which request and response headers are kept per route and which values are captured into variables, from response JSON paths, headers, redirect query strings or form fields.
- Variables: `tide.Store` holds captured values such as tokens and IDs in a mode-0600 file, `tide.Store.Expand` substitutes `{{name}}` placeholders before a request is sent, and `tide.ScrubStep` puts placeholders back into fixtures. Scrubbing fails when a step still holds an unclassified token- or password-shaped value, so credentials do not leak into committed fixtures.
- Replay and diff: `tide.ReplayFlow` re-sends each step, compares status, a fixed set of contract headers and the body, and returns `tide.Result` with per-step `tide.Diff` entries. JSON bodies are compared structurally after masking `id`, `*_id` and `*_ids` values and `*_at` timestamps; other bodies are compared byte for byte. Uploaded-file URLs under `url` and `thumb_url` keys are compared by shape: under the uploads prefix (`tide.ReplayConfig` `UploadPrefix`, default `tide.DefaultUploadPrefix`, `/storage/app/uploads/public`) an original must be `<prefix>/xxx/yyy/zzz/<disk_name>` with the partition taken from the disk name, and a thumbnail `<prefix>/xxx/yyy/zzz/thumb_<id>_<w>_<h>_<ox>_<oy>_<mode>.<ext>`. The partition, disk name and file id are masked; the prefix, size, offsets, mode and extension stay, so a thumbnail of another size or an upload URL under another prefix is still a difference.
- Fake Centrifugo: `tide.NewCentrifugoRecorder` returns an `http.Handler` that records every POST to a path ending in `/publish` or `/broadcast` as a `tide.Publication` (method, path, whether `Authorization: apikey <key>` carried the configured key, JSON body) and answers `{"result":{}}`. Paths ending in `/presence` answer `{"result":{"presence":{}}}`, `/unsubscribe` and `/info` answer `{"result":{}}`, anything else is 404. Bodies are capped at `tide.MaxPublicationBody` (1 MiB). The API key is only compared, never stored. `tide.CentrifugoRecorder.ListenAndServe` binds loopback addresses only, like the recording proxy.
- Broadcast goldens: `tide.RecordBroadcasts` runs a flow against a loopback reference backend whose Centrifugo API URL points at a recorder on `tide.DefaultCentrifugoListen` (`127.0.0.1:8424`). With `tide.BroadcastConfig` `Step` set, earlier steps run as setup and only that step's publications are kept. The result is a `tide.BroadcastGolden`, written with `tide.WriteBroadcastGolden` (which refuses token-shaped bodies) and read strictly with `tide.LoadBroadcastGolden`. A golden with `pending` set is recorded but not yet asserted.
- Broadcast normalisation: `tide.NormalizePublications` masks only `$.data.timestamp` and `$.data.payload.timestamp` (ISO 8601 with an offset) as `"{{timestamp}}"`, `$.data.payload.actor` (an object of exactly `user_id` and `name`) as `"{{actor}}"`, and values equal to an `id:*` variable of a `tide.Store`: numbers or strings under `id`, `*_id` or `*_ids` keys, and the numeric last segment of a channel name such as `room:12`. Carbon `+00:00` values of `*_at` keys anywhere under `$.data.payload.album` become `"{{datetime}}"`; a date of another shape is left as it is, so a format change shows as a difference. A masked number is written as a bare `{{id:name}}`, so a number that becomes a string still differs. A value matching two id variables is an error. `tide.DiffPublications` compares the count, method, path, authorization flag and body (structurally, key order ignored) and reports paths such as `$[0].body.data.payload.id`.
- Manifests: `tide.Manifest` lists routes with auth groups, a pending or ported status, cases and fixture paths; `tide.RecordManifest` records missing cases in batches of at most `tide.MaxBatch`, and `tide.ReplayManifest` replays every recorded case into a `tide.Coverage` table.

## Usage

Record a flow once against the reference backend, then replay it against the port:

```go
spec, err := tide.LoadFlow("testdata/parity/posts.spec.yaml")
if err != nil {
	return err
}
flow, err := tide.RecordFlow(ctx, spec, tide.RecordConfig{Target: "http://127.0.0.1:8000"})
if err != nil {
	return err
}
if err := tide.SaveFlow("testdata/parity/posts.yaml", flow); err != nil {
	return err
}

res, err := tide.ReplayFlow(ctx, flow, tide.ReplayConfig{
	Target:  "http://127.0.0.1:8080",
	BaseDir: "testdata/parity",
})
if err != nil {
	return err
}
for _, step := range res.Steps {
	for _, d := range step.Diffs {
		fmt.Printf("%s %s: want %s, got %s\n", step.ID, d.Path, d.Expected, d.Actual)
	}
}
```

Record the publications a reference backend sends for one flow step, then compare a new backend's publications with them:

```go
golden, err := tide.RecordBroadcasts(ctx, spec, tide.BroadcastConfig{
	Target: "http://127.0.0.1:8000",
	APIKey: "test-only-key",
	Store:  store,
	Step:   "delete",
	IDs:    []string{"id:room", "id:item"},
})
if err != nil {
	return err
}
if err := tide.WriteBroadcastGolden("testdata/broadcasts/deleted.yaml", golden); err != nil {
	return err
}

// Later, with the new backend publishing to rec (a tide.CentrifugoRecorder):
norm, err := tide.NormalizePublications(rec.Publications(), idsOfTheNewBackend)
if err != nil {
	return err
}
for _, d := range tide.DiffPublications(golden.Publications, norm) {
	fmt.Printf("%s: want %s, got %s\n", d.Path, d.Expected, d.Actual)
}
```

## API reference

| Identifier | Description |
|------------|-------------|
| `tide.Flow` | Versioned, ordered list of request and response steps; the fixture format. |
| `tide.Step` | One request and response pair, with capture and normalizer overrides. |
| `tide.LoadFlow` | Reads and validates a flow file. |
| `tide.SaveFlow` | Writes a validated flow atomically. |
| `tide.RecordFlow` | Executes a spec flow against a target and returns the recorded flow. |
| `tide.Request` | The outbound call of a step: method, path, query, headers and a `Body` or multipart `Parts`. |
| `tide.Part` | One multipart field: a text value, or a file beside the fixture pinned by its SHA-256. |
| `tide.MultipartBoundary` | The fixed boundary multipart request bodies are encoded with. |
| `tide.RecordConfig` | Target, client, body cap, vars store, capture rules and part-file directory for `tide.RecordFlow`. |
| `tide.ReplayConfig` | Target, client, body cap, vars store, fixture directory and upload URL prefix for `tide.ReplayFlow`. |
| `tide.DefaultUploadPrefix` | The default uploads URL prefix, `/storage/app/uploads/public`. |
| `tide.ReplayFlow` | Replays a recorded flow against a target and diffs every step. |
| `tide.Result` | Replay outcome: overall status and per-step `tide.StepResult` values. |
| `tide.Diff` | One structural JSON or byte-level mismatch. |
| `tide.MismatchError` | Error that carries a failing `tide.Result`. |
| `tide.NewProxy` | Builds the loopback recording reverse proxy from a `tide.ProxyConfig`. |
| `tide.Proxy` | The recording proxy: `tide.Proxy.Handler`, `tide.Proxy.ListenAndServe`, `tide.Proxy.Flush`. |
| `tide.Rules` | Header keep lists and capture rules for proxy sessions. |
| `tide.Store` | Named capture variables, optionally persisted to a private file. |
| `tide.OpenStore` | Opens a file-backed store, or a memory-only store for an empty path. |
| `tide.Manifest` | Route list with auth groups, status, cases and fixture paths. |
| `tide.ValidateManifest` | Checks a manifest in `tide.ModeAllowIncomplete` or `tide.ModeRequireRecorded` mode. |
| `tide.RecordManifest` | Records a manifest's seed flow and missing route cases in batches. |
| `tide.ReplayManifest` | Replays every recorded route case and builds a coverage table. |
| `tide.CentrifugoRecorder` | Fake Centrifugo HTTP API: `tide.CentrifugoRecorder.Publications`, `tide.CentrifugoRecorder.Reset`, `tide.CentrifugoRecorder.ListenAndServe`. |
| `tide.NewCentrifugoRecorder` | Builds a recorder from `tide.CentrifugoRecorderOptions` (the expected API key). |
| `tide.Publication` | One recorded publish or broadcast request: method, path, authorization flag, JSON body. |
| `tide.RecordBroadcasts` | Runs a flow (or one step of it) against a loopback backend and returns its normalised publications as a golden. |
| `tide.BroadcastConfig` | Target, recorder address, API key, vars store, step, id variables and settle time for `tide.RecordBroadcasts`. |
| `tide.BroadcastGolden` | Versioned broadcast golden: name, flow, optional pending reason, publications. |
| `tide.LoadBroadcastGolden` | Reads a golden strictly and checks every body parses. |
| `tide.WriteBroadcastGolden` | Writes a golden atomically, refusing token-shaped bodies. |
| `tide.NormalizePublications` | Masks timestamps, the actor, album dates and captured ids in publication bodies. |
| `tide.DiffPublications` | Structural diff of two publication lists. |
| `tide.DefaultCentrifugoListen` | Default recorder address, `127.0.0.1:8424`. |
| `tide.Coverage` | Recorded, passing, failing and unrecorded counts, with table rows and a summary line. |

## CLI commands

`summer parity:broadcasts` wraps `tide.RecordBroadcasts` and `tide.WriteBroadcastGolden`:

```sh
summer parity:broadcasts \
  --flow testdata/broadcasts/flows/item-lifecycle.yaml \
  --step delete --name deleted --ids id:room,id:item \
  --target http://127.0.0.1:8000 \
  --vars /tmp/parity/vars.yaml \
  --api-key test-only-key \
  --out testdata/broadcasts/deleted.yaml
```

`--listen` defaults to `127.0.0.1:8424`, `--api-key` to `$PARITY_CENTRIFUGO_API_KEY` and `--settle` to `500ms`. `--ids` defaults to the `id:*` variables the flow mentions. `--pending` stores a reason the golden is not asserted yet. The target and the listen address must be loopback, and the vars file must be outside the golden's directory. The command writes:

```yaml
version: 1
name: deleted
flow: "items/lifecycle#delete"
publications:
  - method: POST
    path: /api/publish
    authorization: true
    body: |-
      {"channel":"room:{{id:room}}","data":{"event":"deleted","payload":{"id":{{id:item}},"actor":"{{actor}}","timestamp":"{{timestamp}}"},"timestamp":"{{timestamp}}"}}
```

## Dependencies

- SummerCMS modules: none.
- Third-party: `github.com/goccy/go-yaml` (fixture, rules and manifest parsing).
- Standard library: `net/http`, `net/http/httputil`, `encoding/json`, `crypto/sha256`, `crypto/subtle`, among others.

## Testing

```sh
go test ./modules/tide/...
```

The tests run recording, the proxy, replay and the fake Centrifugo recorder against local `net/http/httptest` servers and use the sample spec in `modules/tide/testdata/`; they need no external services.