summercms/modules/tide/README.md

tide

HTTP parity toolkit that records request and response fixtures from a reference backend, replays them against a new one and reports normalized differences.

import "git.golem15.com/golem15/summercms/modules/tide"

Overview

tide is the acceptance-test engine for porting an existing WinterCMS or other PHP backend to SummerCMS: the reference backend's real responses define the contract, and the Go port must reproduce them. It records YAML fixtures (flows of request and response steps) either by driving a spec against a target or by sitting as a loopback reverse proxy in front of the reference backend while a real client uses it, then replays those fixtures against the port and diffs the responses after masking values that legitimately differ, such as IDs and timestamps. It also checks realtime side effects: a fake Centrifugo recorder captures the publications a backend sends while a flow runs, and broadcast golden files hold them normalised for comparison. The summer CLI's parity:proxy, parity:record, parity:replay and parity:broadcasts commands are thin wrappers around this package. It has no WinterCMS counterpart.

Features

• Flow fixtures: tide.Flow is a versioned, ordered list of tide.Step values, loaded strictly (unknown fields rejected) with tide.LoadFlow and tide.ParseFlow, and written atomically with tide.SaveFlow or tide.SaveFlowExclusive. Response bodies can live in sidecar files, confined to the fixture directory and checked against an optional SHA-256 digest.
• Recording: tide.RecordFlow executes a spec flow against a target URL and fills in the responses, with a bounded body size (tide.DefaultMaxBody, 8 MiB).
• Multipart uploads: a request can carry parts (tide.Part: a text field with value, or a file field with file, filename, content_type and sha256) instead of a body. File bytes stay in files beside the fixture (for example files/cover.png, read from tide.RecordConfig or tide.ReplayConfig BaseDir) and are never inlined into the YAML. Parts are encoded in declaration order with one fixed boundary, tide.MultipartBoundary, so the reference backend and the port receive byte-identical bodies; the encoded Content-Type replaces a recorded multipart Content-Type. Variables expand in part values, never in file bytes. tide.LoadFlow fails when a part file is missing or its SHA-256 differs, naming the part, and a request with both body and parts is invalid.
• Recording proxy: tide.NewProxy builds a reverse proxy that only binds to and forwards to loopback addresses, groups traffic into named sessions (from the tide.SessionHeader request header or a default session) and writes one fixture per complete session on tide.Proxy.Flush.
• Capture rules: tide.Rules (loaded with tide.LoadRules) decide which request and response headers are kept per route and which values are captured into variables, from response JSON paths, headers, redirect query strings or form fields.
• Variables: tide.Store holds captured values such as tokens and IDs in a mode-0600 file, tide.Store.Expand substitutes {{name}} placeholders before a request is sent, and tide.ScrubStep puts placeholders back into fixtures. Scrubbing fails when a step still holds an unclassified token- or password-shaped value, so credentials do not leak into committed fixtures.
• Replay and diff: tide.ReplayFlow re-sends each step, compares status, a fixed set of contract headers and the body, and returns tide.Result with per-step tide.Diff entries. JSON bodies are compared structurally after masking id, *_id and *_ids values and *_at timestamps; other bodies are compared byte for byte. Uploaded-file URLs under url and thumb_url keys are compared by shape: under the uploads prefix (tide.ReplayConfig UploadPrefix, default tide.DefaultUploadPrefix, /storage/app/uploads/public) an original must be <prefix>/xxx/yyy/zzz/<disk_name> with the partition taken from the disk name, and a thumbnail <prefix>/xxx/yyy/zzz/thumb_<id>_<w>_<h>_<ox>_<oy>_<mode>.<ext>. The partition, disk name and file id are masked; the prefix, size, offsets, mode and extension stay, so a thumbnail of another size or an upload URL under another prefix is still a difference.
• Fake Centrifugo: tide.NewCentrifugoRecorder returns an http.Handler that records every POST to a path ending in /publish or /broadcast as a tide.Publication (method, path, whether Authorization: apikey <key> carried the configured key, JSON body) and answers {"result":{}}. Paths ending in /presence answer {"result":{"presence":{}}}, /unsubscribe and /info answer {"result":{}}, anything else is 404. Bodies are capped at tide.MaxPublicationBody (1 MiB). The API key is only compared, never stored. tide.CentrifugoRecorder.ListenAndServe binds loopback addresses only, like the recording proxy.
• Broadcast goldens: tide.RecordBroadcasts runs a flow against a loopback reference backend whose Centrifugo API URL points at a recorder on tide.DefaultCentrifugoListen (127.0.0.1:8424). With tide.BroadcastConfig Step set, earlier steps run as setup and only that step's publications are kept. The result is a tide.BroadcastGolden, written with tide.WriteBroadcastGolden (which refuses token-shaped bodies) and read strictly with tide.LoadBroadcastGolden. A golden with pending set is recorded but not yet asserted.
• Broadcast normalisation: tide.NormalizePublications masks only $.data.timestamp and $.data.payload.timestamp (ISO 8601 with an offset) as "{{timestamp}}", $.data.payload.actor (an object of exactly user_id and name) as "{{actor}}", and values equal to an id:* variable of a tide.Store: numbers or strings under id, *_id or *_ids keys, and the numeric last segment of a channel name such as room:12. Carbon +00:00 values of *_at keys anywhere under $.data.payload.album become "{{datetime}}"; a date of another shape is left as it is, so a format change shows as a difference. A masked number is written as a bare {{id:name}}, so a number that becomes a string still differs. A value matching two id variables is an error. tide.DiffPublications compares the count, method, path, authorization flag and body (structurally, key order ignored) and reports paths such as $[0].body.data.payload.id.
• Manifests: tide.Manifest lists routes with auth groups, a pending or ported status, cases and fixture paths; tide.RecordManifest records missing cases in batches of at most tide.MaxBatch, and tide.ReplayManifest replays every recorded case into a tide.Coverage table.

Usage

Record a flow once against the reference backend, then replay it against the port:

spec, err := tide.LoadFlow("testdata/parity/posts.spec.yaml")
if err != nil {
	return err
}
flow, err := tide.RecordFlow(ctx, spec, tide.RecordConfig{Target: "http://127.0.0.1:8000"})
if err != nil {
	return err
}
if err := tide.SaveFlow("testdata/parity/posts.yaml", flow); err != nil {
	return err
}

res, err := tide.ReplayFlow(ctx, flow, tide.ReplayConfig{
	Target:  "http://127.0.0.1:8080",
	BaseDir: "testdata/parity",
})
if err != nil {
	return err
}
for _, step := range res.Steps {
	for _, d := range step.Diffs {
		fmt.Printf("%s %s: want %s, got %s\n", step.ID, d.Path, d.Expected, d.Actual)
	}
}

Record the publications a reference backend sends for one flow step, then compare a new backend's publications with them:

golden, err := tide.RecordBroadcasts(ctx, spec, tide.BroadcastConfig{
	Target: "http://127.0.0.1:8000",
	APIKey: "test-only-key",
	Store:  store,
	Step:   "delete",
	IDs:    []string{"id:room", "id:item"},
})
if err != nil {
	return err
}
if err := tide.WriteBroadcastGolden("testdata/broadcasts/deleted.yaml", golden); err != nil {
	return err
}

// Later, with the new backend publishing to rec (a tide.CentrifugoRecorder):
norm, err := tide.NormalizePublications(rec.Publications(), idsOfTheNewBackend)
if err != nil {
	return err
}
for _, d := range tide.DiffPublications(golden.Publications, norm) {
	fmt.Printf("%s: want %s, got %s\n", d.Path, d.Expected, d.Actual)
}

API reference

Identifier	Description
tide.Flow	Versioned, ordered list of request and response steps; the fixture format.
tide.Step	One request and response pair, with capture and normalizer overrides.
tide.LoadFlow	Reads and validates a flow file.
tide.SaveFlow	Writes a validated flow atomically.
tide.RecordFlow	Executes a spec flow against a target and returns the recorded flow.
tide.Request	The outbound call of a step: method, path, query, headers and a Body or multipart Parts.
tide.Part	One multipart field: a text value, or a file beside the fixture pinned by its SHA-256.
tide.MultipartBoundary	The fixed boundary multipart request bodies are encoded with.
tide.RecordConfig	Target, client, body cap, vars store, capture rules and part-file directory for tide.RecordFlow.
tide.ReplayConfig	Target, client, body cap, vars store, fixture directory and upload URL prefix for tide.ReplayFlow.
tide.DefaultUploadPrefix	The default uploads URL prefix, /storage/app/uploads/public.
tide.ReplayFlow	Replays a recorded flow against a target and diffs every step.
tide.Result	Replay outcome: overall status and per-step tide.StepResult values.
tide.Diff	One structural JSON or byte-level mismatch.
tide.MismatchError	Error that carries a failing tide.Result.
tide.NewProxy	Builds the loopback recording reverse proxy from a tide.ProxyConfig.
tide.Proxy	The recording proxy: tide.Proxy.Handler, tide.Proxy.ListenAndServe, tide.Proxy.Flush.
tide.Rules	Header keep lists and capture rules for proxy sessions.
tide.Store	Named capture variables, optionally persisted to a private file.
tide.OpenStore	Opens a file-backed store, or a memory-only store for an empty path.
tide.Manifest	Route list with auth groups, status, cases and fixture paths.
tide.ValidateManifest	Checks a manifest in tide.ModeAllowIncomplete or tide.ModeRequireRecorded mode.
tide.RecordManifest	Records a manifest's seed flow and missing route cases in batches.
tide.ReplayManifest	Replays every recorded route case and builds a coverage table.
tide.CentrifugoRecorder	Fake Centrifugo HTTP API: tide.CentrifugoRecorder.Publications, tide.CentrifugoRecorder.Reset, tide.CentrifugoRecorder.ListenAndServe.
tide.NewCentrifugoRecorder	Builds a recorder from tide.CentrifugoRecorderOptions (the expected API key).
tide.Publication	One recorded publish or broadcast request: method, path, authorization flag, JSON body.
tide.RecordBroadcasts	Runs a flow (or one step of it) against a loopback backend and returns its normalised publications as a golden.
tide.BroadcastConfig	Target, recorder address, API key, vars store, step, id variables and settle time for tide.RecordBroadcasts.
tide.BroadcastGolden	Versioned broadcast golden: name, flow, optional pending reason, publications.
tide.LoadBroadcastGolden	Reads a golden strictly and checks every body parses.
tide.WriteBroadcastGolden	Writes a golden atomically, refusing token-shaped bodies.
tide.NormalizePublications	Masks timestamps, the actor, album dates and captured ids in publication bodies.
tide.DiffPublications	Structural diff of two publication lists.
tide.DefaultCentrifugoListen	Default recorder address, 127.0.0.1:8424.
tide.Coverage	Recorded, passing, failing and unrecorded counts, with table rows and a summary line.

CLI commands

summer parity:broadcasts wraps tide.RecordBroadcasts and tide.WriteBroadcastGolden:

summer parity:broadcasts \
  --flow testdata/broadcasts/flows/item-lifecycle.yaml \
  --step delete --name deleted --ids id:room,id:item \
  --target http://127.0.0.1:8000 \
  --vars /tmp/parity/vars.yaml \
  --api-key test-only-key \
  --out testdata/broadcasts/deleted.yaml

--listen defaults to 127.0.0.1:8424, --api-key to $PARITY_CENTRIFUGO_API_KEY and --settle to 500ms. --ids defaults to the id:* variables the flow mentions. --pending stores a reason the golden is not asserted yet. The target and the listen address must be loopback, and the vars file must be outside the golden's directory. The command writes:

version: 1
name: deleted
flow: "items/lifecycle#delete"
publications:
  - method: POST
    path: /api/publish
    authorization: true
    body: |-
      {"channel":"room:{{id:room}}","data":{"event":"deleted","payload":{"id":{{id:item}},"actor":"{{actor}}","timestamp":"{{timestamp}}"},"timestamp":"{{timestamp}}"}}

Dependencies

• SummerCMS modules: none.
• Third-party: github.com/goccy/go-yaml (fixture, rules and manifest parsing).
• Standard library: net/http, net/http/httputil, encoding/json, crypto/sha256, crypto/subtle, among others.

Testing

go test ./modules/tide/...

The tests run recording, the proxy, replay and the fake Centrifugo recorder against local net/http/httptest servers and use the sample spec in modules/tide/testdata/; they need no external services.