summercms/docs/services/parity-testing.md

title, description, section, order

title	description	section	order
Parity testing	Record the reference backend's responses, broadcasts and vendor calls with tide, replay them against the Go port and diff them after masking IDs and timestamps.	services	150

Parity testing

When a plugin is ported from WinterCMS, its existing clients define the contract: the Go port must answer every request the way the PHP backend did. tide turns that rule into tests. It records the reference backend's real responses as YAML fixtures, replays the same requests against the port, and diffs the responses after masking the values that legitimately differ. WinterCMS has no counterpart.

Flows

A fixture is a tide.Flow: a versioned, ordered list of steps, each a request with its recorded response. You write a spec, a flow with requests only:

version: 1
name: blog-posts
description: List the posts of one blog
steps:
  - id: list-posts
    route_id: GET /api/blog/posts
    request:
      method: GET
      path: /api/blog/posts?page=1

Recording sends each request to the reference backend and fills in the responses. Replaying sends them to the port and compares status, a fixed set of contract headers and the body. JSON bodies are compared structurally after masking id, *_id and *_ids values and *_at timestamps; other bodies byte for byte. Among the headers, Content-Disposition is compared with its dates masked: a CSV download named export-2026-09-17.csv on the recording day replays as export-2026-10-03.csv a few weeks later. Each date must be a real calendar date on both sides, so a renamed file, a broken date or a date that disappears still fails:

ctx := context.Background()
// The reference (PHP) backend and two Go ports. IDs and *_at timestamps
// legitimately differ; the second port changed a title.
reference := backend(`{"data":[{"id":12,"title":"Hello","created_at":"2026-09-30T10:00:00+00:00"}]}`)
defer reference.Close()
port := backend(`{"data":[{"id":3,"title":"Hello","created_at":"2026-10-01T08:30:00+00:00"}]}`)
defer port.Close()
broken := backend(`{"data":[{"id":3,"title":"hello","created_at":"2026-10-01T08:30:00+00:00"}]}`)
defer broken.Close()

raw, err := os.ReadFile("testdata/docs/posts-spec.yaml")
if err != nil {
	fmt.Println(err)
	return
}
spec, err := tide.ParseFlow(raw)
if err != nil {
	fmt.Println(err)
	return
}
// Record the reference once; the flow is what testdata/parity keeps.
flow, err := tide.RecordFlow(ctx, spec, tide.RecordConfig{Target: reference.URL})
if err != nil {
	fmt.Println(err)
	return
}
for _, target := range []string{port.URL, broken.URL} {
	res, err := tide.ReplayFlow(ctx, flow, tide.ReplayConfig{Target: target})
	var mismatch *tide.MismatchError
	if errors.As(err, &mismatch) {
		res = mismatch.Result // a difference is an error carrying the result
	} else if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println("ok:", res.OK)
	for _, step := range res.Steps {
		for _, d := range step.Diffs {
			fmt.Printf("  %s %s: want %s, got %s\n", step.ID, d.Path, d.Expected, d.Actual)
		}
	}
}
// Output:
// ok: true
// ok: false
//   list-posts $.data[0].title: want "Hello", got "hello"

The first port returns other IDs and timestamps and passes; the second changed a title and fails with the JSON path of the difference. A difference makes tide.ReplayFlow return a tide.MismatchError that carries the full tide.Result.

Uploads

A step that uploads a file describes its multipart body as parts instead of a body. Each tide.Part is a text field with a value, or a file field whose bytes live in a file beside the fixture:

    request:
      method: POST
      path: /api/blog/posts/{{id:post}}/photos
      parts:
        - name: caption
          value: Cover
        - name: file
          file: files/cover.png
          content_type: image/png
          sha256: <sha256 of files/cover.png>

Recording and replaying encode the parts in order with one fixed boundary, tide.MultipartBoundary, so the reference backend and the port receive the same bytes. Set BaseDir on tide.RecordConfig and tide.ReplayConfig to the fixture directory the files are read from. tide.LoadFlow refuses a fixture whose part file is missing or no longer matches its sha256, and the file bytes are never copied into the YAML.

Upload responses carry URLs with random parts: the partition and disk name of the original, and the file id in a thumbnail name. Under url and thumb_url keys, the normalizer checks the WinterCMS shape below the uploads prefix (tide.ReplayConfig UploadPrefix, tide.DefaultUploadPrefix by default) and masks only the random parts. The prefix, the thumbnail size and mode and the extension are still compared, so a port that serves /storage/uploads/... or makes 100 by 100 thumbnails instead of 200 by 200 fails the diff.

The parity commands

The summer CLI wraps tide. Run the reference backend and the port on loopback addresses:

summer parity:record --spec testdata/parity/posts.spec.yaml --target http://127.0.0.1:8000 --output testdata/parity/posts.yaml --vars /tmp/parity/vars.yaml
summer parity:replay --fixtures testdata/parity --target http://127.0.0.1:8080 --vars /tmp/parity/vars.yaml

parity:proxy records a real client instead: it runs a reverse proxy in front of the reference backend, and each named session of traffic becomes one fixture. Point the existing frontend at the proxy and click through a feature. The proxy binds to and forwards to loopback addresses only.

Values captured during a flow, such as tokens and created IDs, live in a variables file (--vars) readable only by its owner, and fixtures refer to them as {{name}} placeholders. Recording refuses to write a fixture that still holds a token- or password-shaped value, so credentials do not end up in committed fixtures. Keep the variables file outside the repository.

A route manifest (tide.Manifest) lists a plugin's routes with their auth groups, status and cases; parity:record --manifest records the missing cases in batches, and parity:replay --manifest reports coverage. The Console utilities page lists every flag.

Broadcast goldens

Realtime side effects are part of the contract too. summer parity:broadcasts runs a flow against the reference backend while a fake Centrifugo server (tide.NewCentrifugoRecorder) records the publications the backend sends, and writes them to a golden file:

summer parity:broadcasts --flow testdata/broadcasts/flows/post-lifecycle.yaml --step delete --name deleted --target http://127.0.0.1:8000 --vars /tmp/parity/vars.yaml --out testdata/broadcasts/deleted.yaml

Point the reference backend's Centrifugo API URL at the recorder (127.0.0.1:8424 by default). --step keeps only the publications of one step, running the earlier steps as setup. Timestamps, the actor, the *_at dates inside a published album, a notification's own created_at and id under $.data.payload (the id only when it is a positive integer, as {{id}} unless a captured ID names it) and captured IDs are masked (tide.NormalizePublications), so the Go port's publications, recorded the same way, compare with tide.DiffPublications. A golden with --pending set is recorded but not yet asserted.

On the Go side, the memory realtime driver records publications the same way in tests; see Realtime.

Upstream exchanges

Some routes call outside services: a metadata API, a model vendor, a ticketing system. Two things must hold for them: the port must send the vendor the same requests the reference backend sent, and tests must never call the vendor. Upstream sidecars cover both.

A sidecar sits next to its fixture as <fixture>.upstream.yaml (tide.UpstreamPath) and holds the vendor exchanges of that fixture in recorded order:

version: 1
exchanges:
  - request:
      method: POST
      url: https://api.example.com/v1/items?mode=fast
      headers:
        Authorization: Bearer {{secret:example-token}}
        Content-Type: application/json
        User-Agent: example-client/1.0
      body: '{"name":"widget"}'
    response:
      status: 201
      headers:
        Content-Type: application/json
      body: '{"id":7}'

Only the compared request headers are kept (tide.UpstreamCompareHeaders: User-Agent, Accept, Content-Type, Authorization, X-Api-Key and the model vendors' version headers). A multipart request keeps an ordered list of parts instead of a body, with the SHA-256 of each file instead of its bytes, and a JSON string longer than 1024 characters that is base64 (an uploaded photo, for example) is stored as {{sha256:<hex>}}.

Recording through the proxy

The reference backend's vendor URLs are usually code literals, so the way to capture what it really sends is a recording HTTPS proxy. summer parity:upstream runs one on 127.0.0.1:8425:

summer parity:upstream --ca-dir /tmp/parity/ca --vars /tmp/parity/vars.yaml \
  --script testdata/parity/vendor-script.yaml \
  --out testdata/parity/routes/POST_items_jwt__ok.upstream.yaml

It prints the path of its CA certificate. Start the reference backend with HTTPS_PROXY=http://127.0.0.1:8425 and that certificate as its curl and OpenSSL CA file, run the case, then interrupt the command (Ctrl-C, or SIGTERM when it runs in the background) to write the sidecar. In the default script mode each request is answered from the script file, a list of responses entries (tide.UpstreamScriptResponse: method, host, path and the response to send), so recording needs no real vendor and no real credential; a request no entry matches is answered with status 599 and the sidecar is not written. --mode forward sends each request once to the real vendor through a guarded client instead. Use it only by hand, never in CI.

The rules match the other recorders: the proxy listens on loopback only, the CA key is kept with mode 0600 in --ca-dir outside the fixtures tree, and every value from the variables file is replaced by its {{name}} placeholder. tide.WriteUpstream refuses to write an Authorization or X-Api-Key header that no variable masks.

Replaying offline

In a Go test, load the sidecar with tide.LoadUpstream, build the fake with tide.NewUpstreamFake and hand it to the code under test through fetchguard.WithTransport (see Outbound HTTP). The fake answers each request from the next recorded exchange without dialing and asserts what the port sent: method, scheme, host, path, query (in any order), the compared headers with placeholders expanded from the variables store, and the body (JSON semantically, multipart part by part, base64 payloads by hash). tide.UpstreamFake.Verify then fails on any mismatch, any extra request and any recorded exchange the port never sent.