summercms/docs/services/parity-testing.md

title, description, section, order

title	description	section	order
Parity testing	Record the reference backend's responses and broadcasts with tide, replay them against the Go port and diff them after masking IDs and timestamps.	services	150

Parity testing

When a plugin is ported from WinterCMS, its existing clients define the contract: the Go port must answer every request the way the PHP backend did. tide turns that rule into tests. It records the reference backend's real responses as YAML fixtures, replays the same requests against the port, and diffs the responses after masking the values that legitimately differ. WinterCMS has no counterpart.

Flows

A fixture is a tide.Flow: a versioned, ordered list of steps, each a request with its recorded response. You write a spec, a flow with requests only:

version: 1
name: blog-posts
description: List the posts of one blog
steps:
  - id: list-posts
    route_id: GET /api/blog/posts
    request:
      method: GET
      path: /api/blog/posts?page=1

Recording sends each request to the reference backend and fills in the responses. Replaying sends them to the port and compares status, a fixed set of contract headers and the body. JSON bodies are compared structurally after masking id, *_id and *_ids values and *_at timestamps; other bodies byte for byte:

ctx := context.Background()
// The reference (PHP) backend and two Go ports. IDs and *_at timestamps
// legitimately differ; the second port changed a title.
reference := backend(`{"data":[{"id":12,"title":"Hello","created_at":"2026-09-30T10:00:00+00:00"}]}`)
defer reference.Close()
port := backend(`{"data":[{"id":3,"title":"Hello","created_at":"2026-10-01T08:30:00+00:00"}]}`)
defer port.Close()
broken := backend(`{"data":[{"id":3,"title":"hello","created_at":"2026-10-01T08:30:00+00:00"}]}`)
defer broken.Close()

raw, err := os.ReadFile("testdata/docs/posts-spec.yaml")
if err != nil {
	fmt.Println(err)
	return
}
spec, err := tide.ParseFlow(raw)
if err != nil {
	fmt.Println(err)
	return
}
// Record the reference once; the flow is what testdata/parity keeps.
flow, err := tide.RecordFlow(ctx, spec, tide.RecordConfig{Target: reference.URL})
if err != nil {
	fmt.Println(err)
	return
}
for _, target := range []string{port.URL, broken.URL} {
	res, err := tide.ReplayFlow(ctx, flow, tide.ReplayConfig{Target: target})
	var mismatch *tide.MismatchError
	if errors.As(err, &mismatch) {
		res = mismatch.Result // a difference is an error carrying the result
	} else if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println("ok:", res.OK)
	for _, step := range res.Steps {
		for _, d := range step.Diffs {
			fmt.Printf("  %s %s: want %s, got %s\n", step.ID, d.Path, d.Expected, d.Actual)
		}
	}
}
// Output:
// ok: true
// ok: false
//   list-posts $.data[0].title: want "Hello", got "hello"

The first port returns other IDs and timestamps and passes; the second changed a title and fails with the JSON path of the difference. A difference makes tide.ReplayFlow return a tide.MismatchError that carries the full tide.Result.

Uploads

A step that uploads a file describes its multipart body as parts instead of a body. Each tide.Part is a text field with a value, or a file field whose bytes live in a file beside the fixture:

    request:
      method: POST
      path: /api/blog/posts/{{id:post}}/photos
      parts:
        - name: caption
          value: Cover
        - name: file
          file: files/cover.png
          content_type: image/png
          sha256: <sha256 of files/cover.png>

Recording and replaying encode the parts in order with one fixed boundary, tide.MultipartBoundary, so the reference backend and the port receive the same bytes. Set BaseDir on tide.RecordConfig and tide.ReplayConfig to the fixture directory the files are read from. tide.LoadFlow refuses a fixture whose part file is missing or no longer matches its sha256, and the file bytes are never copied into the YAML.

Upload responses carry URLs with random parts: the partition and disk name of the original, and the file id in a thumbnail name. Under url and thumb_url keys, the normalizer checks the WinterCMS shape below the uploads prefix (tide.ReplayConfig UploadPrefix, tide.DefaultUploadPrefix by default) and masks only the random parts. The prefix, the thumbnail size and mode and the extension are still compared, so a port that serves /storage/uploads/... or makes 100 by 100 thumbnails instead of 200 by 200 fails the diff.

The parity commands

The summer CLI wraps tide. Run the reference backend and the port on loopback addresses:

summer parity:record --spec testdata/parity/posts.spec.yaml --target http://127.0.0.1:8000 --output testdata/parity/posts.yaml --vars /tmp/parity/vars.yaml
summer parity:replay --fixtures testdata/parity --target http://127.0.0.1:8080 --vars /tmp/parity/vars.yaml

parity:proxy records a real client instead: it runs a reverse proxy in front of the reference backend, and each named session of traffic becomes one fixture. Point the existing frontend at the proxy and click through a feature. The proxy binds to and forwards to loopback addresses only.

Values captured during a flow, such as tokens and created IDs, live in a variables file (--vars) readable only by its owner, and fixtures refer to them as {{name}} placeholders. Recording refuses to write a fixture that still holds a token- or password-shaped value, so credentials do not end up in committed fixtures. Keep the variables file outside the repository.

A route manifest (tide.Manifest) lists a plugin's routes with their auth groups, status and cases; parity:record --manifest records the missing cases in batches, and parity:replay --manifest reports coverage. The Console utilities page lists every flag.

Broadcast goldens

Realtime side effects are part of the contract too. summer parity:broadcasts runs a flow against the reference backend while a fake Centrifugo server (tide.NewCentrifugoRecorder) records the publications the backend sends, and writes them to a golden file:

summer parity:broadcasts --flow testdata/broadcasts/flows/post-lifecycle.yaml --step delete --name deleted --target http://127.0.0.1:8000 --vars /tmp/parity/vars.yaml --out testdata/broadcasts/deleted.yaml

Point the reference backend's Centrifugo API URL at the recorder (127.0.0.1:8424 by default). --step keeps only the publications of one step, running the earlier steps as setup. Timestamps, the actor, the *_at dates inside a published album and captured IDs are masked (tide.NormalizePublications), so the Go port's publications, recorded the same way, compare with tide.DiffPublications. A golden with --pending set is recorded but not yet asserted.

On the Go side, the memory realtime driver records publications the same way in tests; see Realtime.