feat(13-01): add the prohibited rule and dated-download and notification masks

- lagoon.ValidateRequest supports Laravel 9 prohibited (!required, not
  implicit); with no catalog line its message is validation.prohibited
- tide compares Content-Disposition with real calendar dates masked on both
  sides; a different name, an invalid date or a one-sided date still diffs
- tide.NormalizePublications masks a Carbon +00:00 $.data.payload.created_at
  and an uncaptured positive integer $.data.payload.id as {{id}}
- the album-date test's outside-album sibling moves off payload.created_at,
  which now has its own mask
- READMEs and docs describe the rule and both masks
This commit is contained in:
Jakub Zych
2026-10-03 06:32:46 +02:00
parent 55a4092019
commit 2b94dfd2d2
10 changed files with 243 additions and 9 deletions

View File

@@ -127,6 +127,7 @@ The behaviour follows Laravel:
- The other rules run only when there is a value: they skip an absent attribute, a blank string, `null` under `nullable` and an absent key under `sometimes`. - The other rules run only when there is a value: they skip an absent attribute, a blank string, `null` under `nullable` and an absent key under `sometimes`.
- `min`, `max`, `size` and `between` measure what Laravel measures: the number on an `integer` or `numeric` attribute, compared exactly as a decimal; the number of elements of an array; the size of an uploaded file in kilobytes; otherwise the length in characters, not bytes. Each picks the matching message, such as `max.string` or `max.file`. - `min`, `max`, `size` and `between` measure what Laravel measures: the number on an `integer` or `numeric` attribute, compared exactly as a decimal; the number of elements of an array; the size of an uploaded file in kilobytes; otherwise the length in characters, not bytes. Each picks the matching message, such as `max.string` or `max.file`.
- `email` is PHP's `FILTER_VALIDATE_EMAIL`, the check WinterCMS uses by default, so `user@localhost` is refused. `date` accepts ISO 8601 dates and date-times, `Y/m/d`, `m/d/Y`, `d.m.Y` and `d-m-Y`, and refuses dates that do not exist. `after_or_equal` and `before_or_equal` take a date, `today`, `tomorrow`, `yesterday` or `now` (in UTC), or the name of another field. - `email` is PHP's `FILTER_VALIDATE_EMAIL`, the check WinterCMS uses by default, so `user@localhost` is refused. `date` accepts ISO 8601 dates and date-times, `Y/m/d`, `m/d/Y`, `d.m.Y` and `d-m-Y`, and refuses dates that do not exist. `after_or_equal` and `before_or_equal` take a date, `today`, `tomorrow`, `yesterday` or `now` (in UTC), or the name of another field.
- `prohibited` refuses a field the endpoint must never accept: it fails for any value that `required` would accept, including `0`, `false` and a non-empty array. Like the other non-implicit rules it never runs for an absent key or a blank string, and `null` or an empty array passes. Laravel 9's catalogs have no line for it, so the message is the key itself, `validation.prohibited`, as the PHP endpoint answers.
- `exists:table,column` counts matching rows in the database, so it needs the transaction handle. Like Laravel it does not skip soft-deleted rows, and it does not run once the attribute already has a message. - `exists:table,column` counts matching rows in the database, so it needs the transaction handle. Like Laravel it does not skip soft-deleted rows, and it does not run once the attribute already has a message.
- File rules (`file`, `image`, `mimes` and the size rules) take a `lagoon.UploadedFile`; `lagoon.UploadedFileFromHeader` builds one from a parsed multipart part. The type is read from the file content, not from its name. - File rules (`file`, `image`, `mimes` and the size rules) take a `lagoon.UploadedFile`; `lagoon.UploadedFileFromHeader` builds one from a parsed multipart part. The type is read from the file content, not from its name.

View File

@@ -24,7 +24,7 @@ steps:
path: /api/blog/posts?page=1 path: /api/blog/posts?page=1
``` ```
Recording sends each request to the reference backend and fills in the responses. Replaying sends them to the port and compares status, a fixed set of contract headers and the body. JSON bodies are compared structurally after masking `id`, `*_id` and `*_ids` values and `*_at` timestamps; other bodies byte for byte: Recording sends each request to the reference backend and fills in the responses. Replaying sends them to the port and compares status, a fixed set of contract headers and the body. JSON bodies are compared structurally after masking `id`, `*_id` and `*_ids` values and `*_at` timestamps; other bodies byte for byte. Among the headers, `Content-Disposition` is compared with its dates masked: a CSV download named `export-2026-09-17.csv` on the recording day replays as `export-2026-10-03.csv` a few weeks later. Each date must be a real calendar date on both sides, so a renamed file, a broken date or a date that disappears still fails:
```go src=modules/tide/example_test.go#ExampleReplayFlow ```go src=modules/tide/example_test.go#ExampleReplayFlow
ctx := context.Background() ctx := context.Background()
@@ -121,6 +121,6 @@ Realtime side effects are part of the contract too. `summer parity:broadcasts` r
summer parity:broadcasts --flow testdata/broadcasts/flows/post-lifecycle.yaml --step delete --name deleted --target http://127.0.0.1:8000 --vars /tmp/parity/vars.yaml --out testdata/broadcasts/deleted.yaml summer parity:broadcasts --flow testdata/broadcasts/flows/post-lifecycle.yaml --step delete --name deleted --target http://127.0.0.1:8000 --vars /tmp/parity/vars.yaml --out testdata/broadcasts/deleted.yaml
``` ```
Point the reference backend's Centrifugo API URL at the recorder (`127.0.0.1:8424` by default). `--step` keeps only the publications of one step, running the earlier steps as setup. Timestamps, the actor, the `*_at` dates inside a published album and captured IDs are masked (`tide.NormalizePublications`), so the Go port's publications, recorded the same way, compare with `tide.DiffPublications`. A golden with `--pending` set is recorded but not yet asserted. Point the reference backend's Centrifugo API URL at the recorder (`127.0.0.1:8424` by default). `--step` keeps only the publications of one step, running the earlier steps as setup. Timestamps, the actor, the `*_at` dates inside a published album, a notification's own `created_at` and `id` under `$.data.payload` (the id only when it is a positive integer, as `{{id}}` unless a captured ID names it) and captured IDs are masked (`tide.NormalizePublications`), so the Go port's publications, recorded the same way, compare with `tide.DiffPublications`. A golden with `--pending` set is recorded but not yet asserted.
On the Go side, the memory realtime driver records publications the same way in tests; see [Realtime](realtime.md). On the Go side, the memory realtime driver records publications the same way in tests; see [Realtime](realtime.md).

View File

@@ -18,7 +18,7 @@ Postgres data layer: the shared GORM connection, per-plugin migrations, model he
- Per-plugin migrations: `lagoon.Migrate` runs the framework's `system_files` set (`attach.Migrations`), backend admin identity set (`lagoon.BackendAdminMigrations`), `deferred_bindings` set (`lagoon.DeferredBindingMigrations`, under the `lagoon.DeferredHistoryID` history) and job-queue set (`lagoon.QueueMigrations`: River's schema pinned at `lagoon.RiverSchemaVersion`, then the `lagoon.JobsTable` record table, under the `lagoon.QueueHistoryID` history), then every `pact.HasMigrations` set in plugin activation order, each in its own `summer_migrations_<plugin_id>` history table (`lagoon.HistoryTableName`). `lagoon.RollbackLast` and `lagoon.Status` cover rollback and history. - Per-plugin migrations: `lagoon.Migrate` runs the framework's `system_files` set (`attach.Migrations`), backend admin identity set (`lagoon.BackendAdminMigrations`), `deferred_bindings` set (`lagoon.DeferredBindingMigrations`, under the `lagoon.DeferredHistoryID` history) and job-queue set (`lagoon.QueueMigrations`: River's schema pinned at `lagoon.RiverSchemaVersion`, then the `lagoon.JobsTable` record table, under the `lagoon.QueueHistoryID` history), then every `pact.HasMigrations` set in plugin activation order, each in its own `summer_migrations_<plugin_id>` history table (`lagoon.HistoryTableName`). `lagoon.RollbackLast` and `lagoon.Status` cover rollback and history.
- Mass assignment: `lagoon.Fill` copies only allow-listed keys onto a model by GORM column name and silently drops the rest, logging each dropped key once outside production. A `json.Number` (from a decoder using `UseNumber`) fills integer, unsigned and float fields. A value that does not fit its column (a fraction, an exponent or an overflow for an integer field, or a value of the wrong type) is a `lagoon.FillTypeError` naming the key, so a caller can answer it as a validation failure on that field. `lagoon.HasFillable` and `lagoon.HasHidden` are the Go forms of `$fillable` and `$hidden`. - Mass assignment: `lagoon.Fill` copies only allow-listed keys onto a model by GORM column name and silently drops the rest, logging each dropped key once outside production. A `json.Number` (from a decoder using `UseNumber`) fills integer, unsigned and float fields. A value that does not fit its column (a fraction, an exponent or an overflow for an integer field, or a value of the wrong type) is a `lagoon.FillTypeError` naming the key, so a caller can answer it as a validation failure on that field. `lagoon.HasFillable` and `lagoon.HasHidden` are the Go forms of `$fillable` and `$hidden`.
- Validation: `lagoon.Validate` (where `required` fails on a zero date or time) accepts Laravel-style rule strings (`required`, `nullable`, `integer`, `numeric`, `between`, `min`, `max`, `in`, `unique`, `boolean`, `email`, `confirmed`, `different`, `mimes`) and returns a field-to-messages map, translated through phrasebook when a translator is given. Unknown rule tokens are an error. A failed numeric range reports the bound that failed: the `min` message below the lower bound, the `max` message above the upper one, and the numeric `between` message when the bound came from `between`. - Validation: `lagoon.Validate` (where `required` fails on a zero date or time) accepts Laravel-style rule strings (`required`, `nullable`, `integer`, `numeric`, `between`, `min`, `max`, `in`, `unique`, `boolean`, `email`, `confirmed`, `different`, `mimes`) and returns a field-to-messages map, translated through phrasebook when a translator is given. Unknown rule tokens are an error. A failed numeric range reports the bound that failed: the `min` message below the lower bound, the `max` message above the upper one, and the numeric `between` message when the bound came from `between`.
- Request validation: `lagoon.ValidateRequest` reproduces Laravel 9 request validation for ported API endpoints, so a 422 body matches the PHP one message for message. It takes the decoded input and an ordered `lagoon.RequestRule` table (attribute names may hold `*` wildcards, expanded against the input to `posts.0.title`), runs the rules of each attribute in order and stops an attribute after a failed implicit rule (`required`, `present`, `filled`, `accepted`) or, under `bail`, after any failure. A non-implicit rule is skipped for an absent attribute, a blank string, a null value under `nullable` and an absent key under `sometimes`. Supported rules: `required`, `present`, `filled`, `accepted`, `nullable`, `sometimes`, `bail`, `array`, `string`, `integer`, `numeric`, `boolean`, `email` (PHP `FILTER_VALIDATE_EMAIL`, WinterCMS's default), `url`, `date`, `after`, `after_or_equal`, `before`, `before_or_equal` (a date, a relative word such as `tomorrow`, or another field), `exists:table,column`, `regex`, `not_regex`, `in`, `not_in`, `file`, `image`, `mimes`, `min`, `max`, `size` and `between`, plus closure rules built with `lagoon.CustomRule`. The size rules compare the number under `numeric` or `integer` (exactly, as decimals), the element count of an array, kilobytes of a `lagoon.UploadedFile`, and otherwise the length in characters, and pick the matching message. Messages come from the `lagoon::validation` catalog in the request locale; `lagoon.ErrorKeys` gives the attribute order of PHP's message bag. - Request validation: `lagoon.ValidateRequest` reproduces Laravel 9 request validation for ported API endpoints, so a 422 body matches the PHP one message for message. It takes the decoded input and an ordered `lagoon.RequestRule` table (attribute names may hold `*` wildcards, expanded against the input to `posts.0.title`), runs the rules of each attribute in order and stops an attribute after a failed implicit rule (`required`, `present`, `filled`, `accepted`) or, under `bail`, after any failure. A non-implicit rule is skipped for an absent attribute, a blank string, a null value under `nullable` and an absent key under `sometimes`. Supported rules: `required`, `present`, `filled`, `accepted`, `nullable`, `sometimes`, `bail`, `array`, `string`, `integer`, `numeric`, `boolean`, `email` (PHP `FILTER_VALIDATE_EMAIL`, WinterCMS's default), `url`, `date`, `after`, `after_or_equal`, `before`, `before_or_equal` (a date, a relative word such as `tomorrow`, or another field), `exists:table,column`, `regex`, `not_regex`, `in`, `not_in`, `file`, `image`, `mimes`, `min`, `max`, `size`, `between` and `prohibited` (fails for any value `required` would accept, `0` and `false` included; not implicit, so an absent key, `null`, a blank string and an empty array pass, and with no catalog line its message is the key `validation.prohibited`), plus closure rules built with `lagoon.CustomRule`. The size rules compare the number under `numeric` or `integer` (exactly, as decimals), the element count of an array, kilobytes of a `lagoon.UploadedFile`, and otherwise the length in characters, and pick the matching message. Messages come from the `lagoon::validation` catalog in the request locale; `lagoon.ErrorKeys` gives the attribute order of PHP's message bag.
- Safe ordering: `lagoon.OrderBy` appends an ORDER BY only for an allow-listed column and an `asc` or `desc` direction, and `lagoon.Collate` adds a validated `COLLATE` clause for language-specific text order (for example the ICU collation `pl-x-icu`); lagoon puts no requirement on the database's default locale. - Safe ordering: `lagoon.OrderBy` appends an ORDER BY only for an allow-listed column and an `asc` or `desc` direction, and `lagoon.Collate` adds a validated `COLLATE` clause for language-specific text order (for example the ICU collation `pl-x-icu`); lagoon puts no requirement on the database's default locale.
- Pagination: `lagoon.Paginate` builds a `lagoon.Page` with `data` and `meta` (`current_page`, `last_page`, `per_page`, `total`). - Pagination: `lagoon.Paginate` builds a `lagoon.Page` with `data` and `meta` (`current_page`, `last_page`, `per_page`, `total`).
- Date and time columns: `lagoon.Date` (a `DATE` column, JSON `"2026-10-02"`) and `lagoon.TimeOfDay` (a `TIME` column, JSON `"14:30:00"`) implement `sql.Scanner`, `driver.Valuer`, JSON and text marshalling, and store NULL for their zero value; `*lagoon.Date` and `*lagoon.TimeOfDay` are the nullable variants, next to `time.Time` and `*time.Time` for `timestamptz`. Build them with `lagoon.NewDate`, `lagoon.DateOf`, `lagoon.ParseDate`, `lagoon.NewTimeOfDay` and `lagoon.ParseTimeOfDay`. `lagoon.Fill` fills all six from JSON strings (RFC 3339 for `time.Time`) through their text unmarshalling, after every conversion it already made. Behaviour change: `required` now treats a zero `time.Time`, `lagoon.Date` or `lagoon.TimeOfDay` (or a pointer to one) as empty, so declare optional dates as pointer fields. - Date and time columns: `lagoon.Date` (a `DATE` column, JSON `"2026-10-02"`) and `lagoon.TimeOfDay` (a `TIME` column, JSON `"14:30:00"`) implement `sql.Scanner`, `driver.Valuer`, JSON and text marshalling, and store NULL for their zero value; `*lagoon.Date` and `*lagoon.TimeOfDay` are the nullable variants, next to `time.Time` and `*time.Time` for `timestamptz`. Build them with `lagoon.NewDate`, `lagoon.DateOf`, `lagoon.ParseDate`, `lagoon.NewTimeOfDay` and `lagoon.ParseTimeOfDay`. `lagoon.Fill` fills all six from JSON strings (RFC 3339 for `time.Time`) through their text unmarshalling, after every conversion it already made. Behaviour change: `required` now treats a zero `time.Time`, `lagoon.Date` or `lagoon.TimeOfDay` (or a pointer to one) as empty, so declare optional dates as pointer fields.

View File

@@ -664,3 +664,52 @@ func TestValidateRequestExistsRule(t *testing.T) {
t.Error("a missing table must be an error for arrays too") t.Error("a missing table must be an error for arrays too")
} }
} }
// TestValidateRequestProhibited ports Laravel 9's prohibited rule: it is
// !validateRequired and not implicit, so an absent key and a blank string
// are never validated, null and an empty array pass, and any value required
// would accept fails, 0 and false included. Neither catalog has a
// validation.prohibited line, so the message is the key itself.
func TestValidateRequestProhibited(t *testing.T) {
rules := []RequestRule{{Field: "condition", Rules: ParseRules("prohibited")}}
pass := map[string]map[string]any{
"absent": {},
"null": {"condition": nil},
"empty string": {"condition": ""},
"blank string": {"condition": " "},
"empty array": {"condition": []any{}},
"empty map": {"condition": map[string]any{}},
}
for name, input := range pass {
if got := mustValidate(t, inLocale("pl"), input, rules); len(got) != 0 {
t.Errorf("%s: errors = %v, want none", name, got)
}
}
fail := map[string]any{
"string": "VG",
"zero": json.Number("0"),
"float zero": float64(0),
"false": false,
"true": true,
"non-empty array": []any{"x"},
"non-empty map": map[string]any{"a": 1},
}
for name, value := range fail {
for _, loc := range []string{"pl", "en"} {
got := mustValidate(t, inLocale(loc), map[string]any{"condition": value}, rules)
want := map[string][]string{"condition": {"validation.prohibited"}}
if !reflect.DeepEqual(got, want) {
t.Errorf("%s (%s): errors = %#v, want %#v", name, loc, got, want)
}
}
}
// Next to other rules, prohibited fails like any non-implicit rule.
mixed := []RequestRule{
{Field: "name", Rules: ParseRules("required|string")},
{Field: "shelf", Rules: ParseRules("prohibited")},
}
got := mustValidate(t, inLocale("en"), map[string]any{"name": "Kind of Blue", "shelf": "A1"}, mixed)
if want := map[string][]string{"shelf": {"validation.prohibited"}}; !reflect.DeepEqual(got, want) {
t.Errorf("mixed = %#v, want %#v", got, want)
}
}

View File

@@ -34,7 +34,7 @@ var imageExtensions = []string{"jpg", "jpeg", "png", "gif", "bmp", "svg", "webp"
// parameter count: -1 any number (at least one), 0 none. // parameter count: -1 any number (at least one), 0 none.
var requestRuleArity = map[string]int{ var requestRuleArity = map[string]int{
"required": 0, "present": 0, "filled": 0, "accepted": 0, "required": 0, "present": 0, "filled": 0, "accepted": 0,
"nullable": 0, "sometimes": 0, "bail": 0, "nullable": 0, "sometimes": 0, "bail": 0, "prohibited": 0,
"array": -2, "string": 0, "integer": 0, "numeric": 0, "boolean": 0, "array": -2, "string": 0, "integer": 0, "numeric": 0, "boolean": 0,
"email": 0, "url": 0, "date": 0, "email": 0, "url": 0, "date": 0,
"after": 1, "after_or_equal": 1, "before": 1, "before_or_equal": 1, "after": 1, "after_or_equal": 1, "before": 1, "before_or_equal": 1,
@@ -278,6 +278,10 @@ func (v *requestValidator) passes(rule Rule, attr string, value any, present boo
return !present || validateRequired(value), nil return !present || validateRequired(value), nil
case "accepted": case "accepted":
return validateRequired(value) && isAccepted(value), nil return validateRequired(value) && isAccepted(value), nil
case "prohibited":
// Laravel 9 validateProhibited: !validateRequired. Not implicit, so it
// never runs for an absent key or a blank string.
return !validateRequired(value), nil
case "nullable", "sometimes", "bail": case "nullable", "sometimes", "bail":
return true, nil return true, nil
case "array": case "array":

View File

@@ -16,10 +16,10 @@ HTTP parity toolkit that records request and response fixtures from a reference
- Recording proxy: `tide.NewProxy` builds a reverse proxy that only binds to and forwards to loopback addresses, groups traffic into named sessions (from the `tide.SessionHeader` request header or a default session) and writes one fixture per complete session on `tide.Proxy.Flush`. - Recording proxy: `tide.NewProxy` builds a reverse proxy that only binds to and forwards to loopback addresses, groups traffic into named sessions (from the `tide.SessionHeader` request header or a default session) and writes one fixture per complete session on `tide.Proxy.Flush`.
- Capture rules: `tide.Rules` (loaded with `tide.LoadRules`) decide which request and response headers are kept per route and which values are captured into variables, from response JSON paths, headers, redirect query strings or form fields. - Capture rules: `tide.Rules` (loaded with `tide.LoadRules`) decide which request and response headers are kept per route and which values are captured into variables, from response JSON paths, headers, redirect query strings or form fields.
- Variables: `tide.Store` holds captured values such as tokens and IDs in a mode-0600 file, `tide.Store.Expand` substitutes `{{name}}` placeholders before a request is sent, and `tide.ScrubStep` puts placeholders back into fixtures. Scrubbing fails when a step still holds an unclassified token- or password-shaped value, so credentials do not leak into committed fixtures. - Variables: `tide.Store` holds captured values such as tokens and IDs in a mode-0600 file, `tide.Store.Expand` substitutes `{{name}}` placeholders before a request is sent, and `tide.ScrubStep` puts placeholders back into fixtures. Scrubbing fails when a step still holds an unclassified token- or password-shaped value, so credentials do not leak into committed fixtures.
- Replay and diff: `tide.ReplayFlow` re-sends each step, compares status, a fixed set of contract headers and the body, and returns `tide.Result` with per-step `tide.Diff` entries. JSON bodies are compared structurally after masking `id`, `*_id` and `*_ids` values and `*_at` timestamps; other bodies are compared byte for byte. Uploaded-file URLs under `url` and `thumb_url` keys are compared by shape: under the uploads prefix (`tide.ReplayConfig` `UploadPrefix`, default `tide.DefaultUploadPrefix`, `/storage/app/uploads/public`) an original must be `<prefix>/xxx/yyy/zzz/<disk_name>` with the partition taken from the disk name, and a thumbnail `<prefix>/xxx/yyy/zzz/thumb_<id>_<w>_<h>_<ox>_<oy>_<mode>.<ext>`. The partition, disk name and file id are masked; the prefix, size, offsets, mode and extension stay, so a thumbnail of another size or an upload URL under another prefix is still a difference. - Replay and diff: `tide.ReplayFlow` re-sends each step, compares status, a fixed set of contract headers and the body, and returns `tide.Result` with per-step `tide.Diff` entries. JSON bodies are compared structurally after masking `id`, `*_id` and `*_ids` values and `*_at` timestamps; other bodies are compared byte for byte. A `Content-Disposition` header is compared after masking its `YYYY-MM-DD` dates on both sides, so a download named after the day it was made, such as `attachment; filename=export-2026-09-17.csv`, replays on a later day; each masked token must be a real calendar date, and a different name, an invalid date or a date on one side only is still a difference. Uploaded-file URLs under `url` and `thumb_url` keys are compared by shape: under the uploads prefix (`tide.ReplayConfig` `UploadPrefix`, default `tide.DefaultUploadPrefix`, `/storage/app/uploads/public`) an original must be `<prefix>/xxx/yyy/zzz/<disk_name>` with the partition taken from the disk name, and a thumbnail `<prefix>/xxx/yyy/zzz/thumb_<id>_<w>_<h>_<ox>_<oy>_<mode>.<ext>`. The partition, disk name and file id are masked; the prefix, size, offsets, mode and extension stay, so a thumbnail of another size or an upload URL under another prefix is still a difference.
- Fake Centrifugo: `tide.NewCentrifugoRecorder` returns an `http.Handler` that records every POST to a path ending in `/publish` or `/broadcast` as a `tide.Publication` (method, path, whether `Authorization: apikey <key>` carried the configured key, JSON body) and answers `{"result":{}}`. Paths ending in `/presence` answer `{"result":{"presence":{}}}`, `/unsubscribe` and `/info` answer `{"result":{}}`, anything else is 404. Bodies are capped at `tide.MaxPublicationBody` (1 MiB). The API key is only compared, never stored. `tide.CentrifugoRecorder.ListenAndServe` binds loopback addresses only, like the recording proxy. - Fake Centrifugo: `tide.NewCentrifugoRecorder` returns an `http.Handler` that records every POST to a path ending in `/publish` or `/broadcast` as a `tide.Publication` (method, path, whether `Authorization: apikey <key>` carried the configured key, JSON body) and answers `{"result":{}}`. Paths ending in `/presence` answer `{"result":{"presence":{}}}`, `/unsubscribe` and `/info` answer `{"result":{}}`, anything else is 404. Bodies are capped at `tide.MaxPublicationBody` (1 MiB). The API key is only compared, never stored. `tide.CentrifugoRecorder.ListenAndServe` binds loopback addresses only, like the recording proxy.
- Broadcast goldens: `tide.RecordBroadcasts` runs a flow against a loopback reference backend whose Centrifugo API URL points at a recorder on `tide.DefaultCentrifugoListen` (`127.0.0.1:8424`). With `tide.BroadcastConfig` `Step` set, earlier steps run as setup and only that step's publications are kept. The result is a `tide.BroadcastGolden`, written with `tide.WriteBroadcastGolden` (which refuses token-shaped bodies) and read strictly with `tide.LoadBroadcastGolden`. A golden with `pending` set is recorded but not yet asserted. - Broadcast goldens: `tide.RecordBroadcasts` runs a flow against a loopback reference backend whose Centrifugo API URL points at a recorder on `tide.DefaultCentrifugoListen` (`127.0.0.1:8424`). With `tide.BroadcastConfig` `Step` set, earlier steps run as setup and only that step's publications are kept. The result is a `tide.BroadcastGolden`, written with `tide.WriteBroadcastGolden` (which refuses token-shaped bodies) and read strictly with `tide.LoadBroadcastGolden`. A golden with `pending` set is recorded but not yet asserted.
- Broadcast normalisation: `tide.NormalizePublications` masks only `$.data.timestamp` and `$.data.payload.timestamp` (ISO 8601 with an offset) as `"{{timestamp}}"`, `$.data.payload.actor` (an object of exactly `user_id` and `name`) as `"{{actor}}"`, and values equal to an `id:*` variable of a `tide.Store`: numbers or strings under `id`, `*_id` or `*_ids` keys, and the numeric last segment of a channel name such as `room:12`. Carbon `+00:00` values of `*_at` keys anywhere under `$.data.payload.album` become `"{{datetime}}"`; a date of another shape is left as it is, so a format change shows as a difference. A masked number is written as a bare `{{id:name}}`, so a number that becomes a string still differs. A value matching two id variables is an error. `tide.DiffPublications` compares the count, method, path, authorization flag and body (structurally, key order ignored) and reports paths such as `$[0].body.data.payload.id`. - Broadcast normalisation: `tide.NormalizePublications` masks only `$.data.timestamp` and `$.data.payload.timestamp` (ISO 8601 with an offset) as `"{{timestamp}}"`, `$.data.payload.actor` (an object of exactly `user_id` and `name`) as `"{{actor}}"`, and values equal to an `id:*` variable of a `tide.Store`: numbers or strings under `id`, `*_id` or `*_ids` keys, and the numeric last segment of a channel name such as `room:12`. Carbon `+00:00` values of `*_at` keys anywhere under `$.data.payload.album` become `"{{datetime}}"`. A notification publication's own row fields are masked too: a Carbon `+00:00` `$.data.payload.created_at` becomes `"{{datetime}}"`, and a positive integer `$.data.payload.id` that no `id:*` variable names becomes a bare `{{id}}` (a captured one keeps `{{id:name}}`); a date of another shape is left as it is, so a format change shows as a difference. A masked number is written as a bare `{{id:name}}`, so a number that becomes a string still differs. A value matching two id variables is an error. `tide.DiffPublications` compares the count, method, path, authorization flag and body (structurally, key order ignored) and reports paths such as `$[0].body.data.payload.id`.
- Manifests: `tide.Manifest` lists routes with auth groups, a pending or ported status, cases and fixture paths; `tide.RecordManifest` records missing cases in batches of at most `tide.MaxBatch`, and `tide.ReplayManifest` replays every recorded case into a `tide.Coverage` table. - Manifests: `tide.Manifest` lists routes with auth groups, a pending or ported status, cases and fixture paths; `tide.RecordManifest` records missing cases in batches of at most `tide.MaxBatch`, and `tide.ReplayManifest` replays every recorded case into a `tide.Coverage` table.
## Usage ## Usage

View File

@@ -275,7 +275,27 @@ func (n *normalizer) lookup(v string) (string, bool) {
} }
} }
// positiveIntRe is a JSON number literal holding a positive integer.
var positiveIntRe = regexp.MustCompile(`^[1-9][0-9]*$`)
func (n *normalizer) walk(path, key string, v *onode) *onode { func (n *normalizer) walk(path, key string, v *onode) *onode {
// A notification publication carries the new row's id and created_at
// under $.data.payload. A Carbon +00:00 created_at is masked like an album
// date; a positive integer id keeps its captured variable's placeholder
// and is {{id}} when no captured variable names it. Any other shape at
// these paths stays visible.
switch {
case path == "$.data.payload.created_at" && v.kind == kindString && carbonOffsetRe.MatchString(v.str):
return &onode{kind: kindString, str: "{{datetime}}"}
case path == "$.data.payload.id" && v.kind == kindNumber && positiveIntRe.MatchString(v.str):
if name, ok := n.lookup(v.str); ok {
return &onode{kind: kindPlaceholder, str: name}
}
if len(n.ids[v.str]) == 0 {
return &onode{kind: kindPlaceholder, str: "id"}
}
return v
}
switch path { switch path {
case "$.data.timestamp", "$.data.payload.timestamp": case "$.data.timestamp", "$.data.payload.timestamp":
if v.kind == kindString && isoOffsetRe.MatchString(v.str) { if v.kind == kindString && isoOffsetRe.MatchString(v.str) {

View File

@@ -6,8 +6,10 @@ import (
"fmt" "fmt"
"mime" "mime"
"net/http" "net/http"
"regexp"
"strconv" "strconv"
"strings" "strings"
"time"
"unicode" "unicode"
"unicode/utf8" "unicode/utf8"
) )
@@ -81,6 +83,9 @@ func compareHeaders(want, got, extra map[string]string) []Diff {
if gv == wv { if gv == wv {
continue continue
} }
if ck == "Content-Disposition" && sameDispositionButDates(wv, gv) {
continue
}
if gv == "" { if gv == "" {
gv = "<missing>" gv = "<missing>"
} }
@@ -290,3 +295,32 @@ func quotePrintable(b []byte) string {
buf.WriteByte('"') buf.WriteByte('"')
return buf.String() return buf.String()
} }
// dispositionDateRe finds YYYY-MM-DD tokens in a Content-Disposition value,
// such as the date in a download name built from the current day.
var dispositionDateRe = regexp.MustCompile(`\b\d{4}-\d{2}-\d{2}\b`)
// maskDispositionDates replaces every date token with one placeholder and
// reports how many it replaced. ok is false when a token is not a real
// calendar date, so the caller falls back to the byte comparison.
func maskDispositionDates(s string) (masked string, n int, ok bool) {
ok = true
masked = dispositionDateRe.ReplaceAllStringFunc(s, func(m string) string {
if _, err := time.Parse("2006-01-02", m); err != nil {
ok = false
return m
}
n++
return "{{date}}"
})
return masked, n, ok
}
// sameDispositionButDates reports whether two Content-Disposition values
// differ only in their dates: both carry the same number of real calendar
// dates and are equal once those are masked. Anything else stays a Diff.
func sameDispositionButDates(want, got string) bool {
wm, wn, wok := maskDispositionDates(want)
gm, gn, gok := maskDispositionDates(got)
return wok && gok && wn > 0 && wn == gn && wm == gm
}

View File

@@ -265,7 +265,7 @@ func TestNormalizePublicationAlbumDates(t *testing.T) {
pub := func(created, updated string) []Publication { pub := func(created, updated string) []Publication {
return []Publication{{Method: "POST", Path: "/api/publish", Body: json.RawMessage( return []Publication{{Method: "POST", Path: "/api/publish", Body: json.RawMessage(
`{"channel":"c","data":{"payload":{"album":{"name":"x","created_at":"` + created + `","updated_at":"` + updated + `{"channel":"c","data":{"payload":{"album":{"name":"x","created_at":"` + created + `","updated_at":"` + updated +
`","market_price_checked_at":null,"photos":[{"created_at":"` + created + `"}]},"created_at":"2026-01-01T00:00:00+00:00"}}}`)}} `","market_price_checked_at":null,"photos":[{"created_at":"` + created + `"}]},"published_at":"2026-01-01T00:00:00+00:00"}}}`)}}
} }
a, err := NormalizePublications(pub("2026-09-30T11:21:55+00:00", "2026-09-30T11:21:56+00:00"), store) a, err := NormalizePublications(pub("2026-09-30T11:21:55+00:00", "2026-09-30T11:21:56+00:00"), store)
if err != nil { if err != nil {
@@ -281,8 +281,10 @@ func TestNormalizePublicationAlbumDates(t *testing.T) {
if !strings.Contains(string(a[0].Body), `"updated_at":"{{datetime}}"`) || !strings.Contains(string(a[0].Body), `"market_price_checked_at":null`) { if !strings.Contains(string(a[0].Body), `"updated_at":"{{datetime}}"`) || !strings.Contains(string(a[0].Body), `"market_price_checked_at":null`) {
t.Fatalf("album dates not masked: %s", a[0].Body) t.Fatalf("album dates not masked: %s", a[0].Body)
} }
// Only the album subtree is masked; a payload date outside it stays. // Only the album subtree is masked; a payload date outside it stays
if !strings.Contains(string(a[0].Body), `},"created_at":"2026-01-01T00:00:00+00:00"`) { // ($.data.payload.created_at itself is the notification row's date and
// has its own mask, see TestNormalizeNotificationPublication).
if !strings.Contains(string(a[0].Body), `},"published_at":"2026-01-01T00:00:00+00:00"`) {
t.Fatalf("over-normalised: %s", a[0].Body) t.Fatalf("over-normalised: %s", a[0].Body)
} }
// A Z-suffixed album date keeps its value, so a format change is a diff. // A Z-suffixed album date keeps its value, so a format change is a diff.

View File

@@ -0,0 +1,124 @@
package tide
import (
"encoding/json"
"strings"
"testing"
)
// TestNormalizeContentDispositionDate: a download name built from the day
// of the request replays on a later day, while a different stem, an invalid
// date, a date on one side only or a missing header still diff.
func TestNormalizeContentDispositionDate(t *testing.T) {
const recorded = "attachment; filename=export-2026-09-17.csv"
cases := []struct {
name string
got map[string]string
diff bool
}{
{"same stem on a later day", map[string]string{"Content-Disposition": "attachment; filename=export-2026-10-03.csv"}, false},
{"same day", map[string]string{"Content-Disposition": recorded}, false},
{"header name case", map[string]string{"content-disposition": "attachment; filename=export-2027-01-01.csv"}, false},
{"different stem", map[string]string{"Content-Disposition": "attachment; filename=report-2026-10-03.csv"}, true},
{"invalid date", map[string]string{"Content-Disposition": "attachment; filename=export-2026-13-45.csv"}, true},
{"date on one side only", map[string]string{"Content-Disposition": "attachment; filename=export.csv"}, true},
{"inline instead of attachment", map[string]string{"Content-Disposition": "inline; filename=export-2026-10-03.csv"}, true},
{"header absent", map[string]string{}, true},
}
for _, c := range cases {
diffs := compareHeaders(map[string]string{"Content-Disposition": recorded}, c.got, nil)
if (len(diffs) > 0) != c.diff {
t.Errorf("%s: diffs = %+v, want diff=%v", c.name, diffs, c.diff)
}
}
// A recorded value that holds an invalid date keeps the byte comparison.
odd := "attachment; filename=export-2026-02-30.csv"
if diffs := compareHeaders(map[string]string{"Content-Disposition": odd}, map[string]string{"Content-Disposition": "attachment; filename=export-2026-02-28.csv"}, nil); len(diffs) != 1 {
t.Errorf("invalid recorded date: diffs = %+v, want one", diffs)
}
// Two dates must both be real and both present.
two := "attachment; filename=export-2026-09-01-2026-09-17.csv"
if diffs := compareHeaders(map[string]string{"Content-Disposition": two}, map[string]string{"Content-Disposition": "attachment; filename=export-2026-10-01-2026-10-03.csv"}, nil); len(diffs) != 0 {
t.Errorf("two dates: diffs = %+v, want none", diffs)
}
// The mask applies to Content-Disposition only.
if diffs := compareHeaders(map[string]string{"Location": "/files/2026-09-17"}, map[string]string{"Location": "/files/2026-10-03"}, nil); len(diffs) != 1 {
t.Errorf("Location: diffs = %+v, want one", diffs)
}
if _, n, ok := maskDispositionDates("x12026-09-170"); n != 0 || !ok {
t.Errorf("a date inside a longer digit run is not a date token")
}
}
// TestNormalizeNotificationPublication: a notification:new publication's id
// and created_at under $.data.payload normalize equal across two runs, while
// a Z date, a string id and every other path stay visible.
func TestNormalizeNotificationPublication(t *testing.T) {
store := mustMemoryStore()
pub := func(id, created string) []Publication {
return []Publication{{Method: "POST", Path: "/api/publish", Body: json.RawMessage(
`{"channel":"acme#5","data":{"event":"notification:new","payload":{"id":` + id +
`,"type":"item_added","payload":{"album_id":3,"created_at":"2026-01-01T00:00:00+00:00"},"read_at":null,"created_at":"` + created + `"}}}`)}}
}
a, err := NormalizePublications(pub("10000001", "2026-09-30T11:21:55+00:00"), store)
if err != nil {
t.Fatal(err)
}
b, err := NormalizePublications(pub("42", "2026-10-03T08:00:00+00:00"), store)
if err != nil {
t.Fatal(err)
}
if diffs := DiffPublications(a, b); len(diffs) != 0 {
t.Fatalf("diffs = %+v", diffs)
}
body := string(a[0].Body)
if !strings.Contains(body, `"payload":{"id":{{id}},`) || !strings.HasSuffix(body, `"read_at":null,"created_at":"{{datetime}}"}}}`) {
t.Fatalf("notification not masked: %s", body)
}
// Only $.data.payload.created_at is masked, not a nested payload date.
if !strings.Contains(body, `"album_id":3,"created_at":"2026-01-01T00:00:00+00:00"`) {
t.Fatalf("over-normalised: %s", body)
}
// A Z date, a zero or negative id and a string id stay visible.
for _, c := range []struct{ id, created, want string }{
{"42", "2026-10-03T08:00:00Z", `"created_at":"2026-10-03T08:00:00Z"`},
{`"42"`, "2026-10-03T08:00:00+00:00", `"payload":{"id":"42",`},
{"0", "2026-10-03T08:00:00+00:00", `"payload":{"id":0,`},
{"-3", "2026-10-03T08:00:00+00:00", `"payload":{"id":-3,`},
{"4.5", "2026-10-03T08:00:00+00:00", `"payload":{"id":4.5,`},
} {
got, err := NormalizePublications(pub(c.id, c.created), store)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(got[0].Body), c.want) {
t.Errorf("id %s created %s: %s lacks %s", c.id, c.created, got[0].Body, c.want)
}
if diffs := DiffPublications(a, got); len(diffs) == 0 {
t.Errorf("id %s created %s must diff against the masked publication", c.id, c.created)
}
}
// A captured id keeps its own placeholder.
store.Set("id:note", "77")
got, err := NormalizePublications(pub("77", "2026-10-03T08:00:00+00:00"), store)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(got[0].Body), `"payload":{"id":{{id:note}},`) {
t.Fatalf("captured id: %s", got[0].Body)
}
// An album publication's existing masks are unchanged.
album := []Publication{{Method: "POST", Path: "/api/publish", Body: json.RawMessage(
`{"channel":"c","data":{"payload":{"album":{"created_at":"2026-09-30T11:21:55+00:00"},"actor":{"user_id":1,"name":null},"timestamp":"2026-09-30T11:21:55+00:00"}}}`)}}
got, err = NormalizePublications(album, store)
if err != nil {
t.Fatal(err)
}
want := `{"channel":"c","data":{"payload":{"album":{"created_at":"{{datetime}}"},"actor":"{{actor}}","timestamp":"{{timestamp}}"}}}`
if string(got[0].Body) != want {
t.Fatalf("album publication\n got %s\nwant %s", got[0].Body, want)
}
}