feat(19.4-03): tide captures a named input of an HTML response
- capture rule from: response.form reads the value of the <input> with the rule's name (a server-rendered form's one-shot token), so a parity flow can post a consent form back - README and parity docs list the capture sources
This commit is contained in:
@@ -109,7 +109,7 @@ summer parity:replay --fixtures testdata/parity --target http://127.0.0.1:8080 -
|
||||
|
||||
`parity:proxy` records a real client instead: it runs a reverse proxy in front of the reference backend, and each named session of traffic becomes one fixture. Point the existing frontend at the proxy and click through a feature. The proxy binds to and forwards to loopback addresses only.
|
||||
|
||||
Values captured during a flow, such as tokens and created IDs, live in a variables file (`--vars`) readable only by its owner, and fixtures refer to them as `{{name}}` placeholders. Recording refuses to write a fixture that still holds a token- or password-shaped value, so credentials do not end up in committed fixtures. Keep the variables file outside the repository.
|
||||
A step captures values with `capture` rules: `from: response.json` with a `path`, `response.header`, `response.location.query`, `request.form`, `request.header` and `request.query` with a `name`, and `response.form` for the `value` of the `<input>` with that `name` in an HTML response, such as the one-shot token a server-rendered consent form posts back. Values captured during a flow, such as tokens and created IDs, live in a variables file (`--vars`) readable only by its owner, and fixtures refer to them as `{{name}}` placeholders. Recording refuses to write a fixture that still holds a token- or password-shaped value, so credentials do not end up in committed fixtures. Keep the variables file outside the repository.
|
||||
|
||||
A route manifest (`tide.Manifest`) lists a plugin's routes with their auth groups, status and cases; `parity:record --manifest` records the missing cases in batches, and `parity:replay --manifest` reports coverage. The [Console utilities](../console/utilities.md) page lists every flag.
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ Use cases:
|
||||
- Recording: `tide.RecordFlow` executes a spec flow against a target URL and fills in the responses, with a bounded body size (`tide.DefaultMaxBody`, 8 MiB).
|
||||
- Multipart uploads: a request can carry `parts` (`tide.Part`: a text field with `value`, or a file field with `file`, `filename`, `content_type` and `sha256`) instead of a `body`. File bytes stay in files beside the fixture (for example `files/cover.png`, read from `tide.RecordConfig` or `tide.ReplayConfig` `BaseDir`) and are never inlined into the YAML. Parts are encoded in declaration order with one fixed boundary, `tide.MultipartBoundary`, so the reference backend and the port receive byte-identical bodies; the encoded Content-Type replaces a recorded multipart Content-Type. Variables expand in part values, never in file bytes. `tide.LoadFlow` fails when a part file is missing or its SHA-256 differs, naming the part, and a request with both `body` and `parts` is invalid.
|
||||
- Recording proxy: `tide.NewProxy` builds a reverse proxy that only binds to and forwards to loopback addresses, groups traffic into named sessions (from the `tide.SessionHeader` request header or a default session) and writes one fixture per complete session on `tide.Proxy.Flush`.
|
||||
- Capture rules: `tide.Rules` (loaded with `tide.LoadRules`) decide which request and response headers are kept per route and which values are captured into variables, from response JSON paths, headers, redirect query strings or form fields.
|
||||
- Capture rules: `tide.Rules` (loaded with `tide.LoadRules`) decide which request and response headers are kept per route and which values are captured into variables, from response JSON paths, headers, redirect query strings, request form fields or the value of a named `<input>` in an HTML response (`from: response.form`, for a server-rendered form's hidden token).
|
||||
- Variables: `tide.Store` holds captured values such as tokens and IDs in a mode-0600 file, `tide.Store.Expand` substitutes `{{name}}` placeholders before a request is sent, and `tide.ScrubStep` puts placeholders back into fixtures. Scrubbing fails when a step still holds an unclassified token- or password-shaped value, so credentials do not leak into committed fixtures.
|
||||
- Replay and diff: `tide.ReplayFlow` re-sends each step, compares status, a fixed set of contract headers and the body, and returns `tide.Result` with per-step `tide.Diff` entries. JSON bodies are compared structurally after masking `id`, `*_id` and `*_ids` values and `*_at` timestamps; other bodies are compared byte for byte. A `Content-Disposition` header is compared after masking its `YYYY-MM-DD` dates on both sides, so a download named after the day it was made, such as `attachment; filename=export-2026-09-17.csv`, replays on a later day; each masked token must be a real calendar date, and a different name, an invalid date or a date on one side only is still a difference. Uploaded-file URLs under the `url` key and every key ending in `_url` (`thumb_url`, `cover_url`) are compared by shape: under the uploads prefix (`tide.ReplayConfig` `UploadPrefix`, default `tide.DefaultUploadPrefix`, `/storage/app/uploads/public`) an original must be `<prefix>/xxx/yyy/zzz/<disk_name>` with the partition taken from the disk name, and a thumbnail `<prefix>/xxx/yyy/zzz/thumb_<id>_<w>_<h>_<ox>_<oy>_<mode>.<ext>`. The partition, disk name and file id are masked; the prefix, size, offsets, mode and extension stay, so a thumbnail of another size or an upload URL under another prefix is still a difference.
|
||||
- Fake Centrifugo: `tide.NewCentrifugoRecorder` returns an `http.Handler` that records every POST to a path ending in `/publish` or `/broadcast` as a `tide.Publication` (method, path, whether `Authorization: apikey <key>` carried the configured key, JSON body) and answers `{"result":{}}`. Paths ending in `/presence` answer `{"result":{"presence":{}}}`, `/unsubscribe` and `/info` answer `{"result":{}}`, anything else is 404. Bodies are capped at `tide.MaxPublicationBody` (1 MiB). The API key is only compared, never stored. `tide.CentrifugoRecorder.ListenAndServe` binds loopback addresses only, like the recording proxy.
|
||||
@@ -198,7 +198,7 @@ summer parity:upstream \
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: [fetchguard](../fetchguard/README.md) (the recording proxy's forward mode).
|
||||
- Third-party: `github.com/goccy/go-yaml` (fixture, rules, manifest and sidecar parsing).
|
||||
- Third-party: `github.com/goccy/go-yaml` (fixture, rules, manifest and sidecar parsing), `golang.org/x/net/html` (the `response.form` capture).
|
||||
- Standard library: `net/http`, `net/http/httputil`, `encoding/json`, `crypto/sha256`, `crypto/subtle`, `crypto/tls`, `crypto/x509`, `crypto/ecdsa`, among others.
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -510,3 +510,42 @@ func TestScrubFormFieldDespiteSubstringSecrets(t *testing.T) {
|
||||
t.Fatalf("code placeholder: %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureResponseFormField(t *testing.T) {
|
||||
store, err := OpenStore("")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
const consent = "q8Zr2nV5xYk1Lw7pT3sA9dF6gH0jK4mN"
|
||||
step := Step{
|
||||
ID: "consent-page",
|
||||
Response: Response{
|
||||
Status: 200,
|
||||
Headers: map[string]string{"Content-Type": "text/html; charset=UTF-8"},
|
||||
Body: Body(`<form method="post" action="/oauth/authorize">` +
|
||||
`<input type="hidden" name="request_id" value="0b9c2c7e-1d0f-4a54-8e0e-3f1c9b6f7a20">` +
|
||||
`<input type=hidden name=consent_token value="` + consent + `"/>` +
|
||||
`<button name="decision" value="allow">Allow</button></form>`),
|
||||
},
|
||||
Capture: []CaptureRule{
|
||||
{From: "response.form", Name: "consent_token", As: "consent_token"},
|
||||
{From: "response.form", Name: "request_id", As: "request_id"},
|
||||
},
|
||||
}
|
||||
if err := CaptureStep(store, &step); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, _ := store.Expand("{{consent_token}}|{{request_id}}"); got != consent+"|0b9c2c7e-1d0f-4a54-8e0e-3f1c9b6f7a20" {
|
||||
t.Fatalf("captured %q", got)
|
||||
}
|
||||
if err := ScrubStep(store, &step); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(step.Response.Body), consent) || !strings.Contains(string(step.Response.Body), `value="{{consent_token}}"`) {
|
||||
t.Fatalf("consent token not scrubbed: %s", step.Response.Body)
|
||||
}
|
||||
missing := Step{ID: "no-form", Response: Response{Body: "<p>nothing</p>"}, Capture: []CaptureRule{{From: "response.form", Name: "consent_token", As: "x"}}}
|
||||
if err := CaptureStep(store, &missing); err == nil || !strings.Contains(err.Error(), "missing response form field consent_token") {
|
||||
t.Fatalf("missing field: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"sync"
|
||||
|
||||
"github.com/goccy/go-yaml"
|
||||
"golang.org/x/net/html"
|
||||
)
|
||||
|
||||
var (
|
||||
@@ -322,6 +323,12 @@ func extractCapture(rule CaptureRule, req Request, resp Response) (string, error
|
||||
return "", fmt.Errorf("missing Location query %s", rule.Name)
|
||||
}
|
||||
return v, nil
|
||||
case "response.form":
|
||||
v := htmlInputValue(string(resp.Body), rule.Name)
|
||||
if v == "" {
|
||||
return "", fmt.Errorf("missing response form field %s", rule.Name)
|
||||
}
|
||||
return v, nil
|
||||
case "request.form":
|
||||
v, err := formValue(string(req.Body), rule.Name)
|
||||
if err != nil {
|
||||
@@ -352,6 +359,36 @@ func extractCapture(rule CaptureRule, req Request, resp Response) (string, error
|
||||
}
|
||||
}
|
||||
|
||||
// htmlInputValue returns the value attribute of the first <input> element
|
||||
// named name in an HTML body (a server-rendered form's hidden field, such
|
||||
// as a one-shot CSRF or consent token), or "".
|
||||
func htmlInputValue(body, name string) string {
|
||||
z := html.NewTokenizer(strings.NewReader(body))
|
||||
for {
|
||||
switch z.Next() {
|
||||
case html.ErrorToken:
|
||||
return ""
|
||||
case html.StartTagToken, html.SelfClosingTagToken:
|
||||
tok := z.Token()
|
||||
if tok.Data != "input" {
|
||||
continue
|
||||
}
|
||||
var n, v string
|
||||
for _, a := range tok.Attr {
|
||||
switch a.Key {
|
||||
case "name":
|
||||
n = a.Val
|
||||
case "value":
|
||||
v = a.Val
|
||||
}
|
||||
}
|
||||
if n == name {
|
||||
return v
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func formValue(body, name string) (string, error) {
|
||||
vals, err := url.ParseQuery(body)
|
||||
if err != nil {
|
||||
|
||||
Reference in New Issue
Block a user