summercms/docs/services/outbound-http.md

title, description, section, order

title	description	section	order
Outbound HTTP	Fetch user-supplied URLs and call vendor APIs through fetchguard, which blocks private addresses at dial time, limits size and time and never redirects.	services	100

Outbound HTTP

A WinterCMS plugin fetches a remote URL with the Laravel HTTP client or Guzzle, and checks the URL by hand when it came from a user. When a URL comes from outside the application, such as a remote image address, fetch it with fetchguard. It is the framework's guard against server-side request forgery: a request that a user can aim at the application's own network, a cloud metadata service or an internal admin panel.

For calls to service APIs, such as a payment provider or a model vendor, use a fetchguard.Client (see Calling a service API). It applies the same guard to every request, caps responses and never follows redirects.

Policies

Every call takes a fetchguard.Policy:

• fetchguard.AllowHostsMode allows only the hosts in AllowHosts, matched exactly or as a dotted suffix.
• fetchguard.PublicOnlyMode allows any public host.

In both modes only https is allowed, and the resolved IP address is checked when the connection is dialled, so a DNS name that resolves into the network is refused too. The check covers private, loopback, link-local, carrier-grade NAT, documentation, multicast and other reserved IPv4 and IPv6 ranges, including IPv4 addresses inside NAT64 and 6to4 addresses. Environment proxy settings are ignored, so the check always sees the real target.

ctx := context.Background()
// Only the application's image host, at most 5 MiB within 5 seconds.
images := fetchguard.Policy{
	Mode:       fetchguard.AllowHostsMode,
	AllowHosts: []string{"images.example.com"},
	MaxBytes:   5 << 20,
	Timeout:    5 * time.Second,
}
// Any public host, for a URL a user pasted.
public := fetchguard.Policy{Mode: fetchguard.PublicOnlyMode}

for _, c := range []struct {
	url    string
	policy fetchguard.Policy
}{
	{"http://images.example.com/cover.jpg", images},
	{"https://cdn.attacker.example/cover.jpg", images},
	{"https://127.0.0.1/admin", public},
	{"https://169.254.169.254/latest/meta-data/", public},
	{"https://[::ffff:10.0.0.1]/", public},
	{"https://%zz", public},
} {
	// The last argument is the application's config (app.Config), for
	// limits the policy leaves at zero; nil uses the framework defaults.
	_, err := fetchguard.Fetch(ctx, c.url, c.policy, nil)
	var fe *fetchguard.Error
	if errors.As(err, &fe) {
		fmt.Println(fe.Reason, c.url)
	}
}
fmt.Println(fetchguard.Defaults())
// Output:
// scheme http://images.example.com/cover.jpg
// invalid_url https://cdn.attacker.example/cover.jpg
// private_ip https://127.0.0.1/admin
// private_ip https://169.254.169.254/latest/meta-data/
// private_ip https://[::ffff:10.0.0.1]/
// invalid_url https://%zz
// 10485760 10s

Code that has to judge a resolved address itself, before it connects, uses the same classification through fetchguard.IsPrivateAddr:

for _, ip := range []string{"8.8.8.8", "10.0.0.7", "::ffff:127.0.0.1"} {
	fmt.Println(ip, fetchguard.IsPrivateAddr(netip.MustParseAddr(ip)))
}
// Output:
// 8.8.8.8 false
// 10.0.0.7 true
// ::ffff:127.0.0.1 true

A failure is always a fetchguard.Error with one fetchguard.Reason from a closed set, so a handler can map it to a stable API error code. A host outside the allow list is reported as invalid_url, as the example shows.

Responses and limits

fetchguard.Fetch returns a fetchguard.Result with the body, the content type and the status code for any response the server completed, including 4xx and 5xx. Check the status yourself.

Redirects are never followed: a 3xx response is returned as a result. To follow it, call fetchguard.Fetch again with the Location URL, which runs every check again.

The body is capped at the policy's MaxBytes (a larger body is fetchguard.ReasonTooLarge) and the call at its Timeout. A limit left at zero falls back to http.fetch.max_bytes and http.fetch.timeout_seconds from the configuration you pass, then to the framework defaults of 10 MiB and 10 seconds (fetchguard.Defaults). A configured value of zero or less is an error, not a way to turn a limit off.

Calling a service API

Build one fetchguard.Client per vendor with fetchguard.NewClient and keep it for the life of the plugin: it keeps connections alive and runs the dial guard on every new one. The client sends any method; the helpers cover the common shapes:

• fetchguard.Client.PostJSON and fetchguard.Client.PutJSON marshal the body, set Content-Type: application/json and then copy your headers over it.
• fetchguard.Client.PostMultipart writes every fetchguard.FormField and then every fetchguard.FormFile in slice order.
• fetchguard.Client.Get sends a GET, and fetchguard.Client.Do or fetchguard.Client.Send take any http.Request you build.
• fetchguard.Bearer builds the Authorization value.

client, err := fetchguard.NewClient(fetchguard.Policy{
	Mode:       fetchguard.AllowHostsMode,
	AllowHosts: []string{"api.example.com"},
	Timeout:    10 * time.Second,
}, nil)
if err != nil {
	panic(err)
}
header := http.Header{}
header.Set("Authorization", fetchguard.Bearer("example-token"))

// Production code passes its own context; the example routes the call to
// a stub instead of the network.
ctx := fetchguard.WithTransport(context.Background(), stubVendor{})
res, err := client.PostJSON(ctx, "https://api.example.com/v1/items", header, map[string]string{"name": "widget"})
if err != nil {
	panic(err)
}
fmt.Println(res.StatusCode, string(res.Body))
// Output:
// 201 {"id":42}

The fetchguard.Result carries the status code and every response header, unjudged: a 401, a 429 with Retry-After or a 3xx is a result, not an error, and the caller decides what it means. Errors are transport failures and guard refusals, always a fetchguard.Error.

Choose the timeout per consumer in the policy: the client has no vendor defaults. A short timeout suits a metadata API; a model call that generates a long answer needs minutes. Request bodies are not capped, so bound uploads before you send them.

Trusted mode

fetchguard.TrustedMode is for endpoints an operator configured, in Go code or in admin settings, for example a model server on the local network. A client in this mode accepts http and https, checks no host list and skips the dial guard; the body cap and the no-redirect rule still apply. Only Go code that builds the policy can choose it: no configuration key, environment variable or request input selects it.

Never use it for a URL that a user or a third party supplied, including a per-user override of a vendor's base URL. Those go through fetchguard.AllowHostsMode or fetchguard.PublicOnlyMode.

Testing outbound calls

Tests never call a vendor. fetchguard.WithTransport returns a context whose client requests go to an http.RoundTripper you supply instead of the network. The URL guard still runs first, so the test also proves the policy accepts the vendor's real host. The override lives only in the context: no policy field, client field, configuration key, environment variable or header can set it.

For API parity work, hand it the tide upstream fake, which replays the vendor exchanges recorded from the reference backend and asserts every request the client sends. See Parity testing.