docs(modules): rewrite lagoon, tide, pact, party, boardwalk, fetchguard READMEs
This commit is contained in:
Jakub Zych
@@ -1,3 +1,86 @@
|
||||
# fetchguard
|
||||
|
||||
`fetchguard` performs policy-controlled outbound HTTP fetches with size, timeout, and address safety checks. No current in-repository package imports it directly; its public entry point is `fetchguard.Fetch` in `fetch.go`.
|
||||
Guarded outbound HTTPS fetcher that blocks private and reserved addresses and enforces host, size and timeout limits.
|
||||
|
||||
`import "git.golem15.com/golem15/summercms/modules/fetchguard"`
|
||||
|
||||
## Overview
|
||||
|
||||
`fetchguard` is the framework's server-side request forgery guard for fetching URLs that come from users or third parties, such as a remote image address. Every call takes a `fetchguard.Policy` that either restricts the target to an allow list of hosts or permits any public host; in both modes the dial-time check refuses private, loopback, link-local, carrier-grade NAT, documentation, multicast and other reserved IPv4 and IPv6 ranges, including IPv4 embedded in NAT64 and 6to4 addresses. Failures come back as a `fetchguard.Error` carrying one `fetchguard.Reason` from a closed set, so callers can map them onto stable API error codes. WinterCMS has no dedicated counterpart; plugins there typically used Guzzle with hand-written checks.
|
||||
|
||||
## Features
|
||||
|
||||
- HTTPS only: any other scheme fails with `fetchguard.ReasonScheme`.
|
||||
- Two modes: `fetchguard.AllowHostsMode` (exact or dotted-suffix host match against `fetchguard.Policy.AllowHosts`) and `fetchguard.PublicOnlyMode` (any public host).
|
||||
- The private and reserved address check runs on the resolved IP at dial time, so DNS answers that point inside the network are refused (`fetchguard.ReasonPrivateIP`); environment proxies are ignored so the check sees the real target.
|
||||
- Redirects are never followed: a 3xx response is returned as a successful `fetchguard.Result`, and a caller that wants to follow the Location header calls `fetchguard.Fetch` again, which re-runs the guard.
|
||||
- The response body is capped at the policy's byte limit (`fetchguard.ReasonTooLarge` when exceeded), with a per-call timeout.
|
||||
- Limits left at zero in the policy fall back to config keys, then to framework defaults of 10 MiB and 10 seconds (`fetchguard.Defaults`, `fetchguard.DefaultsFromConfig`).
|
||||
- Typed failure reasons: `fetchguard.ReasonInvalidURL`, `fetchguard.ReasonScheme`, `fetchguard.ReasonUnresolvable`, `fetchguard.ReasonPrivateIP`, `fetchguard.ReasonNetworkError`, `fetchguard.ReasonTooLarge`.
|
||||
|
||||
## Usage
|
||||
|
||||
```go
|
||||
policy := fetchguard.Policy{
|
||||
Mode: fetchguard.AllowHostsMode,
|
||||
AllowHosts: []string{"images.example.com"},
|
||||
MaxBytes: 5 << 20,
|
||||
Timeout: 5 * time.Second,
|
||||
}
|
||||
|
||||
res, err := fetchguard.Fetch(ctx, imageURL, policy, app.Config)
|
||||
if err != nil {
|
||||
var fe *fetchguard.Error
|
||||
if errors.As(err, &fe) && fe.Reason == fetchguard.ReasonPrivateIP {
|
||||
return errRejectedURL
|
||||
}
|
||||
return err
|
||||
}
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("image fetch: status %d", res.StatusCode)
|
||||
}
|
||||
image := res.Body
|
||||
```
|
||||
|
||||
## API reference
|
||||
|
||||
| Identifier | Description |
|
||||
|------------|-------------|
|
||||
| `fetchguard.Fetch` | Validates the URL against the policy and performs the guarded HTTPS GET; a non-nil error is always a `fetchguard.Error`. |
|
||||
| `fetchguard.Policy` | Per-call settings: mode, allowed hosts, byte limit and timeout (zero means use the configured default). |
|
||||
| `fetchguard.Mode` | Selects `fetchguard.AllowHostsMode` or `fetchguard.PublicOnlyMode`. |
|
||||
| `fetchguard.Result` | Response body, Content-Type header value and status code of any completed response, including 3xx and non-2xx. |
|
||||
| `fetchguard.Error` | Failure carrying a `fetchguard.Reason` and the underlying error for logging. |
|
||||
| `fetchguard.Reason` | Closed set of failure reasons (`invalid_url`, `scheme`, `unresolvable`, `private_ip`, `network_error`, `too_large`). |
|
||||
| `fetchguard.Defaults` | Framework fallback limits: 10 MiB and 10 seconds. |
|
||||
| `fetchguard.DefaultsFromConfig` | Reads the limits from a `compass.Config`, falling back to `fetchguard.Defaults` for absent keys. |
|
||||
|
||||
## Configuration
|
||||
|
||||
`fetchguard.Fetch` and `fetchguard.DefaultsFromConfig` read these keys from the `compass.Config` passed to them. They apply only when the policy leaves the matching limit at zero, and an explicitly configured zero or negative value is an error.
|
||||
|
||||
| Key | Default | Controls |
|
||||
|-----|---------|----------|
|
||||
| `http.fetch.max_bytes` | `10485760` (10 MiB) | Maximum response body size in bytes. |
|
||||
| `http.fetch.timeout_seconds` | `10` | Dial and overall request timeout, in seconds. |
|
||||
|
||||
```yaml
|
||||
http:
|
||||
fetch:
|
||||
max_bytes: 5242880
|
||||
timeout_seconds: 5
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: [compass](../compass/README.md) (config lookup).
|
||||
- Third-party: none.
|
||||
- Standard library: `context`, `crypto/tls`, `errors`, `fmt`, `io`, `math`, `net`, `net/http`, `net/netip`, `net/url`, `strings`, `syscall`, `time`.
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
go test ./modules/fetchguard/...
|
||||
```
|
||||
|
||||
The tests run against local `net/http/httptest` TLS servers and cover the address classifier directly; they need no external services.
|
||||
|
||||
Reference in New Issue
Block a user