---
phase: 06-http-routing-auth-groups-and-rate-limiting
plan: 04
type: execute
wave: 1
depends_on: []
files_modified:
- summercms.go/fetchguard/policy.go
- summercms.go/fetchguard/ip.go
- summercms.go/fetchguard/fetch.go
- summercms.go/fetchguard/fetch_test.go
- summercms.go/fetchguard/ip_test.go
autonomous: true
requirements: [HTTP-07]
must_haves:
truths:
- "A URL whose host is not on Policy.AllowHosts (exact or dotted-suffix match) is rejected before any network I/O (D-11)"
- "PublicOnly mode accepts any hostname but rejects the fetch if ANY resolved IP is private/loopback/reserved, including under an allow-list (D-11)"
- "The private/loopback/reserved IP check runs at dial time via net.Dialer.Control on the actual address being connected, not only against the pre-resolved hostname, closing the DNS-rebinding TOCTOU gap PHP's own code admits it has (D-12)"
- "Only https is accepted; a redirect response is never followed (D-11)"
- "A response body is capped while streaming (io.LimitReader), not after being fully buffered -- a slow-drip origin cannot exceed the cap even within the timeout (D-11, Pitfall 8)"
- "Byte cap and timeout default to config http.fetch.max_bytes/http.fetch.timeout_seconds (10 MiB / 10s when unset) and are overridable per call, but an explicitly-set zero or negative effective value is an error, never treated as unlimited (D-14)"
- "Failure reasons are a closed, typed set matching PHP's invalid_url/scheme/unresolvable/private_ip/network_error/too_large family (D-13)"
- "The private IPv4/IPv6 CIDR table matches ManualCoverUrlFetcher.php's PRIVATE_V4_CIDRS/PRIVATE_V6_PREFIXES constants exactly, including 100.64.0.0/10 (CGNAT) and 169.254.0.0/16 (cloud metadata)"
- "No ManualCoverUrlFetcher or CoverImporter call site exists in this plan -- this plan ships the helper and its tests only (D-13)"
artifacts:
- path: summercms.go/fetchguard/fetch.go
provides: "Fetch(ctx, url string, policy Policy) (*Result, error) -- the SSRF-guarded outbound fetch entry point"
- path: summercms.go/fetchguard/policy.go
provides: "Policy{Mode, AllowHosts, PublicOnly, MaxBytes, Timeout}, Reason type, Defaults/DefaultsFromConfig"
- path: summercms.go/fetchguard/ip.go
provides: "isReservedOrPrivate(netip.Addr) bool -- the private/loopback/reserved/CGNAT CIDR table"
key_links:
- from: summercms.go/fetchguard/fetch.go
to: summercms.go/fetchguard/ip.go
via: "the http.Transport's DialContext Control hook calls isReservedOrPrivate on the address actually being dialed"
pattern: "Control:.*dialControl|isReservedOrPrivate"
---
Ship a framework-owned, SSRF-guarded outbound fetch helper offering both PHP fetch modes (`AllowHosts` exact/dotted-suffix allow-listing and `PublicOnly` any-host-but-only-public-IPs) with the private/loopback/reserved IP check enforced at dial time -- closing the DNS-rebinding gap PHP's own `ManualCoverUrlFetcher` comment admits it has -- plus https-only, no redirects, a streaming byte cap, and a timeout, all with typed failure reasons.
Purpose: HTTP-07 requires a single reusable primitive two future phases' call sites (manual cover URL fetch, Discogs cover import) will use for user-supplied and third-party URLs; getting the SSRF guard right here, once, with tests against real `httptest` servers, is cheaper and safer than each call site re-implementing it.
Output: `fetchguard.Fetch`, `fetchguard.Policy`, typed `Reason` values, and a private/reserved IP classification table -- helper and tests only, no call sites this phase (D-13).
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
@.planning/PROJECT.md
@.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-CONTEXT.md
@.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-RESEARCH.md
@.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-PATTERNS.md
New file summercms.go/fetchguard/policy.go:
package fetchguard
import "time"
type Mode int
const (
AllowHostsMode Mode = iota
PublicOnlyMode
)
type Reason string
const (
ReasonInvalidURL Reason = "invalid_url"
ReasonScheme Reason = "scheme"
ReasonUnresolvable Reason = "unresolvable"
ReasonPrivateIP Reason = "private_ip"
ReasonNetworkError Reason = "network_error"
ReasonTooLarge Reason = "too_large"
)
// Policy is supplied per call. Mode selects AllowHosts vs PublicOnly; the
// private/loopback/reserved IP block is ALWAYS on regardless of Mode (D-11:
// "even under an allow-list").
type Policy struct {
Mode Mode
AllowHosts []string // exact or dotted-suffix match, only used in AllowHostsMode
MaxBytes int64 // 0 means "use the config/framework default", never unlimited
Timeout time.Duration
}
// Error carries the typed Reason plus the underlying error for logging.
type Error struct {
Reason Reason
Err error
}
func (e *Error) Error() string
func (e *Error) Unwrap() error
// Defaults are the framework fallback: 10 MiB, 10s, matching PHP.
func Defaults() (maxBytes int64, timeout time.Duration)
// DefaultsFromConfig reads http.fetch.max_bytes/http.fetch.timeout_seconds
// from cfg, falling back to Defaults() for absent/zero keys. An explicitly
// configured zero or negative value is an error (D-14), returned instead of
// silently falling back -- config typos must not silently become "unlimited".
func DefaultsFromConfig(cfg *compass.Config) (maxBytes int64, timeout time.Duration, err error)
New file summercms.go/fetchguard/fetch.go:
package fetchguard
type Result struct {
Body []byte
ContentType string
StatusCode int
}
// Fetch validates url against policy, resolves defaults for any zero
// MaxBytes/Timeout via DefaultsFromConfig-equivalent (cfg may be nil, in
// which case Defaults() applies), then performs the guarded HTTPS fetch.
// A non-nil error is always *Error with a Reason from the closed set.
func Fetch(ctx context.Context, url string, policy Policy, cfg *compass.Config) (*Result, error)
New file summercms.go/fetchguard/ip.go:
package fetchguard
import "net/netip"
// isReservedOrPrivate classifies addr (already Unmap()-ed by the caller)
// against the exact table ManualCoverUrlFetcher.php hand-rolls:
// v4: 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16,
// 100.64.0.0/10, 0.0.0.0/8
// v6: ::1, fe80::/10, fc00::/7
// plus addr.IsMulticast()/IsUnspecified() as additional stdlib-covered cases.
func isReservedOrPrivate(addr netip.Addr) bool
Task 1: Private/reserved IP classification table
summercms.go/fetchguard/ip.go, summercms.go/fetchguard/ip_test.go
- 127.0.0.1, 10.1.2.3, 172.16.0.1, 172.31.255.255, 192.168.1.1, 169.254.169.254 (cloud metadata), 100.64.0.1 (CGNAT), 0.0.0.1 -> true.
- 8.8.8.8, 1.1.1.1, 93.184.216.34 (a real public IP) -> false.
- ::1 -> true; fe80::1 -> true; fc00::1 -> true; a public v6 address (e.g. 2606:4700:4700::1111) -> false.
- An IPv4-mapped IPv6 literal for a private address (::ffff:169.254.169.254) classifies as true AFTER Unmap() is applied by the caller -- ip_test.go asserts isReservedOrPrivate(addr.Unmap()) for this case, documenting that Unmap() is the CALLER's responsibility (fetch.go's dial hook), not this function's.
- A multicast address (224.0.0.1) and the unspecified address (0.0.0.0) both -> true.
- 172.15.255.255 and 172.32.0.0 (just outside the RFC1918 172.16.0.0/12 range) -> false (boundary test).
/media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/classes/ManualCoverUrlFetcher.php lines 41-55 (PRIVATE_V4_CIDRS, PRIVATE_V6_PREFIXES -- the exact source table, read directly, not from memory)
.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-RESEARCH.md Pitfall 6 (why net.IP.IsPrivate() alone is insufficient; the Unmap()/Is4In6 normalization requirement)
Create ip.go: a package-level `var privateV4 = []netip.Prefix{ netip.MustParsePrefix("127.0.0.0/8"), netip.MustParsePrefix("10.0.0.0/8"), netip.MustParsePrefix("172.16.0.0/12"), netip.MustParsePrefix("192.168.0.0/16"), netip.MustParsePrefix("169.254.0.0/16"), netip.MustParsePrefix("100.64.0.0/10"), netip.MustParsePrefix("0.0.0.0/8") }` and `var privateV6 = []netip.Prefix{ netip.MustParsePrefix("::1/128"), netip.MustParsePrefix("fe80::/10"), netip.MustParsePrefix("fc00::/7") }` (note: PHP's PRIVATE_V6_PREFIXES lists bare "::1" as a prefix-less loopback literal -- express it as the exact /128 host prefix so netip.Prefix.Contains works uniformly with the other CIDR entries). isReservedOrPrivate(addr netip.Addr) bool: if !addr.IsValid() return true (fail closed on garbage input); if addr.IsMulticast() || addr.IsUnspecified() return true; select privateV4 or privateV6 based on addr.Is4() (the caller is documented to pass an already-Unmap()-ed address, so a v4-mapped-v6 literal has already become a plain v4 Addr by this point); iterate the selected table with Prefix.Contains(addr), return true on any match, false otherwise.
cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go test ./fetchguard/... -run TestIsReservedOrPrivate -v
- Every case in the behavior list above is a distinct table-driven subtest and passes.
- go vet ./fetchguard/... is clean.
The private/reserved/CGNAT/metadata IP table is a literal, tested port of ManualCoverUrlFetcher.php's constants -- no range invented, none omitted.
Task 2: Policy, defaults, and the dial-time-guarded, streaming-capped Fetch entry point
summercms.go/fetchguard/policy.go, summercms.go/fetchguard/fetch.go, summercms.go/fetchguard/fetch_test.go
- A malformed URL (missing scheme/host) -> Error{Reason: ReasonInvalidURL}.
- An http:// (non-https) URL -> Error{Reason: ReasonScheme}, no network I/O attempted.
- In AllowHostsMode, a host not present in Policy.AllowHosts (exact match) and not a dotted-suffix of any entry -> Error{Reason: ReasonInvalidURL} before DNS resolution (host-allow-list is a pre-dial check, separate from the IP check).
- A dotted-suffix bypass attempt (host "evil-discogs.com" against an AllowHosts entry "discogs.com") is rejected -- "evil-discogs.com" must NOT match as a suffix of "discogs.com" (only "*.discogs.com" or the exact host matches).
- A hostname that resolves only to a private/loopback address (via an httptest server bound to 127.0.0.1, or a fake net.Resolver/dial override in the test) -> Error{Reason: ReasonPrivateIP}, in BOTH AllowHostsMode (host allow-listed) and PublicOnlyMode -- proving the IP block is unconditional.
- A server that returns a 3xx redirect -> Fetch does not follow it (CheckRedirect returns an error or http.ErrUseLastResponse; test asserts the Result reflects the 3xx response itself, or Error{Reason: ReasonNetworkError}, whichever the implementation picks -- pin ONE behavior and document it in the doc comment on Fetch).
- A server that streams more than policy.MaxBytes (test server writes in a loop past the cap within the timeout) -> Error{Reason: ReasonTooLarge}, and the test asserts via a byte-counting io.Reader on the SERVER side that the client did not request/read more than MaxBytes+1 bytes (proving streaming enforcement, not post-hoc buffering).
- policy.MaxBytes == 0 and policy.Timeout == 0 with a nil cfg -> Defaults() values (10 MiB, 10s) are used.
- DefaultsFromConfig with an explicit http.fetch.max_bytes: 0 (or negative) set in cfg -> returns a non-nil error, never silently falls back to Defaults() (D-14: "zero or negative effective values are an error").
summercms.go/fetchguard/ip.go (Task 1 output)
summercms.go/compass/config.go (Int/Lookup -- for DefaultsFromConfig)
/media/nvme/dev/golem15/fonoteka/plugins/golem15/fonoteka/classes/ManualCoverUrlFetcher.php full file (validation order comment, lines 1-90+ already read this session -- the exact 10-step sequence to mirror minus the MIME-sniffing/attach steps, which are out of scope per D-13)
.planning/phases/06-http-routing-auth-groups-and-rate-limiting/06-RESEARCH.md Code Examples "Dial-time SSRF guard shape" (lines 416-442) and Pitfalls 6-8
Create policy.go with Mode, Reason constants, Policy, Error (Error()/Unwrap() per the interfaces block), Defaults() returning (10*1024*1024, 10*time.Second), and DefaultsFromConfig(cfg) reading "http.fetch.max_bytes"/"http.fetch.timeout_seconds" via cfg.Lookup (not cfg.Int, since Int silently returns 0 for both "absent" and "explicitly zero" -- use Lookup to distinguish "key absent" (fall back to Defaults()) from "key present and <= 0" (return an error) from "key present and positive" (use it)); timeout_seconds converts to time.Duration via time.Duration(n)*time.Second.
Create fetch.go: Fetch(ctx, rawURL string, policy Policy, cfg *compass.Config) (*Result, error). Step 1: url.Parse(rawURL); if err or Scheme=="" or Host=="" -> &Error{ReasonInvalidURL, err}. Step 2: if strings.ToLower(parsed.Scheme) != "https" -> &Error{ReasonScheme, nil}. Step 3 (AllowHostsMode only): strip IPv6 brackets from parsed.Hostname() if present; check exact match against policy.AllowHosts OR a dotted-suffix match (host has a literal "." immediately before the suffix, e.g. strings.HasSuffix(host, "."+allowed) OR host == allowed -- this is what prevents "evil-discogs.com" matching "discogs.com": there is no "." immediately before "discogs.com" in "evil-discogs.com" since the character before it is "-", not "."); no match -> &Error{ReasonInvalidURL, nil}. Step 4: resolve maxBytes/timeout: if policy.MaxBytes > 0 use it, else if cfg != nil use DefaultsFromConfig(cfg) (propagate its error), else use Defaults(); same pattern for Timeout. Step 5: build an http.Client with Timeout: timeout, CheckRedirect: func(...) error { return http.ErrUseLastResponse } (pin this: redirects are never followed, the 3xx response itself is returned to the caller as a non-error Result so the caller can decide -- document this choice in Fetch's doc comment), and Transport: &http.Transport{DialContext: (&net.Dialer{Timeout: timeout, Control: dialControl(policy)}).DialContext}. dialControl(policy) is the unexported func(network, address string, c syscall.RawConn) error from the RESEARCH.md code example: net.SplitHostPort the address, netip.ParseAddr the host, addr.Unmap(), isReservedOrPrivate(addr) -> return an error wrapping ReasonPrivateIP (the DialContext error surfaces through the http.Client call as a network error -- Step 6 below maps it back to ReasonPrivateIP by checking errors.As/a sentinel, not by re-deriving it from scratch). Step 6: perform the GET request with ctx; on any transport-level error, check whether it wraps the private-IP sentinel from Step 5's dial hook (map to ReasonPrivateIP) or a DNS resolution failure specifically (map to ReasonUnresolvable), else map to ReasonNetworkError. Step 7: on a successful round trip, wrap resp.Body in io.LimitReader(resp.Body, maxBytes+1); io.ReadAll it; if len(data) == maxBytes+1, return &Error{ReasonTooLarge, nil} (the limit was hit, meaning the true body is at least one byte larger than allowed); otherwise return &Result{Body: data, ContentType: resp.Header.Get("Content-Type"), StatusCode: resp.StatusCode}, nil.
cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./... && go test ./fetchguard/... -short -race
- Every behavior-list case above is a distinct subtest against a real httptest.Server (or a deliberately-private-bound listener for the private_ip cases) -- no mocked http.RoundTripper standing in for the dial-time check itself, since the dial-time enforcement is the exact thing under test.
- The too_large test proves streaming enforcement: assert (via a counter on the test server's handler) that the server did not need to write more than maxBytes+1 bytes before the client aborted the read, i.e. the client did not buffer an unbounded body first.
- The dotted-suffix bypass test explicitly includes "evil-discogs.com" against an AllowHosts entry of "discogs.com" and asserts rejection.
- DefaultsFromConfig's explicit-zero-is-an-error behavior has a dedicated test using a real *compass.Config loaded from a temp YAML file (not a hand-built struct), so the config-parsing path is exercised too.
fetchguard.Fetch enforces https-only, dial-time private-IP blocking (closing the DNS-rebinding gap), no redirects, streaming byte caps, and typed failure reasons -- proven against real network I/O in every test, with zero production call sites added this phase.
## Trust Boundaries
| Boundary | Description |
|----------|--------------|
| caller-supplied URL -> outbound fetch | the entire point of this package: a user- or third-party-supplied URL must never reach an internal service, cloud metadata endpoint, or loopback service |
| DNS resolution -> TCP connect | the classic SSRF TOCTOU gap (resolve-time check vs connect-time address) |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-06-14 | Elevation of Privilege | SSRF via a caller-supplied URL reaching an internal service or 169.254.169.254 | mitigate | Dial-time net.Dialer.Control check on every connection attempt (D-12), always-on regardless of Mode (D-11), tested against real httptest/private listeners |
| T-06-15 | Tampering | DNS rebinding (resolve-time check passes, connect-time IP differs) | mitigate | The check runs at actual-dial time on the real address being connected, not on a pre-resolved hostname -- this is a deliberate Go-side improvement over PHP's own documented gap (D-12), not parity with PHP's vulnerable pattern |
| T-06-16 | Denial of Service | Unbounded response body from a malicious/slow-drip origin that passes the SSRF gate | mitigate | io.LimitReader enforces the byte cap while streaming, combined with an http.Client.Timeout; a post-hoc len(body) check alone (as PHP's own comment warns against) is never used |
| T-06-17 | Elevation of Privilege | Redirect to a private/internal address after the initial host/IP checks pass | mitigate | CheckRedirect prevents the client from ever following a redirect automatically; the caller receives the 3xx response itself and must explicitly re-invoke Fetch (through the same guard) if it wants to follow it |
| T-06-18 | Spoofing | Dotted-suffix allow-list bypass (e.g. evil-discogs.com against discogs.com) | mitigate | Suffix match requires a literal "." immediately preceding the allowed suffix or an exact match; tested explicitly |
cd summercms.go && go vet ./... && go test ./fetchguard/... -race
- fetchguard.Fetch enforces https-only, host allow-listing (exact + dotted-suffix) or public-only IP mode, always-on private/reserved IP blocking at dial time, no redirects, streaming byte caps, and a timeout.
- Failure reasons are a closed, typed set matching PHP's family.
- Zero call sites exist yet (helper + tests only, per D-13).
- go vet ./... and go test ./... -race are green.