docs(35): decision note for the mail URL scheme check and site name safety
This commit is contained in:
65
.planning/notes/2026-11-14-mail-url-scheme-check.md
Normal file
65
.planning/notes/2026-11-14-mail-url-scheme-check.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# Decision: postcard checks URL schemes in decoded attribute values, never lets a site name stop a send, and prints the footer name literally
|
||||
|
||||
**Date:** 2026-11-14
|
||||
**Status:** Accepted
|
||||
**Context:** Phase 35 re-verification gap (35-VERIFICATION.md `gaps:`, the CR-01 regression), 35-REVIEW.md CR-01, WR-01 and WR-05, Jakub's decisions of 2026-11-14 (two plans: this fix in 35-08, then the regression and unit tests in 35-09; WR-01 and WR-05 bundled with CR-01), plan 35-08
|
||||
|
||||
## The problem
|
||||
|
||||
postcard's `validateHTML` ran `(?i)(?:javascript|vbscript|data):` over the whole rendered document. The built-in layouts print the site name (the Branding Site name, else the display name of `mail.from`) in the header and the footer, so a site name such as `Big Data: Summit` or a sender such as `"Metadata: Weekly" <x@example.com>` failed every send on the default and system layouts. The file version carries the same name, so the override-to-file fallback failed too, and the Editor refused every template and layout save while such a Branding name was stored. An application that sends mail today could stop sending password resets after upgrading, without anyone touching the admin.
|
||||
|
||||
Two related findings: the check read undecoded markup, so `href="javascript:..."` in a layout or partial override passed it and was sent as is when the layout opted out of inlining (WR-05), and an inlined result that failed the post-inline check stopped the message although the docs promise the inliner never does (WR-05). The footer rendered the copyright line, and with it the site name, as Markdown (WR-01).
|
||||
|
||||
## What this supersedes
|
||||
|
||||
- **35-RESEARCH.md Pitfall 8** ("keep the behaviour, it matches sends") and 35-PATTERNS.md "HTML safety gate: reuse them unchanged", for postcard. Prose is no longer treated as a URL.
|
||||
- **The postcard README sentence** saying that text such as `metadata:` is refused at save.
|
||||
- **The README sentence** saying an inlined result that fails the safety check fails the render.
|
||||
- **D-13 itself stands:** `javascript:`, `vbscript:` and `data:` URLs are still refused at save and at send, now also when they are entity-encoded.
|
||||
- `cabana.RenderMarkdown`'s own regexes (the admin Markdown preview) are out of scope and unchanged.
|
||||
|
||||
## Decision
|
||||
|
||||
### What the scheme check reads (`unsafeMarkup`, modules/postcard/safety.go)
|
||||
|
||||
The rendered HTML is read with `golang.org/x/net/html`'s tokenizer (already a direct framework requirement, used by seaglass, tide and cabana). The tokenizer lower-cases attribute names and decodes character references in attribute values (`:`, `:`, `:`, `j`), so the check sees what a mail client sees.
|
||||
|
||||
- **Single-URL attributes** (`urlAttributes`): `action`, `background`, `cite`, `codebase`, `data`, `dynsrc`, `formaction`, `href`, `icon`, `longdesc`, `lowsrc`, `manifest`, `poster`, `profile`, `src`, `usemap`, `xlink:href`, and the SVG animation targets `from`, `to`, `by`. A value is refused when, decoded, with every ASCII whitespace and control character (U+0000 to U+0020 and U+007F) removed and lower-cased, it starts with `javascript:`, `vbscript:` or `data:` (`hasDangerousScheme`). Removing whitespace closes tab- and newline-split schemes.
|
||||
- **List-valued attributes** (`urlListAttributes`: `srcset`, `ping`, `archive`, `values`), **`style` attributes**, the **`content` attribute of `meta` elements**, the **text of `style` elements** and **comment data**: refused when the existing `dangerousScheme` pattern matches anywhere in the decoded value. The conservative rule stays for CSS and lists, where prose does not belong.
|
||||
- **Text content is not checked for schemes.** It is not a URL; a mail client does not follow it. `metadata:` or `Big Data: Summit` in a body, a header or a footer passes.
|
||||
- **Event handlers.** The raw-markup regexes for `script`/`iframe`/`object`/`embed` tags and for whitespace-prefixed `on*=` stay as they are and run first. The tokenizer walk additionally refuses any attribute whose name starts with `on`, which closes `<img src="x"onerror=...>`, the form the regex misses (no legitimate mail markup uses such an attribute).
|
||||
- **Error messages are unchanged** ("rendered HTML contains a dangerous URL scheme", "rendered HTML contains event handlers"). The snippet is the raw markup of the offending token, so an editor error shows the encoded form the administrator wrote.
|
||||
- **Draft layout CSS** is checked as the content of a style element, so the error stays on `content_css`.
|
||||
|
||||
### Site names never stop a send (`postcard.UnsafeText`, `siteNameFrom`, `layoutSiteName`)
|
||||
|
||||
- **`postcard.UnsafeText(s string) bool`** (exported, so cabana can refuse a name at save): a text is unsafe when, HTML-escaped with the standard library's `html.EscapeString` as html/template prints text, it fails the rendered-HTML checks (for example event-handler-shaped text such as `Shop onclick=x`), or when it holds a script URL (`scriptURLText`): `javascript:` or `vbscript:` at a scheme boundary, or `data:` at a scheme boundary directly followed by a non-space character (a data URL). A scheme boundary is the start of the text or a character that cannot belong to a URL scheme (anything but ASCII letters, digits, `+`, `.` and `-`). `Big Data: Summit`, `Metadata: Weekly` and `Metadata:Weekly` are safe; `javascript:alert(1)`, `javascript: Acme` and `data:text/html,x` are not.
|
||||
- **Render-time drop.** An unsafe Branding Site name is treated as empty, so the documented fallback applies (the display name of `mail.from`, if that one is safe); an unsafe `mail.from` display name gives no name. This holds on every render path: the file version, the override render built from it, the override-failure fallback and the editor. One warning per source per minute through the mailer's existing rate-limited `warn`, keyed `site_name:branding` or `site_name:mail.from`, message "postcard: site name left out of the layouts, it fails the mail safety check", with the source and never the value.
|
||||
- **The Branding preview** of an unsaved draft drops the name the same way, so the preview shows what would be sent; the save is what refuses it.
|
||||
- **Save rule.** cabana's `MailBrandSetting.BeforeValidate` refuses a name `postcard.UnsafeText` refuses: a 422 on `site_name` with `backend::lang.mail.brand.site_name_unsafe`, collected with the colour and logo details. Saving a template or a layout while an unsafe name is stored is not refused (the name is left out of the render).
|
||||
- **Copy** (`mail.brand.site_name_unsafe`): en "This name cannot be shown in mail. Remove addresses that start with javascript:, vbscript: or data: and text such as onclick=."; pl "Tej nazwy nie można pokazać w wiadomościach. Usuń adresy zaczynające się od javascript:, vbscript: lub data: oraz tekst w rodzaju onclick=."
|
||||
|
||||
### The inliner never stops mail (WR-05 contradiction)
|
||||
|
||||
The documented behaviour wins. The check sees decoded attribute values and style element CSS before inlining, whether or not the layout sets `disable_auto_inline_css`. An inlined result that still fails it is treated like an inliner error: the un-inlined HTML, which passed the check, is sent with its style block, and a warning keyed `inline:<layout>` is logged ("postcard: inlined CSS failed the safety check, sending the style block only").
|
||||
|
||||
### The footer prints the name literally (WR-01)
|
||||
|
||||
The `footer` partial gains a `text` argument, printed as plain text (a paragraph through html/template in the HTML part, verbatim in the text part); `body` stays Markdown for callers that pass Markdown on purpose. The default and system layouts pass the copyright line as `text`, so `*Acme*`, `Acme <Studio>` and `[x](url)` appear as typed, escaped, as in the header. The footer partial shipped in 35-06 and is in no tag yet (the newest tag, v0.6.4, predates it), so no released override depends on the old call.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
- **Keep the whole-document regex and only guard the site name.** Body text and other configuration text (`metadata:` in a template body, a variable, a partial argument) would keep stopping sends.
|
||||
- **A bluemonday-based mail sanitizer.** It rewrites instead of refusing, and would need a mail allowlist for style elements, tables and layout attributes. The 2026-10-11 note scopes bluemonday to stored rich text; this check only locates attribute values with x/net/html and keeps postcard's refuse-and-fall-back contract.
|
||||
- **Failing the render when the inlined result fails the check** (the reviewer's alternative for WR-05). It stops mail that passed the check, contradicts the documented promise and D-14.
|
||||
|
||||
## Residual risk accepted
|
||||
|
||||
- CSS escape sequences inside CSS (`\6a avascript:`) are not decoded. Mail clients do not run script URLs from CSS.
|
||||
- The unanchored raw handler regex `\son[a-z]+\s*=` still refuses prose such as " online = "; it is unchanged (out of the user's scope for this gap closure).
|
||||
|
||||
## Consequences
|
||||
|
||||
- postcard imports `golang.org/x/net/html` (no go.mod change) and the standard library `html` package.
|
||||
- `postcard.UnsafeText` is new exported API; `Mailer`, `Catalog.Register` and `pact.HasMailTemplates` keep their signatures.
|
||||
- Fixtures that pinned prose as unsafe change with the code (editor, fallback, partials and override smoke tests); the 35-07 subtest that pinned an unsafe stored name is replaced by `TestMailBrandSiteName/unsafe_name_refused`.
|
||||
Reference in New Issue
Block a user