fix(35): CR-02 check raw-text element and comment content as markup

The tokenizer-only walk read noscript, title, textarea, style, xmp and the
other raw-text elements as one text token and never looked inside, so an
unencoded javascript: link in <noscript> or <svg><title> passed the check
and was sent. A mail client renders with scripting off and parses svg
children as markup, so the raw content of those elements, and comment
data (Outlook conditional comments), is now checked again as markup;
nesting deeper than eight levels is refused.
This commit is contained in:
Jakub Zych
2026-11-14 17:30:07 +01:00
parent 2dbee30530
commit 1d481bb02d
4 changed files with 111 additions and 20 deletions

View File

@@ -155,7 +155,7 @@ fmt.Println(err != nil, len(driver.Messages()))
postcard does not pick a locale: the caller passes the full name. A per-language version of a template is a file named after the template, a dash and the locale (`welcome-en.htm`, sent as `acme.blog::mail.welcome-en`). A name with its own file always renders that file. A name the catalog does not hold, of the form `<template>-<locale>` where `<template>` is registered and `<locale>` is a locale tag (two letters, optionally followed by `-` or `_` and a 2 to 8 character subtag, such as `de` or `pt-BR`), sends the default-locale version: the locale's own version when an administrator customized it, else the customized template, else the template's file. Any other unknown name fails with `postcard: unknown template <name>`. So a plugin can pass `<template>-<locale>` for every locale it serves, and a locale without a file still sends mail, in the default language until someone writes that locale's version in the admin.
Before a driver sees a message, postcard refuses a subject or address with a line break, parses every address with `net/mail`, and rejects rendered HTML that contains script, iframe, object or embed tags, inline event handlers, or `javascript:`, `vbscript:` or `data:` URLs. URLs are looked for where a mail client reads one: URL-bearing attributes such as `href` and `src`, `style` attributes, style elements, comments and `meta` content, after character references are decoded, so `href="javascript&#58;..."` is refused. Text is not a URL: a body or a site name such as `metadata:` or `Big Data: Summit` is sent. Variables are escaped by Go's `html/template`.
Before a driver sees a message, postcard refuses a subject or address with a line break, parses every address with `net/mail`, and rejects rendered HTML that contains script, iframe, object or embed tags, inline event handlers, or `javascript:`, `vbscript:` or `data:` URLs. URLs are looked for where a mail client reads one: URL-bearing attributes such as `href` and `src`, `style` attributes, style elements, comments and `meta` content, after character references are decoded, so `href="javascript&#58;..."` is refused. Markup inside `noscript`, `title`, `textarea`, `style`, `xmp` and similar elements, and inside comments, is checked as well, since a mail client may read it as markup. Text is not a URL: a body or a site name such as `metadata:` or `Big Data: Summit` is sent. Variables are escaped by Go's `html/template`.
`postcard.Mailer.Send` delivers before it returns. To keep a request fast, send from a queued job, and send after the write that triggered the mail has committed; see [Queued jobs](jobs.md) and [Transactions](../database/transactions.md).

View File

@@ -27,7 +27,7 @@ Use cases:
- Layouts with a header, a text wrapper and an HTML wrapper, referenced from templates by a short alias (`pact.HasMailTemplates` maps aliases to full names). Three layouts are built in and usable from any plugin's templates: `default`, `system` and `minimal`.
- Partials: reusable blocks named `<plugin>::mail.partials.<name>`, loaded from `views/mail/partials/<name>.htm` and declared through `pact.HasMailPartials`. Bodies and layouts call them with `{{ partial "<name>" "key" value }}`. The partials `button`, `panel`, `table` and `subcopy` are built in.
- Every message gets an HTML part (Markdown converted with goldmark) and a plain-text part. The brand CSS (`mail.brandCss`, or the saved Branding) and `mail.css` go into the layout's style block, and the rendered HTML is then inlined: each CSS rule is copied into the `style` attribute of the elements it matches (go-premailer), so mail clients that drop style blocks still show the design. A layout opts out with `disable_auto_inline_css`.
- Safety checks: rendered HTML with script, iframe, object or embed tags, inline event handlers (any attribute whose name starts with `on`) or `javascript:`/`vbscript:`/`data:` URLs is rejected. Schemes are looked for in URL-bearing attributes, `style` attributes, style elements, comments and `meta` content, after character references are decoded (`javascript&#58;` is `javascript:`); text is not a URL and is not checked for them. CR/LF in the subject or address headers is rejected; addresses are parsed with `net/mail`.
- Safety checks: rendered HTML with script, iframe, object or embed tags, inline event handlers (any attribute whose name starts with `on`) or `javascript:`/`vbscript:`/`data:` URLs is rejected. Schemes are looked for in URL-bearing attributes, `style` attributes, style elements, comments and `meta` content, after character references are decoded (`javascript&#58;` is `javascript:`). The content of `noscript`, `title`, `textarea`, `style`, `xmp` and the other raw-text elements, and of comments, is checked as markup too, because a mail client may read it so (scripting off, `svg` content, Outlook conditional comments); such elements nested more than eight deep are refused. Text is not a URL and is not checked for them. CR/LF in the subject or address headers is rejected; addresses are parsed with `net/mail`.
- Drivers behind the `postcard.Driver` interface: `postcard.MemoryDriver` (keeps messages for tests), `postcard.LogDriver` (logs headers and the text part through `log/slog`, never the HTML or credentials), `postcard.SMTPDriver` (go-mail with an explicit TLS policy) and `postcard.FailDriver` (a deterministic failure for tests).
- Database overrides: a customized row of `system_mail_templates` replaces a template's subject, body and layout, a customized row of `system_mail_layouts` or `system_mail_partials` replaces a layout or a partial, and `system_mail_layouts` also holds layouts that exist only in the database. The mailer reads them on every send, with no cache, in every process that publishes a `*gorm.DB`. A missing database, tables that are not migrated yet, or an override that does not render or fails the safety checks sends the file version.
- Branding: once the branding settings have been saved, the brand CSS is compiled from their colours (`postcard.CompileBrandCSS`) and replaces `mail.brandCss`, and layouts get the logo's absolute URL as `{{ .logoUrl }}` and the site name as `{{ .siteName }}` (else the display name of `mail.from`). The built-in default layout shows them in a header and a copyright footer.

View File

@@ -92,6 +92,25 @@ func hasDangerousScheme(val string) bool {
return strings.HasPrefix(v, "javascript:") || strings.HasPrefix(v, "vbscript:") || strings.HasPrefix(v, "data:")
}
// rawTextElements are the elements whose content x/net/html's tokenizer
// reads as one text token (raw text or RCDATA). A mail client may parse that
// content as markup: it renders with scripting off, so noscript content is
// markup, and title, textarea, style, xmp and the like are foreign elements
// with markup children inside svg or math. unsafeMarkup checks their content
// as markup too.
var rawTextElements = map[string]bool{
"iframe": true,
"noembed": true,
"noframes": true,
"noscript": true,
"plaintext": true,
"script": true,
"style": true,
"textarea": true,
"title": true,
"xmp": true,
}
// unsafeMarkup checks the markup of rendered HTML: event-handler attributes
// and javascript:, vbscript: or data: URLs. It reads doc with x/net/html's
// tokenizer, which lower-cases attribute names and decodes character
@@ -105,19 +124,43 @@ func hasDangerousScheme(val string) bool {
// so prose such as "metadata:" or "Big Data: Summit" is not checked. Any
// attribute whose name starts with "on" is an event handler, with or
// without whitespace before it.
//
// The content of a raw-text element (rawTextElements) and of a comment is
// checked again as markup, because a mail client may read it so (noscript
// with scripting off, title or textarea in svg, Outlook's conditional
// comments). Each nested check reads a strictly shorter text, so the
// recursion ends; markup nested deeper than maxMarkupDepth is refused.
func unsafeMarkup(doc string) error {
return unsafeMarkupAt(doc, 0)
}
// maxMarkupDepth is how deep raw-text elements and comments may nest inside
// each other. Mail markup needs one or two levels (a style element, an
// Outlook conditional comment); a deeper document is refused rather than
// read again and again.
const maxMarkupDepth = 8
func unsafeMarkupAt(doc string, depth int) error {
if depth > maxMarkupDepth {
return &unsafeError{msg: "rendered HTML nests raw-text elements or comments too deeply", snippet: snippetAt(doc, []int{0, min(len(doc), 48)})}
}
z := xhtml.NewTokenizer(strings.NewReader(doc))
offset := 0
inStyle := false
// rawTag is the raw-text element whose content the next text token is.
rawTag := ""
for {
tt := z.Next()
if tt == xhtml.ErrorToken {
return nil
}
raw := string(z.Raw())
start := offset
offset += len(z.Raw())
offset += len(raw)
loc := []int{start, offset}
t := z.Token()
inRaw := rawTag != ""
rawTag = ""
switch tt {
case xhtml.StartTagToken, xhtml.SelfClosingTagToken:
for _, a := range t.Attr {
@@ -135,7 +178,12 @@ func unsafeMarkup(doc string) error {
return &unsafeError{msg: "rendered HTML contains a dangerous URL scheme", snippet: snippetAt(doc, loc)}
}
}
if tt == xhtml.StartTagToken && t.Data == "style" {
// The tokenizer reads the content of a raw-text element as text
// even after a self-closing start tag (<style/>), as browsers do.
if rawTextElements[t.Data] {
rawTag = t.Data
}
if t.Data == "style" {
inStyle = true
}
case xhtml.EndTagToken:
@@ -146,10 +194,21 @@ func unsafeMarkup(doc string) error {
if inStyle && dangerousScheme.MatchString(t.Data) {
return &unsafeError{msg: "rendered HTML contains a dangerous URL scheme", snippet: snippetAt(doc, loc)}
}
// The raw source, not the decoded text: RCDATA (title,
// textarea) is decoded by the tokenizer, and &lt;a&gt; in it is
// text to every reader.
if inRaw {
if err := unsafeMarkupAt(raw, depth+1); err != nil {
return err
}
}
case xhtml.CommentToken:
if dangerousScheme.MatchString(t.Data) {
return &unsafeError{msg: "rendered HTML contains a dangerous URL scheme", snippet: snippetAt(doc, loc)}
}
if err := unsafeMarkupAt(t.Data, depth+1); err != nil {
return err
}
}
}
}

View File

@@ -25,6 +25,11 @@ func TestURLSchemeCheck(t *testing.T) {
{"img_alt", `<img alt="Data: Summit" src="https://cdn.example.com/a.png">`},
{"title_text", `<td title="javascript: tips">Tips</td>`},
{"scheme_in_query", `<a href="https://example.com/?q=javascript:x">x</a>`},
// Escaped markup in a raw-text element is text to every reader.
{"textarea_escaped_link", `<textarea>&lt;a href="javascript:x"&gt;</textarea>`},
{"title_escaped_link", `<svg><title>&lt;a href="javascript:x"&gt;</title></svg>`},
{"noscript_safe_link", `<noscript><a href="https://example.com/">Open</a></noscript>`},
{"outlook_conditional_vml", `<!--[if mso]><v:roundrect href="https://example.com/" style="height:40px"><w:anchorlock/></v:roundrect><![endif]-->`},
}
for _, tc := range accepts {
t.Run("accept/"+tc.name, func(t *testing.T) {
@@ -63,6 +68,17 @@ func TestURLSchemeCheck(t *testing.T) {
{"tab_split", `<a href="java&#9;script:alert(1)">x</a>`, `java&#9;script:alert(1)`, schemeMsg},
{"newline_split", `<a href="java&#x0A;script:alert(1)">x</a>`, `java&#x0A;script:alert(1)`, schemeMsg},
{"handler_without_space", `<img src="x.png"onerror="alert(1)">`, `onerror="alert(1)"`, "event handlers"},
// Raw-text element content that a mail client parses as markup
// (scripting off, foreign content in svg) (CR-02).
{"noscript_href", `<noscript><a href="javascript:alert(1)">x</a></noscript>`, `href="javascript:alert(1)"`, schemeMsg},
{"noscript_handler", `<noscript><img src="x"/onerror=alert(1)></noscript>`, `onerror=alert(1)`, "event handlers"},
{"svg_title_href", `<svg><title><a href="javascript:alert(1)">x</a></title></svg>`, `href="javascript:alert(1)"`, schemeMsg},
{"svg_textarea_href", `<svg><textarea><a href="javascript:alert(1)">x</a></textarea></svg>`, `href="javascript:alert(1)"`, schemeMsg},
{"svg_style_entity", `<svg><style><a href="javascript&#58;alert(1)">x</a></style></svg>`, `javascript&#58;alert(1)`, schemeMsg},
{"xmp_href", `<xmp><a href="javascript:alert(1)">x</a></xmp>`, `href="javascript:alert(1)"`, schemeMsg},
{"self_closing_style", `<style/><a href="javascript&#58;alert(1)">x</a>`, `javascript&#58;alert(1)`, schemeMsg},
{"conditional_comment_handler", `<!--[if mso]><img src="x.png" onerror="alert(1)"><![endif]-->`, `onerror="alert(1)"`, "event handlers"},
{"nested_too_deep", strings.Repeat("<noscript>", maxMarkupDepth+2) + "x", "<noscript>", "too deeply"},
}
for _, tc := range refuses {
t.Run("refuse/"+tc.name, func(t *testing.T) {
@@ -88,6 +104,14 @@ func TestURLSchemeCheck(t *testing.T) {
// send fall back to the file version, whether or not the layout opts
// out of inlining; the encoded link never reaches the driver.
const encodedLink = `<a href="javascript&#58;alert(1)">x</a>`
// The same holds for an unencoded link hidden in a raw-text element that
// html/template leaves as written (CR-02; it escapes title and textarea
// text itself).
links := []struct{ name, link string }{
{"encoded", encodedLink},
{"noscript", `<noscript><a href="javascript:alert(1)">x</a></noscript>`},
{"xmp", `<xmp><a href="javascript:alert(1)">x</a></xmp>`},
}
file := fileWelcome(t)
if strings.Contains(file.HTML, "javascript") {
t.Fatalf("the file version holds a script link:\n%s", file.HTML)
@@ -97,23 +121,25 @@ func TestURLSchemeCheck(t *testing.T) {
if err := cat.Register(plugin.ID(), plugin.MailTemplatesFS(), plugin.MailTemplates(), plugin.MailLayouts()); err != nil {
t.Fatal(err)
}
for _, optOut := range []bool{false, true} {
name := "send/inline"
if optOut {
name = "send/opt_out"
for _, l := range links {
for _, optOut := range []bool{false, true} {
name := "send/" + l.name + "/inline"
if optOut {
name = "send/" + l.name + "/opt_out"
}
t.Run(name, func(t *testing.T) {
t.Parallel()
assertFileFallback(t, cat, file, Options{Overrides: fakeOverrides{ov: Overrides{
Template: &TemplateOverride{Subject: "Hello {{ .name }}", Body: "Hi **{{ .name }}**.", Layout: "encoded"},
Layouts: map[string]LayoutOverride{"encoded": {
HTML: `<html><head><style>{{ .css }}</style></head><body><div>{{ .Content }}` + l.link + `</div></body></html>`,
Text: "{{ .Content }}",
CSS: "p { color: #123456; }",
DisableAutoInlineCSS: optOut,
}},
}}}, true)
})
}
t.Run(name, func(t *testing.T) {
t.Parallel()
assertFileFallback(t, cat, file, Options{Overrides: fakeOverrides{ov: Overrides{
Template: &TemplateOverride{Subject: "Hello {{ .name }}", Body: "Hi **{{ .name }}**.", Layout: "encoded"},
Layouts: map[string]LayoutOverride{"encoded": {
HTML: `<html><head><style>{{ .css }}</style></head><body><div>{{ .Content }}` + encodedLink + `</div></body></html>`,
Text: "{{ .Content }}",
CSS: "p { color: #123456; }",
DisableAutoInlineCSS: optOut,
}},
}}}, true)
})
}
// The editor refuses the same layout draft on content_html, and a
@@ -136,6 +162,12 @@ func TestURLSchemeCheck(t *testing.T) {
if !strings.Contains(de.Message, "javascript&#58;alert(1)") {
t.Errorf("message = %q, want the encoded href", de.Message)
}
for _, l := range links[1:] {
hidden := Draft{Kind: KindLayout, Name: defaultLayoutName, HTML: `<div>{{ .Content }}` + l.link + `</div>`, Text: "{{ .Content }}", DisableAutoInlineCSS: optOut}
if de := ed.Validate(ctx, hidden); de == nil || de.Field != fieldContentHTML || de.Rule != ruleUnsafe {
t.Errorf("%s link in a layout draft: %+v, want content_html/unsafe", l.name, de)
}
}
def, ok := NewCatalog().Layout(defaultLayoutName)
if !ok {