4 Commits

Author SHA1 Message Date
a3551f2bdf fix(classes): sanitize content_html with bluemonday instead of regex checks
- FormatHTML sanitizes the goldmark output with a bluemonday UGC policy
  extended only with what goldmark and the imported posts and categories
  use; unsafe markup is stripped, as Winter's Html::clean strips it
- HTML the policy keeps whole is stored byte for byte (token comparison)
- CheckHTML keeps its signature and 422 contract, refusing HTML the
  policy would change
- the three regexes are gone: the CR-01 bypasses are neutralized and
  "Data:" or "metadata:" text is no longer refused (WR-01)
2026-10-09 23:35:23 +02:00
03907ee387 test: FormatHTML keep and refuse coverage, admin re-save keeps HTML
- TestFormatHTMLKeepsRawHTML: imported-style bodies (paragraphs, attributes, https links and images, entities) round-trip; markdown keeps a trailing raw footer and style
- TestFormatHTMLRejectsUnsafeHTML: every unsafe case must return an error now that the gate is the only defence; adds svg onload, mixed-case schemes and tags, object, embed, script in an HTML block and raw data URLs
- TestPostsAdminResaveKeepsHTML: an admin update of HTML content keeps content_html
2026-10-09 22:48:59 +02:00
a675f2f197 chore(deps): require summercms v0.1.16 2026-10-09 21:32:49 +02:00
528d2d8451 fix(classes): keep raw HTML in content_html behind the safety gate
- FormatHTML renders with goldmark html.WithUnsafe(), as Winter's
  Markdown::parse passes raw HTML through, so an admin save, an import or
  an API write no longer turns imported HTML into raw HTML omitted
- rejectUnsafe still runs after the render and refuses script, iframe,
  object and embed tags, event handlers and javascript, vbscript and data
  URLs; API response shapes are unchanged
- TestFormatHTMLKeepsRawHTML covers HTML paragraphs and a raw footer
2026-10-09 20:58:49 +02:00
7 changed files with 491 additions and 54 deletions

View File

@@ -18,7 +18,7 @@ WinterCMS theme components such as `journalPost` and `journalPosts` are a later
## Features
- **Posts.** `models.Post` maps `golem15_journal_posts`. Translatable fields are title, content, content_html, excerpt, metadata, and slug (indexed). Fillable does not include `redactor_id`, `content_html`, or `user_id`. Attachments `featured_images` and `content_images` use `system_files` under the same MorphName. YAML admin uses `mltext`/`mlmarkdown`. `classes.FormatHTML` regenerates `content_html` on admin save.
- **Posts.** `models.Post` maps `golem15_journal_posts`. Translatable fields are title, content, content_html, excerpt, metadata, and slug (indexed). Fillable does not include `redactor_id`, `content_html`, or `user_id`. Attachments `featured_images` and `content_images` use `system_files` under the same MorphName. YAML admin uses `mltext`/`mlmarkdown`. `classes.FormatHTML` regenerates `content_html` on every write path (admin save, `journal:import-posts`, Journal API writes). It renders markdown and passes raw HTML in `content` through the renderer (Winter parity), then sanitizes the result with a `bluemonday` allow-list: bluemonday's UGC policy plus `footer`, `class`, schema.org microdata, the `color`, `font-size`, `font-weight` and `text-align` styles, goldmark's footnote roles, `dir="auto"` and two pasted-transcript `data-message-*` attributes. Like Winter's `Html::clean`, it strips what the policy does not allow instead of refusing the save: script, style, iframe, object, embed, form, base and meta elements, comments, event handler attributes, and URLs other than http, https, mailto and relative ones. HTML the policy keeps whole is stored byte for byte, so imported bodies survive a save. `content` itself is stored as submitted. The API response shapes are unchanged.
- **Categories and tags.** `models.Category` is translatable (name, description, slug). It also stores `content`, an HTML host column that is not translatable, edited in the admin as a textarea. Fillable is name, slug, code, description, content and parent_id; nest_* columns exist in DDL and are not fillable. `models.Tag` is not translatable. Fillable is name, slug, description. Pivots are `golem15_journal_posts_categories` and `golem15_journal_posts_tags`.
- **Settings.** Singleton row ID=1 on `golem15_journal_settings`, not PHP `system_settings`. `search_use_typesense` defaults to false. A fresh save never contacts Typesense.
- **Public HTTP.** Anonymous GET `/_journal/api/v1/posts`, `/posts/{slug}`, `/categories`, `/tags`, `/rss`, `/authors/{author}`, and `/authors/{author}/posts`. Writes under the same prefix require a cabana backend JWT (`aud=backend`).
@@ -177,7 +177,7 @@ func (Post) MorphName() string { return `Golem15\Journal\Models\Post
| `models.Settings` | Singleton settings row; `TableName` is `golem15_journal_settings`. |
| `models.All` | Models registered by this plugin. |
| `updates.All` | gormigrate steps registered by this plugin. |
| `classes.CheckHTML` | The `FormatHTML` safety gate for HTML stored as is, such as category content. |
| `classes.CheckHTML` | Gate for HTML stored as is, such as category content: refuses HTML that the `FormatHTML` allow-list would strip or change, instead of stripping it. |
## CLI commands
@@ -198,7 +198,7 @@ Both commands write JSON files (one per post slug), matching the PHP `journal:ex
- SummerCMS modules: backpack, beachcomber, bonfire, cabana, lagoon/attach, pact, party, surf.
- Compiled plugin: `golem15.translate`.
- Third-party: `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`, `github.com/yuin/goldmark`, `gocloud.dev`. Test-only: `github.com/testcontainers/testcontainers-go`.
- Third-party: `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`, `github.com/yuin/goldmark`, `github.com/microcosm-cc/bluemonday`, `golang.org/x/net/html`, `gocloud.dev`. Test-only: `github.com/testcontainers/testcontainers-go`.
## Testing

View File

@@ -11,14 +11,29 @@ func TestCheckHTML(t *testing.T) {
{"empty", "", true},
{"heading and link", `<h2>Cordyceps</h2><p>Plain <a href="https://example.test/a">link</a></p>`, true},
{"table", `<table><thead><tr><th>A</th></tr></thead><tbody><tr><td>1</td></tr></tbody></table>`, true},
{"polish date label", `<p>Data: 12 maja 2024</p>`, true},
{"metadata text", `<p>Some metadata: here</p>`, true},
{"quotes, apostrophes and nbsp", `<h2>Co to jest Lion's Mane?&nbsp;</h2><p>"superior herb" &amp; inne</p>`, true},
{"pasted chat transcript", `<div data-message-author-role="assistant" data-message-id="cf51192a-db36-4816-8d29-b0606b53d289" dir="auto"><p>x</p></div>`, true},
{"script", `<script>alert(1)</script>`, false},
{"iframe", `<iframe src="https://example.test/"></iframe>`, false},
{"object", `<object data=x>`, false},
{"embed", `<embed src=x>`, false},
{"event handler", `<img src=x onerror="alert(1)">`, false},
{"event handler without a space", `<p><img src="x"onerror="alert(1)"></p>`, false},
{"slash-separated svg onload", "<div>\n<svg/onload=alert(1)>\n</div>", false},
{"javascript url", `<a href="javascript:alert(1)">`, false},
{"numeric entity colon", `<a href="javascript&#58;alert(1)">x</a>`, false},
{"named entity colon", `<a href="javascript&colon;alert(1)">x</a>`, false},
{"entity inside the scheme", `<a href="&#106;avascript:alert(1)">x</a>`, false},
{"tab inside the scheme", "<a href=\"java\tscript:alert(1)\">x</a>", false},
{"vbscript url mixed case", `<a href="VBScript:x">`, false},
{"data url", `<img src="data:image/png;base64,AAAA">`, false},
{"base", `<base href="https://evil.test/">`, false},
{"meta refresh", `<meta http-equiv="refresh" content="0;url=https://evil.test">`, false},
{"form", `<form action="https://evil.test"><input type=password></form>`, false},
{"style element", `<style>p { color: red }</style>`, false},
{"comment", `<p>a</p><!-- hidden -->`, false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
@@ -31,4 +46,16 @@ func TestCheckHTML(t *testing.T) {
}
})
}
// Every stored-XSS payload FormatHTML strips is refused by CheckHTML.
for _, tc := range unsafePayloads {
if tc.src[0] != '<' {
continue // markdown, not HTML
}
t.Run("payload "+tc.name, func(t *testing.T) {
if err := CheckHTML(tc.src); err == nil {
t.Fatalf("CheckHTML(%q) = nil, want an error", tc.src)
}
})
}
}

View File

@@ -2,57 +2,125 @@ package classes
import (
"bytes"
"errors"
"fmt"
"io"
"regexp"
"slices"
"strings"
"github.com/microcosm-cc/bluemonday"
"github.com/yuin/goldmark"
"github.com/yuin/goldmark/extension"
"github.com/yuin/goldmark/parser"
"github.com/yuin/goldmark/renderer/html"
xhtml "golang.org/x/net/html"
)
var (
markdownEngine = goldmark.New(
goldmark.WithExtensions(extension.Footnote, extension.Table),
goldmark.WithParserOptions(parser.WithAttribute()),
// Winter's Markdown::parse passes raw HTML through, so imported HTML
// bodies round-trip (D-03, PHP parity); htmlPolicy then sanitizes the
// rendered result on every write path.
goldmark.WithRendererOptions(html.WithUnsafe()),
)
markdownUnsafeTag = regexp.MustCompile(`(?i)<(?:script|iframe|object|embed)\b`)
markdownEventHandler = regexp.MustCompile(`(?i)\son[a-z]+\s*=`)
markdownDangerousURL = regexp.MustCompile(`(?i)(?:javascript|vbscript|data):`)
// htmlPolicy is the allow-list for stored journal HTML.
htmlPolicy = newHTMLPolicy()
)
// newHTMLPolicy extends bluemonday's UGC policy only with what goldmark
// renders and what the imported posts and categories use.
func newHTMLPolicy() *bluemonday.Policy {
p := bluemonday.UGCPolicy()
// Winter neither adds nor requires rel="nofollow", so links keep their
// stored form.
p.RequireNoFollowOnLinks(false)
// Imported author footers: <footer class=... itemscope itemtype=...>.
p.AllowElements("footer")
p.AllowAttrs("class").Matching(bluemonday.SpaceSeparatedTokens).Globally()
p.AllowAttrs("itemscope").Matching(regexp.MustCompile(`^(|itemscope)$`)).Globally()
p.AllowAttrs("itemtype").Matching(regexp.MustCompile(`^https?://schema\.org/[A-Za-z]+$`)).Globally()
p.AllowAttrs("itemprop").Matching(bluemonday.SpaceSeparatedTokens).Globally()
// Inline styles in imported footers and goldmark's table cell alignment.
// bluemonday validates each property value.
p.AllowStyles("color", "font-size", "font-weight", "text-align").Globally()
// goldmark's footnote markup.
p.AllowAttrs("role").Matching(regexp.MustCompile(`^doc-(noteref|endnotes|backlink)$`)).OnElements("a", "div")
// Pasted chat transcripts in imported category bodies: dir="auto" (the
// UGC policy allows only ltr and rtl) and two data attributes.
p.AllowAttrs("dir").Matching(regexp.MustCompile(`(?i)^(ltr|rtl|auto)$`)).Globally()
p.AllowAttrs("data-message-author-role", "data-message-id").
Matching(regexp.MustCompile(`^[A-Za-z0-9_-]+$`)).OnElements("div")
return p
}
// FormatHTML converts journal post markdown to stored HTML using goldmark
// footnote, table, and attribute extensions, then the same rejectUnsafe gate
// as cabana.RenderMarkdown. It does not call cabana.RenderMarkdown.
// footnote, table, and attribute extensions. Raw HTML in the source passes
// through the renderer, as it does in Winter's Markdown::parse. The result is
// then sanitized with a bluemonday allow-list that strips, as Winter's
// Html::clean does, what it does not allow: script, style, iframe, object,
// embed, form, base and meta elements, event handler attributes, and URLs
// other than http, https, mailto and relative ones. HTML that the policy
// keeps whole is returned byte for byte, so imported bodies round-trip. It
// does not call cabana.RenderMarkdown, whose admin preview stays without
// unsafe HTML.
func FormatHTML(src string) (string, error) {
var buf bytes.Buffer
if err := markdownEngine.Convert([]byte(strings.TrimSpace(src)), &buf); err != nil {
return "", fmt.Errorf("journal: markdown: %w", err)
}
html := buf.String()
if err := rejectUnsafe(html); err != nil {
return "", err
}
return html, nil
out, _ := sanitizeHTML(buf.String())
return out, nil
}
// CheckHTML applies the FormatHTML safety gate to HTML that is stored as is,
// such as category content. It refuses raw script, iframe, object and embed
// tags, event handler attributes and javascript, vbscript or data URL schemes.
func CheckHTML(html string) error {
return rejectUnsafe(html)
}
func rejectUnsafe(html string) error {
if markdownUnsafeTag.MatchString(html) {
return fmt.Errorf("journal: rendered HTML contains raw unsafe tags")
}
if markdownEventHandler.MatchString(html) {
return fmt.Errorf("journal: rendered HTML contains event handlers")
}
if markdownDangerousURL.MatchString(html) {
return fmt.Errorf("journal: rendered HTML contains a dangerous URL scheme")
// CheckHTML gates HTML that is stored as is, such as category content. It
// uses the FormatHTML allow-list but refuses instead of stripping: doc is
// refused when the policy would remove or change any element, attribute or
// text in it.
func CheckHTML(doc string) error {
if _, changed := sanitizeHTML(doc); changed {
return errors.New("journal: HTML contains elements, attributes or URLs that are not allowed")
}
return nil
}
// sanitizeHTML returns doc unchanged when htmlPolicy keeps all of it, and the
// policy's output with changed set otherwise. Keeping all of it means doc and
// the policy output tokenize to the same HTML tokens, so an output that only
// re-escapes text or attribute values is not a change.
func sanitizeHTML(doc string) (string, bool) {
clean := htmlPolicy.Sanitize(doc)
if clean == doc || sameTokens(doc, clean) {
return doc, false
}
return clean, true
}
// sameTokens reports whether a and b tokenize to the same sequence of HTML
// tokens: type, tag name or decoded text, and the set of decoded attributes.
func sameTokens(a, b string) bool {
ta, oka := htmlTokens(a)
tb, okb := htmlTokens(b)
return oka && okb && slices.Equal(ta, tb)
}
func htmlTokens(doc string) ([]string, bool) {
var out []string
z := xhtml.NewTokenizer(strings.NewReader(doc))
for {
tt := z.Next()
if tt == xhtml.ErrorToken {
return out, errors.Is(z.Err(), io.EOF)
}
t := z.Token()
attrs := make([]string, 0, len(t.Attr))
for _, a := range t.Attr {
attrs = append(attrs, a.Namespace+":"+a.Key+"="+a.Val)
}
slices.Sort(attrs)
out = append(out, fmt.Sprintf("%d|%s|%q", tt, t.Data, attrs))
}
}

View File

@@ -3,59 +3,303 @@ package classes
import (
"strings"
"testing"
xhtml "golang.org/x/net/html"
)
// TestFormatHTMLRejectsUnsafeHTML substitutes JOURNAL-003/004 Twig XSS
// templates (Phase 16). Script, iframe, event handlers, and javascript /
// vbscript / data schemes must fail the reject gate. Footnotes and tables
// from goldmark extensions still pass.
func TestFormatHTMLRejectsUnsafeHTML(t *testing.T) {
// unsafePayloads are the stored-XSS inputs FormatHTML and CheckHTML must
// neutralize: the JOURNAL-003/004 Twig XSS templates (Phase 16) and the
// regex-gate bypasses from the Phase 16.1 code review (CR-01).
var unsafePayloads = []struct {
name string
src string
}{
{"script", "<script>alert(1)</script>"},
{"mixed-case script", "<ScRiPt>alert(1)</sCrIpT>"},
{"script inside an HTML block", "<div>\n<p>Intro</p>\n<script>alert(1)</script>\n</div>"},
{"script after markdown", "# Title\n\nText.\n\n<script src=\"https://evil.test/x.js\"></script>"},
{"iframe", `<iframe src="https://evil.test"></iframe>`},
{"upper-case iframe", `<IFRAME SRC="https://evil.test"></IFRAME>`},
{"object", `<object data="https://evil.test/x.swf"></object>`},
{"embed", `<embed src="https://evil.test/x.swf">`},
{"event", `<img src=x onerror="alert(1)">`},
{"svg onload", `<svg onload="alert(1)"><circle r="1"/></svg>`},
{"mixed-case event handler", `<p OnClick="alert(1)">Click</p>`},
{"event handler in an inline element", "Text with <span onmouseover=\"alert(1)\">hover</span>."},
{"event handler without a space", `<p><img src="x"onerror="alert(1)"></p>`},
{"slash-separated svg onload", "<div>\n<svg/onload=alert(1)>\n</div>"},
{"markdown attribute event handler", "# Title {onclick=\"alert(1)\"}"},
{"javascript", "[x](javascript:alert(1))"},
{"mixed-case javascript href", `<a href="JaVaScRiPt:alert(1)">x</a>`},
{"numeric entity colon", `<a href="javascript&#58;alert(1)">x</a>`},
{"named entity colon", `<a href="javascript&colon;alert(1)">x</a>`},
{"entity inside the scheme", `<a href="&#106;avascript:alert(1)">x</a>`},
{"tab inside the scheme", "<a href=\"java\tscript:alert(1)\">x</a>"},
{"newline inside the scheme", "<p><a href=\"java\nscript:alert(1)\">x</a></p>"},
{"leading space and control characters", "<p><a href=\" \x01javascript:alert(1)\">x</a></p>"},
{"mathml xlink href", "<div>\n<math><maction actiontype=statusline xlink:href=\"javascript&#58;alert(1)\">x</maction></math>\n</div>"},
{"vbscript", "[x](vbscript:msgbox(1))"},
{"raw vbscript href", `<a href="VBScript:msgbox(1)">x</a>`},
{"data", "[x](data:text/html;base64,PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==)"},
{"raw data URL attribute", `<img src="data:image/svg+xml;base64,PHN2Zz48L3N2Zz4=">`},
{"markdown image with a data URL", "![x](data:image/png;base64,iVBORw0KGgo=)"},
{"base", `<base href="https://evil.test/">`},
{"meta refresh", `<meta http-equiv="refresh" content="0;url=https://evil.test">`},
{"form", `<form action="https://evil.test"><input type=password name=p><button formaction="javascript:alert(1)">go</button></form>`},
{"style element", "<style>body { background: url(https://evil.test/x) }</style>"},
{"link stylesheet", `<link rel="stylesheet" href="https://evil.test/x.css">`},
{"html comment", "<p>a</p><!-- <script>alert(1)</script> -->"},
}
// forbiddenElements must never appear in sanitized journal HTML.
var forbiddenElements = map[string]bool{
"script": true, "iframe": true, "object": true, "embed": true, "base": true,
"meta": true, "form": true, "input": true, "button": true, "style": true,
"link": true, "svg": true, "math": true, "maction": true, "frame": true,
"frameset": true, "applet": true, "textarea": true, "select": true,
}
// assertNoActiveContent tokenizes doc and fails on forbidden elements, event
// handler attributes, any style other than the allowed properties, and
// javascript, vbscript or data URLs after entity decoding and after removing
// ASCII whitespace and control characters, as a browser's URL parser does.
func assertNoActiveContent(t *testing.T, doc string) {
t.Helper()
z := xhtml.NewTokenizer(strings.NewReader(doc))
for {
tt := z.Next()
if tt == xhtml.ErrorToken {
return
}
if tt == xhtml.CommentToken {
t.Fatalf("comment survived: %q", doc)
}
if tt != xhtml.StartTagToken && tt != xhtml.SelfClosingTagToken && tt != xhtml.EndTagToken {
continue
}
tok := z.Token()
if forbiddenElements[tok.Data] {
t.Fatalf("element <%s> survived: %q", tok.Data, doc)
}
for _, a := range tok.Attr {
key := strings.ToLower(a.Key)
if strings.HasPrefix(key, "on") {
t.Fatalf("event handler %s survived: %q", a.Key, doc)
}
if key == "action" || key == "formaction" || key == "srcdoc" || key == "http-equiv" {
t.Fatalf("attribute %s survived: %q", a.Key, doc)
}
var b strings.Builder
for _, r := range strings.ToLower(a.Val) {
if r > 0x20 {
b.WriteRune(r)
}
}
v := b.String()
for _, scheme := range []string{"javascript:", "vbscript:", "data:"} {
if strings.Contains(v, scheme) {
t.Fatalf("%s URL survived in %s: %q", scheme, a.Key, doc)
}
}
}
}
}
// TestFormatHTMLSanitizesUnsafeHTML pins CR-01: every payload is rendered
// without an error and with all active content stripped, as Winter's
// Html::clean strips rather than refuses. Footnotes and tables from goldmark
// extensions still render.
func TestFormatHTMLSanitizesUnsafeHTML(t *testing.T) {
html, err := FormatHTML("# Hello")
if err != nil || !strings.Contains(html, "<h1>") {
t.Fatalf("safe heading: html=%s err=%v", html, err)
}
table, err := FormatHTML("| a | b |\n| - | - |\n| 1 | 2 |")
table, err := FormatHTML("| a | b |\n|:- | -:|\n| 1 | 2 |")
if err != nil {
t.Fatalf("table markdown: %v", err)
}
if !strings.Contains(table, "<table") {
t.Fatalf("table did not render: %s", table)
if !strings.Contains(table, "<table") || !strings.Contains(table, "text-align: right") {
t.Fatalf("table or its alignment did not render: %s", table)
}
fn, err := FormatHTML("See footnote.[^1]\n\n[^1]: note")
if err != nil {
t.Fatalf("footnote markdown: %v", err)
}
if !strings.Contains(strings.ToLower(fn), "footnote") && !strings.Contains(fn, "<sup") {
t.Fatalf("footnote did not render: %s", fn)
for _, want := range []string{`<sup id="fnref:1">`, `href="#fn:1"`, `role="doc-noteref"`, `role="doc-endnotes"`, `role="doc-backlink"`} {
if !strings.Contains(fn, want) {
t.Fatalf("footnote lacks %s: %s", want, fn)
}
}
attrs, err := FormatHTML("# Title {#intro .lead}")
if err != nil || !strings.Contains(attrs, `<h1 id="intro" class="lead">Title</h1>`) {
t.Fatalf("markdown attributes: html=%s err=%v", attrs, err)
}
for _, tc := range unsafePayloads {
t.Run(tc.name, func(t *testing.T) {
out, err := FormatHTML(tc.src)
if err != nil {
t.Fatalf("FormatHTML(%q) = %v, want sanitized output", tc.src, err)
}
assertNoActiveContent(t, out)
})
}
t.Run("safe parts of a payload survive", func(t *testing.T) {
out, err := FormatHTML(`<p>Zdjęcie: <img src="https://cdn.example.test/a.jpg" alt="a"onerror="alert(1)"></p>`)
if err != nil {
t.Fatal(err)
}
if want := `<p>Zdjęcie: <img src="https://cdn.example.test/a.jpg" alt="a"></p>`; out != want {
t.Fatalf("got %q, want %q", out, want)
}
})
}
// TestFormatHTMLKeepsSchemeLikeText pins WR-01: text that merely contains
// "data:", "javascript:" or "vbscript:" is not a URL and passes unchanged.
func TestFormatHTMLKeepsSchemeLikeText(t *testing.T) {
cases := []struct {
name string
src string
needle string
name, src, want string
}{
{"script", "<script>alert(1)</script>", "<script"},
{"iframe", `<iframe src="https://evil.test"></iframe>`, "<iframe"},
{"event", `<img src=x onerror="alert(1)">`, "onerror"},
{"javascript", "[x](javascript:alert(1))", "javascript:"},
{"vbscript", "[x](vbscript:msgbox(1))", "vbscript:"},
{"data", "[x](data:text/html;base64,PHNjcmlwdD5hbGVydCgxKTwvc2NyaXB0Pg==)", "data:"},
{"polish date label", "<p>Data: 12 maja 2024</p>", "<p>Data: 12 maja 2024</p>"},
{"markdown date label", "Data publikacji: 12 maja 2024", "<p>Data publikacji: 12 maja 2024</p>\n"},
{"metadata", "Some metadata: here", "<p>Some metadata: here</p>\n"},
{"big data in html", "<p>Big data: trends</p>", "<p>Big data: trends</p>"},
{"scheme names in prose", "<p>Pisze się javascript: i vbscript: w tekście.</p>", "<p>Pisze się javascript: i vbscript: w tekście.</p>"},
{"quotes and apostrophes", "<p>Lion's Mane, \"superior herb\" &amp; inne</p>", "<p>Lion's Mane, \"superior herb\" &amp; inne</p>"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
out, err := FormatHTML(tc.src)
if err == nil && strings.Contains(strings.ToLower(out), tc.needle) {
t.Fatalf("unsafe HTML survived: %s", out)
if err != nil {
t.Fatalf("FormatHTML(%q): %v", tc.src, err)
}
if err == nil && (tc.name == "script" || tc.name == "iframe" || tc.name == "event") && strings.Contains(strings.ToLower(out), tc.needle) {
t.Fatalf("FormatHTML accepted %s: %s", tc.name, out)
if out != tc.want {
t.Fatalf("got %q, want %q", out, tc.want)
}
})
}
}
// TestFormatHTMLKeepsRawHTML pins D-03: raw HTML in content passes through
// the renderer, as Winter's Markdown::parse does, and HTML the policy allows
// is stored byte for byte, so an imported HTML body survives an admin save,
// an import or an API write.
func TestFormatHTMLKeepsRawHTML(t *testing.T) {
const paragraphs = "<p>Grzyby rosną w lesie.</p>\n<p>Druga <b>część</b>.</p>"
out, err := FormatHTML(paragraphs)
if err != nil {
t.Fatalf("html paragraphs: %v", err)
}
if strings.TrimSpace(out) != paragraphs {
t.Fatalf("html paragraphs changed:\n%s", out)
}
mixed, err := FormatHTML("# Tytuł\n\nTekst.\n\n<footer class=\"author\">Autor</footer>")
if err != nil {
t.Fatalf("markdown with a raw footer: %v", err)
}
if !strings.Contains(mixed, "<h1>Tytuł</h1>") || !strings.Contains(mixed, `<footer class="author">Autor</footer>`) {
t.Fatalf("raw footer was dropped:\n%s", mixed)
}
// Imported-style bodies: each round-trips unchanged after trimming.
bodies := []struct {
name string
src string
}{
{"paragraphs", "<p>Pierwszy akapit.</p>\n<p>Drugi akapit z <em>kursywą</em>.</p>\n<p>Trzeci.</p>"},
{"class attributes", `<div class="post-body"><p class="lead">Wstęp</p></div>`},
{"https link", `<p>Zobacz <a href="https://example.test/grzyby?a=1" class="ext" title="Źródło">źródło</a>.</p>`},
{"relative and mailto links", `<p><a href="/blog/grzyby">blog</a> <a href="#przypisy">przypisy</a> <a href="mailto:kontakt@example.test">mail</a></p>`},
{"https image", `<p><img src="https://cdn.example.test/img/fungi.jpg" alt="Grzyby" width="640" height="480"></p>`},
{"entity in an attribute", `<p><a href="https://example.test/?a=1&amp;b=2" title="Tom i Jerry">link</a> <img src="https://cdn.example.test/a.jpg?w=1&amp;h=2" alt="Tom i Jerry"></p>`},
{"table", "<table>\n<tr><td>A</td><td>B</td></tr>\n</table>"},
{"headings", "<h2>Co to jest Lion's Mane?</h2>\n<h3>Właściwości&nbsp;</h3>"},
{"pasted chat transcript", `<div data-message-author-role="assistant" data-message-id="a13366a9-f96b-4d6f-aaa5-0e433176381a" dir="auto"><p>Tekst</p></div>`},
{"surrounding whitespace", "\n\n <p>Trimmed.</p> \n"},
}
for _, tc := range bodies {
t.Run(tc.name, func(t *testing.T) {
out, err := FormatHTML(tc.src)
if err != nil {
t.Fatalf("FormatHTML(%q): %v", tc.src, err)
}
if strings.TrimSpace(out) != strings.TrimSpace(tc.src) {
t.Fatalf("html changed:\n got %q\nwant %q", strings.TrimSpace(out), strings.TrimSpace(tc.src))
}
if strings.Contains(out, "raw HTML omitted") {
t.Fatalf("goldmark placeholder in output: %s", out)
}
})
}
t.Run("markdown with a trailing raw footer and style", func(t *testing.T) {
const footer = `<footer class="source"><p>Źródło: <a href="https://example.test/ref">ref</a></p></footer>`
const style = "<style>\n.source { color: #555; }\n</style>"
src := "## Nagłówek\n\nTekst z **pogrubieniem** i [linkiem](https://example.test).\n\n" + footer + "\n\n" + style
out, err := FormatHTML(src)
if err != nil {
t.Fatalf("markdown with footer and style: %v", err)
}
for _, want := range []string{
"<h2>Nagłówek</h2>",
"<strong>pogrubieniem</strong>",
`<a href="https://example.test">linkiem</a>`,
footer,
} {
if !strings.Contains(out, want) {
t.Fatalf("output lacks %q:\n%s", want, out)
}
}
// The style element is stripped with its content, as Html::clean
// strips <style>.
if strings.Contains(out, "<style") || strings.Contains(out, ".source {") {
t.Fatalf("style element survived:\n%s", out)
}
if strings.Contains(out, "raw HTML omitted") {
t.Fatalf("goldmark placeholder in output: %s", out)
}
})
t.Run("imported author footer with microdata and inline styles", func(t *testing.T) {
src := "Tekst.\n\n<footer class=\"author-footer\" itemscope itemtype=\"https://schema.org/Person\">\n" +
" <div class=\"author-info\">\n" +
" <img src=\"https://cdn.example.test/autor.png\" alt=\"Autor\" class=\"author-photo\" itemprop=\"image\">\n" +
" <div style=\"font-size: 1.5em; font-weight: bold;\" itemprop=\"name\">Jan Kowalski</div>\n" +
" <span style=\"color: #138c74;\" itemprop=\"jobTitle\">Ekspert</span>\n" +
" </div>\n</footer>"
out, err := FormatHTML(src)
if err != nil {
t.Fatal(err)
}
for _, want := range []string{
`<footer class="author-footer" itemscope="" itemtype="https://schema.org/Person">`,
`<img src="https://cdn.example.test/autor.png" alt="Autor" class="author-photo" itemprop="image">`,
`<div style="font-size: 1.5em; font-weight: bold" itemprop="name">Jan Kowalski</div>`,
`<span style="color: #138c74" itemprop="jobTitle">Ekspert</span>`,
} {
if !strings.Contains(out, want) {
t.Fatalf("output lacks %q:\n%s", want, out)
}
}
})
t.Run("disallowed styles and attributes are stripped", func(t *testing.T) {
out, err := FormatHTML(`<p style="position: fixed; color: red" target="_blank" itemtype="https://evil.test/x">x</p>`)
if err != nil {
t.Fatal(err)
}
if want := `<p style="color: red">x</p>`; out != want {
t.Fatalf("got %q, want %q", out, want)
}
})
}
// TestJournalEndToEnd keeps `go test ./... -run TestJournalEndToEnd`
// from reporting [no tests to run] in this package.
func TestJournalEndToEnd(t *testing.T) {}

7
go.mod
View File

@@ -5,13 +5,15 @@ go 1.27.0
require (
git.golem15.com/golem15/sm-translate-plugin v0.1.0
git.golem15.com/golem15/sm-user-plugin v0.1.0
git.golem15.com/golem15/summercms v0.1.9
git.golem15.com/golem15/summercms v0.1.16
github.com/go-gormigrate/gormigrate/v2 v2.1.7
github.com/jackc/pgx/v5 v5.10.0
github.com/microcosm-cc/bluemonday v1.0.27
github.com/testcontainers/testcontainers-go v0.44.0
github.com/testcontainers/testcontainers-go/modules/postgres v0.44.0
github.com/yuin/goldmark v1.8.6
gocloud.dev v0.46.0
golang.org/x/net v0.58.0
gorm.io/gorm v1.31.2
)
@@ -19,6 +21,7 @@ require (
dario.cat/mergo v1.0.2 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
@@ -45,6 +48,7 @@ require (
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/gax-go/v2 v2.19.0 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
@@ -106,7 +110,6 @@ require (
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/image v0.46.0 // indirect
golang.org/x/net v0.58.0 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/term v0.46.0 // indirect

6
go.sum
View File

@@ -72,6 +72,8 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.42.3 h1:ErklX/7uhSbkAAeyQD/Y1OoQ9hO3
github.com/aws/aws-sdk-go-v2/service/sts v1.42.3/go.mod h1:ULe4HCzfKPiR6R3HEurE3b1upEkuk8AkMrOKtaOxKO8=
github.com/aws/smithy-go v1.26.0 h1:9ouqbi+NyKP7fV3Te7UElCwdAb6Y8uk7LGwPE5tVe/s=
github.com/aws/smithy-go v1.26.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
@@ -153,6 +155,8 @@ github.com/googleapis/enterprise-certificate-proxy v0.3.14 h1:yh8ncqsbUY4shRD5dA
github.com/googleapis/enterprise-certificate-proxy v0.3.14/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
github.com/googleapis/gax-go/v2 v2.19.0 h1:fYQaUOiGwll0cGj7jmHT/0nPlcrZDFPrZRhTsoCr8hE=
github.com/googleapis/gax-go/v2 v2.19.0/go.mod h1:w2ROXVdfGEVFXzmlciUU4EdjHgWvB5h2n6x/8XSTTJA=
github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jackc/pgerrcode v0.0.0-20240316143900-6e2875d9b438 h1:Dj0L5fhJ9F82ZJyVOmBx6msDp/kfd1t9GRfny/mfJA0=
@@ -195,6 +199,8 @@ github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mdelapenya/tlscert v0.2.0 h1:7H81W6Z/4weDvZBNOfQte5GpIMo0lGYEeWbkGp5LJHI=
github.com/mdelapenya/tlscert v0.2.0/go.mod h1:O4njj3ELLnJjGdkN7M/vIVCpZ+Cf0L6muqOG4tLSl8o=
github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB5OBs92Dimizgw2cA=
github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s=
github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=

View File

@@ -103,3 +103,92 @@ func TestPostsAdminCreateSmoke(t *testing.T) {
}
})
}
// TestPostsAdminResaveKeepsHTML is the acceptance test of the todo
// journal-imported-html-admin-resave (D-03): an admin save of a post whose
// content is HTML, as imported posts are, keeps that HTML in content_html
// instead of goldmark's raw-HTML placeholder.
func TestPostsAdminResaveKeepsHTML(t *testing.T) {
env := newPostsEnv(t)
token := env.admin(t,
controllers.PermissionAccessPosts,
controllers.PermissionAccessPublish,
controllers.PermissionAccessOtherPosts,
)
const content = "<p>Grzyby rosną w lesie.</p>\n<p>Druga <b>część</b> z <a href=\"https://example.test/zrodlo\">linkiem</a>.</p>"
body := map[string]any{
"title": map[string]any{"en": "Imported", "pl": "Zaimportowany"},
"slug": map[string]any{"en": "imported-html", "pl": "zaimportowany-html"},
"content": map[string]any{"en": content, "pl": content},
"excerpt": map[string]any{"en": "Excerpt", "pl": "Wyjątek"},
}
rec := env.expect(t, http.StatusCreated, http.MethodPost, adminAPI(adminPostsPath), token, body)
var created struct {
ID uint `json:"id"`
}
if err := json.Unmarshal(adminDecode(t, rec).Data, &created); err != nil || created.ID == 0 {
t.Fatalf("create data %s: %v", rec.Body.String(), err)
}
assertHTML := func(stage string) {
t.Helper()
var stored models.Post
if err := env.gdb.First(&stored, created.ID).Error; err != nil {
t.Fatal(err)
}
if strings.Contains(stored.ContentHTML, "raw HTML omitted") {
t.Fatalf("%s: content_html holds the goldmark placeholder: %q", stage, stored.ContentHTML)
}
if strings.TrimSpace(stored.ContentHTML) != content {
t.Fatalf("%s: content_html = %q, want %q", stage, strings.TrimSpace(stored.ContentHTML), content)
}
if stored.Content != content {
t.Fatalf("%s: content = %q, want %q", stage, stored.Content, content)
}
}
assertHTML("create")
env.expect(t, http.StatusOK, http.MethodPut, adminAPI(fmt.Sprintf("%s/%d", adminPostsPath, created.ID)), token, body)
assertHTML("re-save")
}
// TestPostsAdminSaveSanitizesHTML pins CR-01 and WR-01 on the admin write
// path: unsafe markup in content is stripped from content_html, as Winter's
// Html::clean strips it, instead of refusing the save, and a "Data:" label is
// ordinary text. content itself is stored as submitted.
func TestPostsAdminSaveSanitizesHTML(t *testing.T) {
env := newPostsEnv(t)
token := env.admin(t,
controllers.PermissionAccessPosts,
controllers.PermissionAccessPublish,
controllers.PermissionAccessOtherPosts,
)
const content = "<p>Data: 12 maja 2024</p>\n" +
"<p><img src=\"https://cdn.example.test/a.jpg\" alt=\"a\"onerror=\"alert(1)\"></p>\n" +
"<p><a href=\"javascript&#58;alert(1)\">x</a></p>\n" +
"<base href=\"https://evil.test/\">"
rec := env.expect(t, http.StatusCreated, http.MethodPost, adminAPI(adminPostsPath), token, map[string]any{
"title": map[string]any{"en": "Sanitized", "pl": "Oczyszczony"},
"slug": map[string]any{"en": "sanitized-html", "pl": "oczyszczony-html"},
"content": map[string]any{"en": content, "pl": content},
})
var created struct {
ID uint `json:"id"`
}
if err := json.Unmarshal(adminDecode(t, rec).Data, &created); err != nil || created.ID == 0 {
t.Fatalf("create data %s: %v", rec.Body.String(), err)
}
var stored models.Post
if err := env.gdb.First(&stored, created.ID).Error; err != nil {
t.Fatal(err)
}
const want = "<p>Data: 12 maja 2024</p>\n" +
"<p><img src=\"https://cdn.example.test/a.jpg\" alt=\"a\"></p>\n" +
"<p>x</p>"
if got := strings.TrimSpace(stored.ContentHTML); got != want {
t.Fatalf("content_html = %q, want %q", got, want)
}
if stored.Content != content {
t.Fatalf("content = %q, want it stored as submitted", stored.Content)
}
}