feat(breakwater): Markdown renderer and HTML sanitizer with the forum policy (D-17)
- modules/breakwater exports Render(markdown) and Purify(html): goldmark GFM in its default safe mode (raw HTML stripped, unsafe links dropped), then a named bluemonday policy built from NewPolicy, not the UGC preset - the policy is the PHP golem15/forum HTMLPurifier config 1:1: the fixed element list, a[href|rel|target], http/https/mailto only, rel=nofollow on every link, target=_blank plus noopener on fully-qualified links - tests cover the PHP ForumMarkdownRenderTest corpus, links per scheme, blank input, the Purify allow-list and concurrent use; testable examples feed the docs page snippets - module README, root module table row, introduction.md Utilities row and docs/services/markdown.md (which gate serves user content vs admin fields) - go.mod and go.sum unchanged: bluemonday v1.0.27 and goldmark v1.8.6 were already pinned
This commit is contained in:
Jakub Zych
57
modules/breakwater/README.md
Normal file
57
modules/breakwater/README.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# breakwater
|
||||
|
||||
Markdown renderer and HTML sanitizer for user-submitted content: GitHub Flavored Markdown through goldmark, then a fixed bluemonday allow-list that adds nofollow and target=_blank to links.
|
||||
|
||||
`import "git.golem15.com/golem15/summercms/modules/breakwater"`
|
||||
|
||||
**Name:** A breakwater takes the force of the open sea; user Markdown and HTML break on the policy and only calm, allow-listed HTML reaches the shore.
|
||||
|
||||
## Overview
|
||||
|
||||
`breakwater` is the gate for content a site user submits and a plugin stores and serves: forum posts, comments, and any other Markdown or HTML that does not come from an administrator. `breakwater.Render` converts Markdown to HTML and `breakwater.Purify` sanitizes HTML that is already HTML; both return the policy's output, which the plugin stores on write and serves as is.
|
||||
|
||||
The policy is the PHP `golem15/forum` HTMLPurifier config 1:1 (decision note `.planning/notes/2026-10-18-bluemonday-forum-policy.md`): a short element list, links with `http`, `https` or `mailto` only, `rel="nofollow"` on every link and `target="_blank"` on fully-qualified ones. The Markdown pass is [goldmark](https://github.com/yuin/goldmark) with GFM, in its default safe mode: raw HTML in the Markdown is stripped and a link with an unsafe destination is dropped before the policy runs, the analog of the PHP `html_input=strip` and `allow_unsafe_links=false`.
|
||||
|
||||
**Not this module.** [seaglass](../seaglass/README.md) is the policy for rich text an administrator stores through cabana's `richeditor` field: bluemonday's UGC policy, which keeps images, relative and `tel:` links, and links as written without `nofollow`. `cabana.RenderMarkdown` is the admin Markdown field preview: CommonMark without GFM behind a regex gate that refuses unsafe output instead of removing it. Plugins must not use either for user content.
|
||||
|
||||
## Features
|
||||
|
||||
- Allowed elements: `p`, `br`, `strong`, `em`, `b`, `i`, `h1`–`h6`, `ul`, `ol`, `li`, `blockquote`, `pre`, `code`, `a` with `href`, `rel` and `target`, `table`, `thead`, `tbody`, `tr`, `th`, `td`, `hr`, `del`, `ins`, `sub`, `sup`. Everything else, including `img`, `figure`, `script`, `style`, `iframe`, `form`, `svg`, every `on*` attribute, `class` and `id`, is removed.
|
||||
- URL schemes: `http`, `https` and `mailto`. No `tel:` and no relative URLs; a link whose `href` is refused loses its tag and keeps its text.
|
||||
- Every link gets `rel="nofollow"`; every fully-qualified `http` or `https` link also gets `target="_blank"` and `rel="noopener"`. A caller-supplied `rel` or `target` with any other value is dropped and rewritten.
|
||||
- Raw HTML in Markdown is stripped, not escaped into visible text, and HTML comments are removed.
|
||||
- Blank or whitespace-only input renders and purifies to `""`.
|
||||
- Safe for concurrent use: the Markdown engine and the policy are built once and shared.
|
||||
|
||||
## Usage
|
||||
|
||||
```go
|
||||
// A forum post: store the rendered HTML on write and serve it as is.
|
||||
html := breakwater.Render(post.Content)
|
||||
|
||||
// Content stored as HTML: the same allow-list, no Markdown pass.
|
||||
clean := breakwater.Purify(training.ContentHTML)
|
||||
```
|
||||
|
||||
`breakwater.Render("See [the docs](https://example.com/docs) and <b onclick=\"x()\">this</b>.")` gives `<p>See <a href="https://example.com/docs" rel="nofollow noopener" target="_blank">the docs</a> and this.</p>`.
|
||||
|
||||
## API reference
|
||||
|
||||
| Identifier | Description |
|
||||
|------------|-------------|
|
||||
| `breakwater.Render` | Converts user Markdown (GFM) to HTML with raw HTML stripped and unsafe links dropped, then returns it filtered through the policy; `""` for blank input. |
|
||||
| `breakwater.Purify` | Returns HTML filtered through the same policy with no Markdown pass, for content stored as HTML; `""` for blank input. |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: none.
|
||||
- Third-party: `github.com/microcosm-cc/bluemonday` v1.0.27 (the policy; decision note `.planning/notes/2026-10-18-bluemonday-forum-policy.md`), `github.com/yuin/goldmark` v1.8.6 with `extension.GFM` (the Markdown pass).
|
||||
- Standard library: `bytes`, `regexp`, `strings`.
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
go test ./modules/breakwater/...
|
||||
```
|
||||
|
||||
The tests run in memory and need no external services. They cover the PHP `ForumMarkdownRenderTest` corpus (XSS cases, GFM table and fenced code), links and autolinks per scheme, blank input, the `Purify` allow-list and concurrent use.
|
||||
110
modules/breakwater/breakwater.go
Normal file
110
modules/breakwater/breakwater.go
Normal file
@@ -0,0 +1,110 @@
|
||||
// Package breakwater is the framework's gate for user-submitted content: a
|
||||
// Markdown renderer and an HTML sanitizer with one fixed allow-list, for
|
||||
// forum posts, comments and any other Markdown or HTML a site user submits
|
||||
// that a plugin stores and serves.
|
||||
//
|
||||
// Render turns GitHub Flavored Markdown into HTML with raw HTML stripped and
|
||||
// unsafe links dropped, then runs the policy; Purify runs the same policy on
|
||||
// content that is already HTML. The policy is the forum policy of the
|
||||
// 2026-10-18 decision note, 1:1 with the PHP golem15/forum HTMLPurifier
|
||||
// config: a short element list, links with http, https or mailto only,
|
||||
// rel="nofollow" on every link and target="_blank" on fully-qualified ones.
|
||||
//
|
||||
// It is not seaglass, the policy for rich text an administrator stores
|
||||
// (bluemonday's UGC policy: images, relative and tel: links, links kept as
|
||||
// written without nofollow), and it is not cabana.RenderMarkdown, the admin
|
||||
// Markdown field preview (CommonMark without GFM behind a regex gate).
|
||||
// Neither is a gate for user content; breakwater is.
|
||||
package breakwater
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/microcosm-cc/bluemonday"
|
||||
"github.com/yuin/goldmark"
|
||||
"github.com/yuin/goldmark/extension"
|
||||
)
|
||||
|
||||
// engine is the Markdown converter: CommonMark plus GFM (tables,
|
||||
// strikethrough, autolinks, task lists). The HTML renderer keeps goldmark's
|
||||
// default safe mode, so raw HTML in the input is written as an omitted-HTML
|
||||
// comment and a link or autolink with a dangerous destination is written
|
||||
// with an empty href (the PHP html_input=strip and allow_unsafe_links=false).
|
||||
// A goldmark.Markdown is shared across goroutines, as cabana's markdownEngine
|
||||
// is: parse state is per call.
|
||||
var engine = goldmark.New(goldmark.WithExtensions(extension.GFM))
|
||||
|
||||
// policy is the forum policy. It is safe for concurrent use: bluemonday
|
||||
// policies are read-only once built. It stays unexported because a
|
||||
// *bluemonday.Policy is mutable.
|
||||
var policy = newPolicy()
|
||||
|
||||
var (
|
||||
// relTokens accepts one or more space-separated tokens the policy itself
|
||||
// writes. Any other caller-supplied rel is dropped and rewritten.
|
||||
relTokens = regexp.MustCompile(`^(nofollow|noopener|noreferrer)( (nofollow|noopener|noreferrer))*$`)
|
||||
// targetBlank accepts the one target the policy writes.
|
||||
targetBlank = regexp.MustCompile(`^_blank$`)
|
||||
)
|
||||
|
||||
// newPolicy builds the forum policy from an empty bluemonday policy, not
|
||||
// from the UGC preset, so the allow-list is exactly the PHP one (per D-16):
|
||||
//
|
||||
// - HTML.Allowed: p, br, strong, em, b, i, h1-h6, ul, ol, li, blockquote,
|
||||
// pre, code, a[href|rel|target], table, thead, tbody, tr, th, td, hr,
|
||||
// del, ins, sub, sup. No img, no figure, nothing else.
|
||||
// - URI.AllowedSchemes: http, https, mailto. No tel: and no relative URLs;
|
||||
// a link whose href is refused loses its tag and keeps its text.
|
||||
// - HTML.Nofollow: rel="nofollow" on every link.
|
||||
// - HTML.TargetBlank: target="_blank" on every fully-qualified link, with
|
||||
// the rel="noopener" bluemonday adds next to it.
|
||||
//
|
||||
// Comments stay disallowed (the bluemonday default), which is what removes
|
||||
// goldmark's omitted-HTML comment from Render output.
|
||||
func newPolicy() *bluemonday.Policy {
|
||||
p := bluemonday.NewPolicy()
|
||||
p.AllowElements(
|
||||
"p", "br", "strong", "em", "b", "i",
|
||||
"h1", "h2", "h3", "h4", "h5", "h6",
|
||||
"ul", "ol", "li", "blockquote",
|
||||
"pre", "code", "a",
|
||||
"table", "thead", "tbody", "tr", "th", "td",
|
||||
"hr", "del", "ins", "sub", "sup",
|
||||
)
|
||||
p.AllowAttrs("href").OnElements("a")
|
||||
p.AllowAttrs("rel").Matching(relTokens).OnElements("a")
|
||||
p.AllowAttrs("target").Matching(targetBlank).OnElements("a")
|
||||
p.AllowURLSchemes("http", "https", "mailto")
|
||||
p.RequireNoFollowOnLinks(true)
|
||||
p.AddTargetBlankToFullyQualifiedLinks(true)
|
||||
return p
|
||||
}
|
||||
|
||||
// Render converts user Markdown to HTML and returns it filtered through the
|
||||
// policy. The Markdown is GitHub Flavored (tables, strikethrough, autolinks);
|
||||
// raw HTML in it is stripped, not escaped into visible text, and a link with
|
||||
// a javascript:, data: or other unsafe destination is dropped with its text
|
||||
// kept. Blank or whitespace-only markdown renders to "".
|
||||
func Render(markdown string) string {
|
||||
if strings.TrimSpace(markdown) == "" {
|
||||
return ""
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
// Convert only reports its writer's error, and bytes.Buffer.Write never
|
||||
// fails, so the error is unreachable; the buffer always goes through
|
||||
// Purify, so no unsanitized output can be returned.
|
||||
_ = engine.Convert([]byte(markdown), &buf)
|
||||
return Purify(buf.String())
|
||||
}
|
||||
|
||||
// Purify returns html filtered through the policy, with no Markdown pass:
|
||||
// the same allow-list for content a plugin stores as HTML (the PHP purify()
|
||||
// for trainings). Blank or whitespace-only html purifies to "".
|
||||
func Purify(html string) string {
|
||||
if strings.TrimSpace(html) == "" {
|
||||
return ""
|
||||
}
|
||||
return policy.Sanitize(html)
|
||||
}
|
||||
186
modules/breakwater/breakwater_test.go
Normal file
186
modules/breakwater/breakwater_test.go
Normal file
@@ -0,0 +1,186 @@
|
||||
package breakwater_test
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/breakwater"
|
||||
)
|
||||
|
||||
// The corpora of the PHP golem15/forum ForumMarkdownRenderTest, verbatim.
|
||||
const (
|
||||
xssCorpus = "<script>alert(1)</script>\n\n[x](javascript:alert(1))\n\n<iframe src=x></iframe>\n\n<img src=x onerror=alert(1)>"
|
||||
gfmCorpus = "| A | B |\n|---|---|\n| 1 | 2 |\n\n```php\necho 'wave';\n```"
|
||||
)
|
||||
|
||||
func wantContains(t *testing.T, got string, subs ...string) {
|
||||
t.Helper()
|
||||
for _, s := range subs {
|
||||
if !strings.Contains(got, s) {
|
||||
t.Errorf("output lacks %q:\n%s", s, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func wantNotContains(t *testing.T, got string, subs ...string) {
|
||||
t.Helper()
|
||||
for _, s := range subs {
|
||||
if strings.Contains(got, s) {
|
||||
t.Errorf("output contains %q:\n%s", s, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderXSS(t *testing.T) {
|
||||
got := breakwater.Render(xssCorpus)
|
||||
lower := strings.ToLower(got)
|
||||
wantNotContains(t, lower, "<script", "javascript:", "<iframe", "onerror", "<img", "<script", "<!--")
|
||||
wantContains(t, got, "x")
|
||||
}
|
||||
|
||||
func TestRenderStripsRawHTML(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
in string
|
||||
keeps []string
|
||||
}{
|
||||
{"script", "<script>alert(1)</script>", nil},
|
||||
{"inline", "text <b onclick=\"x()\">bold</b> more", []string{"text", "more"}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := breakwater.Render(tc.in)
|
||||
wantNotContains(t, got, "alert(1)", "onclick", "<!--", "<")
|
||||
wantContains(t, got, tc.keeps...)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderGFM(t *testing.T) {
|
||||
got := breakwater.Render(gfmCorpus)
|
||||
wantContains(t, got, "<table>", "<th>A</th>", "<td>1</td>", "<pre>", "<code")
|
||||
wantNotContains(t, got, "class=")
|
||||
|
||||
got = breakwater.Render("~~gone~~ **kept**")
|
||||
wantContains(t, got, "<del>gone</del>", "<strong>kept</strong>")
|
||||
|
||||
got = breakwater.Render("# Title\n\n> quote\n\n- a\n- b\n\n---")
|
||||
wantContains(t, got, "<h1>", "<blockquote>", "<ul>", "<li>", "<hr")
|
||||
}
|
||||
|
||||
func TestRenderLinks(t *testing.T) {
|
||||
t.Run("absolute", func(t *testing.T) {
|
||||
got := breakwater.Render("[site](https://example.com/a?b=1&c=2)")
|
||||
wantContains(t, got, `href="https://example.com/a?b=1&c=2"`, `target="_blank"`)
|
||||
rel := relValue(t, got)
|
||||
wantContains(t, rel, "nofollow", "noopener")
|
||||
})
|
||||
t.Run("autolink", func(t *testing.T) {
|
||||
got := breakwater.Render("visit https://example.com/x now")
|
||||
wantContains(t, got, `<a href="https://example.com/x"`, "nofollow", `target="_blank"`)
|
||||
})
|
||||
t.Run("mailto", func(t *testing.T) {
|
||||
got := breakwater.Render("[mail](mailto:a@example.com)")
|
||||
wantContains(t, got, `href="mailto:a@example.com"`, "nofollow")
|
||||
})
|
||||
dropped := []struct {
|
||||
name, in, scheme, text string
|
||||
}{
|
||||
{"data", "[d](data:text/html;base64,AAAA)", "data:", "d"},
|
||||
{"vbscript", "[v](vbscript:x)", "vbscript:", "v"},
|
||||
{"tel", "[t](tel:+48123456)", "tel:", "t"},
|
||||
}
|
||||
for _, tc := range dropped {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := breakwater.Render(tc.in)
|
||||
wantNotContains(t, got, "data:", "vbscript:", "tel:", "<a")
|
||||
wantContains(t, got, tc.text)
|
||||
})
|
||||
}
|
||||
t.Run("relative", func(t *testing.T) {
|
||||
got := breakwater.Render("[rel](/threads/1)")
|
||||
wantNotContains(t, got, "href")
|
||||
wantContains(t, got, "rel")
|
||||
})
|
||||
}
|
||||
|
||||
// relValue returns the value of the first rel attribute in html.
|
||||
func relValue(t *testing.T, html string) string {
|
||||
t.Helper()
|
||||
_, after, ok := strings.Cut(html, `rel="`)
|
||||
if !ok {
|
||||
t.Fatalf("no rel attribute in:\n%s", html)
|
||||
}
|
||||
val, _, ok := strings.Cut(after, `"`)
|
||||
if !ok {
|
||||
t.Fatalf("unterminated rel attribute in:\n%s", html)
|
||||
}
|
||||
return val
|
||||
}
|
||||
|
||||
func TestBlankInput(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
got string
|
||||
}{
|
||||
{"render empty", breakwater.Render("")},
|
||||
{"render whitespace", breakwater.Render(" \n\t ")},
|
||||
{"purify empty", breakwater.Purify("")},
|
||||
{"purify whitespace", breakwater.Purify(" \n ")},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if tc.got != "" {
|
||||
t.Errorf("%s: got %q, want \"\"", tc.name, tc.got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPurifyAllowList(t *testing.T) {
|
||||
in := `<p onclick="x()">Hi <b>b</b> <i>i</i> <img src=x onerror=alert(1)><script>alert(1)</script></p>` +
|
||||
`<figure><img src="a.jpg"></figure><h2 class="t" id="h">H</h2>` +
|
||||
`<a href="https://e.com/p" target="_top" rel="opener">l</a><a href="tel:+48">t</a>` +
|
||||
`<sub>s</sub><sup>p</sup><ins>i</ins><del>d</del><hr>` +
|
||||
`<table><thead><tr><th>h</th></tr></thead><tbody><tr><td>c</td></tr></tbody></table>` +
|
||||
`<blockquote><pre><code>c < d</code></pre></blockquote>` +
|
||||
`<ul><li>u</li></ul><ol><li>o</li></ol><h6>6</h6><br>`
|
||||
got := breakwater.Purify(in)
|
||||
wantContains(t, got,
|
||||
"<p>", "<b>b</b>", "<i>i</i>", "<h2>H</h2>", "<sub>s</sub>", "<sup>p</sup>", "<ins>i</ins>", "<del>d</del>",
|
||||
"<hr", "<table>", "<thead>", "<th>h</th>", "<tbody>", "<td>c</td>", "<blockquote>", "<pre>",
|
||||
"<code>c < d</code>", "<ul>", "<ol>", "<li>", "<h6>6</h6>", "<br",
|
||||
`href="https://e.com/p"`, `target="_blank"`,
|
||||
)
|
||||
rel := relValue(t, got)
|
||||
wantContains(t, rel, "nofollow", "noopener")
|
||||
lower := strings.ToLower(got)
|
||||
wantNotContains(t, lower, "onclick", "onerror", "<img", "<script", "<figure", "class=", "id=",
|
||||
`target="_top"`, `rel="opener"`, "tel:", "alert(")
|
||||
// The text of the dropped tel link survives.
|
||||
wantContains(t, got, "</a>t<sub>")
|
||||
|
||||
got = breakwater.Purify("<p>plain</p><!-- c --><style>p{}</style><form><input></form><svg onload=x></svg><iframe src=x></iframe>")
|
||||
wantContains(t, got, "<p>plain</p>")
|
||||
wantNotContains(t, got, "<!--", "<style", "<form", "<input", "<svg", "onload", "<iframe")
|
||||
}
|
||||
|
||||
func TestConcurrentUse(t *testing.T) {
|
||||
wantRender := breakwater.Render(gfmCorpus)
|
||||
wantPurify := breakwater.Purify(wantRender)
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 8; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for j := 0; j < 20; j++ {
|
||||
if got := breakwater.Render(gfmCorpus); got != wantRender {
|
||||
t.Errorf("concurrent Render differs:\n%s\nwant:\n%s", got, wantRender)
|
||||
}
|
||||
if got := breakwater.Purify(wantRender); got != wantPurify {
|
||||
t.Errorf("concurrent Purify differs:\n%s\nwant:\n%s", got, wantPurify)
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
}
|
||||
25
modules/breakwater/example_test.go
Normal file
25
modules/breakwater/example_test.go
Normal file
@@ -0,0 +1,25 @@
|
||||
package breakwater_test
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/breakwater"
|
||||
)
|
||||
|
||||
func ExampleRender() {
|
||||
// A forum post: store the rendered HTML on write and serve it as is.
|
||||
post := "See [the docs](https://example.com/docs) and <b onclick=\"x()\">this</b>.\n\n<script>alert(1)</script>"
|
||||
html := breakwater.Render(post)
|
||||
fmt.Println(html)
|
||||
// Output:
|
||||
// <p>See <a href="https://example.com/docs" rel="nofollow noopener" target="_blank">the docs</a> and this.</p>
|
||||
}
|
||||
|
||||
func ExamplePurify() {
|
||||
// Content stored as HTML: the same allow-list, no Markdown pass.
|
||||
stored := `<p onclick="x()">Hi <b>there</b><img src="a.jpg"></p><a href="tel:+48">call</a>`
|
||||
clean := breakwater.Purify(stored)
|
||||
fmt.Println(clean)
|
||||
// Output:
|
||||
// <p>Hi <b>there</b></p>call
|
||||
}
|
||||
Reference in New Issue
Block a user