feat(17-01): richeditor field type on the seaglass sanitizer
- seaglass (new module): bluemonday v1.0.27 policy ported from Journal without the Blog Editor rules, byte-stable token comparison, SanitizeReport, Sanitize, Check, Removals - cabana: type richeditor (common keys plus size); every create and update value sanitized before Fill, non-string values are a 422, removals become meta.notices; settings and relation forms refuse the type - backend lang richeditor.removed_* and markup_changed (en, pl plurals) - TestSeaglassSmoke, TestCompanionsRichEditorSmoke, README and docs
This commit is contained in:
65
modules/seaglass/README.md
Normal file
65
modules/seaglass/README.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# seaglass
|
||||
|
||||
HTML sanitizer for stored rich text: a bluemonday allowlist policy that returns clean HTML byte for byte and reports what it removed from the rest.
|
||||
|
||||
`import "git.golem15.com/golem15/summercms/modules/seaglass"`
|
||||
|
||||
**Name:** Glass the sea has worn smooth; rough HTML goes in and only the safe, smooth part comes out.
|
||||
|
||||
## Overview
|
||||
|
||||
`seaglass` is the framework's one HTML policy for rich text an administrator stores, such as cabana's `type: richeditor` field. It wraps [bluemonday](https://github.com/microcosm-cc/bluemonday) v1.0.27 with the policy the Journal plugin uses for imported WinterCMS content, without that plugin's editor-specific rules. A tokenizing sanitizer checks attributes and URLs the way a browser parses them, so it does not share the bypasses of regex gates (`<img src="x"onerror=…>`, `javascript:`, `<svg/onload=…>`).
|
||||
|
||||
The comparison is byte-stable: when the input and the policy's output tokenize to the same HTML tokens, the input itself is returned, so a value that only differs in escaping is never rewritten. Otherwise the policy's output is returned with a report of the removed elements, attributes and comments, which cabana shows to the administrator as save notices.
|
||||
|
||||
## Role in the framework
|
||||
|
||||
A leaf utility with no SummerCMS dependency. [cabana](../cabana/README.md) runs every richeditor value through it on create and update, top level and inside repeater rows; plugins that serve stored HTML in a public API can run it again on output, for content that was imported without passing the admin save.
|
||||
|
||||
Use cases:
|
||||
|
||||
- Sanitize an HTML field on save and tell the editor what was removed.
|
||||
- Check that stored HTML is already clean (`seaglass.Check`), for example in an import report.
|
||||
- Sanitize HTML on output in a public API.
|
||||
|
||||
## Features
|
||||
|
||||
- bluemonday's UGC policy with links kept as written (no `rel="nofollow"`), `tel:` links next to `http`, `https`, `mailto` and relative URLs, and the `class`, `dir`, microdata, footnote `role` and inline `color`, `font-size`, `font-weight` and `text-align` styles imported WinterCMS content uses.
|
||||
- Everything else is removed: `script`, `style`, `iframe`, `object`, `embed`, `form`, `svg`, `math`, comments, every `on*` attribute, `target`, `rel`, and URLs with any other scheme.
|
||||
- `seaglass.SanitizeReport` returns HTML the policy keeps whole byte for byte with a zero `seaglass.Removals`; a tag that repeats an attribute is always a change.
|
||||
- `seaglass.Removals` names the removed elements and attributes (at most ten distinct names each, in first-seen order) and counts every removal and every removed comment.
|
||||
|
||||
## Usage
|
||||
|
||||
```go
|
||||
clean, report := seaglass.SanitizeReport(`<p onclick="x()">Hi</p><script>alert(1)</script>`)
|
||||
// clean == "<p>Hi</p>"
|
||||
// report.Elements == []string{"script"}, report.Attributes == []string{"onclick"}
|
||||
|
||||
if err := seaglass.Check(stored); err != nil {
|
||||
// stored HTML would change under the policy
|
||||
}
|
||||
```
|
||||
|
||||
## API reference
|
||||
|
||||
| Identifier | Description |
|
||||
|------------|-------------|
|
||||
| `seaglass.Sanitize` | Returns the HTML filtered through the policy: the input itself when the policy keeps all of it. |
|
||||
| `seaglass.SanitizeReport` | `seaglass.Sanitize` with a `seaglass.Removals` report of what was removed; a zero report exactly when the input is returned. |
|
||||
| `seaglass.Check` | Returns an error when the policy would change the HTML, nil when it keeps it whole. |
|
||||
| `seaglass.Removals` | What was removed: `Elements` and `Attributes` (distinct names), `ElementCount` and `AttributeCount` (every removal), `Comments`, and `Changed` (the HTML differs from the input). |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: none.
|
||||
- Third-party: `github.com/microcosm-cc/bluemonday` v1.0.27 (the policy; decision note `.planning/notes/2026-10-11-framework-html-sanitizer-bluemonday.md`), `golang.org/x/net/html` (the token comparison and the removal report).
|
||||
- Standard library: `errors`, `fmt`, `io`, `regexp`, `slices`, `strings`.
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
go test ./modules/seaglass/...
|
||||
```
|
||||
|
||||
The tests run in memory and need no external services.
|
||||
398
modules/seaglass/seaglass.go
Normal file
398
modules/seaglass/seaglass.go
Normal file
@@ -0,0 +1,398 @@
|
||||
// Package seaglass is the framework's HTML sanitizer: a bluemonday allowlist
|
||||
// policy for stored rich text, with a byte-stable comparison so HTML the
|
||||
// policy keeps whole is returned unchanged, and a report of what it removed.
|
||||
package seaglass
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/microcosm-cc/bluemonday"
|
||||
xhtml "golang.org/x/net/html"
|
||||
)
|
||||
|
||||
// policy is the allowlist for stored rich text. It is safe for concurrent
|
||||
// use: bluemonday policies are read-only once built.
|
||||
var policy = newPolicy()
|
||||
|
||||
// newPolicy is bluemonday's UGC policy with the additions stored rich text
|
||||
// needs: links keep their form (no rel="nofollow" is added), tel: links are
|
||||
// allowed next to http, https, mailto and relative URLs, and the class, dir,
|
||||
// microdata, footnote role and inline style attributes imported WinterCMS
|
||||
// content uses. Scripts, styles, frames, forms, embedded objects, SVG and
|
||||
// MathML, comments, every on* attribute, target and rel, and every other URL
|
||||
// scheme are removed.
|
||||
func newPolicy() *bluemonday.Policy {
|
||||
p := bluemonday.UGCPolicy()
|
||||
p.RequireNoFollowOnLinks(false)
|
||||
p.AllowURLSchemes("tel")
|
||||
p.AllowElements("footer")
|
||||
p.AllowAttrs("class").Matching(bluemonday.SpaceSeparatedTokens).Globally()
|
||||
p.AllowAttrs("itemscope").Matching(regexp.MustCompile(`^(|itemscope)$`)).Globally()
|
||||
p.AllowAttrs("itemtype").Matching(regexp.MustCompile(`^https?://schema\.org/[A-Za-z]+$`)).Globally()
|
||||
p.AllowAttrs("itemprop").Matching(bluemonday.SpaceSeparatedTokens).Globally()
|
||||
p.AllowStyles("color", "font-size", "font-weight", "text-align").Globally()
|
||||
p.AllowAttrs("role").Matching(regexp.MustCompile(`^doc-(noteref|endnotes|backlink)$`)).OnElements("a", "div")
|
||||
p.AllowAttrs("dir").Matching(regexp.MustCompile(`(?i)^(ltr|rtl|auto)$`)).Globally()
|
||||
return p
|
||||
}
|
||||
|
||||
// maxRemovalNames is how many distinct names a Removals list keeps.
|
||||
const maxRemovalNames = 10
|
||||
|
||||
// Removals reports what SanitizeReport removed. Elements and Attributes are
|
||||
// the distinct removed element and attribute names in first-seen order, at
|
||||
// most ten each (a cut list ends with "…"); ElementCount and AttributeCount
|
||||
// count every removal, so they stay exact when a list is cut. An attribute
|
||||
// counts as removed when the policy dropped it, when it repeats a name the tag
|
||||
// already has, or when the policy changed its value (a style that lost a
|
||||
// property counts under "style"). Comments counts removed HTML comments.
|
||||
// Changed is true whenever the returned HTML differs from the input; it is the
|
||||
// only field set when the policy changed markup none of the others describe.
|
||||
type Removals struct {
|
||||
Elements []string
|
||||
ElementCount int
|
||||
Attributes []string
|
||||
AttributeCount int
|
||||
Comments int
|
||||
Changed bool
|
||||
}
|
||||
|
||||
func (r *Removals) addElement(name string) {
|
||||
r.ElementCount++
|
||||
r.Elements = addName(r.Elements, name)
|
||||
}
|
||||
|
||||
func (r *Removals) addAttribute(name string) {
|
||||
r.AttributeCount++
|
||||
r.Attributes = addName(r.Attributes, name)
|
||||
}
|
||||
|
||||
func addName(names []string, name string) []string {
|
||||
if slices.Contains(names, name) {
|
||||
return names
|
||||
}
|
||||
if len(names) == maxRemovalNames {
|
||||
return append(names, "…")
|
||||
}
|
||||
if len(names) > maxRemovalNames {
|
||||
return names
|
||||
}
|
||||
return append(names, name)
|
||||
}
|
||||
|
||||
// Sanitize returns html filtered through the policy: html itself, byte for
|
||||
// byte, when the policy keeps all of it, otherwise the policy's output.
|
||||
func Sanitize(html string) string {
|
||||
out, _ := SanitizeReport(html)
|
||||
return out
|
||||
}
|
||||
|
||||
// SanitizeReport is Sanitize with a report of what the policy removed. When
|
||||
// html and the policy's output tokenize to the same HTML tokens (same type,
|
||||
// tag name, decoded text and set of decoded attributes), html is returned
|
||||
// unchanged with a zero Removals, so re-escaping alone is never a change.
|
||||
// Otherwise the policy's output is returned with Changed set and the removed
|
||||
// elements, attributes and comments, found by aligning the start tags of html
|
||||
// with those of the output in order (the policy only removes, it never adds
|
||||
// or reorders).
|
||||
func SanitizeReport(html string) (string, Removals) {
|
||||
clean, changed := sanitize(html)
|
||||
if !changed || clean == html {
|
||||
return html, Removals{}
|
||||
}
|
||||
report := removalReport(html, clean)
|
||||
report.Changed = true
|
||||
return clean, report
|
||||
}
|
||||
|
||||
// Check returns an error when the policy would change any element, attribute
|
||||
// or text of html, and nil when it keeps html whole.
|
||||
func Check(html string) error {
|
||||
if _, changed := sanitize(html); changed {
|
||||
return errors.New("seaglass: HTML contains elements, attributes or URLs that are not allowed")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sanitize returns doc unchanged when the policy keeps all of it, and the
|
||||
// policy's output with changed set otherwise. A tag that repeats an attribute
|
||||
// is always a change: the tokenizer keeps only the first of them, so the
|
||||
// comparison would not see the others in the stored bytes.
|
||||
func sanitize(doc string) (string, bool) {
|
||||
clean := policy.Sanitize(doc)
|
||||
if sameTokens(doc, clean) {
|
||||
return doc, false
|
||||
}
|
||||
return clean, true
|
||||
}
|
||||
|
||||
// sameTokens reports whether a and b tokenize to the same sequence of HTML
|
||||
// tokens. It is false when either repeats an attribute in a tag.
|
||||
func sameTokens(a, b string) bool {
|
||||
ta, oka := htmlTokens(a)
|
||||
if !oka {
|
||||
return false
|
||||
}
|
||||
tb, okb := htmlTokens(b)
|
||||
return okb && slices.Equal(ta, tb)
|
||||
}
|
||||
|
||||
// htmlTokens tokenizes doc. It is not ok when the tokenizer stops on an error
|
||||
// other than EOF, or when a start tag's raw attributes differ from the
|
||||
// attributes the tokenizer kept, which happens when a name repeats.
|
||||
func htmlTokens(doc string) ([]string, bool) {
|
||||
var out []string
|
||||
z := xhtml.NewTokenizer(strings.NewReader(doc))
|
||||
for {
|
||||
tt := z.Next()
|
||||
if tt == xhtml.ErrorToken {
|
||||
return out, errors.Is(z.Err(), io.EOF)
|
||||
}
|
||||
var raw []string
|
||||
if tt == xhtml.StartTagToken || tt == xhtml.SelfClosingTagToken {
|
||||
raw = rawAttrNames(z.Raw())
|
||||
}
|
||||
t := z.Token()
|
||||
if tt == xhtml.StartTagToken || tt == xhtml.SelfClosingTagToken {
|
||||
if len(raw) != len(t.Attr) {
|
||||
return out, false
|
||||
}
|
||||
for i, a := range t.Attr {
|
||||
if raw[i] != a.Key {
|
||||
return out, false
|
||||
}
|
||||
}
|
||||
}
|
||||
attrs := make([]string, 0, len(t.Attr))
|
||||
for _, a := range t.Attr {
|
||||
attrs = append(attrs, a.Namespace+":"+a.Key+"="+a.Val)
|
||||
}
|
||||
slices.Sort(attrs)
|
||||
out = append(out, fmt.Sprintf("%d|%s|%q", tt, t.Data, attrs))
|
||||
}
|
||||
}
|
||||
|
||||
// reportTag is one start tag of a document as removalReport sees it.
|
||||
type reportTag struct {
|
||||
name string
|
||||
raw []string // attribute names in source order, repeats included
|
||||
attrs map[string]string // decoded values, the first of a repeated name
|
||||
}
|
||||
|
||||
// reportTokens returns doc's start tags and its number of comments.
|
||||
func reportTokens(doc string) ([]reportTag, int) {
|
||||
var (
|
||||
tags []reportTag
|
||||
comments int
|
||||
)
|
||||
z := xhtml.NewTokenizer(strings.NewReader(doc))
|
||||
for {
|
||||
tt := z.Next()
|
||||
switch tt {
|
||||
case xhtml.ErrorToken:
|
||||
return tags, comments
|
||||
case xhtml.CommentToken:
|
||||
comments++
|
||||
case xhtml.StartTagToken, xhtml.SelfClosingTagToken:
|
||||
raw := rawAttrNames(z.Raw())
|
||||
t := z.Token()
|
||||
attrs := make(map[string]string, len(t.Attr))
|
||||
for _, a := range t.Attr {
|
||||
key := a.Key
|
||||
if a.Namespace != "" {
|
||||
key = a.Namespace + ":" + a.Key
|
||||
}
|
||||
if _, seen := attrs[key]; !seen {
|
||||
attrs[key] = a.Val
|
||||
}
|
||||
}
|
||||
tags = append(tags, reportTag{name: t.Data, raw: raw, attrs: attrs})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// keeps reports whether out can be the policy's output of in: the same
|
||||
// element, and every output attribute present in the input with the same
|
||||
// value, except style, whose properties the policy filters one by one.
|
||||
func (in reportTag) keeps(out reportTag) bool {
|
||||
if in.name != out.name {
|
||||
return false
|
||||
}
|
||||
for key, val := range out.attrs {
|
||||
got, ok := in.attrs[key]
|
||||
if !ok || (got != val && key != "style") {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// removalReport compares doc with the policy's output clean. It aligns the
|
||||
// start tags greedily, preferring an output tag whose attributes the input
|
||||
// tag can have produced; when that leaves output tags unmatched, it aligns by
|
||||
// element name only.
|
||||
func removalReport(doc, clean string) Removals {
|
||||
in, inComments := reportTokens(doc)
|
||||
out, outComments := reportTokens(clean)
|
||||
match := alignTags(in, out, reportTag.keeps)
|
||||
if match == nil {
|
||||
match = alignTags(in, out, func(a, b reportTag) bool { return a.name == b.name })
|
||||
}
|
||||
var r Removals
|
||||
for i, tag := range in {
|
||||
j := -1
|
||||
if match != nil {
|
||||
j = match[i]
|
||||
}
|
||||
if j < 0 {
|
||||
r.addElement(tag.name)
|
||||
continue
|
||||
}
|
||||
kept := out[j]
|
||||
seen := make(map[string]bool, len(tag.raw))
|
||||
for _, name := range tag.raw {
|
||||
if seen[name] {
|
||||
r.addAttribute(name) // a repeat the tokenizer dropped
|
||||
continue
|
||||
}
|
||||
seen[name] = true
|
||||
val, ok := kept.attrs[name]
|
||||
if !ok || val != tag.attrs[name] {
|
||||
r.addAttribute(name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if inComments > outComments {
|
||||
r.Comments = inComments - outComments
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// alignTags matches every tag of out to a tag of in, in order, with the first
|
||||
// input tag for which same holds. The result maps each input index to its
|
||||
// output index or -1. It is nil when an output tag stays unmatched.
|
||||
func alignTags(in, out []reportTag, same func(in, out reportTag) bool) []int {
|
||||
match := make([]int, len(in))
|
||||
j := 0
|
||||
for i := range in {
|
||||
match[i] = -1
|
||||
if j < len(out) && same(in[i], out[j]) {
|
||||
match[i] = j
|
||||
j++
|
||||
}
|
||||
}
|
||||
if j < len(out) {
|
||||
return nil
|
||||
}
|
||||
return match
|
||||
}
|
||||
|
||||
// rawAttrNames returns the ASCII-lowercased name of every attribute in the
|
||||
// raw start tag, repeats included, in source order. It scans the tag as
|
||||
// golang.org/x/net/html's tokenizer does (readTagName, readTagAttrKey and
|
||||
// readTagAttrVal), which drops every repeat of a name it has seen. A scan that
|
||||
// disagrees with the tokenizer only makes htmlTokens report a change.
|
||||
func rawAttrNames(raw []byte) []string {
|
||||
i, n := 2, len(raw) // past '<' and the first byte of the tag name
|
||||
skipSpace := func() {
|
||||
for i < n && isHTMLSpace(raw[i]) {
|
||||
i++
|
||||
}
|
||||
}
|
||||
for i < n {
|
||||
c := raw[i]
|
||||
i++
|
||||
if isHTMLSpace(c) {
|
||||
break
|
||||
}
|
||||
if c == '/' || c == '>' {
|
||||
i--
|
||||
break
|
||||
}
|
||||
}
|
||||
var names []string
|
||||
skipSpace()
|
||||
for i < n {
|
||||
if raw[i] == '>' {
|
||||
break
|
||||
}
|
||||
// Attribute name: an '=' that starts it is part of the name.
|
||||
start, end := i, n
|
||||
for i < n {
|
||||
c := raw[i]
|
||||
i++
|
||||
if c == '=' && i == start+1 {
|
||||
continue
|
||||
}
|
||||
if c == '=' || c == '/' || c == '>' || isHTMLSpace(c) {
|
||||
i--
|
||||
end = i
|
||||
break
|
||||
}
|
||||
}
|
||||
// Attribute value.
|
||||
skipSpace()
|
||||
if i < n {
|
||||
c := raw[i]
|
||||
i++
|
||||
switch {
|
||||
case c == '/':
|
||||
case c != '=':
|
||||
i--
|
||||
default:
|
||||
skipSpace()
|
||||
if i < n {
|
||||
q := raw[i]
|
||||
i++
|
||||
switch q {
|
||||
case '>':
|
||||
i--
|
||||
case '"', '\'':
|
||||
for i < n {
|
||||
i++
|
||||
if raw[i-1] == q {
|
||||
break
|
||||
}
|
||||
}
|
||||
default:
|
||||
for i < n {
|
||||
c := raw[i]
|
||||
i++
|
||||
if isHTMLSpace(c) {
|
||||
break
|
||||
}
|
||||
if c == '>' {
|
||||
i--
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if end > start {
|
||||
names = append(names, asciiLower(raw[start:end]))
|
||||
}
|
||||
skipSpace()
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
func isHTMLSpace(c byte) bool {
|
||||
return c == ' ' || c == '\n' || c == '\r' || c == '\t' || c == '\f'
|
||||
}
|
||||
|
||||
func asciiLower(b []byte) string {
|
||||
out := []byte(string(b))
|
||||
for i, c := range out {
|
||||
if 'A' <= c && c <= 'Z' {
|
||||
out[i] = c + 'a' - 'A'
|
||||
}
|
||||
}
|
||||
return string(out)
|
||||
}
|
||||
53
modules/seaglass/seaglass_smoke_test.go
Normal file
53
modules/seaglass/seaglass_smoke_test.go
Normal file
@@ -0,0 +1,53 @@
|
||||
package seaglass_test
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/seaglass"
|
||||
)
|
||||
|
||||
// TestSeaglassSmoke checks the two promises of the sanitizer: clean HTML
|
||||
// comes back byte for byte with an empty report, and unsafe constructs are
|
||||
// removed and named.
|
||||
func TestSeaglassSmoke(t *testing.T) {
|
||||
clean := "<p>Mushrooms <strong>can</strong> help:</p>\n<ul><li>focus</li><li><a href=\"https://example.com/a?b=1&c=2\">sleep</a></li></ul>\n<p>Call <a href=\"tel:+48123\">us</a> or <a href=\"/kontakt\">write</a>.</p>"
|
||||
got, report := seaglass.SanitizeReport(clean)
|
||||
if got != clean || report.Changed || report.ElementCount+report.AttributeCount+report.Comments != 0 {
|
||||
t.Fatalf("clean HTML changed: %q %+v", got, report)
|
||||
}
|
||||
if err := seaglass.Check(clean); err != nil {
|
||||
t.Fatalf("Check(clean) = %v", err)
|
||||
}
|
||||
// Re-escaping alone is not a change: bluemonday writes ' here.
|
||||
quoted := `<p>It's "fine"</p>`
|
||||
if got := seaglass.Sanitize(quoted); got != quoted {
|
||||
t.Fatalf("re-escaped text changed: %q", got)
|
||||
}
|
||||
|
||||
dirty := `<p onclick="steal()">Hi <a href="javascript:alert(1)">there</a></p><script>alert(1)</script><!-- note -->`
|
||||
got, report = seaglass.SanitizeReport(dirty)
|
||||
for _, bad := range []string{"onclick", "javascript", "<script", "alert", "<!--"} {
|
||||
if strings.Contains(got, bad) {
|
||||
t.Fatalf("%q survived: %q", bad, got)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(got, "Hi ") || !report.Changed {
|
||||
t.Fatalf("sanitized = %q %+v", got, report)
|
||||
}
|
||||
// A link whose only href is refused loses its tag (its text stays), so
|
||||
// it is reported as a removed a element.
|
||||
if !slices.Contains(report.Elements, "script") || !slices.Contains(report.Elements, "a") || !slices.Contains(report.Attributes, "onclick") || report.Comments != 1 {
|
||||
t.Fatalf("report = %+v", report)
|
||||
}
|
||||
if err := seaglass.Check(dirty); err == nil {
|
||||
t.Fatal("Check(dirty) = nil")
|
||||
}
|
||||
// Bypasses of regex gates are removed too.
|
||||
for _, payload := range []string{`<img src="x"onerror="alert(1)">`, `<a href="javascript:alert(1)">x</a>`, `<svg/onload=alert(1)>`, `<a href="https://a" href="javascript:alert(1)">x</a>`} {
|
||||
if err := seaglass.Check(payload); err == nil {
|
||||
t.Fatalf("payload passed: %s", payload)
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user