feat(17-01): richeditor field type on the seaglass sanitizer
- seaglass (new module): bluemonday v1.0.27 policy ported from Journal without the Blog Editor rules, byte-stable token comparison, SanitizeReport, Sanitize, Check, Removals - cabana: type richeditor (common keys plus size); every create and update value sanitized before Fill, non-string values are a 422, removals become meta.notices; settings and relation forms refuse the type - backend lang richeditor.removed_* and markup_changed (en, pl plurals) - TestSeaglassSmoke, TestCompanionsRichEditorSmoke, README and docs
This commit is contained in:
@@ -108,6 +108,7 @@ The `migrate`, `migrate:status`, `migrate:rollback`, `serve` and admin commands
|
||||
| [party](modules/party/README.md) | Compiled plugin registry that orders plugins by their dependencies and runs their Register and Boot lifecycle. | The guest list and the order people arrive in; it registers plugins and boots them in dependency order. |
|
||||
| [phrasebook](modules/phrasebook/README.md) | Namespaced translation catalogs loaded from plugin YAML, with locale fallback, placeholder interpolation and CLDR pluralization. | What you pack to speak the local language on holiday; it holds the translations. |
|
||||
| [postcard](modules/postcard/README.md) | Transactional mail from plugin-owned Markdown templates and layouts, delivered through a memory, log or SMTP driver. | What you send home in summer; it delivers the application's mail. |
|
||||
| [seaglass](modules/seaglass/README.md) | HTML sanitizer for stored rich text: a bluemonday allowlist policy that returns clean HTML byte for byte and reports what it removed from the rest. | Glass the sea has worn smooth; rough HTML goes in and only the safe, smooth part comes out. |
|
||||
| [sieve](modules/sieve/README.md) | Generic, dependency-free collection helpers that the standard library slices and maps packages lack: mapping, filtering, reducing, keying, grouping, deduplicating, partitioning and finding, plus a small set type. | A sand sieve: you shake a collection through it and keep what you need; map, filter and reduce for slices and maps. |
|
||||
| [signpost](modules/signpost/README.md) | Menu item type registry that collects the typed menu item types plugins register at boot, layers their URL patterns with host overrides and resolves menu and sitemap items through the owning plugin. | A beach signpost points the way to places; the registry tells menus and sitemaps where each item leads. |
|
||||
| [sunrise](modules/sunrise/README.md) | Bootstrap script that installs the summer CLI and creates a new SummerCMS application, or updates an existing one, in one command, embedded so it is versioned with the framework. | The start of a summer day; it starts a new application, or updates one, in one command. |
|
||||
|
||||
@@ -101,8 +101,9 @@ for _, f := range form.Fields {
|
||||
| `mltext` | A text field with one editor per locale; see [Markdown and multilingual fields](#markdown-and-multilingual-fields). |
|
||||
| `mltextarea` | A multi-line text field with one editor per locale; it takes `size` as `textarea` does. See [Markdown and multilingual fields](#markdown-and-multilingual-fields). |
|
||||
| `mlmarkdown` | A markdown field with one editor per locale; see [Markdown and multilingual fields](#markdown-and-multilingual-fields). |
|
||||
| `richeditor` | A WYSIWYG HTML editor for a host text column, sanitized on every save; see [Rich text](#rich-text). |
|
||||
|
||||
The WinterCMS widgets that are not in this list (the rich editor, the code editor, the color picker, the media finder, the repeater, the tag list and the others) are not provided. A field with one of those types stops the start-up, unless a plugin registers that alias as its own field type: plugins add field types with a custom element and Go code that decides what is stored; see [Plugin field types](field-types.md).
|
||||
The WinterCMS widgets that are not in this list (the code editor, the color picker, the media finder, the repeater, the tag list and the others) are not provided. A field with one of those types stops the start-up, unless a plugin registers that alias as its own field type: plugins add field types with a custom element and Go code that decides what is stored; see [Plugin field types](field-types.md).
|
||||
|
||||
### Field options
|
||||
|
||||
@@ -391,6 +392,25 @@ fields:
|
||||
|
||||
A save of `{ "title": { "en": "Hello", "pl": "Witaj" } }` lifts the nested map before `cabana.ProjectWritableFields` drops nested values. The default locale fills the host column; other locales are written through a published `cabana.TranslationWriter` after the host row has a primary key, still inside the controller's permissioned save transaction. The default locale is not duplicated into the translation store. Relation-child create and update on `cabana.RelationService` use the same lift and apply path as controller save; child show hydrates through `hydrateMLRecord` like a host GET. Show and save then run `hydrateMLRecord`: each declared ML field in the record payload is a map of every enabled locale (default from the host column, others from `TranslationWriter.TranslatedExact`; a missing non-default code is an empty string, never D-11 fallback). A non-string value, an undeclared locale, a missing default locale, or a nested map on a field that is not declared multilingual is a 422 on that field. Generic nested objects on other field types are still dropped. Cabana never imports a translate plugin and never exposes a public translate-write or translate-read route: if no writer is published, a nested ML map is a 422.
|
||||
|
||||
## Rich text
|
||||
|
||||
`type: richeditor` edits HTML in a host text column with a WYSIWYG editor in the admin SPA. It takes the common keys (`label`, `comment`, `span`, `required`, `tab`, `context`, `default`) plus `size`; any other key stops the start-up.
|
||||
|
||||
```yaml
|
||||
fields:
|
||||
description:
|
||||
label: acme.faq::lang.faq.description
|
||||
type: richeditor
|
||||
size: large
|
||||
```
|
||||
|
||||
The value travels as a string of HTML, or null. Anything else in a save body is a 422 on the field. Every submitted value is sanitized on the server before it is stored, on create and on update, through [seaglass](../../modules/seaglass/README.md): bluemonday's UGC policy, which keeps paragraphs, headings, lists, emphasis, quotes, tables, images and links with `http`, `https`, `mailto`, `tel` or relative URLs, and removes scripts, styles, frames, forms, comments, every `on*` attribute and every other URL scheme.
|
||||
|
||||
- HTML the policy keeps whole is stored byte for byte, so a value that only differs in escaping is never rewritten.
|
||||
- Otherwise the clean HTML is stored and the save still succeeds. The create or update response then carries `meta.notices` for the field: how many elements and attributes were removed with their names, how many comments, or that some markup changed. The admin shows them under the field.
|
||||
|
||||
The editor's own toolbar keeps to the same allowlist, but the server is the gate: a value posted by any client goes through the policy. The type is supported in controller forms only; settings forms and relation forms refuse it at start-up, so no save path stores an unsanitized value.
|
||||
|
||||
## What a save may write
|
||||
|
||||
The form's writable fields are bound to model columns at boot. A save passes only those fields that are also in the model's `Fillable` list, drops unknown keys, case variants and nested objects, and fills the model with `lagoon.Fill` (see [Models](../database/models.md)). Declared `mltext`, `mltextarea` and `mlmarkdown` locale maps are the exception: they are lifted first, as [Markdown and multilingual fields](#markdown-and-multilingual-fields). The model's validation rules (a `Rules` method returning `lagoon.Validate` rule strings) and the form's `required` flags are checked in the save's transaction, and a failure is a 422 with messages per field. A value that does not fit its column is also a 422 on that field. A controller implementing `pact.FormRules` supplies its own rule set per operation, which replaces the model's rules for admin saves.
|
||||
|
||||
3
go.mod
3
go.mod
@@ -19,6 +19,7 @@ require (
|
||||
github.com/knadh/koanf/providers/env/v2 v2.0.1
|
||||
github.com/knadh/koanf/providers/file v1.2.1
|
||||
github.com/knadh/koanf/v2 v2.3.6
|
||||
github.com/microcosm-cc/bluemonday v1.0.27
|
||||
github.com/nicksnyder/go-i18n/v2 v2.6.1
|
||||
github.com/riverqueue/river v0.47.0
|
||||
github.com/riverqueue/river/riverdriver/riverdatabasesql v0.47.0
|
||||
@@ -42,6 +43,7 @@ require (
|
||||
dario.cat/mergo v1.0.2 // indirect
|
||||
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
|
||||
github.com/Microsoft/go-winio v0.6.2 // indirect
|
||||
github.com/aymerick/douceur v0.2.0 // indirect
|
||||
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/containerd/errdefs v1.0.0 // indirect
|
||||
@@ -64,6 +66,7 @@ require (
|
||||
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/googleapis/gax-go/v2 v2.19.0 // indirect
|
||||
github.com/gorilla/css v1.0.1 // indirect
|
||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||
|
||||
6
go.sum
6
go.sum
@@ -74,6 +74,8 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.42.3 h1:ErklX/7uhSbkAAeyQD/Y1OoQ9hO3
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.42.3/go.mod h1:ULe4HCzfKPiR6R3HEurE3b1upEkuk8AkMrOKtaOxKO8=
|
||||
github.com/aws/smithy-go v1.26.0 h1:9ouqbi+NyKP7fV3Te7UElCwdAb6Y8uk7LGwPE5tVe/s=
|
||||
github.com/aws/smithy-go v1.26.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
|
||||
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
|
||||
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
|
||||
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
|
||||
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
@@ -157,6 +159,8 @@ github.com/googleapis/enterprise-certificate-proxy v0.3.14 h1:yh8ncqsbUY4shRD5dA
|
||||
github.com/googleapis/enterprise-certificate-proxy v0.3.14/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
|
||||
github.com/googleapis/gax-go/v2 v2.19.0 h1:fYQaUOiGwll0cGj7jmHT/0nPlcrZDFPrZRhTsoCr8hE=
|
||||
github.com/googleapis/gax-go/v2 v2.19.0/go.mod h1:w2ROXVdfGEVFXzmlciUU4EdjHgWvB5h2n6x/8XSTTJA=
|
||||
github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
|
||||
github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0=
|
||||
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
|
||||
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
|
||||
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
|
||||
@@ -201,6 +205,8 @@ github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o
|
||||
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
|
||||
github.com/mdelapenya/tlscert v0.2.0 h1:7H81W6Z/4weDvZBNOfQte5GpIMo0lGYEeWbkGp5LJHI=
|
||||
github.com/mdelapenya/tlscert v0.2.0/go.mod h1:O4njj3ELLnJjGdkN7M/vIVCpZ+Cf0L6muqOG4tLSl8o=
|
||||
github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
|
||||
github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB5OBs92Dimizgw2cA=
|
||||
github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
|
||||
github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s=
|
||||
github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=
|
||||
|
||||
@@ -38,6 +38,7 @@ Use cases:
|
||||
- Form-only fields: a controller implementing `pact.FormVirtualFields` lists fields of its `fields.yaml` that are not columns of the form. They are exempt from the column binding, never filled into the model and never part of a record response; the values an administrator submits reach the Form hooks through `cabana.VirtualFieldsFromContext`, only for fields whose `context` allows the operation, and a nested value is a 422 on the field. `type: password` is a masked field that must be listed this way, so a password is sent in a save body and never comes back. A controller implementing `pact.FormRules` supplies the validation rules per operation (`create` or `update`), which replace the model's `Rules()` for admin saves; a rule on a virtual field is checked against the submitted value, never against a model column of the same name. Neither is available on settings forms or relation forms.
|
||||
- Permission editor: a `type: permissioneditor` field with `mode: radio` (allow `1`, inherit, deny `-1`) or `mode: checkbox` (allow `1`) edits a record's permission set as a JSON object of code to integer. The controller implements `cabana.PermissionEditorProvider`: it returns the offered `cabana.PermissionOption` list per request (served on the field as `permissionOptions`, with `locked` for permissions the administrator may not change) and reads and stores the record's values, so the storage shape is the plugin's. A save answers 422 on the field for a value that is not an object of integers, a code that is not offered or a value outside the mode's set, and 403 `forbidden` when a locked code's value changes; stored codes that are not offered are kept. The widget fill contract is unchanged: a widget still writes scalar fields only.
|
||||
- Markdown and multilingual fields: `type: markdown` edits source on a host text column; `cabana.RenderMarkdown` turns that source into HTML with the pinned goldmark engine and no unsafe HTML, and leftover script or iframe tags, event handlers, or javascript, vbscript or data URLs are rejected; the admin form preview renders through it via POST `/markdown/preview`. `type: mltext`, `type: mltextarea` and `type: mlmarkdown` take a JSON object of locale code to string; `mltextarea` is the multi-line variant and takes `size` like `textarea`. The form schema's `cabana.FormMeta.EnabledLocales` lists every enabled code when a `cabana.TranslationWriter` is published, default locale first and then the other enabled codes in the writer's order, so every ML control opens on the locale stored in the host columns; create seeds an empty string per code, and Show/save replace the host scalar with that map via `hydrateMLRecord` and `TranslationWriter.TranslatedExact` (missing non-default codes stay empty; D-11 fallback is not applied). A GET host string is merged onto the seed so sibling locales are not dropped. The default locale fills the host column; other locales reach a plugin-published `cabana.TranslationWriter` after the host row has a primary key, still inside the controller's permissioned save transaction. Relation-child create, update and show on `cabana.RelationService` use the same lift, apply and `hydrateMLRecord` path as controller save, still with no standalone translate-write route. An unknown field type, including an unknown `ml*` type, fails boot.
|
||||
- Rich text: `type: richeditor` edits HTML in a host text column (the admin SPA shows a WYSIWYG editor). It takes the common field keys plus `size`; any other key fails boot, and settings and relation forms refuse the type (supported in controller forms only). On create and update a submitted value must be a string or null (anything else is a 422 on the field) and is sanitized through [seaglass](../seaglass/README.md) before it is filled: HTML the policy keeps whole is stored byte for byte, otherwise the clean HTML is stored and what was removed is reported in `meta.notices` under the field (`backend::lang.richeditor.removed_elements`, `removed_attributes`, `removed_comments` with the count and names, or `markup_changed`).
|
||||
- Dependent dropdowns: `dependsOn: <field>` on a `type: dropdown` field names a sibling with a scalar value (the same form, or the same repeater row); the dropdown takes no `options` and its controller implements `pact.DependentOptionsProvider`, else boot fails. GET `.../fields/{field}/dependent-options?values[<sibling>]=<value>` answers `cabana.AdminDependentOptionsResult` (`applies` and the localized `options` with their `depth`); the form schema carries `dependsOn`. A menu or sitemap controller returns `signpost.ReferenceOptions` here, the one admin binding of the [signpost](../signpost/README.md) menu item type registry. Settings and relation forms refuse `dependsOn` (supported in controller forms only).
|
||||
- Preset fields: `preset` on a `type: text` or `type: mltext` field (a source field name, or a mapping with `field` and `type`, `slug` or `exact`) makes the field follow another text or mltext field of the same form on the create screen until the administrator edits it, per locale for an mltext pair. The schema reports it as `preset` (`cabana.FieldPreset`); the server does not fill the field.
|
||||
- Server-rendered partials: `headerPartial: <name>` in `config_list.yaml` (a strip above the list) and `type: partial` with `path: <name>` in `fields.yaml` render the template `{ConfigDir}/_<name>.htm` with `html/template` against a view model from the controller's `pact.AdminPartialData`. The result reaches the SPA as an allowlisted node tree, never as an HTML string. A missing or unparsable template, a free-form path or a controller without `pact.AdminPartialData` fails boot.
|
||||
@@ -335,7 +336,7 @@ TOKEN=$(./bin/acme admin:token:create admin@example.com --name agent --scope blo
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: [backpack](../backpack/README.md), [boardwalk](../boardwalk/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [lagoon](../lagoon/README.md), [pact](../pact/README.md), [party](../party/README.md), [phrasebook](../phrasebook/README.md), [towel](../towel/README.md).
|
||||
- SummerCMS modules: [backpack](../backpack/README.md), [boardwalk](../boardwalk/README.md), [bonfire](../bonfire/README.md), [bouncer](../bouncer/README.md), [lagoon](../lagoon/README.md), [pact](../pact/README.md), [party](../party/README.md), [phrasebook](../phrasebook/README.md), [seaglass](../seaglass/README.md) (sanitizes `richeditor` values), [towel](../towel/README.md).
|
||||
- Third-party: `gorm.io/gorm` (with `gorm.io/gorm/clause`), `github.com/goccy/go-yaml` (with its `ast` package), `golang.org/x/net/html` (with its `atom` package; parses rendered partials for the allowlist walk).
|
||||
- Standard library: `bytes`, `context`, `crypto/sha256`, `database/sql`, `encoding/hex`, `encoding/json`, `errors`, `fmt`, `html/template`, `io`, `io/fs`, `log/slog`, `math`, `net`, `net/http`, `path`, `reflect`, `regexp`, `sort`, `strconv`, `strings`, `time`, `unicode/utf8`.
|
||||
- Tests additionally use `github.com/testcontainers/testcontainers-go` and its `modules/postgres` package, and [signpost](../signpost/README.md) for the dependent dropdown fixture.
|
||||
|
||||
@@ -3,12 +3,15 @@ package cabana_test
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/backpack"
|
||||
"git.golem15.com/golem15/summercms/modules/cabana"
|
||||
@@ -33,10 +36,14 @@ type companionsLink struct {
|
||||
Title string `gorm:"column:title"`
|
||||
Type *string `gorm:"column:type"`
|
||||
Reference *string `gorm:"column:reference"`
|
||||
// Description is the richeditor field.
|
||||
Description *string `gorm:"column:description"`
|
||||
}
|
||||
|
||||
func (companionsLink) TableName() string { return "cabana_companions_links" }
|
||||
func (companionsLink) Fillable() []string { return []string{"title", "type", "reference"} }
|
||||
func (companionsLink) TableName() string { return "cabana_companions_links" }
|
||||
func (companionsLink) Fillable() []string {
|
||||
return []string{"title", "type", "reference", "description"}
|
||||
}
|
||||
func (companionsLink) Rules() map[string]string { return map[string]string{"title": "required"} }
|
||||
|
||||
// companionsModels are the fixture tables, dropped and recreated per env.
|
||||
@@ -73,7 +80,7 @@ func (companionsPlugin) MenuItemTypes() []pact.MenuItemType {
|
||||
}
|
||||
|
||||
var (
|
||||
_ pact.HasMenuItemTypes = companionsPlugin{}
|
||||
_ pact.HasMenuItemTypes = companionsPlugin{}
|
||||
_ pact.DependentOptionsProvider = companionsLinks{}
|
||||
_ pact.DropdownOptionsProvider = companionsLinks{}
|
||||
)
|
||||
@@ -116,7 +123,7 @@ func newCompanionsEnv(t *testing.T) *conformEnv {
|
||||
t.Helper()
|
||||
gdb := adminGorm(t)
|
||||
resetCompanions(t, gdb)
|
||||
login := "companions-" + filepath.Base(t.TempDir())
|
||||
login := fmt.Sprintf("companions-%d", time.Now().UnixNano())
|
||||
insertAdmin(t, gdb, login, login+"@example.test", adminTestPassword, true, false)
|
||||
dir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(dir, "app.yaml"), []byte("name: cabana-companions\nlocale: en\nfallback_locale: en\n"), 0o644); err != nil {
|
||||
@@ -223,3 +230,72 @@ func TestCompanionsDependentSmoke(t *testing.T) {
|
||||
t.Fatalf("form schema does not carry dependsOn: %s", schema.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// companionsRecord is a create or update answer of the links controller.
|
||||
type companionsRecord struct {
|
||||
Data map[string]any `json:"data"`
|
||||
Meta struct {
|
||||
Notices []cabana.FieldNotice `json:"notices"`
|
||||
} `json:"meta"`
|
||||
}
|
||||
|
||||
func decodeCompanionsRecord(t *testing.T, raw []byte) companionsRecord {
|
||||
t.Helper()
|
||||
var rec companionsRecord
|
||||
if err := json.Unmarshal(raw, &rec); err != nil {
|
||||
t.Fatalf("record: %v\n%s", err, raw)
|
||||
}
|
||||
return rec
|
||||
}
|
||||
|
||||
// TestCompanionsRichEditorSmoke checks the D-10 save gate: a richeditor value
|
||||
// is stored sanitized with notices naming what was removed, clean HTML is
|
||||
// stored byte for byte with no notice, and a value that is not a string is a
|
||||
// 422 on the field.
|
||||
func TestCompanionsRichEditorSmoke(t *testing.T) {
|
||||
e := newCompanionsEnv(t)
|
||||
dirty := `<p onclick="steal()">Answer</p><script>alert(1)</script>`
|
||||
rec := e.send(t, http.MethodPost, "/acme/companions/links", map[string]any{"title": "Dirty", "description": dirty}, true)
|
||||
if rec.Code != http.StatusCreated {
|
||||
t.Fatalf("create status=%d body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
created := decodeCompanionsRecord(t, rec.Body.Bytes())
|
||||
id := dataID(t, rec.Body.Bytes())
|
||||
var stored companionsLink
|
||||
if err := e.db.First(&stored, id).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if stored.Description == nil || *stored.Description != "<p>Answer</p>" {
|
||||
t.Fatalf("stored description = %v", stored.Description)
|
||||
}
|
||||
var fields []string
|
||||
for _, n := range created.Meta.Notices {
|
||||
fields = append(fields, n.Field)
|
||||
if n.Field != "description" || n.Message == "" {
|
||||
t.Fatalf("notice = %+v", n)
|
||||
}
|
||||
}
|
||||
if len(fields) != 2 {
|
||||
t.Fatalf("notices = %+v, want removed elements and attributes on description", created.Meta.Notices)
|
||||
}
|
||||
|
||||
clean := "<p>It's <strong>fine</strong> & \"quoted\"</p>\n<ul><li>one</li></ul>"
|
||||
rec = e.send(t, http.MethodPut, fmt.Sprintf("/acme/companions/links/%d", id), map[string]any{"title": "Clean", "description": clean}, true)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("update status=%d body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if updated := decodeCompanionsRecord(t, rec.Body.Bytes()); len(updated.Meta.Notices) != 0 {
|
||||
t.Fatalf("clean HTML reported notices: %+v", updated.Meta.Notices)
|
||||
}
|
||||
if err := e.db.First(&stored, id).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if stored.Description == nil || *stored.Description != clean {
|
||||
t.Fatalf("clean description not stored byte for byte: %v", stored.Description)
|
||||
}
|
||||
|
||||
rec = e.send(t, http.MethodPut, fmt.Sprintf("/acme/companions/links/%d", id), map[string]any{"title": "Clean", "description": []any{"<p>x</p>"}}, true)
|
||||
if rec.Code != http.StatusUnprocessableEntity || !strings.Contains(rec.Body.String(), `"description"`) {
|
||||
t.Fatalf("non-string description status=%d body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -607,6 +607,11 @@ func (s CRUDService) save(ctx context.Context, cc *CompiledController, id any, i
|
||||
// Plugin field values are lifted the same way and decoded by their type
|
||||
// inside the transaction, once the record is loaded.
|
||||
pluginValues := liftPluginValues(cc, in.Body, op)
|
||||
// A richeditor value is a string or null; anything else is refused on
|
||||
// the field instead of being dropped by the projection.
|
||||
if err := checkRichEditorValues(cc, in.Body, op); err != nil {
|
||||
return RecordResult{}, err
|
||||
}
|
||||
var (
|
||||
result RecordResult
|
||||
notices []pluginNotice
|
||||
@@ -631,7 +636,8 @@ func (s CRUDService) save(ctx context.Context, cc *CompiledController, id any, i
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
notices = decoded.notices
|
||||
// D-10: every richeditor value is sanitized before Fill sees it.
|
||||
notices = append(decoded.notices, sanitizeRichEditors(cc, projected)...)
|
||||
// The Form hooks read what this save writes for plugin field types.
|
||||
ctx = withFieldWrites(ctx, decoded.writes)
|
||||
if err := lagoon.Fill(target, fillAllowed(cc, target, op), projected, false); err != nil {
|
||||
@@ -1244,7 +1250,7 @@ func modelColumns(model any) map[string]struct{} {
|
||||
func scalarFormField(typ string) bool {
|
||||
switch typ {
|
||||
case "text", "textarea", "number", "checkbox", "switch", "dropdown", "datepicker",
|
||||
"markdown", "mltext", "mltextarea", "mlmarkdown":
|
||||
"markdown", "mltext", "mltextarea", "mlmarkdown", "richeditor":
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
|
||||
@@ -106,6 +106,9 @@ func requireDependentProvider(ctl pact.AdminController, fields []FormField) erro
|
||||
// that only an admin controller's save path supports: its checks run in the
|
||||
// controller save and nowhere else.
|
||||
func controllerFormOnly(field FormField) error {
|
||||
if field.Type == richEditorType {
|
||||
return fmt.Errorf("richeditor is supported in controller forms only")
|
||||
}
|
||||
if field.DependsOn != "" {
|
||||
return fmt.Errorf("dependsOn is supported in controller forms only")
|
||||
}
|
||||
|
||||
135
modules/cabana/field_richeditor.go
Normal file
135
modules/cabana/field_richeditor.go
Normal file
@@ -0,0 +1,135 @@
|
||||
package cabana
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/pact"
|
||||
"git.golem15.com/golem15/summercms/modules/seaglass"
|
||||
"github.com/goccy/go-yaml/ast"
|
||||
)
|
||||
|
||||
// richEditorType is the core HTML WYSIWYG field type (D-10).
|
||||
const richEditorType = "richeditor"
|
||||
|
||||
// The phrase keys of the notices a richeditor save reports (backend::lang).
|
||||
const (
|
||||
msgRichRemovedElements = "backend::lang.richeditor.removed_elements"
|
||||
msgRichRemovedAttributes = "backend::lang.richeditor.removed_attributes"
|
||||
msgRichRemovedComments = "backend::lang.richeditor.removed_comments"
|
||||
msgRichMarkupChanged = "backend::lang.richeditor.markup_changed"
|
||||
)
|
||||
|
||||
// richEditorKeys are the fields.yaml keys a richeditor field accepts: the
|
||||
// common field keys plus size.
|
||||
var richEditorKeys = map[string]bool{
|
||||
"label": true, "comment": true, "span": true, "type": true, "required": true,
|
||||
"tab": true, "context": true, "size": true, "default": true,
|
||||
}
|
||||
|
||||
// compileRichEditorKeys refuses every key a richeditor field does not take.
|
||||
func compileRichEditorKeys(typ string, values map[string]ast.Node) error {
|
||||
if typ != richEditorType {
|
||||
return nil
|
||||
}
|
||||
for key := range values {
|
||||
if !richEditorKeys[key] {
|
||||
return fmt.Errorf("%s is not valid on type: richeditor", key)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkRichEditorValues refuses, before the save transaction, a submitted
|
||||
// richeditor value that is neither a string nor null: it is a 422 on the
|
||||
// field, never silently dropped.
|
||||
func checkRichEditorValues(cc *CompiledController, body map[string]any, op string) error {
|
||||
if cc == nil || cc.Form == nil {
|
||||
return nil
|
||||
}
|
||||
for _, field := range cc.Form.Fields {
|
||||
if field.Type != richEditorType || !contextAllows(cc, field.Name, op) {
|
||||
continue
|
||||
}
|
||||
value, present := body[field.Name]
|
||||
if !present {
|
||||
continue
|
||||
}
|
||||
switch value.(type) {
|
||||
case nil, string:
|
||||
default:
|
||||
return &ValidationError{Details: fillTypeDetails(field.Name)}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sanitizeRichEditors runs every projected richeditor value through
|
||||
// seaglass.SanitizeReport and writes the clean value back, so no richeditor
|
||||
// value reaches Fill unsanitized. HTML the policy keeps whole is stored byte
|
||||
// for byte; what it removed is returned as notices on the field.
|
||||
func sanitizeRichEditors(cc *CompiledController, projected map[string]any) []pluginNotice {
|
||||
if cc == nil || cc.Form == nil {
|
||||
return nil
|
||||
}
|
||||
var notices []pluginNotice
|
||||
for _, field := range cc.Form.Fields {
|
||||
if field.Type != richEditorType {
|
||||
continue
|
||||
}
|
||||
key := fillKeyOf(cc, field.Name)
|
||||
value, present := projected[key]
|
||||
if !present {
|
||||
continue
|
||||
}
|
||||
text, ok := value.(string)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
clean, report := seaglass.SanitizeReport(text)
|
||||
projected[key] = clean
|
||||
for _, notice := range richEditorNotices(report) {
|
||||
notices = append(notices, pluginNotice{field: field.Name, notice: notice})
|
||||
}
|
||||
}
|
||||
return notices
|
||||
}
|
||||
|
||||
// fillKeyOf is the fill key bound to a writable field name.
|
||||
func fillKeyOf(cc *CompiledController, name string) string {
|
||||
for _, field := range cc.Writable {
|
||||
if field.Name == name {
|
||||
return field.FillKey
|
||||
}
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
// richEditorNotices turns a removal report into admin notices: removed
|
||||
// elements, removed attributes, removed comments, in that order and only when
|
||||
// non-zero, and markup_changed when the policy changed the HTML in a way none
|
||||
// of them describe.
|
||||
func richEditorNotices(r seaglass.Removals) []pact.FormFieldNotice {
|
||||
var out []pact.FormFieldNotice
|
||||
if r.ElementCount > 0 {
|
||||
out = append(out, pact.FormFieldNotice{
|
||||
Message: msgRichRemovedElements,
|
||||
Params: map[string]string{"names": strings.Join(r.Elements, ", ")},
|
||||
Count: r.ElementCount,
|
||||
})
|
||||
}
|
||||
if r.AttributeCount > 0 {
|
||||
out = append(out, pact.FormFieldNotice{
|
||||
Message: msgRichRemovedAttributes,
|
||||
Params: map[string]string{"names": strings.Join(r.Attributes, ", ")},
|
||||
Count: r.AttributeCount,
|
||||
})
|
||||
}
|
||||
if r.Comments > 0 {
|
||||
out = append(out, pact.FormFieldNotice{Message: msgRichRemovedComments, Count: r.Comments})
|
||||
}
|
||||
if r.Changed && len(out) == 0 {
|
||||
out = append(out, pact.FormFieldNotice{Message: msgRichMarkupChanged})
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -27,6 +27,7 @@ var (
|
||||
"widget": {}, "partial": {}, "fileupload": {}, "datepicker": {},
|
||||
"password": {}, "permissioneditor": {},
|
||||
"markdown": {}, "mltext": {}, "mltextarea": {}, "mlmarkdown": {},
|
||||
"richeditor": {},
|
||||
}
|
||||
formSpans = map[string]struct{}{
|
||||
"left": {}, "right": {}, "full": {}, "auto": {}, "row": {},
|
||||
@@ -512,6 +513,9 @@ func compileFieldNode(name string, node ast.Node, types *fieldTypeRegistry) (For
|
||||
if _, ok := formFieldTypes[typ]; !ok && plugin == nil {
|
||||
return FormField{}, fmt.Errorf("unsupported type %s", typ)
|
||||
}
|
||||
if err := compileRichEditorKeys(typ, values); err != nil {
|
||||
return FormField{}, err
|
||||
}
|
||||
field := FormField{Name: name, Type: typ}
|
||||
if node, ok := values["label"]; ok {
|
||||
field.Label, err = nodeString(node)
|
||||
|
||||
@@ -12,3 +12,7 @@ fields:
|
||||
type: dropdown
|
||||
dependsOn: type
|
||||
emptyOption: backend::lang.form.none
|
||||
description:
|
||||
label: Description
|
||||
type: richeditor
|
||||
size: large
|
||||
|
||||
@@ -323,3 +323,14 @@ myaccount:
|
||||
avatar_hint: JPG, PNG, GIF or WebP, up to 2 MB.
|
||||
avatar_saved: Avatar saved
|
||||
avatar_removed: Avatar removed
|
||||
richeditor:
|
||||
removed_elements:
|
||||
one: ":count element was removed: :names"
|
||||
other: ":count elements were removed: :names"
|
||||
removed_attributes:
|
||||
one: ":count attribute was removed: :names"
|
||||
other: ":count attributes were removed: :names"
|
||||
removed_comments:
|
||||
one: ":count HTML comment was removed"
|
||||
other: ":count HTML comments were removed"
|
||||
markup_changed: The HTML filter changed some markup.
|
||||
|
||||
@@ -345,3 +345,20 @@ myaccount:
|
||||
avatar_hint: JPG, PNG, GIF lub WebP, maksymalnie 2 MB.
|
||||
avatar_saved: Zapisano awatar
|
||||
avatar_removed: Usunięto awatar
|
||||
richeditor:
|
||||
removed_elements:
|
||||
one: "Usunięto :count element: :names"
|
||||
few: "Usunięto :count elementy: :names"
|
||||
many: "Usunięto :count elementów: :names"
|
||||
other: "Usunięto :count elementu: :names"
|
||||
removed_attributes:
|
||||
one: "Usunięto :count atrybut: :names"
|
||||
few: "Usunięto :count atrybuty: :names"
|
||||
many: "Usunięto :count atrybutów: :names"
|
||||
other: "Usunięto :count atrybutu: :names"
|
||||
removed_comments:
|
||||
one: "Usunięto :count komentarz HTML"
|
||||
few: "Usunięto :count komentarze HTML"
|
||||
many: "Usunięto :count komentarzy HTML"
|
||||
other: "Usunięto :count komentarza HTML"
|
||||
markup_changed: Filtr HTML zmienił część znaczników.
|
||||
|
||||
65
modules/seaglass/README.md
Normal file
65
modules/seaglass/README.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# seaglass
|
||||
|
||||
HTML sanitizer for stored rich text: a bluemonday allowlist policy that returns clean HTML byte for byte and reports what it removed from the rest.
|
||||
|
||||
`import "git.golem15.com/golem15/summercms/modules/seaglass"`
|
||||
|
||||
**Name:** Glass the sea has worn smooth; rough HTML goes in and only the safe, smooth part comes out.
|
||||
|
||||
## Overview
|
||||
|
||||
`seaglass` is the framework's one HTML policy for rich text an administrator stores, such as cabana's `type: richeditor` field. It wraps [bluemonday](https://github.com/microcosm-cc/bluemonday) v1.0.27 with the policy the Journal plugin uses for imported WinterCMS content, without that plugin's editor-specific rules. A tokenizing sanitizer checks attributes and URLs the way a browser parses them, so it does not share the bypasses of regex gates (`<img src="x"onerror=…>`, `javascript:`, `<svg/onload=…>`).
|
||||
|
||||
The comparison is byte-stable: when the input and the policy's output tokenize to the same HTML tokens, the input itself is returned, so a value that only differs in escaping is never rewritten. Otherwise the policy's output is returned with a report of the removed elements, attributes and comments, which cabana shows to the administrator as save notices.
|
||||
|
||||
## Role in the framework
|
||||
|
||||
A leaf utility with no SummerCMS dependency. [cabana](../cabana/README.md) runs every richeditor value through it on create and update, top level and inside repeater rows; plugins that serve stored HTML in a public API can run it again on output, for content that was imported without passing the admin save.
|
||||
|
||||
Use cases:
|
||||
|
||||
- Sanitize an HTML field on save and tell the editor what was removed.
|
||||
- Check that stored HTML is already clean (`seaglass.Check`), for example in an import report.
|
||||
- Sanitize HTML on output in a public API.
|
||||
|
||||
## Features
|
||||
|
||||
- bluemonday's UGC policy with links kept as written (no `rel="nofollow"`), `tel:` links next to `http`, `https`, `mailto` and relative URLs, and the `class`, `dir`, microdata, footnote `role` and inline `color`, `font-size`, `font-weight` and `text-align` styles imported WinterCMS content uses.
|
||||
- Everything else is removed: `script`, `style`, `iframe`, `object`, `embed`, `form`, `svg`, `math`, comments, every `on*` attribute, `target`, `rel`, and URLs with any other scheme.
|
||||
- `seaglass.SanitizeReport` returns HTML the policy keeps whole byte for byte with a zero `seaglass.Removals`; a tag that repeats an attribute is always a change.
|
||||
- `seaglass.Removals` names the removed elements and attributes (at most ten distinct names each, in first-seen order) and counts every removal and every removed comment.
|
||||
|
||||
## Usage
|
||||
|
||||
```go
|
||||
clean, report := seaglass.SanitizeReport(`<p onclick="x()">Hi</p><script>alert(1)</script>`)
|
||||
// clean == "<p>Hi</p>"
|
||||
// report.Elements == []string{"script"}, report.Attributes == []string{"onclick"}
|
||||
|
||||
if err := seaglass.Check(stored); err != nil {
|
||||
// stored HTML would change under the policy
|
||||
}
|
||||
```
|
||||
|
||||
## API reference
|
||||
|
||||
| Identifier | Description |
|
||||
|------------|-------------|
|
||||
| `seaglass.Sanitize` | Returns the HTML filtered through the policy: the input itself when the policy keeps all of it. |
|
||||
| `seaglass.SanitizeReport` | `seaglass.Sanitize` with a `seaglass.Removals` report of what was removed; a zero report exactly when the input is returned. |
|
||||
| `seaglass.Check` | Returns an error when the policy would change the HTML, nil when it keeps it whole. |
|
||||
| `seaglass.Removals` | What was removed: `Elements` and `Attributes` (distinct names), `ElementCount` and `AttributeCount` (every removal), `Comments`, and `Changed` (the HTML differs from the input). |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- SummerCMS modules: none.
|
||||
- Third-party: `github.com/microcosm-cc/bluemonday` v1.0.27 (the policy; decision note `.planning/notes/2026-10-11-framework-html-sanitizer-bluemonday.md`), `golang.org/x/net/html` (the token comparison and the removal report).
|
||||
- Standard library: `errors`, `fmt`, `io`, `regexp`, `slices`, `strings`.
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
go test ./modules/seaglass/...
|
||||
```
|
||||
|
||||
The tests run in memory and need no external services.
|
||||
398
modules/seaglass/seaglass.go
Normal file
398
modules/seaglass/seaglass.go
Normal file
@@ -0,0 +1,398 @@
|
||||
// Package seaglass is the framework's HTML sanitizer: a bluemonday allowlist
|
||||
// policy for stored rich text, with a byte-stable comparison so HTML the
|
||||
// policy keeps whole is returned unchanged, and a report of what it removed.
|
||||
package seaglass
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/microcosm-cc/bluemonday"
|
||||
xhtml "golang.org/x/net/html"
|
||||
)
|
||||
|
||||
// policy is the allowlist for stored rich text. It is safe for concurrent
|
||||
// use: bluemonday policies are read-only once built.
|
||||
var policy = newPolicy()
|
||||
|
||||
// newPolicy is bluemonday's UGC policy with the additions stored rich text
|
||||
// needs: links keep their form (no rel="nofollow" is added), tel: links are
|
||||
// allowed next to http, https, mailto and relative URLs, and the class, dir,
|
||||
// microdata, footnote role and inline style attributes imported WinterCMS
|
||||
// content uses. Scripts, styles, frames, forms, embedded objects, SVG and
|
||||
// MathML, comments, every on* attribute, target and rel, and every other URL
|
||||
// scheme are removed.
|
||||
func newPolicy() *bluemonday.Policy {
|
||||
p := bluemonday.UGCPolicy()
|
||||
p.RequireNoFollowOnLinks(false)
|
||||
p.AllowURLSchemes("tel")
|
||||
p.AllowElements("footer")
|
||||
p.AllowAttrs("class").Matching(bluemonday.SpaceSeparatedTokens).Globally()
|
||||
p.AllowAttrs("itemscope").Matching(regexp.MustCompile(`^(|itemscope)$`)).Globally()
|
||||
p.AllowAttrs("itemtype").Matching(regexp.MustCompile(`^https?://schema\.org/[A-Za-z]+$`)).Globally()
|
||||
p.AllowAttrs("itemprop").Matching(bluemonday.SpaceSeparatedTokens).Globally()
|
||||
p.AllowStyles("color", "font-size", "font-weight", "text-align").Globally()
|
||||
p.AllowAttrs("role").Matching(regexp.MustCompile(`^doc-(noteref|endnotes|backlink)$`)).OnElements("a", "div")
|
||||
p.AllowAttrs("dir").Matching(regexp.MustCompile(`(?i)^(ltr|rtl|auto)$`)).Globally()
|
||||
return p
|
||||
}
|
||||
|
||||
// maxRemovalNames is how many distinct names a Removals list keeps.
|
||||
const maxRemovalNames = 10
|
||||
|
||||
// Removals reports what SanitizeReport removed. Elements and Attributes are
|
||||
// the distinct removed element and attribute names in first-seen order, at
|
||||
// most ten each (a cut list ends with "…"); ElementCount and AttributeCount
|
||||
// count every removal, so they stay exact when a list is cut. An attribute
|
||||
// counts as removed when the policy dropped it, when it repeats a name the tag
|
||||
// already has, or when the policy changed its value (a style that lost a
|
||||
// property counts under "style"). Comments counts removed HTML comments.
|
||||
// Changed is true whenever the returned HTML differs from the input; it is the
|
||||
// only field set when the policy changed markup none of the others describe.
|
||||
type Removals struct {
|
||||
Elements []string
|
||||
ElementCount int
|
||||
Attributes []string
|
||||
AttributeCount int
|
||||
Comments int
|
||||
Changed bool
|
||||
}
|
||||
|
||||
func (r *Removals) addElement(name string) {
|
||||
r.ElementCount++
|
||||
r.Elements = addName(r.Elements, name)
|
||||
}
|
||||
|
||||
func (r *Removals) addAttribute(name string) {
|
||||
r.AttributeCount++
|
||||
r.Attributes = addName(r.Attributes, name)
|
||||
}
|
||||
|
||||
func addName(names []string, name string) []string {
|
||||
if slices.Contains(names, name) {
|
||||
return names
|
||||
}
|
||||
if len(names) == maxRemovalNames {
|
||||
return append(names, "…")
|
||||
}
|
||||
if len(names) > maxRemovalNames {
|
||||
return names
|
||||
}
|
||||
return append(names, name)
|
||||
}
|
||||
|
||||
// Sanitize returns html filtered through the policy: html itself, byte for
|
||||
// byte, when the policy keeps all of it, otherwise the policy's output.
|
||||
func Sanitize(html string) string {
|
||||
out, _ := SanitizeReport(html)
|
||||
return out
|
||||
}
|
||||
|
||||
// SanitizeReport is Sanitize with a report of what the policy removed. When
|
||||
// html and the policy's output tokenize to the same HTML tokens (same type,
|
||||
// tag name, decoded text and set of decoded attributes), html is returned
|
||||
// unchanged with a zero Removals, so re-escaping alone is never a change.
|
||||
// Otherwise the policy's output is returned with Changed set and the removed
|
||||
// elements, attributes and comments, found by aligning the start tags of html
|
||||
// with those of the output in order (the policy only removes, it never adds
|
||||
// or reorders).
|
||||
func SanitizeReport(html string) (string, Removals) {
|
||||
clean, changed := sanitize(html)
|
||||
if !changed || clean == html {
|
||||
return html, Removals{}
|
||||
}
|
||||
report := removalReport(html, clean)
|
||||
report.Changed = true
|
||||
return clean, report
|
||||
}
|
||||
|
||||
// Check returns an error when the policy would change any element, attribute
|
||||
// or text of html, and nil when it keeps html whole.
|
||||
func Check(html string) error {
|
||||
if _, changed := sanitize(html); changed {
|
||||
return errors.New("seaglass: HTML contains elements, attributes or URLs that are not allowed")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sanitize returns doc unchanged when the policy keeps all of it, and the
|
||||
// policy's output with changed set otherwise. A tag that repeats an attribute
|
||||
// is always a change: the tokenizer keeps only the first of them, so the
|
||||
// comparison would not see the others in the stored bytes.
|
||||
func sanitize(doc string) (string, bool) {
|
||||
clean := policy.Sanitize(doc)
|
||||
if sameTokens(doc, clean) {
|
||||
return doc, false
|
||||
}
|
||||
return clean, true
|
||||
}
|
||||
|
||||
// sameTokens reports whether a and b tokenize to the same sequence of HTML
|
||||
// tokens. It is false when either repeats an attribute in a tag.
|
||||
func sameTokens(a, b string) bool {
|
||||
ta, oka := htmlTokens(a)
|
||||
if !oka {
|
||||
return false
|
||||
}
|
||||
tb, okb := htmlTokens(b)
|
||||
return okb && slices.Equal(ta, tb)
|
||||
}
|
||||
|
||||
// htmlTokens tokenizes doc. It is not ok when the tokenizer stops on an error
|
||||
// other than EOF, or when a start tag's raw attributes differ from the
|
||||
// attributes the tokenizer kept, which happens when a name repeats.
|
||||
func htmlTokens(doc string) ([]string, bool) {
|
||||
var out []string
|
||||
z := xhtml.NewTokenizer(strings.NewReader(doc))
|
||||
for {
|
||||
tt := z.Next()
|
||||
if tt == xhtml.ErrorToken {
|
||||
return out, errors.Is(z.Err(), io.EOF)
|
||||
}
|
||||
var raw []string
|
||||
if tt == xhtml.StartTagToken || tt == xhtml.SelfClosingTagToken {
|
||||
raw = rawAttrNames(z.Raw())
|
||||
}
|
||||
t := z.Token()
|
||||
if tt == xhtml.StartTagToken || tt == xhtml.SelfClosingTagToken {
|
||||
if len(raw) != len(t.Attr) {
|
||||
return out, false
|
||||
}
|
||||
for i, a := range t.Attr {
|
||||
if raw[i] != a.Key {
|
||||
return out, false
|
||||
}
|
||||
}
|
||||
}
|
||||
attrs := make([]string, 0, len(t.Attr))
|
||||
for _, a := range t.Attr {
|
||||
attrs = append(attrs, a.Namespace+":"+a.Key+"="+a.Val)
|
||||
}
|
||||
slices.Sort(attrs)
|
||||
out = append(out, fmt.Sprintf("%d|%s|%q", tt, t.Data, attrs))
|
||||
}
|
||||
}
|
||||
|
||||
// reportTag is one start tag of a document as removalReport sees it.
|
||||
type reportTag struct {
|
||||
name string
|
||||
raw []string // attribute names in source order, repeats included
|
||||
attrs map[string]string // decoded values, the first of a repeated name
|
||||
}
|
||||
|
||||
// reportTokens returns doc's start tags and its number of comments.
|
||||
func reportTokens(doc string) ([]reportTag, int) {
|
||||
var (
|
||||
tags []reportTag
|
||||
comments int
|
||||
)
|
||||
z := xhtml.NewTokenizer(strings.NewReader(doc))
|
||||
for {
|
||||
tt := z.Next()
|
||||
switch tt {
|
||||
case xhtml.ErrorToken:
|
||||
return tags, comments
|
||||
case xhtml.CommentToken:
|
||||
comments++
|
||||
case xhtml.StartTagToken, xhtml.SelfClosingTagToken:
|
||||
raw := rawAttrNames(z.Raw())
|
||||
t := z.Token()
|
||||
attrs := make(map[string]string, len(t.Attr))
|
||||
for _, a := range t.Attr {
|
||||
key := a.Key
|
||||
if a.Namespace != "" {
|
||||
key = a.Namespace + ":" + a.Key
|
||||
}
|
||||
if _, seen := attrs[key]; !seen {
|
||||
attrs[key] = a.Val
|
||||
}
|
||||
}
|
||||
tags = append(tags, reportTag{name: t.Data, raw: raw, attrs: attrs})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// keeps reports whether out can be the policy's output of in: the same
|
||||
// element, and every output attribute present in the input with the same
|
||||
// value, except style, whose properties the policy filters one by one.
|
||||
func (in reportTag) keeps(out reportTag) bool {
|
||||
if in.name != out.name {
|
||||
return false
|
||||
}
|
||||
for key, val := range out.attrs {
|
||||
got, ok := in.attrs[key]
|
||||
if !ok || (got != val && key != "style") {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// removalReport compares doc with the policy's output clean. It aligns the
|
||||
// start tags greedily, preferring an output tag whose attributes the input
|
||||
// tag can have produced; when that leaves output tags unmatched, it aligns by
|
||||
// element name only.
|
||||
func removalReport(doc, clean string) Removals {
|
||||
in, inComments := reportTokens(doc)
|
||||
out, outComments := reportTokens(clean)
|
||||
match := alignTags(in, out, reportTag.keeps)
|
||||
if match == nil {
|
||||
match = alignTags(in, out, func(a, b reportTag) bool { return a.name == b.name })
|
||||
}
|
||||
var r Removals
|
||||
for i, tag := range in {
|
||||
j := -1
|
||||
if match != nil {
|
||||
j = match[i]
|
||||
}
|
||||
if j < 0 {
|
||||
r.addElement(tag.name)
|
||||
continue
|
||||
}
|
||||
kept := out[j]
|
||||
seen := make(map[string]bool, len(tag.raw))
|
||||
for _, name := range tag.raw {
|
||||
if seen[name] {
|
||||
r.addAttribute(name) // a repeat the tokenizer dropped
|
||||
continue
|
||||
}
|
||||
seen[name] = true
|
||||
val, ok := kept.attrs[name]
|
||||
if !ok || val != tag.attrs[name] {
|
||||
r.addAttribute(name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if inComments > outComments {
|
||||
r.Comments = inComments - outComments
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// alignTags matches every tag of out to a tag of in, in order, with the first
|
||||
// input tag for which same holds. The result maps each input index to its
|
||||
// output index or -1. It is nil when an output tag stays unmatched.
|
||||
func alignTags(in, out []reportTag, same func(in, out reportTag) bool) []int {
|
||||
match := make([]int, len(in))
|
||||
j := 0
|
||||
for i := range in {
|
||||
match[i] = -1
|
||||
if j < len(out) && same(in[i], out[j]) {
|
||||
match[i] = j
|
||||
j++
|
||||
}
|
||||
}
|
||||
if j < len(out) {
|
||||
return nil
|
||||
}
|
||||
return match
|
||||
}
|
||||
|
||||
// rawAttrNames returns the ASCII-lowercased name of every attribute in the
|
||||
// raw start tag, repeats included, in source order. It scans the tag as
|
||||
// golang.org/x/net/html's tokenizer does (readTagName, readTagAttrKey and
|
||||
// readTagAttrVal), which drops every repeat of a name it has seen. A scan that
|
||||
// disagrees with the tokenizer only makes htmlTokens report a change.
|
||||
func rawAttrNames(raw []byte) []string {
|
||||
i, n := 2, len(raw) // past '<' and the first byte of the tag name
|
||||
skipSpace := func() {
|
||||
for i < n && isHTMLSpace(raw[i]) {
|
||||
i++
|
||||
}
|
||||
}
|
||||
for i < n {
|
||||
c := raw[i]
|
||||
i++
|
||||
if isHTMLSpace(c) {
|
||||
break
|
||||
}
|
||||
if c == '/' || c == '>' {
|
||||
i--
|
||||
break
|
||||
}
|
||||
}
|
||||
var names []string
|
||||
skipSpace()
|
||||
for i < n {
|
||||
if raw[i] == '>' {
|
||||
break
|
||||
}
|
||||
// Attribute name: an '=' that starts it is part of the name.
|
||||
start, end := i, n
|
||||
for i < n {
|
||||
c := raw[i]
|
||||
i++
|
||||
if c == '=' && i == start+1 {
|
||||
continue
|
||||
}
|
||||
if c == '=' || c == '/' || c == '>' || isHTMLSpace(c) {
|
||||
i--
|
||||
end = i
|
||||
break
|
||||
}
|
||||
}
|
||||
// Attribute value.
|
||||
skipSpace()
|
||||
if i < n {
|
||||
c := raw[i]
|
||||
i++
|
||||
switch {
|
||||
case c == '/':
|
||||
case c != '=':
|
||||
i--
|
||||
default:
|
||||
skipSpace()
|
||||
if i < n {
|
||||
q := raw[i]
|
||||
i++
|
||||
switch q {
|
||||
case '>':
|
||||
i--
|
||||
case '"', '\'':
|
||||
for i < n {
|
||||
i++
|
||||
if raw[i-1] == q {
|
||||
break
|
||||
}
|
||||
}
|
||||
default:
|
||||
for i < n {
|
||||
c := raw[i]
|
||||
i++
|
||||
if isHTMLSpace(c) {
|
||||
break
|
||||
}
|
||||
if c == '>' {
|
||||
i--
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if end > start {
|
||||
names = append(names, asciiLower(raw[start:end]))
|
||||
}
|
||||
skipSpace()
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
func isHTMLSpace(c byte) bool {
|
||||
return c == ' ' || c == '\n' || c == '\r' || c == '\t' || c == '\f'
|
||||
}
|
||||
|
||||
func asciiLower(b []byte) string {
|
||||
out := []byte(string(b))
|
||||
for i, c := range out {
|
||||
if 'A' <= c && c <= 'Z' {
|
||||
out[i] = c + 'a' - 'A'
|
||||
}
|
||||
}
|
||||
return string(out)
|
||||
}
|
||||
53
modules/seaglass/seaglass_smoke_test.go
Normal file
53
modules/seaglass/seaglass_smoke_test.go
Normal file
@@ -0,0 +1,53 @@
|
||||
package seaglass_test
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.golem15.com/golem15/summercms/modules/seaglass"
|
||||
)
|
||||
|
||||
// TestSeaglassSmoke checks the two promises of the sanitizer: clean HTML
|
||||
// comes back byte for byte with an empty report, and unsafe constructs are
|
||||
// removed and named.
|
||||
func TestSeaglassSmoke(t *testing.T) {
|
||||
clean := "<p>Mushrooms <strong>can</strong> help:</p>\n<ul><li>focus</li><li><a href=\"https://example.com/a?b=1&c=2\">sleep</a></li></ul>\n<p>Call <a href=\"tel:+48123\">us</a> or <a href=\"/kontakt\">write</a>.</p>"
|
||||
got, report := seaglass.SanitizeReport(clean)
|
||||
if got != clean || report.Changed || report.ElementCount+report.AttributeCount+report.Comments != 0 {
|
||||
t.Fatalf("clean HTML changed: %q %+v", got, report)
|
||||
}
|
||||
if err := seaglass.Check(clean); err != nil {
|
||||
t.Fatalf("Check(clean) = %v", err)
|
||||
}
|
||||
// Re-escaping alone is not a change: bluemonday writes ' here.
|
||||
quoted := `<p>It's "fine"</p>`
|
||||
if got := seaglass.Sanitize(quoted); got != quoted {
|
||||
t.Fatalf("re-escaped text changed: %q", got)
|
||||
}
|
||||
|
||||
dirty := `<p onclick="steal()">Hi <a href="javascript:alert(1)">there</a></p><script>alert(1)</script><!-- note -->`
|
||||
got, report = seaglass.SanitizeReport(dirty)
|
||||
for _, bad := range []string{"onclick", "javascript", "<script", "alert", "<!--"} {
|
||||
if strings.Contains(got, bad) {
|
||||
t.Fatalf("%q survived: %q", bad, got)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(got, "Hi ") || !report.Changed {
|
||||
t.Fatalf("sanitized = %q %+v", got, report)
|
||||
}
|
||||
// A link whose only href is refused loses its tag (its text stays), so
|
||||
// it is reported as a removed a element.
|
||||
if !slices.Contains(report.Elements, "script") || !slices.Contains(report.Elements, "a") || !slices.Contains(report.Attributes, "onclick") || report.Comments != 1 {
|
||||
t.Fatalf("report = %+v", report)
|
||||
}
|
||||
if err := seaglass.Check(dirty); err == nil {
|
||||
t.Fatal("Check(dirty) = nil")
|
||||
}
|
||||
// Bypasses of regex gates are removed too.
|
||||
for _, payload := range []string{`<img src="x"onerror="alert(1)">`, `<a href="javascript:alert(1)">x</a>`, `<svg/onload=alert(1)>`, `<a href="https://a" href="javascript:alert(1)">x</a>`} {
|
||||
if err := seaglass.Check(payload); err == nil {
|
||||
t.Fatalf("payload passed: %s", payload)
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user