golang.org/x/net/html's tokenizer keeps only the first of repeated
attribute names, so FormatHTML's byte-preserving comparison saw
<a href="https://x" href="javascript:...">, <img src=ok
src="javascript:..."> and a repeated style as unchanged and stored the
raw bytes, and CheckHTML accepted them.
htmlTokens now scans each start tag's raw bytes for its attribute names,
following the tokenizer's own readTag rules, and reports the document as
changed when that list differs from the attributes the tokenizer kept.
Such input is stored as bluemonday's output, which keeps the first value
as browsers do, and CheckHTML refuses it. Input without repeated
attributes is still stored byte for byte. A differential run of 2M
random tags found no disagreement between the scan and the tokenizer.
A translated content_html sent through the Journal API (syncTranslations
formatted only when content_html was empty) or through
journal:import-posts (no check) was stored verbatim and served by the
anonymous GET /_journal/api/v1/posts and /posts/{slug}. A translation
keyed on the default locale also wrote the host content_html column
past FormatHTML.
classes.FormatTranslation now sets content_html for every translated
locale before it is stored: rendered from a non-blank content with
FormatHTML (host-column parity; a supplied content_html is replaced),
otherwise a supplied content_html is sanitized with the new
classes.SanitizeHTML, which FormatHTML reuses. Both write paths call it.
PHP parity note: Winter's PostApiController::syncTranslations called
Post::formatHtml only when content_html was empty and otherwise stored
the supplied content_html verbatim; ImportPosts::syncTranslations stored
it verbatim; the MLJournalMarkdown admin widget always derived it from
content via formatHtml. The Go port now follows the admin widget on
every path. Response shapes and status codes for valid input are
unchanged; only the stored content_html of unsafe or content-mismatched
translations differs.
- FormatHTML sanitizes the goldmark output with a bluemonday UGC policy
extended only with what goldmark and the imported posts and categories
use; unsafe markup is stripped, as Winter's Html::clean strips it
- HTML the policy keeps whole is stored byte for byte (token comparison)
- CheckHTML keeps its signature and 422 contract, refusing HTML the
policy would change
- the three regexes are gone: the CR-01 bypasses are neutralized and
"Data:" or "metadata:" text is no longer refused (WR-01)
- TestFormatHTMLKeepsRawHTML: imported-style bodies (paragraphs, attributes, https links and images, entities) round-trip; markdown keeps a trailing raw footer and style
- TestFormatHTMLRejectsUnsafeHTML: every unsafe case must return an error now that the gate is the only defence; adds svg onload, mixed-case schemes and tags, object, embed, script in an HTML block and raw data URLs
- TestPostsAdminResaveKeepsHTML: an admin update of HTML content keeps content_html
- FormatHTML renders with goldmark html.WithUnsafe(), as Winter's
Markdown::parse passes raw HTML through, so an admin save, an import or
an API write no longer turns imported HTML into raw HTML omitted
- rejectUnsafe still runs after the render and refuses script, iframe,
object and embed tags, event handlers and javascript, vbscript and data
URLs; API response shapes are unchanged
- TestFormatHTMLKeepsRawHTML covers HTML paragraphs and a raw footer
- add a content textarea (size huge) after description, with en and pl phrases
- add content to Category.Fillable
- export classes.CheckHTML and run it from FormBeforeCreate and FormBeforeUpdate:
unsafe category content answers 422 on content, blank content is stored as NULL
Named JOURNAL-001/002 fillable, FormatHTML unsafe tags, rollback/remigrate, RSS/categories/tags/media, and TestSearchGateOff with zero Typesense HTTP.
Co-authored-by: Cursor <cursoragent@cursor.com>
YAML-driven Posts screen stores translatable markdown and regenerates
content_html in the plugin. Publish writes without access_publish are
refused; list and form queries stay on the principal without
access_other_posts.
Co-authored-by: Cursor <cursoragent@cursor.com>
