golang.org/x/net/html's tokenizer keeps only the first of repeated
attribute names, so FormatHTML's byte-preserving comparison saw
<a href="https://x" href="javascript:...">, <img src=ok
src="javascript:..."> and a repeated style as unchanged and stored the
raw bytes, and CheckHTML accepted them.
htmlTokens now scans each start tag's raw bytes for its attribute names,
following the tokenizer's own readTag rules, and reports the document as
changed when that list differs from the attributes the tokenizer kept.
Such input is stored as bluemonday's output, which keeps the first value
as browsers do, and CheckHTML refuses it. Input without repeated
attributes is still stored byte for byte. A differential run of 2M
random tags found no disagreement between the scan and the tokenizer.
- FormatHTML sanitizes the goldmark output with a bluemonday UGC policy
extended only with what goldmark and the imported posts and categories
use; unsafe markup is stripped, as Winter's Html::clean strips it
- HTML the policy keeps whole is stored byte for byte (token comparison)
- CheckHTML keeps its signature and 422 contract, refusing HTML the
policy would change
- the three regexes are gone: the CR-01 bypasses are neutralized and
"Data:" or "metadata:" text is no longer refused (WR-01)
- TestFormatHTMLKeepsRawHTML: imported-style bodies (paragraphs, attributes, https links and images, entities) round-trip; markdown keeps a trailing raw footer and style
- TestFormatHTMLRejectsUnsafeHTML: every unsafe case must return an error now that the gate is the only defence; adds svg onload, mixed-case schemes and tags, object, embed, script in an HTML block and raw data URLs
- TestPostsAdminResaveKeepsHTML: an admin update of HTML content keeps content_html
- FormatHTML renders with goldmark html.WithUnsafe(), as Winter's
Markdown::parse passes raw HTML through, so an admin save, an import or
an API write no longer turns imported HTML into raw HTML omitted
- rejectUnsafe still runs after the render and refuses script, iframe,
object and embed tags, event handlers and javascript, vbscript and data
URLs; API response shapes are unchanged
- TestFormatHTMLKeepsRawHTML covers HTML paragraphs and a raw footer
Named JOURNAL-001/002 fillable, FormatHTML unsafe tags, rollback/remigrate, RSS/categories/tags/media, and TestSearchGateOff with zero Typesense HTTP.
Co-authored-by: Cursor <cursoragent@cursor.com>
