docs(quick-261114-p5m): fix CR-02 residual quoted end tags in the mail URL check
This commit is contained in:
@@ -5,10 +5,10 @@ current_phase: 35
|
||||
current_phase_name: Mail templates and layouts editor
|
||||
status: executing
|
||||
stopped_at: Completed 35-09-PLAN.md
|
||||
last_updated: "2026-11-14T15:08:54.080Z"
|
||||
last_updated: "2026-11-14T17:38:18.654Z"
|
||||
last_activity: 2026-11-14
|
||||
last_activity_desc: Phase 35 execution started
|
||||
state_head: dce002ee45b49833975a0ae9a5caad63c570107c
|
||||
last_activity_desc: "Completed quick task 261114-p5m: CR-02 residual fix"
|
||||
state_head: f1f284a0f93754c03b2f9f0f1a7c060228d9eb39
|
||||
progress:
|
||||
total_phases: 52
|
||||
completed_phases: 37
|
||||
@@ -32,7 +32,7 @@ See: .planning/PROJECT.md (updated 2026-10-22)
|
||||
Phase: 35 (Mail templates and layouts editor) — EXECUTING
|
||||
Plan: 9 of 9
|
||||
Status: Ready to execute
|
||||
Last activity: 2026-11-14 — Phase 35 execution started
|
||||
Last activity: 2026-11-14 - Completed quick task 261114-p5m: CR-02 residual fix
|
||||
|
||||
Progress: [████████░░] 82%
|
||||
|
||||
@@ -1267,6 +1267,7 @@ Recent decisions affecting current work:
|
||||
| 261106-kt1 | pier MCP 401 challenge names only resource_metadata, so clients request the advertised scopes (write included) | 2026-11-06 | 609caa4 | [261106-kt1-pier-mcp-401-challenge-drops-the-fixed-r](./quick/261106-kt1-pier-mcp-401-challenge-drops-the-fixed-r/) |
|
||||
| 261106-kt2 | deferred:purge, migrate* and queue:clear reuse the app's published database, so the in-server scheduler no longer fails with duplicate provider for *sql.DB | 2026-11-06 | 99abe88 | [261106-kt2-lagoon-deferred-purge-fails-in-the-sched](./quick/261106-kt2-lagoon-deferred-purge-fails-in-the-sched/) |
|
||||
| 261108-pe4 | beachcomber highlights and field projection: Query.IncludeFields/ExcludeFields/Highlight, SearchResult.Hits (plain-text segments), HighlightSearcher + SupportsHighlights; Typesense parses v1 highlights | 2026-11-08 | 7819c67 | [261108-pe4-beachcomber-highlights-and-field-project](./quick/261108-pe4-beachcomber-highlights-and-field-project/) |
|
||||
| 261114-p5m | Fix CR-02 residual: quoted end tags in raw-text elements bypass the postcard scheme check | 2026-11-14 | f1f284a0 | [261114-p5m-fix-cr-02-residual-quoted-end-tags-in-ra](./quick/261114-p5m-fix-cr-02-residual-quoted-end-tags-in-ra/) |
|
||||
|
||||
## Deferred Items
|
||||
|
||||
|
||||
@@ -0,0 +1,235 @@
|
||||
---
|
||||
phase: quick-261114-p5m
|
||||
plan: 01
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- modules/postcard/safety.go
|
||||
- modules/postcard/safety_test.go
|
||||
- modules/postcard/README.md
|
||||
- docs/services/mail.md
|
||||
- .planning/notes/2026-11-14-mail-url-scheme-check.md
|
||||
autonomous: true
|
||||
requirements:
|
||||
- QUICK-261114-p5m
|
||||
|
||||
estimate:
|
||||
tokens: 55000
|
||||
raw_tokens: 55000
|
||||
tasks: 3
|
||||
confidence: low
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "validateHTML refuses all eight verifier probe payloads in which a raw-text end tag or a comment's end sits inside a quoted attribute: noscript href, noscript img onerror, svg style, svg title, svg textarea, noembed, xmp and the Outlook conditional comment (D-13: javascript:/vbscript:/data: URLs and handlers refused). Each refusal carries the existing scheme or handler message and a snippet that points at the offending tag"
|
||||
- "Editor.Validate refuses a default layout draft carrying the svg/style, noscript href or noscript img/onerror end-tag-in-attribute payload on content_html with rule unsafe, with inlining on and with DisableAutoInlineCSS (D-13, save)"
|
||||
- "A send through a layout override that carries one of those three payloads falls back to the file version, logs 'override failed', and never hands the script link to the driver. This holds with DisableAutoInlineCSS and with inlining on (D-13, send)"
|
||||
- "The svg title and svg textarea payloads in a layout override are sent from the override as text that html/template escaped. x/net/html's parser, with scripting on and off, builds no element in the sent HTML that has an on* attribute or a script/data URL"
|
||||
- "Earlier behaviour is unchanged: every existing TestURLSchemeCheck accept and refuse case keeps its message and snippet, the eight-level depth limit still refuses, and the CR-01 and WR-01..03 tests (TestUnsafeText, TestSiteNameSend, TestUnsafeSiteNameDropped, TestMailSafetySmoke, TestInlineUnsafeOutput) pass unchanged. Conditional comments with quoted attributes, CSS with quoted strings, a safe noscript link with a quoted end tag and an attribute value that holds a literal comment opener are all accepted"
|
||||
- "go vet ./..., go test -short ./..., the Docker postcard and cabana tests and scripts/check-phase35.sh --all pass. go.mod and go.sum are unchanged"
|
||||
- "modules/postcard/README.md, docs/services/mail.md and .planning/notes/2026-11-14-mail-url-scheme-check.md say that the check also reads every tag with attribute quoting deciding where the tag ends, so an end tag inside a quoted attribute does not hide the rest of the tag"
|
||||
artifacts:
|
||||
- path: modules/postcard/safety.go
|
||||
provides: "unsafeMarkupAt runs the existing HTML reading and then a new tag reading (unsafeTagReading). The tag reading uses NextIsNotRawText after raw-text start tags and blanks comment openers. Both readings share one attribute check helper (unsafeAttrs)"
|
||||
- path: modules/postcard/safety_test.go
|
||||
provides: "TestURLSchemeCheck with the eight *_end_in_attr refuse cases, three *_end_in_attr send/editor links, five accept cases and escaped_send subtests checked with an x/net/html parser oracle"
|
||||
- path: modules/postcard/README.md
|
||||
provides: "Safety checks bullet describes the quote-aware second reading"
|
||||
- path: docs/services/mail.md
|
||||
provides: "Safety paragraph describes the quote-aware second reading"
|
||||
- path: .planning/notes/2026-11-14-mail-url-scheme-check.md
|
||||
provides: "Decision bullet, residual risk and amendment for the CR-02 residual"
|
||||
key_links:
|
||||
- from: modules/postcard/templates.go validateHTML (renderParts body check, layout HTML check before inlining, inlined result check; partials.go; editor.go content_css)
|
||||
to: modules/postcard/safety.go unsafeMarkup -> unsafeMarkupAt
|
||||
via: "every rendered-HTML check goes through unsafeMarkupAt, so the tag reading applies at save (Editor.Validate) and at send, whatever the opt-out"
|
||||
- from: modules/postcard/safety.go unsafeMarkupAt
|
||||
to: unsafeHTMLReading (existing loop, recursion into raw-text and comment content) then unsafeTagReading
|
||||
via: "HTML reading first, so existing refusals keep their message and snippet; both readings at every depth below maxMarkupDepth"
|
||||
- from: modules/postcard/safety.go unsafeTagReading
|
||||
to: golang.org/x/net/html Tokenizer.NextIsNotRawText
|
||||
via: "called after each start or self-closing tag named in rawTextElements, before the next Next(); without it the tokenizer ends the element at an end tag that sits inside a quoted attribute"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Close the CR-02 residual, which is the remaining gap in 35-VERIFICATION.md for truth 18 (D-13). `unsafeMarkupAt` in `modules/postcard/safety.go` re-checks a raw-text element's content only up to where x/net/html's tokenizer ends that element. That end is the first `</style`, `</noscript`, `</title`, `</textarea`, `</xmp` or `</noembed`, and for a comment the first `-->`, even when that end tag sits inside a quoted attribute. In `<svg><style><a title="</style>" href="javascript:alert(1)">x</a></style></svg>` the rest of the tag, ` href="javascript:alert(1)">`, reaches the outer loop as text and is never checked. A conforming parser keeps the attribute together and builds a live `svg:a` link. With scripting off, the noscript variants become a live link and a live `onerror` handler. `Editor.Validate` accepts such a layout, and a layout with `DisableAutoInlineCSS` sends it intact.
|
||||
|
||||
Fix: in addition to the existing HTML reading, read the document a second time with attribute quoting alone deciding where each tag ends. This tag reading uses the x/net/html tokenizer, calls `NextIsNotRawText()` after every raw-text start tag and does not recognise comment openers. It applies the same attribute checks to every start tag. The planner checked this approach against the unmodified tree with a `go test -overlay` prototype, without changing any repository file. It refused all eight probe payloads (`zz_verifier_probe2_test.go`, `endTagInAttr`), noscript and svg/style fell back with inlining on and off, `Editor.Validate` refused those three, and the short postcard and cabana suites stayed green.
|
||||
|
||||
Purpose: D-13 promises that script and data URLs are refused at save and at send, whether or not the layout opts out of inlining. The README and docs/services/mail.md promise that markup inside these elements is checked. Both promises must hold for this payload class too.
|
||||
Output: the fix with a smoke test and docs (code commit), the decision note amendment (docs commit), and the regression tests with the full gate run (test commit). No new dependency, and no exported API change.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@~/.claude/msd-core/workflows/execute-plan.md
|
||||
@~/.claude/msd-core/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/STATE.md
|
||||
@CLAUDE.md
|
||||
@.planning/phases/35-mail-templates-and-layouts-editor/35-VERIFICATION.md
|
||||
@.planning/phases/35-mail-templates-and-layouts-editor/35-REVIEW-FIX.md
|
||||
@.planning/notes/2026-11-14-mail-url-scheme-check.md
|
||||
@modules/postcard/safety.go
|
||||
@modules/postcard/safety_test.go
|
||||
|
||||
Interfaces the executor uses (all exist at planning time; read the files rather than exploring):
|
||||
|
||||
- safety.go: `unsafeMarkup(doc string) error` calls `unsafeMarkupAt(doc, 0)`; `unsafeMarkupAt(doc string, depth int) error` holds the depth guard (`maxMarkupDepth = 8`, message "rendered HTML nests raw-text elements or comments too deeply") and the tokenizer loop with offsets from `len(z.Raw())`. The attribute switch is at lines 184-198: `on*` keys give "rendered HTML contains event handlers"; `urlAttributes` + `hasDangerousScheme`, `urlListAttributes` / `style` / meta `content` + `containsDangerousScheme` give "rendered HTML contains a dangerous URL scheme"; the snippet is `snippetAt(doc, loc)`. `rawTextElements` lists iframe, noembed, noframes, noscript, plaintext, script, style, textarea, title and xmp. The type `unsafeError{msg, snippet string}` and `snippetAt(s string, loc []int) string` live in templates.go, and `validateHTML` at templates.go:826 runs the `rawUnsafeTag` regex and then `unsafeMarkup`.
|
||||
- x/net/html v0.62.0 (go.mod; already required): `(*Tokenizer).NextIsNotRawText()` clears the tokenizer's raw tag. It must be called after `Next()` has returned the start tag and before the next `Next()`. The tokenizer also sets the raw tag after a self-closing raw-text start tag (`<style/>`). `xhtml.ParseWithOptions(r, xhtml.ParseOptionEnableScripting(bool))` serves as the test oracle.
|
||||
- safety_test.go `TestURLSchemeCheck`: the `accepts` slice is `{name, html}`, wrapped as `"<div>" + schemeFiller + html + schemeFiller + "</div>"`. The `refuses` slice is `{name, html, marker, msg}`; the snippet must contain `marker` and must not start with `<div>`. `links` is `{name, link}`. Each link gets a send subtest `send/<name>/inline` and `send/<name>/opt_out` through `assertFileFallback(t, cat, file, Options{Overrides: fakeOverrides{ov: Overrides{...}}}, true)` with a layout override named `encoded`. The `editor/inline` and `editor/opt_out` subtests loop over `links[1:]` with `newEditor(t, nil)` and `ed.Validate(ctx, Draft{Kind: KindLayout, Name: defaultLayoutName, ...})`, expecting `de.Field == fieldContentHTML && de.Rule == ruleUnsafe`. Helpers for a direct send: `NewMemoryDriver()`, `NewMailer(cat, drv, opts)`, `sendWelcome(t, m, "")`, `lastMessage(t, drv)`, `fileWelcome(t)`.
|
||||
- The verifier's probe, for evidence only (never copy it into the repo): /tmp/claude-1000/-media-nvme-dev-golem15-summercms-io-summercms-summercms-go/0c86abd1-8e0c-485b-8ad1-fd321a8a0fda/scratchpad/probe/overlay2.json overlays zz_verifier_probe_test.go and zz_verifier_probe2_test.go onto modules/postcard.
|
||||
</context>
|
||||
|
||||
<source_audit>
|
||||
| Source | Item | Covered by |
|
||||
|--------|------|-----------|
|
||||
| GOAL (task description) | refuse every payload in the verifier probes | Task 1 (svg/style), Task 3 (all eight) |
|
||||
| GOAL | add the probe payloads to TestURLSchemeCheck, plus an Editor.Validate refusal and a fallback send through a DisableAutoInlineCSS layout | Task 1 smoke, Task 3 |
|
||||
| GOAL | keep CR-01, WR-01..03 and the depth limit intact | Task 1 (HTML reading unchanged and run first), Task 3 (named suites, mutation check, gates) |
|
||||
| GOAL | update README, docs/services/mail.md and the decision note where wording changes | Task 1 (README, docs), Task 2 (note) |
|
||||
| GOAL | closes 35-VERIFICATION.md gap truth 18 | must_haves; the re-verification itself is the verifier's job (output) |
|
||||
| REQ | QUICK-261114-p5m | this plan |
|
||||
| CONTEXT | D-13 (phase 35, restated in the decision note): unsafe URLs refused at save and at send, inlining on or off | Tasks 1 and 3 (editor and send subtests on both inline modes) |
|
||||
| RESEARCH | none (quick task, no research phase); the verifier's advisory on the conditional comment with `-->` in an attribute | Task 3 `conditional_comment_end_in_attr` |
|
||||
</source_audit>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="tracer">
|
||||
<name>Task 1: End to end, one payload: svg/style with an end tag inside an attribute is refused at check, save and send</name>
|
||||
<files>modules/postcard/safety.go, modules/postcard/safety_test.go, modules/postcard/README.md, docs/services/mail.md</files>
|
||||
<action>
|
||||
RED first. In `TestURLSchemeCheck`, add one `refuses` case after the CR-02 group, named `svg_style_end_in_attr`. Its html is `<svg><style><a title="</style>" href="javascript:alert(1)">x</a></style></svg>`, its marker is `href="javascript:alert(1)"` and its message is `schemeMsg`. Also add the same markup to `links`, named `svg_style_end_in_attr`. The existing send loop and the editor's `links[1:]` loop then cover it as `send/svg_style_end_in_attr/inline`, `send/svg_style_end_in_attr/opt_out` and both editor subtests. Above the new entries, add a comment that names the CR-02 residual: an end tag inside a quoted attribute. Run `go test -short -count=1 ./modules/postcard -run TestURLSchemeCheck` and keep the failing subtest names for the SUMMARY: refuse, both sends and both editor subtests fail on the unfixed code.
|
||||
|
||||
GREEN, in modules/postcard/safety.go (per D-13). Keep every existing behaviour and message:
|
||||
(a) Move the attribute checks of the start-tag branch (the `for _, a := range t.Attr` switch, including the namespace prefix for `xlink:href`) into a helper `unsafeAttrs(t xhtml.Token, doc string, loc []int) error`. Keep the same two messages and the same `snippetAt(doc, loc)` snippet. The existing loop calls it.
|
||||
(b) Move the existing tokenizer loop into `unsafeHTMLReading(doc string, depth int) error`. Its behaviour stays the same: the raw-text and comment recursion still calls `unsafeMarkupAt(..., depth+1)` and the style-text and comment scheme checks stay. `unsafeMarkupAt` keeps the depth guard first, then returns the HTML reading's error if there is one, otherwise the tag reading's. The HTML reading runs first, so every refusal that exists today keeps its message and snippet; the tag reading only adds refusals. Both readings run at every depth, so content re-read as markup is also read with quoting deciding where tags end.
|
||||
(c) Add `unsafeTagReading(doc string) error`. It tokenizes a copy of doc in which the `<` of every `<!--` is replaced by one space (`strings.ReplaceAll(doc, "<!--", " !--")`). The copy has the same length, so the offsets accumulated from `len(z.Raw())` index doc, and snippets come from doc. Blank only the `<`: an attribute value that holds a literal comment opener must not become a scheme prefix once `stripURLSpace` removes the spaces (the planner prototype checked that `<a href="<!--javascript:x">` stays accepted). For each start or self-closing tag: take `z.Token()`. If `t.Data` is in `rawTextElements`, call `z.NextIsNotRawText()` before the next `z.Next()`. Then return `unsafeAttrs(t, doc, loc)` if it fails. Ignore text, end tags and comments, and return nil at `xhtml.ErrorToken`. This reading lets attribute quotes alone decide where a tag ends, so an end tag or `-->` inside a quoted attribute does not split the tag.
|
||||
(d) Rejected alternatives, recorded in the doc comment in one line each:
|
||||
- Re-checking from the raw-text content start to the end of the document. Inside the recursion this re-reads the rest of the document once for each raw-text element and comment, which is quadratic or worse on hostile input.
|
||||
- Using the x/net/html parser tree as the check. It loses the raw snippet that the editor shows, and it reads xmp and noembed as raw text and ends a conditional comment at the first `-->`, while CR-02 deliberately treats that content as markup.
|
||||
(e) Doc comments. Extend the `unsafeMarkup` comment with a paragraph on the second reading: why it exists, that it uses quoting only, that it skips comment and raw-text boundaries, and that it runs after the HTML reading at every depth. Also say that each reading is linear and that the unchanged depth limit bounds the recursion. Extend the `rawTextElements` comment to say that the tag reading reads their content as markup with `NextIsNotRawText`.
|
||||
|
||||
Docs, in the same commit, as the CR-02 fix (1d481bb0) did:
|
||||
- modules/postcard/README.md, line 30 (the Safety checks bullet). After the sentence that ends "such elements nested more than eight deep are refused.", add one sentence: every tag is also read with attribute quoting alone deciding where it ends, so an end tag or `-->` inside a quoted attribute (`<a title="</style>" href="javascript:...">`) does not hide the rest of the tag.
|
||||
- docs/services/mail.md, line 158. After "Markup inside `noscript`, `title`, `textarea`, `style`, `xmp` and similar elements, and inside comments, is checked as well, since a mail client may read it as markup.", add the same idea in one sentence.
|
||||
- In both files, backtick only lowercase HTML. The docs checker reads backticked names as package identifiers, so do not backtick x/net/html method names there.
|
||||
|
||||
Commit `fix(quick-261114-p5m): read tags across raw-text and comment ends in the mail URL check`. Stage the four files by path and use no co-author tag. Leave other sessions' uncommitted files alone (G15OFFICE.md, 35-UAT.md, deferred-items.md, state.json, examples/hello/plugins.gen.go).
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./modules/postcard && go test -short -count=1 ./modules/postcard -run TestURLSchemeCheck && go test -count=1 ./cmd/summer -run 'TestDocsTree|TestCommandsDocumented' && go run ./cmd/summer docs:build --check && test "$(go test -short -count=1 -v ./modules/postcard -run 'TestURLSchemeCheck' | grep -cE -- '--- PASS: TestURLSchemeCheck/(refuse|send)/svg_style_end_in_attr')" -ge 3</automated>
|
||||
</verify>
|
||||
<done>The svg/style payload is refused by validateHTML (scheme message, snippet at the `<a title=...>` tag). It is refused by Editor.Validate on content_html/unsafe with inlining on and off, and a send through an `encoded` layout override carrying it falls back to the file version on both inline modes. Every existing TestURLSchemeCheck subtest passes unchanged, and the RED subtest names are recorded. The README and docs sentences are added, the docs checker passes, and one fix commit is made.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Amend the decision note for the CR-02 residual</name>
|
||||
<files>.planning/notes/2026-11-14-mail-url-scheme-check.md</files>
|
||||
<action>
|
||||
Amend the note, which restates D-13 for postcard, in place. Do not rewrite sections that are unaffected.
|
||||
(1) The **Status:** line: after the existing amendment clause, add "; amended again by quick task 261114-p5m (CR-02 residual: an end tag inside a quoted attribute)".
|
||||
(2) The Decision bullet "**Raw-text elements and comments are checked as markup too** (CR-02)": append the second reading. After the HTML reading, `unsafeMarkup` reads the same text once more as tags only. It uses the x/net/html tokenizer with `NextIsNotRawText()` after every raw-text start tag, and the `<` of each `<!--` blanked to a space (same length, so snippets point into the original). Attribute quoting alone decides where a tag ends, so `<svg><style><a title="</style>" href="javascript:...">` and `<!--[if mso]><a title="-->" href="javascript:...">` are refused. The same attribute checks and messages apply. Both readings run at every depth, both are linear, and the depth limit is unchanged.
|
||||
(3) "Residual risk accepted": add one bullet. The tag reading treats comment and raw-text content as markup with quotes honoured. An unbalanced quote inside a comment or raw-text element can therefore join later text into one tag, and text after it that looks like an attribute (`online=yes` after a stray quote) is refused although no reader sees a tag there. No mail markup leaves a quote open in a comment or a style element.
|
||||
(4) "Rejected alternatives": add one bullet for re-checking from the content start to the end of the document (quadratic or worse inside the recursion) and one for using the parser tree as the check (loses raw snippets; reads xmp and noembed as raw text and ends conditional comments at the first `-->`).
|
||||
(5) "Amendment": add a bullet. **CR-02 residual** (35-VERIFICATION.md truth 18, quick 261114-p5m): the re-check read raw-text content only up to the tokenizer's end tag, which an attribute value can contain. It is fixed by the tag reading above, and the verifier's conditional-comment advisory is covered by the same reading.
|
||||
Commit separately: `docs(quick-261114-p5m): amend the mail URL scheme note for quoted end tags`. Stage only this file and use no co-author tag.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && N=.planning/notes/2026-11-14-mail-url-scheme-check.md && grep -q '261114-p5m' "$N" && grep -q 'NextIsNotRawText' "$N" && grep -q 'CR-02 residual' "$N" && S="$(git log -1 --format=%s -- "$N")" && case "$S" in *quick-261114-p5m*) true ;; *) false ;; esac</automated>
|
||||
</verify>
|
||||
<done>The note's status, decision bullet, residual risk, rejected alternatives and amendment describe the tag reading. The note is committed alone in a docs commit.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 3: Regression tests for every probe payload, a parser oracle for escaped sends, and the full gates (tests last)</name>
|
||||
<files>modules/postcard/safety_test.go</files>
|
||||
<action>
|
||||
All changes go in `TestURLSchemeCheck`; the test name is already required by `GO_TESTS` in scripts/check-phase35.sh, so the gate needs no edit. Update the test's doc comment to mention end tags inside quoted attributes.
|
||||
(1) `refuses`: add the seven remaining probe payloads next to `svg_style_end_in_attr`.
|
||||
- `noscript_href_end_in_attr`: `<noscript><a title="</noscript>" href="javascript:alert(1)">x</a></noscript>`, marker `href="javascript:alert(1)"`, schemeMsg
|
||||
- `noscript_handler_end_in_attr`: `<noscript><img title="</noscript>" src=x onerror=alert(1)></noscript>`, marker `onerror=alert(1)`, "event handlers"
|
||||
- `svg_title_end_in_attr`: `<svg><title><a title="</title>" href="javascript:alert(1)">x</a></title></svg>`, marker `href="javascript:alert(1)"`, schemeMsg
|
||||
- `svg_textarea_end_in_attr`: `<svg><textarea><a title="</textarea>" href="javascript:alert(1)">x</a></textarea></svg>`, marker `href="javascript:alert(1)"`, schemeMsg
|
||||
- `noembed_end_in_attr`: `<noembed><a title="</noembed>" href="javascript:alert(1)">x</a></noembed>`, marker `href="javascript:alert(1)"`, schemeMsg
|
||||
- `xmp_end_in_attr`: `<xmp><a title="</xmp>" href="javascript:alert(1)">x</a></xmp>`, marker `href="javascript:alert(1)"`, schemeMsg
|
||||
- `conditional_comment_end_in_attr`: `<!--[if mso]><a title="-->" href="javascript:alert(1)">x</a><![endif]-->`, marker `href="javascript:alert(1)"`, schemeMsg (the verifier's advisory)
|
||||
|
||||
(2) `links`: add `noscript_href_end_in_attr` and `noscript_handler_end_in_attr` with the same markup as their refuse cases. `svg_style_end_in_attr` is already there from Task 1. Each one then gets `send/<name>/inline` and `send/<name>/opt_out`, which must fall back to the file version with the override-failed log, and the editor loops refuse each on content_html/unsafe. Update the `links` comment: html/template escapes literal title and textarea text, so the svg title and textarea variants are checked by (4), not as fallbacks.
|
||||
|
||||
(3) `accepts`: add five cases, so that the tag reading is shown to refuse nothing that mail uses.
|
||||
- `conditional_comment_quoted_attrs`: `<!--[if mso]><table role="presentation" width="600"><tr><td style="padding:0"><![endif]-->`
|
||||
- `conditional_comment_not_mso_safe_link`: `<!--[if !mso]><!--><a href="https://example.com/">x</a><!--<![endif]-->`
|
||||
- `style_css_quoted_strings`: `<style>td[title="a > b"] { font-family: "Helvetica Neue", Arial; }</style>`
|
||||
- `title_escaped_end_in_attr`: `<svg><title><a title="</title>" href="javascript:x"></title></svg>`
|
||||
- `noscript_safe_link_end_in_attr`: `<noscript><a title="</noscript>" href="https://example.com/">Open</a></noscript>`
|
||||
Also add `href_literal_comment_opener` with `<a href="<!--javascript:x">x</a>`, which pins the single-byte blanking.
|
||||
|
||||
(4) Add subtests `escaped_send/svg_title_end_in_attr/inline|opt_out` and `escaped_send/svg_textarea_end_in_attr/inline|opt_out`. Build the mailer as `assertFileFallback` does: `NewMailer(cat, drv, Options{Overrides: fakeOverrides{...}})` with the same `encoded` layout override shape and `DisableAutoInlineCSS` per mode, then `sendWelcome(t, m, "")` and `lastMessage(t, drv)`. Assert three things:
|
||||
- `got.HTML != file.HTML`: the message is sent from the override and not refused.
|
||||
- `got.HTML` contains `<a title=`. html/template escaped the `<`; inlining on re-serializes the quote as `"`, so do not assert a longer string.
|
||||
- A small test helper `liveUnsafeElements(n *xhtml.Node) []string` returns nothing for the parse with `xhtml.ParseWithOptions(strings.NewReader(got.HTML), xhtml.ParseOptionEnableScripting(scripting))`, for scripting true and false. The helper walks the tree and lists every element attribute whose key (with the namespace prefix, as unsafeMarkup builds it) starts with "on", or is in `urlAttributes` with a value for which `hasDangerousScheme` is true. Import `golang.org/x/net/html` as `xhtml` in the test file.
|
||||
This replaces the probe's substring `liveJS` reading, which was a false positive, with what a conforming parser builds.
|
||||
|
||||
(5) Mutation check, not committed. Temporarily make `unsafeMarkupAt` skip the tag reading, then run `go test -short -count=1 ./modules/postcard -run TestURLSchemeCheck`. Record in the SUMMARY that the new `*_end_in_attr` refuse and send cases fail. Restore the file, and confirm `git diff --quiet -- modules/postcard/safety.go` before you go on.
|
||||
|
||||
(6) Full gates. Run them in this order:
|
||||
- `go vet ./...`
|
||||
- `go test -short ./...`
|
||||
- `cd examples/hello && go test ./...`
|
||||
- `go test -count=1 ./modules/postcard ./modules/cabana` (the Docker suites; if the Docker socket is denied, run the tests through `sudo -u golem` as CLAUDE.md describes)
|
||||
- `scripts/check-phase35.sh --all`
|
||||
Then `git diff --quiet -- go.mod go.sum`. If the verifier's overlay file at the scratchpad path in context exists, also run `go test -short -count=1 -overlay <that path> ./modules/postcard -run TestZZV2 -v` and record that all eight payloads are refused and that the noscript and svg/style sends fall back. The overlay is virtual, so no file is written into the repository.
|
||||
Commit `test(quick-261114-p5m): pin end tags inside quoted attributes in the mail URL check`. Stage only safety_test.go and use no co-author tag.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /media/nvme/dev/golem15/summercms.io/summercms/summercms.go && go vet ./... && go test -short ./... >/dev/null && test "$(go test -short -count=1 -v ./modules/postcard -run 'TestURLSchemeCheck' | grep -cE -- '--- PASS: TestURLSchemeCheck/.*end_in_attr')" -ge 20 && go test -short -count=1 ./modules/postcard -run 'TestUnsafeText|TestSiteNameSend|TestUnsafeSiteNameDropped|TestMailSafetySmoke|TestInlineUnsafeOutput' && git diff --quiet -- go.mod go.sum modules/postcard/safety.go && scripts/check-phase35.sh --all</automated>
|
||||
</verify>
|
||||
<done>There are at least 20 passing `*_end_in_attr` subtests: 8 refuse, 6 send, 4 escaped_send and 2 accept. The other new accept cases pass, and the mutation check showed the new cases fail without the tag reading. The CR-01 and WR suites pass unchanged. go vet, the short suite, examples/hello, the Docker postcard and cabana tests and check-phase35.sh --all are green, go.mod and go.sum are unchanged, and one test commit is made.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| admin layout/template author -> rendered mail | A user with the layouts permission writes layout HTML that postcard renders and checks at save (Editor.Validate) and at send. The check is the only gate before a driver hands the HTML to a mail client |
|
||||
| rendered HTML -> mail client parser | The client may read raw-text and comment content as markup (scripting off, svg foreign content, Outlook conditional comments), and attribute quoting decides where its tags end |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|----------|-------------|-----------------|
|
||||
| T-261114-p5m-01 | Tampering | safety.go unsafeMarkupAt: a script URL or handler hidden behind a raw-text end tag or `-->` inside a quoted attribute, sent intact by a layout with DisableAutoInlineCSS (CR-02 residual) | high | mitigate | unsafeTagReading reads every tag with NextIsNotRawText after raw-text start tags and comment openers blanked, using the same attribute checks. The eight probe payloads are refused by validateHTML, the three that survive rendering by Editor.Validate and by send fallback on both inline modes (Task 3 subtests and mutation check) |
|
||||
| T-261114-p5m-02 | Denial of Service | over-refusal: the extra reading refuses legitimate mail markup, so saves fail or sends fall back | medium | mitigate | The HTML reading runs first and the tag reading only checks attributes. Accept cases pin conditional comments with quoted attributes, the not-mso pattern, quoted CSS strings, a safe quoted noscript link and an attribute that holds `<!--`. The full short and Docker suites and the consumers stage of check-phase35.sh (sm-user-plugin and sm-paymentgateway-plugin layouts) stay green |
|
||||
| T-261114-p5m-03 | Denial of Service | CPU on hostile layout HTML (many comments or deeply nested raw-text elements) | low | mitigate | Each reading is one linear tokenizer pass, the depth limit (maxMarkupDepth = 8) is unchanged, and the rest-of-document re-check is rejected because it is quadratic or worse |
|
||||
| T-261114-p5m-04 | Repudiation | new cases that do not run while the gate passes | low | mitigate | All new cases are subtests of TestURLSchemeCheck, which GO_TESTS requires, and the verify step counts at least 20 passing `*_end_in_attr` subtests |
|
||||
| T-261114-p5m-SC | Tampering | npm/pip/cargo installs | low | accept | No package installs and no new Go module; `git diff --quiet -- go.mod go.sum` is part of Task 3's verify |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- `go vet ./...` and `go test -short ./...` are green after each of the three commits; `examples/hello` `go test ./...` is green.
|
||||
- Docker suites: `go test -count=1 ./modules/postcard ./modules/cabana` is green (through `sudo -u golem` if the socket is denied).
|
||||
- `scripts/check-phase35.sh --all` passes (go, docker gate, consumers, admin, docs, marker, additive).
|
||||
- `go.mod` and `go.sum` are unchanged, and no exported identifier is added (the additive stage proves it).
|
||||
- If the verifier's probe overlay exists, re-running it (TestZZV2Validate, TestZZV2LayoutSend) shows all eight payloads refused, and noscript and svg/style falling back with inlining on and off.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- All eight verifier probe payloads are refused by validateHTML. The three payloads that survive html/template rendering are refused by Editor.Validate and fall back at send on both inline modes. The svg title and textarea variants are sent only as escaped text that has no live element under a conforming parser.
|
||||
- No earlier behaviour regresses (CR-01, WR-01..03 and the depth limit), and every existing TestURLSchemeCheck case keeps its message and snippet.
|
||||
- The README, docs/services/mail.md and the decision note describe the quote-aware reading. There are three commits: fix, docs (note) and test.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/quick/261114-p5m-fix-cr-02-residual-quoted-end-tags-in-ra/261114-p5m-SUMMARY.md` when done, with:
|
||||
- the RED subtest names from Task 1 and the mutation-check result from Task 3
|
||||
- the three commit hashes
|
||||
- the gate results (short, Docker, check-phase35.sh --all) and the probe re-run result, if it was run
|
||||
Do not edit 35-VERIFICATION.md or 35-REVIEW-FIX.md. Re-verifying phase 35 truth 18 is the next step, and the verifier does it. Do not tag or push, and do not raw-edit STATE.md (the quick workflow's state handlers do that).
|
||||
</output>
|
||||
@@ -0,0 +1,82 @@
|
||||
---
|
||||
phase: quick-261114-p5m
|
||||
plan: 01
|
||||
status: complete
|
||||
subsystem: postcard (mail safety check)
|
||||
tags: [security, mail, postcard, CR-02, D-13]
|
||||
requires: [phase 35 CR-02 fix (1d481bb0)]
|
||||
provides: [quote-aware tag reading in unsafeMarkup]
|
||||
affects: [modules/postcard validateHTML at save (Editor.Validate) and at send]
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns: ["second tokenizer pass with NextIsNotRawText and blanked comment openers"]
|
||||
key-files:
|
||||
created: []
|
||||
modified:
|
||||
- modules/postcard/safety.go
|
||||
- modules/postcard/safety_test.go
|
||||
- modules/postcard/README.md
|
||||
- docs/services/mail.md
|
||||
- .planning/notes/2026-11-14-mail-url-scheme-check.md
|
||||
decisions:
|
||||
- "unsafeMarkupAt runs the existing HTML reading first, then a tag reading in which attribute quoting alone decides where a tag ends (NextIsNotRawText after raw-text start tags, the < of each <!-- blanked to a space); both share unsafeAttrs"
|
||||
- "Rejected: re-checking from content start to end of document (quadratic inside the recursion) and using the parse tree as the check (loses raw snippets, reads xmp/noembed as raw text)"
|
||||
metrics:
|
||||
duration: ~36min
|
||||
completed: 2026-11-14
|
||||
actuals:
|
||||
tokens: 10200
|
||||
tasks: 3
|
||||
commits: 3
|
||||
plan_head_before: d8137c17d415587b50122b5ee529bc080f08ceec
|
||||
plan_head_after: f1f284a0f93754c03b2f9f0f1a7c060228d9eb39
|
||||
---
|
||||
|
||||
# Quick 261114-p5m: CR-02 residual (end tags inside quoted attributes) Summary
|
||||
|
||||
The mail URL check now reads rendered HTML a second time as tags only, with attribute quoting deciding where each tag ends. A `</style>`, `</noscript>`, `</title>` (and the like) or a `-->` inside a quoted attribute therefore no longer hides a script URL or handler later in the same tag. All eight verifier probe payloads are refused at check, save and send.
|
||||
|
||||
## What changed
|
||||
|
||||
- `modules/postcard/safety.go`: `unsafeMarkupAt` keeps the depth guard, then runs `unsafeHTMLReading` (the old loop, behaviour unchanged, still recursing through `unsafeMarkupAt`), then the new `unsafeTagReading`. The new pass tokenizes a same-length copy in which every `<!--` becomes ` !--` and calls `NextIsNotRawText()` after each raw-text start tag. It checks each start tag with `unsafeAttrs`, the attribute switch both readings now share. Doc comments describe the second reading and the rejected alternatives. No exported API change; go.mod and go.sum are unchanged.
|
||||
- `modules/postcard/README.md` and `docs/services/mail.md` each gained one sentence on the quote-aware reading.
|
||||
- The decision note has an updated status line, decision bullet, residual risk (an unbalanced quote in a comment or raw-text element can over-refuse), two new rejected alternatives and an amendment bullet.
|
||||
- `TestURLSchemeCheck` gained:
|
||||
- 8 `*_end_in_attr` refuse cases
|
||||
- 3 links (svg_style, noscript_href, noscript_handler), each sent with inlining on and off and refused by both editor loops
|
||||
- 6 accept cases
|
||||
- 4 `escaped_send` subtests, checked with an x/net/html parser oracle (`liveUnsafeElements`) with scripting on and off
|
||||
|
||||
## Evidence
|
||||
|
||||
- **RED (Task 1, on the unfixed code):** `refuse/svg_style_end_in_attr`, `send/svg_style_end_in_attr/inline`, `send/svg_style_end_in_attr/opt_out`, `editor/inline` and `editor/opt_out` failed.
|
||||
- **Mutation check (Task 3):** with `unsafeTagReading` disabled, all 8 `refuse/*_end_in_attr` cases failed, as did all 6 `send/*_end_in_attr/{inline,opt_out}` cases and `editor/inline` and `editor/opt_out`. The accept and escaped_send cases still passed. safety.go was restored and `git diff --quiet -- modules/postcard/safety.go` held.
|
||||
- **Passing `*_end_in_attr` subtests:** 20 (8 refuse, 6 send, 4 escaped_send, 2 accept).
|
||||
- **Gates:**
|
||||
- `go vet ./...` and `go test -short ./...` pass, as do `examples/hello` `go test ./...` and the CR-01/WR suites (TestUnsafeText, TestSiteNameSend, TestUnsafeSiteNameDropped, TestMailSafetySmoke, TestInlineUnsafeOutput).
|
||||
- Docker: `go test -count=1 ./modules/postcard ./modules/cabana` passes with Docker available and 0 skips.
|
||||
- `scripts/check-phase35.sh --all` exits 0. Every stage passes: self-test, go, consumers, admin, docs, marker, additive (postcard +122 lines, additive-only) and docker (full `go test ./...`).
|
||||
- `git diff --quiet -- go.mod go.sum` holds.
|
||||
- **Verifier probe re-run** (`-overlay .../probe/overlay2.json`, virtual, no file written):
|
||||
- TestZZV2Validate: all eight payloads are refused.
|
||||
- TestZZV2LayoutSend: noscript href, noscript img/onerror and svg/style fall back with inlining on and off, and Editor.Validate refuses those three.
|
||||
- The probe's `liveJS=true` on the opt_out svg title/textarea lines is its substring false positive. html/template escapes only up to the closing tag, and in svg foreign content the parser builds no link from the text that follows. The escaped_send parser oracle confirms this.
|
||||
|
||||
## Commits
|
||||
|
||||
- d6ac0aeb: fix(quick-261114-p5m): read tags across raw-text and comment ends in the mail URL check
|
||||
- e47d4ae6: docs(quick-261114-p5m): amend the mail URL scheme note for quoted end tags
|
||||
- f1f284a0: test(quick-261114-p5m): pin end tags inside quoted attributes in the mail URL check
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None. The plan was executed as written. On the first mutation attempt (`return nil; _ = unsafeTagReading`) the package did not build. A second attempt with `if false { ... }` gave the result recorded above. Nothing from the first attempt was committed, and the file was restored both times.
|
||||
|
||||
## Threat Flags
|
||||
|
||||
None. No new endpoint, auth path or schema; the change only adds refusals inside the existing check.
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- FOUND: modules/postcard/safety.go, modules/postcard/safety_test.go, modules/postcard/README.md, docs/services/mail.md, .planning/notes/2026-11-14-mail-url-scheme-check.md
|
||||
- FOUND commits: d6ac0aeb, e47d4ae6, f1f284a0
|
||||
Reference in New Issue
Block a user