From 2a7e9761a45ad2c33422a51506f35a844e4a9227 Mon Sep 17 00:00:00 2001 From: Jakub Zych Date: Sat, 10 Oct 2026 12:13:26 +0200 Subject: [PATCH] docs(16.2-03): complete Journal Blog Editor integration plan --- .../16.2-03-SUMMARY.md | 272 ++++++++++++++++++ 1 file changed, 272 insertions(+) create mode 100644 .planning/phases/16.2-plugin-provided-field-types-and-the-blog-editor-formwidget/16.2-03-SUMMARY.md diff --git a/.planning/phases/16.2-plugin-provided-field-types-and-the-blog-editor-formwidget/16.2-03-SUMMARY.md b/.planning/phases/16.2-plugin-provided-field-types-and-the-blog-editor-formwidget/16.2-03-SUMMARY.md new file mode 100644 index 0000000..49aad31 --- /dev/null +++ b/.planning/phases/16.2-plugin-provided-field-types-and-the-blog-editor-formwidget/16.2-03-SUMMARY.md @@ -0,0 +1,272 @@ +--- +phase: 16.2-plugin-provided-field-types-and-the-blog-editor-formwidget +plan: 03 +subsystem: journal-admin +tags: [sm-journal-plugin, field-types, blog-editor, bluemonday, sanitizing, uploads, cabana] + +requires: + - phase: 16.2-01 + provides: pact field type API (FormFieldType with ListValue(ctx, value), FormFieldDecoded, FormFieldNotice, FormFieldUpload, FormFieldError), cabana FieldWritesFromContext, WithFieldWrites, TxFromContext, CompileFormWithTypes, TranslationWriter + - phase: 16.2-02 + provides: golem15-journal-blogeditor element, assets/dist/blogeditor.js and .css in adminFS, tests/blogeditor/fixtures/editor-corpus.json +provides: + - blogeditor and mlblogeditor field types on the Journal plugin (journal.Plugin.FormFieldTypes, classes.BlogEditorTypes, classes.BlogEditorElement, classes.LooksLikeHTML) + - classes.SanitizeHTMLReport and classes.HTMLRemovals (removal report behind the admin notices); SanitizeHTML is its string half + - newHTMLPolicy editor-output extension (figure data-embed/data-id, img alt, ol start) + - classes.StoreMedia, StoredMedia, MediaError, MediaMaxBytes, MediaFolderPattern shared by the public upload route and the editor upload + - postsAdminController FormAfterCreate/FormAfterUpdate writing content_html per written locale + - editor lang keys (content_invalid, image_invalid, upload_failed, removed_elements, removed_attributes, removed_comments, markup_changed) in en and pl +affects: [16.2-04, 16.2-05, 16.2-06, journal] + +actuals: + tokens: 20600 + tasks: 3 + commits: 5 +plan_head_before: 086e485215f69bd8a78f61fb1d5e69b2246366cd +plan_head_after: 2384a61e38f10afa522f38f8180cb62a32a9d9c0 +commits_repo: ../sm-journal-plugin +docs_commits: [332993c] + +tech-stack: + added: [] + patterns: + - "Removal report by in-order start-tag alignment: greedy match on an attribute-compatible output tag, falling back to element-name alignment; Other covers any change nothing else explains, so the report is empty exactly when the output equals the input" + - "Companion column through Form hooks: FieldWritesFromContext Default write to the host column in FormBefore*, non-default writes through the TranslationWriter in FormAfter*" + - "Policy pinned by the editor's own corpus: every corpus entry must survive byte for byte with an empty report" + +key-files: + created: + - ../sm-journal-plugin/classes/blog_editor.go + - ../sm-journal-plugin/classes/media_store.go + - ../sm-journal-plugin/classes/media_store_test.go + - ../sm-journal-plugin/classes/editor_policy_test.go + - ../sm-journal-plugin/field_types.go + - ../sm-journal-plugin/controllers/posts_editor_smoke_test.go + modified: + - ../sm-journal-plugin/classes/format_html.go + - ../sm-journal-plugin/controllers/posts.go + - ../sm-journal-plugin/controllers/admin_registry.go + - ../sm-journal-plugin/controllers/categories.go + - ../sm-journal-plugin/controllers/api/media.go + - ../sm-journal-plugin/controllers/api/media_test.go + - ../sm-journal-plugin/models/post/fields.yaml + - ../sm-journal-plugin/models/category/fields.yaml + - ../sm-journal-plugin/lang/en/lang.yaml + - ../sm-journal-plugin/lang/pl/lang.yaml + - ../sm-journal-plugin/posts_admin_smoke_test.go + - ../sm-journal-plugin/journal_api_categories_test.go + - ../sm-journal-plugin/README.md + - .planning/notes/journal-html-sanitizing-bluemonday.md + +key-decisions: + - "The policy needed exactly the four planned editor-output rules: a probe of the 15-entry corpus against the pre-change policy showed only figure data-embed/data-id, img alt and ol start being stripped; no fifth rule" + - "Editor upload: the sniffed type must be JPEG, PNG, GIF or WebP and must match an allowed extension; a name without an extension gets the sniffed one; a disallowed extension reaches StoreMedia's 'The file must be an image.'" + - "uniqueMediaKey fails instead of overwriting when all 999 numbered names are taken (the old code returned the original path); StoreMedia also checks the folder rule itself" + - "Load without a translation writer (Translated nil) serves FormatHTML(content) for a markdown value" + - "Removal notices end a cut name list with ', …'; HTMLRemovals keeps unexported cut flags" + +patterns-established: + - "A plugin field type's Decode reports changes as notices and never refuses on content it can repair" + +requirements-completed: [D-02, D-03, D-04, D-05, D-06, D-07, D-08, D-09, D-10, D-14, D-18, D-19] + +coverage: + - id: D1 + description: "Post content compiles as mlblogeditor (element, translatable, htmlColumn, uploadFolder; assets exist); Decode keeps clean HTML and the embed placeholder byte for byte with no notice, strips script/onerror with two notices, nil to '', refuses a number; Load serves HTML as stored, markdown as content_html or FormatHTML; ListValue; hooks write content_html per written locale and leave untouched content alone; en/pl plural phrases" + requirement: D-08 + verification: + - kind: unit + ref: "../sm-journal-plugin/controllers/posts_editor_smoke_test.go#TestBlogEditorTracerSmoke" + status: pass + human_judgment: false + - id: D2 + description: "SanitizeHTMLReport: elements, attributes (repeats and changed style), comments, Other, ten-name cut with exact counts; Empty exactly when the output equals the input for every unsafe payload, the imported-post fixture and every corpus entry" + requirement: D-10 + verification: + - kind: unit + ref: "../sm-journal-plugin/classes/editor_policy_test.go#TestSanitizeHTMLReport" + status: pass + human_judgment: false + - id: D3 + description: "Policy admits every editor-corpus entry byte for byte (empty report, nil CheckHTML) and the placeholder through FormatHTML; other providers, malformed ids, data attributes on other elements, other data-* on figure, onload, iframe, non-numeric start and start on ul are still stripped; the 35 unsafe payloads and the CheckHTML table are unchanged (diff against c60abc6 empty)" + requirement: D-05 + verification: + - kind: unit + ref: "../sm-journal-plugin/classes/editor_policy_test.go#TestEditorOutputPassesPolicy" + status: pass + - kind: unit + ref: "../sm-journal-plugin/classes/format_html_test.go#TestFormatHTMLSanitizesUnsafeHTML" + status: pass + - kind: unit + ref: "../sm-journal-plugin/classes/check_html_test.go#TestCheckHTML" + status: pass + human_judgment: false + - id: D4 + description: "StoreMedia keeps the public route's key layout, unique suffix, URL, extension list and size message; editor Upload sniffs the type, uses safe lowercase names and the field's uploadFolder, refuses html/svg/mismatched types, maps MediaError to FormFieldError and a missing bucket to an opaque error; the route's error bodies are unchanged" + requirement: D-18 + verification: + - kind: unit + ref: "../sm-journal-plugin/controllers/posts_editor_smoke_test.go#TestBlogEditorUploadSmoke" + status: pass + - kind: unit + ref: "../sm-journal-plugin/controllers/api/media_test.go#TestWriteMediaStoreError" + status: pass + - kind: unit + ref: "../sm-journal-plugin/classes/media_store_test.go#TestMediaObjectPath" + status: pass + human_judgment: false + - id: D5 + description: "Category content compiles as blogeditor, Load serves the stored *string, Decode strips onclick with a notice, the hooks map blank to NULL and accept any other content (422 retired)" + requirement: D-19 + verification: + - kind: unit + ref: "../sm-journal-plugin/controllers/posts_editor_smoke_test.go#TestBlogEditorUploadSmoke" + status: pass + human_judgment: false + - id: D6 + description: "Public posts API and route table unchanged since c60abc6 (D-09)" + requirement: D-09 + verification: + - kind: other + ref: "git -C ../sm-journal-plugin diff --quiet c60abc6 -- controllers/api/posts_helpers.go controllers/api/posts.go routes.go" + status: pass + human_judgment: false + - id: D7 + description: "Postgres-backed admin saves and the media route: TestPostsFormCompiles, TestPostsAdminCreateSmoke, TestPostsAdminResaveKeepsHTML, TestPostsAdminSaveSanitizesHTML, TestCategoriesAdminContent, journal_api_task3_test.go media upload, plugin boot with the field types" + verification: [] + human_judgment: true + rationale: "No Docker on this host (docker.sock permission denied); the tests compile under go vet and run on a Docker host in plan 06" + +duration: 13min +completed: 2026-10-10 +status: complete +--- + +# Phase 16.2 Plan 03: Journal Blog Editor integration Summary + +**Journal post content (mlblogeditor) and category content (blogeditor) are edited with the Blog Editor; every admin save stores bluemonday's output per locale, byte for byte when nothing is unsafe, writes the same HTML to content and content_html without goldmark, and names what it removed through SanitizeHTMLReport notices; the policy admits exactly the editor's four extra attributes, pinned by the editor corpus, and editor images go through the extracted StoreMedia with a magic-byte check.** + +## Performance + +- **Duration:** about 13 min +- **Started:** 2026-10-10T09:58:55Z +- **Completed:** 2026-10-10T10:12Z +- **Tasks:** 3 (tracer + 2 TDD expansions) +- **Files:** 20 in sm-journal-plugin (6 created, 14 modified), 1 decision note in summercms.go + +## Accomplishments + +- **Removal report (D-10 amended):** `SanitizeHTMLReport(doc) (string, HTMLRemovals)` aligns the input's start tags with bluemonday's output in order; unmatched tags are removed elements, missing, repeated or changed attributes are removed attributes (a `style` that lost a property counts under `style`), comments are counted, `Other` covers anything else. At most ten distinct names per list, exact counts. `SanitizeHTML` is its string half; `FormatHTML`, `FormatTranslation` and `CheckHTML` keep their behaviour. +- **Field types (D-08, D-14):** `classes.BlogEditorTypes` returns `blogeditor` (uploadFolder) and `mlblogeditor` (htmlColumn, uploadFolder) on `golem15-journal-blogeditor` with the plan 02 bundle; `journal.Plugin.FormFieldTypes` registers them. `checkBlogOptions` checks the options at boot. +- **Decode / Load / ListValue (D-04, D-07, D-10):** Decode returns the policy output plus `removed_elements`, `removed_attributes`, `removed_comments` and `markup_changed` notices; nil is `""`, a non-string is a `FormFieldError` (`content_invalid`). Load serves HTML as stored and markdown (`LooksLikeHTML` false) as that locale's `content_html`, else `FormatHTML(content)`. ListValue (ctx form) strips tags and cuts at 120 runes. +- **Admin save path (D-02, D-03):** `preparePostWrite` no longer calls goldmark; it copies the Default `content` write to `ContentHTML`. New `FormAfterCreate`/`FormAfterUpdate` write `content_html` for each non-default written locale through the `TranslationWriter` in the save transaction. Untouched content is not written, so both columns stay byte-identical. +- **Categories (D-19 amended):** `content` is `blogeditor` with `uploadFolder: categories`; the `CheckHTML` 422 gate is gone, blank still becomes NULL; the field comment (en, pl) says unsafe HTML is removed on save. +- **Media (D-18, D-09):** `classes.StoreMedia` holds the public route's storage (extensions, 10240 KiB, `/journal//`, unique suffix, `attach.PublicURL`); `UploadMedia` keeps auth, permissions, folder and form checks and the same bodies. The editor `Upload` sniffs the first 512 bytes (`http.DetectContentType`), requires a matching JPEG/PNG/GIF/WebP type, builds a safe lowercase name and stores in the field's folder (`editor` by default). +- **Policy (D-05 amended):** `figure` `data-embed` `^youtube$` and `data-id` `^[A-Za-z0-9_-]{11}$`, `img` `alt` any text, `ol` `start` `^[0-9]+$`; nothing else. `CheckHTML` documented as a predicate no write path refuses with. +- **Docs:** plugin README (field types, admin save rules, category rule, unchanged public API, editor uploads, the four policy rules, API reference rows); decision note amendment with the four table rows (summercms.go `332993c`). + +## Task Commits + +All code commits are in `../sm-journal-plugin` (branch main): + +1. **Task 1: tracer** - `cbf4891` feat(classes): report what the HTML policy removes; `fbd1c85` feat(admin): post content on the Blog Editor field type +2. **Task 2: categories, uploads, README** - `15fcad7` refactor(api): share media storage with the editor upload; `cb1c688` feat(admin): category content on the Blog Editor and editor image uploads +3. **Task 3: policy extension** - `2384a61` feat(classes): allow the Blog Editor's own markup in the HTML policy; summercms.go `332993c` docs(16.2): record the editor-output policy extension + +## Smoke test output + +``` +--- PASS: TestBlogEditorTracerSmoke (13 subtests: compile, type definitions, decode clean, decode placeholder, + decode strip with notices, decode nil/non-string, load HTML, load markdown content_html, load FormatHTML fallback, + list value, hooks write content_html, hooks leave untouched content, plural phrases) +--- PASS: TestBlogEditorUploadSmoke (9 subtests: StoreMedia layout, StoreMedia refusals, editor upload safe name, + editor upload refusals, missing bucket, category compile, category load, category decode, category hooks) +--- PASS: TestEditorOutputPassesPolicy (15 corpus entries, placeholder, 18 strip cases, alt, ol start) +--- PASS: TestSanitizeHTMLReport +--- PASS: TestFormatHTMLSanitizesUnsafeHTML, TestFormatHTMLDuplicateAttributes, TestFormatHTMLKeepsRawHTML, + TestSanitizeHTML, TestCheckHTML (unchanged since c60abc6) +--- PASS: TestWriteMediaStoreError, TestMediaObjectPath, TestMediaFolderPatternRejectsDotDot +``` + +- `go -C ../sm-journal-plugin vet ./...` clean (compiles the updated Postgres tests). +- `go test -short ./...`: `classes`, `controllers`, `controllers/api` pass; the root package, `models` and `updates` fail only on `docker.sock: permission denied`. +- `npx vitest run` 33/33 and `scripts/check-dist.sh` ("assets/dist matches a fresh build") still pass. + +## Postgres tests that need a Docker host (plan 06) + +- `posts_admin_smoke_test.go`: `TestPostsFormCompiles` (expects `"type":"mlblogeditor"`), `TestPostsAdminCreateSmoke` (HTML content, `ContentHTML == Content`, no `meta.notices`), `TestPostsAdminResaveKeepsHTML` (exact bytes), `TestPostsAdminSaveSanitizesHTML` (save succeeds, both columns sanitized for en host and pl translation, notices name `a, base` and `onerror` per locale). +- `journal_api_categories_test.go`: `TestCategoriesAdminContent` (schema type `blogeditor`, "unsafe create stripped" 201 + `

x

` + onclick notice, "unsafe update stripped" 200 + `x` + `a` notice, safe create without notices). +- `journal_api_task3_test.go` media upload tests (the route now runs through `StoreMedia`), `plugin_test.go`, and a full plugin boot with `FormFieldTypes` (cabana's boot-time asset and alias checks against the embedded `adminFS`). + +## TDD Gate Compliance + +Tasks 2 and 3 are `tdd="true"`. CLAUDE.md requires `go test ./...` green at every commit, so no failing `test(...)` commit exists; as in plans 01 and 02, RED was run and verified before GREEN and tests were committed with the implementation. + +- **Task 2 RED:** compile stubs for `StoreMedia` and the moved helpers, then `TestBlogEditorUploadSmoke` failed on its assertions (`not implemented`, "blogeditor has no Upload") and `TestMediaObjectPath` on its path assertion. `go test -json` was converted to TAP and checked: `msd_run check tdd-red-evidence` returned `RED_EVIDENCE_OK` (`target_test_failed`, target `TestBlogEditorUploadSmoke`). The category subtests were masked by the parent's fatal at RED time; the category schema and hook subtests then failed (type `textarea`, `validation_failed`) after Upload went green and before the category change, and passed after it. +- **Task 3 RED:** `TestEditorOutputPassesPolicy` failed on 18 subtests (corpus entries with the embed, alt and start; placeholder; the negative cases, whose `want` keeps the placeholder) before the policy change: `RED_EVIDENCE_OK`, target `TestEditorOutputPassesPolicy`. `TestSanitizeHTMLReport` was already green because the report shipped in the Task 1 tracer. +- **GREEN:** the implementation commits listed above; no refactor commit was needed. + +## Decisions Made + +See key-decisions in the frontmatter. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Plan text] Host and translation locales in TestPostsAdminSaveSanitizesHTML** +- **Found during:** Task 1 +- **Issue:** The plan says "pl (host) and en (translation)", but the test harness (`bootJournal`) makes `en` the default locale and `pl` the translation. +- **Fix:** The test asserts the host columns for `en` and `golem15_translate_attributes` for `pl`, and notices for both locales. +- **Files modified:** posts_admin_smoke_test.go +- **Commit:** fbd1c85 + +**2. [Rule 2 - Security] uniqueMediaKey no longer overwrites** +- **Found during:** Task 2 +- **Issue:** The moved `uniqueMediaKey` returned the original (existing) path when all 999 numbered names were taken, so a write would overwrite an object; the threat model (T-16.2-16) requires that it never overwrites. +- **Fix:** It returns an error instead (a 500 on the public route in that pathological case only). `StoreMedia` also re-checks the folder rule (MediaError on `folder`), which the route already enforces first, so no response changes. +- **Files modified:** classes/media_store.go +- **Commit:** 15fcad7 + +**3. [Rule 2 - Correctness] Editor upload type must match its extension** +- **Found during:** Task 2 +- **Issue:** The plan's "first bytes sniff as that image type" needs a rule for names whose extension disagrees with the bytes, or that have none. +- **Fix:** A sniffed type outside JPEG/PNG/GIF/WebP, or one that differs from an allowed extension's type, is `image_invalid`; a name without an extension gets the sniffed one; a disallowed extension gets StoreMedia's "The file must be an image." `safeUploadName` also trims leading and trailing `-` and `.` (no hidden or dot-only names). +- **Files modified:** classes/blog_editor.go +- **Commit:** cb1c688 + +**4. [Rule 2 - Correctness] Category field comment** +- **Found during:** Task 2 +- **Issue:** `category.content_comment` (en, pl) told administrators unsafe HTML "is refused", which is no longer true. +- **Fix:** It now says unsafe HTML is removed on save and the form names what was removed. +- **Files modified:** lang/en/lang.yaml, lang/pl/lang.yaml +- **Commit:** cb1c688 + +**5. Files and shapes outside the plan's lists** +- `controllers/admin_registry.go` holds the new `pact.FormAfterCreate`/`pact.FormAfterUpdate` assertions next to the controller's other compile-time assertions (commit fbd1c85). +- `controllers/api/media_test.go` now holds `TestWriteMediaStoreError` (the route's 422 and 500 bodies after the move); the two path tests moved to `classes/media_store_test.go` as planned. +- `HTMLRemovals` carries two unexported flags recording that a name list was cut, so `removalNotices` can append `, …`. +- Task 2 committed the refactor (`15fcad7`) before the feature (`cb1c688`) so that each commit builds on its own. +- `loadBlogHTML` serves `FormatHTML(content)` when no translation writer is published (`Translated` is nil), and serves blank content as it is. + +--- + +**Total deviations:** 4 auto-fixed (1 plan text, 3 correctness/security) plus file and ordering notes. **Impact:** no change to the planned API, the public routes' contract or the four policy rules. + +## Issues Encountered + +- No Docker on this host: the Postgres tests listed above compile but did not run (recorded in `.planning/WINDOWS.md` as unrun-verify entries 15 and 16). + +## Known Stubs + +None. + +## Next Phase Readiness + +- Plan 04 mirrors the same four policy rules in the layer and checks the same `editor-corpus.json`; no fifth rule was needed. +- Plan 05/06: run the Docker-only tests above, then the browser UAT of the editor in the admin (typing, uploads through the cabana route, notices after a stripped save). + +## Self-Check: PASSED + +- Created files exist in ../sm-journal-plugin: classes/blog_editor.go, classes/media_store.go, classes/media_store_test.go, classes/editor_policy_test.go, field_types.go, controllers/posts_editor_smoke_test.go. +- Commits exist in ../sm-journal-plugin: cbf4891, fbd1c85, 15fcad7, cb1c688, 2384a61; in summercms.go: 332993c.