docs(16.2-03): complete Journal Blog Editor integration plan

2026-10-10 12:13:26 +02:00
parent 332993c909
commit 2a7e9761a4
1 changed files with 272 additions and 0 deletions
--- a/.planning/phases/16.2-plugin-provided-field-types-and-the-blog-editor-formwidget/16.2-03-SUMMARY.md
+++ b/.planning/phases/16.2-plugin-provided-field-types-and-the-blog-editor-formwidget/16.2-03-SUMMARY.md
@@ -0,0 +1,272 @@
+---
+phase: 16.2-plugin-provided-field-types-and-the-blog-editor-formwidget
+plan: 03
+subsystem: journal-admin
+tags: [sm-journal-plugin, field-types, blog-editor, bluemonday, sanitizing, uploads, cabana]
+
+requires:
+  - phase: 16.2-01
+    provides: pact field type API (FormFieldType with ListValue(ctx, value), FormFieldDecoded, FormFieldNotice, FormFieldUpload, FormFieldError), cabana FieldWritesFromContext, WithFieldWrites, TxFromContext, CompileFormWithTypes, TranslationWriter
+  - phase: 16.2-02
+    provides: golem15-journal-blogeditor element, assets/dist/blogeditor.js and .css in adminFS, tests/blogeditor/fixtures/editor-corpus.json
+provides:
+  - blogeditor and mlblogeditor field types on the Journal plugin (journal.Plugin.FormFieldTypes, classes.BlogEditorTypes, classes.BlogEditorElement, classes.LooksLikeHTML)
+  - classes.SanitizeHTMLReport and classes.HTMLRemovals (removal report behind the admin notices); SanitizeHTML is its string half
+  - newHTMLPolicy editor-output extension (figure data-embed/data-id, img alt, ol start)
+  - classes.StoreMedia, StoredMedia, MediaError, MediaMaxBytes, MediaFolderPattern shared by the public upload route and the editor upload
+  - postsAdminController FormAfterCreate/FormAfterUpdate writing content_html per written locale
+  - editor lang keys (content_invalid, image_invalid, upload_failed, removed_elements, removed_attributes, removed_comments, markup_changed) in en and pl
+affects: [16.2-04, 16.2-05, 16.2-06, journal]
+
+actuals:
+  tokens: 20600
+  tasks: 3
+  commits: 5
+plan_head_before: 086e485215f69bd8a78f61fb1d5e69b2246366cd
+plan_head_after: 2384a61e38f10afa522f38f8180cb62a32a9d9c0
+commits_repo: ../sm-journal-plugin
+docs_commits: [332993c]
+
+tech-stack:
+  added: []
+  patterns:
+    - "Removal report by in-order start-tag alignment: greedy match on an attribute-compatible output tag, falling back to element-name alignment; Other covers any change nothing else explains, so the report is empty exactly when the output equals the input"
+    - "Companion column through Form hooks: FieldWritesFromContext Default write to the host column in FormBefore*, non-default writes through the TranslationWriter in FormAfter*"
+    - "Policy pinned by the editor's own corpus: every corpus entry must survive byte for byte with an empty report"
+
+key-files:
+  created:
+    - ../sm-journal-plugin/classes/blog_editor.go
+    - ../sm-journal-plugin/classes/media_store.go
+    - ../sm-journal-plugin/classes/media_store_test.go
+    - ../sm-journal-plugin/classes/editor_policy_test.go
+    - ../sm-journal-plugin/field_types.go
+    - ../sm-journal-plugin/controllers/posts_editor_smoke_test.go
+  modified:
+    - ../sm-journal-plugin/classes/format_html.go
+    - ../sm-journal-plugin/controllers/posts.go
+    - ../sm-journal-plugin/controllers/admin_registry.go
+    - ../sm-journal-plugin/controllers/categories.go
+    - ../sm-journal-plugin/controllers/api/media.go
+    - ../sm-journal-plugin/controllers/api/media_test.go
+    - ../sm-journal-plugin/models/post/fields.yaml
+    - ../sm-journal-plugin/models/category/fields.yaml
+    - ../sm-journal-plugin/lang/en/lang.yaml
+    - ../sm-journal-plugin/lang/pl/lang.yaml
+    - ../sm-journal-plugin/posts_admin_smoke_test.go
+    - ../sm-journal-plugin/journal_api_categories_test.go
+    - ../sm-journal-plugin/README.md
+    - .planning/notes/journal-html-sanitizing-bluemonday.md
+
+key-decisions:
+  - "The policy needed exactly the four planned editor-output rules: a probe of the 15-entry corpus against the pre-change policy showed only figure data-embed/data-id, img alt and ol start being stripped; no fifth rule"
+  - "Editor upload: the sniffed type must be JPEG, PNG, GIF or WebP and must match an allowed extension; a name without an extension gets the sniffed one; a disallowed extension reaches StoreMedia's 'The file must be an image.'"
+  - "uniqueMediaKey fails instead of overwriting when all 999 numbered names are taken (the old code returned the original path); StoreMedia also checks the folder rule itself"
+  - "Load without a translation writer (Translated nil) serves FormatHTML(content) for a markdown value"
+  - "Removal notices end a cut name list with ', …'; HTMLRemovals keeps unexported cut flags"
+
+patterns-established:
+  - "A plugin field type's Decode reports changes as notices and never refuses on content it can repair"
+
+requirements-completed: [D-02, D-03, D-04, D-05, D-06, D-07, D-08, D-09, D-10, D-14, D-18, D-19]
+
+coverage:
+  - id: D1
+    description: "Post content compiles as mlblogeditor (element, translatable, htmlColumn, uploadFolder; assets exist); Decode keeps clean HTML and the embed placeholder byte for byte with no notice, strips script/onerror with two notices, nil to '', refuses a number; Load serves HTML as stored, markdown as content_html or FormatHTML; ListValue; hooks write content_html per written locale and leave untouched content alone; en/pl plural phrases"
+    requirement: D-08
+    verification:
+      - kind: unit
+        ref: "../sm-journal-plugin/controllers/posts_editor_smoke_test.go#TestBlogEditorTracerSmoke"
+        status: pass
+    human_judgment: false
+  - id: D2
+    description: "SanitizeHTMLReport: elements, attributes (repeats and changed style), comments, Other, ten-name cut with exact counts; Empty exactly when the output equals the input for every unsafe payload, the imported-post fixture and every corpus entry"
+    requirement: D-10
+    verification:
+      - kind: unit
+        ref: "../sm-journal-plugin/classes/editor_policy_test.go#TestSanitizeHTMLReport"
+        status: pass
+    human_judgment: false
+  - id: D3
+    description: "Policy admits every editor-corpus entry byte for byte (empty report, nil CheckHTML) and the placeholder through FormatHTML; other providers, malformed ids, data attributes on other elements, other data-* on figure, onload, iframe, non-numeric start and start on ul are still stripped; the 35 unsafe payloads and the CheckHTML table are unchanged (diff against c60abc6 empty)"
+    requirement: D-05
+    verification:
+      - kind: unit
+        ref: "../sm-journal-plugin/classes/editor_policy_test.go#TestEditorOutputPassesPolicy"
+        status: pass
+      - kind: unit
+        ref: "../sm-journal-plugin/classes/format_html_test.go#TestFormatHTMLSanitizesUnsafeHTML"
+        status: pass
+      - kind: unit
+        ref: "../sm-journal-plugin/classes/check_html_test.go#TestCheckHTML"
+        status: pass
+    human_judgment: false
+  - id: D4
+    description: "StoreMedia keeps the public route's key layout, unique suffix, URL, extension list and size message; editor Upload sniffs the type, uses safe lowercase names and the field's uploadFolder, refuses html/svg/mismatched types, maps MediaError to FormFieldError and a missing bucket to an opaque error; the route's error bodies are unchanged"
+    requirement: D-18
+    verification:
+      - kind: unit
+        ref: "../sm-journal-plugin/controllers/posts_editor_smoke_test.go#TestBlogEditorUploadSmoke"
+        status: pass
+      - kind: unit
+        ref: "../sm-journal-plugin/controllers/api/media_test.go#TestWriteMediaStoreError"
+        status: pass
+      - kind: unit
+        ref: "../sm-journal-plugin/classes/media_store_test.go#TestMediaObjectPath"
+        status: pass
+    human_judgment: false
+  - id: D5
+    description: "Category content compiles as blogeditor, Load serves the stored *string, Decode strips onclick with a notice, the hooks map blank to NULL and accept any other content (422 retired)"
+    requirement: D-19
+    verification:
+      - kind: unit
+        ref: "../sm-journal-plugin/controllers/posts_editor_smoke_test.go#TestBlogEditorUploadSmoke"
+        status: pass
+    human_judgment: false
+  - id: D6
+    description: "Public posts API and route table unchanged since c60abc6 (D-09)"
+    requirement: D-09
+    verification:
+      - kind: other
+        ref: "git -C ../sm-journal-plugin diff --quiet c60abc6 -- controllers/api/posts_helpers.go controllers/api/posts.go routes.go"
+        status: pass
+    human_judgment: false
+  - id: D7
+    description: "Postgres-backed admin saves and the media route: TestPostsFormCompiles, TestPostsAdminCreateSmoke, TestPostsAdminResaveKeepsHTML, TestPostsAdminSaveSanitizesHTML, TestCategoriesAdminContent, journal_api_task3_test.go media upload, plugin boot with the field types"
+    verification: []
+    human_judgment: true
+    rationale: "No Docker on this host (docker.sock permission denied); the tests compile under go vet and run on a Docker host in plan 06"
+
+duration: 13min
+completed: 2026-10-10
+status: complete
+---
+
+# Phase 16.2 Plan 03: Journal Blog Editor integration Summary
+
+**Journal post content (mlblogeditor) and category content (blogeditor) are edited with the Blog Editor; every admin save stores bluemonday's output per locale, byte for byte when nothing is unsafe, writes the same HTML to content and content_html without goldmark, and names what it removed through SanitizeHTMLReport notices; the policy admits exactly the editor's four extra attributes, pinned by the editor corpus, and editor images go through the extracted StoreMedia with a magic-byte check.**
+
+## Performance
+
+- **Duration:** about 13 min
+- **Started:** 2026-10-10T09:58:55Z
+- **Completed:** 2026-10-10T10:12Z
+- **Tasks:** 3 (tracer + 2 TDD expansions)
+- **Files:** 20 in sm-journal-plugin (6 created, 14 modified), 1 decision note in summercms.go
+
+## Accomplishments
+
+- **Removal report (D-10 amended):** `SanitizeHTMLReport(doc) (string, HTMLRemovals)` aligns the input's start tags with bluemonday's output in order; unmatched tags are removed elements, missing, repeated or changed attributes are removed attributes (a `style` that lost a property counts under `style`), comments are counted, `Other` covers anything else. At most ten distinct names per list, exact counts. `SanitizeHTML` is its string half; `FormatHTML`, `FormatTranslation` and `CheckHTML` keep their behaviour.
+- **Field types (D-08, D-14):** `classes.BlogEditorTypes` returns `blogeditor` (uploadFolder) and `mlblogeditor` (htmlColumn, uploadFolder) on `golem15-journal-blogeditor` with the plan 02 bundle; `journal.Plugin.FormFieldTypes` registers them. `checkBlogOptions` checks the options at boot.
+- **Decode / Load / ListValue (D-04, D-07, D-10):** Decode returns the policy output plus `removed_elements`, `removed_attributes`, `removed_comments` and `markup_changed` notices; nil is `""`, a non-string is a `FormFieldError` (`content_invalid`). Load serves HTML as stored and markdown (`LooksLikeHTML` false) as that locale's `content_html`, else `FormatHTML(content)`. ListValue (ctx form) strips tags and cuts at 120 runes.
+- **Admin save path (D-02, D-03):** `preparePostWrite` no longer calls goldmark; it copies the Default `content` write to `ContentHTML`. New `FormAfterCreate`/`FormAfterUpdate` write `content_html` for each non-default written locale through the `TranslationWriter` in the save transaction. Untouched content is not written, so both columns stay byte-identical.
+- **Categories (D-19 amended):** `content` is `blogeditor` with `uploadFolder: categories`; the `CheckHTML` 422 gate is gone, blank still becomes NULL; the field comment (en, pl) says unsafe HTML is removed on save.
+- **Media (D-18, D-09):** `classes.StoreMedia` holds the public route's storage (extensions, 10240 KiB, `/journal/<folder>/<name>`, unique suffix, `attach.PublicURL`); `UploadMedia` keeps auth, permissions, folder and form checks and the same bodies. The editor `Upload` sniffs the first 512 bytes (`http.DetectContentType`), requires a matching JPEG/PNG/GIF/WebP type, builds a safe lowercase name and stores in the field's folder (`editor` by default).
+- **Policy (D-05 amended):** `figure` `data-embed` `^youtube$` and `data-id` `^[A-Za-z0-9_-]{11}$`, `img` `alt` any text, `ol` `start` `^[0-9]+$`; nothing else. `CheckHTML` documented as a predicate no write path refuses with.
+- **Docs:** plugin README (field types, admin save rules, category rule, unchanged public API, editor uploads, the four policy rules, API reference rows); decision note amendment with the four table rows (summercms.go `332993c`).
+
+## Task Commits
+
+All code commits are in `../sm-journal-plugin` (branch main):
+
+1. **Task 1: tracer** - `cbf4891` feat(classes): report what the HTML policy removes; `fbd1c85` feat(admin): post content on the Blog Editor field type
+2. **Task 2: categories, uploads, README** - `15fcad7` refactor(api): share media storage with the editor upload; `cb1c688` feat(admin): category content on the Blog Editor and editor image uploads
+3. **Task 3: policy extension** - `2384a61` feat(classes): allow the Blog Editor's own markup in the HTML policy; summercms.go `332993c` docs(16.2): record the editor-output policy extension
+
+## Smoke test output
+
+```
+--- PASS: TestBlogEditorTracerSmoke (13 subtests: compile, type definitions, decode clean, decode placeholder,
+    decode strip with notices, decode nil/non-string, load HTML, load markdown content_html, load FormatHTML fallback,
+    list value, hooks write content_html, hooks leave untouched content, plural phrases)
+--- PASS: TestBlogEditorUploadSmoke (9 subtests: StoreMedia layout, StoreMedia refusals, editor upload safe name,
+    editor upload refusals, missing bucket, category compile, category load, category decode, category hooks)
+--- PASS: TestEditorOutputPassesPolicy (15 corpus entries, placeholder, 18 strip cases, alt, ol start)
+--- PASS: TestSanitizeHTMLReport
+--- PASS: TestFormatHTMLSanitizesUnsafeHTML, TestFormatHTMLDuplicateAttributes, TestFormatHTMLKeepsRawHTML,
+          TestSanitizeHTML, TestCheckHTML (unchanged since c60abc6)
+--- PASS: TestWriteMediaStoreError, TestMediaObjectPath, TestMediaFolderPatternRejectsDotDot
+```
+
+- `go -C ../sm-journal-plugin vet ./...` clean (compiles the updated Postgres tests).
+- `go test -short ./...`: `classes`, `controllers`, `controllers/api` pass; the root package, `models` and `updates` fail only on `docker.sock: permission denied`.
+- `npx vitest run` 33/33 and `scripts/check-dist.sh` ("assets/dist matches a fresh build") still pass.
+
+## Postgres tests that need a Docker host (plan 06)
+
+- `posts_admin_smoke_test.go`: `TestPostsFormCompiles` (expects `"type":"mlblogeditor"`), `TestPostsAdminCreateSmoke` (HTML content, `ContentHTML == Content`, no `meta.notices`), `TestPostsAdminResaveKeepsHTML` (exact bytes), `TestPostsAdminSaveSanitizesHTML` (save succeeds, both columns sanitized for en host and pl translation, notices name `a, base` and `onerror` per locale).
+- `journal_api_categories_test.go`: `TestCategoriesAdminContent` (schema type `blogeditor`, "unsafe create stripped" 201 + `<p>x</p>` + onclick notice, "unsafe update stripped" 200 + `x` + `a` notice, safe create without notices).
+- `journal_api_task3_test.go` media upload tests (the route now runs through `StoreMedia`), `plugin_test.go`, and a full plugin boot with `FormFieldTypes` (cabana's boot-time asset and alias checks against the embedded `adminFS`).
+
+## TDD Gate Compliance
+
+Tasks 2 and 3 are `tdd="true"`. CLAUDE.md requires `go test ./...` green at every commit, so no failing `test(...)` commit exists; as in plans 01 and 02, RED was run and verified before GREEN and tests were committed with the implementation.
+
+- **Task 2 RED:** compile stubs for `StoreMedia` and the moved helpers, then `TestBlogEditorUploadSmoke` failed on its assertions (`not implemented`, "blogeditor has no Upload") and `TestMediaObjectPath` on its path assertion. `go test -json` was converted to TAP and checked: `msd_run check tdd-red-evidence` returned `RED_EVIDENCE_OK` (`target_test_failed`, target `TestBlogEditorUploadSmoke`). The category subtests were masked by the parent's fatal at RED time; the category schema and hook subtests then failed (type `textarea`, `validation_failed`) after Upload went green and before the category change, and passed after it.
+- **Task 3 RED:** `TestEditorOutputPassesPolicy` failed on 18 subtests (corpus entries with the embed, alt and start; placeholder; the negative cases, whose `want` keeps the placeholder) before the policy change: `RED_EVIDENCE_OK`, target `TestEditorOutputPassesPolicy`. `TestSanitizeHTMLReport` was already green because the report shipped in the Task 1 tracer.
+- **GREEN:** the implementation commits listed above; no refactor commit was needed.
+
+## Decisions Made
+
+See key-decisions in the frontmatter.
+
+## Deviations from Plan
+
+### Auto-fixed Issues
+
+**1. [Plan text] Host and translation locales in TestPostsAdminSaveSanitizesHTML**
+- **Found during:** Task 1
+- **Issue:** The plan says "pl (host) and en (translation)", but the test harness (`bootJournal`) makes `en` the default locale and `pl` the translation.
+- **Fix:** The test asserts the host columns for `en` and `golem15_translate_attributes` for `pl`, and notices for both locales.
+- **Files modified:** posts_admin_smoke_test.go
+- **Commit:** fbd1c85
+
+**2. [Rule 2 - Security] uniqueMediaKey no longer overwrites**
+- **Found during:** Task 2
+- **Issue:** The moved `uniqueMediaKey` returned the original (existing) path when all 999 numbered names were taken, so a write would overwrite an object; the threat model (T-16.2-16) requires that it never overwrites.
+- **Fix:** It returns an error instead (a 500 on the public route in that pathological case only). `StoreMedia` also re-checks the folder rule (MediaError on `folder`), which the route already enforces first, so no response changes.
+- **Files modified:** classes/media_store.go
+- **Commit:** 15fcad7
+
+**3. [Rule 2 - Correctness] Editor upload type must match its extension**
+- **Found during:** Task 2
+- **Issue:** The plan's "first bytes sniff as that image type" needs a rule for names whose extension disagrees with the bytes, or that have none.
+- **Fix:** A sniffed type outside JPEG/PNG/GIF/WebP, or one that differs from an allowed extension's type, is `image_invalid`; a name without an extension gets the sniffed one; a disallowed extension gets StoreMedia's "The file must be an image." `safeUploadName` also trims leading and trailing `-` and `.` (no hidden or dot-only names).
+- **Files modified:** classes/blog_editor.go
+- **Commit:** cb1c688
+
+**4. [Rule 2 - Correctness] Category field comment**
+- **Found during:** Task 2
+- **Issue:** `category.content_comment` (en, pl) told administrators unsafe HTML "is refused", which is no longer true.
+- **Fix:** It now says unsafe HTML is removed on save and the form names what was removed.
+- **Files modified:** lang/en/lang.yaml, lang/pl/lang.yaml
+- **Commit:** cb1c688
+
+**5. Files and shapes outside the plan's lists**
+- `controllers/admin_registry.go` holds the new `pact.FormAfterCreate`/`pact.FormAfterUpdate` assertions next to the controller's other compile-time assertions (commit fbd1c85).
+- `controllers/api/media_test.go` now holds `TestWriteMediaStoreError` (the route's 422 and 500 bodies after the move); the two path tests moved to `classes/media_store_test.go` as planned.
+- `HTMLRemovals` carries two unexported flags recording that a name list was cut, so `removalNotices` can append `, …`.
+- Task 2 committed the refactor (`15fcad7`) before the feature (`cb1c688`) so that each commit builds on its own.
+- `loadBlogHTML` serves `FormatHTML(content)` when no translation writer is published (`Translated` is nil), and serves blank content as it is.
+
+---
+
+**Total deviations:** 4 auto-fixed (1 plan text, 3 correctness/security) plus file and ordering notes. **Impact:** no change to the planned API, the public routes' contract or the four policy rules.
+
+## Issues Encountered
+
+- No Docker on this host: the Postgres tests listed above compile but did not run (recorded in `.planning/WINDOWS.md` as unrun-verify entries 15 and 16).
+
+## Known Stubs
+
+None.
+
+## Next Phase Readiness
+
+- Plan 04 mirrors the same four policy rules in the layer and checks the same `editor-corpus.json`; no fifth rule was needed.
+- Plan 05/06: run the Docker-only tests above, then the browser UAT of the editor in the admin (typing, uploads through the cabana route, notices after a stripped save).
+
+## Self-Check: PASSED
+
+- Created files exist in ../sm-journal-plugin: classes/blog_editor.go, classes/media_store.go, classes/media_store_test.go, classes/editor_policy_test.go, field_types.go, controllers/posts_editor_smoke_test.go.
+- Commits exist in ../sm-journal-plugin: cbf4891, fbd1c85, 15fcad7, cb1c688, 2384a61; in summercms.go: 332993c.