sm-journal-plugin/README.md

# golem15.journal

The compiled journal plugin for SummerCMS applications: translatable posts and categories, tags, a settings singleton, and the tables a host migrate creates for a blog.

```go
import _ "git.golem15.com/golem15/sm-journal-plugin"
```

Module `git.golem15.com/golem15/sm-journal-plugin`, package `journal`, plugin ID `golem15.journal`.

## Overview

The plugin is a compiled SummerCMS plugin. Its `init` registers it with the framework, so an application loads it by listing it in `summer.yaml`; `summer build` then emits the blank import above into the application's `plugins.gen.go`.

It requires `golem15.translate`. Post and Category store default-locale values on host columns and other locales in `golem15_translate_attributes`. `MorphName` is the PHP class string (`Golem15\Journal\Models\Post`, `Golem15\Journal\Models\Category`); it is never `reflect.Type.String()`.

WinterCMS theme components such as `journalPost` and `journalPosts` are a later successor, not implemented here.

## Features

- **Posts.** `models.Post` maps `golem15_journal_posts`. Translatable fields are title, content, content_html, excerpt, metadata, and slug (indexed). Fillable does not include `redactor_id`, `content_html`, or `user_id`. Attachments `featured_images` and `content_images` use `system_files` under the same MorphName. YAML admin uses `mltext`, `mltextarea` and the plugin's own `mlblogeditor` field type for content (see [Admin: Blog Editor field types](#admin-blog-editor-field-types)); an admin save stores the sanitized HTML in both `content` and `content_html`. `classes.FormatHTML` regenerates `content_html` on the other write paths (`journal:import-posts`, Journal API writes). It renders markdown and passes raw HTML in `content` through the renderer (Winter parity), then sanitizes the result with a `bluemonday` allow-list: bluemonday's UGC policy plus `footer`, `class`, schema.org microdata, the `color`, `font-size`, `font-weight` and `text-align` styles, goldmark's footnote roles, `dir="auto"`, two pasted-transcript `data-message-*` attributes, and exactly four rules for the Blog Editor's own output (Phase 16.2): `data-embed` matching `^youtube$` and `data-id` matching `^[A-Za-z0-9_-]{11}$` on `figure` (the embed placeholder), `alt` with any text on `img`, and `start` matching `^[0-9]+$` on `ol`. No iframe, other data attribute, style property or URL scheme is allowed for the editor; `TestEditorOutputPassesPolicy` checks every entry of the editor corpus against the policy. Like Winter's `Html::clean`, it strips what the policy does not allow instead of refusing the save: script, style, iframe, object, embed, form, base and meta elements, comments, event handler attributes, and URLs other than http, https, mailto and relative ones. HTML the policy keeps whole is stored byte for byte, so imported bodies survive a save. Admin saves strip the same way and name what they removed (`classes.SanitizeHTMLReport`). A tag that repeats an attribute is never kept whole: the sanitized output, which keeps the first value, is stored. On the API and import paths `content` itself is stored as submitted. Translated `content_html` goes through the same policy on the API and `journal:import-posts` write paths (`classes.FormatTranslation`): when the translation carries a non-blank `content`, `content_html` is rendered from it and a supplied `content_html` is replaced, as the host column is always derived; a translation with only `content_html` is sanitized with `classes.SanitizeHTML`. Winter stored a supplied translated `content_html` verbatim. The API response shapes and status codes are unchanged.
- **Categories and tags.** `models.Category` is translatable (name, description, slug). It also stores `content`, an HTML host column that is not translatable, edited in the admin with the `blogeditor` field type. Fillable is name, slug, code, description, content and parent_id; nest_* columns exist in DDL and are not fillable. `models.Tag` is not translatable. Fillable is name, slug, description. Pivots are `golem15_journal_posts_categories` and `golem15_journal_posts_tags`.
- **Settings.** Singleton row ID=1 on `golem15_journal_settings`, not PHP `system_settings`. `search_use_typesense` defaults to false. A fresh save never contacts Typesense.
- **Public HTTP.** Anonymous GET `/_journal/api/v1/posts`, `/posts/{slug}`, `/categories`, `/tags`, `/rss`, `/authors/{author}`, and `/authors/{author}/posts`. Writes under the same prefix require a cabana backend JWT (`aud=backend`).
- **Author slug.** Plugin migration adds `backend_users.golem15_bloghub_author_slug`. The column stores the public author slug that the author routes resolve.
- **Import and export.** `journal:export-posts` and `journal:import-posts` write and read one JSON file per post. The Posts list toolbar exposes the same actions to administrators with `golem15.journal.access_import_export`.
- **Phrasebooks.** Embedded `lang/en/lang.yaml` and `lang/pl/lang.yaml` hold plugin, journal, post, category, and tag UI strings. They are independent of model translation JSON.
- **Schema.** Eight gormigrate steps create the six journal tables, the author-slug column and the category `content` column. AutoMigrate is not used. Table names are `golem15_journal_*` only.

## Usage

Mount the plugin in an application as a git submodule at `plugins/golem15/journal`:

```sh
git submodule add git@git.golem15.com:golem15/sm-journal-plugin.git plugins/golem15/journal
```

Add it to the application's `go.work`:

```text
use (
	.
	./plugins/golem15/journal
)
```

Require it in the application's `go.mod` with a local replace:

```text
require git.golem15.com/golem15/sm-journal-plugin v0.0.0

replace git.golem15.com/golem15/sm-journal-plugin => ./plugins/golem15/journal
```

List it in `summer.yaml` after translate and run `summer build`:

```yaml
module: git.golem15.com/golem15/acme
binary: acme
plugins:
  - id: golem15.translate
    module: git.golem15.com/golem15/sm-translate-plugin
  - id: golem15.journal
    module: git.golem15.com/golem15/sm-journal-plugin
```

The plugin's own `go.mod` replaces the framework with `../summercms.go` for a sibling checkout next to `summercms.go`. A host application that mounts the plugin at `plugins/golem15/journal` should replace the framework from the host `go.work`.

`Requires()` is exactly `golem15.translate`.

A blog frontend lists published posts and fetches one by slug without a token:

```http
GET /_journal/api/v1/posts
GET /_journal/api/v1/posts/hello-world
GET /_journal/api/v1/categories
GET /_journal/api/v1/tags
GET /_journal/api/v1/rss
GET /_journal/api/v1/authors/jane-doe
GET /_journal/api/v1/authors/jane-doe/posts?page=2
```

### Public categories API

`GET /_journal/api/v1/categories` returns `{"data":[...]}`, a tree of category rows. Each row has the keys `id`, `name`, `slug`, `description`, `content`, `parent_id`, `nest_depth` and `post_count`, plus `children` only when the category has listed children. A category with no published posts and no listed children is omitted unless the request adds `include_empty=1`.

`content` is the category's HTML body for a category landing page. It is stored and returned as is: it is not markdown and the API does not render, trim or sanitize it. It is `null` when unset, and it is the same in every locale, because it is a host column and not translatable. A consumer that renders it as HTML must apply its own gate or sanitizer first.

The admin Categories form edits content with the `blogeditor` field type: a save strips what the HTML policy does not allow, names what it removed in the response's `meta.notices`, and is never refused with a `422` because of markup. Blank content is stored as `null`. Rows written by SQL or by import commands are not sanitized by the form, so consumers keep their own gate.

```json
{"data":[{"id":4,"name":"Acme news","slug":"news","description":null,"content":"<h2>Acme news</h2><p>Pillar copy</p>","parent_id":null,"nest_depth":0,"post_count":3}]}
```

### Public author API

`GET /_journal/api/v1/authors/{author}` returns one author's public profile without a token. `{author}` resolves in this order:

1. `backend_users.golem15_bloghub_author_slug`, the public author slug. A user with a slug resolves even before their first post.
2. `backend_users.login`, but only for a user with at least one published post. That login is already public through the post `author` object, so other backend accounts cannot be enumerated.

Soft-deleted backend users never resolve. Unlike the WinterCMS author component, a login also resolves for a user who has a slug, because the post `author` object carries the login and no slug.

```json
{"data":{"id":7,"slug":"jane-doe","first_name":"Jane","last_name":"Doe"}}
```

The author object has exactly these four keys. `slug` is the canonical identifier: the author slug when set, otherwise the login, so a consumer that links by login sets its canonical URL from `slug`. Missing names are empty strings. The object carries no contact, credential, permission or role data and has no `login` key; `slug` falls back to the login only for an author whose login is already public through a published post.

An unknown author, a login without published posts, a soft-deleted user and a malformed value (empty, over 255 bytes, invalid UTF-8 or containing a control character) all answer `404`:

```json
{"error":"Author not found"}
```

`GET /_journal/api/v1/authors/{author}/posts` lists that author's published posts (`published` true and `published_at` in the past), resolving `{author}` the same way. Rows have the same shape as `GET /_journal/api/v1/posts` rows. Bearer tokens are ignored, so an editor also sees published posts only. Query keys:

| Key | Default | Notes |
|-----|---------|-------|
| `page` | `1` | Values below 1 or non-numeric fall back to page 1. |
| `per_page` | `9` | Clamped to 1..30. |
| `sort` | `published_at desc` | Same columns as `/posts`: `published_at`, `created_at`, `updated_at`, `title`, `id`, each `asc` or `desc`. |

The response uses the `/posts` pagination envelope (rows trimmed here):

```json
{"data":[{"id":12,"slug":"hello-world","title":"Hello world","author":{"id":7,"first_name":"Jane","last_name":"Doe","login":"jdoe"}}],"meta":{"current_page":2,"last_page":2,"per_page":9,"total":10}}
```

An author that does not resolve answers the same `404 {"error":"Author not found"}` as the profile route.

Editors create a post with a cabana backend JWT (audience `backend`):

```http
POST /_journal/api/v1/posts
Authorization: Bearer <backend-access-token>
Content-Type: application/json

{"title":"Hello","content":"A blog post"}
```

A blog post model already declares translatable fields:

```go
func (Post) Translatable() []string {
	return []string{"title", "content", "content_html", "excerpt", "metadata", "slug"}
}
func (Post) TranslatableIndexes() []string { return []string{"slug"} }
func (Post) MorphName() string             { return `Golem15\Journal\Models\Post` }
```

### Blog Editor

`assets/src/blogeditor` is the Blog Editor admin element, the custom element `golem15-journal-blogeditor` (`BlogEditorElement`). It is a WYSIWYG HTML editor on TipTap (ProseMirror, MIT) with a Source tab that shows and edits the exact HTML string. The toolbar offers paragraphs and headings 2 to 4, bold, italic, bullet and ordered lists, links, blockquotes, tables, images, YouTube embeds, a raw HTML block, undo and redo, plus a Preview toggle. Labels follow the `ui-locale` attribute (`en`, `pl`).

The element follows the admin field type element contract: the `value` property holds the HTML string; the attributes are `field-name`, `record-id`, `locale`, `ui-locale`, `options`, `disabled` and `invalid`; a bubbling `summer-change` event with `detail.value` fires on user edits only. It styles itself only through the admin `--c-*` variables, in `blogeditor.css` under its tag (TipTap's injected stylesheet is off), and never receives or sends credentials.

- **Untouched values.** The element returns the exact string it was given until the user edits; an untouched imported post is saved byte for byte. Setting `value` to the string it currently returns is a no-op. After an edit, top-level blocks that are still equal to the loaded ones keep their original bytes, so editing one paragraph changes only that paragraph. Markup the schema does not model (`footer`, `section`, `aside`, `header`, `nav`, `details`, `style`, `div` and similar blocks, and a `figure` that is not an image with an optional caption) becomes a raw HTML card that shows its source as text and is written back verbatim. Unmodelled attributes on paragraphs, headings, lists, blockquotes, tables, cells, images and links, and `span` elements with attributes, survive an edit; event handler attributes (`on*`) are never kept.
- **Embeds.** The embed button takes a YouTube URL (`youtube.com/watch?v=`, `youtu.be/`, `youtube.com/shorts/`, `youtube.com/embed/`) and stores exactly `<figure data-embed="youtube" data-id="ID"></figure>`, where `ID` matches `^[A-Za-z0-9_-]{11}$`. The admin shows a card and never an iframe; the site layer expands the placeholder. Another provider is one entry in `EMBED_PROVIDERS` (`embed.ts`) and one in the layer.
- **Preview.** The Preview button (`Podgląd` for `ui-locale` `pl`) is off by default, belongs to each element on its own and shows its state through `aria-pressed`. It splits the editor 50/50; when the element is narrower than about 720px, the pane wraps below the editor. The pane renders the current value from the WYSIWYG or Source tab 300 ms (`PREVIEW_DEBOUNCE_MS` in `preview.ts`) after the last change, and each update reloads the frame. The frame is an `<iframe srcdoc>` with an empty `sandbox` attribute, so no scripts run and its origin is opaque. It has `referrerpolicy="no-referrer"` and a Content-Security-Policy meta (`PREVIEW_CSP`) that allows only inline styles and images. The preview copy leaves out `script`, `iframe`, `frame`, `frameset`, `object`, `embed`, `meta`, `base` and `link` elements and event handler attributes. YouTube placeholders show as thumbnail cards loaded from img.youtube.com, never as a player; these and the post's own images are the only network traffic the pane makes. The styling is post-body typography, with the `--c-*` values read from the element at render time or neutral defaults. Previewing never fires `summer-change` and never changes the value, so an untouched post is still saved byte for byte.
- **Links.** The editor creates and keeps links only to `http`, `https`, `mailto` and relative URLs (`isPolicyUrl` in `extensions.ts`), the schemes both sanitizer policies keep; the scheme is read after removing control characters and spaces. A created link has `href` only; stored links keep their own `target` and `rel`, which the server strips on an edited save.
- **Images.** An image chosen with the file picker (JPEG, PNG, GIF, WebP) or dropped on the editor is handed to the admin page as a bubbling `summer-upload` event with `detail {id, file}`. The admin uploads it with its own session and answers on the element with `summer-upload-result` `{id, url}` or `{id, error}`. On a URL the editor asks for the alt text (required) and an optional caption and inserts `<img src alt>`, or `<figure>` with `<img>` and `<figcaption>` when a caption is given. An error, or no answer within 60 seconds, is shown in the status line and nothing is inserted.
- **Sanitizing.** The element never sanitizes. Every admin save goes through `classes.SanitizeHTMLReport`, which strips what the policy does not allow and reports it; the policy keeps all of the editor's own output.

The element is built by the plugin's own JS build into `assets/dist/blogeditor.js` and `assets/dist/blogeditor.css`. Both files are committed and embedded through `admin.go`, so a host application needs no Node:

```sh
npm ci                      # exact versions from package-lock.json
npm run build               # typecheck, then vite build into assets/dist
npm test                    # vitest on happy-dom
bash scripts/check-dist.sh  # fails when assets/dist differs from a fresh build
```

After changing `assets/src`, run `npm run build` and commit `assets/dist`.

`tests/blogeditor/fixtures/editor-corpus.json` holds the editor's own output, built through editor commands: an embed, images with and without a caption, a table with a header row, links, headings, lists (one starting at 3), a blockquote, inline marks, a code block, a horizontal rule, a hard break, and one document combining them. `tests/blogeditor/corpus.test.ts` rebuilds it and fails when the committed file differs; `UPDATE_CORPUS=1 npm test` rewrites it. The Go policy test and the site layer check the same entries against both sanitizer policies, so entry names stay stable.

### Admin: Blog Editor field types

The plugin registers two admin form field types (`journal.Plugin.FormFieldTypes`, built by `classes.BlogEditorTypes`) that render the Blog Editor element. Any controller's `fields.yaml` may use them:

| Type | Value | Options |
|------|-------|---------|
| `blogeditor` | One HTML string in a host column (category content). | `uploadFolder` |
| `mlblogeditor` | One HTML string per content locale, stored like `mlmarkdown`: the default locale in the host column, the others as translations (post content). | `htmlColumn`, `uploadFolder` |

Both types show the element's Preview pane, and previewing never changes what a save stores.

`htmlColumn` names the translatable attribute that holds the rendered HTML of a markdown value (`content_html` for posts). `uploadFolder` is the folder under `/journal/` that editor images are stored in (`posts`, `categories`; `editor` when unset); it follows the public upload route's folder rule (at most 100 letters, digits, `_`, `-` or `/`, no `..`) and is checked at boot.

```yaml
content:
    label: golem15.journal::lang.post.content
    type: mlblogeditor
    htmlColumn: content_html
    uploadFolder: posts
```

What an admin save stores:

- Each submitted locale's value goes through `classes.SanitizeHTMLReport`. A value the policy keeps whole is stored byte for byte. Anything the policy does not allow is removed, the save goes through, and the response's `meta.notices` names what was removed per field and locale (for example `2 elements were removed: script, iframe`), in English or Polish. A save is never refused because of markup; a value that is not text is a `422` on the field.
- For posts, the stored HTML goes to both `content` and `content_html` of that locale: the host columns through the `FormBeforeCreate`/`FormBeforeUpdate` hooks, the other locales through `FormAfterCreate`/`FormAfterUpdate` and the translation writer. The admin path does not run goldmark.
- A locale whose content the administrator did not change is not written, so an imported post keeps both columns byte for byte until its body is edited.
- A post whose `content` does not start with `<` (`classes.LooksLikeHTML`) is a markdown post: the editor opens it with that locale's `content_html`, or with `FormatHTML(content)` when that is empty. Its first edited save stores HTML in both columns.
- Category content follows the same rule and is no longer refused with a `422`.

The public Journal API is unchanged: `POST` and `PUT /_journal/api/v1/posts` still take markdown `content` and store `FormatHTML` output in `content_html`.

Editor images are uploaded by the admin through the field's upload route and stored by `classes.StoreMedia`, the same storage as `POST /_journal/api/v1/media/upload`: the key is `/journal/<uploadFolder>/<name>` with the same numbered suffix when the name is taken and the same public URL. An editor upload must be a JPEG, PNG, GIF or WebP file whose first bytes sniff as the type its extension names (no SVG), at most 10240 KiB. Its name is lowercased and every character outside `a-z`, `0-9`, `.`, `_` and `-` becomes `-`, so the inserted `src` needs no escaping. The public upload route keeps its own naming, request fields, response and messages.

## API reference

| Identifier | Description |
|------------|-------------|
| `journal.Plugin` | Compiled plugin; `ID` is `golem15.journal`. |
| `journal.Plugin.Requires` | Returns `golem15.translate` only. |
| `journal.Plugin.LangFS` | Embedded English and Polish UI phrasebooks. |
| `journal.Plugin.Migrations` | gormigrate steps registered by this plugin. |
| `journal.Plugin.Models` | Models registered by this plugin. |
| `journal.Plugin.Commands` | Registers `journal:export-posts` and `journal:import-posts`. |
| `journal.Plugin.Routes` | Public GETs (posts, categories, tags, RSS, authors) and backend-Bearer writes under `/_journal/api/v1`. |
| `journal.Plugin.FormFieldTypes` | Registers the `blogeditor` and `mlblogeditor` admin field types. |
| `journal.Plugin.Buckets` | Limiter buckets `journal-public-api` and `journal-api` (Max 120). |
| `models.Post` | Post row; `TableName` is `golem15_journal_posts`. |
| `models.Post.MorphName` | PHP class string `Golem15\Journal\Models\Post`. |
| `models.Post.Fillable` | Allow-list excluding `redactor_id`, `content_html`, and `user_id`. |
| `models.Post.AttachRelations` | `featured_images` and `content_images` attachMany. |
| `models.Post.SearchableAs` | Index name `golem15_journal_posts`. |
| `models.Post.ShouldBeSearchable` | False when unpublished or `search_use_typesense` is off. |
| `models.Post.ToSearchableArray` | Typesense document built after a committed save. |
| `models.Post.CanEdit` | Owner or `golem15.journal.access_other_posts`. |
| `models.Category` | Category row; `TableName` is `golem15_journal_categories`; `Content` holds the HTML body served as `content`. |
| `models.Category.MorphName` | PHP class string `Golem15\Journal\Models\Category`. |
| `models.Category.Fillable` | `name`, `slug`, `code`, `description`, `content`, `parent_id`. |
| `models.Tag` | Tag row; `TableName` is `golem15_journal_tags`. |
| `models.Tag.Fillable` | `name`, `slug`, `description`. |
| `models.Settings` | Singleton settings row; `TableName` is `golem15_journal_settings`. |
| `models.All` | Models registered by this plugin. |
| `updates.All` | gormigrate steps registered by this plugin. |
| `classes.BlogEditorTypes` | The `blogeditor` and `mlblogeditor` field types: Decode through `SanitizeHTMLReport` with removal notices, Load with the markdown fallback, list text and the image upload. |
| `classes.BlogEditorElement` | The editor's custom element tag, `golem15-journal-blogeditor`. |
| `classes.LooksLikeHTML` | Whether stored content is HTML: its first non-space character is `<`. |
| `classes.StoreMedia` | Stores an uploaded image under `/journal/<folder>/<name>` and returns its `StoredMedia`; shared by the public upload route and the editor upload. |
| `classes.StoredMedia` | A stored media object's public `URL` and `Path`. |
| `classes.MediaError` | An upload `StoreMedia` refuses, with the request `Field` and the public route's `Message`. |
| `classes.MediaMaxBytes` | The upload size limit, 10240 KiB. |
| `classes.MediaFolderPattern` | The characters an upload folder may hold. |
| `classes.SanitizeHTML` | Strips from HTML (not markdown) what the `FormatHTML` allow-list does not allow; HTML the policy keeps whole is returned byte for byte. The string half of `SanitizeHTMLReport`. |
| `classes.SanitizeHTMLReport` | `SanitizeHTML` plus an `HTMLRemovals` report of what the policy removed; the report is empty exactly when the input comes back unchanged. |
| `classes.HTMLRemovals` | Removed element and attribute names (distinct, first seen, at most ten each) with exact `ElementCount` and `AttributeCount`, removed `Comments`, and `Other` for any other change; `Empty` reports no removal. |
| `classes.FormatTranslation` | Sets `content_html` in one locale's translated attributes: rendered from a non-blank `content` with `FormatHTML`, otherwise a supplied `content_html` sanitized with `SanitizeHTML`. |
| `classes.CheckHTML` | Predicate: an error when the `FormatHTML` allow-list would strip or change the HTML, including any tag that repeats an attribute. No write path refuses with it any more (admin saves strip and report); it is the counterpart of the site layer's gate and is used by the policy tests. |

## CLI commands

Both commands write JSON files (one per post slug), matching the PHP `journal:export-posts` / `journal:import-posts` tools. They are not a CSV ImportExport framework. A blog application binary registers them through `Plugin.Commands`.

| Command | Arguments and flags | Effect |
|---------|---------------------|--------|
| `journal:export-posts` | `--path` (default `journal-export`), `--dry-run` | Writes one JSON file per post, including non-default-locale translations. Attachments are omitted. |
| `journal:import-posts` | `--path` (default `journal-export`), `--force`, `--dry-run` | Creates posts from those files. Existing slugs are skipped unless `--force` updates them. |

```sh
./bin/acme journal:export-posts --path journal-export
./bin/acme journal:import-posts --path journal-export --dry-run
./bin/acme journal:import-posts --path journal-export --force
```

## Dependencies

- SummerCMS modules: backpack, beachcomber, bonfire, cabana, lagoon/attach, pact, party, surf.
- Compiled plugin: `golem15.translate`.
- Third-party: `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`, `github.com/yuin/goldmark`, `github.com/microcosm-cc/bluemonday`, `golang.org/x/net/html`, `gocloud.dev`. Test-only: `github.com/testcontainers/testcontainers-go`.
- Blog Editor JS build (not needed by a host application): `@tiptap/core`, `@tiptap/pm`, `@tiptap/starter-kit`, `@tiptap/extension-table` and `@tiptap/extension-image` 3.31.4, bundled into `assets/dist`; dev-only `vite`, `vitest`, `happy-dom` and `typescript`, pinned exactly in `package.json`.

## Testing

```sh
go test ./...
```

Postgres integration smokes start Docker unless `-short`. Missing Docker is a failure, not a skip, unless `-short`.

The Blog Editor has its own suite and drift gate:

```sh
npm test
bash scripts/check-dist.sh
```