golem15/summercms
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(phase-16.1): add research and validation strategy

Browse Source
This commit is contained in:
Jakub Zych
2026-10-09 20:19:59 +02:00
parent 1ef21b0e43
commit 17ee383a4f
2 changed files with 755 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

677
.planning/phases/16.1-journal-admin-urgent-fixes-relation-search-list-state-post-f/16.1-RESEARCH.md Normal file
View File

@@ -0,0 +1,677 @@
# Phase 16.1: Journal admin urgent fixes: relation search, list state, post form content and excerpt - Research
**Researched:** 2026-10-09
**Domain:** summercms.go `modules/cabana` (GORM list queries, admin JSON API, Vue admin SPA), `sm-journal-plugin` (post save path), UAT release
**Confidence:** HIGH for the three root causes (each reproduced by a probe or read from the UAT database); MEDIUM for the list-state design (Claude's discretion area)
<user_constraints>
## User Constraints (from CONTEXT.md)
### Locked Decisions
- **D-01: List state lives in both the URL and the server session.** Sort, filters, search and page are reflected in the list URL query string (shareable, bookmarkable, back-button friendly). When the list is opened without list-state query parameters (for example from the admin navigation), the last state stored in the user's session for that list is restored. When the URL carries list-state parameters, the URL wins and also updates the session. User's words: "I may go back to Posts list through navigation and it's great if my filters were remembered + sharable URLs are nice to have."
- **D-02: The relation-search fix is a framework fix in summercms.go `modules/cabana`**, generic for belongsToMany (pivot) relations, not a Journal-specific workaround. Another session is concurrently working in `modules/`; plans must keep cabana changes in their own commits, stage files explicitly by path, and check for conflicting in-flight edits before touching shared cabana files.
- **D-03: Content and excerpt must show the stored values without loss.** Root cause is to be established by research (suspects: translatable-field overlay replacing host values with empty translations; the markdown widget not displaying imported HTML). Saving a post from the admin must not destroy imported HTML (see todo `journal-imported-html-admin-resave.md`); if a full fix needs the Blog Editor, 16.1 at least prevents data loss and 16.2 delivers the editor.
- **D-04: Ship to UAT.** Release a summercms tag carrying the cabana fixes (unless the concurrent session's release already includes them), bump `sm-grzybyfunkcjonalne-app`, redeploy to uat.grzybyfunkcjonalne.pl per its DEPLOY.md, and re-run `scripts/smoke.sh --url` on the server.
### Claude's Discretion
- Session storage mechanism and query-parameter naming for list state, following existing cabana conventions.
- Test strategy, within the lean-mode rule that unit tests are the last plan of the phase.
### Deferred Ideas (OUT OF SCOPE)
- Plugin-provided field types and the Blog Editor formwidget (Phase 16.2). (CONTEXT.md has no separate Deferred section; this is its "Out of scope" line.)
</user_constraints>
<phase_requirements>
## Phase Requirements
No requirement IDs are mapped. The CONTEXT decisions are the requirement set.
| ID | Description | Research Support |
|----|-------------|------------------|
| D-01 | List sort, filters, search and page in the URL and a per-user server store; restore when the list opens without params | "List state" pattern: the URL params already exist; add the `backend_list_states` table, a `GET .../list-state` restore route, a save inside the list handler, and the ListView entry logic |
| D-02 | Generic cabana fix for search over belongsToMany (and sort over to-many) relations | Root cause reproduced: `db.Joins("Categories")` on a many2many builds `"Categories"."post_id"`. Fix: correlated `EXISTS` subqueries built from GORM's parsed relationship |
| D-03 | Post form shows the stored content and excerpt, and a save never destroys imported HTML | Root cause 1: the first enabled locale is `en`, and no `en` translations exist. Fix: default locale first in `meta.enabledLocales`. Root cause 2: `FormatHTML` drops raw HTML. Fix: `html.WithUnsafe()` plus the existing gate. 2 UAT posts already damaged; repair them after the deploy |
| D-04 | Tag, bump, redeploy, smoke | Release mechanics: the host builds against the `../summercms.go` working tree through a replace. The bump is the journal submodule pointer. The new migration needs `bin/grzybyfunkcjonalne migrate` on the server |
</phase_requirements>
## Project Constraints (from CLAUDE.md)
- Go 1.27, standard library first; add a dependency only when the research doc or a phase decision names it. **This phase adds none.** `github.com/yuin/goldmark/renderer/html` is a sub-package of goldmark v1.8.6, which sm-journal-plugin already requires.
- `go vet` and `go test ./...` must be green at every commit.
- Compiled plugins only; no runtime plugin loading.
- API parity: do not "improve" Journal API response shapes. The `content`/`content_html` contract stays unchanged (todo file).
- Unit tests are always the **last plan** of the phase. Before writing plans, present the plan count with one-line scopes (checkpoint).
- Commits:
- no co-author tags;
- one logical change per commit;
- planning docs and code in separate commits.
- D-02 adds three rules for this phase:
- cabana changes go in their own commits;
- stage files by explicit path;
- check for in-flight edits before touching shared cabana files.
- A change to a `modules/` package's exported API, config keys, CLI commands or dependencies updates that module's `README.md` in the same change. It also updates the affected `docs/` pages (`docs/backend/lists-and-filters.md`). `go test ./cmd/summer -run TestDocsTree` and `summer docs:build --check` must pass. Every identifier named in docs must exist.
- Framework READMEs never name a consuming application. Use `blog`/`acme`, not Journal or grzybyfunkcjonalne, in cabana or lagoon docs.
- SPA changes: rebuild `modules/boardwalk/dist` (`npm --prefix admin run build`) and commit it (`scripts/check-admin-dist.sh` is the drift gate). Swag annotation changes: regenerate `admin/openapi/admin.json` and `admin/src/api/schema.d.ts` (`scripts/check-admin-openapi.sh`, `--check` to verify).
- The PHP originals are not changed. Core plugin contracts (user, blog, pages, payment) stay unchanged.
## Summary
Each of the four UAT defects has one confirmed cause.
**(1) Relation search fails.** `applyListSearch` joins every searched relation with `db.Joins(field)` [VERIFIED: modules/cabana/query.go:351-357]. GORM's association `Joins` only fits to-one relations. On a many2many it emits `LEFT JOIN "golem15_journal_categories" "Categories" ON "golem15_journal_posts"."id" = "Categories"."post_id" AND "golem15_journal_posts"."category_id" = "Categories"."id"`, which is the UAT error, reproduced this session with a GORM v1.31.2 DryRun probe.
`applyListOrder` has the same bug for sorting by a to-many column (query.go:309). The code that ran on UAT is v0.1.15. Since v0.1.9, query.go only gained the to-many display join (`relatedSelectMany`). The bug is unchanged at HEAD.
The fix that matches Winter is a correlated `EXISTS` subquery per relation search term. Winter's Lists widget uses `orWhereHas` for search [VERIFIED: examples/golem15-wintercms-starter/modules/backend/widgets/Lists.php:529-531]. Build the subquery from GORM's own parsed relationship (`schema.Relationship.References` / `JoinTable`), not from guessed names.
**(2) List state is lost.** The list state already lives in the URL query (`search`, `sort`, `dir`, `page`, `per_page`, `filter[<name>]`) [VERIFIED: admin/src/app/listQuery.ts; modules/cabana/query.go:62-84]. The form's back link, close and save redirects, and the navigation all go to the bare list path (`go(listPath)`, FormView.vue:60/403/439), so the state is dropped.
There is no server session: admin auth is a stateless JWT cookie whose jti rotates on refresh. The table that would play this role in Winter, `backend_user_preferences`, was explicitly not ported in Phase 9; a test asserts it is absent. Recommendation: a small framework table `backend_list_states` keyed (user_id, controller), a `GET .../list-state` restore route, and a best-effort save inside the existing list handler after a successful query. ListView restores before its first load whenever it is entered with no list params.
**(3)+(4) The form shows empty content and excerpt.** The data is fine. On UAT, `golem15_translate_locales` has `en|f|t|1` and `pl|t|t|2`: en is enabled, sorted first, and not the default. There are **0** rows in `golem15_translate_attributes`. `EnabledLocales` returns codes `ORDER BY sort_order` → `["en","pl"]`. Every ML control opens on `locales[0]`, which is `en`, so the form shows the empty en values of title, slug, content and excerpt. The record heading hides this for the title, because it falls back to the first non-empty locale.
Fix in cabana: send the default locale first in `meta.enabledLocales`. This one server function fixes the form, the preview screen and relation child modals.
Separately, **saving destroys imported HTML**. `preparePostWrite` runs `classes.FormatHTML(post.Content)`, which is goldmark *without* `html.WithUnsafe()`. Probe this session: it turns all 100 imported pl bodies into `<!-- raw HTML omitted -->`. With `html.WithUnsafe()` it returns all 100 unchanged (trim-equal), and the existing 3-pattern gate still rejects script, iframe, event handlers and javascript/data URLs. This is PHP parity: Winter's `Markdown::parse` passes raw HTML through.
UAT already has **2 posts whose `content_html` was wiped at import**. They have no pl translation and raw HTML in the base content:
- id 53 `czy-amanita-muscaria-jest-legalna` (draft);
- id 64 `ekstrakt-z-turkey-tail` (**published**, missing its author footer).
Re-render both after the fix ships.
**Primary recommendation:** five changes, then release v0.1.16 and redeploy:
1. cabana: relation search and sort through `EXISTS` and correlated subqueries built from `gorm.Statement.Parse` relationships.
2. cabana: default locale first in `enabledLocales`.
3. cabana and SPA: list-state table, restore route and ListView entry logic.
4. sm-journal-plugin: `FormatHTML` with `html.WithUnsafe()` behind the existing gate.
5. After the deploy: re-save posts 53 and 64.
## Architectural Responsibility Map
| Capability | Primary Tier | Secondary Tier | Rationale |
|------------|-------------|----------------|-----------|
| Relation search/sort SQL | API / Backend (cabana `query.go`) | Database (Postgres EXISTS) | Allowlisted SQL is built only server-side; the SPA sends only names |
| List state persistence | API / Backend (cabana list handler + new route) | Database (`backend_list_states`) + Browser (ListView entry logic, URL) | D-01 asks for server storage; the URL stays the shareable source |
| Restore-on-entry decision | Browser (ListView) | API (restore route sanitizes) | Only the SPA knows whether the list was entered from navigation or changed by the user |
| ML default locale ordering | API / Backend (cabana `enabledContentLocales`) | Browser (MLField uses `[0]`) | A single server ordering fixes every SPA consumer of `meta.enabledLocales` |
| content → content_html rendering | API / Backend (sm-journal-plugin `classes.FormatHTML`) | — | Shared by admin save, import and Journal API writes |
| Release / deploy | Host repo + server ops | — | The host compiles the framework working tree through a replace |
## Standard Stack
### Core (all already in go.mod / package.json; versions read this session)
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| gorm.io/gorm | v1.31.2 | `gorm.Statement.Parse` → `schema.Relationship` (Type, References, JoinTable, FieldSchema) for the EXISTS builder | Already the ORM; its parsed relationship is the only reliable source of pivot/fk names [VERIFIED: probe output below] |
| gorm.io/gorm/schema | v1.31.2 | `schema.Many2Many` ("many_to_many"), `schema.HasMany` ("has_many"), `schema.BelongsTo`, `schema.HasOne` | Relation type switch [VERIFIED: probe printed `type: many_to_many`, `has_many`] |
| github.com/yuin/goldmark | v1.8.6 (sm-journal-plugin go.mod) | `goldmark.WithRendererOptions(html.WithUnsafe())` from `github.com/yuin/goldmark/renderer/html` | Same module, new import only [VERIFIED: probe compiled against the cached v1.8.6] |
| gormigrate/v2 | v2.1.7 | New migration appended to `lagoon.BackendAdminMigrations` | Existing framework migration list (`summercms.cabana` history) [VERIFIED: modules/lagoon/backend_admin_migrations.go:21, migrations.go:75] |
| vue-router | 5.1.0 | `router.replace` for restore, as ListView already does | Existing [VERIFIED: admin/package.json] |
### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| EXISTS per relation | `Joins` + `DISTINCT` / `GROUP BY` | Duplicate rows inflate `Count`, unqualified outer predicates (`id IN (...)` in Journal `FilterCategories`, `user_id = ?` in `restrictToOwnPosts`) become ambiguous; not Winter parity |
| `backend_list_states` table | `backend_users.metadata` JSON key | metadata is Winter's TEXT jsonable copied at cutover (202610090001); read-modify-write races between tabs; pollutes a copied column |
| `backend_list_states` table | Winter `backend_user_preferences` | Phase 9 D-01 left it unported and `backend_admin_migrations_test.go:116` asserts it is absent; reviving it reverses a decision and its non-unique `user_item_index` cannot back an upsert |
| Server-side table | Browser `sessionStorage` | Contradicts D-01 ("server session") |
| Default-locale-first in cabana | Reorder in sm-translate-plugin `AdminWriter.EnabledLocales` | Plugin-specific; any other TranslationWriter would keep the bug |
| Default-locale-first | New `meta.defaultLocale` + SPA changes | More SPA surface and OpenAPI change; same user-visible result. Dropdown order changes from sort_order to default-first (cosmetic) |
| `html.WithUnsafe()` | Detect HTML (`^\s*<`) and store as-is through `CheckHTML` | Heuristic, and post 64 is markdown with trailing raw HTML (would still be wiped); not PHP parity |
**Installation:** none, because no new modules are added.
## Package Legitimacy Audit
No external packages are installed. `github.com/yuin/goldmark/renderer/html` is a package inside the already-required `github.com/yuin/goldmark v1.8.6`. No npm packages are added.
**Packages removed due to [SLOP] verdict:** none
**Packages flagged as suspicious [SUS]:** none
## Root-cause evidence
### Defect 1: relation search (D-02)
The search code [VERIFIED: modules/cabana/query.go:345-357, read this session]:
```go
field, ok := relationFieldName(model, col.Relation)
physical := listRelationColumn(controller, col.Relation, col.Select)
...
// GORM aliases the joined association with the Go field name.
table = field
column = physical
if _, done := joined[field]; !done {
db = db.Joins(field)
```
The sort code does the same, `db = db.Joins(field)` [VERIFIED: query.go:309].
The Journal model declares the relation as `Categories []Category \`gorm:"many2many:golem15_journal_posts_categories;" json:"-"\`` [VERIFIED: sm-journal-plugin/models/post.go:34]. The posts list column is:
```yaml
categories:
label: golem15.journal::lang.post.categories
relation: categories
select: name
searchable: true
sortable: false
```
[VERIFIED: sm-journal-plugin/models/post/columns.yaml]
GORM v1.31.2 DryRun probe (scratch module, same model shape), output pasted:
```
JOINS: SELECT count(*) FROM "golem15_journal_posts" LEFT JOIN "golem15_journal_categories" "Categories" ON "golem15_journal_posts"."id" = "Categories"."post_id" AND "golem15_journal_posts"."category_id" = "Categories"."id" WHERE LOWER(CAST("Categories"."name" AS TEXT)) LIKE '%x%'
type: many_to_many jointable: golem15_journal_posts_categories
ref: fk=golem15_journal_posts_categories.post_id pk=golem15_journal_posts.id ownPrimaryKey=true pvalue=""
ref: fk=golem15_journal_posts_categories.category_id pk=golem15_journal_categories.id ownPrimaryKey=false pvalue=""
hasmany type: has_many field schema: comments
ref: fk=comments.post_id pk=golem15_journal_posts.id own=true
```
The error happens at `query.go:129`, the `Count`, which is the first statement executed.
The log names `summercms@v0.1.9`, but that is only the version sm-journal-plugin's go.mod *requires* (`git.golem15.com/golem15/summercms v0.1.9`). The code that ran is the `../summercms.go` working tree at 296b9cd, which descends from v0.1.15 with docs-only changes since. At HEAD, query.go differs from v0.1.9 only by `relatedSelectMany` (display), so **the bug is still present at HEAD**.
There are no uncommitted `modules/` edits right now. The checkout is 13 docs commits ahead of origin/master.
### Defect 2: list state (D-01)
- **URL params already exist.** `listQueryFromRequest` reads `search`, `sort`, `dir`, `page`, `per_page` and `filter[<name>]` (`listSearchTerm = "search"` [VERIFIED: modules/cabana/list_schema.go:20]; query.go:62-84). The SPA mirrors them in `parseListQuery`/`toListQuery` [VERIFIED: admin/src/app/listQuery.ts], and ListView changes them with `router.replace` (ListView.vue:114-116).
- **Where the state is dropped.**
- FormView: `const listPath = controllerPath(controllerId) ?? '/'` (FormView.vue:60), `await go(listPath)` after save (403), `await go(backPath.value)` for back (439).
- The navigation and breadcrumb links use the bare controller path.
- Browser Back works today. Every in-app return loses the state.
- **ListView start-up.** It loads through `watch(() => listQueryKey(query.value), ..., { immediate: true })` (ListView.vue:146-153). The first request is fired at setup, before any restore could run.
- **No server session.**
- Login mints a JWT into an HttpOnly cookie (`auth.go` `writeSessionCookie`).
- Refresh rotates the token and jti (`bouncer.RefreshAudienceFor`).
- Logout only blacklists the jti.
- Phase 9 left out Winter's preference table ("`backend_user_groups` and Winter's user-preference and access-log tables: not needed by any Golem15 plugin yet" [VERIFIED: .planning/phases/09-.../09-CONTEXT.md:126]), and `backend_admin_migrations_test.go:116` fails if `backend_user_preferences` exists.
- **What Winter does.** It keeps sort (`putSession('sort', ...)`), search term (`putSession('term', ...)`), filter scopes and `lastVisitedPage` in the session, and per_page in user preferences [VERIFIED: Lists.php:233,289,1773; Search.php:158; Filter.php:959].
### Defects 3 and 4: empty content and excerpt (D-03)
Read-only UAT probe (ssh `grzybyfunkcjonalne-uat`, psql over the peer socket, SELECT only), output pasted:
```
code|is_default|is_enabled|sort_order
en|f|t|1
pl|t|t|2
locale|model_type|count
(0 rows)
posts|content|html|excerpt|title|content_starts_html
104|104|104|100|104|101
```
- **Where the order comes from.** The translate seed inserts `SELECT 'en', 'English', TRUE, TRUE, 1` and `SELECT 'pl', 'Polski', FALSE, TRUE, 2` [VERIFIED: sm-translate-plugin/updates/202610060005_seed_en_pl_locales.go:13-18]. migrate.sh only runs `UPDATE golem15_translate_locales SET is_default = (code = 'pl')` (scripts/migrate.sh:173), so en stays enabled and first.
- **The writer keeps that order.** `AdminWriter.EnabledLocales` runs `Where("is_enabled = ?", true).Order("sort_order asc")` [VERIFIED: sm-translate-plugin/classes/admin_writer.go:31-35], and cabana copies the result into the form schema (`view.Meta.EnabledLocales = append([]string(nil), codes...)`, http.go:727-729).
- **The server sends the right data.** `hydrateMLRecord` puts the host column under the default locale (`if code == defaultLocale { out[code] = host`, field_ml.go:208-211). An en key with no stored translation is `out[code] = ""` (field_ml.go:217-219).
- **The SPA opens on the wrong locale.**
- `const activeLocale = ref(locales.value[0] ?? 'en')` (MLField.vue:30).
- `function firstMLLocale(): string { return schemaEnabledLocales(schema.value?.meta)[0] ?? 'en' }` (FormView.vue:227-229).
- So every ML control (title, slug, content and excerpt) opens on `en`, which is empty. The user reported only content and excerpt, but title and slug are empty in the inputs as well.
**Data loss on save.**
- The save path: `html, err := classes.FormatHTML(post.Content)` … `post.ContentHTML = html` (sm-journal-plugin/controllers/posts.go:213-219). The engine is `goldmark.New(goldmark.WithExtensions(extension.Footnote, extension.Table), goldmark.WithParserOptions(parser.WithAttribute()))`, with no unsafe renderer (classes/format_html.go:15-18).
- PHP parity: `formatHtml` stores `trim($input)` when `use_rich_editor` is on, otherwise `Markdown::enableFootnotes()->enableAttributes()->enableTables()->parse(trim($input))`, then `Html::clean` [VERIFIED: grzybyfunkcjonalne.pl/plugins/golem15/journal/models/Post.php:185-208]. Winter's Markdown passes raw HTML through [ASSUMED: Parsedown default, not re-read this session; consistent with the export, where 83/100 pl `content_html` equal `content` byte for byte and the other 17 differ only by `&quot;` → `"`].
- Probe over the 100 pl bodies of the journal export (goldmark v1.8.6, the same options), output pasted:
```
total=100 safe_omitted=100 unsafe_exact=0 unsafe_trim_equal=100
sample safe="<!-- raw HTML omitted -->\n"
sample unsafe="<p>a</p>\n<p>b</p>"
md unsafe="<h1>T</h1>\n<p>text <b>x</b></p>\n<script>alert(1)</script>"
```
The last line shows why the existing `rejectUnsafe` gate must stay after the unsafe render. None of the 105 export bodies match any of the three gate patterns, so no imported post will be refused on save.
**Already damaged on UAT** (read-only probe):
```
wiped|edited_after_import|admins
2|0|1
id 53 czy-amanita-muscaria-jest-legalna published=f content_html = <!-- raw HTML omitted -->
id 64 ekstrakt-z-turkey-tail published=t content_html ends with 3x <!-- raw HTML omitted --> (author <footer> + <style>)
```
Both come from export files with `"translations": []`: base content containing raw HTML went through the safe `FormatHTML` at import (console/import_posts.go:185). Probe with the unsafe renderer:
- `czy-amanita-muscaria-jest-legalna`: gate passes, equal to the PHP `content_html` after whitespace normalization.
- `ekstrakt-z-turkey-tail`: gate passes, includes `<footer` and `<style>`, 5643 vs 5655 bytes against PHP.
## Architecture Patterns
### System Architecture Diagram
```
Admin SPA (ListView) cabana API (Go) Postgres
──────────────────── ─────────────── ────────
enter list route ──► query empty? ──yes──► GET .../list-state ──► protect() ─► load row(user,ctl)
│ │ sanitize vs compiled schema ─► backend_list_states
│ │◄──────── {search,sort,dir,page,per_page,filter} ◄──┘
│ ├─ non-empty ─► router.replace(query) ─┐
│ no / empty stored │
▼ ▼
user changes search/sort/filter/page ─► router.replace ─► GET /{v}/{p}/{c}?search&sort&dir&page&per_page&filter[x]
│
├─ ExecuteList: ListExtendQuery → applyListSearch
│ own column: LOWER(CAST(t.col AS TEXT)) LIKE ?
│ relation col: EXISTS (SELECT 1 FROM pivot p JOIN rel r ...
│ WHERE p.owner_fk = t.pk AND LOWER(...) LIKE ?)
│ → applyListFilters → Count → applyListOrder → Preload → Find
├─ on success: upsert state(user, controller) if changed (best effort)
└─► rows + meta
FormView (back / save / close) ─► go(listPath) (bare path) ─► ListView entry ─► restore path above
```
### Recommended file touch map
```
summercms.go/
├── modules/cabana/query.go # EXISTS search + to-many sort subquery (D-02)
├── modules/cabana/list_state.go # NEW: store/load/sanitize list state (D-01)
├── modules/cabana/http.go # list(): save after success; new GET .../list-state route; enabledContentLocales default-first (D-03)
├── modules/cabana/admin_openapi.go # swag doc for the new route
├── modules/lagoon/backend_admin_migrations.go # NEW migration backend_list_states
├── modules/cabana/README.md, modules/lagoon/README.md, docs/backend/lists-and-filters.md
├── admin/src/views/ListView.vue # entry restore, non-immediate watch, page clamp
├── admin/openapi/admin.json, admin/src/api/schema.d.ts # regenerated
└── modules/boardwalk/dist/ # rebuilt SPA
sm-journal-plugin/
├── classes/format_html.go # html.WithUnsafe() + existing gate
└── README.md (admin re-save note)
sm-grzybyfunkcjonalne-app/
├── plugins/golem15/journal (submodule pointer), README.md + DEPLOY.md (drop the "do not re-save" warning)
```
### Pattern 1: relation search through EXISTS (generic, Winter `orWhereHas` parity)
**What:** for each searchable column with `relation:`, emit one `EXISTS (...)` term inside the existing OR group instead of `db.Joins(field)`. Read the relationship from GORM's schema parse. Validate the selected column against the related schema.
**When to use:** every relation type in **search**. For **sort**, keep `Joins` for `belongs_to`/`has_one`, which works and is tested (`artist` column in query_test.go). Use a correlated scalar subquery for `has_many`/`many_to_many`.
GORM wraps an OR expression in parentheses when it is combined with other `Where`s. Verified by DryRun:
```
SELECT count(*) FROM "golem15_journal_posts" WHERE user_id = 3 AND (LOWER(CAST("golem15_journal_posts"."title" AS TEXT)) LIKE '%x%' ESCAPE '\' OR EXISTS (SELECT 1 FROM "golem15_journal_posts_categories" AS "p0" JOIN "golem15_journal_categories" AS "r0" ON "r0"."id" = "p0"."category_id" WHERE "p0"."post_id" = "golem15_journal_posts"."id" AND LOWER(CAST("r0"."name" AS TEXT)) LIKE '%x%' ESCAPE '\')) AND published = true
```
Skeleton, using cabana's existing `qualifiedColumn`, `quotedIdent`, `identifier`, `tableName` and `listInvalid` helpers. The relationship API names come from the probe; the shape is a sketch:
```go
// relationExists returns "EXISTS (...)" matching LOWER(CAST(rel.column AS TEXT)) LIKE ?
// for the relation behind Go field `field`, correlated to the main table.
func relationExists(db *gorm.DB, model any, field, column string, n int) (string, error) {
stmt := &gorm.Statement{DB: db}
if err := stmt.Parse(model); err != nil {
return "", err
}
rel, ok := stmt.Schema.Relationships.Relations[field]
if !ok || rel.FieldSchema == nil || rel.FieldSchema.LookUpField(column) == nil {
return "", listInvalid("search", "is not searchable")
}
main := tableName(model) // the unaliased outer FROM table
r := fmt.Sprintf("cabana_r%d", n) // alias: safe for self-referencing relations
match := fmt.Sprintf("LOWER(CAST(%s AS TEXT)) LIKE ? ESCAPE '\\'", qualifiedColumn(db, r, column))
var on, where []string
switch rel.Type {
case schema.Many2Many:
p := fmt.Sprintf("cabana_p%d", n)
for _, ref := range rel.References {
if ref.OwnPrimaryKey { // pivot.owner_fk = main.pk
where = append(where, qualifiedColumn(db, p, ref.ForeignKey.DBName)+" = "+qualifiedColumn(db, main, ref.PrimaryKey.DBName))
} else { // pivot.related_fk = related.pk
on = append(on, qualifiedColumn(db, p, ref.ForeignKey.DBName)+" = "+qualifiedColumn(db, r, ref.PrimaryKey.DBName))
}
}
return fmt.Sprintf("EXISTS (SELECT 1 FROM %s AS %s JOIN %s AS %s ON %s WHERE %s AND %s)",
quotedIdent(db, rel.JoinTable.Table), quotedIdent(db, p),
quotedIdent(db, rel.FieldSchema.Table), quotedIdent(db, r),
strings.Join(on, " AND "), strings.Join(where, " AND "), match), nil
case schema.HasMany, schema.HasOne: // related.fk = main.pk
for _, ref := range rel.References {
where = append(where, qualifiedColumn(db, r, ref.ForeignKey.DBName)+" = "+qualifiedColumn(db, main, ref.PrimaryKey.DBName))
}
case schema.BelongsTo: // related.pk = main.fk
for _, ref := range rel.References {
where = append(where, qualifiedColumn(db, r, ref.PrimaryKey.DBName)+" = "+qualifiedColumn(db, main, ref.ForeignKey.DBName))
}
default:
return "", listInvalid("search", "is not searchable")
}
return fmt.Sprintf("EXISTS (SELECT 1 FROM %s AS %s WHERE %s AND %s)",
quotedIdent(db, rel.FieldSchema.Table), quotedIdent(db, r), strings.Join(where, " AND "), match), nil
}
```
Notes for the planner:
- **Polymorphic references.** A reference with `ref.PrimaryValue != ""` needs an extra bound `= ?` condition. Refuse polymorphic relations with `listInvalid` in this phase unless a test covers them [ASSUMED: GORM polymorphic reference semantics, not probed].
- **Soft deletes.** If the related schema has a `gorm.DeletedAt` field, add `AND r.deleted_at IS NULL`. Winter's `whereHas` applies the related model's SoftDelete scope [ASSUMED].
- **Parse cache.** `stmt.Parse` uses `db.cacheStore`, so the parse is cached.
- **Sort for to-many.** Use `ORDER BY (SELECT string_agg(CAST(r.col AS TEXT), ', ') FROM ... correlated ...) ASC|DESC` with `clause.OrderBy{Expression: clause.Expr{SQL: ...}}`. This mirrors Winter, which sorts on the `group_concat(...)` sub-select [VERIFIED: Lists.php:568-570] and on the comma-joined value `relatedSelectMany` displays. The `sortable` default is true for relation columns (docs/backend/lists-and-filters.md:62), so any other plugin's to-many column crashes on sort today.
### Pattern 2: list state (D-01), recommended design
**Server side (cabana):**
1. **Migration.** Append to `lagoon.BackendAdminMigrations`, ID `"202610100001_backend_list_states"` (follow the `YYYYMMDDNNNN_` pattern of `202610090001_backend_users_metadata`):
```sql
CREATE TABLE IF NOT EXISTS backend_list_states (
user_id INTEGER NOT NULL REFERENCES backend_users(id) ON DELETE CASCADE,
controller TEXT NOT NULL,
state TEXT NOT NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (user_id, controller)
)
```
- `backend_users.id` is `SERIAL` [VERIFIED: backend_admin_migrations.go:36-37].
- Use `state TEXT`, matching the backend tables' TEXT-for-JSON convention (`metadata TEXT`, `permissions TEXT`).
- The table name avoids reviving Winter's `backend_user_preferences`, which the Phase 9 test prohibits.
2. **Store shape.** The same JSON shape as the SPA's `ListQuery`: `{"search":..., "sort":..., "dir":..., "page":..., "per_page":..., "filter":{...}}`. Write it with `INSERT ... ON CONFLICT (user_id, controller) DO UPDATE` (gorm `clause.OnConflict`). Skip the write when the stored state is equal. Cap the stored JSON size (for example 4 KiB) and drop oversize state.
3. **Save (URL wins).** In `service.list`, after `ExecuteList` returns no error, save the request's validated `ListInput` for `(principal.ID, controllerID(cc))`. A save failure is logged with `slog` and does not fail the list. A 422 or 500 list request never saves.
4. **Restore.**
- Route: `g.Get("/{vendor}/{plugin}/{controller}/list-state", s.listState)` + `constrainController(g)`, through `protect()`, so permission is checked before SQL.
- Response: `{data: <ListQuery>}`, `{}` when there is none.
- Sanitize the stored state against the *current* compiled schema before returning it, reusing `normalizePage`, `normalizeSort`, the filter allowlist and the `switchArgument`/`parseDateRange`/`scopeArgument` checks. Drop each invalid part; never 422. A stored filter that a later deploy removed must not break the list forever.
- Order: register the literal route before the `/{id}` show route. ServeMux would pick the literal anyway, but put it next to `schema/list` for readability.
5. **OpenAPI.**
- Add a swag block `AdminListState` for the new route.
- Add one sentence to `AdminList`'s `@Description` ("a successful list request stores its query as the administrator's list state").
- Regenerate with `scripts/check-admin-openapi.sh`.
6. **Logout.** Keep the state across logins by default (see Assumption A1). If the user wants Winter's session-only semantics, delete the user's rows in `service.logout`.
**SPA side (ListView.vue):**
1. **Entry.** Replace the `immediate: true` watch with an explicit entry routine. If `parseListQuery(route.query)` is empty, `await api.GET('/{vendor}/{plugin}/{controller}/list-state')`. If the result is non-empty, `router.replace({ path: route.path, query: toListQuery(state) })` and let the (non-immediate) watch load. Otherwise call `loadList()`. **Never fire an empty list request before the restore decision**: under "save on success", that request would overwrite the stored state with empty.
2. **Same-path re-entry.** ListView is keyed by route path, so clicking the nav item while already on the list with filters changes only the query to `{}`. Remember the key of the last query this view set itself in `replaceQuery`. When the watch sees an empty query that the view did not set, run the entry restore instead of loading empty.
3. **User clears everything.** `replaceQuery` produces `{}`. The key matches the view's own → `loadList()` with no params → the server saves the empty state. Clearing works with no marker param.
4. **Page out of range** after restore (records deleted): if `meta.last_page >= 1 && meta.page > meta.last_page`, `replaceQuery({ ...query, page: meta.last_page })`.
5. **FormView needs no change.** Its bare `listPath` navigation now restores.
### Pattern 3: ML default locale first (D-03)
In `service.enabledContentLocales` (modules/cabana/http.go:966-983), call `writer.DefaultLocale(ctx, db)` and move that code to index 0, keeping the remaining codes in sort_order. If `DefaultLocale` errors, keep the order and log. This changes three things:
- MLField's initial `activeLocale`;
- FormView's `firstMLLocale()`;
- `mergeMLValue`'s fallback for a bare host string, and `recordDisplayName`'s preferred locale. These read `enabledLocales[0]` (formState.ts:205-218, 225-247), so they become the default locale too.
`liftMLValues`/`hydrateMLRecord` call `writer.EnabledLocales` directly and do not depend on order.
### Pattern 4: lossless content_html (D-03, sm-journal-plugin)
```go
import "github.com/yuin/goldmark/renderer/html"
markdownEngine = goldmark.New(
goldmark.WithExtensions(extension.Footnote, extension.Table),
goldmark.WithParserOptions(parser.WithAttribute()),
goldmark.WithRendererOptions(html.WithUnsafe()), // Winter Markdown::parse passes raw HTML; rejectUnsafe still gates it
)
```
- `FormatHTML` keeps `rejectUnsafe(html)` after the render, unchanged. `CheckHTML` is unchanged.
- The one change covers all three write paths: admin save (`preparePostWrite`), import (`console/import_posts.go:185`) and the Journal API writes (`controllers/api/posts_helpers.go:511,567,768`).
- The response shapes are unchanged (API parity).
- Do **not** change `cabana.RenderMarkdown` (the admin preview, `field_markdown.go:21-31`). Its output goes into the SPA via `v-html`, and its "no unsafe HTML" rule is a framework security choice. The HTML preview gap belongs to 16.2's Blog Editor.
### Anti-Patterns to Avoid
- **`Joins(field)` for to-many relations.** It builds a broken ON clause for many2many and duplicates rows for has_many.
- **Guessing pivot or foreign-key names** (`<model>_id`). Read them from `schema.Relationship`. The README says "Framework code never guesses table, pivot or foreign-key names."
- **Saving list state on the SPA's first empty request.** It wipes the stored state (see Pattern 2, SPA step 1).
- **Restoring stored state without re-validation.** A stale filter turns into a permanent 422.
- **Fixing the locale bug by disabling `en` in the database only.** That fixes one site, not the framework. It may still be wanted for grzybyfunkcjonalne (Open Question 2).
- **An HTML detection heuristic instead of the unsafe render.** Post 64 is markdown with trailing raw HTML.
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Pivot, foreign-key and table names of a relation | String conventions | `gorm.Statement{DB: db}.Parse(model)` → `Schema.Relationships.Relations[field]` | Honors custom `joinForeignKey`/`foreignKey` tags and the naming strategy |
| Identifier quoting | `"` concatenation | existing `quotedIdent` / `qualifiedColumn` (query.go:486-497) | Already used by the search SQL |
| Upsert | SELECT then INSERT/UPDATE | `clause.OnConflict{Columns: ..., DoUpdates: ...}` | Race-free across two tabs |
| HTML pass-through | Regex HTML detector | `html.WithUnsafe()` + existing `rejectUnsafe` | PHP parity; verified on 100/100 bodies |
| List param parsing in the SPA | New parser | existing `parseListQuery` / `toListQuery` / `listQueryKey` | Already strict about malformed input |
## Runtime State Inventory (deploy and data, not a rename)
| Category | Items Found | Action Required |
|----------|-------------|------------------|
| Stored data | UAT posts 53 and 64: `content_html` holds `<!-- raw HTML omitted -->` from the import. 0 translate attribute rows. `en` locale enabled with sort_order 1 | After deploy: re-render posts 53 and 64 by re-saving them in `/gfadmin` (this is also D-03's acceptance test), then confirm `content_html NOT LIKE '%raw HTML omitted%'` for all posts. Deciding whether `en` should be disabled is Open Question 2 |
| Live service config | None. nginx vhost and supervisor programs unchanged | None |
| OS-registered state | supervisor programs `grzybyfunkcjonalne-go`/`-nuxt` (names unchanged) | `sudo supervisorctl restart grzybyfunkcjonalne-go` (Nuxt restart optional) |
| Secrets/env vars | None changed | None |
| Build artifacts | `bin/grzybyfunkcjonalne` (sha256 94116f4a…68ea from 296b9cd); `modules/boardwalk/dist` embedded SPA | Rebuild with `scripts/build.sh` from a framework checkout at the release tag; rebuild and commit dist before the tag |
| Schema | New migration `backend_list_states` | On the server: `bin/grzybyfunkcjonalne migrate` (DEPLOY.md Release step 3) **before** the restart |
## Release / deploy mechanics (D-04)
1. **Framework tag.**
- Tags are annotated (`git for-each-ref` shows `tag` objects), cut on master after the work, and pushed to `origin` (`git@git.golem15.com:golem15/summercms.git`).
- The latest tag is `v0.1.15` (2026-10-09). Next: `v0.1.16`, unless the concurrent session tags first; then take the next free number and check `git tag --contains <fix commit>`.
- In Phase 14.3 tagging, the site bump and the deploy were user actions, so make the tag and push a `checkpoint:human-action` or explicit approval.
- Before tagging: `scripts/check-admin-dist.sh`, `scripts/check-admin-openapi.sh --check`, `go vet ./...`, `go test ./...` (needs Docker, see Environment), `go test ./cmd/summer -run TestDocsTree`.
2. **sm-journal-plugin.**
- Commit the FormatHTML change on `main`.
- Set `require git.golem15.com/golem15/summercms v0.1.16` in go.mod. It currently reads `v0.1.9`; there is a `replace ... => ../summercms.go`, so this is a nominal pin and `go mod tidy` works offline. Precedent: sm-translate-plugin's "chore(deps): require summercms v0.1.10".
- Push `origin main`.
3. **Host (`sm-grzybyfunkcjonalne-app`).**
- `go.mod` has `replace git.golem15.com/golem15/summercms => ../summercms.go` and `require ... summercms v0.0.0`, so **there is no framework version to bump**. `scripts/build.sh` compiles the `../summercms.go` working tree, so check out the tag or a clean master there before building.
- The bump is the submodule pointer: `git -C plugins/golem15/journal fetch && git -C plugins/golem15/journal checkout <new main>`, then commit `plugins/golem15/journal` (current pointer 3d533d4).
- Also update README.md:114 and DEPLOY.md "Do not re-save imported posts in /gfadmin". Push origin master.
4. **Deploy** (DEPLOY.md "Release"):
- `scripts/build.sh` and `scripts/check-deploy.sh`.
- rsync the binary to `bin/grzybyfunkcjonalne.new`, then `cp -p bin/grzybyfunkcjonalne bin/grzybyfunkcjonalne.prev && mv bin/grzybyfunkcjonalne.new bin/grzybyfunkcjonalne`.
- rsync `config/` and `scripts/`. Nuxt `.output` is unchanged unless rebuilt.
- On the server in `/srv/grzybyfunkcjonalne/go`: `bin/grzybyfunkcjonalne migrate`, then `sudo supervisorctl restart grzybyfunkcjonalne-go grzybyfunkcjonalne-nuxt`.
- Compare the uploaded binary's sha256 with the local build.
5. **Smoke on the server** (UAT basic auth; loopback bypass):
```sh
C=$(mktemp -d); printf 'resolve = uat.grzybyfunkcjonalne.pl:443:127.0.0.1\n' > "$C/.curlrc"
cd /srv/grzybyfunkcjonalne/go && CURL_HOME="$C" scripts/smoke.sh --url https://uat.grzybyfunkcjonalne.pl
rm -rf "$C"
```
Expect `smoke: ok (98 urls)`. smoke.sh does not cover the admin.
6. **Admin verification** (human, UAT credentials + admin login): see Validation Architecture → Manual. The UAT gate answers 401 off-server, so automated admin checks from the executor machine are impossible without credentials.
7. **Rollback:** move `bin/grzybyfunkcjonalne.prev` back and restart. The new table is additive; the migration Rollback drops it.
## Common Pitfalls
### Pitfall 1: Ambiguous columns once anything joins
**What goes wrong:** Journal's `FilterCategories` uses `db.Where("id IN (SELECT post_id FROM golem15_journal_posts_categories WHERE category_id IN (?))", ids)` and `restrictToOwnPosts` uses `db.Where("user_id = ?", principal.ID)`. Both are unqualified [VERIFIED: models/post.go:107; controllers/posts.go:95]. Any `JOIN` of a table with `id`/`user_id` makes them ambiguous.
**How to avoid:** EXISTS subqueries only, with no joins in the outer query for search. Test "search + category filter + non-superuser" together.
### Pitfall 2: The first list request clobbers the stored state
**What goes wrong:** ListView's `immediate: true` watch fires an empty request at setup. With save-on-success, that request overwrites the stored state before the restore runs.
**How to avoid:** the entry routine decides on restore before any list request. Add a vitest for this: on mount with an empty query, the `list-state` request comes before any list request, and no empty list request is made when the stored state is non-empty.
### Pitfall 3: Stale stored state breaks the list permanently
**What goes wrong:** A filter is renamed in `config_filter.yaml`, the stored state names the old filter, the restore sets it in the URL, and the list answers 422 on every visit.
**How to avoid:** the restore route sanitizes against the current compiled schema and drops invalid parts.
### Pitfall 4: The default locale is not first
**What goes wrong:** Any site where the seeded `en` stays enabled with a lower `sort_order` than the default shows empty ML fields.
**How to avoid:** order in cabana (Pattern 3), and test with a writer that returns `["en","pl"]` and default `pl`. The SPA fixtures use `['en', 'pl']` (MLFields.test.ts:18, FormView.test.ts:560); add a case where the default is not first.
### Pitfall 5: The unsafe render widens what reaches `content_html`
**What goes wrong:** `<style>`, `<footer>` and arbitrary tags now pass through. The gate blocks only `<script|iframe|object|embed`, `on*=` handlers and `javascript:|vbscript:|data:`. PHP additionally ran `Html::clean` for admins without `backend.allow_unsafe_markdown`.
**How to avoid:** keep the gate. Post authors are authenticated admins with `golem15.journal.access_posts`. Note it as an accepted risk in the security review. Re-run `classes` tests: every `TestFormatHTMLRejectsUnsafeHTML` case still errors through the gate.
**Warning sign:** a body containing the Polish word "data:" (for example "Data: 12.03") would be refused with 422. None of the 105 export bodies match today.
### Pitfall 6: The version in the error log misleads
**What goes wrong:** The Go log reports `summercms@v0.1.9`, which is sm-journal-plugin's go.mod require. The binary was built from the working tree at 296b9cd (v0.1.15 code). Do not diff against v0.1.9 to decide what UAT runs.
### Pitfall 7: Docker-only tests cannot run on this machine
**What goes wrong:** `go test ./modules/cabana/ -run TestListQueryContract` fails with `permission denied while trying to connect to the docker API` (run this session). sm-journal-plugin's TestMain is fail-closed too.
**How to avoid:** put the new SQL-shape tests in pure DryRun tests (no DB) that run anywhere. Keep the Postgres tests as integration evidence on a Docker-enabled machine (user / CI). Phase 16-05 already records the same constraint.
### Pitfall 8: Concurrent `modules/` session
**What goes wrong:** The other session edits cabana or lagoon files in parallel and creates a merge conflict or a mixed commit.
**How to avoid:** before each cabana or lagoon task, run `git status --porcelain -- modules admin` and `git log origin/master..` / `git fetch`. Stage by explicit path. Keep one logical change per commit (D-02).
### Pitfall 9: Saving the form writes empty `en` translations
**What goes wrong:** The form sends every enabled locale (`{pl: "...", en: ""}`), so a save creates `en` attribute rows with empty strings, including an empty `en` slug index. `Translated` falls back to the host for empty values, so the public output is unchanged.
**How to avoid:** this is existing behavior. Note it, and do not "fix" it in this phase.
## Code Examples
### Default locale first (cabana, sketch)
```go
// enabledContentLocales: default locale first, others in the writer's order.
codes, err := writer.EnabledLocales(ctx, db)
...
if def, err := writer.DefaultLocale(ctx, db); err == nil && def != "" {
if i := slices.Index(codes, def); i > 0 {
codes = append([]string{def}, slices.Delete(slices.Clone(codes), i, i+1)...)
}
}
return codes
```
### DryRun SQL test (pure, no Docker)
```go
// Source: probe this session; gorm.Config{DryRun: true, DisableAutomaticPing: true}
db, _ := gorm.Open(pgdriver.New(pgdriver.Config{DSN: "host=127.0.0.1 port=1 user=x dbname=x sslmode=disable"}),
&gorm.Config{DryRun: true, DisableAutomaticPing: true, Logger: logger.Discard})
sql := db.ToSQL(func(tx *gorm.DB) *gorm.DB {
q, _ := applyListSearch(tx.Model(&m2mRow{}), ctl, schema, &m2mRow{}, "x")
var n int64
return q.Count(&n)
})
// assert: contains `EXISTS (SELECT 1 FROM "<pivot>"`, contains `."<owner_fk>" = "<main>"."id"`, not contains `"Categories"."post_id"`, no "JOIN" in the outer query
```
## State of the Art
| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| `Joins(field)` for every relation column | EXISTS for search; correlated subquery for to-many sort | this phase | belongsToMany and hasMany search and sort work; counts are exact |
| URL-only list state | URL + per-user `backend_list_states` | this phase | State survives form round-trips and navigation |
| goldmark safe render | goldmark `html.WithUnsafe()` + gate | this phase | Imported HTML round-trips; PHP parity |
## Assumptions Log
| # | Claim | Section | Risk if Wrong |
|---|-------|---------|---------------|
| A1 | List state should persist across logins (durable per user), not end at logout like Winter's PHP session | Pattern 2 | Low: a one-line delete in `logout` switches to session semantics |
| A2 | Winter's `Markdown::parse` passes raw HTML through (Parsedown default) | Root cause D-03 | Low: the export data (content_html == content for 83/100, entities-only diffs for 17) supports it |
| A3 | Winter's `whereHas` applies the related model's soft-delete scope, so EXISTS should add `deleted_at IS NULL` for soft-deleting related models | Pattern 1 | Low: affects only soft-deleting related models; Journal Category has no soft delete [not verified] |
| A4 | GORM polymorphic references carry `PrimaryValue` needing an extra bound condition; refusing polymorphic relations in search is acceptable for now | Pattern 1 | Low: no current list column uses a polymorphic relation [not verified across plugins] |
| A5 | The PHP site's `use_rich_editor` setting value is unknown (Go UAT: `f`); 16.1 does not implement the rich-editor store-as-is branch | Pattern 4 | Low: unsafe markdown already round-trips HTML; the branch belongs with 16.2's Blog Editor |
| A6 | The new route name `list-state` and table name `backend_list_states` are acceptable names | Pattern 2 | Cosmetic |
## Open Questions
1. **Tag number vs the concurrent session.**
- What we know: the latest tag is v0.1.15. No `modules/` edits are in flight in this checkout.
- What's unclear: whether the other session will tag first.
- Recommendation: the release task checks `git fetch --tags` and uses the next free `v0.1.x`.
2. **Should `en` be disabled on grzybyfunkcjonalne?**
- What we know: the site is Polish-only; the export has only `pl` translations. With en enabled, every admin save writes empty en rows (Pitfall 9).
- What's unclear: whether any client requests `locale=en`.
- Recommendation: the framework fix is enough for D-03. Ask the user whether to also set `is_enabled=false` for `en` on UAT (one SQL line, or the Locales admin) and in `migrate.sh`.
3. **Repairing posts 53 and 64: admin re-save or a console command?**
- Recommendation: admin re-save after the deploy. It is two posts and doubles as the D-03 acceptance test. A reusable `journal:rebuild-html` command is out of scope unless the user wants it.
## Environment Availability
| Dependency | Required By | Available | Version | Fallback |
|------------|------------|-----------|---------|----------|
| Go | all Go work | ✓ | go1.27.0 | — |
| Node / npm / pnpm | admin SPA build + vitest; Nuxt build | ✓ | node v22.23.2 | — |
| Docker daemon | testcontainers tests (cabana query/list, journal admin harness) | ✗ (`permission denied` on /var/run/docker.sock; user not in docker group) | client 29.7.2 | Pure DryRun tests here; Postgres suites on a Docker-enabled machine (user/CI) |
| Local Postgres | ad-hoc queries | ✗ for this user (`role "golem" does not exist`) | — | — |
| ssh `grzybyfunkcjonalne-uat` + peer psql | read-only UAT checks, deploy | ✓ | PostgreSQL 17.11 | — |
| swag v1.16.6 | OpenAPI regeneration | ✓ (module cache) | v1.16.6 | — |
| vitest suite | SPA tests | ✓ (`ListView.test.ts` + `MLFields.test.ts`: 44 passed this session) | — | — |
**Missing dependencies with no fallback:** none blocking. Postgres-backed tests need the user's Docker-enabled environment before the tag.
## Validation Architecture
### Test Framework
| Property | Value |
|----------|-------|
| Framework | Go `testing` (+ testcontainers-go v0.44.0 for Postgres); vitest + @vue/test-utils + happy-dom for the SPA |
| Config file | none (Go); `admin/vite.config.*` / `admin/tests/setup.ts` |
| Quick run command | `go test ./modules/cabana -run 'TestListRelation|TestListState|TestEnabledLocales' -count=1` (pure DryRun/unit parts) ; `npm --prefix admin test -- tests/list/ListView.test.ts tests/form` ; `go -C ../sm-journal-plugin test ./classes/` |
| Full suite command | `go vet ./... && go test ./...` (summercms.go, Docker required) ; `go -C ../sm-journal-plugin test ./...` (Docker required) ; `npm --prefix admin test` ; `scripts/check-admin-dist.sh` ; `scripts/check-admin-openapi.sh --check` ; `go test ./cmd/summer -run TestDocsTree` |
### Phase Requirements → Test Map
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| D-02 | Search over a many2many column emits EXISTS through the pivot, no `Joins` | unit (DryRun SQL) | `go test ./modules/cabana -run TestListRelationSearchSQL` | ❌ Wave 0 (new `query_relation_test.go`) |
| D-02 | Search over many2many / hasMany / belongsTo returns the right rows, exact `total`, combined with a scope filter that uses unqualified `id` and a ListExtendQuery `user_id = ?` | integration (Postgres) | `go test ./modules/cabana -run TestListRelationSearch` | ❌ extend `query_test.go` fixtures (add a pivot model) |
| D-02 | Sort by a to-many column orders by the aggregated value, no duplicate rows | integration | `go test ./modules/cabana -run TestListRelationSort` | ❌ |
| D-01 | Successful list GET upserts state; 422 does not; unchanged state does not write | integration | `go test ./modules/cabana -run TestListStateSave` | ❌ |
| D-01 | `GET .../list-state` returns the caller's state only, sanitizes stale filter/sort/per_page, 403 without permission | integration | `go test ./modules/cabana -run TestListStateRestore` | ❌ |
| D-01 | Migration creates `backend_list_states`; rollback drops it; `backend_user_preferences` still absent | integration | `go test ./modules/lagoon -run TestBackendAdminMigrations` | ✅ extend |
| D-01 | ListView: empty query → list-state requested first, then replaced URL; no empty list request before restore; own clear-all saves empty; same-path nav re-entry restores; page clamp | unit (vitest) | `npm --prefix admin test -- tests/list/ListView.test.ts` | ✅ extend |
| D-03 | `meta.enabledLocales` has the default first when the writer returns `["en","pl"]` with default `pl` | unit/integration | `go test ./modules/cabana -run TestEnabledLocalesDefaultFirst` | ❌ (reuse `recordingWriter` in ml_smoke_test.go) |
| D-03 | MLField opens on `enabledLocales[0]`, showing host value | unit (vitest) | `npm --prefix admin test -- tests/form/MLFields.test.ts` | ✅ extend |
| D-03 | `FormatHTML` keeps raw HTML (`<p>a</p>\n<p>b</p>` round-trips; markdown + trailing `<footer>`/`<style>` kept) and still rejects script/iframe/event/javascript/vbscript/data | unit | `go -C ../sm-journal-plugin test ./classes/ -run TestFormatHTML` | ✅ extend `format_html_test.go` |
| D-03 | Admin update of a post whose `content` is HTML leaves `content_html` equal to the HTML (re-save test from the todo) | integration (journal harness) | `go -C ../sm-journal-plugin test -run TestPostsAdminResaveKeepsHTML .` | ❌ add to `posts_admin_smoke_test.go` |
| D-04 | 98-URL smoke on staging | e2e (server) | on-server `CURL_HOME=... scripts/smoke.sh --url https://uat.grzybyfunkcjonalne.pl` | ✅ |
### Manual (UAT, human with credentials)
On `/gfadmin/golem15/journal/posts`:
1. Search "cordyceps": rows return, no "The list could not be loaded".
2. Set a sort and a category filter, open a post, press Back (in-app): the same sort and filter remain and are in the URL. Navigate away via the rail and back: they are restored. Copy the URL into a new tab: the same list.
3. Open any imported post: title, slug, content (HTML source) and excerpt are filled, and the locale selector shows `pl`.
4. Re-save posts 53 and 64, then on the server: `SELECT count(*) FROM golem15_journal_posts WHERE content_html LIKE '%raw HTML omitted%'` → 0. `/blog/ekstrakt-z-turkey-tail` shows the author footer.
### Sampling Rate
- **Per task commit:** the quick-run commands for the touched repo (pure Go tests + vitest file).
- **Per wave merge:** full suite of the touched repo, plus `check-admin-dist.sh` / `check-admin-openapi.sh --check` when SPA or swag files changed.
- **Phase gate:** full suites green on a Docker-enabled machine before the tag; UAT smoke + manual list before `/msd-verify-work`.
### Wave 0 Gaps
- [ ] `modules/cabana/query_relation_test.go`: DryRun SQL tests plus many2many and hasMany fixture models (pivot table) for the Postgres tests.
- [ ] `modules/cabana/list_state_test.go`: save, restore, sanitize, permission.
- [ ] `sm-journal-plugin/posts_admin_smoke_test.go`: re-save keeps HTML.
- [ ] No framework installs needed.
## Security Domain
### Applicable ASVS Categories
| ASVS Category | Applies | Standard Control |
|---------------|---------|-----------------|
| V2 Authentication | no (unchanged) | existing backend guard |
| V3 Session Management | yes (new per-user state, not a session token) | State is keyed by the JWT principal ID; it carries no authority |
| V4 Access Control | yes | The list-state route goes through `protect()` (controller permission before SQL). Rows are scoped to `principal.ID`; no route reads another user's state |
| V5 Input Validation | yes | Stored state is re-validated against the compiled schema on restore. Search SQL uses bound `?` args and `quotedIdent` identifiers from GORM's schema. Size cap on stored JSON |
| V6 Cryptography | no | — |
| V14 / XSS (output) | yes | `content_html` now carries raw admin HTML; the `rejectUnsafe` gate stays on every write path (admin, import, API); the admin preview (`cabana.RenderMarkdown`, v-html) stays safe-mode |
### Known Threat Patterns
| Pattern | STRIDE | Standard Mitigation |
|---------|--------|---------------------|
| SQL injection via relation/column names in search | Tampering | Names come from the compiled schema + `schema.Relationship`; `identifier()` + `LookUpField`; values bound |
| Stored-state poisoning (oversized or crafted state) | Tampering / DoS | Validate on save (only successful list inputs) and on restore; size cap |
| Cross-site GET that writes list state | Tampering (low) | Admin cookie is `SameSite=Strict` (auth.go `sessionCookie`); Bearer clients are not CSRF-able; the impact is only the attacker-chosen filters of the victim |
| Stored XSS through post HTML | Tampering / Elevation | 3-pattern gate (script/iframe/object/embed, `on*=`, javascript/vbscript/data URLs); authors are permissioned admins; accepted residual risk vs PHP `Html::clean` (Pitfall 5) |
## Sources
### Primary (HIGH confidence, read or run this session)
- summercms.go:
- `modules/cabana/query.go` (whole file);
- `field_ml.go`, `crud.go:405-715, 1187-1274`, `http.go:225-300, 700-745, 955-1030`;
- `list_schema.go:20, 526-541`, `schema.go:61-71`, `auth.go:180-340`;
- `modules/lagoon/backend_admin_migrations.go`, `backend_admin_migrations_test.go:100-135`;
- `admin/src/components/form/*`, `admin/src/views/{ListView,FormView}.vue`, `admin/src/app/listQuery.ts`.
- sm-journal-plugin:
- `models/post.go`, `models/post/{columns,fields}.yaml`, `controllers/posts/*.yaml`;
- `controllers/posts.go`, `classes/format_html.go`, `classes/format_html_test.go`, `console/import_posts.go`;
- `admin_harness_test.go`, `go.mod`.
- sm-translate-plugin: `classes/admin_writer.go`, `classes/translatable.go:40-223`, `updates/202610060005_seed_en_pl_locales.go`.
- sm-grzybyfunkcjonalne-app: `go.mod`, `go.work`, `.gitmodules`, `scripts/build.sh`, `scripts/migrate.sh:167-175`, `DEPLOY.md`, `README.md:114`.
- WinterCMS reference:
- `examples/golem15-wintercms-starter/modules/backend/widgets/Lists.php` (500-600, session calls), `Search.php`, `Filter.php`;
- `traits/SessionMaker.php`;
- `database/migrations/2014_01_04_000005_Db_Backend_User_Preferences.php`;
- `grzybyfunkcjonalne.pl/plugins/golem15/journal/models/Post.php:130-208`, `controllers/Posts.php:80-98`.
- Probes:
- GORM v1.31.2 DryRun (many2many Joins SQL, references, EXISTS shape);
- goldmark v1.8.6 safe vs unsafe over 100 export bodies and 2 damaged posts;
- read-only UAT psql (locales, translate attributes, post column counts, wiped posts).
### Secondary
- `.planning/phases/16-*/16-RESEARCH.md` (Pitfalls 1-2), `16-04-SUMMARY.md`, `deferred-items.md`, `16-05-PLAN.md` (Docker constraint).
## Metadata
**Confidence breakdown:**
- Root causes: HIGH. Each is reproduced by a probe or observed in UAT data.
- EXISTS builder: HIGH for many2many and hasMany (probe); MEDIUM for polymorphic and soft-delete handling (assumed).
- List-state design: MEDIUM. Discretion area; the semantics are reasoned through, but there is no Winter-identical storage.
- Release mechanics: HIGH. Read from build.sh, go.mod, go.work and DEPLOY.md.
**Research date:** 2026-10-09
**Valid until:** 2026-10-23 (fast-moving repo with a concurrent `modules/` session)

78
.planning/phases/16.1-journal-admin-urgent-fixes-relation-search-list-state-post-f/16.1-VALIDATION.md Normal file
View File

@@ -0,0 +1,78 @@
---
phase: "16.1"
slug: "journal-admin-urgent-fixes-relation-search-list-state-post-f"
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
status: draft
nyquist_compliant: false
wave_0_complete: false
created: "2026-10-09"
---
# Phase 16.1 — Validation Strategy
> Per-phase validation contract for feedback sampling during execution.
---
## Test Infrastructure
| Property | Value |
|----------|-------|
| **Framework** | {pytest 7.x / jest 29.x / vitest / go test / other} |
| **Config file** | {path or "none — Wave 0 installs"} |
| **Quick run command** | `{quick command}` |
| **Full suite command** | `{full command}` |
| **Estimated runtime** | ~16.1 seconds |
---
## Sampling Rate
- **After every task commit:** Run `{quick run command}`
- **After every plan wave:** Run `{full suite command}`
- **Before `/msd-verify-work`:** Full suite must be green
- **Max feedback latency:** 16.1 seconds
---
## Per-Task Verification Map
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
| 16.1-01-01 | 01 | 1 | REQ-{XX} | T-16.1-01 / — | {expected secure behavior or "N/A"} | unit | `{command}` | ✅ / ❌ W0 | ⬜ pending |
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
---
## Wave 0 Requirements
- [ ] `{tests/test_file.py}` — stubs for REQ-{XX}
- [ ] `{tests/conftest.py}` — shared fixtures
- [ ] `{framework install}` — if no framework detected
*If none: "Existing infrastructure covers all phase requirements."*
---
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|----------|-------------|------------|-------------------|
| {behavior} | REQ-{XX} | {reason} | {steps} |
*If none: "All phase behaviors have automated verification."*
---
## Validation Sign-Off
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
- [ ] Wave 0 covers all MISSING references
- [ ] No watch-mode flags
- [ ] Feedback latency < 16.1s
- [ ] `nyquist_compliant: true` set in frontmatter
**Approval:** {pending / approved YYYY-MM-DD}