docs(17-03): summary of the Journal menu item types and sitemap plugin plan

2026-10-12 17:15:58 +02:00
parent 3e5a12c3ad
commit 362e1a730d
2 changed files with 284 additions and 0 deletions
--- a/.planning/phases/17-blog-companions/17-03-SUMMARY.md
+++ b/.planning/phases/17-blog-companions/17-03-SUMMARY.md
@@ -0,0 +1,277 @@
+---
+phase: 17-blog-companions
+plan: 03
+subsystem: plugins
+tags: [go, sitemap, journal, signpost, cabana, festival, postgres, php-parity, goldens]
+
+requires:
+  - phase: 17-01
+    provides: pact.HasMenuItemTypes / MenuItemType (PatternFrom), signpost registry and ReferenceOptions, cabana repeater, dependsOn, singleton controllers and settings link entries
+  - phase: 17-02
+    provides: admin SPA for repeater, dependent dropdown and singleton forms; summercms v0.1.23
+provides:
+  - sm-journal-plugin registers journal-category, all-journal-categories, journal-post, all-journal-posts, category-journal-posts (pact.HasMenuItemTypes)
+  - new local repository sm-sitemap-plugin (main, no remote), plugin golem15.sitemap
+  - GET /sitemap.xml and /sitemap.xsl byte-identical to the PHP plugin (goldens from PHP)
+  - typed hooks classes.BeforeAddItem and classes.AddItem
+  - sitemap:import and sitemap:import-winter with read-only Winter export SQL
+  - sitemap admin: definitions singleton form (repeater + dependent reference) and settings form
+affects: [17-04 menu plugin (same registry binding pattern), 17-05 site integration (submodule, go.mod bump, Winter export and import), 17-06 unit tests]
+
+actuals:
+  tokens: 84496
+  tasks: 4
+  commits: 5
+plan_head_before: 5fce70b
+plan_head_after: 3e5a12c
+repo_heads:
+  sm-journal-plugin: {before: 58851dd, after: c49d216, commits: 2}
+  sm-sitemap-plugin: {before: "(new repository)", after: 7d937e3, commits: 3}
+
+tech-stack:
+  added: []
+  patterns:
+    - "PHP-parity output pinned by goldens generated from the PHP code with the vendored Winter router (tests/php/gen-goldens.php), committed so CI needs no PHP"
+    - "Hand-written ports where the bytes are the contract: libxml serializer, Url::to/buildUrl/parse_url/rawurlencode, str_replace rewrite"
+    - "List menu item types take the record type's DefaultPattern plus PatternFrom and build children from the layered MenuItemResolve.Pattern"
+    - "Batched translate attribute reads for menu item alternates (one query per model set) with PHP falsy fallback"
+
+key-files:
+  created:
+    - ../sm-journal-plugin/menu_item_types.go
+    - ../sm-journal-plugin/classes/menu_item_types.go
+    - ../sm-journal-plugin/menu_item_types_smoke_test.go
+    - ../sm-sitemap-plugin/plugin.go
+    - ../sm-sitemap-plugin/routes.go
+    - ../sm-sitemap-plugin/admin.go
+    - ../sm-sitemap-plugin/classes/generator.go
+    - ../sm-sitemap-plugin/classes/serialize.go
+    - ../sm-sitemap-plugin/classes/urlto.go
+    - ../sm-sitemap-plugin/classes/tiers.go
+    - ../sm-sitemap-plugin/classes/manifest.go
+    - ../sm-sitemap-plugin/classes/rewrite.go
+    - ../sm-sitemap-plugin/classes/events.go
+    - ../sm-sitemap-plugin/classes/testdata/full.golden
+    - ../sm-sitemap-plugin/tests/php/gen-goldens.php
+    - ../sm-sitemap-plugin/console/import_winter.go
+    - ../sm-sitemap-plugin/controllers/definitions.go
+    - ../sm-sitemap-plugin/README.md
+  modified:
+    - ../sm-journal-plugin/lang/en/lang.yaml
+    - ../sm-journal-plugin/lang/pl/lang.yaml
+    - ../sm-journal-plugin/README.md
+    - .planning/phases/17-blog-companions/deferred-items.md
+
+key-decisions:
+  - "Sitemap definition theme key is the constant default (models.DefinitionTheme); imports always write it and report the PHP theme"
+  - "Journal URL placeholders are replaced in PHP getHeadlessUrlParams order (id, slug, category, year, month, day), not the order the plan text listed"
+  - "Translated slugs and titles use PHP's fallback (missing or falsy translation reads the default value), read in one batched golem15_translate_attributes query per model set"
+  - "journal-category with nesting returns the category's subtree as the plan states; the frozen PHP returned every category (getNested on a new query), logged as a deferred item for plan 05 parity"
+  - "Host override of the tier rules lives in config/golem15.yaml under sitemap.priority_tiers (compass maps a config file name to its top key); a host list replaces the default list"
+  - "Settings entries sit in the category phrase golem15.sitemap::lang.settings.category (CMS); the framework has no shared CMS category key"
+  - "sitemap:import-winter gained --theme for exports with several definitions, and reports dropped cmsPage values"
+
+patterns-established:
+  - "Sitemap tests boot a fake content plugin (acme.sitemaptypes) instead of requiring Journal; the real Journal plus sitemap pair is proven by the host in plan 05"
+
+requirements-completed: [SC-1, SC-2, SC-3, D-01, D-02, D-03, D-04, D-10, D-11, D-12, D-13, D-14]
+
+coverage:
+  - id: D1
+    description: "Journal registers its five PHP menu item types with DefaultPattern/PatternFrom, references, layered patterns, ordered per-locale alternates, nesting subtree, published filters and the x-default rule"
+    requirement: "D-04"
+    verification:
+      - kind: integration
+        ref: "../sm-journal-plugin/menu_item_types_smoke_test.go#TestMenuItemTypesSmoke"
+        status: pass
+    human_judgment: false
+  - id: D2
+    description: "/sitemap.xml from a stored definition (url item + registry item) equals the PHP golden byte for byte with Content-Type application/xml; 404 without a definition, empty 200 without items"
+    requirement: "SC-3"
+    verification:
+      - kind: unit
+        ref: "../sm-sitemap-plugin/classes/golden_test.go#TestSitemapGolden"
+        status: pass
+      - kind: integration
+        ref: "../sm-sitemap-plugin/sitemap_smoke_test.go#TestSitemapTracer"
+        status: pass
+    human_judgment: false
+  - id: D3
+    description: "Url::to port (58 PHP cases), escaping, per-alternate list expansion, nesting, tiers (default rules, custom settings, off), manifest lastmod, frontend_url rewrite, BeforeAddItem skip, AddItem extra, priority precision, 50000 URL cap; tier config refusals and host replacement; manifest fail-safe"
+    requirement: "SC-3"
+    verification:
+      - kind: unit
+        ref: "../sm-sitemap-plugin/classes/golden_test.go#TestSitemapGolden"
+        status: pass
+      - kind: unit
+        ref: "../sm-sitemap-plugin/classes/golden_test.go#TestSitemapTiersConfig"
+        status: pass
+    human_judgment: false
+  - id: D4
+    description: "sitemap:import-winter maps and drops items with a report, refuses a non-empty definition without --force, copies settings; /sitemap.xml rewrite with frontend_url; /sitemap.xsl bytes and headers; BeforeAddItem skip; sitemap:import PHP semantics"
+    requirement: "SC-1"
+    verification:
+      - kind: integration
+        ref: "../sm-sitemap-plugin/sitemap_smoke_test.go#TestSitemapImportSmoke"
+        status: pass
+    human_judgment: false
+  - id: D5
+    description: "Sitemap admin API: singleton created once, repeater rows stored with PHP keys, 422 on a reference not offered for its type, settings form saves, 403 without golem15.sitemap.access_definitions"
+    requirement: "D-11"
+    verification:
+      - kind: integration
+        ref: "../sm-sitemap-plugin/sitemap_smoke_test.go#TestSitemapAdminSmoke"
+        status: pass
+    human_judgment: false
+  - id: D6
+    description: "Sitemap admin screens in the SPA (singleton definition form with the item repeater, settings form) render and read well"
+    requirement: "SC-1"
+    verification: []
+    human_judgment: true
+    rationale: "Visual and UX check of the SPA screens; no browser test runs in this plan, and the definition.view subtitle link waits for a framework schema key"
+  - id: D7
+    description: "Plugin READMEs (Journal menu item types section, sitemap README with Winter adaptations) are accurate and name no consuming application"
+    requirement: "SC-1"
+    verification:
+      - kind: other
+        ref: "grep -ci 'grzyby|horoskopia|fonoteka|www.golem15.com' README.md lang/*/lang.yaml -> 0"
+        status: pass
+    human_judgment: true
+    rationale: "The framework docs checker does not run on sibling plugin repos; identifiers were checked by hand"
+
+duration: 38min
+completed: 2026-10-12
+status: complete
+---
+
+# Phase 17 Plan 03: Journal menu item types and the sitemap plugin Summary
+
+**Journal registers its five PHP menu item types through signpost, and a new sm-sitemap-plugin serves /sitemap.xml and /sitemap.xsl byte-identical to the PHP plugin (goldens generated from PHP), with tiers, manifest lastmod, typed hooks, the frontend_url rewrite, Winter import commands and a singleton admin form**
+
+## Performance
+
+- **Duration:** about 38 min
+- **Started:** 2026-10-12T14:37:00Z
+- **Completed:** 2026-10-12T15:15:00Z
+- **Tasks:** 4 (Task 1 tracer, verified end-to-end before expansion)
+- **Files:** 6 changed in sm-journal-plugin, 47 created in sm-sitemap-plugin
+
+## Accomplishments
+
+- **Journal (additive, D-04):** `(*Plugin).MenuItemTypes` and `classes.MenuItemTypes` register `journal-category` (tree references, nesting subtree), `all-journal-categories` (by name), `journal-post` (by title, no published filter), `all-journal-posts` and `category-journal-posts` (published, by title, category plus descendants). The three list types declare the record type's DefaultPattern and `PatternFrom`, so a host override of `journal-post` and `journal-category` reaches their children. Alternates follow translate `sort_order`, the unprefixed locale has no prefix, slugs and titles are translated with PHP's fallback, and `x-default` is the only alternate without enabled locales. Lang `menuitem.*` (en, pl, UI-SPEC copy), README "Menu item types" section and API rows. No route, payload, table, migration, config key or field definition changed (`git diff 58851dd -- routes.go controllers/api models/category/fields.yaml updates` is empty; category content stays `blogeditor`).
+- **sm-sitemap-plugin (new, `git init -b main`, no remote):** module `git.golem15.com/golem15/sm-sitemap-plugin`, plugin `golem15.sitemap`, go.work with replaces to `../summercms.go` and `../sm-translate-plugin`, go.mod requiring summercms v0.1.23 with the local replace.
+  - Tables `golem15_sitemap_definitions` (PHP columns, theme index) and `golem15_sitemap_settings` (singleton id 1, PHP defaults). `models.Definition` (Fillable exactly theme and data) with tolerant PHP-shaped item decoding and PHP `toArray` encoding.
+  - `classes`: libxml-format serializer, Url::to/buildUrl/parse_url/rawurlencode port, generator in `addItemToSet` order (50000 cap, lastmod from mtime or manifest, BeforeAddItem, empty drop, count, tier or item priority, elements, AddItem), per-alternate expansion for list types, tier rules compiled at boot from `golem15.sitemap.priority_tiers`, manifest reader (regular file, 5 MiB, JSON object, fail safe), `str_replace` rewrite.
+  - Routes `/sitemap.xml` (404 / empty 200 / XML, `application/xml`, `no-cache, private`) and `/sitemap.xsl` (embedded PHP file, `text/xsl; charset=UTF-8`); backend URL from `app.url`, else trusted `X-Forwarded-Proto` or TLS plus Host.
+  - Commands `sitemap:import` (PHP JSON 1:1) and `sitemap:import-winter` (type map, static/CMS drop, reference check, report, `--force`, `--dry-run`, settings copy) with `scripts/winter-export-definitions.sql` and `scripts/winter-export-settings.sql` (SELECT only).
+  - Admin: `golem15.sitemap.definitions` singleton controller (repeater on `data`, type dropdown url plus registry types, reference `dependsOn: type` through `signpost.ReferenceOptions`), settings entries Sitemap (link) and Sitemap settings (form), permission `golem15.sitemap.access_definitions` (developer), en/pl phrasebooks, README.
+
+## Task Commits
+
+**sm-journal-plugin**
+1. **Task 1 (tracer): journal-post type** - `fd1572a` (feat)
+2. **Task 2: the other four Journal types, lang, README** - `c49d216` (feat)
+
+**sm-sitemap-plugin**
+1. **Task 1 (tracer): plugin, definition model, serializer, Url::to, /sitemap.xml, goldens** - `fe39e26` (feat)
+3. **Task 3: full generation, settings, tiers, manifest, hooks, rewrite, XSL, import commands** - `4a82240` (feat)
+4. **Task 4: admin, permissions, lang, README** - `7d937e3` (feat)
+
+**summercms.go:** no code change (plan rule). Plan metadata: the docs commit that adds this SUMMARY and the deferred items, then the STATE/ROADMAP commit. Nothing was pushed or tagged.
+
+## Golden generator
+
+Local PHP 8.5.10 with the vendored Winter router:
+
+```sh
+cd ../sm-sitemap-plugin
+WINTER_VENDOR=/media/nvme/dev/golem15/grzybyfunkcjonalne.pl/vendor php tests/php/gen-goldens.php classes/testdata/tracer.json classes/testdata/tracer.golden
+WINTER_VENDOR=... php tests/php/gen-goldens.php classes/testdata/urlto.json classes/testdata/urlto.golden
+WINTER_VENDOR=... php tests/php/gen-goldens.php classes/testdata/full.json classes/testdata/full.golden
+```
+
+`full.json` holds 15 cases: escaping, empty urlset, no items, hreflang list, nested, default tiers, custom tier settings, tiers off, rewrite, rewrite with equal URLs, manifest, skip and extra hooks, priority precision, null URL, root with a port. The script requires `WINTER_VENDOR`, so the plugin code names no application path.
+
+## Docker smoke tests
+
+All ran on the local Docker daemon and passed: `TestMenuItemTypesSmoke` (Journal), `TestSitemapTracer`, `TestSitemapImportSmoke`, `TestSitemapAdminSmoke` (sitemap). The full Journal suite (`go test ./...`) and the full sitemap suite passed after the last commit; `go vet ./...` and `go test -short ./...` were green in both repos at every commit.
+
+## Decisions Made
+
+See `key-decisions` in the front matter.
+
+## Deviations from Plan
+
+### Auto-fixed Issues
+
+**1. [Rule 3 - Blocking] Definition needs a Rules method for cabana writes**
+- **Found during:** Task 4
+- **Issue:** The singleton route and saves answered 500 "missing Fill/Validate capability": cabana's writable model needs `Rules()`.
+- **Fix:** `models.Definition.Rules` returns an empty map (PHP validated nothing).
+- **Files modified:** ../sm-sitemap-plugin/models/definition.go
+- **Committed in:** 7d937e3
+
+**2. [Rule 1 - Parity] Placeholder replacement order follows PHP**
+- **Found during:** Task 1
+- **Issue:** The plan lists the post placeholders as `:slug, :category, :id, :year, :month, :day`; PHP `buildUrlFromPattern` replaces in `getHeadlessUrlParams` order (`id, slug, category, year, month, day`), which matters when a value contains another placeholder.
+- **Fix:** PHP order (categories `id, slug`).
+- **Committed in:** fd1572a
+
+**3. [Rule 1 - Parity] Translated slugs use PHP's fallback**
+- **Found during:** Task 1
+- **Issue:** The plan names `translate.TranslatedExact`, which returns nothing for a missing translation; PHP `translateContext` falls back to the default value for a missing or falsy translation. Per-call translate helpers would also run three queries per record and locale.
+- **Fix:** One batched `golem15_translate_attributes` read per model set with PHP truthiness and fallback.
+- **Committed in:** fd1572a, c49d216
+
+**4. [Rule 2 - Missing critical] sitemap:import-winter --theme**
+- **Found during:** Task 3
+- **Issue:** A Winter site with several themes exports several definition rows; the command had no way to choose.
+- **Fix:** `--theme`; without it an export with several rows is refused with the list of themes.
+- **Committed in:** 4a82240
+
+**5. [Rule 3 - Blocking] Host tier override file**
+- **Found during:** Task 3
+- **Issue:** The plan's decision names `config/golem15.sitemap.yaml`; compass reads a host config file name as its top key, and the framework docs override plugin keys under the vendor file (`config/acme.yaml`, `blog:`).
+- **Fix:** Documented and tested `config/golem15.yaml` with `sitemap: priority_tiers:` (`TestSitemapTiersConfig`); a host list replaces the default list.
+- **Committed in:** 4a82240
+
+**6. [Rule 3 - Blocking] Settings category key**
+- **Found during:** Task 4
+- **Issue:** The framework has no CMS settings category phrase.
+- **Fix:** `golem15.sitemap::lang.settings.category` ("CMS" in en and pl); the settings index groups by the translated text, so other plugins using "CMS" share the group.
+- **Committed in:** 7d937e3
+
+### Plan discrepancy recorded, not changed
+
+- **journal-category nesting:** the frozen PHP `$category->getNested()` returns every category; the plan's truth asks for the subtree, which is implemented. Logged as deferred item 4 (17-03) for plan 05's live parity check.
+
+---
+
+**Total deviations:** 6 auto-fixed (2 parity, 1 missing critical, 3 blocking). **Impact:** all within the plan's scope; no framework file changed.
+
+## Framework gaps (reported, not patched)
+
+- **Singleton subtitle link:** the "View sitemap.xml" link (`definition.view`) has no form schema key in cabana (17-02 deferred item 1); the phrase is shipped and the README says the link waits for the key.
+- **Shared CMS settings category:** no framework phrase; the plugin carries its own (deviation 6).
+
+## Issues Encountered
+
+- Journal's migrations seed a default category, so the smoke test clears categories before building its tree.
+- `go test -short` still starts Postgres for the Journal suite because its pre-existing TestMain scans `os.Args` (deferred item 1, 17-03).
+
+## Known Stubs
+
+None. The `definition.view` phrase is unused until the framework gap above is closed.
+
+## User Setup Required
+
+None. The remote repository `sm-sitemap-plugin` and the push are plan 05's human-approved step; Journal's go.mod still requires summercms v0.1.21 (it builds through the local replace) and plan 05 bumps it to v0.1.23.
+
+## Next Phase Readiness
+
+- Plan 04 can copy the definitions controller's binding (`DropdownOptions` with url plus registry types, `DependentOptions` through `signpost.ReferenceOptions`) for menu entries.
+- Plan 05 mounts `sm-sitemap-plugin` as a submodule, creates its remote, bumps the go.mod requires, runs the Winter export SQL and `sitemap:import-winter --map ...`, and checks live parity (watch for `journal-category` items with nesting).
+
+---
+*Phase: 17-blog-companions*
+*Completed: 2026-10-12*
--- a/.planning/phases/17-blog-companions/deferred-items.md
+++ b/.planning/phases/17-blog-companions/deferred-items.md
@@ -9,3 +9,10 @@ Out-of-scope findings logged during execution. None blocks the plan that found i
 3. **OpenAPI gaps the SPA works around.** `GET …/fields/{field}/dependent-options` reads `values[<sibling>]` from the query, but the document declares no query (the SPA casts the query). `AdminReorderNode.parent_id`, `AdminReorderRequest.partition` and `AdminReorderTree.partition` are nullable on the wire but typed as plain numbers or objects.
 4. **npm audit in admin/.** Seven advisories in existing dev and test dependencies (vitest and @vitest/mocker, tinypool, @vue/server-renderer through vue, source-map-js, esbuild). None comes from the TipTap packages added in 17-02.
 5. **Main bundle size.** `index-*.js` is about 609 kB minified (591 kB before 17-02) and Vite warns above 500 kB. TipTap itself is a separate lazy chunk (`editor-*.js`, about 381 kB).
+
+## From 17-03 (Journal types and sitemap plugin)
+
+1. **Journal's TestMain misses `-short`.** `sm-journal-plugin/admin_harness_test.go` decides `-short` by scanning `os.Args` for `-test.short`, but `go test -short` passes `-test.short=true`, so the Journal suite starts a Postgres container under `-short`. CLAUDE.md asks for `flag.Parse()` and `testing.Short()`. Pre-existing; not changed in 17-03.
+2. **gofmt drift in `sm-journal-plugin/classes/blog_editor_test.go`.** `gofmt -l` lists the file; pre-existing, not touched in 17-03.
+3. **Singleton subtitle link (17-02 item 1).** The sitemap definition form ships the `definition.view` phrase, but the "View sitemap.xml" subtitle link needs the framework form schema key that 17-02 item 1 describes; the sitemap README says so.
+4. **PHP journal-category nesting returned every category.** The frozen PHP `Category::resolveMenuItem` calls `$category->getNested()`, which in Winter runs the scope on a new query and returns the whole category tree, not the category's subtree. 17-03 implements the subtree, as the plan's D-02/D-03 truth states. A site whose PHP sitemap or menu used `journal-category` with nesting gets fewer URLs from Go; plan 05's live parity check should look for such items.