docs(33-03): complete Golem15.Sitemap rewrite plan
This commit is contained in:
Jakub Zych
@@ -1,10 +1,10 @@
|
||||
---
|
||||
schema_version: 1
|
||||
open_count: 21
|
||||
open_count: 23
|
||||
waived_count: 0
|
||||
fixed_count: 7
|
||||
total_count: 28
|
||||
last_updated: 2026-11-04T09:39:12.905Z
|
||||
total_count: 30
|
||||
last_updated: 2026-11-04T10:26:56.940Z
|
||||
---
|
||||
|
||||
# Broken Windows Ledger
|
||||
@@ -43,6 +43,8 @@ last_updated: 2026-11-04T09:39:12.905Z
|
||||
| 26 | 19.7 | deviation | modules/cabana | | 19.7-01: TestRecordActionPromptRun added on Postgres to prove Run receives the trimmed prompt value | open | | 2026-10-22T14:44:56.449Z | |
|
||||
| 27 | 19.7 | unrun-verify | scripts/check-phase19.6.1.sh | | 19.7-01: check-phase19.6.1.sh --all stopped by the host (memory pressure) after the cabana and surf Postgres suites passed; re-run on the Docker host | open | | 2026-10-22T14:44:56.679Z | |
|
||||
| 28 | 33 | deviation | modules/lagoon/validate.go | | 33-02: lagoon.Validate rejects the string and url rule tokens, so the Journal admin settings save answers 500 (pre-existing); the Headings and links switch is saved through the settings API/MCP until fixed | open | | 2026-11-04T09:39:12.905Z | |
|
||||
| 29 | 33 | stub | contract/contract.go | | sm-sitemap-plugin: contract.ListingSettings HTMLEnabled, HTMLPaths, HTMLPageSize and KeyFile stay zero and golem15_sitemap_urls.images stays [] until 33-04 adds the HTML sitemap, IndexNow and image settings | open | | 2026-11-04T10:26:56.754Z | |
|
||||
| 30 | 33 | deviation | models/settings/fields.yaml | | sm-sitemap-plugin: the Additional addresses lastmod is a text field read as YYYY-MM-DD (server-checked, errors.lastmod) because the framework repeater accepts no datepicker column; swap to a datepicker once cabana supports it in repeaters | open | | 2026-11-04T10:26:56.940Z | |
|
||||
|
||||
````json
|
||||
[
|
||||
@@ -409,6 +411,32 @@ last_updated: 2026-11-04T09:39:12.905Z
|
||||
"recorded_at": "2026-11-04T09:39:12.905Z",
|
||||
"resolved_at": null,
|
||||
"milestone": "v1.0"
|
||||
},
|
||||
{
|
||||
"id": 29,
|
||||
"kind": "stub",
|
||||
"phase": "33",
|
||||
"file": "contract/contract.go",
|
||||
"line": null,
|
||||
"description": "sm-sitemap-plugin: contract.ListingSettings HTMLEnabled, HTMLPaths, HTMLPageSize and KeyFile stay zero and golem15_sitemap_urls.images stays [] until 33-04 adds the HTML sitemap, IndexNow and image settings",
|
||||
"status": "open",
|
||||
"reason": "",
|
||||
"recorded_at": "2026-11-04T10:26:56.754Z",
|
||||
"resolved_at": null,
|
||||
"milestone": "v1.0"
|
||||
},
|
||||
{
|
||||
"id": 30,
|
||||
"kind": "deviation",
|
||||
"phase": "33",
|
||||
"file": "models/settings/fields.yaml",
|
||||
"line": null,
|
||||
"description": "sm-sitemap-plugin: the Additional addresses lastmod is a text field read as YYYY-MM-DD (server-checked, errors.lastmod) because the framework repeater accepts no datepicker column; swap to a datepicker once cabana supports it in repeaters",
|
||||
"status": "open",
|
||||
"reason": "",
|
||||
"recorded_at": "2026-11-04T10:26:56.940Z",
|
||||
"resolved_at": null,
|
||||
"milestone": "v1.0"
|
||||
}
|
||||
]
|
||||
````
|
||||
|
||||
@@ -0,0 +1,437 @@
|
||||
---
|
||||
phase: 33-seo-output-audit-and-content
|
||||
plan: 03
|
||||
subsystem: sitemap
|
||||
tags: [sm-sitemap-plugin, sitemap-index, slot-table, stable-numbering, lastmod, hreflang, filter-contract, listing, conga, settings-singleton, breaking-change]
|
||||
|
||||
requires:
|
||||
- phase: 33-seo-output-audit-and-content
|
||||
provides: "33-01 version ruling (framework target v0.6.1, base v0.6.0) and the signpost content facts"
|
||||
- phase: 32-seo-foundation
|
||||
provides: "signpost content descriptors and Records (D-93), pact.RecordSaved/RecordDeleted (D-92), form extensions (D-91)"
|
||||
provides:
|
||||
- "sm-sitemap-plugin per-type format: GET /sitemap.xml index and GET /_sitemap/files/{file} served from the slot table golem15_sitemap_urls (D-30)"
|
||||
- "Stable file numbering (classes.AssignSlots), file_size setting 1..50 000 (default 1000), renumbering only on a size change"
|
||||
- "Event-driven lastmod (RecordSaved with changes outside golem15.sitemap.lastmod_ignore), RecordDeleted holes"
|
||||
- "Rebuild job golem15.sitemap.rebuild: debounced after record events (rebuild_pending_until), dispatched on settings save, nightly sitemap:rebuild 03:00, sitemap:rebuild [--queue]"
|
||||
- "hreflang xhtml:link alternates with x-default (D-55 to D-57)"
|
||||
- "Package git.golem15.com/golem15/sm-sitemap-plugin/contract: Filter (consumed through app.Lookup), Listing (published by the plugin), reasons (D-95)"
|
||||
- "Settings singleton golem15.sitemap.settings (Files with S1 partial, Content types extension tab, Additional addresses); golem15_sitemap_types"
|
||||
- "Migration of the Phase 17 definition url items into extra_urls, frontend_url into site_url, frontend_default_locale into default_locale (D-32)"
|
||||
- "sitemap:import and sitemap:import-winter retargeted to the additional addresses"
|
||||
affects: [33-04, 33-05, 33-07, 33-08, 33-13, 33-14]
|
||||
|
||||
actuals:
|
||||
tokens: 60600
|
||||
tasks: 4
|
||||
commits: 7
|
||||
plan_head_before: 049f3ee270ed9ac2619f357626914b1a8caac780
|
||||
plan_head_after: 64453d2a914f00a8310f76f806a8858d5d4345da
|
||||
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "Slot table: every address is a row with (grp, file_no); a rebuild keeps numbers, appends oldest first, leaves holes; numbered_size detects a file size change"
|
||||
- "One rebuild transaction that locks settings row 1 (SELECT ... FOR UPDATE), so two rebuilds never interleave"
|
||||
- "Debounce through a conditional UPDATE of rebuild_pending_until plus conga Enqueue in the same transaction; the job clears it when it starts"
|
||||
- "Published contract sub-package without init, so other plugins import the seam without registering the plugin"
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- ../sm-sitemap-plugin/contract/contract.go
|
||||
- ../sm-sitemap-plugin/classes/slots.go
|
||||
- ../sm-sitemap-plugin/classes/rebuild.go
|
||||
- ../sm-sitemap-plugin/classes/writer.go
|
||||
- ../sm-sitemap-plugin/classes/settings.go
|
||||
- ../sm-sitemap-plugin/classes/types.go
|
||||
- ../sm-sitemap-plugin/classes/dispatch.go
|
||||
- ../sm-sitemap-plugin/classes/lastmod.go
|
||||
- ../sm-sitemap-plugin/classes/listing.go
|
||||
- ../sm-sitemap-plugin/models/url.go
|
||||
- ../sm-sitemap-plugin/models/type_setting.go
|
||||
- ../sm-sitemap-plugin/models/legacy.go
|
||||
- ../sm-sitemap-plugin/updates/202610280001_create_golem15_sitemap_urls.go
|
||||
- ../sm-sitemap-plugin/updates/202610280002_update_golem15_sitemap_settings.go
|
||||
- ../sm-sitemap-plugin/updates/202610280003_create_golem15_sitemap_types.go
|
||||
- ../sm-sitemap-plugin/controllers/settings.go
|
||||
- ../sm-sitemap-plugin/controllers/settings/config_form.yaml
|
||||
- ../sm-sitemap-plugin/controllers/settings/_sitemap_files.htm
|
||||
- ../sm-sitemap-plugin/extensions_settings.go
|
||||
- ../sm-sitemap-plugin/jobs.go
|
||||
- ../sm-sitemap-plugin/listeners.go
|
||||
- ../sm-sitemap-plugin/schedule.go
|
||||
- ../sm-sitemap-plugin/console/rebuild.go
|
||||
- ../sm-sitemap-plugin/sitemap_index_test.go
|
||||
- ../sm-sitemap-plugin/content_fixture_test.go
|
||||
- ../sm-sitemap-plugin/filter_fixture_test.go
|
||||
- ../sm-sitemap-plugin/classes/slots_test.go
|
||||
- ../sm-sitemap-plugin/classes/lastmod_test.go
|
||||
modified:
|
||||
- ../sm-sitemap-plugin/plugin.go
|
||||
- ../sm-sitemap-plugin/routes.go
|
||||
- ../sm-sitemap-plugin/admin.go
|
||||
- ../sm-sitemap-plugin/models/settings.go
|
||||
- ../sm-sitemap-plugin/models/settings/fields.yaml
|
||||
- ../sm-sitemap-plugin/controllers/admin_registry.go
|
||||
- ../sm-sitemap-plugin/console/import.go
|
||||
- ../sm-sitemap-plugin/console/import_winter.go
|
||||
- ../sm-sitemap-plugin/console/helpers.go
|
||||
- ../sm-sitemap-plugin/config/config.yaml
|
||||
- ../sm-sitemap-plugin/lang/en/lang.yaml
|
||||
- ../sm-sitemap-plugin/lang/pl/lang.yaml
|
||||
- ../sm-sitemap-plugin/README.md
|
||||
- ../sm-sitemap-plugin/plugin.yaml
|
||||
- ../sm-sitemap-plugin/go.mod
|
||||
|
||||
key-decisions:
|
||||
- "Task 1 ANSWERED: as-proposed (Jakub, 2026-11-04): index stays /sitemap.xml, files /sitemap-<slug>-<n>.xml served through /_sitemap/files/{file}, /sitemap.xsl removed, one settings singleton, url items to Additional addresses, priority/changefreq/tiers/manifest removed, definitions table kept unread"
|
||||
- "Version mapping (33-01 ruling): framework target v0.6.1, base v0.6.0; go.mod still requires summercms v0.6.0 with the existing replace => ../summercms.go; the bump is 33-14's"
|
||||
- "Settings validation runs in the controller's FormBeforeUpdate (Settings.Validate), not lagoon.HasBeforeValidate: cabana's CRUD path turns a BeforeValidate error into a 500"
|
||||
- "Content type titles (the HTML sitemap heading) are stored as a JSONB locale map on golem15_sitemap_types, not through the translation writer"
|
||||
- "Rows left out (deleted, unpublished, noindex, type) keep their file number as a hole; a renumber drops the numbers of left-out rows"
|
||||
- "S1 links files to the backend routes (/sitemap.xml, /_sitemap/files/<name>): the partial kit keeps only root-relative hrefs; the note names the public index URL"
|
||||
- "The index answers 404 while no site address is configured, and an empty sitemapindex before the first rebuild"
|
||||
- "A Filter error or a wrong number of decisions fails the rebuild (the files stay as they were); an invalid Decision.Group is ignored with a warning"
|
||||
- "x-default points to the default_locale setting's address, else the record's first listed locale; alternates list only listed (not excluded) locales"
|
||||
- "Reserved slugs images and news (33-04's image and News sitemaps) are refused as type file names"
|
||||
|
||||
patterns-established:
|
||||
- "Plugins publish narrow seams in a contract sub-package with no init; consumers Lookup the interface and work without it"
|
||||
- "Fake content plugin over an in-memory store for testing signpost consumers without database tables"
|
||||
|
||||
requirements-completed: [SC-1, D-30, D-32, D-55, D-56, D-57, D-93, D-95, D-24, D-22]
|
||||
|
||||
coverage:
|
||||
- id: D1
|
||||
description: "GET /sitemap.xml answers an index of per-type files sitemap-<slug>-<n>.xml with lastmod; each file at most file_size URLs with absolute <loc> and real <lastmod>, no priority or changefreq; drafts absent; escaping; unknown files 404; /sitemap.xsl 404"
|
||||
requirement: "D-30"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestSitemapIndex"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "../sm-sitemap-plugin/routes_test.go#TestFileNames"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D2
|
||||
description: "Stable numbering: holes on delete, appends to the last file with room, renumber only on file_size change"
|
||||
requirement: "D-30"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestStableNumbering"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "../sm-sitemap-plugin/classes/slots_test.go#TestAssignSlots"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D3
|
||||
description: "lastmod moves only on a real change (per locale for translated changes); no-op saves, ignored attributes and rebuilds leave it; deletes leave the files"
|
||||
requirement: "D-30"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestLastmod"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "../sm-sitemap-plugin/classes/lastmod_test.go#TestRealChanges"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D4
|
||||
description: "hreflang alternates for each translated locale plus x-default; none for a missing translation or a single locale"
|
||||
requirement: "D-55"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestHreflangAlternates"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D5
|
||||
description: "contract.Filter batches per page; Exclude leaves the files counted by reason, Group moves to sitemap-offers-1.xml, Groups titles reach Listing.Settings; without a Filter everything published is listed; Listing.Entries paging"
|
||||
requirement: "D-95"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestFilterExcludes"
|
||||
status: pass
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestNoFilter"
|
||||
status: pass
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestListing"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D6
|
||||
description: "Rebuilds: debounced to one job per delay after record events, dispatched on save, nightly 03:00, sitemap:rebuild [--queue]"
|
||||
requirement: "D-30"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestDebouncedRebuild"
|
||||
status: pass
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestRebuildCommand"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "../sm-sitemap-plugin/commands_documented_test.go#TestScheduleAndJobs"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D7
|
||||
description: "Settings singleton (one tile, Files/Content types/Additional addresses), errors.* refusals, save dispatches a rebuild, type off leaves files with reason type, S1 empty/building/files/left-out states"
|
||||
requirement: "D-32"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestSettingsSingleton"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "../sm-sitemap-plugin/models/settings_test.go#TestSettingsValidate"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D8
|
||||
description: "Definitions migration (url items to extra_urls, frontend_url to site_url, locale), definitions table untouched, rollback keeps data; sitemap:import and sitemap:import-winter write the additional addresses"
|
||||
requirement: "D-32"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/sitemap_index_test.go#TestMigrateDefinitions"
|
||||
status: pass
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/import_test.go#TestImportWinter"
|
||||
status: pass
|
||||
- kind: integration
|
||||
ref: "../sm-sitemap-plugin/import_test.go#TestImportFile"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D9
|
||||
description: "Site URL precedence: setting, golem15.sitemap.site_url, app.url; never the Host header"
|
||||
requirement: "D-22"
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "../sm-sitemap-plugin/routes_test.go#TestSiteURLPrecedence"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D10
|
||||
description: "The settings screen's look in the real admin (S1 tables and callouts, the Content types tab placement after Files, the repeater)"
|
||||
verification: []
|
||||
human_judgment: true
|
||||
rationale: "Rendering in the embedded admin SPA; the tests cover the schema, the API and the partial's node tree, not the visual layout"
|
||||
|
||||
duration: 43min
|
||||
completed: 2026-11-04
|
||||
status: complete
|
||||
---
|
||||
|
||||
# Phase 33 Plan 03: Golem15.Sitemap rewrite Summary
|
||||
|
||||
**sm-sitemap-plugin now serves a sitemap index at /sitemap.xml over per-type files `sitemap-<slug>-<n>.xml` from a slot table with stable numbering, event-driven lastmod, hreflang alternates and a debounced conga rebuild, narrowed by a published `contract.Filter` and readable through `contract.Listing`; the PHP-parity format, the definitions screen and the XSL are gone and the old data migrates.**
|
||||
|
||||
## Task 1 ANSWERED: as-proposed (Jakub, 2026-11-04)
|
||||
|
||||
- The index stays at `/sitemap.xml`. The per-type files are `/sitemap-<slug>-<n>.xml`, and Go serves them at `/_sitemap/files/{file}`.
|
||||
- The XSL stylesheet `/sitemap.xsl` is removed and now answers 404. This resolves the todo `.planning/todos/completed/sitemap-xslt-browser-deprecation.md`. If g15office task **w8kupqi3** is still open, it can be closed as resolved by removal (commit `84cb27a` in sm-sitemap-plugin). I could not check its state from here.
|
||||
- The Phase 17 definitions screen and the settings tile are now one settings singleton. The url items moved to "Additional addresses". priority, changefreq, tiers and the manifest are removed. The definitions table stays in the database and is no longer read.
|
||||
|
||||
## Read first: versions, breaking changes, consumers
|
||||
|
||||
- **Versions.** The framework target is v0.6.1 with base v0.6.0 (33-01 ruling). `sm-sitemap-plugin/go.mod` still requires `summercms v0.6.0` through the existing `replace => ../summercms.go`, and that replace was not touched. The plugin was at v0.1.2 (049f3ee). These changes are breaking, so the next minor tag is cut in 33-14. Nothing was pushed or tagged.
|
||||
- **Breaking changes in sm-sitemap-plugin**, approved by the "as-proposed" answer:
|
||||
1. `/sitemap.xml` is now a `<sitemapindex>`, not a `<urlset>`. The URL is unchanged, so robots.txt and the GSC submission need no change.
|
||||
2. The files live at the site root as `/sitemap-<slug>-<n>.xml`, served by Go at `/_sitemap/files/<name>`. A site must map the root names: vue-seo-layer does this (33-07, next tag v0.2.0), or nginx can (the README has the snippet).
|
||||
3. `/sitemap.xsl` answers 404. `<priority>` and `<changefreq>` are no longer written.
|
||||
4. The admin has one "Sitemaps" tile linking to `golem15.sitemap.settings`. The `golem15.sitemap.definitions` controller and the `sitemap_settings` settings form are gone. The permission code `golem15.sitemap.access_definitions` is unchanged, so roles keep their access.
|
||||
5. Config keys `golem15.sitemap.cache_seconds` and `golem15.sitemap.priority_tiers` are removed. New keys: `site_url`, `lastmod_ignore`, `rebuild_delay_seconds`. The festival events `classes.BeforeAddItem` and `classes.AddItem` are removed.
|
||||
6. A `<loc>` never uses the request Host. Without a site address (the setting, `golem15.sitemap.site_url` or `app.url`), the index answers 404.
|
||||
7. `sitemap:import` writes `{url, lastmod}` into the additional addresses and ignores `changefreq` and `priority`. Its success line is now "Imported N addresses into the sitemap's additional addresses." `sitemap:import-winter` keeps its flags but writes url items into the additional addresses and the settings into `site_url` and `default_locale`. Reference items are dropped.
|
||||
8. The application needs a conga worker for rebuilds (record events and saves queue them) and the scheduler for the nightly run. Otherwise someone has to run `sitemap:rebuild`. After deploying, run `sitemap:rebuild` once so the files exist.
|
||||
- **Consumers.** sm-grzybyfunkcjonalne-app and sm-demo-app pin v0.1.2 as a submodule and are untouched. They move to the new URLs in 33-13. No core plugin (user, blog, pages, payment, journal) was changed, and the framework got no code from this plan (D-90; `--marker` passed).
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** about 43 min
|
||||
- **Started:** 2026-11-04T09:47:25Z
|
||||
- **Completed:** 2026-11-04T10:30Z
|
||||
- **Tasks:** 4 (Task 1 decision answered before the run, Tasks 2 to 4 executed)
|
||||
- **Commits:** 7 in sm-sitemap-plugin (branch main, not pushed). 78 files changed: 49 added or modified, 29 removed.
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- **Index and files from a slot table (D-30).** `golem15_sitemap_urls` holds one row per address with its group and file number. Rebuilds keep file numbers, append new addresses to the last file with room, and leave holes for deleted, unpublished or excluded addresses. Only a `file_size` change renumbers. The routes read the table and never generate per request.
|
||||
- **Real lastmod.** A row's lastmod starts at the record's `LastMod`. After that it moves only when `pact.RecordSaved` reports a change outside `golem15.sitemap.lastmod_ignore`. A translated change moves only that locale's row. A no-op save, a counter update or a rebuild moves nothing. `pact.RecordDeleted` takes the rows out at once.
|
||||
- **Rebuilds.** Rebuilds run as the conga job `golem15.sitemap.rebuild`:
|
||||
- debounced after record events through `rebuild_pending_until` (P33-12), defaulting to 60 s;
|
||||
- dispatched after every settings save commits;
|
||||
- run nightly at 03:00 through the scheduled `sitemap:rebuild`;
|
||||
- run by hand with `sitemap:rebuild`, or `--queue` to hand it to a worker.
|
||||
- **hreflang (D-55 to D-57).** A record with two or more listed locales gets `xhtml:link` alternates on every listed address, plus `x-default` on the default locale's address. The xhtml namespace is declared only when used.
|
||||
- **Contract (D-95).** `contract.Filter` is asked once per page of records and once for the additional addresses:
|
||||
- an excluded address keeps its row with the reason and is counted in S1;
|
||||
- `Decision.Group` moves the address to another file group;
|
||||
- `Decision.LastMod` sets the first lastmod of a new address.
|
||||
|
||||
The plugin publishes `classes.Listing` as `contract.Listing` in `Register`. Its `Settings` returns the site address, the index address, the default locale and the groups with counts and titles. Its `Entries` pages through a group's addresses, with a locale filter.
|
||||
- **Settings singleton (C26).** One tile opens it. It has three tabs:
|
||||
- Files: site address, file size, and the S1 partial (index plus files, the left-out table by reason, a note; the empty and building states).
|
||||
- Content types: built at boot from signpost, one block per type (include, file name, HTML heading).
|
||||
- Additional addresses.
|
||||
|
||||
Saving refuses bad input with the `errors.*` copy.
|
||||
- **Migration (D-32).** Migration 202610280002 copies the url items of the served definition into `extra_urls`, `frontend_url` into `site_url` and `frontend_default_locale` into `default_locale`. The definitions table and the Phase 17 columns are untouched, and rollback drops only the new columns.
|
||||
- **Removal.** The generator, the libxml serializer, tiers, the rewrite, the manifest, the `Url::to` port, the events, the PHP goldens with their generator, the XSL, the definition model, the definitions controller and its form are all deleted, along with their tests.
|
||||
|
||||
## Task Commits (sm-sitemap-plugin, branch main, not pushed)
|
||||
|
||||
1. **Task 1: Confirm the public format.** No commit; answered "as-proposed" (above).
|
||||
2. **Task 2: index and per-type files (tracer):** `3359f1a` feat: sitemap index and per-type files from a slot table (tracer)
|
||||
3. **Task 3:**
|
||||
- group A `3ba70bb` feat: sitemap settings singleton with content types and additional addresses
|
||||
- group B `84cb27a` feat!: remove the PHP-parity sitemap, tiers, definitions screen and XSL
|
||||
- group C `62fc7d7` feat: migrate definitions into additional addresses
|
||||
4. **Task 4:**
|
||||
- group A `653627b` feat: real lastmod from record events and debounced rebuilds
|
||||
- group B `ded1095` feat: hreflang alternates and the sitemap filter contract
|
||||
- group C `64453d2` docs: README for the per-type sitemap format
|
||||
|
||||
**Plan metadata:** this SUMMARY commit in summercms.go (docs), then ROADMAP alone.
|
||||
|
||||
## Files Created/Modified (sm-sitemap-plugin)
|
||||
|
||||
- `contract/contract.go`: every P33-08 type. No init and no `party.Register`.
|
||||
- `classes/slots.go`: `AssignSlots` and `Slot`.
|
||||
- `classes/rebuild.go`: `Rebuild`, `RebuildResult`, `RebuildArgs` and `RebuildKind`, `DefaultSlug`, `ExtraRecordID`, the Filter decisions, the alternates.
|
||||
- `classes/writer.go`: `WriteIndex` and `WriteURLSet` on `bufio` with `xml.EscapeText`; `FileName`, `ParseFileName`, `ValidSlug`.
|
||||
- `classes/settings.go`: `LoadSettings`, `SiteURL`, `Absolute`.
|
||||
- `classes/types.go`: `EffectiveType`, `LoadTypeSettings`, `ReservedSlugs`.
|
||||
- `classes/dispatch.go`: `RequestRebuild`, `DispatchRebuild`, `Building`.
|
||||
- `classes/lastmod.go`: `RecordSaved`, `RecordDeleted`, `RealChanges`, `RequestDebouncedRebuild`, `ClearPending`, and the config readers.
|
||||
- `classes/listing.go`: `Listing`.
|
||||
- `models/url.go`, `models/type_setting.go`, `models/legacy.go`: the slot row, the type row, and a reader for Phase 17 and Winter definition data (used by the import and the migration).
|
||||
- `models/settings.go`: the new columns only, plus `Validate` and `Problems`.
|
||||
- `updates/2026102800{01,02,03}_*.go`: the slot table, the settings columns with the definitions copy, and the types table.
|
||||
- `controllers/settings.go` with `settings/config_form.yaml` and `settings/_sitemap_files.htm`: the singleton and S1.
|
||||
- `extensions_settings.go`: the Content types tab.
|
||||
- `admin.go`: one tile; embeds by pattern.
|
||||
- `jobs.go`, `listeners.go`, `schedule.go`, `console/rebuild.go`, `plugin.go`: the job, the listeners, the schedule, the command, and publishing the Listing.
|
||||
- `console/import.go`, `console/import_winter.go`, `console/helpers.go`: these now write to the additional addresses.
|
||||
- `routes.go`: the index and file routes.
|
||||
- `config/config.yaml`: `site_url` (commented), `lastmod_ignore`, `rebuild_delay_seconds`.
|
||||
- `lang/{en,pl}/lang.yaml`: every C26 key, including the images, News, IndexNow and HTML keys that 33-04 uses, plus `errors.lastmod`, `settings.extra_url_lastmod_comment` and `jobs.rebuild`. The Phase 17 item and definition keys are removed.
|
||||
- `README.md` and `plugin.yaml` (regenerated by `summer plugin:manifest`), `go.mod` (`goccy/go-yaml` is now a direct dependency; it is already in the decided set).
|
||||
- Tests:
|
||||
- `sitemap_index_test.go`: 13 Docker tests.
|
||||
- `content_fixture_test.go`, `filter_fixture_test.go`: the fake content plugin and the fake filter plugin.
|
||||
- `import_test.go`, `routes_test.go`, `classes/slots_test.go`, `classes/lastmod_test.go`, `models/settings_test.go`, `console/import_winter_test.go`, `commands_documented_test.go`.
|
||||
|
||||
## Decisions Made
|
||||
|
||||
See `key-decisions` in the frontmatter.
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 3 - Blocking] The framework repeater does not accept a datepicker**
|
||||
- **Found during:** Task 3
|
||||
- **Issue:** C26 asks for a datepicker for the row lastmod. cabana refuses `datepicker` inside a `repeater` when it compiles the schema, so the plugin would not boot.
|
||||
- **Fix:** the lastmod is a text field with the comment "Optional, as YYYY-MM-DD". The server checks it (`models.ValidDate`) and refuses a bad value with the new `errors.lastmod`. This is logged in deferred-items.md and WINDOWS.md #30. The framework fix (datepicker in repeater rows) is separate, additive work.
|
||||
- **Commit:** 3ba70bb
|
||||
|
||||
**2. [Rule 1 - Bug] `BeforeValidate` refusals became 500s**
|
||||
- **Found during:** Task 3
|
||||
- **Issue:** cabana's CRUD path turns any `lagoon.HasBeforeValidate` error into a `CapabilityError` (500). The Phase 17 hook only worked through the old settings-form route.
|
||||
- **Fix:** validation is now `Settings.Validate`, called from the controller's `FormBeforeUpdate` and `FormBeforeCreate` (the SEO plugin's pattern). It returns 422 with field details.
|
||||
- **Commit:** 3ba70bb
|
||||
|
||||
**3. [Rule 3 - Blocking] S1 cannot link to the public file URLs**
|
||||
- **Issue:** the partial renderer keeps only root-relative `href`s (`safePartialURL`).
|
||||
- **Fix:** each file links to the backend route that serves it. The note names the public index address. Logged in deferred-items.md as by design.
|
||||
- **Commit:** 3ba70bb
|
||||
|
||||
**4. [Plan shape] Field labels cannot carry `:type`**
|
||||
- **Issue:** cabana translates field labels without parameters, so "Include :type" cannot be rendered.
|
||||
- **Fix:** each type's switch is labelled with the type's own signpost label. The copy moved to comments without the parameter (`type_include` "Include this content type in the sitemaps.", `type_slug` "File name", `type_title` "Heading in the HTML sitemap", and their pl equivalents).
|
||||
- **Commit:** 3ba70bb
|
||||
|
||||
**5. [Plan shape] Content type titles stored as a JSONB locale map**
|
||||
- **Issue:** the plan said "title via the translation writer", but the extension's `Save` already receives the mltext value as a locale map, and `golem15_sitemap_types` has no host-column model for translate.
|
||||
- **Fix:** titles are stored as `titles jsonb` and fed directly into `GroupInfo.Titles` and `Group.Titles`.
|
||||
- **Commit:** 3ba70bb
|
||||
|
||||
**6. [Rule 3 - Blocking] The Phase 17 tests needed rework**
|
||||
- **Issue:** the route smoke tests (`TestSitemapTracer`, `TestSitemapCacheAndThrottle`, `TestSitemapImportSmoke`), the admin smoke test (`TestSitemapAdminSmoke`, which already failed before this plan because the framework now localizes 422 phrase keys), `TestBackendURL`, and the tier Boot assertion all pinned the removed format.
|
||||
- **Fix:** they were replaced by the new tests, and `TestImportWinter` was rewritten for the new targets. No surviving behaviour lost its test.
|
||||
- **Commits:** 3359f1a, 3ba70bb, 84cb27a
|
||||
|
||||
**7. [Plan shape] Extra settings columns**
|
||||
- The plan did not list `numbered_size` (detects a file size change), `built_at` and `rebuild_requested_at` (the S1 building state).
|
||||
- `extra_urls` is `JSONB NULL DEFAULT '[]'` instead of `NOT NULL`, because a `lagoon.Jsonable` that is not valid writes NULL.
|
||||
- **Commit:** 3359f1a
|
||||
|
||||
**8. [Acceptance grep] Two greps print more than the plan expects**
|
||||
- `grep "priority_tiers\|cache_seconds"` still matches `enable_priority_tiers`, in two places:
|
||||
- the column in the released Phase 17 migration 202610110002, whose DDL must not change;
|
||||
- the PHP export key that `sitemap:import-winter` recognizes so it can report it as ignored.
|
||||
|
||||
Both config keys themselves are gone.
|
||||
- `grep "/_sitemap/files/{file}" routes.go` prints two lines: the route and its doc comment.
|
||||
|
||||
**9. [TDD note] RED evidence**
|
||||
- Before implementing, I ran the tests for 3C and for 4A and 4B:
|
||||
- `TestMigrateDefinitions` failed with `extra_urls = ""`.
|
||||
- `TestLastmod` and `TestDebouncedRebuild` failed on their target assertions.
|
||||
- `TestHreflangAlternates`, `TestFilterExcludes` and `TestListing` failed on theirs.
|
||||
- `TestStableNumbering` and `TestNoFilter` passed at RED. They pin invariants that the tracer slot logic already had.
|
||||
- The plan is `type: execute`, and each task is one commit per the plan's commit-group table, with no separate `test(...)` commit.
|
||||
|
||||
### Auth gates
|
||||
|
||||
None.
|
||||
|
||||
---
|
||||
|
||||
**Total deviations:** 9. One is a framework gap (deferred), one is a bug fix, one is a by-design link limitation, and the rest are plan shape, test rework, acceptance-grep notes and a TDD note.
|
||||
**Impact on plan:** every must-have truth is met. The two visible differences from UI-SPEC C26 are the lastmod text field and the switch labels.
|
||||
|
||||
## Known Stubs
|
||||
|
||||
- `contract.ListingSettings.HTMLEnabled`, `HTMLPaths`, `HTMLPageSize` and `KeyFile` stay zero, and `golem15_sitemap_urls.images` stays `[]`, until 33-04 adds the HTML sitemap, IndexNow and image settings (P33-08; WINDOWS.md #29). The C26 copy for those tabs is already in the phrasebooks.
|
||||
|
||||
## Threat Flags
|
||||
|
||||
None outside the plan's threat model. T-33-09 to T-33-12 are mitigated:
|
||||
- every value goes through `xml.EscapeText`, and the test covers `&`, `<` and quotes;
|
||||
- only an http(s) site URL from the setting or config is used, never the Host;
|
||||
- the routes are table-served, throttled and constrained by `Where`;
|
||||
- only published records are listed, and drafts are tested to be absent;
|
||||
- `site_url` is validated, and the settings are guarded by the existing permission.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
- `go test ./... -run X` prints "no tests to run" for the subpackages, while every named root test passes (same as in 33-02).
|
||||
|
||||
## Verification run
|
||||
|
||||
- `go vet ./...` and `go test -short ./...` in sm-sitemap-plugin: green after every commit.
|
||||
- Docker `go test ./... -count=1` in sm-sitemap-plugin: green after every commit except the tracer. At the tracer the old `TestSitemapAdminSmoke` was red; it was already failing before this plan and was removed in 3A.
|
||||
- The plan's named tests all pass: TestSitemapIndex, TestStableNumbering, TestLastmod, TestHreflangAlternates, TestFilterExcludes, TestNoFilter, TestDebouncedRebuild, TestMigrateDefinitions, TestSettingsSingleton, TestImportWinter. TestListing, TestImportFile and TestRebuildCommand pass too.
|
||||
- Acceptance greps: `type Filter interface` and `type Listing interface`, `xml.EscapeText`, the files route, no init in `contract/`, no files under assets, `classes/testdata` or `tests/php`, `golem15.sitemap.settings`, `Lookup[contract.Filter]`, `Publish[contract.Listing]`, `sitemap:rebuild` in schedule.go and console/rebuild.go, `x-default`. All present; see deviation 8 for the two greps that print extra lines.
|
||||
- From summercms.go:
|
||||
- `bash scripts/check-phase33.sh --plugins --marker`: PASS (sitemap, seo and journal vet and short tests; vitest 77 and 244).
|
||||
- `bash scripts/check-phase33.sh --docker`: PASS (sitemap 23, seo 56, journal 124 top-level Postgres tests).
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None for this plan. When the apps move to the new format in 33-13:
|
||||
- run `sitemap:rebuild` once after the deploy;
|
||||
- make sure a conga worker and the scheduler run;
|
||||
- map the root file names (vue-seo-layer v0.2.0 or the nginx rule in the README).
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
- **33-04:** add images, News, IndexNow and the HTML sitemap on this table and settings. Fill the zero `ListingSettings` fields. Use `RebuildResult.Added/Changed/Removed` for IndexNow.
|
||||
- **33-05:** the SEO plugin implements `contract.Filter` (noindex, canonical, redirect) and may use `Decision.Group` for page-registry kinds.
|
||||
- **33-13:** move grzybyfunkcjonalne and demo to the new sitemap.
|
||||
- **33-14:** bump `go.mod` to v0.6.1 and tag the next minor (breaking) version from `git tag`, after v0.1.2. Write the release notes from "Read first".
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- FOUND: every file in key-files.created (checked with `[ -f ]`).
|
||||
- FOUND commits in sm-sitemap-plugin: 3359f1a, 3ba70bb, 84cb27a, 62fc7d7, 653627b, ded1095, 64453d2. `git rev-list --count 049f3ee..HEAD` = 7.
|
||||
- No uncommitted changes in sm-sitemap-plugin.
|
||||
|
||||
---
|
||||
*Phase: 33-seo-output-audit-and-content*
|
||||
*Completed: 2026-11-04*
|
||||
@@ -14,3 +14,11 @@ Out-of-scope discoveries logged during execution; not fixed in the plan that fou
|
||||
status: open
|
||||
- **Possible fix:** teach `lagoon.Validate` the `string` and `url` tokens (`lagoon.ValidateRequest` already knows `string`). This is a framework change and belongs in v0.6.1 or a follow-up patch.
|
||||
- **Why deferred:** 33-02 commits only to sm-journal-plugin, and the fix belongs in the framework. Dropping the tokens from Journal's `Rules()` would loosen validation in a core plugin.
|
||||
|
||||
## From 33-03 (sm-sitemap-plugin)
|
||||
|
||||
- **A repeater row takes no datepicker (framework).** cabana compiles a `datepicker` inside a `repeater` as an error ("type datepicker is not supported in a repeater (want text, textarea, number, checkbox, switch, dropdown or richeditor)"). The sitemap's Additional addresses row (UI-SPEC C26: `url` text left, `lastmod` datepicker right) therefore uses a text field read as `YYYY-MM-DD`, checked on save (`errors.lastmod`). WINDOWS.md #30.
|
||||
status: open
|
||||
- **Possible fix:** support `datepicker` (mode date) in repeater rows in cabana and the SPA's RepeaterField, then switch the field type in `sm-sitemap-plugin/models/settings/fields.yaml`.
|
||||
- **Why deferred:** 33-03 commits only to sm-sitemap-plugin; the gap is generic framework work (additive, but its own change).
|
||||
- **A partial cannot link off the admin's origin (by design).** cabana's partial renderer keeps only root-relative `a[href]` values (`safePartialURL`), so the S1 files table links each file to the backend route that serves it (`/sitemap.xml`, `/_sitemap/files/<name>`) instead of the public site URL; the note under the table names the public index address. No change needed unless public links in partials become a requirement.
|
||||
|
||||
Reference in New Issue
Block a user