sm-sitemap-plugin/README.md

golem15.sitemap

The compiled sitemap plugin for SummerCMS applications: it serves /sitemap.xml and its stylesheet from one sitemap definition editors manage in the admin, byte for byte the way the WinterCMS Golem15.Sitemap plugin did.

import _ "git.golem15.com/golem15/sm-sitemap-plugin"

Module git.golem15.com/golem15/sm-sitemap-plugin, package sitemap, plugin ID golem15.sitemap.

Overview

The plugin is a compiled SummerCMS plugin. Its init registers it with the framework, so an application loads it by listing it in summer.yaml; summer build then emits the blank import above into the application's plugins.gen.go.

It is a port of the WinterCMS plugin for headless sites. An application has one sitemap definition (models.Definition, theme key default), a list of items: explicit URLs and items of the menu item types other plugins register with the framework's registry (signpost). Requires() is empty: content plugins such as the Journal register their types and the sitemap reads them through the registry.

The output keeps the PHP plugin's bytes: element order, escaping, whitespace, date format, headers and the frontend_url rewrite are not changed. The goldens in classes/testdata are generated from the PHP code and pin them.

Features

• Routes. Anonymous GET /sitemap.xml and GET /sitemap.xsl at the application root, without the backend middleware. /sitemap.xml is throttled per client IP and its output is cached for a short time.
• One definition. Table golem15_sitemap_definitions keeps the PHP columns (id, theme nullable and indexed, data text, timestamps). The theme column holds the fixed key default (models.DefinitionTheme); /sitemap.xml serves that row. models.Definition.Fillable is exactly theme and data, the PHP $fillable with $guarded = ['*'].
• Items. An item is a URL (type: url) or a registry menu item type. Winter.Pages static pages, all static pages and the CMS page field are not ported. Stored item JSON in the PHP shape decodes tolerantly: numbers, strings and booleans are coerced, and unknown keys (cmsPage, code, exists, fillable and any other) are ignored. A save writes the PHP toArray key set (changefreq, priority, nesting, type, url, reference, cmsPage as null, urlPattern).
• Settings. A singleton row (golem15_sitemap_settings, id 1) holds the frontend URL, the frontend default locale, the updates manifest path and the priority tiers, with the PHP defaults.
• Priority tiers. Rules in config (golem15.sitemap.priority_tiers) map URL paths to five priority settings; the shipped defaults are the PHP plugin's rules.
• Hooks. Typed festival events classes.BeforeAddItem and classes.AddItem replace the PHP events golem15.sitemap.beforeAddItem and golem15.sitemap.addItem.
• Admin. The definition opens from the Settings page as a singleton form with a repeater of item rows; a second settings entry holds the settings form. Permission golem15.sitemap.access_definitions.
• Commands. sitemap:import (the PHP JSON import) and sitemap:import-winter (a copy of a WinterCMS site's definition and settings).
• Phrasebooks. lang/en/lang.yaml and lang/pl/lang.yaml keep the PHP key names.

Usage

Mount the plugin in an application as a git submodule at plugins/golem15/sitemap, add it to the application's go.work and go.mod, and list it in summer.yaml:

plugins:
  - id: golem15.sitemap
    module: git.golem15.com/golem15/sm-sitemap-plugin

The plugin's own go.mod and go.work replace the framework with ../summercms.go for a sibling checkout.

/sitemap.xml and /sitemap.xsl

GET /sitemap.xml answers:

• 404 when there is no definition with the theme default;
• 200 with an empty body when the definition has no items;
• otherwise the sitemap, Content-Type: application/xml, Cache-Control: no-cache, private.

Generating the sitemap resolves every registry item (all published posts, all categories, their translations), so the route is guarded:

• Throttle. The surf bucket sitemap-xml (sitemap.BucketSitemap) allows 60 requests a minute per client IP, read behind the host's trusted proxies. The 429 body is surf's {"message":"Too Many Attempts."}. /sitemap.xsl is not throttled.
• Cache. The generated document is kept in memory for golem15.sitemap.cache_seconds seconds (sitemap.CacheConfigKey, default 60; 0 turns it off, a negative value fails the boot). The cache key covers the root URL and the stored definition and settings, so a save of either is served at once; a change of registry content (a new or unpublished post) shows once the entry expires. One generation runs at a time; concurrent requests for a missing entry wait for it.

The document is the PHP DOMDocument::saveXML() format: the XML declaration and the xml-stylesheet processing instruction each followed by a newline, everything else on one line ending with </urlset> and a newline. Items that all resolve to nothing produce a self-closing <urlset .../>. Text is escaped like libxml (&, <, > and carriage return; attributes also ", tab and newline); single quotes and UTF-8 stay literal.

Each URL is written in the PHP order (classes.Generator):

1. The 50000th URL is written, the 50001st is not (classes.MaxURLs).
2. A URL item's address goes through the port of Winter's Url::to (classes.URLTo): root plus the path trimmed of slashes, / keeps its slash, every path segment re-encoded like rawurlencode, valid absolute, mailto:, tel: and fragment URLs kept.
3. lastmod is the record's modification time, else the updates manifest's date for the URL path, else omitted. It is written as 2006-01-02T15:04:05+00:00 in UTC.
4. classes.BeforeAddItem may drop the URL.
5. A URL that does not come out as an http or https URL (for example a javascript: URL an import or a registry type let through) is dropped, and so is such an alternate. The PHP plugin wrote any URL Url::to kept; the stylesheet renders <loc> as a link on the site's origin.
6. The priority is the first matching tier's setting when tiers are on, else the item's priority, written as stored text (1.0 stays 1.0).
7. loc, lastmod, changefreq and priority are written, then the alternates as xhtml:link elements, only when there are more than one.
8. classes.AddItem may append XML before </url>.

A registry item is resolved with its URL pattern, its nesting flag and the settings' frontend default locale as the unprefixed locale. A resolved item with a URL is written, then its sub-items once each with their own URL. A list type (a resolved item without a URL, such as all posts) writes each sub-item once per alternate, with that alternate's URL as loc and every alternate as hreflang, which is what the PHP list types returned.

When the settings hold a frontend URL, the backend URL (app.url when set, else the request's scheme and Host) is replaced literally inside every <loc>, every xhtml:link href and the stylesheet href, as the PHP str_replace did. The rewrite runs only for a frontend URL models.AllowedFrontendURL accepts, and a <loc> or href that would not stay an http or https URL keeps its value. The scheme is https for a TLS connection, or when the immediate peer is in http.trusted_proxies and its X-Forwarded-Proto says https.

GET /sitemap.xsl serves the PHP plugin's stylesheet byte for byte, Content-Type: text/xsl; charset=UTF-8.

A frontend served from another host routes both paths to the application, before its catch-all:

location = /sitemap.xml {
    proxy_pass http://127.0.0.1:8096;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}
location = /sitemap.xsl {
    proxy_pass http://127.0.0.1:8096;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
}

Item types

Type	Item fields used
url	url, changefreq, priority
A registry type (for example journal-post)	reference (when the type takes one), urlPattern (overrides the host and default patterns), nesting, changefreq, priority

The admin's type dropdown offers url and every registry type in registry order with its label. The reference dropdown depends on the row's type and lists the type's references (signpost.ReferenceOptions); url and types without references take none, and a reference the type does not offer is refused on save.

A save also refuses, per row, a missing reference for a registry type that takes one (such a row would resolve to nothing and be left out of the sitemap), a url that is not a path (relative or starting with a single /) or an absolute http/https URL (classes.AllowedItemURL: no other scheme, no protocol-relative //host, no bare fragment, no surrounding whitespace or control characters), and a urlPattern that is not a path starting with a single / (signpost.ValidItemPattern). The errors are keyed data.<row>.reference, data.<row>.url and data.<row>.urlPattern.

Settings

Setting	Default	Meaning
frontend_url	empty	The public frontend URL that replaces the backend URL in the XML: empty, or an absolute http or https URL with a host and without whitespace, control characters, a double quote, a less-than sign, a greater-than sign or an ampersand (models.AllowedFrontendURL). A stored value outside this form is not applied and the sitemap keeps the backend URL.
frontend_default_locale	empty	The locale whose URLs carry no prefix, passed to registry types; empty leaves it to the type.
updates_manifest_path	empty	A JSON file mapping a URL path to an ISO 8601 date, used as lastmod for items without their own date.
enable_priority_tiers	on	Tier priorities replace item priorities.
priority_homepage, priority_services, priority_blog, priority_categories, priority_legal	1.0, 0.8, 0.6, 0.4, 0.2	The tier values, stored as text.

A save refuses a frontend_url outside the accepted form with a 422 keyed frontend_url (phrase golem15.sitemap::lang.errors.frontend_url), and sitemap:import-winter stops with an error. The PHP plugin stored any text and wrote it unescaped into <loc>, the hreflang links and the stylesheet instruction.

The updates manifest is read only when it is a regular file of at most 5 MiB (classes.ManifestMaxBytes) holding a JSON object; paths match with the leading slash added and the trailing slash removed (/ stays). Any error omits lastmod and is logged without the file's content. The manifest is a frontend build output, for example a script that writes the git commit date of each static page's source:

{
  "/": "2026-06-10T09:12:44+00:00",
  "/about": "2026-05-02T14:31:00+00:00"
}

Priority tiers

With enable_priority_tiers on, the URL's path without trailing slashes ("" for the homepage) is matched against the rules in golem15.sitemap.priority_tiers in order; the first match's setting value is the priority, and a URL no rule matches keeps its item's priority. A rule names a setting and either exact (a list of paths) or pattern (a regular expression in RE2 syntax). The rules are compiled at boot: an unknown setting, a rule with neither or both of exact and pattern, or a pattern that does not compile fails the boot (classes.CompileTiers).

The plugin's config/config.yaml ships the rules the PHP plugin had in code, so a site configured the same way gets the same priorities. A host replaces the whole list in its own config, for example config/golem15.yaml:

sitemap:
  priority_tiers:
    - setting: priority_homepage
      exact: ["", "/en"]
    - setting: priority_blog
      pattern: '^/(en/)?articles/[^/]+$'

Hooks

classes.BeforeAddItem fires through festival.Bus.UntilHandled before a URL is counted; a listener that sets Skip drops it (the PHP listener returned false). classes.AddItem fires through festival.Bus.Fire after a URL's elements are built; a listener may append escaped child XML to Extra. Both carry the definition item and a classes.ItemInfo (URL, modification time, written lastmod, alternates).

func (p *Plugin) Boot(app *backpack.App) error {
	app.Events.Listen("acme.seo", func(ctx context.Context, e *classes.BeforeAddItem) error {
		if strings.HasSuffix(e.Info.URL, "/draft") {
			e.Skip = true
		}
		return nil
	})
	app.Events.Listen("acme.seo", func(ctx context.Context, e *classes.AddItem) error {
		e.Extra = append(e.Extra, "<mobile>"+classes.EscapeText("yes")+"</mobile>")
		return nil
	})
	return nil
}

Admin

Two entries on the Settings page, category CMS, both requiring golem15.sitemap.access_definitions (developer role):

• Sitemap links to the definitions controller (golem15.sitemap.definitions), a singleton form: the server opens the definition with the theme default, creating it when absent, and the form has no list, create or delete. Its one field is a repeater on data whose rows hold type, reference, URL, URL pattern, priority, change frequency and the nesting checkbox.
• Sitemap settings is the settings form.

The WinterCMS forms are adapted to the admin's YAML:

• The definition form's tab is flattened, and the SitemapItems form widget is replaced by a type: repeater field on data.
• span: none and cssClass become left and right.
• The settings form's nginx hint and Priority Tiers section are field comments and this README; the trigger conditions on the tier fields are dropped, so those fields always show and their comments say when they apply.
• The tier comments no longer name one site's paths.
• The CMS page field and its messages are not ported; the editor's empty-definition action (empty_confirm) is not ported.
• The form's "View sitemap.xml" subtitle link (definition.view) waits for a form schema key in the framework; the phrase exists.

API reference

Identifier	Description
sitemap.Plugin	Compiled plugin; ID is golem15.sitemap.
sitemap.Plugin.Routes	GET /sitemap.xml (throttled, cached) and GET /sitemap.xsl.
sitemap.Plugin.Buckets	Registers the sitemap-xml throttle bucket: 60 requests a minute per client IP.
sitemap.BucketSitemap	sitemap-xml, the throttle bucket of /sitemap.xml.
sitemap.CacheConfigKey	golem15.sitemap.cache_seconds, the seconds a generated sitemap is cached.
sitemap.Plugin.Boot	Compiles the tier rules and starts an empty sitemap cache; a bad rule or a negative cache_seconds fails the boot.
sitemap.Plugin.ConfigFS	The default config/config.yaml (cache time and tier rules).
sitemap.Plugin.Commands	Registers sitemap:import and sitemap:import-winter.
sitemap.Plugin.AdminControllers	The definitions controller.
sitemap.Plugin.Settings	The Sitemap link and the Sitemap settings form.
sitemap.Plugin.Permissions	golem15.sitemap.access_definitions.
sitemap.Plugin.LangFS, sitemap.Plugin.AdminFS	Embedded phrasebooks and admin YAML.
sitemap.Plugin.Migrations, sitemap.Plugin.Models	The two migrations and the two models.
models.Definition	The definition row; Items and SetItems read and write its item list.
models.DefinitionTheme	The fixed theme key default.
models.DefinitionItem, models.DefinitionItems	One item and the stored list, with the tolerant PHP-shaped decoding.
models.ItemTypeURL	The built-in item type url.
models.Text, models.DecodeText	A text field's value; the tolerant decoding of one stored text value.
models.Settings, models.DefaultSettings	The settings row and its PHP defaults; Settings.Priority reads a tier value, Settings.BeforeValidate refuses a frontend_url outside the accepted form on a save.
models.AllowedFrontendURL	Whether a value may be the frontend_url setting: empty, or an absolute http/https URL with a host and without whitespace, control characters, ", <, > or &.
models.PriorityHomepage, models.PriorityServices, models.PriorityBlog, models.PriorityCategories, models.PriorityLegal, models.PriorityDefaults	The tier setting keys and their defaults.
controllers.DefinitionsControllerID, controllers.PermissionAccessDefinitions	golem15.sitemap.definitions and golem15.sitemap.access_definitions.
classes.Generator	Writes a definition's sitemap; Generate, Stylesheet.
classes.Resolver	What the generator resolves registry items through (*signpost.Registry).
classes.Serialize, classes.URLEntry, classes.Alternate	The libxml-format writer and its entries.
classes.EscapeText, classes.EscapeAttr	libxml's text and attribute escaping.
classes.URLTo, classes.BuildURL, classes.URLPath	Ports of Winter Url::to, Url::buildUrl and parse_url(..., PHP_URL_PATH).
classes.AllowedItemURL	Whether a url item's address may be stored: a path or an absolute http/https URL.
classes.Rewrite, classes.BackendURL	The frontend_url rewrite, applied only for a value models.AllowedFrontendURL accepts, and the backend URL it replaces.
classes.TiersConfigKey, classes.TierConfig, classes.TierRule, classes.Tiers, classes.CompileTiers, classes.LoadTiers	The tier rules.
classes.Manifest, classes.LoadManifest, classes.ManifestMaxBytes	The updates manifest.
classes.BeforeAddItem, classes.AddItem, classes.ItemInfo	The hooks.
classes.MaxURLs	50000.
console.Import, console.ImportFile	sitemap:import.
console.ImportWinter, console.ImportWinterFiles, console.WinterOptions, console.WinterReport	sitemap:import-winter.

CLI commands

Command	Arguments and flags	Effect
sitemap:import	FILE, --theme, --append	The PHP ImportCommand: reads a JSON array of {url, changefreq, priority} entries (monthly and 0.5 by default). Without --append the definition keeps its non-URL items first and the imported URLs replace its URL items; with it the URLs follow every existing item. The definition is always default; a --theme other than default is reported. An entry whose URL classes.AllowedItemURL refuses stops the import.
sitemap:import-winter	--definitions FILE, --settings FILE, --theme, --map old=new (repeatable), --page-url reference=/path (repeatable), --force, --dry-run	Copies a WinterCMS export into the definition and the settings row. URL items are kept; static and CMS page items are dropped unless --page-url names the URL of their reference (for example --page-url blog=/blog copies the CMS page item blog as a url item /blog); --map renames types first (for example --map blog-post=journal-post); an item of an unknown type, or whose reference is not among its type's current references, is dropped; a url item whose URL classes.AllowedItemURL refuses is dropped, and so is a page item whose --page-url it refuses; a urlPattern that is not a path is cleared. Every dropped, remapped or cleared item and every dropped cmsPage is printed. A definition that already has items is refused without --force; --dry-run writes nothing. Settings missing from the export take the defaults; a frontend_url which models.AllowedFrontendURL refuses stops the import before anything is written, also with --dry-run. --theme picks a definition when the export holds several.

Both commands open the database themselves when the application has not published one.

The export runs on the WinterCMS database with the read-only scripts in scripts/:

mysql -N -B --raw <database> < scripts/winter-export-definitions.sql > sitemap-definitions.jsonl
mysql -N -B --raw <database> < scripts/winter-export-settings.sql > sitemap-settings.json
./bin/acme sitemap:import-winter --definitions sitemap-definitions.jsonl --settings sitemap-settings.json --dry-run
./bin/acme sitemap:import-winter --definitions sitemap-definitions.jsonl --settings sitemap-settings.json

Dependencies

• SummerCMS modules: backpack, bonfire, cabana, compass, festival, lagoon, pact, party, signpost, surf; bouncer in tests. cabana is a regular dependency: models and controllers return its validation error.
• Third-party: gorm.io/gorm, github.com/go-gormigrate/gormigrate/v2. Test-only: github.com/testcontainers/testcontainers-go, github.com/jackc/pgx/v5.

Testing

go test -short ./...   # goldens and config checks, no Docker
go test ./...          # also the Postgres smoke tests

TestSitemapGolden (classes) feeds testdata/tracer.json, urlto.json and full.json through the Go port and compares bytes with the goldens; it also checks the 50000 URL cap. TestSitemapTiersConfig checks the tier rule refusals, a host's replacement of the rule list and the manifest's fail-safe reads. TestRewriteUnsafeFrontendURL (classes), TestAllowedFrontendURL and TestSettingsBeforeValidate (models) and TestWinterSettingsFrontendURL (console) are the Docker-free checks of the frontend_url rules: the accepted form, the rewrite that is not applied for any other value, the save hook and the import refusal. The Postgres smokes (TestSitemapTracer, TestSitemapImportSmoke, TestSitemapAdminSmoke, TestImportWinter, TestSitemapCacheAndThrottle) boot the plugin with a test content plugin and drive the routes, the import commands and the admin API; they cover the 422 of a refused frontend_url save and a value stored straight in the row, which the sitemap does not apply.

The goldens are generated from the PHP plugin's generation code and the vendored Winter URL generator of any WinterCMS application, and committed, so CI needs no PHP:

WINTER_VENDOR=/path/to/winter-app/vendor php tests/php/gen-goldens.php classes/testdata/full.json classes/testdata/full.golden