Jakub Zych 4e55939a1f test(17-06): sitemap parity unit tests
- TestURLTo (every Url::to golden case, BackendURL, URLPath),
  TestXMLEscaping (PHP XmlEscapingTest as behaviour), TestTiers (each
  default rule, first match, off, bad pattern fails Boot), TestRewrite,
  TestHooks, TestURLCap, TestManifest (fail-safe reads, key
  normalization, dates), TestFillable (PHP MassAssignmentTest),
  TestDefinitionItemJSON, TestBackendURL (trusted forwarded proto)
- TestImportWinter on Postgres: maps and drops with report, --theme,
  --dry-run, --force, settings upsert, refusals
2026-10-12 19:38:04 +02:00

golem15.sitemap

The compiled sitemap plugin for SummerCMS applications: it serves /sitemap.xml and its stylesheet from one sitemap definition editors manage in the admin, byte for byte the way the WinterCMS Golem15.Sitemap plugin did.

import _ "git.golem15.com/golem15/sm-sitemap-plugin"

Module git.golem15.com/golem15/sm-sitemap-plugin, package sitemap, plugin ID golem15.sitemap.

Overview

The plugin is a compiled SummerCMS plugin. Its init registers it with the framework, so an application loads it by listing it in summer.yaml; summer build then emits the blank import above into the application's plugins.gen.go.

It is a port of the WinterCMS plugin for headless sites. An application has one sitemap definition (models.Definition, theme key default), a list of items: explicit URLs and items of the menu item types other plugins register with the framework's registry (signpost). Requires() is empty: content plugins such as the Journal register their types and the sitemap reads them through the registry.

The output keeps the PHP plugin's bytes: element order, escaping, whitespace, date format, headers and the frontend_url rewrite are not changed. The goldens in classes/testdata are generated from the PHP code and pin them.

Features

  • Routes. Anonymous GET /sitemap.xml and GET /sitemap.xsl at the application root, outside any throttle group and without the backend middleware.
  • One definition. Table golem15_sitemap_definitions keeps the PHP columns (id, theme nullable and indexed, data text, timestamps). The theme column holds the fixed key default (models.DefinitionTheme); /sitemap.xml serves that row. models.Definition.Fillable is exactly theme and data, the PHP $fillable with $guarded = ['*'].
  • Items. An item is a URL (type: url) or a registry menu item type. Winter.Pages static pages, all static pages and the CMS page field are not ported. Stored item JSON in the PHP shape decodes tolerantly: numbers, strings and booleans are coerced, and unknown keys (cmsPage, code, exists, fillable and any other) are ignored. A save writes the PHP toArray key set (changefreq, priority, nesting, type, url, reference, cmsPage as null, urlPattern).
  • Settings. A singleton row (golem15_sitemap_settings, id 1) holds the frontend URL, the frontend default locale, the updates manifest path and the priority tiers, with the PHP defaults.
  • Priority tiers. Rules in config (golem15.sitemap.priority_tiers) map URL paths to five priority settings; the shipped defaults are the PHP plugin's rules.
  • Hooks. Typed festival events classes.BeforeAddItem and classes.AddItem replace the PHP events golem15.sitemap.beforeAddItem and golem15.sitemap.addItem.
  • Admin. The definition opens from the Settings page as a singleton form with a repeater of item rows; a second settings entry holds the settings form. Permission golem15.sitemap.access_definitions.
  • Commands. sitemap:import (the PHP JSON import) and sitemap:import-winter (a copy of a WinterCMS site's definition and settings).
  • Phrasebooks. lang/en/lang.yaml and lang/pl/lang.yaml keep the PHP key names.

Usage

Mount the plugin in an application as a git submodule at plugins/golem15/sitemap, add it to the application's go.work and go.mod, and list it in summer.yaml:

plugins:
  - id: golem15.sitemap
    module: git.golem15.com/golem15/sm-sitemap-plugin

The plugin's own go.mod and go.work replace the framework with ../summercms.go for a sibling checkout.

/sitemap.xml and /sitemap.xsl

GET /sitemap.xml answers:

  • 404 when there is no definition with the theme default;
  • 200 with an empty body when the definition has no items;
  • otherwise the sitemap, Content-Type: application/xml, Cache-Control: no-cache, private.

The document is the PHP DOMDocument::saveXML() format: the XML declaration and the xml-stylesheet processing instruction each followed by a newline, everything else on one line ending with </urlset> and a newline. Items that all resolve to nothing produce a self-closing <urlset .../>. Text is escaped like libxml (&, <, > and carriage return; attributes also ", tab and newline); single quotes and UTF-8 stay literal.

Each URL is written in the PHP order (classes.Generator):

  1. The 50000th URL is written, the 50001st is not (classes.MaxURLs).
  2. A URL item's address goes through the port of Winter's Url::to (classes.URLTo): root plus the path trimmed of slashes, / keeps its slash, every path segment re-encoded like rawurlencode, valid absolute, mailto:, tel: and fragment URLs kept.
  3. lastmod is the record's modification time, else the updates manifest's date for the URL path, else omitted. It is written as 2006-01-02T15:04:05+00:00 in UTC.
  4. classes.BeforeAddItem may drop the URL.
  5. The priority is the first matching tier's setting when tiers are on, else the item's priority, written as stored text (1.0 stays 1.0).
  6. loc, lastmod, changefreq and priority are written, then the alternates as xhtml:link elements, only when there are more than one.
  7. classes.AddItem may append XML before </url>.

A registry item is resolved with its URL pattern, its nesting flag and the settings' frontend default locale as the unprefixed locale. A resolved item with a URL is written, then its sub-items once each with their own URL. A list type (a resolved item without a URL, such as all posts) writes each sub-item once per alternate, with that alternate's URL as loc and every alternate as hreflang, which is what the PHP list types returned.

When the settings hold a frontend URL, the backend URL (app.url when set, else the request's scheme and Host) is replaced literally inside every <loc>, every xhtml:link href and the stylesheet href, as the PHP str_replace did. The scheme is https for a TLS connection, or when the immediate peer is in http.trusted_proxies and its X-Forwarded-Proto says https.

GET /sitemap.xsl serves the PHP plugin's stylesheet byte for byte, Content-Type: text/xsl; charset=UTF-8.

A frontend served from another host routes both paths to the application, before its catch-all:

location = /sitemap.xml {
    proxy_pass http://127.0.0.1:8096;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}
location = /sitemap.xsl {
    proxy_pass http://127.0.0.1:8096;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
}

Item types

Type Item fields used
url url, changefreq, priority
A registry type (for example journal-post) reference (when the type takes one), urlPattern (overrides the host and default patterns), nesting, changefreq, priority

The admin's type dropdown offers url and every registry type in registry order with its label. The reference dropdown depends on the row's type and lists the type's references (signpost.ReferenceOptions); url and types without references take none, and a reference the type does not offer is refused on save.

Settings

Setting Default Meaning
frontend_url empty The public frontend URL that replaces the backend URL in the XML.
frontend_default_locale empty The locale whose URLs carry no prefix, passed to registry types; empty leaves it to the type.
updates_manifest_path empty A JSON file mapping a URL path to an ISO 8601 date, used as lastmod for items without their own date.
enable_priority_tiers on Tier priorities replace item priorities.
priority_homepage, priority_services, priority_blog, priority_categories, priority_legal 1.0, 0.8, 0.6, 0.4, 0.2 The tier values, stored as text.

The updates manifest is read only when it is a regular file of at most 5 MiB (classes.ManifestMaxBytes) holding a JSON object; paths match with the leading slash added and the trailing slash removed (/ stays). Any error omits lastmod and is logged without the file's content. The manifest is a frontend build output, for example a script that writes the git commit date of each static page's source:

{
  "/": "2026-06-10T09:12:44+00:00",
  "/about": "2026-05-02T14:31:00+00:00"
}

Priority tiers

With enable_priority_tiers on, the URL's path without trailing slashes ("" for the homepage) is matched against the rules in golem15.sitemap.priority_tiers in order; the first match's setting value is the priority, and a URL no rule matches keeps its item's priority. A rule names a setting and either exact (a list of paths) or pattern (a regular expression in RE2 syntax). The rules are compiled at boot: an unknown setting, a rule with neither or both of exact and pattern, or a pattern that does not compile fails the boot (classes.CompileTiers).

The plugin's config/config.yaml ships the rules the PHP plugin had in code, so a site configured the same way gets the same priorities. A host replaces the whole list in its own config, for example config/golem15.yaml:

sitemap:
  priority_tiers:
    - setting: priority_homepage
      exact: ["", "/en"]
    - setting: priority_blog
      pattern: '^/(en/)?articles/[^/]+$'

Hooks

classes.BeforeAddItem fires through festival.Bus.UntilHandled before a URL is counted; a listener that sets Skip drops it (the PHP listener returned false). classes.AddItem fires through festival.Bus.Fire after a URL's elements are built; a listener may append escaped child XML to Extra. Both carry the definition item and a classes.ItemInfo (URL, modification time, written lastmod, alternates).

func (p *Plugin) Boot(app *backpack.App) error {
	app.Events.Listen("acme.seo", func(ctx context.Context, e *classes.BeforeAddItem) error {
		if strings.HasSuffix(e.Info.URL, "/draft") {
			e.Skip = true
		}
		return nil
	})
	app.Events.Listen("acme.seo", func(ctx context.Context, e *classes.AddItem) error {
		e.Extra = append(e.Extra, "<mobile>"+classes.EscapeText("yes")+"</mobile>")
		return nil
	})
	return nil
}

Admin

Two entries on the Settings page, category CMS, both requiring golem15.sitemap.access_definitions (developer role):

  • Sitemap links to the definitions controller (golem15.sitemap.definitions), a singleton form: the server opens the definition with the theme default, creating it when absent, and the form has no list, create or delete. Its one field is a repeater on data whose rows hold type, reference, URL, URL pattern, priority, change frequency and the nesting checkbox.
  • Sitemap settings is the settings form.

The WinterCMS forms are adapted to the admin's YAML:

  • The definition form's tab is flattened, and the SitemapItems form widget is replaced by a type: repeater field on data.
  • span: none and cssClass become left and right.
  • The settings form's nginx hint and Priority Tiers section are field comments and this README; the trigger conditions on the tier fields are dropped, so those fields always show and their comments say when they apply.
  • The tier comments no longer name one site's paths.
  • The CMS page field and its messages are not ported; the editor's empty-definition action (empty_confirm) is not ported.
  • The form's "View sitemap.xml" subtitle link (definition.view) waits for a form schema key in the framework; the phrase exists.

API reference

Identifier Description
sitemap.Plugin Compiled plugin; ID is golem15.sitemap.
sitemap.Plugin.Routes GET /sitemap.xml and GET /sitemap.xsl.
sitemap.Plugin.Boot Compiles the tier rules; a bad rule fails the boot.
sitemap.Plugin.ConfigFS The default config/config.yaml (tier rules).
sitemap.Plugin.Commands Registers sitemap:import and sitemap:import-winter.
sitemap.Plugin.AdminControllers The definitions controller.
sitemap.Plugin.Settings The Sitemap link and the Sitemap settings form.
sitemap.Plugin.Permissions golem15.sitemap.access_definitions.
sitemap.Plugin.LangFS, sitemap.Plugin.AdminFS Embedded phrasebooks and admin YAML.
sitemap.Plugin.Migrations, sitemap.Plugin.Models The two migrations and the two models.
models.Definition The definition row; Items and SetItems read and write its item list.
models.DefinitionTheme The fixed theme key default.
models.DefinitionItem, models.DefinitionItems One item and the stored list, with the tolerant PHP-shaped decoding.
models.ItemTypeURL The built-in item type url.
models.Text, models.DecodeText A text field's value; the tolerant decoding of one stored text value.
models.Settings, models.DefaultSettings The settings row and its PHP defaults; Settings.Priority reads a tier value.
models.PriorityHomepage, models.PriorityServices, models.PriorityBlog, models.PriorityCategories, models.PriorityLegal, models.PriorityDefaults The tier setting keys and their defaults.
controllers.DefinitionsControllerID, controllers.PermissionAccessDefinitions golem15.sitemap.definitions and golem15.sitemap.access_definitions.
classes.Generator Writes a definition's sitemap; Generate, Stylesheet.
classes.Resolver What the generator resolves registry items through (*signpost.Registry).
classes.Serialize, classes.URLEntry, classes.Alternate The libxml-format writer and its entries.
classes.EscapeText, classes.EscapeAttr libxml's text and attribute escaping.
classes.URLTo, classes.BuildURL, classes.URLPath Ports of Winter Url::to, Url::buildUrl and parse_url(..., PHP_URL_PATH).
classes.Rewrite, classes.BackendURL The frontend_url rewrite and the backend URL it replaces.
classes.TiersConfigKey, classes.TierConfig, classes.TierRule, classes.Tiers, classes.CompileTiers, classes.LoadTiers The tier rules.
classes.Manifest, classes.LoadManifest, classes.ManifestMaxBytes The updates manifest.
classes.BeforeAddItem, classes.AddItem, classes.ItemInfo The hooks.
classes.MaxURLs 50000.
console.Import, console.ImportFile sitemap:import.
console.ImportWinter, console.ImportWinterFiles, console.WinterOptions, console.WinterReport sitemap:import-winter.

CLI commands

Command Arguments and flags Effect
sitemap:import FILE, --theme, --append The PHP ImportCommand: reads a JSON array of {url, changefreq, priority} entries (monthly and 0.5 by default). Without --append the definition keeps its non-URL items first and the imported URLs replace its URL items; with it the URLs follow every existing item. The definition is always default; a --theme other than default is reported.
sitemap:import-winter --definitions FILE, --settings FILE, --theme, --map old=new (repeatable), --page-url reference=/path (repeatable), --force, --dry-run Copies a WinterCMS export into the definition and the settings row. URL items are kept; static and CMS page items are dropped unless --page-url names the URL of their reference (for example --page-url blog=/blog copies the CMS page item blog as a url item /blog); --map renames types first (for example --map blog-post=journal-post); an item of an unknown type, or whose reference is not among its type's current references, is dropped. Every dropped or remapped item and every dropped cmsPage is printed. A definition that already has items is refused without --force; --dry-run writes nothing. Settings missing from the export take the defaults. --theme picks a definition when the export holds several.

Both commands open the database themselves when the application has not published one.

The export runs on the WinterCMS database with the read-only scripts in scripts/:

mysql -N -B --raw <database> < scripts/winter-export-definitions.sql > sitemap-definitions.jsonl
mysql -N -B --raw <database> < scripts/winter-export-settings.sql > sitemap-settings.json
./bin/acme sitemap:import-winter --definitions sitemap-definitions.jsonl --settings sitemap-settings.json --dry-run
./bin/acme sitemap:import-winter --definitions sitemap-definitions.jsonl --settings sitemap-settings.json

Dependencies

  • SummerCMS modules: backpack, bonfire, compass, festival, lagoon, pact, party, signpost, surf; bouncer and cabana in tests.
  • Third-party: gorm.io/gorm, github.com/go-gormigrate/gormigrate/v2. Test-only: github.com/testcontainers/testcontainers-go, github.com/jackc/pgx/v5.

Testing

go test -short ./...   # goldens and config checks, no Docker
go test ./...          # also the Postgres smoke tests

TestSitemapGolden (classes) feeds testdata/tracer.json, urlto.json and full.json through the Go port and compares bytes with the goldens; it also checks the 50000 URL cap. TestSitemapTiersConfig checks the tier rule refusals, a host's replacement of the rule list and the manifest's fail-safe reads. The Postgres smokes (TestSitemapTracer, TestSitemapImportSmoke, TestSitemapAdminSmoke) boot the plugin with a test content plugin and drive the routes, the import commands and the admin API.

The goldens are generated from the PHP plugin's generation code and the vendored Winter URL generator of any WinterCMS application, and committed, so CI needs no PHP:

WINTER_VENDOR=/path/to/winter-app/vendor php tests/php/gen-goldens.php classes/testdata/full.json classes/testdata/full.golden
Description
No description provided
Readme 320 KiB
Languages
Go 87.9%
PHP 6.9%
XSLT 5.2%