golem15.sitemap

The compiled sitemap plugin for SummerCMS applications: it serves a sitemap index at /sitemap.xml whose per-type files of a configurable size keep their numbers across rebuilds and carry real lastmod and hreflang alternates, built from every content type's published records.

import _ "git.golem15.com/golem15/sm-sitemap-plugin"

Module git.golem15.com/golem15/sm-sitemap-plugin, package sitemap, plugin ID golem15.sitemap.

Overview

The plugin is a compiled SummerCMS plugin. Its init registers it with the framework, so an application loads it by listing it in summer.yaml; summer build then emits the blank import above into the application's plugins.gen.go.

Content plugins describe their records to the framework's registry (signpost content types). The sitemap reads them through signpost.Registry.Records, page by page, and never imports a content plugin: Requires() is empty. Every listed address is a row of the slot table golem15_sitemap_urls, which places it in a file sitemap-<slug>-<n>.xml of its group. A rebuild keeps every address in its file, appends new ones to the last file with room and leaves a hole where an address left; only a changed file size renumbers the files. The routes serve from the table and never generate per request.

Other plugins narrow the sitemap through the contract package: the SEO plugin publishes a contract.Filter that leaves noindex, canonicalised and redirected addresses out, and reads the sitemap through the contract.Listing the plugin publishes. Without a Filter every published address is listed.

Breaking change from v0.1 (D-32)

v0.1 served one /sitemap.xml in the WinterCMS byte format from a definition of items, with priority, change frequency, priority tiers, an update manifest and the /sitemap.xsl stylesheet. The per-type format replaces it:

• /sitemap.xml is now a sitemap index; its URL is unchanged, so robots.txt Sitemap: lines and Search Console submissions keep working.
• The files are /sitemap-<slug>-<n>.xml at the site root. Go serves them at /_sitemap/files/{file}; the site's SEO layer (vue-seo-layer) or its proxy maps the root names there.
• /sitemap.xsl answers 404, and no <priority> or <changefreq> is written.
• The definitions screen and the old settings tile are one settings singleton. The url items of the definition move to the additional addresses, frontend_url to the site address and frontend_default_locale to the default locale (migration 202610280002). Reference items are dropped: the content types list their records. The golem15_sitemap_definitions table and the Phase 17 settings columns stay in the database, unread, so a rollback loses nothing.
• The config keys golem15.sitemap.cache_seconds and golem15.sitemap.priority_tiers and the festival events BeforeAddItem and AddItem are gone.

Features

• Index and files. GET /sitemap.xml lists one <sitemap> per file with an address, with the newest lastmod of its addresses. GET /_sitemap/files/{file} serves one file: <loc>, <lastmod> and xhtml:link alternates per address, X-Robots-Tag: noindex. Both are application/xml, anonymous and throttled per client IP.
• Stable numbering. A file holds at most the file size setting's addresses (default 1000, 1 to 50 000). Addresses keep their file number; new ones are appended oldest first; a deleted, unpublished or excluded address leaves a hole. A changed file size renumbers every file in the next rebuild.
• Real lastmod. A row's lastmod starts at the record's last change and then moves only when a pact.RecordSaved reports a change outside golem15.sitemap.lastmod_ignore: a no-op save, a counter update or a rebuild leaves it as it was.
• hreflang. Every address of a record with more than one listed locale carries an alternate per locale, itself included, plus x-default on the default locale's address. A locale without an address gets none, a single-locale record none.
• Groups. A content type's group is the last segment of its key in plural form (journal-post: posts, journal-category: categories, seo-page: pages), or the file name set in the settings. Additional addresses go to pages. A Filter may move an address to another group (offers).
• Rebuilds. A conga job (golem15.sitemap.rebuild): queued after record changes, debounced to one per golem15.sitemap.rebuild_delay_seconds; dispatched on every settings save; run every night at 03:00 by the scheduled sitemap:rebuild; and by hand.
• Settings singleton. One tile in the CMS settings category: Files (site address, file size, the files table), Content types (per type: include, file name, HTML sitemap heading) and Additional addresses.
• Contract. contract.Filter for other plugins to implement and contract.Listing for other plugins to read (D-95).

Usage

Mount the plugin in an application as a git submodule at plugins/golem15/sitemap, add it to the application's go.work and go.mod, and list it in summer.yaml:

plugins:
  - id: golem15.sitemap
    module: git.golem15.com/golem15/sm-sitemap-plugin

The plugin's own go.mod and go.work replace the framework with ../summercms.go for a sibling checkout.

The application needs a conga worker (summer queue:work) for the rebuild job, and the scheduler for the nightly rebuild. Without a worker, run sitemap:rebuild yourself.

Routes

Route	Answer
GET /sitemap.xml	The index. 404 while no site address is configured (a <loc> is never built from the request's Host). Before the first rebuild it is an empty <sitemapindex>.
GET /_sitemap/files/{file}	One file, {file} matching sitemap-[a-z0-9-]+-[0-9]+\.xml. 404 for an unknown file or one whose addresses all left.
GET /sitemap.xsl	Gone: 404.

Both routes use the surf throttle bucket sitemap-xml (sitemap.BucketSitemap): 60 requests a minute per client IP, read behind the host's trusted proxies. Every value is written through encoding/xml's escaping.

The index lists the files at the site root, for example:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap><loc>https://www.example.com/sitemap-posts-1.xml</loc><lastmod>2026-10-28T09:00:00Z</lastmod></sitemap>
</sitemapindex>

A site whose frontend proxies the backend maps /sitemap-<slug>-<n>.xml to /_sitemap/files/sitemap-<slug>-<n>.xml. vue-seo-layer does it from v0.2.0; with nginx:

location ~ ^/(sitemap-[a-z0-9-]+-[0-9]+\.xml)$ {
    proxy_pass http://backend/_sitemap/files/$1;
}

Site address

Every <loc> starts with the Site address setting; when it is empty, with the config key golem15.sitemap.site_url; else with app.url. Each must be an absolute http/https URL without whitespace, quotes, <, > or & (models.AllowedFrontendURL); a value outside that form is skipped.

Lastmod

classes.RecordSaved handles every pact.RecordSaved of a model a content type declares:

• a change of an attribute on the record's own row moves the lastmod of all its addresses to now;
• a translated change (RecordChange.Locale set) moves only that locale's address;
• changes of attributes in golem15.sitemap.lastmod_ignore (default updated_at, created_at, views, sort_order) and a save without changes move nothing.

A pact.RecordDeleted takes the record's addresses out of the files at once. Both queue a debounced rebuild. A rebuild sets the lastmod of a new address from the record's LastMod (or the Filter's Decision.LastMod) and never moves an existing one; an additional address carries the lastmod the editor gives it.

Rebuilds

classes.Rebuild walks signpost.Registry.ContentTypes in registration order, pages Records 500 at a time, keeps the published records' locales with an address, asks the Filter per page, adds the additional addresses and writes everything in one transaction that holds the settings row (two rebuilds never interleave). It returns the added, changed and removed absolute URLs.

Record events set rebuild_pending_until with a conditional update and enqueue one rebuild with that delay only when no pending time lies in the future; the job clears it when it starts. A settings save dispatches a rebuild at once (a summer_jobs row labelled "Rebuild the sitemaps").

Filter

A plugin that decides which addresses are indexable implements contract.Filter and publishes it in its Register. It imports the contract package only:

import "git.golem15.com/golem15/sm-sitemap-plugin/contract"

type noindexFilter struct{}

func (noindexFilter) Decide(ctx context.Context, batch []contract.Candidate) ([]contract.Decision, error) {
	out := make([]contract.Decision, len(batch))
	for i, c := range batch {
		if strings.HasPrefix(c.URL, "/drafts/") {
			out[i] = contract.Decision{Exclude: true, Reason: contract.ReasonNoindex}
		}
	}
	return out, nil
}

func (noindexFilter) Groups(context.Context) ([]contract.Group, error) { return nil, nil }

func (p *Plugin) Register(app *backpack.App) error {
	return app.Publish[contract.Filter](noindexFilter{})
}

Decide gets one page of records of one content type (or the additional addresses, type key extra) and answers one decision per candidate in order. An error or a wrong number of decisions fails the rebuild, and the files stay as they were. Excluded addresses are counted by reason in the settings screen (noindex, canonical, redirect; a type switched off counts as type).

Listing

The plugin publishes classes.Listing as contract.Listing:

• Settings answers the site address, the index address, the default locale and the groups with their listed addresses, their slug and their titles (the content type's HTML sitemap heading, else the Filter's group titles);
• Entries pages a group's listed addresses (EntriesQuery{Group, Locale, After, Limit}, 100 by default, at most 1000) with absolute URLs, titles and lastmod, Next and Total.

The HTML sitemap and IndexNow fields of contract.ListingSettings stay zero until those settings exist.

Admin

The Settings page has one Sitemaps tile (icon network, category CMS, permission golem15.sitemap.access_definitions). It opens the singleton controller golem15.sitemap.settings:

• Files: Site address, Addresses per file and the files table: the index and every file with its address count and newest lastmod, the addresses left out by reason, and a note naming the public index address. Before the first build it says there are no files yet; after a save and until the rebuild ran, that it is rebuilding.
• Content types: per signpost content type, in registration order, a switch labelled with the type's label, the file name ([a-z0-9-], unique, images and news reserved) and the HTML sitemap heading per locale. Stored in golem15_sitemap_types.
• Additional addresses: a repeater of paths (or absolute URLs) with an optional lastmod as YYYY-MM-DD. They go into the pages group.

A save refuses an invalid site address, a file size outside 1 to 50 000, an address that is not a path or an http/https URL, a lastmod that is not a date and an invalid or taken file name, and dispatches a rebuild.

API reference

Identifier	Description
sitemap.Plugin	Compiled plugin; ID is golem15.sitemap, Requires is empty.
sitemap.Plugin.Register	Publishes classes.Listing as contract.Listing.
sitemap.Plugin.Boot	Subscribes to pact.RecordSaved and pact.RecordDeleted.
sitemap.Plugin.Routes	GET /sitemap.xml and GET /_sitemap/files/{file}.
sitemap.Plugin.Buckets, sitemap.BucketSitemap	The sitemap-xml throttle bucket: 60 requests a minute per client IP.
sitemap.Plugin.Jobs	The rebuild job (classes.RebuildKind), three attempts.
sitemap.Plugin.Schedule	sitemap:rebuild daily at 03:00.
sitemap.Plugin.Commands	sitemap:rebuild, sitemap:import, sitemap:import-winter.
sitemap.Plugin.ConfigFS	The default config/config.yaml.
sitemap.Plugin.AdminControllers, sitemap.Plugin.Settings, sitemap.Plugin.Permissions	The settings singleton, its tile and golem15.sitemap.access_definitions.
sitemap.Plugin.AdminActionExtensions	The Content types tab on the settings singleton.
sitemap.TabFiles, sitemap.TabTypes	The tab keys of the settings form.
sitemap.Plugin.LangFS, sitemap.Plugin.AdminFS	Embedded phrasebooks, admin YAML and partials.
sitemap.Plugin.Migrations, sitemap.Plugin.Models	The migrations and models.
contract.Filter, contract.Candidate, contract.Decision, contract.Group	The filter other plugins implement.
contract.ReasonNoindex, contract.ReasonCanonical, contract.ReasonRedirect, contract.ReasonType	The left-out reasons.
contract.Listing, contract.ListingSettings, contract.GroupInfo, contract.EntriesQuery, contract.Entry, contract.EntriesPage	What the sitemap publishes for other plugins.
classes.Rebuild, classes.RebuildResult	One rebuild of the slot table and what it changed.
classes.RebuildArgs, classes.RebuildKind	The rebuild job's arguments and kind golem15.sitemap.rebuild.
classes.RequestRebuild, classes.DispatchRebuild, classes.RequestDebouncedRebuild, classes.ClearPending, classes.Building	Dispatching rebuilds and the debounce window.
classes.RecordSaved, classes.RecordDeleted, classes.RealChanges, classes.Changed	The record event handling.
classes.LastmodIgnore, classes.RebuildDelay, classes.DefaultLastmodIgnore, classes.LastmodIgnoreConfigKey, classes.RebuildDelayConfigKey	The lastmod and debounce config.
classes.AssignSlots, classes.Slot	Stable file numbering.
classes.WriteIndex, classes.WriteURLSet, classes.IndexFile, classes.FileEntry, classes.FileAlternate, classes.NSSitemap, classes.NSXHTML	The XML writers.
classes.FileName, classes.ParseFileName, classes.ValidSlug, classes.DefaultSlug, classes.ReservedSlugs	File and group names.
classes.SiteURL, classes.SiteURLConfigKey, classes.Absolute, classes.LoadSettings	The site address and the settings row.
classes.TypeConfig, classes.EffectiveType, classes.LoadTypeSettings	The per-type settings.
classes.Listing	The contract.Listing implementation.
classes.ExtraRecordID	The record id of an additional address.
models.URL, models.Alternate	A row of the slot table and an hreflang alternate.
models.ExtraTypeKey, models.ExtraGroup, models.ReasonDeleted, models.ReasonUnpublished	Additional addresses and the holes' reasons.
models.Settings, models.DefaultSettings, models.ExtraURL, models.DefaultFileSize, models.MaxFileSize	The settings row; Validate and Problems are its save checks.
models.TypeSetting	A row of golem15_sitemap_types.
models.AllowedFrontendURL, models.AllowedAddress, models.ParseDate, models.ValidDate	The value checks.
models.LegacyItem, models.ParseLegacyItems, models.ParseLegacyItem, models.LegacyEntries, models.LegacyJSON, models.DecodeText, models.Text	Reading Phase 17 and WinterCMS definition data.
controllers.SettingsControllerID, controllers.PermissionAccessDefinitions	golem15.sitemap.settings and golem15.sitemap.access_definitions.
console.Rebuild, console.Import, console.ImportFile, console.ImportWinter, console.ImportWinterFiles, console.WinterOptions, console.WinterReport	The commands.

Configuration

config/config.yaml, read as golem15.sitemap.<key>:

Key	Default	Meaning
site_url	unset	The site address when the setting is empty (before app.url).
lastmod_ignore	[updated_at, created_at, views, sort_order]	Attributes whose change does not move lastmod. A host replaces the whole list.
rebuild_delay_seconds	60	The delay of the rebuild a record event queues; events within it share one rebuild.

CLI commands

Command	Arguments and flags	Effect
sitemap:rebuild	--queue	Rebuilds the files in the process and prints Rebuilt the sitemaps: N addresses in N files (N added, N changed, N removed).; with --queue dispatches the rebuild job instead. Scheduled daily at 03:00.
sitemap:import	FILE, --append, --theme	Reads a JSON array of {url, lastmod} entries into the additional addresses (replacing them; --append adds and skips listed ones). changefreq and priority are ignored, --theme is accepted and does nothing. One bad entry stops the import.
sitemap:import-winter	--definitions FILE, --settings FILE, --theme, --map old=new (repeatable), --page-url reference=/path (repeatable), --force, --dry-run	Copies a WinterCMS export: url items become additional addresses (--page-url turns a static or CMS page item into one, --map renames types first), every other item is dropped and listed; frontend_url becomes the site address and frontend_default_locale the default locale, the PHP-parity keys are reported as ignored. Stored addresses are replaced only with --force; --dry-run writes nothing; an unsafe frontend_url stops the import.

The commands open the database themselves when the application has not published one. ./bin/acme help <command> shows each command's long help and examples. The plugin logs through the component sitemap (log.components.sitemap in config/log.yaml).

The WinterCMS export runs on the WinterCMS database with the read-only scripts in scripts/:

mysql -N -B --raw <database> < scripts/winter-export-definitions.sql > sitemap-definitions.jsonl
mysql -N -B --raw <database> < scripts/winter-export-settings.sql > sitemap-settings.json
./bin/acme sitemap:import-winter --definitions sitemap-definitions.jsonl --settings sitemap-settings.json --dry-run
./bin/acme sitemap:import-winter --definitions sitemap-definitions.jsonl --settings sitemap-settings.json
./bin/acme sitemap:rebuild

Dependencies

• SummerCMS v0.6.1 modules: backpack, bonfire, cabana, compass, conga, diary, lagoon, pact, party, phrasebook, signpost, surf; bouncer and kiosk in tests. Until v0.6.1 is tagged, go.mod requires v0.6.0 and develops against ../summercms.go.
• Third-party: gorm.io/gorm, github.com/go-gormigrate/gormigrate/v2, github.com/goccy/go-yaml. Test-only: github.com/testcontainers/testcontainers-go, github.com/jackc/pgx/v5.

Testing

go test -short ./...   # unit tests, no Docker
go test ./...          # also the Postgres tests

Docker-free: TestAssignSlots and TestRealChanges (classes), TestAllowedFrontendURL and TestSettingsValidate (models), TestWinterSettingsFrontendURL (console), TestSiteURLPrecedence, TestFileNames, TestScheduleAndJobs, TestCommandsDocumented and TestPluginManifest (root).

The Postgres tests boot the plugin with a fake content plugin (two content types over an in-memory store) and, for the Filter, a fake plugin that publishes one: TestSitemapIndex (the index and files after a rebuild job, drafts absent, escaping, 404s), TestStableNumbering, TestLastmod, TestDebouncedRebuild, TestHreflangAlternates, TestFilterExcludes, TestNoFilter, TestListing, TestSettingsSingleton (the admin API, its refusals and the files partial), TestMigrateDefinitions, TestImportWinter, TestImportFile and TestRebuildCommand.