A rebuild takes the transaction-scoped advisory lock
golem15.sitemap.rebuild before it reads the settings and collects, and
applies in the same transaction, so a second rebuild collects only after
the first has committed. apply no longer writes the lastmod of a content
row it did not create, so a lastmod RecordSaved moved meanwhile stays.
RecordDeleted marks the listed rows it excludes and RecordSaved the rows
whose lastmod it moves (indexnow_pending_at); the next rebuild sends
them to IndexNow as removed or changed and clears the marks. RecordSaved
checks for a nil event before reading it. TestIndexNow now deletes and
edits through the record events, as the admin does.
The sitemap publishes classes.Listing as contract.Rebuilder too, so a
plugin whose Filter reads data of its own can queue the debounced rebuild
a record event queues when that data changes.
- XML escaping and namespaces, no priority, the image cap and file names
- News window and cap, sources and languages, the publication name
- IndexNow batches of 10 000, answer outcomes, keys and the host filter
- HTML sitemap group order, tree order and paging
- default slugs, the effective type setting, lastmod config, slot edges
No behaviour change: NotifyIndexNow splits through indexNowBatches and
SendIndexNow maps the answer through indexNowOutcome, so both can be unit
tested without a queue or the network.
- an additional address whose absolute URL a listed record already has is
left out with the new reason duplicate (models.ReasonDuplicate)
- the settings screen counts it under Already listed by a content type
- a record left out by the filter or the type switch does not count
contract.ListingSettings.TypesOff lists the signpost keys of the content
types switched off in the sitemap settings, so another plugin (the SEO
tab's sitemap status) can tell an address left out with ReasonType
without reading the sitemap's tables. Additive.
- GET and PUT /_sitemap/api/v1/settings and POST /_sitemap/api/v1/rebuild
resolve the administrator from a backend token, a personal API token or
the MCP dispatch and require the sitemap permission (401/403 otherwise);
bucket sitemap-api
- PUT changes the fields sent with the settings screen's checks (422
validation_failed, nothing stored), merges the HTML sitemap addresses,
writes content types, generates the IndexNow key and queues a rebuild
- GET answers the settings, the content types, the index files with their
counts, the left-out counts, the IndexNow state and the News source
values
- MCP tools sitemap_get_settings, sitemap_update_settings and
sitemap_rebuild bound to those routes, listed with the permission
- the content type settings write and the file listing move to classes,
shared by the form, S1, the index and the API; S1 and the index now list
the image and News files
- GET /_sitemap/api/v1/html?locale= answers the overview: the listed
addresses of the XML sitemaps by group, in the order pages, offers,
services, categories (as a tree), posts (newest first), tags, authors,
other content types, each with its title in the locale, total, the
first html_page_size links and more
- GET /_sitemap/api/v1/html/{group}?locale=&page= answers one page of a
group; past the last page, an unknown group or the HTML sitemap off
answer 404; every answer is noindex; per-IP bucket sitemap-html
- the HTML sitemap tab: html_enabled, the address per locale (html_paths)
and html_page_size 50 to 500, with errors.html_path and errors.page_size
- contract.Listing.Settings answers HTMLEnabled, HTMLPaths, HTMLPageSize
and KeyFile
- saving with IndexNow on generates a 32-hex key (crypto/rand);
indexnow_new_key generates a new one, so the old key file answers 404
- GET /_sitemap/indexnow/{file} serves <key>.txt as text/plain while
IndexNow is on; the SEO layer maps it to the site root
- after each rebuild in production the added, changed and removed
addresses on the site host are queued as golem15.sitemap.indexnow jobs
(at most 10 000 per request, five attempts)
- the job posts {host, key, urlList} to api.indexnow.org through a
fetchguard client limited to that host and records the answer; 429 and
5xx retry, 400, 403 and 422 do not
- the IndexNow tab shows the S2 status partial: key or no key, last
notification or never, and the 403, 422 and 429 callouts
- sitemap-news.xml lists the articles (content kind article) first
published in the last two days, newest first, at most 1000, with the
publication name (news_publication, else app.name), the language of the
locale, the publication date and the title
- news_sources narrows it to type:<key> and category:<key>:<id> values;
the Images and News tab lists the values to use in a partial
- the index lists the file while news_enabled is on; otherwise it is 404
- migration 202610280004: image_file_no, kind, published_at and parent_refs
on the slot table; the image, News, IndexNow and HTML sitemap settings
- the rebuild stores each page's featured and body images (absolute on the
site, deduplicated, at most 1000) and numbers the rows with images into
sitemap-images-<n>.xml with the per-type files' stable rules
- the index lists the image files while images_enabled is on; the files
route serves them with image:image and image:loc only
- every listed address of a record with more than one listed locale
carries xhtml:link alternates for each locale plus x-default on the
default locale (D-55 to D-57); no alternate for a missing translation
- each page of records and the additional addresses go to the published
contract.Filter in one batch: Exclude leaves the files with its reason,
Group moves the address, LastMod seeds a new row (D-95, D-24)
- classes.Listing implements contract.Listing and is published in Register
- RecordSaved with a change outside golem15.sitemap.lastmod_ignore moves
the lastmod of the record's rows (only the changed locale's for a
translated change); a no-op save, an ignored change or a rebuild does not
- RecordDeleted takes the record's rows out of the files at once
- record events queue one rebuild per golem15.sitemap.rebuild_delay_seconds
(rebuild_pending_until, P33-12); the job clears it when it starts
- sitemap:rebuild [--queue], scheduled nightly at 03:00 (pact.HasSchedule)
BREAKING CHANGE: /sitemap.xsl is gone (404), the definitions screen and
the Phase 17 settings tile are replaced by the settings singleton, and
priority, change frequency, priority tiers and the update manifest are
no longer written or read. The config keys golem15.sitemap.cache_seconds
and golem15.sitemap.priority_tiers are removed; site_url, lastmod_ignore
and rebuild_delay_seconds are new.
- sitemap:import and sitemap:import-winter write the additional addresses
(and site_url, default_locale) instead of the definition
- the golem15_sitemap_definitions table stays in the database, unread
- one settings tile opening the golem15.sitemap.settings singleton
- Files tab (site address, file size, S1 files partial), Additional
addresses repeater, Content types tab built from signpost at boot
- golem15_sitemap_types; a type switched off leaves the files (reason type)
- a save validates (errors.*) and dispatches a rebuild after commit
- contract package: Filter, Listing and their types (D-95)
- golem15_sitemap_urls slot table and the new settings columns
- Rebuild over signpost Records, stable slot assignment, rebuild job
- GET /sitemap.xml index and GET /_sitemap/files/{file} from the table
Every command gets long help and examples, descriptions in imperative
sentence case, and a warn-only TestCommandsDocumented over bonfire.CheckDocs.
The plugin logs through a diary component logger instead of the process
default logger. Requires SummerCMS v0.4.0; command names and flags unchanged.
- models.AllowedFrontendURL: empty, or an absolute http(s) URL with a host
and without whitespace, control characters, a double quote, <, > or &
- classes.Rewrite leaves the document unchanged for a refused value,
inserts through EscapeText/EscapeAttr and keeps a loc or href that would
not stay an http(s) URL
- TestRewriteUnsafeFrontendURL, TestAllowedFrontendURL and a stored-row
check in TestSitemapImportSmoke
A url item stored as javascript://%0aalert(1)%0a//x passed Url::to and
reached <loc>, which sitemap.xsl renders as a link on the site origin.
- The definitions form refuses, per row, a url that is not a path or an
http(s) URL (classes.AllowedItemURL) and a urlPattern that is not a
path (signpost.ValidItemPattern), keyed data.<i>.url / .urlPattern.
- Generator.addItem drops a loc or alternate that is not http(s).
- sitemap:import-winter drops such url items and clears such patterns
with a report line; sitemap:import refuses such an entry.
- generator in Definition::addItemToSet order: 50000 URL cap, lastmod from
the record or the updates manifest (regular file, 5 MiB cap, fail safe),
BeforeAddItem (skip) and AddItem (extra XML) festival events, tier or item
priority, per-alternate expansion of list type children
- priority tiers from golem15.sitemap.priority_tiers, compiled at boot,
defaults equal to the PHP getPriorityForUrl rules
- frontend_url rewrite of loc, xhtml:link href and the stylesheet href;
GET /sitemap.xsl serves the PHP stylesheet byte for byte
- golem15_sitemap_settings singleton and Settings model
- sitemap:import (PHP ImportCommand) and sitemap:import-winter with the
read-only Winter export SQL
- urlto and full goldens generated from PHP; TestSitemapImportSmoke
- golem15.sitemap plugin: golem15_sitemap_definitions migration, Definition
model (Fillable theme and data) with tolerant PHP-shaped item decoding and
PHP toArray item encoding
- Url::to and buildUrl port, libxml-format serializer, generator for url and
registry items
- GET /sitemap.xml serves the definition with theme default (404 without one,
empty body without items)
- tests/php/gen-goldens.php and the tracer golden; TestSitemapGolden and the
Postgres TestSitemapTracer
