diff --git a/README.md b/README.md new file mode 100644 index 0000000..298be31 --- /dev/null +++ b/README.md @@ -0,0 +1,235 @@ +# golem15.sitemap + +The compiled sitemap plugin for SummerCMS applications: it serves `/sitemap.xml` and its stylesheet from one sitemap definition editors manage in the admin, byte for byte the way the WinterCMS Golem15.Sitemap plugin did. + +```go +import _ "git.golem15.com/golem15/sm-sitemap-plugin" +``` + +Module `git.golem15.com/golem15/sm-sitemap-plugin`, package `sitemap`, plugin ID `golem15.sitemap`. + +## Overview + +The plugin is a compiled SummerCMS plugin. Its `init` registers it with the framework, so an application loads it by listing it in `summer.yaml`; `summer build` then emits the blank import above into the application's `plugins.gen.go`. + +It is a port of the WinterCMS plugin for headless sites. An application has one sitemap definition (`models.Definition`, theme key `default`), a list of items: explicit URLs and items of the menu item types other plugins register with the framework's registry (`signpost`). `Requires()` is empty: content plugins such as the Journal register their types and the sitemap reads them through the registry. + +The output keeps the PHP plugin's bytes: element order, escaping, whitespace, date format, headers and the `frontend_url` rewrite are not changed. The goldens in `classes/testdata` are generated from the PHP code and pin them. + +## Features + +- **Routes.** Anonymous `GET /sitemap.xml` and `GET /sitemap.xsl` at the application root, outside any throttle group and without the backend middleware. +- **One definition.** Table `golem15_sitemap_definitions` keeps the PHP columns (`id`, `theme` nullable and indexed, `data` text, timestamps). The `theme` column holds the fixed key `default` (`models.DefinitionTheme`); `/sitemap.xml` serves that row. `models.Definition.Fillable` is exactly `theme` and `data`, the PHP `$fillable` with `$guarded = ['*']`. +- **Items.** An item is a URL (`type: url`) or a registry menu item type. Winter.Pages static pages, all static pages and the CMS page field are not ported. Stored item JSON in the PHP shape decodes tolerantly: numbers, strings and booleans are coerced, and unknown keys (`cmsPage`, `code`, `exists`, `fillable` and any other) are ignored. A save writes the PHP `toArray` key set (`changefreq`, `priority`, `nesting`, `type`, `url`, `reference`, `cmsPage` as null, `urlPattern`). +- **Settings.** A singleton row (`golem15_sitemap_settings`, id 1) holds the frontend URL, the frontend default locale, the updates manifest path and the priority tiers, with the PHP defaults. +- **Priority tiers.** Rules in config (`golem15.sitemap.priority_tiers`) map URL paths to five priority settings; the shipped defaults are the PHP plugin's rules. +- **Hooks.** Typed festival events `classes.BeforeAddItem` and `classes.AddItem` replace the PHP events `golem15.sitemap.beforeAddItem` and `golem15.sitemap.addItem`. +- **Admin.** The definition opens from the Settings page as a singleton form with a repeater of item rows; a second settings entry holds the settings form. Permission `golem15.sitemap.access_definitions`. +- **Commands.** `sitemap:import` (the PHP JSON import) and `sitemap:import-winter` (a copy of a WinterCMS site's definition and settings). +- **Phrasebooks.** `lang/en/lang.yaml` and `lang/pl/lang.yaml` keep the PHP key names. + +## Usage + +Mount the plugin in an application as a git submodule at `plugins/golem15/sitemap`, add it to the application's `go.work` and `go.mod`, and list it in `summer.yaml`: + +```yaml +plugins: + - id: golem15.sitemap + module: git.golem15.com/golem15/sm-sitemap-plugin +``` + +The plugin's own `go.mod` and `go.work` replace the framework with `../summercms.go` for a sibling checkout. + +### /sitemap.xml and /sitemap.xsl + +`GET /sitemap.xml` answers: + +- `404` when there is no definition with the theme `default`; +- `200` with an empty body when the definition has no items; +- otherwise the sitemap, `Content-Type: application/xml`, `Cache-Control: no-cache, private`. + +The document is the PHP `DOMDocument::saveXML()` format: the XML declaration and the `xml-stylesheet` processing instruction each followed by a newline, everything else on one line ending with `` and a newline. Items that all resolve to nothing produce a self-closing ``. Text is escaped like libxml (`&`, `<`, `>` and carriage return; attributes also `"`, tab and newline); single quotes and UTF-8 stay literal. + +Each URL is written in the PHP order (`classes.Generator`): + +1. The 50000th URL is written, the 50001st is not (`classes.MaxURLs`). +2. A URL item's address goes through the port of Winter's `Url::to` (`classes.URLTo`): root plus the path trimmed of slashes, `/` keeps its slash, every path segment re-encoded like `rawurlencode`, valid absolute, `mailto:`, `tel:` and fragment URLs kept. +3. `lastmod` is the record's modification time, else the updates manifest's date for the URL path, else omitted. It is written as `2006-01-02T15:04:05+00:00` in UTC. +4. `classes.BeforeAddItem` may drop the URL. +5. The priority is the first matching tier's setting when tiers are on, else the item's priority, written as stored text (`1.0` stays `1.0`). +6. `loc`, `lastmod`, `changefreq` and `priority` are written, then the alternates as `xhtml:link` elements, only when there are more than one. +7. `classes.AddItem` may append XML before ``. + +A registry item is resolved with its URL pattern, its nesting flag and the settings' frontend default locale as the unprefixed locale. A resolved item with a URL is written, then its sub-items once each with their own URL. A list type (a resolved item without a URL, such as all posts) writes each sub-item once per alternate, with that alternate's URL as `loc` and every alternate as `hreflang`, which is what the PHP list types returned. + +When the settings hold a frontend URL, the backend URL (`app.url` when set, else the request's scheme and `Host`) is replaced literally inside every ``, every `xhtml:link href` and the stylesheet href, as the PHP `str_replace` did. The scheme is `https` for a TLS connection, or when the immediate peer is in `http.trusted_proxies` and its `X-Forwarded-Proto` says `https`. + +`GET /sitemap.xsl` serves the PHP plugin's stylesheet byte for byte, `Content-Type: text/xsl; charset=UTF-8`. + +A frontend served from another host routes both paths to the application, before its catch-all: + +```nginx +location = /sitemap.xml { + proxy_pass http://127.0.0.1:8096; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; +} +location = /sitemap.xsl { + proxy_pass http://127.0.0.1:8096; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto $scheme; +} +``` + +### Item types + +| Type | Item fields used | +|------|------------------| +| `url` | `url`, `changefreq`, `priority` | +| A registry type (for example `journal-post`) | `reference` (when the type takes one), `urlPattern` (overrides the host and default patterns), `nesting`, `changefreq`, `priority` | + +The admin's type dropdown offers `url` and every registry type in registry order with its label. The reference dropdown depends on the row's type and lists the type's references (`signpost.ReferenceOptions`); `url` and types without references take none, and a reference the type does not offer is refused on save. + +### Settings + +| Setting | Default | Meaning | +|---------|---------|---------| +| `frontend_url` | empty | The public frontend URL that replaces the backend URL in the XML. | +| `frontend_default_locale` | empty | The locale whose URLs carry no prefix, passed to registry types; empty leaves it to the type. | +| `updates_manifest_path` | empty | A JSON file mapping a URL path to an ISO 8601 date, used as `lastmod` for items without their own date. | +| `enable_priority_tiers` | on | Tier priorities replace item priorities. | +| `priority_homepage`, `priority_services`, `priority_blog`, `priority_categories`, `priority_legal` | `1.0`, `0.8`, `0.6`, `0.4`, `0.2` | The tier values, stored as text. | + +The updates manifest is read only when it is a regular file of at most 5 MiB (`classes.ManifestMaxBytes`) holding a JSON object; paths match with the leading slash added and the trailing slash removed (`/` stays). Any error omits `lastmod` and is logged without the file's content. The manifest is a frontend build output, for example a script that writes the git commit date of each static page's source: + +```json +{ + "/": "2026-06-10T09:12:44+00:00", + "/about": "2026-05-02T14:31:00+00:00" +} +``` + +### Priority tiers + +With `enable_priority_tiers` on, the URL's path without trailing slashes (`""` for the homepage) is matched against the rules in `golem15.sitemap.priority_tiers` in order; the first match's setting value is the priority, and a URL no rule matches keeps its item's priority. A rule names a `setting` and either `exact` (a list of paths) or `pattern` (a regular expression in RE2 syntax). The rules are compiled at boot: an unknown setting, a rule with neither or both of `exact` and `pattern`, or a pattern that does not compile fails the boot (`classes.CompileTiers`). + +The plugin's `config/config.yaml` ships the rules the PHP plugin had in code, so a site configured the same way gets the same priorities. A host replaces the whole list in its own config, for example `config/golem15.yaml`: + +```yaml +sitemap: + priority_tiers: + - setting: priority_homepage + exact: ["", "/en"] + - setting: priority_blog + pattern: '^/(en/)?articles/[^/]+$' +``` + +### Hooks + +`classes.BeforeAddItem` fires through `festival.Bus.UntilHandled` before a URL is counted; a listener that sets `Skip` drops it (the PHP listener returned `false`). `classes.AddItem` fires through `festival.Bus.Fire` after a URL's elements are built; a listener may append escaped child XML to `Extra`. Both carry the definition item and a `classes.ItemInfo` (URL, modification time, written `lastmod`, alternates). + +```go +func (p *Plugin) Boot(app *backpack.App) error { + app.Events.Listen("acme.seo", func(ctx context.Context, e *classes.BeforeAddItem) error { + if strings.HasSuffix(e.Info.URL, "/draft") { + e.Skip = true + } + return nil + }) + app.Events.Listen("acme.seo", func(ctx context.Context, e *classes.AddItem) error { + e.Extra = append(e.Extra, ""+classes.EscapeText("yes")+"") + return nil + }) + return nil +} +``` + +### Admin + +Two entries on the Settings page, category CMS, both requiring `golem15.sitemap.access_definitions` (developer role): + +- **Sitemap** links to the definitions controller (`golem15.sitemap.definitions`), a singleton form: the server opens the definition with the theme `default`, creating it when absent, and the form has no list, create or delete. Its one field is a repeater on `data` whose rows hold type, reference, URL, URL pattern, priority, change frequency and the nesting checkbox. +- **Sitemap settings** is the settings form. + +The WinterCMS forms are adapted to the admin's YAML: + +- The definition form's tab is flattened, and the `SitemapItems` form widget is replaced by a `type: repeater` field on `data`. +- `span: none` and `cssClass` become `left` and `right`. +- The settings form's nginx hint and Priority Tiers section are field comments and this README; the `trigger` conditions on the tier fields are dropped, so those fields always show and their comments say when they apply. +- The tier comments no longer name one site's paths. +- The CMS page field and its messages are not ported; the editor's empty-definition action (`empty_confirm`) is not ported. +- The form's "View sitemap.xml" subtitle link (`definition.view`) waits for a form schema key in the framework; the phrase exists. + +## API reference + +| Identifier | Description | +|------------|-------------| +| `sitemap.Plugin` | Compiled plugin; `ID` is `golem15.sitemap`. | +| `sitemap.Plugin.Routes` | `GET /sitemap.xml` and `GET /sitemap.xsl`. | +| `sitemap.Plugin.Boot` | Compiles the tier rules; a bad rule fails the boot. | +| `sitemap.Plugin.ConfigFS` | The default `config/config.yaml` (tier rules). | +| `sitemap.Plugin.Commands` | Registers `sitemap:import` and `sitemap:import-winter`. | +| `sitemap.Plugin.AdminControllers` | The definitions controller. | +| `sitemap.Plugin.Settings` | The Sitemap link and the Sitemap settings form. | +| `sitemap.Plugin.Permissions` | `golem15.sitemap.access_definitions`. | +| `sitemap.Plugin.LangFS`, `sitemap.Plugin.AdminFS` | Embedded phrasebooks and admin YAML. | +| `sitemap.Plugin.Migrations`, `sitemap.Plugin.Models` | The two migrations and the two models. | +| `models.Definition` | The definition row; `Items` and `SetItems` read and write its item list. | +| `models.DefinitionTheme` | The fixed theme key `default`. | +| `models.DefinitionItem`, `models.DefinitionItems` | One item and the stored list, with the tolerant PHP-shaped decoding. | +| `models.ItemTypeURL` | The built-in item type `url`. | +| `models.Text`, `models.DecodeText` | A text field's value; the tolerant decoding of one stored text value. | +| `models.Settings`, `models.DefaultSettings` | The settings row and its PHP defaults; `Settings.Priority` reads a tier value. | +| `models.PriorityHomepage`, `models.PriorityServices`, `models.PriorityBlog`, `models.PriorityCategories`, `models.PriorityLegal`, `models.PriorityDefaults` | The tier setting keys and their defaults. | +| `controllers.DefinitionsControllerID`, `controllers.PermissionAccessDefinitions` | `golem15.sitemap.definitions` and `golem15.sitemap.access_definitions`. | +| `classes.Generator` | Writes a definition's sitemap; `Generate`, `Stylesheet`. | +| `classes.Resolver` | What the generator resolves registry items through (`*signpost.Registry`). | +| `classes.Serialize`, `classes.URLEntry`, `classes.Alternate` | The libxml-format writer and its entries. | +| `classes.EscapeText`, `classes.EscapeAttr` | libxml's text and attribute escaping. | +| `classes.URLTo`, `classes.BuildURL`, `classes.URLPath` | Ports of Winter `Url::to`, `Url::buildUrl` and `parse_url(..., PHP_URL_PATH)`. | +| `classes.Rewrite`, `classes.BackendURL` | The `frontend_url` rewrite and the backend URL it replaces. | +| `classes.TiersConfigKey`, `classes.TierConfig`, `classes.TierRule`, `classes.Tiers`, `classes.CompileTiers`, `classes.LoadTiers` | The tier rules. | +| `classes.Manifest`, `classes.LoadManifest`, `classes.ManifestMaxBytes` | The updates manifest. | +| `classes.BeforeAddItem`, `classes.AddItem`, `classes.ItemInfo` | The hooks. | +| `classes.MaxURLs` | 50000. | +| `console.Import`, `console.ImportFile` | `sitemap:import`. | +| `console.ImportWinter`, `console.ImportWinterFiles`, `console.WinterOptions`, `console.WinterReport` | `sitemap:import-winter`. | + +## CLI commands + +| Command | Arguments and flags | Effect | +|---------|---------------------|--------| +| `sitemap:import` | `FILE`, `--theme`, `--append` | The PHP `ImportCommand`: reads a JSON array of `{url, changefreq, priority}` entries (`monthly` and `0.5` by default). Without `--append` the definition keeps its non-URL items first and the imported URLs replace its URL items; with it the URLs follow every existing item. The definition is always `default`; a `--theme` other than `default` is reported. | +| `sitemap:import-winter` | `--definitions FILE`, `--settings FILE`, `--theme`, `--map old=new` (repeatable), `--force`, `--dry-run` | Copies a WinterCMS export into the definition and the settings row. URL items are kept; static and CMS page items are dropped; `--map` renames types first (for example `--map blog-post=journal-post`); an item of an unknown type, or whose reference is not among its type's current references, is dropped. Every dropped or remapped item and every dropped `cmsPage` is printed. A definition that already has items is refused without `--force`; `--dry-run` writes nothing. Settings missing from the export take the defaults. `--theme` picks a definition when the export holds several. | + +Both commands open the database themselves when the application has not published one. + +The export runs on the WinterCMS database with the read-only scripts in `scripts/`: + +```sh +mysql -N -B --raw < scripts/winter-export-definitions.sql > sitemap-definitions.jsonl +mysql -N -B --raw < scripts/winter-export-settings.sql > sitemap-settings.json +./bin/acme sitemap:import-winter --definitions sitemap-definitions.jsonl --settings sitemap-settings.json --dry-run +./bin/acme sitemap:import-winter --definitions sitemap-definitions.jsonl --settings sitemap-settings.json +``` + +## Dependencies + +- SummerCMS modules: backpack, bonfire, compass, festival, lagoon, pact, party, signpost, surf; bouncer and cabana in tests. +- Third-party: `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`. Test-only: `github.com/testcontainers/testcontainers-go`, `github.com/jackc/pgx/v5`. + +## Testing + +```sh +go test -short ./... # goldens and config checks, no Docker +go test ./... # also the Postgres smoke tests +``` + +`TestSitemapGolden` (`classes`) feeds `testdata/tracer.json`, `urlto.json` and `full.json` through the Go port and compares bytes with the goldens; it also checks the 50000 URL cap. `TestSitemapTiersConfig` checks the tier rule refusals, a host's replacement of the rule list and the manifest's fail-safe reads. The Postgres smokes (`TestSitemapTracer`, `TestSitemapImportSmoke`, `TestSitemapAdminSmoke`) boot the plugin with a test content plugin and drive the routes, the import commands and the admin API. + +The goldens are generated from the PHP plugin's generation code and the vendored Winter URL generator of any WinterCMS application, and committed, so CI needs no PHP: + +```sh +WINTER_VENDOR=/path/to/winter-app/vendor php tests/php/gen-goldens.php classes/testdata/full.json classes/testdata/full.golden +``` diff --git a/admin.go b/admin.go new file mode 100644 index 0000000..c299864 --- /dev/null +++ b/admin.go @@ -0,0 +1,72 @@ +package sitemap + +import ( + "embed" + "io/fs" + + "git.golem15.com/golem15/sm-sitemap-plugin/controllers" + "git.golem15.com/golem15/sm-sitemap-plugin/models" + "git.golem15.com/golem15/summercms/modules/backpack" + "git.golem15.com/golem15/summercms/modules/pact" +) + +// adminFS holds the admin YAML file by file: the controllers directory also +// holds Go sources. +// +//go:embed controllers/definitions/config_form.yaml models/definition/fields.yaml models/settings/fields.yaml +var adminFS embed.FS + +//go:embed lang +var langFS embed.FS + +// AdminFS serves the admin YAML. +func (p *Plugin) AdminFS() fs.FS { return adminFS } + +// LangFS is the embedded English and Polish phrasebooks. +func (p *Plugin) LangFS() fs.FS { return langFS } + +// AdminControllers hands the controllers a lazy accessor of the booted app, +// so they read the database and the menu item type registry at request +// time. +func (p *Plugin) AdminControllers() []pact.AdminController { + return controllers.AdminControllers(func() *backpack.App { + if p == nil { + return nil + } + return p.app + }) +} + +// Settings registers the two PHP settings entries in the CMS category: the +// sitemap definition (a link to the singleton definitions controller) and +// the sitemap settings form. Both require golem15.sitemap.access_definitions. +func (p *Plugin) Settings() []pact.SettingsItem { + return []pact.SettingsItem{ + { + Code: "sitemap", + Label: "golem15.sitemap::lang.plugin.name", + Description: "golem15.sitemap::lang.plugin.description", + Category: "golem15.sitemap::lang.settings.category", + Icon: "network", + Permissions: []string{controllers.PermissionAccessDefinitions}, + Controller: controllers.DefinitionsControllerID, + }, + { + Code: "sitemap_settings", + Label: "golem15.sitemap::lang.settings.label", + Description: "golem15.sitemap::lang.settings.description", + Category: "golem15.sitemap::lang.settings.category", + Icon: "settings", + Permissions: []string{controllers.PermissionAccessDefinitions}, + Form: "models/settings/fields.yaml", + NewModel: func() any { return &models.Settings{} }, + }, + } +} + +var ( + _ pact.AdminAssets = (*Plugin)(nil) + _ pact.HasAdminControllers = (*Plugin)(nil) + _ pact.HasSettings = (*Plugin)(nil) + _ pact.HasLang = (*Plugin)(nil) +) diff --git a/admin_permissions.go b/admin_permissions.go new file mode 100644 index 0000000..7fd18a0 --- /dev/null +++ b/admin_permissions.go @@ -0,0 +1,19 @@ +package sitemap + +import ( + "git.golem15.com/golem15/sm-sitemap-plugin/controllers" + "git.golem15.com/golem15/summercms/modules/pact" +) + +// Permissions is the plugin's Winter registerPermissions() catalog: the PHP +// code golem15.sitemap.access_definitions on the developer role. +func (*Plugin) Permissions() []pact.Permission { + return []pact.Permission{{ + Code: controllers.PermissionAccessDefinitions, + Tab: "golem15.sitemap::lang.plugin.name", + Label: "golem15.sitemap::lang.plugin.permissions.access_definitions", + Roles: []string{"developer"}, + }} +} + +var _ pact.HasPermissions = (*Plugin)(nil) diff --git a/controllers/admin_registry.go b/controllers/admin_registry.go new file mode 100644 index 0000000..5e8130d --- /dev/null +++ b/controllers/admin_registry.go @@ -0,0 +1,15 @@ +package controllers + +import ( + "git.golem15.com/golem15/summercms/modules/backpack" + "git.golem15.com/golem15/summercms/modules/pact" +) + +// AdminControllers is the plugin's admin controller list. app returns the +// booted application; it is asked on every call, so a controller never keeps +// a database handle or registry of its own. +func AdminControllers(app func() *backpack.App) []pact.AdminController { + return []pact.AdminController{ + definitionsAdminController{app: app}, + } +} diff --git a/controllers/definitions.go b/controllers/definitions.go new file mode 100644 index 0000000..a6963c3 --- /dev/null +++ b/controllers/definitions.go @@ -0,0 +1,106 @@ +// Package controllers holds the sitemap plugin's admin controllers. +package controllers + +import ( + "context" + "errors" + + "git.golem15.com/golem15/sm-sitemap-plugin/models" + "git.golem15.com/golem15/summercms/modules/backpack" + "git.golem15.com/golem15/summercms/modules/pact" + "git.golem15.com/golem15/summercms/modules/signpost" + "gorm.io/gorm" +) + +const ( + // DefinitionsControllerID is the cabana controller id of the sitemap + // definition form. + DefinitionsControllerID = "golem15.sitemap.definitions" + // PermissionAccessDefinitions is Definitions::$requiredPermissions. + PermissionAccessDefinitions = "golem15.sitemap.access_definitions" +) + +// typeURLLabel is the label of the built-in url item type. +const typeURLLabel = "golem15.sitemap::lang.item.url" + +// definitionsAdminController edits the one sitemap definition of the +// application (theme default): a singleton form whose items are a repeater +// with a type dropdown (url plus the registry types) and a reference +// dropdown that depends on it. +type definitionsAdminController struct { + app func() *backpack.App +} + +func (definitionsAdminController) ID() string { return DefinitionsControllerID } + +func (definitionsAdminController) ModelName() string { return `Golem15\Sitemap\Models\Definition` } + +func (definitionsAdminController) ConfigDir() string { return "controllers/definitions" } + +func (definitionsAdminController) RequiredPermissions() []string { + return []string{PermissionAccessDefinitions} +} + +func (definitionsAdminController) NewRecord() any { return &models.Definition{} } + +// The singleton record is the definition with the fixed theme key, created +// with no items when there is none (PHP index(): firstOrCreate, then the +// update form). +func (definitionsAdminController) SingletonRecord(_ context.Context, db *gorm.DB) (any, error) { + if db == nil { + return nil, errors.New("golem15.sitemap: database is not configured") + } + var defs []models.Definition + if err := db.Where("theme = ?", models.DefinitionTheme).Order("id ASC").Limit(1).Find(&defs).Error; err != nil { + return nil, err + } + if len(defs) > 0 { + return &defs[0], nil + } + theme := models.DefinitionTheme + def := &models.Definition{Theme: &theme} + def.SetItems(nil) + if err := db.Create(def).Error; err != nil { + return nil, err + } + return def, nil +} + +func (c definitionsAdminController) registry() *signpost.Registry { + if c.app == nil { + return nil + } + app := c.app() + if app == nil { + return nil + } + reg, _ := app.Lookup[*signpost.Registry]() + return reg +} + +// DropdownOptions serves the item type dropdown: url, then every registry +// type in registry order with its label (PHP getTypeOptions). +func (c definitionsAdminController) DropdownOptions(string) []pact.Option { + out := []pact.Option{{Value: models.ItemTypeURL, Label: typeURLLabel}} + for _, t := range c.registry().Types() { + out = append(out, pact.Option{Value: t.Key, Label: t.Label}) + } + return out +} + +// DependentOptions serves the reference dropdown of an item row: the +// references of the row's type through signpost.ReferenceOptions; url and a +// type without references answer Applies false. +func (c definitionsAdminController) DependentOptions(ctx context.Context, _ string, values map[string]any) (pact.DependentOptionSet, error) { + typ, _ := values["type"].(string) + return signpost.ReferenceOptions(ctx, c.registry(), typ) +} + +var ( + _ pact.AdminController = definitionsAdminController{} + _ pact.AdminPermissioned = definitionsAdminController{} + _ pact.AdminRecordSource = definitionsAdminController{} + _ pact.AdminSingleton = definitionsAdminController{} + _ pact.DropdownOptionsProvider = definitionsAdminController{} + _ pact.DependentOptionsProvider = definitionsAdminController{} +) diff --git a/controllers/definitions/config_form.yaml b/controllers/definitions/config_form.yaml new file mode 100644 index 0000000..89f044c --- /dev/null +++ b/controllers/definitions/config_form.yaml @@ -0,0 +1,24 @@ +# =================================== +# Form Behavior Config +# =================================== + +# Record name +name: golem15.sitemap::lang.definition.title + +# Model Form Field configuration +form: ~/plugins/golem15/sitemap/models/definition/fields.yaml + +# Model Class name +modelClass: Golem15\Sitemap\Models\Definition + +# Default redirect location +defaultRedirect: system/settings + +# Update page +update: + redirect: golem15/sitemap/definitions + redirectClose: system/settings + +messages: + update: golem15.sitemap::lang.definition.title + saved: backend::lang.messages.form.saved diff --git a/lang/en/lang.yaml b/lang/en/lang.yaml new file mode 100644 index 0000000..cafdcf2 --- /dev/null +++ b/lang/en/lang.yaml @@ -0,0 +1,65 @@ +plugin: + name: Sitemap + description: Generate a sitemap.xml file for your website. + permissions: + access_settings: Access sitemap configuration settings + access_definitions: Access sitemap definitions page +item: + location: 'Location:' + priority: Priority + changefreq: Change frequency + always: always + hourly: hourly + daily: daily + weekly: weekly + monthly: monthly + yearly: yearly + never: never + editor_title: Edit Sitemap Item + type: Type + allow_nested_items: Allow nested items + allow_nested_items_comment: Nested items could be generated dynamically by static page and some other item types + url: URL + reference: Reference + title_required: The title is required + unknown_type: Unknown item type + unnamed: Unnamed item + add_item: Add item + new_item: New item + reference_required: The item reference is required. + url_required: The URL is required + url_pattern: URL Pattern + url_pattern_comment: Use :placeholder syntax, e.g. /blog/:slug + url_pattern_required: Please enter a URL pattern + page: Page + check: Check + definition: Definition + save_definition: Saving Definition... + load_indicator: Emptying Definition... + empty_confirm: Empty this definition? +definition: + not_found: No sitemap definition was found. Try creating one first. + title: Sitemap definition + view: View sitemap.xml +settings: + label: Sitemap settings + description: Frontend URL, locale and priority options. + category: CMS + frontend_url: Frontend URL + frontend_url_comment: The public URL of the frontend app (e.g. https://www.example.com). When set, sitemap URLs use this domain instead of the backend URL. + frontend_default_locale: Frontend Default Locale + frontend_default_locale_comment: The unprefixed locale on the frontend (e.g. pl for a prefix_except_default strategy). Other locales get a /{code}/ prefix in sitemap URLs. + updates_manifest_path: Updates Manifest Path + updates_manifest_path_comment: Absolute path to a JSON file mapping URL path => ISO 8601 last-modified date, produced by the frontend build. Used as the for URL items that have no date of their own (e.g. static pages). When empty or unreadable, those items omit . Dynamic sets keep their own dates. + enable_priority_tiers: Enable Priority Tiers + enable_priority_tiers_comment: When enabled, priorities are auto-assigned based on the URL path rules in the config, overriding per-item priorities. When disabled, per-item priorities are used. + priority_homepage: Homepage Priority + priority_homepage_comment: Used for URLs that match the homepage tier rules in the config. Default 1.0. Applies only when priority tiers are enabled. + priority_services: Services & Work Priority + priority_services_comment: Used for URLs that match the services tier rules in the config. Default 0.8. Applies only when priority tiers are enabled. + priority_blog: Blog Posts Priority + priority_blog_comment: Used for URLs that match the blog tier rules in the config. Default 0.6. Applies only when priority tiers are enabled. + priority_categories: Categories & Hub Pages Priority + priority_categories_comment: Used for URLs that match the categories tier rules in the config. Default 0.4. Applies only when priority tiers are enabled. + priority_legal: Legal Pages Priority + priority_legal_comment: Used for URLs that match the legal tier rules in the config. Default 0.2. Applies only when priority tiers are enabled. diff --git a/lang/pl/lang.yaml b/lang/pl/lang.yaml new file mode 100644 index 0000000..8e8ad8f --- /dev/null +++ b/lang/pl/lang.yaml @@ -0,0 +1,65 @@ +plugin: + name: Mapa strony + description: Generuje plik sitemap.xml dla Twojej strony. + permissions: + access_settings: Zarządzaj ustawieniami wtyczki + access_definitions: Zarządzaj definicją mapy strony +item: + location: 'Lokacja:' + priority: Priorytet + changefreq: Częstotliwość zmian + always: cały czas + hourly: co godzinę + daily: codziennie + weekly: co tydzień + monthly: co miesiąc + yearly: co rok + never: nigdy + editor_title: Edytuj pozycję mapy + type: Typ + allow_nested_items: Pozwól na zagnieżdżanie pozycji + allow_nested_items_comment: Zagnieżdżone pozycje mogą zostać wygenerowane dynamicznie przez statyczne strony i niektóre rodzaje typów pozycji. + url: URL + reference: Odnośnik + title_required: Tytuł jest wymagany + unknown_type: Nieznany typ pozycji + unnamed: Pozycja bez nazwy + add_item: Dodaj pozycję + new_item: Nowa pozycja + reference_required: Odnośnik dla pozycji jest wymagany. + url_required: Adres URL jest wymagany + url_pattern: Wzorzec URL + url_pattern_comment: Użyj składni :placeholder, np. /blog/:slug + url_pattern_required: Podaj wzorzec URL + page: Strona + check: Sprawdź + definition: Definicja + save_definition: Zapisywanie definicji... + load_indicator: Czyszczenie definicji... + empty_confirm: Wyczyścić tę definicję? +definition: + not_found: Nie znaleziono definicji mapy strony. Musisz ją utworzyć. + title: Definicja mapy strony + view: Zobacz sitemap.xml +settings: + label: Ustawienia mapy strony + description: Adres frontendu, język i priorytety. + category: CMS + frontend_url: Adres URL frontendu + frontend_url_comment: Publiczny adres aplikacji frontendowej (np. https://www.example.com). Gdy jest ustawiony, adresy w mapie strony używają tej domeny zamiast adresu backendu. + frontend_default_locale: Domyślny język frontendu + frontend_default_locale_comment: Język bez prefiksu na frontendzie (np. pl przy strategii prefix_except_default). Pozostałe języki dostają w mapie strony prefiks /{kod}/. + updates_manifest_path: Ścieżka manifestu aktualizacji + updates_manifest_path_comment: Bezwzględna ścieżka do pliku JSON, który mapuje ścieżkę URL na datę ostatniej zmiany ISO 8601, tworzonego przez build frontendu. Służy jako dla pozycji URL bez własnej daty (np. stron statycznych). Gdy jest pusta lub plik jest nieczytelny, te pozycje nie mają . Zestawy dynamiczne zachowują własne daty. + enable_priority_tiers: Włącz poziomy priorytetów + enable_priority_tiers_comment: Po włączeniu priorytety są nadawane według reguł ścieżek URL z konfiguracji i zastępują priorytety pozycji. Po wyłączeniu używane są priorytety pozycji. + priority_homepage: Priorytet strony głównej + priority_homepage_comment: Dla adresów pasujących do reguł poziomu strony głównej z konfiguracji. Domyślnie 1.0. Działa tylko przy włączonych poziomach priorytetów. + priority_services: Priorytet usług i realizacji + priority_services_comment: Dla adresów pasujących do reguł poziomu usług z konfiguracji. Domyślnie 0.8. Działa tylko przy włączonych poziomach priorytetów. + priority_blog: Priorytet wpisów bloga + priority_blog_comment: Dla adresów pasujących do reguł poziomu bloga z konfiguracji. Domyślnie 0.6. Działa tylko przy włączonych poziomach priorytetów. + priority_categories: Priorytet kategorii i stron zbiorczych + priority_categories_comment: Dla adresów pasujących do reguł poziomu kategorii z konfiguracji. Domyślnie 0.4. Działa tylko przy włączonych poziomach priorytetów. + priority_legal: Priorytet stron prawnych + priority_legal_comment: Dla adresów pasujących do reguł poziomu stron prawnych z konfiguracji. Domyślnie 0.2. Działa tylko przy włączonych poziomach priorytetów. diff --git a/models/definition.go b/models/definition.go index 5a3dc7a..d0a1cf0 100644 --- a/models/definition.go +++ b/models/definition.go @@ -31,6 +31,10 @@ func (Definition) MorphName() string { return `Golem15\Sitemap\Models\Definition // (MassAssignmentTest). func (Definition) Fillable() []string { return []string{"theme", "data"} } +// Rules is empty: the PHP model validated nothing. The admin form checks +// each item row's reference against its type. +func (Definition) Rules() map[string]string { return map[string]string{} } + // Items returns the stored items. func (d Definition) Items() []DefinitionItem { return d.Data.Get() } diff --git a/models/definition/fields.yaml b/models/definition/fields.yaml new file mode 100644 index 0000000..e90e6e9 --- /dev/null +++ b/models/definition/fields.yaml @@ -0,0 +1,69 @@ +# =================================== +# Form Field Definitions +# =================================== +# +# The WinterCMS form was one tab holding the SitemapItems widget. Here the +# items are a repeater on data; each row is one DefinitionItem. The CMS page +# field is not ported: items are URLs or registry types. + +fields: + data: + label: golem15.sitemap::lang.item.definition + type: repeater + prompt: golem15.sitemap::lang.item.add_item + titleFrom: url + form: + fields: + type: + label: golem15.sitemap::lang.item.type + type: dropdown + options: getTypeOptions + span: left + reference: + label: golem15.sitemap::lang.item.reference + type: dropdown + dependsOn: type + emptyOption: backend::lang.form.none + span: right + url: + label: golem15.sitemap::lang.item.url + type: text + span: left + urlPattern: + label: golem15.sitemap::lang.item.url_pattern + comment: golem15.sitemap::lang.item.url_pattern_comment + type: text + span: right + priority: + label: golem15.sitemap::lang.item.priority + type: dropdown + span: left + options: + '0.1': '0.1' + '0.2': '0.2' + '0.3': '0.3' + '0.4': '0.4' + '0.5': '0.5' + '0.6': '0.6' + '0.7': '0.7' + '0.8': '0.8' + '0.9': '0.9' + '1.0': '1.0' + changefreq: + label: golem15.sitemap::lang.item.changefreq + type: dropdown + span: right + options: + always: golem15.sitemap::lang.item.always + hourly: golem15.sitemap::lang.item.hourly + daily: golem15.sitemap::lang.item.daily + weekly: golem15.sitemap::lang.item.weekly + monthly: golem15.sitemap::lang.item.monthly + yearly: golem15.sitemap::lang.item.yearly + never: golem15.sitemap::lang.item.never + nesting: + label: golem15.sitemap::lang.item.allow_nested_items + comment: golem15.sitemap::lang.item.allow_nested_items_comment + type: checkbox + default: true + span: full diff --git a/models/settings/fields.yaml b/models/settings/fields.yaml new file mode 100644 index 0000000..2670aeb --- /dev/null +++ b/models/settings/fields.yaml @@ -0,0 +1,68 @@ +# =================================== +# Form Field Definitions +# =================================== +# +# The WinterCMS form's nginx hint and Priority Tiers section are field +# comments and README text here, and the tier fields always show (the +# trigger on enable_priority_tiers is not ported). + +fields: + frontend_url: + label: golem15.sitemap::lang.settings.frontend_url + comment: golem15.sitemap::lang.settings.frontend_url_comment + type: text + span: left + + frontend_default_locale: + label: golem15.sitemap::lang.settings.frontend_default_locale + comment: golem15.sitemap::lang.settings.frontend_default_locale_comment + type: text + span: right + + updates_manifest_path: + label: golem15.sitemap::lang.settings.updates_manifest_path + comment: golem15.sitemap::lang.settings.updates_manifest_path_comment + type: text + span: full + + enable_priority_tiers: + label: golem15.sitemap::lang.settings.enable_priority_tiers + comment: golem15.sitemap::lang.settings.enable_priority_tiers_comment + type: checkbox + default: true + span: full + + priority_homepage: + label: golem15.sitemap::lang.settings.priority_homepage + comment: golem15.sitemap::lang.settings.priority_homepage_comment + type: text + default: '1.0' + span: auto + + priority_services: + label: golem15.sitemap::lang.settings.priority_services + comment: golem15.sitemap::lang.settings.priority_services_comment + type: text + default: '0.8' + span: auto + + priority_blog: + label: golem15.sitemap::lang.settings.priority_blog + comment: golem15.sitemap::lang.settings.priority_blog_comment + type: text + default: '0.6' + span: auto + + priority_categories: + label: golem15.sitemap::lang.settings.priority_categories + comment: golem15.sitemap::lang.settings.priority_categories_comment + type: text + default: '0.4' + span: auto + + priority_legal: + label: golem15.sitemap::lang.settings.priority_legal + comment: golem15.sitemap::lang.settings.priority_legal_comment + type: text + default: '0.2' + span: auto diff --git a/sitemap_smoke_test.go b/sitemap_smoke_test.go index 79884d5..a1ae643 100644 --- a/sitemap_smoke_test.go +++ b/sitemap_smoke_test.go @@ -4,11 +4,14 @@ import ( "bytes" "context" "encoding/json" + "fmt" + "io" "net/http" "net/http/httptest" "os" "path/filepath" "strings" + "sync/atomic" "testing" "time" @@ -18,6 +21,8 @@ import ( "git.golem15.com/golem15/sm-sitemap-plugin/models" "git.golem15.com/golem15/summercms/modules/backpack" "git.golem15.com/golem15/summercms/modules/bonfire" + "git.golem15.com/golem15/summercms/modules/bouncer" + "git.golem15.com/golem15/summercms/modules/cabana" "git.golem15.com/golem15/summercms/modules/compass" "git.golem15.com/golem15/summercms/modules/lagoon" "git.golem15.com/golem15/summercms/modules/pact" @@ -359,3 +364,160 @@ func TestSitemapImportSmoke(t *testing.T) { t.Fatalf("missing url: %v", err) } } + +var adminSeq atomic.Uint64 + +// call sends an admin API request with an optional bearer token and JSON +// body. +func (e *sitemapEnv) call(t *testing.T, method, path, token string, body any) *httptest.ResponseRecorder { + t.Helper() + var reader io.Reader + if body != nil { + raw, err := json.Marshal(body) + if err != nil { + t.Fatal(err) + } + reader = bytes.NewReader(raw) + } + req := httptest.NewRequest(method, cabana.DefaultAdminPrefix+"/api/v1"+path, reader) + req.Host = "api.example.test" + req.RemoteAddr = fmt.Sprintf("198.51.100.%d:4000", adminSeq.Add(1)%250+1) + req.Header.Set("Accept-Language", "en") + if body != nil { + req.Header.Set("Content-Type", "application/json") + } + if token != "" { + req.Header.Set("Authorization", "Bearer "+token) + } + rec := httptest.NewRecorder() + e.h.ServeHTTP(rec, req) + return rec +} + +// admin signs in a new administrator whose role grants permissions. +func (e *sitemapEnv) admin(t *testing.T, permissions ...string) string { + t.Helper() + stamp := fmt.Sprintf("%d%d", time.Now().UnixNano()%1_000_000_000, adminSeq.Add(1)) + login, password := "admin-"+stamp, "admin-secret-"+stamp + hash, err := bouncer.HashPassword(10, password) + if err != nil { + t.Fatal(err) + } + grants := map[string]int{} + for _, code := range permissions { + grants[code] = 1 + } + raw, _ := json.Marshal(grants) + var roleID uint + if err := e.gdb.Raw(`INSERT INTO backend_user_roles (name, code, permissions, is_system, created_at, updated_at) VALUES (?, ?, ?, FALSE, NOW(), NOW()) RETURNING id`, + "Role "+stamp, "role-"+stamp, string(raw)).Scan(&roleID).Error; err != nil || roleID == 0 { + t.Fatalf("backend role: id=%d err=%v", roleID, err) + } + if err := e.gdb.Exec(`INSERT INTO backend_users (login, email, password, is_activated, is_superuser, role_id, created_at, updated_at) VALUES (?, ?, ?, TRUE, FALSE, ?, NOW(), NOW())`, + login, login+"@example.test", hash, roleID).Error; err != nil { + t.Fatal(err) + } + rec := e.call(t, http.MethodPost, "/auth/login", "", map[string]string{"login": login, "password": password}) + var body struct { + Data struct { + AccessToken string `json:"access_token"` + } `json:"data"` + } + if rec.Code != http.StatusOK || json.Unmarshal(rec.Body.Bytes(), &body) != nil || body.Data.AccessToken == "" { + t.Fatalf("admin login: %d %s", rec.Code, rec.Body.String()) + } + return body.Data.AccessToken +} + +// TestSitemapAdminSmoke drives the definition singleton, the item repeater +// with its dependent reference, the settings form and the permission. +func TestSitemapAdminSmoke(t *testing.T) { + e := bootSitemap(t) + const perm = "golem15.sitemap.access_definitions" + token := e.admin(t, perm) + const base = "/golem15/sitemap/definitions" + + // The singleton route creates the definition once. + var ids []float64 + for range 2 { + rec := e.call(t, http.MethodGet, base+"/singleton", token, nil) + var body struct { + Data struct { + ID float64 `json:"id"` + } `json:"data"` + } + if rec.Code != http.StatusOK || json.Unmarshal(rec.Body.Bytes(), &body) != nil || body.Data.ID == 0 { + t.Fatalf("GET singleton: %d %s", rec.Code, rec.Body.String()) + } + ids = append(ids, body.Data.ID) + } + var n int64 + e.gdb.Model(&models.Definition{}).Count(&n) + if ids[0] != ids[1] || n != 1 { + t.Fatalf("singleton ids %v, %d rows", ids, n) + } + id := fmt.Sprintf("%d", int(ids[0])) + + // The form schema compiles: a singleton with the type options and the + // dependent reference. + rec := e.call(t, http.MethodGet, base+"/schema/form", token, nil) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"singleton":true`) || !strings.Contains(rec.Body.String(), `"acme-post"`) || !strings.Contains(rec.Body.String(), `"dependsOn":"type"`) { + t.Fatalf("form schema: %d %s", rec.Code, rec.Body.String()) + } + rec = e.call(t, http.MethodGet, base+"/fields/data.reference/dependent-options?values%5Btype%5D=acme-post", token, nil) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"value":"7"`) { + t.Fatalf("dependent options: %d %s", rec.Code, rec.Body.String()) + } + + // Saving rows stores PHP-keyed item JSON. + rows := []map[string]any{ + {"type": "url", "url": "/", "reference": "", "urlPattern": "", "priority": "1.0", "changefreq": "always", "nesting": false}, + {"type": "acme-post", "url": "", "reference": "7", "urlPattern": "/n/:slug", "priority": "0.5", "changefreq": "weekly", "nesting": true}, + } + rec = e.call(t, http.MethodPut, base+"/"+id, token, map[string]any{"data": rows}) + if rec.Code != http.StatusOK { + t.Fatalf("PUT rows: %d %s", rec.Code, rec.Body.String()) + } + var raw string + if err := e.gdb.Raw(`SELECT data FROM golem15_sitemap_definitions WHERE id = ?`, id).Scan(&raw).Error; err != nil { + t.Fatal(err) + } + const wantData = `[{"changefreq":"always","priority":"1.0","nesting":false,"type":"url","url":"/","reference":null,"cmsPage":null,"urlPattern":""},{"changefreq":"weekly","priority":"0.5","nesting":true,"type":"acme-post","url":"","reference":"7","cmsPage":null,"urlPattern":"/n/:slug"}]` + if raw != wantData { + t.Fatalf("stored data = %s", raw) + } + + // A reference not offered for its row type is a 422 on the row. + rows[1]["reference"] = "99" + rec = e.call(t, http.MethodPut, base+"/"+id, token, map[string]any{"data": rows}) + if rec.Code != http.StatusUnprocessableEntity || !strings.Contains(rec.Body.String(), "data.1.reference") { + t.Fatalf("PUT unknown reference: %d %s", rec.Code, rec.Body.String()) + } + + // The settings form saves the frontend URL. + rec = e.call(t, http.MethodPut, "/settings/sitemap_settings", token, map[string]any{"frontend_url": "https://www.example.test", "enable_priority_tiers": true}) + if rec.Code != http.StatusOK { + t.Fatalf("PUT settings: %d %s", rec.Code, rec.Body.String()) + } + var st models.Settings + if err := e.gdb.First(&st, 1).Error; err != nil || models.Text(st.FrontendURL) != "https://www.example.test" { + t.Fatalf("settings = %+v %v", st, err) + } + rec = e.call(t, http.MethodGet, "/settings", token, nil) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"controller":"golem15.sitemap.definitions"`) { + t.Fatalf("settings index: %d %s", rec.Code, rec.Body.String()) + } + + // Without the permission every route answers 403. + other := e.admin(t) + for _, c := range []struct{ method, path string }{ + {http.MethodGet, base + "/singleton"}, + {http.MethodPut, base + "/" + id}, + {http.MethodGet, "/settings/sitemap_settings"}, + {http.MethodPut, "/settings/sitemap_settings"}, + } { + if rec := e.call(t, c.method, c.path, other, map[string]any{}); rec.Code != http.StatusForbidden { + t.Errorf("%s %s without permission: %d %s", c.method, c.path, rec.Code, rec.Body.String()) + } + } +}