diff --git a/.planning/phases/32-seo-foundation/32-01-PLAN.md b/.planning/phases/32-seo-foundation/32-01-PLAN.md index 30debd20..346afa38 100644 --- a/.planning/phases/32-seo-foundation/32-01-PLAN.md +++ b/.planning/phases/32-seo-foundation/32-01-PLAN.md @@ -43,7 +43,7 @@ must_haves: - "Extension field values never reach lagoon.Fill, BindWritableFields or the owner's columns, translation rows or attachments (D-91)" - "An admin without the extension's Permissions, or a request where Visible answers false, gets no extension field in schema or show, and a save ignores posted extension keys and does not call Save (D-91, UI-SPEC C1 Permission)" - "A form that no extension targets compiles byte-identically to before this plan: its CanonicalJSON is unchanged and owner tab interleaving is not canonicalised (strictly additive)" - - "The framework change is strictly additive: new optional struct fields and new types only; existing pact, cabana and signpost exported signatures and behaviour are unchanged and existing plugins compile and behave the same without edits" + - "The framework change is strictly additive: new optional struct fields and new types only; existing pact, cabana and signpost exported signatures and behaviour are unchanged, no method is added to an existing interface, and existing plugins compile and behave the same without edits (D-106)" - "The SPA renders the compiled order as sent and adds no ordering code: admin/src is unchanged by this plan" - "Field span (left, right, full) keeps working next to inserted fields (D-91)" - "One plugin extending two controllers and two plugins extending one controller both compile deterministically (assumption-delta invariant)" @@ -103,7 +103,7 @@ Implement D-91 cross-plugin admin form extensions in the framework: `pact.AdminA Purpose: SC-1 ("A, B plus C after A renders A, C, B"; a tab between two tabs; clear boot errors). Every later plan of this phase (D-98 order) builds the SEO tab and the per-type settings tabs on this seam. -Output: the pact types, `modules/cabana/form_extensions.go`, the save/show/schema wiring, a tracer test through the admin HTTP API, docs, and the UI-SPEC C1 backstop test. +Output: the pact types, `modules/cabana/form_extensions.go`, the save/show/schema wiring, a tracer test through the admin HTTP API, docs, and the UI-SPEC C1 backstop test. Which plan implements each decision of the phase, and what Phase 33 takes, is in `32-DECISIONS-MAP.md`. **Planner decisions recorded here (Claude's discretion: the exact Go shape of the seam, CONTEXT):** - **P-01 (placement data in Go, definitions in fields.yaml keys):** an extension declares tabs and fields as Go structs (`AdminFormTab`, `AdminFormField` with `Before`, `After`, `Zone`, matching D-91's and D-101's wording) and each field's ordinary fields.yaml keys in `AdminFormField.Config` (a `map[string]any` holding label, type, span, comment, options and the rest). cabana encodes `Config` with goccy/go-yaml and compiles it through the same `decodeFieldMapping`/`compileFieldNode` path as a fields.yaml entry, so every type, option and plugin-field-type check applies unchanged. Go declarations let a plugin generate fields at boot (the per-type settings tabs of D-104) without templating YAML. Confirmed or replaced at the Task 1 checkpoint. diff --git a/.planning/phases/32-seo-foundation/32-02-PLAN.md b/.planning/phases/32-seo-foundation/32-02-PLAN.md index 9c32556c..d2d48b44 100644 --- a/.planning/phases/32-seo-foundation/32-02-PLAN.md +++ b/.planning/phases/32-seo-foundation/32-02-PLAN.md @@ -52,7 +52,7 @@ must_haves: - "Journal registers descriptors for journal-post, journal-category and the new journal-tag type and imports neither sm-seo-plugin nor sm-sitemap-plugin (D-93, D-96, D-102, SC-2)" - "Journal's post API and post descriptor resolve translated slugs for the request locale (D-58, D-96)" - "AdminFormSave carries the same Changes list, so an extension's Save sees a slug change inside the save transaction (D-92, D-91)" - - "The framework change is strictly additive: new optional fields, new types and new functions only; existing pact, cabana and signpost signatures and behaviour are unchanged, and journal, sitemap and the demo app build unchanged against it" + - "The framework change is strictly additive (D-106): new optional fields, new types and new functions only, and no method added to an existing interface; existing pact, cabana and signpost signatures and behaviour are unchanged, and journal, sitemap and the demo app build unchanged against it" prohibitions: - statement: "No breaking change to existing framework API: MenuItemType, AdminFormSave and every existing cabana and signpost function keep their signatures and behaviour; a plugin that registers no Content descriptor and no listener sees no difference" status: resolved @@ -111,7 +111,7 @@ Implement D-92 record events and D-93 content descriptors in the framework, and Purpose: SC-3 needs a slug change from the admin, MCP or an import to reach the SEO plugin; SC-2 needs Journal posts, categories and tags to take part without importing sm-seo-plugin; D-11 and D-20c need a public path mapped to its record. The framework names no SEO concept (D-90). -Output: the pact types, `modules/cabana/record_events.go`, `modules/signpost/content.go`, docs, and the Journal changes committed in `../sm-journal-plugin` (branch main). +Output: the pact types, `modules/cabana/record_events.go`, `modules/signpost/content.go`, docs, and the Journal changes committed in `../sm-journal-plugin` (branch main). Which plan implements each decision of the phase, and what Phase 33 takes, is in `32-DECISIONS-MAP.md`. **Planner decisions recorded here (Claude's discretion: the exact Go shape, CONTEXT):** - **P-07 (event shape):** `type RecordChange struct { Attribute, Locale string; Old, New any }`; `type RecordSaved struct { Controller, Model, ID, Locale string; Created bool; Changes []RecordChange }`; `type RecordDeleted struct { Controller, Model, ID string }`. `Model` is `lagoon.MorphType(db, record)` (the attach.Owner morph name, else the table name), so the event and the descriptor agree. `ID` is the primary key as text. `Locale` is the request's content locale (towel). On create, `Changes` lists every written column with a nil Old. Fired and listened to as pointers (`app.Events.Listen("vendor.plugin", func(ctx, *pact.RecordSaved) error)`). diff --git a/.planning/phases/32-seo-foundation/32-03-PLAN.md b/.planning/phases/32-seo-foundation/32-03-PLAN.md index 1cd0d3f9..208985e2 100644 --- a/.planning/phases/32-seo-foundation/32-03-PLAN.md +++ b/.planning/phases/32-seo-foundation/32-03-PLAN.md @@ -56,7 +56,6 @@ files_modified: - ../sm-seo-plugin/controllers/history/config_form.yaml - ../sm-seo-plugin/controllers/history/_history_diff.htm - ../sm-seo-plugin/controllers/api/resolve.go - - ../sm-seo-plugin/controllers/api/helpers.go - ../sm-seo-plugin/extension/_seo_status.htm - ../sm-seo-plugin/extension/_seo_effective.htm - ../sm-seo-plugin/extension/_placeholders.htm @@ -195,10 +194,10 @@ Redirects, normalisation, strict URLs and robots.txt are plan 32-04; the managem Purpose: SC-2 (Journal posts, categories and tags show the SEO tab without importing sm-seo-plugin; the frontend gets the effective SEO of any URL with the precedence URL rule > record > global > derived). -Output: the new repository with commits on branch `main`, nothing pushed (pushing waits for Jakub's go-ahead in plan 32-06). +Output: the new repository with commits on branch `main`, nothing pushed (pushing waits for Jakub's go-ahead in plan 32-06). Which plan implements each decision of the phase, and what Phase 33 takes, is in `32-DECISIONS-MAP.md`. **Planner decisions recorded here (Claude's discretion: table and package names, CONTEXT):** -- **P-12 (repository):** clone `git@git.golem15.com:golem15/sm-seo-plugin.git` (it exists, empty, public) into `../sm-seo-plugin` and work on branch `main`; if cloning an empty repository is refused, `git init -b main` and `git remote add origin git@git.golem15.com:golem15/sm-seo-plugin.git`. No push in this plan. `go.mod` requires the framework at the latest existing tag from `git -C ../summercms.go tag --sort=-v:refname | head -1` (no version written into the plan); `go.work` (tracked, like the sibling plugins) uses `.` and replaces `git.golem15.com/golem15/summercms => ../summercms.go`. +- **P-12 (repository):** clone `git@git.golem15.com:golem15/sm-seo-plugin.git` (it exists, empty, public: created on 2026-10-26 with `tea`, D-106) into `../sm-seo-plugin` and work on branch `main`; if cloning an empty repository is refused, `git init -b main` and `git remote add origin git@git.golem15.com:golem15/sm-seo-plugin.git`. No push in this plan. `go.mod` requires the framework at the latest existing tag from `git -C ../summercms.go tag --sort=-v:refname | head -1` (no version written into the plan); `go.work` (tracked, like the sibling plugins) uses `.` and replaces `git.golem15.com/golem15/summercms => ../summercms.go`. - **P-13 (tables):** prefix `golem15_seo_`; `entries` (id, model_type, record_id text, locale, type_key, url, title, description, keyphrase, canonical, robots, og_title, og_description, og_image, effective_title, effective_description, data jsonb not null default '{}', created_at, updated_at; unique (model_type, record_id, locale)); `revisions` (id, batch_id text null, author_id null, author_label, via admin|mcp|import|auto|undo, subject_kind entry|rule|redirect|robots|settings|notfound, subject_id text, what, before jsonb, after jsonb, undone_at null, undone_by_id null, created_at); `settings` (singleton row 1; site_name, twitter_site, title_template default '%title% | %site%', home_template default '%site%', default_description, default_og_image, query_params jsonb seeded with page=content, utm_*/fbclid/gclid/msclkid=tracking, sort=filter; noindex_paths, search_paths, url_host keep|www|bare, url_slash keep|add|remove, url_lowercase bool default false, verify_google, verify_bing, verify_yandex, verify_other jsonb, verify_files jsonb, csv_delimiter, robots columns added in plan 32-04); `type_settings` (type_key primary key, inject bool default true, title_template, description_template, robots); `rules` (id, url, match exact|pattern, title, description, canonical, robots, og_title, og_description, og_image, note, timestamps; unique (url, match)). Raw SQL DDL, timestamped migration IDs, one file per table (sm-sitemap-plugin pattern). - **P-14 (precedence per value, D-02 with the approved UI-SPEC copy):** title: rule > record > type title template, else the general template, applied to the content title (source template) > content title > site name; description: rule > record > type description template > content description (excerpt) > settings default description (source global); og:title, og:description: rule > record > effective title/description; og:image: rule > record > content image > settings default social image; canonical: rule > record > self; robots per directive: rule > record > never-indexed paths (noindex,nofollow), search paths and URLs carrying a filter parameter (noindex,follow) > type default > index,follow. The per-type template is the "global defaults" layer of D-02; the site-wide default description and default social image are last-resort fallbacks after content, as the approved copy `settings.default_description_comment` states. - **P-15 (resolve answer):** `GET /_seo/api/v1/resolve?url=` (throttle bucket `seo-resolve`, no auth) answers `{"action":"ok"|"not_found","status":200|404,"url":,"locale":...,"staging":bool,"seo":{"title","description","canonical","robots","x_robots_tag","og":{"title","description","image","url"},"twitter":{"card","site"},"verification":[{"name","content"}]}}`; plan 32-04 adds `redirect` and `gone`. Absolute URLs use the config key `golem15.seo.site_url` (fallback `app.url`), never the request Host. Unpublished records and unknown paths without a rule answer `not_found`. @@ -228,13 +227,28 @@ Framework: cooler.PurgeSiteCacheAfterCommit(ctx, db, purger, kind, slug); compas +## Execution units and resume points + +This plan creates a repository and has five tasks, so each task is a resumable unit and the plan can run across several executor contexts without losing work: +- Every task ends with the commit groups listed below, in order, in `../sm-seo-plugin`: `git -c safe.directory='*' -C ../sm-seo-plugin add ` (paths only, never `-A` or `.`), then `commit -m ""`. No task leaves uncommitted files for the next one, every commit keeps `go vet ./...` and `go test -short ./...` green in the plugin, and each group adds the lang keys it uses to both lang files. +- Resume check before each task: `git -c safe.directory='*' -C ../sm-seo-plugin log --format=%s | grep -Fx ''` for each subject of the task. All present: the task is done; re-run its `` and go on. Some present: continue at the first missing group. None: start the task. +- An executor whose context is getting heavy stops after a task's last commit and returns its completed-task table (task, subjects, SHAs); the continuation starts at the next task from that table. The Task 4 checkpoint is always such a boundary: Tasks 1 to 3 run in one context and Task 5 in a fresh one. + +| Task | Commit groups, in order | Resume point after the task | +|------|-------------------------|-----------------------------| +| 1 | `feat: plugin scaffold, entries and the SEO tab tracer` | TestSEOTabTracer passes: the SEO tab saves a title and resolve returns it | +| 2 | A `feat: URL rules and the settings singleton`; B `feat: per-type settings tabs and the SEO tab permission`; C `feat: full precedence resolver with value sources` | rules, settings and the full resolver work; Task 3 builds on `classes/service.go` only | +| 3 | A `feat: SEO change history`; B `feat: entry generation job and cleanup on delete` | history, the generation job and the delete listener work; nothing of the element toolchain is installed | +| 4 | none (checkpoint); the continuation prompt carries the answer, and 32-03-SUMMARY.md records it | the four npm pins and grobotstxt approved or refused | +| 5 | A `feat: SEO field elements and bundle`; B `feat: SEO field types and the complete SEO tab`; C `docs: plugin README` | plan complete; write 32-03-SUMMARY.md | + Task 1: End-to-end "an editor sets a record's SEO title in its SEO tab and the frontend's resolve answer returns it" — one path only `git ls-remote git@git.golem15.com:golem15/sm-seo-plugin.git` exits 0 (the repository exists; empty output is fine); plans 32-01 and 32-02 are committed in summercms.go The entries table keyed by (model type, record id, locale) and the opt-in through signpost descriptors are what every content plugin builds on; confirmed by Jakub as D-01 (already decided, no checkpoint). - ../sm-seo-plugin/go.mod, ../sm-seo-plugin/go.sum, ../sm-seo-plugin/go.work, ../sm-seo-plugin/.gitignore, ../sm-seo-plugin/README.md, ../sm-seo-plugin/plugin.go, ../sm-seo-plugin/logger.go, ../sm-seo-plugin/extensions.go, ../sm-seo-plugin/routes.go, ../sm-seo-plugin/models/registry.go, ../sm-seo-plugin/models/entry.go, ../sm-seo-plugin/updates/registry.go, ../sm-seo-plugin/updates/202610270001_create_golem15_seo_entries.go, ../sm-seo-plugin/classes/service.go, ../sm-seo-plugin/classes/resolver.go, ../sm-seo-plugin/classes/entries.go, ../sm-seo-plugin/controllers/api/resolve.go, ../sm-seo-plugin/controllers/api/helpers.go, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/testmain_test.go, ../sm-seo-plugin/fixture_test.go, ../sm-seo-plugin/seo_tab_test.go + ../sm-seo-plugin/go.mod, ../sm-seo-plugin/go.sum, ../sm-seo-plugin/go.work, ../sm-seo-plugin/.gitignore, ../sm-seo-plugin/plugin.go, ../sm-seo-plugin/logger.go, ../sm-seo-plugin/extensions.go, ../sm-seo-plugin/routes.go, ../sm-seo-plugin/models/registry.go, ../sm-seo-plugin/models/entry.go, ../sm-seo-plugin/updates/registry.go, ../sm-seo-plugin/updates/202610270001_create_golem15_seo_entries.go, ../sm-seo-plugin/classes/service.go, ../sm-seo-plugin/classes/resolver.go, ../sm-seo-plugin/classes/entries.go, ../sm-seo-plugin/controllers/api/resolve.go, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/testmain_test.go, ../sm-seo-plugin/fixture_test.go, ../sm-seo-plugin/seo_tab_test.go - ../sm-sitemap-plugin/plugin.go, admin.go, go.mod, go.work, .gitignore, testmain_test.go, models/registry.go, updates/registry.go, updates/202610110002_create_golem15_sitemap_settings.go - ../sm-journal-plugin/routes.go and controllers/api/posts.go (handler shape, wire.WriteJSON) @@ -245,13 +259,13 @@ Framework: cooler.PurgeSiteCacheAfterCommit(ctx, db, purger, kind, slug); compas Per D-01, D-90, D-91, D-93, D-94 and P-12/P-13/P-15: - 1. Repository (P-12): clone into `../sm-seo-plugin`, branch `main`; `go.mod` module `git.golem15.com/golem15/sm-seo-plugin`, go 1.27.0, requiring the framework at the latest existing tag and gorm, gormigrate, pgx, testcontainers at the versions sm-sitemap-plugin pins; tracked `go.work` with the framework replace; `.gitignore` with `node_modules/` and `go.work.sum`; a README skeleton in the standard plugin structure. - 2. `plugin.go`: `Plugin{app *backpack.App; service *classes.Service}`, `ID() "golem15.seo"`, `Requires() nil`, Register stores the app, Boot builds the service (DB, signpost registry, config) and fails with `golem15.seo: %w`; compile-time assertions for party.Plugin, pact.HasConfig, HasMigrations, HasModels, HasRoutes, HasAdminActionExtensions, HasLang. + 1. Repository (P-12): clone into `../sm-seo-plugin`, branch `main`; `go.mod` module `git.golem15.com/golem15/sm-seo-plugin`, go 1.27.0, requiring the framework at the latest existing tag and gorm, gormigrate, pgx, testcontainers at the versions sm-sitemap-plugin pins; tracked `go.work` with the framework replace; `.gitignore` with `node_modules/` and `go.work.sum` (the README is written in Task 5). + 2. `logger.go`: `pluginLog = diary.For("seo")` (sm-sitemap-plugin pattern). `plugin.go`: `Plugin{app *backpack.App; service *classes.Service}`, `ID() "golem15.seo"`, `Requires() nil`, Register stores the app, Boot builds the service (DB, signpost registry, config) and fails with `golem15.seo: %w`; compile-time assertions for party.Plugin, pact.HasConfig, HasMigrations, HasModels, HasRoutes, HasAdminActionExtensions, HasLang. 3. `models/entry.go` + the entries migration (P-13); `classes/entries.go` upsert/load per (model_type, record_id) returning locale maps; `classes/service.go` the single service every writer calls. 4. `extensions.go`: `AdminActionExtensions()` reads `signpost` `ContentTypes()` lazily and returns, for each type with `Content.Controller`, an extension adding tab `golem15.seo::lang.tab.seo` (no anchor, P-17) with, for the tracer, one field `seo_title` of built-in type `mltext` (replaced by `mlseotext` in Task 5); Load returns `{"seo_title": map[locale]title}`; Save upserts the entry rows inside `in.DB` with the URL snapshot from signpost `Describe` (reading through the save transaction). 5. `classes/resolver.go` (tracer subset): `Resolver.Resolve(ctx, rawURL string) (Result, error)` normalises the input to a path, finds the owner through signpost `MatchPath`, loads the entry for the matched locale and answers the title with source `record`, else the content title with source `content`; unpublished or unmatched → `not_found`. `routes.go`: `GET /_seo/api/v1/resolve` (throttle `seo-resolve` bucket via Buckets with `surf.TrustedProxies`); `controllers/api/resolve.go` writes the P-15 JSON with `wire.WriteJSON`. 6. Tests: `testmain_test.go` copied from sm-sitemap-plugin (DB/user `golem15_seo`, `flag.Parse()` then `testing.Short()`); `fixture_test.go`: a fake content plugin `acme.pages` with admin controller `acme.pages.pages` (model with title, slug; a fields.yaml with tabs), a menu item type `acme-page` (`/pages/:slug`) with a Content descriptor; `TestSEOTabTracer` in `seo_tab_test.go`: boot both plugins with cabana and signpost, GET the form schema of `acme.pages.pages` (contains `seo_title` in tab `golem15.seo::lang.tab.seo`), PUT a page with `seo_title` {en: "Custom"}, then `GET /_seo/api/v1/resolve?url=/pages/` answers `seo.title` "Custom" and the owner table has no SEO column. - Commit in `../sm-seo-plugin` with `git -c safe.directory='*'`: `feat: plugin scaffold, entries and the SEO tab tracer`. Do not push. + 7. Commit (resume unit, see Execution units): stage exactly this task's `` paths in `../sm-seo-plugin` with `git -c safe.directory='*'` and commit `feat: plugin scaffold, entries and the SEO tab tracer`. Do not push. go -C ../sm-seo-plugin test ./... -run 'TestSEOTabTracer' -count=1 -v 2>&1 | tail -20 @@ -272,7 +286,7 @@ Framework: cooler.PurgeSiteCacheAfterCommit(ctx, db, purger, kind, slug); compas Task 2: URL rules, global and per-type settings (AdminSingleton + D-91 on itself) and the full precedence resolver - ../sm-seo-plugin/models/settings.go, ../sm-seo-plugin/models/type_setting.go, ../sm-seo-plugin/models/rule.go, ../sm-seo-plugin/models/settings/fields.yaml, ../sm-seo-plugin/models/rule/fields.yaml, ../sm-seo-plugin/models/rule/columns.yaml, ../sm-seo-plugin/updates/202610270003_create_golem15_seo_settings.go, ../sm-seo-plugin/updates/202610270004_create_golem15_seo_type_settings.go, ../sm-seo-plugin/updates/202610270005_create_golem15_seo_rules.go, ../sm-seo-plugin/controllers/admin_registry.go, ../sm-seo-plugin/controllers/settings.go, ../sm-seo-plugin/controllers/settings/config_form.yaml, ../sm-seo-plugin/controllers/settings/_staging.htm, ../sm-seo-plugin/controllers/settings/_placeholders.htm, ../sm-seo-plugin/controllers/rules.go, ../sm-seo-plugin/controllers/rules/config_list.yaml, ../sm-seo-plugin/controllers/rules/config_form.yaml, ../sm-seo-plugin/admin.go, ../sm-seo-plugin/admin_permissions.go, ../sm-seo-plugin/admin_navigation.go, ../sm-seo-plugin/extensions.go, ../sm-seo-plugin/classes/resolver.go, ../sm-seo-plugin/classes/templates.go, ../sm-seo-plugin/classes/urls.go, ../sm-seo-plugin/classes/robots_value.go, ../sm-seo-plugin/config/config.yaml, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/settings_test.go, ../sm-seo-plugin/seo_tab_test.go + ../sm-seo-plugin/models/settings.go, ../sm-seo-plugin/models/type_setting.go, ../sm-seo-plugin/models/rule.go, ../sm-seo-plugin/models/settings/fields.yaml, ../sm-seo-plugin/models/rule/fields.yaml, ../sm-seo-plugin/models/rule/columns.yaml, ../sm-seo-plugin/updates/202610270003_create_golem15_seo_settings.go, ../sm-seo-plugin/updates/202610270004_create_golem15_seo_type_settings.go, ../sm-seo-plugin/updates/202610270005_create_golem15_seo_rules.go, ../sm-seo-plugin/controllers/admin_registry.go, ../sm-seo-plugin/controllers/settings.go, ../sm-seo-plugin/controllers/settings/config_form.yaml, ../sm-seo-plugin/controllers/settings/_staging.htm, ../sm-seo-plugin/controllers/rules.go, ../sm-seo-plugin/controllers/rules/config_list.yaml, ../sm-seo-plugin/controllers/rules/config_form.yaml, ../sm-seo-plugin/admin.go, ../sm-seo-plugin/admin_permissions.go, ../sm-seo-plugin/admin_navigation.go, ../sm-seo-plugin/extensions.go, ../sm-seo-plugin/classes/resolver.go, ../sm-seo-plugin/classes/templates.go, ../sm-seo-plugin/classes/urls.go, ../sm-seo-plugin/classes/robots_value.go, ../sm-seo-plugin/config/config.yaml, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/settings_test.go, ../sm-seo-plugin/seo_tab_test.go - ../sm-sitemap-plugin/controllers/definitions.go (AdminSingleton), controllers/admin_registry.go, admin.go (Settings link entry), admin_permissions.go, models/settings.go, models/settings/fields.yaml - ../sm-paymentgateway-plugin/controllers/orders.go 24-50, 106-155 (list/form controller hooks, FilterOptions, ListExtendRecords) @@ -284,11 +298,14 @@ Framework: cooler.PurgeSiteCacheAfterCommit(ctx, db, purger, kind, slug); compas Per D-02, D-03, D-04, D-05, D-06, D-07, D-08, D-15, D-21, D-22, D-23, D-104 and P-13/P-14/P-15: 1. Models and migrations for settings (row 1 inserted by the migration with the seeded query parameters), type_settings, rules. Rule validation: url starts with "/", `*` only in a pattern (`rules.url_invalid`), unique (url, match) (`rules.url_taken`); settings: verification file names `^[A-Za-z0-9._-]+\.(html|txt|xml)$` (`settings.verify_filename_invalid`), unique names (`settings.verify_filename_taken`), templates with only known placeholders %title%, %description%, %site% (`settings.template_unknown`). - 2. Controllers: `golem15.seo.settings` (`AdminSingleton`, tabs General, Indexing, Addresses, Verification per UI-SPEC C11, staging partial, P3 placeholders partial); `golem15.seo.rules` (list + form per C7, delete confirms, `ListExtendRecords` builds the `overrides` column). The settings tile is a link entry (`pact.SettingsItem{Code: "seo_settings", Controller: "golem15.seo.settings", Icon: "search", Order: 100, ...}`, D-104); one permission `golem15.seo.manage` (D-15) on every controller and the tile; navigation item `seo` (icon `search`, order 600) landing on `golem15.seo.rules` for now with side-menu items URL rules and History (plan 32-04 re-points it to Redirects per C2). - 3. D-104: in `extensions.go` add one extension on `golem15.seo.settings` whose Tabs place one tab per content type (Name = the type's signpost Label key, the first After the General tab key, each next After the previous) and whose Fields are `type__inject` (switch, default on), `type__title_template`, `type__description_template`, `type__robots` (built-in dropdown until Task 5 swaps in `seorobots` with `scope: type`; defaults per D-21: a tag type `noindex,follow`, other types empty) and a `type__placeholders` partial; Load/Save read and upsert `type_settings`. The SEO tab extension's Visible reads `type_settings.inject` per request (answers true when it cannot decide). + 2. Controllers: `golem15.seo.settings` (`AdminSingleton`, tabs General, Indexing, Addresses, Verification per UI-SPEC C11, staging partial; the P3 placeholders partial comes with the other partials in Task 5); `golem15.seo.rules` (list + form per C7, delete confirms, `ListExtendRecords` builds the `overrides` column). The settings tile is a link entry (`pact.SettingsItem{Code: "seo_settings", Controller: "golem15.seo.settings", Icon: "search", Order: 100, ...}`, D-104); one permission `golem15.seo.manage` (D-15) on every controller and the tile; navigation item `seo` (icon `search`, order 600) landing on `golem15.seo.rules` for now with the side-menu item URL rules (Task 3 adds History when its controller exists; plan 32-04 re-points the item to Redirects per C2). `admin.go` embeds the admin files by glob (`controllers/*/*.yaml controllers/*/_*.htm models/*/*.yaml`) instead of file by file as sm-sitemap-plugin does, so later tasks and plans add screens and partials without editing it (Task 5 adds `extension/_*.htm`). + 3. D-104: in `extensions.go` add one extension on `golem15.seo.settings` whose Tabs place one tab per content type (Name = the type's signpost Label key, the first After the General tab key, each next After the previous) and whose Fields are `type__inject` (switch, default on), `type__title_template`, `type__description_template`, `type__robots` (built-in dropdown until Task 5 swaps in `seorobots` with `scope: type`; defaults per D-21: a tag type `noindex,follow`, other types empty); Task 5 adds the `type__placeholders` P3 partial field. Load/Save read and upsert `type_settings`. The SEO tab extension's Visible reads `type_settings.inject` per request (answers true when it cannot decide). 4. Resolver complete per P-14: templates with placeholders (`classes/templates.go`); canonical building and the D-07 parameter list with path refinement (`classes/urls.go`; site URL from `golem15.seo.site_url`, fallback `app.url`); robots per directive with the value grammar of UI-SPEC C5 (`classes/robots_value.go`: parse and serialise in the fixed order, refuse malformed input with `pact.FormFieldError`); never-indexed and search paths (one pattern per line, `*` wildcard) for D-21/D-23; the X-Robots-Tag value equal to the robots directives; OG/Twitter values (D-04) with twitter:card summary_large_image when an image exists; verification meta on the home path (D-05); the non-production lock from `app.Config.Environment()` != "production" forcing noindex,nofollow and `staging: true` (D-22). `Result` carries a source per value for P2 and plan 32-05's read tool. - 5. Tests (smoke): `TestResolverPrecedence` table in `settings_test.go` covering each P-14 chain, a pattern rule `/blog?tag=*`, the parameter list (utm stripped, page kept, sort → noindex,follow) and the staging lock; `TestSettingsTypeTabs` (two content types → General, type A, type B, Indexing, Addresses, Verification; zero types → four fixed tabs); `TestSEOTabPermission` (an admin without golem15.seo.manage sees no SEO tab and a posted seo_title changes nothing). - Commit per logical change in `../sm-seo-plugin`, staged by path. + 5. Tests (smoke): `TestResolverPrecedence` table in `settings_test.go` covering each P-14 chain, a pattern rule `/blog?tag=*`, the parameter list (utm stripped, page kept, sort → noindex,follow) and the staging lock; `TestSettingsTypeTabs` (two content types → General, type A, type B, Indexing, Addresses, Verification; zero types → four fixed tabs); `TestSEOTabPermission` in `seo_tab_test.go` (an admin without golem15.seo.manage sees no SEO tab and a posted seo_title changes nothing). + 6. Commits (resume units, see Execution units), staged by path in `../sm-seo-plugin` (paths relative to it), in this order: + - A `feat: URL rules and the settings singleton`: models/settings.go, models/rule.go, models/settings/fields.yaml, models/rule/fields.yaml, models/rule/columns.yaml, the settings and rules migrations, controllers/admin_registry.go, controllers/settings.go, controllers/settings/config_form.yaml, controllers/settings/_staging.htm, controllers/rules.go, controllers/rules/config_list.yaml, controllers/rules/config_form.yaml, admin.go, admin_permissions.go, admin_navigation.go, lang/en/lang.yaml, lang/pl/lang.yaml; + - B `feat: per-type settings tabs and the SEO tab permission`: models/type_setting.go, the type_settings migration, extensions.go, lang files, settings_test.go (TestSettingsTypeTabs), seo_tab_test.go (TestSEOTabPermission); + - C `feat: full precedence resolver with value sources`: classes/resolver.go, classes/templates.go, classes/urls.go, classes/robots_value.go, config/config.yaml, settings_test.go (TestResolverPrecedence). go -C ../sm-seo-plugin test ./... -run 'TestResolverPrecedence|TestSettingsTypeTabs|TestSEOTabPermission|TestSEOTabTracer' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -30 @@ -308,7 +325,7 @@ Framework: cooler.PurgeSiteCacheAfterCommit(ctx, db, purger, kind, slug); compas Task 3: History core, entry generation job, entry cleanup on delete - ../sm-seo-plugin/models/revision.go, ../sm-seo-plugin/models/revision/fields.yaml, ../sm-seo-plugin/models/revision/columns.yaml, ../sm-seo-plugin/updates/202610270002_create_golem15_seo_revisions.go, ../sm-seo-plugin/classes/history.go, ../sm-seo-plugin/classes/generate.go, ../sm-seo-plugin/classes/service.go, ../sm-seo-plugin/controllers/history.go, ../sm-seo-plugin/controllers/history/config_list.yaml, ../sm-seo-plugin/controllers/history/config_form.yaml, ../sm-seo-plugin/controllers/history/_history_diff.htm, ../sm-seo-plugin/controllers/admin_registry.go, ../sm-seo-plugin/jobs.go, ../sm-seo-plugin/commands.go, ../sm-seo-plugin/listeners.go, ../sm-seo-plugin/plugin.go, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/seo_tab_test.go + ../sm-seo-plugin/models/revision.go, ../sm-seo-plugin/models/revision/fields.yaml, ../sm-seo-plugin/models/revision/columns.yaml, ../sm-seo-plugin/updates/202610270002_create_golem15_seo_revisions.go, ../sm-seo-plugin/classes/history.go, ../sm-seo-plugin/classes/generate.go, ../sm-seo-plugin/classes/service.go, ../sm-seo-plugin/controllers/history.go, ../sm-seo-plugin/controllers/history/config_list.yaml, ../sm-seo-plugin/controllers/history/config_form.yaml, ../sm-seo-plugin/controllers/history/_history_diff.htm, ../sm-seo-plugin/controllers/admin_registry.go, ../sm-seo-plugin/admin_navigation.go, ../sm-seo-plugin/controllers/settings.go, ../sm-seo-plugin/jobs.go, ../sm-seo-plugin/commands.go, ../sm-seo-plugin/listeners.go, ../sm-seo-plugin/plugin.go, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/seo_tab_test.go - ../sm-chat-plugin/plugin.go 85-100 (conga.Job, OnQueue, MaxAttempts) and its jobs package - summercms.go/modules/pact/capabilities.go (HasJobs, HasCommands) and docs/plugins/scheduling.md @@ -318,11 +335,13 @@ Framework: cooler.PurgeSiteCacheAfterCommit(ctx, db, purger, kind, slug); compas Per D-13, D-92, D-94: - 1. History core: every service write (entry, rule, settings, type settings) appends one revision row per changed object with author id/label from the towel actor, `via` = `mcp` when `bouncer.Dispatched(ctx, "backend")` answers true else `admin` (import/auto/undo set by later plans), `what` built from the UI-SPEC `history.what_*` keys, before/after jsonb of the changed fields only, all inside the caller's transaction. History screen `golem15.seo.history` per C12: list (columns, filters via/created_at/author, `history.list_empty`, row state `disabled` for undone), preview with P5 `history_diff` (Field / Before / After, DM Mono for URL values, created/deleted markers). Undo record actions are plan 32-05. - 2. GenerateEntries (D-94): `jobs.go` registers `conga.Job` for `GenerateEntriesArgs{Type string}`; `classes/generate.go` pages signpost `Records` per content type (limit 200) and upserts entry rows with each locale's URL, type key and the effective-title/description cache, never overwriting editor values; `commands.go` adds the console command `seo:generate-entries` with an optional `--type` flag that enqueues the job; the settings singleton's FormAfterUpdate enqueues it too. + 1. History core: every service write (entry, rule, settings, type settings) appends one revision row per changed object with author id/label from the towel actor, `via` = `mcp` when `bouncer.Dispatched(ctx, "backend")` answers true else `admin` (import/auto/undo set by later plans), `what` built from the UI-SPEC `history.what_*` keys, before/after jsonb of the changed fields only, all inside the caller's transaction. History screen `golem15.seo.history` per C12: list (columns, filters via/created_at/author, `history.list_empty`, row state `disabled` for undone), preview with P5 `history_diff` (Field / Before / After, DM Mono for URL values, created/deleted markers), and its side-menu entry History in `admin_navigation.go`. Undo record actions are plan 32-05. + 2. GenerateEntries (D-94): `jobs.go` registers `conga.Job` for `GenerateEntriesArgs{Type string}`; `classes/generate.go` pages signpost `Records` per content type (limit 200) and upserts entry rows with each locale's URL, type key and the effective-title/description cache, never overwriting editor values; `commands.go` adds the console command `seo:generate-entries` with an optional `--type` flag that enqueues the job; the settings singleton's FormAfterUpdate (`controllers/settings.go`) enqueues it too. 3. `listeners.go` (registered in Boot): `*pact.RecordDeleted` for a model with a content type deletes its entry rows inside a transaction and records a `deleted` revision so the values stay in history (D-92, D-94); `*pact.RecordSaved` with Created true upserts the URL snapshot of the new record (address-change redirects are plan 32-04). 4. Tests (smoke) in `seo_tab_test.go`: `TestHistoryRecordsAdminWrite` (saving the SEO tab writes one revision with via admin and before/after), `TestEntryDeletedWithRecord`, `TestGenerateEntries` (two fake records → two entry rows with URLs; editor values untouched on a second run). - Commit in `../sm-seo-plugin`, staged by path. + 5. Commits (resume units, see Execution units), staged by path in `../sm-seo-plugin`, in this order: + - A `feat: SEO change history`: models/revision.go, models/revision/fields.yaml, models/revision/columns.yaml, the revisions migration, classes/history.go, classes/service.go, controllers/history.go, controllers/history/config_list.yaml, controllers/history/config_form.yaml, controllers/history/_history_diff.htm, controllers/admin_registry.go, admin_navigation.go, lang files, seo_tab_test.go (TestHistoryRecordsAdminWrite); + - B `feat: entry generation job and cleanup on delete`: classes/generate.go, jobs.go, commands.go, listeners.go, plugin.go, controllers/settings.go, lang files, seo_tab_test.go (TestEntryDeletedWithRecord, TestGenerateEntries). go -C ../sm-seo-plugin test ./... -run 'TestHistoryRecordsAdminWrite|TestEntryDeletedWithRecord|TestGenerateEntries' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -20 @@ -353,7 +372,7 @@ Framework: cooler.PurgeSiteCacheAfterCommit(ctx, db, purger, kind, slug); compas Task 5: Plugin field types and elements E1, E2, E3, E7, partials P1/P2/P3 and the complete SEO tab (UI-SPEC C3-C6) Task 4 answered "approved" for typescript 5.9.3, vite 7.3.5, vitest 3.2.7 and happy-dom 20.11.6 - ../sm-seo-plugin/package.json, ../sm-seo-plugin/package-lock.json, ../sm-seo-plugin/tsconfig.json, ../sm-seo-plugin/vite.config.ts, ../sm-seo-plugin/scripts/check-dist.sh, ../sm-seo-plugin/assets/src/elements/index.ts, ../sm-seo-plugin/assets/src/elements/measure.ts, ../sm-seo-plugin/assets/src/elements/labels.ts, ../sm-seo-plugin/assets/src/elements/text.ts, ../sm-seo-plugin/assets/src/elements/robots.ts, ../sm-seo-plugin/assets/src/elements/serp.ts, ../sm-seo-plugin/assets/src/elements/image.ts, ../sm-seo-plugin/assets/src/elements/styles.css, ../sm-seo-plugin/assets/dist/seo.js, ../sm-seo-plugin/assets/dist/seo.css, ../sm-seo-plugin/field_types.go, ../sm-seo-plugin/classes/field_types.go, ../sm-seo-plugin/classes/image.go, ../sm-seo-plugin/classes/snippet.go, ../sm-seo-plugin/extensions.go, ../sm-seo-plugin/extension/_seo_status.htm, ../sm-seo-plugin/extension/_seo_effective.htm, ../sm-seo-plugin/extension/_placeholders.htm, ../sm-seo-plugin/controllers/rules/_seo_status.htm, ../sm-seo-plugin/controllers/rules/_seo_effective.htm, ../sm-seo-plugin/models/rule/fields.yaml, ../sm-seo-plugin/models/settings/fields.yaml, ../sm-seo-plugin/admin.go, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/README.md, ../sm-seo-plugin/seo_tab_test.go + ../sm-seo-plugin/package.json, ../sm-seo-plugin/package-lock.json, ../sm-seo-plugin/tsconfig.json, ../sm-seo-plugin/vite.config.ts, ../sm-seo-plugin/scripts/check-dist.sh, ../sm-seo-plugin/assets/src/elements/index.ts, ../sm-seo-plugin/assets/src/elements/measure.ts, ../sm-seo-plugin/assets/src/elements/labels.ts, ../sm-seo-plugin/assets/src/elements/text.ts, ../sm-seo-plugin/assets/src/elements/robots.ts, ../sm-seo-plugin/assets/src/elements/serp.ts, ../sm-seo-plugin/assets/src/elements/image.ts, ../sm-seo-plugin/assets/src/elements/styles.css, ../sm-seo-plugin/assets/dist/seo.js, ../sm-seo-plugin/assets/dist/seo.css, ../sm-seo-plugin/field_types.go, ../sm-seo-plugin/classes/field_types.go, ../sm-seo-plugin/classes/image.go, ../sm-seo-plugin/classes/snippet.go, ../sm-seo-plugin/extensions.go, ../sm-seo-plugin/extension/_seo_status.htm, ../sm-seo-plugin/extension/_seo_effective.htm, ../sm-seo-plugin/extension/_placeholders.htm, ../sm-seo-plugin/controllers/settings/_placeholders.htm, ../sm-seo-plugin/controllers/rules/_seo_status.htm, ../sm-seo-plugin/controllers/rules/_seo_effective.htm, ../sm-seo-plugin/models/rule/fields.yaml, ../sm-seo-plugin/models/settings/fields.yaml, ../sm-seo-plugin/admin.go, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/README.md, ../sm-seo-plugin/seo_tab_test.go - ../sm-journal-plugin/package.json, vite.config.ts, tsconfig.json, scripts/check-dist.sh, assets/src/blogeditor/ (index.ts, labels.ts, element structure), field_types.go, classes/blog_editor.go (80-109 type pair, 343-380 Upload) - ../sm-paymentgateway-plugin/assets/css/quote-cart.css (scoped element CSS with --c-* variables) @@ -365,10 +384,13 @@ Framework: cooler.PurgeSiteCacheAfterCommit(ctx, db, purger, kind, slug); compas Per D-04, D-06, D-08, D-09, D-100 and UI-SPEC: 1. Element bundle: `package.json` (name `golem15-seo-elements`, private, type module, scripts typecheck/build/test, devDependencies pinned exactly as approved, no runtime dependencies); `npm --prefix ../sm-seo-plugin install --save-dev --save-exact` with those pins; `vite.config.ts` copying Journal's (entry `assets/src/elements/index.ts`, outputs `seo.js` and `seo.css`, vitest with happy-dom and `tests/**/*.test.ts`); `scripts/check-dist.sh` copying Journal's drift gate. Elements in light DOM with CSS scoped under each tag and `--c-*` variables only (E-1 palette only inside `golem15-seo-serp .serp-card`): E1 `golem15-seo-text` (C4, shared `measure.ts` canvas measurement, thresholds of Q-3), E2 `golem15-seo-robots` (C5, value grammar identical to `classes/robots_value.go`), E3 `golem15-seo-serp` (C6: context and duplicates ops, desktop/mobile, pixel truncation, keyphrase checks with the C6 stem rule, one queued request while busy), E7 `golem15-seo-image` (upload through the field type's Upload, preview, remove). `labels.ts` holds the en/pl element copy of UI-SPEC with CLDR plural maps. `npm run build` and commit `assets/dist`. 2. Field types (`field_types.go` + `classes/field_types.go`): `seotext`/`mlseotext` (option `measure`), `seorobots`/`mlseorobots` (option `scope`, Decode refuses a malformed directive string), `seoimage` (P-16 Upload into `*blob.Bucket`, folder `seo`, image content sniffing, `FormFieldError` on a non-image). Each lists `assets/dist/seo.js` and `assets/dist/seo.css` (if cabana refuses the same asset in several types, list it on `mlseotext` only and record that in the SUMMARY). The extension's `seo_snippet` widget action lives in `AdminFormExtension.Actions` (`classes/snippet.go`: `context` answers locales with url, h1, siteName, titleTemplate, defaultTitle, defaultDescription, descriptionSource, bodyWords and host; `duplicates` answers other records' and rules' labels and admin URLs whose effective title or description equals the submitted one in that locale). - 3. Complete SEO tab per C3 (fields `seo_status` P1, `seo_snippet` E3, `seo_title` mlseotext, `seo_description` mlseotext, `seo_keyphrase` mltext left, `seo_canonical` mltext right with the `validation.url` check for a path or http(s) URL, `seo_robots` mlseorobots scope record, `seo_og_title`, `seo_og_description` mltextarea small, `seo_og_image` seoimage, `seo_effective` P2), partials from `extension/` through `Dir` and `PartialData` (P1: URL rule override and canonical-to-noindex callouts; P2: effective values per locale with source labels and the staging note); the rule form and settings use `seotext`, `seorobots` and `seoimage` where C7/C11 name text, robots and image fields; P3 placeholders partial on General and each type tab. + 3. Complete SEO tab per C3 (fields `seo_status` P1, `seo_snippet` E3, `seo_title` mlseotext, `seo_description` mlseotext, `seo_keyphrase` mltext left, `seo_canonical` mltext right with the `validation.url` check for a path or http(s) URL, `seo_robots` mlseorobots scope record, `seo_og_title`, `seo_og_description` mltextarea small, `seo_og_image` seoimage, `seo_effective` P2), partials from `extension/` through `Dir` and `PartialData` (P1: URL rule override and canonical-to-noindex callouts; P2: effective values per locale with source labels and the staging note); the rule form and settings use `seotext`, `seorobots` and `seoimage` where C7/C11 name text, robots and image fields; the P3 placeholders partial on General (`controllers/settings/_placeholders.htm`) and on each type tab (the `type__placeholders` field of the settings extension, `extension/_placeholders.htm`); `admin.go` adds `extension/_*.htm` to its embed patterns. 4. Full en/pl `lang.yaml` for every key of the UI-SPEC sections this plan renders (navigation for rules/history, settings tile, permission, SEO tab, P1, P2, SEO settings, URL rules, History). 5. Smoke: `TestSEOTab` in `seo_tab_test.go`: with the fake content plugin carrying a tabbed form and a tab-less form, the compiled schema lists the C3 fields in order inside the SEO tab placed last, the tab-less form has default then SEO, a malformed `seo_robots` value answers 422, and `seo_og_image` upload rejects a non-image. `npm --prefix ../sm-seo-plugin run typecheck && npm --prefix ../sm-seo-plugin run build` succeed (element unit tests are plan 32-06). README completed in the standard structure (overview, features, usage for content plugins: register a descriptor with Controller and Model, config keys `golem15.seo.site_url`, the resolve endpoint, permissions). - Commits in `../sm-seo-plugin`: elements + dist; field types + SEO tab; lang + README. + 6. Commits (resume units, see Execution units), staged by path in `../sm-seo-plugin`, in this order: + - A `feat: SEO field elements and bundle`: package.json, package-lock.json, tsconfig.json, vite.config.ts, scripts/check-dist.sh, assets/src/elements/index.ts, measure.ts, labels.ts, text.ts, robots.ts, serp.ts, image.ts, styles.css, assets/dist/seo.js, assets/dist/seo.css; + - B `feat: SEO field types and the complete SEO tab`: field_types.go, classes/field_types.go, classes/image.go, classes/snippet.go, extensions.go, extension/_seo_status.htm, extension/_seo_effective.htm, extension/_placeholders.htm, controllers/settings/_placeholders.htm, controllers/rules/_seo_status.htm, controllers/rules/_seo_effective.htm, models/rule/fields.yaml, models/settings/fields.yaml, admin.go, lang/en/lang.yaml, lang/pl/lang.yaml, seo_tab_test.go (TestSEOTab); + - C `docs: plugin README`: README.md. npm --prefix ../sm-seo-plugin run typecheck && bash ../sm-seo-plugin/scripts/check-dist.sh @@ -436,5 +458,5 @@ Any content type registered with a descriptor gets the SEO tab without importing - Tests `TestSEOTabTracer`, `TestSEOTab`, `TestSEOTabPermission`, `TestResolverPrecedence`, `TestSettingsTypeTabs`, `TestHistoryRecordsAdminWrite`, `TestEntryDeletedWithRecord`, `TestGenerateEntries` -Create `.planning/phases/32-seo-foundation/32-03-SUMMARY.md` when done, listing the sm-seo-plugin commit SHAs, the framework tag go.mod requires, the Task 4 answer and the cabana asset-dedup observation. +Create `.planning/phases/32-seo-foundation/32-03-SUMMARY.md` when done, listing the sm-seo-plugin commit SHAs per task and commit group, the framework tag go.mod requires, the Task 4 answer and the cabana asset-dedup observation. diff --git a/.planning/phases/32-seo-foundation/32-04-PLAN.md b/.planning/phases/32-seo-foundation/32-04-PLAN.md index ffd37fa2..1702cf44 100644 --- a/.planning/phases/32-seo-foundation/32-04-PLAN.md +++ b/.planning/phases/32-seo-foundation/32-04-PLAN.md @@ -9,8 +9,6 @@ files_modified: - .planning/research/STACK.md - ../sm-seo-plugin/go.mod - ../sm-seo-plugin/go.sum - - ../sm-seo-plugin/plugin.go - - ../sm-seo-plugin/admin.go - ../sm-seo-plugin/admin_navigation.go - ../sm-seo-plugin/extensions.go - ../sm-seo-plugin/listeners.go @@ -20,7 +18,6 @@ files_modified: - ../sm-seo-plugin/lang/en/lang.yaml - ../sm-seo-plugin/lang/pl/lang.yaml - ../sm-seo-plugin/README.md - - ../sm-seo-plugin/models/registry.go - ../sm-seo-plugin/models/redirect.go - ../sm-seo-plugin/models/redirect/fields.yaml - ../sm-seo-plugin/models/redirect/columns.yaml @@ -163,7 +160,7 @@ Extend `sm-seo-plugin` with D-16 to D-20c and D-22 to D-25: Purpose: SC-3 (changing a record's slug creates a 301 and flattens existing chains; loops are refused from the admin; MCP and import refusal is plan 32-05 on the same service). -Output: commits in `../sm-seo-plugin` (main, not pushed), the decision note and STACK row in this repository as a planning-docs commit. +Output: commits in `../sm-seo-plugin` (main, not pushed), the decision note and STACK row in this repository as a planning-docs commit. Which plan implements each decision of the phase, and what Phase 33 takes, is in `32-DECISIONS-MAP.md`. **Planner decisions recorded here:** - **P-18 (one service path for redirects):** every redirect write (admin form, address change, 404 actions, and later MCP/batch/CSV) goes through `classes.Service` methods `SaveRedirect`, `AddressChanged`, `DeleteRedirect`, which run the loop simulation and flattening inside the caller's transaction and write history. The Redirects form gets its loop error and chain notice from a field-less D-91 extension the plugin registers on its own `golem15.seo.redirects` controller (Save returns `pact.AdminFormErrors{"target": ...}` or a notice on `target`), because owner controllers have no notice hook. @@ -196,11 +193,26 @@ Route analog: ../sm-sitemap-plugin/routes.go 43-60 (r.Get("/sitemap.xml", handle +## Execution units and resume points + +Each task is a resumable unit, and Task 4 (robots.txt, AI bots, verification files and the decision note) is split into five commit groups, so the plan can run across several executor contexts without losing work: +- Every task ends with the commit groups listed below, in order: `git -c safe.directory='*' -C ../sm-seo-plugin add ` (paths only, never `-A` or `.`), then `commit -m ""`; G1 is a planning-docs commit in this repository. No task leaves uncommitted files for the next one, every commit keeps `go vet ./...` and `go test -short ./...` green, and each group adds the lang keys it uses to both lang files. TDD test commits may precede a group's commit. +- Resume check before each task: `git -c safe.directory='*' -C ../sm-seo-plugin log --format=%s | grep -Fx ''` for each plugin subject, and `git log --format=%s | grep -Fx ''` here for G1. All present: the task is done; re-run its `` and go on. Some present: continue at the first missing group. None: start the task. +- An executor whose context is getting heavy stops after a task's (or a Task 4 group's) last commit and returns its completed table (task, group, subject, SHA); the continuation starts at the next missing group. +- `admin.go` embeds admin YAML and partials by glob since plan 32-03 Task 2, so new screens need no edit there. If 32-03-SUMMARY.md records file-by-file embeds instead, add each new file to `admin.go` and stage it with its group. + +| Task | Commit groups, in order | Resume point after the task | +|------|-------------------------|-----------------------------| +| 1 | `feat: address-change 301 (tracer)` | TestSlugChange301 passes | +| 2 | A `feat: redirect engine with loop refusal and chain notices`; B `feat: redirects screen`; C `feat: 404 log and report endpoint`; D `feat: address-change redirects for writes outside the admin` | the engine, both screens and the listener work | +| 3 | `feat: URL normalisation, strict URLs and the one-hop redirect decision` | resolve is the redirect decision endpoint | +| 4 | G1 (this repository) `docs(32-04): grobotstxt decision note`; G2 `feat: robots.txt served file, AI bots and the non-production lock`; G3 `feat: robots.txt editor and tester screen`; G4 `feat: verification files API`; G5 `docs: robots.txt, AI bots and verification files in README` | plan complete; write 32-04-SUMMARY.md | + Task 1: End-to-end "an editor changes a record's slug and the old URL resolves to a 301 to the new one" — one path only - ../sm-seo-plugin/models/redirect.go, ../sm-seo-plugin/models/registry.go, ../sm-seo-plugin/updates/202610270006_create_golem15_seo_redirects.go, ../sm-seo-plugin/classes/redirects.go, ../sm-seo-plugin/classes/addresses.go, ../sm-seo-plugin/classes/resolver.go, ../sm-seo-plugin/extensions.go, ../sm-seo-plugin/controllers/api/resolve.go, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/redirects_test.go + ../sm-seo-plugin/models/redirect.go, ../sm-seo-plugin/updates/202610270006_create_golem15_seo_redirects.go, ../sm-seo-plugin/classes/redirects.go, ../sm-seo-plugin/classes/addresses.go, ../sm-seo-plugin/classes/resolver.go, ../sm-seo-plugin/extensions.go, ../sm-seo-plugin/controllers/api/resolve.go, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/redirects_test.go - ../sm-seo-plugin/extensions.go, classes/service.go, classes/resolver.go, classes/entries.go (plan 32-03) - ../sm-seo-plugin/fixture_test.go and seo_tab_test.go (fake content plugin and admin harness) @@ -208,7 +220,7 @@ Route analog: ../sm-sitemap-plugin/routes.go 43-60 (r.Get("/sitemap.xml", handle - .planning/phases/32-seo-foundation/32-RESEARCH.md (Pattern 6, Pitfall 11) - Per D-18 and P-20: add the redirects table (id, source, match exact|pattern|regex, target, status 301|302|410, query ignore|pass, ignore_case bool, trailing_slash bool default true, origin manual|slug|import|mcp|notfound, note, timestamps; unique (source, match)); `classes/redirects.go` exact-source lookup only for the tracer; `classes/addresses.go` `(*Service).AddressChanged(ctx, tx, change AddressChange) (AddressResult, error)` implementing P-20; in the SEO tab extension's Save compare `Describe` URLs (read through the transaction) with the stored entry URLs and call AddressChanged per changed locale, returning the `redirects.auto_created` notice (P-20 field choice); the resolver answers `{"action":"redirect","status":301,"location":}` for an exact redirect source. `TestSlugChange301` in `redirects_test.go`: create a published fake page /pages/a with an entry, PUT slug b through the admin API (response Meta.Notices contains the auto_created notice), then resolve /pages/a → redirect 301 to the absolute /pages/b, resolve /pages/b → ok; an earlier redirect /pages/x → /pages/a now points to /pages/b. Commit `feat: address-change 301 (tracer)` in ../sm-seo-plugin. + Per D-18 and P-20: add the redirects table (id, source, match exact|pattern|regex, target, status 301|302|410, query ignore|pass, ignore_case bool, trailing_slash bool default true, origin manual|slug|import|mcp|notfound, note, timestamps; unique (source, match); the model and the migration register themselves through `models.Register` and `updates.Register` in their own files); `classes/redirects.go` exact-source lookup only for the tracer; `classes/addresses.go` `(*Service).AddressChanged(ctx, tx, change AddressChange) (AddressResult, error)` implementing P-20; in the SEO tab extension's Save compare `Describe` URLs (read through the transaction) with the stored entry URLs and call AddressChanged per changed locale, returning the `redirects.auto_created` notice (P-20 field choice); the resolver answers `{"action":"redirect","status":301,"location":}` for an exact redirect source. `TestSlugChange301` in `redirects_test.go`: create a published fake page /pages/a with an entry, PUT slug b through the admin API (response Meta.Notices contains the auto_created notice), then resolve /pages/a → redirect 301 to the absolute /pages/b, resolve /pages/b → ok; an earlier redirect /pages/x → /pages/a now points to /pages/b. Commit (resume unit, see Execution units): stage exactly this task's `` paths in ../sm-seo-plugin and commit `feat: address-change 301 (tracer)`. go -C ../sm-seo-plugin test ./... -run 'TestSlugChange301' -count=1 -v 2>&1 | tail -20 @@ -246,7 +258,12 @@ Route analog: ../sm-sitemap-plugin/routes.go 43-60 (r.Get("/sitemap.xml", handle - Journal-style write outside cabana (RecordSaved with a slug change and Controller "") creates the 301 after commit; RecordSaved without a URL change creates nothing; an unpublished record's change creates nothing - Implement D-16, D-17, D-19, D-20, D-18 for non-admin writers, per P-18, P-19, P-22 and the behaviour list. `classes/redirects.go`: compiled rule set (exact map, wildcard rules ordered by longest literal prefix then id, regex rules by id, compiled once per change with stdlib regexp; wildcard via `regexp.QuoteMeta` with `*` → `(.*)`); `classes/redirect_graph.go`: simulation, loop and chain detection, flattening. Redirects screen `golem15.seo.redirects` per C8 (list columns, filters, row state `negative` labelled `redirects.state_chain` when the target is another rule's source; form layout) and the field-less self-extension of P-18. 404 log `golem15.seo.notfound` per C9 (list by hits desc, preview with `_hint.htm`, prompted `redirect` and danger `gone` record actions, bulk `gone`, row state `disabled` labelled Resolved computed through the rule set). `controllers/api/notfound.go` + route `POST /_seo/api/v1/notfound` (throttle bucket `seo-notfound`) per P-22, config keys `golem15.seo.notfound_token` (default empty) and `golem15.seo.notfound_max_rows` (default 10000) in `config/config.yaml`. `listeners.go`: `*pact.RecordSaved` for a model with a content type re-describes the record after commit in its own transaction and calls AddressChanged per changed locale (no-op when the admin Save already updated the entry). Navigation per UI-SPEC C2: item `seo` lands on `golem15.seo.redirects`, side menu Redirects, 404 log, URL rules, robots.txt, History, Import and export (the last two screens' controllers come in Task 4 and plan 32-05; add each side-menu entry when its controller exists). Write the tests first (`TestRedirectEngine`, `TestLoopRefused` admin path, `TestNotFoundLog`, `TestAddressChangeFromEvent` in `redirects_test.go`). Commits in ../sm-seo-plugin by logical change. + Implement D-16, D-17, D-19, D-20, D-18 for non-admin writers, per P-18, P-19, P-22 and the behaviour list. `classes/redirects.go`: compiled rule set (exact map, wildcard rules ordered by longest literal prefix then id, regex rules by id, compiled once per change with stdlib regexp; wildcard via `regexp.QuoteMeta` with `*` → `(.*)`); `classes/redirect_graph.go`: simulation, loop and chain detection, flattening. Redirects screen `golem15.seo.redirects` per C8 (list columns, filters, row state `negative` labelled `redirects.state_chain` when the target is another rule's source; form layout) and the field-less self-extension of P-18. 404 log `golem15.seo.notfound` per C9 (list by hits desc, preview with `_hint.htm`, prompted `redirect` and danger `gone` record actions, bulk `gone`, row state `disabled` labelled Resolved computed through the rule set). `controllers/api/notfound.go` + route `POST /_seo/api/v1/notfound` (throttle bucket `seo-notfound`) per P-22, config keys `golem15.seo.notfound_token` (default empty) and `golem15.seo.notfound_max_rows` (default 10000) in `config/config.yaml`. `listeners.go`: `*pact.RecordSaved` for a model with a content type re-describes the record after commit in its own transaction and calls AddressChanged per changed locale (no-op when the admin Save already updated the entry). Navigation per UI-SPEC C2: item `seo` lands on `golem15.seo.redirects`, side menu Redirects, 404 log, URL rules, robots.txt, History, Import and export (robots.txt comes in Task 4 and Import and export in plan 32-05; add each side-menu entry when its controller exists). Write the tests first (`TestRedirectEngine`, `TestLoopRefused` admin path, `TestNotFoundLog`, `TestAddressChangeFromEvent` in `redirects_test.go`). + Commits (resume units, see Execution units), staged by path in ../sm-seo-plugin, in this order: + - A `feat: redirect engine with loop refusal and chain notices`: classes/redirects.go, classes/redirect_graph.go, classes/addresses.go, models/redirect.go, redirects_test.go (TestRedirectEngine); + - B `feat: redirects screen`: models/redirect/fields.yaml, models/redirect/columns.yaml, controllers/redirects.go, controllers/redirects/config_list.yaml, controllers/redirects/config_form.yaml, controllers/admin_registry.go, extensions.go, admin_navigation.go, lang files, redirects_test.go (TestLoopRefused admin path); + - C `feat: 404 log and report endpoint`: models/notfound.go, models/notfound/fields.yaml, models/notfound/columns.yaml, the notfound migration, classes/notfound.go, controllers/notfound.go, controllers/notfound/config_list.yaml, controllers/notfound/config_form.yaml, controllers/notfound/_hint.htm, controllers/api/notfound.go, controllers/admin_registry.go, admin_navigation.go, routes.go, config/config.yaml, lang files, redirects_test.go (TestNotFoundLog); + - D `feat: address-change redirects for writes outside the admin`: listeners.go, redirects_test.go (TestAddressChangeFromEvent). go -C ../sm-seo-plugin test ./... -run 'TestRedirectEngine|TestLoopRefused|TestNotFoundLog|TestAddressChangeFromEvent|TestSlugChange301' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -40 @@ -283,7 +300,7 @@ Route analog: ../sm-sitemap-plugin/routes.go 43-60 (r.Get("/sitemap.xml", handle - P1 shows status.canonical_redirected when the record's canonical URL is a redirect source - Implement D-20a, D-20b, D-20c and D-103 per P-21 and the behaviour list: `classes/normalize.go` pure `Normalize(u *url.URL, opts NormalizeOptions) (*url.URL, bool)`; `classes/strict.go` through signpost `MatchPath` and `Describe`; the resolver pipeline order of P-21 with the runtime hop cap; the resolve answer gains `redirect` and `gone` actions (status, location absolute on the site URL, never from the request Host); P1 gains the canonical-redirected callout (D-06). Tests first: `TestNormalize`, `TestStrictURL`, `TestOneHop` in `addresses_test.go`. Commit in ../sm-seo-plugin. + Implement D-20a, D-20b, D-20c and D-103 per P-21 and the behaviour list: `classes/normalize.go` pure `Normalize(u *url.URL, opts NormalizeOptions) (*url.URL, bool)`; `classes/strict.go` through signpost `MatchPath` and `Describe`; the resolver pipeline order of P-21 with the runtime hop cap; the resolve answer gains `redirect` and `gone` actions (status, location absolute on the site URL, never from the request Host); P1 gains the canonical-redirected callout (D-06). Tests first: `TestNormalize`, `TestStrictURL`, `TestOneHop` in `addresses_test.go`. Commit (resume unit, see Execution units): stage exactly this task's `` paths in ../sm-seo-plugin and commit `feat: URL normalisation, strict URLs and the one-hop redirect decision`. go -C ../sm-seo-plugin test ./... -run 'TestNormalize|TestStrictURL|TestOneHop' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -30 @@ -303,7 +320,7 @@ Route analog: ../sm-sitemap-plugin/routes.go 43-60 (r.Get("/sitemap.xml", handle Task 4: robots.txt editor, tester, AI bots, GET /robots.txt with the non-production lock, verification files, and the grobotstxt decision note Plan 32-03 Task 4 answered "approved" for github.com/jimsmart/grobotstxt v1.0.3 (recorded in 32-03-SUMMARY.md) - .planning/notes/2026-10-27-robots-txt-grobotstxt.md, .planning/research/STACK.md, ../sm-seo-plugin/go.mod, ../sm-seo-plugin/go.sum, ../sm-seo-plugin/models/robots.go, ../sm-seo-plugin/models/robots/fields.yaml, ../sm-seo-plugin/updates/202610270008_create_golem15_seo_robots.go, ../sm-seo-plugin/controllers/robots.go, ../sm-seo-plugin/controllers/robots/config_form.yaml, ../sm-seo-plugin/controllers/robots/_staging.htm, ../sm-seo-plugin/controllers/robots/_robots_served.htm, ../sm-seo-plugin/controllers/api/robots.go, ../sm-seo-plugin/controllers/api/verification.go, ../sm-seo-plugin/classes/robots.go, ../sm-seo-plugin/classes/bots.go, ../sm-seo-plugin/classes/verification.go, ../sm-seo-plugin/classes/field_types.go, ../sm-seo-plugin/field_types.go, ../sm-seo-plugin/routes.go, ../sm-seo-plugin/plugin.go, ../sm-seo-plugin/admin.go, ../sm-seo-plugin/admin_navigation.go, ../sm-seo-plugin/config/config.yaml, ../sm-seo-plugin/assets/src/elements/index.ts, ../sm-seo-plugin/assets/src/elements/labels.ts, ../sm-seo-plugin/assets/src/elements/robotstxt.ts, ../sm-seo-plugin/assets/src/elements/robotstester.ts, ../sm-seo-plugin/assets/src/elements/styles.css, ../sm-seo-plugin/assets/dist/seo.js, ../sm-seo-plugin/assets/dist/seo.css, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/README.md, ../sm-seo-plugin/robots_test.go + .planning/notes/2026-10-27-robots-txt-grobotstxt.md, .planning/research/STACK.md, ../sm-seo-plugin/go.mod, ../sm-seo-plugin/go.sum, ../sm-seo-plugin/models/robots.go, ../sm-seo-plugin/models/robots/fields.yaml, ../sm-seo-plugin/updates/202610270008_create_golem15_seo_robots.go, ../sm-seo-plugin/controllers/robots.go, ../sm-seo-plugin/controllers/robots/config_form.yaml, ../sm-seo-plugin/controllers/robots/_staging.htm, ../sm-seo-plugin/controllers/robots/_robots_served.htm, ../sm-seo-plugin/controllers/api/robots.go, ../sm-seo-plugin/controllers/api/verification.go, ../sm-seo-plugin/classes/robots.go, ../sm-seo-plugin/classes/bots.go, ../sm-seo-plugin/classes/verification.go, ../sm-seo-plugin/classes/field_types.go, ../sm-seo-plugin/field_types.go, ../sm-seo-plugin/routes.go, ../sm-seo-plugin/controllers/admin_registry.go, ../sm-seo-plugin/admin_navigation.go, ../sm-seo-plugin/config/config.yaml, ../sm-seo-plugin/assets/src/elements/index.ts, ../sm-seo-plugin/assets/src/elements/labels.ts, ../sm-seo-plugin/assets/src/elements/robotstxt.ts, ../sm-seo-plugin/assets/src/elements/robotstester.ts, ../sm-seo-plugin/assets/src/elements/styles.css, ../sm-seo-plugin/assets/dist/seo.js, ../sm-seo-plugin/assets/dist/seo.css, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/README.md, ../sm-seo-plugin/robots_test.go - .planning/notes/2026-10-20-sociallogin-goth.md (decision note format) and .planning/research/STACK.md (Supporting Libraries table, the goth row) - ../sm-sitemap-plugin/routes.go (root route with throttle bucket) @@ -313,14 +330,18 @@ Route analog: ../sm-sitemap-plugin/routes.go 43-60 (r.Get("/sitemap.xml", handle Per D-22, D-24, D-25, D-99, D-105, D-05, D-103, D-08 and P-23: - 1. Planning docs (framework repo, separate commit `docs(32-04): grobotstxt decision note`): `.planning/notes/2026-10-27-robots-txt-grobotstxt.md` (use the execution date in the file name if it differs) naming `github.com/jimsmart/grobotstxt` v1.0.3, Apache-2.0, why (Google's parser semantics: longest match, allow wins ties, `*`/`$`, `MatchingLine()`), the alternatives (temoto/robotstxt, hand-rolled RFC 9309), that it is plugin-only (the framework go.mod does not require it) and D-99; one row in `.planning/research/STACK.md` Supporting Libraries in the goth row's format. + 1. Planning docs (G1, framework repo, separate commit `docs(32-04): grobotstxt decision note` staging only the note and `.planning/research/STACK.md`): `.planning/notes/2026-10-27-robots-txt-grobotstxt.md` (use the execution date in the file name if it differs) naming `github.com/jimsmart/grobotstxt` v1.0.3, Apache-2.0, why (Google's parser semantics: longest match, allow wins ties, `*`/`$`, `MatchingLine()`), the alternatives (temoto/robotstxt, hand-rolled RFC 9309), that it is plugin-only (the framework go.mod does not require it) and D-99; one row in `.planning/research/STACK.md` Supporting Libraries in the goth row's format. 2. `go get github.com/jimsmart/grobotstxt@v1.0.3` in ../sm-seo-plugin. - 3. robots singleton table (id 1, content text, bots jsonb) and `golem15.seo.robots` (`AdminSingleton`) per C10: tab robots.txt (staging partial, `content` of field type `robotstxt` with Decode refusing more than 500 KB with `robots.too_large`, `tester` widget E5 filled from content and every bot switch, `served` P4), tab AI bots with one switch per bot from `classes/bots.go` in the C10 order plus `Claude-User` as a user-fetch bot operated by Anthropic (D-105) placed after ChatGPT-User; comments `bots.search_comment`, `bots.user_comment`, `bots.training_comment` with `:operator`. + 3. robots singleton table (id 1, content text, bots jsonb) and `golem15.seo.robots` (`AdminSingleton`, registered in `controllers/admin_registry.go`, with its robots.txt side-menu entry in `admin_navigation.go`) per C10: tab robots.txt (staging partial, `content` of field type `robotstxt` with Decode refusing more than 500 KB with `robots.too_large`, `tester` widget E5 filled from content and every bot switch, `served` P4), tab AI bots with one switch per bot from `classes/bots.go` in the C10 order plus `Claude-User` as a user-fetch bot operated by Anthropic (D-105) placed after ChatGPT-User; comments `bots.search_comment`, `bots.user_comment`, `bots.training_comment` with `:operator`. 4. `classes/robots.go`: `ServedFile(ctx, production bool)` composes the stored rules, a `User-agent: ` + `Disallow: /` group per switched-off bot, and the P-23 Sitemap lines; non-production answers `User-agent: *` / `Disallow: /`. The tester action `robots_test` composes the production file from the unsaved content and switches, runs grobotstxt `NewRobotsMatcher().AgentAllowed` and `MatchingLine`, and answers allowed/blocked, the line number and text, and whether a bot switch blocked it. On save, a notice `robots.blocks_noindex` lists noindex URLs (rules with noindex and entries whose robots contain noindex) that Googlebot is blocked from (D-24). 5. Elements E4 `golem15-seo-robotstxt` (live lint per the E4 copy: unknown directive with suggestion, rule before any User-agent, bad path, missing colon, own Sitemap line, Crawl-delay; size meter; 10 findings max) and E5 `golem15-seo-robotstester` (input row, crawler select listing Googlebot, Googlebot-Image, Bingbot, every bot token and `*`, result line, staging and unsaved notes); rebuild and commit `assets/dist`. 6. Routes: `GET /robots.txt` (throttle bucket `seo-robots`, no backend middleware) serving text/plain; on any error it serves the last good body cached in memory, else `User-agent: *` / `Disallow:` with status 200, and logs; outside production it adds `X-Robots-Tag: noindex, nofollow`. `GET /_seo/api/v1/verification/{name}` (name pattern `[A-Za-z0-9._-]+\.(html|txt|xml)`) serves the stored file body with a matching content type, 404 otherwise. Every `/_seo/api/v1` response carries `X-Robots-Tag: noindex`. No route is registered at `/{name}` (RESEARCH anti-pattern). 7. Tests `TestRobotsServed` (stored + bots + Sitemap; staging lock; DB failure still 200), `TestRobotsTester` (Google documented examples: longest match, allow wins ties, `$` anchor; a switched-off bot reported as blocked by its switch), `TestVerificationFile` in `robots_test.go`. README sections for robots.txt, AI bots, verification files and the new config keys. - Plugin commits in ../sm-seo-plugin by logical change. + 8. Commits (resume units, see Execution units), staged by path in ../sm-seo-plugin after G1, in this order; a resumed run skips every group whose subject exists: + - G2 `feat: robots.txt served file, AI bots and the non-production lock` (steps 2, 4 and the robots.txt route of step 6): go.mod, go.sum, models/robots.go, the robots migration, classes/robots.go, classes/bots.go, controllers/api/robots.go, routes.go, config/config.yaml, robots_test.go (TestRobotsServed, TestRobotsTester); + - G3 `feat: robots.txt editor and tester screen` (steps 3 and 5): models/robots/fields.yaml, controllers/robots.go, controllers/robots/config_form.yaml, controllers/robots/_staging.htm, controllers/robots/_robots_served.htm, controllers/admin_registry.go, admin_navigation.go, field_types.go, classes/field_types.go, assets/src/elements/index.ts, labels.ts, robotstxt.ts, robotstester.ts, styles.css, assets/dist/seo.js, assets/dist/seo.css, lang/en/lang.yaml, lang/pl/lang.yaml; + - G4 `feat: verification files API` (the verification route of step 6): classes/verification.go, controllers/api/verification.go, routes.go, robots_test.go (TestVerificationFile); + - G5 `docs: robots.txt, AI bots and verification files in README` (step 7's README sections): README.md. go -C ../sm-seo-plugin test ./... -run 'TestRobotsServed|TestRobotsTester|TestVerificationFile' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -30 @@ -389,5 +410,5 @@ Changing a record's slug creates a 301 and flattens chains, from the admin and f - Tests `TestSlugChange301`, `TestRedirectEngine`, `TestLoopRefused`, `TestNotFoundLog`, `TestAddressChangeFromEvent`, `TestNormalize`, `TestStrictURL`, `TestOneHop`, `TestRobotsServed`, `TestRobotsTester`, `TestVerificationFile` -Create `.planning/phases/32-seo-foundation/32-04-SUMMARY.md` when done with every commit SHA in both repositories. +Create `.planning/phases/32-seo-foundation/32-04-SUMMARY.md` when done with every commit SHA in both repositories, per task and commit group. diff --git a/.planning/phases/32-seo-foundation/32-05-PLAN.md b/.planning/phases/32-seo-foundation/32-05-PLAN.md index 05240c5c..fafbd759 100644 --- a/.planning/phases/32-seo-foundation/32-05-PLAN.md +++ b/.planning/phases/32-seo-foundation/32-05-PLAN.md @@ -137,7 +137,7 @@ Add the `/_seo/api/v1` management API and pier MCP tools for every SEO admin ope Purpose: SC-4 (every SEO operation in the admin is available as an MCP tool; a batch change can be previewed with dry-run and undone from history) and the MCP/import half of SC-3. The crawler half of D-14 is Phase 33. -Output: commits in `../sm-seo-plugin` (main, not pushed), the demo test files committed in `../sm-demo-app` (they skip when the plugin is not mounted), the demo mount patch in this phase directory, and `scripts/check-phase32.sh` in this repository. +Output: commits in `../sm-seo-plugin` (main, not pushed), the demo test files committed in `../sm-demo-app` (they skip when the plugin is not mounted), the demo mount patch in this phase directory, and `scripts/check-phase32.sh` in this repository. Which plan implements each decision of the phase, and what Phase 33 takes, is in `32-DECISIONS-MAP.md`. **Planner decisions recorded here:** - **P-24 (routes and tools):** management routes in a `/_seo/api/v1` group with throttle bucket `seo-api`, each handler resolving the backend principal (Journal's `requireBackendPrincipal` pattern through `cabana.ResolveBackend`) and requiring `golem15.seo.manage` with `cabana.Allows`. Tools (pier prefixes them with `seo_`), all `Guard: "backend"`, `Permissions: ["golem15.seo.manage"]`: GET `get_url_seo` (/seo?url=), `list_rules`, `show_rule`, `list_redirects`, `show_redirect`, `list_not_found`, `show_robots`, `test_robots` (/robots/test?url=&agent=), `show_settings`, `list_history`, `show_history`, `export_csv` (/export?dataset=&delimiter=&page=), `list_issues`; writes `set_url_seo` (PUT /seo), `create_rule`, `update_rule`, `delete_rule` (destructive), `create_redirect`, `update_redirect`, `delete_redirect` (destructive), `redirect_not_found`, `gone_not_found` (destructive), `delete_not_found` (destructive), `update_robots`, `update_settings`, `undo_change`, `undo_batch`, `import_csv` (with dry_run), `apply_batch` (with dry_run). `operations.go` exports `AdminOperations` mapping each admin operation (form save, list delete, record action, bulk action, widget action, singleton save) to its tool name; `TestEveryAdminOperationHasTool` fails on a missing or unbound one. History `via = mcp` comes from `bouncer.Dispatched(ctx, "backend")`. diff --git a/.planning/phases/32-seo-foundation/32-06-PLAN.md b/.planning/phases/32-seo-foundation/32-06-PLAN.md index b9eb721a..82db4075 100644 --- a/.planning/phases/32-seo-foundation/32-06-PLAN.md +++ b/.planning/phases/32-seo-foundation/32-06-PLAN.md @@ -6,6 +6,7 @@ wave: 6 depends_on: ["32-05"] files_modified: - scripts/check-phase32.sh + - scripts/apisurface/main.go - modules/cabana/form_extensions_test.go - modules/cabana/record_events_test.go - modules/signpost/content_test.go @@ -44,8 +45,9 @@ files_modified: - ../sm-demo-app/plugins/golem15/journal - .planning/phases/32-seo-foundation/32-VALIDATION.md - .planning/phases/32-seo-foundation/deferred-items.md + - .planning/phases/32-seo-foundation/32-06-release-log.md autonomous: false -requirements: [SC-5, SC-1, SC-2, SC-3, SC-4, Q-01, D-90, D-91, D-92, D-93, D-98, D-22, D-25, D-09] +requirements: [SC-5, SC-1, SC-2, SC-3, SC-4, Q-01, D-106, D-90, D-91, D-92, D-93, D-98, D-22, D-25, D-09] estimate: tokens: 160000 @@ -55,12 +57,13 @@ estimate: must_haves: truths: - - "The framework change of Phase 32 is strictly additive: every exported declaration of pact, cabana and signpost at the latest release tag exists unchanged at the release commit, and existing plugins compile and behave the same without edits (Jakub, 2026-10-26)" - - "Existing consumers build and pass their short tests against the new framework with no change on their side: sm-sitemap-plugin at HEAD, sm-journal-plugin at its commit before Phase 32, and sm-demo-app at its commit before Phase 32 with its pinned submodules" + - "The framework change of Phase 32 is strictly additive (D-106): the exported API surface of pact, cabana, signpost and every other framework package changed since the latest release tag keeps every element of that tag unchanged (no removed declaration, field or method; no changed signature, field type, struct tag, constant value or underlying type) and adds no method to an interface that exists at that tag; a new capability method goes on a new, separate optional interface; existing plugins compile and behave the same without edits" + - "Existing consumers build and pass their short tests against the new framework with no change on their side: sm-sitemap-plugin at HEAD, sm-journal-plugin at its commit before Phase 32, and sm-demo-app at its commit before Phase 32 with its pinned submodules; every other sibling plugin and application that requires the framework builds at its HEAD against it (D-106)" - "The new code has unit tests in this last plan: cabana placement and boot errors, record events, signpost descriptors, the SEO plugin's resolver, redirect graph, normalisation, robots grammar and tester, templates, CSV, owner resolution and issues, Journal descriptors and the element backstops (SC-5)" - - "The framework release is the next v0.5.x patch tag, computed from git tag at release time and confirmed by Jakub at the checkpoint; no version number is fixed in advance (SC-5, Q-01)" + - "The framework release is the next v0.5.x patch tag, computed from git tag at release time and confirmed by Jakub at the checkpoint; no version number is fixed in advance (SC-5, Q-01, D-106)" - "sm-seo-plugin and sm-journal-plugin require that framework tag, are tagged and pushed only after Jakub's go-ahead, and sm-demo-app mounts golem15.seo and bumps Journal in separate commits that pass TestGoModRequiresTags, TestSEOTabMounted and TestSEOToolsCatalog (SC-2, SC-4, SC-5)" - - "scripts/check-phase32.sh --all is green with the demo mounted, including the additive and compat stages" + - "scripts/check-phase32.sh --all is green with the demo mounted, including the additive and compat stages, and its self-test proves that the additive stage refuses a removed declaration, a changed signature and a method added to an existing interface" + - "A resumed release run never tags, pushes or posts twice: every release step is recorded in 32-06-release-log.md as it completes" - "The release tag gets a summercms.io changelog post the same day, written from changelog-post.py source output without planning IDs" - statement: "E1 counter line at 320px container width: status text and count wrap onto two lines without overlap (UI-SPEC long-text backstop)" verification: backstop @@ -79,7 +82,7 @@ must_haves: - statement: "E3 keyphrase stem matching: grzyby/grzybów match, grzyb/grzbiet do not, diacritics and case ignored, URL hyphens equal spaces (UI-SPEC populated backstop, Q-4)" verification: backstop prohibitions: - - statement: "No breaking change to existing framework API: no exported pact, cabana or signpost declaration of the latest tag is removed or changed, and no existing consumer needs an edit to build against the release" + - statement: "No breaking change to existing framework API (D-106): no exported declaration of the latest tag is removed or changed, no method is added to an interface that exists at that tag, and no existing consumer needs an edit to build against the release" status: resolved verification: test - statement: "Nothing is pushed, tagged, deployed or posted to the changelog without Jakub's go-ahead at the checkpoint, and no version number is hard-coded before that checkpoint computes it" @@ -92,6 +95,9 @@ must_haves: - path: "scripts/check-phase32.sh" provides: "additive and compat stages added to the phase gate" contains: "compat" + - path: "scripts/apisurface/main.go" + provides: "standard-library API surface printer the additive stage compares: signatures without parameter names, interface method sets, struct fields" + contains: "imethod" - path: "modules/cabana/form_extensions_test.go" provides: "the D-91 unit matrix" contains: "func TestFormExtension" @@ -105,6 +111,10 @@ must_haves: provides: "per-task verification map, nyquist_compliant true" contains: "nyquist_compliant: true" key_links: + - from: "scripts/check-phase32.sh" + to: "scripts/apisurface/main.go" + via: "additive stage runs go run ./scripts/apisurface on the last tag's worktree and on HEAD per package and refuses removed or changed lines and new methods on interfaces that exist at the tag" + pattern: "apisurface" - from: "scripts/check-phase32.sh" to: "../sm-journal-plugin" via: "compat stage builds a detached worktree of Journal's pre-phase commit against this framework checkout through a scratch go.work" @@ -124,12 +134,14 @@ must_haves: This plan proves the phase with unit tests, proves the framework change is additive, and releases it. -Write the phase's unit tests (project rule: unit tests are the last plan) per the 32-VALIDATION.md map, add the `additive` and `compat` stages to `scripts/check-phase32.sh` (Jakub, 2026-10-26: the framework release is a patch and must be strictly additive), then, after Jakub's go-ahead, tag the framework with the next v0.5.x patch computed at release time (Q-01: Phase 33 builds on this tag, not on a `replace`), release sm-seo-plugin and sm-journal-plugin on that tag, mount the plugin in sm-demo-app with the plan 32-05 patch, and post the changelog. The framework names no SEO concept and no consuming application (D-90); the seams D-91, D-92 and D-93 ship as published API (D-98 plan order). +Write the phase's unit tests (project rule: unit tests are the last plan) per the 32-VALIDATION.md map, add the `additive` and `compat` stages to `scripts/check-phase32.sh` (D-106: the framework release is a patch and must be strictly additive), then, after Jakub's go-ahead, tag the framework with the next v0.5.x patch computed at release time (D-106) (Q-01: Phase 33 builds on this tag, not on a `replace`), release sm-seo-plugin and sm-journal-plugin on that tag, mount the plugin in sm-demo-app with the plan 32-05 patch, and post the changelog. The framework names no SEO concept and no consuming application (D-90); the seams D-91, D-92 and D-93 ship as published API (D-98 plan order). Purpose: SC-5 (the framework release with the seams is tagged, and the new code has unit tests delivered in the phase's last plan), and the final evidence for SC-1 to SC-4. Output: test files in three repositories, the gate stages, the updated VALIDATION.md, tags and pushes (after the go-ahead), demo commits, the changelog post. +**Decision coverage.** `32-DECISIONS-MAP.md` maps every decision of CONTEXT.md (D-01 to D-106, D-20a to D-20c, Q-01) to the plan and task that implements it, or to Phase 33; the boundary below is its Phase 33 column, and this plan's verification checks that the map lists every decision. + **Phase 33 boundary (Q-01).** These decisions are not implemented in Phase 32 and are explicitly deferred to Phase 33 ("SEO output, audit and content"); Phase 32 leaves room for them as noted: - D-30, D-31, D-32, D-34 (sitemap rewrite, image/News/IndexNow, HTML sitemap) and D-95 (the sitemap filter interface sm-seo-plugin will implement, with `Requires()` on Golem15.Sitemap); D-24's sitemap consistency (noindex, canonicalised and redirected pages leave the sitemaps; the "sitemap URL blocked" warning). Room: signpost content descriptors list records for sitemaps; the plugin's batch dry-run already flags noindex on published records' URLs. The reviewed todo `sitemap-xslt-browser-deprecation.md` stays with that rewrite. - D-33 (page registry; it registers its types through the same D-93 descriptors). @@ -152,6 +164,7 @@ Output: test files in three repositories, the gate stages, the updated VALIDATIO @.planning/phases/32-seo-foundation/32-CONTEXT.md @.planning/phases/32-seo-foundation/32-VALIDATION.md @.planning/phases/32-seo-foundation/32-UI-SPEC.md +@.planning/phases/32-seo-foundation/32-DECISIONS-MAP.md @.planning/phases/32-seo-foundation/32-01-SUMMARY.md @.planning/phases/32-seo-foundation/32-02-SUMMARY.md @.planning/phases/32-seo-foundation/32-05-SUMMARY.md @@ -159,39 +172,66 @@ Output: test files in three repositories, the gate stages, the updated VALIDATIO @../CLAUDE.md -Release facts: framework tags are vX.Y.Z on master (`git tag --sort=-v:refname`); other sessions may tag before this plan runs, so the version is computed at Task 4, never earlier. sm-journal-plugin is on branch main; sm-seo-plugin on main with an empty origin until its first push. Demo rules: go_mod_test.go TestGoModRequiresTags (no replace, every golem15 requirement a canonical tag, every summer.yaml plugin required); scripts/check-pins.sh (submodule commits equal the required versions). Changelog: `../sm-summercmsio-app/scripts/changelog-post.py source vX.Y.Z` prints the tag message and commits since the previous tag; `publish vX.Y.Z notes.md --excerpt "..."` posts; the token is read from ~/.config/summercms/journal-token. +Release facts: framework tags are vX.Y.Z on master (`git tag --sort=-v:refname`); other sessions may tag before this plan runs, so the version is computed at Task 4, never earlier (D-106: a patch tag, only the third number moves). sm-journal-plugin is on branch main; sm-seo-plugin on main with an empty origin until its first push. Demo rules: go_mod_test.go TestGoModRequiresTags (no replace, every golem15 requirement a canonical tag, every summer.yaml plugin required); scripts/check-pins.sh (submodule commits equal the required versions). Changelog: `../sm-summercmsio-app/scripts/changelog-post.py source vX.Y.Z` prints the tag message and commits since the previous tag; `publish vX.Y.Z notes.md --excerpt "..."` posts; the token is read from ~/.config/summercms/journal-token. Pre-phase commits: the framework base SHA and the Journal SHA before plan 32-02's first commit are recorded in 32-01-SUMMARY.md and 32-02-SUMMARY.md; the demo SHA before plan 32-05's test commit is in 32-05-SUMMARY.md. +## Execution units and resume points + +Five tasks across three repositories plus a one-way release, so each task is a resumable unit and the plan runs in at least two executor contexts: +- Every task ends with the commits listed below, staged by path (`git add `, with `-C ` and `-c safe.directory='*'` for sibling repositories; never `-A` or `.`). No task leaves uncommitted files for the next one. Defect fixes go in their own `fix(32-06): ...` (framework) or `fix: ...` (plugins) commits before the group's commit. +- Resume check before each task: `git -C log --format=%s | grep -Fx ''` for each subject of the task. All present: the task is done; re-run its `` and go on. Some present: continue at the first missing group. None: start the task. +- An executor whose context is getting heavy stops after a task's last commit and returns its completed-task table (task, repository, subject, SHA); the continuation starts at the next task. The Task 4 checkpoint is always such a boundary: Tasks 1 to 3 run in one context and Task 5 in a fresh one. +- Task 5 is resumed from `.planning/phases/32-seo-foundation/32-06-release-log.md`, never from memory (see its step 0). + +| Task | Commit groups, in order (repository) | Resume point after the task | +|------|--------------------------------------|-----------------------------| +| 1 | `test(32-06): additive and compat stages in check-phase32` (this repository) | `--self-test`, `--additive` and `--compat` green | +| 2 | `test(32-06): framework unit matrices for form extensions, record events and descriptors` (this repository) | framework seams fully unit-tested | +| 3 | A `test: unit tests for the SEO classes` (../sm-seo-plugin); B `test: element backstops` (../sm-seo-plugin); C `test: content descriptors, journal-tag type and translated slugs` (../sm-journal-plugin); D `docs(32-06): complete the validation map` (this repository) | all unit tests in; VALIDATION.md complete | +| 4 | none (checkpoint); the continuation prompt carries the answer and the confirmed versions | release, release-no-apps or hold | +| 5 | one line per step in 32-06-release-log.md, then the demo commits named in step 4 | release done or hold recorded; write 32-06-SUMMARY.md | + Task 1: End-to-end compatibility proof: the additive and compat gate stages build existing consumers unchanged against the new framework — one path through every repository - scripts/check-phase32.sh + scripts/check-phase32.sh, scripts/apisurface/main.go - scripts/check-phase32.sh (plan 32-05) - - scripts/check-phase19.7.sh (the additive check and the scratch go.work technique for consumers) + - scripts/check-phase19.7.sh (its additive check and self-test layout, and the scratch go.work technique for consumers; its line-based check misses methods added to existing interfaces and parameters added to multi-line signatures, which is why this plan compares API surfaces) - ../sm-journal-plugin/go.work and ../sm-demo-app/go.work (replace lines to mirror in scratch go.work files) - .planning/phases/32-seo-foundation/32-01-SUMMARY.md, 32-02-SUMMARY.md, 32-05-SUMMARY.md (pre-phase SHAs) - Per the strictly-additive rule (Jakub, 2026-10-26), D-91, D-92, D-93: - 1. Stage `additive`: `LAST_TAG=$(git tag --sort=-v:refname | head -1)`; `git worktree add --detach "$T/base" "$LAST_TAG"`; for each of `./modules/pact`, `./modules/cabana`, `./modules/signpost` run `go -C "$T/base" doc -all ` and `go doc -all `, keep declaration lines only (lines starting with `func `, `type `, `var `, `const `, and tab-indented member lines inside type, var and const blocks; doc-comment lines are indented with four spaces and are ignored), and refuse with `additive: removed or changed in : ` for every base declaration line missing verbatim from HEAD's output; remove the worktree on exit. - 2. Stage `compat`: (a) `../sm-sitemap-plugin` must have no change since the phase base (`git status --porcelain` empty and HEAD equal to the SHA recorded at stage start of the phase, or simply no commit since the base date), then `go -C ../sm-sitemap-plugin build ./... && go -C ../sm-sitemap-plugin test -short ./...`; (b) `git -C ../sm-journal-plugin worktree add --detach "$T/journal" ` and a scratch go.work with absolute paths (use the worktree; replace summercms with this checkout, sm-translate-plugin and sm-user-plugin as in Journal's go.work), then `GOWORK=... go -C "$T/journal" build ./... && go test -short ./...`; (c) `git -C ../sm-demo-app worktree add --detach "$T/demo" `, then for each gitlink under plugins/golem15 in that commit (`git -C "$T/demo" ls-tree HEAD plugins/golem15/`) `git -C ../sm-demo-app/ worktree add --detach "$T/sub/" `, a scratch go.work using `$T/demo` and every `$T/sub/` with the demo's replace lines remapped and summercms replaced by this checkout; `go build ./...`, `go build -tags summercms_mcp ./...`, `go test -short ./...` in `$T/demo`. Every worktree is removed on exit (trap). Print `compat: ok ` per consumer. - 3. Add both stages to `--all` and to `--self-test` (plant a removed declaration line in a fake doc output and expect a refusal). Run them now; if `additive` refuses a line that another session changed (not Phase 32), stop and report it to Jakub with the line and the commit that changed it (`git log -S`), because a patch tag cannot carry it. - Commit `test(32-06): additive and compat stages in check-phase32`. + Per D-106 (patch release, strictly additive), D-91, D-92, D-93: + 1. `scripts/apisurface/main.go` (package main, standard library only: go/build, go/parser, go/ast, go/printer, go/token, sort, strings): `go run ./scripts/apisurface ` prints the exported API of the package in that directory, one element per line, sorted, each line collapsed to single spaces; it exits 2 with a message when the directory has no Go files. Files are the non-test files go/build's default context selects for the directory. Before printing, it clears the parameter and result names of every func type (functions, methods, interface methods, func-typed fields and parameters), so renamed parameters, realigned fields and edited comments print identically. Line kinds: `func F[]() `; `method () M() ` for exported methods of exported types; `type T struct` plus `field T.Name ` per exported field and `embed T.` per embedded field; `type T interface` plus `imethod T.M() ` per method (unexported methods too: they decide who can implement it) and `iembed T.` per embedded interface or type-set term; `type T ` for other defined types and `type T = ` for aliases (type parameters printed); `const X = @` with an omitted type and value carried from the previous spec of the group, as Go does; `var X ` (the declared type only, `var X` when it has none). + 2. Stage `additive` (D-106): `LAST_TAG=$(git tag --sort=-v:refname | head -1)`; `git worktree add --detach "$T/base" "$LAST_TAG"`. Packages: `modules/pact`, `modules/cabana`, `modules/signpost`, plus every other directory under `modules/` that holds a non-test `.go` file changed since `LAST_TAG` (`git diff --name-only "$LAST_TAG" -- modules`, which includes uncommitted changes) and exists at the tag. For each package run the tool on `$T/base/` and on `` into `$T/api/.base` and `$T/api/.head` (`LC_ALL=C sort`), then refuse with: + (a) `additive: removed or changed in : ` for every base line missing from head (`LC_ALL=C comm -23`): removed declarations, fields and methods, and every changed signature, field type, struct tag, constant value and underlying type, including a signature written over several lines that gains a parameter; + (b) `additive: method added to existing interface in : ` for every head `imethod T.` or `iembed T.` line that is not in base while base has `type T interface`: a method added to an existing exported interface breaks every type that implements it, so a new capability method (pact `Has*` interfaces included, such as `HasAdminActionExtensions`) must go on a new, separate optional interface, which this rule allows with any methods. + Allowed, and printed as `additive: ok (+ lines)`: new declarations, new interfaces, new methods on concrete types and new fields on existing structs (Go 1 compatibility with keyed literals; the compat stage proves the consumers still build). Remove the worktree on exit (trap). + 3. Stage `compat`: (a) `../sm-sitemap-plugin` must have no change since the phase base (`git status --porcelain` empty and HEAD equal to the SHA recorded at stage start of the phase, or simply no commit since the base date), then `go -C ../sm-sitemap-plugin build ./... && go -C ../sm-sitemap-plugin test -short ./...`; (b) `git -C ../sm-journal-plugin worktree add --detach "$T/journal" ` and a scratch go.work with absolute paths (use the worktree; replace summercms with this checkout, sm-translate-plugin and sm-user-plugin as in Journal's go.work), then `GOWORK=... go -C "$T/journal" build ./... && go test -short ./...`; (c) `git -C ../sm-demo-app worktree add --detach "$T/demo" `, then for each gitlink under plugins/golem15 in that commit (`git -C "$T/demo" ls-tree HEAD plugins/golem15/`) `git -C ../sm-demo-app/ worktree add --detach "$T/sub/" `, a scratch go.work using `$T/demo` and every `$T/sub/` with the demo's replace lines remapped and summercms replaced by this checkout; `go build ./...`, `go build -tags summercms_mcp ./...`, `go test -short ./...` in `$T/demo`. Then (d): every other sibling directory of this checkout whose `go.mod` requires `git.golem15.com/golem15/summercms` (the other `sm-*-plugin` and `sm-*-app` repositories and Go application workspaces), except this checkout, the three above and `../sm-seo-plugin`: a detached worktree of its HEAD, detached worktrees of its gitlinked submodules at their pinned commits as in (c), a scratch go.work mirroring its own go.work (or `use .` when it has none) with absolute paths and summercms replaced by this checkout, then `go build ./...`, printing `compat: ok (build)`. When that build fails, build the same worktree against the last tag's worktree (created as in `additive` when compat runs alone): failing there too prints `compat: skipped (fails at too)`, otherwise refuse `compat: builds at but not at HEAD`. Every worktree is removed on exit (trap). Print `compat: ok ` per consumer of (a) to (c). + 4. Add both stages to `--all`, and extend `--self-test` with planted package pairs in a temporary directory (a `base/` and a `head/` Go package, run through the tool and the stage's comparison), each printing `PASS self-test additive ` or `refuse: self-test additive `: + - refused: `removed-func` (an exported func deleted); `changed-param` (`F(int)` becomes `F(int64)`); `multiline-param-added` (a signature written over several lines gains a parameter); `interface-method-added` (`type HasThing interface { Thing() string }` gains `Other() error`, modelled on a pact capability interface); `field-type-changed`; `const-value-changed`; + - accepted: `func-added`; `struct-field-added` (the other fields realigned by gofmt); `optional-interface-added` (a new `HasOther` interface with methods); `method-added-to-struct`; `param-renamed`; `comment-edited`. + 5. Run both stages now. If `additive` refuses a line that another session changed (not Phase 32), stop and report it to Jakub with the line and the commit that changed it (`git log -S`), because a patch tag cannot carry it (D-106). If it refuses a Phase 32 line, make the seam additive in its own `fix(32-06): ...` commit (for example, move a new interface method onto a new optional interface and update its callers) before continuing. + 6. Commit (resume unit, see Execution units) staging only `scripts/check-phase32.sh` and `scripts/apisurface/main.go`: `test(32-06): additive and compat stages in check-phase32`. bash scripts/check-phase32.sh --self-test && bash scripts/check-phase32.sh --additive --compat - non-zero exit, an "additive: removed or changed" line, a missing "compat: ok" line for sm-sitemap-plugin, sm-journal-plugin or sm-demo-app, or "FAIL" in the output + non-zero exit, an "additive: removed or changed" or "additive: method added to existing interface" line, a "refuse:" line, a missing "compat: ok" line for sm-sitemap-plugin, sm-journal-plugin or sm-demo-app, or "FAIL" in the output + go vet ./scripts/apisurface && go run ./scripts/apisurface modules/pact | grep -c '^type HasAdminActionExtensions interface$' + non-zero exit, or a count of 0 (the surface does not list pact's capability interfaces) - - `bash scripts/check-phase32.sh --additive --compat 2>&1 | grep -c "compat: ok"` prints 3 + - `bash scripts/check-phase32.sh --additive --compat 2>&1 | grep -c "compat: ok"` is at least 3, with one line each for sm-sitemap-plugin, sm-journal-plugin and sm-demo-app, and every other sibling repository that requires the framework has a `compat: ok (build)` or `compat: skipped` line - `git -c safe.directory='*' -C ../sm-journal-plugin worktree list | wc -l` and the same for sm-demo-app print 1 after the run (no leftover worktrees) - - The self-test proves the additive stage refuses a planted removed declaration + - `bash scripts/check-phase32.sh --self-test 2>&1 | grep -c "PASS self-test additive"` prints 12, including the `interface-method-added`, `changed-param` and `multiline-param-added` refusals + - `go run ./scripts/apisurface modules/pact | grep -c "^imethod HasAdminActionExtensions\."` is at least 1 (interface method sets are part of the surface) + - `go list -deps ./scripts/apisurface | grep '\.' | grep -v '/scripts/apisurface$'` prints nothing (standard library only) - The gate proves end to end that the framework change is additive and that existing consumers build and test unchanged against it. + The gate proves end to end that the framework change is additive (D-106): no removed or changed API element and no method added to an existing interface; existing consumers build and test unchanged against it. @@ -210,7 +250,7 @@ Pre-phase commits: the framework base SHA and the Journal SHA before plan 32-02' - Descriptors: MatchPattern edge cases (trailing slash, encoded segments, extra and missing segments, empty segment), MatchPath ordering, duplicate Model/Controller boot errors, types without Content unchanged - Write the exhaustive unit matrices named in 32-VALIDATION.md (`TestFormExtension*`, `TestFormExtensionBootErrors`, `TestRecordEventDiff`, `TestContent*`) as table tests per the behaviour list (SC-1, D-91, D-92, D-93). Pure tests run under `-short`; Docker-backed cases stay in the existing `_postgres_test.go` files. Fix any defect the tests expose in the same task with its own `fix(32-06): ...` commit. Commit tests as `test(32-06): framework unit matrices for form extensions, record events and descriptors`. + Write the exhaustive unit matrices named in 32-VALIDATION.md (`TestFormExtension*`, `TestFormExtensionBootErrors`, `TestRecordEventDiff`, `TestContent*`) as table tests per the behaviour list (SC-1, D-91, D-92, D-93). Pure tests run under `-short`; Docker-backed cases stay in the existing `_postgres_test.go` files. Fix any defect the tests expose in the same task with its own `fix(32-06): ...` commit. Commit (resume unit, see Execution units) staging only the three test files: `test(32-06): framework unit matrices for form extensions, record events and descriptors`. go test -short ./modules/cabana ./modules/signpost -run 'TestFormExtension|TestRecordEventDiff|TestContent' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -40 @@ -243,7 +283,8 @@ Pre-phase commits: the framework base SHA and the Journal SHA before plan 32-02' - Journal: descriptor matrix for post, category, tag (Describe per locale, Match exact/non-exact, Records paging), journal-tag type with tag_pages on and off, translated slug lookup - Write the unit tests per the behaviour list (SC-2, SC-3, SC-4, D-09, D-22, D-25 and the UI-SPEC backstops); pure Go tests run under `-short`. Element tests run with `npm --prefix ../sm-seo-plugin run test` (vitest + happy-dom, `tests/**/*.test.ts`). Fix defects in their own commits. Then update `32-VALIDATION.md`: the per-task verification map (every task of plans 32-01 to 32-06 with its automated command and status), Wave 0 items checked, `nyquist_compliant: true`, `wave_0_complete: true`, `status: validated` only if the full gate is green; commit it separately as a planning-docs commit. Plugin commits in each repository staged by path. + Write the unit tests per the behaviour list (SC-2, SC-3, SC-4, D-09, D-22, D-25 and the UI-SPEC backstops); pure Go tests run under `-short`. Element tests run with `npm --prefix ../sm-seo-plugin run test` (vitest + happy-dom, `tests/**/*.test.ts`). Fix defects in their own commits. Then update `32-VALIDATION.md`: the per-task verification map (every task of plans 32-01 to 32-06 with its automated command and status), Wave 0 items checked, `nyquist_compliant: true`, `wave_0_complete: true`, `status: validated` only if the full gate is green. + Commits (resume units, see Execution units), staged by path, in this order: A in ../sm-seo-plugin `test: unit tests for the SEO classes` (the nine `classes/*_test.go` files of this task); B in ../sm-seo-plugin `test: element backstops` (the five `tests/elements/*.test.ts` files); C in ../sm-journal-plugin `test: content descriptors, journal-tag type and translated slugs` (content_descriptors_test.go, menu_item_types_test.go); D in this repository, as a planning-docs commit, `docs(32-06): complete the validation map` (32-VALIDATION.md only). go -C ../sm-seo-plugin test -short ./classes -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -40 @@ -268,7 +309,7 @@ Pre-phase commits: the framework base SHA and the Journal SHA before plan 32-02' Task 4: Release go-ahead: confirm the framework patch version, the plugin versions and the push list A pushed tag is pinned by plugin go.mod files and application submodules; Phase 33 builds on it, and the seams become published API. Release Phase 32 now with the computed versions, and which remote operations are approved? - Before asking, the executor runs and reports: `git fetch --tags origin` in the framework, sm-journal-plugin and sm-seo-plugin; `git tag --sort=-v:refname | head -3` in the framework; the next patch = the newest framework tag with its patch number plus one (v0.5.x today; a later 0.5.x if another session tagged meanwhile; only the third number moves); Journal's next patch from its newest tag; sm-seo-plugin's first version following the sibling plugins' first-release convention; `bash scripts/check-phase32.sh --all` with `PHASE32_ALLOW_UNMOUNTED=1` (green except the mount, which Task 5 completes); clean trees in the three repositories apart from other sessions' known uncommitted demo entries (listed). The push list it presents: framework master and the tag; sm-journal-plugin main and its tag; sm-seo-plugin main (first push) and its tag; then local demo commits (apply the 32-05 patch, add the seo submodule, bump Journal, require the framework tag), pushed only if named; the summercms.io changelog post for the framework tag; optional Journal bumps in sm-grzybyfunkcjonalne-app (with `golem15.journal.tag_pages: false`, its layer has no tag pages) and sm-summercmsio-app. Nothing is deployed in any option. + Before asking, the executor runs and reports: `git fetch --tags origin` in the framework, sm-journal-plugin and sm-seo-plugin; `git tag --sort=-v:refname | head -3` in the framework; the next patch = the newest framework tag with its patch number plus one (D-106: v0.5.x today; a later 0.5.x if another session tagged meanwhile; only the third number moves); Journal's next patch from its newest tag; sm-seo-plugin's first version following the sibling plugins' first-release convention; `bash scripts/check-phase32.sh --all` with `PHASE32_ALLOW_UNMOUNTED=1` (green except the mount, which Task 5 completes); clean trees in the three repositories apart from other sessions' known uncommitted demo entries (listed). The push list it presents: framework master and the tag; sm-journal-plugin main and its tag; sm-seo-plugin main (first push) and its tag; then local demo commits (apply the 32-05 patch, add the seo submodule, bump Journal, require the framework tag), pushed only if named; the summercms.io changelog post for the framework tag; optional Journal bumps in sm-grzybyfunkcjonalne-app (with `golem15.journal.tag_pages: false`, its layer has no tag pages) and sm-summercmsio-app. Nothing is deployed in any option. @@ -324,6 +366,7 @@ Pre-phase commits: the framework base SHA and the Journal SHA before plan 32-02' - `grep -n "golem15.seo" ../sm-demo-app/summer.yaml` prints one line and `bash ../sm-demo-app/scripts/check-pins.sh` prints `check-pins: ok` - The changelog post URL for the tag is recorded in 32-06-SUMMARY.md - Every demo commit of this task lists only this plan's paths (`git show --stat`) + - `32-06-release-log.md` has one `step done` line per performed step and no step twice The framework patch release carrying the three seams is tagged and announced, both plugins are released on it, the demo mounts the SEO plugin, and the full phase gate is green. @@ -342,32 +385,34 @@ Pre-phase commits: the framework base SHA and the Journal SHA before plan 32-02' | Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | |-----------|----------|-----------|----------|-------------|-----------------| -| T-32-30 | Tampering | published framework API (patch tag) | high | mitigate | additive stage refuses any removed or changed exported declaration of pact, cabana and signpost; compat stage builds existing consumers unchanged | +| T-32-30 | Tampering | published framework API (patch tag) | high | mitigate | additive stage compares the API surface (scripts/apisurface) of pact, cabana, signpost and every changed framework package with the last tag and refuses removed or changed elements and methods added to existing interfaces, with a self-test that plants each case; compat stage tests three consumers and builds every other sibling that requires the framework (D-106) | | T-32-31 | Repudiation | pushes and tags without consent | high | mitigate | Task 4 blocking decision checkpoint; Task 5 performs exactly the named operations and records each in the SUMMARY | | T-32-32 | Information disclosure | changelog token | high | mitigate | changelog-post.py reads ~/.config/summercms/journal-token itself; the executor never prints, copies or mints it and asks Jakub on a missing or refused token | | T-32-33 | Tampering | demo submodules carrying foreign work | medium | mitigate | the Journal submodule moves only when its HEAD is an ancestor of the release tag; other sessions' uncommitted entries are never staged | -| T-32-SC | Tampering | npm/pip/cargo installs | high | mitigate | No new package; tests use the toolchain approved at plan 32-03 Task 4 | +| T-32-34 | Tampering | a resumed release task repeating one-way steps | medium | mitigate | 32-06-release-log.md records each step as it completes; a resumed run confirms and skips logged steps, never re-tags, force-pushes or re-posts | +| T-32-SC | Tampering | npm/pip/cargo installs | high | mitigate | No new package; tests use the toolchain approved at plan 32-03 Task 4; scripts/apisurface uses the standard library only | - `bash scripts/check-phase32.sh --all` green with Docker and the demo mounted (or the hold recorded in deferred-items.md). - `go vet ./... && go test ./...` green in the framework, sm-seo-plugin, sm-journal-plugin and sm-demo-app; `npm --prefix ../sm-seo-plugin run test` green. - 32-VALIDATION.md `nyquist_compliant: true`. +- Decision coverage: `comm -23 <(grep -oE '^- \*\*D-[0-9]+[abc]?' .planning/phases/32-seo-foundation/32-CONTEXT.md | sed 's/^- \*\*//' | LC_ALL=C sort -u) <(grep -oE '^\| D-[0-9]+[abc]?' .planning/phases/32-seo-foundation/32-DECISIONS-MAP.md | sed 's/^| //' | LC_ALL=C sort -u)` prints nothing: every decision is mapped to a plan or to Phase 33. -SC-5: the framework release with the seams is tagged as the next v0.5.x patch, it is strictly additive, and the new code has unit tests delivered in this last plan; SC-1 to SC-4 are proven by the gate and the demo. +SC-5: the framework release with the seams is tagged as the next v0.5.x patch, it is strictly additive (D-106: no removed or changed API element, no method added to an existing interface), and the new code has unit tests delivered in this last plan; SC-1 to SC-4 are proven by the gate and the demo. ## Artifacts this phase produces (plan 06) -- `scripts/check-phase32.sh` stages `additive` and `compat` +- `scripts/check-phase32.sh` stages `additive` and `compat`; `scripts/apisurface/main.go` (the API surface printer the additive stage compares) - Unit tests: `modules/cabana/form_extensions_test.go`, `modules/cabana/record_events_test.go`, `modules/signpost/content_test.go`; `../sm-seo-plugin/classes/*_test.go` (TestResolverPrecedence, TestRedirectGraph, TestNormalizeMatrix, TestRobotsValueRoundTrip, TestRobotsMatcherGoogleExamples, TestTemplates, TestCSV, TestOwnerResolution, TestIssues); `../sm-seo-plugin/tests/elements/{text,robots,serp,robotstxt,transfer}.test.ts`; Journal descriptor, tag-type and slug tests -- Updated `32-VALIDATION.md`; `deferred-items.md` when anything is held +- Updated `32-VALIDATION.md`; `deferred-items.md` when anything is held; `32-06-release-log.md` (the release steps as performed) - Tags (after the go-ahead): the framework's next v0.5.x patch, sm-journal-plugin's next patch, sm-seo-plugin's first release - sm-demo-app commits: framework pin, Journal bump, golem15.seo mount (summer.yaml, go.work, go.mod, .gitmodules, plugins.gen.go, main.go, config/golem15.seo.yaml, nginx `location = /robots.txt`, DEPLOY.md) - The summercms.io changelog post for the framework tag -Create `.planning/phases/32-seo-foundation/32-06-SUMMARY.md` when done with the Task 4 answer, every version, tag, push and commit SHA, the gate output summary and the changelog URL. +Create `.planning/phases/32-seo-foundation/32-06-SUMMARY.md` when done with the Task 4 answer, every version, tag, push and commit SHA (per task and commit group, the release steps copied from 32-06-release-log.md), the gate output summary and the changelog URL. diff --git a/.planning/phases/32-seo-foundation/32-DECISIONS-MAP.md b/.planning/phases/32-seo-foundation/32-DECISIONS-MAP.md new file mode 100644 index 00000000..faa2686c --- /dev/null +++ b/.planning/phases/32-seo-foundation/32-DECISIONS-MAP.md @@ -0,0 +1,157 @@ +# Phase 32: decision map + +Every decision in `32-CONTEXT.md` (D-01 to D-106, with D-20a, D-20b and D-20c, and Q-01), mapped to the Phase 32 plan and task that implements it, or deferred to Phase 33 ("SEO output, audit and content", Q-01). A decision with entries in both columns is split: Phase 32 delivers the part named in the first column, and Phase 33 delivers the rest. IDs that do not appear here (D-26 to D-29, D-35 to D-39, D-47 to D-49, D-53, D-54, D-59 to D-69, D-73 to D-79, D-84, D-87 to D-89) do not exist in CONTEXT.md. + +Plan references use `plan/Tn` (for example `32-03/T2` is plan 32-03, Task 2). Plan 32-06 cites the Phase 33 column in its objective ("Phase 33 boundary") and checks in its verification that every CONTEXT.md decision appears in this file. + +## Code layout and seams + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-90 | The framework carries no SEO code; SEO lives in two shared plugins | 32-01, 32-02 (neutral seams, neutral docs); 32-05/T4 (`marker` gate stage); 32-06/T1, T5 (`--all` green) | — | +| D-91 | Cross-plugin admin form extensions with Before/After placement | 32-01/T1 to T4 (seam, boot errors, docs, SPA backstop); used by 32-03/T1, T2, T5 and 32-04/T2; 32-06/T2 (unit matrix) | — | +| D-92 | Record events from cabana | 32-02/T1 to T3; used by 32-03/T3 (cleanup) and 32-04/T2 (address change after commit); 32-06/T2 (diff tests) | — | +| D-93 | Content descriptors in signpost | 32-02/T1, T2, T4; used by 32-03/T1 to T3; 32-06/T2, T3 | Sitemaps and the page registry consume `Records` (D-30, D-33) | +| D-94 | `sm-seo-plugin` scope | 32-03 (models, SEO tab, settings, resolver, generation job, cleanup, preview/placeholder/revision widgets); 32-04 (redirects, 404 log, robots.txt, verification API, redirect decision); 32-05 (MCP, CSV, issue report) | Page registry, llms.txt, JSON-LD `@graph` and breadcrumbs APIs, crawler | +| D-95 | `sm-sitemap-plugin` with its own filter interface that SEO implements | — (`Requires()` stays empty in 32-03) | Whole decision, with the sitemap rewrite | +| D-96 | Content plugins keep only their own fixes (Journal) | 32-02/T4 (Journal descriptors, journal-tag type, D-58 fix) | H1 to H2 conversion (D-50), Journal schema types (D-43) | +| D-97 | One shared Nuxt SEO layer | — | Whole decision | +| D-98 | Six lean plans in a fixed order | 32-01 to 32-06 (this plan set) | — | +| D-99 | robots.txt tester on `github.com/jimsmart/grobotstxt` v1.0.3 with a decision note | 32-03/T4 (legitimacy checkpoint); 32-04/T4 (install, tester, decision note, STACK row); 32-06/T3 (Google examples test) | — | +| D-100 | OG image through an `seoimage` field type; extensions own no attachments | 32-01/T3 (fileupload extension fields refused); 32-03/T5 (`seoimage`, E7) | — | +| D-101 | Form extensions accept only the tab zone | 32-01/T1 to T3 (`Zone` field, other zones are boot errors) | — | +| D-102 | Journal API emits record events; `journal-tag` type | 32-02/T3 (API emission), 32-02/T4 (journal-tag); 32-04/T2 (listener for writes outside the admin) | — | +| D-103 | Redirect decision endpoint and verification-file API only | 32-04/T3 (decision in resolve), 32-04/T4 (verification API) | Layer enforcement of redirects and root mapping of verification files | +| D-104 | SEO settings as an `AdminSingleton` linked from a settings tile, per-type tabs through D-91 | 32-01 (generated fields declared in Go, P-01); 32-03/T2 | — | +| D-105 | `Claude-User` joins the AI bot list | 32-04/T4 | — | +| D-106 | Patch release (next v0.5.x), strictly additive seams; `sm-seo-plugin` repo created with `tea` | 32-01, 32-02 (additive truths); 32-03/T1 (clones the existing repo, P-12); 32-06/T1 (`additive` stage on API surfaces, `compat` stage), 32-06/T4, T5 (patch tag computed at release) | — | + +## 1. SEO data model + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-01 | Per-record SEO owned by the plugin, one table keyed by (model type, record id, locale), tab switched on per type | 32-03/T1 (entries table, SEO tab), 32-03/T2 (inject switch per type) | — | +| D-02 | Per-URL rules; precedence URL rule > record > global > derived | 32-03/T2 (rules screen, resolver P-14) | — | +| D-03 | Global settings: templates, defaults, organisation data, verification, robots per type | 32-03/T2 (templates, default description and image, site name, verification, per-type robots) | Organisation or person data for JSON-LD (with D-43, D-45) | +| D-04 | Social tags inherit, optional overrides, `twitter:site` | 32-03/T2 (resolver OG and Twitter values), 32-03/T5 (Social fields) | — | +| D-05 | Search engine verification: meta codes and files | 32-03/T2 (settings, meta on the home path), 32-04/T4 (`/_seo/api/v1/verification/{name}`) | Serving the files at the site root from the layer (D-103) | +| D-06 | Canonical: self-referencing absolute, overrides, warnings | 32-03/T2 (builder, overrides), 32-03/T5 (P1 canonical-to-noindex callout), 32-04/T3 (canonical-redirected callout), 32-05/T2, T3 (issues, batch conflicts) | — | +| D-07 | Query-parameter list with path refinement; drives filter-URL noindex | 32-03/T2 (`classes/urls.go`) | — | +| D-08 | Meta robots switches, advanced directives, `X-Robots-Tag` | 32-03/T2 (value grammar, `x_robots_tag` in resolve), 32-03/T5 (E2), 32-04/T4 (header on robots.txt and API answers); 32-06/T3 | Sending the header on frontend-rendered pages and files (layer, D-97) | +| D-09 | SERP preview, counters, keyphrase checks | 32-03/T5 (E1, E3); 32-06/T3 (element backstops) | — | + +## 2. MCP management + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-10 | The whole of SEO is manageable through MCP on `pier` | 32-05/T1, T2 (one tool per admin operation, parity test) | Tools for the crawler and the other Phase 33 screens | +| D-11 | Tools address pages by URL; owner resolution | 32-02/T4 (`MatchPath`), 32-05/T1 (`classes/owner.go`) | — | +| D-12 | Batch tools, `dry_run` with conflicts, all-or-nothing | 32-05/T3 | "Noindex on a page in a sitemap" refined by the D-95 filter | +| D-13 | Read and audit tools: effective SEO with sources, CSV, history with undo | 32-03/T2 (value sources), 32-03/T3 (history core), 32-05/T2 (read tools), 32-05/T3 (undo, CSV, E6) | — | +| D-14 | Issue report: CMS data checks plus a crawler | 32-05/T2 (CMS data checks) | The built-in crawler and its checks | +| D-15 | One SEO permission | 32-03/T2, 32-05/T1, T2 | — | + +## 3. Redirects, URL normalisation, strict URLs + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-16 | 301, 302 and 410 | 32-04/T2 | — | +| D-17 | Exact, wildcard and regex sources; options; evaluation order | 32-04/T2 | — | +| D-18 | Slug or path change creates a 301 and flattens chains; delete or unpublish creates nothing | 32-02/T2, T3 (events carry the change), 32-04/T1 (admin path), 32-04/T2 (other writers) | — | +| D-19 | Loops always refused; chains saved with a warning | 32-04/T2 (admin), 32-05/T2, T3 (MCP, import, issue report) | — | +| D-20 | 404 log with one-click redirect or 410 | 32-04/T2 (log, report endpoint, actions), 32-05/T2 (tools) | The layer's call to the report endpoint | +| D-20a | Where redirects execute (discretion) | 32-04/T3 (decision endpoint, Go-served paths) | Nitro middleware in the layer (D-97, D-103) | +| D-20b | URL normalisation per site | 32-03/T2 (Addresses settings), 32-04/T3 (`Normalize`, one hop) | Enforcement on frontend routes through the layer | +| D-20c | Strict URLs | 32-02/T4 (`Match` with `Exact`), 32-04/T3 (strict 301); 32-05/T4 (route drift check) | Layer routes reading their patterns from the CMS | + +## 4. robots.txt and noindex policy + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-21 | Default robots per page kind | 32-03/T2 (per-type defaults, never-indexed and search paths, filter-parameter URLs) | Pagination page 2+ (D-41) and signed-in routes (D-23) in the layer | +| D-22 | robots.txt editor, validation, tester, `Sitemap:` line, non-production lock | 32-03/T2 (lock in resolve), 32-04/T4 (editor, tester, served file, lock); 32-06/T3 | The `Sitemap:` default moves to the D-30 index | +| D-23 | Private frontend pages marked noindex automatically | 32-03/T2 (path patterns in the admin, applied by the resolver) | Automatic marking of signed-in routes in the layer and the user plugin's auth layer | +| D-24 | noindex, robots.txt and sitemaps stay consistent | 32-04/T4 (warning when robots.txt blocks a noindex URL), 32-05/T2, T3 (issues, batch conflict) | Noindex, canonicalised and redirected pages leave the sitemaps; "sitemap URL blocked" warning | +| D-25 | AI bot switches; optional llms.txt | 32-04/T4 (switches); 32-06/T3 | llms.txt generated from the sitemap | + +## 5. Sitemaps and page registry + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-30 | Sitemap index split by type, 1000-URL files, real `lastmod` | — | Whole decision | +| D-31 | Image, News sitemaps, IndexNow | — | Whole decision | +| D-32 | The new format replaces PHP parity (breaking, confirmed) | — | Whole decision; the XSLT todo stays with it | +| D-33 | Page registry in the CMS | — | Whole decision | +| D-34 | HTML sitemap | — | Whole decision | + +## 6. Pagination + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-40 | URL form chosen per site | — | Whole decision | +| D-41 | Page 2 and later: robots, suffixes, H1, pillar text | — (the canonical already keeps the `page` parameter, 32-03/T2) | Whole decision | +| D-42 | Numbered server-rendered links, rel prev/next, 301 and 404 rules | — | Whole decision | + +## 7. Structured data + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-43 | One JSON-LD `@graph` | — (room: the entry's `data` jsonb, 32-03/T1) | Whole decision | +| D-44 | Schema type, type-dependent fields, custom JSON-LD | — (room: reserved SEO tab anchors `seo_robots`, `seo_og_image`, 32-03/T5) | Whole decision | +| D-45 | Publisher in the global settings | — | Whole decision | +| D-46 | Validation in admin, crawler and Rich Results link | — | Whole decision | + +## 8. Headings + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-50 | No H1 in body content | — | Whole decision, including Journal's conversion (D-96) | +| D-51 | Live heading outline and crawler checks | — | Whole decision | +| D-52 | One heading convention in the layers | — | Whole decision | + +## 9. hreflang + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-55 | hreflang in head and sitemaps from one source | — | Whole decision | +| D-56 | `x-default` points to the default locale | — | Whole decision | +| D-57 | No hreflang for a missing translation | — | Whole decision | +| D-58 | Journal API translated-slug fix | 32-02/T4; 32-06/T3 (`TestTranslatedSlug`) | — | + +## 10. SSR guarantee + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-70 | Indexable pages always SSR or SSG; build guard; raw versus rendered check | — | Whole decision | +| D-71 | Real status codes, server-rendered links, content in HTML, image rules | — (the resolve answer already returns real `not_found`, `redirect` and `gone` decisions, 32-03, 32-04) | Whole decision | +| D-72 | `sm-bm-app` gets noindex and `Disallow` only | — | Whole decision | + +## 11. Internal linking + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-80 | One link per post card | — | Whole decision | +| D-81 | No generic anchors | — | Whole decision | +| D-82 | Related posts, phrase auto-linking, suggestions, orphan report | — | Whole decision | +| D-83 | Outbound link rules | — | Whole decision | + +## 12. Media and breadcrumbs + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| D-85 | Media SEO | — | Whole decision | +| D-86 | Breadcrumbs component and primary category | — | Whole decision | + +## Phase split + +| ID | Decision | Phase 32 | Phase 33 | +|----|----------|----------|----------| +| Q-01 | Two phases; Phase 33 builds on the Phase 32 framework tag | 32-06/T4, T5 (the patch tag); this map's Phase 33 column is the boundary | Builds on the tag, not on a `replace` | + +## Claude's discretion items + +- Where redirects execute (D-20a), the translated-slug fix (D-58): as above. HTML sitemap grouping (D-34): Phase 33. +- Table and package names: 32-03 P-12, P-13; 32-04 P-18 to P-23; 32-05 P-24 to P-28. +- Exact Go shapes of the seams: 32-01/T1 and 32-02/T1 (blocking decision checkpoints). +- Effective SEO fetched through one resolver endpoint per URL: 32-03/T1, T2 (P-15). Purging the site cache after SEO writes is not a Phase 32 task step; it pays off once the Phase 33 layer caches the resolve answer, so Phase 33 planning should pick it up with D-97. +- Crawler implementation: Phase 33 (D-14).