golem15/summercms
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(32): research phase domain

Browse Source
This commit is contained in:
Jakub Zych
2026-10-26 20:03:37 +01:00
parent d661e1de95
commit 54d770740c
1 changed files with 839 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

839
.planning/phases/32-seo-foundation/32-RESEARCH.md Normal file
View File

@@ -0,0 +1,839 @@
# Phase 32: SEO foundation - Research
**Researched:** 2026-10-26
**Domain:** Go framework seams (pact, cabana, signpost), a new shared plugin (`sm-seo-plugin`), redirects and robots.txt protocol code, MCP through `pier`
**Confidence:** HIGH for the code seams (read this session), MEDIUM for the plugin design (recommendations), MEDIUM for external protocol facts (official Google docs)
<user_constraints>
## User Constraints (from CONTEXT.md)
The decisions below are copied verbatim from `32-CONTEXT.md` for the parts that bind Phase 32 (phase split, sections 0 to 4). Sections 5 to 12 (D-30 to D-86) are Phase 33 scope under Q-01; they are listed by ID at the end and matter here only where a Phase 32 shape must leave room for them.
### Locked Decisions
#### Phase split (resolved)
- **Q-01 (resolved by Jakub, 2026-10-25): two consecutive phases, the second one large.** Dawid left the split to Jakub. Every decision in this document applies to both phases:
- **32 Foundation:**
- the three framework seams (D-91 to D-93), with the field placement rules of D-91;
- `sm-seo-plugin` core: data model, SEO tab, global settings;
- meta, canonical, robots;
- redirects, URL normalisation, strict URLs;
- robots.txt, MCP tools, history.
The seams are one-way framework API. Phase 32 ends with a framework release tag, and Phase 33 builds on that tagged version rather than on a `replace`.
- **33 Output, audit and content** (about 12–15 plans):
- `sm-sitemap-plugin` rewrite (D-32): XML and HTML sitemaps;
- page registry;
- JSON-LD, pagination, hreflang;
- headings, breadcrumbs;
- the Nuxt SEO layer and the SSR rules in the layers;
- crawler and issue report;
- internal linking and auto-links;
- media (alt, file names, formats).
**Exception to the lean-planning rule, approved by Jakub:** Phase 33 is planned as many smaller plans instead of a few large ones, so the planner must not merge them into 3–4 plans. The other project rules still apply: the plan list with a one-line scope each is shown to Jakub before PLAN.md files are written, and unit tests are the last plan.
ROADMAP.md still lists one Phase 32 with the full scope. Before `/msd-plan-phase 32`, narrow Phase 32 and add Phase 33 through `/msd-phase`, with its g15office task. The planner must not split or merge phases on its own.
(Research note: ROADMAP.md already carries the narrowed Phase 32 and the new Phase 33 [VERIFIED: .planning/ROADMAP.md:1576-1620].)
#### 0. Code layout (Jakub, 2026-10-25)
- **D-90:** **The framework carries no SEO code.** SEO lives in two shared plugins: a new `sm-seo-plugin` (Golem15.SEO) and an updated `sm-sitemap-plugin` (Golem15.Sitemap). The framework gets only the generic seams those plugins need (D-91 to D-93); none of them names SEO, and each is useful to other plugins. The pattern is the PixelPixel `mpseo` WinterCMS plugin (see Reference implementations):
- content plugins did not import anything from it; they registered a content type (model, controller, a mapper for default title/description/image, a URL resolver, an `is_published` check, the tab position) through the `seoinjection.extend` event;
- the SEO plugin owned one `entries` table keyed by source type and id plus the URL, injected the SEO tab into the registered controllers through `extendFormFields`, deleted the entry with the record, kept per-type defaults with placeholders in its settings (with an inject switch per type), generated entries in a job, had a Google result preview and revisions, and resolved SEO by URL on the frontend.
SummerCMS keeps that division and replaces the event with D-91 to D-93. — **Reversibility:** costly — the seams become framework API, and moving SEO back into the framework later would mean migrating the plugin's tables.
- **D-91:** **Cross-plugin admin form extensions, with real field placement.** `pact.AdminActionExtension` (controller ID plus record actions; its comment already leaves room for more kinds) gains form fields and tabs. An extension targets a controller ID, adds fields and tabs to its form, and loads and saves their values itself through `Load` and `Save` hooks that run inside the record's save transaction. The owner's model and tables are untouched.
Placement must work properly. In WinterCMS, adding field C at the end of A, B was easy, but placing it between them (A, C, B) meant fighting `extendFormFields`. Here:
- every field and tab in a compiled form has a stable name: the field key from `fields.yaml`, the tab key;
- an extension places each field with `Before` or `After` an anchor field, inside an existing tab, a new tab, or a zone (outside the tabs, primary tabs, secondary tabs); with no anchor the field goes to the end of its tab, as in WinterCMS;
- a new tab is placed the same way, `Before` or `After` another tab;
- an extension may hide an owner's field, for example a plugin's own meta-title field that the SEO tab replaces (`mpseo` removed `viewBag[meta_title]` the same way);
- the result is deterministic: extensions apply in plugin order (`party`, by `Requires()`); several extensions anchored at the same point keep that order; an anchor may name a field added by another extension only if that plugin is in `Requires()`;
- an unknown anchor, a duplicate name or an anchor cycle stops the boot with a clear error, like the existing `AdminActionExtension` checks;
- cabana compiles the final order at boot; the SPA renders the compiled form as it is and never reorders it; field `span` (left, right, full) keeps working next to inserted fields;
- top-level fields and tabs only; fields inside repeaters and nested forms are out of scope for now.
Tests cover at least: C after A gives A, C, B; C before the first field; a new tab between two existing tabs; two plugins anchored at the same field; hiding a field; each boot error. — **Reversibility:** one-way — the extension struct and the anchor rules become a published framework API.
- **D-92:** **Record events from cabana.** Other plugins can subscribe to "record saved" (created or updated) and "record deleted" for a controller or model type. The event carries the record id, the locale and the changed attributes with old and new values (so a slug or parent change is visible), and fires after commit. Admin, MCP and imports all go through cabana and emit it; plugin code that writes records directly uses a cabana helper to emit the same event. Today only GORM hooks on the owner's model exist, which another plugin cannot use. The SEO plugin uses the events for the automatic 301 (D-18) and to delete SEO entries with their record. — **Reversibility:** costly — subscribers in shared plugins depend on the event shape.
- **D-93:** **Content descriptors in signpost.** `pact.MenuItemType` already has URL patterns, `Resolve` and locale alternates. It gains optional functions that describe the type's content: list the records (for sitemaps and the registry), `lastmod`, whether a record is published, and default title, description and image (the `mapper` of `mpseo`). Content plugins (journal, FAQ, future ones) only register types in signpost, as they already do for menus and the sitemap, and never import `sm-seo-plugin` or `sm-sitemap-plugin`. — **Reversibility:** costly — every content plugin implements these functions.
- **D-94:** **`sm-seo-plugin` (Golem15.SEO)**, a new repo mounted like the other core plugins:
- models: the SEO entry (type, id, locale, URL, the SEO tab's values, `data` jsonb), entry revisions (history and undo), settings (sections 1 and 4, with a tab per content type: templates with placeholders and an inject switch, as in `mpseo`), URL rules, redirects, the 404 log, robots.txt, the page registry (D-33);
- uses D-91 to add the SEO tab to every registered type's controller, D-92 for the automatic 301 and cleanup, D-93 for defaults and URL ownership;
- a job that creates missing entries (the `mpseo` GenerateSeoEntriesJob), run through `conga`;
- the frontend API: effective SEO by URL, redirects, robots.txt, llms.txt, verification files, JSON-LD `@graph`, breadcrumbs;
- admin widgets: the result preview, the placeholder list, revisions;
- MCP through `pier` from its admin routes, plus CSV import/export and the issue report; the crawler (Phase 33) lives here too.
- **D-95:** **`sm-sitemap-plugin` (Golem15.Sitemap)** produces the sitemaps of D-30, D-31 and D-34 from signpost (D-93). It knows nothing about SEO: it defines its own small filter interface (exclude a URL, noindex, `lastmod` override) and looks it up with `app.Lookup`. `sm-seo-plugin` implements it and requires Sitemap (`Requires()`); Sitemap works without SEO.
- **D-96:** **Content plugins keep only their own fixes.** Journal gets the translated-slug fix (D-58), the H1 → H2 conversion (D-50), its schema types and the D-93 descriptors. It does not depend on the SEO plugin.
- **D-97:** **The Nuxt side is one new shared layer** (`vue-seo-layer`, name to confirm): `<head>` from the effective-SEO API, redirect middleware, JSON-LD, breadcrumbs, the SSR build guard (D-70). Journal, menu and FAQ layers and the apps extend it.
#### 1. SEO data model
- **D-01 (revised by Jakub, 2026-10-25):** **Per-record SEO is owned by `sm-seo-plugin`.** A content type takes part by registering in signpost (D-93); the SEO settings switch the tab on per type, as `mpseo` did. The plugin adds a standard "SEO" tab to the type's admin form through the generic form extension (D-91) and stores the data in its one shared table keyed by (model type, record id, locale). Every plugin, present and future, gets the same tab without its own migrations, and MCP has one place to read and write. Originally recorded as a framework mechanism; moved to the plugin under D-90. — **Reversibility:** one-way — the shared table and the opt-in contract are what every content plugin builds on.
- **D-02:** **Per-URL SEO rules** live on a separate screen, with exact URLs and patterns (e.g. `/blog?tag=*`). They cover pages with no record behind them: the home page, Nuxt pages, filter URLs. **Precedence: URL rule > record SEO > global defaults > values derived from content.** — **Reversibility:** costly — the precedence order is visible to editors and MCP clients and is baked into the effective-SEO resolver.
- **D-03:** **Global SEO settings** contain:
- title templates and defaults: a template such as `%title% | %site%`, a separate home-page template, the default meta description, the default OG image and the site name;
- organisation or person data for JSON-LD (D-43);
- search engine verification (D-05);
- default robots per content type (D-21).
- **D-04:** **Social tags inherit with optional overrides.** By default OG and Twitter take the title, description and image from SEO or from the content. A "Social" section in the SEO tab can override `og:title`, `og:description` and `og:image`. Twitter Card (`summary_large_image`) falls back to OG, and `twitter:site` is set in the global settings. There is no social card preview.
- **D-05:** **Search engine verification** supports two methods:
- meta tag codes (Google, Bing, Yandex, others), emitted on the home page;
- verification files (name and content) served from the site root, e.g. `/google1234.html`.
- **D-06:** **Canonical:**
- Every page is self-canonical by default: an absolute URL, with junk parameters (utm, fbclid and similar) stripped and pagination kept.
- Overrides are possible per record and per URL rule. An override may be relative or absolute and may point to another domain.
- The admin warns when a canonical points to a noindex or redirected page.
- **D-07:** **Global canonical rules use a query-parameter list.** Parameters marked as content (e.g. `page`) stay in the canonical; filter and sort parameters (e.g. `sort`, `color`) are dropped. The list can be refined per path pattern (e.g. `/katalog/*`). The same list drives the noindex of filter URLs (D-21).
- **D-08:** **Meta robots:**
- index/noindex and follow/nofollow switches, defaulting to "inherit";
- an advanced section: `max-snippet`, `max-image-preview`, `max-video-preview`, `noarchive`, `nosnippet`, `noimageindex`, `unavailable_after`;
- the same directives are also sent as an `X-Robots-Tag` header, so they work for PDFs and other files.
- **D-09:** **Editor aids in the SEO tab:**
- a live Google SERP preview (desktop and mobile, with pixel truncation);
- character and pixel counters for title and description, with warnings for too long, too short, empty, and duplicates of another record;
- a focus keyphrase with Yoast-style checks: the phrase in the title, H1, description and URL, plus text length.
#### 2. MCP management
- **D-10:** **The whole of SEO is manageable through MCP.** Every admin operation in this phase has an MCP tool, built on the 19.7 `pier` module: admin routes become tools under the linked admin's permissions, and read grants see only read tools. Typical use: Dawid audits a site, then tells his Claude "noindex this page, 301 that one to this one".
- **D-11:** **Tools address pages by URL.** The server resolves the owner:
- if a record is behind the URL, the change goes into the record's SEO tab;
- otherwise a URL rule is created.
The answer says where the change was written, and the caller can force a URL rule.
- **D-12:** **Bulk changes:**
- Batch tools take a list of changes, e.g. 50 redirects from one audit.
- `dry_run` returns the diff and the conflicts: redirect loops and chains, noindex on a page that is in a sitemap, a canonical pointing to a redirected page.
- Applying a batch is all-or-nothing.
- **D-13:** **Read and audit tools:**
- the effective SEO of a URL, with the source of every value (URL rule, record, global or derived);
- the site issue report (D-14);
- CSV export and import with a preview, also in the admin;
- a change history with author, time, and before/after values, from both the admin and MCP, with undo of one change or of a whole batch.
- **D-14:** **The issue report combines two sources:**
- CMS data checks: duplicate or missing titles and descriptions, conflicting rules;
- a built-in crawler, run on demand or on a schedule, that fetches rendered pages from the sitemaps and from links. It checks status codes, H1/H2/H3, canonical, robots, JSON-LD, internal links, orphan pages, generic anchors and missing alt.
Results are available in the admin and over MCP. The crawler is the only way to check the Nuxt-rendered output and static frontend pages.
- **D-15:** **One SEO permission** covers all SEO operations: meta, rules, redirects, robots.txt, settings. It is not split per area.
#### 3. Redirects, URL normalisation, strict URLs
- **D-16:** **Status codes: 301, 302 and 410 Gone.** No 307/308.
- **D-17:** **Source matching:**
- exact URLs (the default), wildcard patterns (`/old/*` → `/new/*`), and regex with capture groups;
- options: ignore case, trailing slash, and query string (ignore it or pass it through);
- evaluation order: exact, then patterns, then regex.
- **D-18:** **A slug or path change creates a 301** from the old URL to the new one and flattens existing chains (A→B, then B→C becomes A→C). Deleting or unpublishing a record creates nothing: the URL returns 404 and appears in the 404 log.
- **D-19:** **Loops are always refused**, whether they come from the admin, MCP or an import. Chains are saved with a warning and listed in the issue report.
- **D-20:** **404 log:** URL, hit count, last hit and referer. A redirect or 410 can be created from an entry with one click or through MCP. There are no per-rule hit counters and no validity dates.
- **D-20a:** **Where redirects execute is Claude's discretion.** The rules live in the CMS and are enforced by whatever serves the page: Nuxt Nitro middleware in a shared layer for frontend pages, Go for its own paths and for statically embedded sites.
- **D-20b:** **URL normalisation is set per site:**
- https always;
- www or bare domain;
- trailing slash or none;
- lowercase paths;
- double slashes, `/index.html` and similar collapse to the clean form.
Any other form reaches the final form in one 301 hop, never through a chain.
- **D-20c:** **Strict URLs.** WinterCMS answered 200 to `/<any-category>/<post-slug>` because it checked only the post slug, which created duplicates. In SummerCMS every URL segment (category, parent categories, author, locale prefix) must match the record. A URL whose segments do not match a record that exists 301s to the record's canonical URL. This also covers a post moving to another category. A wrong URL never answers 200. — **Reversibility:** costly — it changes how every layer route and every Journal API lookup resolves records.
#### 4. robots.txt and noindex policy
- **D-21:** **Default robots per page kind** (editable per type in the global settings, and per record):
| Page kind | Robots |
|---|---|
| Login, register, password reset, every signed-in user page | noindex |
| Internal search results | noindex,follow |
| Filter and sort parameter URLs (D-07 list) | noindex,follow |
| Technical pages: cart and checkout (future), draft previews, thank-you pages, 404 and error pages | noindex |
| Tag archives | noindex,follow |
| Author pages | index,follow (always) |
| Pagination page 2+ | index,follow, self-referencing canonical |
- **D-22:** **robots.txt editor:**
- a text editor with syntax validation and a tester ("is this URL blocked for Googlebot?");
- the `Sitemap:` line is appended automatically;
- every non-production environment (staging, demo) always serves `Disallow: /` and `X-Robots-Tag: noindex`, whatever the stored content. — **Reversibility:** reversible — but the staging lock must never be disabled silently, since an indexed staging copy is costly to clean up.
- **D-23:** **Private frontend pages are marked automatically.** Every route that requires sign-in gets `noindex,nofollow` and an `X-Robots-Tag` header without per-page code. Path patterns in the admin cover public pages that should not be indexed (e.g. `/konto/*`, `/logowanie`). The user plugin's auth layer marks its own pages.
- **D-24:** **noindex, robots.txt and sitemaps stay consistent automatically.**
- Pages that are noindex, canonicalised elsewhere or redirected drop out of the XML and HTML sitemaps.
- The admin warns when a noindex page is blocked in robots.txt (Googlebot cannot see the noindex) and when a sitemap URL is blocked.
- **D-25:** **AI bots:**
- per-bot switches (GPTBot, OAI-SearchBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot, Applebot-Extended and others), with AI search bots distinguished from training bots;
- all bots allowed by default;
- an optional `llms.txt` generated from the sitemap.
#### Phase 33 decisions (referenced, not in Phase 32 scope)
D-30 to D-34 (sitemaps, page registry), D-40 to D-42 (pagination), D-43 to D-46 (structured data), D-50 to D-52 (headings), D-55 to D-58 (hreflang, Journal translated-slug fix), D-70 to D-72 (SSR), D-80 to D-83 (internal linking), D-85 to D-86 (media, breadcrumbs). Phase 32 must leave room for D-33 (registry types in signpost), D-95 (the sitemap filter interface the SEO plugin will implement), D-43/D-44 (`data` jsonb on the entry, reserved SEO tab anchors) and D-25's `llms.txt`.
### Claude's Discretion
- Where redirects execute (D-20a), HTML sitemap grouping and paging (D-34), the Journal translated-slug fix (D-58).
- Table and package names inside the plugins, and the exact Go shape of the three seams (D-91 to D-93). The seams go into the existing `pact`, `cabana` and `signpost` modules; no new framework module is planned.
- How the frontend fetches effective SEO: one resolver endpoint per URL, cached and purged through the existing site-cache purge.
- Crawler implementation (Go, in-process job via `conga`/River, politeness limits).
### Deferred Ideas (OUT OF SCOPE)
- **Social card preview** (Facebook/LinkedIn/X) in the SEO tab: offered, not chosen.
- **Per-redirect hit counters and validity dates:** offered, not chosen.
- **307/308 redirects:** not chosen.
- **E-commerce structured data** (Product, offers on category pages): no shops yet.
#### Reviewed Todos (not folded)
- `sitemap-xslt-browser-deprecation.md` (g15office w8kupqi3): replace the browser XSLT view of `/sitemap.xml`. The user chose to keep it as a separate backlog task. The D-32 rewrite of the sitemap plugin touches the same code, so the planner should note that the XSL output may be removed or kept there.
- `sitemap-plugin-port.md`: stale. The port was delivered in Phase 17 and should be closed separately.
</user_constraints>
<phase_requirements>
## Phase Requirements
No requirement IDs are mapped (ROADMAP: "Requirements: TBD"). The success criteria are the coverage source.
| ID | Description | Research Support |
|----|-------------|------------------|
| SC-1 | A plugin can add a field between two fields of another plugin's admin form (A, B plus C after A renders A, C, B), and add a tab between two existing tabs; unknown anchors, duplicate names and anchor cycles stop the boot with a clear error. | Pattern 1 (placement pass in `cabana.Activate` next to `compileActionExtensions`), Pitfalls 1–6, SPA tab derivation facts |
| SC-2 | Journal posts, categories and tags show the SEO tab without importing `sm-seo-plugin`, and the frontend gets the effective SEO of any URL with the precedence URL rule > record > global > derived. | Pattern 3 (D-93 descriptors incl. controller ID), Pattern 5 (resolver), Pitfall 9 (no tag type today), journal findings |
| SC-3 | Changing a record's slug creates a 301 and flattens existing chains; redirect loops are refused from the admin, MCP and imports. | Pattern 2 (D-92 events), Pattern 6 (URL-diff 301 + flatten), Pitfalls 10–12, journal API write path finding |
| SC-4 | Every SEO operation in the admin is available as an MCP tool, and a batch change can be previewed with dry-run and undone from history. | Pattern 7 (own `/_seo/api/v1` routes for pier), Pattern 8 (dry-run by rollback, history/undo) |
| SC-5 | The framework release with the seams is tagged, and the new code has unit tests, delivered in the phase's last plan. | Release section, Validation Architecture |
</phase_requirements>
## Project Constraints (from CLAUDE.md)
From `summercms.go/CLAUDE.md`, the meta repo `CLAUDE.md` and the user's global instructions:
- **No co-author tags in commits** (user global and project). One logical change per commit; planning docs and code in separate commits. Stage files by explicit path, never `git add -A`; leave other sessions' uncommitted edits (`STATE.md`, `state.json`) alone; check for in-flight edits before touching shared files such as cabana.
- **Do not push, tag or deploy without the user's go-ahead.** Every pushed `vX.Y.Z` tag gets a summercms.io changelog post the same day (`../sm-summercmsio-app/scripts/changelog-post.py`); never print or mint the journal token.
- **Framework code never names a consuming application**, and per D-90 never names SEO.
- **Dependencies:** standard library first; a new dependency needs a research doc or a phase/decision note naming it (`.planning/research/STACK.md`). YAML is parsed with `goccy/go-yaml`, never `gopkg.in/yaml.v3`.
- **No runtime plugin loading** (stdlib `plugin`, yaegi) without a decision note.
- **Documentation rule:** a change to a module's exported API, config keys, CLI commands or dependencies updates that module's `README.md` and the affected `docs/` pages in the same commit; every identifier in a README or docs page must exist (the docs checker enforces it). Affected today: `modules/pact/README.md`, `modules/cabana/README.md`, `modules/signpost/README.md`, `modules/party/README.md`, `docs/backend/admin-controllers.md`, `docs/plugins/registration.md`, `docs/plugins/menu-item-types.md`, `docs/setup/coming-from-wintercms.md` [VERIFIED: grep of the seam identifiers this session].
- **Keep `go vet ./...` and `go test ./...` green at every commit.** Docker-backed tests skip under `-short`; a `TestMain` that starts containers calls `flag.Parse()` and checks `testing.Short()`.
- **Admin release gate:** a tag that includes `admin/` changes needs `npm --prefix admin run build`, the committed `modules/boardwalk/dist`, and `scripts/check-admin-dist.sh` passing.
- **Core plugin contracts stay non-breaking** (routes, payloads, tables, migration IDs, config keys) unless the user asks. Change a submodule plugin in its own checkout, push it first, then bump the pointer in the app in a separate commit.
- **MSD project rules:** lean planning (few, large plans; show the plan list with one-line scopes and wait for confirmation), unit tests are the last plan, STATE.md only through `msd_run query state.*` handlers, g15office sync with a short comment when the phase advances.
- **User memory:** prefers an established, maintained library plus a decision note for protocol and security code; Dawid decides SEO behaviour (talk in Polish, files in English), Jakub decides phase and scope.
## Summary
Phase 32 is mostly framework surgery with a large plugin on top. The three seams land in code that exists and is well-factored: `cabana.Activate` already runs a post-compile pass for cross-plugin record actions (`compileActionExtensions`), cabana already fires after-commit events through `lagoon.AfterCommit` plus the `festival` bus (`fireSubjectChanged`), and `signpost` already owns per-type URL resolution with locale alternates. The D-91 form extension is the hardest seam: the current compile pipeline assumes every form field belongs to the controller's own plugin, so plugin field types, widgets, partials, `fileupload` and the writable-column binding all need an "owned by an extension" branch. The SPA needs no ordering code, because it derives tabs by first field appearance and puts the default tab first.
The investigation turned up several facts the UI-SPEC and CONTEXT do not account for, and the planner must handle them. First, a settings form (`pact.SettingsItem.Form`) refuses `widget`, `partial`, `fileupload` and every plugin field type, so the SEO settings screen must be an `AdminSingleton` controller linked from the settings tile. Second, `pier` binds MCP tools to exact method and path routes, while cabana's admin routes are generic `{vendor}/{plugin}/{controller}` patterns behind `requireAjax`, so the SEO plugin needs its own token-guarded JSON API under `/_seo/api/v1`, as Journal has. Third, Journal's MCP writes go through Journal's own API, not through cabana, so Journal's API handlers must call the new cabana record-event helper. Without that, a slug change made through MCP creates no 301. Fourth, Journal has no tag menu item type, and one site turns tag pages off. Fifth, Go cannot serve dynamic root-level verification files without shadowing app catch-all routes.
**Primary recommendation:** Build D-91 as a new `pact.AdminFormExtension` (a sibling kind carried next to `AdminActionExtension`). Each extension field is tagged with its contributing plugin, so that partials, widget actions, field types and storage dispatch to the extension. D-92 events are `pact` types fired by cabana through `lagoon.AfterCommit`, and the SEO plugin derives the 301 by comparing resolved URLs before and after the change. D-93 descriptor functions become optional fields on `pact.MenuItemType`, including the admin controller ID. `sm-seo-plugin` exposes one service layer, used by its admin controllers, its D-91 extension hooks and a `/_seo/api/v1` JSON API, and pier tools bind to that API.
## Architectural Responsibility Map
| Capability | Primary Tier | Secondary Tier | Rationale |
|------------|-------------|----------------|-----------|
| Form extension placement (D-91) | API / Backend (cabana, boot) | Browser (SPA renders as compiled) | Order is compiled once at boot; the SPA must never reorder (C1) |
| Extension field storage (Load/Save) | API / Backend (extension hooks in the save tx) | Database (SEO tables) | Owner's model and tables untouched (D-91) |
| Record events (D-92) | API / Backend (cabana, after commit) | Plugins (listeners) | One emission point per write path; direct writers call a helper |
| Content descriptors (D-93) | API / Backend (content plugins via signpost) | — | Content plugins never import SEO or sitemap |
| SEO data, rules, redirects, history | Database / Storage (sm-seo-plugin tables) | API (service layer) | One shared table keyed by type, id, locale (D-01) |
| Effective SEO resolution | API / Backend (public resolver endpoint) | Frontend Server (Nuxt layer, Phase 33) | Precedence baked into the resolver (D-02) |
| Redirect / normalisation enforcement | Frontend Server (Nitro middleware, Phase 33) | API / Backend (decision endpoint; Go for `/robots.txt` and its own paths) | D-20a: whoever serves the page enforces |
| robots.txt serving and staging lock | API / Backend (exact `GET /robots.txt` route) | CDN/proxy (nginx location per app) | Same deployment pattern as `/sitemap.xml` |
| Verification files at site root | Frontend Server / proxy | API (content via `/_seo/api/v1`) | A Go `/{name}` route would shadow app catch-alls |
| MCP tools | API / Backend (pier → `/_seo/api/v1`) | — | pier binds exact method + path routes |
| SERP preview, counters, robots editor | Browser (plugin custom elements E1–E6) | API (widget actions) | Light-DOM elements through WidgetField/PluginField |
## Standard Stack
### Core (all already in the project)
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| Go stdlib `regexp` | Go 1.27 | Redirect regex sources | RE2, linear time: no catastrophic backtracking from admin- or MCP-supplied patterns [CITED: pkg.go.dev/regexp] |
| Go stdlib `net/url`, `path` | Go 1.27 | URL normalisation, canonical building | Stdlib-first rule; no maintained normaliser library needed [ASSUMED: purell is archived] |
| Go stdlib `encoding/csv` | Go 1.27 | CSV import/export | Stdlib; BOM and `;` delimiter handled by hand |
| `gorm.io/gorm` | v1.31.2 | Models | Decided [VERIFIED: .planning/research/STACK.md:18] |
| `github.com/go-gormigrate/gormigrate/v2` | v2.1.7 (as pinned by sm-sitemap-plugin) | Per-plugin migrations | Decided [VERIFIED: ../sm-sitemap-plugin/go.mod] |
| `conga` (River) | framework | GenerateSeoEntries job, async purges | Decided (`pact.HasJobs`) |
| `cooler.SiteCachePurger` | framework | Purge rendered pages after SEO changes | Exists: `SiteCachePurgeURLKey = "site_cache.purge_url"` [VERIFIED: modules/cooler/sitecache.go:26] |
| `festival.Bus` (`app.Events`) | framework | Record events | Synchronous typed bus: "Dispatch is synchronous on the caller goroutine." [VERIFIED: modules/festival/bus.go:29-30] |
| `testcontainers-go` + postgres module | v0.44.0 | Plugin DB tests | Decided, same as sm-sitemap-plugin |
### Supporting (new, needs a decision note)
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| `github.com/jimsmart/grobotstxt` | v1.0.3 (2022-03-16) | Server-side robots.txt tester (E5 `robots_test`), `Sitemap:` extraction, D-24 "noindex URL blocked" check | Function-for-function port of Google's C++ robots.txt parser; exposes `RobotsMatcher.MatchingLine()` (the line E5 shows) and a `ParseHandler` with line numbers [CITED: pkg.go.dev/github.com/jimsmart/grobotstxt]. Apache-2.0 [CITED: github.com/jimsmart/grobotstxt]. Version and date [VERIFIED: proxy.golang.org @latest]. `[ASSUMED]` for legitimacy: the Go ecosystem is not covered by the package-legitimacy seam |
Element bundle (dev-only, `sm-seo-plugin/package.json`), same pins as the journal Blog Editor:
| Package | Version | Purpose |
|---------|---------|---------|
| typescript | 5.9.3 | Element typecheck |
| vite | 7.3.5 | ES module bundle for E1–E6 |
| vitest | 3.2.7 | Element unit tests (UI-SPEC backstops) |
| happy-dom | 20.11.6 | DOM for vitest |
[VERIFIED: ../sm-journal-plugin/package.json; `npm view` confirms each exact version exists]
### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| grobotstxt | `github.com/temoto/robotstxt` v1.1.2 (2021, MIT) | Older, its own parser, no matching-line API; the README does not document Google's longest-match semantics [CITED: github.com/temoto/robotstxt] |
| grobotstxt | Hand-rolled RFC 9309 matcher (~200 lines) | Stdlib-first and no stale dependency, but protocol code the user prefers to take from a library. RFC 9309 is frozen, so a 2022 release of a faithful Google port is not a staleness risk in practice. **Decision for Jakub; recommend grobotstxt + decision note.** |
| Own `/_seo/api/v1` routes for MCP | Binding pier tools to cabana's generic admin routes | Not workable: path segments would be agent-supplied arguments, and the routes sit behind `requireAjax` (see Pattern 7) |
**Installation:**
```bash
# sm-seo-plugin (Go)
go get github.com/jimsmart/grobotstxt@v1.0.3 # only after the decision note is accepted
# sm-seo-plugin (element bundle)
npm install --save-dev --save-exact typescript@5.9.3 vite@7.3.5 vitest@3.2.7 happy-dom@20.11.6
```
## Package Legitimacy Audit
| Package | Registry | Age | Downloads | Source Repo | Verdict | Disposition |
|---------|----------|-----|-----------|-------------|---------|-------------|
| github.com/jimsmart/grobotstxt | Go proxy | v1.0.3, 2022-03-16 | n/a (Go proxy has no counts) | github.com/jimsmart/grobotstxt | not run (seam supports npm/pypi/crates only) | `[ASSUMED]`: planner adds `checkpoint:human-verify` + decision note before `go get` |
| typescript | npm | pinned 5.9.3 | 210M/wk | github.com/microsoft/TypeScript | OK | Approved |
| vite | npm | pinned 7.3.5 | 144M/wk | github.com/vitejs/vite | SUS ("too-new", the *latest* release is 5 days old; the pin is older) | Flagged: `vite` [WARNING: flagged as suspicious — verify before using.] Pin exactly 7.3.5, the version already in sm-journal-plugin's lockfile |
| vitest | npm | pinned 3.2.7 | 68M/wk | github.com/vitest-dev/vitest | SUS ("too-new", same reason) | Flagged: `vitest` [WARNING: flagged as suspicious — verify before using.] Pin exactly 3.2.7 |
| happy-dom | npm | pinned 20.11.6 | 10M/wk | github.com/capricorn86/happy-dom | SUS ("too-new", same reason) | Flagged: `happy-dom` [WARNING: flagged as suspicious — verify before using.] Pin exactly 20.11.6 |
**Packages removed due to [SLOP] verdict:** none
**Packages flagged as suspicious [SUS]:** vite, vitest, happy-dom. The flag comes from the publish date of the latest version, not of the pinned one. The pins match an approved sibling plugin, so the checkpoint can be a one-line confirmation. `postinstall` is null for all four.
## Architecture Patterns
### System Architecture Diagram
```
┌──────────────────────── boot (party → signpost → Boot → surf → cabana.Activate → pier) ───────────────────────┐
content plugin │ MenuItemTypes() + D-93 descriptor funcs ──► signpost.Registry (published on app) │
(journal) │ │ │
│ sm-seo-plugin.Boot reads registry ──► builds AdminFormExtensions (SEO tab per type, per-type settings tabs) │
│ cabana.Activate: compileRegistry → compileFormExtensions(placement, boot errors) → compileActionExtensions │
│ pier.Mount: binds SEO tools to /_seo/api/v1/* routes │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Admin editor ──PUT /backend/api/golem15/journal/posts/{id}──► cabana.save (one tx)
│ ├─ lift extension values out of body (never reach Fill / owner columns)
│ ├─ owner Fill, Validate, row write, translations
│ ├─ extension Save(tx, record, values, changes) ──► seo entries + history (same tx)
│ │ └─ returns per-field notices / FormFieldError (422, rolls back)
│ └─ lagoon.AfterCommit ─► app.Events.Fire(pact.RecordSaved{...})
│ │
MCP client ─► pier ─► PUT /_journal/api/v1/posts/{id} (journal handler) ─────┤ cabana.EmitRecordSaved(helper) after commit
▼
sm-seo-plugin listener: re-resolve URLs via descriptor
├─ URL changed? ── yes ─► redirect old→new (301), flatten chains, refuse loops,
│ update entry URL, history (via=auto), site-cache purge
└─ RecordDeleted ─► delete entries (history keeps the values)
Nuxt SSR (Phase 33 layer) ─GET /_seo/api/v1/resolve?url=…─► resolver
├─ normalisation (one 301 hop) ├─ redirects (exact → pattern → regex) ├─ strict URL (descriptor Match → canonical 301)
└─ effective SEO: URL rule > record entry > per-type template/global > descriptor-derived; + X-Robots-Tag value; + staging lock
Crawlers ─GET /robots.txt─► Go route: stored rules + AI-bot groups + Sitemap lines; non-production → "Disallow: /"; never 5xx
```
### Recommended Project Structure (new repo `sm-seo-plugin`, module `git.golem15.com/golem15/sm-seo-plugin`, package `seo`, ID `golem15.seo`)
```
sm-seo-plugin/
├── plugin.go # party.Register, Requires(), Register/Boot (reads signpost, builds extensions)
├── admin.go # AdminFS/LangFS embeds, AdminControllers, Navigation, Settings (link entry), Permissions
├── extensions.go # AdminFormExtensions(): SEO tab per descriptor type; per-type tabs on own settings controller
├── field_types.go # mlseotext/seotext, mlseorobots/seorobots, robotstxt (+ og image type if chosen, see Open Q)
├── listeners.go # RecordSaved / RecordDeleted → redirects, entry cleanup
├── routes.go # GET /robots.txt; /_seo/api/v1/* (public resolve + backend-guarded management)
├── mcp.go # pact.HasMCP tools bound to /_seo/api/v1
├── jobs.go # GenerateEntries job (conga), registered via pact.HasJobs
├── classes/ # service layer: resolver, redirects graph, normaliser, robots, history, batch, csv, templates
├── controllers/ # redirects, rules, notfound, history (list/form/preview), robots + transfer + settings (AdminSingleton)
├── models/ # entry, revision(history), settings, type_settings, rule, redirect, notfound, robots
├── updates/ # gormigrate migrations (timestamped IDs like sm-sitemap-plugin)
├── lang/{en,pl}/lang.yaml # golem15.seo::lang.*
├── assets/src/ # E1–E6 custom elements (Vite), assets/dist committed + scripts/check-dist.sh
├── tests/ + *_test.go # testmain_test.go (testcontainers), fake content plugin fixture
└── go.mod / go.work # replace framework => ../summercms.go (dev), require tagged framework at release
```
### Pattern 1: D-91 form extension compiled after the registry
**What:** A new `pact` kind, compiled by cabana in `Activate` right after `compileRegistryWithTypes` and before `compileActionExtensions`/`compileContributions`. That is where the existing cross-plugin pass runs today:
```go
// modules/cabana/http.go:135-143 (verbatim)
// Contributed record actions are merged before compileContributions,
// which validates every record action's permissions.
if err := compileActionExtensions(reg, plugins); err != nil {
return nil, err
}
if err := compileAccountContributions(reg, plugins); err != nil {
return nil, err
}
if err := compileContributions(reg, plugins); err != nil {
```
[VERIFIED: modules/cabana/http.go:135-143]
The struct to extend, verbatim:
```go
type AdminActionExtension struct {
Controller string
RecordActions []AdminRecordAction
}
```
[VERIFIED: modules/pact/capabilities.go:429-432], with the doc comment ending "Only record actions are carried for now; the struct leaves room for more kinds." [VERIFIED: modules/pact/capabilities.go:427-428]
**Recommended Go shape (Claude's discretion, sketch, all names `[ASSUMED]`):**
```go
// pact — added fields on AdminActionExtension keep existing keyed literals compiling.
type AdminActionExtension struct {
Controller string
RecordActions []AdminRecordAction
Form *AdminFormExtension // nil: no form contribution
}
type AdminFormExtension struct {
Fields string // path in the contributing plugin's AdminFS to a fields.yaml with placement keys
Hide []string // owner field names removed from the compiled form
Permissions []string // an admin without them gets no extension fields (schema, show, save)
Visible func(ctx context.Context) bool // per request, like NavigationItem.Visible
Actions []AdminAction // widget actions of the extension's `type: widget` fields
PartialData func(ctx context.Context, name string, record any) (any, error)
Load func(ctx context.Context, in AdminFormLoad) (map[string]any, error)
Save func(ctx context.Context, in AdminFormSave) (AdminFormSaveResult, error)
}
```
In the extension's fields file, each field takes the ordinary `fields.yaml` keys plus `before:`/`after:` (anchor field) and `zone:`. A top-level `tabs:` list places new tabs (`name`, `before`/`after`). Parsing the file with the existing `decodeFieldMapping`/`compileFieldNode` reuses every type, option and plugin-field-type check, and keeps YAML-defined admin forms consistent with the rest of the framework.
**Placement algorithm (compile once, deterministic):**
1. Canonicalise the owner's list: group the fields by tab in first-appearance order. The default tab (no `tab`) comes first, as the SPA renders it. Within a tab, source order is kept. The rendered result is identical, because the SPA already groups this way.
2. For each plugin in `party` order (already topologically sorted by `Requires()` [VERIFIED: modules/party/registry.go:103-104 `ordered, err := topoSort(selected, index)`]), apply its extension: insert its new tabs at `before`/`after` tab anchors, then its fields at `before`/`after` field anchors. With no anchor, a field goes to the end of its tab. Several extensions on the same anchor keep plugin order: insert after the last field that was inserted at that anchor (an "anchor cursor"), not directly after the anchor.
3. Boot errors (each names the plugin, the controller and the anchor): an unknown field or tab anchor; an anchor that names a field from a plugin outside `Requires()`; a duplicate field name, whether against the owner or an earlier extension; a duplicate tab key; a cycle among an extension's own `before`/`after` references (topologically sort the extension's fields, and report a cycle when the sort stalls); `before:` the default tab (UI-SPEC C1); any `zone` other than the tab zone (UI-SPEC Q-1 default); `Hide` naming an unknown field or a field that another field's `fill`, `dependsOn`, `trigger` or `preset` references.
4. Run the per-field compile steps again on the merged list, with an owner-aware branch (Pitfalls 3–6).
**SPA facts that make "no SPA change" true:**
```ts
// admin/src/components/form/formState.ts:21-26 (verbatim)
export const DEFAULT_TAB = 'default'
/** Tab key of a field: its tab label, or the default tab. */
export function tabOf(field: FormField): string {
return field.tab ? `tab:${field.tab}` : DEFAULT_TAB
}
```
[VERIFIED: admin/src/components/form/formState.ts:21-26]. `FormView.vue` builds `tabs` in first-appearance order and calls `out.unshift(tab)` only for `DEFAULT_TAB` [VERIFIED: admin/src/views/FormView.vue:151-171].
### Pattern 2: D-92 record events fired after commit
**What:** Use the established after-commit firing pattern. Event types live in `pact`, like `pact.MCPSubjectChanged`, so that subscribers depend only on `pact`.
```go
// modules/cabana/subject_events.go:31-41 (verbatim excerpt)
tx, _ := TxFromContext(ctx)
lagoon.AfterCommit(ctx, tx, func(ctx context.Context, _ *gorm.DB) {
for _, id := range adminIDs {
if id == 0 {
continue
}
event := &pact.MCPSubjectChanged{Guard: backendSubjectGuard, UserID: id, Kind: kind}
if err := app.Events.Fire(ctx, event); err != nil {
moduleLog.WarnContext(ctx, "cabana: subject changed listener failed", "kind", kind, "admin_id", id, "error", err)
}
```
[VERIFIED: modules/cabana/subject_events.go:31-41]. `runAfterCommit` calls each callback synchronously in a loop with a `recover` [VERIFIED: modules/lagoon/transaction.go:193-204], so listeners run before the HTTP response is written.
**Recommended shape (sketch, `[ASSUMED]` names):**
```go
// pact
type RecordChange struct {
Attribute string // column or ML field name; relation names for belongsTo keys
Locale string // "" for non-translatable; the locale for ML values (translated slugs live in the translate table)
Old, New any
}
type RecordSaved struct {
Controller string // admin controller ID; "" for direct writes
Model string // MorphName() when the model implements it, else TableName()
ID string // primary key as text (uint and string keys)
Created bool
Changes []RecordChange
}
type RecordDeleted struct {
Controller, Model, ID string
}
// cabana helper for direct writers (journal API handlers, record/bulk actions, console imports):
func EmitRecordSaved(ctx context.Context, app *backpack.App, tx *gorm.DB, ev pact.RecordSaved)
func EmitRecordDeleted(ctx context.Context, app *backpack.App, tx *gorm.DB, ev pact.RecordDeleted)
```
Emission points in cabana: `CRUDService.save` (create/update; snapshot the host columns after `loadRecord` and before `lagoon.Fill`, and read `TranslatedExact` for submitted ML fields before `applyMLTranslations`), `Delete`, `BulkDelete`, `reorderPost` (a tree move changes `parent_id`, and with it nested URLs), and the relation-manager child writes (`relation_child.go`). A listener error is logged and never fails the save, the same rule as `fireSubjectChanged`.
### Pattern 3: D-93 descriptors as optional `MenuItemType` fields
The current struct, verbatim:
```go
type MenuItemType struct {
Key string
Label string
DefaultPattern string
PatternFrom string
Placeholders []string
Nesting bool
DynamicItems bool
References func(ctx context.Context) ([]MenuItemReference, error) `json:"-"`
Resolve func(ctx context.Context, in MenuItemResolve) (*ResolvedMenuItem, error) `json:"-"`
}
```
[VERIFIED: modules/pact/capabilities.go:798-808]. The only implementer across the sibling repos is `sm-journal-plugin/classes/menu_item_types.go` (and its vendored copies in apps), and it uses keyed literals (`Key: MenuTypeCategory, …`) [VERIFIED: grep + ../sm-journal-plugin/classes/menu_item_types.go:100-110]. Adding fields is therefore source-compatible.
**Recommended additions (sketch, `[ASSUMED]`):**
```go
// D-93 content descriptor; all optional. A type with Content == nil takes no part.
Content *ContentDescriptor `json:"-"`
type ContentDescriptor struct {
Controller string // admin controller ID whose form edits the records (needed for D-91 targeting)
Model string // the morph name the record events carry, to map events → type
Records func(ctx context.Context, after string, limit int) ([]ContentRecord, string, error) // paged, for the generate job / registry / sitemaps
Describe func(ctx context.Context, id string) (*ContentRecord, error)
Match func(ctx context.Context, values map[string]string) (id string, ok bool, err error) // pattern placeholders → record, for D-11 ownership and D-20c strict URLs
}
type ContentRecord struct {
ID string
Published bool
LastMod *time.Time
Locales []ContentLocale // per locale: URL (same layering as Resolve), Title (H1), Description, Image URL, BodyWords
}
```
`Controller` and `Model` are generic (a sitemap, a search indexer and an audit tool all need them) and name nothing SEO-specific. signpost gains read helpers (`ContentTypes()`, `TypeForModel(model)`, and `MatchPath(ctx, path)`, which compiles each type's layered pattern into a matcher). D-33 (Phase 33) later registers the page-registry types through the same struct.
### Pattern 4: SEO settings as an `AdminSingleton` controller, per-type tabs via D-91 on itself
A settings form refuses exactly the field kinds the UI-SPEC puts on the SEO settings screen:
```go
// modules/cabana/settings.go:96-104 (verbatim)
fields, err := decodeFields(raw, types.refusing())
...
for _, field := range fields {
// A settings screen has no admin controller to own actions or view models.
if field.Type == "widget" || field.Type == "partial" || field.Type == "fileupload" || field.Type == permissionFieldType {
return nil, fmt.Errorf("cabana: setting %s field %s: type %s is not supported on a settings form", item.Code, field.Name, field.Type)
}
```
[VERIFIED: modules/cabana/settings.go:96-104]. `types.refusing()` also rejects plugin field types such as `seorobots`; the docstring above it reads "a settings form refuses them (supported in controller forms only)" [VERIFIED: modules/cabana/settings.go:71-72].
So: `golem15.seo.settings` implements `pact.AdminSingleton`. The tile is a link entry (`SettingsItem.Controller`, as sitemap does with `Controller: controllers.DefinitionsControllerID` [VERIFIED: ../sm-sitemap-plugin/admin.go:50]). Per-type tabs (`type_<key>_*`) cannot be model columns, because types are dynamic. The plugin registers one D-91 extension per descriptor type on its own settings controller, with Load/Save hooks against a `type_settings` table. This dogfoods the seam, and the UI-SPEC already allows it ("applies D-91 placement to its own settings form", C11).
### Pattern 5: One effective-SEO resolver, one service layer
- `classes.Resolver.Resolve(ctx, absoluteURL, locale) (Result, error)` runs, in order: the staging lock, normalisation (a single 301 to the final form), redirects (exact, then pattern, then regex), strict URL (descriptor `Match` → canonical URL; a mismatch gives a 301), then effective SEO with a per-value `source` (rule / record / template / global / content / default), and the `X-Robots-Tag` value.
- Every writer (admin controllers' Form hooks, D-91 Save, `/_seo/api/v1` handlers, CSV import, record-event listeners) calls one service package, so history, loop checks and cache purges cannot be bypassed.
- Public read: `GET /_seo/api/v1/resolve?url=…` (throttled; answers only published records; absolute URLs built from configured `app.url`/frontend URL, never from `Host`).
### Pattern 6: URL-diff 301 instead of attribute parsing (D-18)
On `RecordSaved`, call the type's `Describe(id)` to get the current per-locale URLs and compare them with the URLs stored on the SEO entries. Any difference creates `old → new` (301, origin "Address change"), flattens rules whose target is `old`, deletes any rule whose **source** equals `new` (otherwise the record would be shadowed), drops self-loops created by flattening, then updates the entry URL. This covers slug changes, translated-slug changes, category moves, parent changes and host pattern changes, without the listener knowing which attribute drives the URL. `RecordChange` values still let the extension's Save hook show the `redirects.auto_created` notice inside the same request.
### Pattern 7: MCP through the plugin's own routes
pier binds by exact method and pattern and looks the route up in the route table:
```go
// modules/pier/catalog.go:178-182 (verbatim)
route, ok := routes[method+" "+pattern]
if !ok {
return nil, fmt.Errorf("%s: binds to %s %s, which no plugin registers", errPrefix, method, pattern)
}
```
[VERIFIED: modules/pier/catalog.go:178-182]. Cabana's admin routes are generic, e.g. `g.Put("/{vendor}/{plugin}/{controller}/{id}", requireAjax(s.update))` [VERIFIED: modules/cabana/http.go:371], and a pier `path:` field must be a required agent argument [VERIFIED: modules/pier/catalog.go:220-223]. Copy Journal's pattern instead: `/_journal/api/v1/...` routes with `Guard: "backend"` and declared `Permissions` [VERIFIED: ../sm-journal-plugin/mcp.go:14-20, 78-100; ../sm-journal-plugin/routes.go:14-34]. The handlers detect MCP calls with `bouncer.Dispatched(ctx, "backend")` [VERIFIED: modules/bouncer/dispatch.go:40] and record `via = mcp` in history.
### Pattern 8: Dry-run by rollback, apply by token, undo with a staleness guard
- Batch: run the batch in `lagoon.Transaction`, collecting the diff and conflicts; on `dry_run` return the diff and roll back on purpose. Validation then runs the same code in both modes. Apply is the same call without the rollback, all-or-nothing (D-12).
- E6 apply token: an HMAC over (dataset, normalised rows, the data fingerprint of the touched rows). Apply runs the dry-run again and refuses with `transfer.stale` when the fingerprint differs (UI-SPEC C13).
- History: one revision row per changed object with `batch_id`, `author`, `via`, `before`/`after` jsonb. Undo refuses when the current value differs from `after` (`history.undo_stale`), when the undo would create a loop (`history.undo_loop`), or when the record is gone (`history.undo_gone`). An undo is itself a new revision (`via = undo`).
### Anti-Patterns to Avoid
- **Putting SEO columns on content models or naming SEO in framework code:** violates D-90.
- **Binding extension fields through `BindWritableFields`:** the binding refuses any scalar field that is not a model column (`"cabana: controller " + id + ": field " + field.Name + " is not a model column"` [VERIFIED: modules/cabana/crud.go:256]). Extension fields must be lifted out before projection.
- **A Go `GET /{name}` route for verification files:** it is more specific than an app's `GET /` and would capture every one-segment page of a statically served site (summercms.io serves its site through `g.Get("/", site.ServeHTTP)` [VERIFIED: ../sm-summercmsio-app/plugins/golem15/summercms/plugin.go:62-65]).
- **Answering 5xx on `/robots.txt`:** Google stops crawling for 12 hours, then uses a cached copy [CITED: developers.google.com/search/docs/crawling-indexing/robots/robots_txt].
- **Building canonical/absolute URLs from the request `Host`:** cache poisoning and host-header injection; use the configured site URL.
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| robots.txt matching (longest match, allow wins ties, `*`/`$`, UA groups) | Own matcher | grobotstxt (decision note) | Google semantics have many edge cases; the tester must agree with Googlebot |
| Regex engine for redirect sources | PCRE-style lib | stdlib `regexp` | RE2 guarantees linear time on untrusted patterns |
| After-commit event dispatch | Goroutines / channels | `lagoon.AfterCommit` + `app.Events.Fire` | Rolled-back saves fire nothing; the existing pattern is proven |
| Site cache invalidation | New HTTP purge client | `cooler.SiteCachePurger` / `PurgeSiteCacheAfterCommit` [VERIFIED: modules/cooler/sitecache.go:302] | Coalescing, loopback checks, token rules exist |
| Admin forms for redirects/rules/404/history | Custom SPA screens | cabana lists, forms, previews, record/bulk actions | UI-SPEC: "no SPA change" |
| MCP plumbing | Own MCP server | `pact.HasMCP` → pier | One endpoint, grants, audit |
| CSV | Own parser | `encoding/csv` (+ BOM strip/prepend, `;` delimiter) | Quoting rules |
| Job queue for entry generation | goroutine loops | `conga` (`pact.HasJobs`) | Retries, one pool |
**Key insight:** most of this phase is composition. The only new algorithms are field placement, redirect graph maintenance (flatten, loop detection), normalisation and the precedence resolver. Each is a pure function that can be unit-tested without a database.
## Common Pitfalls
### Pitfall 1: Tabs merge by localized label in the SPA
**What goes wrong:** `tabOf` keys tabs by `tab:${field.tab}`, and cabana localizes `field.Tab` per request (`field.Tab = translateKey(ctx, tr, src.Tab)` [VERIFIED: modules/cabana/form_schema.go:331]). Two different tab keys that translate to the same text in some locale merge into one tab.
**How to avoid:** check duplicate tab keys on the raw key at boot, and also check that localized labels do not collide for each locale the phrasebook loads (the translator is available in `Activate`).
### Pitfall 2: Interleaved owner tabs make "after tab X" ambiguous
**What goes wrong:** Journal post fields interleave tabs (title/slug/content in Edit, excerpt in Manage, categories/tags in Categories, then published/is_pinned/user/published_at/featured_images back in Manage [VERIFIED: ../sm-journal-plugin/models/post/fields.yaml:2-72]). "After Manage" by field index is ill-defined.
**How to avoid:** canonicalise to tab-grouped order first (Pattern 1, step 1). Consequence for the UI-SPEC: the Post tab order is **Edit | Manage | Categories | SEO** (first appearance), not "Edit | Categories | Manage | SEO" as written in C3.
### Pitfall 3: Extension widgets fail the owner's widget checks
**What goes wrong:** `compileExtension` checks the widget tag against the **owner's** plugin prefix, requires `fill` keys to be writable scalar fields, and requires the owner controller to declare `AdminClientAssets` JS: `"field %s: fill %s is not a writable scalar field"` and `"field %s: a widget needs the controller to declare its JS through pact.AdminClientAssets"` [VERIFIED: modules/cabana/extension.go:132, 145, 150]. E3 (`golem15-seo-serp`, fill `seo_title`/`seo_description`/`seo_keyphrase`) fails all three on `golem15.journal.posts`.
**How to avoid:** for extension fields, check the prefix against the contributing plugin, accept fill keys that are fields of the same extension, accept the contributing plugin's field-type scripts as the element source, and dispatch `widgets/{field}` to `AdminFormExtension.Actions` (name collisions with the owner's actions are a boot error).
### Pitfall 4: Extension partials resolve against the owner's files
**What goes wrong:** `compilePartials` reads `path.Join(dir, "_"+name+".htm")` from the owner's FS and requires the owner controller to implement `pact.AdminPartialData` [VERIFIED: modules/cabana/extension.go:231-233]. P1/P2 would fail the boot.
**How to avoid:** compile extension partials from the contributing plugin's AdminFS, dispatch them to `AdminFormExtension.PartialData`, and refuse a partial name that clashes with the owner's. The view-model guard (no GORM models) applies unchanged.
### Pitfall 5: `fileupload` cannot be an extension field as-is
**What goes wrong:** `compileFileFields` requires the owner's model to implement `attach.Owner` and to declare the field in `AttachRelations` (`"is not an attachment relation the model declares in AttachRelations"` [VERIFIED: modules/cabana/field_file.go:309-326]). The UI-SPEC's `seo_og_image` (C3 #10) cannot attach to a Journal post without touching its model.
**How to avoid:** choose one (Open Question 2): (a) D-91 lets an extension declare an attachment owner of its own (the SEO entry), which touches upload, deferred binding and hydration; (b) an `seoimage` plugin field type using `pact.FormFieldType.Upload` ("Upload, when set, stores a file the element uploads and returns its URL." [VERIFIED: modules/pact/capabilities.go:531-532]), stored as a file reference in the entry. Recommendation: (b) for Phase 32. It needs a seventh element, E7, which is a UI-SPEC delta.
### Pitfall 6: Translatable plugin field types write to the host column
**What goes wrong:** `FormFieldType.Translatable` means "the default locale in the host column, the others as translations" [VERIFIED: modules/pact/capabilities.go:519-521]. An `mlseotext` extension field would write into the owner's columns.
**How to avoid:** for extension fields, run `Decode` (validation and notices) but divert storage to `AdminFormExtension.Save` as `map[locale]string`. On show, run the extension `Load` first and the type's `Load` after it.
### Pitfall 7: Hidden SEO fields must also be skipped on save
**What goes wrong:** the per-request removal (`hiddenSchemaFields` [VERIFIED: modules/cabana/http.go:857-866]) is cabana-internal and schema-only. An admin without `golem15.seo.manage` could still post `seo_*` keys.
**How to avoid:** apply `Permissions`/`Visible` in schema, show and save. On save, drop the keys and skip `Save` when the extension is not visible, so stored values stay untouched (UI-SPEC C1 "Permission").
### Pitfall 8: Journal's MCP writes bypass cabana
**What goes wrong:** Journal's MCP tools bind to `PUT /_journal/api/v1/posts/{id}` and the other `/_journal/api/v1` write routes, which its own handlers serve [VERIFIED: ../sm-journal-plugin/routes.go:25-33; ../sm-journal-plugin/mcp.go:78-100]. A slug changed through MCP creates no `RecordSaved`, so no 301.
**How to avoid:** Journal's `Store`/`Update`/`Destroy` (and featured-image writes, if they ever affect the URL) call the cabana helper inside their transaction. This is a Phase 32 change in sm-journal-plugin. D-96 lists only "descriptors" for Journal, so flag the extra change to Jakub as a consequence of D-92.
### Pitfall 9: Tags have no menu item type, and some hosts have no tag pages
**What goes wrong:** Journal registers `journal-category`, `all-journal-categories`, `journal-post`, `all-journal-posts`, `category-journal-posts` and no tag type [VERIFIED: ../sm-journal-plugin/classes/menu_item_types.go:20-26]. SC-2 requires the SEO tab on tags. The grzyby layer config sets `tag: false` [VERIFIED: ../sm-grzybyfunkcjonalne-app/vue-grzybyfunkcjonalne-app/nuxt.config.ts:23-26], while signpost requires a `DefaultPattern` starting with "/" for every type [VERIFIED: modules/signpost/signpost.go:79-81].
**How to avoid:** add a `journal-tag` type with a descriptor, and give hosts a way to say "this type has no public URL here" (for example `ContentDescriptor` answering no URL when a host config flag is off). The SEO tab still shows; the resolver and P2 show `effective.no_url`.
### Pitfall 10: Loop detection for patterns and regex is not a graph walk
**What goes wrong:** cycles among exact rules are found by following targets. A pattern rule such as `/old/*` → `/old/new/*` loops forever, and a regex source cannot be enumerated.
**How to avoid:** at save, simulate. Take the rule's target, or a probe instance of a pattern source with `*` replaced by a probe token. For regex, build a probe string by walking `regexp/syntax` literals where that is possible. Run the full resolver up to N=10 hops: a repeated URL means a loop (refused), more than one hop means a chain (saved with a notice). At runtime the resolver also stops at N hops and answers 404 plus a log line. It never redirects in circles.
### Pitfall 11: Slug changed back and forth
**What goes wrong:** A→B and then B→A produce `A→B` plus `B→A`, a loop that flattening turns into `A→A`.
**How to avoid:** when creating `old→new`, first delete rules whose source is `new`, then flatten, then drop self-loops. Test case: A→B→A→C ends with exactly `A→C` and `B→C`.
### Pitfall 12: Old values of translated slugs are invisible to a column snapshot
**What goes wrong:** Journal's slug is translatable (`Translatable()` includes `"slug"`, with `TranslatableIndexes()` `{"slug"}` [VERIFIED: ../sm-journal-plugin/models/post.go:43-47]). The non-default values live in the translate plugin's table, written through `TranslationWriter.WriteTranslated` [VERIFIED: modules/cabana/field_ml.go:31-33].
**How to avoid:** read `TranslatedExact` for the submitted ML fields before writing them, and emit per-locale `RecordChange`s. The URL-diff approach (Pattern 6) does not depend on this, but the event shape promises old/new values.
### Pitfall 13: Lowercasing paths changes resources
**What goes wrong:** lowercasing non-ASCII letters (`Ł` → `ł`) changes the path, while percent-encoding hex case is equivalent. Blindly lowercasing `%C5%81` to `%c5%81` is harmless, but lowercasing decoded text is not.
**How to avoid:** lowercase only ASCII letters of the decoded path, keep the switch off by default (UI-SPEC Q-7), and test with Polish slugs.
### Pitfall 14: Body and payload caps
**What goes wrong:** widget action payloads are capped at 64 KiB and `Data` at 256 KiB [VERIFIED: modules/pact/capabilities.go:242, 259]. The default HTTP body limit is `default_bytes: 1048576` in the app template [VERIFIED: internal/build/stubs/new.tmpl:26], and pier's `mcp.max_result_bytes` defaults to 1 MiB (pier README).
**How to avoid:** a 60 KB admin import limit (UI-SPEC Q-8), MCP imports in chunks of at most about 900 KB, exports paged.
### Pitfall 15: Route-pattern drift between Nuxt and signpost
**What goes wrong:** the frontend routes live in the layer config, and the same patterns are mirrored in Go config `menu_items.patterns` (grzyby keeps them in `config/menu_items.yaml` [VERIFIED: ../sm-grzybyfunkcjonalne-app/config/menu_items.yaml:1-7]). When they drift, strict URLs 301 valid pages.
**How to avoid:** the resolver's strict-URL check uses only signpost patterns. Add a boot or deploy check in the apps, and note it for the Phase 33 layer (the layer could read patterns from an API).
## Code Examples
### Event listener registration (festival generic method; Go 1.27 generic methods are already used in the codebase)
```go
// Source: modules/festival/bus.go:43-46 signature (verbatim): func (b *Bus) Listen[T any](owner string, fn func(context.Context, T) error)
app.Events.Listen("golem15.seo", func(ctx context.Context, ev *pact.RecordSaved) error {
return p.service.OnRecordSaved(ctx, ev) // URL diff → redirect, flatten, entry update, history(via=auto), purge
})
```
(`pact.RecordSaved` is the proposed type `[ASSUMED]`.)
### Detecting an MCP call for history
```go
// Source: modules/bouncer/dispatch.go:40 — func Dispatched(ctx context.Context, guard string) (*Dispatch, bool)
via := "admin"
if _, ok := bouncer.Dispatched(r.Context(), "backend"); ok {
via = "mcp"
}
```
### Wildcard source → anchored regex (stdlib)
```go
func wildcardRegexp(src string, ignoreCase bool) (*regexp.Regexp, error) {
quoted := regexp.QuoteMeta(src) // "/old/\\*"
expr := "^" + strings.ReplaceAll(quoted, `\*`, `(.*)`) + "$"
if ignoreCase {
expr = "(?i)" + expr
}
return regexp.Compile(expr)
}
```
### robots.txt test with the matching line (grobotstxt)
```go
// Source: pkg.go.dev/github.com/jimsmart/grobotstxt — NewRobotsMatcher, AgentAllowed, MatchingLine
m := grobotstxt.NewRobotsMatcher()
allowed := m.AgentAllowed(robotsBody, agent, absoluteURL)
line := m.MatchingLine() // 0 when no rule matched → E5 "allowedNone"
```
## State of the Art
| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| `noarchive` honoured by Google | "no longer used by Google Search" (cache link removed); Bing still reads it | Google docs, current | Keep the directive (D-08), but the UI should not imply a Google effect [CITED: developers.google.com/search/docs/crawling-indexing/robots-meta-tag] |
| `crawl-delay` | Ignored by Google; supported by Bing/Yandex and Anthropic bots | current | E4 lint message `crawlDelay` is accurate [CITED: Google robots.txt doc; support.claude.com/en/articles/8896518] |
| Ad-hoc robots parsing | RFC 9309 (2022) standard; Google 500 KiB limit | 2022 | E4/Decode 500 KB cap matches [CITED: Google robots.txt doc] |
| WinterCMS `extendFormFields` mutation | Compiled placement at boot (D-91) | this phase | Deterministic, testable order |
**Deprecated/outdated:**
- WinterCMS `viewBag[meta_title]` per-page SEO: replaced by the SEO tab plus URL rules.
- The AI bot list in UI-SPEC C10 omits `Claude-User`, Anthropic's user-initiated fetcher [CITED: support.claude.com/en/articles/8896518]. Other operators' user-fetch agents (for example `Perplexity-User`) are `[ASSUMED]` and should be checked. This goes to Dawid under Q-6.
## Assumptions Log
| # | Claim | Section | Risk if Wrong |
|---|-------|---------|---------------|
| A1 | Go shapes of `AdminFormExtension`, `RecordSaved`/`RecordDeleted`, `ContentDescriptor` and helper names | Patterns 1–3 | One-way API: the shape needs Jakub's review before the release tag |
| A2 | grobotstxt is a faithful, safe port suitable for production | Standard Stack | A wrong tester answer; mitigated by golden tests against Google's documented examples |
| A3 | The framework release after the seams is `v0.6.0` (minor bump over `v0.5.0`) | Release | Tag naming only |
| A4 | Phase 32 delivers the decision endpoint for redirects, normalisation and strict URLs; enforcement in Nuxt is Phase 33 (the layer) | Arch map | If Jakub expects enforcement in 32, a Nitro middleware task is missing |
| A5 | purell (URL normaliser) is archived, so the stdlib is used | Standard Stack | None (stdlib is chosen anyway) |
| A6 | The 404 log is fed by an authenticated server-to-server report from the frontend (a shared token like the site-cache purge), not by a public endpoint | Security | Log flooding if left public |
| A7 | Other AI user-fetch tokens (for example Perplexity-User) | State of the Art | Bot list completeness (Dawid, Q-6) |
| A8 | `sm-seo-plugin` repo creation on git.golem15.com is a human step (`git ls-remote` answered "Cannot find repository") | Environment | Blocks the first push |
## Open Questions
1. **Zones (D-91 vs UI-SPEC Q-1).**
- What we know: the SPA renders one tab zone, and fields.yaml has no `tabs:`/`secondaryTabs:` blocks.
- What's unclear: whether Jakub accepts "tab zone only, other zones are a boot error" for the published API.
- Recommendation: put `Zone` in the struct now (it is one-way API) and accept only the tab zone in Phase 32.
2. **OG image on extension fields (Pitfall 5).**
- Recommendation: an `seoimage` plugin field type with `Upload` (E7), recorded as a UI-SPEC delta. The alternative is extension-owned attachments in D-91.
3. **Journal changes beyond D-96.** The API handlers must emit record events (Pitfall 8), and a `journal-tag` type is needed (Pitfall 9).
- Recommendation: include both in the Phase 32 Journal plan and tell Jakub.
4. **Where Go enforces redirects.** No framework hook exists for "any unmatched path" (surf only wraps ServeMux 404/405 into the error page [VERIFIED: modules/surf/errorpage.go:328-359]). A fourth seam would be needed for Go-served static sites.
- Recommendation: none in Phase 32. The Nuxt layer enforces in Phase 33, and static-site plugins can call the resolver service later.
5. **Verification files at the site root.** Serve their content from `/_seo/api/v1/verification/{name}` and map root paths in the Phase 33 layer (or an nginx regex location).
6. **grobotstxt vs hand-rolled RFC 9309.** Jakub decides; a decision note is needed either way.
7. **Settings tile entry type.** It must be a `SettingsItem.Controller` link to an `AdminSingleton`, not `Form` (Pattern 4). This changes UI-SPEC C2's wording; the rendered result is unchanged.
## Environment Availability
| Dependency | Required By | Available | Version | Fallback |
|------------|------------|-----------|---------|----------|
| Go toolchain | all Go code | ✓ | go1.27.2 | — |
| Node / npm | admin SPA tests, element bundle | ✓ | v24.22.0 / 12.2.1 | — |
| Docker daemon | testcontainers suites | ✓ (`docker ps` works, ryuk running) | 29.9.1 | `sudo -u golem` per CLAUDE.md if the socket is denied |
| PostgreSQL client | manual checks | ✓ | psql 18.6 | testcontainers |
| `sm-seo-plugin` remote repo | first push, app submodules | ✗ | — | Human creates `golem15/sm-seo-plugin` on git.golem15.com |
| graphify | graph context | ✗ (disabled) | — | not needed |
| msd package-legitimacy for Go | Go dependency audit | ✗ (npm/pypi/crates only) | — | Manual proxy.golang.org check + decision note |
**Missing dependencies with no fallback:** the remote repository for `sm-seo-plugin` (human checkpoint).
**Missing dependencies with fallback:** Go legitimacy seam (manual check).
## Validation Architecture
### Test Framework
| Property | Value |
|----------|-------|
| Framework | Go `testing` (+ testify), testcontainers-go v0.44.0 Postgres; vitest 3.2.7 + happy-dom 20.11.6 for admin and elements |
| Config file | none for Go; `admin/vitest.config.ts`; `sm-seo-plugin/vite.config.ts` (Wave 0) |
| Quick run command | `go test -short ./modules/cabana ./modules/pact ./modules/signpost` and `go -C ../sm-seo-plugin test -short ./...` |
| Full suite command | `go vet ./... && go test ./...` (framework), the same in `sm-seo-plugin`, `sm-journal-plugin`, `sm-demo-app`; `npm --prefix admin run test`; `npm --prefix ../sm-seo-plugin run test`; `scripts/check-phase32.sh --all` |
### Phase Requirements → Test Map
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| SC-1 | C after A → A, C, B; C before first; new tab between tabs; two plugins on one anchor keep order; hide field; default-tab-first; span preserved | unit (pure compile, no DB) | `go test -short ./modules/cabana -run 'TestFormExtension'` | ❌ Wave 0 |
| SC-1 | Boot errors: unknown field/tab anchor, anchor outside `Requires()`, duplicate field/tab, cycle, `before` default tab, unsupported zone, hide of referenced field | unit | `go test -short ./modules/cabana -run 'TestFormExtensionBootErrors'` | ❌ Wave 0 |
| SC-1 | Extension Load/Save in the save tx: rollback on Save error, 422 per field, notices per field, owner columns untouched, hidden extension skipped on save | integration (testcontainers) | `go test ./modules/cabana -run 'TestFormExtensionSave'` | ❌ Wave 0 |
| SC-1 | SPA renders compiled order A, C, B; tab-less form + extension tab renders "Basic", then the new tab (UI-SPEC C1 backstop) | vitest | `npm --prefix admin run test -- tests/form/FormView.test.ts` | ✅ file exists, add cases |
| SC-2 | D-93 descriptor optional fields: types without `Content` unchanged; `MatchPath` maps the layered pattern → record | unit | `go test -short ./modules/signpost ./modules/pact -run 'TestContent'` | ❌ Wave 0 |
| SC-2 | Precedence URL rule > record > template/global > derived, with per-value source; staging lock forces noindex | unit (pure resolver with fakes) | `go -C ../sm-seo-plugin test -short ./classes -run TestResolverPrecedence` | ❌ Wave 0 |
| SC-2 | SEO tab on posts, categories and tags via a fake content plugin; Journal never imports sm-seo-plugin | integration + gate grep | `go -C ../sm-seo-plugin test ./... -run TestSEOTab`; `check-phase32.sh` greps `../sm-journal-plugin` for `sm-seo-plugin` | ❌ Wave 0 |
| SC-2 | Demo app boots with journal + seo; `golem15.journal.posts` form schema contains the SEO tab after Manage | integration (app) | `go -C ../sm-demo-app test ./... -run TestSEOTabMounted` | ❌ Wave 0 |
| SC-3 | Redirect graph: flatten A→B,B→C ⇒ A→C; slug back-and-forth; loop refusal for exact/pattern/regex probes; chain notice; N-hop runtime stop | unit (pure) | `go -C ../sm-seo-plugin test -short ./classes -run 'TestRedirectGraph'` | ❌ Wave 0 |
| SC-3 | cabana save changing slug fires `RecordSaved` after commit (not on rollback); listener creates 301 | integration | `go test ./modules/cabana -run TestRecordEvents` and `go -C ../sm-seo-plugin test ./... -run TestSlugChange301` | ❌ Wave 0 |
| SC-3 | Journal API `PUT /_journal/api/v1/posts/{id}` slug change emits the event (MCP path) | integration | `go -C ../sm-journal-plugin test ./... -run TestAPIRecordEvents` | ❌ Wave 0 |
| SC-3 | Loops refused from admin (422), MCP API (4xx tool error), CSV import (refused row) | integration | `go -C ../sm-seo-plugin test ./... -run 'TestLoopRefused'` | ❌ Wave 0 |
| SC-4 | Every SEO admin operation has a tool: compile pier catalog in a tagged test app; tool route list matches the declared operation list | integration (`-tags summercms_mcp`) | `go -C ../sm-demo-app test -tags summercms_mcp ./... -run TestSEOToolsCatalog` | ❌ Wave 0 |
| SC-4 | Batch dry-run returns the diff and conflicts and writes nothing; apply all-or-nothing; undo change and batch; stale undo refused | integration | `go -C ../sm-seo-plugin test ./... -run 'TestBatch|TestUndo'` | ❌ Wave 0 |
| SC-4 | Element backstops (E1 thresholds, E2 grammar round-trip, E3 effective title + stem matching, E4 lint rules, E6 apply gating) | vitest | `npm --prefix ../sm-seo-plugin run test` | ❌ Wave 0 |
| SC-5 | Framework tag exists and consumers build against it; docs checker green | gate | `scripts/check-phase32.sh --all`, `summer docs:build --check`, `go test ./cmd/summer -run TestDocsTree` | ❌ Wave 0 |
### Sampling Rate
- **Per task commit:** `go vet ./... && go test -short ./...` in the touched repo (+ `npm --prefix admin run test` if `admin/` changed).
- **Per wave merge:** the full `go test ./...` with Docker in each touched repo, `scripts/check-admin-openapi.sh --check` if cabana payloads changed.
- **Phase gate:** `scripts/check-phase32.sh --all` green (self-test, go, plugins, consumers, docker, admin, docs, marker-style "framework names no SEO / no consuming app" grep) before `/msd-verify-work`.
### Wave 0 Gaps
- [ ] `modules/cabana/form_extension_test.go`: placement and boot-error tables (fake plugins, `CompileFormWithTypes` fixtures in `testdata/`)
- [ ] `modules/cabana/record_events_test.go`: after-commit and rollback behaviour (testcontainers, the existing cabana TestMain)
- [ ] `modules/signpost` content-descriptor tests
- [ ] `sm-seo-plugin/testmain_test.go` (copy sm-sitemap-plugin's: `flag.Parse()` + `testing.Short()`), plus a fake content plugin fixture registering a `MenuItemType` with `Content` and an admin controller
- [ ] `sm-seo-plugin/package.json`, `vite.config.ts`, `vitest` setup (copy the journal Blog Editor)
- [ ] `scripts/check-phase32.sh` modelled on `scripts/check-phase19.7.sh` (stages self-test/go/plugins/consumers/docker/admin/docs), with a grep stage: no `seo`/`SEO` identifiers added under `modules/` (D-90), and no consuming app named
- [ ] Fixture: a `journal-tag` menu item type test in sm-journal-plugin
## Security Domain
### Applicable ASVS Categories
| ASVS Category | Applies | Standard Control |
|---------------|---------|-----------------|
| V2 Authentication | no (reuses cabana backend guard and pier identity) | `bouncer` backend guard; pier OAuth/personal tokens |
| V3 Session Management | no | existing |
| V4 Access Control | yes | `golem15.seo.manage` on every admin controller, extension (`Permissions`), `/_seo/api/v1` management route and MCP tool (`Permissions` + route check); read grants see only GET tools (pier ceiling) |
| V5 Input Validation | yes | Field-type `Decode` (`FormFieldError`), `lagoon.Validate` rules; URL sources must start with `/`; targets path or http(s) only; regex compiled with stdlib and length-capped; robots.txt ≤ 500 KB; verification file names `^[A-Za-z0-9._-]+\.(html|txt|xml)$` without slashes |
| V6 Cryptography | limited | HMAC import tokens via stdlib `crypto/hmac` keyed by the app key; never hand-roll |
| V12 Files | yes | CSV size caps; OG image upload through the framework upload path and its limits |
| V13 API | yes | Public `resolve` answers only published content; throttled; JSON only |
| V14 Configuration | yes | Non-production lock decided by `app.Config.Environment()` (used the same way in surf/cabana [VERIFIED: modules/surf/router.go:732]); cannot be disabled from the admin |
### Known Threat Patterns for this stack
| Pattern | STRIDE | Standard Mitigation |
|---------|--------|---------------------|
| Open redirect via pass-through query or regex capture into an external host | Tampering | Targets set only by authorized editors; capture substitution only into the path part; external targets only when the rule's target literal is absolute |
| ReDoS from admin- or MCP-supplied regex | DoS | stdlib RE2 (linear); cap pattern length and rule count; precompile once |
| Host-header injection into canonical or absolute URLs | Spoofing | Build from configured site URL, never `r.Host` |
| Draft leakage through `resolve` | Info disclosure | Descriptor `Published` gate; unpublished → 404 semantics |
| 404-log flooding | DoS | Server-to-server report with a shared token (A6), URL length cap, row cap, throttle |
| CSV formula injection on export | Tampering | Prefix cells starting with `= + - @ \t \r` with `'` |
| Stored XSS via SEO strings in admin partials | Tampering | `html/template` escaping and cabana's partial sanitizer (`sanitizePartialNodes` [VERIFIED: modules/cabana/partial_render.go:144]) |
| Staging copy indexed | Info disclosure | Forced `Disallow: /` + `X-Robots-Tag: noindex` by environment, not by stored setting (D-22) |
| Privilege bypass by posting hidden extension fields | Elevation | Pitfall 7: extension visibility enforced on save |
## Release and Sequencing Notes (SC-5)
- Framework tags so far: `v0.5.0` is the latest [VERIFIED: `git tag --sort=-v:refname`]. The seams are additive, so recommend `v0.6.0` [ASSUMED].
- Order: framework seams (with `replace` during development) → sm-journal-plugin descriptors and event emission → `sm-seo-plugin` → sm-demo-app mounts both (the demo has MCP config `config/mcp.yaml` and mounts journal and sitemap [VERIFIED: ../sm-demo-app/summer.yaml, ../sm-demo-app/config]) → unit-test plan → framework tag (if `admin/` changed: rebuild dist, commit, `check-admin-dist.sh`) → plugins require the tag → app pointer bumps → changelog post. Push, tag and deploy need Jakub's go-ahead.
- Suggested lean plan split (for the plan-count confirmation with Jakub):
1. Framework D-91 form extensions (pact, cabana compile/save/show/schema, README/docs, SPA backstop test).
2. Framework D-92 record events + D-93 descriptors (pact, cabana emit points, signpost helpers, docs); Journal adoption (descriptors for post/category/tag, `journal-tag`, API event emission).
3. sm-seo-plugin core: repo, models/migrations, settings singleton + per-type tabs, SEO tab extension, URL rules, resolver + public API, generate job, history core, elements E1–E3 (+E7 if chosen).
4. Redirects, 404 log, normalisation, strict URLs, slug-change 301, robots.txt + AI bots + verification + staging lock, elements E4–E5.
5. `/_seo/api/v1` management API + MCP tools, batch dry-run, undo, CSV import/export (E6), demo mount, `check-phase32.sh`.
6. Unit tests (last plan), framework tag, plugin tags, pin bumps.
## Sources
### Primary (HIGH confidence)
- Read this session: `modules/pact/capabilities.go`, `modules/pact/mcp.go`, `modules/cabana/{action_extensions,crud,extension,form_schema,schema_types,settings,field_ml,field_type,field_file,http,registry,subject_events}.go`, `modules/lagoon/transaction.go`, `modules/festival/bus.go`, `modules/signpost/signpost.go`, `modules/party/registry.go`, `modules/surf/{router,errorpage}.go`, `modules/pier/{README.md,catalog.go}`, `modules/bouncer/dispatch.go`, `modules/cooler/sitecache.go`, `admin/src/views/FormView.vue`, `admin/src/components/form/formState.ts`
- Sibling repos: `sm-journal-plugin` (routes, mcp, models, menu item types, fields.yaml, package.json), `sm-sitemap-plugin` (plugin, routes, admin, go.mod, testmain), `sm-grzybyfunkcjonalne-app` (nginx, menu_items, nuxt.config), `sm-summercmsio-app` (site plugin routes), `sm-demo-app/summer.yaml`
- Reference: mpseo `classes/SeoInjectorManager.php`, `models/Entry.php`, `classes/PlaceholderResolver.php`
- proxy.golang.org (grobotstxt, temoto/robotstxt versions and go.mod); npm registry (`npm view` + package-legitimacy seam)
### Secondary (MEDIUM confidence)
- developers.google.com/search/docs/crawling-indexing/robots/robots_txt (500 KiB, longest match, 5xx behaviour, crawl-delay)
- developers.google.com/search/docs/crawling-indexing/robots-meta-tag (directives, noarchive status, X-Robots-Tag syntax)
- pkg.go.dev/github.com/jimsmart/grobotstxt (API: MatchingLine, ParseHandler)
- support.claude.com/en/articles/8896518 (ClaudeBot, Claude-User, Claude-SearchBot)
### Tertiary (LOW confidence)
- github.com/temoto/robotstxt README (comparison only)
## Metadata
**Confidence breakdown:**
- Standard stack: HIGH, since nearly everything is existing framework code; the one new dependency is MEDIUM.
- Architecture: MEDIUM-HIGH. The seam insertion points are verified; the Go shapes are recommendations needing Jakub's API review.
- Pitfalls: HIGH. Each one is backed by a verified line in the current code.
**Research date:** 2026-10-26
**Valid until:** 2026-11-25 (code seams move fast with parallel sessions in cabana; recheck `cabana/http.go` Activate order before planning)