diff --git a/.planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md b/.planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md new file mode 100644 index 00000000..5ddf7a32 --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md @@ -0,0 +1,877 @@ +# Phase 33: SEO output, audit and content - Research + +**Researched:** 2026-10-28 +**Domain:** SEO output across a Go headless CMS (compiled plugins) and Nuxt 4 layers: sitemaps, structured data, hreflang, pagination, headings, breadcrumbs, SSR guarantee, crawler audit, internal linking, media +**Confidence:** MEDIUM overall. Code seams are HIGH (read this session). External specs are cited from official docs (the classify seam rates WebFetch LOW, so they are tagged `[CITED]`). Several design choices are recommendations that need Jakub's confirmation; they are listed in the Assumptions Log and Open Questions. + + +## User Constraints (from CONTEXT.md) + +Source: `.planning/phases/33-seo-output-audit-and-content/33-CONTEXT.md`, copied verbatim. The full decision text is in `.planning/phases/32-seo-foundation/32-CONTEXT.md`, which wins when the two disagree. + +### Locked Decisions + +#### Planning (Jakub, 2026-10-25) +- **About 12–15 smaller plans: an exception to the lean-planning rule.** The planner must not merge them into 3–4 plans. The other project rules still apply: the plan list with a one-line scope each is shown to Jakub before PLAN.md files are written, and unit tests are the last plan. +- **Mode:** mvp (ROADMAP.md). +- **The framework still carries no SEO code (D-90).** If Phase 33 needs a new framework seam, it must be generic and additive (D-106 rules: patch tag, no breaking change); D-103 rules out a fourth seam for redirects and verification files. + +#### Decisions implemented entirely in Phase 33 +Section numbers refer to 32-CONTEXT.md. + +| ID | Decision | 32-CONTEXT section | +|----|----------|--------------------| +| D-30 | Sitemap index split by type, 1000-URL files, real `lastmod` | 5 | +| D-31 | Image, News sitemaps, IndexNow | 5 | +| D-32 | The new format replaces PHP parity (breaking, confirmed); the XSLT todo goes with it | 5 | +| D-33 | Page registry in the CMS | 5 | +| D-34 | HTML sitemap | 5 | +| D-40 | URL form chosen per site | 6 | +| D-41 | Page 2 and later: robots, suffixes, H1, pillar text | 6 | +| D-42 | Numbered server-rendered links, rel prev/next, 301 and 404 rules | 6 | +| D-43 | One JSON-LD `@graph` | 7 | +| D-44 | Schema type, type-dependent fields, custom JSON-LD | 7 | +| D-45 | Publisher in the global settings | 7 | +| D-46 | Validation in admin, crawler and Rich Results link | 7 | +| D-50 | No H1 in body content, including Journal's conversion (D-96) | 8 | +| D-51 | Live heading outline and crawler checks | 8 | +| D-52 | One heading convention in the layers | 8 | +| D-55 | hreflang in head and sitemaps from one source | 9 | +| D-56 | `x-default` points to the default locale | 9 | +| D-57 | No hreflang for a missing translation | 9 | +| D-70 | Indexable pages always SSR or SSG; build guard; raw versus rendered check | 10 | +| D-71 | Real status codes, server-rendered links, content in HTML, image rules | 10 | +| D-72 | `sm-bm-app` gets noindex and `Disallow` only | 10 | +| D-80 | One link per post card | 11 | +| D-81 | No generic anchors | 11 | +| D-82 | Related posts, phrase auto-linking, suggestions, orphan report | 11 | +| D-83 | Outbound link rules | 11 | +| D-85 | Media SEO | 12 | +| D-86 | Breadcrumbs component and primary category | 12 | +| D-95 | `sm-sitemap-plugin` with its own filter interface that SEO implements | 0 | +| D-97 | One shared Nuxt SEO layer | 0 | + +#### Decisions split between Phases 32 and 33 (Phase 33 part only) +Phase 32 delivered the rest of each; see the Phase 32 column of the decisions map. + +| ID | Decision | Phase 33 part | +|----|----------|---------------| +| D-03 | Global settings | Organisation or person data for JSON-LD (with D-43, D-45) | +| D-05 | Search engine verification | Serving the files at the site root from the layer (D-103) | +| D-08 | Meta robots, `X-Robots-Tag` | Sending the header on frontend-rendered pages and files (layer, D-97) | +| D-10 | All SEO manageable through MCP on `pier` | Tools for the crawler and the other Phase 33 screens | +| D-12 | Batch tools with `dry_run` | "Noindex on a page in a sitemap" conflict refined by the D-95 filter | +| D-14 | Issue report: CMS data checks plus a crawler | The built-in crawler and its checks | +| D-20 | 404 log | The layer's call to the report endpoint | +| D-20a | Where redirects execute | Nitro middleware in the layer (D-97, D-103) | +| D-20b | URL normalisation per site | Enforcement on frontend routes through the layer | +| D-20c | Strict URLs | Layer routes reading their patterns from the CMS | +| D-21 | Default robots per page kind | Pagination page 2+ (D-41) and signed-in routes (D-23) in the layer | +| D-22 | robots.txt | The `Sitemap:` default moves to the D-30 index | +| D-23 | Private frontend pages noindex | Automatic marking of signed-in routes in the layer and the user plugin's auth layer | +| D-24 | noindex, robots.txt and sitemaps consistent | Noindex, canonicalised and redirected pages leave the sitemaps; "sitemap URL blocked" warning | +| D-25 | AI bot switches, optional llms.txt | llms.txt generated from the sitemap | +| D-93 | Content descriptors in signpost | Sitemaps and the page registry consume `Records` (D-30, D-33) | +| D-94 | `sm-seo-plugin` scope | Page registry, llms.txt, JSON-LD `@graph` and breadcrumbs APIs, crawler | +| D-96 | Content plugins keep only their own fixes | Journal's H1 to H2 conversion (D-50) and schema types (D-43) | +| D-103 | Redirect decision endpoint and verification-file API only | Layer enforcement of redirects and root mapping of verification files | + +### Claude's Discretion +- HTML sitemap grouping and paging (D-34). +- Crawler implementation: Go, an in-process job through `conga`/River, politeness limits (D-14). +- Table and package names inside the plugins. +- Site-cache purge after SEO writes: Phase 32 serves effective SEO through one resolver endpoint per URL (P-15) but does not purge. Once the D-97 layer caches the resolve answer, purging through the existing site-cache purge becomes needed; plan it with D-97. + +### Deferred Ideas (OUT OF SCOPE) +- From 32-CONTEXT: social card preview, per-redirect hit counters and validity dates, 307/308 redirects, e-commerce structured data. +- `sitemap-xslt-browser-deprecation.md` (g15office w8kupqi3) stays a separate backlog task; the D-32 rewrite touches the same code, so the plan should say whether the XSL output is removed or kept. + +Also out of scope (33-CONTEXT domain): e-commerce structured data and shop pages; migrating `sm-bm-app` to SSR (D-72 gives it noindex and `Disallow` only). + + + +## Phase Requirements + +No REQUIREMENTS.md IDs are assigned (ROADMAP: "Requirements: TBD"). The coverage list is the decision IDs of 33-CONTEXT. Plan numbers refer to the suggested breakdown below. + +| ID | Description | Research support (section) | Suggested plan | +|----|-------------|----------------------------|----------------| +| D-30 | Index split by type, 1000-URL files, real lastmod | Sitemap rewrite: slot table, lastmod store | 33-03 | +| D-31 | Image, News sitemaps, IndexNow | Sitemap extras; specs cited | 33-04 | +| D-32 | New format replaces PHP parity; XSL decision | Sitemap rewrite; XSL removal recommended (Chrome 158) | 33-03 | +| D-33 | Page registry | SEO plugin registry + signpost type + layer route report | 33-05, 33-08 | +| D-34 | HTML sitemap | Sitemap JSON groups API + layer page | 33-04, 33-08 | +| D-40/41/42 | Pagination form, page 2+ rules, numbered links, 301/404 | Resolve answer `pagination` + layer composable | 33-06, 33-08, 33-09 | +| D-43/44/45/46 | @graph, schema type and fields, publisher, validation | Structured-data builder in SEO plugin; descriptor fields | 33-01, 33-06 | +| D-50/51/52 | No H1 in body, outline, heading convention | Journal gate + editor; layer htmlGate h1→h2; layer tests | 33-02, 33-09 | +| D-55/56/57 | hreflang head + sitemap, x-default, missing translations | Resolve `alternates`; sitemap xhtml:link | 33-03, 33-06, 33-07 | +| D-70 | SSR/SSG only for indexable routes; build guard; raw vs rendered | Nuxt module guard; crawler raw-HTML check | 33-08, 33-10 | +| D-71 | Real status codes, server links, content in HTML, image rules | Layer middleware + components; crawler checks | 33-07, 33-09, 33-10 | +| D-72 | sm-bm-app noindex + Disallow | Static robots.txt + meta + header | 33-12 | +| D-80/81 | One link per card, no generic anchors | PostCard change; editor warning; crawler check | 33-02, 33-09, 33-10 | +| D-82 | Related posts, auto-links, suggestions, orphan report | Autolink table + layer transform; crawler link graph | 33-11, 33-09 | +| D-83 | Outbound link rules | Editor rel checkboxes; domain list; layer transform | 33-02, 33-11, 33-09 | +| D-85 | Media SEO | Editor alt/decorative; missing-alt report; bulk alt tool; formats decision | 33-02, 33-11 | +| D-86 | Breadcrumbs + primary category | Resolve `breadcrumbs`; layer component | 33-06, 33-08 | +| D-95 | Sitemap filter interface implemented by SEO | `app.Publish[sitemap.Filter]` | 33-03, 33-05 | +| D-97 | One shared Nuxt SEO layer | New `vue-seo-layer` repo | 33-07, 33-08 | +| D-03 (33 part) | Organisation/person data | Publisher settings | 33-06 | +| D-05 (33 part) | Verification files at site root | Layer root mapping | 33-07 | +| D-08 (33 part) | X-Robots-Tag on frontend pages and files | Layer middleware header | 33-07 | +| D-10 (33 part) | MCP tools for Phase 33 screens | `AdminOperations` parity test extended | 33-05, 33-10, 33-11 | +| D-12 (33 part) | Batch conflict refined by filter | `noindex_published` conflict reads the filter | 33-05 | +| D-14 (33 part) | Crawler | conga job, fetchguard, x/net/html | 33-10 | +| D-20, D-20a/b/c (33 part) | 404 report call, redirect middleware, normalisation, strict URLs | Layer middleware; route drift check | 33-07, 33-08 | +| D-21, D-23 (33 part) | Page 2+ robots; signed-in routes noindex | Layer route meta + middleware | 33-08 | +| D-22 (33 part) | `Sitemap:` → index | Index stays at `/sitemap.xml` | 33-03, 33-05 | +| D-24 (33 part) | Sitemaps exclude noindex/canonicalised/redirected; "sitemap URL blocked" | Filter + issue kind | 33-05 | +| D-25 (33 part) | llms.txt from the sitemap | SEO route reading sitemap URL set | 33-05 | +| D-93 (33 part) | Sitemaps and registry consume Records | Rebuild job over `signpost.Records` | 33-03 | +| D-94 (33 part) | Registry, llms.txt, @graph, breadcrumbs APIs, crawler | SEO plugin plans | 33-05, 33-06, 33-10 | +| D-96 (33 part) | Journal H1→H2, schema types | Journal plan + descriptor kind | 33-02 | +| D-103 (33 part) | Layer enforcement of redirects, root verification files | Layer middleware | 33-07 | + + +## Project Constraints (from CLAUDE.md) + +From `summercms.go/CLAUDE.md`, `../CLAUDE.md` and the user's global instructions. These carry the authority of locked decisions. + +- **No co-author tags in commits.** One logical change per commit; planning docs and code in separate commits. Stage files by explicit path, never `git add -A`; several sessions work concurrently (leave other sessions' uncommitted `STATE.md`, `state.json`, submodule drift alone). +- **Don't push, tag or deploy without Jakub's go-ahead.** Every pushed framework release tag `vX.Y.Z` gets a changelog post on summercms.io the same day (`../sm-summercmsio-app/scripts/changelog-post.py`; never print, copy or mint the `g15_` token). +- **Framework code never names a consuming application**, and (D-90) never names SEO. New framework seams are generic and additive (D-106: patch tag). +- **Dependencies: standard library first.** A new Go or npm dependency needs a research doc or a decision note naming it (`.planning/research/STACK.md` holds the decided set). YAML via `goccy/go-yaml`, never `gopkg.in/yaml.v3`. +- **Docs rules:** a change to a module's exported API, config keys, CLI commands or dependencies updates that module's README and the affected `docs/` pages in the same commit; the docs checker (`summer docs:build --check`, `TestDocsTree`) enforces identifiers. Plugin READMEs follow the standard structure. +- **Core plugins live in their own repos**, mounted in apps as submodules at `plugins//`; commit and push the plugin first, then the pointer bump in the app as a separate commit. Consumers `require` a tagged framework version and `replace` it with `../summercms.go` during development. +- **Core plugins (user, blog, pages, payment) work across many projects: avoid breaking changes unless asked.** D-32 is the explicitly confirmed breaking change for `sm-sitemap-plugin`; no other breaking change is authorised. +- **Porting rule:** the existing frontend's requests define the contract; response shapes are not "improved". Phase 33 adds fields; it must not change existing Journal/SEO answer shapes. +- **Tests:** keep `go vet ./...` and `go test ./...` green at every commit; Docker tests skip under `-short` via `testing.Short()` after `flag.Parse()`. +- **MSD rules:** plan list with one-line scopes shown to Jakub before PLAN.md files; unit tests are the last plan; STATE.md only through `msd_run query state.*`; g15office comment when the phase advances. +- **Memory:** prefer the maintained library plus a decision note for protocol and security code; additive framework changes are a patch tag (0.5.x), never hard-code the version; create new repos with `tea`. + +## Summary + +Phase 32 left a working backend (sm-seo-plugin v0.1.0: resolver, redirects, robots.txt, MCP, CMS-data issues) and three framework seams, but nothing on the frontend consumes them and the sitemap plugin is still the Phase 17 PHP-parity single file. Phase 33 spans **ten repositories**: framework, four Go plugins (sitemap, seo, journal, optionally faq), a new Nuxt layer plus three existing layers, and four apps. The dominant planning problem is therefore **release ordering across repos**, not any single technique: framework tag → Journal → sitemap → SEO → `vue-seo-layer` → journal/menu/FAQ layers → apps. Apps pin layers by git tag (pnpm) and plugins by Go tag, so app plans either wait for tags or develop against local links and re-pin in the final plan. + +Two prerequisites block a clean start. (1) Phase 32's post-review fixes are **unreleased**: framework is 18 commits past v0.5.1 (CR-01, CR-02, CR-04, WR-01, WR-02, WR-07), sm-seo-plugin 21 commits past v0.1.0 (CR-03 plus a new migration), Journal 3 commits past v0.2.5. 32-UAT test 3, the release decision, is still pending. (2) The D-93 descriptor (`pact.ContentRecord` / `ContentLocale`) lacks what JSON-LD, breadcrumbs, the image sitemap and the News sitemap need: a publication date, a parent trail, authors, body images and a content kind. Phase 33 therefore needs one more **generic, additive** framework seam. It can ship in the same v0.5.2 patch as the unreleased fixes. + +Technically the phase splits into well-understood pieces: +- **XML sitemaps** are a stored-slot rewrite: a persistent URL→file-number table gives stable numbering, and record events give real `lastmod`. +- **The resolver** gains additive answer fields: `alternates`, `jsonld`, `breadcrumbs`, `pagination`, `autolinks`. +- **The new `vue-seo-layer`** renders that answer with Nuxt core `useHead`/`useSeoMeta`, enforces redirects and 410 in a Nitro middleware, proxies root files to Go, and fails the build on an indexable `ssr: false` route. +- **The crawler** is a conga job on the existing `fetchguard` client, parsing HTML with `golang.org/x/net/html`. + +The riskiest items are product decisions, not code: +- D-70 asks for a raw-versus-rendered comparison, which needs a headless browser. +- D-85 asks for WebP/AVIF, which needs a new dependency. +- Success criterion 3 says every crawler issue is fixable through MCP, which cannot hold for layout bugs. +- grzybyfunkcjonalne.pl moves a live production site several versions at once. + +**Primary recommendation:** +- Start with a Wave-1 plan that ships the additive descriptor seam together with the pending Phase 32 fixes as framework v0.5.2. +- Build `vue-seo-layer` without `@nuxtjs/seo` and have journal, menu and FAQ extend it. +- Keep `/sitemap.xml` as the index URL so GSC and robots.txt need no URL change. +- Drop the XSL output: Chrome stops supporting XSLT on 2026-11-17. + +## Architectural Responsibility Map + +| Capability | Primary tier | Secondary tier | Rationale | +|------------|-------------|----------------|-----------| +| Sitemap index, per-type files, image/News, IndexNow pings | API/Backend (`sm-sitemap-plugin`, Go) | Frontend server (proxy of `/sitemap*.xml` when nginx does not route them) | D-95: sitemap owns generation from signpost; Go has the records and events | +| Exclusion of noindex/canonicalised/redirected URLs from sitemaps | API/Backend (`sm-seo-plugin` implements `sitemap.Filter`) | — | D-95: sitemap knows nothing about SEO | +| Page registry (D-33) | API/Backend (`sm-seo-plugin`, signpost type) | Frontend server (reports routes at server start) | Registry is CMS data; Nuxt knows its static routes | +| Effective SEO, JSON-LD @graph, breadcrumbs, hreflang, pagination metadata | API/Backend (resolver) | Frontend server (renders into ``, adds layer-owned nodes such as FAQPage) | D-43/D-94: graph generated in CMS; one resolver call per URL (P-15) | +| Redirect/410 enforcement, X-Robots-Tag on pages | Frontend server (Nitro middleware in `vue-seo-layer`) | API/Backend (decision) | D-20a/D-103: CMS decides, the serving tier enforces | +| Verification files, robots.txt, llms.txt, IndexNow key file at site root | API/Backend (served by Go) | Frontend server (root proxy) | D-103: no fourth framework seam; the layer maps the root | +| SSR build guard | Frontend build (Nuxt module in `vue-seo-layer`) | — | Only the Nuxt build knows routeRules and `.client.vue` pages | +| Visible breadcrumbs, pagination links, heading convention, one link per card | Browser/Client markup rendered by Frontend server (SSR) | — | D-71: all navigation server-rendered `` | +| H1→H2 conversion of stored content | API/Backend (Journal write gate) | Frontend server (htmlGate demotes h1 as defence in depth) | D-50/D-96: the content plugin fixes its own data | +| Phrase auto-links, outbound rel rules | Frontend server (body HTML transform at SSR) | API/Backend (phrase table and domain list in resolve answer) | Keeps Journal unaware of SEO (D-96); output is server-rendered HTML | +| Crawler, issue report, orphan report, link suggestions | API/Backend (`sm-seo-plugin` conga job) | — | D-14: Go, in-process job | +| Media alt / decorative / file-name slug in the editor | API/Backend + admin element (Journal editor) | — | The editor and its media store are Journal's | +| WebP/AVIF/srcset generation | Undecided (see Open Question 3) | — | Needs a new dependency in Go, or a Nuxt image pipeline | + +## Prerequisites (resolve before or in the first plan) + +1. **Unreleased Phase 32 fixes.** Framework `git log v0.5.1..HEAD` lists the fixes CR-01 (`676ca53`), CR-02 (`5471b66`), CR-04 (`d660cd0`), WR-01 (`468af06`), WR-02 (`1583651`) and WR-07 (`0e187cf`). sm-seo-plugin has 21 commits after v0.1.0, including CR-03 (`2d81c26`) and migration `202610280001_add_golem15_seo_redirects_source_key`. sm-journal-plugin has 3 commits after v0.2.5 [VERIFIED: git log in each repo, this session]. 32-UAT test 3 (release or hold) and test 4 (the meaning of `RecordSaved.Locale`, WR-02) are pending. Phase 33 builds on tags (Q-01). Recommendation: fold the release of these fixes into 33-01, together with the new additive seam, as framework **v0.5.2**, sm-seo-plugin **v0.1.1** and Journal **v0.2.6** or later. The exact numbers are read from `git tag` at release time. +2. **UI design contract.** `.planning/config.json` has `workflow.ui_phase: true`. Phase 33 adds admin screens: crawler runs and issues, page registry, auto-links, sitemap settings, publisher and schema fields in the SEO tab. It also adds layer components: breadcrumbs, pagination, HTML sitemap. A `33-UI-SPEC.md` following 32-UI-SPEC conventions (E-elements, C-contracts) is expected before plans that build these screens [VERIFIED: .planning/config.json read via shell]. +3. **Release cadence decision (Open Question 1).** Can intermediate tags be pushed mid-phase, so app plans pin real tags? Or do app plans work against local links and the final plan tags and re-pins everything, as Phase 32 did with `32-05-demo-mount.patch`? + +## Standard Stack + +No new runtime dependency is required for the recommended MVP scope. Every item below is already in the decided set or already in the repos. + +### Core (Go) +| Library / module | Version | Purpose | Why standard | +|---------|---------|---------|--------------| +| SummerCMS `signpost` | v0.5.1 + additive v0.5.2 fields | `ContentTypes`, `Records`, `Describe`, `MatchPath` feed sitemaps, registry, resolver | D-93 seam [VERIFIED: modules/signpost/content.go func list] | +| SummerCMS `conga` | framework | Sitemap rebuild, IndexNow ping, crawl jobs; `pact.HasSchedule` for nightly runs (`conga.Daily`, `conga.Every`) | Decided job system [VERIFIED: modules/conga/README.md] | +| SummerCMS `fetchguard` | framework | Crawler and IndexNow HTTP: no redirect following, body cap, timeout, host allow-list | Framework SSRF guard [VERIFIED: modules/fetchguard/README.md] | +| SummerCMS `cooler` | framework | `cooler.PurgeSiteCacheAfterCommit` after SEO writes (keys `site_cache.purge_url`, `site_cache.purge_token`) | Existing site-cache purge [VERIFIED: modules/cooler/sitecache.go:26,29] | +| `golang.org/x/net/html` | already required by sm-journal-plugin | Crawler HTML parsing, auto-link/heading transforms in Go | x/ family, already in the dependency graph [VERIFIED: sm-journal-plugin/classes/format_html.go imports `xhtml "golang.org/x/net/html"`] | +| `github.com/jimsmart/grobotstxt` | v1.0.3 | "Sitemap URL blocked" and crawler robots checks | Decided D-99 [VERIFIED: .planning/research/STACK.md line 50] | +| stdlib `encoding/xml` | Go 1.27 | Sitemap XML escaping (`xml.EscapeText`) now that byte parity with PHP is dropped | Standard library first | +| stdlib `encoding/json` | — | JSON-LD graph building and custom JSON-LD validation | Standard library first | + +### Core (Nuxt) +| Library | Version | Purpose | Why standard | +|---------|---------|---------|--------------| +| `nuxt` | 4.4.8 (pinned across layers) | `useHead`/`useSeoMeta` (core unhead), routeRules, Nitro middleware, module hooks | Existing pin [VERIFIED: vue-journal-layer/package.json] | +| `unhead` | 2.1.17 (via Nuxt) | Escapes `<` in JSON script `innerHTML` (`<`), so JSON-LD cannot break out of `` escaping | unhead `useHead({ script: [{ type: 'application/ld+json', innerHTML: graph }] })` | unhead escapes `<` as `<` for json types | +| Background work / schedules | goroutines + tickers | `conga` jobs + `pact.HasSchedule` | Leader election, unique periods, progress records | +| Cache purge of rendered pages | new purge endpoint | `cooler.PurgeSiteCacheAfterCommit` + existing `/api/__site__/cache/purge` | Already wired to every site | +| Redirect decision in the layer | re-implementing rules in TS | `GET /_seo/api/v1/resolve` | One source of truth (D-103) | +| Head tags | `@nuxtjs/seo` modules | Nuxt core `useHead`/`useSeoMeta` | Core composables; no competing graph | + +**Key insight:** in this phase the CMS already holds every decision. Each frontend piece that "computes SEO" itself is a second source of truth that will drift: hreflang from i18n, canonical from seo-utils, schema from nuxt-schema-org, sitemap from @nuxtjs/sitemap. The layer should only render the answer and enforce it. + +## Runtime State Inventory + +D-32 is a migration of live sites, so this inventory applies. + +| Category | Items found | Action required | +|----------|-------------|-----------------| +| Stored data | `golem15_sitemap_definitions` rows (grzyby, demo): `url` items and registry items with `priority`/`changefreq` (not read in the databases this session; the shape is known from code) [VERIFIED: models/definition.go, models/definition_item.go]. `golem15_sitemap_settings`: `frontend_url`, `frontend_default_locale`, `updates_manifest_path`, tier priorities [VERIFIED: models/settings.go] | **Data migration:** keep `url` items as the "pages" group; map `frontend_url` → site URL; drop the tier columns in a new migration (or leave them unused); first rebuild job fills the slot table | +| Live service config | Google Search Console sitemap submissions for grzybyfunkcjonalne.pl point at `/sitemap.xml` (assumed, not checked) | Keep the index at `/sitemap.xml`; optionally submit the per-type files for monitoring (D-30); a manual GSC task, not code | +| Live service config | grzyby nginx routes exactly `location = /sitemap.xml` and `location = /sitemap.xsl` to Go [VERIFIED: deploy/nginx/grzybyfunkcjonalne-staging.conf:145-161]; `/robots.txt` goes to Nuxt (@nuxtjs/robots), whose `robots` config sets `sitemap: '/sitemap.xml'` | Add a location for `^/sitemap(-[a-z0-9-]+)?\.xml$`, `/robots.txt`, `/llms.txt` and the IndexNow key file (or let the layer proxy them); remove the `sitemap.xsl` location; the production conf was not found in repo (only staging), so check with Jakub | +| OS-registered state | supervisor programs for grzyby (deploy/supervisor) | None expected (same binaries); verify after deploy | +| Secrets/env vars | `NUXT_CACHE_PURGE_TOKEN` / `site_cache.purge_token`; `golem15.seo.notfound_token` (new on grzyby); new IndexNow key (generated in admin, stored in DB, not an env var) | Add the notfound token and seo site URL to grzyby env; no rename of existing keys | +| Build artifacts | Nitro route cache on running Nuxt servers holds pages with the old head (two graphs from nuxt-schema-org) | Purge after deploy (the purge route) | + +## Common Pitfalls + +### Pitfall 1: Building on unreleased fixes +**What goes wrong:** Phase 33 plugins `require` v0.5.1 / seo v0.1.0 and silently miss CR-01..CR-04, or develop against `replace` and fail at tag time. +**How to avoid:** 33-01 releases the fixes together with the new seam; check that every plugin's `go.mod` requires the new tags in 33-14. +**Warning signs:** `go list -m` shows v0.5.1 in a plugin that needs a v0.5.2 field. + +### Pitfall 2: Two JSON-LD graphs / two canonicals +**What goes wrong:** the journal layer keeps `@nuxtjs/seo`; nuxt-schema-org adds WebSite and WebPage; seo-utils adds a canonical. The CMS graph and canonical come in addition. +**How to avoid:** remove `@nuxtjs/seo` from the layers. A smoke expectation asserts exactly one `application/ld+json` script and one `rel="canonical"`. + +### Pitfall 3: hreflang for missing translations +**What goes wrong:** using i18n's locale head, or the sitemap listing every enabled locale. +**How to avoid:** use only `ContentLocale` entries with a non-empty `URL`. Journal emits a single alternate with locale `"x-default"` when translate has no enabled locale [VERIFIED: sm-journal-plugin/classes/menu_item_types.go:53-55], so emit no hreflang when only one locale exists. hreflang must list itself and be reciprocal [CITED: developers.google.com/search/docs/specialty/international/localized-versions]. + +### Pitfall 4: lastmod changes on every save +**What goes wrong:** Journal's descriptor `LastMod` is `UpdatedAt`, which moves on any save. Google uses lastmod only "if it's consistently and verifiably accurate" [CITED: developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap]. +**How to avoid:** use the event-driven lastmod store (Pattern 2); ignore empty `Changes`. + +### Pitfall 5: Stable numbering broken by deletes +**What goes wrong:** re-chunking an ordered list after a delete shifts every later URL into an earlier file. +**How to avoid:** slot table with holes; only append. + +### Pitfall 6: Shared purge file duplicated across layers +**What goes wrong:** journal, menu and FAQ ship `server/api/__site__/cache/purge.post.ts` and `server/lib/siteCachePurge.ts` "byte for byte" [VERIFIED: vue-journal-layer/server/api/__site__/cache/purge.post.ts header]. Nitro keeps one handler per route, so a new SEO-layer behaviour (clearing the resolve LRU) is lost when another layer's copy wins. +**How to avoid:** move the route and lib into `vue-seo-layer` and delete the copies in 33-09. The purge clears `nitro:routes` [VERIFIED: siteCachePurge.ts:16] and the resolve LRU. + +### Pitfall 7: Generic "read more" and the image as a second link +**What goes wrong:** PostCard renders an image `NuxtLink` (`tabindex="-1" aria-hidden="true"`) and a `$t('journal.readMore')` link in addition to the title link [VERIFIED: vue-journal-layer/app/components/Journal/PostCard.vue:15-24]. That is three anchors to one URL, one of them generic. +**How to avoid:** title link only; make the card clickable with a CSS stretched link; drop the separate read-more anchor, or make its text "Czytaj: " (D-81). Note the tension with D-80 in Open Question 8. + +### Pitfall 8: XSL breaks in browsers +**What goes wrong:** keeping `<?xml-stylesheet?>` while Chrome 158 (2026-11-17) stops running XSLT on Stable [CITED: developer.chrome.com/docs/web-platform/deprecating-xslt]. +**How to avoid:** remove the PI and `/sitemap.xsl`. The HTML sitemap (D-34) is the human view. Record in the plan that backlog todo w8kupqi3 is resolved by removal, and close it in g15office separately. + +### Pitfall 9: News sitemap misuse +**What goes wrong:** listing all posts. Only articles "created in the last two days" belong; up to 1,000 `news:news` per sitemap; title without author or date [CITED: developers.google.com/search/docs/crawling-indexing/sitemaps/news-sitemap]. +**How to avoid:** the window is computed from the new `PublishedAt`; off by default. + +### Pitfall 10: Build guard false confidence +**What goes wrong:** checking only `nuxt.options.routeRules`, while rules also come from `nitro.routeRules`, layers, `.client.vue` pages and experimental inline rules. +**How to avoid:** check all of them in `nitro:config` after layers merge. Test with a playground that has each kind. + +### Pitfall 11: FAQ rich results are gone +**What goes wrong:** treating FAQPage validation as a Google rich-result requirement. Google removed the FAQ rich result: "This feature will no longer appear in Google Search starting May 7, 2026" [CITED: developers.google.com/search/docs/appearance/structured-data/faqpage]. +**How to avoid:** still emit FAQPage, since it is valid schema.org and D-43 asks for it, but give it no rich-result validation rules. The Article type has "no required properties" in Google's docs; its recommended ones are author, dateModified, datePublished, headline and image [CITED: developers.google.com/search/docs/appearance/structured-data/article]. + +### Pitfall 12: Crawling staging +**What goes wrong:** every page reports noindex because of the D-22 staging lock. +**How to avoid:** suppress lock-caused robots issues when `Staging()` is true. Crawl over http on staging needs `fetchguard.TrustedMode`. + +### Pitfall 13: grzyby jumps several versions at once +**What goes wrong:** framework v0.4.4→v0.5.2, Journal pseudo-version → v0.2.x, layers v0.4.0→v0.7.0 with breaking layer changes, plus SEO mount and sitemap migration, all on a live site. +**How to avoid:** a separate plan (33-13) with smoke-ssr expectations covering the public URLs that must stay identical (Phase 16 D-15), staging deploy first, production deploy as a human checkpoint. + +## Code Examples + +### Sitemap index and urlset with hreflang and images (Go, stdlib escaping) +```go +// Source: sitemaps.org protocol + Google image/hreflang docs (cited above). Names [ASSUMED]. +const ( + nsSitemap = "http://www.sitemaps.org/schemas/sitemap/0.9" + nsXHTML = "http://www.w3.org/1999/xhtml" + nsImage = "http://www.google.com/schemas/sitemap-image/1.1" + nsNews = "http://www.google.com/schemas/sitemap-news/0.9" +) + +func writeIndex(w io.Writer, files []IndexFile) error { + bw := bufio.NewWriter(w) + bw.WriteString(xml.Header + `<sitemapindex xmlns="` + nsSitemap + `">`) + for _, f := range files { + bw.WriteString("<sitemap><loc>") + xml.EscapeText(bw, []byte(f.Loc)) + bw.WriteString("</loc>") + if !f.LastMod.IsZero() { + bw.WriteString("<lastmod>" + f.LastMod.UTC().Format(time.RFC3339) + "</lastmod>") + } + bw.WriteString("</sitemap>") + } + bw.WriteString("</sitemapindex>\n") + return bw.Flush() +} +// Each <url>: <loc>, optional <lastmod>, then when >1 locale every <xhtml:link rel="alternate" hreflang=".." href=".."/> +// including itself and x-default, then up to 1000 <image:image><image:loc>..</image:loc></image:image>. +// No <priority>, no <changefreq> (D-30; Google ignores them). +``` + +### IndexNow ping (Go, fetchguard) +```go +// Source: indexnow.org/documentation (cited). Key: 8–128 chars [a-zA-Z0-9-], served at /<key>.txt. +body := map[string]any{"host": host, "key": key, "keyLocation": site + "/" + key + ".txt", "urlList": urls} // ≤10,000 URLs +res, err := client.PostJSON(ctx, "https://api.indexnow.org/indexnow", body) // fetchguard.Client, AllowHosts api.indexnow.org +// 200/202 ok; 403 key file missing; 422 host mismatch; 429 back off (retry via conga attempts) +``` + +### Layer head from the resolve answer (Nuxt core only) +```ts +// Source: Nuxt core useHead/useSeoMeta; field names of `seo` VERIFIED (resolver.go:72-101), new fields [ASSUMED]. +export function useCmsSeo() { + const res = useRequestEvent()?.context.seo ?? useState('cms-seo').value + if (!res?.seo) return + const s = res.seo + useSeoMeta({ title: s.title, description: s.description, robots: s.robots, + ogTitle: s.og.title, ogDescription: s.og.description, ogImage: s.og.image || undefined, ogUrl: s.og.url, + twitterCard: s.twitter.card, twitterSite: s.twitter.site || undefined }) + useHead({ + htmlAttrs: { lang: res.html_lang }, + link: [{ rel: 'canonical', href: s.canonical, key: 'canonical' }, + ...(res.alternates ?? []).map((a) => ({ rel: 'alternate', hreflang: a.hreflang, href: a.url, key: `alt-${a.hreflang}` }))], + meta: s.verification.map((v) => ({ name: v.name, content: v.content })), + script: [{ type: 'application/ld+json', key: 'jsonld', innerHTML: mergeGraph(res.jsonld, useSchemaNodes()) }], // unhead escapes '<' + }) +} +``` + +### Phrase auto-linking over sanitize-html output (TS) +```ts +// sanitize-html serializes normalized HTML with escaped text, so a tag/text split is reliable on ITS output. +const SKIP = new Set(['a', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'code', 'pre', 'figcaption', 'button']) +export function autolink(html: string, links: { phrase: string; url: string }[], max: number, self: string): string { + const stack: string[] = []; let used = 0; const done = new Set<string>() + return html.split(/(<[^>]+>)/).map((part) => { + if (part.startsWith('<')) { trackTag(stack, part); return part } + if (used >= max || stack.some((t) => SKIP.has(t))) return part + for (const l of links) { + if (done.has(l.url) || l.url === self || used >= max) continue + const i = indexOfWord(part, escapeHtml(l.phrase)) // case-insensitive, word boundary, first occurrence + if (i < 0) continue + done.add(l.url); used++ + part = part.slice(0, i) + `<a href="${escapeAttr(l.url)}">` + part.slice(i, i + l.phrase.length) + '</a>' + part.slice(i + l.phrase.length) + } + return part + }).join('') +} +``` + +### Journal H1 → H2 in the write gate (Go) +```go +// After SanitizeHTMLReport: rewrite <h1 ...> / </h1> start and end tags to h2 using the x/net/html tokenizer, +// count conversions, and surface "n headings converted" as a save notice (admin) / response field (API, MCP). +// Editor: StarterKit.configure({ heading: { levels: [2, 3, 4] } }) plus a parse rule mapping <h1> to level 2. +// Layer defence in depth: sanitize-html transformTags: { h1: 'h2' } in htmlGate. +``` + +## State of the Art + +| Old approach | Current approach | When changed | Impact | +|--------------|------------------|--------------|--------| +| `<priority>`, `<changefreq>` | Omitted; Google ignores them | long-standing [CITED: build-sitemap docs] | D-30 drops them | +| image:caption/title/geo_location/license | Only `image:image` + `image:loc` | Google removed the others [CITED: image-sitemaps docs] | Smaller image entries | +| `rel=prev/next` for Google | "Google no longer uses these tags"; other engines may [CITED: pagination docs] | — | D-42 keeps them for Bing; numbered `<a href>` links are what matters | +| Browser XSLT view of sitemaps | Removed from Chrome Stable in Chrome 158, 2026-11-17 [CITED: Chrome deprecating-xslt] | 2026 | Drop the XSL output | +| FAQ rich results | Not shown in Google Search from 2026-05-07 [CITED: faqpage docs] | 2026 | Keep FAQPage, no rich-result validation | +| `@nuxtjs/seo` all-in-one | CMS-driven head via core composables for headless CMS sites | — | Layer replaces the module | + +**Deprecated/outdated in the repos:** +- sm-sitemap-plugin priority tiers config (`golem15.sitemap.priority_tiers`), the frontend_url rewrite and the PHP goldens (`classes/testdata/*.golden`). D-32 retires them. +- `classes.BeforeAddItem`/`AddItem` festival events: no consumer outside the plugin [VERIFIED: grep across siblings]. Replace them with the `Filter` interface. + +## Assumptions Log + +| # | Claim | Section | Risk if wrong | +|---|-------|---------|---------------| +| A1 | New pact fields named `PublishedAt`, `Parents`, `Authors`, `Images`, `Videos`, `Kind` are acceptable as generic, additive seam (D-90/D-106) | Pattern 1 | Seam shape rework; needs Jakub's blocking checkpoint like 32-01/T1 | +| A2 | `sitemap.Filter` interface with batched `Decide` and a `Group` override | Pattern 3 | Page-registry kinds cannot split into per-kind files otherwise | +| A3 | Resolve answer field names (`alternates`, `jsonld`, `breadcrumbs`, `pagination`, `autolinks`, `outbound`, `html_lang`, `og_locale`) | Pattern 4 | Layer/plugin contract; consumed by every app (D-33 "costly") | +| A4 | Keep the index at `/sitemap.xml` (per-type files `sitemap-<slug>-<n>.xml`) | Plan 33-03 | GSC submission changes if a different index URL is chosen | +| A5 | Remove XSL output now | Pitfall 8 | Editors lose the styled XML view (HTML sitemap replaces it) | +| A6 | Layer drops `@nuxtjs/seo` entirely (journal layer), summercms.io keeps its own | Standard Stack | `useSiteConfig` callers and `linkChecker`/`robots`/`sitemap` options in apps must be removed | +| A7 | Middleware enforces only redirect and gone; not_found left to Nuxt routing | Pattern 5 | Soft-404 risk for unknown paths that a catch-all page renders with 200 | +| A8 | Crawler default limits (5000 pages, concurrency 2, 250 ms, 10 s, 5 MiB) | Pattern 7 | Load on small servers; configurable | +| A9 | Event-driven lastmod with an attribute ignore list | Pattern 2 | Lastmod too eager or too lazy | +| A10 | Auto-links applied at render in the layer, not stored in content | Code Examples | Links absent from the API/RSS output; crawler still sees them | +| A11 | grzyby production nginx mirrors the staging conf (only staging conf found in repo) | Runtime State | Missing sitemap routes after deploy | +| A12 | GSC for grzyby has `/sitemap.xml` submitted | Runtime State | None if index stays at the same URL | +| A13 | Optional Go modules gen2brain/webp, gen2brain/avif, chromedp are legitimate and maintained | Package audit | Supply-chain risk; gated behind checkpoint + decision note | + +## Open Questions + +1. **Release cadence for a 10-repo phase.** + - Known: apps pin plugins by Go tag and layers by pnpm git tag; Phase 32 tagged only in its last plan and staged the demo mount as a patch. + - Unclear: whether Jakub allows intermediate pushes and tags. Plugin tags after 33-03..06, layer tags after 33-08/09. + - Recommendation: allow intermediate tags, each behind a `checkpoint:human-verify` "push and tag" step. Otherwise 33-12 and 33-13 can only produce patches applied in 33-14, as 32-05 did. +2. **D-70 "rendered" HTML.** + - Known: a no-JS comparison needs no browser; a rendered DOM needs headless Chromium (chromedp + Chrome on the server). Chromium 154 exists on this dev machine; production servers unknown. + - Recommendation (MVP): the raw HTML is checked against CMS expectations (descriptor title in H1, word count vs `Words`, CMS-known links present). Rendered comparison deferred, or optional when a Chrome path is configured. +3. **D-85 WebP/AVIF/srcset.** + - Known: `lagoon/attach` decodes webp but cannot encode it ("a webp thumbnail holds JPEG bytes") [VERIFIED: modules/lagoon/attach/thumb.go:22-24]. Encoding needs a new dependency. + - Options: (a) gen2brain/webp+avif in a plugin (WASM, no cgo); (b) Nuxt image pipeline (new npm deps, sharp native); (c) defer formats, ship srcset widths from existing JPEG/PNG thumbs plus alt/filename rules. + - Recommendation: (c) for MVP, with (a) as a follow-up decision note. +4. **Success criterion 3 vs layout issues.** Some crawler findings (two H1 from a layout, a missing breadcrumb component) are frontend code, not data. Recommendation: issue kinds carry either a `tool` or `fix: "code"`. The criterion then reads as: every data issue has a tool. Confirm with Jakub and Dawid. +5. **Site URL in the sitemap plugin.** Sitemap cannot read `golem15.seo.site_url` (D-95). Recommendation: a sitemap setting or config `golem15.sitemap.site_url` defaulting to `app.url`; the SEO plugin reports a mismatch as an issue. +6. **Related posts (D-82)** changes Journal API semantics (today: same category, limit 3, `published_at DESC`) [VERIFIED: sm-journal-plugin/controllers/api/posts_helpers.go:152-172]. Recommendation: opt-in config in Journal (count 3–6, tag+category scoring), default = current behaviour (porting contract). +7. **summercms.io scope.** It is a `nuxt generate` site with no sitemap or SEO plugin mounted, and it uses `@nuxtjs/seo` for its own sitemap. Recommendation: Phase 33 only bumps its Journal and layer pins and checks the build guard; a full move is deferred. +8. **D-80 vs D-81 "read more".** D-80 says only the title is a text link; D-81 allows "read more" carrying the title. Recommendation: no separate read-more anchor in the layer component. +9. **D-20c "layer routes reading their patterns from the CMS".** Nuxt routes are build-time. Recommendation: build-time options stay, plus a startup drift check against CMS patterns (logged, and reported by the crawler), rather than a build-time network fetch. +10. **"The user plugin's auth layer" (D-23)** does not exist as a Nuxt layer today; there are only vue-journal, menu, FAQ and paymentgateway layers. Recommendation: the SEO layer honours `definePageMeta({ requiresAuth: true })` / `seo: { robots }`; apps mark their auth pages; a user layer is out of scope. + +## Environment Availability + +| Dependency | Required by | Available | Version | Fallback | +|------------|------------|-----------|---------|----------| +| Go | all Go repos | ✓ | go1.27.2 | — | +| Node | layers, apps | ✓ | v24.22.0 (engines >=22.18) | — | +| pnpm | layer installs | ✓ | 12.14.0 | npm 12.2.1 | +| Docker | testcontainers Postgres suites | ✓ | 29.9.1 (server reachable as golem) | `-short` only | +| tea | create `vue-seo-layer` repo | ✓ | /usr/bin/tea | — | +| Chromium | only if rendered crawl chosen | ✓ (dev) | 154.0.8037.97 | raw-HTML checks | +| Write access to summercms.go `.git` and phase dir | docs commits | ✓ | owned by golem now (32-03 had a jin:jin blocker) | — | + +**Missing dependencies with no fallback:** none for the MVP scope. +**Missing with fallback:** headless browser on production (raw-HTML check instead). + +## Validation Architecture + +### Test framework +| Property | Value | +|----------|-------| +| Framework (Go) | `go test` + testcontainers Postgres (`TestMain` skips Docker under `testing.Short()`) | +| Framework (layers) | `node --test --test-reporter=tap tests/*.test.ts` (pure TS utils) + `testing/smoke-ssr.mjs` against a built playground with a fixture API | +| Framework (admin elements) | vitest 3.2.7 + happy-dom 20.11.6 in sm-seo-plugin / sm-journal-plugin | +| Config file | per repo (`package.json` scripts, `vite.config.ts`); new `scripts/check-phase33.sh` in summercms.go | +| Quick run command | `go test -short ./...` in the touched repo; `npm test` / `pnpm test` in the touched layer | +| Full suite command | `bash scripts/check-phase33.sh --all` (modelled on check-phase32.sh: self-test, go, plugins, layers, consumers, docker, docs, marker, additive, compat) | + +### Phase requirements → test map +| Req ID | Behavior | Test type | Automated command | File exists? | +|--------|----------|-----------|-------------------|-------------| +| D-30 | index + per-type files of N URLs, stable numbering after a delete, lastmod only on real change | integration (Postgres) | `go test ./... -run 'TestSitemapIndex\|TestStableNumbering\|TestLastmod'` (sm-sitemap-plugin) | ❌ Wave 0 | +| D-31 | image entries ≤1000/url; news window 2 days; IndexNow ping on URL diff (fake transport via `fetchguard.WithTransport`) | unit + integration | `go test ./... -run 'TestImageSitemap\|TestNewsSitemap\|TestIndexNow'` | ❌ | +| D-32 | no priority/changefreq/XSL; url items migrated | integration | `go test ./... -run TestMigrateDefinitions` | ❌ | +| D-95/D-24 | noindex/canonical/redirected excluded via SEO filter; sitemap works without SEO | integration (two apps) | `go test ./... -run 'TestFilterExcludes\|TestNoFilter'` | ❌ | +| D-33 | registry page resolves ok; route report upserts; page kinds group into files | integration | `go test ./... -run 'TestRegistry'` (sm-seo-plugin) | ❌ | +| D-43..46 | @graph per kind with @id links; custom JSON-LD refused when invalid; validator warnings | unit | `go test ./classes/schema/...` | ❌ | +| D-55..57 | alternates only for translated locales; x-default = default locale | integration | `go test ./... -run TestAlternates` | ❌ | +| D-40..42 | `?page=1` → 301; other form → 301 to configured; suffix templates | integration | `go test ./... -run TestPagination` | ❌ | +| D-50/51 | H1→H2 on admin/API/MCP/import, notice; command dry run | integration (Journal) | `go test ./... -run 'TestHeadingConversion'` | ❌ | +| D-70 | build fails for indexable `ssr:false`, `.client.vue`; passes when declared noindex | build smoke | `pnpm build:playground` with a fixture playground per case (expect exit 1/0) | ❌ | +| D-20a/D-08/D-103 | middleware: 301/410, X-Robots-Tag, verification file at root, LRU purged | SSR smoke | `node testing/smoke-ssr.mjs --expect testing/expect-seo.txt` | ❌ | +| D-43 (layer) | exactly one ld+json script, one canonical | SSR smoke | same expect file (`contains=`/`notcontains=`) | ❌ | +| D-80/81/52 | one anchor per card, H2 cards, no generic anchor | SSR smoke + unit | journal layer `pnpm test` + smoke | ❌ | +| D-14 et al. | crawler checks against an httptest site with planted issues; each issue has tool or `fix:code` | integration | `go test ./... -run TestCrawler` | ❌ | +| D-10 | every new admin operation has an MCP tool | unit | `go test -short ./... -run TestEveryAdminOperationHasTool` | ✅ (extend) | +| D-82/83 | autolink first occurrence, max N, not in headings/links/self; outbound rel | unit (TS + Go) | `pnpm test` (vue-seo-layer / journal layer) | ❌ | +| D-72 | bm serves `Disallow: /` and noindex | smoke | `curl -sI` in check script against a local build | ❌ | +| D-90/D-106 | framework additions name no SEO/app; additive only | gate | `check-phase33.sh --marker --additive` | ❌ (copy from phase32) | + +### Sampling rate +- **Per task commit:** `go vet ./... && go test -short ./...` in the touched Go repo; `pnpm test` in the touched layer. +- **Per wave merge:** Docker suites of the touched repos + layer playground build + smoke-ssr. +- **Phase gate:** `scripts/check-phase33.sh --all` green before `/msd-verify-work`. + +### Wave 0 gaps +- [ ] `scripts/check-phase33.sh` (copy the stage engine of check-phase32.sh; add a `layers` stage: node tests, playground builds, smoke-ssr). +- [ ] sm-sitemap-plugin: retire `classes/golden_test.go` + `testdata/*.golden`; new test fixtures for multi-type, multi-locale content (fake content type plugin, as sm-seo-plugin's `fixture_test.go` does). +- [ ] vue-seo-layer: `.playground`, `testing/` (copy smoke-ssr, fixture-api, nuxt-server from vue-journal-layer), fixture `/_seo/api/v1/resolve` answers. +- [ ] Build-guard fixture playgrounds (indexable ssr:false → fail; declared noindex → pass; `.client.vue` page → fail). +- [ ] Crawler httptest site with planted issues. + +## Security Domain + +### Applicable ASVS categories +| ASVS category | Applies | Standard control | +|---------------|---------|-----------------| +| V2 Authentication | yes (route report, 404 report, purge) | bearer tokens ≥32 chars (existing purge rule), backend guard + `golem15.seo.manage` for management routes | +| V3 Session Management | no | — | +| V4 Access Control | yes | one SEO permission (D-15); MCP tools listed only with the permission; read grants see GET tools only | +| V5 Input Validation | yes | custom JSON-LD: parse, size/depth caps, objects only, merge by `@id`; autolink URLs internal paths or http(s); IndexNow key `[A-Za-z0-9-]{8,128}`; verification names `[A-Za-z0-9._-]+\.(html\|txt\|xml)` [VERIFIED: sm-seo-plugin/classes/verification.go:11] | +| V6 Cryptography | yes (IndexNow key) | `crypto/rand` hex key; never hand-roll | +| V12/V13 SSRF / outbound | yes | `fetchguard` for crawler and IndexNow; crawler follows only same-host links | + +### Known threat patterns +| Pattern | STRIDE | Standard mitigation | +|---------|--------|---------------------| +| SSRF through crawled links or a site URL pointing inside the network | Elevation | fetchguard host allow-list = site host; TrustedMode only from config, never from page content | +| Script breakout from JSON-LD (editor custom JSON with `</script>`) | Tampering/XSS | unhead escapes `<` in json scripts; also validate on save | +| Open redirect via middleware | Spoofing | Location always from the resolver, absolute on the site URL (Phase 32 guarantee); layer never builds Location from the request | +| XML injection in sitemaps | Tampering | `xml.EscapeText` on every value; only http(s) locs | +| Crawl as DoS on own site | DoS | concurrency/delay caps, one crawl at a time (conga unique), max pages | +| Resolve endpoint flooding through the Nuxt server | DoS | LRU cache in the layer; forward X-Forwarded-For so the per-IP bucket applies per visitor | +| Route report poisoning the registry | Tampering | bearer token; report only inserts unknown static paths, never deletes editor data | +| Auto-link injection | XSS | phrase/URL validated on save; layer escapes attribute values; links only into text nodes | + +## Sources + +### Primary (HIGH confidence, code read this session) +- `summercms.go/modules/pact/content.go`, `modules/pact/capabilities.go:956-1010`, `modules/signpost/content.go`, `modules/signpost/signpost.go`, `modules/backpack/services.go`, `modules/cooler/sitecache.go`, `modules/fetchguard/README.md`, `modules/conga/README.md`, `modules/lagoon/attach/thumb.go` +- `sm-sitemap-plugin/` (plugin.go, routes.go, classes/generator.go, serialize.go, events.go, models/*, config/config.yaml, README) +- `sm-seo-plugin/` (routes.go, mcp.go, operations.go, classes/resolver.go, issues.go, verification.go, robots.go, config/config.yaml) +- `sm-journal-plugin/` (classes/content_descriptors.go, menu_item_types.go, format_html.go, assets/src/blogeditor/*, controllers/api/posts_helpers.go) +- `vue-journal-layer/` (nuxt.config.ts, PostCard.vue, PostPage.vue, Pagination.vue, seo.ts, sitemap source, purge route + lib, htmlGate.ts, testing/smoke-ssr.mjs); node_modules h3 1.15.11, unhead 2.1.17, sanitize-html 2.18.0 +- Apps: grzyby `nuxt.config.ts`, `package.json`, nginx staging conf, `.gitmodules`, `summer.yaml`; demo `nuxt.config.ts`; summercms.io `nuxt.config.ts`; bm `frontend/` +- Planning: 33-CONTEXT, 32-CONTEXT, 32-DECISIONS-MAP, 32-02..06 SUMMARYs, 32-UAT, 32-VERIFICATION, deferred-items, 32-06-release-log, STATE.md, ROADMAP.md + +### Secondary (cited official docs; seam tier LOW for WebFetch) +- https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap +- https://developers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps +- https://developers.google.com/search/docs/crawling-indexing/sitemaps/image-sitemaps +- https://developers.google.com/search/docs/crawling-indexing/sitemaps/news-sitemap +- https://developers.google.com/search/docs/specialty/international/localized-versions +- https://developers.google.com/search/docs/specialty/ecommerce/pagination-and-incremental-page-loading +- https://developers.google.com/search/docs/appearance/structured-data/article +- https://developers.google.com/search/docs/appearance/structured-data/breadcrumb +- https://developers.google.com/search/docs/appearance/structured-data/faqpage +- https://www.sitemaps.org/protocol.html +- https://www.indexnow.org/documentation +- https://developer.chrome.com/docs/web-platform/deprecating-xslt +- https://nuxt.com/docs/4.x/guide/concepts/rendering, /directory-structure/app/pages, /api/utils/define-route-rules, /api/advanced/hooks, /directory-structure/server +- https://nuxtseo.com/docs/schema-org/guides/default-schema-org, https://nuxtseo.com/docs/nuxt-seo/getting-started/introduction + +### Tertiary (LOW, needs validation) +- Optional Go modules (gen2brain/webp, gen2brain/avif, chromedp): only registry existence checked on proxy.golang.org. + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH for what exists (read in repos); MEDIUM for the removal of `@nuxtjs/seo` (design judgement backed by cited module behaviour). +- Architecture: MEDIUM. The seams are verified; the new interfaces and answer fields are recommendations needing a shape checkpoint. +- Pitfalls: HIGH for repo-derived ones (PostCard, purge duplication, lastmod, staging); MEDIUM for external spec ones (cited). +- Plan breakdown: MEDIUM. It depends on Open Question 1 (release cadence) and the UI-SPEC. + +**Research date:** 2026-10-28 +**Valid until:** 2026-11-17 for the XSL item (Chrome 158); about 30 days for the rest (stable specs, pinned toolchain). diff --git a/.planning/phases/33-seo-output-audit-and-content/33-VALIDATION.md b/.planning/phases/33-seo-output-audit-and-content/33-VALIDATION.md new file mode 100644 index 00000000..7cd80143 --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-VALIDATION.md @@ -0,0 +1,97 @@ +--- +phase: "33" +slug: "seo-output-audit-and-content" +# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6) +# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117) +status: draft +nyquist_compliant: false +wave_0_complete: false +created: "2026-10-28" +--- + +# Phase 33 — Validation Strategy + +> Per-phase validation contract for feedback sampling during execution. Source: `33-RESEARCH.md` § Validation Architecture. + +--- + +## Test Infrastructure + +| Property | Value | +|----------|-------| +| **Framework** | `go test` + testcontainers Postgres (Go repos); `node --test` + `testing/smoke-ssr.mjs` (Nuxt layers); vitest + happy-dom (admin elements) | +| **Config file** | per repo (`package.json` scripts, `vite.config.ts`); new `scripts/check-phase33.sh` in summercms.go (Wave 0) | +| **Quick run command** | `go vet ./... && go test -short ./...` in the touched Go repo; `pnpm test` in the touched layer | +| **Full suite command** | `bash scripts/check-phase33.sh --all` | +| **Estimated runtime** | ~300 seconds (full, with Docker and playground builds) | + +--- + +## Sampling Rate + +- **After every task commit:** Run the quick command in the touched repo +- **After every plan wave:** Docker suites of the touched repos + layer playground build + smoke-ssr +- **Before `/msd-verify-work`:** `scripts/check-phase33.sh --all` must be green +- **Max feedback latency:** 120 seconds (quick loop) + +--- + +## Per-Task Verification Map + +Filled by the planner per task. Decision → test map from research: + +| Decision | Behavior | Test Type | Automated Command | File Exists | Status | +|----------|----------|-----------|-------------------|-------------|--------| +| D-30 | index + per-type files, stable numbering after delete, lastmod only on real change | integration | `go test ./... -run 'TestSitemapIndex\|TestStableNumbering\|TestLastmod'` (sm-sitemap-plugin) | ❌ W0 | ⬜ pending | +| D-31 | image ≤1000/url, news 2-day window, IndexNow ping on diff | unit + integration | `go test ./... -run 'TestImageSitemap\|TestNewsSitemap\|TestIndexNow'` | ❌ W0 | ⬜ pending | +| D-32 | no priority/changefreq/XSL; url items migrated | integration | `go test ./... -run TestMigrateDefinitions` | ❌ W0 | ⬜ pending | +| D-95/D-24 | SEO filter excludes noindex/canonicalised/redirected; sitemap works without SEO | integration | `go test ./... -run 'TestFilterExcludes\|TestNoFilter'` | ❌ W0 | ⬜ pending | +| D-33 | registry pages resolve; route report upserts | integration | `go test ./... -run TestRegistry` (sm-seo-plugin) | ❌ W0 | ⬜ pending | +| D-43..46 | @graph per kind with @id links; invalid custom JSON-LD refused | unit | `go test ./classes/schema/...` | ❌ W0 | ⬜ pending | +| D-55..57 | alternates only for translated locales; x-default = default | integration | `go test ./... -run TestAlternates` | ❌ W0 | ⬜ pending | +| D-40..42 | `?page=1` → 301; form normalisation; suffix templates | integration | `go test ./... -run TestPagination` | ❌ W0 | ⬜ pending | +| D-50/51 | H1→H2 on every write path; command dry run | integration | `go test ./... -run TestHeadingConversion` (Journal) | ❌ W0 | ⬜ pending | +| D-70 | build fails for indexable `ssr:false` / `.client.vue`; passes when noindex | build smoke | fixture playground builds (expect exit 1/0) | ❌ W0 | ⬜ pending | +| D-20a/D-08/D-103 | middleware 301/410, X-Robots-Tag, verification file at root, cache purge | SSR smoke | `node testing/smoke-ssr.mjs --expect testing/expect-seo.txt` | ❌ W0 | ⬜ pending | +| D-43 (layer) | exactly one ld+json script, one canonical | SSR smoke | same expect file | ❌ W0 | ⬜ pending | +| D-80/81/52 | one anchor per card, H2 cards, no generic anchor | SSR smoke + unit | journal layer `pnpm test` + smoke | ❌ W0 | ⬜ pending | +| D-14 | crawler finds planted issues; each issue has a tool or `fix: code` | integration | `go test ./... -run TestCrawler` | ❌ W0 | ⬜ pending | +| D-10 | every new admin operation has an MCP tool | unit | `go test -short ./... -run TestEveryAdminOperationHasTool` | ✅ (extend) | ⬜ pending | +| D-82/83 | autolink rules; outbound rel | unit | `pnpm test` (vue-seo-layer / journal layer) | ❌ W0 | ⬜ pending | +| D-72 | bm serves `Disallow: /` and noindex | smoke | `curl -sI` in check script | ❌ W0 | ⬜ pending | +| D-90/D-106 | framework additions generic and additive | gate | `scripts/check-phase33.sh --marker --additive` | ❌ W0 | ⬜ pending | + +*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky* + +--- + +## Wave 0 Requirements + +- [ ] `scripts/check-phase33.sh` — stage engine from check-phase32.sh plus a `layers` stage +- [ ] sm-sitemap-plugin: retire `classes/golden_test.go` + `testdata/*.golden`; multi-type, multi-locale fixtures +- [ ] vue-seo-layer: `.playground`, `testing/` (smoke-ssr, fixture-api, nuxt-server), fixture resolve answers +- [ ] Build-guard fixture playgrounds (indexable ssr:false → fail; noindex → pass; `.client.vue` → fail) +- [ ] Crawler httptest site with planted issues + +--- + +## Manual-Only Verifications + +| Behavior | Requirement | Why Manual | Test Instructions | +|----------|-------------|------------|-------------------| +| grzybyfunkcjonalne.pl serves the new sitemap index in production | Success criterion 1 | Production deploy needs go-ahead | After deploy: fetch `/sitemap.xml`, follow each child, submit index in GSC | +| Rich Results test passes for a post, a category and an FAQ page | D-46 | External Google tool | Run the admin "Rich Results" link per page kind | +| Crawl → MCP dry-run fix workflow | Success criterion 3 | Agent-driven workflow | Dawid runs a crawl, asks his Claude client to fix issues with `dry_run`, then applies | + +--- + +## Validation Sign-Off + +- [ ] All tasks have `<automated>` verify or Wave 0 dependencies +- [ ] Sampling continuity: no 3 consecutive tasks without automated verify +- [ ] Wave 0 covers all MISSING references +- [ ] No watch-mode flags +- [ ] Feedback latency < 120s +- [ ] `nyquist_compliant: true` set in frontmatter + +**Approval:** pending