diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index e76463ed..75e24731 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -1640,10 +1640,40 @@ Plans: 3. The crawler reports SEO issues per URL, and each issue can be fixed through MCP. 4. The new code has unit tests, delivered in the phase's last plan. -**Plans:** 0 plans +**Plans:** 14 plans Plans: -- [ ] TBD (run /msd-plan-phase 33 to break down) +**Wave 1** +- [ ] 33-01-PLAN.md — framework: additive content descriptor fields and per-request field options, check-phase33.sh, v0.5.2 release log (no tag) +- [ ] 33-07-PLAN.md — vue-seo-layer (new repository): CMS head and one JSON-LD graph, redirect/410 enforcement, root files, 404 report, shared purge route + +**Wave 2** *(blocked on Wave 1 completion)* +- [ ] 33-02-PLAN.md — sm-journal-plugin: per-site H1 setting on every write path, editor J1–J4, descriptor facts, journal-author type, MCP tools +- [ ] 33-03-PLAN.md — sm-sitemap-plugin: sitemap index with per-type files, slot table, real lastmod, hreflang, filter contract (breaking, D-32) +- [ ] 33-08-PLAN.md — vue-seo-layer: SSR build guard, noindex routes, route report, drift check, breadcrumbs, pagination, HTML sitemap + +**Wave 3** *(blocked on Wave 2 completion)* +- [ ] 33-04-PLAN.md — sm-sitemap-plugin: image and News sitemaps, IndexNow, HTML-sitemap API, MCP tools +- [ ] 33-09-PLAN.md — journal, menu and FAQ layers on the SEO layer: headings in both H1 modes, one link per card, body links, breadcrumbs, pagination + +**Wave 4** *(blocked on Wave 3 completion)* +- [ ] 33-05-PLAN.md — sm-seo-plugin I: sitemap filter, page registry with route report, site endpoint, llms.txt, Links settings, purge after writes + +**Wave 5** *(blocked on Wave 4 completion)* +- [ ] 33-06-PLAN.md — sm-seo-plugin II: JSON-LD @graph, publisher, schema fields (E8/E9), breadcrumbs, hreflang, pagination + +**Wave 6** *(blocked on Wave 5 completion)* +- [ ] 33-10-PLAN.md — sm-seo-plugin: crawler, issue report with fix routes, Site audit and Crawls, MCP tools + +**Wave 7** *(blocked on Wave 6 completion)* +- [ ] 33-11-PLAN.md — internal linking and media: auto-links, link suggestions (E10), related posts, WebP variants with srcset + +**Wave 8** *(blocked on Wave 7 completion)* +- [ ] 33-12-PLAN.md — apps I: demo end to end on local links, summercms.io patches, sm-bm-app noindex (D-72) +- [ ] 33-13-PLAN.md — grzybyfunkcjonalne.pl on the new sitemap format and the SEO layer; uat deploy by decision + +**Wave 9** *(blocked on Wave 8 completion)* +- [ ] 33-14-PLAN.md — unit tests, check-phase33.sh --all, release tags in order, app re-pins, changelog post ## Progress diff --git a/.planning/phases/33-seo-output-audit-and-content/33-01-PLAN.md b/.planning/phases/33-seo-output-audit-and-content/33-01-PLAN.md new file mode 100644 index 00000000..e2a6a37f --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-01-PLAN.md @@ -0,0 +1,334 @@ +--- +phase: 33-seo-output-audit-and-content +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - modules/pact/content.go + - modules/pact/capabilities.go + - modules/pact/README.md + - modules/signpost/README.md + - modules/signpost/example_content_test.go + - modules/signpost/content_fields_smoke_test.go + - modules/cabana/field_type.go + - modules/cabana/http.go + - modules/cabana/admin_openapi.go + - modules/cabana/README.md + - modules/cabana/field_request_options_smoke_test.go + - admin/openapi/admin.json + - admin/src/api/schema.d.ts + - docs/plugins/menu-item-types.md + - docs/backend/field-types.md + - scripts/check-phase33.sh + - .planning/phases/33-seo-output-audit-and-content/33-release-log.md +autonomous: false +requirements: [SC-2, SC-3, D-93, D-90, D-106, D-43, D-86, D-31, D-85, D-50, D-51, D-96, D-10] + +estimate: + tokens: 90000 + raw_tokens: 90000 + tasks: 4 + confidence: low + +must_haves: + truths: + - "pact.ContentDescriptor gains Kind and Tools, pact.ContentRecord gains PublishedAt, Parents, Authors and TopHeading, and pact.ContentLocale gains Images, Videos and Headings, with the value types ContentRef, ContentPerson, ContentImage, ContentVideo and ContentHeading; every field is optional and names nothing SEO-specific (D-93, D-90)" + - "A content type that sets none of the new fields describes, lists and matches exactly as in v0.5.1; signpost.Describe and signpost.Records hand the new fields through unchanged, proven by TestContentDescriptorFields and the runnable signpost example (D-93: sitemaps and the page registry consume Records)" + - "The descriptor fields carry what JSON-LD, breadcrumbs, the image and News sitemaps, the missing-alt report and the heading checks need: first publication date, ordered candidate parents (first is the default primary), authors with an optional reference to their own page, body images with alt and a decorative flag, videos, the body's heading outline and the top heading level the body may use (D-43, D-86, D-31, D-85, D-50, D-51)" + - "ContentDescriptor.Tools names the MCP tools that change a type's records by operation (edit, headings, image_alt), so an audit tool can point every content issue at a fixing tool without importing the content plugin (SC-3, D-10, D-96)" + - "A plugin field type can supply per-request options through pact.FormFieldType.RequestOptions; cabana merges them into the field's typeOptions in the served form schema on create, update and preview, an option set in fields.yaml wins over a request option of the same key, and an element therefore reads a site setting without a token, cookie or admin route (UI-SPEC Q-2, 2026-10-28 Journal H1 setting)" + - "A RequestOptions error never fails the form: the schema is served with the fields.yaml options only and the error is logged once per request" + - "The exported API surface of the framework after this plan is strictly additive against v0.5.1: no element removed or changed and no method added to an existing interface (D-106), checked by scripts/check-phase33.sh --additive" + - "scripts/check-phase33.sh exists with the stages self-test, go, plugins, layers, consumers, docker, admin, docs, marker, additive and compat; --self-test passes and a stage whose repository is missing fails with the repository named (Wave 0)" + - "33-release-log.md lists what v0.5.2 will carry (the six Phase 32 fix commits CR-01, CR-02, CR-04, WR-01, WR-02, WR-07 plus this plan's seams) and the unreleased commits of sm-seo-plugin and sm-journal-plugin; nothing is pushed or tagged in this plan (2026-10-28 release cadence: all tags in 33-14)" + - "Jakub's answer on the meaning of pact.RecordSaved.Locale (32-UAT test 4, WR-02) is recorded before v0.5.2 freezes it" + - "The plan list of Phase 33 is 14 plans, as Jakub confirmed on 2026-10-28; this plan is the first of them and the framework part of v0.5.2 (2026-10-28 plan list)" + - {statement: "A descriptor that leaves the new slices nil and one that sets them to empty slices are treated alike by signpost and by the docs examples (no nil-versus-empty drift for consumers)", verification: backstop} + prohibitions: + - requirement_id: "D-90" + category: values + status: resolved + verification: test + statement: "No framework file added or changed by this plan names SEO, a sitemap, JSON-LD or a consuming application; the marker stage of check-phase33.sh proves it" + - requirement_id: "D-106" + category: safety + status: resolved + verification: test + statement: "No existing exported element of pact, signpost or cabana changes its signature or meaning, and no interface gains a method; v0.5.1 consumers build and test unchanged" + artifacts: + - path: "modules/pact/content.go" + provides: "additive descriptor fields and value types" + contains: "PublishedAt *time.Time" + - path: "modules/pact/capabilities.go" + provides: "FormFieldType.RequestOptions" + contains: "RequestOptions" + - path: "modules/signpost/content_fields_smoke_test.go" + provides: "TestContentDescriptorFields (tracer)" + contains: "func TestContentDescriptorFields(" + - path: "modules/cabana/field_request_options_smoke_test.go" + provides: "TestFieldTypeRequestOptions" + contains: "func TestFieldTypeRequestOptions(" + - path: "scripts/check-phase33.sh" + provides: "phase gate engine (Wave 0)" + contains: "--additive" + - path: ".planning/phases/33-seo-output-audit-and-content/33-release-log.md" + provides: "v0.5.2 contents and the release order 33-14 follows" + contains: "v0.5.2" + key_links: + - from: "modules/pact/content.go" + to: "modules/signpost/content.go" + via: "Registry.Describe and Registry.Records return the type's *pact.ContentRecord values as they come, new fields included" + pattern: "func \\(r \\*Registry\\) (Describe|Records)\\(" + - from: "modules/pact/capabilities.go" + to: "modules/cabana/http.go" + via: "the form schema handler calls each plugin field type's RequestOptions once per request and merges the map into TypeOptions" + pattern: "RequestOptions" +--- + + + +## Phase Goal + +**As a** site editor and as an SEO specialist auditing a SummerCMS site, **I want to** have the rendered sites emit complete SEO output (sitemaps, structured data, pagination, hreflang, headings, breadcrumbs) from the CMS, and an audit that finds the remaining problems, **so that** sites are indexed correctly and issues can be fixed through MCP. + +This plan gives every later plan what it needs from the framework: the content facts that JSON-LD, breadcrumbs, image and News sitemaps and the audit read from a content type (one generic, additive seam), a way for a plugin field type to receive per-request options (the Blog Editor's heading levels follow the Journal H1 setting), the phase gate script, and the release log that 33-14 follows. + + +Extend the framework additively (D-90, D-106) and prepare v0.5.2 without tagging it: +- `pact.ContentDescriptor`, `pact.ContentRecord`, `pact.ContentLocale` gain optional fields (Pattern 1 of 33-RESEARCH.md, names confirmed at the Task 1 checkpoint); +- `pact.FormFieldType.RequestOptions` lets a plugin field type add per-request options to its schema entry (UI-SPEC Q-2, planner decision: a generic seam rather than an element-side fetch); +- `scripts/check-phase33.sh` with every stage the phase gate needs (Wave 0 of 33-VALIDATION.md); +- `33-release-log.md`: v0.5.2 = the unreleased Phase 32 fixes plus these seams; plugin versions are read from `git tag` in 33-14. + +Purpose: Phase 33 builds on tagged versions (Q-01); every plugin in this phase develops against `../summercms.go` through `replace`, and 33-14 tags v0.5.2 with everything this plan adds. + +Output: commits in this repository (code and docs separately from the planning-docs commit of the release log). Nothing is pushed or tagged. + +**Planner decisions recorded here:** +- **P33-01 (descriptor shape, `[ASSUMED]` until Task 1):** `ContentDescriptor.Kind string` with constants `ContentKindArticle = "article"`, `ContentKindCollection = "collection"`, `ContentKindProfile = "profile"`, `ContentKindPage = "page"`; `ContentDescriptor.Tools map[string]string` with constants `ContentToolEdit = "edit"`, `ContentToolHeadings = "headings"`, `ContentToolImageAlt = "image_alt"` (the value is a full MCP tool name such as `journal_update_post`); `ContentRecord.PublishedAt *time.Time`, `ContentRecord.Parents []ContentRef` (ordered candidates, first is the default primary), `ContentRecord.Authors []ContentPerson`, `ContentRecord.TopHeading int` (the highest heading level the body should contain: 2 when the page renders the title as the only H1, 0 when the type sets no rule); `ContentLocale.Images []ContentImage`, `ContentLocale.Videos []ContentVideo`, `ContentLocale.Headings []ContentHeading`; `ContentRef{Type, ID string}` (Type is a menu item type key), `ContentPerson{Name string; Ref ContentRef}` (Ref.Type "" when the person has no page), `ContentImage{URL, Alt string; Decorative bool; Width, Height int}`, `ContentVideo{Provider, ID string}`, `ContentHeading{Level int; Text string}`. Each field gets a one-sentence doc comment in the existing style. +- **P33-02 (per-request field options):** `FormFieldType.RequestOptions func(ctx context.Context, db *gorm.DB) (map[string]any, error)` with `json:"-"`. cabana calls it at most once per field type per schema request, merges the map under the field's `typeOptions` (keys present in fields.yaml win), and logs and ignores an error. The Blog Editor (33-02) uses it for `headingLevels` and `genericAnchors`. +- **P33-03 (release log):** `.planning/phases/33-seo-output-audit-and-content/33-release-log.md` is the phase's release ledger, written like `32-06-release-log.md`: planned contents per repository now, and each executed release step appended in 33-14 so a resumed run never re-tags. + + + +@~/.claude/msd-core/workflows/execute-plan.md +@~/.claude/msd-core/templates/summary.md + + + +@.planning/phases/33-seo-output-audit-and-content/33-CONTEXT.md +@.planning/phases/32-seo-foundation/32-CONTEXT.md +@.planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md +@.planning/phases/33-seo-output-audit-and-content/33-PATTERNS.md +@.planning/phases/32-seo-foundation/32-02-SUMMARY.md +@.planning/phases/32-seo-foundation/32-06-SUMMARY.md +@.planning/phases/32-seo-foundation/32-UAT.md +@CLAUDE.md + + +modules/pact/content.go today (v0.5.1): ContentDescriptor{Controller, Model string; Records, Describe, Match funcs}; ContentList{Pattern, After, Limit, UnprefixedLocale}; ContentPage{Records []ContentRecord; Next}; ContentDescribe{ID, Pattern, UnprefixedLocale}; ContentRecord{ID string; Published bool; LastMod *time.Time; Locales []ContentLocale}; ContentLocale{Locale, URL, Title, Description, Image string; Words *int}; ContentMatch / ContentMatchResult{ID, Locale string; Exact bool}. +modules/pact/capabilities.go FormFieldType{Name, Element string; Translatable bool; Scripts, Styles, Options []string; CheckOptions, Decode, Load, ListValue, Upload funcs}; FormFieldValue{DB, Record, Field, Options, Locale, Default, Value, Translated}. +modules/cabana/schema_types.go: the served field carries TypeOptions map[string]any `json:"typeOptions,omitempty"`; modules/cabana/field_type.go:263 sets it at compile time. +admin/src/components/form/fields/PluginField.vue:44 serializes props.field.typeOptions into the element's options attribute (no SPA change needed). +scripts/check-phase32.sh: stage engine (detect_go, go_named, run_* stages, additive via scripts/apisurface against the newest tag not containing HEAD, compat via snapshots/worktrees in a scratch sibling farm, --self-test planting each case). +Framework commits after v0.5.1 that v0.5.2 carries: 676ca53 (CR-01), 5471b66 (CR-02), d660cd0 (CR-04), 468af06 (WR-01), 1583651 (WR-02), 0e187cf (WR-07). + + + +## Execution units and resume points + +- Each task ends with the commit groups below, staged by explicit path (never `-A` or `.`); another session may have uncommitted `.planning/STATE.md`, `.planning/state.json`, `.planning/ROADMAP.md` or `.planning/G15OFFICE.md` changes, which stay untouched. Code and docs commits are separate from planning-docs commits. +- Resume check: `git log --format=%s | grep -Fx ''` for each subject. All present: re-run the task's `` and continue; some present: continue at the first missing group. + +| Task | Commit groups, in order | Resume point | +|------|-------------------------|--------------| +| 1 | none (checkpoint answer recorded in the SUMMARY) | answer recorded | +| 2 | `feat(signpost): content descriptor fields for dates, parents, authors, media and outline` | TestContentDescriptorFields passes | +| 3 | `feat(cabana): per-request options for plugin field types` | TestFieldTypeRequestOptions passes | +| 4 | A `test: phase 33 gate script`; B `docs(33-01): release log for v0.5.2` | gate self-test and stages pass | + + + + + Task 1: Confirm the Go shape of the descriptor fields and RequestOptions, and the meaning of RecordSaved.Locale + The fields become published framework API in v0.5.2; every content plugin fills them and sm-seo-plugin, sm-sitemap-plugin and the layers read them, so renaming later breaks consumers. + Adopt P33-01 and P33-02 as written, adjust names, or drop RequestOptions; and confirm WR-02 (32-UAT test 4) before v0.5.2 freezes it. + + P33-01 lists the proposed fields (Pattern 1 of 33-RESEARCH.md, plus Tools for SC-3 and TopHeading/Headings for the Journal H1 setting's CMS-side check). They name no SEO concept: feeds, search indexers and audit tools use the same facts. P33-02 is the planner's resolution of UI-SPEC Q-2: the Blog Editor needs the per-site heading levels and generic-anchor list on every request, and field options are compiled at boot today. The alternative is that the editor fetches a public Journal endpoint, which breaks the element rule "no direct route" (UI-SPEC Shared rules) less cleanly. WR-02 (commit 1583651) set RecordSaved.Locale to the primary tag of the request locale; 32-UAT test 4 asks Jakub to accept that or ask for the content-locale meaning. + + + + + + + Reply "as-planned", "adjust: " or "no-request-options", plus "WR-02: accept" or "WR-02: content locale" + + + + Task 2: End-to-end "a content type describes a record with dates, parents, authors, images, videos and its outline, and signpost hands them through" — one path only + modules/pact/content.go, modules/signpost/content_fields_smoke_test.go, modules/signpost/example_content_test.go, modules/pact/README.md, modules/signpost/README.md, docs/plugins/menu-item-types.md + + - modules/pact/content.go (whole file; the doc-comment style to copy) + - modules/signpost/content.go (Describe, Records, ContentTypes, ContentTypeForModel) + - modules/signpost/content_test.go and modules/signpost/example_content_test.go (the fake content type the tests use) + - docs/plugins/menu-item-types.md (the content descriptor section) + - .planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md (Pattern 1) + + + Per D-93 and P33-01 (as answered in Task 1): add the fields and the five value types to `modules/pact/content.go`, each with a one-sentence comment in the file's style and no SEO wording (D-90): Kind and Tools on ContentDescriptor with the Kind and Tool constants; PublishedAt, Parents, Authors, TopHeading on ContentRecord; Images, Videos, Headings on ContentLocale. Tools values are full MCP tool names. No existing field, signature or comment meaning changes (D-106). signpost needs no logic change if Describe and Records already return the type's records as given; read `modules/signpost/content.go` and change it only if it copies records field by field. Add `modules/signpost/content_fields_smoke_test.go` with `TestContentDescriptorFields`: register a fake type with Kind article and Tools {edit: acme_update_page}, whose Describe answers PublishedAt, two Parents, one Author with a Ref, two Images (one decorative with empty alt), one Video and three Headings with TopHeading 2; assert signpost.Describe and signpost.Records return every value unchanged and that `ContentType(key)` exposes Kind and Tools; a second fake type that sets none of them describes exactly as before. Extend the runnable example in `example_content_test.go` with the new fields (Output comment updated). Document the fields in the descriptor tables of `modules/pact/README.md`, `modules/signpost/README.md` and `docs/plugins/menu-item-types.md` (every identifier must exist; the docs checker enforces it). Commit as listed in Execution units. + + + go test ./modules/signpost -run 'TestContentDescriptorFields|Example' -count=1 -v 2>&1 | tail -30 + non-zero exit, a "--- FAIL" line, "no tests to run", or no "--- PASS: TestContentDescriptorFields" line + go vet ./... && go test -short ./... + non-zero exit or a "FAIL" line + go test ./cmd/summer -run TestDocsTree -count=1 2>&1 | tail -3 && go run ./cmd/summer docs:build --check 2>&1 | tail -5 + non-zero exit, or a "FAIL" or "unknown identifier" line in the output + + + - `grep -n "PublishedAt \*time.Time" modules/pact/content.go` prints one line; `grep -n "^type Content\(Ref\|Person\|Image\|Video\|Heading\) struct" modules/pact/content.go` prints five lines + - `grep -n "ContentKindArticle\|ContentToolImageAlt" modules/pact/content.go` prints at least two lines + - `go doc ./modules/pact ContentRecord` lists PublishedAt, Parents, Authors and TopHeading + - TestContentDescriptorFields covers the set-everything type and the set-nothing type and passes + - docs/plugins/menu-item-types.md names every new field + + A content type can describe the new facts and every consumer of signpost receives them; types that do not set them are unaffected. + + + + Task 3: Per-request options for plugin field types (RequestOptions) merged into the served schema + modules/pact/capabilities.go, modules/cabana/field_type.go, modules/cabana/http.go, modules/cabana/admin_openapi.go, modules/cabana/README.md, modules/cabana/field_request_options_smoke_test.go, modules/pact/README.md, docs/backend/field-types.md, admin/openapi/admin.json, admin/src/api/schema.d.ts + + - modules/pact/capabilities.go (FormFieldType ~689-725) + - modules/cabana/field_type.go (compile of TypeOptions ~240-270, Decode/Load call sites) + - modules/cabana/http.go (the per-request form schema filter, where visible fields are chosen per administrator) + - modules/cabana/schema_types.go (TypeOptions JSON) + - modules/cabana/admin_openapi.go (~566, the form description that names typeOptions) + - an existing plugin field type test in modules/cabana (grep -ln "FormFieldType{" modules/cabana/*_test.go) for the fixture pattern + - docs/backend/field-types.md + + + - A field type with RequestOptions returning {headingLevels: [2,3,4]} serves typeOptions containing headingLevels on the create form schema, the update form schema and the preview schema + - The same key set in fields.yaml for that field wins over the request value + - RequestOptions runs once per schema request for each field type, however many fields use the type + - RequestOptions returning an error serves the schema with the fields.yaml options only, answers 200, and logs one warning naming the field type + - A field type without RequestOptions serves byte-identical schema JSON to v0.5.1 (CanonicalJSON comparison like TestFormExtensionUnextendedSchemaUnchanged) + - A form extension's plugin field (32-01) gets its request options the same way + + + Per P33-02: add `RequestOptions func(ctx context.Context, db *gorm.DB) (map[string]any, error)` with `json:"-"` and a doc comment to `pact.FormFieldType`. In cabana's form schema path (http.go, where the per-request schema copy is built), for each plugin field whose type has RequestOptions, call it once per type per request with the request's read DB, and merge the result into a copy of that field's TypeOptions (never mutate the compiled schema; keys already present win). On error, log once with the type name and serve the unmerged options. Write `TestFieldTypeRequestOptions` (subtests per behavior line) in `modules/cabana/field_request_options_smoke_test.go` first, then implement. Update the swag description in `admin_openapi.go` (typeOptions: "the type's own fields.yaml keys, plus per-request options the type supplies") and regenerate with `scripts/check-admin-openapi.sh`; commit `admin/openapi/admin.json` and `admin/src/api/schema.d.ts` only if they change. Document RequestOptions in `docs/backend/field-types.md`, `modules/cabana/README.md` and `modules/pact/README.md`. Commit as listed in Execution units. + + + go test ./modules/cabana -run 'TestFieldTypeRequestOptions|TestFormExtensionUnextendedSchemaUnchanged' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok|PASS)' | tail -30 + a "FAIL" line, "no tests to run", or no "--- PASS: TestFieldTypeRequestOptions" line + scripts/check-admin-openapi.sh --check + non-zero exit (the committed OpenAPI or schema.d.ts differs from the generated one) + go vet ./... && go test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "RequestOptions func(ctx context.Context, db \*gorm.DB) (map\[string\]any, error)" modules/pact/capabilities.go` prints one line + - `grep -n "RequestOptions" modules/cabana/http.go modules/cabana/field_type.go` prints at least one line + - `grep -n "RequestOptions" docs/backend/field-types.md` prints at least one line + - TestFieldTypeRequestOptions has a subtest for each behavior line and passes + + A plugin field type can deliver per-request options to its element through the existing typeOptions attribute; nothing changes for other field types. + + + + Task 4: Phase gate script and the v0.5.2 release log + scripts/check-phase33.sh, .planning/phases/33-seo-output-audit-and-content/33-release-log.md + + - scripts/check-phase32.sh (whole file: usage, detect_go, go_named, run_* stages, run_self_test, additive_compare, run_compat, run_all, run_stage) + - scripts/apisurface/main.go + - .planning/phases/32-seo-foundation/32-06-release-log.md (ledger format) + - .planning/phases/33-seo-output-audit-and-content/33-VALIDATION.md (Wave 0 and the decision-to-test map) + - .planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md (Validation Architecture, release order) + + + Create `scripts/check-phase33.sh` from check-phase32.sh's engine (keep detect_go, go_named, the scratch sibling farm, additive and compat logic and the self-test planting style; label everything Phase 33). Stages and what each runs: + - `--go`: `go vet ./...` and `go test -short ./...` here; + - `--plugins`: vet and `test -short` in ../sm-sitemap-plugin, ../sm-seo-plugin, ../sm-journal-plugin, plus `npm --prefix run test` (vitest) in sm-seo-plugin and sm-journal-plugin; + - `--layers`: for ../vue-seo-layer, ../vue-journal-layer, ../vue-menu-layer, ../vue-faq-layer: `npm --prefix run test`, `npm --prefix run build:playground`, and the layer's SSR smoke (journal: `node testing/smoke-ssr.mjs --output .playground/.output --fixtures testing/fixtures/acme --expect testing/expect-acme.txt`; seo: `npm --prefix ../vue-seo-layer run smoke:ssr`; menu and faq: `npm --prefix run smoke:ssr`), and `node testing/guard-check.mjs` in vue-seo-layer (the build-guard fixtures 33-08 adds); + - `--consumers`: build sm-demo-app, sm-grzybyfunkcjonalne-app, sm-summercmsio-app and sm-bm-app (Go `go build ./...`; the Nuxt apps' `build` script); + - `--docker`: the Docker suites (`go test ./...` without -short) of the three Go plugins; + - `--admin`: `scripts/check-admin-openapi.sh --check` and `scripts/check-admin-dist.sh`; + - `--docs`: TestDocsTree and `go run ./cmd/summer docs:build --check`; + - `--marker`: the framework diff since v0.5.1 (`git diff v0.5.1 -- modules docs`) names no SEO word, sitemap, JSON-LD or consuming application (the D-90 word list of check-phase32.sh's marker stage, plus the app names grzyby, fonoteka, plytarium, summercmsio, demo.summercms); + - `--additive`: apisurface of every changed framework package against the newest tag not containing HEAD; + - `--compat`: three consumers tested (sm-sitemap-plugin, sm-journal-plugin, sm-demo-app) and every other sibling that requires the framework built. + A stage whose repository or directory is missing fails with "missing " (33-14's `--all` needs every repository; vue-seo-layer appears in 33-07). `--self-test` plants: an additive removal, an interface method added, a marker hit, a failing go test, a missing repo; each must be detected. Then write `33-release-log.md` with sections "Planned contents" (framework v0.5.2 = the six fix SHAs above plus this plan's two seams; `git -C ../sm-seo-plugin log --oneline v0.1.0..HEAD` and `git -C ../sm-journal-plugin log --oneline v0.2.5..HEAD` pasted as the unreleased plugin fixes), "Release order" (framework → sm-journal-plugin → sm-sitemap-plugin → sm-seo-plugin → vue-seo-layer → vue-journal-layer, vue-menu-layer, vue-faq-layer → apps; versions read from `git tag` in 33-14), and an empty "Executed steps" table. Commit group A (script) and group B (release log, planning docs). + + + bash scripts/check-phase33.sh --self-test + non-zero exit, or a planted case reported as undetected + bash scripts/check-phase33.sh --go --docs --marker --additive + non-zero exit, or a "FAIL", "refuse:" or marker hit line + + + - `bash scripts/check-phase33.sh` with no argument prints a usage listing all eleven stage flags and exits 2 + - `grep -c "missing " scripts/check-phase33.sh` is at least 1 (missing-repository failure path) + - `grep -n "676ca53\|5471b66\|d660cd0\|468af06\|1583651\|0e187cf" .planning/phases/33-seo-output-audit-and-content/33-release-log.md` prints six lines + - the additive stage compares against v0.5.1 and passes + + The phase has its gate script and a release ledger; v0.5.2's contents are written down and nothing is released. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| plugin code → framework API | plugins fill descriptor fields and RequestOptions; consumers trust the values only as data | +| admin SPA ← form schema | request options reach the browser inside typeOptions | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-33-01 | Information disclosure | RequestOptions in the served schema | medium | mitigate | the schema is served only to authenticated administrators with access to the form (existing cabana guard); the doc comment states that request options must hold no secret, and TestFieldTypeRequestOptions asserts the merge happens only in the authenticated schema handler | +| T-33-02 | Denial of service | RequestOptions called per field | low | mitigate | called once per field type per request (behavior test), errors fail open | +| T-33-03 | Tampering | breaking published API | high | mitigate | additive stage of check-phase33.sh (apisurface vs v0.5.1) and the Task 1 checkpoint | +| T-33-SC | Tampering | npm/pip/cargo installs | high | accept | this plan installs no package | + + + +- `go vet ./... && go test -short ./...` green after every commit. +- `bash scripts/check-phase33.sh --self-test` and `--go --docs --marker --additive` pass. +- `scripts/check-admin-openapi.sh --check` passes. + + + +- The descriptor fields and RequestOptions exist with docs and smoke tests, strictly additive against v0.5.1. +- check-phase33.sh exists and its self-test passes. +- The release ledger records v0.5.2's contents; no push, no tag. + + +## Artifacts this phase produces + +- Struct fields: `pact.ContentDescriptor.Kind`, `pact.ContentDescriptor.Tools`, `pact.ContentRecord.PublishedAt`, `pact.ContentRecord.Parents`, `pact.ContentRecord.Authors`, `pact.ContentRecord.TopHeading`, `pact.ContentLocale.Images`, `pact.ContentLocale.Videos`, `pact.ContentLocale.Headings`, `pact.FormFieldType.RequestOptions`. +- Types: `pact.ContentRef`, `pact.ContentPerson`, `pact.ContentImage`, `pact.ContentVideo`, `pact.ContentHeading`. +- Constants: `pact.ContentKindArticle`, `pact.ContentKindCollection`, `pact.ContentKindProfile`, `pact.ContentKindPage`, `pact.ContentToolEdit`, `pact.ContentToolHeadings`, `pact.ContentToolImageAlt`. +- Tests: `TestContentDescriptorFields` (modules/signpost/content_fields_smoke_test.go), `TestFieldTypeRequestOptions` (modules/cabana/field_request_options_smoke_test.go). +- CLI: `scripts/check-phase33.sh` with `--self-test`, `--go`, `--plugins`, `--layers`, `--consumers`, `--docker`, `--admin`, `--docs`, `--marker`, `--additive`, `--compat`, `--all`. +- Files: `.planning/phases/33-seo-output-audit-and-content/33-release-log.md`. +- Config keys: none. + + +Create `.planning/phases/33-seo-output-audit-and-content/33-01-SUMMARY.md` when done; record the Task 1 answer (including any renamed field) at the top so plans 33-02 to 33-14 use the final names. + diff --git a/.planning/phases/33-seo-output-audit-and-content/33-02-PLAN.md b/.planning/phases/33-seo-output-audit-and-content/33-02-PLAN.md new file mode 100644 index 00000000..d9eebc23 --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-02-PLAN.md @@ -0,0 +1,363 @@ +--- +phase: 33-seo-output-audit-and-content +plan: 02 +type: execute +wave: 2 +depends_on: ["33-01"] +files_modified: + - ../sm-journal-plugin/models/settings.go + - ../sm-journal-plugin/models/settings/fields.yaml + - ../sm-journal-plugin/updates/202610280001_add_golem15_journal_settings_headings.go + - ../sm-journal-plugin/config/config.yaml + - ../sm-journal-plugin/plugin.go + - ../sm-journal-plugin/classes/headings.go + - ../sm-journal-plugin/classes/journal_settings.go + - ../sm-journal-plugin/classes/blog_editor.go + - ../sm-journal-plugin/classes/format_html.go + - ../sm-journal-plugin/classes/media_store.go + - ../sm-journal-plugin/classes/media_names.go + - ../sm-journal-plugin/classes/image_alt.go + - ../sm-journal-plugin/classes/content_descriptors.go + - ../sm-journal-plugin/classes/content_facts.go + - ../sm-journal-plugin/classes/menu_item_types.go + - ../sm-journal-plugin/menu_item_types.go + - ../sm-journal-plugin/controllers/api/posts.go + - ../sm-journal-plugin/controllers/api/settings.go + - ../sm-journal-plugin/controllers/api/maintenance.go + - ../sm-journal-plugin/routes.go + - ../sm-journal-plugin/mcp.go + - ../sm-journal-plugin/console/fix_headings.go + - ../sm-journal-plugin/console/import_posts.go + - ../sm-journal-plugin/assets/src/blogeditor/element.ts + - ../sm-journal-plugin/assets/src/blogeditor/extensions.ts + - ../sm-journal-plugin/assets/src/blogeditor/toolbar.ts + - ../sm-journal-plugin/assets/src/blogeditor/outline.ts + - ../sm-journal-plugin/assets/src/blogeditor/linkform.ts + - ../sm-journal-plugin/assets/src/blogeditor/imageform.ts + - ../sm-journal-plugin/assets/src/blogeditor/labels.ts + - ../sm-journal-plugin/assets/src/blogeditor/blogeditor.css + - ../sm-journal-plugin/assets/dist/blogeditor.js + - ../sm-journal-plugin/assets/dist/blogeditor.css + - ../sm-journal-plugin/tests/blogeditor/fixtures/editor-corpus.json + - ../sm-journal-plugin/lang/en/lang.yaml + - ../sm-journal-plugin/lang/pl/lang.yaml + - ../sm-journal-plugin/README.md + - ../sm-journal-plugin/headings_test.go + - ../sm-journal-plugin/journal_descriptor_fields_test.go + - ../sm-journal-plugin/classes/media_names_test.go +autonomous: true +requirements: [SC-2, SC-3, D-50, D-51, D-81, D-83, D-85, D-96, D-43, D-93, D-10] + +estimate: + tokens: 150000 + raw_tokens: 150000 + tasks: 4 + confidence: low + +must_haves: + truths: + - "Journal has a per-site setting content_h1_to_h2 (Settings › Journal, new tab Headings and links, switch default on) with the config key golem15.journal.content_h1_to_h2 as the value when no settings row exists (2026-10-28 Journal H1 setting, amends D-50)" + - "While the setting is on, every H1 saved in post content becomes an H2 on every write path: admin save, the /_journal/api/v1 write API (and so MCP), the markdown path and the import command; the admin save answers the plural notice journal:editor.h1_converted on the editor field and the API answer carries the same phrase in its notices (D-50, D-96, UI-SPEC C27)" + - "While the setting is off, content H1 is stored as written, the editor offers H1 to H4, no conversion notice appears, and journal:fix-headings refuses with exit code 1 and the copy that names the setting (2026-10-28 Journal H1 setting, UI-SPEC partial: H1 setting off)" + - "Switching the setting on or off never rewrites stored content; only journal:fix-headings --apply (dry run by default) and the MCP tool journal_fix_headings with dry_run false convert stored H1 headings (D-50, UI-SPEC Destructive actions)" + - "GET /_journal/api/v1/posts/{slug} carries content_h1_to_h2 (boolean, additive key) so the journal layer reads the setting from the Journal API, not from a layer option (2026-10-28 Journal H1 setting)" + - "The Blog Editor receives headingLevels and genericAnchors per request through RequestOptions: H2 to H4 with a pasted H1 parsed as H2 while the setting is on, H1 to H4 while it is off (D-51, UI-SPEC C28 J1)" + - "The editor's outline panel (J2) lists headings in document order and flags skipped levels, empty headings, link-only headings, an H1 in Source mode while the setting is on, and links whose whole text is on the generic list; it recomputes 300 ms after a change, starts collapsed, never takes focus on load and never edits content (D-51, D-81)" + - "The link form (J3) offers nofollow, sponsored and ugc checkboxes only for absolute http(s) URLs, writes rel with the checked values, never puts rel on internal links, and warns without blocking when the link text is on the generic list; the stored-HTML policy keeps rel with only those three tokens (D-83, D-81, D-80)" + - "The image form (J4) sets a slugged file name before upload (lowercase, Polish diacritics transliterated, non-alphanumeric runs to one hyphen, at most 80 characters, empty becomes image), requires alt unless Decorative image is checked, and inserts alt=\"\" for a decorative image; the server applies the same slug rule to uploaded file names (D-85)" + - "Journal's descriptors fill the 33-01 fields: posts give PublishedAt, Parents (their categories in order), Authors (with a journal-author reference when author pages are on), Images (featured image plus body images with alt and decorative), Videos (YouTube embeds), Headings and TopHeading (2 while the setting is on, 0 while off), Kind article and Tools {edit: journal_update_post, headings: journal_fix_headings, image_alt: journal_set_image_alt}; categories give Parents (their parent) and Kind collection; tags give Kind collection (D-93, D-43, D-96)" + - "A new journal-author menu item type describes author pages (Kind profile, Records of authors with at least one published post, exact Match on the author pattern, host key golem15.journal.author_pages default true), so author pages get the SEO tab, ProfilePage data and a sitemap group (D-43, D-34)" + - "MCP has journal_get_settings, journal_update_settings, journal_fix_headings (dry_run) and journal_set_image_alt (dry_run, bulk), each bound to a /_journal/api/v1 route with golem15.journal.manage_settings or golem15.journal.access_posts (D-10, D-85, UI-SPEC MCP parity)" + - "The admin save notice, the outline counts and every plural phrase use CLDR maps (en one/other, pl one/few/many/other) (UI-SPEC zero-one-many)" + - "J2 shows outlineEmpty when the text has no heading (UI-SPEC empty: J2)" + - "J4 keeps the existing uploading status line until the upload result arrives, then inserts the image (UI-SPEC loading: J4)" + - {statement: "J4 slug rule: \"Lwia Grzywa – w lesie!.JPG\" becomes lwia-grzywa-w-lesie with the extension .jpg, an empty name becomes image, and names are cut at 80 characters (UI-SPEC J4 backstop)", verification: backstop} + - "Planner assumption (UI-SPEC unresolved, J2 long text): outline items with long heading text wrap inside the panel with a hanging indent under the level tag and are never truncated; verify in the editor at 360 px" + - "Planner assumption: the H1 setting covers post content only (the setting's label says post content); category pillar content is not converted, and a second H1 there is reported by the crawler as h1_multiple" + prohibitions: + - requirement_id: "D-50" + category: values + status: resolved + verification: test + statement: "Turning the H1 setting on or off never rewrites already stored content" + - requirement_id: "D-96" + category: values + status: resolved + verification: judgment + statement: "Journal never imports sm-seo-plugin or names SEO settings; everything SEO reads comes through the signpost descriptor" + artifacts: + - path: "../sm-journal-plugin/classes/headings.go" + provides: "ConvertH1 on the x/net/html tokenizer" + contains: "func ConvertH1(" + - path: "../sm-journal-plugin/console/fix_headings.go" + provides: "journal:fix-headings" + contains: "journal:fix-headings" + - path: "../sm-journal-plugin/classes/content_facts.go" + provides: "body images, videos and headings from content_html" + contains: "pact.ContentHeading" + - path: "../sm-journal-plugin/headings_test.go" + provides: "TestHeadingConversionTracer, TestHeadingConversion, TestFixHeadingsCommand, TestSetImageAlt, TestJournalSettingsTools" + contains: "func TestHeadingConversionTracer(" + - path: "../sm-journal-plugin/journal_descriptor_fields_test.go" + provides: "TestJournalDescriptorFields, TestJournalAuthorType" + contains: "func TestJournalDescriptorFields(" + key_links: + - from: "../sm-journal-plugin/classes/blog_editor.go" + to: "../sm-journal-plugin/classes/headings.go" + via: "decodeBlogHTML calls ConvertH1 when the setting is on and returns the h1_converted notice" + pattern: "ConvertH1\\(" + - from: "../sm-journal-plugin/controllers/api/posts.go" + to: "../sm-journal-plugin/classes/journal_settings.go" + via: "Show adds content_h1_to_h2 from the effective setting; Store and Update convert and return notices" + pattern: "content_h1_to_h2" + - from: "../sm-journal-plugin/classes/blog_editor.go" + to: "summercms.go/modules/pact/capabilities.go" + via: "the blogeditor and mlblogeditor types set RequestOptions to the settings' heading levels and generic list" + pattern: "RequestOptions" +--- + + + +## Phase Goal + +**As a** site editor and as an SEO specialist auditing a SummerCMS site, **I want to** have the rendered sites emit complete SEO output (sitemaps, structured data, pagination, hreflang, headings, breadcrumbs) from the CMS, and an audit that finds the remaining problems, **so that** sites are indexed correctly and issues can be fixed through MCP. + +This plan makes Journal, the first content plugin, supply what the rest of the phase consumes and fix its own content (D-96): the per-site H1 setting with conversion on every write path, the editor's heading levels, outline, link rel and image naming, the new descriptor facts, author pages as a content type, and MCP tools for its settings, stored headings and image alt. + + +Implement in `../sm-journal-plugin` (branch main, not pushed): +- the Journal H1 setting (2026-10-28, amends D-50/D-51): column, config key, settings tab, API flag, conversion with notices on every write path, `journal:fix-headings`; +- Blog Editor J1–J4 (UI-SPEC C28) on RequestOptions (33-01 P33-02), with the stored-HTML policy keeping `rel` (D-83) and server-side slugged media names (D-85); +- descriptor facts for posts, categories and tags plus a `journal-author` type (D-93, D-43); +- MCP tools `journal_get_settings`, `journal_update_settings`, `journal_fix_headings`, `journal_set_image_alt` (D-10, D-85). + +Purpose: the layer's title element and htmlGate demotion (33-09), JSON-LD and breadcrumbs (33-06), image and News sitemaps (33-04) and the audit's CMS checks (33-10) all read what this plan provides. + +Output: commits in ../sm-journal-plugin (main, not pushed; 33-14 tags). The framework is used through the existing `replace => ../summercms.go`. + +**Planner decisions recorded here:** +- **P33-04 (setting precedence):** the settings row's `content_h1_to_h2` decides; the config key `golem15.journal.content_h1_to_h2` (default true, new `config/config.yaml` shipped through `ConfigFS`) is used only when no settings row exists. `classes.EffectiveH1ToH2(ctx, db, cfg) bool` is the one reader; README documents it. +- **P33-05 (bulk alt tool lives in Journal):** the UI-SPEC's `[ASSUMED]` `seo_set_image_alt` becomes `journal_set_image_alt`: alt text is post content, which only Journal may write (D-96); the SEO issue report points at it through `ContentDescriptor.Tools["image_alt"]` (33-10). Research had the tool in 33-11; it moves here with Journal's other content fixes. +- **P33-06 (featured image alt):** the featured image alt in descriptors and in the API stays as today (UI-SPEC Q-6): the attachment's title when set, otherwise the post title. +- **P33-07 (author type URL):** `journal-author` default pattern `/blog/author/:author` (the journal layer's default route), host override through `menu_items.patterns`, `:author` is the backend user's author_slug. + + + +@~/.claude/msd-core/workflows/execute-plan.md +@~/.claude/msd-core/templates/summary.md + + + +@.planning/phases/33-seo-output-audit-and-content/33-CONTEXT.md +@.planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md +@.planning/phases/33-seo-output-audit-and-content/33-PATTERNS.md +@.planning/phases/33-seo-output-audit-and-content/33-01-SUMMARY.md +@.planning/phases/32-seo-foundation/32-02-SUMMARY.md +@CLAUDE.md + + +From 33-01 (final names in 33-01-SUMMARY.md): pact.ContentDescriptor{Kind, Tools}; pact.ContentRecord{PublishedAt, Parents []ContentRef, Authors []ContentPerson, TopHeading}; pact.ContentLocale{Images []ContentImage, Videos []ContentVideo, Headings []ContentHeading}; pact.FormFieldType.RequestOptions func(ctx, *gorm.DB) (map[string]any, error). +sm-journal-plugin today: models/settings.go (row ID 1, Fillable/Rules), models/settings/fields.yaml (tabs general, search, rss), classes/blog_editor.go (BlogEditorTypes, decodeBlogHTML returns pact.FormFieldDecoded{Value, Notices}), classes/format_html.go (bluemonday UGC policy, RequireNoFollowOnLinks(false), SanitizeHTMLReport), classes/media_store.go (StoredMedia{URL, Path}), classes/content_descriptors.go (describePosts, firstImages, describeCategories, describeTag, TagPagesKey), classes/menu_item_types.go (MenuItemTypes, menuAlternates), controllers/api/posts.go (Show ~122, Store, Update, serializePost ~587, featuredImages ~629), controllers/api/rss.go:17-40 (settings row read with defaults), console/export_posts.go:24-60 (bonfire.Command with Bare dry-run flag), mcp.go (tools bound to /_journal/api/v1 routes, permission mcpAccessPosts), admin_permissions.go (golem15.journal.manage_settings, golem15.journal.access_posts), routes.go (public group throttle journal-public-api, write group throttle journal-api). +Editor: assets/src/blogeditor/{element.ts (attributes field-name, record-id, locale, ui-locale, options JSON, disabled), toolbar.ts:45-47 heading2..4, extensions.ts:80 heading, labels.ts (format with {name}), blogeditor.css}; tests/blogeditor/*; scripts/check-dist.sh. +Copy: UI-SPEC "Journal settings, save notice and console (C27)" and "Blog Editor additions (C28)" tables, en and pl, verbatim. + + + +## Execution units and resume points + +- Commit groups per task, staged by path in ../sm-journal-plugin with `git -c safe.directory='*' -C ../sm-journal-plugin add ` then `commit -m ""`. Leave `go.work.sum` (untracked, another session's) alone. Every commit keeps `go vet ./...` and `go test -short ./...` green there; each group adds the lang keys it uses to both lang files. +- Resume check: `git -c safe.directory='*' -C ../sm-journal-plugin log --format=%s | grep -Fx ''`. + +| Task | Commit groups, in order | Resume point | +|------|-------------------------|--------------| +| 1 | `feat: H1 to H2 setting with the admin save conversion (tracer)` | TestHeadingConversionTracer passes | +| 2 | A `feat: H1 conversion on the API, markdown and import paths`; B `feat: journal:fix-headings and the settings API`; C `feat: MCP tools for settings, headings and image alt` | all write paths and tools pass | +| 3 | A `feat: descriptor dates, parents, authors, media and outline`; B `feat: journal-author content type` | descriptor tests pass | +| 4 | A `feat: stored HTML keeps link rel and media names are slugged`; B `feat: editor heading levels and outline panel`; C `feat: editor link rel, generic-anchor warning and image naming`; D `docs: headings, editor and MCP tools in README` | dist check passes; plan complete | + + + + + Task 1: End-to-end "an editor saves a post with an H1 and gets it back as an H2 with a notice, and the public API reports the setting" — one path only + ../sm-journal-plugin/models/settings.go, ../sm-journal-plugin/models/settings/fields.yaml, ../sm-journal-plugin/updates/202610280001_add_golem15_journal_settings_headings.go, ../sm-journal-plugin/config/config.yaml, ../sm-journal-plugin/plugin.go, ../sm-journal-plugin/classes/headings.go, ../sm-journal-plugin/classes/journal_settings.go, ../sm-journal-plugin/classes/blog_editor.go, ../sm-journal-plugin/controllers/api/posts.go, ../sm-journal-plugin/lang/en/lang.yaml, ../sm-journal-plugin/lang/pl/lang.yaml, ../sm-journal-plugin/headings_test.go + + - ../sm-journal-plugin/models/settings.go, models/settings/fields.yaml, updates/202610060006_create_golem15_journal_settings.go, updates/registry.go + - ../sm-seo-plugin/updates/202610280001_add_golem15_seo_redirects_source_key.go (additive ALTER with Rollback) + - ../sm-seo-plugin/config/config.yaml and ../sm-seo-plugin/plugin.go:20-30,79-80 (ConfigFS, HasConfig) + - ../sm-journal-plugin/classes/blog_editor.go (decodeBlogHTML, removalNotices) and classes/format_html.go (SanitizeHTMLReport) + - ../sm-journal-plugin/controllers/api/posts.go (Show, serializePost) and controllers/api/rss.go:17-40 + - ../sm-journal-plugin/posts_admin_smoke_test.go (admin harness for a post save) + - .planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md (C27 and its copy) + + + Per the 2026-10-28 Journal H1 setting and P33-04: migration `202610280001_add_golem15_journal_settings_headings` adds `content_h1_to_h2 boolean NOT NULL DEFAULT true` and `generic_anchors text NULL` to golem15_journal_settings (Rollback drops both), registered in `updates/registry.go` style; model fields ContentH1ToH2 bool and GenericAnchors *string added to Fillable and Rules (boolean; nullable|string|max:5000). fields.yaml gets the tab `golem15.journal::lang.journal.tab_headings` after General with `content_h1_to_h2` (switch, span left, default true, label h1_to_h2_label, comment h1_to_h2_comment) and `generic_anchors` (textarea size small, span full, default = the en and pl seed lists of C27 joined by newlines). New `config/config.yaml` with the commented key `content_h1_to_h2: true`; plugin.go embeds it, implements `ConfigFS()` and asserts `pact.HasConfig`. `classes/journal_settings.go`: `EffectiveH1ToH2(ctx, db, cfg) bool` and `GenericAnchors(ctx, db) []string` (row value, else the seed lists). `classes/headings.go`: `ConvertH1(html string) (string, int)` rewriting `

`/`

` start and end tags to h2 with the x/net/html tokenizer, keeping attributes and every other byte. decodeBlogHTML calls it when EffectiveH1ToH2 is true and appends the notice `golem15.journal::lang.editor.h1_converted` with Count n (FormFieldNotice). Show adds `content_h1_to_h2` to the post payload (additive key; no existing key changes). Lang keys of C27 in en and pl (CLDR plurals). `TestHeadingConversionTracer` in headings_test.go: admin save of a post whose content holds `

A

x

` stores `

A

x

` and the save response meta.notices has the converted phrase with count 1; GET /_journal/api/v1/posts/{slug} answers content_h1_to_h2 true; with the setting switched off the same save stores the H1 unchanged, adds no notice, and Show answers false. Commit as listed. + + + go -C ../sm-journal-plugin test ./... -run 'TestHeadingConversionTracer' -count=1 -v 2>&1 | tail -20 + non-zero exit, "FAIL" or "no tests to run" in the output, or no "--- PASS: TestHeadingConversionTracer" line + go -C ../sm-journal-plugin vet ./... && go -C ../sm-journal-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "func ConvertH1(" ../sm-journal-plugin/classes/headings.go` prints one line + - `grep -n "content_h1_to_h2" ../sm-journal-plugin/updates/202610280001_add_golem15_journal_settings_headings.go ../sm-journal-plugin/models/settings/fields.yaml ../sm-journal-plugin/controllers/api/posts.go` prints a line in each file + - `grep -n "h1_converted" ../sm-journal-plugin/lang/en/lang.yaml ../sm-journal-plugin/lang/pl/lang.yaml` prints a line in each + - TestHeadingConversionTracer asserts both setting states and passes + + The setting exists, the admin save converts H1 with a notice while it is on, and the public post answer reports it. + + + + Task 2: Conversion on every other write path, journal:fix-headings, the settings API and the MCP tools (settings, fix headings, set image alt) + ../sm-journal-plugin/controllers/api/posts.go, ../sm-journal-plugin/controllers/api/settings.go, ../sm-journal-plugin/controllers/api/maintenance.go, ../sm-journal-plugin/classes/format_html.go, ../sm-journal-plugin/classes/image_alt.go, ../sm-journal-plugin/console/fix_headings.go, ../sm-journal-plugin/console/import_posts.go, ../sm-journal-plugin/routes.go, ../sm-journal-plugin/mcp.go, ../sm-journal-plugin/plugin.go, ../sm-journal-plugin/lang/en/lang.yaml, ../sm-journal-plugin/lang/pl/lang.yaml, ../sm-journal-plugin/headings_test.go + + - ../sm-journal-plugin/controllers/api/posts.go (Store, Update, the transaction and RecordSaved emission from 62c75b1/7a03640) + - ../sm-journal-plugin/controllers/api/record_events.go and auth.go (backend-audience JWT, permissions) + - ../sm-journal-plugin/console/export_posts.go:24-60 and console/import_posts.go + - ../sm-journal-plugin/mcp.go and mcp_test.go (tool binding pattern) + - ../sm-journal-plugin/journal_api_writes_test.go (API write harness) + - .planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md (C27 console copy, MCP parity row) + + + - API Store and Update of content `

T

` store `

T

` and the JSON answer gains `notices: [{message, count}]` with the h1_converted phrase; translated content in a locale converts the same way; with the setting off nothing converts and `notices` is absent + - Markdown content (FormatHTML path) with `# T` stores an h2 while on + - `journal:import-posts` converts while on and prints the converted count + - `journal:fix-headings` (dry run default) prints "Would convert :count H1 headings in :posts posts. Run with --apply to save." and changes nothing; `--apply` converts default and translated content, fires RecordSaved per changed post, prints "Converted :count H1 headings in :posts posts."; nothing to do prints "No H1 headings found in post content."; with the setting off it exits 1 with the refusal copy + - GET/PUT /_journal/api/v1/settings read and write content_h1_to_h2 and generic_anchors (other settings columns too) with golem15.journal.manage_settings; 403 without it + - POST /_journal/api/v1/fix-headings {dry_run} answers {posts, headings, dry_run} and refuses 409 with the refusal phrase while the setting is off + - POST /_journal/api/v1/images/alt {dry_run, changes:[{post_id, locale, src, alt}]} sets alt on the matching img in content_html (default or translated), answers the per-change diff, refuses the whole batch 422 when one src is not in the post, and with dry_run writes nothing + - MCP lists journal_get_settings, journal_update_settings, journal_fix_headings and journal_set_image_alt, each bound to its route + + + Per D-50, D-96, D-85 and P33-05: call ConvertH1 in the API write paths after sanitizing (default and translated content_html, inside the existing transaction) and return the additive `notices` key; apply it in FormatHTML's caller for markdown content and in console/import_posts.go. `console/fix_headings.go`: bonfire command `journal:fix-headings` modeled on export_posts.go (flags `apply` Bare, `dry-run` implied default; Examples; English output exactly as UI-SPEC), registered in plugin.go Commands(). `controllers/api/settings.go`: ShowSettings/UpdateSettings (validation via the model's Rules). `controllers/api/maintenance.go`: FixHeadings and SetImageAlt handlers running in one transaction (dry_run rolls back), with RecordSaved per changed post through the existing emission helper; `classes/image_alt.go` rewrites the alt attribute of `` whose src equals the change's src with the x/net/html tokenizer (escaping through the tokenizer, never string concatenation). Routes in the write group (throttle journal-api). MCP tools in mcp.go: `get_settings` (GET /settings), `update_settings` (PUT /settings), `fix_headings` (POST /fix-headings), `set_image_alt` (POST /images/alt, destructive true), permissions golem15.journal.manage_settings for the first three and golem15.journal.access_posts for set_image_alt. Tests first in headings_test.go: `TestHeadingConversion` (API, markdown, import, translated), `TestFixHeadingsCommand`, `TestJournalSettingsTools`, `TestSetImageAlt`. Commit groups A, B, C as listed. + + + go -C ../sm-journal-plugin test ./... -run 'TestHeadingConversion|TestFixHeadingsCommand|TestJournalSettingsTools|TestSetImageAlt' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -30 + a "FAIL" line, "no tests to run", or one of the four test names missing from the "--- PASS" lines + go -C ../sm-journal-plugin vet ./... && go -C ../sm-journal-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "journal:fix-headings" ../sm-journal-plugin/console/fix_headings.go` prints at least one line and `commands_documented_test.go` passes (command documented) + - `grep -n '"fix_headings"\|"set_image_alt"\|"get_settings"\|"update_settings"' ../sm-journal-plugin/mcp.go` prints four lines + - the refusal copy "H1 to H2 conversion is turned off in the Journal settings (Headings and links). Turn it on first, or keep the H1 headings." appears in console/fix_headings.go + - the existing Journal API answer keys are unchanged (journal_api_matrix_test.go passes) + + Every write path converts while the setting is on, stored content is only rewritten on request, and Journal's settings and content fixes are MCP tools. + + + + Task 3: Descriptor facts for posts, categories and tags, and the journal-author content type + ../sm-journal-plugin/classes/content_descriptors.go, ../sm-journal-plugin/classes/content_facts.go, ../sm-journal-plugin/classes/menu_item_types.go, ../sm-journal-plugin/menu_item_types.go, ../sm-journal-plugin/lang/en/lang.yaml, ../sm-journal-plugin/lang/pl/lang.yaml, ../sm-journal-plugin/journal_descriptor_fields_test.go + + - ../sm-journal-plugin/classes/content_descriptors.go (describePosts, firstImages, describeCategories, describeTag, tagPages, TagPagesKey) + - ../sm-journal-plugin/classes/menu_item_types.go (MenuItemTypes, tag type registration, menuAlternates) and menu_item_types.go + - ../sm-journal-plugin/controllers/api/authors.go (author lookup by author_slug) + - ../sm-journal-plugin/content_descriptors_test.go (fixture style) + - summercms.go/modules/pact/content.go (33-01 fields) + + + - A published post with categories C1, C2 describes Parents [journal-category C1, journal-category C2] in that order, PublishedAt = published_at, Authors [{name, Ref journal-author }] while author pages are on and Ref.Type "" while off + - Its locales list Images: the featured image first (alt per P33-06), then body images in document order with alt, Decorative true for alt="" and width/height from attributes; Videos for each figure data-embed youtube; Headings from h1 to h6 with trimmed text + - TopHeading is 2 while content_h1_to_h2 is on and 0 while off; Kind article; Tools has edit, headings and image_alt with the journal_ tool names + - A child category describes Parents [its parent]; categories and tags have Kind collection + - journal-author: Records lists only authors with a published post, Describe answers the name as Title and the author URL per locale, Match is exact on the pattern, Kind profile; with golem15.journal.author_pages false Describe answers empty URLs and Match never matches + + + Per D-93, D-43 and P33-06/P33-07: `classes/content_facts.go` parses content_html once per locale with x/net/html into images (src, alt, decorative, width, height), YouTube videos and headings. Fill the 33-01 fields in describePosts, describeCategories and describeTag (same queries batched per page; no N+1 per record). Add the `journal-author` menu item type next to journal-tag (references and Resolve as the tag type does, Content descriptor with Model = the backend user morph type, Controller "", Kind profile), host key constant `AuthorPagesKey = "golem15.journal.author_pages"` (default true when unset), lang keys for its label. Tests first in journal_descriptor_fields_test.go (`TestJournalDescriptorFields`, `TestJournalAuthorType`). Commit groups A and B. + + + go -C ../sm-journal-plugin test ./... -run 'TestJournalDescriptorFields|TestJournalAuthorType|TestRecordSavedDescribesNewURL' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -20 + a "FAIL" line, "no tests to run", or TestJournalDescriptorFields or TestJournalAuthorType missing from the "--- PASS" lines + go -C ../sm-journal-plugin vet ./... && go -C ../sm-journal-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "journal-author" ../sm-journal-plugin/classes/menu_item_types.go` prints at least one line + - `grep -n "PublishedAt:\|Parents:\|TopHeading:\|Tools:" ../sm-journal-plugin/classes/content_descriptors.go` prints at least four lines + - `grep -n "golem15.journal.author_pages" ../sm-journal-plugin/classes/content_descriptors.go` prints one line + + Every Journal content type describes the facts the sitemaps, JSON-LD, breadcrumbs and the audit need, and author pages are a content type. + + + + Task 4: Blog Editor J1–J4 on RequestOptions, link rel in the stored-HTML policy, slugged media names + ../sm-journal-plugin/classes/blog_editor.go, ../sm-journal-plugin/classes/format_html.go, ../sm-journal-plugin/classes/media_store.go, ../sm-journal-plugin/classes/media_names.go, ../sm-journal-plugin/classes/media_names_test.go, ../sm-journal-plugin/assets/src/blogeditor/element.ts, ../sm-journal-plugin/assets/src/blogeditor/extensions.ts, ../sm-journal-plugin/assets/src/blogeditor/toolbar.ts, ../sm-journal-plugin/assets/src/blogeditor/outline.ts, ../sm-journal-plugin/assets/src/blogeditor/linkform.ts, ../sm-journal-plugin/assets/src/blogeditor/imageform.ts, ../sm-journal-plugin/assets/src/blogeditor/labels.ts, ../sm-journal-plugin/assets/src/blogeditor/blogeditor.css, ../sm-journal-plugin/assets/dist/blogeditor.js, ../sm-journal-plugin/assets/dist/blogeditor.css, ../sm-journal-plugin/tests/blogeditor/fixtures/editor-corpus.json, ../sm-journal-plugin/lang/en/lang.yaml, ../sm-journal-plugin/lang/pl/lang.yaml, ../sm-journal-plugin/README.md + + - ../sm-journal-plugin/assets/src/blogeditor/{element.ts, extensions.ts, toolbar.ts, labels.ts, blogeditor.css, serializer.ts} and tests/blogeditor/{smoke.test.ts, corpus.test.ts} + - ../sm-journal-plugin/classes/blog_editor.go (BlogEditorTypes), format_html.go (newHTMLPolicy), media_store.go + - ../sm-journal-plugin/classes/editor_policy_test.go (TestEditorOutputPassesPolicy and the corpus) + - .planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md (C28 J1–J4, Blog Editor copy table, Shared rules, spacing exceptions) + - ../sm-journal-plugin/scripts/check-dist.sh + + + Per UI-SPEC C28 and D-51, D-81, D-83, D-85: BlogEditorTypes sets RequestOptions to {headingLevels: [2,3,4] or [1,2,3,4] by EffectiveH1ToH2, genericAnchors: GenericAnchors()}. element.ts reads `options.headingLevels` (default [2,3,4]) into the TipTap heading levels with a parse rule mapping `

` to level 2 when 1 is absent; toolbar.ts adds `heading1` first when level 1 is allowed (label heading1). `outline.ts`: the J2 bar `blogeditor__outline-toggle` (aria-expanded, collapsed by default, never focused on load) and panel (ol of headings with level tag, flags flagSkipped, flagEmpty, flagLinkOnly, flagH1 only while level 1 is not allowed and Source mode holds an H1; outlineLinks with flagGeneric per link whose trimmed, lowercased, punctuation-stripped text is on genericAnchors; outlineEmpty; 12px indent per level below the first content level; 300 ms debounce; activating an item moves the caret to the heading; never edits content; long items wrap with a hanging indent). `linkform.ts`: J3 fieldset linkRel with relNofollow, relSponsored, relUgc enabled only for absolute http(s) URLs (relHint otherwise), Apply writes rel, internal links get none, genericWarning line (aria-live polite) without blocking Apply. `imageform.ts`: J4 queue form per file with fileName (slug rule on blur and Insert, extension shown fixed), altText (altRequired unless decorative), decorative checkbox (clears and disables alt, inserts alt=""), caption; Insert uploads the renamed file and inserts on the upload result. labels.ts gets every C28 key in en and pl (CLDR plurals for outlineCount and outlineProblems); blogeditor.css copies the editor metrics listed in UI-SPEC Spacing exceptions, light DOM, `--c-*` colours only. Server: format_html.go allows `rel` on `a` matching `^(nofollow|sponsored|ugc)( (nofollow|sponsored|ugc))*$` (bluemonday AllowAttrs.Matching.OnElements); `classes/media_names.go` `MediaFileName(name string) string` implements the slug rule (smoke test `TestMediaFileName` in classes/media_names_test.go with the UI-SPEC example, an empty name and an 81-character name) and media_store.go applies it to uploads; editor-corpus.json gains a rel link and a decorative image so TestEditorOutputPassesPolicy pins them. Rebuild the bundle with `npm --prefix ../sm-journal-plugin run build` and commit assets/dist with the sources. README: the setting, its precedence, the command, the MCP tools, journal-author and the editor additions. Commit groups A to D as listed. + + + npm --prefix ../sm-journal-plugin run test 2>&1 | tail -15 + non-zero exit, or a "FAIL" or "failed" count above 0 in the vitest summary + bash ../sm-journal-plugin/scripts/check-dist.sh + non-zero exit (the committed bundle differs from a fresh build or typecheck fails) + go -C ../sm-journal-plugin test ./... -run 'TestEditorOutputPassesPolicy|TestMediaFileName|TestBlogEditor' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -20 + a "FAIL" line or "no tests to run" + + + - `grep -n "headingLevels" ../sm-journal-plugin/assets/src/blogeditor/element.ts ../sm-journal-plugin/classes/blog_editor.go` prints a line in each + - `grep -n "nofollow|sponsored|ugc" ../sm-journal-plugin/classes/format_html.go` prints one line + - `grep -n "flagSkipped\|genericWarning\|decorative" ../sm-journal-plugin/assets/src/blogeditor/labels.ts` prints at least three lines + - `grep -n "func MediaFileName(" ../sm-journal-plugin/classes/media_names.go` prints one line, and a Go test asserts "Lwia Grzywa – w lesie!.JPG" → "lwia-grzywa-w-lesie.jpg" + - check-dist.sh passes after the dist commit + + The editor follows the per-site heading setting, shows the outline and link warnings, writes rel on external links, names and describes images properly, and the server enforces the same rules. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| admin/API/MCP client → Journal write paths | untrusted HTML content and alt text | +| editor element ← typeOptions | per-request options shown to administrators | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-33-04 | Tampering | ConvertH1 / image alt rewrite | high | mitigate | both work on x/net/html tokens after the bluemonday policy, never on raw string replacement; attribute values are re-escaped by the tokenizer; TestSetImageAlt includes an alt with quotes and angle brackets | +| T-33-05 | Elevation of privilege | settings and maintenance routes | high | mitigate | backend-audience JWT plus golem15.journal.manage_settings / access_posts, 403 tests in TestJournalSettingsTools | +| T-33-06 | Tampering | link rel attribute | medium | mitigate | the policy accepts only the three rel tokens (regex), the corpus test pins it | +| T-33-07 | Tampering | uploaded file names | medium | mitigate | MediaFileName slugs server-side ([a-z0-9-], 80 characters) whatever the client sends | +| T-33-08 | Denial of service | fix-headings over a large site | low | accept | CLI/MCP operation by an administrator, one transaction per post, dry run by default | +| T-33-SC | Tampering | npm/pip/cargo installs | high | accept | no new package; the bundle rebuilds from the committed lockfile | + + + +- `go -C ../sm-journal-plugin vet ./... && go -C ../sm-journal-plugin test -short ./...` green after every commit. +- Docker suites: TestHeadingConversionTracer, TestHeadingConversion, TestFixHeadingsCommand, TestJournalSettingsTools, TestSetImageAlt, TestJournalDescriptorFields, TestJournalAuthorType. +- `npm --prefix ../sm-journal-plugin run test` and `bash ../sm-journal-plugin/scripts/check-dist.sh` pass. + + + +- The H1 setting works in both modes on every write path, and the API exposes it. +- The editor additions J1–J4 work as UI-SPEC C28 describes. +- Descriptors carry the new facts and author pages are a content type. +- Four Journal MCP tools exist. + + +## Artifacts this phase produces + +- Config key: `golem15.journal.content_h1_to_h2` (default true), `golem15.journal.author_pages` (default true). +- Settings columns: `golem15_journal_settings.content_h1_to_h2`, `golem15_journal_settings.generic_anchors`; migration `202610280001_add_golem15_journal_settings_headings`. +- Go symbols: `classes.ConvertH1`, `classes.EffectiveH1ToH2`, `classes.GenericAnchors`, `classes.MediaFileName`, `classes.AuthorPagesKey`, API handlers `ShowSettings`, `UpdateSettings`, `FixHeadings`, `SetImageAlt`; `Plugin.ConfigFS`. +- Menu item type: `journal-author`. +- Routes: `GET /_journal/api/v1/settings`, `PUT /_journal/api/v1/settings`, `POST /_journal/api/v1/fix-headings`, `POST /_journal/api/v1/images/alt`; additive keys `content_h1_to_h2` (post Show) and `notices` (write answers). +- CLI: `journal:fix-headings [--apply]`. +- MCP tools: `journal_get_settings`, `journal_update_settings`, `journal_fix_headings`, `journal_set_image_alt`. +- Files: `config/config.yaml`, `classes/headings.go`, `classes/journal_settings.go`, `classes/content_facts.go`, `classes/image_alt.go`, `classes/media_names.go`, `console/fix_headings.go`, `controllers/api/settings.go`, `controllers/api/maintenance.go`, `assets/src/blogeditor/{outline,linkform,imageform}.ts`, `headings_test.go`, `journal_descriptor_fields_test.go`. +- Lang keys: the C27 and C28 keys (`journal.tab_headings`, `journal.h1_to_h2_label`, `journal.h1_to_h2_comment`, `journal.generic_anchors_label`, `journal.generic_anchors_comment`, `editor.h1_converted`, and the element labels). + + +Create `.planning/phases/33-seo-output-audit-and-content/33-02-SUMMARY.md` when done. + diff --git a/.planning/phases/33-seo-output-audit-and-content/33-03-PLAN.md b/.planning/phases/33-seo-output-audit-and-content/33-03-PLAN.md new file mode 100644 index 00000000..8d295e62 --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-03-PLAN.md @@ -0,0 +1,396 @@ +--- +phase: 33-seo-output-audit-and-content +plan: 03 +type: execute +wave: 2 +depends_on: ["33-01"] +files_modified: + - ../sm-sitemap-plugin/contract/contract.go + - ../sm-sitemap-plugin/classes/slots.go + - ../sm-sitemap-plugin/classes/rebuild.go + - ../sm-sitemap-plugin/classes/writer.go + - ../sm-sitemap-plugin/classes/lastmod.go + - ../sm-sitemap-plugin/classes/types.go + - ../sm-sitemap-plugin/classes/listing.go + - ../sm-sitemap-plugin/classes/generator.go + - ../sm-sitemap-plugin/classes/generator_test.go + - ../sm-sitemap-plugin/classes/serialize.go + - ../sm-sitemap-plugin/classes/serialize_test.go + - ../sm-sitemap-plugin/classes/golden_test.go + - ../sm-sitemap-plugin/classes/testdata/full.golden + - ../sm-sitemap-plugin/classes/testdata/full.json + - ../sm-sitemap-plugin/classes/testdata/tracer.golden + - ../sm-sitemap-plugin/classes/testdata/tracer.json + - ../sm-sitemap-plugin/classes/testdata/urlto.golden + - ../sm-sitemap-plugin/classes/testdata/urlto.json + - ../sm-sitemap-plugin/classes/tiers.go + - ../sm-sitemap-plugin/classes/tiers_test.go + - ../sm-sitemap-plugin/classes/rewrite.go + - ../sm-sitemap-plugin/classes/rewrite_test.go + - ../sm-sitemap-plugin/classes/manifest.go + - ../sm-sitemap-plugin/classes/manifest_test.go + - ../sm-sitemap-plugin/classes/urlto.go + - ../sm-sitemap-plugin/classes/urlto_test.go + - ../sm-sitemap-plugin/classes/events.go + - ../sm-sitemap-plugin/assets/sitemap.xsl + - ../sm-sitemap-plugin/tests/php/gen-goldens.php + - ../sm-sitemap-plugin/models/settings.go + - ../sm-sitemap-plugin/models/settings/fields.yaml + - ../sm-sitemap-plugin/models/settings_test.go + - ../sm-sitemap-plugin/models/url.go + - ../sm-sitemap-plugin/models/type_setting.go + - ../sm-sitemap-plugin/models/definition.go + - ../sm-sitemap-plugin/models/definition_item.go + - ../sm-sitemap-plugin/models/definition/fields.yaml + - ../sm-sitemap-plugin/models/definition_test.go + - ../sm-sitemap-plugin/models/registry.go + - ../sm-sitemap-plugin/updates/202610280001_create_golem15_sitemap_urls.go + - ../sm-sitemap-plugin/updates/202610280002_update_golem15_sitemap_settings.go + - ../sm-sitemap-plugin/updates/202610280003_create_golem15_sitemap_types.go + - ../sm-sitemap-plugin/updates/registry.go + - ../sm-sitemap-plugin/controllers/admin_registry.go + - ../sm-sitemap-plugin/controllers/settings.go + - ../sm-sitemap-plugin/controllers/settings/config_form.yaml + - ../sm-sitemap-plugin/controllers/settings/_sitemap_files.htm + - ../sm-sitemap-plugin/controllers/definitions.go + - ../sm-sitemap-plugin/controllers/definitions/config_form.yaml + - ../sm-sitemap-plugin/controllers/definitions_test.go + - ../sm-sitemap-plugin/admin.go + - ../sm-sitemap-plugin/extensions_settings.go + - ../sm-sitemap-plugin/listeners.go + - ../sm-sitemap-plugin/jobs.go + - ../sm-sitemap-plugin/schedule.go + - ../sm-sitemap-plugin/console/rebuild.go + - ../sm-sitemap-plugin/console/import_winter.go + - ../sm-sitemap-plugin/console/import.go + - ../sm-sitemap-plugin/console/import_winter_test.go + - ../sm-sitemap-plugin/import_test.go + - ../sm-sitemap-plugin/plugin.go + - ../sm-sitemap-plugin/routes.go + - ../sm-sitemap-plugin/routes_test.go + - ../sm-sitemap-plugin/sitemap_smoke_test.go + - ../sm-sitemap-plugin/sitemap_index_test.go + - ../sm-sitemap-plugin/config/config.yaml + - ../sm-sitemap-plugin/go.mod + - ../sm-sitemap-plugin/go.sum + - ../sm-sitemap-plugin/lang/en/lang.yaml + - ../sm-sitemap-plugin/lang/pl/lang.yaml + - ../sm-sitemap-plugin/README.md +autonomous: false +requirements: [SC-1, D-30, D-32, D-55, D-56, D-57, D-93, D-95, D-24, D-22] + +estimate: + tokens: 160000 + raw_tokens: 160000 + tasks: 4 + confidence: low + +must_haves: + truths: + - "GET /sitemap.xml answers a sitemap index (application/xml) whose entries are per-type files sitemap--.xml with a lastmod each; each file lists at most file_size URLs (setting, default 1000, range 1 to 50 000) with and a real , and no or (D-30)" + - "File numbering is stable: URLs keep their file number across rebuilds, new URLs append to the last file of their group with room, a deleted, unpublished or excluded URL leaves a hole instead of shifting later files, and only a change of file_size renumbers (D-30, RESEARCH Pattern 2, Pitfall 5)" + - "A URL's lastmod moves only when its record is saved with a real change (RecordSaved with a non-empty Changes list outside the ignore list golem15.sitemap.lastmod_ignore); a no-op save, a counter update or a rebuild leaves it unchanged (D-30, RESEARCH Pitfall 4)" + - "Every URL of a record with more than one translated locale carries xhtml:link alternates for each translated locale including itself, plus x-default pointing to the default locale's URL; a locale without a URL gets no alternate and a single-locale record gets none (D-55, D-56, D-57)" + - "The sitemap consumes signpost.Records per content type (D-93) and asks the contract.Filter published by another plugin for each batch; without a Filter everything published is included, and with one, excluded URLs (noindex, canonical, redirect) leave the files and are counted by reason (D-95, D-24)" + - "The settings singleton (one tile, AdminSingleton) has the tabs Files (site_url, file_size, the S1 files partial), Content types (per signpost type include switch, file slug, HTML heading) and Additional addresses (url and lastmod rows); it replaces the Phase 17 definitions screen and the old settings tile (D-32, UI-SPEC C26, Q-4)" + - "The PHP-parity output is gone: priority tiers, the definitions screen, the update manifest, the festival BeforeAddItem/AddItem events, the PHP goldens and the XSL stylesheet (/sitemap.xsl answers 404); this removal is what resolves the todo .planning/todos/completed/sitemap-xslt-browser-deprecation.md (g15office w8kupqi3), which Jakub moved to completed on 2026-10-28 (D-32)" + - "A migration moves every url item of golem15_sitemap_definitions into the Additional addresses rows and frontend_url into site_url; the definitions table stays in the database unread, so a rollback keeps the data (D-32)" + - "The site URL of every is the setting site_url, else the config key golem15.sitemap.site_url, else app.url; the request's Host header is never used" + - "Rebuilds run as a conga job: debounced to one queued rebuild per golem15.sitemap.rebuild_delay_seconds (default 60) after record events, on every settings save, nightly through the scheduled command sitemap:rebuild (pact.HasSchedule, 03:00), and by hand with sitemap:rebuild" + - "The sitemap plugin publishes contract.Listing (Settings with SiteURL, IndexURL, Groups and the HTML-sitemap and IndexNow fields filled by 33-04; Entries paging the included URLs of a group) for other plugins, without depending on any of them (D-95)" + - "The index URL stays /sitemap.xml, so robots.txt Sitemap: lines and Search Console submissions need no change (D-22)" + - "S1 shows files.empty before the first build and the files.building callout while a rebuild job runs (UI-SPEC empty and loading: S1, sitemap rebuild)" + - "The settings form refuses an invalid file size, file slug, duplicate slug or site address with the errors.* copy of UI-SPEC C26 (UI-SPEC error: save refused)" + - {statement: "A file size change renumbers every file in one rebuild and the index lists the new files at once (documented on the field comment)", verification: backstop} + prohibitions: + - requirement_id: "D-95" + category: values + status: resolved + verification: test + statement: "The sitemap never lists a URL that the published Filter excluded, and never imports or names the SEO plugin" + - requirement_id: "D-30" + category: safety + status: resolved + verification: test + statement: "A rebuild never moves an existing URL to another file unless file_size changed" + artifacts: + - path: "../sm-sitemap-plugin/contract/contract.go" + provides: "Filter, Candidate, Decision, Group, Listing, ListingSettings, Entry, EntriesQuery, EntriesPage, reasons" + contains: "type Filter interface" + - path: "../sm-sitemap-plugin/classes/slots.go" + provides: "stable slot assignment" + contains: "func " + - path: "../sm-sitemap-plugin/classes/writer.go" + provides: "index and urlset writers on encoding/xml escaping" + contains: "xml.EscapeText" + - path: "../sm-sitemap-plugin/updates/202610280002_update_golem15_sitemap_settings.go" + provides: "settings columns and the definitions url-item migration" + contains: "extra_urls" + - path: "../sm-sitemap-plugin/sitemap_index_test.go" + provides: "TestSitemapIndex, TestStableNumbering, TestLastmod, TestFilterExcludes, TestNoFilter, TestMigrateDefinitions, TestHreflangAlternates" + contains: "func TestSitemapIndex(" + key_links: + - from: "../sm-sitemap-plugin/classes/rebuild.go" + to: "summercms.go/modules/signpost/content.go" + via: "Records per included content type, paged by Next" + pattern: "Records\\(" + - from: "../sm-sitemap-plugin/classes/rebuild.go" + to: "../sm-sitemap-plugin/contract/contract.go" + via: "app.Lookup[contract.Filter]() decides each batch; absent means include everything" + pattern: "Lookup\\[contract\\.Filter\\]" + - from: "../sm-sitemap-plugin/listeners.go" + to: "../sm-sitemap-plugin/classes/lastmod.go" + via: "*pact.RecordSaved with non-ignored Changes sets lastmod; RecordSaved/RecordDeleted queue a debounced rebuild" + pattern: "\\*pact\\.RecordSaved" +--- + + + + + + + +## Phase Goal + +**As a** site editor and as an SEO specialist auditing a SummerCMS site, **I want to** have the rendered sites emit complete SEO output (sitemaps, structured data, pagination, hreflang, headings, breadcrumbs) from the CMS, and an audit that finds the remaining problems, **so that** sites are indexed correctly and issues can be fixed through MCP. + +This plan rewrites `sm-sitemap-plugin` to the D-30 format (the confirmed breaking change D-32): an index at `/sitemap.xml`, per-type files of a configurable size with stable numbering, real lastmod, hreflang alternates, and the filter contract the SEO plugin implements. + + +Rewrite ../sm-sitemap-plugin (main, not pushed; 33-14 tags the next minor version, breaking): +- a published contract sub-package (`contract`) with the Filter other plugins implement and the Listing other plugins read (D-95); +- a stored slot table and a rebuild job over signpost content types (D-30, D-93); +- event-driven lastmod, hreflang alternates, index and file routes; +- the settings singleton replacing the Phase 17 screens, the migration of definitions, and the removal of the PHP-parity code (D-32). + +Purpose: SC-1 ("every site serves a sitemap index with per-type files of a configurable size, stable numbering and real lastmod, consistent with noindex and robots.txt"). + +Output: commits in ../sm-sitemap-plugin. Image and News sitemaps, IndexNow, the HTML-sitemap API, the management API and MCP tools are plan 33-04. + +**Planner decisions recorded here:** +- **P33-08 (contract package):** `git.golem15.com/golem15/sm-sitemap-plugin/contract` (package `contract`), not the plugin's root package, because importing the root package runs its `init` and registers the plugin. Types: `Candidate{TypeKey, RecordID, Locale, URL string}`; `Decision{Exclude bool; Reason string; Group string; LastMod *time.Time}`; `Group{Key string; Titles map[string]string}`; `Filter interface { Decide(ctx, []Candidate) ([]Decision, error); Groups(ctx) ([]Group, error) }`; reasons `ReasonNoindex = "noindex"`, `ReasonCanonical = "canonical"`, `ReasonRedirect = "redirect"`, `ReasonType = "type"`; `Listing interface { Settings(ctx) (ListingSettings, error); Entries(ctx, EntriesQuery) (EntriesPage, error) }`; `ListingSettings{SiteURL, IndexURL string; DefaultLocale string; Groups []GroupInfo; HTMLEnabled bool; HTMLPaths map[string]string; HTMLPageSize int; KeyFile string}`; `GroupInfo{Key, Slug string; Titles map[string]string; Count int}`; `EntriesQuery{Group, Locale, After string; Limit int}`; `Entry{Group, TypeKey, RecordID, Locale, URL, Title string; LastMod *time.Time}`; `EntriesPage{Entries []Entry; Next string; Total int}`. HTMLEnabled, HTMLPaths, HTMLPageSize and KeyFile stay zero until 33-04 adds their settings. +- **P33-09 (routes):** Go serves `GET /sitemap.xml` (index, literal root route) and `GET /_sitemap/files/{file}` (`Where("file", "sitemap-[a-z0-9-]+-[0-9]+\\.xml")`); the index's `` entries are the public root URLs `/sitemap--.xml`, which the SEO layer (33-07) or an nginx rewrite maps to `/_sitemap/files/`. No root wildcard route is registered (it would capture every one-segment path of a site Go serves). +- **P33-10 (slot table):** `golem15_sitemap_urls` (id, grp, type_key, record_id, locale, url (path), title, file_no, first_seen, lastmod, alternates jsonb, images jsonb, excluded_reason, updated_at; unique (type_key, record_id, locale); index (grp, file_no)). Extra addresses use type_key `extra` and record_id = hex sha256 of the path. Group = per-type slug unless the Filter's Decision.Group overrides it; extra addresses go to `pages`. Default slug: last segment of the type key pluralised (y→ies, else +s unless it ends in s): journal-post → posts, journal-category → categories, journal-tag → tags, journal-author → authors, seo-page → pages. +- **P33-11 (reasons shown):** S1's left-out table shows noindex, canonical, redirect and type; unpublished and deleted rows are holes, not "left out". +- **P33-12 (debounce):** `golem15_sitemap_settings.rebuild_pending_until timestamptz`; an event sets it with a conditional UPDATE and enqueues one rebuild with that delay only when no pending time lies in the future; the job clears it. + + + +Signal: "chosen" (the ROADMAP's "per-type files of a configurable size"). +Primary noun: a sitemap file is (group slug, file number) under a per-site file size; the single PHP file is gone. +Decision: promote. The per-type index becomes the only format (D-32); file size is a real setting (default 1000, 1 to 50 000, renumbering on change), not a constant. +Invariant test: TestStableNumbering (deletes leave holes, new URLs append, only a file-size change renumbers). + + + +@~/.claude/msd-core/workflows/execute-plan.md +@~/.claude/msd-core/templates/summary.md + + + +@.planning/phases/33-seo-output-audit-and-content/33-CONTEXT.md +@.planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md +@.planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md +@.planning/phases/33-seo-output-audit-and-content/33-PATTERNS.md +@.planning/phases/33-seo-output-audit-and-content/33-01-SUMMARY.md +@CLAUDE.md + + +signpost (framework): Registry.ContentTypes() / ContentType(key) / Records(ctx, key, ContentList) / Describe(ctx, key, id, unprefixed); pact.ContentRecord{ID, Published, LastMod, PublishedAt, Parents, Authors, TopHeading, Locales[]{Locale, URL, Title, Description, Image, Words, Images, Videos, Headings}}; pact.RecordSaved{Controller, Model, ID, Locale, Created, Changes []RecordChange{Attribute, Locale, Old, New}}; pact.RecordDeleted. +conga: conga.Job(fn, conga.MaxAttempts(n)); conga.From(app).Dispatch(ctx, db, args, conga.DispatchOpts{Label, Delay}); Enqueue(ctx, db, args, conga.EnqueueOpts{Delay}); pact.HasJobs; pact.HasSchedule → []pact.ScheduledCommand{Command, Args, Cadence: pact.DailyAt(3, 0)}. +Analogs: ../sm-seo-plugin/jobs.go + classes/generate.go (job kind and dispatch after commit with lagoon.AfterCommit), ../sm-seo-plugin/extensions_settings.go (per-type settings tabs as a FieldsYAML document built at boot from signpost ContentTypes on the plugin's own singleton), ../sm-seo-plugin/controllers/settings.go (AdminSingleton linked from SettingsItem.Controller, D-104), ../sm-seo-plugin/classes/settings_write.go (settings singleton + history pattern). +Today's plugin: routes.go (GET /sitemap.xml, /sitemap.xsl, cache), models/settings.go (frontend_url, frontend_default_locale, updates_manifest_path, tiers), models/definition*.go, controllers/definitions.go, classes/{generator,serialize,tiers,rewrite,manifest,urlto,events}.go, console/import_winter.go (PHP rows), config/config.yaml (cache_seconds, priority_tiers), plugin.go (Requires nil, init registers). +XML namespaces: sitemap 0.9 http://www.sitemaps.org/schemas/sitemap/0.9, xhtml http://www.w3.org/1999/xhtml. + + + +## Execution units and resume points + +- Commit groups staged by path in ../sm-sitemap-plugin (`git -c safe.directory='*' -C ../sm-sitemap-plugin add `; deletions with `git rm`). Every commit keeps `go vet ./...` and `go test -short ./...` green there. The go.mod require stays on the existing framework tag with `replace => ../summercms.go` (already present); 33-14 bumps it. +- Resume check: `git -c safe.directory='*' -C ../sm-sitemap-plugin log --format=%s | grep -Fx ''`. + +| Task | Commit groups, in order | Resume point | +|------|-------------------------|--------------| +| 1 | none (checkpoint answer recorded in the SUMMARY) | answer recorded | +| 2 | `feat: sitemap index and per-type files from a slot table (tracer)` | TestSitemapIndex passes | +| 3 | A `feat: sitemap settings singleton with content types and additional addresses`; B `feat!: remove the PHP-parity sitemap, tiers, definitions screen and XSL`; C `feat: migrate definitions into additional addresses` | settings and migration tests pass | +| 4 | A `feat: real lastmod from record events and debounced rebuilds`; B `feat: hreflang alternates and the sitemap filter contract`; C `docs: README for the per-type sitemap format` | all sitemap_index_test.go tests pass; plan complete | + + + + + Task 1: Confirm the concrete public format of the new sitemaps + The file names and the index URL are public: Search Console submissions, robots.txt and crawlers' caches follow them, and grzybyfunkcjonalne.pl moves to them in 33-13. + D-32 (the breaking switch) is decided; this confirms the concrete URLs, the XSL removal and the folding of the Phase 17 screen before code is written. + + Proposed (RESEARCH A4, A5, Pitfall 8; UI-SPEC Q-4): (1) the index stays at /sitemap.xml, so GSC and robots.txt keep their URL; (2) files are /sitemap--.xml at the site root, slug per content type (posts, categories, tags, authors, pages; registry kinds offers, services, landing-pages), plus sitemap-images-.xml and sitemap-news.xml in 33-04; Go serves them under /_sitemap/files/ and the SEO layer or nginx maps the root; (3) the XSL view is removed (Chrome 158 stops running XSLT on 2026-11-17; the HTML sitemap is the human view), which resolves the todo sitemap-xslt-browser-deprecation.md (g15office w8kupqi3; already moved to .planning/todos/completed on 2026-10-28); (4) the Phase 17 "Sitemap" definitions screen and the old settings tile become one settings singleton; url items migrate to Additional addresses; priority, change frequency, tiers and the update manifest are removed; the definitions table stays unread in the database. + + + + + + + Reply "as-proposed", "keep-xsl" or "other-names: " + + + + Task 2: End-to-end "a content type's published records appear in an index at /sitemap.xml that links per-type files of N URLs" — one path only + ../sm-sitemap-plugin/contract/contract.go, ../sm-sitemap-plugin/models/url.go, ../sm-sitemap-plugin/models/registry.go, ../sm-sitemap-plugin/updates/202610280001_create_golem15_sitemap_urls.go, ../sm-sitemap-plugin/updates/202610280002_update_golem15_sitemap_settings.go, ../sm-sitemap-plugin/updates/registry.go, ../sm-sitemap-plugin/models/settings.go, ../sm-sitemap-plugin/classes/slots.go, ../sm-sitemap-plugin/classes/rebuild.go, ../sm-sitemap-plugin/classes/writer.go, ../sm-sitemap-plugin/jobs.go, ../sm-sitemap-plugin/routes.go, ../sm-sitemap-plugin/plugin.go, ../sm-sitemap-plugin/sitemap_index_test.go + + - ../sm-sitemap-plugin/plugin.go, routes.go, models/settings.go, updates/202610110002_create_golem15_sitemap_settings.go, testmain_test.go, sitemap_smoke_test.go + - ../sm-seo-plugin/jobs.go and classes/generate.go:14-28,168-177 (job kind, dispatch) + - ../sm-seo-plugin/fixture_test.go (a fake content type plugin in tests) + - summercms.go/modules/signpost/content.go (Records paging) + - .planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md (Pattern 2, Code Examples: index writer) + + + Per D-30, D-93 and P33-08 to P33-10: create the contract package with every type of P33-08. Migration 202610280001 creates golem15_sitemap_urls (P33-10); migration 202610280002 adds to golem15_sitemap_settings: site_url text, file_size integer NOT NULL DEFAULT 1000, default_locale text, extra_urls jsonb NOT NULL DEFAULT '[]', rebuild_pending_until timestamptz (the url-item copy is Task 3's). models/url.go and the new Settings fields. `classes/slots.go`: assign file numbers per group (keep existing numbers, append to the last file with room by first_seen then id, holes stay). `classes/rebuild.go`: `Rebuild(ctx, app) (RebuildResult, error)` walks signpost.ContentTypes, pages Records (Limit 500), keeps Published records with a URL per locale, upserts rows (title from ContentLocale.Title, lastmod initialised from the record's LastMod), marks vanished rows excluded "deleted", assigns slots in one transaction, and returns added/changed/removed URL lists (IndexNow in 33-04 uses them). `classes/writer.go`: index and urlset writers with bufio and `xml.EscapeText`, RFC 3339 UTC lastmod, no priority and no change frequency. jobs.go: `RebuildArgs` (kind `golem15.sitemap.rebuild`) with conga.MaxAttempts(3); plugin.go asserts pact.HasJobs. routes.go: replace the old handlers with the index (`GET /sitemap.xml`, absolute on the site URL per the precedence in must_haves) and `GET /_sitemap/files/{file}` (P33-09), both reading the table, content type application/xml, X-Robots-Tag noindex on files, 404 for an unknown or empty file; keep the throttle bucket. `TestSitemapIndex` in sitemap_index_test.go (Docker): a fake content type with 5 published records and 1 draft, file_size 2 → after Rebuild the index lists sitemap-pages-1.xml to -3.xml with lastmod; file 1 lists 2 URLs with absolute and ; the draft is absent; no priority or change frequency element anywhere; an unknown file answers 404. Commit as listed. + + + go -C ../sm-sitemap-plugin test ./... -run 'TestSitemapIndex' -count=1 -v 2>&1 | tail -20 + non-zero exit, "FAIL" or "no tests to run" in the output, or no "--- PASS: TestSitemapIndex" line + go -C ../sm-sitemap-plugin vet ./... && go -C ../sm-sitemap-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "type Filter interface\|type Listing interface" ../sm-sitemap-plugin/contract/contract.go` prints two lines + - `grep -n "xml.EscapeText" ../sm-sitemap-plugin/classes/writer.go` prints at least one line + - `grep -n "/_sitemap/files/{file}" ../sm-sitemap-plugin/routes.go` prints one line + - `grep -rn "party.Register\|func init" ../sm-sitemap-plugin/contract/` prints nothing + + A real index and per-type files are served from the slot table after a rebuild job. + + + + Task 3: Settings singleton (Files, Content types, Additional addresses), removal of the PHP-parity code, and the definitions migration + ../sm-sitemap-plugin/models/settings.go, ../sm-sitemap-plugin/models/settings/fields.yaml, ../sm-sitemap-plugin/models/settings_test.go, ../sm-sitemap-plugin/models/type_setting.go, ../sm-sitemap-plugin/models/registry.go, ../sm-sitemap-plugin/updates/202610280002_update_golem15_sitemap_settings.go, ../sm-sitemap-plugin/updates/202610280003_create_golem15_sitemap_types.go, ../sm-sitemap-plugin/updates/registry.go, ../sm-sitemap-plugin/classes/types.go, ../sm-sitemap-plugin/controllers/admin_registry.go, ../sm-sitemap-plugin/controllers/settings.go, ../sm-sitemap-plugin/controllers/settings/config_form.yaml, ../sm-sitemap-plugin/controllers/settings/_sitemap_files.htm, ../sm-sitemap-plugin/admin.go, ../sm-sitemap-plugin/extensions_settings.go, ../sm-sitemap-plugin/console/import_winter.go, ../sm-sitemap-plugin/console/import.go, ../sm-sitemap-plugin/console/import_winter_test.go, ../sm-sitemap-plugin/import_test.go, ../sm-sitemap-plugin/routes_test.go, ../sm-sitemap-plugin/sitemap_smoke_test.go, ../sm-sitemap-plugin/config/config.yaml, ../sm-sitemap-plugin/plugin.go, ../sm-sitemap-plugin/lang/en/lang.yaml, ../sm-sitemap-plugin/lang/pl/lang.yaml, ../sm-sitemap-plugin/sitemap_index_test.go, ../sm-sitemap-plugin/classes/generator.go, ../sm-sitemap-plugin/classes/generator_test.go, ../sm-sitemap-plugin/classes/serialize.go, ../sm-sitemap-plugin/classes/serialize_test.go, ../sm-sitemap-plugin/classes/golden_test.go, ../sm-sitemap-plugin/classes/testdata/full.golden, ../sm-sitemap-plugin/classes/testdata/full.json, ../sm-sitemap-plugin/classes/testdata/tracer.golden, ../sm-sitemap-plugin/classes/testdata/tracer.json, ../sm-sitemap-plugin/classes/testdata/urlto.golden, ../sm-sitemap-plugin/classes/testdata/urlto.json, ../sm-sitemap-plugin/classes/tiers.go, ../sm-sitemap-plugin/classes/tiers_test.go, ../sm-sitemap-plugin/classes/rewrite.go, ../sm-sitemap-plugin/classes/rewrite_test.go, ../sm-sitemap-plugin/classes/manifest.go, ../sm-sitemap-plugin/classes/manifest_test.go, ../sm-sitemap-plugin/classes/urlto.go, ../sm-sitemap-plugin/classes/urlto_test.go, ../sm-sitemap-plugin/classes/events.go, ../sm-sitemap-plugin/assets/sitemap.xsl, ../sm-sitemap-plugin/tests/php/gen-goldens.php, ../sm-sitemap-plugin/models/definition.go, ../sm-sitemap-plugin/models/definition_item.go, ../sm-sitemap-plugin/models/definition/fields.yaml, ../sm-sitemap-plugin/models/definition_test.go, ../sm-sitemap-plugin/controllers/definitions.go, ../sm-sitemap-plugin/controllers/definitions/config_form.yaml, ../sm-sitemap-plugin/controllers/definitions_test.go + + - ../sm-sitemap-plugin/admin.go, controllers/definitions.go, models/definition*.go, console/import_winter.go and import_test.go + - ../sm-seo-plugin/controllers/settings.go, controllers/settings/config_form.yaml, extensions_settings.go (singleton + per-type tabs pattern) + - ../sm-seo-plugin/controllers/rules/_seo_status.htm (partial view-model pattern with summer-callout and summer-table classes) + - .planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md (C26, S1, the sitemap copy table, Destructive actions: file size) + + + - The settings tile (icon network, existing category, label sitemap:settings.label) opens the singleton; the old Phase 17 tile and the definitions screen are gone + - Content types tab lists one block per registered signpost type in registration order: type__include (switch default on), type__slug (default per P33-10), type__title (mltext, default the type label); a duplicate slug refuses with errors.slug_taken, a slug outside [a-z0-9-] with errors.slug + - file_size outside 1..50000 refuses with errors.file_size; site_url refuses with the existing errors.frontend_url phrase when not an absolute http(s) URL + - Saving dispatches a rebuild; a type switched off moves its rows to excluded reason type + - Additional addresses rows (url path, optional lastmod) appear in the pages group with their lastmod + - Migration: definitions url items become extra_urls rows (url as given, no lastmod), frontend_url becomes site_url, frontend_default_locale becomes default_locale; reference items are dropped (content types cover them); the definitions table is untouched; Rollback clears the copied values only + - sitemap:import-winter writes PHP url items into extra_urls (the companion import path of the apps keeps working) + - GET /sitemap.xsl answers 404; the old cache config key is gone + + + Per D-32, UI-SPEC C26 and Task 1's answer: build the singleton controller `golem15.sitemap.settings` (AdminSingleton, settings tile via SettingsItem.Controller) with fields.yaml tabs Files (site_url text left, file_size number right, files partial S1 full), Additional addresses (extra_urls repeater, titleFrom url, rows url text left and lastmod datepicker right); the Content types tab is a FieldsYAML document built at boot from signpost ContentTypes by a self-extension on the singleton (the SEO plugin's extensions_settings.go pattern), stored in golem15_sitemap_types (type_key unique, include bool, slug, title via the translation writer) created by migration 202610280003. S1 partial `_sitemap_files.htm`: summer-table of files (index first; File in DM Mono linked to the public URL, Addresses, Last modified), then the Left out table by reason (P33-11), then the files.index_note callout; files.empty and files.building per C26. Remove the PHP-parity code and its tests (git rm the files listed in files_modified), the definitions controller, model and form, and the `cache_seconds` and `priority_tiers` config keys; config/config.yaml gets commented `site_url`, `lastmod_ignore` (default [updated_at, created_at, views, sort_order]) and `rebuild_delay_seconds` (60). Extend migration 202610280002 with the url-item copy (write the TestMigrateDefinitions case first). Retarget console/import_winter.go to extra_urls. Every copy key of the C26 sitemap table in both lang files. Commit groups A, B, C. + + + go -C ../sm-sitemap-plugin test ./... -run 'TestSettingsSingleton|TestMigrateDefinitions|TestImportWinter|TestSitemapIndex' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -20 + a "FAIL" line, "no tests to run", or TestSettingsSingleton or TestMigrateDefinitions missing from the "--- PASS" lines + go -C ../sm-sitemap-plugin vet ./... && go -C ../sm-sitemap-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `git -C ../sm-sitemap-plugin ls-files assets classes/testdata tests/php` prints nothing + - `grep -rn "priority_tiers\|cache_seconds" ../sm-sitemap-plugin --include=*.go --include=*.yaml` prints nothing + - `grep -n "golem15.sitemap.settings" ../sm-sitemap-plugin/controllers/settings.go` prints at least one line + - TestMigrateDefinitions asserts the copied extra_urls, the site_url and that golem15_sitemap_definitions still holds its rows + + One settings screen manages the new format, the PHP-parity code is gone and existing sites' data migrates. + + + + Task 4: Real lastmod from record events, debounced and nightly rebuilds, hreflang alternates, and the Filter and Listing contract + ../sm-sitemap-plugin/classes/lastmod.go, ../sm-sitemap-plugin/classes/rebuild.go, ../sm-sitemap-plugin/classes/writer.go, ../sm-sitemap-plugin/classes/listing.go, ../sm-sitemap-plugin/listeners.go, ../sm-sitemap-plugin/schedule.go, ../sm-sitemap-plugin/console/rebuild.go, ../sm-sitemap-plugin/plugin.go, ../sm-sitemap-plugin/README.md, ../sm-sitemap-plugin/sitemap_index_test.go, ../sm-sitemap-plugin/commands_documented_test.go + + - summercms.go/modules/pact/record_events.go (RecordSaved, RecordChange, no-op saves fire with empty Changes) + - ../sm-seo-plugin/listeners.go (subscribing to *pact.RecordSaved / *pact.RecordDeleted on app.Events) + - ../sm-seo-plugin/classes/settings_write.go:301-313 (lagoon.AfterCommit enqueue) + - summercms.go/modules/pact/capabilities.go (HasSchedule, ScheduledCommand, DailyAt) and docs/plugins/scheduling.md + - ../sm-journal-plugin/classes/menu_item_types.go:53-55 (the single x-default alternate when translate has no locale) + - .planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md (Pitfalls 3, 4, 5; Pattern 3) + + + - TestStableNumbering: 5 URLs at size 2 → files 1,1,2,2,3; delete the second → file 1 keeps one URL (hole), files 2 and 3 unchanged; add two → they go to file 3 then a new file 4; change size to 3 → renumbered 1,1,1,2,2,2 + - TestLastmod: a save whose Changes touch title moves lastmod of that record's rows (only the changed locale's row for a translated change); a save with empty Changes or only updated_at/views does not; a rebuild never moves lastmod + - TestHreflangAlternates: a record with pl and en URLs → each url element carries xhtml:link for pl, en and x-default (default locale's URL); a record with only pl → none; a locale with an empty URL → absent + - TestFilterExcludes: with a published Filter returning Exclude noindex for one candidate and Group offers for another, the first leaves the files (counted under noindex), the second lands in sitemap-offers-1.xml; Filter.Groups titles reach Listing.Settings Groups + - TestNoFilter: without a Filter, everything published is listed + - TestDebouncedRebuild: three RecordSaved events within the delay queue exactly one rebuild job + - Listing.Entries pages a group's included URLs by after/limit with Total; Listing.Settings answers SiteURL, IndexURL and Groups with counts + + + Per D-30, D-55 to D-57, D-95 and P33-12: listeners.go subscribes `*pact.RecordSaved` and `*pact.RecordDeleted` (type looked up with signpost ContentTypeForModel); `classes/lastmod.go` sets lastmod = now on the record's rows when a change outside `golem15.sitemap.lastmod_ignore` exists (Locale "" → every locale row, else that locale's row), then requests a debounced rebuild (P33-12) after commit; RecordDeleted marks rows excluded deleted and requests a rebuild. rebuild.go: batch candidates per page through `app.Lookup[contract.Filter]()` when present (Decide; Exclude → excluded_reason; Group → grp; LastMod → initial lastmod for new rows only), excluded types (include off) → reason type; alternates per record = translated locales with a URL (≥2) plus x-default on default_locale (settings, else the record's first locale). writer.go emits xhtml:link alternates (namespace declared on the urlset only when used). `classes/listing.go` implements contract.Listing over the table and settings and plugin.go publishes it in Register with `app.Publish[contract.Listing]`. schedule.go: pact.HasSchedule with `{Command: "sitemap:rebuild", Cadence: pact.DailyAt(3, 0)}`; console/rebuild.go runs a rebuild synchronously (`--queue` dispatches the job instead), documented for commands_documented_test.go. README rewritten for the new format: routes, settings, contract package (with a Filter example), lastmod rules, config keys, the breaking change and the migration. Tests first per behavior list. Commit groups A, B, C. + + + go -C ../sm-sitemap-plugin test ./... -run 'TestSitemapIndex|TestStableNumbering|TestLastmod|TestHreflangAlternates|TestFilterExcludes|TestNoFilter|TestDebouncedRebuild|TestMigrateDefinitions' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -30 + a "FAIL" line, "no tests to run", or any of the eight names missing from the "--- PASS" lines + go -C ../sm-sitemap-plugin vet ./... && go -C ../sm-sitemap-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "Lookup\[contract.Filter\]" ../sm-sitemap-plugin/classes/rebuild.go` prints one line + - `grep -n "Publish\[contract.Listing\]\|Publish(" ../sm-sitemap-plugin/plugin.go` prints at least one line + - `grep -n "sitemap:rebuild" ../sm-sitemap-plugin/schedule.go ../sm-sitemap-plugin/console/rebuild.go` prints a line in each + - `grep -n "x-default" ../sm-sitemap-plugin/classes/rebuild.go ../sm-sitemap-plugin/classes/writer.go` prints at least one line + + The sitemap is event-driven, multilingual, filterable by another plugin and readable through the published Listing. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| crawlers → /sitemap.xml, /_sitemap/files | anonymous, throttled reads | +| content data → XML output | titles and URLs written into XML | +| other plugins → contract.Filter | in-process decisions | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-33-09 | Tampering | XML writer | high | mitigate | every value through xml.EscapeText; only http(s) absolute locs built from the site URL setting/config, never from Host; TestSitemapIndex includes a title and path with &, < and quotes | +| T-33-10 | Denial of service | file and index routes | medium | mitigate | served from the table (no per-request generation), existing per-IP bucket kept, files route constrained by Where | +| T-33-11 | Information disclosure | unpublished records | high | mitigate | only Published records with a URL are stored as included; drafts tested absent | +| T-33-12 | Tampering | settings site_url | medium | mitigate | absolute http(s) validation, settings guarded by the existing sitemap permission | +| T-33-SC | Tampering | npm/pip/cargo installs | high | accept | no new package | + + + +- `go -C ../sm-sitemap-plugin vet ./... && go -C ../sm-sitemap-plugin test -short ./...` after every commit. +- Docker suite: the eight sitemap_index_test.go tests plus TestSettingsSingleton and TestImportWinter. + + + +- /sitemap.xml is an index of per-type files with stable numbering, real lastmod and hreflang. +- The sitemap works with and without a Filter; the PHP-parity mode is gone and old data migrates. + + +## Artifacts this phase produces + +- Package `git.golem15.com/golem15/sm-sitemap-plugin/contract`: `Filter`, `Candidate`, `Decision`, `Group`, `Listing`, `ListingSettings`, `GroupInfo`, `EntriesQuery`, `Entry`, `EntriesPage`, `ReasonNoindex`, `ReasonCanonical`, `ReasonRedirect`, `ReasonType`. +- Go symbols: `classes.Rebuild`, `classes.RebuildResult`, `RebuildArgs` (kind `golem15.sitemap.rebuild`), the Listing implementation. +- Tables/columns: `golem15_sitemap_urls`; `golem15_sitemap_types`; settings columns `site_url`, `file_size`, `default_locale`, `extra_urls`, `rebuild_pending_until`; migrations 202610280001 to 202610280003. +- Routes: `GET /sitemap.xml` (index), `GET /_sitemap/files/{file}`; removed `GET /sitemap.xsl`. +- Admin: controller `golem15.sitemap.settings` (singleton), partial `sitemap_files` (S1). +- CLI: `sitemap:rebuild [--queue]`; schedule entry `sitemap:rebuild` daily 03:00. +- Config keys: `golem15.sitemap.site_url`, `golem15.sitemap.lastmod_ignore`, `golem15.sitemap.rebuild_delay_seconds`; removed `golem15.sitemap.cache_seconds`, `golem15.sitemap.priority_tiers`. +- Tests: `TestSitemapIndex`, `TestStableNumbering`, `TestLastmod`, `TestHreflangAlternates`, `TestFilterExcludes`, `TestNoFilter`, `TestDebouncedRebuild`, `TestMigrateDefinitions`, `TestSettingsSingleton`. + + +Create `.planning/phases/33-seo-output-audit-and-content/33-03-SUMMARY.md` when done; note the Task 1 answer and, if g15office task w8kupqi3 is still open, that it can be closed as resolved by removal. + diff --git a/.planning/phases/33-seo-output-audit-and-content/33-04-PLAN.md b/.planning/phases/33-seo-output-audit-and-content/33-04-PLAN.md new file mode 100644 index 00000000..40226fa5 --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-04-PLAN.md @@ -0,0 +1,289 @@ +--- +phase: 33-seo-output-audit-and-content +plan: 04 +type: execute +wave: 3 +depends_on: ["33-03"] +files_modified: + - ../sm-sitemap-plugin/classes/images.go + - ../sm-sitemap-plugin/classes/news.go + - ../sm-sitemap-plugin/classes/indexnow.go + - ../sm-sitemap-plugin/classes/htmlgroups.go + - ../sm-sitemap-plugin/classes/rebuild.go + - ../sm-sitemap-plugin/classes/slots.go + - ../sm-sitemap-plugin/classes/writer.go + - ../sm-sitemap-plugin/classes/listing.go + - ../sm-sitemap-plugin/models/settings.go + - ../sm-sitemap-plugin/models/settings/fields.yaml + - ../sm-sitemap-plugin/models/url.go + - ../sm-sitemap-plugin/updates/202610280004_add_golem15_sitemap_media_news_indexnow.go + - ../sm-sitemap-plugin/updates/registry.go + - ../sm-sitemap-plugin/controllers/settings.go + - ../sm-sitemap-plugin/controllers/settings/config_form.yaml + - ../sm-sitemap-plugin/controllers/settings/_indexnow_status.htm + - ../sm-sitemap-plugin/controllers/api/auth.go + - ../sm-sitemap-plugin/controllers/api/html.go + - ../sm-sitemap-plugin/controllers/api/settings.go + - ../sm-sitemap-plugin/controllers/api/indexnow.go + - ../sm-sitemap-plugin/jobs.go + - ../sm-sitemap-plugin/mcp.go + - ../sm-sitemap-plugin/routes.go + - ../sm-sitemap-plugin/plugin.go + - ../sm-sitemap-plugin/config/config.yaml + - ../sm-sitemap-plugin/lang/en/lang.yaml + - ../sm-sitemap-plugin/lang/pl/lang.yaml + - ../sm-sitemap-plugin/README.md + - ../sm-sitemap-plugin/sitemap_extras_test.go + - .planning/phases/33-seo-output-audit-and-content/COVERAGE.md +autonomous: true +requirements: [SC-1, D-31, D-34, D-24, D-10, D-30] + +estimate: + tokens: 130000 + raw_tokens: 130000 + tasks: 3 + confidence: low + +must_haves: + truths: + - "With images_enabled (default on) the index lists sitemap-images-.xml files whose entries carry the page and one per featured or body image of that page, at most 1000 images per page, numbered stably like the per-type files (D-31)" + - "With news_enabled (default off) the index lists sitemap-news.xml with at most 1000 articles (content Kind article) first published in the last two days, filtered by news_sources (article types and their categories; empty = every article type), each with publication name (news_publication, else app.name), language, publication_date from PublishedAt and title (D-31, RESEARCH Pitfall 9)" + - "With indexnow_enabled (default off) the plugin generates a 32-character hex key with crypto/rand on save, serves it at GET /_sitemap/indexnow/.txt (the SEO layer maps the root /.txt), and after each rebuild posts the added, changed and removed URLs (at most 10 000 per request) to https://api.indexnow.org/indexnow through a fetchguard client limited to that host; 403, 422 and 429 answers are recorded and shown in S2, 429 retries through the job's attempts (D-31)" + - "Generating a new key (indexnow_new_key on save) makes the old key file answer 404 at once (UI-SPEC Destructive actions)" + - "No IndexNow request is ever sent outside the production environment" + - "GET /_sitemap/api/v1/html?locale= answers the HTML-sitemap overview (groups in the order pages, offers, services, categories as a tree, posts newest first, tags, authors, other types; each group with its title in the locale, total, the first html_page_size entries and more=true when larger) and GET /_sitemap/api/v1/html/{group}?locale=&page= one page of a group; both list exactly the indexable set of the XML sitemaps (no excluded URL), answer 404 when the HTML sitemap is off, for an unknown group or a page past the last, and carry X-Robots-Tag noindex (D-34, D-24)" + - "The settings singleton gains the tabs Images and News, IndexNow (with the S2 indexnow_status partial) and HTML sitemap (html_enabled default on, html_path per locale default /mapa-strony pl and /sitemap en, html_page_size 50 to 500 default 150), with the C26 copy and errors.html_path / errors.page_size refusals (UI-SPEC C26)" + - "contract.Listing.Settings answers HTMLEnabled, HTMLPaths, HTMLPageSize and KeyFile, so the SEO plugin and the layer read them without importing sitemap internals (D-95)" + - "Every sitemap admin operation is an MCP tool: sitemap_get_settings (GET /_sitemap/api/v1/settings), sitemap_update_settings (PUT) and sitemap_rebuild (POST /_sitemap/api/v1/rebuild), listed only with the sitemap permission (D-10, UI-SPEC MCP parity)" + - "S2 shows indexnow.no_key before a key exists and indexnow.never before the first notification, then the key, its URL and the last notification (UI-SPEC empty: S2)" + - "S2 shows the indexnow.err_key, err_host or err_rate callout matching the last 403, 422 or 429 answer (UI-SPEC error: IndexNow)" + - "The external API this plan integrates (IndexNow) has its capability matrix in COVERAGE.md, every opt-out with a reason" + - {statement: "A HTML-sitemap group larger than html_page_size shows its first page plus the sitemap.all link and its group page is paginated (UI-SPEC zero-one-many: large groups; layer rendering in 33-08)", verification: backstop} + prohibitions: + - requirement_id: "D-31" + category: safety + status: resolved + verification: test + statement: "IndexNow never submits a URL whose host differs from the site URL's host, and never sends a request outside production" + artifacts: + - path: "../sm-sitemap-plugin/classes/indexnow.go" + provides: "key generation, ping job body, answer recording" + contains: "api.indexnow.org" + - path: "../sm-sitemap-plugin/classes/htmlgroups.go" + provides: "HTML sitemap groups, order, tree and paging" + contains: "func " + - path: "../sm-sitemap-plugin/mcp.go" + provides: "sitemap MCP tools" + contains: "pact.HasMCP" + - path: "../sm-sitemap-plugin/sitemap_extras_test.go" + provides: "TestImageSitemap, TestNewsSitemap, TestIndexNow, TestHTMLGroups, TestSitemapTools" + contains: "func TestIndexNow(" + - path: ".planning/phases/33-seo-output-audit-and-content/COVERAGE.md" + provides: "IndexNow capability matrix" + contains: "INTEGRATE" + key_links: + - from: "../sm-sitemap-plugin/classes/rebuild.go" + to: "../sm-sitemap-plugin/classes/indexnow.go" + via: "a rebuild's added/changed/removed URLs dispatch the IndexNow job when enabled in production" + pattern: "IndexNowArgs" + - from: "../sm-sitemap-plugin/classes/indexnow.go" + to: "summercms.go/modules/fetchguard" + via: "fetchguard.NewClient(Policy{Mode: AllowHostsMode, AllowHosts: [api.indexnow.org]}).PostJSON" + pattern: "fetchguard\\.NewClient" +--- + + + +## Phase Goal + +**As a** site editor and as an SEO specialist auditing a SummerCMS site, **I want to** have the rendered sites emit complete SEO output (sitemaps, structured data, pagination, hreflang, headings, breadcrumbs) from the CMS, and an audit that finds the remaining problems, **so that** sites are indexed correctly and issues can be fixed through MCP. + +This plan adds the image and News sitemaps, IndexNow pings, the data API behind the HTML sitemap page, and MCP tools for the sitemap settings. + + +Extend ../sm-sitemap-plugin (after 33-03; main, not pushed): +- image sitemap files and the News sitemap from the 33-01 descriptor facts (D-31); +- IndexNow: key, key file, ping job from rebuild diffs, S2 status (D-31); +- the HTML-sitemap groups API that the layer's page renders (D-34, layer page in 33-08); +- the remaining settings tabs, the management API and the MCP tools (D-10); +- keeping `COVERAGE.md` (the IndexNow capability matrix written at planning time) in sync with what is built. + +Purpose: SC-1 (image and News sitemaps, IndexNow) and the data side of the HTML sitemap. + +Output: commits in ../sm-sitemap-plugin; a planning-docs commit here only if COVERAGE.md has to change. + +**Planner decisions recorded here:** +- **P33-13 (image slots):** `golem15_sitemap_urls.image_file_no` (stable numbering of rows that have images, same rules as file_no) plus `published_at`, `kind` and `parent_refs jsonb` columns (for News and the category tree), filled by the rebuild; migration 202610280004 adds them with the media, News, IndexNow and HTML settings columns: images_enabled bool default true, news_enabled bool default false, news_publication text, news_sources jsonb default '[]', indexnow_enabled bool default false, indexnow_key text, indexnow_last_at timestamptz, indexnow_last_count int, indexnow_last_status int, html_enabled bool default true, html_path (translatable through the translation writer, default value pl /mapa-strony and en /sitemap), html_page_size int default 150. +- **P33-14 (IndexNow request):** body `{host, key, urlList}`; no keyLocation (the key file is served at the root, so it covers every URL of the host); endpoint `https://api.indexnow.org/indexnow`; 200 and 202 are success; 400, 403, 422 recorded without retry; 429 makes the job return an error so conga retries (MaxAttempts 5). +- **P33-15 (news sources values):** taglist values `type:` and `category::`; a News candidate matches when its type or one of its Parents is listed. + + + +@~/.claude/msd-core/workflows/execute-plan.md +@~/.claude/msd-core/templates/summary.md + + + +@.planning/phases/33-seo-output-audit-and-content/33-CONTEXT.md +@.planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md +@.planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md +@.planning/phases/33-seo-output-audit-and-content/33-03-SUMMARY.md +@CLAUDE.md + + +From 33-03: contract.Listing / ListingSettings{SiteURL, IndexURL, DefaultLocale, Groups, HTMLEnabled, HTMLPaths, HTMLPageSize, KeyFile}; classes.Rebuild → RebuildResult{Added, Changed, Removed []string}; golem15_sitemap_urls (grp, type_key, record_id, locale, url, title, file_no, first_seen, lastmod, alternates, images, excluded_reason); routes GET /sitemap.xml, GET /_sitemap/files/{file}; settings singleton golem15.sitemap.settings with Files, Content types, Additional addresses tabs. +fetchguard: NewClient(Policy{Mode: AllowHostsMode, AllowHosts, Timeout}, cfg) → Client.PostJSON(ctx, url, header, payload) (*Result, error); WithTransport(ctx, rt) routes a test's requests to an httptest handler. +Management API analog: ../sm-seo-plugin/controllers/api/auth.go `manage(...)` (cabana.ResolveBackend + cabana.Allows) and mcp.go (pact.MCP tools bound to routes, pier prefixes the plugin id). +Specs (cited in 33-RESEARCH.md): image sitemap only image:image + image:loc, ≤1000 per url; News: articles from the last two days, ≤1000 news:news per sitemap; IndexNow key 8–128 [A-Za-z0-9-], ≤10 000 URLs per POST. +Copy: UI-SPEC C26 tables (tabs, images, News, IndexNow, S2, HTML sitemap, errors). + + + +## Execution units and resume points + +- Commit groups staged by path in ../sm-sitemap-plugin; COVERAGE.md committed here as planning docs. Every commit keeps vet and `test -short` green. +- Resume check: `git -c safe.directory='*' -C ../sm-sitemap-plugin log --format=%s | grep -Fx ''`; here `git log --format=%s | grep -Fx 'docs(33-04): update the IndexNow coverage matrix'` (only when group C was needed). + +| Task | Commit groups, in order | Resume point | +|------|-------------------------|--------------| +| 1 | `feat: image sitemap files (tracer)` | TestImageSitemap passes | +| 2 | A `feat: Google News sitemap`; B `feat: IndexNow key, key file and pings`; C (here, only if the matrix changed) `docs(33-04): update the IndexNow coverage matrix` | News and IndexNow tests pass | +| 3 | A `feat: HTML sitemap groups API`; B `feat: sitemap management API and MCP tools`; C `docs: images, News, IndexNow, HTML sitemap and MCP in README` | plan complete | + + + + + Task 1: End-to-end "a post's featured and body images appear in sitemap-images-1.xml linked from the index" — one path only + ../sm-sitemap-plugin/updates/202610280004_add_golem15_sitemap_media_news_indexnow.go, ../sm-sitemap-plugin/updates/registry.go, ../sm-sitemap-plugin/models/url.go, ../sm-sitemap-plugin/models/settings.go, ../sm-sitemap-plugin/classes/images.go, ../sm-sitemap-plugin/classes/rebuild.go, ../sm-sitemap-plugin/classes/slots.go, ../sm-sitemap-plugin/classes/writer.go, ../sm-sitemap-plugin/routes.go, ../sm-sitemap-plugin/sitemap_extras_test.go + + - ../sm-sitemap-plugin/classes/{rebuild,slots,writer}.go and sitemap_index_test.go (33-03) + - summercms.go/modules/pact/content.go (ContentLocale.Images, Image) + - .planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md (Code Examples: urlset with images; State of the Art: image tags) + + + Per D-31 and P33-13: migration 202610280004 adds the P33-13 columns; the rebuild stores each locale row's images (featured Image first, then Images, absolute on the site URL when relative, deduplicated, at most 1000) and kind, published_at, parent_refs; slots.go assigns image_file_no to rows with images (stable, holes kept). writer.go emits the image urlset (namespace http://www.google.com/schemas/sitemap-image/1.1, only image:image and image:loc). The index adds sitemap-images-.xml when images_enabled and the file has rows; the files route serves them. `TestImageSitemap`: a post with a featured image and two body images → sitemap-images-1.xml lists its page with three image:loc; images_enabled false removes the files from the index; a page with 1200 images lists 1000. Commit as listed. + + + go -C ../sm-sitemap-plugin test ./... -run 'TestImageSitemap|TestSitemapIndex' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -10 + a "FAIL" line, "no tests to run", or no "--- PASS: TestImageSitemap" line + go -C ../sm-sitemap-plugin vet ./... && go -C ../sm-sitemap-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "sitemap-image/1.1" ../sm-sitemap-plugin/classes/writer.go` prints one line + - `grep -n "image_file_no" ../sm-sitemap-plugin/updates/202610280004_add_golem15_sitemap_media_news_indexnow.go` prints one line + + Images of every listed page reach Google through the image sitemap files. + + + + Task 2: Google News sitemap and IndexNow (key, key file, ping job, S2) + ../sm-sitemap-plugin/classes/news.go, ../sm-sitemap-plugin/classes/indexnow.go, ../sm-sitemap-plugin/classes/rebuild.go, ../sm-sitemap-plugin/classes/writer.go, ../sm-sitemap-plugin/controllers/api/indexnow.go, ../sm-sitemap-plugin/controllers/settings.go, ../sm-sitemap-plugin/controllers/settings/config_form.yaml, ../sm-sitemap-plugin/controllers/settings/_indexnow_status.htm, ../sm-sitemap-plugin/models/settings/fields.yaml, ../sm-sitemap-plugin/jobs.go, ../sm-sitemap-plugin/routes.go, ../sm-sitemap-plugin/config/config.yaml, ../sm-sitemap-plugin/lang/en/lang.yaml, ../sm-sitemap-plugin/lang/pl/lang.yaml, ../sm-sitemap-plugin/sitemap_extras_test.go, .planning/phases/33-seo-output-audit-and-content/COVERAGE.md + + - summercms.go/modules/fetchguard/README.md (Client, PostJSON, WithTransport, AllowHostsMode) + - ../sm-seo-plugin/classes/robots.go (how Staging()/non-production is detected) and classes/service.go Staging() + - ../sm-sitemap-plugin/controllers/settings/_sitemap_files.htm (partial style from 33-03) + - .planning/phases/33-seo-output-audit-and-content/COVERAGE.md (the matrix this task implements) + - .planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md (C26 Images and News, IndexNow tabs, S2 copy) + + + - TestNewsSitemap: articles published 1 hour and 47 hours ago are listed, one published 49 hours ago is not; a non-article type is never listed; news_sources [category:journal-category:3] keeps only articles with that parent; more than 1000 candidates list the newest 1000; news_enabled false: no sitemap-news.xml in the index and the file answers 404 + - TestIndexNow (fake transport): enabling and saving generates a 32-hex key; GET /_sitemap/indexnow/.txt answers the key as text/plain; another name answers 404; after a rebuild with one new and one removed URL the job posts {host, key, urlList[2]} to api.indexnow.org; 202 recorded as last status; 403 and 422 recorded without retry; 429 makes the job fail so it retries; indexnow_new_key replaces the key and the old file answers 404; with the app environment not production no request is sent; a URL of another host is never in urlList + + + Per D-31, P33-14 and P33-15: `classes/news.go` builds the News urlset (namespace http://www.google.com/schemas/sitemap-news/0.9) from rows with kind article and published_at in the last 48 hours, newest first, limit 1000, publication name news_publication else config app.name, language = the row's locale. `classes/indexnow.go`: `GenerateKey()` (crypto/rand, hex, 32 characters), `IndexNowArgs` (kind golem15.sitemap.indexnow, URLs) dispatched by the rebuild after commit when enabled, in production and the diff is non-empty (chunks of 10 000), the job posting through a fetchguard client (AllowHostsMode, AllowHosts [api.indexnow.org], Timeout 10 s) and recording indexnow_last_at, _count, _status. `controllers/api/indexnow.go` serves the key file (`GET /_sitemap/indexnow/{file}` with Where `[A-Za-z0-9-]{8,128}\.txt`). Settings tabs Images and News (images_enabled, news_enabled, news_publication and news_sources hidden by trigger while news_enabled is unchecked; news_sources taglist mode array, customTags false, options from article types and their categories) and IndexNow (indexnow_enabled, partial S2 `_indexnow_status.htm`, indexnow_new_key checkbox hidden while disabled; the form's save generates the key). Config keys documented: `indexnow_endpoint` is not configurable (fixed host). COVERAGE.md was written at planning time (IndexNow capability matrix); keep it in sync: when the implementation drops, adds or changes a capability, update its row (every OPT-OUT keeps a reason) and commit it here as group C, otherwise there is no group C. Tests first. Commit groups A and B (plugin), C only if COVERAGE.md changed. + + + go -C ../sm-sitemap-plugin test ./... -run 'TestNewsSitemap|TestIndexNow' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -20 + a "FAIL" line, "no tests to run", or TestNewsSitemap or TestIndexNow missing from the "--- PASS" lines + grep -c "| OPT-OUT |" .planning/phases/33-seo-output-audit-and-content/COVERAGE.md + output is 0, or the command exits non-zero because the file is missing + + + - `grep -n "crypto/rand" ../sm-sitemap-plugin/classes/indexnow.go` prints one line + - `grep -n "AllowHostsMode" ../sm-sitemap-plugin/classes/indexnow.go` prints one line + - `grep -n "sitemap-news/0.9" ../sm-sitemap-plugin/classes/news.go` prints one line + - every OPT-OUT row in COVERAGE.md has a non-empty reason column + + News and IndexNow work behind their switches, never outside production, and the integration's coverage is on record. + + + + Task 3: HTML-sitemap groups API, the HTML sitemap tab, Listing fields, management API and MCP tools + ../sm-sitemap-plugin/classes/htmlgroups.go, ../sm-sitemap-plugin/classes/listing.go, ../sm-sitemap-plugin/controllers/api/auth.go, ../sm-sitemap-plugin/controllers/api/html.go, ../sm-sitemap-plugin/controllers/api/settings.go, ../sm-sitemap-plugin/models/settings/fields.yaml, ../sm-sitemap-plugin/mcp.go, ../sm-sitemap-plugin/routes.go, ../sm-sitemap-plugin/plugin.go, ../sm-sitemap-plugin/lang/en/lang.yaml, ../sm-sitemap-plugin/lang/pl/lang.yaml, ../sm-sitemap-plugin/README.md, ../sm-sitemap-plugin/sitemap_extras_test.go + + - ../sm-seo-plugin/controllers/api/auth.go (manage wrapper), mcp.go, mcp_test.go + - ../sm-seo-plugin/routes.go:79-85 (noindex wrapper) and Buckets() + - ../sm-sitemap-plugin/classes/listing.go (33-03) + - .planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md (L3 group order and markup data, C26 HTML sitemap tab, MCP parity) + + + - TestHTMLGroups: overview for pl lists groups in the L3 order with titles from type__title in pl, categories nested by parent, posts newest first, others alphabetical; a group with 160 entries at page size 150 shows 150 and more=true; GET /html/posts?page=2 answers the rest with pages=2; page 3 → 404; unknown group → 404; html_enabled false → 404 for both; excluded URLs never appear; answers carry X-Robots-Tag noindex + - Listing.Settings answers HTMLEnabled, HTMLPaths {pl:/mapa-strony, en:/sitemap}, HTMLPageSize and KeyFile (.txt when IndexNow is on) + - TestSitemapTools: GET/PUT /_sitemap/api/v1/settings and POST /_sitemap/api/v1/rebuild need a backend principal with the sitemap permission (401/403 otherwise); PUT validates like the form; the MCP catalog lists get_settings, update_settings and rebuild bound to those routes + + + Per D-34, D-10 and UI-SPEC L3/C26: `classes/htmlgroups.go` builds the overview and group pages from the slot table (included rows only, locale filter, L3 order, category tree from parent_refs, paging by html_page_size); `controllers/api/html.go` serves `GET /_sitemap/api/v1/html` and `GET /_sitemap/api/v1/html/{group}` (public, noindex header, throttle bucket `sitemap-html` 300/min per client IP behind trusted proxies). HTML sitemap tab fields (html_enabled switch, html_path mltext with the defaults, html_page_size number) and errors.html_path / errors.page_size validation. listing.go fills the HTML and KeyFile fields. Management API (`controllers/api/auth.go` manage wrapper modeled on the SEO plugin's; `controllers/api/settings.go` show/update; rebuild dispatch) under `/_sitemap/api/v1` with bucket `sitemap-api`; `mcp.go` declares get_settings, update_settings, rebuild (pact.HasMCP, guard backend, the plugin's existing permission). README documents every route, setting, config key and tool. Tests first. Commit groups A, B, C. + + + go -C ../sm-sitemap-plugin test ./... -run 'TestHTMLGroups|TestSitemapTools|TestImageSitemap|TestNewsSitemap|TestIndexNow' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -20 + a "FAIL" line, "no tests to run", or TestHTMLGroups or TestSitemapTools missing from the "--- PASS" lines + go -C ../sm-sitemap-plugin vet ./... && go -C ../sm-sitemap-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "/_sitemap/api/v1/html" ../sm-sitemap-plugin/routes.go ../sm-sitemap-plugin/README.md` prints a line in each + - `grep -n '"get_settings"\|"update_settings"\|"rebuild"' ../sm-sitemap-plugin/mcp.go` prints three lines + - `grep -n "X-Robots-Tag" ../sm-sitemap-plugin/controllers/api/html.go ../sm-sitemap-plugin/routes.go` prints at least one line + + The HTML sitemap has its data API, the sitemap settings are complete and every sitemap operation is reachable through MCP. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| sitemap job → api.indexnow.org | outbound HTTP from the server | +| visitors → HTML-sitemap API | anonymous reads | +| admin/MCP → management API | authenticated writes | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-33-13 | Elevation of privilege (SSRF) | IndexNow ping | high | mitigate | fetchguard AllowHostsMode with the single host api.indexnow.org, HTTPS only, no redirects followed; URL list filtered to the site host (TestIndexNow) | +| T-33-14 | Information disclosure | IndexNow key | low | accept | the key is public by design (served at the root); rotation through indexnow_new_key | +| T-33-15 | Spoofing | staging pings | medium | mitigate | no request outside production (TestIndexNow non-production case) | +| T-33-16 | Denial of service | HTML-sitemap API | medium | mitigate | table reads with paging capped at html_page_size ≤500, per-IP bucket sitemap-html | +| T-33-17 | Elevation of privilege | management API and MCP | high | mitigate | cabana.ResolveBackend + permission check on every route, 401/403 tests in TestSitemapTools | +| T-33-SC | Tampering | npm/pip/cargo installs | high | accept | no new package (fetchguard and crypto/rand are framework/stdlib) | + + + +- `go -C ../sm-sitemap-plugin vet ./... && go -C ../sm-sitemap-plugin test -short ./...` after every commit. +- Docker suite: TestImageSitemap, TestNewsSitemap, TestIndexNow, TestHTMLGroups, TestSitemapTools, plus the 33-03 suite. +- COVERAGE.md has no OPT-OUT row without a reason. + + + +- Image and News sitemaps and IndexNow work behind their switches. +- The HTML-sitemap API answers exactly the indexable set. +- Sitemap settings are manageable through MCP. + + +## Artifacts this phase produces + +- Columns: `golem15_sitemap_urls.image_file_no`, `published_at`, `kind`, `parent_refs`; settings `images_enabled`, `news_enabled`, `news_publication`, `news_sources`, `indexnow_enabled`, `indexnow_key`, `indexnow_last_at`, `indexnow_last_count`, `indexnow_last_status`, `html_enabled`, `html_path`, `html_page_size`; migration 202610280004. +- Go symbols: `classes.GenerateKey`, `IndexNowArgs` (kind `golem15.sitemap.indexnow`), the News and image writers, HTML group builder, API handlers for html, settings, rebuild and the key file. +- Routes: `GET /_sitemap/indexnow/{file}`, `GET /_sitemap/api/v1/html`, `GET /_sitemap/api/v1/html/{group}`, `GET /_sitemap/api/v1/settings`, `PUT /_sitemap/api/v1/settings`, `POST /_sitemap/api/v1/rebuild`; files `sitemap-images-.xml`, `sitemap-news.xml`. +- Throttle buckets: `sitemap-html`, `sitemap-api`. +- MCP tools: `sitemap_get_settings`, `sitemap_update_settings`, `sitemap_rebuild`. +- Partial: `indexnow_status` (S2). +- Files: `classes/images.go`, `classes/news.go`, `classes/indexnow.go`, `classes/htmlgroups.go`, `controllers/api/{auth,html,settings,indexnow}.go`, `mcp.go`, `sitemap_extras_test.go`, `.planning/phases/33-seo-output-audit-and-content/COVERAGE.md` (written at planning time, kept in sync here). + + +Create `.planning/phases/33-seo-output-audit-and-content/33-04-SUMMARY.md` when done. + diff --git a/.planning/phases/33-seo-output-audit-and-content/33-05-PLAN.md b/.planning/phases/33-seo-output-audit-and-content/33-05-PLAN.md new file mode 100644 index 00000000..d179bb39 --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-05-PLAN.md @@ -0,0 +1,341 @@ +--- +phase: 33-seo-output-audit-and-content +plan: 05 +type: execute +wave: 4 +depends_on: ["33-04"] +files_modified: + - ../sm-seo-plugin/go.mod + - ../sm-seo-plugin/go.sum + - ../sm-seo-plugin/go.work + - ../sm-seo-plugin/plugin.go + - ../sm-seo-plugin/menu_item_types.go + - ../sm-seo-plugin/classes/sitemap_filter.go + - ../sm-seo-plugin/classes/registry.go + - ../sm-seo-plugin/classes/site.go + - ../sm-seo-plugin/classes/llms.go + - ../sm-seo-plugin/classes/history.go + - ../sm-seo-plugin/classes/batch.go + - ../sm-seo-plugin/classes/issues.go + - ../sm-seo-plugin/classes/robots.go + - ../sm-seo-plugin/classes/resolver.go + - ../sm-seo-plugin/classes/settings_write.go + - ../sm-seo-plugin/classes/partials.go + - ../sm-seo-plugin/models/page.go + - ../sm-seo-plugin/models/page/columns.yaml + - ../sm-seo-plugin/models/page/fields.yaml + - ../sm-seo-plugin/models/settings.go + - ../sm-seo-plugin/models/settings/fields.yaml + - ../sm-seo-plugin/models/robots.go + - ../sm-seo-plugin/models/registry.go + - ../sm-seo-plugin/updates/202610280002_create_golem15_seo_pages.go + - ../sm-seo-plugin/updates/202610280003_add_golem15_seo_links_and_llms.go + - ../sm-seo-plugin/updates/registry.go + - ../sm-seo-plugin/extensions_robots.go + - ../sm-seo-plugin/extension/_seo_status.htm + - ../sm-seo-plugin/controllers/admin_registry.go + - ../sm-seo-plugin/controllers/pages.go + - ../sm-seo-plugin/controllers/pages/config_list.yaml + - ../sm-seo-plugin/controllers/pages/config_form.yaml + - ../sm-seo-plugin/controllers/pages/config_filter.yaml + - ../sm-seo-plugin/controllers/pages/_page_status.htm + - ../sm-seo-plugin/controllers/api/registry.go + - ../sm-seo-plugin/controllers/api/site.go + - ../sm-seo-plugin/controllers/api/llms.go + - ../sm-seo-plugin/admin_navigation.go + - ../sm-seo-plugin/routes.go + - ../sm-seo-plugin/mcp.go + - ../sm-seo-plugin/operations.go + - ../sm-seo-plugin/config/config.yaml + - ../sm-seo-plugin/lang/en/lang.yaml + - ../sm-seo-plugin/lang/pl/lang.yaml + - ../sm-seo-plugin/README.md + - ../sm-seo-plugin/fixture_test.go + - ../sm-seo-plugin/sitemap_filter_test.go + - ../sm-seo-plugin/registry_test.go +autonomous: true +requirements: [SC-1, SC-3, D-95, D-24, D-33, D-12, D-22, D-25, D-83, D-94, D-10] + +estimate: + tokens: 160000 + raw_tokens: 160000 + tasks: 3 + confidence: low + +must_haves: + truths: + - "sm-seo-plugin publishes a contract.Filter (sm-sitemap-plugin/contract) and requires golem15.sitemap: a URL whose effective SEO is noindex, whose canonical points elsewhere or that redirects leaves the XML and HTML sitemaps with the reason noindex, canonical or redirect; the non-production lock never excludes a URL, so staging and production list the same set (D-95, D-24)" + - "The page registry (SEO › Pages, controller golem15.seo.pages) holds frontend pages with an address per locale, a kind (page, offer, service, landing), a name, a parent, a content-changed date and a note; it is the signpost content type seo-page, so each page gets the SEO tab, resolves ok, and is listed in the sitemaps grouped by kind into sitemap-pages, sitemap-offers, sitemap-services and sitemap-landing-pages files (D-33, D-30)" + - "The frontend server reports its static routes with POST /_seo/api/v1/registry/routes (bearer golem15.seo.route_report_token, else notfound_token): unknown paths are inserted with origin reported and kind page, known ones get reported_at, nothing is ever deleted or overwritten, and a reported page missing from a newer report shows the Not on the site badge and the P9 warning (D-33, UI-SPEC C21)" + - "A registry address that another content type owns refuses with pages.url_owned; a duplicate, an invalid address, a missing default-locale address and a parent loop refuse with their C21 copy (UI-SPEC C21, error: save refused)" + - "GET /_seo/api/v1/site answers the site facts the layer needs: site_url, default_locale, locales, unprefixed_locale, links.new_tab, html_sitemap {enabled, paths, page_size}, routes [{type, pattern}] for the drift check, root_files (verification file names and the IndexNow key file) and llms (D-97 support, D-20c, D-103)" + - "With llms_enabled (robots.txt screen, AI bots tab, default off) GET /llms.txt serves the site's indexable pages from the sitemaps grouped like the HTML sitemap, headed by llms_intro or the default meta description; off, it answers 404 (D-25, UI-SPEC C25)" + - "robots.txt Sitemap: lines name the sitemap index from the sitemap plugin's Listing (/sitemap.xml) unless golem15.seo.robots_sitemaps overrides them; per-type files are never listed (D-22)" + - "The SEO settings get a Links tab with links_new_tab (switch, default on), nofollow_domains and sponsored_domains (one domain per line, subdomains included, settings.domain_invalid on a bad line); all three are MCP-writable through seo_update_settings and reach the resolve answer as outbound {new_tab, nofollow_domains, sponsored_domains} (2026-10-28 layer options: links_new_tab; D-83)" + - "The batch conflict noindex_published fires only when the URL is currently listed in a sitemap file (read through contract.Listing), and a CMS issue sitemap_blocked (Error, tool seo_update_robots) reports sitemap URLs that the production robots.txt blocks for Googlebot (D-12, D-24)" + - "Every SEO write (admin, MCP, batch, import, undo) purges the rendered-site cache after commit through the framework's site-cache purge, so the layer's cached resolve answers and pages refresh (33-CONTEXT Claude's discretion: purge with D-97)" + - "Pages tools seo_list_pages, seo_create_page, seo_update_page and seo_delete_page exist and TestEveryAdminOperationHasTool maps every Pages operation (D-10)" + - "SEO › Pages sits between URL rules and robots.txt in the side menu with the icon files, behind golem15.seo.manage (UI-SPEC C15)" + - "The Pages list shows pages.list_empty when empty (UI-SPEC empty: Pages list)" + - "The SEO tab's P1 shows status.excluded_title with the matching status.excluded_noindex, _canonical, _redirect or _type text when the page is left out of the sitemaps (UI-SPEC partial: page excluded)" + - "A reported page missing from the newest report keeps its data, shows the negative badge pages.state_missing and the P9 warning pages.status_missing, and is never deleted automatically (UI-SPEC partial: registry page missing)" + - {statement: "Two route reports in a row with the same list change only reported_at; a report from a server with a stale build never removes an editor's title, kind or SEO values", verification: backstop} + prohibitions: + - requirement_id: "D-33" + category: safety + status: resolved + verification: test + statement: "The route report never deletes a registry page and never overwrites a value an editor set" + - requirement_id: "D-95" + category: values + status: resolved + verification: judgment + statement: "The SEO plugin reads sitemap data only through the published contract package; the sitemap plugin keeps working without SEO" + artifacts: + - path: "../sm-seo-plugin/classes/sitemap_filter.go" + provides: "contract.Filter implementation on the resolver" + contains: "Decide(" + - path: "../sm-seo-plugin/classes/registry.go" + provides: "page registry service, signpost descriptor, route report" + contains: "func " + - path: "../sm-seo-plugin/controllers/api/site.go" + provides: "GET /_seo/api/v1/site" + contains: "func " + - path: "../sm-seo-plugin/sitemap_filter_test.go" + provides: "TestSitemapFilterTracer, TestFilterReasons, TestSitemapBlockedIssue, TestNoindexPublishedConflict" + contains: "func TestSitemapFilterTracer(" + - path: "../sm-seo-plugin/registry_test.go" + provides: "TestRegistry, TestRouteReport, TestSiteEndpoint, TestLLMSTxt, TestPurgeAfterWrite" + contains: "func TestRegistry(" + key_links: + - from: "../sm-seo-plugin/plugin.go" + to: "../sm-sitemap-plugin/contract/contract.go" + via: "Register publishes the Filter with app.Publish[contract.Filter]; Requires includes golem15.sitemap" + pattern: "contract\\.Filter" + - from: "../sm-seo-plugin/classes/history.go" + to: "summercms.go/modules/cooler/sitecache.go" + via: "each recorded write schedules the site-cache purge after commit" + pattern: "Purge" + - from: "../sm-seo-plugin/menu_item_types.go" + to: "../sm-seo-plugin/classes/registry.go" + via: "the seo-page MenuItemType's Content descriptor (Records, Describe, Match) reads the registry" + pattern: "seo-page" +--- + + + +## Phase Goal + +**As a** site editor and as an SEO specialist auditing a SummerCMS site, **I want to** have the rendered sites emit complete SEO output (sitemaps, structured data, pagination, hreflang, headings, breadcrumbs) from the CMS, and an audit that finds the remaining problems, **so that** sites are indexed correctly and issues can be fixed through MCP. + +This plan connects the SEO plugin to the new sitemaps and gives the frontend layer its site-level facts: the sitemap filter, the page registry with the route report, the site endpoint, llms.txt, the Sitemap: line, the Links settings (new tab, nofollow and sponsored domains) and cache purging after SEO writes. + + +Extend ../sm-seo-plugin (main, not pushed): +- implement and publish `contract.Filter` (D-95, D-24) and require golem15.sitemap; +- the page registry: model, admin screen (C21), signpost type, route report endpoint, MCP tools (D-33, D-10); +- `GET /_seo/api/v1/site`, `GET /llms.txt` (D-25), robots `Sitemap:` from the index (D-22); +- the Links tab (`links_new_tab`, nofollow and sponsored domains) and `outbound` in the resolve answer (2026-10-28 layer options, D-83); +- the refined batch conflict and the `sitemap_blocked` CMS issue (D-12, D-24); +- site-cache purge after every SEO write. + +Purpose: SC-1 (sitemaps consistent with noindex and robots.txt), D-33, and the contract the layer (33-07/33-08) builds on. + +Output: commits in ../sm-seo-plugin. This plan runs after 33-04 (not beside it) because its tests mount the sibling sm-sitemap-plugin checkout through go.work, which 33-04 edits; a concurrent edit there would break this plan's builds. go.mod requires `git.golem15.com/golem15/sm-sitemap-plugin` at its existing tag with the sibling replaced in go.work; 33-14 bumps the require to the new sitemap tag. + +**Planner decisions recorded here:** +- **P33-16 (resolve answer, additive, `costly`):** `Result.Outbound *Outbound` with JSON `outbound: {new_tab: bool, nofollow_domains: [string], sponsored_domains: [string]}`, present on ok answers. No existing key changes. 33-06 and 33-11 add further keys; the layer treats absent keys as empty. +- **P33-17 (site answer, `costly`):** `GET /_seo/api/v1/site` (public group, noindex, bucket seo-resolve) answers `{site_url, default_locale, locales: [..], unprefixed_locale, links: {new_tab}, html_sitemap: {enabled, paths: {locale: path}, page_size}, routes: [{type, pattern}], root_files: [..], llms: bool}`; 33-06 adds `pagination` and `home_crumb`. +- **P33-18 (registry table):** `golem15_seo_pages` (id, url text unique (default-locale path; other locales through the translation writer), title, kind page|offer|service|landing, origin manual|reported|mcp, parent_id, content_updated_at date, reported_at timestamptz, note, timestamps); settings column `routes_reported_at`. Groups: Filter.Groups answers pages, offers, services, landing-pages with titles from `pages.kind_*` per locale; Decide sets Group from the page kind. +- **P33-19 (purge):** the purge is scheduled where history records a write (classes/history.go), so every write path that records history purges once per transaction (kind `seo`, slug = the URL or `*` for settings). +- **P33-20 (route report token):** config `golem15.seo.route_report_token` (empty uses `notfound_token`; both empty → 404), constant-time comparison, body `{routes: [path...]}` of at most 2000 paths of at most 2048 bytes each. + + + +@~/.claude/msd-core/workflows/execute-plan.md +@~/.claude/msd-core/templates/summary.md + + + +@.planning/phases/33-seo-output-audit-and-content/33-CONTEXT.md +@.planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md +@.planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md +@.planning/phases/33-seo-output-audit-and-content/33-PATTERNS.md +@.planning/phases/33-seo-output-audit-and-content/33-03-SUMMARY.md +@.planning/phases/33-seo-output-audit-and-content/33-04-SUMMARY.md +@.planning/phases/32-seo-foundation/32-05-SUMMARY.md +@CLAUDE.md + + +From 33-03: package git.golem15.com/golem15/sm-sitemap-plugin/contract — Filter{Decide(ctx, []Candidate) ([]Decision, error); Groups(ctx) ([]Group, error)}; Candidate{TypeKey, RecordID, Locale, URL}; Decision{Exclude, Reason, Group, LastMod}; Group{Key, Titles}; reasons noindex/canonical/redirect/type; Listing{Settings(ctx) (ListingSettings, error); Entries(ctx, EntriesQuery) (EntriesPage, error)}; ListingSettings{SiteURL, IndexURL, DefaultLocale, Groups, HTMLEnabled, HTMLPaths, HTMLPageSize, KeyFile}. +sm-seo-plugin (Phase 32): classes.Service (NewService(app), SiteURL, Staging, DefaultLocale), Resolver.Resolve(ctx, rawURL) (Result{Action, Status, Location, URL, Locale, Staging, SEO, Sources, TypeKey, RecordID}) and ResolveRecord; classes/settings_write.go (allowed columns, SettingsValues, SetSettingsValue: four places per new column); classes/history.go (Record, WithVia); classes/batch.go (conflict kinds incl. noindex_published); classes/issues.go (Issue{Kind, URL, Locale, Details, Tool}, toolSetURLSEO = "seo_set_url_seo"); classes/robots.go (served body, robots_sitemaps); extensions_robots.go (fields on the robots singleton); controllers/redirects.go + config_*.yaml (list/form pattern); controllers/api/notfound.go (bearer token pattern); mcp.go (tools), operations.go (AdminOperations), operations_test.go (TestEveryAdminOperationHasTool, TestMCPReadToolsAreGET); routes.go (public group with noindex(), management group with BucketAPI); fixture_test.go (fake content plugin acme.pages, fake TranslationWriter); testmain_test.go (Docker Postgres). +cooler: SiteCachePurger (Purge(ctx, kind, slug)) and the after-commit purge helper in modules/cooler/sitecache.go (read the file for the exact name). +Copy: UI-SPEC C15, C20 Links tab, C21, C25 and their copy tables. + + + +## Execution units and resume points + +- Commit groups staged by path in ../sm-seo-plugin; vet and `test -short` green at every commit; lang keys in both files per group; `assets/dist` untouched (no element in this plan). +- Resume check: `git -c safe.directory='*' -C ../sm-seo-plugin log --format=%s | grep -Fx ''`. + +| Task | Commit groups, in order | Resume point | +|------|-------------------------|--------------| +| 1 | `feat: sitemap filter excludes noindex, canonicalised and redirected URLs (tracer)` | TestSitemapFilterTracer passes | +| 2 | A `feat: page registry with its admin screen and signpost type`; B `feat: route report endpoint`; C `feat: MCP tools for registry pages` | registry tests and the operations parity pass | +| 3 | A `feat: site endpoint, llms.txt and the sitemap index in robots.txt`; B `feat: links settings and outbound rules in resolve`; C `feat: sitemap-aware batch conflict and the sitemap_blocked issue`; D `feat: purge the site cache after SEO writes`; E `docs: registry, site endpoint, llms.txt and links in README` | plan complete | + + + + + Task 1: End-to-end "an editor sets a page to noindex and it leaves the XML sitemap with reason noindex" — one path only + ../sm-seo-plugin/go.mod, ../sm-seo-plugin/go.sum, ../sm-seo-plugin/go.work, ../sm-seo-plugin/plugin.go, ../sm-seo-plugin/classes/sitemap_filter.go, ../sm-seo-plugin/fixture_test.go, ../sm-seo-plugin/sitemap_filter_test.go + + - ../sm-sitemap-plugin/contract/contract.go and classes/rebuild.go (33-03) + - ../sm-seo-plugin/plugin.go, classes/resolver.go (ResolveRecord), fixture_test.go, testmain_test.go + - summercms.go/modules/backpack/services.go (Publish keyed by the type argument; duplicate is an error) + - .planning/phases/33-seo-output-audit-and-content/33-RESEARCH.md (Pattern 3) + + + Per D-95, D-24 and P33-18: add the require of git.golem15.com/golem15/sm-sitemap-plugin (its current tag) and `replace git.golem15.com/golem15/sm-sitemap-plugin => ../sm-sitemap-plugin` in go.work; plugin.go adds "golem15.sitemap" to Requires and publishes `classes.SitemapFilter` with `app.Publish[contract.Filter]` in Register. `classes/sitemap_filter.go`: Decide resolves each candidate's record (ResolveRecord by type, id, locale; batched through one DB handle) ignoring the non-production lock: robots noindex → Exclude noindex; canonical whose path differs from the candidate's → Exclude canonical; a redirect source → Exclude redirect; else include. Groups answers nothing yet (Task 2 adds the registry kinds). The test fixture app mounts the sitemap plugin (import of its package in the test) next to the fake content plugin. `TestSitemapFilterTracer` (Docker): two published fake pages; set the first to noindex through the SEO tab admin save; run the sitemap rebuild; the sitemap file lists only the second; the excluded row's reason is noindex; the same holds with the app environment set to staging. Commit as listed. + + + go -C ../sm-seo-plugin test ./... -run 'TestSitemapFilterTracer' -count=1 -v 2>&1 | tail -20 + non-zero exit, "FAIL" or "no tests to run" in the output, or no "--- PASS: TestSitemapFilterTracer" line + go -C ../sm-seo-plugin vet ./... && go -C ../sm-seo-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n "golem15.sitemap" ../sm-seo-plugin/plugin.go` prints at least one line + - `grep -n "sm-sitemap-plugin/contract" ../sm-seo-plugin/classes/sitemap_filter.go ../sm-seo-plugin/plugin.go` prints a line in each + - `grep -n "sm-sitemap-plugin =>" ../sm-seo-plugin/go.work` prints one line + + The SEO plugin decides what the sitemap lists, and a noindex page leaves the sitemap. + + + + Task 2: Page registry (model, admin screen, signpost type seo-page, sitemap groups), route report endpoint and Pages MCP tools + ../sm-seo-plugin/models/page.go, ../sm-seo-plugin/models/page/columns.yaml, ../sm-seo-plugin/models/page/fields.yaml, ../sm-seo-plugin/models/registry.go, ../sm-seo-plugin/updates/202610280002_create_golem15_seo_pages.go, ../sm-seo-plugin/updates/registry.go, ../sm-seo-plugin/classes/registry.go, ../sm-seo-plugin/classes/sitemap_filter.go, ../sm-seo-plugin/menu_item_types.go, ../sm-seo-plugin/controllers/admin_registry.go, ../sm-seo-plugin/controllers/pages.go, ../sm-seo-plugin/controllers/pages/config_list.yaml, ../sm-seo-plugin/controllers/pages/config_form.yaml, ../sm-seo-plugin/controllers/pages/config_filter.yaml, ../sm-seo-plugin/controllers/pages/_page_status.htm, ../sm-seo-plugin/controllers/api/registry.go, ../sm-seo-plugin/admin_navigation.go, ../sm-seo-plugin/routes.go, ../sm-seo-plugin/mcp.go, ../sm-seo-plugin/operations.go, ../sm-seo-plugin/config/config.yaml, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/registry_test.go + + - ../sm-seo-plugin/controllers/redirects.go, controllers/redirects/config_{list,form,filter}.yaml, models/redirect.go and models/redirect/{columns,fields}.yaml + - ../sm-seo-plugin/controllers/api/notfound.go (token check) and controllers/api/redirects.go (management CRUD) + - ../sm-seo-plugin/classes/owner.go (FindOwner via MatchPath) and classes/rules.go + - ../sm-journal-plugin/classes/menu_item_types.go (a MenuItemType with a Content descriptor) + - ../sm-seo-plugin/operations.go and operations_test.go + - .planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md (C15, C21, Pages copy table, Destructive actions) + + + - Create /kontakt (kind page, title Kontakt) → it resolves ok with the SEO tab values; it is a seo-page record in signpost; the sitemap lists it in sitemap-pages-1.xml; a kind offer page lands in sitemap-offers-1.xml + - A page with en address /contact and pl /kontakt has two locale URLs and hreflang alternates; an empty en address means no en URL (UI-SPEC Q-15) + - Address /blog/lwia-grzywa owned by a fake record → 422 pages.url_owned naming the type and title; duplicate → pages.url_taken; "kontakt" → pages.url_invalid; empty default-locale address → pages.url_required; parent loop → pages.parent_loop + - Route report with the right token and ["/", "/kontakt", "/o-nas"] inserts /o-nas (origin reported, kind page, title derived from the last segment) and updates reported_at of the others; a later report without /o-nas marks it missing (badge, P9 warning) and deletes nothing; wrong token 401; no token configured 404; 2001 paths 422; a path owned by a content type is skipped + - Deleting a page deletes its SEO entries (the existing RecordDeleted cleanup) and it leaves the sitemaps on the next rebuild + - Filter.Groups answers pages, offers, services, landing-pages with the kind titles per locale; Decide groups registry candidates by kind + - MCP lists list_pages, create_page, update_page, delete_page; TestEveryAdminOperationHasTool passes with the Pages controller + + + Per D-33, D-10, UI-SPEC C21 and P33-18/P33-20: migration 202610280002 creates golem15_seo_pages and the settings column routes_reported_at; models/page.go with url and title translatable through the translation writer (as the SEO settings' ml fields are), Rules and the C21 validation (FormBeforeCreate/Update hook in the controller translating :type/:title params, the Phase 32 pattern); `classes/registry.go`: CRUD service with history, `ReportRoutes(ctx, paths) (ReportResult, error)`, and the descriptor functions; `menu_item_types.go` registers MenuItemType `seo-page` (label pages.name, Content{Controller golem15.seo.pages, Model page morph, Kind page, Records, Describe (per-locale URLs, Title, LastMod content_updated_at, Parents [seo-page parent], Published true), Match exact}). Pages screen: config_list.yaml (title pages.list_title, recordUrl update, 20 per page, checkboxes, setup, toolbar [create, delete], noRecordsMessage pages.list_empty, defaultSort url asc, row state negative pages.state_missing for reported pages with reported_at before routes_reported_at), columns and filters per C21, form per C21 with P9 `_page_status.htm` (pages.status_reported plain or pages.status_missing warning, nothing for manual pages); navigation item Pages (icon files) after URL rules, permission golem15.seo.manage. `controllers/api/registry.go`: `POST /_seo/api/v1/registry/routes` (public group, bucket seo-notfound, P33-20 token) and the management CRUD `GET/POST /_seo/api/v1/pages`, `PUT/DELETE /_seo/api/v1/pages/{id}`; MCP tools list_pages, create_page, update_page, delete_page (destructive) and the AdminOperations rows for controller golem15.seo.pages. Config key `route_report_token` documented in config/config.yaml. Every C21 copy key in both lang files. Tests first in registry_test.go (`TestRegistry`, `TestRouteReport`). Commit groups A, B, C. + + + go -C ../sm-seo-plugin test ./... -run 'TestRegistry|TestRouteReport|TestEveryAdminOperationHasTool|TestMCPReadToolsAreGET|TestSitemapFilterTracer' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -20 + a "FAIL" line, "no tests to run", or TestRegistry, TestRouteReport or TestEveryAdminOperationHasTool missing from the "--- PASS" lines + go -C ../sm-seo-plugin vet ./... && go -C ../sm-seo-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n '"seo-page"' ../sm-seo-plugin/menu_item_types.go ../sm-seo-plugin/classes/registry.go` prints at least one line + - `grep -n "/registry/routes" ../sm-seo-plugin/routes.go` prints one line and `grep -n "subtle.ConstantTimeCompare" ../sm-seo-plugin/controllers/api/registry.go` prints one line + - `grep -n '"list_pages"\|"create_page"\|"update_page"\|"delete_page"' ../sm-seo-plugin/mcp.go` prints four lines + - TestRouteReport asserts that a later report deletes nothing and keeps editor-set titles + + Frontend pages live in a CMS registry with the SEO tab, sitemap groups by kind, a safe route report and MCP tools. + + + + Task 3: Site endpoint, llms.txt, robots Sitemap: from the index, Links settings with outbound rules in resolve, sitemap-aware conflict and issue, purge after writes + ../sm-seo-plugin/classes/site.go, ../sm-seo-plugin/classes/llms.go, ../sm-seo-plugin/classes/robots.go, ../sm-seo-plugin/classes/resolver.go, ../sm-seo-plugin/classes/settings_write.go, ../sm-seo-plugin/classes/batch.go, ../sm-seo-plugin/classes/issues.go, ../sm-seo-plugin/classes/history.go, ../sm-seo-plugin/classes/partials.go, ../sm-seo-plugin/models/settings.go, ../sm-seo-plugin/models/settings/fields.yaml, ../sm-seo-plugin/models/robots.go, ../sm-seo-plugin/updates/202610280003_add_golem15_seo_links_and_llms.go, ../sm-seo-plugin/updates/registry.go, ../sm-seo-plugin/extensions_robots.go, ../sm-seo-plugin/extension/_seo_status.htm, ../sm-seo-plugin/controllers/api/site.go, ../sm-seo-plugin/controllers/api/llms.go, ../sm-seo-plugin/routes.go, ../sm-seo-plugin/config/config.yaml, ../sm-seo-plugin/lang/en/lang.yaml, ../sm-seo-plugin/lang/pl/lang.yaml, ../sm-seo-plugin/README.md, ../sm-seo-plugin/registry_test.go, ../sm-seo-plugin/sitemap_filter_test.go + + - ../sm-seo-plugin/classes/settings_write.go (allowed columns ~36-41, SettingsValues 46-69, SetSettingsValue 72+, history and after-commit job 301-313) + - ../sm-seo-plugin/classes/robots.go (served body, Sitemap lines), classes/batch.go (conflicts), classes/issues.go, classes/history.go + - ../sm-seo-plugin/extensions_robots.go and extension/_seo_status.htm + - summercms.go/modules/cooler/sitecache.go (the after-commit purge helper) + - .planning/phases/33-seo-output-audit-and-content/33-UI-SPEC.md (C20 Links tab, C25, SEO tab additions copy for status.excluded_*) + + + - TestSiteEndpoint: /_seo/api/v1/site answers every P33-17 key; root_files lists each verification file name and, when the sitemap's IndexNow is on, its key file; routes lists each content type's layered pattern; X-Robots-Tag noindex + - TestLLMSTxt: off → 404; on → text/plain with "# ", the intro (or default description) as a quote line, then one "## " section per non-empty group with "- [title](absolute url)" lines; excluded URLs never appear + - Robots: without robots_sitemaps override the served file has exactly "Sitemap: /sitemap.xml"; with the override, the configured lines + - Links tab: links_new_tab default true; nofollow_domains "example.com\nhttps://x.pl/a" → 422 settings.domain_invalid naming line 2; the resolve answer of an ok page has outbound {new_tab: true, nofollow_domains: [example.com], sponsored_domains: []}; seo_update_settings with links_new_tab false turns it off + - TestNoindexPublishedConflict: a batch noindex on a URL that is in a sitemap file reports noindex_published; on a URL not listed (type switched off) it does not + - TestSitemapBlockedIssue: robots.txt "Disallow: /blog/" with a listed /blog/x → issue sitemap_blocked (Error, tool seo_update_robots, details.line) + - TestPurgeAfterWrite: an SEO tab save, a redirect create through MCP and a settings save each post one purge after commit; a rolled-back batch posts none + - P1 shows status.excluded_title with the reason text for a noindex page and for a page whose type is off in the sitemap settings + + + Per D-22, D-24, D-25, D-83, the 2026-10-28 layer options and P33-16/P33-17/P33-19: migration 202610280003 adds settings links_new_tab boolean NOT NULL DEFAULT true, nofollow_domains text, sponsored_domains text, and robots llms_enabled boolean NOT NULL DEFAULT false, llms_intro text. Settings: the four-place pattern of settings_write.go for the three Links columns with domain validation (lowercase host names, one per line, no scheme or path), the Links tab in models/settings/fields.yaml (links_new_tab switch right with settings.links_new_tab_comment; nofollow_domains and sponsored_domains textarea small left/right with settings.domains_comment); robots singleton fields llms_enabled and llms_intro (hide trigger) through extensions_robots.go. `classes/site.go` builds the P33-17 answer (Listing.Settings via app.Lookup[contract.Listing], verification names, signpost patterns); `controllers/api/site.go` serves it; `classes/llms.go` + `controllers/api/llms.go` serve the literal root route `GET /llms.txt` (bucket seo-robots). robots.go reads the index URL from Listing.Settings when `robots_sitemaps` is not set in config (the config default line changes to a comment explaining the fallback). resolver.go adds Outbound (P33-16). batch.go: noindex_published only when a contract.Listing entry lists the URL. issues.go: kind `sitemap_blocked` (Error, tool seo_update_robots) using the grobotstxt matcher against the production body. history.go schedules the after-commit purge (P33-19). partials.go/_seo_status.htm: the excluded callout from Filter decisions for the record. README: routes, config keys, the registry, the site answer, llms.txt, Links and purge. Tests first. Commit groups A to E. + + + go -C ../sm-seo-plugin test ./... -run 'TestSiteEndpoint|TestLLMSTxt|TestNoindexPublishedConflict|TestSitemapBlockedIssue|TestPurgeAfterWrite|TestRobots|TestSettingsLinks' -count=1 -v 2>&1 | grep -E '^(--- |FAIL|ok)' | tail -30 + a "FAIL" line, "no tests to run", or TestSiteEndpoint, TestLLMSTxt, TestNoindexPublishedConflict, TestSitemapBlockedIssue or TestPurgeAfterWrite missing from the "--- PASS" lines + go -C ../sm-seo-plugin vet ./... && go -C ../sm-seo-plugin test -short ./... + non-zero exit or a "FAIL" line + + + - `grep -n '"/llms.txt"\|"/site"' ../sm-seo-plugin/routes.go` prints two lines + - `grep -n "links_new_tab" ../sm-seo-plugin/classes/settings_write.go ../sm-seo-plugin/models/settings/fields.yaml ../sm-seo-plugin/classes/resolver.go` prints a line in each + - `grep -n "sitemap_blocked" ../sm-seo-plugin/classes/issues.go` prints at least one line + - existing resolve keys are unchanged: TestResolverPrecedence and the Phase 32 resolve tests pass + + The layer has a site endpoint, robots and llms.txt follow the sitemap, outbound link rules are settings, conflicts and issues know the sitemap, and every SEO write refreshes the rendered site. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| frontend server → route report | token-authenticated server-to-server POST | +| visitors → /site, /llms.txt | anonymous reads | +| admin/MCP → registry and settings | authenticated writes | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-33-18 | Tampering | route report poisoning the registry | high | mitigate | bearer token with constant-time comparison, size caps, insert-only (never delete or overwrite editor values), owned paths skipped; TestRouteReport | +| T-33-19 | Information disclosure | /site answer | medium | mitigate | answers only public facts (site URL, locales, patterns, public root file names); no token, no unpublished data; reviewed against P33-17 in TestSiteEndpoint | +| T-33-20 | Information disclosure | llms.txt | medium | mitigate | built from the sitemap's included set only (noindex/unpublished never present), off by default | +| T-33-21 | Elevation of privilege | Pages admin and MCP | high | mitigate | golem15.seo.manage on every operation; TestEveryAdminOperationHasTool and the manage wrapper | +| T-33-22 | Tampering | domain lists | low | mitigate | validated host names only; rendered by the layer as rel tokens, never as markup | +| T-33-23 | Denial of service | purge after writes | low | mitigate | one purge per transaction after commit, coalesced by the framework purger | +| T-33-SC | Tampering | npm/pip/cargo installs | high | accept | no new package; sm-sitemap-plugin is a first-party sibling module | + + + +- `go -C ../sm-seo-plugin vet ./... && go -C ../sm-seo-plugin test -short ./...` after every commit. +- Docker suite: TestSitemapFilterTracer, TestRegistry, TestRouteReport, TestSiteEndpoint, TestLLMSTxt, TestNoindexPublishedConflict, TestSitemapBlockedIssue, TestPurgeAfterWrite and the Phase 32 suites. +- `go -C ../sm-sitemap-plugin test -short ./...` still passes (sitemap without SEO). + + + +- Sitemaps leave out noindex, canonicalised and redirected URLs through the published filter. +- The page registry works from the admin, MCP and the route report. +- The site endpoint, llms.txt, robots Sitemap:, the Links settings and purge after writes work. + + +## Artifacts this phase produces + +- Go symbols: `classes.SitemapFilter` (Decide, Groups), registry service with `ReportRoutes`, `classes.Outbound` and `Result.Outbound`, site and llms builders, API handlers for registry, pages, site and llms. +- Menu item type: `seo-page` (Kind page). +- Tables/columns: `golem15_seo_pages`; settings `routes_reported_at`, `links_new_tab`, `nofollow_domains`, `sponsored_domains`; robots `llms_enabled`, `llms_intro`; migrations 202610280002, 202610280003. +- Routes: `POST /_seo/api/v1/registry/routes`, `GET /_seo/api/v1/site`, `GET /llms.txt`, `GET/POST /_seo/api/v1/pages`, `PUT/DELETE /_seo/api/v1/pages/{id}`; resolve answer key `outbound`. +- Admin: controller `golem15.seo.pages` (Pages), partial `page_status` (P9), Links tab, llms fields on the robots screen; issue kind `sitemap_blocked`. +- MCP tools: `seo_list_pages`, `seo_create_page`, `seo_update_page`, `seo_delete_page`. +- Config key: `golem15.seo.route_report_token`. +- Tests: `TestSitemapFilterTracer`, `TestRegistry`, `TestRouteReport`, `TestSiteEndpoint`, `TestLLMSTxt`, `TestNoindexPublishedConflict`, `TestSitemapBlockedIssue`, `TestPurgeAfterWrite`. + + +Create `.planning/phases/33-seo-output-audit-and-content/33-05-SUMMARY.md` when done. + diff --git a/.planning/phases/33-seo-output-audit-and-content/33-06-PLAN.md b/.planning/phases/33-seo-output-audit-and-content/33-06-PLAN.md new file mode 100644 index 00000000..403e7a14 --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-06-PLAN.md @@ -0,0 +1,320 @@ +--- +phase: 33-seo-output-audit-and-content +plan: 06 +type: execute +wave: 5 +depends_on: ["33-05"] +files_modified: + - ../sm-seo-plugin/classes/schema/graph.go + - ../sm-seo-plugin/classes/schema/nodes.go + - ../sm-seo-plugin/classes/schema/validate.go + - ../sm-seo-plugin/classes/schema/custom.go + - ../sm-seo-plugin/classes/breadcrumbs.go + - ../sm-seo-plugin/classes/alternates.go + - ../sm-seo-plugin/classes/pagination.go + - ../sm-seo-plugin/classes/resolver.go + - ../sm-seo-plugin/classes/normalize.go + - ../sm-seo-plugin/classes/site.go + - ../sm-seo-plugin/classes/settings_write.go + - ../sm-seo-plugin/classes/entries.go + - ../sm-seo-plugin/classes/field_types.go + - ../sm-seo-plugin/classes/partials.go + - ../sm-seo-plugin/classes/manage.go + - ../sm-seo-plugin/models/settings.go + - ../sm-seo-plugin/models/settings/fields.yaml + - ../sm-seo-plugin/updates/202610280004_add_golem15_seo_publisher_pagination.go + - ../sm-seo-plugin/updates/registry.go + - ../sm-seo-plugin/extensions.go + - ../sm-seo-plugin/extension/_seo_status.htm + - ../sm-seo-plugin/extension/_seo_effective.htm + - ../sm-seo-plugin/controllers/settings/_publisher_preview.htm + - ../sm-seo-plugin/controllers/settings/config_form.yaml + - ../sm-seo-plugin/controllers/api/graph.go + - ../sm-seo-plugin/controllers/api/seo.go + - ../sm-seo-plugin/field_types.go + - ../sm-seo-plugin/routes.go + - ../sm-seo-plugin/mcp.go + - ../sm-seo-plugin/operations.go + - ../sm-seo-plugin/config/config.yaml + - ../sm-seo-plugin/assets/src/elements/schema.ts + - ../sm-seo-plugin/assets/src/elements/graph.ts + - ../sm-seo-plugin/assets/src/elements/parent.ts + - ../sm-seo-plugin/assets/src/elements/index.ts + - ../sm-seo-plugin/assets/src/elements/labels.ts + - ../sm-seo-plugin/assets/src/elements/styles.css + - ../sm-seo-plugin/assets/dist/seo.js + - ../sm-seo-plugin/assets/dist/seo.css + - ../sm-seo-plugin/lang/en/lang.yaml + - ../sm-seo-plugin/lang/pl/lang.yaml + - ../sm-seo-plugin/README.md + - ../sm-seo-plugin/structured_data_test.go +autonomous: true +requirements: [SC-2, SC-3, D-43, D-44, D-45, D-46, D-03, D-86, D-55, D-56, D-57, D-40, D-41, D-42, D-21, D-94, D-10] + +estimate: + tokens: 170000 + raw_tokens: 170000 + tasks: 3 + confidence: low + +must_haves: + truths: + - "The resolve answer of every ok page carries jsonld: one {\"@context\": \"https://schema.org\", \"@graph\": [...]} whose nodes link by @id: WebPage (or its subtype) + BreadcrumbList + references to WebSite and the publisher on every page; WebSite (+ SearchAction when golem15.seo.search_url is set) and the publisher on the home page; BlogPosting/Article/NewsArticle with author Person, publisher, image, datePublished, dateModified, wordCount and articleSection for article content; CollectionPage with mainEntity pointing to #itemlist for collections (the layer adds the ItemList node); ProfilePage + Person for profiles; Service with provider, areaServed and an optional Offer for registry offers and services; ContactPage + the publisher with address and opening hours; a VideoObject per YouTube video (D-43, D-94, D-96)" + - "The publisher is set in the global settings (Publisher tab: Organisation or Person, name, logo through seoimage, url, email, phone, sameAs; Local business tab behind a switch with type, address, geo, opening hours and price range); a person cannot be a local business (settings.local_person) and every C20 refusal answers 422 (D-45, D-03)" + - "The SEO tab gains Structured data (E8 seoschema: type select per kind, Service fields, custom JSON-LD with live parse feedback), Structured data preview (E9: the effective @graph of the saved page plus unsaved schema, required and recommended findings, the Rich Results Test link) and Primary category for content whose descriptor supplies Parents (D-44, D-46, D-86)" + - "Saving custom JSON-LD refuses invalid JSON, non-objects, more than 16 KB, nesting deeper than 8, a literal