diff --git a/.planning/phases/33-seo-output-audit-and-content/33-VERIFICATION.md b/.planning/phases/33-seo-output-audit-and-content/33-VERIFICATION.md new file mode 100644 index 00000000..bb16d60e --- /dev/null +++ b/.planning/phases/33-seo-output-audit-and-content/33-VERIFICATION.md @@ -0,0 +1,304 @@ +--- +phase: 33-seo-output-audit-and-content +verified: 2026-11-04T20:20:34Z +status: gaps_found +score: "Roadmap success criteria 2/4 verified (SC3, SC4); SC1 pending the release plus partial gap G1; SC2 partial gap G3. Plan must-haves 191/202 truths verified: 2 failed (G3), 5 pending the 33-14 Task 4 release, 3 not applicable by Jakub's 2026-11-04 decisions, 1 human" +covered_files: + - .planning/phases/33-seo-output-audit-and-content/33-01-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-01-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-02-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-02-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-03-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-03-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-04-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-04-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-05-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-05-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-06-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-06-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-07-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-07-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-08-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-08-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-09-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-09-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-10-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-10-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-11-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-11-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-12-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-12-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-13-PLAN.md + - .planning/phases/33-seo-output-audit-and-content/33-13-SUMMARY.md + - .planning/phases/33-seo-output-audit-and-content/33-14-PLAN.md + - admin/openapi/admin.json + - docs/backend/field-types.md + - docs/database/attachments.md + - docs/database/casts-and-validation.md + - docs/plugins/menu-item-types.md + - go.mod + - modules/cabana/README.md + - modules/cabana/admin_openapi.go + - modules/cabana/backend_users_admin.go + - modules/cabana/field_type.go + - modules/cabana/http.go + - modules/cabana/schema_types.go + - modules/lagoon/README.md + - modules/lagoon/attach/variant.go + - modules/lagoon/attach/webp/webp.go + - modules/lagoon/validate.go + - modules/pact/README.md + - modules/pact/capabilities.go + - modules/pact/content.go + - modules/phrasebook/lang/en/validate.yaml + - modules/phrasebook/lang/pl/validate.yaml + - modules/phrasebook/lang/pl/validation.yaml + - modules/signpost/README.md + - scripts/check-phase33.sh +covered_digest: "v2:sha256:26f6b27b662a671e43464f8c68ec483f4712b0c68be80466e07a95b00c59654d" +behavior_unverified: 0 +overrides_applied: 0 +gaps: + - truth: "D-57 / SC2 hreflang: a page without a translation has no hreflang for that locale (33-06: 'a locale without a translation is absent'; 33-03: alternates 'for each translated locale')" + status: failed + reason: "The SEO and sitemap plugins drop only locales without a URL (sm-seo-plugin classes/alternates.go translatedLocales; sm-sitemap-plugin classes/rebuild.go). Journal, the main content type, gives every enabled translate locale a URL: classes/content_descriptors.go loops items[i].Alternates, which menuAlternates (classes/menu_item_types.go:686) builds for every enabled locale with fallback values. So on a site with an enabled locale that a record has no translation for, the record gets hreflang links and sitemap addresses for that locale, serving default-locale content under the other locale's URL (or a 404). 33-13 observed exactly this on grzyby (every post listed twice, /en/ addresses all 404) and fixed it by data only. The tests that prove D-57 (TestHreflangAlternates, TestAlternates) use fixture records with an empty URL, which the Journal never produces (fixture-only proof). deferred-items.md: status open." + artifacts: + - path: "../sm-journal-plugin/classes/content_descriptors.go" + issue: "post, category and tag descriptors emit a pact.ContentLocale for every enabled locale, translated or not" + - path: "../sm-journal-plugin/classes/menu_item_types.go" + issue: "menuAlternates: one alternate per enabled locale with fallback (Phase 17 PHP parity, used by the descriptors)" + missing: + - "Either the Journal descriptors list a locale only when the record has content in it (the default locale or a translation row), leaving the menu item alternates unchanged, or Jakub accepts the data-only mitigation (disable locales without content) with an override" + - "A test on the real Journal descriptor path (not a fixture) asserting no alternate for an untranslated enabled locale" + - truth: "SC1 / D-24: the sitemaps stay consistent with noindex (noindex pages leave the XML and HTML sitemaps)" + status: partial + reason: "Recorded by Jakub as a gap, not fixed in 33-14 (deferred-items.md, 33-12 'An additional address that the site serves noindex stays in the sitemap'). CMS-side noindex, canonical and redirect exclusion works and is tested (TestFilterExcludes, TestSitemapFilterTracer, TestFilterReasons), and Nuxt-side noindex pages never enter the registry because the route report leaves them out (vue-seo-layer lib/routes.ts staticRoutes). But a page whose noindex lives only in Nuxt (seo.noindexRoutes, requiresAuth, seo.robots) and that an editor or migration 202610280002 puts into Additional addresses stays in the sitemap with X-Robots-Tag noindex. The crawler reports it as sitemap_noindex, but routes it to seo_set_url_seo (classes/crawl/checks.go:355), which cannot fix a Nuxt-side noindex; the working fix is removing the address with sitemap_update_settings. The demo and grzyby deploy steps remove the migrated addresses by hand." + artifacts: + - path: "../vue-seo-layer/server/plugins/seo-route-report.ts" + issue: "the route report carries no noindex paths, so the CMS cannot know a Nuxt-only noindex" + - path: "../sm-seo-plugin/classes/sitemap_filter.go" + issue: "the filter judges the CMS's effective SEO only" + - path: "../sm-seo-plugin/classes/crawl/checks.go" + issue: "sitemap_noindex names seo_set_url_seo, which does not fix this case" + missing: + - "Route report sends the noindex static paths; sm-seo-plugin stores them and the filter excludes them (contract change across vue-seo-layer and sm-seo-plugin), or an accepted override for this release" + - "Optionally: sitemap_noindex on an additional address names sitemap_update_settings" +human_verification: + - test: "Release decision on the gaps (escalation): G3 (Journal fallback hreflang, D-57) and G1 (Nuxt-only noindex address in the sitemap). For each, fix before the tags or accept with an override in this file's frontmatter (ready-to-paste blocks in 'Gaps Summary')." + expected: "Both resolved or overridden; then re-run verification, which should report human_needed for the release-time items below" + why_human: "Product and release decision: G3's fix changes a core plugin's descriptor output; G1 is a recorded contract gap Jakub chose not to fix in 33-14" + - test: "SC1 production, after 33-14 Task 4 (tags, pins, 33-13 patches, deploy to grzybyfunkcjonalne.pl per DEPLOY.md steps 1-12): run the 33-13-deploy-log.md checklist items 1-8 on https://grzybyfunkcjonalne.pl" + expected: "/sitemap.xml is a sitemapindex of sitemap-posts-1, -categories-1, -authors-1, -pages-1 and -images-1; posts file lists the 62 posts once each with lastmod and no /en/ or hreflang; /sitemap.xsl 404; a post has one canonical, one JSON-LD script, no hreflang=en, breadcrumbs; SEO › Pages and Settings › Sitemaps as listed; a crawl finishes without /en/ 404s or duplicates; GSC reads the index" + why_human: "Pending the release: the production deploy happens after the tags by Jakub's decision (33-13-deploy-log.md, after-tags; there is no uat any more). Not a failure." + - test: "Before the grzyby deploy, on the tagged build: scripts/urls.txt check of sm-grzybyfunkcjonalne-app (every public URL answers as before)" + expected: "Every urls.txt line passes (33-13 truth 1, prohibition D-32)" + why_human: "Pending the release: 33-13 proved it on local links only; it must be repeated on the tagged versions in Task 4" + - test: "SC3 live workflow: run a crawl from SEO › Crawls on a deployed site, then from a real Claude client linked over MCP ask it to fix the reported issues with dry_run, review the preview, apply" + expected: "Every data issue names a tool the client can call; dry runs write nothing; applying fixes the data issues; fix: code issues are named as template work" + why_human: "External agent-driven workflow (33-VALIDATION.md manual-only row). TestCrawlerChecks and TestEveryAdminOperationHasTool prove the routing, not a live client session" + - test: "D-46: Google's Rich Results Test for a post, a category and an FAQ page from the admin's Rich Results link" + expected: "No errors for BlogPosting, CollectionPage/ItemList and FAQPage" + why_human: "External Google tool (33-VALIDATION.md manual-only row)" + - test: "Visual check after the deploys: grzyby home, /blog, a category, a post, /autor/admin on desktop and mobile (whole-card links, no 'more' link, post title markup, breadcrumbs, footer 'Mapa strony'); the demo on prefix_except_default (/pl/... addresses, LanguageSwitcher, /pl/mapa-strony); the new admin screens (SEO tab E8 to E10, Site audit, Crawls, Auto-links, Pages, Settings › Sitemaps) including the Blog Editor outline with long headings at 360px (33-02 planner assumption)" + expected: "Layouts render as in 33-UI-SPEC.md; long outline headings wrap with a hanging indent and are not truncated" + why_human: "Visual rendering; element tests run under happy-dom without layout" + - test: "33-14 Task 4 release items not yet in any patch: add -tags nodynamic to the WebP-linking app builds (sm-demo-app, sm-grzybyfunkcjonalne-app, sm-summercmsio-app scripts/build.sh or summer.yaml build.tags) and correct the 33-13 patch's DEPLOY.md 'dynamically linked' row; apply 33-14-*-locale.patch on top of the 33-12 demo patches; vue-journal-layer check-git-install.sh against the tag; changelog posts" + expected: "Release builds are static (file bin/: statically linked); every step logged in 33-release-log.md" + why_human: "Pending the release by design (Task 4 runs after this verification)" +--- + +# Phase 33: SEO output, audit and content Verification Report + +**Phase Goal:** As a site editor and as an SEO specialist auditing a SummerCMS site, I want to have the rendered sites emit complete SEO output (sitemaps, structured data, pagination, hreflang, headings, breadcrumbs) from the CMS, and an audit that finds the remaining problems, so that sites are indexed correctly and issues can be fixed through MCP. +**Verified:** 2026-11-04T20:20:34Z +**Status:** gaps_found +**Re-verification:** No, initial verification (plan 33-14 paused at its release checkpoint; Tasks 1 and 2 done) +**Mode:** mvp. The ROADMAP goal reads "I want the rendered sites"; the plans' normalised form above passes `user-story.validate`. + +Repositories checked (commits since the base tag; none tagged, all ahead of origin): summercms.go 72 since v0.6.0, sm-sitemap-plugin 17 since v0.1.2, sm-seo-plugin 29 since v0.1.1, sm-journal-plugin 14 since v0.3.1, vue-seo-layer 18 since v0.1.1, vue-journal-layer 11 since v0.7.1, vue-menu-layer 1 and vue-faq-layer 3 since v0.4.0, sm-bm-app bd7ee84 (local D-72 commit). + +## User Flow Coverage + +| Step | Expected | Evidence | Status | +|------|----------|----------|--------| +| Editor publishes content | Sitemap index with per-type files, real lastmod, stable numbering | sm-sitemap-plugin classes/{slots,writer,lastmod,rebuild}.go; TestSitemapIndex, TestStableNumbering, TestLastmod (docker stage PASS) | ✓ | +| Crawler/visitor opens a page | SSR head from the CMS, one JSON-LD @graph, hreflang, breadcrumbs, pagination | vue-seo-layer useCmsSeo.ts, seo.ts, Breadcrumbs.vue, Pagination.vue; smoke:ssr expect-seo.txt (count=1 canonical/ld+json); layers stage PASS | ✓ except hreflang on untranslated Journal locales (G3) | +| Build of an indexable client-only route | Build fails | lib/guard.ts, testing/guard-check.mjs (guard:check in layers stage PASS) | ✓ | +| SEO specialist runs the audit | Issues per URL, each with an MCP tool or fix: code | classes/crawl/{crawler,checks,fix}.go; TestCrawlerChecks, TestCrawlerTracer; MCP seo_start_crawl … seo_unignore_issues | ✓ (live MCP session: human) | +| Outcome | "sites are indexed correctly and issues can be fixed through MCP" | Holds on the code and fixture sites; production grzybyfunkcjonalne.pl pending the release; G1 and G3 remain | ⚠ gaps + pending release | + +## Goal Achievement + +### Roadmap Success Criteria + +| # | Truth | Status | Evidence | +|---|-------|--------|----------| +| SC1 | Every site serves a sitemap index with per-type files of a configurable size, stable numbering and real lastmod, consistent with noindex and robots.txt, and grzybyfunkcjonalne.pl runs on the new format | ⚠ PENDING RELEASE + partial gap G1 | Index, per-type files, file_size 1–50 000, stable slots, real lastmod, filter, robots.txt `Sitemap:` line, sitemap_blocked issue: code and Docker suites green. **grzybyfunkcjonalne.pl on the new format is pending the post-tag production deploy (33-13 after-tags decision, no uat any more): human_needed, not failed.** Noindex consistency fails for Nuxt-only noindex pages listed as Additional addresses (G1, recorded by Jakub as a gap) | +| SC2 | Indexable pages render server-side with head tags, one JSON-LD @graph, hreflang and breadcrumbs from the CMS, and a build fails when an indexable route has ssr: false | ✗ PARTIAL (G3) | Head, one @graph, breadcrumbs, pagination, build guard: verified (layers stage: tests, playground builds, smokes, guard:check; demo strict smoke rehearsal in 33-14). hreflang is emitted from the CMS, but for Journal records it lists every enabled locale, translated or not (D-57 violated, G3) | +| SC3 | The crawler reports SEO issues per URL, and each issue can be fixed through MCP | ✓ VERIFIED (automated) | Per-URL issues of every C23 kind; `Route()` in classes/crawl/fix.go gives every finding a tool or fix: code (Jakub's 2026-10-28 reading of SC3); TestCrawlerChecks finds each planted issue once and none on /clean; TestEveryAdminOperationHasTool. Caveat inside G1: sitemap_noindex on a Nuxt-only noindex address names a tool that does not fix it. Live agent session: human item | +| SC4 | The new code has unit tests, delivered in the phase's last plan | ✓ VERIFIED | All 39 unit test files of 33-14 exist (Go table tests; TS files with 4–15 cases) in commits 513793a, eb8a028, fbe9816 (framework), 1048c01 (sitemap), 82b0dac, 075cf17 (seo), e7832a8, 311c28a (journal), 75cc100 (vue-seo-layer), 9e66cbe (journal layer), 7d952fa (faq layer); `check-phase33.sh --all` green in this run with required test names per repository | + +**Score:** 2/4 roadmap criteria verified; plan must-haves 191/202 truths verified (0 present-but-behaviour-unverified). + +### Plan Must-Haves (202 truths, 15 backstops, 27 prohibitions) + +| Plan | Truths | Result | Exceptions | +|------|--------|--------|-----------| +| 33-01 framework seams, gate | 12 | 12 verified | WR-02 meaning recorded in 33-01-SUMMARY; version text v0.5.1/v0.5.2 superseded by v0.6.0/v0.6.1 (release log, 2026-11-04) | +| 33-02 Journal H1 setting, editor, descriptors | 18 | 17 verified, 1 human | J2 long-text wrap at 360px (planner assumption, visual) | +| 33-03 sitemap index | 15 | 14 verified, 1 failed | Truth 4 (alternates only for translated locales) fails on Journal records: G3 | +| 33-04 images, News, IndexNow, HTML API | 13 | 13 verified | IndexNow only in production and only on the site host (TestIndexNow, NotifyIndexNow) | +| 33-05 filter, registry, site endpoint | 16 | 16 verified | G1 is outside its wording ("effective SEO"), but limits SC1 | +| 33-06 graph, breadcrumbs, hreflang, pagination | 20 | 19 verified, 1 failed | Truth 7 (hreflang from translated locales only) fails on Journal records: G3 | +| 33-07 vue-seo-layer head and root files | 14 | 14 verified | | +| 33-08 build guard, route report, drift check | 15 | 15 verified | Drift check emits a false journal-tag warning while tag_pages is false (G2, warning) | +| 33-09 journal, menu, FAQ layers | 17 | 17 verified | | +| 33-10 crawler and issue report | 22 | 22 verified | TestChromeRenderer ran against /usr/bin/chromium | +| 33-11 auto-links, WebP | 13 | 13 verified | `go test -tags nodynamic ./modules/lagoon/attach/webp` passes | +| 33-12 apps I | 8 | 8 verified | Patches apply; sm-bm-app check-noindex ok | +| 33-13 grzyby | 8 | 5 verified, 1 pending, 2 superseded | Truth 1 (urls.txt identical) proven on local links, repeat on tags pending; truths 7–8 (uat) superseded: no uat, production after tags | +| 33-14 tests, gate, release | 11 | 6 verified, 4 pending, 1 superseded | Pending Task 4: release order, patches applied with app builds and check-pins, changelog post, release log steps. Superseded: the uat check. Verified: nothing pushed or tagged yet (0 tags contain HEAD in any repository) | + +Backstop truths (15): each has a named passing test in the green gate (for example paginationWindow.test.ts `[1,'gap',3,4,5,6,7,'gap',14]`, expect-seo.txt `count=