golem15/summercms
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
3cbc8d32d15ca53ad9c81b8a202137bfb39bd133
summercms/.planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VERIFICATION.md
Jakub Zych 565ce982d9 test(11.1): persist human verification items as UAT
2026-10-01 09:36:25 +02:00

497 lines
37 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
phase: 11.1-summercms-documentation-for-humans-and-ai-agents
verified: 2026-10-01T07:31:53Z
status: human_needed
score: 13/13 must-haves verified
covered_files:
- ".gitignore"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-01-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-01-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-02-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-02-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-03-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-03-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-04-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-04-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-05-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-05-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-06-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-06-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-07-PLAN.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-07-SUMMARY.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-REVIEW-DISPOSITION.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-REVIEW.md"
- ".planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VALIDATION.md"
- ".planning/todos/pending/readme-go-fences-src.md"
- "CLAUDE.md"
- "README.md"
- "cmd/summer/docs.go"
- "cmd/summer/docs_test.go"
- "cmd/summer/main.go"
- "cmd/summer/main_test.go"
- "cmd/summer/phase11_1_acceptance_test.go"
- "docs/architecture/application-lifecycle.md"
- "docs/architecture/go-modules-and-workspaces.md"
- "docs/architecture/introduction.md"
- "docs/architecture/request-lifecycle.md"
- "docs/backend/admin-controllers.md"
- "docs/backend/admin-spa.md"
- "docs/backend/forms.md"
- "docs/backend/lists-and-filters.md"
- "docs/backend/partials-and-widgets.md"
- "docs/backend/relation-manager.md"
- "docs/backend/settings.md"
- "docs/backend/users-and-permissions.md"
- "docs/console/introduction.md"
- "docs/console/scaffolding.md"
- "docs/console/setup-and-maintenance.md"
- "docs/console/utilities.md"
- "docs/console/writing-commands.md"
- "docs/database/attachments.md"
- "docs/database/casts-and-validation.md"
- "docs/database/migrations.md"
- "docs/database/models.md"
- "docs/database/queries-and-pagination.md"
- "docs/database/relations.md"
- "docs/database/transactions.md"
- "docs/examples/blog/blog_test.go"
- "docs/examples/blog/classes/doc.go"
- "docs/examples/blog/config/config.yaml"
- "docs/examples/blog/console/doc.go"
- "docs/examples/blog/console/publish.go"
- "docs/examples/blog/console/publish_test.go"
- "docs/examples/blog/controllers/doc.go"
- "docs/examples/blog/controllers/posts.go"
- "docs/examples/blog/controllers/posts/config_form.yaml"
- "docs/examples/blog/controllers/posts/config_list.yaml"
- "docs/examples/blog/controllers/posts_test.go"
- "docs/examples/blog/jobs/doc.go"
- "docs/examples/blog/lang/en/lang.yaml"
- "docs/examples/blog/middleware/doc.go"
- "docs/examples/blog/models/doc.go"
- "docs/examples/blog/models/post.go"
- "docs/examples/blog/models/post_test.go"
- "docs/examples/blog/models/posts/columns.yaml"
- "docs/examples/blog/models/posts/fields.yaml"
- "docs/examples/blog/plugin.go"
- "docs/examples/blog/postgres_test.go"
- "docs/examples/blog/registry.gen.go"
- "docs/examples/blog/routes.go"
- "docs/examples/blog/scaffold_layout_test.go"
- "docs/examples/blog/updates/20260101000000_create_acme_blog_posts.go"
- "docs/examples/blog/updates/20260101000100_add_published_at.go"
- "docs/examples/blog/updates/doc.go"
- "docs/examples/blog/updates/updates_test.go"
- "docs/examples/blog/views/mail/welcome.htm"
- "docs/index.md"
- "docs/plugins/extending.md"
- "docs/plugins/registration.md"
- "docs/plugins/scheduling.md"
- "docs/plugins/testing.md"
- "docs/services/authentication.md"
- "docs/services/configuration.md"
- "docs/services/events.md"
- "docs/services/frontend-and-ajax.md"
- "docs/services/jobs.md"
- "docs/services/localization.md"
- "docs/services/mail.md"
- "docs/services/oauth-server.md"
- "docs/services/outbound-http.md"
- "docs/services/parity-testing.md"
- "docs/services/push.md"
- "docs/services/rate-limiting.md"
- "docs/services/realtime.md"
- "docs/services/routing.md"
- "docs/services/search.md"
- "docs/services/storage.md"
- "docs/setup/coming-from-wintercms.md"
- "docs/setup/configuration.md"
- "docs/setup/installation.md"
- "docs/setup/introduction.md"
- "docs/setup/porting-a-plugin.md"
- "docs/site.yaml"
- "go.mod"
- "go.sum"
- "internal/docsite/check_commands.go"
- "internal/docsite/check_forbidden.go"
- "internal/docsite/check_identifiers.go"
- "internal/docsite/check_links.go"
- "internal/docsite/check_policy.go"
- "internal/docsite/checks_test.go"
- "internal/docsite/docsite.go"
- "internal/docsite/docsite_test.go"
- "internal/docsite/emit.go"
- "internal/docsite/emit_test.go"
- "internal/docsite/fences.go"
- "internal/docsite/fences_test.go"
- "internal/docsite/highlight.go"
- "internal/docsite/highlight_test.go"
- "internal/docsite/load.go"
- "internal/docsite/load_test.go"
- "internal/docsite/render.go"
- "internal/docsite/render_test.go"
- "internal/docsite/serve.go"
- "internal/docsite/serve_test.go"
- "internal/docsite/snippet.go"
- "internal/docsite/snippet_test.go"
- "internal/docsite/testdata/violations/command-bin-no-dot/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-bin-no-dot/want.txt"
- "internal/docsite/testdata/violations/command-env-prefix/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-env-prefix/want.txt"
- "internal/docsite/testdata/violations/command-flag-first/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-flag-first/want.txt"
- "internal/docsite/testdata/violations/command-go-run/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-go-run/want.txt"
- "internal/docsite/testdata/violations/command-in-callout/docs/extras/faq.md"
- "internal/docsite/testdata/violations/command-in-callout/want.txt"
- "internal/docsite/testdata/violations/go-fence-callout-no-src/docs/extras/faq.md"
- "internal/docsite/testdata/violations/go-fence-callout-no-src/want.txt"
- "internal/docsite/testdata/violations/go-fence-golang/docs/extras/faq.md"
- "internal/docsite/testdata/violations/go-fence-golang/want.txt"
- "internal/docsite/testdata/violations/identifier-wrong-case/docs/extras/faq.md"
- "internal/docsite/testdata/violations/identifier-wrong-case/modules/demo/open.go"
- "internal/docsite/testdata/violations/identifier-wrong-case/want.txt"
- "internal/docsite/testdata/violations/snippet-blockquote/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-blockquote/want.txt"
- "internal/docsite/testdata/violations/snippet-build-ignored-test-root/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-build-ignored-test-root/modules/demo/demo_test.go"
- "internal/docsite/testdata/violations/snippet-build-ignored-test-root/modules/demo/ignored_test.go"
- "internal/docsite/testdata/violations/snippet-build-ignored-test-root/want.txt"
- "internal/docsite/testdata/violations/snippet-build-ignored/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-build-ignored/modules/demo/broken.go"
- "internal/docsite/testdata/violations/snippet-build-ignored/want.txt"
- "internal/docsite/testdata/violations/snippet-callout-drift/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-callout-drift/want.txt"
- "internal/docsite/testdata/violations/snippet-callout-missing/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-callout-missing/want.txt"
- "internal/docsite/testdata/violations/snippet-example-no-output-helper/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-example-no-output-helper/modules/demo/never_test.go"
- "internal/docsite/testdata/violations/snippet-example-no-output-helper/want.txt"
- "internal/docsite/testdata/violations/snippet-example-no-output-region/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-example-no-output-region/modules/demo/never_test.go"
- "internal/docsite/testdata/violations/snippet-example-no-output-region/want.txt"
- "internal/docsite/testdata/violations/snippet-list-item/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-list-item/want.txt"
- "internal/docsite/testdata/violations/snippet-readme-src/modules/demo/README.md"
- "internal/docsite/testdata/violations/snippet-readme-src/want.txt"
- "internal/docsite/testdata/violations/snippet-testdata-go/docs/extras/faq.md"
- "internal/docsite/testdata/violations/snippet-testdata-go/modules/demo/testdata/fixture_test.go"
- "internal/docsite/testdata/violations/snippet-testdata-go/want.txt"
- "internal/docsite/theme/assets/search.js"
- "internal/docsite/theme/assets/site.css"
- "internal/docsite/theme/assets/site.js"
- "internal/docsite/theme/assets/theme-init.js"
- "internal/docsite/theme/templates/404.html"
- "internal/docsite/theme/templates/footer.html"
- "internal/docsite/theme/templates/header.html"
- "internal/docsite/theme/templates/icons.html"
- "internal/docsite/theme/templates/page.html"
- "internal/docsite/theme/templates/pager.html"
- "internal/docsite/theme/templates/search.html"
- "internal/docsite/theme/templates/sidebar.html"
- "internal/docsite/theme/templates/toc.html"
- "internal/docsite/theme_test.go"
- "internal/docsite/violations_test.go"
- "modules/backpack/example_test.go"
- "modules/beachcomber/example_test.go"
- "modules/beachcomber/export_docs_test.go"
- "modules/beachcomber/typesense/example_test.go"
- "modules/bonfire/example_test.go"
- "modules/bouncer/example_test.go"
- "modules/cabana/example_controller_test.go"
- "modules/cabana/example_test.go"
- "modules/cabana/testdata/docs/controllers/posts/config_filter.yaml"
- "modules/cabana/testdata/docs/controllers/posts/config_form.yaml"
- "modules/cabana/testdata/docs/controllers/posts/config_list.yaml"
- "modules/cabana/testdata/docs/models/post/columns.yaml"
- "modules/cabana/testdata/docs/models/post/fields.yaml"
- "modules/cabana/testdata/docs/models/settings/fields.yaml"
- "modules/compass/example_test.go"
- "modules/conga/example_test.go"
- "modules/conga/export_docs_test.go"
- "modules/festival/example_test.go"
- "modules/fetchguard/example_test.go"
- "modules/flare/example_test.go"
- "modules/lagoon/attach/example_test.go"
- "modules/lagoon/example_test.go"
- "modules/lagoon/export_docs_test.go"
- "modules/lighthouse/centrifugo/example_test.go"
- "modules/lighthouse/example_test.go"
- "modules/lighthouse/export_docs_test.go"
- "modules/pact/example_test.go"
- "modules/party/example_plugin_test.go"
- "modules/party/example_test.go"
- "modules/phrasebook/example_test.go"
- "modules/postcard/example_test.go"
- "modules/surf/example_test.go"
- "modules/tide/example_test.go"
- "modules/tide/testdata/docs/posts-spec.yaml"
- "modules/towel/example_test.go"
- "modules/wire/example_test.go"
- "modules/wristband/example_test.go"
- "scripts/check-phase11.1.sh"
covered_digest: "v2:sha256:69e8741c6e6d238b3ae5164f594e9dce56016ddf6cda5b738f3fb81f3fec780e"
behavior_unverified: 0
overrides_applied: 0
re_verification:
previous_status: gaps_found
previous_score: 10/13
gaps_closed:
- "DOCS-04 fence discovery: AST collection refuses nested and README src= fences, Go-lexer aliases need src=, and captions cover only checked fences"
- "D-07 proof of execution: src= targets must be in the default build and reached from a Test or an Example with output"
- "DOCS-05 name forms: go doc -c is case-sensitive, and command words cover env prefixes, flags, go run ./cmd/summer and bin/{app}"
gaps_remaining: []
regressions: []
decision_coverage:
honored: 18
total: 18
not_honored: []
advisory:
- finding: "The identifier index still reads build-ignored Go files, so a declaration go test never compiles can still satisfy an identifier span."
category: other
reason: "Recorded by plan 11.1-07 as outside the gap-closure contract. It is not one of the three prior gaps. No red test was run for it on this pass, so it does not reopen DOCS-05."
evidence_status: "none provided"
human_verification:
- test: "Run summer docs:serve, open the printed address, search for a module name and a CLI command, and use the arrow keys and Enter."
expected: "Rows show section, title, heading and a 2-line excerpt with mark highlights; arrow keys move the selection and Enter opens the page; the live region announces the count; empty and zero-result states show the UI-SPEC copy."
why_human: "Plan 02 and plan 06 backstop truths, and the two manual rows in 11.1-VALIDATION.md. search.js has no JS test runner."
- test: "Cycle the theme toggle through light, dark and system, reload in each mode, and check a guide page and an API page at 1280px, 1024px and 375px."
expected: "No flash of the wrong theme; the choice persists; UI-SPEC contrast pairs hold in both themes."
why_human: "Plan 02 backstop truths. Visual timing is not observable from Go tests."
- test: "Disable JavaScript and load a guide page below 1024px."
expected: "The sidebar renders above the content and the search and theme buttons are hidden."
why_human: "Plan 02 backstop truth (no-JS layout)."
- test: "Read llms.txt, one page .md and llms-full.txt as an agent would."
expected: "Spec shape with predictable .md URLs that are useful without the HTML."
why_human: "Plan 01 backstop truth. Usefulness is non-inferable from structure. unverified — held-out test recommended."
- test: "Read Setup, Backend, Database and Services, and follow docs/setup/porting-a-plugin.md top to bottom."
expected: "WinterCMS docs tone, accurate to the module READMEs, ending at the same acme.blog plugin as docs/examples/blog."
why_human: "Plans 03, 04 and 05 backstop truths."
- test: "Resolve the seven judgment-tier prohibitions in the report body."
expected: "Each holds. The verifier's non-authoritative verdict is holds for all seven."
why_human: "unverified-prohibition — human review recommended. Judgment-tier prohibitions are not a silent pass."
---
# Phase 11.1: SummerCMS documentation for humans and AI agents Verification Report
**Phase Goal:** SummerCMS has a WinterCMS-style documentation set that serves both humans and AI agents. The Markdown source lives in `summercms.go/docs/` and a `summer` CLI command builds it into a static site with sidebar navigation, search, `llms.txt`/`llms-full.txt` and a raw `.md` per page. Every code example compiles and is tested, and a "Coming from WinterCMS" map plus an `acme/blog` porting walkthrough cover the migration path. It documents the framework as it stands after Phase 11 and never names a consuming application.
**Verified:** 2026-10-01T07:31:53Z
**Status:** human_needed
**Re-verification:** Yes — after gap closure (plan 11.1-07)
## Verdict in one paragraph
The three gaps from 2026-09-30 are closed in the code, and the phase gate is green. `collectFences` walks the goldmark AST the renderer uses. A `src=` fence inside a callout, blockquote or list item, and any `src=` fence in a module README, fails Check and is not captioned. Go-lexer aliases need `src=`. A `.go` target must be in the default build and reached from a `Test` or an Example with output. `go doc -c` is case-sensitive, and `commandWord` accepts env prefixes, cobra-style flags, `go run ./cmd/summer` and `bin/{app}`. `bash scripts/check-phase11.1.sh --all` printed `phase11.1 all passed` (exit 0), including a scratch-tree self-test that plants the old holes. Automated must-haves are 13/13. Status stays `human_needed` because the plans' backstop truths (search, theme, no-JS layout, editorial usefulness) and seven judgment-tier prohibitions still need a person. That is not a code gap.
## Goal Achievement
### Observable Truths
The scored set is the same 13 truths as the previous report. Plan 11.1-07's ten truths are the re-test of truths 6–8; they are listed under that table and are not a second denominator.
| # | Truth | Status | Evidence |
| --- | --- | --- | --- |
| 1 | SC1: docs/ holds Markdown pages with frontmatter in Setup, Architecture, Plugins, Backend, Database, Services, Console, API reference, and every framework module is reachable from the sidebar | ✓ VERIFIED | Regression: `docs/site.yaml` still lists those eight sections. `scripts/check-phase11.1.sh --all` ran `TestPhase11_1Acceptance` under `--named` (`-count=1`, skips refused) and printed `phase11.1 named passed`. `docs:build` wrote 72 pages. |
| 2 | SC2: a `summer` command builds a self-contained static site with sidebar, TOC, prev/next, client-side search and dark mode, no Node toolchain; only new dep is approved | ✓ VERIFIED | `--deps` passed (chroma/v2 only). `--docs` built the site. Search keystrokes and theme flash stay in human verification; markup is covered by the acceptance test the gate ran. |
| 3 | SC3/DOCS-03: `llms.txt`, `llms-full.txt` and a clean `.md` per page, with a test asserting sync with the page tree | ✓ VERIFIED | `--docs` requires `llms.txt`, `llms-full.txt` and `index.md`. `TestDocsAIOutputsInSync` is in the named set that passed. Usefulness of the raw files stays a backstop. |
| 4 | SC4 (current content): every Go example in a docs/ page is compiled and run by `go test ./...` | ✓ VERIFIED | `TestPhase11_1Acceptance` passed via `--named`. The acceptance scanner now marks blockquoted fences nested (`scanDocFences` / `TestAcceptanceFenceScanner`) and the real tree has no nested `src=` fence. |
| 5 | SC5 / DOCS-07: the concept map and the acme/blog walkthrough exist, and the walkthrough's code is verified under SC4 | ✓ VERIFIED | Both pages are still present. `--named` ran the blog package tests, including the Docker migrate/route/command tests, and refuses a skip. `phase11.1 named passed`. |
| 6 | DOCS-04 enforcement: a test fails on a missing or drifted snippet; every Go fence must carry src= | ✓ VERIFIED | Closed by plan 07 truths G1–G5 and G10. Self-test on a scratch copy of the real tree refused a callout `src=` fence (`must be a top-level block`) and a `golang` fence with no `src=`. `TestPlantedViolations` walks all 81 violation directories, including the new plants, and the self-test stage passed. |
| 7 | D-07 proof of execution: src= must name compiled source, and code no Test or Example runs is a snippet problem | ✓ VERIFIED | Closed by G6–G7. `loadTestGraph` roots are `isTestName` plus `doc.Examples` with `Output` or `EmptyOutput`. `checkBuiltGo` uses `build.ImportDir` on `build.Default` and refuses `testdata` and `_` directories. `TestSnippetRootsAndBuild` and the planted fixtures `snippet-example-no-output-helper`, `snippet-build-ignored` and `snippet-testdata-go` are in the corpus the self-test ran. |
| 8 | DOCS-05: checkers fail on stale identifiers, broken links/anchors, unknown command names and forbidden names | ✓ VERIFIED | Closed by G8–G9 for the two holes. `goDoc` runs `go doc -c`. Self-test refused `lagoon.Openfromapp`, `FOO=1 summer no:such` and `summer --root . no:such`. Links, anchors and forbidden names were already refused and the same self-test still plants them. |
| 9 | DOCS-06: "Coming from WinterCMS" maps Winter concepts to SummerCMS, with checker-verified identifiers and explicit Not provided rows | ✓ VERIFIED | Regression: the Not provided table in `docs/setup/coming-from-wintercms.md` still links `docs/services/frontend-and-ajax.md`. Identifiers are inside the checkers the gate ran. |
| 10 | DOCS-08: CLAUDE.md records the docs update rule and names the checkers | ✓ VERIFIED | `CLAUDE.md` `## Documentation` still names `TestDocsTree` and `summer docs:build --check`. `--claude` passed. |
| 11 | D-11: no page, snippet or built output names a consuming application | ✓ VERIFIED | `grep -rliE` over `docs/**/*.md` found no consuming-application spelling. `--forbidden` passed. A planted name is still in the self-test. |
| 12 | Phase-end tests: docsite coverage ≥85%, a planted fixture per rule, and acceptance subtests SC1–SC5 | ✓ VERIFIED | This pass: `go test -count=1 -cover ./internal/docsite` reported `coverage: 94.1% of statements`. 81 violation directories. `TestPlantedViolations` and `TestPhase11_1Acceptance` passed inside the gate. |
| 13 | The gate: `check-phase11.1.sh --all` is green and `--named` refuses skipped or zero-match runs | ✓ VERIFIED | This pass ran `bash scripts/check-phase11.1.sh --all`. Every stage passed and it printed `phase11.1 all passed`, exit 0. `detect_json` still exits on skip, "no tests to run" and a missing required name. |
**Score:** 13/13 truths verified (0 present, behavior-unverified)
Five backstop truths (search rows and keys, no theme flash, contrast, no-JS layout, llms usefulness, editorial and walkthrough followability) stay non-inferable. They are not in the 13. They route to human verification with reason `insufficient_spec`.
### Plan 11.1-07 closure truths
All ten are verified. They are the detailed contract for truths 6–8, not extra score rows.
| # | Truth | Status | Evidence |
| --- | --- | --- | --- |
| G1 | Snippet, policy and command checks find every goldmark `*ast.FencedCodeBlock` | ✓ VERIFIED | `collectFences` in `internal/docsite/fences.go` walks the document from `newMarkdown`. `checkSnippets`, `checkPolicy` and `checkCommands` call it. `TestCollectFences` expects the callout, blockquote and list fences and rejects a four-space indented fence. |
| G2 | A non-top-level `src=` fence fails Check, docs:build --check and docs:sync, and sync writes nothing | ✓ VERIFIED | `nestedSrcMessage` in `checkFences` returns before `Extract`. `TestSyncParsedFences` expects one problem and an unchanged tree. Self-test plant on the real-tree copy was refused. |
| G3 | A module README `src=` fence is refused and no unchecked fence gets a figcaption | ✓ VERIFIED | `readmeSrcMessage` when `page.Module != ""`. `fenceAnnotator` sets `data-src` only when `Module == ""` and the parent is the document. `TestFenceCaptionsOnlyVerified` expects one guide caption and none on the API page. |
| G4 | Any fence whose chroma lexer is Go needs `src=`, at any depth; go-html-template, text and empty do not | ✓ VERIFIED | `goLang` uses `lexerFor`. `checkPolicy` applies it to every collected fence of a docs page. `TestGoLang` and `TestNestedFenceChecks` (callout `go`, `GO`, `main.go`). Self-test refused `golang`. |
| G5 | Shell fences inside callouts are command-checked; four or more leading spaces are not a fence | ✓ VERIFIED | `checkCommands` reads `f.code` from the AST for `sh`/`shell`/`bash`/`console`. `openFence` rejects indent above 3. `TestCollectFences` wants 6 fences, not 7. Plant `command-in-callout`. |
| G6 | Reachability roots are Test functions plus Examples with output, from test files in the default build | ✓ VERIFIED | `loadTestGraph` skips Examples with empty `Output` and no `EmptyOutput`, and only parses `TestGoFiles`/`XTestGoFiles`. `TestSnippetRootsAndBuild` asserts `ExampleNone` and `onlyFromIgnored` are not reachable. |
| G7 | A `.go` src= target fails when it is under `testdata` or `_`, or `build.ImportDir` does not list it | ✓ VERIFIED | `checkBuiltGo`. Plants `snippet-build-ignored`, `snippet-build-ignored-test-root`, `snippet-testdata-go`. |
| G8 | `go doc -c`, so `demo.Openfromapp` fails | ✓ VERIFIED | `check_identifiers.go` argv is `go`, `doc`, `-c`. `TestIdentifierGoDocCaseSensitive` expects `Openfromapp` false. Self-test refused `lagoon.Openfromapp`. Plant `identifier-wrong-case`. |
| G9 | Command word after `VAR=value` and flags, for `summer`, `go run ./cmd/summer`, `./bin/{app}` and `bin/{app}`; `summer --help` alone is not a command | ✓ VERIFIED | `commandWord` in `check_commands.go`. `TestCommandWord` covers those forms. Self-test refused the env-prefix and flag-first plants. |
| G10 | Each hole has a planted fixture; the real tree passes TestDocsTree, docs:build --check and TestPhase11_1Acceptance; the gate prints `phase11.1 all passed` | ✓ VERIFIED | 18 new plants are in `testdata/violations` and `TestPlantedViolations` runs every directory. Gate exit 0, quoted above. `scanDocFences` strips `>` and `goFenceLang` uses chroma. SC4 cross-checks figcaption counts. |
### Required Artifacts
| Artifact | Expected | Status | Details |
| --- | --- | --- | --- |
| `internal/docsite/fences.go` | AST fence discovery | ✓ VERIFIED | `func collectFences`; wired from snippet, policy and command checks |
| `internal/docsite/snippet.go` | top-level src=, README refusal, build membership, Example roots | ✓ VERIFIED | `build.ImportDir`, `doc.Examples`, `nestedSrcMessage`, `readmeSrcMessage` |
| `internal/docsite/highlight.go` | `goLang` via the highlighter lookup | ✓ VERIFIED | `func goLang`; `checkPolicy` calls `goLang(` |
| `internal/docsite/check_commands.go` | `commandWord` | ✓ VERIFIED | env prefix, flags, `go run ./cmd/summer`, `bin/` |
| `internal/docsite/check_identifiers.go` | case-sensitive go doc fallback | ✓ VERIFIED | `"doc", "-c"` |
| `internal/docsite/render.go` | captions only on verified fences | ✓ VERIFIED | `Module == ""` and document parent |
| `internal/docsite/testdata/violations/snippet-callout-drift/want.txt` | CR-01 plant | ✓ VERIFIED | message contains `top-level block` |
| `cmd/summer/phase11_1_acceptance_test.go` | independent scanner and caption cross-check | ✓ VERIFIED | `TestAcceptanceFenceScanner`, `scanDocFences` |
| `scripts/check-phase11.1.sh` | gate plants the holes; `--all` | ✓ VERIFIED | ran this pass |
| `internal/docsite/docsite.go` | Check calls `checkSnippets` after parse | ✓ VERIFIED | call at `checkContent`; not left only in assemble |
| `docs/setup/coming-from-wintercms.md` | concept map | ✓ VERIFIED | regression, file present, Not provided rows |
| `docs/setup/porting-a-plugin.md` | walkthrough | ✓ VERIFIED | regression, file present |
| `docs/examples/blog/*` | in-root plugin | ✓ VERIFIED | named blog tests passed inside the gate |
| `CLAUDE.md` | documentation rule | ✓ VERIFIED | `--claude` passed |
### Key Link Verification
| From | To | Via | Status | Details |
| --- | --- | --- | --- | --- |
| `internal/docsite/render.go` | `internal/docsite/snippet.go` | caption set equals the fences `checkSnippets` compares | WIRED | `Module == ""` and `KindDocument` in `fenceAnnotator`; `TestFenceCaptionsOnlyVerified` |
| `internal/docsite/check_policy.go` | `internal/docsite/highlight.go` | `goLang` reuses `lexerFor` | WIRED | `goLang(` at the policy loop |
| `internal/docsite/snippet.go` | `go/build` and `go/doc` | `build.ImportDir` and `doc.Examples` | WIRED | `buildPackage`, `exampleHasOutput`, `loadTestGraph` |
| `internal/docsite/check_identifiers.go` | `go doc -c` | `goDoc` argv | WIRED | `exec.CommandContext(ctx, "go", "doc", "-c", ...)` |
| `scripts/check-phase11.1.sh` | new tests and fixtures | `--named` list includes `TestNestedFenceChecks`, `TestCommandWord`, `TestSnippetRootsAndBuild` | WIRED | `--named` passed |
| `cmd/summer/docs.go` | `internal/docsite` | `Build` / `Check` / `Sync` | WIRED | regression; gate builds `./cmd/summer` and runs `docs:build --check` |
| `docs/setup/porting-a-plugin.md` | `docs/examples/blog` | `src=` fences | WIRED | acceptance SC4/SC5 inside the gate |
### Data-Flow Trace (Level 4)
The docsite does not render application data from a database. Page HTML is produced from the Markdown sources.
| Artifact | Data Variable | Source | Produces Real Data | Status |
| --- | --- | --- | --- | --- |
| Built guide HTML | fenced code and figcaption | docs page body plus `Extract` of a `src=` target, caption only when the checker compared it | Yes | ✓ FLOWING |
| `llms.txt` / per-page `.md` | title, description, body | loaded page tree (`emit.go`) | Yes | ✓ FLOWING |
| Search index | page titles and text | the same page tree written by the build | Yes | ✓ FLOWING |
| API reference pages | README body | `modules/*/README.md` via `loadModules` | Yes | ✓ FLOWING |
### Behavioral Spot-Checks
| Behavior | Command | Result | Status |
| --- | --- | --- | --- |
| Full phase gate, including scratch-tree plants of the old holes | `bash scripts/check-phase11.1.sh --all` | all stages passed, `phase11.1 all passed`, exit 0 | ✓ PASS |
| Docsite coverage at least 85% | `go test -count=1 -cover ./internal/docsite` | `coverage: 94.1% of statements` | ✓ PASS |
| Callout src=, golang fence, wrong-case ident, env-prefix and flag-first commands | self-test plants inside the gate (real-tree copy) | each `expect_refusal` returned; stage printed `phase11.1 self-test passed` | ✓ PASS |
| Named tests, skips refused | gate `--named` (`go test -json -count=1`) | `phase11.1 named passed` | ✓ PASS |
| Full module tests | gate `--go` (`go vet ./...` then `go test ./...`) | `phase11.1 go passed` (many packages `(cached)` after `--named` had already run the docs packages with `-count=1`) | ✓ PASS |
### Probe Execution
| Probe | Command | Result | Status |
| --- | --- | --- | --- |
| `scripts/check-phase11.1.sh` | `bash scripts/check-phase11.1.sh --all` | exit 0, `phase11.1 all passed` | PASS |
### Requirements Coverage
| Requirement | Source Plan | Description | Status | Evidence |
| --- | --- | --- | --- | --- |
| DOCS-01 | 01, 03, 04, 06 | Markdown pages with strict frontmatter in Winter sections; every module reachable via an API page | ✓ SATISFIED | Truth 1 |
| DOCS-02 | 01, 02, 06 | self-contained site, docs:serve on loopback, no Node, only chroma/v2 added | ✓ SATISFIED | Truth 2. Search and theme interaction remain human UAT, as 11.1-VALIDATION.md already records. |
| DOCS-03 | 01, 06 | llms.txt, llms-full.txt, .md per page, sync test | ✓ SATISFIED | Truth 3 |
| DOCS-04 | 01–07 | every Go fence in docs/ has src=; a test fails on a missing or drifted snippet; Examples carry Output and run | ✓ SATISFIED | Truths 4, 6, 7 and G1–G7, G10. REQUIREMENTS.md still shows this box open and the traceability row as Gaps Found; this dispatch was not allowed to edit that file. The code satisfies the requirement. |
| DOCS-05 | 02, 06, 07 | checkers fail on stale identifiers, broken links and anchors, unknown commands, forbidden names | ✓ SATISFIED | Truth 8 and G8–G9. Same note: the REQUIREMENTS.md checkbox was left for a later edit and is not evidence. |
| DOCS-06 | 03, 04, 06 | concept map with checker-verified identifiers | ✓ SATISFIED | Truth 9 |
| DOCS-07 | 05, 06 | acme/blog walkthrough as a real in-root package | ✓ SATISFIED | Truth 5 |
| DOCS-08 | 02, 06 | CLAUDE.md documentation rule naming the checkers | ✓ SATISFIED | Truth 10 |
No orphaned requirement. REQUIREMENTS.md maps exactly DOCS-01 through DOCS-08 to Phase 11.1, and every ID is claimed by at least one plan.
### Decision Coverage
All trackable CONTEXT.md decisions are honored by shipped artifacts. 18 honored, 18 total, none missing. Non-blocking.
### Prohibitions (judgment-tier)
**unverified-prohibition — human review recommended.** These verdicts are a non-authoritative LLM judge. They are not a silent pass. Autonomous completion is complete with 7 flagged prohibitions.
| # | Statement | Verdict | Evidence |
| --- | --- | --- | --- |
| 1 | The built site must not reference a third-party origin for assets (plans 01, 02) | holds | Theme templates and assets were not modified by plan 07. The gate rebuilt the site. Prior inspection (no external `src=`, vendored font `url()`) still applies to those unchanged files. |
| 2 | The theme must not load third-party fonts, icons or scripts, and has no analytics | holds | Same unchanged theme tree. Licences ship under the theme assets. |
| 3 | The docs must not describe an unprovided WinterCMS feature as available (plans 03, 04) | holds | Not provided table re-read this pass; it links Frontend and AJAX. `docs/site.yaml` `llms_notes` still say the framework is headless. |
| 4 | The docs must not present an insecure setting as the default (plan 04) | holds (spot-check carried forward) | Mail, push, search and serve pages were not in the plan 07 diff. A full read is still a human item. |
| 5 | The walkthrough must not show a write without a fill allow-list, or an admin controller without a permission | holds | `lagoon.Fill(post, post.Fillable(), ...)` in `docs/examples/blog/models/post.go`; `RequiredPermissions` in `controllers/posts.go`. |
| 6 | The gate must not count a skipped, filtered or zero-match run as a pass | holds | `detect_json` exits on `skip`, `no tests to run`, zero passes and a missing required name. `--named` passed under that detector. |
| 7 | The site must not show a source caption over a code block the snippet checker did not compare (plan 07) | holds | `fenceAnnotator` matches the checked set. `TestFenceCaptionsOnlyVerified` and the acceptance figcaption cross-check ran inside the gate. |
### Test Quality Audit
| Test File | Linked Req | Active | Skipped | Circular | Assertion Level | Verdict |
| --- | --- | --- | --- | --- | --- | --- |
| `internal/docsite/violations_test.go` | DOCS-04, DOCS-05 | 81 directory cases plus runtime forbidden plants; each expects exactly one problem and a refused Build | 0 (`t.Skip` absent in `internal/docsite/*_test.go`) | No. Expected text is the planted `want.txt`, not output of the checker | Value | OK |
| `internal/docsite/snippet_test.go` `TestSnippetRootsAndBuild` | DOCS-04 | reachability and `Extract` error substrings | 0 | No. Oracle is `go/doc` and `go/build`, not the checker echoing itself | Behavioral | OK |
| `internal/docsite/checks_test.go` `TestCommandWord`, `TestIdentifierGoDocCaseSensitive`, `TestNestedFenceChecks` | DOCS-04, DOCS-05 | table and fixture assertions | 0 | No | Value | OK |
| `cmd/summer/phase11_1_acceptance_test.go` | DOCS-01..07 | SC1–SC5 plus `TestAcceptanceFenceScanner` | 0 in the named run (a skip would have failed the gate) | No. Scanner is independent of `docsite.collectFences` | Behavioral | OK |
| `scripts/check-phase11.1.sh` self-test | DOCS-04, DOCS-05 | plants on a copy of the real tree | n/a | No. The plant is a hand-written bad fence, not generated by the checker | Behavioral | OK |
**Disabled tests on requirements:** 0
**Circular patterns detected:** 0
**Insufficient assertions:** 0
### Anti-Patterns Found
| File | Line | Pattern | Severity | Impact |
| --- | --- | --- | --- | --- |
| phase Go, the gate script, `docs/console/utilities.md`, `README.md` | — | TBD / FIXME / XXX | — | none |
| `internal/docsite` | — | `t.Skip` | — | none |
### Advisory (New Scope, Unevidenced)
New-scope findings from this re-verification with no deterministic evidence. Not blocking. They do not reopen a closed must-have.
| # | Finding | Category | Why Advisory |
| --- | --- | --- | --- |
| 1 | Identifier index still parses build-ignored Go files | other | Named in the plan 07 summary as out of scope. Not a prior gap. No failing test was run. |
### Human Verification Required
Automated checks passed. These items are why status is `human_needed` rather than `passed`.
### 1. Search interaction
**Test:** Run `summer docs:serve`, search for a module name and a CLI command, use arrow keys and Enter.
**Expected:** Rows show section, title, heading and a marked excerpt; Enter opens the page; the live region announces the count.
**Why human:** Plan 02 and plan 06 backstop. No JS test runner.
### 2. Theme, contrast, no flash
**Test:** Cycle light, dark and system; reload; check a guide page and an API page at 1280px, 1024px and 375px.
**Expected:** No flash; the choice persists; UI-SPEC contrast holds.
**Why human:** Visual and first-paint timing.
### 3. No-JS layout
**Test:** Disable JavaScript and load a guide page below 1024px.
**Expected:** Sidebar above the content; search and theme buttons hidden.
**Why human:** Plan 02 backstop.
### 4. Raw files for an agent
**Test:** Read `llms.txt`, one page `.md` and `llms-full.txt`.
**Expected:** Spec shape and URLs that are useful without HTML.
**Why human:** Plan 01 backstop (`insufficient_spec`). unverified — held-out test recommended.
### 5. Editorial and walkthrough
**Test:** Read Setup, Backend, Database and Services, and follow `docs/setup/porting-a-plugin.md`.
**Expected:** WinterCMS tone, accurate to the READMEs, ending at `docs/examples/blog`.
**Why human:** Plans 03, 04 and 05 backstop.
### 6. Judgment-tier prohibitions
**Test:** Confirm the seven prohibition rows above.
**Expected:** Each holds.
**Why human:** unverified-prohibition — human review recommended.
### Gaps Summary
No code gap remains from the previous report. Fence discovery, proof of execution, and the identifier and command forms now fail closed, with planted fixtures and a green `scripts/check-phase11.1.sh --all`. Nothing in a later milestone phase was needed to defer them. The open work is human UAT already scheduled in `11.1-VALIDATION.md`, plus explicit resolution of the judgment-tier prohibitions. Do not open another gap-closure plan for those.
---
_Verified: 2026-10-01T07:31:53Z_
_Verifier: the agent (gsd-verifier)_
Reference in New Issue View Git Blame Copy Permalink